From my testing, it appears you do need to keep the file there permanently. If it is removed, BSky may revoke your username.