Hey, in the UK, is a sim swap attack really that easy?
https://nordvpn.com/blog/sim-swap-attack/
It sounds like the fault of the provider more than anything else, which should / would be easy to make sure there is a firm process in place to stop this from happening (surely it is madness to have this service at the end of a phone call with no clear steps in place to keep things secure?)
https://ee.co.uk/help/help-new/managing-and-using-my-account/leaving-ee/what-is-a-pac-code
text PAC to 65075
log in to My EE and go to Menu > Account settings > Leave EE
So your phone needs to be secure, and you provider account needs to be secure.
Seems simple enough, they you can rely on phone number as your gateway back in, store everything in cloud provider, and get back up to speed?
Perhaps I am missing something, I am not militant about security...