@Edent I, um, thanks very much for this article. It's extremely clear that the main thing I should do is, as you say, record what key is used in what accounts. But...

If someone gets into my house or physically steals my keys I have bigger problems 🤣 besides which, taking a copy of my keys, which provide access to my house, is trivially easier (a photo would be enough) and at least as dangerous. So if I'm comfortable with my house key security model, then my yubikey is still better than that.

So basically: I fully agree with your assessment. Thanks for clearly explaining that I can ignore this concern.