I completely agree this is a huge barrier to mainstream adoption of smart devices. I think that there are lot of fundamental UX challenges that aren't being addressed in smart devices at the moment probably due to the demographic designing them.
Hopefully this doesn't mark me as "part of the problem" in suggesting different solutions, but accessibility for children and guests was part of my buying decisions for my smart home devices. I've preferred to have solutions that have physical controllers which are augmented by app control rather than pure app controlled devices. Hence I've chosen smart switches (LightwaveRF) over smart bulbs and Evohome for heating which as Quentin points out has physical controllers at each radiator (and a base unit). The TV has a physical remote which will turn the volume up and down on the Sonos playbar and we leave an old unlocked iPad kicking around to control the wider Sonos system in the house - this isn't ideal but few of our guests really need to access the Sonos system anyway.
Guest accounts would be a good feature, but I think they would be insufficient to address a lot of the usability challenges I encounter.