My email client (Airmail) has a useful feature here, if it doens’t find an avatar for that specific sender (I think thats a gmail/google thing) it uses the favicon of the senders domain so if bob@example.com emails me I see http://example.com/favicon.ico as the icon for his email, this means that at least you can see the company someone has emailed from and its useful for picking out external emails from the corporate noise.

Seems like it would be a good fallback to the way gravatar works for people that use unique addresses as they’re generally always on the same domain.