More and more browsers have origin-isolated their caches. Safari — including on mobile— has been doing it for years! There is no such thing as a shared cache in these browsers.

P.S.: You should swap out Gravatar for Libreavatar with a proxied image cache. It's an open alternative that uses an open standard and a federated system. Your blog would query the user's email domain for the face icon (or only as a fallback).