While I like the idea, Libravatar’s API is exceptionally slow and also error-prone. I scripted a benchmark for making 100 requests for 100 email addresses with the default size of 80x80px. The email addresses were a combination of Gravatar email addresses and Libravatar specific email addresses. The average file size is ±10KB while the average response time is ±6.8s. The 99th percentile of the response time is ±21s. That’s really not acceptable.
If privacy is really a concern here, get rid of Gravatar altogether.