You’ve confused two different things there. Nothing I do at the weekend gives you any indication about whether I can learn on the job.
Culture fit is an interesting question. Will you hire me if I support a different football team from you? If I like gardening at the weekends whereas all your team prefer sky-diving? How about if I volunteer for a charity which you don’t support?
Much better to build a diverse culture which is resilient than a fragile monoculture.