Trim contributors who don't have contributions

https://github.com/bevry/istextorbinary has rob and michael, who turns out don't have any contributions... this likely would have been from using bevry org members to prefill the data rather than the actual contributor graph - if the repo is a github repo, and if we can fetch contributors, we should dump such contirbutors like we do with inactive sponsors