I know this thread is old, but i am replying just in case someone else is looking for an answer:

I found this useful site in which someone is maintaining a big list of bots and a ready made robots.txt file.

https://baccyflap.com/res/robots/