robots.txt and other txt files? (HTML, JS, another tricks)
Yeah that definitely is a much simpler approach haha, on the list I found there are hundreds of bots already.
Seems like it is a difficult balance between keeping bad bots out and being findable by others. I applied to join a bunch of webrings in the hope that it helps with the second part, at least.