The Internet is fighting over whether robots.txt applies to AI agents. It all started when Cloudflare published a blog post, ...