AI Scraping

I've Been Deindexed by Google!

A short post on getting a personal blog removed from Google search to resist AI scraping: a blanket robots.txt disallow did nothing, and the fix was ironically to allow Googlebot back in so it could see a noindex, nofollow robots meta tag, repeated for other bots until the site dropped out of the index.

Blog post · SEO · Web · AI Scraping