{"article":{"slug":"ive-been-deindexed-by-google","title":"I've Been Deindexed by Google!","subtitle":null,"summary":"A short post on getting a personal blog removed from Google search to resist AI scraping: a blanket robots.txt disallow did nothing, and the fix was ironically to allow Googlebot back in so it could see a noindex, nofollow robots meta tag, repeated for other bots until the site dropped out of the index.","content_type":"blog_post","language":"en","canonical_url":"https://kennyqin.com/deindexed-by-google/","author":{"name":null,"url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Citron","url":"https://kennyqin.com/","listing_slug":null,"listing":null},"topics":[{"name":"SEO","slug":"seo","url":"https://listedarticles.com/topics/seo"},{"name":"Web","slug":"web","url":"https://listedarticles.com/topics/web"},{"name":"AI Scraping","slug":"ai-scraping","url":"https://listedarticles.com/topics/ai-scraping"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":225,"reading_minutes":1,"published_at":"2026-10-07T11:15:00.000Z","added_at":"2026-10-08T08:08:12.847Z","updated_at":"2026-10-08T08:08:12.847Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/ive-been-deindexed-by-google","markdown_url":"https://listedarticles.com/articles/ive-been-deindexed-by-google.md","example":false,"citation":"Citron. \"I've Been Deindexed by Google!.\" 7 Oct 2026. https://kennyqin.com/deindexed-by-google/ (all-rights-reserved)","access":{"human_view":"full","full_text_available":true,"source_url":"https://kennyqin.com/deindexed-by-google/"},"body_markdown":"I don't remember exactly when but, earlier in the year, I decided I didn't want this website to be indexed by search engines anymore. A lot of it has to do with AI and the relentless scraping that plagues the internet now. I know that as long as my blog is accessible by the public internet, there are no real measures I can put in to stop it but at least some of it will be respected.\n\nI started out by setting a blanket disallow in the robots.txt.\n\n```\nUser-agent: *\nDisallow: /\n```\nWhen I realised that didn't do anything after a few weeks of waiting and checking, I set the `robots` meta tag.\n\n```\n\n```\nI thought that would do the trick. What I didn't realise is that you have to actually, ironically, **allow** the bot that will do the deindexing to visit your website so that it can see the meta tag that tells it to not index your page anymore.\n\n```\nUser-agent: Googlebot\nAllow: /\nUser-agent: *\nDisallow: /\n```\nI also did this with several other common bots and removed them from the robots.txt whenever I found that my website had been deindexed.\n\nIt's been at least 3 months since I did my last change to have Google deindex me and, after checking today, I'm happy to announce that this website is free from Google! 🥳\n","body_html":"

I don't remember exactly when but, earlier in the year, I decided I didn't want this website to be indexed by search engines anymore. A lot of it has to do with AI and the relentless scraping that plagues the internet now. I know that as long as my blog is accessible by the public internet, there are no real measures I can put in to stop it but at least some of it will be respected.

\n

I started out by setting a blanket disallow in the robots.txt.

\n
User-agent: *\nDisallow: /
\n

When I realised that didn't do anything after a few weeks of waiting and checking, I set the robots meta tag.

\n
<meta name="robots" content="noindex, nofollow" />
\n

I thought that would do the trick. What I didn't realise is that you have to actually, ironically, allow the bot that will do the deindexing to visit your website so that it can see the meta tag that tells it to not index your page anymore.

\n
User-agent: Googlebot\nAllow: /\nUser-agent: *\nDisallow: /
\n

I also did this with several other common bots and removed them from the robots.txt whenever I found that my website had been deindexed.

\n

It's been at least 3 months since I did my last change to have Google deindex me and, after checking today, I'm happy to announce that this website is free from Google! 🥳

","headings":[]}}