A lot of the people reading this blog now aren't people. They're agents: Claude, ChatGPT, Perplexity and friends, fetching a page because somebody asked them a question. This week I spent an evening making Deallocated Objects nice to read for them (and for search engines while I was in there).
The blog is a static site served by nginx, so everything here is either a file generated at build time or a few lines of nginx config. No framework required.
Markdown For Every Post
HTML is a lot of noise for an agent to wade through. Every post now also exists as plain markdown. Add .md to any post URL:
https://deallocatedobjects.com/posts/the-art-of-the-reveal.mdEach file starts with a little YAML frontmatter (title, canonical URL, author, dates, a short description), then the post. The older posts were written in markdown to begin with, so the build just cleans those up and points their images at the self-hosted copies.
The nicer trick is content negotiation. If a client asks for markdown in its Accept header, it gets markdown from the normal URL, no .md needed. In nginx that's a map plus one line per location:
# http context
map $http_accept $deallocated_md {
default "";
"~*text/markdown" ".md";
}
# server block
location ~ ^/posts/[^/.]+$ {
if ($deallocated_md) { rewrite ^ $uri.md last; }
try_files $uri.html =404;
}Try it:
curl -H "Accept: text/markdown" https://deallocatedobjects.com/posts/the-art-of-the-revealTwo details matter here. Every response sends Vary: Accept, so a cache in the middle never hands HTML to an agent or markdown to a browser. And the .md responses send a Link: <...>; rel="canonical" header pointing at the HTML page, plus X-Robots-Tag: noindex, so Google doesn't treat them as duplicate copies of every post.
llms.txt
/llms.txt is a plain markdown file at the root of the site that tells an agent what it's looking at. Mine has a one-line summary of the blog, how to get markdown, where the feeds are, how I'd like posts cited, and a list of every post with a link to its markdown and a one-line summary.
There's also /llms-full.txt, which is the full text of every post in one file (about 225 KB), and /index.md, which is the homepage as markdown. Every response also carries a Link header pointing at llms.txt, so an agent can find it without guessing.
robots.txt
Silence in robots.txt technically means "allowed", but I'd rather be explicit. Mine allows everyone, then names the big ones explicitly (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and about a dozen more) and adds a Content-Signal line:
User-agent: *
Allow: /
Content-Signal: search=yes, ai-input=yes, ai-train=yesThat's the right call for a blog. I want to be cited. A product site might choose differently.
Feeds
The old Atom feed at /feed still works, and every entry now has a summary and a text/markdown alternate link. I also added a JSON Feed, where each item carries a _markdown_url (JSON Feed allows custom keys that start with an underscore).
Meta Tags
The old site had almost none of these, so every page got the full set:
- a real meta description, a canonical URL, a robots tag and
lang="en" - Open Graph and Twitter card tags, including
article:published_timeandmodified_time - JSON-LD structured data:
BlogPostingand a breadcrumb on every post, withWebSite,Blogand aPersonfor me on the index pages
The descriptions were sneakier than I expected. The old database stored a pre-made description for each post, but it had run the paragraphs together ("...your app.The RevealWhat do I mean...") and a few contained raw <img> HTML. Now the build makes its summaries from the rendered post text instead.
Two more fixes from the old 2014 markup. The viewport tag had maximum-scale=1, which stops people from pinch-zooming on their phones, so that's gone. And there was no og:image at all, so when I shared a post, X and Threads grabbed the first image on the page, which happens to be the Little Bites of Cocoa banner at the top. Every post has a proper 1200x630 preview card now.
IndexNow
Last, an IndexNow key and a script that pings Bing and friends whenever a post is published. Google doesn't participate, but Bing's index feeds a lot of AI search tools, so it's worth a 30-line script.
None of this took long, and most of it is just files. If you have a blog, llms.txt and .md versions of your posts are a nice place to start. Here's this post:
curl -H "Accept: text/markdown" https://deallocatedobjects.com/posts/making-my-blog-readable-by-ai-agentsThe rest of how this site is built is in the colophon.