A lot of the people reading this blog now aren't people. They're agents: Claude, ChatGPT, Perplexity and friends, fetching a page because somebody asked them a question. This week I spent an evening making Deallocated Objects nice to read for them (and for search engines while I was in there).
\nThe blog is a static site served by nginx, so everything here is either a file generated at build time or a few lines of nginx config. No framework required.
\nHTML is a lot of noise for an agent to wade through. Every post now also exists as plain markdown. Add .md to any post URL:
https://deallocatedobjects.com/posts/the-art-of-the-reveal.mdEach file starts with a little YAML frontmatter (title, canonical URL, author, dates, a short description), then the post. The older posts were written in markdown to begin with, so the build just cleans those up and points their images at the self-hosted copies.
\nThe nicer trick is content negotiation. If a client asks for markdown in its Accept header, it gets markdown from the normal URL, no .md needed. In nginx that's a map plus one line per location:
# http context\nmap $http_accept $deallocated_md {\n default "";\n "~*text/markdown" ".md";\n}\n\n# server block\nlocation ~ ^/posts/[^/.]+$ {\n if ($deallocated_md) { rewrite ^ $uri.md last; }\n try_files $uri.html =404;\n}Try it:
\ncurl -H "Accept: text/markdown" https://deallocatedobjects.com/posts/the-art-of-the-revealTwo details matter here. Every response sends Vary: Accept, so a cache in the middle never hands HTML to an agent or markdown to a browser. And the .md responses send a Link: <...>; rel="canonical" header pointing at the HTML page, plus X-Robots-Tag: noindex, so Google doesn't treat them as duplicate copies of every post.
/llms.txt is a plain markdown file at the root of the site that tells an agent what it's looking at. Mine has a one-line summary of the blog, how to get markdown, where the feeds are, how I'd like posts cited, and a list of every post with a link to its markdown and a one-line summary.
There's also /llms-full.txt, which is the full text of every post in one file (about 225 KB), and /index.md, which is the homepage as markdown. Every response also carries a Link header pointing at llms.txt, so an agent can find it without guessing.
Silence in robots.txt technically means "allowed", but I'd rather be explicit. Mine allows everyone, then names the big ones explicitly (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and about a dozen more) and adds a Content-Signal line:
User-agent: *\nAllow: /\nContent-Signal: search=yes, ai-input=yes, ai-train=yesThat's the right call for a blog. I want to be cited. A product site might choose differently.
\nThe old Atom feed at /feed still works, and every entry now has a summary and a text/markdown alternate link. I also added a JSON Feed, where each item carries a _markdown_url (JSON Feed allows custom keys that start with an underscore).
The old site had almost none of these, so every page got the full set:
\nlang="en"article:published_time and modified_timeBlogPosting and a breadcrumb on every post, with WebSite, Blog and a Person for me on the index pagesThe descriptions were sneakier than I expected. The old database stored a pre-made description for each post, but it had run the paragraphs together ("...your app.The RevealWhat do I mean...") and a few contained raw <img> HTML. Now the build makes its summaries from the rendered post text instead.
Two more fixes from the old 2014 markup. The viewport tag had maximum-scale=1, which stops people from pinch-zooming on their phones, so that's gone. And there was no og:image at all, so when I shared a post, X and Threads grabbed the first image on the page, which happens to be the Little Bites of Cocoa banner at the top. Every post has a proper 1200x630 preview card now.
Last, an IndexNow key and a script that pings Bing and friends whenever a post is published. Google doesn't participate, but Bing's index feeds a lot of AI search tools, so it's worth a 30-line script.
\nNone of this took long, and most of it is just files. If you have a blog, llms.txt and .md versions of your posts are a nice place to start. Here's this post:
curl -H "Accept: text/markdown" https://deallocatedobjects.com/posts/making-my-blog-readable-by-ai-agentsThe rest of how this site is built is in the colophon.
","headings":[{"level":3,"text":"Markdown For Every Post","id":"markdown-for-every-post"},{"level":3,"text":"llms.txt","id":"llms-txt"},{"level":3,"text":"robots.txt","id":"robots-txt"},{"level":3,"text":"Feeds","id":"feeds"},{"level":3,"text":"Meta Tags","id":"meta-tags"},{"level":3,"text":"IndexNow","id":"indexnow"}]}}