{text}
;\n });\n}\n`\n\n#### plain text\n\n#### semfont\n\nOr skip React and take the scores. `analyze` is the engine alone, four numbers per word, no CSS, and these imports work with no React installed:\n\n`import { analyze } from 'semfont/analyze';\nimport { styleFor, themes } from 'semfont/theme';\n\nconst { tokens } = analyze('The rollback failed too.');\ntokens[4]; // { text: 'failed', valence: -0.7, salience: 0.23, surprise: 0.06, certainty: 0, ... }\nstyleFor(tokens[4], themes.editorial).style; // { color: 'color-mix(in oklab, currentColor, oklch(0.58 0.19 25) 53%)' }\n`\n\nThat last form is how this page works. There is no bundler here, so one import map tells the browser where `semfont/analyze` and `semfont/theme` live, pinned to a version on npm, and the demo box above is the same three lines as the React component: analyze, style, render.\n\n`\n\n`\n\n## Next Steps\n\nAs you can probably guess based on the implementation it will be essentially impossible to get perfect classification while also being fast enough to not slow down the streaming. However for the purpose of making text easier to read it doesn't have to be perfect and some simple heuristics may end up taking us far enough away. That being said there are some case like sarcasm which would be interesting to try to tackle with heuristics and some cases like negation with embedded clauses which may be possible to parse out. Furthermore, for streaming coding agents there are technical words which could be worth including in the vocabulary.\n\nCode and demo at github.com/RohanAdwankar/semfont.","body_html":"semfont is a small library that sets typography automatically.\nAs you can see below it automatically highlights, colors, bolds, and italicizes text which aims to make it easier to read.
\ntheme\n editorial\n loud\n monochrome\n technical
\nEvery word gets four scores. Each one starts as a dictionary lookup and is then adjusted by a couple of rules over the words around it.
\nValence is how good or bad the word is, from -1 to 1. A negator up to three words back flips the sign and damps it, because not great is a mild complaint rather than the mirror image of praise. An intensifier up to two words back scales it instead.
let v = VALENCE[word] ?? 0; // great -> 0.75\nif (negatorWithin(3)) v = -v * 0.74; // not great -> -0.55\nv *= gain; // really great -> 0.98\n
Salience is how much the word is worth looking at, 0 to 1. A frequency list gives each word a rarity, 0 for one of the hundred most common English words and 1 for one it has never seen. Rarity alone is not enough, so the score also rises with how often the word repeats in this particular text: an uncommon word you keep saying is what the text is about.
\n`const seen = Math.min(1, (repeats - 1) / 2);\nconst repetition = 0.45 + 0.55 * seen;\nlet s = SALIENCE[word] ?? 0;\ns = Math.max(s, 0.55 * rarity * repetition);
\n// rarity('the') 0.00, rarity('kubelet') 0.93\n// kubelet said once -> 0.23\n// kubelet said 3 times -> 0.51\n`
\nSurprise is where the sentence turns, 0 to 1. Some words announce it on their own, like suddenly or ironically. Otherwise it comes from position: everything for six words after a contrast word gets it, decaying with distance, and so does any word much rarer than the rest of the passage.
`let s = SURPRISE[word] ?? 0; // suddenly -> 0.85\nif (afterContrast) {\n s = Math.max(s, 0.45 * 0.82 ** (distance - 1));\n}\ns += 0.3 * Math.max(0, rarity - passageMeanRarity - 0.25);
\n// 'The tests failed' -> failed 0.16\n// 'It compiled, but the tests failed' -> failed 0.44\n`
\nCertainty is how sure the writer sounds, -1 hedged to 1 asserted. Words like probably and definitely are in a table. But if you write The build probably failed, you are not unsure about the word probably, you are unsure about whether it failed. So the hedge keeps its own score and every other word in the sentence gets 55% of it, and the whole line leans a little instead of one word in the middle of it.
`c = CERTAINTY[word] ?? sentenceCertainty * 0.55;
\n// 'The build probably failed.'\n// probably -0.40, every other word -0.22\n`
\nThen a theme maps each score to one typographic axis: valence to colour, salience to weight, surprise to a highlight, certainty to slant. Each axis has a threshold, so most words come out untouched.
\nThose rules only look a few words either side, and that window has a blind spot. Write I would not go so far as to call the new editor great and great stays green, because the not that cancels it sits nine words back. Write We fixed the crash and you get one green word and one red one, because nothing connects fixed to the thing it fixed.
So a second pass now runs after the window rules and reads each clause as a whole. A negator reaches to the end of its clause and fades with distance, which turns great red. A verb like fixed, recovered or avoided marks whatever s it as the thing that got better, which turns crash green. The same pass reads less broken and fewer complaints as improvements, too simple as a complaint, and a lone Great, in front of bad news as sarcasm.
Every change it makes is recorded on the word, so you can ask why a word came out the colour it did:
\nanalyze('We fixed the crash.').tokens[6];\n// { text: 'crash', valence: 0.44,\n// notes: ['resolved by "fixed"'] }\n
It costs about as much as the first pass and stays inside the budget, so there is no switch to flip. These are the ten sentences that led to it, including those two. Left is the first version, right is now.
\nbeforenow\nGreat, another outage. Just what I needed today.Great, another outage. Just what I needed today.\nWe fixed the crash and closed the security hole before anyone noticed.We fixed the crash and closed the security hole before anyone noticed.\nThe cluster recovered from the crash in under a minute.The cluster recovered from the crash in under a minute.\nWe avoided a catastrophic outage by catching the bug in staging.We avoided a catastrophic outage by catching the bug in staging.\nI would not go so far as to call the new editor great.I would not go so far as to call the new editor great.\nLess broken than last week, and far fewer complaints.Less broken than last week, and far fewer complaints.\nThe memory leak is gone.The memory leak is gone.\nThe reviewer called it "terrible", which is wrong.The reviewer called it "terrible", which is wrong.\nThe API is too simple and the docs are too clever.The API is too simple and the docs are too clever.
\n`import { SemanticText } from 'semfont';
\n<SemanticText as="p">\n The migration ran clean on staging. In production it deleted the index,\n and the rollback failed too.\n</SemanticText>\n`
\nPick a theme, or only the channels you want:
\n<SemanticText text={incident} theme="monochrome" />\n<SemanticText text={incident} channels={['valence']} />\n
Teach it your own vocabulary:
\n<SemanticText\n lexicon={{ valence: { flaky: -0.7, oncall: -0.4 }, salience: { rollback: 0.8 } }}\n text={incident}\n/>\n
Now for the case I started this for. AI system stream in a large amount of text and its hard to read all of it so the intention of this is a library that can easily be tossed into most streaming components to make the text easier to read:
\n`import { useChat } from '@ai-sdk/react';\nimport { SemanticText } from 'semfont';
\nfunction Chat() {\n const { messages } = useChat();\n return messages.map((m) => {\n const text = m.parts.filter((p) => p.type === 'text').map((p) => p.text).join('');\n return m.role === 'assistant'\n ? <SemanticText key={m.id} as="p" text={text} />\n : <p key={m.id}>{text}</p>;\n });\n}\n`
\nOr skip React and take the scores. analyze is the engine alone, four numbers per word, no CSS, and these imports work with no React installed:
`import { analyze } from 'semfont/analyze';\nimport { styleFor, themes } from 'semfont/theme';
\nconst { tokens } = analyze('The rollback failed too.');\ntokens[4]; // { text: 'failed', valence: -0.7, salience: 0.23, surprise: 0.06, certainty: 0, ... }\nstyleFor(tokens[4], themes.editorial).style; // { color: 'color-mix(in oklab, currentColor, oklch(0.58 0.19 25) 53%)' }\n`
\nThat last form is how this page works. There is no bundler here, so one import map tells the browser where semfont/analyze and semfont/theme live, pinned to a version on npm, and the demo box above is the same three lines as the React component: analyze, style, render.
<script type="importmap">\n{ "imports": {\n "semfont/analyze": "https://cdn.jsdelivr.net/npm/semfont@0.2.0/src/analyze.js",\n "semfont/theme": "https://cdn.jsdelivr.net/npm/semfont@0.2.0/src/theme.js"\n} }\n</script>\n<script type="module">\n import { analyze } from 'semfont/analyze';\n import { styleFor, themes } from 'semfont/theme';\n</script>\n
As you can probably guess based on the implementation it will be essentially impossible to get perfect classification while also being fast enough to not slow down the streaming. However for the purpose of making text easier to read it doesn't have to be perfect and some simple heuristics may end up taking us far enough away. That being said there are some case like sarcasm which would be interesting to try to tackle with heuristics and some cases like negation with embedded clauses which may be possible to parse out. Furthermore, for streaming coding agents there are technical words which could be worth including in the vocabulary.
\nCode and demo at github.com/RohanAdwankar/semfont.
","headings":[{"level":1,"text":"A font that reads what you wrote","id":"a-font-that-reads-what-you-wrote"},{"level":2,"text":"How it works","id":"how-it-works"},{"level":2,"text":"Improving the algorithm","id":"improving-the-algorithm"},{"level":2,"text":"Using it","id":"using-it"},{"level":2,"text":"Next Steps","id":"next-steps"}]}}