Karpathy's tips for understanding LLM output, and the ASD-STE100 cheat sheet that gets the standard wrong
Andrej Karpathy's ladder for reading model output runs from controlled English to diagrams, HTML pages and explainer videos. The cheat sheet attached to his post summarizes ASD-STE100 cleanly and confidently, and gets three things wrong. I go through what the standard asks for, rewrite one answer before and after, and give a prompt that keeps identifiers and hedges intact.
Contents
On 2 October Andrej Karpathy posted a short guide to a problem most of us now have every day. It starts like this:
We'll be spending a lot more time trying to understand the outputs of language models.
A few thoughts, tips & tricks:
When I captured it on 4 October it showed 6.4 million views. What follows is a ladder of four output formats, each step up announced with “But even better:”, and an image: a one-page, engineering-drawing summary of ASD-STE100, the controlled English that aircraft maintenance manuals are written in.
I read the post, then read the standard it recommends, then checked the image against the standard. The layout makes the rules easy to scan, most of it is right, and three of its dictionary rows would teach a reader the wrong thing. That turns out to be the most useful part of the whole thread, because it is the post’s own subject, demonstrated by its own attachment.
If you came here for ASD-STE100 itself, the rules at a glance, a before-and-after example and a copyable prompt follow the section on what the standard is.
What did Karpathy recommend for understanding LLM output?
Four formats, in increasing order of ambition. In his words, trimmed:
Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100,
it's a controlled language specification originally developed for aerospace maintenance
documentation.
Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot
easier to process, parse, and understand.
Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage.
Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer
videos generated on any arbitrary topic.
For the writing rung he adds a practical softening: he sometimes asks for “80% of the way to ASD-STE100” because “the spec is quite stringent.” For video he gives a sample prompt, “Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration”, and notes that you can instead ask the model to find “decent free alternatives that use your local compute.”
The summary is the actual thesis. As models do more of the work, “a lot more of our work will rise up the abstractions into oversight and understanding.” Because intelligence and code are getting cheap, “you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before.” He has been heading here for a while: his 2025 year in review already called code “discardable after single use” and said models should speak to us in our favored format.
I agree with the direction. The rest of this piece is about what each format lets you check.
What is ASD-STE100, and why ask an LLM to write in it?
ASD-STE100 is Simplified Technical English, a controlled language that began in 1979, when European airlines asked aircraft manufacturers for maintenance documentation that a mechanic reading in a second language could not misread. The first guide appeared in 1986. It is maintained by ASD, the European aerospace and defence association, and the current version, Issue 9 of 15 January 2025, turned it from a specification into an international standard.
It has two parts. The writing rules: 53 of them in nine sections, covering words, multi-word nouns, verbs, sentences, procedural and descriptive writing, safety instructions, punctuation and writing practices. And a dictionary of about 900 approved words, generally one meaning and one part of speech each, plus about 1,200 unapproved words with approved alternatives. Writers can also use technical nouns and technical verbs from defined categories, so STE is not confined to those 900 words. The limits most people quote are real: 20 words in an instruction, 25 in a description, active voice, no semicolons. The next section lists them with their rule numbers.
Why it suits a model: the rules remove exactly the habits that make LLM prose tiring. No “it is imperative that,” no “prior to commencing,” no stacked clauses, and a strong preference for saying who does what. Karpathy’s point is that models know the style well and that the result is “a lot more readable.” And for human readers, the evidence that it helps is real: a 1996 study of 175 aircraft maintenance technicians found comprehension improved with Simplified English, most for the hardest work cards and for non-native readers. Readable is not the same as human-sounding, which is a separate problem I tested in how to make Claude write like a human.
Before you use it, know what you are borrowing:
- It is free to obtain and restricted to redistribute. You request a copy through the official downloads page; the document is ASD’s copyright, with reproduction limited to the permissions in its notice.
- Its own FAQ says it “is not intended for general-purpose writing,” while allowing that its principles, such as short sentences and the active voice, travel well. Karpathy’s “80% of the way” is that same compromise.
- It is hard to write well: “STE was created for the maximum benefit of the reader. This does not necessarily mean that it is simple to write.”
Karpathy did not start this. A wave of “make the model write in STE” skills hit GitHub and Hacker News in July 2026, two of them with more than 3,000 stars each. His post put a much larger audience on it.
ASD-STE100 rules at a glance: the rules you can check without the dictionary
The dictionary can’t be summarized fairly in a table, and it is ASD’s copyright, so I won’t try. The writing rules are a different matter. These are the ones a reader, a reviewer or a script can check without it, with their numbers in Issue 9:
| Rule (Issue 9) | What it asks |
|---|---|
| 5.1 | Procedural sentences: 20 words at most |
| 5.2 | One instruction per sentence, unless the actions happen at the same time |
| 6.3 | Descriptive sentences: 25 words at most |
| 6.6 | Descriptive paragraphs: six sentences at most |
| 2.1 | Multi-word nouns: three words at most |
| 3.2 | Verb forms: infinitive, imperative, simple present, simple past, simple future, and the past participle used as an adjective |
| 3.5 | An “-ing” form only as a technical noun or as a modifier in one |
| 3.6 | Active voice; in descriptive writing, the passive only when the agent is unknown |
| 4.2 | No contractions, and no words left out to make a sentence shorter |
| 8.1 | Any standard punctuation except the semicolon |
What the table leaves out is the part a model is worst at: rules 1.2 and 1.3, approved words used only as their listed part of speech and only with their listed meaning. That is where the cheat sheet below goes wrong, and where “80% of the way” quietly lets a model off.
ASD-STE100 example: an LLM answer before and after
Here is a paragraph of the kind models write every day. I made it up for this example, but you have seen its cousins:
In order to ensure that the deployment doesn't fail, it is imperative that the cache is
invalidated prior to running the migration. The stale reads you've been seeing in prod-eu-1
may possibly be caused by the cache, but this hasn't been verified yet; it's also worth
noting that CACHE_TTL=900 is only set on the EU nodes.
And the same content at about 80% of the way to ASD-STE100:
Invalidate the cache before you run the migration. If you do not invalidate it, the deployment
can fail.
The cache is possibly the cause of the stale reads in prod-eu-1. This is not verified yet.
Only the EU nodes have CACHE_TTL=900.
The instruction comes first and stands alone (rules 5.1 and 5.2). The passives are gone, contractions with them, and “in order to” became nothing at all. The 34-word sentence with a semicolon is now three sentences, and “hasn’t been verified” became “is not verified”: a past participle doing the job of an adjective, which rule 3.2 allows.
The parts that matter most are the ones that did not change. prod-eu-1 and CACHE_TTL=900 are copied character for character. “Only the EU nodes” keeps its “only”. And the cause stays a possibility that is “not verified yet”, not a finding. A careless simplifier writes “The cache causes the stale reads”, which is shorter, cleaner and wrong. STE invites that trade. Most of the prompt below exists to refuse it.
ASD-STE100 prompt for LLMs: 80% mode and strict mode
Karpathy’s “80% of the way” works better when you say which 80%. The version below keeps the countable rules and lets only the vocabulary slide. It also adds an instruction no style guide gives human writers, because they don’t need it: do not lose anything.
Rewrite the text below in ASD-STE100 Simplified Technical English (Issue 9), about 80% of the way.
Follow these rules exactly. Only the vocabulary may be looser than the standard:
- Instructions: 20 words at most, one instruction per sentence unless actions are simultaneous.
- Descriptions: 25 words at most per sentence, six sentences at most per paragraph.
- Active voice. Passive only in descriptive text when you do not know who does the action.
- Verbs: infinitive, imperative, simple present, simple past, simple future, or a past
participle used as an adjective. No -ing verb forms.
- Multi-word nouns: three words at most. No semicolons. No contractions.
Keep identifiers, commands, paths, error strings, numbers and versions exactly as written.
Keep every condition and qualifier ("only", "per user", "older than 72 hours").
Keep every uncertainty at its original strength: "may" or "possibly" never becomes "is",
and "not verified" stays "not verified". Add nothing that is not in the source.
Do not claim the result is STE-compliant.
For strict mode, used for procedures and warnings, delete “about 80% of the way” and the sentence about looser vocabulary, then add two lines: use only words approved in the ASD-STE100 dictionary, as their approved part of speech and with their approved meaning, plus technical nouns and technical verbs as the writing rules define them; and mark every word you are not sure about. The second line matters more than the first. A model does not have the dictionary, and a request for “approved words only” without it produces the confident errors in the cheat sheet below.
I use a version of this prompt in my own agent setup: task notes for my task list are written in the 80% mode, and a small script checks sentence length, semicolons, passive candidates and whether every identifier and hedge from the source survived. It can’t check vocabulary or meaning, and every report it prints ends by saying so. If you copy one idea from my setup, copy that disclaimer.
Does the ASD-STE100 cheat sheet in Karpathy’s post get the standard right?
Many details are right, and several entries would teach the wrong rule. Here is the image attached to the post:
Image from Karpathy’s post on X, 2 October 2026, reproduced for review; its creator is not stated. It is not an ASD publication.
I checked it panel by panel against Issue 9, and where useful against Issue 8. A lot of it holds: every numeric limit (20, 25, 6, 3, one instruction per sentence), the six approved verb forms, the two-part, nine-section structure, the uppercase convention for approved words, and the example sentences with their word counts. Seven of the ten dictionary rows have the right word and status. A tired reader would trust it, and mostly should.
Now the dictionary panel, where it goes wrong:
Detail of the same image.
- approximately is approved, and the sheet says it isn’t. The row marks “approximately (adv)” as not approved and offers ABOUT instead, with “Wait for ABOUT 10 minutes” as the correct form. In the standard it is the other way round. APPROXIMATELY is an approved adverb meaning “almost correct or accurate.” ABOUT is approved only as a preposition meaning “concerned with,” and the standard uses this exact pair as a teaching example: “Drain about 2 liters of fuel from the tank” is its example of what not to write. The sheet teaches the error the standard warns against.
- TEST is not an approved verb. The row marks “TEST (v)” approved, with a definition, “To find if it operates correctly,” that I could not find in the standard. TEST is approved only as a noun; the standard’s own example is “DO A FUNCTIONAL TEST OF THE SOFTWARE,” with “Functionally test the software” as the version to avoid. Someone quoted this exact rule on Hacker News in July.
- “in order to” has no entry I could find. Shortening it to TO is sensible advice and in the spirit of the standard, but I found no such row in the dictionaries of Issue 9 or Issue 8. The sheet presents a recommendation in the format of a dictionary entry.
Outside the dictionary there are two more substantive problems. The verb panel says descriptive text “can use the passive only when it is necessary.” The rule is narrower: in descriptive writing “you can use the passive voice only when the agent is unknown.” “When necessary” is a judgement call; “when the agent is unknown” is a test. And the structure panel lists technical names and technical verbs as dictionary content, while the FAQ says plainly that the dictionary does not list them; they are categories defined in the writing rules.
There is also a quieter signal across the sheet: its terminology predates Issue 9 and matches Issue 8. It says “noun clusters” (now “multi-word nouns”), “technical name” (now “technical noun”), “Specification” in the title block (now a standard), and its history timeline stops before the 2025 change. One section title, “Procedures,” matches neither edition.
The post doesn’t say who made the image or how, and whatever its origin, the result is the same: useful summaries next to incorrect dictionary advice, in a layout that makes both look equally authoritative. It is already being used as a source. One repository created the day of the post takes its limits from “Karpathy’s sheet” (the limits happen to be the correct part). And a separate STE skill written in July, before the post, has an open issue listing twelve rows that ban approved words, approximately among them, so machine-built STE word tables fail the same way independently. Readers did catch part of this: about 20 hours after the post, a reply flagged the approximately row by pasting a ChatGPT answer, and a GitHub reference file documented the same error and the older terms. The TEST row, the “in order to” row and the passive-voice paraphrase I found no one flagging.
The standard’s own maintainers saw this coming. In June 2026 they published a white paper on STE and AI, and the downloads page summarizes it in two sentences I would put above every AI style guide:
AI-generated text can appear clear, authoritative, and consistent with STE, even when it
does not correctly apply the rules and vocabulary of the standard. Plausibility must not
be confused with verified compliance.
Do LLMs actually follow ASD-STE100 when you ask?
Loosely, and they tend to believe they follow it better than they do.
A technical writer who looked closely calls what models produce “STE-flavored English, not STE”: without checking against the current dictionary, a model can imitate the style while breaking its vocabulary rules. In the replies under Karpathy’s post, one person reports that three current models “simply ignore” the request; others saw little difference between the STE answer and the normal one. On Hacker News in July, a recurring view was that compliance drifts quickly and only a linter or a commit hook holds it.
The measurements are thin. The most-starred STE skill advertised that it “cuts slop 72.9%”; that number counted violations of the skill’s own linter rules, and in a later eight-scenario audit of version 2.0.0 its author found that a short prompt produced shorter replies with fewer formatting defects. That was not a comprehension study, and the project has since rebuilt the skill. Lucian Ghinda ran a small, explicitly informal test on code explanations: with Claude, a loose “simple technical English” prompt mentioned 8.5% fewer of the scored facts than the baseline answer, and a strict ASD-STE100 prompt 46.8% fewer; with Codex the corresponding losses were 43.3% and 40.0%. Four code samples, two sessions. Treat it as a warning that simplification can drop facts, not as a ranking. Two 2026 preprints on constraint following find what you would expect: models overestimate their own compliance, and they can restate a rule while breaking it.
The practical conclusion is the one I made about prompts in general: a style rule in a prompt is a request, not a guarantee. If you actually need STE, check it outside the model. The prose linter Vale can encode the rules you choose to enforce, which also replaces “80%” with a list you decided on. A linter checks the rules you configured; it doesn’t certify full STE compliance, or that the text is true.
Are diagrams better than text for understanding LLM output?
Larkin and Simon put the answer in the title of their 1987 paper: “Why a Diagram is (Sometimes) Worth Ten Thousand Words.” A diagram holding the same information as text can be far cheaper to use, because location does the indexing that a reader would otherwise do in their head. A 2025 meta-analysis by Cromley and Chen, across 181 studies of multimedia learning, found an overall effect around 0.37, with large variation by design principle and outcome; text-and-diagram results were among the more consistent ones, animation much less so.
Two cautions specific to models. Generated diagrams are still unreliable as diagrams: benchmarks on SVG, Mermaid and similar formats show real gaps. And a diagram has its own way of overstating. One reply put it well: when you simplify, protect words like “if,” “unless” and “not yet verified,” because without them “a clearer sentence can become a stronger claim,” and the same goes for a diagram, where a missing arrow quietly asserts that there is no dependency. The counter-case also came up: someone caught a real error because one arrow pointed the wrong way, which the prose had hidden. Both are true. A diagram makes structure visible, including wrong structure, and makes omissions invisible.
How I would use it: ask for a diagram in a text format you can read and diff (Mermaid, Graphviz, a simple SVG), look at the rendered result, and ask the model to list every relationship it drew as a sentence underneath. Then read the list.
Why ask an LLM for output in HTML?
Bret Victor described the goal in 2011, in “Explorable Explanations”: “A reactive document allows the reader to play with the author’s assumptions and analyses, and see the consequences.” That is what an HTML answer offers that prose doesn’t: tables and sliders, collapsible sections, small simulations. It was the rung the replies liked best, and Claude’s artifacts and ChatGPT’s canvas brought generated content into dedicated workspaces in 2024.
The best suggestion in the thread, made independently by several people, was a variation: don’t ask for a page that shows the answer, ask for a page that lets you change one assumption and watch what breaks. That makes the explanation’s assumptions inspectable. It is not yet a test of the real thing. If a model explains a caching bug with an interactive page, the page’s toggle verifies the page’s model of the bug, not your system; to check the diagnosis, reproduce it with the actual code and inputs.
The costs: an HTML artifact takes more tokens and time than an answer, and it can be polished and wrong in the same way the cheat sheet is. Ask for a self-contained file with its assumptions listed visibly on the page.
Can an LLM make a 3Blue1Brown-style explainer video?
Increasingly, and this is where Karpathy is most excited and the evidence is thinnest. One way to get the “3b1b style” is Manim, the animation engine Grant Sanderson wrote for 3Blue1Brown (MIT licensed, about 94,500 stars), or ManimCE, the better-documented community fork; their setup differs, so say which one you want. A model writes Manim code, the code renders the animation, a text-to-speech service narrates it. In the replies, people did exactly that within a day: on stochastic calculus, on how payments work, one entirely with local tools and no API keys, which is the alternative Karpathy himself mentions.
There is real research behind the idea. TheoremExplainAgent (ACL 2025), evaluated on a benchmark of 240 theorems, found something encouraging: the videos exposed flaws in the model’s reasoning that its text explanations had hidden. Animation forces you to commit to what happens next. Whether that improves what viewers learn is a separate question it didn’t test.
Three practical notes. ElevenLabs, which Karpathy’s sample prompt names, logs the text you send by default, and turning that off is an enterprise feature, so keep private material out of hosted narration or use the local route. Put the key in the tool’s credential settings rather than in a prompt. And video is harder to scan without a transcript or chapters: in a study of 6.9 million viewing sessions across 862 edX videos, median engagement time topped out at about six minutes. Keep the script, the captions and the source project, so the claims stay searchable. One early reply put the risk in a sentence: “a wrong claim narrated over a 3b1b animation is way harder to catch than a wrong sentence.”
Does easier-to-read LLM output mean easier-to-verify output?
Not by itself. A clearer format can expose an error, as the arrow and the theorem videos did, and it can make an unsupported claim easier to accept. The useful question is what each format lets you inspect.
The research on the second effect is worth knowing. In a 2024 study (Si et al., NAACL), about 80 crowdworkers fact-checking claims with an LLM’s explanation were right 87% of the time when the explanation was correct and 35% when it was wrong, below the 49% they managed with no evidence at all on those cases. A CHI 2021 study found explanations increased people’s acceptance of AI advice whether or not the advice was right. Work on text simplification (Devaraj et al., ACL 2022) found factual errors common in simplified versions and argued that a readable but inaccurate version can be worse than no access at all. And the illusion of explanatory depth, our habit of overrating how well we understand mechanisms, makes me cautious about mistaking a visible mechanism for a complete explanation.
None of those studies compared Karpathy’s four formats, so I won’t claim each rung is harder to check than the last. What I’d say is narrower: each rung moves the claims somewhere new. Prose you can quote and search. A diagram puts them in arrows and absences. A page puts them in code and layout. A video puts them in timing and voice. The checking has to follow them there, and none of the four formats does it for you.
Karpathy’s summary says our work will rise “into oversight and understanding.” I think that is right, and I’d add that the two aren’t the same activity. Understanding a translation is not the same as verifying it, and the same is true of any output you couldn’t have produced yourself.
How should you use Karpathy’s techniques without being fooled?
Use all four rungs; they are good. Keep one thing on each rung that you can check.
For writing in STE or anything like it:
- Ask for the simplified version and keep the original next to it. Simplification should be a view, not a replacement.
- Tell the model to keep identifiers, parameter names, error strings and technical terms exactly as written, and to keep every “if,” “unless” and “not verified.” Those are the words a simplifier drops first.
- If the style matters, check it with a linter, not with the model’s word.
For diagrams: request a text format, look at the render, then read a sentence per edge.
For HTML: ask for the page that lets you change an assumption, with its assumptions listed, and test the real system separately.
For video: keep the script as text, read it once before you watch, and keep private material out of hosted narration.
And for anything you will act on: pick one load-bearing claim and check it against the primary source. That is how the cheat sheet comes apart. It took one look at one dictionary entry.
What does Karpathy’s post say about where work is going?
That the scarce skill is moving from producing output to judging it, and that models can help with the judging too, by producing artifacts that would never have been worth making by hand. The discardable explainer, the one-off interactive page, the video made for an audience of one: these are real and new.
What I would add is a distinction the post leaves implicit. Feeling that you understand something is not evidence that it is correct. The ladder is very good at the first. The second has to be built in on purpose, in whatever format you choose. The most widely shared summary of a standard for clear, unambiguous writing this week inverted one of that standard’s own teaching examples, in the clearest possible layout. I would keep the primary source open next to the beautiful thing.
01 / Discussion
No comment section here — all discussions happen on X.
02 / More articles
Auditing CLAUDE.md with Claude Code's /doctor: keep the reason in the file, the war story in git
The new audit treats every emphatic rule in your agent config as a hypothesis about a model you may no longer run. It asks where each rule came from, and in the same file it tells you to delete the story of where it came from. Both instructions are right once the reason and the history live in different places.
Claude Haiku 5.5 as a Claude Code subagent: effort, the 100K tier and the cost per task
How to route a Claude Code subagent to Haiku 5.5 on purpose, which effort level to pin it to, how to keep it under the long-prompt rate with /autocompact, and what the system card says to check before you trust it with a tool loop.
Claude Code vs Codex: seven ways developers split work between the two instead of picking one
The handoffs are vendor-documented now: a review command, a session transfer, an importer in each direction. Who should get which job is documented nowhere. Seven splits, each traced to its origin and marked documented or reported, plus the sandbox defaults that differ and the integrations that were removed this year.