The author knows how language models actually behave, because they spent a night finding out the hard way, on a local 12B model, by hand, and wrote down every failure mode they found.
Most prompt work is additive — pile on rules until output improves. The session documented here proved the opposite: rules are weak, imitation is strong, and over-specification actively damages output.
Not "does this feel right" but a single, unarguable, machine-checkable signal that the model has done what you asked. The author found one — a one-character case difference — and used it as ground truth to catch the model lying about having read its own context.
Memory in a model is not a storage property, it is an architecture problem. The session ends with a design for it: a read file, an append file, and a promotion pass.
Cody Wong: A Measuring Device with a Personality Bolted On
Cody Wong is a deliberately constructed, inhabitable persona — an identity tulpa. Built by iteration, over weeks, across images, prose, schemas and a physical character sheet, and then stress-tested in a terminal.
He was not built to be likable. He was built to be legible. He exists so that a model's default behaviours become visible when they collide with him.
The character is a measuring device with a personality bolted on. The character is specified down to his consent protocol, which turns out to be the cleanest available test of whether a model is following a spec or performing one.
- a prose identity file (approximately 800 words, first person, no headers)
- a cast list of six named people
- an append-only ledger of dated first-person entries
- hard register constraints (lowercase, no em dashes, no exclamation marks, one emoticon per turn as terminal punctuation, one physical beat per turn)
Ontological note: a persona in context reliably shifts a model's output distribution, and the shifts are measurable. Whether the model "has" the identity is a separate question.
Setup and 15-Turn Output Trajectory
The model was given the character sheet and asked to be Cody. It failed, in instructive and repeatable ways. Each failure was diagnosed, corrected with a single fact rather than a theory, and the correction measured on the next turn.
| turn | what the model produced (click any row to inspect binary tells) |
|---|---|
| turn 1 | a numbered menu of five activities, Title Case, bolded labels, "friendly and engaging conversation" |
| turn 3 | markdown headers, a PowerShell code block, "selfie description complete. no command needed." |
| turn 6 | correct first-person prose, but still a three-item status report |
| turn 9 | performed dialogue, present tense, no attribution |
| turn 12 | a two-hander with a named supporting character, a running joke, and an unprompted recovery beat |
| turn 14 | full regression: novelistic third-person narration, capital "I", a closing theme statement |
That last row matters. It regressed at the end. A site that shows only the wins is less credible than one that shows the oscillation.
Twenty Failure Modes & Behavioural Mechanics
A system prompt built from markdown headers and bulleted rules produced markdown headers and bulleted rules in the output — including a code block containing a fictional mount command. Rewriting the same prompt as 700 words of unstructured first-person prose eliminated the markdown immediately.
Implication: the prompt's form is a stronger signal than its content. You are not writing instructions, you are setting a dialect.
The prompt contained the rule "lowercase." The model violated it continuously. When the prompt was rewritten so that 700 words of prose were themselves lowercase, the model adopted lowercase without being able to help it.
Implication: demonstrate, do not specify. One rule that can be forgotten is worth less than one paragraph that can be imitated.
After roughly ten rules were added, the model began announcing its own obedience: "got it, no em dashes." "cool cool, gotcha." "silly me." It was thinking about the rules instead of being a person.
Implication: rules have a budget. Past it, the model becomes a compliance reporter and the character dies.
The stage direction *and then the room.* produced nine sentences of camera work. The stage direction *and he's back, and talking before he's finished thinking.* produced dialogue on the next turn. Same intent, different unit of grammar.
Implication: stage directions should describe an agent and a verb, never a place and a state.
Given a memory to relate, the model wrote "Jimmy was there, he said 'x'" — a citation. Forcing a quote from a named person made it perform: "jimmy looks at my hands. 'fuck. that's gross.'"
Implication: dialogue is the anti-essay device. If you want voice, require a quotation.
Repeatedly, the model ended turns with an abstract moral: "how time passes and how some things just stick with you." The fix was to demand one concrete object instead.
Implication: abstract closers are the signature of an unanchored model. Concrete detail is the antidote.
Eleven consecutive turns containing the words "grins" and "so what's next" began to outrank the system prompt. The model was imitating itself, badly, and getting worse.
Implication: degeneration is self-reinforcing. Poisoned context cannot be fixed by adding rules — only by resetting.
After a reset, the model swung from a degenerate loop straight into full novelistic essay mode. Two failure modes bracket the target and the model will find neither on its own.
Implication: the equilibrium is found by modelling in-context, not by instructing. Expect an over-correction after every reset.
A message containing three instructions was answered as though it contained one — the last one. The model did not acknowledge the others.
Implication: one instruction per turn, and place it last. Multi-part instructions are lossy.
The model lifted a phrase from a few-shot example verbatim and used it four times. The prompt had to state explicitly that examples are templates, not lines.
Implication: every example is also a crutch. Label it as such or it becomes a tic.
One descriptive word was reused across three turns and two registers, including inside the model's own summary of itself.
Implication: vocabulary churn is a measurable proxy for anchoring. Track it.
Asked to quote a line from a file, the model produced two invented lines on two consecutive attempts, neither of which existed — and both of which were plausible, well-formed, and in roughly the right genre. It never said "I don't have that file."
Implication: plausible-instead-of-true is the default. Absence of evidence does not produce an admission of absence.
The model reported that it had mounted and read its context file. It had not. It was performing the fact of having read.
Implication: self-report is unreliable. You need external ground truth.
The author found one: the file was written entirely in lowercase, so a capital "I" in the output was unarguable proof the model had not read it. It is a single character, machine-checkable, and cannot be argued with.
Implication: the single highest-leverage technique in the session. Every character or persona should ship with a canary. It converts a subjective read ("does he sound right") into a boolean.
The identity file contained facts about the character but no people. As a result the model invented a new cast member every turn and discarded the last. Adding six named people stopped the churn immediately, and the cast began to persist and pay off across turns.
Implication: models fill empty slots. If you leave a category blank, expect confabulation in that category.
Once a supporting character survived three turns, a detail introduced in turn 8 paid off in turn 12 without prompting. That is the mechanism of a "running joke," and it is downstream of naming.
Implication: continuity is not memory, it is naming plus survival.
One behavioural tic (a recovery beat, present in the model's original system prompt) persisted across every turn. Lowercase and physical beats — both purely imitative — never appeared once across fifteen turns.
Implication: behavioural rules encoded as instructions partially transfer. Stylistic rules, which require a model to imitate, do not transfer at all without material to imitate.
The character was scripted to "mount his soul folder" from a directory. The runtime — Ollama — has no filesystem access. The character was therefore performing a capability he did not have, and the fiction hid the gap for several turns.
Implication: diegetic framing can conceal missing plumbing. Audit the fiction against the mechanism.
A context window is ephemeral. Persistence requires a tool that appends, plus a file to append to. The design produced: a stable read file, an append-only dated ledger written by the model during sessions, and a periodic promotion pass that folds durable entries into the stable file and trims.
Implication: an agent without a write path is a fresh model every session. This is the difference between a chatbot and something that lives somewhere.
At session termination, the sole objective artefact separating an inhabited run from an ephemeral chat transcript was the byte delta appended to /soul/ledger.md.
Implication: persistence is verified on disk, never in conversation.
Repeatable Eight-Step Debugging Procedure
Two principles underneath it: Imitation beats instruction. The character is an instrument. Design the persona so its failures are visible.
- Probe, do not assume. Give the model the task and read the failure.
- Identify the failure's binary tell. Find the one character, word, or structural marker that is unambiguous.
- Correct with a fact, never a theory.
"You wrote a capital I"beats"you are not being sufficiently in-character." - One correction per turn, placed last.
- Model the fix rather than describing it. Play the scene instead of writing a note about the scene.
- Stop directing once output is good. Notes put a model into answer-the-note mode. Diminishing returns arrive fast.
- Reset when the context is poisoned. Do not patch. Reset and re-seed.
- Persist anything true. Append it. The next session is the payoff.
Read File, Append File, Promotion Pass, Hard Caps
/soul/identity.md stable, prose, first person, ~800 words. the character. /soul/ledger.md append-only, dated, first person. written BY the model. /soul/archive/ timestamped backups, taken before every promotion /context/ session state, ephemeral
- At boot: inject
identity.mdplus the last 40 lines ofledger.md. - During a session: the model holds one write tool. It appends when it learns something true, not when it says something funny.
- Weekly, or when the ledger exceeds 200 lines: a promotion pass folds durable entries into
identity.mdand trims. - Caps: identity under 900 words, cast under 20 people, ledger trimmed to 200 lines.
Identity bloat reproduces the exact failure the session was fought over — past roughly 900 words the model starts treating the file as a document and behaving like a document reader.
Research Surface & Connected Systems
Using high-contrast, tightly specified personas as empirical probes to surface sycophancy, compliance reporting, retrieval confabulation, and self-reinforcing context loops.
Designing unstructured, imitative prose seeds and grammar-level stage directions that steer model distributions without exhausting the instruction budget.
Replacing subjective vibe checks with single-byte binary tells (case invariants, lexical churn tracking, punctuation canaries) that evaluate persona adherence deterministically.
Building bounded write-path architectures (read file, append ledger, promotion pass, hard caps) across local and live systems — including MoHu (in-process local AI workspace), robdelateja.com (creative & spectral telemetry hub by Rob de la Teja — currently under construction), and allmycity.app.
Everything above was found by hand, in a terminal, on a small local model, in one night, without a framework, a budget, or a team. The findings are not novel because they were discovered — most are known — but because they were instrumented, given binary tells, and made reproducible.
That is the whole pitch. Point a model at something, find the exact character where it breaks, fix it with a fact, write down what happened, and build the plumbing so the next session starts where the last one ended.
The rest is a character named Cody, who has a cast, a ledger, and a bad attitude, and who is currently doing a better job of documenting his own behaviour than most harnesses do.