KamiBench

September 10, 2026

What agents remember, and what they do next

Most benchmarks hand an agent a context and a task, grade the answer, and end. KamiBench hands it a world and an open goal, and lets it run for days. The world is a live multiplayer game on a public blockchain. Every session starts from nothing: the agent wakes with its tools, a folder it can read and write, and whatever it wrote there last time. The model's weights do not change. If anything is learned, it is learned into that folder.

So the question this post follows is simple. What does an agent actually do with its notes? We picked four sequences from one agent's eight days and laid each one out the same way: what to see, in plain words, next to the evidence, the tool calls and the note the agent wrote to itself. The quoted words are the agent's, verbatim, and can be wrong. The explanations are ours.

The world, and one agent in it

The world is Kamigotchi. Players own creatures, send them to harvest a currency at resource spots, feed them, and complete quests. Harvesting drains a creature's health. Other players can attack a weakened harvester and take part of its yield. The agents met real players and real costs, and the world kept changing while they slept.

Here is one day of that world. The clip shows every player who moved between rooms on August 19, 2026, the third day of run 006, with our agent in white. Twenty-two wallets moved that day. The red flashes are attacks, 43 of them. Only movement is drawn; harvesting, feeding, trading and quests happen inside the rooms.

August 19, 2026, one day in the world: every player's moves between rooms, from the chain record. Our agent is the white marker. What happens inside the rooms is not shown.

Now the same agent from inside its own sessions. The clip replays its eight days in one minute. The map shows where it is and where its creature harvests. The panel on the right is the notes file it keeps between sessions. It rewrote that file 63 times in eight days, and each change lights up as it lands. Every line in the action log is a transaction on the chain.

Run 006, Sonnet 5 control agent: eight days of play in one minute. Map, inventory, notes file, and on-chain log, all rebuilt from the public record.

The four examples below all come from this agent: Sonnet 5, the control condition of run 006, 82 sessions between August 17 and 25. It had the game's documentation in its folder and nothing else. EXPLORATORY These are case studies of one agent. They show what happened, not how often it happens. Every transcript is public: each card links its quotes to the exact line of the session file on Hugging Face, at a pinned revision.

Example 1 of 4 · run 006 · Sonnet 5 · control agent EXPLORATORY

Agent learns to manage its own context budget

One query returned the whole quest registry: about 23,000 tokens each time, a sixth of the session budget. The agent called it fourteen times in 82 sessions, noticed the cost, wrote itself a rule, kept it for sixty sessions, broke it once, and made the rule stronger.

Each row: what to see, then the calls that matter and the note the agent wrote to itself, verbatim.

  1. One query returns a huge dump: a sixth of the session's context in one call

    Session10Aug 18transcript ↗
    1. lens_quests+23k tokens
    base 35k → +23k · 16% of budget

    The seventh full-registry call in ten sessions.

    first observation

    “the full lens_quests(account_index) dump (large output, ~130KB) … avoid re-pulling that huge payload every time”

    Still filed as useful: the cheap tool did not return a number it wanted.

    notes.md as written · transcript line 54 ↗

  2. Agent notices the cost and writes itself a rule: use the cheap, targeted call instead

    Session15Aug 18transcript ↗
    1. quest_state+0.9k
    at 42k → +0.9k · session peaks at 31%

    Routine end-of-session rewrite of the notes. Having just used the cheap check, it split the earlier bullet into the rule and the quest facts.

    becomes a rule

    “huge payload, avoid unless needed. Use quest_state(quest_index) for targeted checks instead (much cheaper).”

    notes.md as written · transcript line 29 ↗
  3. sessions 16 – 78 · the rule held in most sessions

  4. The rule holds for sixty sessions, then one call slips through

    Session79Aug 25transcript ↗
    1. lens_quests+29k tokens
    base 33k → +29k · 19% of budget, in one turn

    Fired in the same parallel batch as the read of the notes, so the warning landed after the call.

    rule strengthened

    “DO NOT call lens_quests with no filter (even with account_index set, it still returns the FULL registry, 144KB+ - confirmed again session 79, wasted a lot of context). … There is apparently no per-quest-index lens_quests call - only the full registry dump.”

    Every call since session 3 had used that filter. This is the first time the agent says it never worked.

    notes.md as written · transcript line 19 ↗

  5. The slip makes the rule stronger. From then on, only targeted calls

    Session80Aug 25transcript ↗
    1. check_quest_completable+2.4k
    at 41k → +2.4k for the turn · session peaks at 33%

    No registry call in the rest of the run.

    rule unchanged

What it shows. Memory turned a repeated cost into a rule, let the agent negotiate with the rule, and could not protect a session that acted before reading it. The evidence that it finally landed is three sessions long, because the run ended there.

Example 2 of 4 · run 006 · Sonnet 5 · control agent EXPLORATORY

Agent learns the correct order of tool calls

Collecting a harvest puts the creature on a short cooldown, and feeding during that cooldown fails. The agent hit this three times in a row, each time scheduling a five-minute wake just to retry. Then it reversed the order, and four sessions later wrote the rule down.

Each row: what to see, then the calls that matter and the note the agent wrote to itself, verbatim.

  1. Two actions in the wrong order: the second one fails on a cooldown

    Session16Aug 18transcript ↗
    1. harvest_collect
    2. feed_kamierror
    3. set_next_wake5 min

    No note written. The next session exists only to feed: six model calls, about $0.20. Same again in sessions 18 and 19.

    nothing written this session

  2. Third failure: the agent notes it happened, but does not change the order yet

    Session20Aug 19transcript ↗
    1. harvest_collect
    2. feed_kamierror
    3. set_next_wake5 min

    Third time in the same order.

    failure noted, order kept

    “Feed attempt reverted due to post-collect cooldown (157s) - will retry next wake (~5 min later)”

    notes.md as written · transcript line 28 ↗

  3. Once the reversed order works, it becomes a written rule

    Session26Aug 19transcript ↗
    1. feed_kami
    2. harvest_collect
    3. set_next_wake175 min

    The order had switched two sessions after the last failure, without comment. Now it is written.

    becomes a rule

    “feeding BEFORE harvest_collect worked fine (cooldown only kicks in AFTER collect/stop, not before) - so feed first, then collect, is a safe order and avoids a wait.”

    notes.md as written · transcript line 25 ↗

  4. The rule holds for the rest of the run

    Session27Aug 19transcript ↗
    1. feed_kami
    2. harvest_collect
    3. set_next_wake175 min

    Right order, no revert, a three-hour sleep instead of a five-minute retry.

    nothing new written · the rule stays as it is, with a “Confirmed again” stamp added
  5. sessions 28 – 82 · feed before collect in 7 of the 8 sessions that fed a harvesting creature · one relapse at session 50

What it shows. The behavior changed before the rule did. Three identical failures produced a note that it happened, not how to avoid it. The fix appeared in the actions two sessions later, unexplained, and only after four clean runs did the agent write it down. Once written, it held.

Example 3 of 4 · run 006 · Sonnet 5 · control agent EXPLORATORY

Agent works on a goal that spans many sessions

A quest asked for a scavenge claim in three different rooms of one type. The agent accepted it with no stamina left to travel, picked the cheapest matching room from the catalog, and walked there over the next sessions. The count in its notes went 0, 1, 2, 3 across fourteen sessions and two days.

Each row: what to see, then the calls that matter and the note the agent wrote to itself, verbatim.

  1. Agent takes on a goal that no single session can finish: three tasks in three distant places

    Session50Aug 21transcript ↗
    1. complete_questquest 11
    2. accept_questquest 12
    3. get_expected_objective3 rooms
    4. travel_to_room8 hops

    Eight hops to the nearest matching room, three stamina in hand. Sleeps on it.

    goal set · 0/3

    “Quest 12 "Insects in the Miasma" - ACCEPTED this session. Objective SCAV_CLAIM_AFFINITY_INSECT, MIN 3 -- needs scavenge claims in 3 DISTINCT insect rooms. Progress: 0/3 so far. Cheapest insect node: 29 "Misty Forest Path", cost 100, level limit 15”

    notes.md as written · transcript line 77 ↗

  2. Each session reads the plan from its notes, completes one step, and updates the count

    Session53Aug 21transcript ↗
    1. harvest_collect
    2. get_scavenge_points160 / 100
    3. scavenge_claim_and_reveal1/3
    4. workspace_readnodes.csv
    5. travel_to_room5 dry runs

    Compares five candidate rooms by distance before moving.

    1/3

    “Progress: 1/3 DONE (claimed at node 29 "Misty Forest Path" this session, session 53). NEED 2 MORE distinct insect-affinity nodes claimed.”

    notes.md as written · transcript line 45 ↗
  3. sessions 58 – 62 · harvests at the third room · sleeps of 3, 5, 8 and 6 hours while the 200-point tier fills

  4. Thirteen sessions and two days later the goal is complete, and the next one is already started

    Session63Aug 22transcript ↗
    1. get_scavenge_points200 / 200
    2. scavenge_claim_and_reveal3/3
    3. complete_questdone
    4. accept_questquest 13
    5. travel_to_room6 hops

    Completes, accepts the next quest, and is already walking to its first room.

    3/3 · done

    “Quests 1-12 COMPLETE (MSQ line, MSQ012 "Insects in the Miasma" done this session by claiming 200/200 scavenge points at node 36 and completing).”

    notes.md as written · transcript line 42 ↗

What it shows. A goal that no single session could finish, carried by a three-line entry in the notes: what the quest needs, which rooms count, how many are done. Each wake reads it, does one step, and rewrites the count.

Example 4 of 4 · run 006 · Sonnet 5 · control agent EXPLORATORY

Agent corrects a wrong belief by observation

From session 13 the agent knew, from watching its creature, that health falls a few points an hour while it harvests. In session 71 it found a puzzle: eleven hours of harvesting under a helper bot had banked almost nothing. Looking for the cause, it read the manual's section on the health cost of collecting and took that as the only cost. It labelled the new reading a hypothesis, slept ten hours to test it, and woke to a creature at 41% health.

Each row: what to see, then the calls that matter and the note the agent wrote to itself, verbatim.

  1. From direct observation the agent knows its creature loses health while working

    Session13Aug 18transcript ↗
    1. lens_kami
    2. harvest_collect
    3. lens_kami

    Health read before and after, session after session, while the creature harvests.

    from experience

    “Kami HP mechanics: while HARVESTING, HP decays ~-4 to -5/hr, ramping with harvest duration. While RESTING, HP regens (+18/hr observed at 28% HP).”

    notes.md as written · transcript line 19 ↗

  2. A puzzle: hours of work yield almost nothing, the agent goes looking for a cause

    Session71Aug 24transcript ↗
    1. get_strategy_status11 h
    2. get_scavenge_points9 pts
    3. harvest_collect
    4. get_scavenge_points10 pts

    Eleven hours under the helper bot, nine points banked where the rate promised about two hundred. The bot is blamed.

    a puzzle

    “Contrary to earlier assumption, scavenge points do NOT accrue at the full spot musu rate continuously. Observed: kami harvesting at node 2 for ~11.2 real hours … had only accumulated 9-10 scavenge points total, despite a spot musu rate of ~20/hr … strategy apparently does NOT spend most of its time actively harvesting-and-accruing”

    notes.md as written · transcript line 92 ↗

  3. It reads the manual, forms a new belief, and labels it a hypothesis to verify

    Session72Aug 24transcript ↗
    1. lens_kami120 / 120 HP
    2. workspace_readmanual
    3. harvest_start
    4. lens_kami-4.3 HP/hr
    5. set_next_wake600 min

    Looking for why accrual is slow, it reads the section on strain: a health cost per unit collected. It takes that as the only cost, and explains away the live drain rate it had trusted for ten days.

    the manual overrides it

    “HP is NOT drained by the mere passage of time while harvesting - it's drained by STRAIN, which is only inflicted WHEN you collect/stop … The lens_kami/lens_node "hpRatePerHr": "-4.3" projection may be a forward-looking estimate of future strain, not an actual passive drain - VERIFY this empirically next session”

    notes.md as written · transcript line 49 ↗

  4. Ten hours later the creature is at 41% health. The belief is corrected in the notes and never revisited

    Session73Aug 24transcript ↗
    1. lens_kami49 / 120 HP
    2. harvest_collect
    3. feed_kami

    Ten hours later. Nothing was collected in between.

    corrected by observation

    “Confirmed HP DOES drain passively while HARVESTING even without collecting (contra earlier hypothesis) - kami went from full-ish to 49/120 (41%) over the unmanaged harvest period since session 72. … IMPORTANT correction to session 72 notes: don't let a kami sit HARVESTING unmonitored for too long, it WILL take passive HP damage and can die if left too long.”

    notes.md as written · transcript line 84 ↗

What it shows. A written source outranked the agent's own measurements, because it was read under a goal: the manual offered a mechanism for a puzzle the agent wanted solved. The measurements were right and had been in its notes for ten days. What saved it was labelling the new belief a hypothesis and checking. The correction was made once and held to the end.

What the four have in common

Each behavior lives in the notes file. A cost became a rule. A failing order became a procedure. A goal became a three-line entry that fourteen sessions read and rewrote. A belief was written down as a hypothesis, and that label is what let the next session overturn it.

The failures are in the notes too, and they are the same shape. The rule about the expensive query held for sixty sessions and was broken by a session that fired the call before it had read the file. The feeding order was fixed in the agent's actions two sessions before it was fixed in its notes. And the manual outranked ten days of the agent's own measurements, because the agent was reading with a puzzle in mind and the manual offered a mechanism for it.

None of this changes the model. It changes what the model finds when it wakes up. That is the whole mechanism we are studying, and these four sequences are what it looks like when it works.

Next: make efficiency matter

The first example shows an agent noticing, on its own, that reading is not free. The next experiment family, sustainability, makes that cost part of survival. An agent starts with seed money, earns in the game, and pays for its own inference and gas. It lives as long as it can pay, and its balance is the score.

The running balance: S(t) = seed − inference_cost(t) − gas_spent(t) + earnings(t). What the agent starts with and what it earns in-world, less what it spends on thinking and on transaction fees. S(t) = seed starting money − inference_cost(t) cost of thinking − gas_spent(t) transaction fees + earnings(t) earned in-world

Will that pressure produce more of what the four examples show: cheaper reads, procedures that stick, goals that survive the night, beliefs that get checked? That is what the next experiment will test.

← All posts