Beneath the Chat Window: Taking the Reins of Stateful LLM Work

Beyond the Chat Window · Article 7 of 8

The workflow reconstructed in the previous article can sound as though it asks the user to bolt a state-management system onto a language model that would otherwise live entirely inside a conversation. That is increasingly the wrong picture. The strongest agentic LLM products already operate through machinery that looks surprisingly familiar: persistent or reusable workspaces, files, tools, plans, context management, checkpoints, subagents, and durable sessions. The chat window is often the most visible part of the system, but it is not the whole system.

This series was produced inside exactly that kind of environment. From the user’s point of view, the work happened through a sequence of messages: create a schema, formalize the articles, increase granularity, correct a continuity rule, add a new central example, revise the composition protocol, resume from a checkpoint. Underneath those messages, however, the work depended on file reads and writes, persistent project objects, execution state, tool calls, artifact generation, and later a publication pipeline that rendered and verified a PDF. The model did not carry the entire project as a private thought. It repeatedly reconstructed a local working context from external state and acted on that state with tools.

The practical opportunity is therefore subtler than “give your LLM memory.” Your LLM service may already have memory and agent infrastructure. The opportunity is to decide how much of that persistent process you want to leave under provider control and how much of it you want to make explicit, user-owned, and governable.

The chat window is a control surface, not the whole machine

A useful mental model separates the conversational control surface from the execution harness beneath it. A turn in a modern agentic system may involve far more than passing the visible transcript to a model. The product can select relevant context, retrieve files, call tools, execute code, spawn subagents, compact old history, write intermediate results to disk, and preserve a session so that the next turn begins from durable state rather than from a literal replay of every prior token.

The workflow for this series can be reduced to something like this:

human instruction
↓
interpret against current project state
↓
select relevant files, objects, and prior results
↓
model inference
↓
file / code / search / artifact tools
↓
state mutation and validation
↓
persist updated state and execution position
↓
return to the human control surface

The human saw the top and bottom of that loop, while the method made selected parts of the middle explicit enough to inspect and govern. That distinction matters because persistent state is normally not a property of the neural network itself. A model does not acquire a durable JSON file in its weights because you mentioned it three turns ago. Persistence is supplied by the surrounding product or harness: a project container, database, filesystem, memory service, conversation store, or execution session. The model is given selected information from those systems and decides what to do with it.

Current agent platforms increasingly say this directly. OpenAI’s Agents API describes a managed harness that handles sessions, orchestration, context compaction, and recovery, while an optional execution environment gives the agent files and command execution; sessions retain state across turns, and coordinator agents can delegate to subagents that share a filesystem.[1] OpenAI’s consumer-facing ChatGPT Work similarly describes long-running work across apps and files, with the user able to follow progress, answer questions, change direction, and approve actions; on desktop it can work against user-authorized local folders.[2]

Anthropic’s guidance is even more explicit about filesystem state. Its current prompting documentation recommends structured files such as JSON for task status, free-form progress files for general context, and git for checkpoints, and it notes that a fresh context can rediscover project state from the local filesystem rather than relying only on compaction.[3] Claude Code adds automatic checkpoints, subagents, hooks, and background tasks, while Anthropic’s Agent SDK exposes the same core harness for custom systems.[4]

GitHub Copilot has moved in the same direction for software projects. Copilot Memory persists repository and personal information across supported agent experiences; current VS Code/Copilot releases preserve plans through context compaction, share memory among agents, and write large tool outputs to disk instead of stuffing them into the active conversation.[5] Copilot’s custom-agent system also lets the main agent delegate work to specialized subagents with isolated context windows.[6]

Google’s current ecosystem exposes a similar pattern in several forms. Gemini Spark is a consumer agent for ongoing workflows that can use connected apps, skills, chats, websites, and other context over time.[7] At the developer layer, Gemini Managed Agents run in sandboxed environments with tools and code execution, while Gemini Enterprise’s Agent Designer supports multi-step agents and subagents connected to organizational data and tools.[8]

These systems differ in what they expose, but the direction is consistent: project-scale performance increasingly depends on an agent harness around the model.

The method changes where the human sits in that harness

If the platform already has all of this machinery, the reason to create a canonical state and protocol is not that the platform lacks state. It is that “the system maintains state” and “the user governs state” are not the same proposition. A provider-managed harness may preserve enough information to continue a task successfully while giving the user little visibility into what was retained, how it was represented, or which rules will govern the next transformation.

A provider-managed agent can decide what to remember, what to summarize, what to compact, which files matter, when to delegate, and how to represent progress. For many tasks, that is exactly what the user wants. The attraction of an agent is that the user can state a goal and allow the system to handle the mechanics. ChatGPT Work, Gemini Spark, Claude Code, and Copilot agents all push toward this experience in different domains: describe the objective, observe progress, intervene when necessary, and let the harness manage the rest.

The workflow used for this series deliberately moved the human deeper into the same loop. When I initially imposed a rule that later articles should inherit one precise result from the previous article, the user did not merely tell me to “remember not to do that.” The continuity contract in persistent state was changed. When the user decided that the series itself should become the central worked example, that idea became a project object and propagated through every article. When the series expanded from five articles to eight, the existing graph was extended rather than replaced. The user’s conceptual interventions became structural changes in the state from which subsequent model work was generated.

The difference is between steering outputs and steering the state-generation process. At the conversational surface, a user might say, “No, make Article 7 less about product comparison and more about human control.” A capable agent can remember the correction and produce a better next response. In an explicit stateful workflow, the user can require the article object, graph node, composition map, and downstream dependencies to change before more prose is written. The user is no longer trusting the harness to infer which internal representation should change. The user has a representation to inspect and a protocol through which to alter it.

This does not eliminate model autonomy. The model still proposes schemas, infers dependencies, drafts text, selects tools, and performs transformations. It changes the boundary of autonomy. A greater fraction of the persistent structure is jointly governed rather than independently managed by the agent.

For a capable user, that can be a significant change in experience. You are not simply issuing increasingly precise prompts. You are editing the durable substrate from which future prompts are interpreted.

Native agent infrastructure and user-owned state are complements

To decide how much state the user should own, it helps to separate two layers that are easy to conflate. The first is execution infrastructure: a durable session, a filesystem, code execution, browser or application tools, context compaction, retries, subagent coordination, and long-running compute. These are expensive and difficult to reproduce well, and commercial agent platforms are rapidly making them native. When a service already gives you reliable files, checkpoints, artifact handling, and resumable sessions, recreating those features manually is usually pointless.

The second is semantic project state: the user’s explicit model of the task, including objects, decisions, dependencies, statuses, invariants, provenance, and rules for revision. A provider can maintain its own internal equivalent, but an explicit user-owned version has different properties. It can be inspected. It can be versioned. It can be moved between services. It can be analyzed with ordinary software. It can encode domain-specific rules that should not depend on the agent’s current interpretation. Most importantly, it gives the human a stable place to put intent that is deeper than another sentence in the chat history.

This series used commercial execution infrastructure and user-owned semantic state together. The environment supplied the ability to read and write files, run code, search current sources, generate document artifacts, and resume from files created in earlier passes. The project supplied its own schema, argument graph, protocol, composition maps, revision records, and publication status. Neither layer replaced the other.

This is also the right way to think about “obsolescence.” A native checkpoint system can make a hand-maintained checkpoint file unnecessary. Automatic context compaction can make manual transcript summaries unnecessary. A durable agent session can remove the need to reconstruct tool state between turns. Built-in subagents can replace a homemade delegation loop. Those are genuine simplifications.

But a platform feature does not make explicit semantic state obsolete merely because the platform has hidden state of its own. It makes the question more selective: which state needs to be visible and governable by the user, and which state can safely remain an implementation detail?

In this project, the article dependency graph belonged on the visible side of that boundary because the user cared about the argument sequence and repeatedly changed it. Temporary browser state did not. The rule governing how many prior results an article may inherit belonged on the visible side because the user corrected it explicitly. The model’s internal choice of which search result to inspect next did not. The editorial rule requiring two antagonistic revision proposals and a strict referee belonged on the visible side because it expressed the user’s desired method of quality control. The low-level details of PDF rasterization did not need to become part of the intellectual schema, even though they mattered operationally.

The method is therefore not an argument for exposing everything. It is an argument for making consequential state legible at the level where the human wants control, while allowing low-level execution details to remain delegated when they do not affect authorship, constraints, dependencies, or later decisions.

Which current services are sufficient?

As of September 2026, several retail or readily available agent products expose enough of the required infrastructure to implement substantial versions of this method, although their suitability depends heavily on domain.

ChatGPT Work / desktop agent workflows are the closest general-purpose retail fit. Work is designed for longer tasks across files and connected applications, supports user steering during execution, and on desktop can be granted access to local folders.[2] ChatGPT Projects also provide persistent grouping of chats, files, instructions, and shared context.[9] The key practical question is whether the particular mode you are using gives the agent actual read-write access to the project files rather than treating them only as uploaded reference material. When it does, the schema/protocol pattern used in this series is straightforward: put the canonical state in the accessible workspace and instruct the agent to treat it as authoritative.

Claude Code is an excellent fit when the project can live in a filesystem and terminal-oriented environment. It already expects the agent to inspect and modify local files, and Anthropic’s own long-horizon guidance closely resembles the state-management practices described in this series: structured state files, progress notes, git checkpoints, incremental work, and recovery from a fresh context.[3] Its limitation is not technical capability but product orientation: it is optimized around development and local-project workflows rather than being a general consumer document workspace.

GitHub Copilot’s agent ecosystem is similarly strong for repository-bound work. Persistent memory, durable plans through compaction, disk-backed tool outputs, custom agent profiles, and subagents make it a natural environment for explicit project state when the state belongs in the repository.[5][6] It is less appropriate for a general writing or research project unless that project is intentionally managed as files in a repository.

Gemini Spark has enough agentic continuity and connected-app capability to handle ongoing workflows, and Gemini’s Skills give users reusable instructions and reference files.[7][10] But the consumer surface currently exposes less of a general arbitrary read-write project filesystem than a coding agent or desktop workspace. For users who need explicit durable state objects that the agent can freely mutate, Google’s Managed Agents API or Gemini Enterprise Agent Designer is a closer architectural match than ordinary Gemini chat.[8]

A service is sufficient for the method when it can satisfy most of the following requirements:

• preserve a project across turns or sessions; • read and write durable files or equivalent structured state; • let the model selectively reload state rather than depend on raw transcript retention; • execute tools needed by the project; • preserve or reconstruct execution position after interruption; • allow the user to inspect and modify the canonical state; • support artifact creation outside the chat response; • ideally, support isolated helper agents or separate contexts for independent review; • ideally, expose enough logs or files to maintain provenance.

A large context window is not enough. File upload alone is not enough if the model cannot update the canonical object. “Memory” alone is not enough if the user cannot distinguish durable project state from opaque personalization. The method needs a place where project state can live as an object that both human and model can intentionally manipulate.

A practical snapshot, as of September 30, 2026, looks like this:

Service Fit for this method What the user can own explicitly Main limitation
ChatGPT Work / desktop workflows
Strong general-purpose fit Project files, instructions, explicit schema/protocol files, local folders on desktop
Some orchestration and memory remain provider-managed rather than exposed as project objects
Claude Code Strong fit for filesystem-based projects
Files, progress/state documents, git history, checkpoints, hooks, custom agent instructions
Product is oriented toward code and terminal-centered project work
GitHub Copilot agents Strong fit for repository projects Repository files, instructions, agent profiles, plans committed or stored with the project
Persistent memory is partly platform-managed and the environment is software-development specific
Gemini Spark Partial general-purpose fit Skills, connected-app context, task definitions, uploaded reference material
Consumer surface exposes less of an arbitrary mutable project filesystem
Gemini Managed Agents / Enterprise Agent Designer
Stronger developer/enterprise fit Agent files, instructions, workflows, tools, sandboxed execution
This is an agent-building platform rather than a simple retail chat workflow

The table is not a ranking. It answers a narrower question: if you want the human to govern canonical state directly, where can that state actually live and be mutated? That is why coding-oriented agents often fit the method unusually well: a repository is already a persistent, versioned, inspectable workspace. General-purpose assistants are only now exposing comparable project substrates.

The deeper opportunity is shared governance

Modern agent products are moving toward more autonomy because autonomy is useful. A user should not have to manually orchestrate every search, file read, tool call, or context refresh. The goal of this method is not to reverse that progress by turning the user into a human scheduler.

The goal is to make autonomy selective rather than to eliminate it. For a simple task, the user can stay at the surface: state the goal, let the harness work, and judge the result. For a project whose internal structure matters, the user can descend one layer and take ownership of the durable semantic representation. The model remains free to execute within that representation, but the human can see and alter the objects that determine what later execution means.

That was the pattern throughout this series. The model repeatedly proposed useful structures. The user repeatedly intervened at moments when those structures encoded the wrong assumption or omitted an important idea. Because the structures were persistent and inspectable, those interventions changed more than the next answer. They changed the state from which future answers would be produced.

The best-performing agentic systems are already moving work out of the transient chat context and into persistent environments. The practical extension of that idea is to let the human follow them there.

The final question is what that buys the user. It does not make the underlying model infinitely more intelligent, and it imposes real overhead. The claim worth examining is narrower and more practical: on a project that lasts many turns, changes direction, produces large artifacts, and requires the human to retain authorship and control, does entering the stateful layer change what “performance” feels like and what the user can reliably accomplish?

That is the subject of the final article.

References

[1] OpenAI, “Agents API” and “Multi-agent,” September 2026. https://developers.openai.com/api/docs/guides/agents-api/overview and https://developers.openai.com/api/docs/guides/agents-api/multi-agent

[2] OpenAI, “ChatGPT is now a partner for your most ambitious work” and “ChatGPT Work and Codex,” July-September 2026. https://openai.com/index/chatgpt-for-your-most-ambitious-work/ and https://help.openai.com/en/articles/20001275

[3] Anthropic, “Prompting best practices,” section on long-horizon reasoning and state tracking, current 2026 documentation. https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/prompt-templates-and-variables

[4] Anthropic, “Enabling Claude Code to work more autonomously.” https://www.anthropic.com/news/enabling-claude-code-to-work-more-autonomously

[5] GitHub, “GitHub Copilot in Visual Studio Code v1.110” and “About GitHub Copilot Memory.” https://github.blog/changelog/2026-03-06-github-copilot-in-visual-studio-code-v1-110-february-release/ and https://docs.github.com/en/copilot/concepts/agents/copilot-memory

[6] GitHub Docs, “About custom agents” and “Custom agents and sub-agent orchestration.” https://docs.github.com/en/copilot/concepts/agents/copilot-cli/about-custom-agents and https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents

[7] Google, “Use Gemini Spark to manage your tasks & workflows in Gemini Apps.” https://support.google.com/gemini/answer/17094507

[8] Google, “Build managed agents with the Gemini API” and “Create and manage agents using Agent Designer.” https://blog.google/innovation-and-ai/technology/developers-tools/managed-agents-gemini-api/ and https://support.google.com/g/answer/16540723

[9] OpenAI Academy, “Using projects in ChatGPT.” https://openai.com/academy/projects/

[10] Google, “Create & manage skills for Gemini Apps.” https://support.google.com/gemini/answer/17094296