Prompting agents at scale: context windows, compaction, and thread management

The hardest problem in long-running agent sessions isn’t model intelligence. It’s context management.

Every file read, every tool output, every intermediate result fills the context window. As it fills, the agent degrades — it loses track of earlier decisions, makes contradictory choices, and produces worse output. You can have the most capable model available and still get bad results if the context window is drowning in irrelevant history.

Understanding how context works, how compaction tries to help, and how to structure work so agents stay effective over hours is the difference between agents that work and agents that fall apart.

What’s in a context window

When you submit a prompt, several things fill the context window:

The IDE extension automatically includes open files and selected text ranges as context. The CLI requires you to explicitly mention files with @ or /mention. The difference matters: more automatic context means faster starts but less control over what fills the window.

Context window limits by model

Different models have different context windows. The limit matters because once you hit it, the agent must compact or stop:

Codex monitors remaining context space and reports it. You don’t need to count tokens — but you do need to recognize when an agent has been running long enough that compaction is inevitable.

How compaction works

Compaction is the process of summarizing conversation history to reclaim context space. When the context window fills past a threshold, Codex:

1. Identifies what’s still relevant (current task, constraints, decisions made)
2. Summarizes it into a compact representation
3. Discards the original verbose history
4. Continues with the summary as context

The compaction prompt is configurable:

compact_prompt = "Prioritize architectural decisions and unresolved issues. Discard successful command output."

You can also set the auto-compaction threshold:

model_auto_compact_token_limit = 80000

Compaction is automatic and mostly invisible. But it has a cost: summarized history is lossy. Details that seemed unimportant during compaction might turn out to be critical two turns later.

AGENTS.md as persistent infrastructure

AGENTS.md is the most important file for agent effectiveness at scale. It’s loaded into every session as always-on context — it survives compaction because it’s re-injected from disk, not carried in conversation history.

# Project Conventions

## Architecture
- Services communicate over gRPC, not REST
- All new endpoints require an integration test
- Database migrations must be reversible

## Testing
- Test coverage target: 85% for new code
- Run `npm test -- --coverage` to check
- Flaky tests: tag with `@flaky` and create a Linear issue

## Code Style
- Prefer functional composition over class inheritance
- Async/await over raw promises
- Error messages must include the operation that failed

## Before Marking Work Complete
1. Verify all tests pass (`npm test`)
2. Verify lint is clean (`npm run lint`)
3. Check no TODOs remain in changed files
4. Run the agent's own verification steps

Good AGENTS.md files are concrete, testable, and scoped to what the agent can verify itself. “Write clean code” is useless. “Run npm run lint and fix all warnings before marking work complete” is actionable.

Breaking work into compact-friendly chunks

The most important technique for long-running sessions: break complex tasks into independent, verifiable pieces. Each piece should:

1. Have a clear, testable definition of done
2. Produce output that can be verified without carrying forward all intermediate context
3. Be small enough that the agent can complete it before compaction becomes lossy

Example — instead of “Implement the new payment flow”:

Step 1: Write the data model. Verify it matches the spec. Commit.
Step 2: Implement the API endpoint. Verify it returns correct responses for valid and invalid inputs. Commit.
Step 3: Wire the frontend. Verify the happy path and error states in the browser. Commit.

Each step is independently verifiable. Each step produces a commit. The agent can compact between steps without losing critical context because the decisions are recorded in code and commit messages.

Thread patterns for long-running work

Single thread, sequential — Simplest. The agent carries all context. Works for 1-2 hour tasks. Fails when the task is long enough that compaction loses critical decisions.

New thread per step — Cleanest. Each step gets a fresh context window. The handoff is the commit message and the code. Works well when each step has a clear deliverable.

Main thread + subagents — Best for parallel work. The main thread orchestrates; subagents do isolated research or review. Each subagent gets a clean context window. Only the summary returns.

Cloud delegation — For work that can run in parallel with local work. Push to a cloud environment, let it run, pull the diff when done. Useful when you want to keep working locally while a long task runs elsewhere.

Practical rules for context management

1. Commit between major steps. The commit message is context that survives compaction. An agent resuming work can read the git log instead of relying on summarized history.

2. Use structured output formats. When an agent produces a plan, design doc, or review, have it write it to a file. Files survive compaction. Inline conversation output doesn’t.

3. Be selective about attachments. In the CLI, use @ to attach only the files relevant to the current prompt. Don’t attach the entire codebase. More context isn’t better — it’s just more that needs to be summarized later.

4. Reset threads for new topics. Don’t ask an agent that just debugged a payment issue to now refactor the auth system. Start a new thread. The debugging history is noise for the refactoring task.

5. Monitor compaction frequency. If Codex is compacting every few turns, the task is too large for a single thread. Split it. Frequent compaction means the agent is spending more time summarizing history than doing work.

The limits of compaction

Compaction works well for:

• Filtering successful command output (you don’t need every npm test run in history)
• Summarizing file reads that didn’t lead to changes
• Keeping the current task and constraints in focus

Compaction struggles with:

• Nuanced design decisions where the reasoning chain matters
• Debugging sessions where the exact error message from 20 turns ago is the key clue
• Multi-step workflows where step 3 depends on a detail from step 1

When the reasoning matters more than the outcome, record it in a file before compaction hits. The agent can re-read the file — it can’t recover discarded conversation history.