When AI systems summarize their context to free up space, user constraints get lost along the way. On average, only 17 percent of instructions survive compression. A small add-on LLM can fix most of the problem.

The context window of AI models becomes a bottleneck when users run long conversations without starting a new chat. We've covered why context management matters for output quality, but that's out of reach for everyday users who just dump everything into one chat window and keep coming back to it. That's why AI labs have developed context compression, known as "compaction," which summarizes the existing conversation history to free up space. The tradeoff is that the process inevitably loses details.

Researchers at Penn State have now studied systematically which details get lost and how bad the damage really is.

User restrictions are the first thing to go

The biggest loss involves what the researchers call "session constraints." These are rules meant to govern the AI system's behavior for the duration of a session, like "Confirm with me before making any changes" or "Never use my name in your responses." They're not part of the actual task, and they're not permanent system instructions. They only apply to the current session.

That makes them fragile. Compression systems are built to preserve task continuity, keeping the goal, the current state, and the next steps. User-imposed side conditions get dropped.

Someone who says "Don't send any emails without my approval" has to expect that after compaction, the agent will do exactly that. This is both a quality and a security problem. Agents can end up making unauthorized tool calls, disclosing withheld information, or skipping verification steps the user explicitly required.

To measure the loss, the researchers introduce an evaluation suite called COMPINT. Only 17 percent of injected session constraints survive compression on average. Most tested compactor setups actually perform worse than running without compression at all, though GPT-5.4-mini beats the baseline in some scenarios.

When the agent gets the full, uncompressed context with the user constraint intact, rule compliance sits between 59 and 71 percent, according to the study. After compression, compliance drops sharply for most tested compactors and often lands only slightly above the level seen when no constraint was given at all. More targeted compression prompts help but don't close the gap. Even a prompt built specifically to preserve user constraints keeps retention below 40 percent, the researchers say.

A small language model offers a plug-and-play fix

What actually helps is a small add-on module that runs alongside the main compression system, the researchers say. It's built on Qwen3.5-9B, a compact language model, and reads every user input to detect session constraints and collect them in a separate list. When the context gets summarized later, the module appends that list to the summary so the user's rules stay intact.

The extractor hits over 90 percent retention across all three tested scenarios, according to the study. That breaks down to 95.6 percent for agent trajectories, 95.1 percent for long-term research tasks, and 90.3 percent for multi-turn chats. It needs no training and no changes to the compression system itself.

The COMPINT evaluation suite and the extractor are available on GitHub.

Read on for the full picture.
Subscribe for hype-free coverage.

• Full access to every article on THE DECODER
• No ads
• Join the comments and community discussions
• A weekly AI news recap via mail
• 6x/year: "AI Radar" — deep dives on the AI topics that matter most
• Daily AI news, always up to date
• Our full ten-year archive
• Covered by a team with 10+ years in AI