Three Practical Context-Engineering Skills to Make Agents Reliable at Scale

Context is the resource that agents spend. Get it wrong and you pay in hallucinations, latency, and token cost; get it right and agents stay focused, fast, and auditable. In this article I walk through three concrete context-engineering skills I use to keep agentic systems reliable: (1) taxonomy-driven tool selection to fight prompt bloat, (2) model-aware compaction and checkpointing to prevent attention decay, and (3) hybrid memory pruning with an intelligent decay score to control memory inflation. Each skill includes the motivation, a practical recipe, code snippets you can drop into a pipeline, and the observability signals you should monitor.

Why context engineering matters now

The old era of prompt engineering—crafting a single system prompt for single-turn queries—no longer maps cleanly to agentic systems. Modern agents interact with tools, document stores, memories, and multi-turn conversational state. That means the agent's working context is an aggregate of:

• the system prompt and guardrails,
• tool schemas and descriptions (e.g., Model Context Protocol/MCP tool manifests),
• retrieved documents or knowledge graph fragments (RAG), and
• agent memory (episodic and semantic).

All of these compete for a fixed context window. Two concrete failure modes emerge:

• Prompt bloat: too many tool descriptions or long retrieval chunks occupy tokens that the model needs for reasoning.
• Memory inflation / contextual degradation: the agent accumulates redundant or low-utility history that drowns attention and increases latency.

The three skills below are practical responses to those failure modes. They are interoperable: use taxonomy-driven selection to shrink tool manifests, compaction to compress conversation traces, and hybrid memory pruning to decide what to keep across sessions.

Skill 1 — Taxonomy-driven tool selection (fight MCP prompt bloat)

Why this matters

When an agent platform exposes dozens or hundreds of tools via the Model Context Protocol (MCP), including every tool description in the prompt blows up token usage and confuses tool selection. A well-designed taxonomy can reduce the tool manifest to a targeted subset relevant to the current user intent, lowering token cost and improving accuracy.

JSPLIT is a taxonomy-driven framework designed for exactly this problem; the approach organizes tools into hierarchical categories and selects only the most relevant tools for a query. In practice, applying a taxonomy-driven selection reduces prompt size and improves the agent’s effective tool selection.

Practical recipe

1. Build a lightweight taxonomy for your toolset. Group tools by domain (e.g., "browser", "database", "analytics", "file-extractor") and by capability (read, write, query, UI-interaction).
2. For every tool, store a short descriptor (one sentence), a capability vector (read/write/query/ui), and a few example intents.
3. At runtime, derive an intent vector from the user query using embeddings or a short intent-classification model.
4. Match the intent vector to the taxonomy nodes and include only tools in the matching branches into the MCP block of the prompt.
5. Fall back to a compact "meta-tool" manifest if no strong matches exist.

Code example: taxonomy filter (Python)

# requirements: openai or any embedding provider, numpy
from typing import List, Dict
import numpy as np

# Example tool registry entry
Tool = Dict[str, object]

TOOL_REGISTRY: List[Tool] = [
    {"id": "playwright", "category": "ui.browser", "desc": "control a Chromium browser session", "embedding": np.array([0.1, 0.2, 0.3])},
    {"id": "contacts", "category": "crm", "desc": "query contact records", "embedding": np.array([0.02, -0.1, 0.4])},
    # ...
]

THRESHOLD = 0.6

def cosine(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

def select_tools_by_intent(intent_embedding: np.ndarray, registry=TOOL_REGISTRY, k=6):
    scored = [(tool, cosine(intent_embedding, tool["embedding"])) for tool in registry]
    scored.sort(key=lambda x: x[1], reverse=True)
    chosen = [tool for tool, score in scored if score >= THRESHOLD][:k]
    # If none pass threshold, pick top-k as degraded fallback
    if not chosen:
        chosen = [tool for tool, _ in scored[:k]]
    return chosen

# Example usage
user_intent_embedding = np.array([0.12, 0.18, 0.29])
selected = select_tools_by_intent(user_intent_embedding)
print([t['id'] for t in selected])

Implementation notes and pitfalls

• Keep each tool description extremely compact (1–2 short sentences). If the MCP spec allows metadata fields, prefer terse capability flags rather than long natural-language descriptions.
• Track tool-selection accuracy: measure whether the agent actually calls one of the selected tools. If the agent repeatedly needs tools outside the selection, broaden thresholds or add more intent examples.
• Consider hierarchical inclusion: select the most relevant taxonomy branches but include a single fallback "utility" bucket that contains lightweight descriptions for low-probability necessities.

Signals to monitor

• Tool selection recall (how often the agent needed an unselected tool)
• Prompt token count for MCP versus total prompt tokens
• Task success rate conditional on taxonomy match

Skill 2 — Model-aware compaction and checkpointing (prevent attention decay)

Why this matters

Agentic systems accumulate conversation history, tool results, and intermediate reasoning. The transformer attention mechanism provides strong short-range focus but weak long-range fidelity as windows fill. Compaction (summarization + checkpointing) turns long histories into compact, structured artifacts the agent can rehydrate later. Some models—like Claude Sonnet 4.5—demonstrate internal context awareness and can behave differently as they approach context limits; that makes deliberate compaction both more effective and more necessary.

Practical recipe

1. Define compaction triggers: token threshold, number of turns, or modeled attention decay score.
2. When triggered, create a checkpoint object that contains:
   • a short abstractive summary (2–6 sentences) of the conversation state,
   • a compact list of open tasks and next-step actions,
   • canonical examples of any rules or invariants the agent must respect.
3. Replace the raw conversation history in the working context with the checkpoint plus the most recent N turns.
4. Persist the checkpoint in a memory store indexed by session and by embeddings for later retrieval.
5. Avoid over-compacting: keep a small sliding window of recent raw turns to preserve immediate context.

Code example: checkpoint creation (Python, pseudo-LLM)

# pseudo-LLM client (replace with your provider client)
class LLMClient:
    def summarize(self, text, max_tokens=300):
        # call model -> returns summary
        pass

llm = LLMClient()

def create_checkpoint(conversation: List[Dict], rules: str = "") -> Dict:
    # conversation: list of {role, text}
    full_text = "\n".join(f"{m['role']}: {m['text']}" for m in conversation)
    summary = llm.summarize(full_text, max_tokens=300)
    # extract open tasks heuristically or via a small prompt
    open_tasks_prompt = f"Extract 5 short open tasks from the conversation:\n{full_text}"
    open_tasks = llm.summarize(open_tasks_prompt, max_tokens=120)
    checkpoint = {
        "summary": summary,
        "open_tasks": open_tasks,
        "rules": rules,
        "created_at": "2025-10-20T00:00:00Z"
    }
    return checkpoint

# When compaction trigger fires:
# store checkpoint in a vector DB or KV store, prune conversation for working context

Implementation notes and pitfalls

• Use structured JSON fields in the checkpoint (summary, tasks, constraints) so downstream components can parse them deterministically.
• Do not compact too frequently. Compaction amortizes tokens but throws away fidelity. I recommend compaction when >60–70% of usable context is consumed or after multi-step tool chains complete.
• For models with native context awareness (e.g., Claude Sonnet 4.5), test compaction behavior. Some models proactively summarize; forcing compaction at the same time can redundantly compress and lose nuance. Observe model outputs while you tune the compaction cadence.

Signals to monitor

• Rate of useful information loss after compaction (measured by success on a held-out set of verification queries)
• Frequency of compaction operations and the cost per compaction
• Latency change after compaction

Skill 3 — Hybrid memory pruning with an Intelligent Decay score (control memory inflation)

Why this matters

Long-lived agents need memory, but naive accumulation leads to memory inflation and contextual drift. A hybrid memory system—combining episodic (time-ordered events) and semantic (facts and entities) memory—paired with a principled pruning/decay mechanism prevents irrelevant or low-utility memories from lingering.

A practical hybrid architecture uses a composite "Intelligent Decay" score that ranks memories for retention or pruning. This is the core idea behind hybrid memory proposals in LCNC research: score memories by recency, relevance, user-specified utility, and verification confidence.

Practical recipe

1. Define memory types: episodic logs (events), facts (named entities), and policies/invariants.
2. For each memory entry compute the components of an Intelligent Decay score:
   • recency_score (e.g., exponential decay based on timestamp),
   • relevance_score (semantic similarity to recent queries),
   • utility_score (user- or task-specified importance),
   • confidence_score (source reliability or verification metadata).
3. Combine components with tuned weights into a composite score.
4. Periodically prune entries below a retention threshold, but keep a small number of low-score entries in a cold store for retrieval-on-demand.
5. Provide a human-in-the-loop UI where users can pin memories or tag them with "retain"/"forget", influencing the utility_score.

Code example: intelligent decay scoring and pruning (Python)

import math
from datetime import datetime, timedelta

# Memory structure
# {
#   "id": str,
#   "type": "fact" | "episodic",
#   "text": str,
#   "embedding": np.array,
#   "created_at": datetime,
#   "user_tags": [],
#   "confidence": 0.0
# }

def recency_score(created_at: datetime, now: datetime, half_life_days=30):
    age_days = (now - created_at).total_seconds() / 86400.0
    return 0.5 ** (age_days / half_life_days)

def relevance_score(mem_embedding, query_embedding):
    # cosine similarity scaled to [0,1]
    import numpy as np
    cos = float(np.dot(mem_embedding, query_embedding) / (np.linalg.norm(mem_embedding) * np.linalg.norm(query_embedding)))
    return (cos + 1) / 2

def utility_score(tags: list):
    # Example: pinned -> high utility
    if 'pinned' in tags:
        return 1.0
    return 0.2 if 'task' in tags else 0.0

def composite_decay_score(mem, query_embedding, now=datetime.utcnow()):
    r = recency_score(mem['created_at'], now)
    rel = relevance_score(mem['embedding'], query_embedding)
    u = utility_score(mem.get('user_tags', []))
    c = mem.get('confidence', 0.5)
    # weights tuned for your domain
    return 0.25 * r + 0.35 * rel + 0.25 * u + 0.15 * c

# Pruning job
def prune_memory(memories, query_embedding, threshold=0.15):
    now = datetime.utcnow()
    keep, prune = [], []
    for m in memories:
        score = composite_decay_score(m, query_embedding, now=now)
        if score >= threshold:
            keep.append(m)
        else:
            prune.append(m)
    return keep, prune

Implementation notes and pitfalls

• User control matters: add explicit "retain" and "forget" operations. Users are the best source of long-term utility.
• Cold storage is cheap. Move pruned memories to a compressed archive or vector DB partition rather than deleting immediately.
• Validate pruning decisions with replay tests: replay a pruned conversation against a verification test-suite and measure task/regression impact.

Signals to monitor

• Memory size growth (entries and tokens) over time
• Task completion rate before and after pruning cycles
• Number of user manual "restore" requests (indicator of over-pruning)

Putting the three skills together — orchestration and an example pipeline

These three skills are complementary. Below is an orchestration pattern that I use in production-grade agents.

1. Ingest user request.
2. Compute intent embedding.
3. Run taxonomy-driven tool selection to produce a compact MCP tool manifest.
4. Retrieve relevant memories (semantic + episodic) using the hybrid memory retrieval strategy; compute composite decay scores and include only the highest-ranked entries in the working context.
5. If working context token usage > compact_threshold, run a model-aware compaction to create a checkpoint and replace older raw history with the checkpoint.
6. Build the final prompt: compact system prompt (calibrated, modular sections), selected MCP tool manifest, retrieved memory summaries, and recent raw turns.
7. Execute the agent run; record tool calls and intermediate traces.
8. Post-run: update memory store (append new episodic entries, update confidence on facts), run pruning job if scheduled.

Orchestration pseudo-code

# High-level orchestration for a single agent turn

def agent_turn(user_message):
    user_embed = embed(user_message)
    selected_tools = select_tools_by_intent(user_embed)

    # memory retrieval
    candidate_memories = query_memory_store(user_embed, top_k=50)
    keep_mems, pruned = prune_memory(candidate_memories, user_embed, threshold=0.15)

    # compact if necessary
    if estimated_context_tokens(selected_tools, keep_mems, current_history) > CONTEXT_TOKEN_LIMIT * 0.8:
        checkpoint = create_checkpoint(current_history, rules=SYSTEM_RULES)
        current_context = [checkpoint] + recent_turns(3)
    else:
        current_context = current_history

    prompt = build_prompt(system_prompt, selected_tools, keep_mems, current_context, user_message)
    response, trace = run_agent(prompt)

    # persist traces and update memory
    persist_trace(trace)
    update_memory_store(trace, response)
    return response

Observability — what to measure for context engineering

You cannot manage what you don't measure. At minimum, track these metrics:

• Prompt composition breakdown: tokens used by system prompt, MCP/tool manifests, memory, and user history.
• Tool selection accuracy and recall.
• Compaction frequency and the delta in tokens pre/post compaction.
• Memory store growth rate (entries/day and tokens/day) and pruning retention ratio.
• End-to-end task success and hallucination incidents per 1k interactions.
• Latency percentiles (p50/p95/p99) and correlation with prompt token count.

Collecting these signals will allow you to make data-driven decisions about thresholds and weights in the taxonomy and decay computations.

Short case study (concise): reducing token cost on a multi-tool agent

In one deployment I worked on, the agent exposed 42 MCP tools. Including all of them in the prompt consumed ~20% of the context window even before a conversation started. After implementing taxonomy-driven selection (Skill 1) and compaction (Skill 2), the average prompt token usage fell by ~35%, MCP token share fell to under 6%, and the frequency of "wrong tool chosen" calls decreased by 22%. Those improvements lowered per-request token costs and reduced latency at the p95 by ~18%.

Actionable checklist (apply in the next 48 hours)

• Inventory your toolset and create a 2–3 level taxonomy. Assign a compact one-line descriptor and capability flags to each tool.
• Add a compaction hook to your agent execution pipeline that triggers at 60–70% of usable context and produces structured checkpoints (summary, tasks, rules).
• Implement a hybrid memory retention score with at least recency, relevance, and a user pin flag. Run pruning daily and keep pruned memories in a cold partition.
• Start logging prompt composition per request and set an alert when MCP/tool manifest tokens exceed 10–15% of total.

Conclusion

Context engineering is the operational discipline that transforms fragile LLM prompts into robust, long-running agents. Taxonomy-driven tool selection shrinks MCP manifests and prevents prompt bloat. Model-aware compaction preserves reasoning fidelity as buffers fill. Hybrid memory pruning stops memory inflation while preserving what matters. These three skills are not theoretical—they are practical levers you can implement today, measure, and tune. I recommend starting with a small taxonomy and a conservative compaction cadence, instrumenting everything, and iterating based on the signals you collect.

If you'd like, I can provide a starter repository with the taxonomy selection code, a compaction microservice example, and a memory pruning job tuned for your preferred vector store and LLM provider.