Skip to content
Home › Architect's Playbook Patterns

Parallelization and Caching

Architect's Playbook Patterns

Definition

A latency-reduction pattern where a coordinator fans out work across parallel subagents and applies prompt caching on the shared/accumulated context, instead of processing items serially.

Key points

  • Serial processing: Sequentially processing 12 precedents in a complex legal case takes over 3 minutes (~T+180s) — unacceptable latency.
  • Parallel execution: Coordinator spawns ~12 parallel subagents → synthesis with prompt caching; completes in the ~T+20s range.
  • Subagent parallelism: For independent data (e.g. 12 legal precedents), the coordinator spawns parallel subagents each handling a subset, then aggregates results.
  • Prompt caching: When follow-up summaries consistently take 40+ seconds passing 80K+ tokens of accumulated findings, enable prompt caching on the synthesis subagent to drastically reduce transfer overhead.

Why it matters for the exam

  • This is the Multi-Agent × Latency cell in the Reference Matrix ("Parallelization & Caching").
  • The trigger numbers are memorizable: 12 subagents, 80K+ tokens, 40+ seconds, ~180s → ~20s.

Common gotchas

  • Parallelization and caching are two distinct levers: fan-out cuts wall-clock time; caching cuts the transfer/overhead cost of re-sending large shared context.
  • Prompt caching is applied on the synthesis subagent (the shared context), not on each independent worker.

See also

Sources

Referenced by