ArisGlobal cut inference spend about 40% by caching its shared prompt, not documents
AWS says ArisGlobal's shared prompt grew from about 72,000 to 180,000 tokens in 6 months. Putting it first and caching it served 65% of input tokens from cache at more than 19 billion tokens a month.
Why it matters: In multi-tenant agents, the part that repeats is worth caching, not the biggest part. Check that stable instructions come before per-request data, and alert when your cache-hit rate drops.
From Drop #092 · Tue, Nov 17, 2026 · Evening