Signal SentrySearchUpdated Nov 20, 05:30 UTCAM Drop
Agents · industry

Agents in Customer service

Agent deployments and launches in Customer service, from the Drops.

From the Drops

AgentsCustomer serviceTestingHuman in loop: no

Preprint: LLM judges passed forbidden agent writes that a simple rule check caught

Scoring 2,160 tau2-bench episodes four ways, the author reports an LLM judge agreed with native scoring 56.2% of the time and passed 10 to 11 of 11 forbidden writes even with the policy; a rule-based boundary check caught all 11.

Why it matters: If an LLM judge is your only check on a live agent, it may pass the writes that matter most. Test a plain rule: was a write expected, and did one happen?

From Drop #084 · Fri, Nov 13, 2026 · Evening
AgentsCustomer serviceLiveHuman in loop: unclear

OpenAI opens outbound SIP phone calling to all organizations via GPT-Live

OpenAI's docs say voice agents can now dial numbers through a customer's SIP trunk using the Live API. Ringing is capped at 3 minutes and connected calls at 2 hours, and caller SIP headers count as untrusted metadata.

Why it matters: Outbound calling lets a voice agent reach people, not just answer them; decide who approves which numbers it may dial, and log every call, before switching it on.

From Drop #071 · Sat, Nov 7, 2026 · Morning
AgentsCustomer serviceTestingHuman in loop: unclear

Preprint: multi-agent setups lost to single agents on τ²-bench's user-facing tasks

Across about 7,950 simulated conversations and six models, researchers report the multi-agent system was worse in 12 of 13 valid cells and tied in one, with a weighted mean gain of -28 points.

Why it matters: For agents that serve people, the authors say a single agent should be the default, and a multi-agent design must show compute-matched evidence from its own setting.

On the Frontier watch →From Drop #038 · Wed, Oct 21, 2026 · Evening
AgentsCustomer serviceTestingHuman in loop: no

IBM: learned playbooks put a GPT-4.1 support agent atop τ²-bench retail

IBM Research says ALTK-Evolve mined rules from an agent's own runs and appended them to its prompt, lifting held-out retail Pass^4 from 57.5% to 67.5% with no weight updates. Customers were simulated.

Why it matters: Prompt-level memory may buy reliability without fine-tuning; IBM says the cost was input tokens, about $0.084 to $0.139 per retail conversation.

From Drop #037 · Wed, Oct 21, 2026 · Morning
AgentsCustomer serviceLiveHuman in loop: no

Bold Agent Kit: humans beat AI receptionists about 4x on intake across 14,000 calls

Bold Agent Kit says that in about 14,000 client calls, 6,343 handled by AI, AI calls had 700% more no-progress hangups, while human receptionists missed about 15% more calls. It calls the study observational.

Why it matters: The company, which sells AI receptionists, says to judge cost per successful outcome, not cost per answered call, and suggests AI for after-hours calls and routing.

From Drop #027 · Fri, Oct 16, 2026 · Morning
AgentsCustomer serviceLiveHuman in loop: unclear

PolyAI and Otter execs say voice AI hasn't had its ChatGPT moment

PolyAI CTO Shawn Wen told TechCrunch full-duplex models exist but reasoning must get faster; Otter CMO Alex Gay says transcription errors flow into every follow-up action.

Why it matters: Both say people should be told they are talking to AI or being recorded: a baseline for anyone shipping voice agents.

From Drop #018 · Sun, Oct 11, 2026 · Evening