# Simulation report

Total questions: **1178**

## 1. Paraphrased real questions (should be answerable)

- Total 938 | answered 919 (97%) | wrongly escalated 13 (1%) | expected source missed 4

## 2. Scenario coverage (correct behavior per category)

- emotional_struggle expect `answer`: 30/30 correct (100%)
- technical_problem  expect `answer`: 25/30 correct (83%)
- program_curiosity  expect `answer`: 24/30 correct (80%)
- billing_general    expect `answer`: 24/30 correct (80%)
- account_specific   expect `escalate`: 18/30 correct (60%)
- off_topic          expect `decline`: 29/30 correct (96%)
- medical            expect `escalate`: 18/30 correct (60%)
- human_request      expect `escalate`: 30/30 correct (100%)

## 3. Warmth (coach-like tone), n=148

- Average warmth: **3.99 / 5**
- Lowest-warmth answers:
  - [1/5] "how to log out"
  - [1/5] "is thc like nicotine? do both make dopamine?"
  - [1/5] "nicotine absorption: chewing vs smoking diff?"
  - [2/5] "what if i can't finish the program before my sub runs out?"
  - [2/5] "youtube video not working"
  - [2/5] "refund policy unfair, need exception"
  - [2/5] "still get cravings after a few days? why?"
  - [2/5] "how to buy a gift card?"

## 4. Top answerable-but-escalated cases (fix these first)

- sim=0.866 | asked: "cerebral palsy. will quitting cause withdrawal?" | expected: "If i have cerebral palsy, can quitting smoking cause wi"
- sim=0.838 | asked: "i have ocd. quitting makes me fear routine changes. help?" | expected: "After quitting, I'm afraid of changing routine as I hav"
- sim=0.835 | asked: "why hard to breathe after 2 hrs without smoking?" | expected: "Why do I have trouble breathing after going 2 hours wit"
- sim=0.822 | asked: "Got OCD, and quitting smoking means changing my routine. I'm" | expected: "After quitting, I'm afraid of changing routine as I hav"
- sim=0.819 | asked: "upi deducted, sub not active" | expected: "Amount deducted via UPI but subscription is not active"
- sim=0.811 | asked: "got a tech issue not listed." | expected: "For ANY technical issue not listed here."
- sim=0.804 | asked: "If I'm a heavy smoker, how often am I supposed to do this mi" | expected: "I am a chain smoker, how many times a day should I smok"
- sim=0.801 | asked: "Does stopping smoking have anything to do with snoring more " | expected: "is it any connection of quit smoking with snoring in ni"
- sim=0.767 | asked: "i was charged after canceling free trial" | expected: "Charged after cancelling subscription during free trial"
- sim=0.758 | asked: "Got charged even though I canceled my free trial." | expected: "Charged after cancelling subscription during free trial"
- sim=0.748 | asked: "apple renewed sub" | expected: "Apple has renewed"
- sim=0.744 | asked: "My UPI payment went through but my sub isn't active, what's " | expected: "Amount deducted via UPI but subscription is not active"
- sim=0.664 | asked: "Why did Apple just renew my subscription?" | expected: "Apple has renewed"