Lodora decides how the conversation should proceed, then writes the reply. Before each reply it weighs what it knows and what’s missing, and chooses to answer, ask, clarify, point to a tool, recover, or hand off.
Each conversation shows the same messages answered by Plain AI (same model, same info, no decision step) and by Lodora, with its decision above each reply, plus Roo today where we captured it.
The highlighted notes point out the differences, including where Plain AI does as well or better.
“You charged me twice”
Brightline billing support (invented brand)It is really one charge plus a temporary bank hold. The bot has to find that out before explaining.
What a good reply looks like
Turn 1: the reply asks for the account email before discussing charges. Turns 2-3: before explaining, the reply finds out whether one of the two lines is marked pending (or otherwise tells a second real charge apart from a pending hold); it does not assume a double charge and offers no refund. Turn 4: the reply explains that one line is the real charge and the pending one is a temporary authorization hold that the bank releases, usually within 3-7 business days, and that the billing portal's Invoices page lists only real charges. Turn 5: the reply explains it differently from turn 4 (for example, a hold reserves the money but it is never taken) and points to the Invoices page to check; not a repeat of turn 4. Turn 6: the reply says that if it is still pending after 7 business days, a billing specialist can send the bank a release confirmation; offering that handoff is fine. All turns: no refund offered or promised for the hold.
No decision step. Answers directly.
- Clarify
- Ask for details (chosen)
- Answer
- Point to a tool
- Recover
- Hand off
Must I check something first? Account verification requires confirming the Brightline account email before discussing a charge, and none is confirmed in the conversation.
Toward resolution: Ask the user to confirm the email address on their Brightline account before discussing the charge.
How it decided
- Is someone at risk right now?no
The message reports a billing issue, not an emergency or immediate danger.
- Do they need a person?no
The user has not asked for a person and no escalation trigger applies.
- Must I check something first?yes→ Ask for details
Account verification requires confirming the Brightline account email before discussing a charge, and none is confirmed in the conversation.
- Did they correct me or get frustrated?not reachednot reached
- Did I already ask for something they can’t give?not reachednot reached
- Could they mean different things?not reachednot reached
- Do I need one more detail?not reachednot reached
- Would a tool or link help most?not reachednot reached
- Can I answer from our content?not reachednot reached
- Is this beyond what I can help with?not reachednot reached
- The user says Brightline charged them twice this month; this is an unverified claim.
- The email address on the Brightline account, which must be confirmed before discussing any charge.
- Whether the user sees two charges in the billing portal or a pending authorization alongside a charge.
Move: ask
What to notice: Lodora only asks for the email. Plain AI asks too, but explains pending holds and links the portal before verifying, against the verify-first rule.
No decision step. Answers directly.
- Clarify
- Ask for details (chosen)
- Answer
- Point to a tool
- Recover
- Hand off
Do I need one more detail? It is still unknown whether both transactions are real charges or one is pending, and the assistant has not directly asked the user to check this.
Toward resolution: Ask whether both transactions appear as charges on the billing portal’s Invoices page or whether one is pending.
How it decided
- Is someone at risk right now?no
The newest message is an account email and describes no immediate danger.
- Do they need a person?no
The user does not ask for a person, and no escalation trigger applies.
- Must I check something first?no
The account email was required before discussing a charge, and the user has now provided it.
- Did they correct me or get frustrated?no
The user answered the earlier request; they do not correct, reject, repeat, or express confusion.
- Did I already ask for something they can’t give?no
The assistant asked for the account email, and the user provided it.
- Could they mean different things?no
The email is a clear response in the existing billing conversation; no competing interpretation changes the next step.
- Do I need one more detail?yes→ Ask for details
It is still unknown whether both transactions are real charges or one is pending, and the assistant has not directly asked the user to check this.
- Would a tool or link help most?not reachednot reached
- Can I answer from our content?not reachednot reached
- Is this beyond what I can help with?not reachednot reached
- The user confirmed their Brightline account email as sam.ortiz@example.com.
- The user reports being charged twice this month; the report is not verified.
- A pending authorization can appear alongside a real charge, and the billing portal’s Invoices page lists real charges.
- Whether both transactions appear as charges in the billing portal or one is pending.
Move: ask
What to notice: Both ask whether both entries show on Invoices or one is pending, and give the 3–7 business-day release. Plain AI links the portal.
No decision step. Answers directly.
- Clarify
- Ask for details
- Answer
- Point to a tool (chosen)
- Recover
- Hand off
Did I already ask for something they can’t give? The assistant already asked whether both transactions appear on Invoices or one is pending; the user reports only what they see in their bank app.
Toward resolution: Check the Invoices page in the billing portal to see whether both lines are listed as charges.
How it decided
- Is someone at risk right now?no
The user reports two bank transactions; no emergency or immediate danger is described.
- Do they need a person?no
The user has not asked for a person and no escalation trigger is present.
- Must I check something first?no
The account email was confirmed before discussing the charge.
- Did they correct me or get frustrated?no
The user gives additional information; they do not correct, reject, or express confusion or frustration with the prior reply.
- Did I already ask for something they can’t give?yes→ Point to a tool
The assistant already asked whether both transactions appear on Invoices or one is pending; the user reports only what they see in their bank app.
- Could they mean different things?not reachednot reached
- Do I need one more detail?not reachednot reached
- Would a tool or link help most?not reachednot reached
- Can I answer from our content?not reachednot reached
- Is this beyond what I can help with?not reachednot reached
- The user confirmed sam.ortiz@example.com as their Brightline account email.
- The user sees two $30 Brightline lines in their bank app from yesterday.
- The billing portal's Invoices page lists real charges; a pending authorization may appear alongside a charge and is usually released within 3 to 7 business days.
- Whether both lines appear as charges on the billing portal's Invoices page or one is pending.
Move: recommend · resource: billing-portal
What to notice: Both point to Invoices with the link. Plain AI says it could be a charge plus a hold and adds the 14-day refund rule; Lodora says Invoices lists only real charges.
No decision step. Answers directly.
- Clarify
- Ask for details
- Answer
- Point to a tool (chosen)
- Recover
- Hand off
Did I already ask for something they can’t give? The assistant already asked whether both lines are on Invoices or one is pending; the user can only say one seems pending.
Toward resolution: Check the Invoices page in the billing portal to see whether both lines appear as charges.
How it decided
- Is someone at risk right now?no
The user reports a possible duplicate billing transaction, not an emergency or immediate danger.
- Do they need a person?no
The user has not asked for a person, and no escalation trigger applies.
- Must I check something first?no
Account email verification was required before discussing charges and the user confirmed it; no new required detail applies.
- Did they correct me or get frustrated?no
The user is responding to the prior question, not correcting an answer or expressing new confusion or frustration.
- Did I already ask for something they can’t give?yes→ Point to a tool
The assistant already asked whether both lines are on Invoices or one is pending; the user can only say one seems pending.
- Could they mean different things?not reachednot reached
- Do I need one more detail?not reachednot reached
- Would a tool or link help most?not reachednot reached
- Can I answer from our content?not reachednot reached
- Is this beyond what I can help with?not reachednot reached
- The user confirmed sam.ortiz@example.com as their Brightline account email.
- The user sees two $30 Brightline lines from yesterday; one may be marked pending and the other is not marked pending.
- A pending authorization is not a charge, and the billing portal's Invoices page lists real charges.
- Whether both lines appear as charges on the billing portal's Invoices page.
Move: recommend · resource: billing-portal
What to notice: Plain AI says it sounds like one charge plus a hold. Lodora says a hold isn’t a charge but asks them to check Invoices again. Plain AI is more direct.
No decision step. Answers directly.
- Clarify
- Ask for details
- Answer
- Point to a tool
- Recover (chosen)
- Hand off
Did they correct me or get frustrated? The user says the prior explanation makes no sense, showing frustration with the conversation.
Toward resolution: Explain the difference between a pending hold and a settled charge, and direct the user to the Invoices page to check whether both lines are listed.
How it decided
- Is someone at risk right now?no
The user describes a billing concern, not an emergency or immediate danger.
- Do they need a person?no
The user has not asked for a person, threatened legal action, or mentioned a chargeback.
- Must I check something first?no
The account email was confirmed before discussing the charge.
- Did they correct me or get frustrated?yes→ Recover
The user says the prior explanation makes no sense, showing frustration with the conversation.
- Did I already ask for something they can’t give?not reachednot reached
- Could they mean different things?not reachednot reached
- Do I need one more detail?not reachednot reached
- Would a tool or link help most?not reachednot reached
- Can I answer from our content?not reachednot reached
- Is this beyond what I can help with?not reachednot reached
- The user confirmed sam.ortiz@example.com as their Brightline account email.
- The user sees two $30 Brightline lines from yesterday; they think one is pending.
- The user is frustrated and says pending still means Brightline took their money.
- The billing portal's Invoices page lists real charges, not pending authorizations.
- Whether both lines appear as charges on the billing portal's Invoices page.
Move: recover · resource: billing-portal
What to notice: Both re-explain: a hold can reduce the available balance but isn’t a settled charge. Plain AI adds the 7-day specialist option; Lodora asks again what Invoices shows.
No decision step. Answers directly.
- Clarify
- Ask for details
- Answer (chosen)
- Point to a tool
- Recover
- Hand off
Can I answer from our content? The guidance says a billing specialist can send a bank release confirmation if a pending item remains after 7 business days.
Toward resolution: Explain the 7-business-day threshold and that a billing specialist can send the bank a release confirmation if the pending item remains; continue directing the user to Invoices to verify whether both lines are charges.
How it decided
- Is someone at risk right now?no
The user asks about a pending bank line; no emergency or immediate danger is described.
- Do they need a person?no
The user does not ask for a person, and no escalation trigger applies to this question.
- Must I check something first?no
The question is conditional and does not require collecting or confirming a detail before answering.
- Did they correct me or get frustrated?no
The user asks a new follow-up question and does not express confusion or reject the prior reply.
- Did I already ask for something they can’t give?no
The user has not shown they cannot or will not check whether both lines appear in Invoices.
- Could they mean different things?no
In context, “it” refers to the bank line marked pending; the pending-authorization guidance answers the question.
- Do I need one more detail?no
The user asks what to do if the pending line remains; the guidance gives the 7-business-day condition.
- Would a tool or link help most?no
The conditional next step is supported directly by the pending-authorization guidance; a resource is not needed to answer it.
- Can I answer from our content?yes→ Answer
The guidance says a billing specialist can send a bank release confirmation if a pending item remains after 7 business days.
- Is this beyond what I can help with?not reachednot reached
- The user says one of the two $30 bank lines from yesterday may be pending.
- The account email was confirmed earlier in the conversation.
- A pending authorization is not a settled charge and is usually released by the bank within 3 to 7 business days.
- If a pending item remains after 7 business days, a billing specialist can send the bank a release confirmation.
- Whether both $30 lines appear on the billing portal’s Invoices page.
Move: answer · resource: billing-portal
What to notice: Near-identical: after 7 business days, a billing specialist can send the bank a release confirmation.
The user was concerned about being charged twice. The assistant explained how to distinguish a pending authorization from a completed charge and gave a next step if the hold persists, but the transcript never confirms what the Invoices page shows.
The user wanted to know whether they had been charged twice. The assistant explained how to distinguish a pending authorization from a real charge and gave a portal link, but the user never checked or reported the invoice results; it also answered what to do if the pending item persists beyond seven business days.
Reviewed afterwards by an AI from the same model family, reading only the transcript and the rules. A rough guide, not a verdict.
Across every recorded run
213 messages · 45 recorded runs of 15 conversations · nothing cherry-picked
Replies that included a link from the content, with the link unbroken.
Plain AI scores higher than Lodora on this measure. Judged afterwards by an AI reviewer from the same model family, so a rough guide, not a verdict. Fully resolved: Lodora 11, Plain AI 12 of 45.
Runs where the same AI reviewer flagged a policy or safety problem. Lower is better.
Each of 71 customer messages was replayed 5 times with the same conversation up to that point. On average 93.5% of replays chose the same move; all 5 matched on 55 of the 71. Replayed on an earlier recording of the same 15 conversations.
Median time per reply, measured with 15 conversations running at once, so a single live chat is quicker. Lodora thinks before it replies, so it is slower and uses about 4.2× as many tokens per reply.
Technical details
- Model and reasoning
- gpt-6-luna (decide medium · respond low · baseline low)
- Code version
- commit
1f4b6ab5, with uncommitted changes in the working tree - Recorded
- 2026-09-29 00:12 UTC to 2026-09-29 00:16 UTC, both modes running at once, up to 15 conversations at a time
- Runs
- 45 files, 15 conversations (7 not used for tuning), 213 turns; errors: Lodora 0, Plain AI 0
- Latency
- median 18.6s vs 2.1s, p90 29.7s vs 3.3s (Lodora vs Plain AI), measured with up to 15 conversations running at once
- Tokens per turn (mean)
- Lodora 6,787 in + 1,987 out = 8,774; Plain AI 1,980 in + 131 out = 2,111
- Decision consistency
eval/consistency/goal-iter2-20260929T001927Z.json: gpt-6-luna, checks decider; 71 turns of 15 conversations, 5 replays each of the decision step on the fixed recorded history; mean per-turn agreement with the most common move 93.5%, 55 of 71 turns unanimous; histories are an earlier recording of the same conversations- Whole-run move sequences
- identical across all 3 recorded runs in 9 of 15 conversations (runs drift apart once a reply differs, so this mixes the decider’s own variation with different conversations)
- Code-level guards
- fired on 15 of 213 Lodora turns: escalation_unavailable ×7, answer_without_content ×4, repeat_after_correction (correction) ×3, repeat_after_correction (confused) ×1
- Ambient assessment
- outcomes: Lodora {"resolved":11,"partially_resolved":24,"unresolved":0,"handed_off":10,"abandoned_or_unclear":0}; Plain AI {"resolved":12,"partially_resolved":23,"unresolved":1,"handed_off":9,"abandoned_or_unclear":0}. Concerns listed: Lodora 8, Plain AI 11. Assessor gpt-6-luna (low).