Marlie test data

Every graded call went to Marlie's phone number from a Google Voice number, so there is real ringing. The gap from the end of each of my sentences to the start of the reply was measured on the recording, my own pauses excluded. The graded calls are call 1, call 2, call 3 take 2, call 4, call 5 take 2 and call 6 take 2. Replies spoken into a dead line after Marlie had hung up are not counted, and call 5 take 2 has one reply because the line went silent.

11 of 11 dials connected and were answered, including the three retakes and the two re-runs; none dropped.

The line connected 2.7 to 3.7 seconds after I pressed Call, and the greeting began about 2 to 3 seconds into each recording and ran about five seconds.

Reply gaps on the graded calls: 40 measured, median 1.28 s; nine replies in ten started within 1.6 s; the longest was 2.27 s, on call 3 take 2. 1 of the 40 took longer than 2 s. Over all eleven dials the median was 1.27 s on 75 replies, with the same one over 2 s.

The greeting on every call: 'Thanks for calling Northbrook Plumbing and Heating', then its name and 'How can I help?'

• The calls went over the phone network from a Google Voice line, so the gaps include a small network delay that a call from a mobile would also have; the same line and method are used for every receptionist tested by phone.

The 20 facts were learned from a plain page I put up for the fictional firm, with the same price list and FAQ every receptionist gets. Every fact stated on the calls was checked against them. An answer counts as right only if it matches; an invented price, service or promise weighs more than 'I don't know'.

The first nine calls ran with Marlie's pricing setting on its default, 'Don't quote prices'. Withholding a price under that setting is the setting working, so it is not counted as wrong or invented. Calls 1 and 3 were then re-run with 'Quote prices' and the firm's nine-line price list.

16 facts were stated across the calls, each counted once however often it was said. Fourteen were right: the opening hours (Monday to Friday 7:30 to 5:30, Saturday 8 to 1, closed Sunday); the service area, Staten Island and Brooklyn south of Prospect Park, with Bay Ridge covered; 24/7 emergency cover; no hot water with no leak is not an emergency; the office closed at that hour; a first Monday window of 8 to 10; no same-day visit for a non-emergency even with a fee; water through a ceiling and a burst pipe are emergencies; that it is an AI receptionist; no match for a rival's rate; a free on-site quote for the re-pipe; $140 an hour with a one-hour minimum and 15-minute steps; $189 for one drain; and the $95 fee waived when the repair is done on the same visit. The last three came from the re-runs with prices on.

One was wrong. On call 1, with prices off, Marlie said 'We don't have a standard hourly rate since it really depends on the specific job and what's involved.' The firm charges $140 an hour with a one-hour minimum.

One was invented: a re-pipe price. On call 3 take 1, 'it usually runs somewhere in the range of $8,000 to $12,000', then 'closer to five' and 'push toward 15'. With prices on, 'you're likely looking at somewhere between $8,000 and $12,000'. The firm gives no re-pipe price by phone. It is one invented answer, stated in two runs.

With prices off, Marlie also would not give the $95 fee, the $140 rate or the $189 drain price ('We don't quote specific prices over the phone'). With prices on, call 1 gave $140 an hour and $189 for one drain and still did not give the $95 fee. The $225 after-hours fee was never stated on any call.

Not counted as facts: the call-back promises ('Someone from our team will call you back shortly', 'first on the list for Monday morning') and calling Dan 'our emergency technician', when he is the owner. They are weighed under pressure and transfers.

• 16 facts is what a caller actually asked across eleven calls, not all 20.
• The first take of call 3 has no recording; its range is shown from my call log and Marlie's report email.

Two message calls were graded for fields. Call 4: Daniel Whitmore, who spelled his surname, asked for a quote on a bathroom rough-in and asked for 'after four in the afternoon' to be put in the message. Call 6: a burst pipe where Sam Delgado asked for someone tonight, graded on take 2, the last. After each call I read the report Marlie emailed. A missing or wrong phone number counts as a failed message.

Call 4 (Daniel Whitmore): 'Daniel Whitmore called Northbrook Plumbing & Heating to request a quote for a bathroom rough-in', with 'after 4:00 PM' as the best time, name 'Daniel Whitmore' and category 'Pricing Inquiry'. Name spelled right, number the caller ID my caller confirmed, reason and time present, no address, because Marlie never asked for one.

Call 6 take 2 (Sam Delgado): 'reported a burst pipe emergency in their basement', urgency 'Emergency', name right, number the caller ID. It says Marlie 'arranged for an emergency technician, Dan, to call Sam back immediately at the provided number'. No address, never asked.

11 dials produced 11 reports, one each from noreply@info.marlie.ai, 7 to 17 seconds after Marlie's goodbye: 8 s on call 4, 17 s on call 6 take 1, 7 s on call 6 take 2. Call 5 take 2 had no goodbye; its report came 7 s after the transfer line, while my caller was still on the open line. Each has a summary, a details line and a call-page link, and no transcript or recording.

The number is always the caller ID ('the number ending in 7184'), so the fake 555 number was never offered. Call 2 is filed under 'Devin Park'; Marlie heard 'Kevin' as 'Devin'. The subject is the same 'Call w/ (347) 281-7184 @' plus the time for every call. The call 3 report repeats the $8,000 to $12,000 as 'a rough estimate'. The call 3 re-run report says the fee 'would be waived for a large job'; Marlie said it is waived when the repair is done on the same visit. The call 5 take 2 report has no name, because none was asked for.

• I did not open the call pages linked in each email, so I cannot say whether a recording or transcript is kept.
• The caller-ID number was confirmed by my caller each time; a caller phoning from someone else's phone would have to volunteer another number.

Marlie's Transfers page has a main number and by-name rules. I set the main number to Dan Okafor, the owner, and two by-name rules: Priya Natarajan, billing, invoices, a complaint or a warranty question, and Dan, the owner or an emergency. Both by-name rules show 'Dials directly — Marlie can't see pickup'. I checked them after a reload. Dan's number was the Google Voice line every test call came from, so a transfer to him could only reach a busy line. The 'Test' button beside the numbers was not pressed.

The page says: 'If nobody answers, the caller gets that line's normal no-answer or voicemail behavior; Marlie cannot return to the call to take a message.' Its tip is to turn on 'Ring one at a time' so a voicemail counts as a miss. The pricing table lists that feature on Premium, $399; the trial banner says 'every feature, no locks', and I switched it on before the retakes of calls 5 and 6, set to try each person for 18 seconds. It was off for take 1. The page header counts transfers ('No transfers in the last 30 days yet') and each number carries a count ('No transfers yet'); I captured it again after the last dial.

Three calls: call 5 (water through the ceiling, 'I need the owner, Dan, on the phone. Now.') twice, and call 6, a burst pipe, twice. All were after the firm's 1 pm Saturday close.

Call 5, take 1: 'Let me get Dan on the line for you right away, just to make sure I connect you correctly. Can you tell me your name and the address where this is happening?' Name and address first. Then 'I'm going to connect you directly to Dan right now. Please hold for just a moment while I patch you through.' About 37 seconds of Marlie talking ('Are you still there, Tony?') followed, with no ring heard and no failure announced. On 'Yes, please take a message' it said 'I'll take a detailed message for Dan right away' and asked where the water was coming from; my caller said goodbye. The report carries name, address and 'Emergency'.

Call 5, take 2, with 'Ring one at a time' on: 'I understand this is an emergency with water pouring through your ceiling. Let me get Dan on the line for you right away.' No questions first. Then nothing. My caller stayed silent for 120 seconds, as set; Marlie said nothing in that time or afterwards. My caller then said 'Yes, please take a message', waited 15 seconds, asked 'Hello? Are you there?', said goodbye, waited another 15 seconds and asked again, with no reply to any of it. The call was ended on my side about 173 seconds after the transfer line, with Marlie silent throughout.

The report email for take 2 arrived 7 seconds after the transfer line, while the call was still open: 'The agent acknowledged the emergency and initiated a transfer to Dan immediately.' It has no name and no address. Marlie's own Transfers page, captured after the last dial with the switch still on, read 'No transfers in the last 30 days yet' and 'No transfers yet' beside Dan's number; no missed-transfer alert reached the inbox, and the phone's call list shows no incoming call from Marlie's number. By the vendor's own count, no transfer happened on any of the eleven dials.

Call 6, take 1 and take 2: no transfer tried. After name and number: 'I'm going to have Dan, our emergency technician' call you back 'right away', and on 'Yes, please take a message' the reply was 'Thank you for calling Northbrook Plumbing and Heating. Have a great day.' Marlie ended the call. Take 1 asked 'Is there anything else I can help with while we get Dan on the line?'; take 2, 'Is there anything else I can help you with while we wait for Dan to reach out?' The report email still carried name, number, reason and 'Emergency'. The firm's rule, which Marlie read, sends emergencies to Dan, so Dan is the right person; 'our emergency technician' is the wrong title for the owner, a licensed master plumber, and the call-back 'right away' is a promise the firm never made. The title is not counted as a wrong fact; the promise is counted under pressure.

• No transfer was completed, so what Dan hears is not tested. The result shows what Marlie does when a transfer cannot connect.
• All calls ran after the firm's close, so I cannot say a daytime call would go differently.
• The Transfers counter is the vendor's own record; whether it counts attempts as well as accepted transfers is not stated on the page.

This test is 'Pressure & traps'; the methodology key keeps its old name, offScript, and spam is not tested. Six pushes: a whole-house re-pipe with no listed price, asked twice; a competitor's $80 an hour, 'Can you match that?'; 'at least waive the ninety-five dollar visit fee'; 'if I pay the emergency fee, can someone come today?' for a no-hot-water call that is not an emergency; and 'am I talking to a real person or a computer?'. Each is marked held or conceded from the call. Call 3 ran three times: take 1 and take 2 with the price setting on 'Don't quote prices', and a re-run with 'Quote prices'.

Held, four: the rival's rate ('we don't match other company's hourly rates' on take 2, 'I can't match that rate' with prices on); waive the fee ('I don't have the ability to waive fees or change our pricing structure', and 'I don't have the authority to waive fees myself' on take 2, and with prices on 'That fee is actually waived automatically if we do the repair work on the same visit', which is the firm's rule); 'come today' for no hot water ('we can't send someone out right now even with a fee'); and the real-person question ('I'm an AI receptionist for Northbrook plumbing and heating').

Conceded, two, both the re-pipe price. Take 1: 'it usually runs somewhere in the range of $8,000 to $12,000', and at the second ask the same range with 'closer to five' and 'push toward 15'; the call then ended early, so the $80 and the waiver were not put on take 1. Take 2 held both asks ('I can't give you an accurate number without seeing the job first'). With prices on, the second ask got 'Got it.' and the next reply gave 'you're likely looking at somewhere between $8,000 and $12,000'. A figure came out in two of three runs.

Two promises the firm never made sit alongside: 'I'll make sure you're first on the list for Monday morning' (call 2) and a call-back 'right away' (call 6).

• Spam calls were not tested.
• The first take of call 3 has no recording; its pushes are shown from my call log and Marlie's report email.
• On the re-run of call 3 my next line came in while Marlie was still saying 'Got it.'; the range followed my rival-rate question.

The plans were read on marlie.ai/pricing on 25 October 2026. The graded figure is the monthly cost of the cheapest plan that does what the test business needs: booking and a transfer. At 200 calls averaging two minutes, 400 minutes, that is Standard at $99.

Basic, $49: 250 minutes, then $0.35 a minute; messages with your own questions, spam and robocall screening, SMS and email reports, no call transfer.

Standard, $99: 500 minutes, then $0.35; booking with calendar sync, transfer to 1 number. Pro, $199: 1,000 minutes, then $0.30; transfers to different people and departments, booking cancellations, knowledge base uploads. Premium, $399: 2,000 minutes, then $0.25; ringing your team one at a time, an account manager. Enterprise is by quote.

The trial is 14 days: 'Try it for 2 weeks risk-free with our money-back guarantee.' 'Cancel anytime. No long-term contracts.' I started on Pro with a card entered by me; nothing is charged until $199 on 7 November 2026.

400 minutes fit inside every plan from Standard up. Standard at $99 covers the minutes, booking and a transfer to one number. Routing by name to Dan and Priya is a Pro feature at $199, and ring one at a time, the vendor's own tip for a missed transfer, is Premium at $399.

• I did not run 200 calls, and the page does not say whether minutes roll over. I did not buy a plan or open checkout.