Key takeaways
- Stress-test escalation handoff in a live pilot with your actual CRM, not a controlled demo with dummy data.
- Get the contract definition of "deflection" in writing before you sign; it changes the reported ROI figure significantly.
- Identify who owns knowledge base updates after go-live, and confirm they can act without an IT ticket.
- Run the 2x volume scenario in any proof of concept to expose pricing spikes, routing limits, and workflow caps before they become migration costs.
Does this platform integrate with our CRM at the routing level, or just the surface?
Most chatbot platforms claim CRM integration. What they usually mean is a one-way connector that loses routing context the moment a conversation escalates. True integration, as Infobip's evaluation criteria describes, preserves account owner, customer segment, and full conversation history at the exact moment of handoff.
The difference matters operationally. A surface-level connector creates a new reconciliation task: someone manually matches the bot conversation to the CRM record after the fact. That is not a reduction in workload.
Run these three tests during any pilot:
- Bidirectional data sync: Confirm that bot actions write back to the CRM record in real time, not in a nightly batch.
- Context preservation at escalation: Verify the agent receives the customer's full conversation history and CRM profile the moment handoff occurs.
- Live edge-case routing: Route a scenario your team actually handles, because as ZoomInfo Pipeline notes, integration failures hide in real conversations, not scripted demos.
Passing all three is the only reliable signal the integration will hold in production.
What happens when our volume doubles?
Ask every vendor to model the platform at 1x, 2x, and 5x your current monthly conversation volume. Pricing tiers, routing logic limits, and workflow complexity caps behave differently under load. The failure modes are rarely visible in a standard demo.
Platforms that look capable today can hit concurrent session ceilings, per-resolution pricing spikes, or routing rule limits at growth scale. These are the constraints that trigger painful migrations 18 months after go-live, when switching costs are high. As Botpress's scalability criteria notes, coding requirements, channel expansion, and load behavior all change materially as usage grows.
The scale question is not theoretical. CGI deployed an AI chatbot for an international financial services firm that now handles 500,000 support conversations per year, with only 6% requiring live-agent intervention. The CGI case study attributes approximately €2 million in annual cost reduction to that deployment.
Ask vendors to confirm in writing what changes at 2x volume. Vagueness here is your answer.
What is the true cost of ownership beyond the monthly fee?
The sticker price covers the subscription. Everything else appears after sign-off, and that is where budgets break.
Ask vendors for a written cost model at your projected volume, not a flat monthly rate. Usage-based, seat-based, and hybrid pricing structures each behave differently at scale. The cheapest option today can become the most expensive option at growth. Formstack's pricing framework recommends a weighted cost breakdown before any procurement decision.
The hidden cost categories to ask about explicitly include:
- Implementation and custom integration build fees
- Per-resolution or per-conversation overage charges
- Knowledge base migration costs from your current tool
- Ongoing support tier costs and what the base plan excludes
- Pricing changes when you add channels, routing rules, or workflow complexity
Do not accept a verbal answer on any of these. A vendor confident in their pricing will put it in writing.
For a full breakdown of SaaS vs custom chatbot pricing, see how much does a chatbot cost before your next vendor call.
Who owns this platform after go-live, and how much IT time does it need?
The day-to-day ownership question determines whether the platform reduces your team's workload or creates a new dependency. Ask these three questions in a live admin console demo, not a verbal walkthrough.
Knowledge base updates: When a product changes or pricing is corrected, can an ops team member make that update without an IT ticket? Ask the vendor to demonstrate this during your evaluation.
Escalation rule changes: When you add a new team or update a routing assignment, who makes that change and how long does it take? If the answer involves a developer or a support request, factor that into your total cost estimate.
Reporting access: Can your ops team pull deflection rates, escalation rates, and handoff data without a developer? Formstack's admin governance framing identifies reporting access as one of the clearest signals of whether a platform is built for ops ownership or IT ownership.
If any of these three questions requires IT involvement as a default, price that dependency into your evaluation before you sign.
Is this a SaaS chatbot or a custom build, and does that distinction matter for our workflows?
The SaaS vs custom build decision is a structural fork, not a preference question. Getting it wrong means overpaying for capability you do not need, or committing to a platform that hits its ceiling before your business does.
SaaS works when your use cases are standard: FAQ deflection, appointment scheduling, order status. If your integrations are pre-built connectors and compliance requirements are low, a SaaS chatbot will serve you well.
A custom build is more likely the right answer when your workflows are non-standard or multi-step. It also fits when compliance requires data residency control, or when your business needs domain-trained AI for healthcare intake, recruitment screening, or financial services.
- SaaS sufficient: standard use cases, pre-built connectors, low compliance risk
- Custom required: non-standard workflows, strict data residency or HIPAA/GDPR compliance requirements, deep CRM integration, domain-trained models
arsuno.ai builds custom AI automation systems for businesses where SaaS chatbots have hit their ceiling. If your workflows fall into the custom column, the next step is a conversation about what that build looks like for your specific stack.
What does your security and compliance documentation actually cover?
"GDPR compliant" as a self-declared checkbox is not the same as documented compliance with an audit trail, data processing agreements, and a defined breach response procedure. Ask for the documentation, not the claim.
Before procurement, request three specific documents from every vendor. First, data residency confirmation: where is data stored, where is it processed, and does that change if you expand to a new region? Second, breach notification SLA: what is the vendor's response timeline, who is notified, and what is your team's responsibility? Third, compliance audit documentation: what does the vendor provide for your internal or external audit, and in what format?
Nextiva's compliance selection criteria frames these as operational questions, not legal ones. Your legal team will need to verify the answers, but your procurement process should surface them before the contract stage, not after.
Frame these questions for your legal counsel to review. This article gives you the evaluation criteria; your counsel determines whether the answers meet your obligations.
How will we measure whether this chatbot is actually working?
Ask vendors to define "deflection" in writing before the contract. A bot reply is not a resolved outcome. The definition changes the reported ROI figure significantly, and vendors who resist this question are telling you something important about how they calculate success.
The metrics that matter for leadership reporting are specific. Containment rate measures conversations resolved without human intervention. Escalation rate tracks how often the bot hands off, and why. First-contact resolution rate shows whether escalated conversations are resolved on the first human touch. Average handle time for escalated conversations tells you whether the handoff is adding efficiency or friction.
"Conversations handled" is not a metric; it is a count. Ask whether reporting is accessible to your ops team without a developer. If pulling a deflection report requires a support ticket, that is a governance risk, not a minor inconvenience.
Reporting access is also a signal of platform maturity. A vendor confident in their results makes those results easy to see.
What does your escalation flow look like, and what routing logic do you support?
Escalation quality is the clearest test of whether a chatbot platform is built for real operations. Ask vendors to demonstrate the handoff live, using your actual CRM in a pilot environment, not a controlled demo with dummy data. This single test separates a platform that reduces workload from one that creates a parallel queue your team manages manually.
Routing logic must support account owner assignment, territory rules, segment-based routing, and round-robin fallback. Ask whether these are configurable without code changes. As ZoomInfo Pipeline's routing framing notes, routing configuration depth is one of the clearest indicators of whether a platform is built for real operations or demo environments.
At handoff, the human agent must receive three things: the full conversation transcript, the CRM record context including account tier and last interaction, and the reason for escalation. Infobip's handoff criteria identifies context preservation as the most operationally significant integration requirement.
Treat a forced escalation scenario as a mandatory pilot test case.
Frequently Asked Questions
What happens when the chatbot doesn't know the answer?
The platform should route to a human agent rather than returning a dead-end response.Verify that the bot preserves the full conversation context during that fallback, so the agent does not start from scratch. Ask vendors to show you this scenario in a live pilot using a real edge case from your support queue. A platform that handles unknowns gracefully is a signal of production-grade design, not just demo polish.
How does the knowledge base stay current?
After go-live, a named person on your ops team should own knowledge base updates without needing IT support. Ask vendors to demonstrate the update workflow in the admin console, including how quickly a change goes live after it is made. Policy changes, pricing corrections, and new product information should be deployable within hours, not days.
What does the escalation flow look like for the human agent?
At handoff, the agent should see the full conversation transcript, the customer's CRM profile, and the reason for escalation on a single screen. Routing should assign the conversation to the correct team or individual based on account owner, region, or segment, not just a general queue. Ask vendors to run a live escalation in your pilot environment using a real scenario, not a scripted demo flow.
How is deflection defined in the contract?
A bot reply and a resolved outcome are not the same thing, and the contract definition determines which one counts as a deflection. Ask vendors to provide the exact definition in writing before you sign, and confirm whether it is tied to customer confirmation, session closure, or simply a bot response. The definition affects both your reported ROI and any pricing tied to resolution volume.
Does pricing scale with usage or team size?
Pricing models vary across usage-based, seat-based, and hybrid structures, and each behaves differently as volume grows. Ask vendors to model your cost at current volume, 2x volume, and 5x volume, including any overage charges, routing rule limits, or tier changes. Hidden implementation fees, support tier costs, and per-channel charges often appear only after the base contract is signed.
Can a human agent see the full conversation history at handoff?
Yes, but verify this in a pilot rather than taking a vendor's word for it. The agent should receive the full transcript, including any data the customer submitted during the bot interaction, alongside the CRM record on a single screen. Ask vendors to demonstrate this with a real customer scenario, not a controlled demo with dummy data.
Is a custom chatbot build worth the investment for mid-market businesses?
A custom build is worth evaluating when pre-built connectors cannot support your CRM workflows or compliance requirements. Mid-market businesses with non-standard routing, strict data residency needs, or domain-specific AI requirements often find SaaS platforms hit a ceiling before their business does. The decision comes down to whether the long-term cost of workarounds and manual reconciliation exceeds the upfront build investment.
.webp)
