Build a customer support chatbot that learns from past tickets and conversations

Your support inbox fills up by 9 a.m. The same five questions appear across email, Slack, and your help desk system. Someone asks about billing cycles. Another customer wants to reset their password. A third needs clarification on your refund policy. Your team answers these tickets individually, manually, every single day. Meanwhile, your best support agents are trapped answering FAQ variations instead of solving actual problems. The cost: salary hours burned on repetition, slower response times, and customers who could have solved their problem in two minutes instead waiting for a human. You don't need a dedicated engineering team or a six-month project to fix this. You can build a working customer support chatbot that learns from your actual ticket history and past conversations within a week, using tools that already exist. The chatbot won't be generic; it will know your products, your policies, and the specific way your team explains solutions. When it hits something outside its knowledge base, it will escalate cleanly to a human. The workflow runs in the background, continuously improving as you feed it new resolved tickets. This is the Alchemy approach: connect existing tools in a way that eliminates handoff friction and creates a system that compounds knowledge over time.

The Automated Workflow

The core architecture runs like this: your resolved support tickets (the ground truth) feed into a searchable knowledge base; new incoming customer queries hit that base first; a conversational AI answers if confidence is high, escalates if uncertain; and every resolved interaction loops back to improve future responses.

Step 1: Extract knowledge from past tickets

Start by exporting your support ticket history (CSV or JSON format) from your ticketing system. Use Chat With PDF by Copilot.us to ingest this data. Although it's built for PDFs, you can convert your CSV to a structured text document, or export ticket transcripts as formatted documents. The tool will let you upload multiple files, which is helpful if you split tickets by year or category. Your initial prompt to Chat With PDF should be something like:

CopyYou have access to 200 resolved customer support tickets. For each ticket, extract:
1. The customer's original question or problem
2. The support agent's final resolution or answer
3. Any relevant product features mentioned
4. Refund, billing, or policy decisions Output as a structured JSON list. Include only tickets marked "resolved" or "closed".

This gives you a rough knowledge base. It's not perfect, but it's a starting point and requires zero manual formatting.

Step 2: Enrich the knowledge base with meeting context

Use Cogram to attend your weekly support team meetings. Set it to record and transcribe, then extract action items and discussion topics. Cogram will create a summary that includes clarifications your team made about policy, workarounds for known bugs, or patterns in what customers ask about. Export this summary and feed it to MindPal as a secondary knowledge source. Why? Because your tickets show what was resolved, but your meetings show what your team actually thinks about those resolutions. A ticket might say "restart the app," but in the meeting someone mentioned a deeper root cause that doesn't appear in the ticket text. Cogram captures that.

Step 3: Build the multi-agent system in MindPal

MindPal is where the workflow gets intelligent. Create a team of two agents:

Agent 1: Knowledge Retriever.

Takes incoming customer questions and searches your exported ticket data and Cogram summaries. Returns the top three matching resolutions from past tickets, along with confidence scores.

Agent 2: Conversational Responder.

Takes the retrieved results and crafts a human-sounding answer. If confidence is below 60%, it flags the ticket as "needs human review." Your MindPal configuration might look like this conceptually:

CopyAgent 1 (Retriever): Input: customer_question (string) Action: Search ticket_database and meeting_summaries Output: top_3_matches with confidence_scores Agent 2 (Responder): Input: customer_question + top_3_matches Model: Claude Sonnet 4.6 (fast, good for conversational tone) Prompt: "Use the matching past resolutions to answer this question. If none of the matches have >70% relevance, output {escalate: true, reason: 'No confident match found'}" Output: response (string) or escalation_flag (boolean)

Step 4: Connect to your ticketing system with n8n or Zapier

Use n8n (self-hosted or cloud) or Zapier to listen for new incoming support tickets. The workflow should: 1. Receive a new ticket via webhook or API polling.

2. Extract the customer's question.

3. Send it to your MindPal workflow.

4. Wait for the response.

5. If confidence is high, post the reply as an automated comment and mark the ticket as "auto-responded."

6. If escalation is flagged, assign it to the next available support agent with MindPal's findings as context. Here's a pseudocode representation for n8n:

CopyTrigger: Webhook (incoming ticket from Help Scout, Zendesk, or your system) Step 1: Extract question from ticket JSON const question = incoming_payload.description; Step 2: Call MindPal API with question POST https://api.mindpal.io/v1/workflows/support-chatbot/run Body: { "workflow_id": "support_chatbot", "input": { "customer_question": question, "ticket_id": ticket_id } } Step 3: Check response if (response.escalate === false && response.confidence > 0.70) { // Auto-reply POST to_ticketing_system/tickets/{ticket_id}/comments Body: { "body": response.answer, "is_internal": false, "from": "Support Bot" } // Mark as resolved PATCH to_ticketing_system/tickets/{ticket_id} Body: {"status": "closed"} } else { // Escalate to human PATCH to_ticketing_system/tickets/{ticket_id} Body: { "status": "pending", "internal_note": response.reasoning, "priority": "normal" } } Step 4: Log outcome Log ticket_id, question, response, confidence to analytics database

Step 5: Use Okara AI for sensitive replies

If your support involves sensitive information (payment details, account credentials, health data), use Okara AI to draft replies that handle that information securely. Okara encrypts everything until you explicitly enable it, which is important if your MindPal workflow or n8n instance sits in a shared environment.

Step 6: Human oversight via Twig

Twig sits on top of the escalated tickets. It's designed to help support agents resolve issues faster by providing suggestions and context. When a ticket is escalated from your chatbot, Twig will see the bot's reasoning, search results, and confidence score, then suggest answers to the human agent. The agent can accept the suggestion, edit it, or reject it entirely. Each decision trains the system further. Wire Twig to your n8n workflow so that whenever a ticket is marked as "escalated," it's automatically available in Twig's queue with full context pre-loaded.

CopyTrigger: Ticket escalated from MindPal Action: POST to Twig API POST https://api.twig.ai/v1/cases Body: { "title": customer_question, "description": "Bot escalated: " + mindpal_reasoning, "context": { "suggested_answer": mindpal_response, "confidence_score": confidence, "ticket_id": ticket_id, "ticket_url": link_to_ticketing_system }, "priority": "normal" }

Step 7: Continuous loop

Every Friday, export all resolved tickets from the week (both bot-resolved and human-resolved) and re-ingest them into your Chat With PDF instance. This means your knowledge base grows automatically. Over time, the bot answers more tickets with higher confidence, and human agents only see the genuinely tricky cases.

The Manual Alternative

If you want tighter control over which answers go live, you can pause the automation before the "mark as resolved" step. Instead, have n8n create a Slack channel called #bot-review where proposed answers appear for 30 minutes. A human clicks "approve" or "reject," and only approved answers post to tickets. This adds friction but reduces the risk of a bad answer going live. Alternatively, use Claude Code to write a custom approval UI. Feed it the structured ticket data and MindPal responses, and it will generate a simple web interface where your team reviews batches of answers before they ship.

Pro Tips

Monitor confidence drift.

After the first month, check how many escalations are happening.

If it's above 40%, your ticket data was probably messy or inconsistent. Spend an afternoon cleaning up your CSV (removing duplicates, standardising product names) and re-uploading to Chat With PDF.

Watch rate limits on the ticketing system API.

If you have more than 50 incoming tickets per day, n8n can hit Zendesk or Help Scout API limits. Add a queue step that batches requests and spreads them over time. Use Claude Sonnet 4.6, not o4-mini, for the Responder agent in MindPal. Sonnet is faster and cheaper for this use case. The o-series models are overkill for answering questions from a fixed knowledge base.

Set a confidence threshold, then measure it.

Start at 70%. If you're getting complaints about wrong answers, raise it to 75% or 80%. If escalations are too high and your team is overwhelmed, lower it to 60%. Track the threshold weekly.

Segment your knowledge base by product or customer type.

If you sell both B2B and B2C, create separate MindPal workflows for each. A B2C customer asking about a password reset should not see answers meant for enterprise billing teams.

Cost Breakdown