Build an AI Email Agent with Inbound Parsing

An AI email agent reads incoming emails, understands them with an LLM, and takes action — auto-replying, routing to the right team, creating tickets, or extracting data. This guide builds one from scratch using inbound email parsing and Claude.

---

How It Works

Incoming email
    ↓
MX record → SendPigeon parses it
    ↓
Webhook POST → your server
    ↓
LLM classifies intent + extracts data
    ↓
Agent acts (reply, route, tag, create ticket)

The inbound parsing service handles the hard part (SMTP, TLS, spam filtering, MIME parsing). Your agent gets clean JSON.

---

Step 1: Set Up Inbound Parsing

If you haven't already, set up inbound email parsing:

1. Add MX record: 10 inbound.sendpigeon.dev
2. Configure webhook URL in the SendPigeon dashboard
3. Verify it works with the built-in webhook tester

---

Step 2: Classify Incoming Emails

Start with classification — understand what each email is about before taking action:

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic();

type EmailClassification = {
  category: "support" | "billing" | "sales" | "spam" | "other";
  priority: "high" | "medium" | "low";
  summary: string;
  sentiment: "positive" | "neutral" | "negative";
  actionRequired: boolean;
};

async function classifyEmail(email: {
  from: string;
  subject: string;
  text: string;
}): Promise<EmailClassification> {
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 500,
    messages: [
      {
        role: "user",
        content: `Classify this email. Return JSON only.

From: ${email.from}
Subject: ${email.subject}
Body: ${email.text}

Return this exact JSON structure:
{
  "category": "support" | "billing" | "sales" | "spam" | "other",
  "priority": "high" | "medium" | "low",
  "summary": "one sentence summary",
  "sentiment": "positive" | "neutral" | "negative",
  "actionRequired": true/false
}`,
      },
    ],
  });

  const text = response.content[0].type === "text" ? response.content[0].text : "";
  return JSON.parse(text);
}

---

Step 3: Build the Webhook Handler

Wire classification into your inbound webhook:

import express from "express";
import { createHmac } from "crypto";

const app = express();
app.use(express.json());

const WEBHOOK_SECRET = process.env.SENDPIGEON_WEBHOOK_SECRET!;

app.post("/api/inbound", async (req, res) => {
  // Verify signature
  const signature = req.headers["x-webhook-signature"] as string;
  const timestamp = req.headers["x-webhook-timestamp"] as string;
  const payload = JSON.stringify(req.body);

  const expected = createHmac("sha256", WEBHOOK_SECRET)
    .update(`${timestamp}.${payload}`)
    .digest("hex");

  if (signature !== expected) {
    return res.status(401).send("Invalid signature");
  }

  // Acknowledge immediately, process async
  res.status(200).send("OK");

  const { data } = req.body;
  await handleInboundEmail(data);
});

async function handleInboundEmail(data: {
  id: string;
  from: string;
  to: string;
  subject: string;
  text: string;
  html: string;
}) {
  // 1. Classify
  const classification = await classifyEmail({
    from: data.from,
    subject: data.subject,
    text: data.text,
  });

  console.log(`[${classification.category}] ${classification.priority} — ${classification.summary}`);

  // 2. Skip spam
  if (classification.category === "spam") {
    return;
  }

  // 3. Route based on classification
  switch (classification.category) {
    case "support":
      await createSupportTicket(data, classification);
      if (classification.priority === "high") {
        await notifySlack(`🚨 High priority support: ${classification.summary}`);
      }
      break;
    case "billing":
      await forwardToBilling(data, classification);
      break;
    case "sales":
      await createLeadInCRM(data, classification);
      break;
    default:
      await forwardToGeneral(data, classification);
  }
}

app.listen(3000);

---

Step 4: Add Auto-Replies (Carefully)

Once classification is reliable, add auto-replies for well-understood categories:

async function generateReply(email: {
  from: string;
  subject: string;
  text: string;
  classification: EmailClassification;
}): Promise<{ html: string; confidence: number }> {
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 1000,
    system: `You are a helpful support agent for [Your Company].
Be concise, friendly, and professional.
If you're not confident you can fully resolve the issue, say so and let them know a human will follow up.
Return JSON: { "reply": "your HTML reply", "confidence": 0.0-1.0 }`,
    messages: [
      {
        role: "user",
        content: `Category: ${email.classification.category}
Priority: ${email.classification.priority}

From: ${email.from}
Subject: ${email.subject}
Body: ${email.text}

Draft a reply.`,
      },
    ],
  });

  const text = response.content[0].type === "text" ? response.content[0].text : "";
  const result = JSON.parse(text);
  return { html: result.reply, confidence: result.confidence };
}

Send with confidence threshold

async function autoReplyIfConfident(
  inboundId: string,
  email: { from: string; subject: string; text: string },
  classification: EmailClassification
) {
  const { html, confidence } = await generateReply({ ...email, classification });

  if (confidence >= 0.85) {
    // High confidence — auto-send
    await sendReply(inboundId, html);
    console.log(`Auto-replied to ${email.from} (confidence: ${confidence})`);
  } else {
    // Low confidence — queue for human review
    await queueForReview(inboundId, { email, classification, draftReply: html, confidence });
    console.log(`Queued for review: ${email.from} (confidence: ${confidence})`);
  }
}

async function sendReply(inboundId: string, html: string) {
  await fetch(`https://api.sendpigeon.dev/v1/inbound/emails/${inboundId}/reply`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.SENDPIGEON_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ html }),
  });
}

---

Step 5: Extract Structured Data

Beyond classification, use the LLM to extract actionable data:

async function extractData(email: { subject: string; text: string }) {
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 500,
    messages: [
      {
        role: "user",
        content: `Extract structured data from this email. Return JSON only.

Subject: ${email.subject}
Body: ${email.text}

Extract:
{
  "orderNumber": "string or null",
  "accountEmail": "string or null",
  "issueType": "string description",
  "requestedAction": "string description",
  "urgency": "immediate" | "soon" | "whenever"
}`,
      },
    ],
  });

  const text = response.content[0].type === "text" ? response.content[0].text : "";
  return JSON.parse(text);
}

---

Architecture for Production

                    ┌──────────────┐
                    │  MX Record   │
                    └──────┬───────┘
                           ↓
                    ┌──────────────┐
                    │  SendPigeon  │
                    │   Inbound    │
                    └──────┬───────┘
                           ↓
                    ┌──────────────┐
                    │   Webhook    │
                    │   Handler    │
                    └──────┬───────┘
                           ↓
                    ┌──────────────┐
                    │  Job Queue   │ ← Don't block the webhook
                    └──────┬───────┘
                           ↓
              ┌────────────┼────────────┐
              ↓            ↓            ↓
        ┌──────────┐ ┌──────────┐ ┌──────────┐
        │ Classify │ │ Extract  │ │   Route  │
        └────┬─────┘ └────┬─────┘ └────┬─────┘
             ↓            ↓            ↓
        ┌──────────┐ ┌──────────┐ ┌──────────┐
        │  Reply?  │ │   Save   │ │  Notify  │
        └──────────┘ └──────────┘ └──────────┘

Key principles:

1. Acknowledge the webhook immediately — return 200, process async via a job queue
2. Classify first, act second — get routing right before enabling auto-replies
3. Confidence thresholds — only auto-send when the model is sure
4. Human fallback — queue uncertain cases for review
5. Log everything — store the classification, confidence, and any generated reply for auditing

---

Cost Considerations

For most support use cases, Haiku handles classification and Sonnet handles reply generation. At 1,000 emails/day, that's ~$5-10/day in LLM costs.

---

What You Can Build

• AI support agent — auto-classify and reply to common questions
• Email-to-Slack router — classify and post to the right Slack channel
• Invoice processor — extract amounts, dates, and vendor info from emailed invoices
• Lead qualifier — score inbound sales emails and route to the right rep
• Document intake — classify and store emailed documents (contracts, reports, receipts)
• Auto-labeler — tag and organize emails in a shared inbox

---

Next Steps

• Set up inbound email parsing if you haven't already
• Receive emails in Next.js or Node.js
• Learn about email sequences to build automated follow-up flows
• Check the SendPigeon docs for full inbound API reference