Narrative Violation: In B2B customer support, AI is a Copilot, Not a Replacement

The state of agentic customer support, featuring data from Pylon

America | Tech | Opinion | Culture | Charts

We’re excited to announce a16z’s FDE Fellowship: an 8-week cohort for FDEs and Applied AI leaders at the forefront of deploying AI into real-world enterprises. - AD

The two functions where AI has made the most obvious progress in doing real work have been coding and customer support. So, naturally, they’re the two places we look for early clues as to the question, “Is AI replacing jobs, or is it augmenting jobs?” With coding, it certainly seems to be the latter: developers are busier than ever. But people don’t usually think of customer support as a discipline where AI will augment humans: they think it’ll simply replace them. (Even though, as we posted in our chart roundup last week, customer support jobs are outpacing the broader job market.)

In collaboration with Pylon, the agentic support platform for B2B companies, we looked at the data and we found a pretty resounding answer: AI is acting much more as an invisible triage agent or copilot than it is as an end-to-end support handler. It’s changing the front line of customer support by filtering noise from real inbound, routing tickets, handing the hard ones to specialists with customer context attached, and improving human efficiency and capability on those hard tickets. And when AI actively engages on a ticket, it cuts the human’s workload by a third, relative to when it silently hands off.

Pylon’s CEO Marty Kausas put it straightforwardly: “The best AI companies are not replacing people. Cursor (coding), Harvey (law), Abridge (health), Clay (GTM), are all augmenting humans, not replacing them. We’re seeing this in the B2B support category. Easy questions can be automated, but those questions also take up the least amount of time. Most of the opportunity in B2B support is still augmentation, and helping support teams move faster.”

Here’s the data that we found:

The journey of a ticket, from AI to human

The traditional headline metric for AI-assisted customer support is the “deflection rate”: what percentage of inbound requests make it to a human, and what percent don’t. When we ask questions like, “What jobs is AI successfully completing, partially completing, or assisting?”, deflection rate is your headline number. And it involves four steps:

As you’d probably expect, B2B and B2C companies have markedly different deflection rates. We’re only at a point where AI is resolving B2B customer support requests end-to-end about 15% of the time, whereas in B2C businesses, it’s more like 35%. This makes sense, given there’s likely higher stakes (in dollars and customer relationships!) riding on your average B2B ticket.

The problem with deflection rate as a metric is, it doesn’t actually tell you all the hard work AI is doing. It shows the very basic filtering work, but not any of the ways it can actually solve a problem for a customer, get work done for the human working customer support, or iteratively improve a process.

So what work is it doing?

Garbage in, garbage out

To state the obvious, early: not all tickets are real, and not all resolutions are real.

About a third of all AI-assigned inbound is unwanted noise: marketing inbound, misdirected mail, and system notifications. Email is the worst-offending channel by far: two-thirds of it is just noise. Dedicated forms or chat widgets, thankfully, aren’t as noisy.

Meanwhile, a lot of the ‘resolutions’ in customer support flows are, you know, not actually a resolution. It’s just that the human asking the question never pursued it.

So, at a baseline, you’re dealing with a fairly leaky funnel of “questions in, answers out.” The question is, can AI help with this?

Hot take: most of the time, AI is just quietly helping

As it turns out, across Pylon’s data set, a clean two-thirds of the time in human-AI hybrid setups, the AI looks at the question first, and then triages it to a human without saying anything.

The AI is getting two things right here. First, it’s successfully identifying which questions are “hard enough to go directly to a human”: requests where the AI immediately triages it to a human rep take 5.3 messages to resolve on average, compared to 3.9 messages at baseline (when it’s only humans.) And second, when AI tries to assist the customer first, even in the situations where it does escalate to a human, it takes the human fewer messages than baseline to complete the request.

This is strong evidence that the AI is effectively helping to triage tickets to specialists, and not just creating more work for humans to do.

To underscore this point, in tickets where the AI triages the request, attempts to help if able, and hands off to a human at the appropriate point, customer outcomes are just as good as in human-only support systems. Unsurprisingly, in tickets where it’s only assisted by an AI, the outcomes aren’t nearly as good - you’re twice as likely to get a one- or two-star CSAT rating.

So, what is it going to take for AI to improve its resolution rate, and make customers happier?

This one isn’t much of a surprise: you have to give it more context about the customer. For businesses that use the account intelligence they already have (which Pylon integrates with their appropriately named product, Account Intelligence), AI resolves more tickets end-to-end, and gets higher customer satisfaction ratings. It gets better on both dimensions; unsurprisingly.

Similarly, if your company as a whole is pretty AI-native, there’s a much higher likelihood that your customer support AI will be good at resolving questions end-to-end. To be more specific about it, the “AI-resolved” tickets are more likely to be actual resolutions, as opposed to “the customer gave up and dropped off.” This is a really encouraging sign.

But if you’re a really important, big-spending B2B customer? You’re probably still gonna get a human.

Pylon’s data is a refreshing reminder of what really counts in customer support, and why AI isn’t a zero-sum competition with people for jobs; it’s a copilot, when the best teams use it. As tooling gets better and organizations get more AI-native, we won’t be surprised to see these trends keep going, and serve as a leading indicator for jobs generally.

Subscribe for more from a16z, including weekly data on how sectors are evolving - straight from the companies operating at the frontier.

The internet ruined customer service. AI could save it.

Avoiding Death on the Yellow Brick Road

This newsletter is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. Furthermore, this content is not investment advice, nor is it intended for use by any investors or prospective investors in any a16z funds. This newsletter may link to other websites or contain other information obtained from third-party sources - a16z has not independently verified nor makes any representations about the current or enduring accuracy of such information. If this content includes third-party advertisements, a16z has not reviewed such advertisements and does not endorse any advertising content or related companies contained therein. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z; visit https://a16z.com/investment-list/ for a full list of investments. Other important information can be found at a16z.com/disclosures. You’re receiving this newsletter since you opted in earlier; if you would like to opt out of future newsletters you may unsubscribe immediately.