Name: Product Roadmap Review
Uploaded: 2025-12-09T02:46:07.636Z
Duration: 24 min 28 s
Description: Product Roadmap Review

Transcript for "Product Roadmap Review": Hi, folks. I'm Brian. I'm VP of product here at Intercom. And, today, I wanna take, maybe twenty five minutes or so and kinda give you, a curated version of the road map, that we shared in at the November And maybe a slightly rawer version, and kinda share some of the thinking of, where my head's at, where the team's head is at, on some of the, items that we're working on now. So start at the start at the start. What we believe, we believe AI agents will do the vast majority of customer service work. We still have to figure out where that number is. Is it 80%? Is it 90%? For a lot of customers now, it's in that 60%. We do believe this number will keep increasing, but we don't think it will really get to a 100%, and that humans will be durably relevant. We believe you've gotta innovate at all layers of the technology stack, and we believe agents will just inevitably expand beyond customer service to do the sales and success jobs. So we do think there'll be a single AI customer agent category. So what this means for our investments is that kind of raw technology investment, bucket one, the thin flywheel, and that's what I'll focus most of, today talking about. And I won't actually talk about our investments in the customer agent. We'll leave that, for another time. And what I actually just going back, I'd also say we are investing more in the help desk. That's a decision we've just made recently there. So I won't go into details of that one. I'll just focus really on number two, on our fin flywheel. Now just framing the technology stack. The way we think about this is there's an app layer. That's what you use and and work with in intercom. The AI layer, which sits in between the models at the bottom and that app layer. And, really, what this looks like so I was mentioning this. This is the thin flywheel at the app layer. The AI layer is is easiest to think of as our RAG system. That's the system that goes, uses the model to figure out, what Finn should say, and then the raw model layer, which all those components in there. And the key point to make is that, we are still investing here at this model out to expand our footprint of where, we actually own that full technology. And the reason we do this is for latency improvements, for efficiency improvements, and for product quality improvements. We think the best product here ultimately will come from owning that full stack. So, and look looks like this number will increase a percent. We'll have to wait a few more days to lock this one in. All of that work is what really drives this continued performance improvement that we're able to give to all of our customers. Okay. So let's jump into to the flywheel. And I actually wanna start so the flywheel train, test, deploy, analyze the bedrock of how we framed our our fin product. And, I wanna start with, analyze, and I wanna start with automation rate. And, this is a a new metric that we're, gonna be bringing in very shortly, hopefully, in the next, couple weeks. But it's a simple metric. And let me explain why this metric is so important is that we have emphasized resolution rate, front and center, and people think, great. I've got, when Finn's involved, I got a 66% resolution rate. That sounds fantastic. I'm really, happy with that. But the reality is it enables people to ignore the fact that they're not letting Finn, have a chance to answer at a meaningful chunk of conversations. So you see here if someone's got, Finn not involved in 52% of it, that their actual automation rate, which is of all your inbound conversations, how many is Finn actually resolving? And that number is 32%. And we really wanna ensure customers aren't too anchoring to just resolution rate when, ultimately, automation rate is really the overall impact that Fin is having. And the result is that we think we've encouraged customers to not focus enough on involvement rate. They've just focused on resolution rate. So we're bringing that new number automation rate, which is just involvement rate times resolution rate, more front and center. Fin's performance and impact to your team, automation rate, the quality of Fin, that's the CX score. That's front and center. Now one of the big levers, this is not a roadmap one, this is now available for everyone, is Fin attributes. And this is one of the, big product improvements we tried to make to really encourage people to have that effectively approaching a 100% involvement rate is what we want. And and Fin attributes is really what we saw with so many customers, particularly who were using started with the help desk, is that Fin was kind of buried in complex workflows, and this was really constraining, Fin's involvement rate. And so Fin attributes is a way to use AI to do the job that people were using deterministic workflows, button based workflows before, which translated into, a less than ideal experience in the messenger, and to let AI do the work of those workflows for you. And then you can bring Fin up front and center and have that AI agent concierge experience so that when someone comes in, just tell us what you're coming here for. Fin will do the work to triage and route that, and you can still use, escalation guidance and even better escalation rules, which work more reliably when you wanna put attributes in, to ensure if there are conversations that you aren't comfortable with Fin answering, Fin can completely just skip those and say, I'm gonna hand you straight to the team. And so I'll actually just show you. This is this is literally designed from last night, draft design that someone shared. But, what I wanna highlight is this this part over here, and we'll try to make this more clear of, like, hey. So this number, we want to be almost a 100%, ideally a 100%, and that will show you more clearly, hey. Finn was involved, but some of these Finn escalated. Some because the user requested and said, no. You didn't answer my question. I need to talk to the team. Or some because you've set up your configuration rules and said, Finn, don't answer that. And kind of make that much more clear of what's going on. So we're really gonna try to expose a little more of the system at a little more complexity of this diagram so you can really get a sense of the levers and see, where your opportunities are, not just for resolution rate, but for involvement rate. Okay. So now let's move to resolution rate. Hopefully, this is, repetitive, and you you've heard this before, but it's just foundational to how we think about things is that conversational taxonomy break breakdown. What type of of question is your customer coming with? Is it and these map too. Informational maps to content can answer it. Personalized means there's some data, but, usually, you need to, like, read a database. For example, what's your account balance? Actions is usually multistep business logic or going and changing a value in a database. Hey. Can I cancel that order? And then investigative is where someone on your team had to go back and check with other people, and there it it was content alone couldn't answer it. So, the key point here is that, first of all, for, content, we've done a lot of work to real and this is more continued work from our insights product to really point you at content improvements to make to increase that resolution rate. So from these AI powered suggestions, like, here's the exact content that we saw from your human answering a question, and now you need to add this to your content so Finn can answer it, and then helping you clean up your content. Here's contradictory or duplicate suggestions. Here's actually suggestions by audience and brand to so you can prove that for that next level down, of complexity of getting that content right. So we're really trying to help you get that, informational resolution rate, maxing that out. And, actually, what we see in our customers look looking, in aggregate here of, like, where is that unresolved breakdown? As most customers are having the most success for informational, and that the the bigger opportunities more generally are in the actions in that personalized space. And, hopefully, everyone's aware of procedures. We've talked, a fair amount, about this, of this is our big product, effort to really tackle complex business processes, and when you actually have to change data. And procedures is where it's based on a a more straightforward doc format where we integrate deterministic controls inside of that. This is the huge design challenge of how do you balance generative AI with deterministic controls because you have to get these things right. So that's the modern natural language editor that incorporates those deterministic controls within it. So, we're making great progress. We're really excited with where we're seeing with, customers. This is still in in closed beta. You can see here is where you've got you can even incorporate code in there to ensure that Fin is doing exactly what you want, in certain situations. So this is progressing well. We're really excited about this. And what we've also talked about here is where we're actually using, an AI assistant to to help out as well because frequently that getting from zero to one, this AI assistant can get that first pass of that procedure way faster and really sets customers up well. But the key thing here and and just going back as this is, still in private beta, and our hope is to get this to manage availability, which is essentially work through sales, to get this into that's our hope is to get there in January. Our intent is to get this to general release so everyone can use these, but we are not yet sure if we can do that. And the reason why is because it's easy to go wrong here. It looks easy, but customers can actually set these up in a way that, isn't working, isn't successful. And so, we want to ensure we're not letting customers dig a hole and degrade the FIN quality, and the experience here unintentionally. So our our reason for holding this back to to that is just to ensure we have confidence that customers can successfully do this on their own. So we're working through that more. So, hopefully, this is in January. This gets to manage availability, which is working through sales. But what I wanna add, and and this is actually pretty fresh what we've realized, is us recognizing that data connectors are actually the the big unlock for customers and where almost all customers should put more emphasis first before thinking about procedures. So data connectors are there. They're available to everyone, today, and we know there is more work to do here. Where there's authentication improvements, other, technical improvements we're making here. We need a lot more UX improvements to make data connectors easier to use because, what we've learned is is that they're very effective for those personalized answers. And, actually, what we've learned sorry. To interrupt, what we're also gonna point to with insights is, hey. We're gonna point to here's improvements you can make with data connectors. Here's data that you can inject and try to point you how to do this. And this is the key point that is pretty fresh from where our thinking is is that, look, everyone should lean into content first. Why? It's easiest. It's fastest. Yes. Sometimes it's still a big lift in getting your content to the right place, but it doesn't require any technical, expertise at all. And you can get huge value out of from Fin so so fast, and there's so much opportunity there that, to go with. So content first, then go to data connectors, then bring in that data for the account balance for what type of account do you have that can really and what Fin does so well is combine those first two. So feed Fin the data, and you can give the like, going back and kinda control, give guidance on here's how to use this more. And and then Fin can combine content answers with personalized answers and really give this beautiful synthesized answer. And then the third one is when you've got complex business processes that need multistep laying out, that maybe you're going in and changing a database. That's the most complex. That's, like, the third bucket to go after, but it really should be that sequence. So one content, two data connectors available today. We are gonna make these easier. We're gonna make these more prominent in the product, but the k the raw capability is there today to go. So that's the thing I'd love for folks who are keen to get at procedures, but it's still restricted to, like, lean in to data connectors now. Fin works great for this, and we will make this easier, but you can this is unrestricted now. Okay. Next, I wanted to touch, quickly on, test. So, simulations, we talked a lot about these at Pioneer where, these are really, right now, framed as almost part of procedures. And that's how we're framing simulations, which is where you go in and describe an automated test and give the evaluation criteria, and then, we run that test, simulate an end user based on the criteria you said, and then evaluate if the procedure worked effect effectively. So simulations and procedures really go hand in hand, and that's what's innate enables you to confidently build out a procedure step by step and sequence your way to, like, yep. It's doing step one right, steps two, step three, and it's working for all the different, customer attributes and types and variations and edge cases to build that confidence there and to build the confidence so that it didn't break, unintentionally afterwards. So, really, procedures and simulations go, hand in hand to building confidence for for automating those complex queries. So, again, we're building in an assistant to help with these. Often getting the first pass is is is better. But what it so but those are right now, those simulations are are are tied directly with procedures. But what, where we're leaning more into, and this is pretty fresher thinking, is the importance of version control. So we're bringing this in sooner, across multiple parts of the product. And then that version control also tying that in with the preview capability, with that testing, capability, and these will be for procedures, but also beyond that. So, that's where we're leaning into now is better version control and better preview control so that, across the product across the Fin product space, you can have more confidence that changes you're implementing are right before going live. And we're also gonna, lean more into Fin QA, which is when it Fin is live, there may be certain areas you wanna ensure Fin is doing right on. Hey. I wanna just ensure for this specific area, say, this billing type query or or this thing. I wanna be sure I'm happy with how Fin is working or the improvements I made that I wanna check up and validate they're working well, as well as a broader just make it easier for humans to do the QA work here, not requiring them to build their own spreadsheets or or workarounds here. So we're gonna also this is coming soon. We'll share more details when we have more, confidence of the shape of what this product is. Now, from a Deploy point of view, voice, I hope everyone knows about this. It's just super exciting to see the progress here and and see this, landing in the real world and bringing procedures into voice. There's a big meaty road map here, but I'm I'm, I won't go into, the details, of voice for here. What I last one to jump down to is, is back to insights and and to CX score. So, we launched, CX score, going back, I think, at the start of summer. And this is a really significant iteration that we've, made to the under the hood, how we're calculating it. And, really, what we've brought in is these additional attributes that we're evaluating Fin against, these score reasons. And these score reasons have multiple benefits. First off, we're actually able to increase the coverage of conversation that we can apply a CX score to because going into the details is a bit messy, but, basically, using these CX score reasons gives us more confidence that we can give an appropriate rating to a score, which previously, if there weren't many parts to it, we just couldn't rate it. So this has helped increase the coverage of, the score. It also will give you more confidence in the accuracy of that score because it's giving you the the specific reasons, not just in the the summary that we gave overall, but these specific reasons because they're attributes. It means you can go in and say, hey. I wanna find all the low AI agent answer quality ones where there was a negative emotion and then be able to dig into those conversations at scale. So these really unlock huge new power. They increase the quality of the score. They increase the coverage of the score, and they unlock huge new value of you really being able to dig in, understand what's going on, and improve, improve, improve. So we're really excited about this. We're gonna share, benchmarks so folks can have a sense of, how they map to other people in their industry for the CX score. And we really feel with this iteration of the CX score, it can now be your absolutely primary, metric for reporting on fin and and human quality. And there's a bunch that you can build off, now when it's built into the system there. So, those are the, key highlights that I wanted to touch on for for this road map. There's heaps more in there. I don't wanna bombard you with data. That gives you a sense of of where we're at, where we're thinking, and but, ultimately, the the biggest picture for us is thinking about, that automation rate for Fin and getting that automation rate, which is both involvement, get Fin on all the channels, and the resolution rate, get Fin answering all the question types, the query types, combining those together so that you're getting to the where the peak is in that 90% Fin automation rate. That is achievable today. We have customers getting that today, that's what we want everyone putting that deliberate effort in and setting that goal high. And we're trying to help you get there as much as possible, and we will keep doing that. So, I'll wrap there. Thanks, everyone. Hey, guys. Thank you so much for joining, and thanks to Brian for presenting the part of the road map for us today. We also have Jared here on the call. Hi, Jared. Hi, everybody. Does anyone have any questions in terms of the product road map? Feel free to put them in the q and a. Or anything related to procedures or testing, anything at all. Yeah. While we're waiting, if anyone has any questions, just to mention on the CX score, we currently have some changes. They will be implemented tomorrow. The CX score now evaluates a wide wider set of, signals, which includes financer quality, teammates, answer quality, customer effort, strong emotion, product service feedback, and also your policy feedback. So, yeah, very interesting as well. If nobody has any questions for today, you can always reach out to your RM or CSM if you have one. Otherwise, you can ask Finn or, yeah, reach out to us if you have any questions. Thank you so much for attending. Oh, there we go. I have a question. I'm gonna share the screen. And for. There was a lot of info to digest compared to the dev team. How much is this preferred to discuss with? Very interesting question. Not offhand. Anecdotally, like what we talk about, we leave it up to our customers to decide whether they disclose to their customers whether they're when they're engaging with Fin or our AI. Again, anecdotally, we have heard that there may be there may be less friction and a slightly different way to communicate when people engage with AI. But very similarly to how when Google first became something, you know, when Google first came around two decades ago, it's all about just training your users and getting them comfortable with, like, how best to engage with the AI and get the most out of AI responses. So as an example, you know, Google's taught us for a very long time to only put in keywords when doing searches, so two or three words, and that gets you very good outcomes. But with AI, obviously, there's a lot of retraining that needs to happen because the more words and the more context you can give the AI, the better the response you're gonna get every single time. So it was quite a long winded answer. It very likely that there will be some, blog posts and information we can share about how much more willing people are to engage with, Fin versus human agent. But we generally don't see all that much difference. Yep. We also had get a lot of feedback of that you can't even, identify if it's a human agent or an AI bot nowadays. So yep. Yeah. Its responses are conversational enough that it is very hard to tell. Cool. We got another question. Don't get me off and very happy with it. And most definitely. I think it's all the same messaging. Cool. Yeah. So if you have a relationship manager, definitely reach out to the relationship manager. Otherwise, yeah, reach out to our support team for any questions, or even ask Finn for any tips. Did you have any suggestions? When you're efficient, come in, we're having queues. Definitely. Well, like, the best thing to actually do is to reach out to Finn. What you'll notice happening is that we will actually get looped into those questions after you reach out to Finn. So our advice is, like, any question you have, actually ask our version of Finn in the bottom right hand corner of your intercom screen. And then, of course, you can reach out to your account team or you can tell Finn to escalate, and that escalation will generally loop in someone in our region. So in this case, APAC, you'll get myself, Francie, or another one of the APAC team members who'll be able to help out. Perfect. Awesome. No worries. But, yeah, if anyone has any questions that are coming up, please feel free to reach out to us, or Fin or our support team or your relationship manager. But, yeah, thank you so much for joining today's webinar. Also let us know, if you need the full product road map. We we're very happy to send that over to you because Brian couldn't cover it all today because it's over a 100 pages long and it would have taken quite a long time probably to present. Cool. Cool. Thank you so much, and I'll see you hopefully at the next webinar. Thanks so much for joining team. See you next time. Bye.