Name: Beyond the North Star: Turning Data into Impact with Metric Trees
Uploaded: 2025-08-06T18:17:02.088Z
Duration: 56 min 44 s
Description: Beyond the North Star: Turning Data into Impact with Metric Trees

Transcript for "Beyond the North Star: Turning Data into Impact with Metric Trees": Where you work, that'd be great too. It'd be great to see, all the various backgrounds coming in. Looks like we have about 30 attendees or so. Let's wait for everyone to join. It's like we have a bit of a shy audience. Who wants to start sharing? There we go. That way, someone from Chicago. Alright. Donnie at Beats for Dre. Very cool. Stephanie, multitasking, very common. I think everybody needs to do it these days, but welcome. Samith from Double Loop. Welcome. We're very familiar with Double Loop. I see a SoCal. I'm based here in, Irvine in Southern California also. Glad to see some SoCal representation. Yes. I see a question about a recording. There will be a a recording link, that will get sent out after the session's done. Oh, we got our first, first Canadian audience member. Welcome. Karl. Let's give it maybe thirty more seconds, and, then we'll get rolling. Alright. Let's get started. So, good morning, everyone again. Good afternoon, depending on where you are. Thank you so much for, for joining us today for our, beyond the north star turning data into impact with metric tree session. I'm your host. My name is Abhi Srikhande. I lead professional services, here at Mixpanel. You know, I've worked with a number of customers over the last twenty years or so. And, you know, one thing that remains consistent and makes my job really interesting is helping teams move beyond data. Right? To genuinely understand how data and insights lead to a better operating model for customers. How it helps them evolve their product development, their engineering, their marketing, their data teams and just their overall ways of working to become more data driven as an organization. And, you know, that's what brings us here today. We are going beyond simply looking at data to truly making it impactful. Now I think everyone has heard of North Star metrics or focus metrics and how they are crucial for digital product companies. They're designed to align all team towards a single business goal, right? But what does putting this into practice actually mean? So in this session, we'll dig a little deeper into how you can build a comprehensive connected plan to drive your north star or focus metrics, effectively analyze your data to track your progress. We'll show you how a unified metric tree can be your strategic blueprint, really linking those top level business goals to daily team actions. And most importantly, you'll hear some real transformational stories and walk away with some actionable advice on on what to do next. And so speaking of real transformational stories, I am incredibly excited to be joined by Karl Thumm who will share his insights and experiences on this topic. Carl is senior director of data and analytics at Zola where he leads the product analytics function and owns the company's experimentation framework. So, Carl, first of all, it is absolutely wonderful to have you here today with us. Let's start off with, you know, tell us a little bit about yourself, about Zola, and maybe some quick insights from your background as they relate to this topic. Thanks, Abhi. Really excited to be here. So my role at Zola involves a close partnership with our product and marketing teams. I help them define KPIs and a OKRs, size and prioritize projects, measure the impact of experiments, and ultimately empower them to make better decisions for the business. For those of you who don't know, Zola is an all in one wedding platform. We simplify planning with tools for wedding websites, universal registries, stationery, and connecting couples to over 50,000 wedding vendors. Previously, I was also at Shutterfly focused on product analytics on the mobile app. So I was the counterpart to all product managers across all product categories. My role there was primarily about helping the product team prioritize new product opportunities, optimizing all the customization funnels, measuring the performance across push and in app notification strategies, and just helping the team use data to drive growth. The reason why I'm here today is to talk about a critical challenge, connecting daily work to high level company goals. It's so important to give stakeholders and product owners the proper measurement and visualization tools to help them be able to zoom in and out and understand how their day to day work impacts the highest level success metrics. Amazing. Really excited to hear more of your, your, you know, real life examples and and how we can apply this to practice, Karl. So, you know, let let's maybe start off with businesses today are just awash in data. Karl, you've worked in data for for a very long time. Product development and data have always been tied, closely together especially these days. And and businesses are just collecting more and more information about their customers and their products than ever before. This explosion of data obviously has tremendous promise, right? I mean, the potential to understand user behavior, optimize experience, unlock real growth. It's huge. However, one of the things that we consistently observe is the significant transformational gap. Many companies despite having large sets of data collecting a lot of information, just find themselves stuck in merely reporting what happened. Right? They can tell you that sales are up or engagement is down, but struggle to answer why and very importantly struggle to answer so what to do next that will have the biggest impact. And this leap from descriptive, analytics to prescriptive actionable insights is a major hurdle to product strategy or digital transformation. So, I mean, I start us off with a question called in, you know, in your experience, what are some of the the common pitfalls or challenges that you've seen prevent companies from beyond, you know, going beyond the understanding what happened to the actual prescriptive stage? And then, what can companies do to move to that next stage of maturity? Before you respond, I'm going to get the audience, involved here too. So while Karl responds to this question, I'm gonna open up a audience poll over here. You should see the poll coming up. We'd love to understand where all of you are are in your journeys and kinda where we would place your organizations. Karl? Yeah. So what really struck me first by your question was just about more and more and more and more and more. And more data isn't always better. And I think before you get too descriptive, too prescriptive, there's another stage in between. So personally at Zola, a common pitfall that we've had to overcome is what I'd like to call, like, the glass half full versus glass half empty. So this stems from, like, surface level descriptive analytics. So let me give you an example of imagining a a funnel with a 100,000 users and a 5% conversion rate. So this looks like a huge glass half empty opportunity, and now compare that to a glass half full funnel with 50,000 users and a 10% conversion. So a product owner might prioritize the first funnel based off volume alone, and they feel like they're being data driven. So the main pitfall here is not getting descriptive enough. So what if I told you, like, the 50,000 users in the second phone funnel had double the lifetime value? Then the priority completely flips. So the key, is that the prioritization has to be tied to a big business goal, like driving users with higher lifetime value and not just the raw conversion volume. So this is an example of, like, how we've personally evolved from taking descriptive analytics to the next level by adding a little more description to, what success means, and it's made a really huge difference in how we've been able to achieve impact and focus. Definitely. A crucial point, Karl, it really resonates also with what we see at at, Mixpanel with customers. Right? Often wrestling with the sort of overwhelming data volume, struggling to set signal from noise and connecting strategy with execution. So, you know, while a North Star metric can provide that crucial high level direction, you can only focus on that single metrics because it's a black box. Right? It tells you where you're going, but you want to connect it to what are the the components that actually get me there. Right? Without this sort of multi metric interconnected view, the detailed why becomes elusive. And then, you know, actions are hard to define. So, I I really wanna see what, what our audience is is saying here with our polls. So, let's see if we can share the results of the poll. I'm gonna read the results here but, oh, there we go. Thank you. So, it looks like we have a number of folks at that that foundational level, number at in the insightful level. So thank you for sharing. I think this this underscores, what is not an uncommon point. Right? The solid grasp of what happened, good foundational insightful sort of knowledge, but, maybe a gap and perhaps how do we use data for for driving decision making, for driving priorities, for being more predictive and and prescriptive. Hopefully, this webinar is a is a good guide to to navigating the next step and we can give you a couple of tips, as we move into that direction. Awesome. So now that we've diagnosed the the problem a a little bit, let's talk about how we might be able to to address this. I'm gonna start my screen sharing again. So what I call the strategic blueprint is really, a comprehensive metric strategy. Right? One that is specifically centered around a unified metric tree as a core component. A metric strategy really involves creating this clear connected plan and the mechanism that allow you to bridge your strategic priorities down to the daily actions for your team. And, obviously, that means having some of those components of governance and the metrics catalog and your training and an operating model well defined. But the center of all of this is, metrics trees. Metric trees, I'm sure everyone is aware of them, but, they're essentially a hierarchical decomposition of your business goals. Right? What they represent is your business strategy because they connect your top level goals to your individual metrics to daily actions of the team. And so, Karl, maybe let's explore this further. You know, how exactly does a unified metric bridge that gap between overarching business goals and daily actions? Maybe also walk us through your focus metrics at Zola and how that help you drive, conversations. Yeah. So I'd love to share in the next slide. Our focus metric here is total new acquisitions for our paper and website products. So just so you can understand how we're arranged organizationally, we have product managers for our paper business, which is our ecommerce silo, and we have product managers for website, which is our planning tool silo. So we break this into two separate trees, one for website, one for paper. The bottom two levels are really about people starting the onboarding flow and completing the onboarding flow, and we've broken that out further by device class so that we can see exactly where the growth is coming from for each one of the verticals, and how many people are coming into this funnel. So looking at level one, we can immediately see the top level story that website acquisitions are driving our overall month over month growth, while the paper acquisitions are offsetting some of that gain. Year over year, both are growing, but website is clearly the primary growth engine. So drilling down to level two, we can pinpoint the source of these trends. For website, we see month over month is overwhelmingly coming from the mobile platform. And for paper, the month over month decline is primarily driven by mobile with some minor softness in desktop as well. And finally, when we get to our third level, we can see who is entering the flow. So for website, the growth is, people it's consistent across, basically, people across both platforms. And for paper, we can trace the month over month decline to fewer users entering the onboarding flow on mobile web. So very crucially, like, because the drop in complete acquisitions level two is steeper than the drop of people entering onboarding, it signals a two part problem. Fewer people are starting the funnel, and the conversion rate is also lower. So we can properly diagnose the the paper softness to mobile web and that it's both the top of funnel and conversion rate. So that's just a high level example of one of the many metric trees that we have, and that's just a a a monthly review, an example of breaking down exactly what's happening in our acquisition funnel for those two product lines. Yeah. And and, Karl, the like, even as you describe this. Right? Complicated business, multiple channels, and yet you can tell that story of how the business is doing so clearly in what took forty five seconds. Right? And that's that's really powerful to be able to do that and then be able to deep dive as needed. As you started putting this in practice, what was the impact on your overall orgs analytics maturity? Right? Like, did you find your team and you asking more sophisticated questions as this visibility became clearer? Yep. Absolutely. I mean, the the questions became more sophisticated because the tree gives everyone a structured way to diagnose their product domain. It's like a visual way of asking the five whys to get to the root cause of a change. So but the most significant impact apart from the questions becoming more sophisticated was on feature prioritization. So prioritization is one of the biggest challenges any organization faces. So it helped us, like, really decide what not only what to do, but what not to do. Right. And we used a a a data backed solution to create an above and a below the line for a road map. So one of the key things that MetrcTrees offers is a correlation feature. So if you can see if your level one, level two, and level three are highly correlated, you understand that they're really, really important for driving improvements or declines to your focus metric. So that was our way of using a quantitative, kind of metric to decide what's important and what's not. Another great evolution that happened with metric trees is just collaboration and moving collaboration out of Slack and into the tool where the data is. So this helps in a variety of different factors where people don't need to contact switch between a conversation in Slack to data in Slack to Mixpanel. And it really keeps the discussions focused on what's happening in the data, which is really, really powerful, to to kind of consolidate and centralize that that discussion in one place. Definitely. You you hit on my two favorite or least favorite topics, prioritization. Alright? Everyone's been in I mean, regardless of the function you're in, product, data, marketing, engineering, we've all been in these big room planning sessions where very often it turns into this discussion argument of what we should do next, what has the biggest impact. And obviously, everybody has an opinion. And so many times, it's about how much of this can we make math versus art. There's always gonna be a little bit of art in in this, but how much of this can we make, quantifiable so we can arrive at the same result? The the second piece you mentioned is is huge. Team collaboration for a shared understanding of success. It is exactly how we see organizations who effectively operationalize, the the strategic blueprint operate. When teams can share these metrics, have this kind of visibility. I mean, to what you were describing with your own tree. Whether you're the CEO of Zola or you're, an engineer working on a very specific product feature, everyone can see that tree and the impact of how certain decisions are are impacting it. And I think that's that's huge to driving a true data driven culture. And so, you know, the we've kinda talked about how you thought about it at Zola. I wanna switch the conversation a little bit to for our audience who's wanting to to do this, what some of those those concrete next steps might be, in order to get to that next step. As we do that, I have a question for the audience. So this is not a poll. This is, I'm gonna ask the audience to share this in, in the chat. You know, what's one decision at your company, or maybe just for you personally in your role that currently requires sort of this painful multi team effort. I already used big room so you can't use big room as an example. No. I'm kidding. You can if you want to. But, what is something that a shared metric tree could could potentially simplify? Right? So put your put your, responses in the in the chat. And then, as you're typing your your responses, we'll kind of start to get into, how do we take your focus metrics and start mapping them into those those primary drivers. Right? You want to think about as your first step, what's the two or three main levers that directly impact your strategic priorities? That's going to be the foundational layer of your metric tree. Connecting the the ultimate goal to eventually what, actions your team can take. And, this step is something we call deconstructing the business. Right? You wanna start figuring out what those those components are. And so, Karl, I'm gonna I'm gonna, punt this back to you, which is for that important first step. From your experience with Zola and Shutterfly, what approach did you take to deconstructing the business into metric trees? And, you know, kinda share your insights also on evolving and refining the the tree over a period of time. Sure. So we used a phased approach starting with a pilot program. So we gathered a small cross functional team, one member for each team to build a set a single metric tree together. So this really created, like, a a cohesion that this is how we do this, and we took a very standard funnel of looking at our ecommerce funnel from checkout started to order completed. Next, we sent these people off on their own to create their own metric trees for their own domain. Then we gathered all those metric trees, and we reviewed them together, shared ideas, and how we could kind of standardize the definitions across all the metric trees. So one of the ideas that we came up with was creating, like, a new versus existing user because a lot of the people, getting married, it takes a long time to get married. So that was an important cohort distinction that we wanted to make. Another one was really focusing on unique users, so that we could really understand the one to one impact across all the different product lines. So products with lots of frequency then get normalized down to the user level, which is the lowest common denominator. So the final step, which was my favorite, was really mapping out how these individual trees from each team are intersected. And it really was inspiring to see just how shared everyone's focus areas and how not different everyone's success metrics really were. So over time, we added more targeted trees that helped us really understand our business during key seasonality periods throughout the year. So what we found was, like, it's really, really important to have foundational metric trees that are standard across all the product lines, but that doesn't mean that you can't create an ad hoc metric tree for a specific use case or a specific audience. So we'd like to, embody both and, trees grow, they change, the branches cross, and we treat it very much like our metric trees as an evolution as well. As our our business changes, as the dynamics change, there's still opportunity to optimize and and to refine those trees further. Definitely. Karl, as as you were talking, I'm looking at some of the the comments from the audience coming in. I think very much resonates with with what you're saying. Right? I'm gonna read out just a a couple of them not to to leave anybody out, but the the sort of idea of what's what designs are working best for for users. I think there's a similar comment around what big bets we can make, and the idea of having that shared understanding of what drivers we're all trying to prioritize is so important to defining what what bets are we going to make. I love the comment. What what a metric tree is composed of? Well, that is very much your first starting point, John, asking the question. Definitely step step zero, right, is figuring out what the most important drivers for you are. I mean, Karl just mentioned one point about, you know, multiple, teams having their own metric trees. That doesn't mean that they're disconnected. Right? Because very often what can happen is, in a company, we can fall into the trap of saying, well, our our our most important metric is revenue. Of course, it is. All companies wanna make money. Yep. But if we just say revenue, it sometimes takes us away from the fact that individual teams have different focus metrics that may all eventually point towards revenue. And we want those teams to think about what truly their drivers are so we can make those those prioritization decisions. I saw a comment about what marketing channels and approaches, can drive growth. There'll actually be a a comment on this coming up right next, which is step two, which may be a great segue into into discussing. So first step obviously, Karl, you deconstructed the business. Various teams figured out, you know, what their priorities are. You constructed the metric trees. The next logical step then was, how did you wire your inputs into the tree? Right? How do you manage the process of not just the calculation of the metric, but what other things can be input into your, your metrics to understand how you're you're doing as a business? So there's two ways of doing this, and it it really depends on your company's maturity in terms of, like, how well defined your overall company success metrics are. But they could be top down where you know that your company's success metrics are new customers, retaining existing customers, driving total revenue, and driving order value, order frequency increases. So you start with those tops, and then you've you work backwards. So you work back to the checkout flow. You work back to the product ads and ads to cart. You work back to the product detail pages. You work back to the search results pages, and then you get a semblance of how all those metrics flow from a top down perspective. Another way of doing it is bottoms up. So you work in each domain and you work at where people start. So all the acquisition landing pages, all the traffic to the site, and then you they go to the search results pages, or they navigate, or they search, and you go the other direction all the way up to the top. So it really depends on your business and kind of, like, how well defined the overarching company goals are. And if they're not defined, like asking the questions and getting alignment over, like, hey. What are the company's overarching goals is the definitely the place to start. But whether you work backwards or work from the beginning to the end, or do a combination of both is a great exercise for kind of seeing what different answers you may get. You start to better understand what your inputs are, and and then you get to then see where the the weaknesses are in the funnel in kind of that 50,000 foot view that the metric tree so helpfully, visualizes. So, less is more here to begin. It's so easy to get into all the nuances and add so many metrics. Oh, it's just take a phased approach where you start with the minimum amount of milestones, and then layering complexity as, as it yields value. So like I said before, if you're adding a ton of metrics that aren't correlated, you know that they're not driving, a positive or negative relationship with your focus metrics. So that begs the question of, like, are they really truly focus areas? Definitely. Great points, Karl. I'll I'll add a I'll add a a color to a couple of things you said too. In the meantime, if you have questions for Karl or for me, please add them to the the q and a tab, and we can cover them, right after some of the these initial slides are done. You know, one of the things you mentioned, Karl, the, coming from from our audience comments also. Things like the big bets and what channel should I prioritize. I'll also tie that to experiments. And metrics can help you define which ones you should pick. Right? Because you can if you've tied your metrics, and your actions appropriately, I can now run a b test and and identify which one's actually resulting in a bigger impact, and then automatically prioritize that. Right. So it really helps. You also mentioned about less is more. Great point to highlight things like vanity metrics where sometimes we can fall into the trap of measuring things that look good on paper, but really if we double click into it, it may not have a lot of value. So, you know, common example for this that's used is, app downloads. App downloads certainly sounds like a great metric. A lot of people are are downloading my app, so that that's great. But that doesn't really help if people aren't signing on to the app or signing on to your your paid, features or actually using the app. App. Right? So app downloads may be a little bit of a vanity metric, until you also associate it with what does it actually have a monetized impact. And so kinda thinking about what you want to measure, is really important. Almost think of this as the the minimum lovable product of what you want to measure. Alright. Great. So this was, you know, it was about deconstructing the the business into a tree. It was about connecting your inputs. I think where the rubber meets the road is the next step, which is how do you make it real for your team? Right? So what's one piece of practical advice or strategic mindset shift, would you recommend, Karl, for those who are beginning to implement sort of this data driven approach? Yeah. So my my main advice is really to ensure your trees ladder up first to the company's key goals and priorities. And if there's not clarity on what those goals are, to get alignment on that. And then when teams start to see exactly how their work affects the bigger picture, it creates, like, a powerful sense of ownership, and you can even establish that ownership inside the metric trees tool. People begin to see that their metrics matter, and when things matter, that's when you see the tipping point of when it organically becomes ritual. So the tree is really a powerful tool for displaying what to prioritize, and I feel like that's the real differentiator that it reduces the time to figure out what the area of focus is. And another huge benefit is, like, scalability. The same tree structure could be used for, like, weekly business reviews or a high level quarterly review. So this gives you a lot of flexibility to zoom in and out for those cadence reports, and it saves your teams, like, an immense amount of time for figuring out, like, the drivers and what's going on with your your business vertical. And because, like, businesses are constantly evolving, so will your metric trace. So I'd highly recommend, and this is something we learn pretty early on to schedule, like, either quarterly or biannually reviews depending on how mature your metrics are. So this really ensures that there's a framework to keep the pace with your product changes and to also ensure that your metric tree is maximizing its value and relevance to the overall business. Definitely. That's really valuable, Karl. You know, as as you were describing this kinda why I say this also is the rubber meets the road step is because a lot of this is about changing behaviors, changing how we work to be more data driven. Right? So this is the transformation part of digital transformation. We can take care of all the tooling and technology and data, but unless we start using it that way, it's not really useful. And so to your point of, you know, using it for MBRs, QVRs, using it for your sprint planning, I also think about kind of root cause analysis. If you have a a metric tree properly built out and everyone's inputs are are connected, teams know what they're doing according to certain metrics, Root cause analysis becomes a little easier. You know exactly where to deep dive and figure out the cause of an issue. And the the flip side of that is is planning. Like, long term planning becomes a little more structured and driven. Right? Because when you're placing big bets, now you know why I'm prioritizing certain things. And if they don't work, which is the whole point of an experiment, you can come back and and and change the the specific things that you need to change. So, this is, this was a lot of valuable information of what you're doing at, at Zola, Karl. So so thank you for, for everything you shared. So, you know, for our audience, I'd say one immediate takeaway and as much as it applies to to what you're doing is, is is start sort of building your first tree. Right? Deconstructing the business and figuring out what inputs fit in. And for a second, if you allow me to make this Mixpanel specific, this is also precisely where where Mixpanel empowers that part of your journey. The the platform is purpose built to help you track and visually build and manage these views. The great part about this is ultimately each of those metrics connects to way more detailed reporting about, you know, retention reports and funnel analysis and flows that are all part of, of the product. And so you get that connectedness, which really, you know, helps to to make this real. So before we go into the q and a, I did want to highlight that, hopefully you can apply, you know, a number of things that we've discussed here here today. There is a, a great practical resource that we recently released called how to operationalize your growth with, with metric trees. You can download it, using this QR code. It should also be on, you know, the mixpanel.com website. Hopefully, this helps you, you know, get a good summary of what we've discussed, understand those common pitfalls, kick start your journey towards really turning data into into business impact. So, we've discussed a lot. I'd love to to flip it around here and and look at some of the the questions coming from the audience. We may not be able to get to all the the questions, but please put them in because we will come back and, you know, at least, give you some responses offline also. Karl, I'm gonna start you with with one of the the questions, which I think is is very important. So in trying to use metrics that measure customer engagement and correlation to churn and retention, how do you account for false positives? So this is a question from Joe, and I think the the point is, you know, you get insights from customers that are engaged, but how do you get insights for customers that aren't as engaged? Right? So that's definitely part of the equation. Yeah. So my my first question would be, how would you define the first and the second of customers that are engaged versus not? And then I would go to, like, yes, there are potential false positives, but it's more about identifying the areas to maybe test into whether, there's something there. So correlation isn't causation, and that's where at Zola, we have a experimentation framework where product teams can run their own tests to first identify the area to see whether there's an opportunity there, and then they can see through a randomized test whether or not they can really move those metrics. So I'd say, like, mostly, you will never be able to minimize or eliminate false positives. It just helps you kind of, guide you to the path or maybe prioritizing areas of focus and then running further analysis and experiments to see whether that's really a lever you can move or not. Yeah. Good point, Karl. Also add if this is relevant, Joe, the sort of doing the funnel and flow analysis to figure out customers that aren't engaging as much. Where are they dropping off in the flow? You can even use, techniques like session replay to figure out, you know, where in the journey are they, you know, getting getting across sort of this unhappy path where they where they drop off. And then, defining your metrics accordingly. And then obviously defining some of those experiments and testing of saying, well, if I change my onboarding flow steps, does that help me, help me achieve more acquisition or conversion? You can also test it by different cohorts and and segments to see if there are specific nuances by region demographics, etcetera, that are that are driving it. Here's an interesting question, Karl. In, in deconstruct whoops. I suddenly lost the question. Oh, here we go. In deconstructing the business, you end up with more levels beyond level two. Is there a point at which it can start to become too complex? And, if that's the case, any recommendations on what to do? The one thing I will say is, you will most certainly have more than two levels. Right? Because, even a simple metric, is is going to or simple strategic metric is going to, to have a few levels before it gets to team level or product or service level specific metrics. But your point is absolutely valid, which is at some point, it becomes too complex and you want to keep it simple to call the original point of, less is more. So, Karl, maybe a maybe an example from your side on that. Yeah. This is a great question because this is exactly what happened to us in the creation of all of our metric trays. And what I showed you was a pretty detailed example of, the growth team, the acquisition team's funnel from onboarding started to completed. So everyone has their individual funnels to a metric like acquisitions, and then there's a holistic company wide funnel where a lot of the in betweens are eliminated. So that's where we kinda, like, have the very detailed trees for the individual domains. But then for the overarching company tree, none of those steps are really listed. It's more like acquisitions. So each team's focus metric becomes a level on the overarching company, metric tree. So that's how we did it, and it it took time to get to the right grain of figuring out what the milestones were, but it's definitely achievable. I mean, we have over five different product lines. We have a very complex and very challenging, business, multiple businesses of planning someone's wedding, which for those of you who have gotten married, understand that it's a very multifaceted, complicated process. So keeping that's where keeping it simple, but really having the detailed view of the metric tree and then having that summarized view of the overarching company ones are that that's a very, very important aspect to making it, successful. Thank you, Karl. Another question here about any examples where metrics streams drove false insights and how best to avoid them. I'll just start up since I just mentioned this. I I think one of them is the the the vanity metrics. Right? The ultimately, the tree is is as helpful as we make it. And so it's paramount that we pick the the right set of metrics. And so if we make pick metrics that look good on paper but don't really drive ultimate business value, I think that's a trap we we want to avoid. So healthy engagement is important. Connection and correlation to your top level strategic priorities is important. So you really want to think about those vanity metrics and how to convert them into more actionable metrics. The other thing I'll add also is, is trade offs. Right? Very often, I'll take an example of, marketing campaigns. So revenue may be a function of, traffic driven by your marketing campaigns. And so you're looking at the tree, you're seeing that, traffic's going up a lot, but your conversion rate is dropping. And so your natural inclination might be to say, well, somebody's gotta go look at the conversion rate. But the context is important, which is, well, maybe you had a marketing campaign that just got launched, maybe discount codes or something like that. That really drove traffic incrementally. But there was a reason for that, and that may not incrementally drive conversion to the same percentage. Right? Because you just created a an inorganic event to do that. And so having that context and figuring out the trade offs is very important. So it kinda helps you away get away from those false insights. Karl, any anything else you might add to that? I feel like this could be a whole separate webinar on vanity metrics, but and, at Solow, we're not immune to them, but that's a really great piece of advice, I feel like, for minimizing, red herrings or false positives or areas that don't actually move the needle. And how we came up with a great means of of differentiating quality from not quality is just a very simple definition of coming back and doing it on two separate days. So if someone does it once, like install the app, that's a very top level metric. They may not have they might not be invested in using your app. But if they come back to the app on the second day, we've done a lot of analysis on, like, a simplistic way of looking at quality across our metrics, and we've found great success with just doing the behavior twice is a real, super high quality indicator if especially when it's done on two separate days. Got it. I'm also gonna use this, opportunity to connect this question with that earlier question about, you know, two levels in the tree or three or where does it, where does it get more complicated? Sort of thinking about, if you measure your total revenue as, average revenue per user and then monthly active users. Right? Perfectly logical combination. And you see that, average revenue per user is up. So you think your strategy is working, everything's great. Great. Keep going. But the reason those that those next levels are important is because you may again be missing context. So let's say you just launched a new product or a new feature that targeted a specific segment in the market. Maybe that is one segment that is driving up monthly active users and driving up average revenue per user, because that segment is spending way more than everybody else, which is helping everything else look good. But unless you break it down to that level, you won't know which segment actually drove revenue versus the others are actually decreasing. But it was making your average revenue per user look the the same. So, obviously, that's a, you know, averaging, example, but, this is where you need metrics for individual teams, individual products also. I think we have time for a couple more. So, let's see. Curious how you use forecasting condition with your trees to make those bigger prioritization decisions. Yeah. This is this is great because like most other companies, financing is or forecasting is mostly done by strategic planning or the financing the finance team. So there's these overarching goals are kinda set in a totally separate, probably, process tools. So we use forecasting mostly as the overarching benchmark of, like, that's what the goal is. That's what we expect all of our high level focus metrics to move on. So we use it at the highest level, and then we use what's happening inside the inner metric trees to see where the inputs are of trying to drive those overall focus metrics upward. That's just germane to Zola's business because having targets for every single aspect of the business and putting that into Mixpanel would be challenging. I could see how that might work that you might be able to have all the inner goals all the way down. But I think back to the, like, start simply is if you have a forecasting process to, see what the goal is of what the growth is for a particular metric to keeping it at that highest level and just knowing that that's what you're trying to move and that's what you're trying to do, that's where I think you will have a minimum viable chance of success with with combining those two workflows. Yeah. I I like that. And, you know, with with with larger companies, you're going to have multiple business units that obviously all are working towards the same strategic goal. Each will have their own metric tree. And so kind of when you do forecasting is is taking the combinations of all of those and the causal relationships between metrics to figure out how they apply. You know, the your tree can also serve as a great baseline. Right? I mean, very often forecasting is annual planning, which gets into some real factors of budget and resources and and and all of that. And so the very common question you'll get from your leaders is, well, what what is the baseline? What can we do next year with the resources that we have today? And then if I if I add 5% more, what can we do? So now that that tree based on your current metrics is your baseline. And so now you have the ability to experiment and say, well, if I if I invest in in acquisition type priorities, then I can do this much. If I invest in churn related priorities, I can do this much. So it gives you a kind of a means of comparing, you know, apples and apples or apples and oranges. To kinda go that a little bit more, the like, most planning and most forecasts are set on a year over year basis. And what I really like about the visualization and you can change this and customize your metric tree to be however you want, but it has all the year over year and month over month metrics at every single level. So if you know you need to grow revenue 10% year over year and you there's a new customer 15% and existing customers are 5%, you start to understand all those inner dynamics of how you're hitting your target. And, it makes it super, like, summarized and easy to understand whether you're making improvements month over month, which is kinda like during the course of the year, or whether you're tracking well against your year over year, targets for the company. Thank you for bringing that up, Karl. I I I do wanna tell the audience, I did not ping Karl on the side to pick up the expiry, but that is a mixed value. You can compare month over month, quarter over quarter, year over year. Great point. Alright. One more here. So, Shane says we're a b to b company and I feel like we have three to six different flavors of engagement. So some groups, of stats matter for one flavor but not for the other. Karl, any advice in terms of how to avoid overcomplicating it, or under simplifying either scenario? Okay. So, like, I actually thought of a b to b example before this. So let's just go back to, like, sales qualified leads, marketing qualified leads, just simplifying the funnel down to, like, what are the milestones of each one of those funnels and teams, and how do they flow into each other step by step? Now to look at different audiences, I I'd highly suggest mutual exclusivity of, like, how you define the audiences. And there are drawbacks to making all the audiences mutually exclusive. But one thing that it creates and simplifies is it makes a nonlinear process linear, and that you can't get to this stage without doing this, and you can't get to that stage without doing that. So, where I've seen a lot of success is that maybe start by scaling back the number of definitions or looking at your definitions and seeing how much overlap and where you can simplify, where you can consolidate, and then make it linear in terms of, like, that you can't progress to a certain step without, doing these, and then see how that leads to your success metrics, would be my, my my first level of advice there of simplifying, making them mutually exclusive, and, making the the milestones linear towards your success goal. Makes sense, Karl. If I understood the question right also, I would say the like, each of your, five to six flavors may have a different set of focus metrics themselves. Now they may all tie up to certain common metrics, but each of those may have different priorities. And so kind of building your trees according to that will help you from, sort of trying to create one that addresses all of those those different flavors. Let's see. One question on, how do you execute metrics trees in Mixpanel? Are there documentations that describe how to create metrics trees? It's a great question. Metrics trees, we've been in beta for metrics trees, and metrics trees launches general availability next week. So, if you're a Mixpanel user, talk to your account managers, customer success reps on how to get that functionality. There is documentation, will be documentation up on the website as part of Mixpanel docs on how you can create it. You should certainly use the the ebook that we shared earlier. And then obviously, if you have more questions, please reach out to us on, on on Mixpanel.com. I think that was the easiest question we're gonna get today. Alright. Maybe one more question here before we end. In terms of root cause analysis, particularly external factors such as trends in the market, how do we fix such a situation to get the team back on track? So external factors like external events that may occur, is that the I assume so. Yeah. Karl. So I the year over year and month over month metric is is definitely your friend. So if you see that, that they're they're over and under deliveries where no nothing's changing, there's probably an external event that's occurring, But without knowing more about external events, I I don't I don't know aside from using that to get to the root cause. I I guess maybe to work a little bit backwards from that is using, process of elimination of your metric trade to understand that there's there's no findings there, but there's an external event that must be tied to that external event because there have been no new product changes. There have been no, basically, large changes in the overall experience or marketing channels that would have to be attributed to that. But this is a challenging problem that all companies deal with with regards to, like, things that may happen that's beyond everyone's control and then pinpointing it, to that event. So both process of elimination and, using year over year, month over month, and and doing an audit of what's changed to try to line up the trend change with what's going on in the metrics, I think, would be how I would do it. That was my thought also, Karl, is the sort of, if you have your metrics well defined, you can do root cause analysis, obviously, by diving into which specific metric caused the change. And then an external, an external reason or external trend, presumably impacts one or more of those metrics. And so building that in to figure out what the change was. I think this is part of where the, where metrics help is because you have your metrics already defined, you know what factors are under your control. And so then you can at least isolate what that external event was even though you may not be able to determine the exact impact of that external event yet. But hopefully, that gives you a a mechanism to add it in for for the future. Oh, Donnie had a had a comment. So so there are there are annotations that you can put inside the tool, and I've actually mentioned this in the Mixpanel product road map about being able to import your, promotional calendars. You can't quite do that yet. It's definitely, been a proposed idea. I think it'd be because everyone wants to know when promotions are starting and ending and how they impact the business, but you can manually add annotations when things do occur and you can collaborate inside the tool currently. Great point, Karl. Yeah. There's, along with the wiring of the inputs to Karl point, there's, there's sort of notes and annotations around strategy, around, tags, around, collaboration elements so you can share with, with other team members on what happens. You can keep a historical track of, of what's happening. Alright. Let's see. We are just about at the top of the hour. So, thank you everyone for your fantastic engagement. The the questions that we haven't been able to answer, we've had a a team here, Claire and Kara, who've been capturing them. So hopefully, we'll be able to get back to you offline and answer some of your questions. If you have more, obviously, please reach out to us. I do want to be one reminder on this again. Karl, thank you so much for for sharing your insights, your expertise, kind of bringing these concepts to life. I think it's extremely valuable not just to look at the, you know, the the functionality, but to hear how somebody has actually implemented this and how you think about, you know, changes and driving changes in your in your organization. So thank you so much for your time. Really appreciate it. For the audience, one last reminder, if you haven't downloaded the book, here's the link. If, if you don't get the link or it doesn't work, reach out to us on mixpanel.com and happy to share over there. And for everybody, thank you so much for taking an hour of your time to to attend this morning. Hopefully, this was helpful, and look forward to to more of these in the future. Thank you all. Thank you, Karl. Thank you.