Video: Leading the Way: A CIO’s Guide to AI Adoption | Duration: 4208s | Summary: Leading the Way: A CIO’s Guide to AI Adoption | Chapters: Welcome and Introduction (24.99s), AI Maturity Pillars (84.31s), Introducing Expert Panelists (152.04501s), Workday's AI Journey (266.945s), AI Strategy Evolution (351.415s), AI Council Formation (523.91003s), AI Experimentation Journey (709.225s), Prioritizing AI Initiatives (846.17s), Evolving AI Budgets (1254.115s), AI Governance Framework (1517.2499s), Managing AI Agents (1680.215s), Operational AI Readiness (2068.78s), Effective AI Enablement (2654.965s), Measuring AI Outcomes (3177.1602s), AI Implementation Insights (3633.7952s), AI for Good (3867.62s)
Transcript for "Leading the Way: A CIO’s Guide to AI Adoption":
SPLITT: Hello, everybody, and welcome. Thank you for joining us for this Looking Forward with Workday webinar, Leading the Way A CIO's Guide to AI Adoption. You're going to be hearing from two esteemed guests, Mae Yap and Ronnie Johnson, who forged incredible trails in their own enterprise AI journeys. By way of quick introductions, my name is John Parkhalber, and I'll be moderating this session. You'll meet our expert panelists shortly, but first, a few housekeeping items. During this session, we may share a few forward looking statements that are subject to change. Please note that these are covered under our product statement agreement. And with that, some additional housekeeping items. This webinar is being recorded and will be emailed to you within twenty four hours. We've uploaded some resources for you to view. Just simply select docs on the right side of your screen to review and download those materials. If you have any technical issues, please use the Q and A tool for any technical assistance. With that, onto the agenda. Our goal today is for you to walk away with greater insights into how you advance your AI maturity journeys. We'll start with a brief framing of the five pillars of AI maturity from some recent Workday research. We'll hear from our two esteemed CIO guests on how they've worked across those pillars, and we'll wrap things up with some additional resources you can use to smooth your path to AI adoption. These are the five drivers of AI maturity that will frame the rest of today's discussion. This framework includes strategy, how AI serves your business objectives and how this influences prioritization. Executive sponsorship, how you work with other leaders in the organization to drive AI impact. Organizational knowledge, the crucial people readiness aspect of AI maturity. Operational readiness, the technology and operations frameworks that support your AI initiatives. And then finally, last but not least, governance, a foundational piece to how risk is managed across data security, ethics, and compliance. Let's hear about how our expert panelists have approached this. With that, I'd like to introduce Meiyah, Senior Vice President and Chief Information Officer at Jabil, and Ronnie Johnson, Chief Information Officer at Workday. Now, Mae, to start things off, let's start with you. Can you share a little bit more about yourself and what Jabil does? Yeah, definitely. So once again, thanks John. It's great to be here. Rami, great to see you again. It's been a year, right? So like what John introduced, I'm the CIO at Jabil, right? Jabil is a global manufacturing solution provider, operating in more than 25 countries, with over 150,000 employees worldwide. So, like what you can see on screen right now, these are actually a part of the history of Jabil. In fact, we just celebrated sixty years of JBA inaugurations a few days ago, right? So I'm really, really proud about it. So in JBA, our business actually span across eight different industries, from healthcare to cloud infrastructure, to automotive, consumer lifestyles, consumer electronics, so on and so forth. So technology has been a very, very critical part in connecting our value chain across JAPER, right, from product design, to supply chain orchestration, to factory operation, definitely to customer collaboration. So one of the interesting things that have happened right now is manufacturing is transforming, manufacturing is changing, manufacturing is becoming from a hardcore manufacturing to interestingly digital first. So, this is actually where I will share a little bit of the insight today. Thank you, John. Fantastic, May. It's incredible the scale of impact you can have in Jabil, an organization that operates so broadly across the globe. Roni, I want to ask the same question to you. Can you introduce yourself and share a little bit about Workday Business Technology? Happy to. I'm Roni Johnson, CIO at Workday. I have been enrolled now just at three years, so really excited to be here with you all today. I lead Workday's business technology function and that includes our enterprise data, our AI strategy and product management functions, it includes our go to market systems, it includes our Workday on Workday practice and our core infrastructure teams. One of the things that I love the most about my job is that our teams, especially our Workday on Workday, what we call our Wow teams, are customer zero. We get to experiment with our product capabilities and features, sometimes influence the product roadmap, but really we get to be Workday's first customer and so it's really, really exciting the role we get to play, especially when rolling out new AI capabilities. Ronnie, I'm lucky to be one of the end users in that environment as well. Ronnie, I'd like to stick with you for a moment. So Workday is obviously a leader in the AI conversation for enterprise technology. You, in particular, are in a unique position to talk to our customer CIOs about their AI strategies, your AI strategy, and how you lead internal adoption across these tools. What has your AI journey been like at Workday? It's been quite unique, I'd say. I mentioned I joined three years ago. It was literally the week that GPT-four came out and it's the week that frankly everybody and their mothers started experimenting with generative AI and had lots and lots of opinions about it. And so really, really quickly I realized we needed to have a cohesive internal AI strategy and plan for how we would select and evaluate technologies, how we would roll them out. And so we began a partnership and that initiated with creating an AI exec advisory for the senior most leaders of our company to weigh in on what their objectives and priorities and frankly the outcomes they were looking for in AI. And then it helped to actually create what we call a technology council, but essentially it's an operating committee where we brought in our legal partners, product and technology leadership or frankly thought leadership in AI as well as our procurement teams because we realized from an operational standpoint we really needed to figure out how are we going to set the guardrails for how our teams would internally use it. So we had these two councils that helped really inform and guide our strategy and what we decided to do essentially was to look for use cases that had the highest impact and highest feasibility and so we were looking for the intersection between those two things and we frankly started to prioritize the use cases and present them back to these councils for them to weigh in. I bring that up because there was a lot of learnings in the three years that I've been here. Our early use cases, frankly, we let a thousand flowers bloom and we knew that we were going to make commitments that were pretty short lived as we were experimenting. So our focus now has really been on how do we truly deliver transformational impact. So it's more of not the functional or team level use cases but the macro use cases where we can actually think about how we do cross functional or enterprise wide change. So, it's been a fantastic journey, the maturity of it, some of it's been painful, some of the lessons been painful, but it's been a wonderful, wonderful ride. Ronnie, something that stands out to me from what you said is that both scale and focus went hand in hand. So thank you for sharing that. May, I'd like to switch over to you. In a recent interview, you mentioned you established a data and AI council three years ago at Jabil. What's your AI journey been like since then? Yep. So, John, maybe a little bit before the three years mark that I actually established the Data and AI Council. We started the AI projects like way before the three years mark, right? When ChatGPT came about, they were running and saying everybody's excited about ChatGPT. But we didn't actually start from there. We actually started much earlier, probably eight, nine years ago, right? Where, you know, manufacturing is like a melting pot. That's where a lot of technology conglomerates. Actually, Industry four point zero is a very big push factor for us to actually innovate and do something different. So, like eight, nine years ago, we already started with what we call the AI computer vision solutions, that we actually deploy computer vision solutions into our manufacturing shop floor to actually augment, you know, our inspectors who are actually there, using what we call their golden eye to actually inspect cosmetic inspection. So that was a very, very painful process, right? At that point in time, it was very new, right? People are not familiar with AI, even though AI has been there for a while. So the context was very different. It was actually about machine learning, deep learning, and actually using the data that we have collected over the years to actually improve the inspection process. So that aside, right, coming back to your question about, you know, when we set up the Data and AI Council three years ago, the AI journey has been great, right? So I think the biggest realization that we, when we started like eight, nine years ago, is that we realized that, hey, IT is not about AI, right? Neither is AI equals to IT. So that was a very, very clear proposition for us. When we actually wanted to do that, the idea was simple, right? AI decisions shouldn't should not happen in silo, neither should it just be happening in operation, in business, in IT. It should actually happen together. And that was the main objective as to why we set up the Data Council, where we get people like what Ronnie is saying. We have legal, we have HR, we have finance, we have operation, we have the business unit actually sitting together, and we collectively decide on what we wanted to do. So when we set up the Data and and AI Council, it was very, very clear that what we wanted to do is we wanted to make sure that we give clear guidance to the organization about, first, what to invest in AI. Second, what are the gut reels that we need to put in place? And third, what should be the what are the things that we can actually do right? How should the ingestion be? What is the literacy program that we wanted to actually introduce? So it was with that mindset that we actually started this AI journey, right? So there was actually a few key steps that we actually embarked on. And that was actually about, we wanted to focus on strategy, we wanted to focus on governance, We wanted to focus on use case that can be applicable across the enterprise. So, the first thing that we were focusing on in Data and AI Council is really about data foundation. Remember, I didn't just talk about AI Council. I talked about Data and AI Council, right? So, data becomes such an important part of this whole AI foundation. It is the foundation for everything. So, we wanted to make sure that when we focus on data foundation, it is making sure that how data can be trusted. Our data is governed. We have a very clear policy we have about data. We have a clear guidance about data management. We have a clear data governance practice put in place. So that was really about data foundation. Then it becomes AI experimentation, right? So remember, we started like three years ago, way before ChatGPT actually was there, right? And when people actually get overly excited. So our focus was actually, hey, we know we wanted to do AI, we wanted to do it right, but it's also important that we need to make sure that people have their hands dirty, they can actually experiment on POC in a gut guided platform, in a guided way. So that's what we wanted to do on the second part of it. And then the third part of it is really when Gen AI actually came about, right? So we wanted to really have, have the platform for people to actually know how to use Gen AI properly, how to use agentic AI properly, and where AI can actually be embedded into the business workflow, right? So if you ask me, looking back at how we start, I think we started well. We include everybody into the Data and AI Council, right? A few lessons learned is that data maturity really determines how fast you can actually scale your AI, right? So, like what I say, become a foundation. The governance is super important, and it is good that we have actually set up the gut real. And then there are people that challenge us, right? Hey, why do you actually start with a policy instead of, you know, letting it free for everybody to do, So in the hindsight, when we look at it, I think it's really good that we tell people what you can do and what you cannot do, right? We say upfront what are some of the things that we do not allow them to do, like loading customer data or our proprietary data onto, you know, the World Wide Web or onto all the free model that's actually up there. That's a no no. And that there are consequences if you do that. So I think the gut real helps to actually guide people. It doesn't actually stop innovation. That's one thing I must say, right? I think it's a super important learning lesson. It actually helps to give people a framework on what we can actually do and what not do, right? So the last thing is really, you know, when we actually talk about real transformation about AI, the involvement of everybody, the business, the operation, getting involved and discuss it together, is super important. That's where we can actually generate AI outcome. That's really interesting, Mae. And a few things stood out to me. Number one, I might want to double click on some of your comments about governance a little bit later. But one thing that really stood out was the importance that you held in cross functional collaboration to be successful around AI. And I'd like to double click around that a little bit. As you got more people from across the organization on board with your AI initiatives, how did the way that you prioritize those initiatives change over time? I think that's a super good question, right? So, again, if we look back at our own industry, we are manufacturing solution provider. So, we need to make sure that first, whatever that we need to do needs to help the factory. It needs to help the factory to be more efficient and effective. So that's the first priority that we have, right? We orchestrate supply chain. Think about how much supply chain, how much of BOM we have to deal with. So the other priority is really making sure that, hey, whatever that we do on supply chain or how we manage, you know, all the back end workflow needs to be really, really efficient. So I think and engineering, not to forget, right, we are engineering company. So that's tons of data. There's a lot of tested knowledge in whatever that we are doing. So if you look back at that, so it was very clear that when we set up the Data and AI Council, the three prioritization on how we choose the right AI use cases is that exactly along that line. So we kind of actually put in three pillars of AI focus that we will look at. One is AI computer vision, because that's super important. That actually help us improve our quality inspection a lot more. So AI computer vision, that's one area that we focus. The second area that we focus is really on the machine learning and using data, advanced data analytics, to help us in supply chain orchestration, to help us in forecasting. So that's another big pillar. And last but not least, the third pillar that we actually prioritize our use cases is really about agentic AI and generative AI. How we can actually turn what we call the tested knowledge from the engineers, the old engineers, from the old technicians, into explicit knowledge that we can use, right? So that's the three parts of the prioritization that we look at. Makes a lot of sense, very organized process. Roni, I'm curious about your thoughts on this as well. How has AI or what, I should say, influenced AI prioritization at Workday? I mentioned that intersection of feasibility and impact. Let me double click on both of those. Impact, we really think about the key business drivers. For us, in prioritized order, it's really been around revenue uplift where we can actually earn more revenue, where we can reduce our overall expense, where we can improve productivity, we can improve our customer experience, our CSAT, and then also where we can reduce risk. So we look at each one of those factors as a key business driver around impact. Then when we think about feasibility, really how mature is that AI and frankly do we have the skill sets to effectively deliver. Every once in a while we will take some moonshots that will be some pretty big bets that maybe, we know that there is probably a higher level of risk in actually accomplishing the reward, but for the most part we try to do work that we believe we can accomplish. And so we actually have our business spend time with us to actually build a hypothesis around what we think the outcome will be. And in doing so, we actually measure ourselves against that, which I think has been a lot of fun in the last few years, like how are we performing against the outcomes that we set out to do. There have been a few scenarios where we thought really big and we kind of didn't deliver so big and in some scenarios, especially where there was pretty significant expense for those investments, we've actually pulled some of those investments back or in some cases replatformed to a capability that either turned out to be cheaper or we had more skills available to do some of that work. So, it's been really, really fun to test ourselves against what we thought we would deliver versus what we have delivered. And for the most part, we've been pretty close to what we planned on. When we talked about strategy earlier in our different councils, one thing I left out and it's really frankly one of Workday's secret sauces. As an IT leader, you think, know, if you build it, they will come, you know, like the field of dreams. And initially when we sought to start rolling out some of our AI capabilities in our journey, we're like, here it is! And we realized that you cannot just, you know, welcome to your AI. You really need to kind of think about how you do your customer or your workforce enablement. And I'm really fortunate at Workday that we've got such a strong learning and development team. They really kind of taught us that the best practices around change enablement and for us and even the first year in our AI journey we had a programme we called Everyday AI where we made AI available to every workmate for use every day and within the first year we were able to get upwards of 80%, I think we're 87% utilization for our collaboration suite. And what we really found too is when we leveraged this Izzy kind of an intro tool or an entry tool into AI, it really got the experimentation kind of muscles going, the juices flowing and people started to actually help curate use cases that were functional. I never thought I'd see our legal team but they actually were one of the first teams to actually bring in a firm to actually teach them prompt engineering and this is you know, this is early twenty twenty four. So it's really been exciting to see the different functions get a sense of just from basic collaboration tools and understanding the power of the art of the possible, really curating use cases that actually became functionally the things that actually provided the highest impact and frankly the highest feasibility. It's been wonderful to actually see those use cases in production actually serving our business well. Yeah, that's fantastic. The outcomes focused prioritization you mentioned before really stood out and the people readiness item, which we're going to drill into a little later, is really interesting. And I feel like that everyday AI initiative really creates community as well around AI usage and sharing the best practices among the user community. So just fantastic. Now, Ronnie, you're talking about strategy a little bit, and I find that, at least from the research, what do I know, that strategy and executive sponsorship often go hand in hand. And with increasing sponsorship, budgetary approaches might change between a segment of IT funding to a dedicated AI budget or a hybrid between the two. How do you see that playing out? How has that played out at Workday and how do you see that playing out among your peers? I'll keep speaking in the three years that I've been here, but in my first year we actually had corporate funded, a lot of corporate funded initiatives. Our business partners were just learning the art of the possible. And in fact, of the things we had to do from a strategy and planning perspective is we brought in a few management consulting firms to help actually tease out, here is what we think the real sweet spot is for finance, here is what we think the real opportunity is for customer success, here is what we think the real opportunity is for, let's just say, even engineering. And so, when thought, when we figured out those highest feasibility, highest impact options, those businesses hadn't yet planned in my first year to make some material investments. And so, in our first year we really saw corporate funding being leveraged. In those following years, we really got thoughtful about building that into our fiscal year planning process where we went into the beginning of those years really looking at what do we think we can accomplish, sizing what is that going to take, and actually building those investments into our strategy. One of the really cool things after we had our first years of outcomes is because several of those levers that we were pulling on, those key value drivers, were around cost reduction, we were able to leverage the savings that we garnered in reducing expense to actually fund some of the next year's AI programs. So I am hopeful that now AI just becomes a part of everybody's normal budget planning processes and then it kind of gets baked into it. I frankly think in the first years of any kind of material movement you'll see in AI that you're going to probably need some unplanned or outside appropriate investments. Sure, makes a lot of sense. Mae, would love to hear your perspectives as well. As you've gotten more executive sponsorship in your AI initiatives, how has that influenced budgeting over at Jabil? I think the journey is very similar to what Rani just said, right? So, well, I will also start from the three years when we actually started to establish the Data and Council, right? So that's when, well, a big part of that initiative to set up that Data and AI Council is every of those members that had joined from legal, from operation, from supply chain, they become the change agent of their own function, right? So while they are being educated, they are becoming the change agent. They are transforming themselves. So they bring back that good knowledge back to their own function. And very much more like what Rani is saying, as part of the budgeting process, they will start to actually put budget into their own function to actually say, hey, this is what we wanted to do, and this is so important that we need to budget for our long term planning, our fore planning, that it needs to actually be budgeted into our function. So this is actually where the Data and Area Council, if I may go back to that, actually becomes a real changing agent for the people to actually think differently, right? So the budget actually comes from the function. And not to say IT actually does support quite a lot of those initiatives to using IT budget, because I always feel that you need somebody to be actually the change agent to initiate that, right? So, IT, like what Roni, I'm sure you have experienced that, we actually funded quite a lot of the projects, right? Be it, you know, people who actually built the model, it actually the data scientists, the data engineers. So, we funded all that to actually get the ball rolling and get people excited. And then really, then it actually comes to the recent twelve months, right? And I think we have actually seen some good progresses. We have seen some good results that, hey, the board actually gets really excited. The board really wanted to actually see, 'Hey, can we actually do more in this area? Can we actually pump in more money?' So we do actually have a corporate budget now, actually allocated for us to actually really make sure that we put out the right platform for the people, such that it can actually do what we want it to do. It can actually achieve the thought process that we were talking about, right? So, it is actually going through the range of evolution. Yeah. That's fantastic. So, the board is seeing justification in your investments and expanding them as a result Now, of I want to stick with the Data and AI Council as well to talk a little bit about governance. And I think the first thing I'd like to double click on is the kinds of ethics and security policies that the committee is most focused on. Definitely. So really, when we talk about the governance, right, that's really a few parts of the governance that we manage in the Data and AI Council. So I kind of categorized that into three areas. One is definitely about responsible AI, right? The governance on how do we actually ensure that it, you know, the data that we actually do, the model that we actually do is ensuring fairness, transparency, accountability. So that's one part of the responsible AI that we govern, right? The other part is really about coming back to data. You know, I will not give enough justice if I do not talk enough about data, right? So it is really about data protection, especially about, you know, how do we actually protect our intellectual property? How do we protect our customer data? How do we protect our proprietary data on manufacturing? So it's really about data protection, what you can do and what you cannot do, right? And then the third part of the governance is definitely on vendor governance, right? Can you imagine that, you know, we have 25 countries, have 150,000 people. There's lots of vendors actually going after us, right, and say that, hey, I can provide you a free model, I can provide you a free POC. People actually got interested in that, right? But that brings us even more trouble, because then if everybody, if every site, 100 sites actually wanted to go on different vendors, then we kind of lose control about how do we actually govern and do things properly. So that is actually a specific part about vendor governance, that we wanted to make sure that the AI capabilities that come from the external providers have been supervised, right? We actually have a process that is governed by IT called the Architecture Review Board to actually come and review what the external providers is actually providing. Some are good solution, but some doesn't actually probably fit into our architectural structure. So that's one thing that we talk about when we do governance, right? So the other thing that I do actually want to bring out to the audience also is don't think about governance as a root block. It is really not a root block. It's actually trying to lay out the foundation so that we can actually run faster next time, so that we can actually do something better next time, right? It is not to kill innovation, but it is really a guardrail for us to do things faster, better, and even better, right? Yep. That's helpful, Mae. And one thing I want to double click on as far as governance is concerned, is a lot of people see agentic AI in different ways. Some people see them as entities that can execute tasks. Others see them as digital workers that need to be governed. What's your perspective on that? And how does that influence the way that you manage agents? Yeah. Yeah. So, John, I think that's a fascinating topic. I think you and I were actually also debating on it like a few months ago, right? So, so if we start thinking about AI agents, in my perspective, they are no different from digital employees. And only if we manage them as a digital employee, then we would not lose touch about who are the agents that you have actually deployed. So if you just relate them as employees, but they become a digitalised form, in a normal employee is what you would do as human employees, you will actually have a life cycle on how you track the employees, right? You will actually have an onboarding process. You will train them, you will deploy them, you will make sure that you have performance review on your employee. And then to the later stage, they off board, maybe they actually retire from the job, they actually switch jobs. So that is a whole ecosystem as to how you manage employees. But digital workforce should not be different from that, right? So if you think about AI agents, you need to have something very similar. You need to know how to onboard them, which is equivalent to how you deploy them. You need to train them and that's really about the model training and the prompts engineering that you need to actually continue to upscale your agent. And we need to actually give them the proper assess control, right, and identity management. You need to actually monitor their performance. You need to actually eventually retire them or replace them or actually put them into another place. So that's what you need to do about AI agents and what we need to do about digital workforce. So I think it's super important that agent needs to be managed properly, so that we're aware of what they are doing when they created them. How do you actually intend to deploy them and keep track of them and actually use them? So I'll just divert a little bit to talk about, you know, I was talking to some other CIOs. I was very, very surprised that they told me that they have deployed 2,000 over agents within their environment, right, which is all good, but they do not know where their agents are. Right, so that's a little bit scary to me, right? So yeah, you deploy 2,000 agents, but you do not know where they are right now. That is a problem. So, really advocate that we do need to actually track agents properly, none other than how you actually track your human resources. Yeah, makes a lot of sense. It's important to know where your agents are and what they're doing. Ronnie, I'd love to switch over to you and talk a little bit about some of your governance considerations with AI in Focus. There are a lot of things that need to be checked off before an agent is greenlit to be released in the organization. How should the people who are listening to this think about balancing governance with velocity? So, think every company is a little bit different. Every company has a different risk appetite based on the customers they serve and then the sensitivity of your data. And so, me, two councils I talked about, our AI executive advisory and our operations council were key. I mentioned they were staffed by legal and we had a responsible AI team that helped give us some very thoughtful guardrails. We had obviously our cybersecurity friends who were making sure that they understood how to get ahead of the threats that might potentially be introduced by new technology and really just making sure that we understood what are the security compliance, the intellectual property protections we need to have in place Then just the responsible AI and the ethics that we need to make sure were the guiding principles about how we manage. And I love what Mae said earlier, guardrails don't slow you down, they actually let you go faster. Imagine if you were just trying to run on a dirt road or drive a car on a dirt road, when you pave it and you put some lanes that means multiple agents can now run faster than they would if they didn't have these rules, didn't have this paved road. So, us, the infrastructure of governance really, really let us run a lot faster. It included making sure that we actually did register our agents. It meant that we knew exactly what they were doing. To Maeve's point, we actually do performance manage our agents. We retired some of our first ones because as we were AB testing our capabilities that come online a year later, we found that we could actually, frankly the times of value in delivering with another platform was five times as fast. We had more skilled workforce to deliver in a certain manner. So, we were able to, in some cases, almost 25x the delivery velocity of retired some things that had served us well, but could not perform in the same way that we could get value out of some other ones. So, I think performance management was really interesting. Another thing that three years ago we were all sitting in middle. And we now have some agents that are directly in front of our customers, but one of our guardrails to let us go really fast, especially when we were deploying third party agents, was to develop our own capability to do LLM monitoring, where we're obviously looking at the basic things like accuracy, toxicity, and bias, but even tone, making sure that the agent reflected the tone. If someone was telling us that something awful had happened, it sounded empathetic. If someone was suggesting to do something we know was wrong, we made sure it had a firm tone back to them. And so, it was really, really important for us to be able to get trust in our agents, be able to monitor them. And this wasn't just a monitor at go live, this is monitor wide. It's like it's nightly regression testing or frankly, more than that. So, it was really, really important for us to get some level of confidence with those agents that we did have some cadence management and monitoring of them. So, I definitely think no matter what, especially things in production, when you have sensitive data, putting those guardrails in place let you go faster so that you have more confidence and faith. Because frankly, you have a bad, everybody knows this from security events, when you have a bad event, everything slows back down to a halt and everyone wants to re examine everything. So, I think those guardrails actually help engender trust that's lasting. Yeah, it's really interesting that you say that, Ronnie, that governance helps accelerate adoption velocity, but trust trade offs don't end up being made. It's important to be diligent about governance and monitoring. So it's really helpful to hear your insights on that. May, I'd like to switch back to you and move from governance to operational and technology readiness. From your perspective, what are the essential ingredients to AI tech readiness? Yeah, I think, so, I think we always hear the terms about people, process and system. I think that three elements really jump well into operation readiness, right? So we have the people readiness. How do we actually get people trained to know what they need to know about AI, right? So that we do not actually, you know, so that we set the right expectation about what AI can do and cannot do, right? So I think that's super important. So that, I think we will dwell a little bit more about people readiness, but definitely about people, right? And the process is actually a very key thing in this whole equation, right? And process actually encompasses a lot of things. So if you think about AI, right? If you, if you just talk about AI without the process, it doesn't deliver value. It is just sitting like a model repository. It's just sitting right there with lots of different models, right? But when you actually integrate the processes, you integrate workflow into AI, that's where the value actually started, right? So I think that's super important. And then definitely on technology. So technology is a glue that actually kind of integrates the people process and the systems together, right? So, technology is actually the easiest part of all equation, to be honest. And I think all CIO actually knows about that, right? So, how do we actually get our people ready from technology perspective? How do we actually get the platform ready, so that our user doesn't need to consider about that? So, but if I actually kind of double click more into really, you know, like operationally, what is really needed, right? We talk about process, but there's more than process. So I cannot stop emphasising about data. And I think Ronnie will agree with me, right? So data is the new goal, right? Data is fundamental. And we need to make sure that, you know, the data that has been created needs to be trustable. It needs to be contextualised, so that we can actually really use it for the enterprise data, right? Tons of data out there. So, we encountered some of the, you know, nuances as to, you know, this data means something in this place, but it doesn't mean something in the finance community. So, we need to actually make sure that we have, give the data the right context, right? We actually, we actually look at data properly. We manage the data properly. And trust me, it's a very, very difficult thing to do, right? So, one thing that we started on cleaning the data and managing the data and putting the right data governance is that we set up the data governance rules. We put in a governed data platform that people can actually start to clean the data and put the data into that format, into a governed data platform. And it's very use case driven. It's very use case driven, meaning that, you know, there's so much data to be cleaned up and every time everybody is actually cleaning up the above data. So we need to be focused on what data are we going to clean up right now. So it is use case driven. So whatever use case that we're working on, that's where we actually spend a lot of effort to actually clean up the data while we are actually doing the other things, right? So data is definitely one part of it, right? The other pillar that we are talking about in terms of that process and system is really the platform. How do we actually make sure that we have a scalable AI infrastructure and make it available for everybody, right? So who is eligible for it? Who has the, who can actually assess to it? I think that's also important. So it's really the platform side of things. That is really the integration part of it. How do you actually enable AI to be integrated right into your workflow? I think that's like what we say, it's super important. If without workflow, that's nothing, it's just model, right? And then the last thing is about the governance, the responsible use of AI. I view this as the foundation of getting ready for AI and making sure that we are operationally ready. Yeah, that completely makes sense. And one thing I want to double click on is, along with the foundations you mentioned, where has it made sense to consolidate systems? Well, I think that's a very heavy loaded question, but I think it's super important, right? I think a lot of the CIOs out there are actually like us with many, many, many systems, right? And how do we actually make sure that when we look at the consolidation, where are the three areas? So, for me, there's a few areas that stood out, right? First is actually about enterprise, enterprise platform, right? We are very focused on consolidating cost systems around strategic platform that we are using, be it for financial purpose, operational data, manufacturing data. I think that those are the core systems that we are talking about. And I think when we consolidate around those few core areas, it actually makes it easier for us to actually link the data together, for us to actually make use of the AI, you know, insights much better. So things like HR and finance, right, and operation when they're unified, AI can really provide a lot more deep insights about HR, finance, operation can work it out together. So I think enterprise platform is one area that you look at. The second platform is really about data platform, right? So again, 100 over location, there are many people who have different way on how they wanted to actually handle the data, what kind of platform they wanted to use. So we, we are kind of mandating people to actually kind of use the platform that we have selected. So one advocate that we actually push very, very strongly in Jabil is that leave the technology to the IT people, right? So operation, finance, supply chain, you focus on your workflow, you focus on what you need to do, and what you really, really know well, right? So, let IT do the technology platform. Let IT actually decide on the right data platform, so that, you know, if we, if we actually can't consolidate data in, in the right structure, in the right governed data platform, that makes our data management much easier, right? So, enterprise platform, data platform, and really the workflow system that we talk about, right? So, what are the right workflow systems that can actually integrate well into all the AI business intelligence? I think that's the third part that we talk about in terms of technology consolidation. And May, we're fortunate to be partnered with you as a part of that, so we really appreciate that. Ronnie, I want to switch over to you on the data readiness piece of these foundations. How would you say that data management approaches have changed in your office with the introduction of generative and AgenTAKI? I think Mae actually gave a really robust answer to that. So, I'm going kind of double click on one of her comments that I think was really important around consolidation. One of the things that we've seen, I mentioned that we let a thousand flowers bloom back three years ago because we were doing little point solutions. The really interesting thing that's happening where we're starting to see consolidation is where we're choosing what we call as the enterprise kind of systems of record to do that body of work. Where we had like maybe 20 things doing that work in the ecosystem that we can consolidate it down to kind of these real enterprise grade platforms. That's also where kind of making sure you've got very clean, high quality data becomes interesting. We were realizing when we had multiple little point solutions answering across an ecosystem, we were actually getting conflicting answers. And so, getting to that single consolidated centralized platform where we had high quality data that we can trust in a true system of record really, really let us to go across and start to not just do these kind of team based or function based AI initiatives but let us do real transformation because we had high quality answers that we could truly rely on that we knew were coming out of a system of record with high quality data. So, I cannot underscore what Mae was saying enough, really getting to that high quality data and actually putting it in a real system of record is so, so important. You will, if you unfortunately place it in different places with little context or that doesn't have the enterprise grade context, you will start to get answers that really aren't the intended outcome. So, I really think that's an important part of more of an enterprise data strategy. That makes sense, Ronnie. And just to double click on that, you mentioned earlier in today's discussion that Workday is customer zero for the Workday platform. So how is the platform supporting you in some of the efforts that you talked about? I mean, it's one, I get it for free, which is awesome, but we get a chance to actually truly get enterprise grade data and Workday has made a number of acquisitions recently and one of them, Asana, which we had a big launch today. It's allowing us to have a single unified front door for our overall employee experience. It's integrated with a lot of our collaboration systems and all kinds of sources of data and it's actually helping create a better unified employee experience across multiple different platforms still enabled and able to trust this high quality rich data in a single source of truth like Workday. So for us it's been fantastic to be able to one, be customer zero and be able to share with you all what may be to come for you. A true enterprise grade platform that you can trust that starts to spin outside of what people normally would call ERP but really just becomes the front door to how your employees can work. Yeah, thank you for those insights, Ronnie. May, I'd like to go back to you. I don't want to put words in your mouth, but it sounds like you're talking about the high priority you have about people readiness compared to tech readiness, and how it's perhaps a little more difficult on the people readiness side of things. Which enablement approaches have you found to be more effective to bring about adoption and change management at Jabil? Yeah, definitely. I think that's really, really important, right. So I don't think people readiness is the most difficult, but it is actually equally difficult, right? Because when we are talking about, you know, the whole change of the AI adoption, it's a big transformation. It's a, you know, it's a change process. Anytime when you have a transformation, when you have a change process, you need to make sure that these are not daunting words, right? These are not scary words to people, right? So how do we actually manage that? I think definitely important is that, you know, it is really trying to help our people to actually understand how AI can actually help them in their work, right? So I think we are only successful as leaders, are only successful if we can actually educate our people on how they can actually use AI to even do better job, to actually help them to improve productivity, to help them in making decisions. That's where we can actually call ourselves to be successful, right? So, when we actually focus on people readiness, there are a few things that we focus on. Definitely the fluency, right? Getting people to actually understand the context of what we are doing. Understand, you know, the AI literacy. We call it AI literacy. So what we have done is, using the WhoWorkday platform, we actually roll out programmes to actually every employee, to make sure that they have to go through very, very simple training, right? Basic one, three training, or one on one training on what is AI, some of the common terminology that you see in AI and data, right? So, that actually sets the foundation on literacy. I'm not saying that's the only thing, right? But that's step number one. Everybody go through work day, they have to complete their basic training one on one on data and AI, right? So then we also organise a lot of brown bag session, lunch and learn sessions, to make sure that, you know, there are interesting topics that people get excited about, right? Like Chechippity, like all these models that you actually often see. So we organise Brownback session. We invite, users to actually come in and talk about their own experience. So usually we have a few thousand people that sign up for those kind of programs, right? So those are the brown bag sessions. Then we also do clinic sessions, right? Very dedicated clinic sessions that we send our data engineers, we send our data scientists to the site, that when they have questions, and we sit down with them, literally put out a table, have our engineers sitting right there, and people come in and out to actually bring in their question, to actually query about it. So all these are big part of, you know, literacy, right? And then really, then the second thing is about, after you have actually gone through the literacy, we need to make sure that they have hands on experience, right? So they can only learn when they have that hands on experience. In fact, we are the first one that signed out for co pilot, right? We also have CHET GPT, enterprise license that's available for people who needs it. Not everybody will get it, but the managers are the best judge as to, you know, who needs to actually have their hands on this productivity tool. So, there are actually, again, training sessions that pair up with all these hands on tools that we actually allow them to actually go use it. So, that's about fluency that we talk about, right? So, when they become not scared about that, then they can actually start to use that. And I talk about change agent being in my data and AI council. I think they are excellent, right? Because we have representatives, senior representatives from each and every function. They now become the change agent and actually bring back the knowledge to their own function. They are the ones that curate and actually see which are the right AI use cases that we should be investing. So everything that we talk about in the Data and AI Council, every governance that we set up, they become the best person to actually kind of bring it back to their function. I think that's a very effective model that we are doing. Mei, it sounds like you have such a great focus on AI literacy, and one thing that also stood out is it's not just the how, but it's the why. Why should your users adopt this? And you're explicit about how it helps them work better in the context of the job. So, thought that was fantastic. Ronnie, I want to switch over to you for a bit. You know, enablement isn't a one size fits all thing because IT employee readiness might be different from end user community readiness. How do you think about that at Workday? Well, one of the things that surprised me in my first year of my AI journey, at least at Workday, was that my agents mostly were being managed by my infrastructure teams. As we went out to try to enable the business, we realized my infrastructure team at the time did not have the business intimacy that was necessary to really kind of tease out the art of the possible or when it was time to even begin some of the implementations, they didn't communicate in a way that was effectively driving the outcomes at the speed we were looking for and so there were a couple of things that we needed to do. One of it was that we created our AI strategy and product management function that really partnered with each one of our business functions, developed that business intimacy and can communicate effectively in business languages in a very different way than sometimes your traditional infrastructure teams. Now my infrastructure team is not traditional in any way, but it became the model for how we thought about what is a frankly, what we call an AI pod look like versus an agile squad and so it started to affect even what our product operating model looks like and it changed what our engagement model looks like. It had us moving from, you know, some cases we had three and four week sprints down to two week sprints and frankly the sprints was a deliverable, know, you could actually deliver AI capabilities in some cases in a single sprint and so it really, really changed how we thought about things. I think about how it changed product management, it absolutely changed, you know, where some of the AI lived, but it changed the requirement across the, frankly, our entire IT team to be able to be better communicators to influence. When you're doing transformation, you have to come from a trusted place, means you have to be a great listener, you have to be an effective communicator, and you have to be able to make sure you speak the language of the business, not in some complex LLM talk that our business stakeholders couldn't understand. So, thought that was really an important migration of just thought process from us. The one thing I will say that Mae touched on, think is really, really important, I want to have another exclamation point on that, was that when you get a chance to actually monitor if you will, or see how are your employees adopting, there's a way you can see and frankly we do it in Workday, we actually can see their utilization of many of our core platforms. So, if you were in an engineering function and you were using some engineering coding tool, we could see your adoption of whatever that technology or platform was. If you were in marketing and you were using some marketing automation tool that was leveraging AI, we were seeking to baseline not just your adoption, your utilization and the flow of work enough to determine whether or not you are effectively using it. So, moving from understanding utilization to actually making sure that we truly call it fluency. And now we're at the stages where for those folks who we consider to be kind of our advanced AI users or the folks who are in kind of our data science AI and ML practices, we actually are getting to the point where we are expecting to actually test your proficiency. So, I think it's really, really important. When we show people what is the benchmark, you see people kind of striving for more than what we were expecting them to get to, but it comes a competition with themselves and with other teams to really, really kind of, to grow that skill set and to be able to deliver value back to our business stakeholders. I'd love to see us kind of internally, non intentionally gamify this, but it's got a lot of fun around watching people reskill themselves just because it's fun. This is just an incredible time to be in IT, a wonderful time to be at CIO, I'm sure it may get the same. Hey, Ronnie, if I can interrupt, right? I think I'm really going to steal that, right? To gamify it, I think that's super interesting. Yeah, that's fantastic, Ronnie. And I feel like it's interesting that you were talking about business acumen for IT. You're talking about adoption and how adoption is a real value driver and the importance of visibility into those items. And I'd like to transition over to AI outcomes and ask both of you about how you're thinking about AI outcomes. Mae, perhaps we can start with you. At Jabil, what kinds of things are you monitoring to measure the efficacy of your AI investments? Yeah. So, I think everybody started, we'll start off with, hey, what's the ROI, right? So, how do you, what kind of ROI are we actually achieving with all this AI invention? What kind of additional productivity? How is it helping to reduce cycle time or improve our efficiency? So, we are no different, right? But I'm saying that this is not the only matrix, and it's not the only important matrix, so definitely on our own. But I think what's more important to us is really adoption, right? So, the other AI outcome that we do measure is really adoption. How many employees are actively using the AI tools? I think that's important that we actually do understand the adoption rate. And, you know, I think we are at a stage that we have enough POC, right? So, we need to actually start industrialise our AI solutions. So, that's where the adoption matrix actually becomes important. I do not just want to deploy AI solution in one site. I have 99 more sites that I do actually need to apply those invention, right? So, that's really where adoption matrix actually matters to me. And then the third one is really about innovations, right? How many new ideas? How many more ideas? How many more use cases that can actually deliver value are we actually generating internally? I think that's super important. That also sends the right signal that, hey, we are not trying to kill innovation. It's really important that we encourage, and we continue to encourage innovation. And usually the best innovation comes from our sites, a 100 different sites, because they are meeting the issues on a day to day basis. They have new ways of actually understanding how to actually deploy new ways to do things better. So for Jabil, we, I think we have done it quite well. Year we do actually host deliver best practice, right? So where we actually select the best team that actually, actually have solution deployed, and we bring them to headquarter and actually let them share, record the session and let them share their best practices, their best solution. And the next best thing that we are doing, we are starting to track how many other sites have actually adopted these new practices and actually deploy them into, into their site, right? So I think that that's actually back to adoption and actually innovation. So those are the AI outcomes that we are measuring right now. What a great way to drive community ownership among the people who are across the Jabil organization and then see the efficacy of those and the great outcomes that those produce. That's fantastic. Thank you, Mae. Ronnie, curious about how Workday is doing this as well. Some of the same. I love that your adoption is going viral, so I may be borrowing that from you. So, adoption in AI fluency is one of the key measures for us too. We like to actually track the percentage of our employees who are active users on specific platforms, especially when they become functional based. We track training completion rates and then we actually started up the process for our technical users of fluency testing. And so, making sure that we truly believe that they are getting to the level of proficiency we expect for them to deliver those outcomes. Productivity, and these are all measured in different functions, different ways, but for those who are technical or developers, code completion acceleration becomes important. For those in operations roles, we expect to get to a certain level of case deflection or getting to a support resolution or what we call mean time to resolve MTTR. We look at delivery velocity, we look at sales efficiency for our revenue functions, and then also for our legal functions things like contract review time, how much can we reduce that. When I think about a third business impact layer, are looking at what we call saved revenue impact or cost avoidance or growth reductions. So, instance, in areas where we actually do get to high case deflection, we end up with the business impact of a reduction in expense because the human agent now is not doing what they were doing 100 cases a day, they are now really only having to interact in one or two cases a day and they are creating a much higher quality customer experience which actually impacts our CSAT. And so, we also like to measure as far as business impact what we call our use case ROI. Do we actually deliver on the return on investment? That one is one of the hardest ones to measure and so we always like to find some other ways to look at things but for some of the use cases that we've had in production a little bit longer for our operations functions we actually are getting to the point where we can get hard numbers around ROI. One that's delighted me I think the most has been risk and compliance. Where before we may never have ever gotten to 100% test coverage, we're really getting close to those numbers. For our folks in audit functions, especially financial or SOX controls or even security audit functions, being able to get to 100% coverage is really, really good. Usually, have to sample because you only can do so much. But because of some of the new capabilities around generative AI, we've really been able to do some interesting things. I had a team member present to me earlier today on, what do you call it? Oh man, I'm afraid it's man, synthetic data. And it's been a long time since we were able to truly trust synthetic data because oftentimes you can actually get back to the root of what was created. So, you do synthetic data, often there is a formula to actually create this data so that you have test data that doesn't actually look like production or it's not copies of production. And so, the idea that you can actually create synthetic data that can't be replicated back to the source, can see what the actual data source started with was not really real and so I'm really excited to actually now be able to put at scale the same size or quantity of production data in test environments so that we can actually test with way better results than we've ever had before with synthetic data. And then one of the things I think that's awesome too, just because we can get to much better coverage is around our reduction in security incidents. So, risk and compliance has been an area of kind of a key impact area. And then, as Mae mentioned, innovation. We like to see how many of our employees are actually contributing to new use cases, especially when they're frankly curated by our business partners versus ourselves. The number of new experiments that we put in production in a year and frankly the number of pilots that we actually graduate to production. So, we love monitoring innovation and seeing a winnow, especially when it's organic, comes from individuals, employees or just submitting them in contests and experiments, really seeing those go into production is really a rewarding feeling. Great suggestions. I hope the audience can take some of those back to their organizations and start doing more impact measurement at their area of investments as well. A couple of questions left. I'm sad that this is almost over. May, I wanted to go back to you and I'm curious, how has Workday been helping you realize value from your AI investments on your journey? Yeah, so John, if you recall what I said, right, so when we consolidate enterprise platform, and what Rani says, system of record, I think those are very, very important concepts. So Workday definitely has played a critical role in that, right, because all our human resource data are actually captured into Workday. So that actually creates a very natural system of records right there. So I think that's super important that we now actually have trusted enterprise data around people and around, you know, what the people are doing, what kind of training records, everything that we are talking about that, right? But the other thing that really make me excited is really the early adoption program that we are working and partnering with Workday, right? So remember I talked about digital workforce, the digital agents, right? So, I really like the thought process that, hey, now we are actually working with Workday together to actually see how we can actually start to track all this digital workforce, digital agent in Workday. So, I think the early adoption program really helped us to actually get access into that. So, to me, it's extremely helpful. And I look forward to even more of that coming in, right? We really appreciate that, May. All right, Ronnie, over to you. Any key takeaways that you'd like for today's audience to keep in mind from the great discussion we had? You know, I mean, everybody's at different stages in their AI journey. I think the one thing I would recommend if you want to kind of get the shortcuts to avoid some of the missteps is like, start with platforms that you not just trust, but platforms that are going to scale and give you leverage over time. We're at the stage where we're mowing the lawn on those thousand flowers we let bloom in the beginning, and it was great for fun experimentation, but some of those platforms didn't scale to the size of growth that we aspire to. And so, if there's any recommendation I make, it's really think about enterprise grade platforms that actually can grow the size of the company, that give you the level of frankly trust, confidence frankly vendor partnership that you'd be looking for. The other thing I'd say if you're really, really early and you're in the beginning, just get started. There's no perfect way to begin the journey, but every month or year or quarter you wait, you really have cost your team the opportunity to learn. Mean the learnings that happened in that first year are pretty tremendous and so frankly just get started and you know introduce great partners. I think the other thing that I'd say and it's probably me and I, we met a few times, I think we both believe is that there's a sense of responsibility I think that all of us have as CIOs as we roll out AI. We understand it has an impact obviously on great business outcomes but it also does, if introduce it the right way, really spark a sense of excitement and curiosity in your workforce. So, be really thoughtful about how you engage, thoughtful about how you deploy, don't just build it and they will come. Really think about your change enablement strategy to really ignite a wonderful passion for technology curiosity that really can create wonderful outcomes for your companies. Fantastic, Rani, thank you for that. Mae, what closing thoughts do you have for everybody? Yeah, so I think Rani actually did summarise it really, really well and I actually agree with everything that she's saying, right? So, for a CIO out there, I think you just need to get your hands dirty and start, start doing it, right? I think that's super important. We are at this incredible time, exciting moment in technology that, you know, we have not seen it for decades, but we need to actually start using it. But technology is not the only thing, right? So, one thing that I do actually wanted to highlight is that, you know, when we actually start to deploy any kind of technology, including AI technology, let's focus on how this technology can actually empower the people, help the people, right? So that's what as leaderhood is a very important leadership focus that we need to actually start it right, right? It's not about technology. AI adoption is not just another technology tools or silver lining that actually helps. It is really talking about changing the whole mindset, changing your workflow, changing how the processes actually wanted to actually work together, right? It is trying to make people actually more creative, more productive, and more impactful. I think that's about the people part that we really, really need to emphasise on this whole thing. And I told John, right, I do actually want to close this statement with AI for good. And I truly, truly believe that, right. So when we talk about AI for good as CIO, where our role is actually trying to help and empower the people, we need to actually use AI and deploy them in an ethical, transparent, and beneficial way for people, right? It is through responsible AI, which Ronnie talked about, right? It is through empowering the people. Is really using AI to solve meaningful problem. And that's really what I mean by AI for good, right? So I do actually want to end in this statement that says that AI is one of the most transformative technology of our time. But the real opportunity isn't just what AI can do, it's what the people can do and can achieve when they work alongside it. Thank you, May. AI that for good and AI that supports people are such important things for us to close on. Thank you for those insights. I wish that we could end the presentation here, but unfortunately I have to show just a few more slides to the folks in the audience just to show them some additional things they can check out if they would like to. Just for quick reference, we have some resources that you can use from Workday to help begin your AI adoption journey. In the documents tab, you can look at our AI maturity models documentation if you'd like to learn more, an AI masterclass that's been delivered by our Workday leaders. And if you're interested in where you sit in the AI maturity journey, go ahead and take our AI maturity benchmarking test. And if you'd like to take an even deeper dive, we'd love for you to attend our upcoming webinar on new AI features that are coming in Workday's upcoming R1 release and to join Workday's AI strategy Special Interest Group if you are a member of the Workday community. There are additional resources on the Docs tab Beyond AI Maturity Models and Masterclasses that you can see here on this slide. And I just want to wrap things up by saying thank you for joining us today and we hope to see you soon.