Name: Demystifying Books and Records in the Age of AI: What Regulators Expect Now
Uploaded: 2026-03-31T16:31:14.938Z
Duration: 1 h 15 min 8 s
Description: Demystifying Books and Records in the Age of AI: What Regulators Expect Now

Transcript for "Demystifying Books and Records in the Age of AI: What Regulators Expect Now": Hello, everyone. Welcome to today's webinar, Demystifying Books and Records in the Age of AI. I am so excited about our session today. Amber and I have been talking about this for what feels like years, but really a few weeks now. We have so many questions and people in attendance, so I'm gonna get through a couple of housekeeping items before we get started today. If I could just go to the next slide. So, of course, we're doing introductions and housekeeping. We're going to get into a refresher with Amber of just around books and records in general for anyone who may be state registered and going to SEC or just wants a refresher in general now that AI has been introduced into the mix. Then we're going to get into just AI. Like, what's the difference between an AI chatbot and an agent that performs tasks, and how is that important to compliance team members? And then we're going to get into the questions that everyone's been asking. So we've collected those a lot up front. We have a lot to talk about today, with regards to all of the questions about disclosures and Form 80 B and all the things. But before we get started, I just wanted to let everyone know we are going to have this entire recording available on demand. If you would like, and you are a live session attendee and we have your attendance records, to receive one IACCP continuing education credit, we are giving that out for free. So thank you for being here. Just get in touch with us, and we can process that for you. And then we have a lot of prompts today, and so I have a couple of members of my team. I'm gonna try and stay focused with Amber because once the two of us get talking, we can't stop, but I I believe Caroline and my team is gonna be prompting a few questions and engaging. So feel free. We're going to go through all these questions with our team. We're going to bring them to the team that Amber works on, our regulatory consulting team, and help develop more materials or answers for those who are asking those questions. So without further ado, I have said your name about five times, but I have not introduced you yet. So before we get into this prompt question, Amber, I would love for you to introduce yourself to the group. Great. Thank you, Elsa. Excited to be here on this topic, and you know, we'll try to keep it under five hours because there's so much to talk about! We're so excited! I'm Amber Tapman. I'm Senior Director in Compliance Advisory, the Partnership Group. I've been here about a dozen years, been in the industry much longer I won't do math here, just to preserve some self esteem which is why this topic is so interesting to me, because we weren't talking about this a couple years ago, the industry has changed so much, and probably never as much as this will change life for all of us. So I'm so excited and looking forward to conversation. I lead an AI working group internally among consultants, so it's something that's top of mind. We're always looking and exploring and trying to find, you know, new use cases and kind of guidelines for people, because as you're going to find out, if you don't know already, the SEC, we don't have finite rules that are specific AI rules. So that requires us to be a little more creative. So, excited. Yeah, I mean, I could go on and on about how excited I am about this conversation. But, you know, going back to that chat prompt, like, think the most interesting thing right now to me there's about five different things that Splinter off. But we're acting like we all know what AI is. Right? And so, like, my knowledge of AI three sixty five days ago is dramatically different as a as personally and professionally. And so when I thought about this prompt question, and I'm sure it's populating in chat now, it's just like, what is the challenge? Is it that I don't understand the differences of it? Is it are you getting granular, you're like, exactly where do I need to report it? It is so different. And it's also you know, we've brought in, you know, OpenAI ChatGPT into our organization. Like, was part of that group, and you're on the task force. And so it's like, how are we leveraging these tools? And then on top of that, how are we teaching people an entirely different way to work? Right? And so it's just ripe for a lot of conversations. And then it's also ripe for a lot of risk. It's what are people doing with so fast, Allison. It's like, by the time people think they understand it, it's moved on. And I think for a lot of people like my clients that I work with, advisors, let's just say a lot of them are not tech forward, right? Didn't come into this business a million years ago or even recently to be tech experts or AI experts. So it is a bit of a different language for them. So what we try to do is say, Okay, we know that it's a fascinating topic, and you can dig deep and never come out if you want to explore it, but we're trying to focus on those things that perhaps you need to know, right? At least right now. Because I am always saying, and I think you've heard me say this, this is such a big change that, you know, I think at some point we will be talking about specialized audits in just AI, because it's going to be woven into everything advisors do. And, you know, we're going to talk today a lot about some cautionary things. That's what we do. We're in the risk business, so we talk about worst case scenarios a lot. But by no means does that mean we are or I am anti AI. I think it's going to change everything, and it should. This is what we need to do. The SEC expects us to implement, they are themselves, right? So it's not at all a negativity, it's just a caution because we're dealing in a space that's quite new, rapid pace of change, and it's new to a lot of people, and we don't have specific rules. So we're just saying, hey, here's some things to think about, but please know, think everybody is going to have to use AI, and they should. The question is, how do you use it? And kind of to tee up on your point, Allison, is that I think a lot of firms, one of the things I run into first, just out of the gate is, I'll say, Do you use AI in your firm? And I hear a lot from CCOs like, Nope. And I'm like, Are you sure? Because I feel like you probably are, and let's explore that. And so I may be in their office and doing a review, talking to some of the people in different areas, come to find out, yeah, absolutely they are. And, you know, I think it's natural that people would be curious. You're going to have tech forward people in some departments that really didn't know they're doing anything wrong, especially if you haven't talked about it yet as a firm, and you don't have procedures yet. So they're using it anyway. So I think it's really important to understand that you now really have to analyze, talk to your people and say, Okay, what are we using? What are you using? And have an understanding of what's happening now, and ask them, What do you want to use? What do Because if you start saying, oh, this is scary, stay away from it. And I realize as compliance people, that's not unusual. But if we say that they're going to use it, and then you have no guardrails around it. And once, you know, you miss the boat if it's that far It becomes progress. Yeah. I keep trying to interrupt you. But it becomes one of these things that it's like, maybe compliance isn't using it. But why would you ever stop someone who has to run massive things in Excel from not using AI to help them with it if it takes them a minute and it took them five So that's where we're at with this. Right? Absolutely. It's efficiency. And if you do it well, everybody does better. The firm does better, the clients. It's just about deciding which tools and we'll talk about that how you know and being really open minded, because this is not one that you sit on the sidelines and say, Nope, we're not going to use it. It's too scary. Just too big. It's too big. I think it's really important. So I appreciate your comments about people that maybe don't know the terminology, because I run into that a lot. And, again, it's a broad area. You don't need to be an expert to know some things, some takeaways, what you need to go back and do in your own office, at least to get started. So that's exciting. Let's segue Everyone's favorite topic, books and records requirements. Yes. People love that. I would love it if you could just and you don't have to read this whole slide, everyone. But I would love it if you could just take some time. Everyone's diligently reading it. We know these people. But let's take some time here and walk us through the We do books and records, webinars regularly, because it is important. It's a list. We know there are a lot of topics, you know, client advisory records, copies of agreements, discretionary accounts, Form ADB policies and procedures, finding I'm not going in order, sorry the trading records. There's so many this is just a summary. We have like five categories here. But if you go and look at 2042, there's a long list. The thing that's fun to me about this list is it's really indicative of where we are as an SEC framework, right? Because it's principles based. It's not prescriptive rules based. So this is from essentially 1940, with little tweaks here and there. So you're not going to see AI, right? Nothing here is going to say AI. In fact, there's a part that I love I'm sorry, I'm jumping ahead the next slide, we're going to talk about retention. In the retention part of this, it talks about microfilm and microfiche. So I don't know about you, but I have never seen an advisor who's keeping books and records on microfilm or microfiche. And I feel like that was technology, if you could call it, that was like from the 1800s. So it still says that in our rule here. So while we need to follow the rule, I think the idea is not necessarily to get bogged down and specific. We're going to have to apply these principles based rules to things that were not thought about, you know, in 1940, right? And I hope you're not using microfilm and microfiche. But if you are, I'd love to hear more about that. Hit me up. Seriously. Or when was the last time you used it? At? That can be in the I know. Right. Scary! Although maybe I'm missing something and we need to know that. The thing about Books and Records and how it ties into AI, we have the same rule for Books and Records requirements. That hasn't changed, clearly. So, AI brings some new kind of points of interest, right? How we have to use these old rules, apply them to new things, to make sure or try to make sure that we are consistent with these rules. And again, some of it you don't know until you see it, right, in an exam or otherwise. Some of the ways that things have changed, We have such a volume, velocity, like the movement of records coming out. Like before, think about anything you would generate, right? You have a person probably that is doing a lot of work and like ta da, here's my spreadsheet or document or my paper or letter to a client. Remember those? Like when you actually had it, that took a lot of time. And so you didn't have as many. Now it is conceivable you will have thousands, right, of little bits and pieces you're going to need to not only know about and review, and we'll talk about that, but retain. So that is a big challenge for all of us. We have new and different types of records. We weren't talking about transcripts, you know, a couple of years ago. We weren't talking about summaries generated by AI of our conversations a few years ago. We weren't talking about prompts. Okay? This is a new world. These are things, different types of records, we're going to have to understand what they are and keep these records. Because as we'll see, it's not just important for these record keeping requirements for books and records purposes, 2042, it's important for all of your record keeping purposes. I think those of you who are in compliance and have been for a while will understand that while this is a list of general topics we need to keep, it's not an exhaustive list. You look at your policies and procedures. You look at your disclosures. How many times does it say, we're going to do A, B, C, we're going to do it this way, we're going to you've got to be able to document that you're doing all of those things. So our books and records, if you will, rule requirements sometimes overlap with those things, sometimes they don't. So we're talking about a whole world of items we need to know about and we need to retain beyond what is mentioned in 2042. So I think people should remember that. Other changes here are accuracy, right? These tools are great. And I know you're going to kind of walk us through different types of AI technologies here because that can be confusing. And we're just kind of all calling it AI, right? The word AI. But, you know, they're great for efficiency and some other things we definitely should use them. You're going to have to have people looking at them in a way you didn't before when people were creating them. Right? So Yeah. I think I've told you this, but so we I I mentioned earlier, we have OpenAI. We use ChatGPT, and so I'm very big AI person. And so I've created custom GPTs, we'll get into what that means if you're not familiar. But we have the most used custom GPT among the people on the marketing team is what's called the regulatory fact checker. And this one gives me giggles because they're like, oh, we don't have to bother Amber and Ish and Ben, the rest of our user services teams, which is wonderful, right? But then it tells me that AML for investment advisors is still a rule. And anyone on this call is an investment advisor and remembers that that got pushed to 2028. Didn't Don't have a heart attack or not now. But I also kind of love that it keeps getting it wrong because it keeps me on my toes because, like, I'm, like, making sure it's not introducing another bull. So it's it's one of those things that I almost appreciate the hallucination because it keeps me on chat. But the whole marketing team they're not programmed. My colleagues are constantly going make sure the AML thing is hard. And so, that's just one nuisance Such good point. And I think it's probably the main point here. Right. Because, you know, you are savvy enough and knowledgeable enough, you're going to recognize when something's not right. I'm more worried about because I'll talk to advisors, my clients all the time, that will say, Oh, I ran this through whatever tool, right? And sometimes I train them, Please ask me! Because don't just assume it's right. And oftentimes, it's not, or it's wrong in different ways. But the confidence, the authority with which Hickoff gives these inputs, I mean, you would believe it if you didn't know better. And I have seen incorrect sources. It's quoting rules that don't exist. It's quoting sources that don't exist. Now I kind of have my own collection of crazy outfits that I'm like, wow, how did this happen? And know, rules, you know, because that's kind of our thing. You know, I've had, I've runnified multiple different, you know, just personally before we started looking at this at work to say, okay, I know this answer. If I ask the question of multiple, what will it tell me? And about half the time on these, it's been wrong. And so I will go back and say, Well, what about this? And it's like, Oh yeah, that's right, Amber. Do you want me include that in the summary? And I'm like, Yes, yes, I do. It's important Right, know that because it is, I think, people that are maybe not tech people or others, we want to believe that it knows everything, because that would be so great and easy. But it is not thinking. It does not know. It is educated. It is trained based on data, right? And it's only as good as the data that it has access to. And then it tries to predict what a response would be based on And your the idea that you're prompt could maybe not be what you really want and could create a different path. But, you know, one of the things I was shocked about, I a study that was talking about the sources of information. So when, let's say you go in to just make, you know, for a query, like, tell me about this rule, tell me about whatever, it's if you're not asking it to do anything, you know, it's not agentic, it's not it's tell me this. And the common sources, the most common sources, you know, across the board that are not only sourced but cited as, you know, places referenced. Number one is Reddit. Also in the top five are some And Wikipedia. Of That's what I was going to say. Like, the very thing, those of you who are a lot closer to being in school than I am, or have kids like I do, and what do you always say? Okay, look at your sources, try to get primary sources, but you know, Wikipedia, use a little caution. This is where a lot of our AI information, that's where they're getting the information today. And Instagram and things. So there are some crosswords. It's great if you want to test sentiment, I think, like what are people talking about? What is out there? And it's not that it's always wrong. It's that it's wrong often enough, or the context is different. And if you don't yourself know what you're asking for, and you have no way to understand, is it right? Is it off? You know, how do I know? It sounds so authoritative, people are believing it. So I think that was probably the most important in my opinion, Alisa, thanks for bringing that up because it's true: if it's wrong, it doesn't do you any good. And God forbid, you're basing your decisions, client decisions, whether that's portfolio construction, whether that's other advice to clients, if you're basing that on inaccurate information, you won't be able to say if question AI made me do it, right? It's just not going to apply. So as part of your fiduciary duty, you really need to know, you know, I have a reason, it's a reasoned ability looking at it, I believe, I have a reasonable belief that this information is correct before I then, you know, use it as the basis for advice, recommendations, anything you might do, right? It starts there. And if it's wrong at the get go, everything later is wrong. Okay, so I think that's the challenge. That is the main challenge. I mean, the hallucinations we've heard about, that comes up with crazy things, but bias. I'm really concerned about bias, because not only in the sources of information that it's kind of, you know, reviewing, if you will, but also the way we prompt. We don't even know sometimes if we are creating, you know, a different response than what we're intending. And then it magnifies over time if we have this inherent bias that we're just not aware of. And we're basing decisions based on this. I think as people start firms start to use this more for portfolio construction, we may see more because it's going to know. And remember, it's looking for data sometimes to say, what do other firms do? What do other people you don't know who it's looking at. That's why we don't it's not a case where you want to look off our neighbor's paper, if you know what I mean. We need to use it to supplement our unique skill set and knowledge and save us for our highest and best use, to shortcut the important things. And I think that's how people are having to learn that. And that's not easy. I want to get more into our slides, because I know we've a lot of people And still I don't want to cut you off. But you know, it feels like a lifetime ago, but I think it was sometime in November, December where FINRA and the SEC released their annual report, so the SEC exam priorities, and then, of course, the FINRA regulatory oversight report. So, I was waiting in anticipation because I was so curious, and I knew you were and several others here at Comply, about how AI was gonna show up. And so, you know, it showed up. It didn't show up loud, but it did it was in the exam priority type priorities. And it's also so the the SEC exam, of course, is a little, more vague in their, you know, surprise in in what they're saying. I think it really just gets back to it is that this is anywhere the AI is touching needs to be documented. It needs to be reported, and you need to know what's going on. And so, you know, getting into the FINRA ones, you know, they get a little more specific about Gen AI. And so that's a lot of the stuff we were talking about. So generative AI is when you're using an AI, and it's creating a piece of content or an output that you are then gonna leverage. And so, you know, if you've got a marketing team that's AI savvy and they're producing, you know, content materials and blogs and website copy and all that, and it's still going through the regular old, you know, marketing review that you have set up, great use case. But if your marketing team's going rogue and there's no use cases and there's no defined process, that's problematic. But it's still a little on the safer side because of that Right. Marketing review piece. The part that keeps me up at night is the research. Right? The Right. What other people can be doing. Because I think, like, even my first experiences with AI, just watching it magically produce something that would have taken me in the past, like, five hours is, like, kind of a a a that gives you whiplash a little bit because you're like, this is incredible. And then you see how much easier life is gonna become. But then it's a matter of you know? And if you're a manager of people, you get this. Like, now you gotta review the AI work. Right? And you're like, I wouldn't have done it like this or that. But when it comes to the investment decisions or portfolio decisions Right. Or any of that decision making, even if it's that's where it gets really hairy, and that's what we're gonna get into. That's where the risks definitely increase, right? And so that's, you know, and understanding humans in the loop, I think people are all worried, you know, about what AI is going to impact jobs, and I think that's true. But maybe it's taking away those functions that, you know, it's designed to do and saving people for this other, like I said, highest and best use of your viewing and making you more efficient. And if you're savvy with AI or become savvier with AI, the cool thing about AI is it can make you savvier while you're using it. You're like, help me do this better. And it's like, oh, Okay. Let me help you prompt me. But I completely lost my train of thought. It's that I don't earth shattering. It changes everything. That's why it's hard to define. And you were right about the SEC. We thought maybe hoped, to some degree, because this is such a wide open space, there are a lot of people that find comfort in regulation, specific regulation, and we're hoping. But that's the difference between well, one of the differences between FINRA and SEC. FINRA is rules based. They're, you know, they're very prescriptive, whereas SEC has always been principles based. So we thought when there was a rule proposal that had to do with conflicts of interest and predictive, you know, technologies that we thought that was a precursor to rules, right? The SEC seemed to be a little hesitant or cautious. And then, last summer, they withdrew the proposal and it wasn't replaced at least not today, so far, it hasn't been replaced. And people, I think, were thinking some of them firms saying, Oh, good, we don't have to do it or they're not concerned. And that is not the case. I want to caution people, it is absolutely not the case that interested in this because they didn't issue it. All they're doing is saying our existing framework should apply to this. It's principles based, risk based. You should be able to apply what you know to all of these new elements. Now, I do think we need to be aware that the SEC is going to be. That's what the priority set is. It's just woven into everything. So you can't really pull out and say AI is a topic. No, it's a topic in every other topic. You need to expect them to be asking a lot of questions and being prepared for a lot of scrutiny in this area. And they themselves are using it, so their attitude definitely has changed. And they're going to expect us to use it too. So I think it's really important to know. No, I think it's really important, and I think this is a really good segue, because I was talking with someone recently, and they were like, oh, there hasn't been AI enforcement actions. And I was like, no, no, no. There sure have been AI enforcement actions. Absolutely. And it's technology based. Right? So it's like AI equals technology, full stop. There's no difference between the two. And so these ones just happened to I would love for you to walk us through kind of, like, the highlights of I'm I'm sure everyone on this webinar can read it. But, you know, like, you know, what happened here is, can be existing maybe in your firm or in either, you know? Absolutely. And Alison, this is just the precursor to more actions we're going to see on this topic. It just happened to be that this was the first area that was really easy to find. And we have seen cases, AI washing, you may have heard that term. It reminds you a lot of those of you who remember ESG with greenwashing when that became popular a few years ago. I think the problem is, we're always looking to better serve our clients. We're looking for new things. Oh! Clients are interested in EFG. Clients are interested in AI. So we try to, you know, accommodate them. We want them to know, Hey, we're looking out for you. We're trying to advance with the times. But some people go too far. Some firms go too far. And then they start saying, Oh, we better get some buzzwords in our disclosures or on our website. We need people to think for all this, because that's a great marketing pitch. I'm no marketing expert, but I know you can tell us that is probably number one marketing I'm guessing. No, my role is product marketing. And, you know, as we, you know, your task force and here at Comprise, we develop our own technology and AI enabled things, we are being very responsible the way we do it. Because we deeply believe that it needs to benefit the compliance team. It does not need to add extra weight on here. But getting into one of these examples. So I found this one the other day, and it was interesting to me. This isn't a firm, but Nate Incorporated basically said that it's got this shopping app that uses AI. And come to find out, it's not. It's outsourced overseas. So with a combo, you know, on the right side, you're just gonna see the SEC charges. But it was DOJ combo ruling. And so they had were hit with fines as well on the civil side. It's kind of one of those things where it's like they're they it's the lines in the sand. They're looking. They're wanting to figure out, like, if you're using AI to power something and really Mhmm. 9% of it is offshore, you know, humans doing the thing with a little bit of AI assistance. That's misleading the investors. Absolutely. And Alison, the thing that's, I think, reassuring, or I hope it's reassuring to our listeners is, this is not new. Like, if you were to say, you know, these violations there are three cases, I encourage you to go look at them involving greenwashing. It's basically misstating or exaggerating what you're doing, how you're doing it. And we've always said, you know, don't use hyperbole, don't exaggerate, because you need to be able to demonstrate you are doing these things. And it's really hard to say, we're cutting edge. What does that mean? How do you demonstrate that? But if you're saying, we're using this advanced algorithm, number one, how do you define advanced? Number two, if you say, well, that's the intern studying math, and he does some things. Like, he is not an advanced algorithm. The spreadsheet is not an advanced algorithm. It's misleading. It's fraudulent. It's fraudulent. And and so this is not new. The next one I think is pretty interesting. Oh. Yeah. It's it's Hold on. My little Is my mouse not working? And it wasn't working earlier. Okay. So this one is also huge AI washing. Right? Yeah. Yeah. They're all And so I'm just sorry. It's just on my screen. That's okay. And it's the same concept. I mean, facts are a bit different. But it's about, you know, if you're holding something out, whether it's specifically said or you're implying you're doing something in a certain way for clients. Clients rely on it, and if you can't demonstrate that, or if something goes wrong in that process, you can't hold that out. It is promissory, it is a violation of anti fraud provisions, and of course they're going to find these. These are easier to find too than some of the other AI issues because marketing has long been a scrutinized thing. So I think it's definitely important, you can see it's got to match. If you can't do it, if you don't do it the way you're saying you are, know, don't pass go. You can't do this. And advisors should be used to this. You couldn't say it even if it didn't involve AI. If it involved, you know, hey, we used, you know, an abacus to do all the work. Like if you're not, you're still fraudulent, right? It's still misleading. And I think people are getting excited because they want to meet clients where they are and say they're using AI without maybe understanding the definitions. Or some skeptically might be just saying, hey, let's get it in there so we can talk to clients who are interested. So there's a lot of similarities. And I'm like, okay. Well, do you you know, it's like it's a mixed bag. It's like I'm getting pressure, or it's like I'm really interested, or we can augment everything. But, you know, this last one is also very similar with the misleading AI assessment claims. I think the TLDR in all of it is that, you know, you need to do what you said you did. Absolutely. And again, hasn't changed. As AI continues to change and evolve, the technology grows, this will still be here. You can't mislead people. You can't commit fraud. Yeah. So if you're getting pressure from someone to AI wash something that you guys are doing, here are three great examples of how that's gonna land you in hot water. So this is gonna lead me into my next prompt because the next section is going to be super fun. On a scale of one to five, just put one, two, three, four, five, or you can elaborate if you'd like, how would you rate your knowledge of AI? So, around January, we had an AI consultant come in for the marketing team to help us understand the changes it would make to the way people search. Right? You know, we need to appear in, you know, AI search or whatever it may be. And I think we all ranked ourselves very high. And then after the session, we were like, we're more like a two, a three. But since then, we've worked with that consultant, and we've gotten really up to speed. But I think some of the biggest things that I'm gonna get into here is just level setting of understanding AI in general. And so last year, we ran a survey, which we are rerunning again this summer. This one's gonna be way more on AI, so look forward to sharing that with the group and in broader groups. But we asked a bunch of CCOs, what are the use cases that you're using for AI? And, of course, risk scoring and alerts, document summarization, and marketing communications review ranked among the top. So going back to that entire developing AI for things that are really going to free up your time and let you have that human expertise level of supervisory control is really, really great. And it's great to hear that firms this was done, I want to say, like, nine months ago data, you know, are that actively using it there. So, of course, with new technology comes the risks. And so none of these should be a surprise to anyone. They probably, you know, probably have an uneasy feeling, at least one or two of them. But, you know, the 57% showing up saying regulatory expectations and disclosures. Like, Amber, you know, what's the question they're going to ask? Like Mhmm. And it sounds, even from just picking up on what you've been saying, it's that they're going to ask about AI within all of the questions. So how is AI showing up in this versus, like, let's now tune to AI, right? Which is is why you have to know what people are doing. You also have to understand if AI is embedded in any of the products your vendors use, because I don't think some people understand that. And that's hard, but it's a whole different ballgame. So that's a really, you know, good that's a good, you know, identification of risk. Absolutely. And then you get into data privacy and security. And that's the one where it's like, if you've got people at your firm going into the free version of Chatuchiki or Claude or Gemini or Perplexity or whatever it may be, and they're just doing their own thing in there, like, all of that is fair game open. You know? I was actually, like, thinking to myself, you know, how many people who have gotten, like, job offers and contracts that they've uploaded to, like, reopen AI. Like, just because they're not even Right. Who we get. Like, there's probably so much information, personal information going into these systems Right. And innocently. Because it's like, oh, I'm just using it for that. But you just don't know the source of something like that And or a data we should say this. And I know you're going there, and everybody might already know this. But our client data that is such an important thing. Yes, you can still tell us, I know about this, you know, a closed system, you know, versus open and don't put client don't put PII or NPI in these. Also understand, no matter what you're doing from proprietary information, anything, are the tools using what information you're providing or it has access to if you've set it up to link to something else? Are they using that for training purposes? Because if they are, who knows how far this information will go. Yeah, what is the definition of training purposes? Look how. That's a big, check mark, question mark. Absolutely, and this is what you like your prompts or information you upload. Some of the tools, as we progress, will connect with other tools. So it might overlay your email system. So now we're talking about, you know, our, you know, meeting summaries and transcriptions. You know, what's said there? Where is it going? Who's, you know, other third parties? Are they going to give it to those parties? This is a really scary area and it ties into REGSP. Know everyone's aware of Reg SP and being really sure what is it the breach on the vendor side are they telling you, because now you have a lot more vendors to scrutinize and be aware of. And the risks are real there. Agreed. And the last three, or the last one and then the other two, like the model trainability and explainability is what truly keeps me up at night if regulatory compliance were to keep me up at night. But it really genuinely is like putting your faith in the hands of a vendor that's gonna say this is how it works, and that's the definitive thing. And I think, like, just looking at, you know, the people who choose this profession and all of us at work in a professional level, I really do feel like I need to understand the output to stand behind it. And so if you're starting to get that gut feeling that the vendor you're using for AI, whether it's a point solution in regulatory compliance or broader implementation across the firm, you need to feel like you can testify in front of Congress about it. Have to explain how it works. Absolutely. That's what we're going to get to. And I think the last two in this slide actually perfectly separate. The next thing is the lack of internal expertise and the cultural resistance, I think, are in a lot of ways intertwined, right? Where it's like, I've done the social media. I've done the transition to Zoom. I've done all these things. Enough already. I don't want any more tech. But this is the slide that gives me heartburn, right? And so in this poll, we had I think it was 67% of firms were using AI, and then another 20 something were about to start leveraging it back when this data was done. Those 49% have a formal AI policy governance structure in place. And of course, at what point are you going be like, oh, let's completely overhaul this now that we have this technology Especially since you don't even know it's be used. Right? It's not going to have a policy if you don't even know people are using it. Where it all boils down to is, how are you using AI? Have you defined it? What are the acceptable use cases? Where does regulatory compliance sit in that Because procurement if you are not in the procurement group, anytime AI or machine learning or anything really touches the business, you need to be. And you can send them those three examples of enforcement actions. But it's scary because it's I remember this early in my career, I was doing public relations and social media, and this was sixteen, seventeen years ago. And I had to write a social media policy for the company because they didn't have one. And they were like, we don't really know it, so can you write it, and we'll just let you know if it's acceptable legally? And I'm like, okay. But that was kind of one of the ways that businesses handled people being online and still a representation of the company. You can't act afoul on the internet and expect to come into work the next day. Vice versa, you can't be promoting our stuff in a way that's not approved by the company as well. And so that kind of shifted. And it reminds me a little bit about that. And then I think the marketing social media pretty quickly after, as all that kind of devolved. Right. You need to tell people what they can and can't do, which tools they can and can't use, how are they using it, how are they documenting who's reviewing it, you know, all these things. But it's not different in principle from what you need to do before on any other business. This is how we design policies and procedures. And when you update, and you should, your policies and procedures, don't be surprised if you're having to update many, if not most, sections in your manuals. Would you say that people should update sections? Or do you think that people need to do an AI policy and update sections? Well, it's easy for me to say because unless I'm doing it for clients You're not doing it. I would all I'm a risk averse. Both. And the reason I say this is because it would be hard in a standalone policy to cover all of the little nuances that are going to be important in every single use case. And I always advocate when creating policies and procedures for the those closest to it, those actually doing it, to be involved in the process. So I like marketing people involved when we build marketing related policies and procedures, because they're the ones doing it, we need to make sure it reflects that, and they will, you know, kind of buy in and help us evolve You over can't do that with an AI only. You need to get people that are closest to it and really tell you how it's changing, how their use is changing. So that's where I kind of leap back to an audit before, because it's really hard when it's you. And tools. You know, of the things that we're starting to see, not only us, and I think we'll see more of, is how do we use AI and the power of AI to make life easier for our clients, right? From a product standpoint, too. It reminds me a lot of when cybersecurity became a big issue and people were getting vendors, and I was talking to clients saying, they don't understand what we do. This is, you know, can we find a vendor that specializes in our space? That's one of the challenges I'm seeing with, I think, the use of vendors. It's great if you find vendors who understand the industry, have some expertise in the industry, because think about the data sets they're trained on. Think about, like, it won't understand industry jargon, you know, there's some rules that, you know, I think a tool that is specific, has a lot of use in our industry, it will benefit you greatly. So I just wanted to say that because I think about people kind of using all of these random tools without governance. It's much easier if you are careful and judicious in how you select your tools, the credibility of those tools, not to say you don't need to review, but you really need to, you know, that'll shortcut your effort a little bit to know who you're working with, what their history and data set they're trained on. So I think that's just my 2¢ because it's really hard when you're dealing with a tool that really doesn't have a lot of industry It's the air first thing I ask when anyone purchased me. I was speaking at an event recently, and a bunch of you came up afterwards with very specific questions about Form 80 B and policies and procedures. And the first question I ask when anyone we had a client reach out about our AI roadmap, I was like, What's your governance look like? We want to know first what your governance looks like, what your audit looks like, a full audit of every AI you get in The US listing that you don't know about. Think it gets looking at this slide now of just the misconceptions, I want to kind of focus there for a minute because the AI chat tools are you downloading the app for ChatGPT or Gemini or whatever it may be and chatting in the free space, right? Or you could have one walled off, like we have the enterprise OpenAI contract where all of our data, everything that we have use cases for it, we have use cases for pulling any sort of information in there, there are certain things that are completely walled off, but we know it's a safe and protected environment. IT team went through it and all of that stuff. So, that's like the AI chat, And that's why Amber's talking a lot about prompt engineering. And I can follow-up with some supplemental material on this because it's really important how you craft your prompt. It's also cool because you can tell you can word vomit to the AI that you want them to help you with the prompt, and then they will turn that into a prompt that then you didn't prompt them with, and then you get the output you want, which is kind of hysterical because you're learning as you're going, which is personally my favorite part about AI. But then we get into the custom GBTs. So these are my example earlier is a custom GBT. So we have this regulatory fact checker, and in its information, I'm like, you're a former SEC regulator, you're a former FINMA regulator, you know FCA compliance, like any regulatory regimes that comply specializes in. And, you know, I've got, you know, the different data points and telling it to look up things on the sec.gov website, the FINRA website, like it needs to be these sources. That one is great because you've got it pointed. You've got it focused on the task it needs to do versus, like, just the chat where you're saying to the broad AI gods, do this thing out of context, it's narrowed down even more. And so one of the things that Comply is about to release is a ComplyAID policy guide. So it's that custom walled off enterprise, you know, solution that would allow our clients, many of you on the call, who get the most common questions over and over and over again. What's my gift limit? What's my this limit? What's the policy for this or that? Now your employees would be able to engage with that chat and then say, I want to go to, you know, the open with a client. What's the rules around that? Or whatever it may be, to then reduce the time that your team has. Because we hear it all the time. Like, we are the help desk for policy questions, and we can't take it anymore. So we hear you, and that's one of the things we've been developing. The agentic AI piece is, in my personal opinion, I have not seen show up a lot in compliance yet, which is when you have an AI agent, not a chatbot, but a mechanism with workflows that perform a task. So that would be like, hey, I need you to go into the comply platform and pre clear a trade for me. And you'd be typing it into the chatbot, and it would take care of that for you. I have not personally seen that in the compliance No, but I think it's coming. Think it's coming. And I think it might be also something like we were talking about before, the vendors. Make sure you understand how these vendors work, I think we're applying some of that embedded there already. Yeah. So back to not trying to spill over secrets on this. But Amber and I obviously are with this AI task force, and we sit with product a lot. And so anything we develop will have that regulatory backbone because we have the regulatory expertise here. But if you are working with a vendor who doesn't have regulatory expertise or a team of former regulators on staff, it might get a little more dicey with that whole explainability. And we've tested that, Allison. You and I were talking about that. We've tested just you know, for fun, because I guess that's fun for people like us. It's a blast. Get a widely varying and you can tell it doesn't a lot of times understand the context despite whatever prompt you use. It's why it's so important to have, you know, this credible source to go to. All right. The one thing I want to touch on before we get into the real heart of this conversation, and I wanna be mindful of time as well, is Black Box AI. And so you might see this term out in the wild. It is what model trans transparency and explainability. It's the other word for that. If you're talking to a vendor, if you're talking to anyone, if you're talking to a person on some department that's bought some AI tool and now it's your mess to kind of figure out with regards to the policies and how the use case is, and they cannot tell you exactly how it's done, then you have a situation on your hands that needs to be escalated with that vendor. Because when is what the FCC will ask you. And This is I really think Absolutely not. Because they have control, let's think about it, over the firms. And they have control over this. But they know more than anyone that AI washing is happening. And so they want to make sure that it's being For end use. And I think you need to understand not only what tools you're using for what purposes, and then you need to be tested, whether it's an AI audit, whether it's internally. Try to explain: How does it work? What is that result based on? Whatever form you're using, whatever task you apply it to, whatever output you expect, you need to be able to explain. And it's hard and that's why we're calling it a black box because it's like, well, it takes some stuff and voila! It comes out with this! You can't see where that came from. You know, this is why testing is going to be a huge thing with this, to be able to test the output. Is it what you would have expected? Is there a flaw? Is there bias? But also vendors. We're going to be in this space where a lot of vendors can't explain it. And if they can't explain it to you, you can't explain it to the SEC, which is why when you do your vendor due diligence, which is going to have to be more robust than ever, it's really key that you understand how they're using it, where their point is, if you're using it for compliance, for flagging, problematic or high risk transactions, whatever use it is, if you can explain what it's looking at, what data it's referring to, what are the criteria for flagging, whatever it is, and this true whether it's marketing or compliance or trading, any research anything, any level of AI use or model you need to know it. If you can't explain it, that's a sign that you need to go back and do some more homework, understand it better, talk to your people if they're using it now. Because really, ideally, you won't use it until you can explain it. And if you're working with a vendor that won't be transparent and tell you kind of what it's about, what it's doing I'm not saying give away the secret sauce. I'm saying give you enough information to understand, then it may not be your most credible choice. That's a good point, though, Amber, because they they're not the vendor might not say, oh, we're not going to tell you our proprietary thing. They might make it seem like they don't have to tell you. Right? Like, we're not a regular services provider. So, you know, it's how you use the technology and what you do. That's something that, you know, software first vendors will often, you know, hang their hat on with liability, so to speak. And this is a room full of compliance and risk people, so we can talk about liability pretty freely. But that is, and I'm not calling out anyone specifically, but that is an easy way to say it. If I'm a lawyer, you use it at your own risk at the end of the day. Right. And I am. And I see it in legal applications, similar issues. Again, it's, I think, nobody maybe you don't want to dig deep enough to be able to explain it because you're not a techy person, maybe you're not interested in AI, but you know, you need to know enough to feel comfortable and be able to have a reasonable explanation. You don't need to know everything about the firm. We're not talking about trade secrets here. We're talking about enough to have an idea of what's happening and be able to put that into words. And if you are getting, you know, kind of a runaround from a firm who doesn't understand why you need to know, that's a sign. They don't maybe have to tell you, but you don't have to work with that vendor either. There should be a meeting of the minds there where they understand what you need. And that's why I was saying before that vendors that specialize in our industry are going to be more likely to get in front of that and understand you're going to need the answers to some questions. All right. Let's get into the QA section. So, one more I should have said this earlier, but there's another chat prompt. So, if you haven't already, let us know some of the stuff that you guys are thinking about of having policies and procedures in place and things of that nature. But, I mean, let's just jump right into it, Amber. What do adequate policies and procedures? Those are questions we're getting all the time, right? And so, we're going to talk high level. Some of this we've already covered, actually. We're looking at adequate, right? Your implementation must be effective, but you've got to start somewhere. So you need to understand what you're doing. Who's doing it? How are they doing it? Put some guardrails around it. But, you know, this idea of an AI tool inventory: you're going to be asked for this: What tools are you using, including embedded? Start there. Have that. Have these conversations. And when you work down through your policies and procedures, you'll find out from the policy level, what do we allow? You know, not only tools, but use cases. How do we want you to use it? You're going to put that in, and again, we talked about having maybe a standalone with kind of an overarching description, but also in specific sections, and that's going to vary from firm to firm, because every firm is going to use AI differently. And we all know that you need to customize policies and procedures. We can't use generic we never were supposed to be able to, but now I think we're going to see more differentiation between firms, because of this technology and the capabilities. So it's more important than ever, work with an expert in this area, do it yourself, whatever you do, just make sure that you build in how you do it, what are you using, how are you retaining it, who's reviewing it, right? And that includes testing and validation. You know, whatever use, know, again, use case is different, but if you're using it for something, you're relying on output, what are you going to, you know, if you're questioned, you know, you've got to be able to explain it, but you've got to be able to show that you had a reasonable belief that it's giving you the correct data and that you're using whatever data that might be researched, that might be portfolio construction information, know, whatever it is, how did you reach a level of confidence that the information is correct? So you're going to have to create testing protocols. That's a little difficult sometimes when you don't maybe know enough or you're using a black box tool. So it's really important to match up the tool, the use, and really sit there and think. And you may need help with this, to find out how can I test this? You know, is it I know one of the things I do a lot or recommend a lot is, you know, creates a scenario or situation that you know the outcome, you know what it should be, or what you'd like to see. And then, you know, run it through. And again, I'm talking broadly because everyone's using different tools in different ways. But if the output is not what you would have expected, it's something you need to go back and look at. If you have two similarly similar situations, for example, and you get different output. It's important also to remember that, especially if we're talking about that first level of AI that Alison was talking about, I highly encourage you to keep track of, you know, which AI tool you're using, which version you're using, what date you're looking at, because as those of you who've been in the space playing around maybe with it know, the answer, given the same prompts and the same tool, the answer you will get may be different a month later because in the meantime, it's learning, right? Being exposed to more things. You need to know That's like software as a service, so it's updating without you, know, like, we're not reading the release notes on everything. Like, we love we love products. We love release notes. We appreciate them. But we are not, like, every single time a change is being made to a software system, you know, across all of that. Right. And so, you know, this whole thing, the next question I have hopefully, it doesn't skip to two because I clicked it twice. But the regulatory risk of AI use. But even when you were talking through the policies, I'm sure it is very tempting to use AI to write your policies or edit your policies. You know, I had was chatting with one of our consultants and they had mentioned, you know, sweetly that they had received, you know, some RegSP ChatGPT output to review, from from a cloud. And so it's we're kind of at this stage of of, yes, that's super helpful, but what is the risk if you're using AMP developer policies? And then just basic I think we've talked a lot about the regulatory risk. Mean, nothing's worse than a fine and public embarrassment, right? Well, Or a fiduciary duty. Like, most firms are But well if you can't say there's a legal concept sienter, the intention to do something that poor, when we're talking about anti fraud, you don't need an intention to do something wrong. It's that you didn't do what would have been reasonable under the circumstance to find out that this information is not right, that you should have kicked the tires more, you should have had somebody else look at it. Policies and procedures I've seen these. Clients have sent me things. And remember, we've done anecdotal examples where it doesn't have all the right information, especially general tools that are not necessarily well trained on the nuance of policies and procedures. And I've seen many versions that don't have key points. The problem is, unless you're a compliance a person that lives and breathes compliance, you may not recognize it. What it does provide might seem credible, and it's definitely authoritative, they can give you a chart and a checklist, too. But that doesn't mean it includes everything, and you may not recognize that it doesn't without somebody looking at that. And that's why, again, I encourage people to if you're going to rely on tools, rely on tools that are created by people in the space, or at least vetted and designed by people, or whether that's through automation or an individual person. Because it really is scary because you don't know what you don't know. Yeah, it's that and having an AI savvy expert in the loop. And maybe AI savvy looks like a level one right and it's level two and six months now. Don't impostor syndrome yourself into not thinking that you can tackle all of this and understand all of it. But I think it is just so vital to have a seat at that table with procurement to understand what's going on across the business with all this technology. I talk about this all the time, not that everyone on this webinar has heard me talk about this at all, but there was a venture capital company a couple months ago that you know which one story I'm talking about because I talk about it a little bit. But they had a meeting with the startup, and meeting ended. They shared some not so maybe disparaging things about that startup. Then they went on to talk about other companies, and the meeting wrapped. They said their goodbyes, and then off went the transcript to the startup, not realizing that the department had the meeting notetaker set to just send it. Absolutely. In a normal vanilla call would have been like, eek, this is bad. Like, we don't have this in the policies and procedures. We don't know what this is about. Like, when did this start? But then to be breaking, like, six rules at once and then emailing the transcript, that's now books on record. I'm literally on the hook for Absolutely. Then PI, for me, really. Right. How many times do people historically chat amongst themselves before people join? I've seen that example of it's like, you might be talking about something harmless. But now it's going to be part of the record. And I'm glad you brought this up because that I think is one of the most frequently asked questions about what do I need about to the chat, the mute button. And again, you need to know the tools. Some, you know, just kind of transcribe or try to what you said. Some summarize. Some provide, you know, an analysis or something and they all store it differently. You need to understand: what does it store? You know, some have voice, some have just, you know, their own created transcript. Do they use that to train? Because again, we don't want confidential things in here if we can avoid it, but you're talking to clients. Clients are not going to say, Well, I need to really talk to you about my issues, but I can't say it because I'm on this AI agenda. Like, it's going to happen. So you want to be really sure it's in a secure environment. And then, there is going to be, you know, a summary transcript or whatever, and you need to review that. Ideally, someone that's in that meeting, because I know even from personal experience, Alice, that I've done things and I'm like, that's not what I said! Sometimes it's a miss you know, it doesn't know whether Alice was at it or not. Genetically doing It's treatment. It's not thinking. It's a bizarre word, they're not going to catch it. Or if it's, you know Right. I was raised in Texas. Instead of saying pen, I say pen. Yeah. Like, you can write with. And, like, those sorts of things, it's not necessarily picking up on. So it is it's like a couple of things like jargon too. It doesn't always know what we need. Right. And if it's Mhmm. It's like if this is turned on for you and you wanna use it, wonderful. Let's get someone in to review the transcript and meeting notes and be real about it and not send off something that looks bad. 100%. But if you're sitting here thinking, oh my gosh. Are we sending meeting transcripts around? You need to talk to whoever Because manages it's really, really important. It's really important. And it's not just that's true for written communication, which is an important part of two zero four two. But we also have things. It may not be transmitted to a client, it may be transmitted to you from the vendor. You don't know where it lives, so to speak. But if you are going to put it, you know, be able to access it later and use that as the basis, say the client is talking about, you know, updates to their personal situations, their risks, objectives, you know, all of these things that you're then going to be using to create, you know, advice, portfolio changes, something. Again, if this is wrong, you're transcripting, you're not going back and looking at this. And I have had it say the complete opposite thing, or emphasize one piece, you know, that you said five, you know, spent a minute talking about in an hour or something else, the bias is there. If you then use that as a basis for recommendations, it's passed off to other people, or people look back at those notes, or God forbid, you have a regulatory or litigation request. So then you're going to pull this up later and it says, oh, well, your own meeting, it said this.' You're like, we wouldn't say that, I didn't say it.' Well, saved it. You had the chance to review it. You should have. Because it's discoverable, it's going to come back up, and then it might magnify all the problems when you keep using it to provide additional advice and guidance in different ways. So it's tremendously important that you review and yes, if it's being distributed, you know, that's a whole different concern. Absolutely. And we see a lot of errors there. We see them mixing up things. Client A with accounts belonging to Client B. There's all sorts of problems. The technology will get better. I believe that, Allison, but for now, it's us. We are on the line. It is our duty. You cannot blame a vendor. You cannot blame know, it told me to do it. It's you. It's a liability thing. I can talk to you about this for about five minutes. I know. I know. I want to be mindful of time, and I want to touch on just two more questions I think that we haven't already covered. This one, you've kind of touched on, because you brought up Red SP, brought up cybersecurity. Anything else that you want to touch on with AI as it relates to these kind of business continuity, cyber, kind of technical things about the compliance program? It's similar to what we should already be doing. It just puts a new spin on it because of AI vendors. So not only our vendor management issues, which we've talked about, you're looking at different things because it's AI. But with our business continuity planning, if you're relying on this, if it's a crucial part of your business, what happens that should be in your business continuity plan, it should be probably in your risk assessment. What happens if something goes wrong? What happens if they go out of business? What happens if they are subject to breach? You know, they're hacked? You need to be mindful of the more you rely on any tool, the more you could be at risk in terms of continuing to provide the services that you're contracted to provide for your clients, in addition to inconveniencing yourself and losing some of that efficiency that we're using AI for in the first place. So make sure, if you're relying on tools, yes, the contract should specify certain things, you should review that, you should kick the tires. You should be all over that from a due diligence perspective. But you really need to think worst case scenario, the more you rely on them, what are you going to do? If you're letting them store your books and records, what are you going to do if they go out of business? Do you know? Does your contract cover that? Because if you can't access those, it's the same as not having any records. So, you know, I think it's not, in principle, different from what we're supposed to do already. It just puts a new spin because we're going to have probably a lot more records, maybe more vendors, so it just magnifies that concern. And we do know that the SEC, this is one of the things that they've always been looking at, but in the context of AI, they're really going to dig deep. So I think it's really important that people update that in addition to everything else, understand how they work, how can you create this if you don't understand what the risk is. I don't want to drag this on longer because I know we're out of time, but you know, the whole but I will. But, you know, mentioning the record, it's like, if you if you have records of multiple, that's expensive to get out as well. Right? To To get those records out if if something does change in that relationship. That's another factor that I'm sure everyone's aware of. But, you know, if you're not, like, once your business communications are archived with the vendor, you know, you have to hold on to those records for five years, I believe, or five, I think. Right. And it's not just those records. Remember, we talked about 02/00/1942, those things specified there. Anything in your policies and procedures, it's a lot. Research, the underpinnings for any recommendation made or proposed to be made, advice made or proposed to make. So that is a lot. So you're theoretically talking similar to what we do with email now. You know, we don't I've seen maybe one firm in my whole career that went through manually and decided, oh, I need to keep the email. No, don't. It doesn't meet the requirement. That's inefficient. It's ridiculous for firms of any size. So what do we do? We archive everything, right? Because the risk of not having it when we need it, you know, we'll get those off channel communications cases in the last couple of years. The violations were because they didn't have books and records, they didn't archive. Now we have many tools potentially, and you need to know what they're using and what that output is because if you need to look back and say, this is why we made this recommendation, this is why we advised this, if that's the reason, you're going to have to archive that somehow. And if you have a million tools, it's going to be harder. So the more you can get and again, a reliable firm that is going to be there tomorrow, is not going to go out of business, it understands the concept, right? It understands why this is important, why you need to keep these records, which is one of the things I think that vendors in our space, rather than just general providers, really are more in tune with. I would agree with you wholeheartedly on that. And I feel like we could have three parts of this webinar. I'm really excited to love that. Check out to it. Don't stop us from having a good time. But I cannot wait to check-in with the chat. I'm so excited to see the answers. Yeah. Thank you for everyone participating in it. I'm I would be shocked if there aren't a bunch of questions, Amber, for me to to sync with you after this. But, overall, just thank you for your time. Thank you for everyone for being here, our clients, our prospects, you know, everyone in the industry that's really interested in this. I think it's just a hot topic and a very big nut to crack with regards to things. So hopefully, you leave here equal parts more informed and ready to make some changes within your business. And we're around if you have questions. I've put my email address up there. We'll we're gonna follow-up with the recording of today's session. If anyone who has a question, our team will get back to you. Give us a little bit of time. You know? We are regulatory experts, so we like to give you the best answer possible and often that means, pinging someone that is an expert in that area and so we'll get back to you shortly. Amber thank you and thank you everyone and have a great rest of your day.