Video: Identity First Recovery: A Clean-Room Approach to AD Resilience | Duration: 3008s | Summary: Identity First Recovery: A Clean-Room Approach to AD Resilience | Chapters: Introduction and Overview (145.615s), Forest Recovery Setup (238.72s), Isolated Recovery Environment (884.03s), Forest Recovery Process (1255.32s), Best Practices & Implementation (2213.94s), Q&A and Resources (2517.48s), Certificate Authority Support (2581.87s), Managing IRE Access (2648.375s), Q&A Session (2745.5s), Azure Recovery Timeline (2915.435s), Closing Remarks (2953.625s)
Transcript for "Identity First Recovery: A Clean-Room Approach to AD Resilience":
Okay. Hey, folks. Hope everyone's doing well today. My name is Mike Schmidt. I'm one of the identity architects here at Rubrik. And today, we're gonna be talking about identity first recovery, essentially a clean room approach to, AD resilience and forced recovery. This is a a large release from Rubrik focused on making the forest recovery process even simpler, making recovery hosts, very simple to deploy and configure for, the various DCs that we're recovering. And so today, I'm gonna talk through some of the details there, things we typically look out for when doing a forest recovery from a planning and preparation standpoint. And then I'm gonna go through a quick run through in the UI, in Rubrik Security Cloud to show what this looks like, and then just wrap up with some guidance around, you know, working with a lot of different customers, things that we see, and things that we would suggest, to make this process simpler. And as a follow-up, we have lots of different, runbooks, documentation, Oasis labs, which are our sandbox labs where you can actually test this out. So, feel free to ask any questions in the chat or save until the end, and, thank you for joining today. So as we jump into this, just a quick safe harbor statement related to, features and whatnot. You know, everything I'm talking about today is available. The main thing is just making sure that the Rubrik clusters that you own are at a certain, code version as a minimum, which at the moment is nine five three, for those that, are, you know, familiar and know the cluster versions offhand. But as we jump into this, we're really gonna be just talking about the high level features, what the benefits are. And so when we think about forest recovery, this is a big topic that is a focus for a lot of folks. And the issue is that this is a very laborious and time intensive process, on its own. You know, if anyone on this call here has gone through this and, you know, it could have been ten years ago, it could have been last week, the Microsoft's guidance here hasn't really changed in a in a long time. You have to... Essentially, if if you as a threat or some persistent actor in the AD environment and you can't clear them out through your traditional means or quarantine or whatnot, you know, the next option is to do a full forest recovery, which essentially a rebuild of this active directory forest from a previous time stamp. Now what Rubrik adds to the mix is we have things like threat intelligence to understand that, hey. We need to restore back from four days ago because the attacker was introduced, two days prior instead of, kind of throwing darts at the wall, figuring out what version we need to run back to. And then, we also facilitate a much more hands off expedient recovery where we can, especially with this methodology, this identity first recovery, we typically see, full recoveries within an hour, time frame, give or take. And there... And there's some variability based on the size of the, AD database, but it's just a very condensed automated, recovery methodology that Rubik provides. Now in terms of what you really need for this type of testing, it really breaks down to three items. The first is that, for folks that are familiar with Rubrik and, our existing customers of ours, You know, active directory is a workload just like anything else. Your... Like your on prem, your cloud, your SaaS, workloads. And so we would, at a minimum, need a subset of DCs per domain being protected and, backed up using our Rubrik backup service. At a minimum, we need one domain controller per domain, because, essentially, we're doing an authoritative restore. But we always suggest having two or more backups per domain just for the sake of resiliency. And so that backup is your foundational layer. What we do is create a system state image using the Windows backup utility, and that's protected on the Rubrik clusters as an immutable data store. And that becomes the basis of our recovery data for when we need to trigger a forced recovery. The second piece that's really important is the actual recovery environment itself. Now the terminology that's very common is the isolated recovery environment or an IRE. This could be known as a clean room, could be known as a bubble network. You know, it has a lot of different names and connotations. Really, what this means in this scenario is a isolated segment of the network in which a Rubrik cluster can communicate with over, simple SMB for data transfer. But the big focus here is that when we recover these domain controllers, we want no interaction between these recovered domain controllers and production, especially from a lab and testing scenario. Now, obviously, if this is a production use case and your active directory is actually attacked, that same isolated environment, can be used, and then we just simply open up the networking side. And now it's your new production, use case and environment. So from a... Most times, our customers are using this, ADFR process to repeatedly test this every three months, six months, twelve months, update documentation. So they'll use these IREs to, create the recovery hosts, do the ADFR, and then blow them away. And you can do all of that without any impact to production, and I'll showcase what that looks like here on the next, page. And then finally, the third thing is the actual recovery host themselves. So, at a minimum, we need a single recovery host per domain being restored. So if your active directory forest has five domains, we would need five recovery hosts. If it was a single domain, single recovery host. Now the good thing with this process is that these recovery hosts are essentially just vanilla brand new, Windows Server hosts. The really... The only stipulation is that they match the same major version. So if your source data is, you know, Windows Server 2016, then your recovery host would want a a blank Windows Server '20 16 box. There's some minor, you know, software items we need, like the Visual c plus redistributable. We need the Rubrik backup service installed, and it needs to be registered to Rubrik Security Cloud. But that can be fully templated, and, so we could deploy these recovery hosts, whether manually or via Rubik's Orchestrate Recovery in a matter of minutes. So that prep step can be, very straightforward and very simple. And we're essentially putting these recovery hosts in the IRE that we built out, and then we're triggering the ADFR process to restore those domain controllers in a safe isolated environment. Working with a lot of customers, step one may be the most time intensive just to make sure you have, the Rubrik cluster set up, the DCs protected, all that looking clean. You know, maybe you're doing replication between clusters. The isolate environment may require some coordination between your data center team, your virtualization team, whether it's, you know, any of the major hypervisors that we would support, your networking folks potentially if we have to, make firewall changes or whatnot. But, usually, that's very straightforward, and we have all that documented. And then the recovery host would just be, you know, your server admins and those that own those, instances, so we can create those for testing. Now what this looks like from a visual standpoint is, pretty straightforward. And for those that know Rubrik, you know, a very similar construct compared to any other workload that we typically provide. And for those that are maybe looking at this for the first time, just to highlight some of the components here, at the top, you have the Rubrik Security Cloud, what we would call RSC. This is a Rubrik hosted management plane. So you would actually log in to, you know, customername.myrubrik.com. You would sign in either with local credentials or single sign on, and that's basically your one stop shop for all things related to rubric management. From the actual control and data plane standpoint, that's where the clusters come in. And these come in many different flavors, virtual, physical, on prem, cloud, colocation. The good thing is that the code that's actually running on these clusters is identical. The only difference is the under arcing architecture. You know, an on prem virtualized appliance is gonna be a bit different than an Azure native appliance compared to a physical, brick that's in one of your racks in a data center. But the actual code base is exactly the same. They run the same code levels. So it's very easy to mix and match different environments where we have customers that may run a multi domain environment where DCs are spread across on premises cloud, around the world, and we can still support that model because RSC essentially is the orchestrator. That would help kick off something like forest recovery. That would instruct the local clusters, to restore data, and then that would, communicate with these domain controllers to recover. On the left side here, we have, call it your primary location production. We've got multiple data... Excuse me, domain controllers running the, Rubrik backup service. The actual communication is strictly between the cluster and those domain controllers. So none of that data actually goes up to Rubrik Security Cloud. It's only within that location. So this is where we can do some, you know, air gapping or, other items where we can keep things isolated to that site. Really, the only ports we need open are SMB and the RBS communication ports. Now the cluster itself could be on the same segment or subnet as these domain controllers, but it... There's no issue with having it in another, subnet or VNet as long as we have the correct ports opened up through the firewall, that would be present there. And then once you have those backups in the primary location, those can be replicated to a a second location, a Doctor site. And, typically, that's where we see the isolated environment, which would be, you know, highlighted there in the bottom, which is, where our recovery hosts are gonna be. So the main thing to keep in mind is depending on what you're looking to actually test and, trying to accomplish with this type of model, we typically want these tests to be as realistic as possible. So the idea is that if your primary location was to go down or it was breached or attacked, that data should be replicated already to a second location. We should have already, done tests to recover these, domain controllers in an isolated network. And so if and when we need this in production, it's the same exact process, and we should be able to get up and running as quick as possible. Now that is is the ultimate goal here is, if and when there is this type of cyberattack, which we see is a very prevalent attack vector today. We wanna attempt to get back up to production as quick as possible with active directory being that tier zero application. So it's... You know, that's ultimately the main focus here. Now this isolated environment from a Rubik lens, you know, these are mainly domain controllers we're focused on at least in this conversation. But you could always bring in your third party DNS servers, any other services, computer hosts, and other items you'd wanna test as well. Because when we do this recovery, these are essentially clones of your production domain controllers. So you would be able to authenticate to them. You can log in with your AD credentials even though it's still kind of in an isolated network and, be able to test out any scenario that you'd like. Now a few things we highlight on the right there is the main priorities of these clean room environments or these isolated environments is we want zero risk to production. So a lot of that's focused through, firewall ruling or not allowing the... This network to have a default gateway. Essentially just no routable path out that would be able to communicate with production. What I'll show in a minute here is that within the UI, we actually have a compatibility check where we're just making sure the, source DCs and production... Or excuse me, the recovery DCs are compatible with each other, but we also make sure that, there's no traffic being allowed between them. So we actually probe between the production servers and the isolated recovery host to make sure that, you know, IP connectivity isn't allowed. Different ports like LDAP and whatnot for peering are not allowed. And if it is, we'll call it out and say, hey. Your isolated environment is not as isolated as you thought it was. Some other things to think about are DNS and network time protocol, NTP, are are very important here. We will need some sort of DNS in the isolated environment. Most times, we would just... If there's a third party involved, for DNS, you know, it's not AD hosted, we can just simply configure the local DC for DNS, and it's gonna work fine. Other times, customers will clone in their, third party DNS server so that we can resolve to it for the sake of testing. But the the topology here is meant to be very straightforward. We've got two sites. This could be done on one site off one cluster if we have some restrictions there or just a smaller environment. The primary location could be on premises. The Doctor location could be in the cloud. All that we really need is, a cluster that holds the active directory information and a recovery host to communicate with. Outside of that, the topology is extremely flexible. And since we're using, Windows system state snapshots instead of, think of, like, a virtualized disk backup, that makes it really portable as well. I'd say we compare that type of model between, like, virtualization and containers. You know, the benefit of a container compared to virtualized environments is that a container is extremely portable that you can just take that same package and use that application literally anywhere. It's the same idea here. It's... Even though you're taking an on prem, physical DC backup, that data can still be used to restore a a domain controller in Azure. So just moving on here. And the only other thing I'd show is just when we have the the cloud environments, it just becomes another leg of the topology, fully supported there, as scalable as you need it to be. Now one other thing to cover is when we start thinking about these recovery hosts that we have to deploy, typically, that would just be a manual process where, you know, someone on your cloud team or your virtualization team would turn up a template of a Windows Server, do some quick prep on it, and have it ready to go. But we do have this feature also available in our orchestrated recovery feature set. Today, that would be supported with VMware, soon to be Microsoft Azure. And as we expand out more workloads, we'll be able to support more target, sources. But the idea here is that we combine both the forced recovery process and the host creation process, and then we essentially preplan this in the orchestration or orchestrate recovery tool to the point that you can just click recover, and this now becomes your one click, recovery test, which is, extremely powerful, and it could even be used in production. You know, it's the exact same process. Again, the only difference is that once these are up and running, we change up the networking a little bit, and and now you've got a production forest. That... That's, you know, publicly accessible. So let me jump real quick into the, the lab environment. So what I'm gonna do here is I'm in, Rubrik Security Cloud, and this is one of our lab environments. And within the different modules here, we have, identity resilience, and this is gonna be where you find all things identity. Now within the different in... Inventory here, because we support, you know, Entre ID, Active Directory, and Okta, we're looking at Active Directory depending on how many forests you have in your environment. In this case, in our lab, we have... What's that? Six. You could have one with a single domain. You could have one forest with 20 domains. We've seen it all. We have no limit on the mix and match there. So from here, what I'll do is kinda showcase what this process looks like. And the assumption here is that we already have domain controllers in this specific tenant and forest, already backed up. So what I'm gonna do is click this radio button here and click recover forest. And so, we have a few different options here. Forest recovery is typically what we're doing. Now clean recovery utilizes some additional integrations from, Workday and SailPoint to look a bit deeper at things like activity logs of different users to help further confirm the the correct point of recovery. And then this infrastructure recovery is what I was mentioning before where we can actually build out the infrastructure, the recovery host alongside the forced recovery itself. In this environment, we'll just do the forced recovery option. It's gonna give you a page where you can look over the prereqs and everything else. We have a lot of documentation that's published, but we'll go through this. And the first thing you're gonna see is when do you wanna recover. So it's either a specific point in time that you're aware of. You have a very explicit, you know, something happened two days ago at 11:00, and we need to use snapshots before that. The other side of it is using the latest known good snapshot where we essentially automatically, provide the best one for the use case. The next thing... Screen you're gonna see here is on a per domain basis. As you could imagine, we need to essentially pick, hey. What's the source domain controller we wanna pull data from? What method do we wanna utilize for recovery? And what is our target host to publish this data to? So if I click on the, configure tab here, you'll see two options, and I'll cover these quickly as well. By default, we really only need to restore one DC per domain. You know, your your primary DC, that is enough to essentially bring up the skeleton of your forest, and then you could worry about downstream secondary DCs later. We also do provide the mechanism to promote downstream hosts either via replication, which is, you know, just saving you a few clicks, Or we also utilize install for media, which, essentially live mounts a dataset, creates a package kinda like a zip file, and sends that out to the downstreams, which accelerates, convergence times greatly compared to typical replication. So we do have those options here. For now, I'm just gonna do the single, PDC restore and, you know, just going left to right here. The first thing we're gonna do is, pick... don't mean to interrupt, but, folks are not seeing your your demo screen. Just wanted to, call that out for you. Okay. Seeing. is that... a second. Yeah. They're just seeing that the the slide. Well, didn't mean to do that. So let me let me go back, two pages there. You can see it okay now? Yes. Okay. Thanks, Mike. Sorry about that. Well, let me step back a few then. I I was talking about it, but let me, just reshare this again real quick. So we've got our identity resilience page here, our different forests and domains. I'm clicking the radio button here by the one we would like to recover. I go to recover forest, and then we'll go to, forest recovery. These are our different options here. We'll go to start recovery, and then, the timeline here we'll pick is our, in this case, our latest snapshot, but we could pick a point in time. We now see our specific domain that we're recovering in the forest. We also know exactly what snapshot we're pulling as well as some default filled in attributes, but we're gonna configure this explicitly. And, as I mentioned before, just since folks can see it, we can do downstream host promotion via replication install for media. In this case, we're just gonna do the single, primary DC restoration. We're at this, selection screen here where the first thing we're gonna do is be able to select which source DC we wanna pull data from. Now in a domain, typically, every DC has an exact copy of the data. So if I picked DC two or DC one here, we would get to the same point because we're gonna re seize the re seize the, FSMO roles and rebuild the roles and everything else. I personally usually select just the schema master, just to make sure we have the absolute latest. So I'll click save here. So we're going through, and we're gonna restore the various, FSMA roles. It's a DNS server, so that's gonna be rebuilt and restored as well. We're gonna pick which cluster we wanna restore the data from. And so this is where we get into things like disaster recovery planning. So, obviously, we're gonna have a source cluster. Now this data was replicated to a secondary cluster as is, you know, best practice. We would see that cluster here, and and we're basically telling the ADFR process, you know, what dataset do we wanna use, and what cluster do we wanna use it from. Now on the recovery method side, we're using identity first recovery. System state and bare metal are other methodologies that have, you know, pros and cons. Identity first recovery is definitely our go to method, at the moment here. And so we're gonna, click back from here. And then the next thing is we're gonna select a recovery host. And so this is gonna run a quick compatibility check, to make sure that... You know, there's a few things that need to match in this scenario, but, generally, the major version, just making sure the recovery host has any other minor prereqs we need, you know, small software packages, RBS installed, etcetera. So it's gonna go through and do compatibility, and and we're saying it's good. The other thing I mentioned is we're doing a... Essentially, a network leak test between the source DC and the recovery host, to make sure that, you know, typical AD protocols aren't allowed between them so that this recovery host won't try to peer with, production, especially in a a test scenario. So we're good there. One other thing to note is when we restore this data, you know, think about it as we're essentially cloning this host more or less. So things like IP addressing, some of the DNS settings would get carried over. What we're asking for here is, essentially, do you wanna keep the IP address of the target host? Because this is typically gonna be a different network, or do you wanna use the source DC IP addresses? We always default to target host because that makes the most sense, you know, 99 out of a 100 times. But if for whatever reason you wanna use the source DC IPs, you can do that as well. This item here, the, cleanup metadata step is really important. So this is a bit of an overlooked step during typical manual forced recoveries, but it's usually the most time consuming. Think about a scenario if you have, you know, 20 domain controllers in a domain. One of the domains gets tainted, and that gets replicated to all the DCs. So the whole, you know, domain and essentially the forest is tainted and we're driving this forest recovery. What this step does is when we restore the data, you know, all the references to the old tainted DCs are still in that database. What we do is actually go through and clean all those references up. So, you know, peering relationships, DNS entries, anything else that would reference these old, domain controllers, because we don't essentially wanna know about them. We we assume everything has basically been, decommissioned at that point. The only reason you'd leave it intact is, you know, some customers when maybe there's a breach or maybe they're doing testing, they wanna be able to send a clean copy of this into, like, a security enclave so that the teams can troubleshoot it and understand what happened. That's really the only time you'd have it here, but other times, you'd almost always wanna do this cleanup step. It's also gonna ask you to re... Type in a new DSRM password just for the sake of security, because what we're doing is actually taking that recovery host, which is a a simple Windows server. We're gonna install the domain services role on it. We're gonna promote it to a domain controller, and that will require a a DSRM password, which is essentially the safe mode for a domain controller. And, you know, we'll create that as we're going through this process. So now we're showing that everything's configured for this domain. The one last step here is to determine what we wanna do around DNS and NTP. Now depending on this is something that we would vet out during planning. The biggest question we have to ask is, do you have DNS configured on the domain controllers themselves, or are you using a third party DNS server? And then in the recovery environment, we will need some sort of DNS to resolve to to do the promotion, action. So then it just becomes a question of, you know, maybe if I have a copy of the domain... Or excuse me, the DNS in the isolated network, I would put the IP addresses here. Otherwise, if we just wanna run DNS off the domain controller, this would configure it for us automatically, or you could leave it intact if if it makes sense for the situation. But, you know, DNS is really important in this conversation. The other point here is NTP, where a lot of times, there's an external NTP server that we're looking to point to. Since these environments are isolated, you know, maybe I just wanna point it to myself as a DC just so we can get through quickly without it having tried to ping out and be unsuccessful. Again, just depends on the scenario and and what's available and not available. These last two here, similar to the metadata cleanup step, almost always would leave on unless you're doing a a security review, where we can rebuild the global catalog server, from the backup, as well as this being very important where we reset the Kerberos, ticket generating... Ticket password. We actually do this twice to eliminate, golden, ticket attacks, which is very prevalent so that someone who had compromised your previous forest wouldn't be able to access your new forest, because the ticketing had been reset. Finally, we just get to a review page where we just see the configurations, and then you click go, and, we go recover it. And so for the sake of time, I'll jump to a successful recovery. What it's gonna do is when you click recover, you're gonna get this, tracker. You're gonna get an exact timestamp of, excuse me, start and end times. You're gonna see every step along the way where, you know, in this case, we start at 10:21 wrapped up at effectively 10:30. So call it fourteen, fifteen minutes for the the core recovery, and then we perform a set of health checks to make sure that everything looks good, replication looks good, the domain controllers are accessible, and this would be for both domains. So, you know, this one, 10:21 to effectively 10:34 for the root domain. And then, for secondary here, similarly, you know, 10:21 to 10:41, just as it waits for certain things for the root domain to complete. But all in all, you would, you know, see all these steps. It's completely hands off, so we're doing that all for you. And, this recovery took a little less than thirty minutes. Now, obviously, this is a lab environment, but, you know, anecdotally, working with customers large and small, the, the delta here is not heavy at all. You know, I've... Most times, I see it somewhere in the hour to... You know, forty five minutes to an hour is very typical. Again, if if you have a a 50 gigabyte, NTDS, directory compared to something that's, 600 gigabyte, you know, there's gonna be a delta there just with data transfer and whatnot. But, generally, that's gonna be the the most time intensive step, and then you'll see... You know, we run through the rest of these items in less than five minutes. And and just to hit on those for a second here, you know, we store the data from the backup. We, then it's an authoritative restore, so we mark the SysPhalls authoritative. We resease the FSMO roles. Then we go through that metadata cleanup step, reset the passwords, rebuild the catalogs, re enable replication, because this new DC is gonna be communicating with this new DC. But they... Because we did the cleanup step, all the old DCs that, you know, should be disconnected or decommissioned, we won't have to worry about. So that's ultimate, you know, power of what we're doing here. And the good thing is because we created these... The clean rooms, the isolated environments, this is a repeatable test. You know, we can create a host, you know, say we're we're using a hypervisor. We can take a snapshot of that host, perform this test, make sure everything looks good. You could revert that host back to what you, previously, you know, had before executing this, run it again, tweak it. And, a big benefit there is you can very easily update your disaster recovery plans, your documentation, vet out everything here where, you know, you'll start small and then build it up. So let me show just one more thing here, and I'll make sure I'm sharing. So just some some final closing thoughts as we as we wrap up here. A few things just to call out for best practices when we go through something like a forced recovery and identity recovery is the first one here is very important, and this is gonna be true for any type of identity recovery type solution. Obviously, if AD or Entre Okta is down, your single sign on is down as well. So it's very, very important that we create local non SSO accounts. You can do that on the RSC UI, as well as, you know, the other tool sets so that you can access what you need to access in these times of need. We also have quorum authorization, which is essentially a multi user approval for changes to snapshots, backups, SLA changes, which is our, you know, backup and retention intervals. So that's very important. From an, like an RPO and RTO and SLA perspective, Entre ID, we could support as hourly backups, or higher. Active directory is four hours. Most customers of ours are somewhere between eight and twelve hours, so maybe two, maybe three backups a day. You could do four hours as well, and, we're we're just taking diffs every time that we, we hit that interval. So it's it's low intensity, especially after that first full backup, but that just gives you a good idea of, you know, if you have to go back four hours or eight hours to the last backup for these recoveries. That's essentially your your recovery point objective and and trying to match that to your plans. The other last two here is within the Rubik UI, we have an extensive set of role based access control. Every single click in the UI is an API call. So we can... For example, if you have an identity team and a security team and a, you know, Rubrik admin team, you could say that your identity folks only can do, you know, granular recovery of objects. And maybe when you need to do something in a forced recovery, you could use a a privileged access management, SSO group that would escalate to be able to actually trigger a forced recovery process. Or, you know, your AD admins couldn't access Entre ID settings and vice versa. So a lot of granularity there, and we highly encourage, using role based access as much as possible. And then just something to note that we have built in features like intelligent data lock so that, you know, even if backups are logically deleted within the UI, there's gonna be a, kind of like a recycle bin function there, just in case there's a fat fingers issue or someone deleted something by accident. We've got your back there. The last thing I'll say to to wrap up is, you know, I've worked with a ton of customers around forced recovery, building isolated environments, and these go from, you know, smaller enterprises to multinational global organizations. But the key factor is that you would need to think about this as a phased approach. It's okay to start small. An isolated environment, a clean room really is as simple as just a segmented part of the network. You know, it doesn't have to be a duplicate data center. It doesn't have to be a whole facility. It can be as simple as a VLAN or a separate subnet in your network, whether that's in the cloud, on prem, etcetera. So, you know, start small. Just make sure that the, technology fits your dataset and use case, that it restores like you need to. And then we build up from there where we can add more and more services, users, objects that more mimic a realistic scenario. You know, we call this a phased approach, crawl, rock, run. You know, we wanna make sure that the foundational layer is built before we add more and more. And, what that gets us to is starting to think about the minimum viable business of how do we... When we, experience an attack or a cyber event, what's the most important thing to come back up first? What sites are the most important to come back up? How quickly do we need to restore business? And then we build the ADFR plan and the rubric footprint around it to make sure we support your RTO and RPO objectives. So that's, that's the extent of one that I wanted to cover today. Really appreciate the time. In terms of resources that we have, we have plenty of technical white papers, videos, guides on our support portal. The lab I was showing is also, something that you can schedule with your, Rubrik counterparts, which is a a free to use lab to be actual... Or to be able to actually go out and execute in a forest recovery test and see it work in a live environment where we have, you know, eight or nine domain controllers running, and you can see exactly what's going on there, before maybe testing in your own environment. So feel free to reach out and let us know if you have any questions, but thank you very much. Take care. And so, Ed, I was gonna see is... Were there any questions or anything that I can answer? And, again, I apologize for the screen share component. I... When I was sharing, I could not see the chat, and so, sorry Yeah, Mike. Thanks for the great presentation. We do have four. questions in the q and a section to the right. Does the recovery support CAs too or just DCs? Certificate authorities. So if... Currently, certificate authorities are supported in our bare metal and system state recovery methods because those are more disk level or image level recoveries. At the moment, as of today, Identity First Recovery as a method does not support certificate authorities, but that is a feature being added, within the next few weeks here. So we're adding additional roles and services that get restored in that method, and, certificate authorities are gonna be part of that. So it's... That one's right around the corner for full support there. Awesome. Sergio asks, what recommendations do you have to manage things in the IRE? Yep. So that... It's a good question. It depends on where the IRE lives. So if it's on premises, tends to be a little bit simpler because you typically have, some sort of local... You know, you could patch a cable into that network so you have, like, a local laptop sitting there. If you're using a hypervisor, you can easily console into those hosts just through the hypervisor console. But an interesting thing that's come up more recently is if those IREs are in a cloud provider, like Azure or AWS, it actually becomes trickier because you have to think about how can you actually access that DC in an isolated network. So what we typically, recommend is either you can poke a hole through the, security groups in the cloud to allow something like, you know, basic RDP, especially for test environments where everything's internal. The other side of it is configuring a a bastion host or a jump box that you can access via, you know, VPN or RDP that gives you a, you know, not direct access to the DC, but a host that's sitting in the IRE that then you can jump to the domain controller itself. So I think that's... It's just a matter of access, you know, making sure that you can reach the domain controller in some form or fashion, when it's in that isolated network. Alright. Siva asks, for selective time recovery, how how do you handle deltas, like password or GPO changes or new accounts? Selective time recovery. I'll answer the question that's below it, I think, from Steve as well. Yes. It keeps the host name. So when you recover, you know, DC one in production to one of the recovery hosts, it's gonna mimic it as close as possible. So it's changing the host name and other settings so that if you were to release that to the public network as your choice, it's gonna basically look like that DC. The only difference is the, the IP addressing under the hood is different. For selective time recovery, I'm gonna answer what I I believe you're asking. Maybe I'll hit the mark, maybe not. But if we're restoring three days back, we're using three day old data. So... But we're copying the whole database over. So as long as it was already in the database at that time, passwords, GPO changes, etcetera are in there. Now where I think you're going with it and where maybe I would talk about is that delta between three days ago and today, obviously, is important, you know, especially if it's not three days, it's two weeks, or something longer. So we do have a feature called roll forward, that is available where, as part of our identity resilience feature set, which we didn't cover too much today, but we're also looking at activity logs in AD and Entre and Okta where we can essentially say, okay. You restored two weeks ago, say, you know, Halloween night just because it's recent. Since we already have the activity logs from Halloween night to today, November 12, we can actually replay those, obviously, without the attackers' actions in there, and and then, you know, that's its own conversation. But we can replay all those new accounts, GPO changes, etcetera, up to present so that you don't have to manually go through those and and redo them all. So so that is a feature we have today called roll forward for that specific use case. And, you know, definitely feel free to reach out to your Rubrik counterparts for more details there, and it's also on the the Rubrik website that you can check out. Great. And lastly, Chris asks how soon for Azure Orchestrated Recovery? Yep. So that... It's a big focus for folks like myself and the product team. You know, within the realm of safe harbor, I would say that we are looking to, release that as soon as possible end of year q one. I... Again, I would say reach out to your Rubrik counterparts on a more specific ETA, but I do know it's a very high focus at the moment, because we have so many customers that run-in Azure, and to be able to create Azure recovery hosts as part of, forced recovery planning would be a huge asset. So, you know, it's it's a high priority for the team. I would say it's coming sooner than later. You know, I would love to see it before the end of the year. It may creep into early next year, but it's, it's coming around shortly. Fantastic. That's all the questions we have, Mike. Wanna thank you so much for a great presentation, and thanks to everyone who, submitted questions. And just for the overall, interest in in the product. We seem to have a lot of good interest in all of this. So, as Mike said, we encourage everyone to reach out to your Rubik folks, and get a demo, a further demo of of what we have to offer here. Do you like what you saw here then? Take a deeper dive. Thanks so much, Mike, and, until the next one. Yes? Yep. Thank you. Take care. Appreciate it.