RLC-H & Ascender Pro: How to Deploy Consistent Secure Linux With Ease

During today's session, we'll discuss how to eliminate this false choice with an integrated approach that addresses both the foundation layer, which is your operating system or your kernel, and the operations layer, which is deployment and maintenance simultaneously. So, let me ask you something. How many of you have been caught in the middle of this fight? DevOps is pushing for faster deployments and security is demanding that nothing moves until it's fully hardened. So, that's the impossible choice. Deploy fast or deploy secure. Pick one. The bottom line is your business needs both. The good news is you're here because there's a better way that Brian and Jimmy are going to describe.

So, let's get into it. Mr. Brian Dawson, thank you so much for being here and for your um very wonderful song that defines a lot about your personality, which I tend to love. You're one of my favorites. Um you're our RLCH product manager with decades of DevOps experience. go ahead and introduce yourself. >> Cool. Um, I'll start with my background and I'll talk a bit about the role. So, Brian Dawson, um, started my career actually, uh, interestingly enough, uh, building developer tools and supporting developers on, uh, the PlayStation. That was where I got my introduction to Linux. If people aren't aware, uh, uh, at a later stage of the PlayStation, we delivered PS4 Linux.

Um it was always built on GCC tools for open source. That led to a career in open source which included uh a number of years in the CI/CD devops and infrastructure as code space as that was emerging. Um I luckily have now landed here at CIQ uh on the Linux product team where we are working to deliver workload optimized versions of enterprise Linux. So, enterprise Linux with um with with specialization to support your unique workloads. >> Awesome. Thank you so much for that. Jimmy, I've learned so much from you listening to the way you describe um Ascender Pro for some of our customers. It's it's incredible how much knowledge that you have inside this space and you're definitely our lead developer and Ansible expert.

It's it's quite clear you demonstrate that in almost every conversation I have with you. Thank you so much for being here. Can you please go ahead and introduce yourself? >> Yeah, so I'm Jimmy Connor. I started off in IT over 25 years ago as a Windows and Linux system admin. You know, down in the trenches doing the dirty work. Uh worked my way up eventually to working as a contractor hardening government uh systems, you know, doing security harden on them. Moving over to the consultant side and then jumped over into Ansible. I've been doing anible itself for over nine years now. Uh but I've been doing automation all the way back from beginning because again, you know, when you're doing a system in midwork, you don't want to do the same things over and over.

You don't want to do all the the little, you know, tedious things. You know, you automate that. Uh so I've learned automation all the way back from Puppet to Chef all the way up to an at this point. And I came over here at CIQ to help lead actually building the Ascender product, getting it up and running to basically share that with the world. >> That's so great. Yeah. I I can't wait to hear what you share with this audience with regards to Ascender Pro. It's it's really incredible stuff that you've done with your team. So, let's go ahead and get into the challenge. What are some of the teams stuck in?

So, starting with you, Brian, considering your DevOps and Linux background, which is extensive, what's the fundamental problem that you've seen? >> Well, um yeah, how much time do I have? Well, now I'll boil it down to the fundamental problem. As I said, I started in the early days of CIC dev CIC DevOps after you know years of of of uh you know actually really at a time when even centralized and shared repos were a reasonably new concept and the goal at that time was to make software development less errorprone, less manual, repeatable and and then ultimately faster. Right? So this idea with continuous integration, continuous delivery kind of a predecessor or component of DevOps is that we need to be able to deliver at speed.

The faster we could build, deploy and get software and infrastructure out to customers, the the faster we one can deliver value, but we can also learn where we may have issues and iterate again. Right? So now as I come and I look at uh actually delivering a Linux distro I go well okay when we apply DevOps to to you know kind of your core and foundational infrastructure the fundamental problems are the same how do we deliver repeatably reliably fast >> right but then how do we do that in the face of diversity of our customers needs or our infrastructures needs and what I will say

is so that problem is the same but in a sense I think when we talk about OS and infrastructure deployment, especially with harden Linux, um the problem's a little bit more challenging and you know that's why I'm excited to have met Jimmy and kind of work on solving this together. >> Yeah. And you guys have shown that you work incredibly well together. I've seen some of the dialogues that you have just trying to really get into the need of how do we solve this problem on both sides of the table. It's been incredible to watch. Um you make my job very easy actually. Um, and so to you Jimmy, what are you hearing from the teams struggling with this from your side?

And you have so much automation experience. What have you seen long term and what's kind of come up new as well? >> So in regards to like the operations side, the main thing I hear from teams, you know, I talk to them every week is that everything's moving so fast now that so much is being thrown at them that they don't really have time to actually do all the work that they're, you know, told to do. You know, we live in this wondrous world of AI where everything is automagically trademarked uh done for you, but in reality, you know, all your all these teams are just pushed to do more work faster with less resources.

>> You know, >> everything's moving so fast they can barely keep up. And then on top of that, you know, we have now new security requirements coming down, being thrown at them. You have, oh, we have to make every sure everything's secure. We have to patch faster, you know, because the cost of actually being hacked and having all your data leaked or having, you know, exploits, it's expensive. >> Yes. >> Uh so I I read an interesting stat the other day, if I can remember, it was uh on average over a 100 new CVs are published every single day. So when you start thinking about it and then on some single days it's over a thousand.

So I started thinking about my past. You know, back in the day, we only patch every quarter. You know, a quarter basically 90 days we would go without patching. You know, you start thinking about it. 90 100 average CDs. If we were doing that now, 100 CDs a day, that's over 9,000 CVES every quarter. So, you know, how many of those affect my servers? And even scarier since, you know, Halloween's coming up, you know, so close. How many of those affect my internet facing services? So we have to move so much faster now to ensure security and ensure everything's up to date, everything's patched, you know, and this is a new world we live in.

You know, you have to patch your servers, you know, but the problem is you go out there, you patch your servers on Sunday, security sends you a scan that says everything's aokay on Monday. Come Tuesday, new, you know, new zero day. You got to start all over. >> But surprisingly, you know, there's not a lot of people out there, or at least some people that still aren't automating this. You know, they're manually logging into the boxes. I talked to a customer the other day. They have one guy that that's all he does all day. He just logs in and runs DNF update all day long. >> Yeah, Jimmy.

Oh, sorry. Go ahead. Yeah. >> Yeah. And when you, you know, ask him why aren't you automating this? It all comes back to I just don't have enough time. If I stop to write all the automation to do all this, then I'm just that much farther behind over here. So, >> chicken. >> Well said. And and I'd like to say it's the classic, if I had more time, I would have written you a shorter letter. Right. if I if I had time to automate, I would have automated.

Um, but look, I was just having a conversation with an RLCH customer uh the other the other day and it's it's you know what I constantly hear reiterated um we just don't have enough time to do what we need to do and the time required for us to ensure we're secure and stay up to date is hard is is getting um more shorter and more demanding now. Even moreover they were saying as soon as we deploy our OS the app team has already configuration drift instantly starts the app team has already started turning off turning services back on and kind of undoing the hardening we've done and that turns into an extra sort of constant challenge.

Um I will add that in this situation we've talked about DevOps which is this DevOps but downstream in a sense um what I'm seeing is the problem of ops to dev right how does ops build out the infrastructure to support dev at the pace they need it supported while ensuring that it's meeting the security requirements of the business. Isn't it so funny that the common theme here is always if I had more time and with the onset of AI, I mean AI is here. This is no longer the age. It's already here. It's landed. We have to live with it. And for my role, it helps me go faster.

But nobody's thinking about the infrastructure layer. Now your CVE remediation is the velocity is just going crazy. Like it's just increasing tenfold, nanofold. It's going nuts. And you got to remember the other side of the the aisle is actually using it too. They're using AI to actually create exploits. Backter. >> Correct. Right. >> So, let's you guys both went off script. I'm going to pull you back on. Um, so let's get into it. Let's get into kind of like the meat and potatoes at the foundation layer, which is the operating system foundation, the kernel. Brian, what is RLCH and what makes it different from your standard enterprise Linux?

I'm going to start with the second piece first. Key thing is um it is intentionally not very different from your standard enterprise Linux, right? People trust uh enterprise Linux. They build their businesses on top of it. One because it's an open source foundation that the community contributes to. There's transparency, but then two because it prioritizes stability and security. right now. Where does it differ is a generalurpose sort of out ofthe-box enterprise Linux distro um is not uh uh ter is not does not come out of the box compliant. It does not come out of the box tuned for your security needs. So then therefore you have to put in manual effort to configure that for the specific needs of your secure secure workloads.

So now we get into what is RLCH? RLCH is what we call a workload optimized variant of upstream of Rocky Linux which is a trusted enterprise Linux distribution built such that it is preh hardened and pre-mediated for compliance across an increasing set of standards. Let's go backwards and start with the pre-mediated. When you get Rocky Linux from CIQ hardened or what we call RLCH, you can get an image that comes out of the box 95 plus percent pre-mediated for disastic uh with a few things that you got to switch on your side. You hit 100%. You get 99 plus% compliance or pre-remediated for CIS and you get 99% pre-mediated for Nest 800-171.

But moreover, what you also get is harden modules that provide what we call proactive security. Um, so now you're not just checking compliance boxes that used to take you, you know, five hours per server or more, but you're also getting proactive hardening modules that don't exist in any other um, enterprise Linux distribution today. >> Yeah. And talk about saving time, you just authored um, that by line on the difference between manual hardening and preh hardening. And I think that that was it really showed how having to do that manually could potentially slow teams down even more. And if you have this preh hardened solution, it allows you to ensure that things are locked down so you can really focus on some of the other strategic operations that the team really needs to tackle.

So I love that and thank you so much for contributing that. If you haven't seen it, it's on the ci.com website from a couple days ago. Um Jimmy on the operations layer so even with this hardened operating system you need to manage it. Is that where ascender pro comes in and how do they work kind of hand in hand together? >> Yeah. So basically when you have an hardware image it's great but with all security you're going to find that you're going to have exceptions. You're you know not every application is going to work 95% with a 95% step compliant server. But with ROCH, it's great because you're kind of flipping how everything works and you're reducing the amount of automation you actually need to run on the servers to deploy them.

Uh this allows you of course to deploy faster. So typically when you're thinking about hardening servers, even with automation that can take 20, you know, 20 to 30 minutes. You know, it's a lot of time especially when you're doing that across thousands of thousands of servers. So the traditional way of hardening is you basically start with an image here and you start stacking and running tasks against it. you know, running 300 different tasks to get it up to this level to be compliant. But when you start think about ROC, it kind of flips that script. You're starting at the top and then all you're really managing with your automation now is backing it off the exceptions that you need to get the application working.

So you're basically now changing from automating and handling all the hardening to just automate the exception way way faster. And then of course with a center pro, you're also still going to do all your normal configuration management. you're going to do your application appointment, whatnot. Uh, but the in the end, Harding itself is not a one-time thing. You're going to have to constantly recheck to ensure your security policies in place that someone hasn't logged in and make change that. And because of all this, you're going to need an automation product to basically ensure you stay compliant. >> And from you've been on so many customer calls.

I mean, they really look to you as an expert of Ascender Pro. I I love that because we got the guy, we got you. Um what are some of the examples that you might have seen some of the the challenges that a sender pro really solves for based on some of those conversations? Uh so you know we started Ascender Pro you know from the upstream AWX you know but we wanted to go our own direction with it because we wanted to focus on stability and security >> you know uh it needs to stay stable because again this software runs your company you know you can't

have things like arback permissions breaking when you go from one version to another it just you know it's bad uh it needs to be stable because people are depending on this but it also needs to be secure again needs it runs your company it needs to touch it touches every aspect of your infrastructure. So security on top of that is paramount. So you know as far as what it does for you, you know, again, we talked about all the configuration management, everything else. It's making your life easier by doing all the day-to-day tasks that you know, you don't have typical time to do and do your normal job, >> right?

Yeah. So it's I'm on both levels. This is really freeing up a ton of time for the team >> in an era where we just have none of it across all three of us. I mean, that's the goal. I I need a solution for myself. >> Well, you know, and I'd like to call out though, Jimmy, and I think maybe in your modesty, um, you are selling a sender pro a little short in terms of, right, it is much more than standard automation, right? Like I, you know, we can achieve a lot of this with, you know, again, agents, Anible, >> right? Um, but you were, we are going above and beyond that with ascender.

>> Yeah. And there's a lot of stuff we're doing with a center that goes above and beyond that. Um, basically automation itself, it fixes a lot of issues, but again, we're taking it further and making it easy for you to access all the information behind the scenes that the automation collects, you know, so you're able to then have all this wealth information because you get a lot of data from your automation. Uh, but in, you know, normally you don't have access to actually see that data, search that data, run queries on the data, build reports off of that. And a lot of the stuff we built behind the scenes with this allows you to do that.

You know, easy things like going in and, you know, a simple example, if I have a, you know, I was swapping from one DNS server to another DNS servers. DNS servers are in the fax. I can run a simple uh report pulls up all data servers that are using the old one. I know which servers now I need to swap over. You know, simple things like that make it easy with all the stuff we're doing behind the scenes. >> Yeah. Yeah. So let's let's go another step further guys. Um the integration and I'm going to echo our cloud team and their favorite line which is better together.

RLCH and Ascender Pro are they definitely fit that. They they work hand in hand. Why is the combination of the two more powerful than themselves on their own? >> Let's start with you. >> Okay. Well um uh again and we've laid this out. You said it, Lindsay. Jimmy said it is um because it it enables us to overcome the tension between speed and security which in reality is always a tension right when we one take a prevetted preh hardened pre-mediated image and then we extract that hardening into an anible playbook we now have the ability to automate that hardening at scale while still having the ability to customize it for our particular environments, our particular sites, our particular workloads.

But now we're 80% of the way there before we even start to think about how we automate that deployment. So we have to turn some dials to make sure that it addresses um you know our our particular needs. And before I pass to you, Jimmy, I also call out the thing that I get a lot of excitement about from customers um is configuration drift, right? So now it is one thing to say, okay, we were able to quickly get this out to a bunch of servers, but now I have no idea what the server state is in terms of configuration and it's going to very immediately as soon as that OS hits a box.

um an app dev team, a business team is going to be tweaking it, undoing some of the security, but then also um at any given time those boxes that are running different loads may be um um exposed or vulner or or exposed to different vulnerabilities. So now I have to manage that. Ascender Pro, which Jimmy can explain better than I can. Um, now not only gives this the ability to get it deployed, but to get visibility into what has happened post deployment to make sure it's secure. >> Yeah. So, a sender pro, you know, again, automation in general, you're going to be running that constantly against your machines.

You need to make sure they stay compliant. So again the anible playbooks we talked about earlier the hardness servers you can use those same ones to actually recheck compliance over and over and over again. Uh you can even take small little patches. You know a lot of our customers are running these nightly just rechecking automation re-checking ensuring everything's compliant. But you know we have other customers they'll actually pull out some key little actual tasks and say we need to make sure these are the most important ones. We want to run these ones every 15 minutes against all these servers. Again, very easy to do, very quick to do within there, but you are basically moving at a faster pace to try to keep up with the pace that the world's moving world's moving faster.

You need to make sure you're actually hard and secure. But what we're doing a lot of on the back end now is again a lot of the data we're collecting. So like package facts, you start thinking about package facts. Every package that's installed on a system, you're able to now really quickly view across your entire organization, your entire infrastructure. you know, if I need to know what versions of the kernel I have deployed everywhere, I have that information at my fingertips. On top of that, we now take the aata date from rocky Linux.org and actually apply that to the packages. So, we know exactly when packages are vulnerable.

So, if I actually run package fax on the system, it pulls that down. Say someone logged into that box and installed TNET on it manually. That's going to get picked up in that package fax. We're actually logging changes to packages, too. So if a package is added, we log that. If a package is removed, we log that. When you change from one version to another version of package, we log that. So for instance, I'm troubleshooting. I have an issue with this server and it started going berserk. I can see, you know, hey, look, go from this time period to this time period. We show that we upgraded from engine X 1.21 to 1.24.

You know, very simple. It's there. All the data there for you. However many ways you want to actually look at is there. uh my si your sizzle comes to you tomorrow and says hey I just ran in this journal about this open SSH CVE we need to get dispatched you know we have the arata data you can now look across your entire organization go in check on the CVE say okay let me search for this CVE it now shows every package that is vulnerable to that CVE and then every server that is running that package across your entire organization again it's just taking all the wealth of information and just however you want to look at it you know you can build reports for that.

>> Yeah. So, it's deploy fast or deploy secure. Now, it's deploy fast and secure. And it was funny. I was actually talking to Greg Kurtzer last night. He thinks it's three things you have to focus on. Security, performance, and cost. And we won't get into it here, but we definitely have the the cost advantage for these two products together and the loans. We're kind of like checking all the boxes now. deploy fast, deploy secure at a cost that makes things very efficient for the entire team. Um, so Brian, what about scale in the Linux community? Gueies can be viewed as costing performance. What do you have to say about that?

>> Well, so I am going to have to redirect this to to to Jimmy, but I but this is actually a good leadin for me. So the reality is in any enterprise that is which is you know I believe something of 78 plus percent of enterprises uh run Linux kind of as their core OS or backbone um you are deploying it at scale. So you have in a small org fleets of hundreds in a large org you have fleets of thousands.

So now think about thousands of servers packages randomly getting installed on those you're hard con your you know all the work that you put into getting this secure baseline image that you know will pass an audit is now changing thousands of times across an or so as Jimmy talked about when when your CISO comes as and says hey we're up for an audit imagine how long it takes right um so you know a a key thing here Jimmy before I hand it to you is this solution can't just work in a clean room, right? It has to work at it has to work at um scale and it also has to work in real world performant workloads.

So we've defi designed RLCH to support both and the better together story really based on CIQ principles is about we can't just be secure we have to be performing. >> Yeah. And again, not only that, you also have to be proactive. One of the great things about this is now you're seeing in real time as new packages come in whether they're actually vulnerable to something. So if I go and upgrade a package on the system, as soon as that automation runs, comes over, hits me, and I know immediately that this is actually vulnerable to something. So I can patch it before my sizzle comes to me and says, "Hey, we need to fix this." You say, "Already done." But within there scaling as far as the automation side, you know, again, automation isn't hard to scale for the most part, it's resources.

You throw resources. Go ahead. >> But so I'm curious, Jimmy, and I don't mind if I put on your host hat for a minute, Lindsay. >> Um uh because I I had to answer this question for somebody um recently, right? If you have a large fleet, you could be looking at 10, 20, 30,000 nodes. >> Yeah. Um, one of the big value ads ascender pro has is it adds a guey so you can actually visualize the state of your fleet. >> Um, do we have any issues? How have we considered the necessary scale in an enterprise use case? >> Again, we run on Kubernetes. So, you're throwing more hardware at it when you want to scale because again, you're making connections out.

It's not is agentless. So, you're basically making connections to whatever devices you have out there. Uh we've done tests up to 50,000 devices at this point on the back end. Um just millions and millions even on the logging side with packages. You know, you're talking about uh I think we're at like 50 million packages in the database we're running right now just for the full scale of it. But again, as Kubernetes, you can scale it very easily. You add more hardware, you add more resources, but it's all based upon the amount of connections you're making out at one time for scale your automation. But it's, you know, something we've actually worked on, something we've actually made sure it works, uh, here at C CIQ.

>> Awesome. You guys, you make it sound so simple. Um, I love it. It's just it's such an easy thing to understand because it's such a common pain point that we're really attempting to solve with this better together story. Um, let's go a little bit let's go another layer deeper. Um, you guys have both participated in so many different conversations with our customers and people that we're talking to as potential customers. Um, what are the specific use cases where this really really shines >> and supports the internal teams? >> Um, secrets, Brian. >> Okay. All right. Well, never mind then. No. Um, no. I No, I'm I'm proud and I think we're all proud and I'm excited to share that.

Um, you know, beyond compliance, we've talked about proactive hardening, and I'm going to take a minute because I didn't get to earlier. Some of the things we do in RLCH and proactive hardening is we install LKRG. Linux kernel runtime guard root detection kit. Consider it anti virus for Linux systems. Windows ships with anti virus. Mac OS sort of has built-in anti virus and gatekeeper. Linux doesn't. Um, LKRG presents that. We take common packages such as gibbc, open ssh, which everybody uses unless you're in an airgapped environment, and we slim those down and harden those to reverse the attack surface. Um, and then we do things like we take mallet, the user space memory allocator, which every program uses via gibb c.

Um, that's where a lot of um attacks come. They manipulate memory. It's very easy to leave sensitive data in memory, dirty memory and we do things to help prevent ensure that that doesn't happen or our engineering team um decades of security um experience have built that. Um so because of that RLCH we always had planned and identified before shipping that it would mitigate entire classes of vulnerabilities just because those hard modules would mean that a RLC8 system would not be susceptible to vulnerabilities. Now recently I'm not going to remember the CVE name but we will post it in chat the actual CVE number. There was a systemd core dump vulnerability that came out right about the time that we were getting ready to ship RLCH.

Now to explain how serious this vulnerability is is when exploited it will dump password hashes for every user on that server on that system including the root password. Moreover, because those passwords are are are not, I'd say um um um the highest potential strength of hashing, those passwords can then be cracked in about 3 minutes. So, there'll be a blog post that is a real world example uh proof of concept of how you can dump all of the passwords and you could crack the root password in a matter of seconds. Right? So, what are what is the real world example we're proud of? RLCH is one of the few enterprise Linux distributions that um is not affected by that.

It's shipped out of the box with that vulnerability mitigated while upstream Enterprise Linux 9 distributions still have the vulnerability. Um um so you know that is exciting and it's actually one of uh a number of vulnerabilities we've identified that just don't affect the system and over you know the evolution of RLCH we'll be continuing to expand that um that list. So, you know, not only, as we've talked about, can you quickly remediate CVS, not only do we deliver monthly releases and playbooks with those remediated, there's a number of CVS that you're not even going to have to worry about because you've deployed RLCH and we've proven that.

>> Yep. Yeah. It's almost zero CVES, imagine. Um, stay on that for a second, though. How do you measure LKRG's performance impact? um you how do you measure well first I'll say I'll answer it a different way um LKRG is a KM mod that installs against the kernel as basically monitoring the kernel for to put it very high level um for suspicious behavior privilege escalations etc so the reality is it's going to have a performance impact right because it's involved in you know practically every system call um >> um so but um uh solar designer um and Adam and his co-author Adam uh or original creator of LKRG have been uh uh very diligent about ensuring that they're minimizing performance impact.

So as we've measured it and third parties have measured it LKRG will have a roughly 2.5% performance impact right now for certain servers 2.5% is still a lot. So, uh, what the team has done is they've, um, shipped all of these hardening packages such that they're configurable. So, you can deploy the same base image and then you can tune your hardening up and down um, uh, for it to fit the performance requirements of your workload. >> Yeah. And just to acknowledge um, Paul Brunk who is um, participating in today's webinar. He says, "I use LKRG in prod and the secure rocky repo has helped us with CVE mitigation." So, well done, Brian, on that one.

And definitely solar. >> Solar is an insane expert at this material. Every time we put him on a platform and just let him speak to the masses, it's just everybody wants to hear him speak. >> I can't I can't think and I know we're not getting into things, but I have to call out that this is an important thing. if we get a minute is um uh we actually have two lead engineers that have decades of experience in the DoD, one in decades of experience in DoD on compliance and then we have solar designer with decades of recognized experience and offensive security. And so what's neat I realized is we basically were able to put those two in a room and enable people to have their cake and eat it too, right?

Which is which really nobody else is doing out there. um sort of um offensive security knowledge providing for a defensive security posture and then um you know all the requirements and benefits that you get from pre compliance remediation. >> Yep. >> I love my product. Sorry, I could talk about it for >> Well, let's let's give all that energy over to Jimmy now. Jimmy, was there anything else that you wanted to add just in terms of some of the real world the any other real world use case that you've heard? >> So, this as an obvious solution.

>> One of the thing one of the clients we helped was basically you know again they were using automation already so they have already automated some of what they were actually doing but they were moving towards the whole infrastructure as code for everything to the point to where again you know it's cliche but they were treating their servers as cows not pet. So if you think about their in their dev environment, you know, they have their developers actually working on code. They have their own little branch they're working on. When they push it up to that dev branch, that dev branch then calls a web hook over to a sender, which then rebuilds their entire dev environment every single time.

And what it does is it deploys the new servers. So it builds out new servers. It runs all the tests against them. Runs all the checks against them. Make sure everything works. And then it connects over to the load balancer and swaps it over and then waits until everything bleeds off the old dev environment and then access the old dev environment. And for every push, that's what's happening every single time. And because of ROC, they're not have to do that whole hardening anymore. So they're able to deploy these servers. They're using VMware on the back end with some link clones. So it makes it really fast.

Um, you know, they're setting up a new dev environment in two minutes, you know, flat. It's very very easy, quick, goes through, but they and take that entire process and move it all the way up the stack. So now when those developers actually take that code, move it over into their staging environment, same process kicks off. It actually builds out a new staging environment. Then they test everything there. Uh flops it over, removes the old one, and then when they do prod deployments, they do a push in the middle of the night. It then goes and takes that and builds out a new prod development system and replaces it.

So no longer are they actually logging into boxes and doing DNF update on servers or going in and actually updating code on servers themselves. They rebuild every single time. They update their image and now that image is deployed across everywhere. You know they don't have to log in individual boxes. They update here, deploy there. And then you start thinking about now they're in the process of okay well when we have actually system downtime. Say our production server has an issue you know application dies. We no longer have to log into that box and just immediately start troubleshooting. What we do is we tell it to deploy a new prod system, but we tell it to retain the old one.

So it goes in, deploys a new prod, swaps it over into the load balancer, points to the new prod is back up and running. And now our developers can log in over here and figure out what's going on with the prod system and not have to be frantic that oh, we have to get this back up and running right now. You know, this deploy new then figure out what actually happened. So that was one of great examples we had recently of helping one of our clients. >> So are you basically saying that they take action before they go and investigate what happened? >> They actually build out the new servers and get everything back up and running before they then investigate why it broke.

>> Well, assuming it's not like a network issue or something else and then they have a whole new different procedure and it's always DNS. Always DNS, >> of course. Um so Jimmy what's the migration path from AWX or Ansible automation platform to Ascender? >> So to migrate from AWX to Ascender uh there are two different paths. Well the first easy one is if you're already on say AWX 24 or below uh which I hope you are not because even the latest AWX version out there has over 100 CBS against it. So again, but if you're over in that 24 or below, uh there's a lot of customers out there even all the way down to 17, which is kind of scary, but you're able to actually port that database directly over to us and we'll actually migrate it up.

So you just basically take it back up there, deploy ours, apply your database over the top, and then the migrations will actually bring it up to our standard and it'll run. All your stuff is in there. If you're past that point or if you have another system that um out there that isn't AWX, you can actually do a migration via the API. So we have playbooks that can actually slurp everything down from your current system, stores it and actually kind of helps you build infrastructures as code and then pushes that over to the ascender install. >> Incredible. Sounds relatively easy. Um Brian, I have custom compliance needs.

How do I address that? This is the core thing of the better together story that frankly um we learned from deploying RLCH in the field and that is uh while we I'll talk real quick about how we deliver RLCH or Rocky LX from CIQ harden um we deliver multiple golden images as ISOs QCS or cloud images across all the major cloud providers that you can deploy remediated right away, right? But what we've heard is I have custom compliance needs. How do I address that? Right? And that is, you know, kind of leans in the better gather story.

We learned at that point that not only do we need to deliver gold images and deliver the repos of those secure packages, but we need to take kind of our hardening goodness, the scripts our engineers spent weeks on um to get, you know, compliant images and proactive harden images and extract those and provide those as variableized playbooks, right? So it then makes it easy for you to go into the declarative language you of anible and update those playbooks to configure things that are specific to your system or in reality go through and kind of wholesale ch change those playbooks as needed to meet your your specific compliance needs because the reality is um as we used to say in DevOps um um all organizations are similar but no organization is 100% alike.

there always going to be that 20% difference you need to account for. >> Yeah, exactly. Um there's just, you know, 20 or so minutes left. I wanted to take a moment to just remind the audience to send in your questions so that we can ensure that this is we're answering some of the burning thoughts that you have in your mind. And there's one from Jeff Muny already. Um I think I'm not sure which one this is for, but you guys >> probably for me. >> Great. Yeah, that would be they're building a release n minus one. So it depends on what the issue is because most time again they're testing prod fully before it actually deploys and swatches to new prod.

So the application itself isn't broken. So they generally don't have to jump back to another release. >> So they're not really have to do n minus one. You know if they think it may be a problem they have that ability to go in there and actually tell it okay jump back jump back. you know, they may jump back two or three if they notice a big issue, but within there, you know, it's more that, hey, this thing's actually running. You know, it may f, you know, one of my log files has groove rapidly and filled up the whole hard drive. You know, it could be as simple as that.

So, deploy the new one, then figure out and then go in and fix the code one and then you deploy that to the other. >> And in this case, Jimmy, um, because I had not heard about this case yet, this is almost like so you have the application layer and you have the OS layer. This is ensuring you always have a a clean o environment on standby. Yes. >> So if it's an environment issue is that no >> well it's not on standby because again they can build it out. >> Right. I I guess I don't mean warm failover. I I figure that was ambiguous. But really what you're doing is you're refreshing the environment.

>> Yes. >> To get it back to a known state and then you're then you're diagnosing what it actually happened. >> Yeah. And again it's still using the live database. So it's not, you know, recreating the database from scratch. It's still using the live database for a lot of their applications. So there still can be issues with that, but again, now that's just a normal issue. You go in troubleshoots, >> right? >> Yeah. >> You're never going to get away from that fully. >> Very cool. >> Yep. Um, any other questions from the group here? Um, if not, there's a few other things that we wanted to cover just to ensure we gave you the full picture.

um within this period of time >> and if not I have one here um Jimmy what other visibility do you get from a sender pro into CBEES? >> So into CV so with Vera in general we talked about it a little bit but we give the data to you in different ways. So you can first see basically at a host level that hey these are all my packages and these are vulnerabilities they have for individual servers but also at a overview level for your entire infrastructure you can see at the level saying okay I have this a came down from rocky Linux this is all

the packages underneath and then these are all the CVEs we give you data you know all the data as far as what that CVE is what the issue is what was fixed what the new package version is for had um even links over to you know workarounds etc from upstream that you can run if necessary but we also give that to you in a couple different ways. So that was at the view. You then have the overview package view which I can go down to any package that's vulnerable. Say engine X. Click on engine X and it shows me every engineX version across my entire infrastructure that has vulnerabilities against it.

Expand out that out. Shows you the vulnerabilities but also I can show for each one of those packages every single host in my environment that has that vulnerability. And then jump over to the CV view that we talked about earlier. That's where I'm presenting it a different way via CVE. So, basically, it's a huge list of CVEs. Uh, like one of my test system has, you know, 1,700 different CVs against it because I haven't patched it on purpose just to showcase it. And within there has the CVS, the CD names. I can actually go through and search if I'm looking for a specific one like, you know, the Open SSH one and then expand that.

It shows me all the data about the CVES. Expand that. It shows me all the data about the packages vulnerable. expand that shows me every system running those packages and what version they are. So again, just a wealth of information and all this data comes from just that simple package box module importing it to us and then us handle it in, you know, different ways. >> Awesome. Um Brian, back to you because I asked the migration path um from upstream to Ascender Pro, right? >> What's the migration path from Rocky Linux to RLCH? >> Yeah. Um, and as usual, I'm going to I'm going to take the scenic route to the answer.

I'm going to start with saying one again, one of our key principles is that and I think we take pride in in all we deliver is uh we build our commercial offerings on top of open source. Um, so we don't get our value from locking you in to our district, right? So RLCH and a number of the hardened components um are actually available upstream, right? But we take those, we package it, we test it, we provide different deployment methods, we add value on top of the bits which are coming from upstream. Um, and so now if we sort of invert that to go, okay, great. Well, I'm upstream.

Um, I can fall from RLCH to RLC, but what does that mean if I want to go from RLC to RLCH? Um, and there's a couple of Glenn >> RLC to RLCH, right? >> Um, no, in this particular case, I'm saying Rocky Linux to RLCH. >> Okay. Yeah, keep going >> because it's a longer answer. Let's call out while Rocky and RLCH is built to be ABAI or bug for bug compatible with Red Hat Linux as well as mimicking though it uses DNF instead of uh RPM but mimicking the user experience of Red Hat. So it's easy translation easy translation but this is how do I get from Rocky to RLCH?

Well, what we've done is we've built the hardening such that it is a layer on top of community rocky edition. Um, so effectively in this context of talking about anible, if you buy ROC, we will deliver you anible playbooks which will effectively turn your current Rocky deployment into a Rocky Linux from CIQ harden deployment. >> Awesome. We are right on schedule. Look at that. Are there any other questions from the audience? If not, I'll just give it a moment. Cool. Well, I think guys, that's a wrap. Um, thank you so much for making the time to just share so much information. Um, if you have questions, if you watch this again, it'll be up on our YouTube.

Definitely, you might hear something else that you missed when you watch it the second time around. If a question comes up, email us at infogiq.com. Um, you can also use that form to schedule an expert demo which would be customized to your environment as well as a security assessment. Um, there's tons of resources available on ciq.com. A lot of the the thought leadership stuff that Brian and Jimmy have authored either separately or together are available on the blog at ciq.com as well. Please uh be sure to follow us on LinkedIn so you can catch word of the next webinar. There will be a lot more on this topic because security and performance is definitely a strong focus for CIQ going forward.

Um it resonates with a lot of our customer conversations. So we're going to continue to double down on this topic and look forward to your attendance in the future. Um subscribe to our newsletter. You can catch that at the bottom of our website. And thank you very much. I hope you make it a great day. >> Thanks so much everybody. >> Thank you Lindsay. Thank you everybody. >> Bye guys.