How to manage a Linux fleet you can actually account for

I guess that's that's probably the really seed of everything we're trying to do here. We're trying to maintain that control uh so that servers don't get arbitrary packages. Somebody can't install something they're not supposed to. You don't get versions you don't want um or before something else is supposed to happen. So, we're trying to control the content uh that our systems have access to. Got it. Got it. So where does that content start? And inside uh the video and in enterprise Linux manager itself uh it starts with adding sources and and putting them into your system. So let me bring up the video right now. We'll drop into it and let's see if we can't talk a little bit about what's going on here.

>> All right. So in this example here, you can see I've already installed um I've got several uh systems already or not systems uh repositories that I've synced from upstream uh in this case from RL from CIQ's uh secure depot. Um but this is what I'm going this is what my source of truth is coming from, right? I pick where I'm going to pull my packages from so that the rest of my systems can have uh get to them. And in this example here, I've already pulled in RLC Pro Core, AppStream, and the RLC Pro main. And so this is what, you know, upstream, this is what's going to pull into my Elm um uh host here that I'm going to make available to my systems um from within Elm.

So that's what I'm doing here. Um you know, if I wanted to create a new one, it's very simple here and you know, give it a name, for example, um core. You can see I selected depot server, but I could have chosen a local ISO, another HTTP source, an RSYNC source, wherever those RPMs might be. I can then pull those into here. Um, I've already, you know, selected my I've already set up my my authentication for uh depot server, for example. And you can see here I've got RLC Pro 10 from CIQ. And I'm going to pull that in. I'm going to click sync. And then that's going to go ahead and pull in from everything from upstream, you know, from the CIQ depot server into my Elm server.

So it's going to rest locally here inside my Elm server um that I'm going to use later when we get into content views. >> So does that mean that uh I can do multiple versions of not just RLC, but uh can I pull in sources from all sorts of different places and versions? >> Yep, exactly. Um if it's RPM based uh we can pull that into um Elm. >> Okay. So does that mean if I wanted to manage more than just RLC um this can kind of handle some of that? >> Yep. I certainly can. >> Nice. Nice. Okay. Uh so after you've set up your sources and kind of where everything's pulling from, the next uh next step in the process is uh pulling those together into what we call a content view.

This should be fairly familiar to people using other tools out there. Uh the way the content views works, but for those who don't know, uh can we talk through this and tell me kind of what a content view is and why you want to use it? Well, content views are what our actual hosts are going to have access to. Right? This is where our hosts are going to point to the the content views uh which create repos behind the scenes that our systems are going to have access to. In this case, you can see I've got several content views. Um and each time I do something with a content view, it's going to version that content view.

Um and so then I will talk about that in a little bit here. But you can see I give it again a simple display name um here for example and I'm going to create a oneto one content view with upstream from that source. Uh you can see it's going to sit there and pending until I actually go over there to the right and in a second here it'll click on uh compose. Um and so it's going to show you all the packages that that's going to then uh compose into this content view. Um and this is a very quick uh uh setup here. it's not, you know, this lengthy hourong change.

It's gonna uh set up this compose um very quickly so that now I can then go through and then start doing something with that particular content view um you know which we'll get into like making sure that I want to uh exclude packages or pin particular versions of packages. I can do that now within content views and set that available to my downstream hosts.

So when I'm setting up a content view basically can I say like hey this is the content view I'm getting from the RLC upstream uh appstream let's say and I want to make sure that on these sets of hosts uh they we do not allow them to have access to engine X for instance is that where is that where in that when you're setting up that content view you can say exclude this don't include that >> exactly thing >> so once I do this initial compose here. Then I would go back into that and that's where then I would then uh change um and set up those excludes um and um um we're going to show another one here.

>> Yeah. Yeah. I'm trying to get to the po point there's a part here where it actually shows the packages part. There you go. Yeah. Is this where you would actually say like hey don't include this. >> Not this not this view here. >> This actually just shows you kind of what's in what's inside that source. >> What's coming down from the source? Exactly. Yep. >> Got it. Got it. Okay. Um and so uh but I can go in through there and then start to curate that view and say exclude this don't have this in there. >> Um or or or or pin a specific package.

>> Is that right? >> Yep. So if I need a, you know, particular kernel that I don't need, I've got packages that are dependent on an exact kernel version >> and I don't need that even if one newer version were to come down um because of some you know uh virus or or bug rather that was found and they updated the kernel. But I don't I can't use that yet because I've got downstream third-party applications that need that specific kernel that I've got that I'm using. I can pin to that specific kernel. And so even though there may be a newer package, my systems aren't going to have access to the newer package because I pinned it to the to the exact kernel that I want.

>> Got it. Got it. So then also in Elm, I mean obviously part of this and I kind of alluded to it, which is there's a place inside Elm where you're going to go in and you're actually going to set up your hosts. Um uh what's affected, who gets access to what, uh assign hosts to different groups, and then you can assign and set me through this here. You can assign different groups different content views. Is that correct or is it you >> can? Yep. >> And can you do multiple content views per like group of hosts to say, hey, this is the core the core set of you uh of RLC9 and then here's the the app content view and then here's this.

>> Yep. Yep. >> Okay. And and now talk me through why that's important because to me it would be like don't you just just go one source to one set of hosts at that point. >> Well because all the different sources have you know like appstream is going to have very different packages than core right. Um and so I need to make sure that I have if I need packages I have appstream I need to have that content view available to the to the host as well or there could be dependencies um uh based upon something that I'm pulling that may need you know something out of core if I'm installing something out of uh appstream.

So I need I need all the pieces there together to to resolve any kind of dependency issues um and make available what the actual systems need and that may require different uh content view pieces um put together as a whole. >> Got it. Got it. So, one of the other concepts inside the video itself that it talks about, and this was one again kind of hard for me to wrap my head around, which is you can base content views off of other content views. >> You can absolutely that's the that's kind of a core piece within within Elm.

Um u the whole idea of um the you know you've got stages in your in your environment whether it be you know appdev test prepra prod you know these different layers right and you want to go through each of the layers so that prod doesn't get a surprise you know production layer doesn't get a surprise so in this example here I've got a content view called dev um you can see that it started out every time I do a a compose it's going to increment the version number. Um and if you pause here then the uh so in the example here so at version two of the dev content view I then picked vers so that became my version one of my prod.

So u of so the content view version one of prod is actually back is pointing back to version two of my dev content view and version two of prod is actually pointing back to version four of my dev content view. And on the up the chain we go. You can go ahead and let it continue. Um and so I have already done the testing like for example in my dev I may have done a whole list of excludes and a bunch of pinning. I don't have to do that again in the prod content view. It's because it's pulling back from that particular version of the dev content view.

And it's a very simple then uh relationship. Uh so now an example version eight here um um is going to point to then becomes version four of the prod um and so I can do my live testing of new versions, new new uh exclusions, new packages in my dev content view, test it on my dev uh uh host environment and make sure that everything works. Once they work, then I can then simply move it to uh the produ and point prod to that that the version I just tested in dev. So that makes sure that that that helps then prevent anything landing in the prod servers, the prod hosts that uh haven't been purely uh tested um u prior to you know being promoted into prod.

>> So you're actually not maintaining kind of two lists at that point. you're not having like here's my list of everything that we have on dev um and then I have a separately like built from scratch version of what's on prod and I have to do these manual sync between these two lists right you're actually >> no yeah none none of this you know rs syncing between you know directories you know for prod and another you know rync you know to prepro or anything like that and and duplicating all these RPMs over and over and over and over again because there's a different view I have one RPM M stored one time uh in the uh uh on the Elm host.

>> Got it. Got it. So, does that also allow you when you're actually kind of like pinning a version of prod to a version of dev, does that mean that like if something for whatever reason might go wrong even in production? I know it happens every once in a while. Does that mean that you can actually kind of roll back and say like, "No, no, no. We really need to not have this. So I'm going to roll back prod to to look at a different version an older version of dev. >> Exactly. So instead of I can go the other way. So in version you know in that example I had version four of prod.

Maybe we found something that we didn't catch. I can take that and go back to you know a different version maybe version you know whatever version was below that in dev um you know version six for example of the dev curated view um of the content view. >> Okay. Look, I'm I'm asking all these questions because Elm seems like a fairly straightforward tool. Um, it can seem like it can do a lot, but there are certain key concepts that from my perspective are like they they seem like their core parts, but are maybe a little bit hard to grasp. And that's one of them for me is like why is dev based prod based on dev and how can I roll things back if there's problems?

If I even if I taking away the dev and and prod if I have dev on version eight can I roll dev back to version seven as well just within contact view. >> Yeah absolutely. >> Okay. >> Yep. >> Okay. And basically what that's doing is saying like hey on version 8 I we maybe added in some new versions of these packages uh or these RPMs uh and we had a couple extra things that got added. We did some testing in dev. We don't like it. We need to roll it back. So I'm going to roll go from from v version eight back to version seven.

And all that's doing at that point is when that particular host says hey what packages are available to me in what versions that list that comes back to that host is curated by Elm itself to say this is what you have. Is that right? >> Exactly. That's exactly right. So then the next question from my perspective is so what happens if I actually said at some point we'll go out and install those new versions of this and new versions of that and then you roll it back or you're going to skip over things. How do do we do that? So in the video you talk a little bit we talk through the the relationship between uh Enterprise Linux Manager Elm and another tool called Ascender Pro um and what those things look like.

So I want to kind of talk through that for just a little bit here um and what those are and kind of define what paths they run in and how they work together. >> Absolutely. So again as we mentioned Enterprise Linux Manager is our content um uh curator or the view of what our systems are going to have access to where Ascender Pro gives us the ability to then manage and orchestrate what's going to happen on those systems right uh we can set up playbooks to to run against a system um um to then you know do that DNF upgrade or DNF install package you know whatever Um and so we can do that through the playbooks through Ascender.

We can then manage that. Um so the Ascender Pro becomes our workflow manager where Enterprise Linux Manager is our curated uh content view of what the systems can use. Um so if we've we've set up Enterprise Linux Manager so that it only has access to a particular view of a package um and available you know what what upgrades are available when I run my Cinder pro playbook to do the upgrades. It's only going to get what uh has been ava made available in um um Elm. >> Got it. Got it. So they kind of work together um and they can kind of hook up together to kind of run things um and to a certain extent kind of >> talk to each other.

>> Exactly. So in that in that case where I can set up so that one can talk to the other and I can even use a sender to deploy the small binary that we use. It's a small go binary that is the that goes uh client on the host. So I can use a sender to actually deploy that host um agent um to the boxes so that when they connect to Elm they know exactly what repositories to pull from um which content views they're part of which groups they're part of and and and uh uh Elm and then you know from that point then when a sender needs to pull something else it it it's getting the right packages at the right time.

>> Okay. So, let's talk about the shape a little bit about what Elm looks like from an install base. You touched on a little bit. Each host is going to have a very small agent that kind of runs on it. >> Um, and then in addition to that, where where is Elm running? Is it running like traditional, >> if you look at at other products, they run as a series of services that you kind of have to maintain the whole kitten kaboodleoodle together. How is Elm laid out?

uh in addition to like those little host agents that run there >> elements actually um a single um Kubernetes workst binary right um and so there's no extra services no layering of packages this package needs that package to do this thing um it is a single install um uh ran on a single host very easily you know in a in a Kubernetes uh fashion >> so it can run like as a Kubernetes container or like a Docker container or something like that. >> Um, and it's >> it's then self-contained. Do you need Elm on every single machine or is it just a single install for your whole fleet?

>> Single install. Uh, depending on, of course, you know, uh, making sure your your downstream hosts have access to Elm, right, in the same network segment or firewall allows, right? But it's it's a single host um, with the disk space that it needs to hold the reposi hold all those repositories. It's syncing from its source. um so that it can have access and then all the other hosts can pull in from from that particular Elm server. >> Got it. Got it. So when you say that like everybody needs access those agents on those hosts need to talk to the whatever wherever you set up that Elm server >> or container >> uh to to run but other than that there's not a lot of limitations on that.

and it's not uh it's it's the single place where it pulls down everything and the metadata that it needs for the lists and all those types of things to kind of work that way. Um, so does that mean that each host, let's say I have the Elm server and each and all the hosts, they're all on a local like pretty much uh locked down uh network. Uh not completely air gap because somebody needs to go out and and grab the the package list, but uh fairly locked down. Um, you know, a lot of the way that most a lot of fleets work that I know of is that uh they all kind of phone home to uh through the internet to the same servers and they're all built uh pulling down packages.

The same package they're pulling down at the same time and so you're just eating up bandwidth. Um, Elm kind of kind of consolidates that. Is that right or am I >> No, that's exactly right. and and and there are other, you know, solutions that, you know, kind of do the same as Elm, but it they add so much more weight and and and uh uh extra stuff that is just not needed to curate what we're trying to do here back in the, you know, kind of the original Linux mentality of one tool, one job. And that's what we're doing with Elm. um it curating our content and making that available to downstream servers um without a lot of extra you know uh bells and whistles that just aren't used just aren't needed.

Um yeah, >> so that also then goes back to the idea of you mentioned the phone home idea. Um neither Elm nor the clients then have to phone home uh and let you know uh the mothership know what's going on, right? Elm walks, you know, uh uh reaches out to whether it be depot server, whether it be an, you know, an ISO box, um arbitrary HTTP service, it's just going to pull in those packages and make them available to our clients. our clients are going to use those packages uh to you know do the jobs that they are designed to do. Um there's no licensing management done via Elm.

There's no phone home to let people know what's being installed by anybody. So um >> so >> yeah [clears throat] >> is Elm the only one who knows like I'm talking to a 100 hosts uh or versus 10 versus a thousand. Uh the upstreams don't know. All they know is that the Elm server itself pulled down that ISO or the Elm server itself put pulled down that that package list just did it once and that's all it knows. >> So you're really kind of not having to and from a phone home standpoint a lot of times that that ends up being uh licensing right to make sure that you're under licensing and the oh nope you your your server is talking to a thousand nodes so we're going to charge you for a thousand nodes.

Whereas Elm says >> it doesn't matter how many you connect to. Uh we're doing whatever you want there. >> And you mentioned the uh um isolated area, right? You know that that u so again Elm is the only one that needs you know remote access um to wherever it's going to like in the example of a depot server or some internetbased HTTP you know service. Um that's the only one that that's Elm is the only one that needs access to that. All the clients need access to Elm and that's it. So, um, >> it's also worth noting, I believe, uh, that that Elm is not like a separate purchase skew.

This is something that comes with an RLC Pro subscription. Is that correct? >> That is correct. Yep. It's just built right into RLC Pro. It's just another piece that we provide as you know, um, you know, table stakes for enterprise Linux. Um, ought to be no extra skew, no extra cost. >> Nice. Nice. Well, I this was this has been great. Uh that's just the surface of what Elm can do. It's just the surface of kind of what the video kind of touches on as well. Uh we only pulled a few pieces out of of the demo. Specific ones that are like those key concepts that are maybe not necessarily super trivial to understand, but are really important to understand as well.

Uh so if you want to see the whole thing uh we have the video what is Enterprise Linux Manager up on YouTube and again we'll give you the link uh in the chat. Um if you've got any additional questions or want to dig into Elm or the rest of RLC Pro lineup. We'd love to set up a discovery call for you and get into your specific needs and setup and what we can do for you. Patrick, thanks for walking me through all of this. This has been fantastic. I appreciate it. >> Thanks all. All right.