CIQ’s Brown Bag Webinar Series
Webinar Synopsis:
Speakers:
-
Zane Hamilton, Director of Sales, CIQ
-
Michael Young, IT Manager, CIQ
-
Robert Adolph, Chief Product Officer, CIQ
Note: This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors.
Full Webinar Transcript:
Zane Hamilton:
Good morning, good afternoon, and good evening, wherever you are. Welcome to another CIQ webcast.This week we'll try to do something a little bit different. We've titled this one a Brown Bag webinar. We want to talk about some of the things we’re hearing from our customers, some of the things we’re hearing from the community, and then some things we've been working on personally or even for CIQ. So today, I have Michael Young, and I believe Robert Adolph is with us as well, so welcome. It's been a while since you've been on. Would you mind introducing yourself and tell us what you do at CIQ and with Rocky?
Michael Young:
I'm a Linux software engineer here at CIQ. I help out a lot with fielding questions that we receive on how to run applications or whether this or that works on Rocky. I also hang out in the community forums and help there as well.
Zane Hamilton:
Thank you. Robert, you've been on here a lot. Everybody should know who you are, but if they don't, please tell us who you are.
Robert Adolph:
My name is Robert, I'm our Chief Product Officer, and I’m grateful Michael could join us today as well as yourself.
Zane Hamilton:
Robert's very active in testing kernels and playing around with Rocky Linux, so getting his take on things is always fun. One of the things that I've been hearing a lot lately, Michael, is people talking about Kubernetes on Rocky. Obviously, Kubernetes is not new, but it's something that maybe people who are just getting into the Rocky community are starting to deploy on Rocky in their environments. It's something that I'm running across a lot. Is that something you’re seeing as well?
Michael Young:
Yes. A lot of people are asking, Hey, can I run Kubernetes on Rocky? How well does it perform on Rocky? So yes, we've been getting a lot of questions in that regard.
Kubernetes on Rocky [01:55]
Zane Hamilton:
One of the things I noticed, especially with a chip shortage and people not wanting to spend a bunch of money, but they want to learn and play with Kubernetes; they want to build it out. I know the cloud is always going to be an option. Being able to go up, spin up some Rocky instances, and deploy some things in the cloud is a great way to go. Some people want to get more involved and understand, maybe even learn something new.
One of the things I've been seeing, and I had a friend of mine's son ask me, “I want to learn something new, and I'm trying to get into IT. What should I do?” I started talking about Rocky, obviously: “Go learn Rocky Linux!” And he started asking about Kubernetes: “I want to build a Kubernetes cluster. How do I get started?”
One of the things that I remember, probably the last two or three years, especially pre-pandemic, everybody was building out Linux clusters on Raspberry Pi, deploying Kubernetes clusters. It was a really cheap, easy way to go play. But now, it's really hard to find a Raspberry Pi, especially if you're looking for an 8GB model. I've seen people paying $750 to $800 for one, when you could go buy a laptop and do the same thing. But for people who have one, that seems to be the problem: they have one. What I've been running across lately is people building out something like this. You take one of those 8GB Pis and attach an SSD, and you can install VMware ESXi on this thing now. You can run a full-blown ESXi cluster, and install Rocky Linux on it, and have multi-nodes, so you have an entire multi-node Kubernetes cluster on a Raspberry Pi, which is something that I’ve started obviously building out, and that’s why it looks like this today. I've started down that road to play with it and see if I can get it to work. And it's been interesting to try to get that type of thing running. Obviously, it's never going to be a great performer. It's not really what it's intended for, but it's a great place to learn.
And one of the other questions I've been getting is: I've got VMware installed on it; I've got some Rocky boxes – now what? One of the things I've also started noticing and looking around for is that obviously there's a bunch of Ansible Playbooks out there to go do this for a simple Rocky cluster. I mean, it's not like it’s that hard, playing with those and getting them up and running. I've seen some different blogs out there that are talking about getting a Kubernetes cluster up and running in five minutes. What have you seen people doing, or what have you done yourself?
Learning Linux on Your Own [04:07]
Michael Young:
I've also been looking into it as well. My son has been showing an interest as well as learning some Linux. And obviously, with me working with Rocky Linux, he's heard a lot about it. Even my brother-in-law has been wanting to pick up Linux. So I'm running into the same issue as many are not finding too many Raspberry Pis out there. But that was the idea: I talked to my brother-in-law and said, “Hey, what do you think about this project? We'll do it with our kids!” He has a son as well, same age, and they're really getting into technology. And I said, “What do you think of this idea: we get some Pis, and we'll start learning and building some clusters, and obviously we’re going to pick Rocky Linux to run on it.”
I've been doing my research and have not been successful in getting some Pis yet. That's about as far as I've gotten right now. But I have seen some blog posts on how to use Ansible, on how to just run a simple playbook. Somebody's already done the work on how to put Kubernetes on to a cluster of Rocky servers. So it's basically just doing a little bit of Google searching, and you could easily get a playbook, run it, and put it on Rocky that way. I would love to get my hands on some more Rocky Pis. If you go on YouTube, there are a lot of YouTubers doing the same thing: building clusters for learning. And maybe they’re not exactly using Rocky, but Rocky would work just as well as the operating system.
Zane Hamilton:
Most people have a three-lane around somewhere, if not several of them, because they were significantly cheaper, and that's what I started off doing. But like I said, I was interested in doing something else, and now that 64-bit is available or has been available for Rocky on the Pi, I've been interested in playing with that. Robert, have you been playing yet?
Robert Adolph:
Our Fuzzball stack uses a little bit of Kubernetes and a lot of Rocky. Michael has done many deployments of our own Kubernetes stack that we containerized and has a systematic way to deploy that as well. I wouldn't say I was on Raspberry Pi. I am an old hardware person. I have like 15 old laptops lying around the house that I string up. Mine looks not as pretty as yours, probably; let’s just put it that way.
Zane Hamilton:
It's got a backup battery built-in, nothing wrong with that either. Michael, you've had some people call in and ask questions about installing Kubernetes. How does CIQ feel about it? There are a lot of options when you go to install Kubernetes. There are a lot of different things you need to think about. From a CIQ perspective, what are we working on for that?
CIQ Approach to Installing Kubernetes [07:29]
Michael Young:
They know that Kubernetes will run on Rocky, but they want to know the best practice. Do we have any suggestions? Do we have anything to guide them in regards to installing Kubernetes on Rocky? I've been trying to work on a document in that regard, because there are different ways to install Kubernetes – for all those familiar with Kubernetes –there are different paths that you can go on installing and setting it up, so basically going through the documentation from Kubernetes and digesting and just breaking down the best path using Rocky as an operating system. I have a document, and I should have shared this before, so it was all queued up.
Zane Hamilton:
Michael, you were looking at this from the perspective of: if I wanted to run this in production. We talked about a hobbyist playing with it, trying to learn to run it on a Pi, or doing it at home. Those installs, you're going to get something from Ansible. It will probably be pretty vanilla and not necessarily something you want to run and stick in production. But you're looking at this really from an operational standpoint of: if I go this route, these are the types of decisions I need to make, and also, where do we stand from a CIQ perspective on what that would look like.
What Is the Best Way to Set Kubernetes on My Production Server? [08:59]
Michael Young:
Exactly. Thank you for clarifying that. Most people starting out typically grab the quick starts or use Rancher or some way to get Kubernetes up and running quickly to play around with and learn. This is more from that perspective of: this is going to go into production. What is the best way to set this up on my production server? There are different ways to set up and run Kubernetes. That's the purpose of this document that I've been working on. I am going through and putting down what are the base requirements, and also what are the things a system admin or a person going to deploy this needs to think about in deploying Kubernetes on their Rocky system.
It’s still a work in progress, but I've actually been using this document for some support cases to go through and set up an environment to help tackle some support tickets that have come our way. I need to polish it up. I'm not going to show you the whole document because it's just a bunch of notes for me to put my thoughts together. We just wanted to share with those watching that we are working on stuff like that here at CIQ – that we get these questions and we say, “Oh, you know what? Let’s see if there's a better way to do this or what is the best way to do this?” And we'll start digging into it and try to figure it out to help everyone else out.
Zane Hamilton:
It's something that’s very important to us, to have a nice streamlined answer, for this is how we believe you should be doing it. I know, Robert, you're an advocate of helping people and making sure people are successful. Being able to produce something like this and share it with the community is important and can add value to the community.
Robert Adolph:
Thanks to folks like Michael for doing all this research over time. But we’re going to have multiple pieces of information like this that we'll be putting out from our support team and our lead engineers like Michael. Over time, we're getting asked these questions, and we'll have accessible information, whether it's in our learning portals or our support portals or generally available across the internet in blogs, on our website, etc. One as important as this, in particular, we'll definitely do a video and show people step-by-step guides on how to do it. Again, that’s thanks to our support team that's really putting a lot of effort into pre-thinking and pre-analyzing how to do these things effectively for our customers.
Zane Hamilton:
I can also imagine that once these documents are created, they will probably be followed by, if not the community, one of us that will put together Ansible Playbooks or Puppet manifests to go do this kind of stuff. So that not only do you have the document that you can view and follow if you want to, but also be able to do this very quickly and from something that is a known good. One of the other things that gets brought up, I know this is something that Robert you've talked about quite a bit, is that universal base image. When we're talking about containers, we're talking about Kubernetes, it's something that’s important for people to understand. What we are talking about when we say UBI or universal base image. What is it? And then why is it important?
Universal Base Image [12:47]
Robert Adolph:
I'll let Michael get into the weeds for you, but at the end of the day, we want a freely available way for everybody to build their containers at scale, in the ways that will suit the applications that they need to run. Whether it's a Kubernetes environment or for science and on the Fuzzball platform, we want people to have a universal way to make it easy and effective, so they're up and running in a streamlined way. Mike, I'm happy to let you dig in deeper on what that looks like in practice, but we're getting asked for this a lot. It's something that we're putting a lot of effort into from the team, and it's something that we're going to see down the road with a lot of the major distributions of Kubernetes.
Michael Young:
A lot of people like the UBI because when they think of the Enterprise Linux operating system, they think of stability and security. It’s been proven. A lot of people want to start off using that as their base for whatever the application is that they're going to be providing or distributing. But you also want a base that is not too big in size being distributed around. With the UBI, the nice thing is that it's condensed down; it’s a small image, just what's needed, but then it's also an image that can be run anywhere. Hopefully, you're running it on Rocky, but you don't necessarily have to run it on Rocky. But you know that the libraries and everything that's in that base image is coming from a source that everyone trusts and relies on for stability. You can run different platforms. I think we’ve talked about VMWare has their platform. Tanzu supports running UBIs on their platform. Someone that wants to use Rocky as their base image for distributing their app, that's one of the benefits. That's being worked on actively as well, to have that available. We did some testing of one recently. Neil, who works with us as well as in RESF, was putting together an image, and we were doing some testing with that.
Zane Hamilton:
Another thing, Robert: we talk about Alkemist quite often. Is RunSafe Alkemist something that can also be put into that universal base image that can make that even more secure?
Can Alkemist Be Put into UBI? [15:51]
Robert Adolph:
It's actually an extension beyond just having it into the build process of Rocky. It’s having it in a container, having it in the provisioning solution, having it in Fuzzball itself. Originally, they designed it to protect applications. That is a natural fit for what we're talking about. It's definitely something that we're going to have available for customers.
Zane Hamilton:
One of our previous webcasts showed going through the process of using repo as a service and enabling Alkemist, and it was very simple and very straightforward to do. I was shocked. It was cool. Usually, when you go through something like that, it seems like it's a whole rebuild and the new install of the operating system, and then it works, but this is truly just running a command, and it's there, it's working. It's exciting that that is going to be available also, in something like a container as well. So I want to open this up, and I want to let people ask questions, if there are any out there. It doesn't just have to be on what we talked about today. I know there are a lot of different things that are going on in the community. There are a lot of different questions that I get quite often. We were trying to have a topic today to spark some interest. If you guys have questions, please feel free to jump in and start asking those. If not, I'm going to make Michael start telling stories.
Michael Young:
Oh no.
Zane Hamilton:
Or Robert. Any good stories lately, Robert, while we wait for a question?
Robert Adolph:
There are always good stories, but while we got Michael on, it might be beneficial to talk through some of the support initiatives we're bringing further down the road and having a way to understand the different packages that different applications need or want. Helping customers with potentially moving from CentOS 7 and preparing to see what they need to go to Rocky 8, and potentially 9 coming soon too. There's a lot of value from a support team perspective that our leads like Michael want to bring to our customers. Michael, feel free to express your thoughts on that and tell us where you're thinking of taking some of that.
Transitioning to Rocky 8 [18:22]
Michael Young:
We get a lot of questions, like: “I’m on CentOS 7, and we’re looking to move to Rocky 8. Does this application run on Rocky?” They're not looking at doing an in-place migrate; they’re just wanting to know: “Hey, can I install the same application over here on Rocky and just expect it to work? That way, I can port over my application and know it’s going to work over there.” We get a lot of questions. “Here's my app. Here's the version. Does it run on Rocky?” So one of the things we've been discussing internally is that we would like to have a way – we're still in the brainstorming processes here – but talking about is there a way where we could go basically run an assessment tool in a sense, pull out your packages and figure out through the tool, is this package (put aside the version, for now) available at least by that name in the repository for Rocky? If it is there, then we have to look and see if it's different versions – we might need to flag that as something to look at. But the idea being that any packages we don't find here, we’d like to start noting that at our support department, so that when we get future questions about that same package, we've already done the research for our customers. From a support perspective, we want to make it easier to fill in those questions to be able to give a good response: “Yes, this will work, but you need to look out for this and look out for that.” That way, it helps with migration plans because there are many people out there who are planning on upgrading to Rocky Linux or changing from CentOS 7 over to Rocky 8. But there's always that unknown: will it work? We're trying to come up with ways from a support perspective to help ease that nervousness and ease the planning for the migration. If that was the direction you were thinking of, Robert?
Robert Adolph:
Definitely. And then a similar mechanism, a similar tool also helps with triage, also helps with understanding validations on different hardware, different capabilities. The support team is looking at multiple different ways to do testing and validation on those hardwares as well. Over time, we're going to have a database full, if you will, of a lot of that information. It helps people make decisions. We're also going to develop a lot of performance-tuning capabilities. We're born out of an HPC company. We're growing beyond that, obviously, into full-on enterprise and hyperscale as well. But at the end of the day, at the core of some of our key folks is performance. We'll have a lot of capabilities around how to tune Rocky, how to get the most out of Rocky and/or variations of Rocky that come to market.
Zane Hamilton:
Outside of customers asking for this, do we have vendors coming and asking for this preemptively – so is their customer maybe asking them to support Rocky? Is that something that we see as well?
Are Vendors Coming and Asking to Support Rocky? [22:13]
Robert Adolph:
We did have a nice partnership announcement a couple of weeks back with our friends at Google Cloud. Clearly, they're getting asked, but I can tell you that every cloud partner we're working with is seeing a similar uptake. We're talking to multiple different large organizations, whether they’re government or commercial, that are saying the same thing. The same conversation is happening over and over, and we're preparing solutions to enable customers faster and faster.
Zane Hamilton:
Michael, you hit on something that seems to come up quite often. It is that migration from CentOS 7. I know there are many ways to go about it, but are you seeing a specific trend? Are there people that are doing more of trying to go from CentOS 7 to 8 and then to Rocky? Or is it truly the Wild West? Everybody's doing it differently, and whatever makes sense for them?
Michael Young:
A good majority have already done the transition from CentOS 8 up to Rocky 8. There's a migration script that has been put to its test. It's working very well for many people. But the questions we've been filling lately have been from CentOS 7 straight to Rocky. I don't recall anyone discussing going to 8 first, CentOS 8 first then to Rocky. The discussion is more around, “I'm on CentOS 7. We know we have a couple more years of support left, so we're starting to think now about transitioning over to Rocky 8.”
Zane Hamilton:
So the SOS tool gets used quite often to see what's on those machines? And how does that process work? So they run or execute the sos report. They give it to us, and then what happens?
Sos Report [24:17]
Michael Young:
So there's a really neat tool called sos report that gives someone a good snapshot of everything on their system – hardware, software, the RPM packages installed, storage, or anything you would need to try to figure out: are there any gotchas? That's one thing when we are engaged to help with migration, we ask to run that tool first because with that we’re able to get a good idea of what we're working with. What's on the system? What do we need to look out for? That's an excellent tool to start with for gathering data and figuring out what is needed to be considered in a migration plan.
Zane Hamilton:
I'm assuming that is also going to feed back into what Robert's talking about of that central point of ‘here are the gotchas that we’ve run across before.’ So as it comes in, you run the report, and immediately we know that this is going to potentially cause a problem or we know this is going to work.
Michael Young:
Definitely. I was looking at it just yesterday. There was a discussion around having tools to help us with assessing systems. When someone says, “I have a problem,” we usually say, “Okay, give us that report just so that we can have a good feel without having access directly to their system. That gives us a good way to understand what is there and what is present. But definitely, when I looked at the sos report tool, I was looking at the code a little bit. I haven't dug into it enough, but it's made to be extensible. There are ways to do plug-ins. That's an area that would be good to explore. See what we can leverage in this regard as far as helping with migration. Maybe there's something we could add to it. That would definitely give us the ability to direct someone who wants to migrate in a better way. Then right now, it's a lot of manual work, but maybe we could leverage that tool to help us since we're going to have information about their system. We can immediately, through some scripting, pick up on, “Hey, we see this here. We’ve got to take care of this before they can migrate.”
Zane Hamilton:
Very interesting. Thank you. That's fantastic. I don't know, Robert, if you have anything you want to add to that. That is a great opportunity to start adding functionality and helping solve that problem.
Robert Adolph:
At the end of the day, just keep in the back of your mind, our support team is made up of folks like Michael. We're not just going to say: “We only support the OS. We only support an issue or a ticket.” We're also here for advice and architecture and helping people be successful. Our goal is to help all of the folks that are using Rocky to be successful. We're trying to do that in the community. As you heard, Michael's in the community, helping there, like tons of volunteers as well as our customers individually. We are here to help you be successful. Whether it's about a migration conversation or architectural conversation that's coming up, know that that’s what our team is trying to do.
Zane Hamilton:
That's great. Thanks, Robert. We are almost out of time, and I'm trying to see if we have any questions that have come in. It doesn’t look like we have yet. If you guys have questions, please shoot them our way real quick before we end this week. If not, we'll be back next week. There we go: “Does the migration tool work for going from CentOS 7 to Rocky 8, or do you advise I go to CentOS 8 first?” And that was kind of what I was asking. I've seen that there is a tool, Michael, that will take you from CentOS 7 to CentOS 8. And I can't remember the name of it, and then you could run the Rocky script, but involved in that, there are at least two reboots from 7 to 8 and then from CentOS to Rocky. I don't know if that would make people nervous.
Michael Young:
There are some tools we are recommending as well as even the RESF community, the Rocky community. The consensus is that the preference, if possible, is to start from scratch, as far as your upgrades. Try not to jump major versions from CentOS 7 – even going from CentOS 7 to CentOS 8, there could be a lot of gotchas there. That's the recommendation, not even to go from a major version, like CentOS 7, straight to CentOS 8, and then go, “Oh, now I'm going to migrate to Rocky,” so not necessarily going to recommend that. There is a tool out there that some have had very good success on. There is an open source tool called ELevate that some have used.
Zane Hamilton:
That's the one I was, yes. ELevate.
Michael Young:
That is an option. But we are still recommending to avoid any potential gotchas. If you're on CentOS 7, It's better to do a fresh install of Rocky 8 and then migrate your apps over. Have a good backup and then migrate over. That's still our recommended path as far as that goes. I don't see a need to first go to CentOS 8 and then do the migrate to Rocky. If you're going to go through all that trouble of jumping from CentOS 7 to CentOS 8, you might as well go straight to Rocky.
Zane Hamilton:
One of the issues I've had in a previous life of moving from major version to major version is you don't notice whenever you first finish. It seemed like everything was fine. You go through that first patching cycle, and things start to explode, and you don't know why. And you have to go back and start digging apart, well, what went wrong, where, and it's usually stuff that got left over or that didn't get brought over as a part of that migration. I have always tried to install new to save myself some pain later, but again, we've had some customers go through this and do just fine. Whatever your risk tolerance is.
Michael Young:
Yep.
Zane Hamilton:
Thank you for the question, John. I appreciate it. Anyone else? If not, we will see you again next week. Michael, thank you very much for all you do and for your time today. Robert, as always, good to see you. Thank you. Thanks, guys.