Optimizing AI & HPC Workloads with Nvidia Partnership

Um we have a a product called Ascender uh which is built on top of AWX in the middle of the stack and then um at the top we have a product called fuzzball which does high performance compute orchestration. Taken together those products are focused on um reducing the friction for building and scaling AI and HPC workloads and kind of re and changing how organizations build and grow those pipelines. >> Excellent. Um, so I I just want to make sure our audience and our audience probably is familiar with this because they're very open source savvy as well, right? So Rocky Linux is an open-source Linux distro. >> Yes.

>> Right. And and CIQ is the was the main benefactor, if you will, of of Rocky coming out there. But CIQ has I don't want to use the word proprietary but commercial distros of Rocky that are geared towards certain you know uh certain use cases such as you know high security kind of setting harded and so forth like that. Does that make sense? >> It does. Yeah. And so to to so to to to expand on that a little bit. So Rocky Linux is maintained by the RESF. Um and it is it is completely open source. Um and CIQ is one of the major founding sponsors of the >> um now CIQ has optimized versions builds on top of the open- source um community version.

Now those are all still open source as well. Um they are CIQ is an open source for company and so Rocky Linux from CIQ which does contain modifications and enhancements still is an open source project but it is maintained by CIQ. >> Excellent. All right. I think I think we got we got that. Let me ask you a question. Beyond Rocky though, beyond, you know, the Linux distro, what else is CIQ about? Beard because there's an AI mission here, right? >> It's not just AI, though AI is obviously the topic of the day. Um, for us, it's about making performant workloads accessible and optimized. Um, one thing that we're seeing in the market today, primarily driven by AI, is that productivity is uh skyrocketing, but costs are skyrocketing as well.

Um, and one of the big reasons for that is there's an enormous amount of waste and inefficiency in how systems get built. And you end up embedding that over time into your infrastructure. And so one of the things we talk about a lot internally is that today AI is very focused on human efficiency. How do you augment and enhance or in some cases replace the work that humans do? Um, we believe that the next generation of that is going to be machine efficiency. How do you make infrastructure much more scalable and efficient? And that is really our core mission. >> Got it. Um, if you don't mind, I want to kind of pivot and talk about some recent news.

>> Yeah. >> Out of CIQ around working with Nvidia. Um, well, I you know it better than I. Why don't you tell tell us what it's about? >> Um, yeah. So, we we recently announced um a an expansion of our our work with Nvidia. So CI Rocky Linux from CIQ is the first Linux distribution authorized to in addition to redistributing uh the CUDA and the CUDA toolkit um the DOA offed drivers from from Nvidia. And the the goal of this partnership is is really simple. It's to make it easier and faster to run and deploy uh machine workloads. And so we believe fundamentally that engineers want to focus on building their AI applications, not on debugging uh kernel integrations with the with the operating system.

And so we are able to redistribute that entire stack and have a much faster time to first inference. We've seen as much as a 9x production in getting set up and running your first workloads, bringing hours down to minutes. >> Got it. Um you I so again our audience they've they've unpacked enough Linux to know we're talking about these are packages right that did they come default now in in the uh CIQ Rocky DRO now the these packages or they're still optional? >> No. So you you can choose not to install them. Um but if you and and not download them um at at runtime but yes if you want them baked in natively um that is one click away.

Let's talk a little bit about CUDA, right? We we spend a fair amount of time here talking about it. There are parts of it and and again, you probably know better than me here, Bejorn, but there are parts of it that are open source that they clearly open sourced, but then there are parts that I don't know how to say this. They're not licensed open source, though they may in fact be quote unquote open. Do do you follow what I'm saying? >> I do. >> How how do you explain that for our audience? Like there's a distinction there. >> There are certain packages within the CUDA toolkit that are fully open and redistributable and then there are others that require um explicit partnership and permission and licensing from Nvidia.

Um now I I don't want to speak on on behalf of Nvidia. I imagine that's a a user experience um decision. They want to make sure that they that when you are using CUDA it is done in a way that is in line with their expectations. And you know I think Nvidia has done a very good job of opening that up over the past several months. um uh CIQ Rocky Rocky Linux from CIQ uh along with uh two other Linux distributions uh was able to redistribute uh CUDA in partnership with Nvidia.

We've now expanded that into DOA oped and I think one of the major focuses from both organizations over the next couple of quarters is going to be how do you lower the barrier for use right how do you get up and running quicker and also how do you make sure that what you're running at scale is validated tested and secure and so from that standpoint that really can only come through deep integrations at the code level and at the package level. >> Got it. I mean you know the The point I was looking to make there was that because someone out there may be saying, "Oh, but I heard cooter is open source.

What, you know, what's the big deal about it being included in CIQ? I could get it." But there were parts that you can't just download right from anywhere. >> Um, let let's dive into DOA off a little bit. How would you describe that to the audience? So, Doka OED is uh Nvidia's replacement for their previous Melanoxed drivers. And so, essentially, this is the core component of their data infrastructure and networking component. So, it allows you to go deeper into the NVIDIA stack and program and work with their data center infrastructure. >> Got it. And you know what? Just for the folks like me a little slow on the uptake, what does OFed stand for?

Bjorn, >> that is a good question. It is um open fabrics enterprise distribution. It is it is it is not often uh actually uh set out like the full the full length. But yes, that is what it stands for. >> Excellent. I caught you. I you know, hey, I I thought you would you know, if I didn't know, you certainly would. And you did. Good for you. You know, let let's do some housekeeping beyond CIQ like for people who maybe want to download or at least look at some of the CIQ download, you know, distros and downloads and so forth. What's it is it a GitHub thing?

Should they go to the CIQ website? If so, what is the CIQ website? What where where should we go? >> Yeah, I recommend that everyone go to uh to ciq.com. just go to ciq.com the main the main site there and go from there. >> Very cool. Um, you know, there are other players in this AI market that I'm trying to say this diplomatically that are a little nervous about Nvidia's position visa vuda while also being the main uh GPU provider. And so there's a lot of talk of other CUDA like if you will toolkits and stacks that you know will allow people to build on whether it's from public cloud providers or you know some of the other layers here.

Any plans in terms of support for that with CIQ or Cud is your coup is the horse that got you here and that's what you're riding. No, we are very much and I'm glad you asked that. We believe sim kind of similar to our open source roots. We believe in kind of meeting our customers where they are. Um so we are proud to partner with Nvidia. Um we are obviously excited to >> uh be able to talk soon about some of our other partnerships that we're working on. Um but we believe that it is not necessarily a winner take all race in the future and we want to make sure to enable customers on whatever hardware and software stacks that they choose.

>> I love it. Bejorn, I want to thank you for coming here up on Techstrong TV today and explaining this all to us. You know, some of these, as I was I was mentioning off camera, there's nuances to open source. There's nuances to distros and and open versus open source and licensing. And of course with AI now we're kind of relearning all this over again as we examine, you know, probably some of the biggest changes since the dot days, right? >> Yeah. >> So I I appreciate it. Continued success with CIQ and come back. You you know you intimated you have more partnership of announcements coming soon.

So you've got an invite to come on and tell us about them. >> I appreciate that, Alan. Wonderful to meet you and thank you for having me on. >> All righty. Bejorn Hovindland, president of CIQ. Go check them out at CIQ. That's CIQ.com. Um, and you know, Rocky Linux is is obviously something that's made their mark in a relatively short time in the industry. So, they're doing something right there. We're going to take a break here on Textron TV. We'll be back in a minute. >>