Announcement From CIQ: Fuzzball Has Been Released!

that a lot we talk about it a lot but what actually makes it the Next Generation so why is it different so at its core uh buzz ball kind of does away with some of the Legacy paradigms and architectures that we've used in high performance Computing for a long time uh and replaces those with a much more modern architecture that's based around the best practices that have been established by places like cloud hyperscale and Enterprise um so fuzzball essentially integrates for example tools like kubernetes and cicd platforms things like that into a platform that is easy to deploy both on-prem or in the cloud and

allows you to write up for example workflows for your users so they can codify the work that they do in Computing and run that on the reconfigurable resources available through the Fuzzball platform uh so that means AI machine learning data analytics simulation and cfd or find an element analysis fuzzball is for all use cases and as I said is the integration between kubernetes and kind of the world of high performance Computing that uh the industry has been looking for for a long time and that's something I want to go back to here in just a second I want to talk about the kubernetes specific piece

but I think we have graduating in the back as well in fact there he is hi everybody good afternoon Greg welcome it's still morning here Zane that's true it is still morning it's not for me but it's for you phone force was just telling us kind of high level what fuzzball is but I want to take a step back and I want to ask you Greg why did you start building fuzzball what brought this to life what made you want to go do this so I spent a large very large portion of my career uh working doing high performance Computing and people that have done

that um no I mean we've been kind of doing the same thing for the last 30-ish almost 30 years now and which isn't necessarily a bad thing I mean when something works right you don't don't fix it if it ain't broke uh but what we started to see uh I think really ever since the Advent of containerization in high performance Computing we started seeing kind of a new way of doing things and we started seeing uh more and different types of of science and research and Computing needs being thrown at these these high performance Computing resources and as a result of that like this this

this architecture that we've been using started to show its its age a little bit it became you know it was a little you know long in the tooth and uh how do we start moving this towards a more modern infrastructure so that was one side of it the other side of it was uh we started seeing more people in Enterprise being interested in something called performance intensive Computing um and when they're looking at performance intensive Computing workloads they're looking at things like AIML and compute and data-driven analytics and and so on and so forth and they're looking at this in terms of well what kind

of an infrastructure do we need to run this on and they immediately went to well well okay let's look at the cloud infrastructure let's look at kubernetes let's look at what everything's offering everything that's being offered out there today but historically Enterprise hasn't had this sort of a need before like most of the time they're doing Services they're doing like um you know infrastructure type requirements which the application requirements of that infrastructure is actually very low this is why virtualization took off so big in Enterprise because you can have hundreds of VMS running simultaneously on a single server when you're talking about performance intensive Computing

you can't do that you really need to be focused on how do you most efficiently and and um how do you run your applications and if you're trying to run 100 at a time you're not going to run them efficiently so how do you start thinking about this in a different way now I went to countless meetings at various Enterprise organizations about four or five years ago now who basically outlined to me they've got an HBC problem to solve and as I looked at you know what they're trying to solve the answer was clear to me go build a Beowulf this is what you need

to build and their response is oh my gosh I remember those architectures back when I was a student in the you know the late 80s um certainly there's a more newer way of doing this that's more in alignment with Howard what you want us to use SSH we're trying to stop using SSH now you want our users running SSH and logging in getting actual shells they don't know how to use Linux I mean this is the kind of responses we would get and um so Enterprise really started going to the tools that they know which are things like kubernetes and kubernetes is great for microservice

it's a fantastic microservice platform I have yet to see an extremely compelling HPC use case using kubernetes it'll come I'm sure someone you know it'll it'll get there um I don't believe it's it's there yet and I believe that you know it's you know the expression you know putting a saddle on a cow that's kind of what it reminds me of and I found some cool pictures I actually asked um some AIS to actually draw me some pictures of saddles on cows it was pretty cool where Greg where is this Forest that could be one of our next webinars maybe a demo we did this

in the past a while ago together um but but to me that's kind of like what what you know doing a high performance Computing and kubernetes is like it's just not designed for it it's fantastic at what it's designed for just that's not what it's designed for so um could you could you do it yeah maybe um I don't think it's been done yet though so when we started looking at how do we modernize high performance Computing we really took the best of all the different Industries the best capabilities the best Technologies and and and ideologies and put them together into a unified solution

that we call fuzzball and in Rose I love the hat that is so cool you remember those in the 80s the ones are on the pencils and you can spin them real fast absolutely can you do that you know I actually have a turning chair but I don't know if it's real fast that's awesome that is awesome so so we started pulling the best kind of of all these different sectors of the these different Technologies within the ecosystems and um and that's where we ended up with fuzzball it is a cloud native on-prem hybrid Computing solution the infrastructure of it sits on top of

kubernetes because that is a microservice platform and fuzzball orchestrate is a microservice solution and um but when we do the compute we do it outside of kubernetes because we wanted to make sure that we're getting the most efficient use of that hardware and kubernetes hasn't proven to be extremely lightweight at this point so it would typically get in the way of of when really running a lot of performance intensive Computing and so we we can build this whole thing up and uh and literally create a service for performance intensive Computing and this isn't our idea I've now seen Gardner and other analysts actually talking about

performance intensive Computing as a service or pick ass for short and that's just awesome is it I'm sorry whoever came up for that acronym really I was kind of wondering about that one guys I saw p-i-c-a-a-s and I was like you gotta be kidding me it's more like a picas like a Picasso oh Picasso in my mind that's where I was going with it it just sounds like you're saying pick-ass really really elegantly just like how the number of letters that you're allowed to put in front of the as a service there is gradually growing as the years go by it used to just be

yes there what next so that's what fuzzball really is and that's why um one of the reasons why you know I started I started thinking about this and the last thing I'll just mention on this thread and then I promise I'll shut up for at least a minute or two is um the the topper on the cake what and and some people may have seen some of my other talks and whatnot know this story but uh you know I was hearing Enterprises say they need something more modern they need something they can do more than just a traditional HPC and honestly I was still kind

of like yeah I got my blinders on this is the right solution you just you you all are just um you know too too cool for for this but yeah then a big huge social media company uh called me in and said we're going to be the biggest HPC system in the world here's how we want to do it and they described an architecture to me that literally just I've never heard of and and I never considered and it was so massive it was so big that um honestly it just it was so cool that it really just kind of put my brain into overdrive

and I joke around about going completely OCD uh thinking about how to actually solve this problem so um started thinking about this um this social media company um you know we even talked about you know me working there and me helping them build this but I actually thought it would be much more interesting to go and build this as a company and offer this back to the world as a solution and um and that's what we did and thus ciq was born um thank you Greg thank you for that I think all of us working at ciq is happy that it exists and it is

happening appreciate that absolutely it looks like we have somebody from Mexico City watching Lanier I appreciate it thank you very much for watching and welcome you you both have mentioned kubernetes in this and I I think that was an important distinction I thank you for diving into that Greg and calling out where kubernetes sits in this and that it's not running the actual compute it's running the orchestration piece of this and it's actually once you get off to do the compute it's different so thank you for clarifying that uh Forest I know you brought some things for us to look at I would love to

for you to show us I feel like we've done some of these over time but I the catalog just keeps getting bigger and bigger of the stuff that you had in the show it does give me just a moment so while Forest is bringing this up and and force don't just just talk over me as soon as you're ready but I want to stress like this is this is not just yet another Beowulf we have completely re-architected the structure of a high performance Computing system and created what we jokingly call HPC 2.0 this really is an entirely new generation of how to think about workload

management workflows um how to think about orchestration and and potentially even orchestration at a much wider scale than what we've been thinking about it so I'm going to be quiet now the forest is ready all right yeah everyone uh so this is the main fuzzball graphical user interface um so we're just going to kind of go through a couple of quick demos of what some workflows look like in this uh one of the big as we've kind of discussed Concepts in Fuzzball is the ability of users to be able to write workflows that codify end-to-end their computational pipelines they can then go and take and

run through fuzzball on you know whenever heterogeneous architectures they happen to have for their underlying compute resources um so this is the main place called GUI um this is what a user in fuzzball would see when they hop onto the system I've gone and done a standard Google SSO login flow before this so just logging in with my uh just standard company email and everything um we have a couple of different things in this that are kind of Worth showing off real quick um these definitions over here are compute definitions that we have available for the fuzzball cluster what we're looking at here is a

deployment of fuzzball running up on AWS eks so obviously most Cloud providers have some type of dedicated kubernetes service that one's aws's when we deploy fuzzball out onto that we can you know leverage obviously those engines that are out there on those public clouds so this is based on AWS eks on the instances that we're going to be provisioning in this demo are different types of AWS instances that we've set up so for example we can use some large uh CPU only instances here um that are the c5n.9x large types we have enabled EFA placement groups stuff like that for optimal MPI on this type

of thing you can also do GPU based instances so in this case we've got a p3.2x large and so on and so forth um over here we can Define secrets and users uh those um I'm gonna more so get to the workflow side of things but this is uh secrets that you can Define within the cluster that essentially allow you to template out on the server side different credentials things like that so you have to keep those in your workflows so it adds security and then users is just essentially the list of users on the cluster uh members owners that type of thing um so

I'm gonna go ahead and run a couple workflows this is what they look like in the fuzzball system you can see I have my whole list of workflows that I previously run here um but we're going to start here in the workflow editor and just go ahead and open a few different use cases uh run them and we'll see how it goes um the first one that we're going to look at is a pretty simple pretty basic one uh this one is gromax computational molecular Dynamics done for you know pharmaceutical drug Discovery all kinds of different things um and fuzzball as I said users can

codify their workflows from end to end so data movements image pulls things like that because fuzzball is based on kubernetes everything is done up container images so each one of these jobs is a separate container image and first of all will end to end orchestrate the execution of this workflow on those resources that we saw to find over here based on what I've told this workflow that it needs to go out and run on um so first off we can Define the concept of a data volume in fuzzball uh so in this case we have the volume here and that's going to reach out to

uh this link on the internet and pull down this file drop it into our data volume at the top level directory with that name um and jobs will then be able to mount that volume you can see the volume name at that pal so that for example in this first job we can untar that tarball that we pulled down over here in the zingras that's got our Benchmark data in it so that's data movement in you can see we have all these jobs that do different things this one's pretty simple just an untarred job you can see that we're pulling from a container registry here

this is kind of one that we use for different testing different uh kind of workflows that type of thing um so in this case we have this container gromac c3r with this tag Apptainer stable on it we're going to reach out to it and use essentially a service account type tooling to get into it one big thing about flexible is how well it integrates with cicd systems so you can see that we can use secrets and credentialservice accounts that type of thing to make it very seamless for in this case you need to just utilize the secret and be able to pull that container into

all these workflows as I need or in all these workflow jobs as I need um over here we have resources so one core 1GB item memory this one's a pretty basic job if we want to look at one that's a little bit more complex uh this one right here um is doing about the same thing as this previous one mounting that volume pulling the same container but we're going to use two cores 14 gigabytes of memory and one Nvidia GPU so this is going to map to uh g4dn.x large instance on AWS so a simple GPU instance but we got the idea that you know

we're running on a GPU buzz ball also supports common paradigms in HPC like multi-node workflows through MPI or task arrays also also called embarrassingly parallel workflows in this case we're running an MPI workflow through openmpi we're going to use two nodes uh and this is the actual you know run Benchmark step of this so we're going to go ahead and pull basically the same resources that we pulled for this but we're going to get two nodes each of these resources on it Fuzzball is going to wire those together up in the cloud and then allow this compute work to proceed on those instances as noted

the fuzzle orchestrate stack is run as microservices through kubernetes so we're not actually doing the batch compute work inside of kubernetes PODS here which has a significant performance benefits and is one of the big kind of best practices that fuzzball leverages for how it uses kubernetes um and then this final job is just uh cutting the logs so uh we'll go ahead and run this we can uh add a name if we want demo go ahead and start this workflow and you'll see that we have a success workflow starts successfully we can go to the status of that um and this is what the actual

workflow uh kind of interface looks like as a workflow is running um so you can see we have all the steps this is our definition uh what that looked like in yaml fuzzball workflows can be written in yaml as well if you prefer to just type them out as opposed to using the interactive editor so all of that everything that we just saw in that graphical interface is encoded within the CML definition here as well and you can see that we're executing on the different parts of that workflow we're creating the volume we're pulling the image uh and we're starting on a file transferring from

the internet so here in a little bit these will start to run uh uh while we're waiting and we're actually gonna let this onto our one uh run really quickly so we can see that go on and then we're gonna um we uh we can talk in the background if we need to while these run but I'm going to kick off a few more and we're gonna just kind of check back into them uh throughout the um webinar here so you can see the antar started this is essentially reached out to AWS with those resource requirements that I gave it uh that one um core

and one gigabyte of memory it spun up a very minimal instance for that uh and has landed the container on that landed the command all that on it run it uh and you there's no logs from tar but we now have that data in there untard um in just a moment on one of these longer running jobs around the terminal into one of these and we'll be able to actually kind of um look around see that data see what's up but that untar one finishes very quickly so um once that comes up we'll take a look in there but in the meantime we're gonna go

ahead and kick off a couple more of these really quickly um let me take a look at uh a little bit of sequencing so I said fuzzball works for a lot of different workflows this is some sequencing genomic sequencing using a common HTC star Sam tools type pipeline in genomics so this is doing a lot of the similar stuff that we just saw in that last one um but this introduces the concept of S3 Ingress in fuzzball so first of all uh can reach out to any S3 API compliant object storage and be able to pull down data from that um so in this case

you can see I've got my key or my key and my key ID here that I've generated from you know aws's authentication type panel I've got my region and then if I wanted to use a different this by default targets AWS S3 but if you wanted to use a different S3 compliant object storage endpoint you could put it in an error um you know there's a couple different Services different things that offer that so it's compatible there and you can see we're doing two ingresses one is a file of fast queue input files so like little genomic reads like base paradel and then this is

a genome that we're pulling down to compare those two you'll notice we also have an egress here an egress um essentially allows us to move data back out of this job once it's done so we have two egresses here one is going to reach into this data volume and pull out the file at results output and upload it to our bucket uh into this bucket name and then those kind of subdirectories in it we have another one that we're also moving these HTC count.txt and then HTC countdemo1.txt um so we've got uh obviously a couple different egresses going on there so a couple different results

files we want to preserve from this workflow um we're pulling down once again another container from our registry it's worth noting that uh fuzzball supports not only just um you know private Registries obviously but any public registry and for that matter any private registry um so you can pull down from Docker Hub you can pull down from the Nvidia NGC uh any number of given private registry Solutions etc etc um I'll show you guys that in a bit the rest of this workflow is is kind of what we've seen before um different commands in this case we're running some Python scripts inside of this to

do a little bit of multi-processing and at the end we're going to wrap up our results into a table so go ahead and start this um let's see here we'll check back in on grow Max really quickly oh and we can see that this uh our second job here prepare Benchmark data is finished um and fuzzball you can set up obviously as uh you kind of saw here this directed a cyclic graph of job dependencies um a better one uh kind of shows off a little bit more of a complex field here but um in fuzzball you can set up for example this requires field

so this job will only run after this one is done and you can set up you know pipeline or workflows of execution there have we gone to this run Benchmark we can see that this is printing us out some logs but furthermore we can get a terminal directly into the instance running bus so if we go ahead and do say Nvidia SMI because we're using a GPU you can see that we've got our Nvidia Sni output here we've got 34 GPU usage 343 megabytes of memory we don't see the exact process name because of how namespacing Works between containers and the host in this case

but it is indeed running so we've got Nvidia SMI we can look at we can check top so for example we've got that GMX MPI running on the cores of this instance that we're on um so you can see GMX MPI going there we can also do things like LS slash data because we attach that data volume to this workflow at slash date or just a workflow job we're on Benchmark and slash data so we can do LS data and you'll see that carball and the unpacked tarball that we set up in some of our previous jobs um and so yeah so that runs we

can do our logs uh that's kind of interesting um you can see those are running there you can go back in here and look at this going and uh yeah first of all uh has support for not only just uh multi-node paradigms in MPI but also uh pgas networks through gas now um so for example you can run the chapel programming language on fuzzball uh and do workflows with that type of thing um just to point out if you want to for example edit the ammo directly in your workflow screen you can do so you can directly type into that if you want to edit

something there um what was that that I heard earlier about cows on a saddle or something like our saddle on a cow or something like that saddle on a cow let's see here uh first of all it's obviously for all use cases including the latest and generative AI data analytics all that type of thing so let's see here subtle on a cow oil painting somebody who's right we have done this similarly before um what we did with Dolly mini back in the day so this is genuine stable diffusion running on fuzzball um here in just a little bit we'll be able to kind of see

a little bit more about this um but for example we're pulling down our uh checkpoint file here for our weights uh test image and then once we're done with this we're going to upload our generated images back to our S3 bucket that we've been kind of working with before and once our results are in these um we'll be able to go look at those through that um yeah so first of all uh obviously for all different types of use cases um it's great for cfd that type of thing this is a little bit more complexible workflow but I believe we just put a video out

on our YouTube channel today that shows this workflow running in action and kind of how the results and stuff come out of that um I'll go ahead and fire it off here and we'll see if it gets done by the end we can see it upload live but we've got like I said a great video on our YouTube of this workflow specifically going right now um we say that our Chapel hello world is finished because it's pretty quick and we have Oliver logs from our four locales because I believe this is a yes four node workflow um as you can see we have all of

those you can see we have all of our logs for example there um as I said it also supports embarrassingly parallel workflows uh I am not a task programmer so this is a little bit of a of a rough script um but we should get results and everything here as expected uh we can obviously do task arrays and things like that embarrassingly parallel workflows and fuzzball um so if you've got you know 10 000 input files and you want to be able to process them all at once we can set this up so we have 12 jobs that are going to be done here on

three instances at once so we can tell fuzzball to spin up three instances matching these resource specifications here um and then we'll start iterating through spells this script kind of uses an FB task ID here sort of like what you might expect with like swarm or something similar to map these tasks around but in this case yeah we'll spin up three instances matching these resource requirements and then we'll start executing 12 tasks on them so go ahead and start this up yeah so these are going um and we are kind of just waiting at this point for these to finish some of these will take

a little bit but we uh actually I do have one more um so as I mentioned uh in fuzzball you can kind of do different uh types of workflows that maybe are a little bit more complex than just these straight line ones that I've seen um in fuzzball our Tech on a on a technical level workflows invisible are directed acyclic graphs so they're essentially a graph that has no cycles and has only one way forward um like you can't go backwards basically uh and so with that you can solidly create like for example in this case this first job create directories will run once that's

done you'll notice that this Retreat query sequence and retrieve database sequences both depend upon this create directories one so both of those jobs will be spawned once this one is done will start running concurrently once both of those are done make blast database will start and you'll see that it depends on both of those and then run blast at the very end of this uh just depends on that so you can imagine this being a little bit more complex I don't think I have my much more complex on one hand unfortunately but you can imagine this being much more complex you can kind of extend

this out add Infinium to create arbitrarily complex workflow graphs of you know whatever complex data processing with tons of different steps that uh you're having to do normally on your hvc resources so yeah um we can see this grow Max is finished here so if we go back and look kind of through this we can see this is all finished execution uh we've got our logs here I always get a little nervous because grow Max has some cheeky little quote but it shares that I always get concerned it's going to be a little bit too much but that I don't want to get rid of

because it seems like it's um like the grow Max Community Values seeing the quote so I keep it around but yeah so you can see we have our grumex results we've got our quote we've got our nanoseconds per day performance um if we cut our full log file you can see that we've got uh everything here kind of showing our CMD instructions GPU support as I kind of scroll through here running on two nodes with total four cores four processing units two compatible gpus so kind of just our standard grommax results that we would expect we then get down here to the bottom your mega

flaps accounting load balancing and then obviously that's our results so um if we want to preserve this file we can upload them down to S3 I don't in this case just because it's not terribly useful um but these are running and uh yeah we can go back and view previous workflows so these are all you'll notice I've been running these for a little bit um making sure things are in order but if we go look at you know for example um any one of these past ones like actually let me just go through these really quickly where's that one uh let's you can see kind

of what the interface looks like for finished failed or canceled workflows um a couple days ago I was working on getting the HBL linpak Benchmark working on fuzzball so for example you know I can go back and retrieve this workflow that I ran a couple of days ago this was 516 so two days ago I can go retrieve my logs from it um I'm gonna do whatever I want here I can go ahead and rerun this directly from that and if we go back to our workflows thing you can see we now have that workflow running here again um so y'all those are uh that's

the the fuzzball oh there is one more thing we can look up uh or another thing here this ball isn't just for uh batch Computing it's also for interactive Computing so in this case we're going to do a simple jupyter notebook type thing um uh so um basically bringing up the Jupiter notebook being able to connect to it from our terminal here normally you know as we've discussed you'd have to SSH into a cluster kind of this is a little bit of a um you know a headache in the process to get connected to one of these uh you'll see on fuzzball I'm able to

port forward this Jupiter notebook server running out on cloud resources directly back to my user terminal here um and how I'm able to just interact with that without having to um directly SSH to the cluster just in my own web browser uh some first of all to support that we have the concept of an isolated Network namespace so I can tick this back and forth and it'll create essentially kind of like a network for this container that it can I can connect to uh and then we're just going to port forward this port back to my local we're running this on a GPU we're going

to kind of see a few AI use cases in this um so go ahead and run that and just to point out I don't think I made it explicit this last one which we can now see is finished and we can unlock screen that one there we go so we can see the logs from our blast workflow here um this is uh obviously ncdi blast doing genomic mapping and just to point out the container we're using in this case is directly something pulled from Docker so this is just ncbi blast directly from there um we can also do things like this is lamps I believe

pulled directly from the uh Nvidia NGC as well we can go ahead and probably run this I might hold off on this one for a moment just until these so yeah so our star sequencing is done uh we'll go look at that in a moment I'm gonna fire this one off so yeah you can see that we're pulling like I said in this case a Docker image down from the Nvidia NGC and then we're gonna land this just on a simple CPU node um because we're just running this on CPU um our grow Max one was on GPU but this one uh just to show

you some CPU only stuff as I noted um just to point this out we have the star sequencing that's done um I just lost my windows there we go and as I mentioned this one has an output that comes from it um let's see here hold on one moment in fact I'll cover up in different bucket and don't want to you know too much now that we're on camera again I was reminded to sit up straight and look professional let's see here surprise you're live yeah here we go let's go back to this that has got to be the next conference like swag or something

we give out I like that idea I agree I don't think Robert would agree but I agree you know I would just I just want to see Robert wearing that oh me too that would be great I'll send him one awesome Reds yeah uh so here on the interface like I said I had to go open up an additional uh directory in the same bucket that we're in here um you can see that we've got for example this HTC countdemo.txt and starresults demo.txt here that have been uploaded here today I don't think there's anything particularly interesting in this file but you can see this is

our like HTC results here um so you can see that from uh foreign that's represented here that we defined in this volume right here we've gone ahead and to this code ciq misc support demo output bucket uploaded those couple of files the HTC countdown of one and then star results demo one um so looking at these we can see these are all starting to finish out uh this task array one I believe I missed the tasks themselves um because I was off explaining other things but uh in this case this is kind of like a fintech type thing this is running some wrapper scripts they're

just doing uh kind of some number munging um it's got it spins up like I said three different instances it lands kind of a python multi-processing library on them called dasc and then starts running um these different tasks that use this python script that utilizes to ask using multiprocessing on that node um so we've got our three nodes these just rerun this one in the background so I can show this sequentially coming out here in a moment but you can see we have all these tasks that respond from this 12.

each one of these has its own set of kind of logs and everything like that we'll grab one of these and you can see that each one of these is just grabbing a uh kind of just a file with like 20 000 numbers on it doing some averaging but you can imagine um you know some fintech Black Shoals numbers stuff like that analyzing you know say 100 200 stocks for back testing all in um you know kind of embarrassingly parallel format or you've got 20 000 input files from your space telescope and you need to process them all sequentially um you know this workflow right

here is a fairly small task array so it's only got three uh no or uh nodes basically that are going to run on it but you can scale this up to 100 a thousand ten thousand and have you know if you've got a million files you could have ten thousand tasks running at once it's um it's fairly extensible and as a built for scale so we can see that hpl is finished uh so we've got kind of our uh um linear algebra system results here and obviously as I mentioned I ran this workflow uh first uh for the first time a couple of days ago

and I was able to just rerun it and now we've got it completed again today here uh lamps CPU NGC this is completed we pull down an input file that's kind of just a common um like the standard hello world input file for lamps and you can see we've got all all of our logs here um I'm not running a GPU enabled node so it kind of complains a little bit about not finding Cuda but you can see we have all of our results um CPU usage uh Etc and Pi tasks this is just running on some smaller instances not BFA enabled ones um so

we kind of see that um yeah we've got just 32 cores there um but yeah if we wanted to scale this up we could run this out on EFI enabled instances we can run this out on um uh yeah the best essentially and best practices that there are for uh MPI on the cloud and AWS um this right here we should jump into really quickly this is The jupyter Notebook workflow that we've been working on uh so you can see this is started and we've got this link here but if I go oops you can see that if I go to my logs here and

I try to open this I get this problem loading page if I stop sharing my screen for a second I will give you all a quick view of what the other way you can use fuzzball looks like um so one of the things about Fuzzball is that it's entirely API driven obviously this gives us the great benefits around not sshing that type of thing but it also being API driven means that we can pretty easily wrap most of the um uh actions that a user can take versus a cluster and Fuzzball um we can wrap those in essentially you know whatever raffle we want so

we've got our graphical user interface but we also have this CLI that allows you to do all of the same things that you just saw me do on our GUI but through a graph or through a CLI based form so I'll go ahead and do fuzzball context login this allows me to log into the cluster context this is going to open um so I'm trying to find my stream yard really quickly service so this opens when we control click that I get this right here so this also goes back to a question that George asked earlier saying that the GUI looks amazing but do I

have to use it and I think it's kind of what you're alluding to now of course oh I see what's going on here um my string yard is on Seminole sorry here we go just don't hit back the GUI is amazing and it's been fun to watch uh people who typically don't use gui's actually start using the GUI but you do not have to use the GUI to do this you can actually do this all via command line like Boris was talking about any alluded to very shortly there you can do everything via command line if you'd like this one I don't know I don't

think extremely hard is like giving me the option to open the window that I'm expecting it to give me to open here uh so I think I'm not sure why that's doing that but here in just a moment I'll go ahead and do this SSL flow um and I'm able to this will complete there we go okay so like I said that opens up you control click this it opens up just the standard Google SSO select what Google account you want to log in with and once we're in here we can do fuzzball workflow list for example uh tail 10.

and here are all the workflows that we've just been running as a part of this kind of webinar so there's star sequencing Chapel Hello World um the stable diffusion one which is finished uh open from Power abuse there's everything we've been running um we can go ahead and do a fuzzball uh let's see workflow port forward click and type grab the ID of the workflow from here and then the name of the job and the workflow they want to connect to in the ports so you can see we've now got this listening on our remote I'll go ahead and stop the screen here open up

this um and wherever we at so back in this window here you can see we had a problem loading page but once I yes I am trying the right window cool um once I go ahead and refresh you can see that we're connected in here as we would expect uh so this is as I said did you put a notebook instance running out on some Cloud resources on AWS we've got a workspace that we can go into here with some notebooks um so this is gonna for example this is just AI training with pi torch we can go ahead and run through these we're going

to train a simple model on the sidebar 10 data set um we've got our data set downloaded and extracted go ahead and show a couple of the images from that data set um since we've got a plain bird dog bird go ahead and set up the code for the model itself and then set up a little bit more code and then this right here actually starts training this model so if we go over here um as we might want to in our Jupiter notebook we can do a quick Nvidia some line and we can see 28 GPU usage 947 megabytes of memory again because of

namespacing we don't see the exact process but we can do top and some Python 3 running there um if we go back over here we can see our loss values starting to be printed from this training if we go over here refresh this is we can also access that same information through here um so you can see we've got that python3 there Nvidia SMI space and we've got there are notebooks and everything as I just showed in the other window this will finish training here in a little bit um and you can see we've got kind of our 302 and stuff like that uh coming

here from the output of this so like this right there some of these different bits of information being captured in the logs um and yeah once this is trained we can pretty easily uh just go right over here download this directly to our laptop once we save right here the scifar network dot pytorch file will be able to really easily sooner you might be able to see this but if I go back over here you can see we have sci-fi net.pth and go ahead and download and I got my sci-fi net file I'm not sure if you can see that pop up but I can

go ahead and download it that goes right to my local um and yeah this will stay alive for as long as we set the timeout value right here too so this will stay alive for 30 minutes before it'll close um and go back over here and check our of this one right here so dask oh I missed it again my apologies well you can see our last task here running on this task one um so that uh normally this has been spin up and you can kind of see them in uh actually but I've kind of missed that anyway you can see our last task

running there uh you can see we've also got for example stable diffusion is finished we've gone ahead and uploaded our images um if we go into leave this bucket right here I should be able to refresh yeah and then we have our blast results from that blast workflow that we ran earlier and we've got our stable diffusion images so I'll go ahead and download these um and yeah you can see the only thing that's still running is the Alex net notebook because that uh is set to run just indefinitely for about a half hour um but everything else we ran here is finished correctly um

task is once again finished uh lamps is finished hpl blast um oh open Chrome power but you also finished awesome so this is a cfd one we're doing a whole bunch of different cfd tasks this for example uh has been uploaded to this S3 bucket our final photo result if I can just open it yeah I can there we go so there's for example our crash test image result you can see a video of this as well on our YouTube channel um so there's that uh I believe our stable diffusion images have downloaded so let me pre-screen those really quickly I don't see many Saddles

but we do indeed have our results as expected thank you for some cows there um oil painting like uh yeah okay cool um that's about what I have uh for the demo here today um that is a fairly comprehensive view of what you can do with the fuzzball system um we've got a lot uh of different stuff that we're working on but this is uh this is Osmond so excellent thank you for watching my apologies if that was a little long-winded oh it's good it's good it's the fastest full training of fuzzball I've ever seen excellent thank you for us we do have several questions

to get to yeah absolutely throwing those up uh so Nicholas asked if kubernetes doesn't do the compute why add the overhead of kubernetes at all instead of just traditional RPM Dev install Services great question thank you Nicholas for that so fuzzball is built out of uh right now two pieces and there's a third one coming the the base piece is called substrate this what's wrong this is what runs on the compute cluster itself fuzzball orchestrate is what orchestrates all the different substrate instances to do all the different things that you saw for us just demonstrate everything from Ingress of data managing the workflows and when

I say the workflows it's much more than just what you have in your uh yaml file there's a lot of context in that in in the workflow that fuzzball has to manage um again all the Ingress the egress the volume setup setting up containers downloading containers building the images for those containers persisting and storing those containers just to say a few there's a lot of things that it has to do fuzzball itself or fuzzball orchestrate of it as itself runs as a microservice uh platform so there's a lot of different pieces to it and they all fit together via microservices so being that fun osball

orchestrate is a microservice or a cloud native platform it wants to run on some some sort of microservice solution and kubernetes is is just the one that most people are uh very comfortable with and very familiar with at this point uh also many of the clouds will offer kubernetes services so it's very easy to stand up fuzzball orchestrate on top of these various services so once you have this fuzzball service running on top of kubernetes you can now start running workloads so one other piece as well is I believe Forest you ran all of these different workflows in the in AWS in AWS you can

kind of think of it as it scales as needed right and it'll bring so every time Forest would run run jobs uh it would actually spin up the appropriate number of of instances run those on the right type of instances and then tear them down when it's all done you're running this on-prem you have a finite number of of compute resource so it will appropriately schedule we didn't talk about the scheduling features but it will be able to schedule all of those workflows appropriately for running on an on-prem solution which again the scheduling parameters of that is very different than scheduling in the cloud hopefully

I answered that um fuzzball substrate is a traditional RPM Dev installed service fuzzball orchestrate is just a lot more complicated I like that it's just a lot more complicated I'll get into that later that actually depends true um James thank you for uh posting a question here and he says are you showing this in the cloud today and you mentioned hybrid Cloud will this be a true hybrid Cloud platform so I just mentioned that there's two pieces of fuzzball that we're showing off today and there's a third so the substrate at the bottom fuzzball orchestrate which orchestrates all the different substrates and then there's fuzzball

Federate which sits on top of orchestrate because as Forest mentioned there's no SSH involved in this the entire thing is API driven it makes it very easy now for us to extend fuzzball clusters and unite them so to the question yes we can actually absolutely you can run a fuzzball orchestrate instance in AWS in one availability Zone a fuzzball orchestrate instance and a second availability Zone Fuzzball orchestrate instance in a different cloud and on-prem and then Federate them all together and when you do that level of federation fuzzball or fuzzball Federate is going to be making a meta orchestration decisions based on things like uh

resource availability cost how much does it cost to run so obviously if you've got on-prem resources it may be much cheaper to run than having to go spin up Cloud instances and then lastly data where is that data and when you look at all of these together you actually get some really interesting possibilities in terms of scheduling so as an example well it's cheaper to run on-prem but my data is up in S3 all right now you just have to start making decisions on when is that when and how is that job going to run and those decisions are tuned via policies by the organization

the organization will control that we today we are releasing fuzzball substrate and fuzzball orchestrate uh fuzzball Federate keep an eye out it's coming soon I like that it's a good tease oh another question from James awesome does fuzzball know what resources you have available or are you guessing or what if you say I want this resource and that one like you were doing Forest but then that resource actually doesn't exist or it's being used of course you want to grab that one yeah yeah so uh fuzzball does know what resources you have available in the cloud as I believe I showed at the very start

of that there's definitions that you can set that map to for example different instance types out on the cloud provider so you can set up different compute node definitions for example that map to different AWS instance types like you know their P3 GPU series um their C uh c5n like CPU series um you can like I said essentially map those two different instance types on-prem um fuzzball becomes aware of what resources it has uh kind of as a part of the bootstrapping and setup process with orchestrate and substrate there um so it kind of is able to programmatically determine what it's aware of what's out

there what resources are on those nodes Etc if you say I want this but that resource doesn't exist uh in the cloud you will essentially just get a workflow failure that says no sufficient uh provision definitions exist and you'll have to head to your sysadmin and get a definition put to that um if that so that's just if like your system doesn't have a matching resource defined in it it does have a matching resource defined uh it's as Greg mentioned gonna just reach right out to the cloud provider um provision that instance type live and then route it back to the workflow to use the

compute resources for that um on-prem and Ascension does the same thing it takes some of the resources that are available and assigns them to the workflow um but yeah so first of all is aware of what you have available you can Define that you can also you know discover some of that on its own and then if you say I want this and it doesn't exist but it yeah but it knows how to find it it'll go spin it up or like get some of it but if it doesn't exist and you don't have it defined you just got an error as you would expect

get some different nodes working thank you for us so I think that kind of leads into the next question from silly yeah what is the overhead like on this since you're actually putting something on each compute node what is the overhead like so this goes back to oh uh I guess forest and I are tag teaming I'll take this one jump in if there's anything else you want to add um it kind of goes back to one of the earlier questions about kubernetes versus you know why I use kubernetes and and versus not so a lot of people they're trying to solve these problems are

actually trying to go the other direction where they're basically saying we know what kubernetes is like we know how that operates let's just go run everything in kubernetes uh a lot of the feedback we got is kubernetes is just way too uh too much overhead and you're not getting a lot of the performance out of the underlying resource due to kubernetes just honestly just getting in the way it's just too big uh so uh we spent a lot of effort and this is why fuzzball orchestrates sits on kubernetes while fuzzball substrate does not fuzzball substrate takes a lot of the experience that we glean from

you know open source you know applications like Singularity and Apptainer and um and and knows how to run these containers and know how to run this infrastructure in a way that gets out of the way and gives you direct access to that underlying resource and is the most efficient way is absolutely possible so we have Benchmark fuzzball in a variety of different ways at this point and we are getting bare metal speed always thank you thank you really quickly I know we're very close to time but just to elaborate again we've um just to put some numbers on kind of performance and overhead we run

benchmarks on some puzzle clusters like hplai we've found that we've got um the numbers out of those that we've been told by different uh you know manufacturer and that type of thing in that case that we should get out of that Benchmark um so we've seen that fuzzball uh provably doesn't add overhead to your hvc we've also run up in the cloud comparing uh doing um kind of benchmarks around gromax comparing to some of the numbers AWS has published and we are dead on to what's out there so this is uh as we've noted this adds no performance overhead and you know like a more

standard kubernetes installation does and we've got you know numbers that we've had to uh to back the our overhead is minimal appreciate that thank you for us oh Ian what's up Ian thanks for watching okay so I know that we're like at time of like where how we usually do about an hour but this is really exciting this is our release that we've been working on for years and so we're going to stay on and answer a few more questions okay Ian for those who are familiar with workflow solutions for the life sciences for example um Cromwell I'm not going to read that there blah

blah blah is there any possibility of a brief comparison or contrast to better appreciate fuzzball other than just the name thank you again it is a cool name Forrest you want to take it you want me to my understanding around K and uh like next flow things like that or that those are mostly uh workflow description languages that give you a very robust way to codify a workflow but fuzzball includes a massive software platform back end that actually goes ahead and doesn't just provide a workflow engine which is one of the micro Services that's a part of the fuzzball orchestrate stock there's a workflow engine

that takes in users workflows parses them out sends the information um to the different parts of the uh the other microservices that it needs to be in so overall I would say the biggest difference is that buzz ball versus like next flow that type of thing Fuzzball is not just a workflow description language it's an entire kubernetes based platform that you can deploy itself as a cluster not just take like workflows and run it versus existing resources fuzzball is its own entire Computing platform uh platform that a workflow engine like that is just one component the other thing I would just mention as well is

there's nothing stopping you as a matter of fact it's encouraged to use those workflow managers inside of fuzzball so you can absolutely use that you can provision out your resources with fuzzball and then leverage all of those same tooling that you're already familiar with I've done things like for example taking uh you know existing um next flow workflows and converted them fairly simply over into fuzzball workflow formats as well so it's all kind of translatable Fairly easily we can also in theory next flows next flow is a little specific as well because it can actually spin up Cloud resources and interact with clouds so we've

actually brainstormed in terms of can we leverage next flow as a fuzzball client and so that may also come in the future very interesting Theory we haven't actually done it yet so Theory it should work by the way hey Ian great to see you well kind of so here's a good question that Nicholas asks uh will fuzzball be free open source with the option for paid support likes to learn yeah we give this one a lot I'll let Greg answer that one yeah so um it is Our intention to open source fuzzball but we're not going to open source it on day one we put

a lot of research into this a lot of work into this and to be blunt some of the feedback that we got from uh some of uh the other companies in HPC is we're looking forward for you to open source this so we can go and run this and and sell it to our customers again with not including us so we are not open sourcing it simply because we want to encourage good behavior and good Partnerships with our our wonderful Partners so um that that's our initial goal as a purchaser of this we've got no issue whatsoever with providing source code if that's something that

you're interested in and be happy to provide source code licenses thank you Greg yeah and Ian's back okay uh enough of that thank you I'm getting a great sense from the excellent UI yes it is but I am gapping on the execution overall architecture other than use of containers from any repo yeah there's a lot in in terms of the amount of architecture that we built into this so again it starts at substrate working up from there substrate is a container run time um even though we took and leverage a lot of the knowledge we had from Singularity and Apptainer uh we didn't use any

of that code actually we it is a complete new uh container run time specifically about um and leveraging apis to control it completely so it is an API based container runtime that API uses something uh around the idea of leasing so you can say you can make a request to substrate uh I I'm interested in this eight gpus 24 cores this much memory and it's on this architecture and so on and so forth and substrate will say I have that available here's a lease I can provide and so on and so forth now orchestrate will manage all of the substrate instances for you so as

workflows come in orchestrate will know what each one of the substrate instances are doing but at the end of the day each substrate instance itself is the the source of Truth so it will confirm and check and then it builds a a graph of all of the different places and all these different workflows can run it breaks apart the workflows and then runs everything on that that that or that platform again it also does data management so it'll do you know pulling in data volume management managing the volume it runs on Parallel storage as well as straight NFS as well as local different types of

volumes we can manage and we can build does all of that embedded into the platform so it is an incredibly robust compute and orchestration platform specifically for performance intensive computing as of service thank you we have more keep going so Nicholas asks again what kind of metrics are available to admins so he's looking for something like xmod slurp what do we have in fuzzball uh of course you want that one or you want me to um uh the metrics back on the side of it is the latest status of that is a little bit more of an engineering question I know that we have for

example um wired up our fuzzball installs to grafana um in different common bonding tools like that uh and we've been able to go in and see um you know how the cluster is running on eks all that stuff um so the best thing I can say here is that it yeah works with um for example one new platforms like rafana to ingest the logs from a puzzball cluster and put those into a searchable format Greg I don't know if you have anything to add there yeah I would just say is is well from the uh administrative side it's using kubernetes it's using a very uh

more traditional Enterprise stack on the orchestrate side so all of the typical tooling that you have for monitoring at kubernetes cluster you can use on the on the orchestrate side on the substrate side we're as close to bare metal is absolutely possible and in that side you you could just use fuzzball in terms of what it is providing which to be clear it's not a huge amount today it's enough that you can get by you can figure out what everything's happening and whatnot but you can't do kind of a high level profiling or real-time metrics or or analysis on what jobs are doing through that

if you wanted to do that there's there's a variety of solutions you can use as you mentioned um really it's just then a matter of just tallying that data back up and how do you want to present that back to the users but yeah buzzwell's very very flexible with regards to that okay hey are you guys going to how do you pronounce this Isaac yeah I see what is that so um the International supercomputing Conference so it's it's the Europe version of super Computing in a nutshell um fantastic conference um I'm not planning on going and I think we do what we are sending some

people um but uh but yeah we're still figuring that out there will be at least several of us over there um we're we're pretty big company at this point well big right I guess it's all relative for me it's a pretty big company and um you know we're at about 85 people right now so and we we are international so you know we do have a number of people in Europe so we'll probably send some people from the United States over as well as have some of our European contingent uh join us over there so yes but not me I I have maybe a silly

question Greg does so does fuzzball really take away the need to have a head node I thought yeah you can just get rid of that is that gone now yeah it's complete it it is gone in a traditional sense so um what Rose is asking is in a traditional Beowulf architecture we typically see things like um uh you have a control node sitting up on top and you've got all of these these compute or sub nodes sitting below that control node acts as an interactive server where people will SSH into that that control node and then do all their kind of major interactive work and

then interact with the scheduler that will then run the jobs on the compute resource on all those compute nodes uh fuzzball does not have the same architecture and this is what I was referring to earlier Fuzzball is a new way of looking and thinking about high performance Computing uh systems and there is no kind of control or interactive system as as we've we've had in the past now what we have is the notion of apis and there's a number of different interfaces and tooling you can use against those apis and terms in HPC says system more into a Computing Appliance really just a massive scaled

Computing Appliance so you're now interacting with this giant Computing compliance via these apis which again users don't usually interact with apis but they're interacting with the clients that are communicating over these apis so you can use a fuzzball cluster even no matter wherever it is in the world via your laptop or via your workstation and again just connecting and leveraging these apis so that brings up a question about security which I imagine just because I've known you for a little while that that has been that's top of mind like making sure that I mean I imagine that fuzzball is more secure but can you can

you touch on that yeah yeah so one of the biggest um difficulties or challenges with securing a traditional HPC system is the fact that you are allowing over SSH over a secure shell you're allowing people to have full access to that to that interactive resource up on top and then to the compute resources when their jobs are running and it's very hard to validate that a user coming in is in fact the right user right this started off thinking about passwords as an example right a password is something that only you know well that's the Hope right only you know that and then you can

use that password to go get access to the system but what people found is this well it's pretty easy to either brute force or hack passwords and get in so now all of a sudden people are getting in with with you know your private password um but they're they're they're hacking it right so then we started using one-time passwords or or additional you know tokens to get in so multi-factor tokens or multi-factor MFA multi-factor authentication um to get into these systems but again that just kind of solved the problem for a little while giving users full access to pretty much do whatever they want is

a very difficult thing to secure fuzzball because everything is going through those dsls that forest was showing those workflows everything is defined you can reproduce you can replay and you can audit every single action that's happened as a matter of fact all of the data Ingress and egress could could be completely locked down to the point where a we're actually getting authentication or authorization for any sort of i o going in or out of the system via another source so if if a if a lab or or or a classified facility has limitation in terms of where they can pull data from or who can

pull that data or what applications are allowed to be used against that data we can integrate that via the Ingress and egress and we can also lock down when the job is running we take it off the network so they CA they have to do all of their Ingress and egress via that section of the DSL so we can now start securing these workflows in a way that honestly we just haven't been able to do before and it gives a lot of capability in terms of of management right this is the type of thing that a CSO or or somebody who's really focused on the

security side is going to be uh very interested in making sure that everything is is auditable it's reproducible and it's secure not just secure from a hacker perspective but secure from a supply chain perspective keep in mind everything that we're running now is also coming out of something that can be verified and validated each one of those containers can be validated so excuse me so we can very easily have complete uh transparency on everything that's happening and then trust on everything that's happening so yeah security is definitely top of mind yeah thank you for that Greg I know I feel like we've uh kept you

a little bit longer I think you might have a couple other meetings to go to but thank you so much for coming on and sharing with us the story and building this amazing company and you know releasing fuzzball and making it fun and awesome um I really appreciate your your time being here and um I don't know Zane if you have something else to say but you guys we are open we are open for business we are ready for you know your your comments your questions you can go to our website you can schedule with us and and dive a little bit deeper on to

how fuzzball can really benefit uh your environment and how we can work together and we'd love to chat with you so leave a comment leave a like say hello um share with your friends schedule a meeting we're here for you absolutely thank you all for watching and I want to thank the engineering team I know you guys have put a lot of time and a lot of effort into this so we really appreciate it for us I appreciate all the work that you've done I know you spent a lot of time working on these workflows to be able to show people and show off fuzzball

and it's been fun watching over the last almost two years now play with this thing and watch it grow so really appreciate everybody out there like Rose said go uh like subscribe if you want to talk to us more about this go to the fuzzball page and sign up and give us some time appreciate it good to see you guys