Answering Your Questions - Fuzzball HPC 2.0

that it's just a lot more complicated let's get into that later that actually depends true um James thank you for uh posting a question here and he says are you showing this in the cloud today and you mentioned hybrid Cloud will this be a true hybrid Cloud platform so I just mentioned that there's two pieces of fuzzball that we're showing off today and there's a third the substrate at the bottom Fuzzball orchestrate which orchestrates all the different substrates and then there's fuzzball Federate which sits on top of orchestrate because as Forest mentioned there's no SSH involved in this the entire thing is API driven it

makes it very easy now for us to extend Fuzzball clusters and unite them so to the question yes we can actually absolutely you can run a fuzzball orchestrate instance in AWS in one availability Zone a fuzzball orchestrate instance in a second availability Zone Fuzzball orchestrate instance in a different cloud and on-prem and then Federate them all together and when you do that level of federation buzz ball or fuzzball Federate is going to be making a meta orchestration decisions based on things like uh resource availability cost how much does it cost to run so obviously if you've got on-prem resources it may be much cheaper to

run than having to go spin up Cloud instances and then lastly data where's that data and when you look at all of these together you actually get some really interesting possibilities in terms of scheduling so as an example well it's cheaper to run on-prem but my data is up in S3 all right now you just have to start making decisions on when is that when and how is that job going to run and those decisions are tuned via policies by the organization the organization will control that we today we are releasing fuzzball substrate and fuzzball orchestrate uh fuzzball Federate keep an eye out it's coming

soon oh another question from James awesome does fuzzball know what resources you have available or are you guessing or what if you say I want this resource and that one like you were doing Forest but then that resource actually doesn't exist or it's being used of course you want to grab that one yeah yeah so uh fuzzball does know what resources you have available in the cloud as I believe I showed at the very start of that there's definitions that you can set that map to for example different instance types out on the cloud provider so you can set up different compute node definitions for

example that map to different AWS instance types like you know their P3 GPU series um their C uh c5n like CPU series um you can like I said essentially map those two different instance types on-prem fuzzball becomes aware of what resources it has kind of as a part of the bootstrapping and setup process with orchestrate and substrate there um so it kind of is able to grammatically determine what it's aware of what's out there what resources are on those nodes Etc if you say I want this but that resource doesn't exist uh in the cloud you will essentially just get a workflow failure that says

no sufficient uh you know provision definitions exist and you'll have to head to your sysadmin and get a definition put to that um if that so that's just if like your system doesn't have a matching resource defined in it it does have a matching resource to find uh it's as Greg mentioned gonna just reach right out to the cloud provider um provision that instance type live and then route it back to the workflow to use the compute resources for that um on-prem and Ascension does the same thing it takes some of the resources that are available and assigns them to the workflow um but yeah

so fuzzball is aware of what you have available you can Define that you can also you know discover some of that on its own and then if you say I want this and it doesn't exist but it yeah but it knows how to find it it'll go in and out or like get some of it but if it doesn't exist and you don't have it defined you just got an error as you would expect to get some different nodes working thank you for us so I think that kind of leads into the next question from silly yeah what is the overhead like on this since

you're actually putting something on each compute node what is the overhead like so this goes back to oh uh I guess Forest denier tag teaming I'll take this one it'll work jump in if there's anything else you want to add um it kind of goes back to one of the earlier questions about kubernetes versus you know why I use kubernetes and and versus not so a lot of people that are trying to solve these problems are actually trying to go the other direction where they're basically saying we know what kubernetes is like we know how that operates let's just go run everything in kubernetes uh

a lot of the feedback we got is kubernetes is just way too uh too much overhead and you're not getting a lot of the performance out of the underlying resource due to kubernetes just honestly just getting in the way it's just too big uh so uh we spent a lot of effort and this is why Fuzzball orchestrate sits on kubernetes while fuzzball substrate does not Buzz Ball's substrate takes a lot of the experience that we glean from you know open source you know applications like Singularity and Apptainer and um and knows how to run these containers and know how to run this infrastructure in a

way that gets out of the way and gives you direct access to that underlying resource and is the most most efficient way is absolutely possible so we have Benchmark fuzzball in a variety of different ways at this point and we are getting bare metal speed always thank you really quickly I know we're very close to time but just to elaborate we've just put some numbers on kind of performance and overhead we run benchmarks on some puzzle clusters like hplai we've found that we've got um the numbers out of those that we've been told by different uh you know manufacturer and that type of thing in

that case that we should get out of that Benchmark so we've seen that fuzzball uh provably doesn't add overhead to your HPC we've also run up in the cloud comparing uh doing um kind of benchmarks around gromax comparing to some of the numbers AWS has published and we are dead on to what's out there so this is uh as we've noted this adds no performance overhead and you know like a more standard kubernetes installation does and we've got you know numbers that we've had to uh to back the our overhead is minimal appreciate that thank you for us oh Ian what's up Ian thanks for

watching okay so I know that we're like at time of like where how we usually do about an hour but this is really exciting this is our release that we've been working on for years and so we're gonna stay on and answer a few more questions okay Ian for those who are familiar with workflow solutions for the life sciences for example um Cromwell I'm not going to read that there blah blah blah is there any possibility of a brief comparison or contrast to better appreciate fuzzball other than just the name thank you again it is a cool name Forrest you want to take it you

want me to my understanding around K and uh like next flow things like that or that those are mostly uh workflow description languages that give you a very robust way to codify a workflow but fuzzball includes a massive software platform back end that actually goes ahead and doesn't just provide a workflow engine which is one of the micro Services that's a part of the fuzzball orchestrate stack there's a workflow engine that takes in users workflows parses them out sends the information um to the different parts of the the other microservices that it needs to be in so overall I would say the biggest difference is

that buzz ball versus like next flow that type of thing first of all is not just a workflow description language it's an entire kubernetes based platform that you can deploy itself as a cluster not just take like workflows and run it versus existing resources fuzzball is its own entire Computing platform uh platform that a workflow engine like that is just one component the other thing I would just mention as well is there's nothing stopping you as a matter of fact it's encouraged to use those workflow managers inside of fuzzball so you can absolutely use that you can provision out your resources with fuzzball and then

leverage all of those same tooling that you're already familiar with I've done things like for example taking uh you know existing next flow workflows and converted them fairly simply over into fuzzball workflow formats as well so it's all kind of translatable Fairly easily we can also in theory next flow is next flow is a little specific as well because it can actually spin up Cloud resources and interact with clouds so we've actually brainstormed in terms of can we leverage next flow as a buzz ball client and so that may also come in the future very interesting Theory we haven't actually done it yet so Theory

it should work by the way hey Ian great to see you well kind of so here's a good question that Nicholas asks uh will fuzzball be free open source with the option for paid support likes to learn we give this one a like I'll let Greg answer that one yeah so um it is Our intention to open source fuzzball but we're not going to open source it on day one we put a lot of research into this a lot of work into this and to be blunt some of the feedback that we got from uh some of uh the other companies in HPC is we're

looking forward for you to open sources so we can go and run this and and sell it to our customers again with not including us so we are not open sourcing it simply because we want to encourage good behavior and good Partnerships with our our wonderful Partners so um that that's our initial goal as a purchaser of this we've got no issue whatsoever with providing source code if that's something that you're interested in I'd be happy to provide source code licenses thank you Greg yeah okay uh enough of that thank you I'm getting a great sense from the excellent UI yes it is but I

am gapping on the execution and overall architecture other than use of containers from any repo yeah there's a lot in in terms of the amount of architecture that we built into this so again it starts at substrate working up from there substrate is a container run time um even though we took and leverage a lot of the knowledge we had from Singularity and Apptainer uh we didn't use any of that code actually we it is a complete new uh container run time specifically about um and leveraging apis to control it completely so it is an API based container runtime that API uses something uh around

the idea of leasing so you can say you can make a request to substrate uh I I'm interested in this eight gpus 24 cores this much memory and it's on this architecture and so on and so forth and substrate will say I have that available here's a lease I can provide and so on and so forth now orchestrate will manage all of the substrate instances for you because workflows come in orchestrate will know what each one of the substrate instances are doing but at the end of the day each substrate instance itself is um the the source of Truth so it will confirm and check

and then it builds a a graph of all of the different places and all these different workflows can run it breaks apart the workflows and then runs everything on that that that or that platform again it also does data management so it'll do you know pulling in data volume management managing the volume it runs on Parallel storage as well as straight NFS as well as local different types of volumes we can manage and we can build does all of that embedded into the platform so it is an incredibly robust compute and orchestration platform specifically for performance intensive computing as of service thank you we have

more keep going so Nicholas asked again what kind of metrics are available to admins so he's looking for something like xmod slurp what do we have in fuzzball uh of course you want that one or you want me to um uh the metrics back end side of it is the latest status of that is a little bit more of an engineering question I know that we have for example um wired up our fuzzball installs to grafana in different common blogging tools like that and we've been able to go in and see um you know how the cluster is running on eks all that stuff so

the best thing I can say here is that it yeah works with um for example modeling platforms like rafana to ingest the logs from a fuzzball clustering put those into a searchable format Dragon if you have anything out of there yeah I would just say is as well from the administrative side it's using kubernetes it's using a very a more traditional Enterprise stack on the orchestrate side so all of the typical tooling that you have for monitoring at kubernetes cluster you can use on the on the orchestrate side on the substrate side we're as close to bare metal is absolutely possible and in that side

you you could just use fuzzball in terms of what it is providing which to be clear it's not a huge amount today it's enough that you can get by you can figure out what everything's happening and whatnot but you can't do kind of a high level profiling or real-time metrics or or analysis on what jobs are doing through that if you wanted to do that there's a variety of solutions you can use as you mentioned um really it's just then a matter of just tallying that data back up and how do you want to present that back to the users but yeah buzzwell's very very

flexible with regards to that okay hey are you guys going to how do you pronounce this Isaac yeah I see what is that so um the International supercomputing Conference so it's it's the Europe version of super Computing in a nutshell um fantastic conference um I'm not planning on going and I think we do what we are sending some people um but uh but yeah we're still figuring that out there will be at least several of us over there um we're we're pretty big company at this point well big right I guess it's all relative for me it's a pretty big company and um you know

we're at about 85 people right now so and we we are international so you know we do have a number of people in Europe so we'll probably send some people from the United States over as well as have some of our European contingent uh join us over there so yes but Nami I I have maybe a silly question Greg so does fuzzball really take away the need to have a head node did that yeah people just get rid of that is that gone now yeah it's complete it it is gone in a traditional sense so um what Rose is asking is in a traditional Beowulf

architecture we typically see things like um uh you have a control node sitting up on top and you've got all of these these compute or sub nodes sitting below that control node acts as an interactive server where people will SSH into that that control node and then do all their kind of major interactive work and then interact with the scheduler that will then run the jobs on the compute resource on all those compute nodes uh fuzzball does not have the same architecture and this is what I was referring to earlier Fuzzball is a new way of looking and thinking about high performance Computing uh systems

and there is no kind of control or interactive system as as we've we've had in the past now what we have is the notion of apis and there's a number of different interfaces and tooling you can use against those apis and terms in HPC says system more into a Computing Appliance really just a massive scaled Computing Appliance so you're now interacting with this giant Computing compliance via these apis which again users don't usually interact with apis but they're interacting with the clients that are communicating over these apis so you can use a fuzzball cluster even no matter wherever it is in the world via your

laptop or via your workstation and again just connecting and leveraging these apis so that brings up a question about security which I imagine just because I've known you for a little while that that has been that's top of mind like making sure that I mean I imagine that fuzzball is more secure but can you can you touch on that yeah yeah so one of the biggest um difficulties or challenges with securing a traditional HPC system is the fact that you are allowing over SSH over a secure shell you're allowing people to have full access to that to that interactive resource up on top and then

to the compute resources when their jobs are running and it's very hard to validate that a user coming in is in fact the right user right it just started off thinking about passwords as an example right a password is something that only you know well that's the Hope right only you know that and then you can use that password to go get access to the system but what people found is this well it's pretty easy to either brute force or hack passwords and get in so now all of a sudden people are getting in with with you know your private password um but they're they're

they're hacking it right so then we started using one-time passwords or or additional you know tokens to get in so multi-factor tokens or multi-factor MFA multi-factor authentication to get into these systems but again that's just kind of solved the problem for a little while giving users full access to pretty much do whatever they want is a very difficult thing to secure Fuzzball because everything is going through those dsls that forest was showing those workflows everything is defined you can reproduce you can replay and you can audit every single action that's happened as a matter of fact all of the data Ingress and egress could could

be completely locked down to the point where a we're actually getting authentication or authorization for any sort of i o going in or out of the system via another source so if if a if a lab or or or classified facility has limitation in terms of where they can pull data from or who can pull that data or what applications are allowed to be used against that data we can integrate that via the Ingress and egress and we can also lock down when the job is running we take it off the network so they CA they have to do all of their Ingress and egress

via that section of the DSL so we can now start securing these workflows in a way that honestly we just haven't been able to do before and it gives a lot of capability in terms of of management right this is the type of thing that a CSO or or somebody who's really focused on the security side is going to be uh very interested in making sure that everything is is auditable it's reproducible and it's secure not just secure from a hacker perspective but secure from a supply chain perspective keep in mind everything that we're running now is also coming out of something that can be

verified and validated each one of those containers can be validated so excuse me so we can very easily have complete uh transparency everything that's happening and then trust on everything that's happening so yeah security is definitely top of mind yeah thank you for that Greg I know I feel like we've uh kept you a little bit longer I think you might have a couple other meetings to go to but thank you so much for coming on and sharing with us the story and building this amazing company and you know releasing fuzzball and making it fun and awesome um I really appreciate your your time being

here and um I don't know saying if you have something else to say but you guys we are open we are open for business we are ready for you know your your comments your questions you can go to our website you can schedule with us and and dive a little bit deeper on to how fuzzball can really benefit uh your environment and how we can work together and we'd love to chat with you so leave a comment leave a like say hello um share with your friends schedule a meeting we're here for you absolutely thank you all for watching and I want to thank the

engineering team I know you guys have put a lot of time and a lot of effort into this so we really appreciate it for us I appreciate all the work that you've done I know you spent a lot of time working on these workflows to be able to show people and show off fuzzball and it's been fun watching over the last almost two years now play with this thing and watch it grow so really appreciate everybody out there like Rose said go uh like subscribe if you want to talk to us more about this go to the puzzball page and sign up and give us

some time appreciate it