Research Computing Roundtable - Turnkey HPC: What is HPC?

big and small so um been around for quite a while um they used to be rockwell long long ago um so some of you may for me may be familiar with that it depends on depends on your age i suppose but uh i've been in hpc specifically for a little over 20 years i've been in the it industry for 29 29 or so years so that's about it it goes fast does it patrick it does it does indeed so next we have john aka all grizznock uh john hanks been doing this for a long time i don't remember what i said last time when i

got introduced so i hope i don't contradict myself this time um i i was thinking about the topic today and i think maybe i'm inherently unqualified to answer this question or even try to answer this question because in in pondering it i i think all i do is manage workstations now they just happen to look a lot like a cluster because of the way i manage them so i don't know that i even do any hpc anymore no i think that's great that's something that's going to be interesting to kind of see i mean what is hpc to everyone else so i'll move next kind

of skip over christian because we had you greg you're in the middle hi everybody i i feel like who is in the brady bunch uh in the middle was that alice jan no i'm kidding thanks um but uh hi everybody uh i've been doing hpc pretty much my whole career started with a degree in biochemistry and then failed to continue getting my phd because i got more interested in computing and linux and open source than than than doing the science so i've been really focused on hpc and scientific computing ever since and as christian mentioned um i i've worked with krishna and gary for ages

um and at berkeley lab so uh it's great to have great to have this middle line across the board is is is the berkeley is the berkeley crew here i think very nice we'll just call it the berkeley line there so next person from berkeley gary gary hi hi my name is gary chung i'm i manage the institutional hpc computing at berkeley lab greg and i started the program back in 2003 and it's been going great since i also have a dual appointment with uc berkeley so i managed the hpc effort for the institutional uh institution down at uc berkeley also um yeah and uh

krishna and greg and i we are way back doing this very nice glenn you're up next can you guys hear me now we got you yay uh glenn you gotta you gotta you gotta show us your shirt today too by the way [Laughter] uh been at ciq for 16 days i'm the uh vp of freaking sweet ideas the iq and uh my background uh was a doctorate in immunology of microbiology postdoc in hiv research and then um left academia and fell into hpc with my uh second postdoc at the san diego supercomputer center and uh from then on i was like in 99 from then

on it's been you know bioinformatics and uh hpc has been has been my livelihood so uh incredible and since about 2000 i've known greg i think so i blame him it's an easy target right yeah he's in the middle exactly hey everyone my name is forest bert i'm a high performance computing systems engineer here at ciq i've been with ciq for about a year before that i was the student hpc system administrator at boise state university while i was getting my bachelor's in computer science i kind of got my start in the cloud and in computing and stuff like that um you know in the

early 2010s messing around with um you know the tech that i had available and uh you know cloud gaming servers and stuff like that i'm real pleased these days to work with a little bit bigger systems um but yeah i love hpc got super interested in it while i was in college and uh yeah i'm really pleased to have been working at ciq and to uh be working on everything that we do with fuzzball and that type of stuff so great to be here thank you forest so kind of jumping back into the question of of kind of what john started to allude to a

little bit and i think that the the definition of hpc is going to be a little bit different to everyone here so kind of like to go around and ask what does hpc really mean to each of you and just kind of tell us what that means tell us a little bit about maybe where you got to that point and how you got there and start off with patrick and keep picking on me adam you're right next to me uh i would say what does hpc mean to me so hpc to me is any service which is designed specifically to perform a set of complex

tasks which i know that is a very broad definition but it's very um it's very true because i've dealt with hpc in all of its various forms from um from craze to to working on loose grid systems to working on top uh uh close coupled very tight systems with infiniband and mirinet and you know just all sorts of different interconnects of all kinds and it really just depends on what tasks you're trying to accomplish what you need in order to meet that task out um you know it kind of there's a blending of what is it there's there's hpc for high performance computing and then

there's now high throughput computing and now there's like all these other sub variants but in my mind it's it's really it's all hpc it's just with different tasks in mind to accomplish and and and different hardware and software solutions to accomplish those tasks so for me it's just kind of one really big bucket of uh designing clusters and computers to meet tasks needs so that's great thank you patrick and john you kind of started down the road of talking about desktops and that's kind of what you would deal with today i know they may have been a little bit tongue-in-cheek but in reality that can

kind of be something that that really is the case so tell me about your your definition of hpc for for me hbc the primary thing there is high performance and for it to be hpc it needs to be high performance and that's why a lot of what i do now is not hpc because i got people getting you know this massive pile of gpu hardware and and memory and cores and then sitting there typing in a jupiter notebook on that system it's like the very as far away from high performance as you could ever hope to get you know somebody checking out a seven thousand

dollar gpu to type and hit enter in a command line prompt um so i i think it's the hpc high throughput computing those are really the two main designations and then for me it's getting even further away from high throughput computing where yeah we run some embarrassing and parallel workloads but the most important thing in our environment is on demand for people interacting so we're we're pushing that long tail even further out by pulling in people that don't really need a cluster they just need a well-managed desktop workstation that has a lot of memory very nice thank you very much krishna how do you define

hpc i i think it is changing it is my audio better more um you're good okay um i think the definition of hpc is changing 2005 when i started my first job in us it was at san diego super computing center and the first machine i worked on was from ibm it's called data star it has a very specialized interconnect uh very specialized work servers all put together but that's not the high-end machine back then a san diego supercomputing center had a good size machine data star that's how the hpc mission used to be and the applications that run on used to run on that

application was very optimized tuned for the architecture that ibm architecture that we used but these days i think it's not the case anymore a lot of technologies have commoditized and interconnect infiniband as it has is is treated as a commodity a lot of servers that we use on even the top machines the top 500 machines our commodity servers from supermicro and things like that i think it is morphing the definition for hpc is becoming a lot more commoditized and not specialized machines anymore the employer who i work for right now we build stream computers all that we have is a lot of data streaming at

us um wafer inspections right so we are we have really high high definition cameras uh looking at this nano scale uh layer width uh transistor width and things like that so it's really high volume data just fire hose of data being shoved at us the biggest machine that my uh my employer built is just three racks not hundreds of racks or even tens of taxes just three racks that's the biggest that we have so and it's all commodity hardware commodity we we don't have a specialized server design team at kla we don't have a specialized architecture team but really we all take white boxes super

micro band commodity and just try to build that this is i would consider what we are doing at kla is hpc because the data volumes that we are dealing with um if that's the definition it does come under hpc but the hardware wise it's all off the shelf so it is changing hpc definition is surely changing is what i would say oh that's very interesting i mean talking about commodity hardware versus very specialized hardware that it used to be so i think like you said things don't think there's one way i don't think that's inherently a bad thing though i really think that commoditization of

the hardware has brought hpc into the hands of so many more people that it's actually a very good thing that you know you can build a super equivalent with off-the-shelf parts i mean i i did it for for a a very very small um but one could argue quite important x86 design company where they didn't have the money to go out and and and really invest in a massive super to to kind of solve their complex problems uh but instead i was able to put together desktop machines literally on a folded piece of metal and i would go to fry's rest in peace and purchase

off-the-shelf hardware and literally take it back and have one of my guys put it on a folded piece of aluminum and bolt the the the bits to the they'll fold a piece of lumen and have bread racks from sam's club with like literally 44 machines per rack in an open air data center because it was designed originally for uh for uh mainframes so it was you know designed to to kind of have an open-air concept uh and you know roll in the bread racks of 44 machines and we could do it and add thousands of cars at a fraction of the price that it would

take for us to go to ibm or super micro even or dell or anybody to have it done and so i mean just the ability to do that um it brings you know it allowed it allowed that small little hundred person company to compete with the likes of amd and intel and not necessarily effectively compete um as they got bought by intel last year but um they were able to compete so i mean just that that that alone you know that commoditization of hardware it's not a bad thing in my opinion yeah it's really cool to work on those really highly specialized systems um but

it also is it's amazing what bringing that capacity for computing to the general populace can really do well before i go to greg for his definition of it i i coming from an enterprise background i've watched people build commodity infrastructure to solve enterprise problems that probably should have been hpc in the past and it's interesting to see kind of that merging of the two worlds where you see hpc becoming more commodity hardware and maybe those workloads are moving from just general enterprise into an hpc environment where they actually get handled in a much more efficient fashion so i i've been excited to kind of see

that merging but now i'll go to you greg what's your definition of hpc well before i even mention that i just want to respond to patrick's point i completely agree we've seen this happen so many times in research organizations especially governments in academia where in many cases people you know that have never done system administration before are even buying clusters and you know they're not managing them very effectively but they're getting research done and because of small budgets they're actually able to do uh very good research on that at some point typically they'll graduate and and be able to then to have uh you know

proper support for those systems so it's not you know a grad student or or even an undergrad or a postdoc who lost a bet and now has to go maintain that system uh in many cases then they can actually reach out to somebody like you know gary's group um at berkeley and and lbl to actually solve that problem and actually manage it appropriately but but it hitting commodity has lowered that barrier of entry so high that it's been it's been fantastic now to get to the question you asked me on zayn um you know i've been doing this for long enough to have a very

strong opinion of what high performance computing is or rather was uh it was it was tightly coupled uh parallel processing at massive scale and i remember times where unless you can prove that your application could run effectively at massive scale you didn't get time uh you were actually uh you know you were down revved in terms of um uh how much time you can get and what priority you get to run in unless you can demonstrate that level of efficiency now we used we called it the long tail for a long time which was all of the scientific applications and work workloads that are running

on these systems that are not that massively traditional high performance computing uh profile and what we've seen over time at least what i've seen over time is that that long tail has just continued to grow grisnog you kind of alluded to that as well that the long tail has now grown so far out that the long tail is now becoming the dominant form of workload on many of these systems and i've had many many arguments over drinks at super computing on what is high performance computing and um and is these sorts of applications actually high performance computing it took me a long time to actually

um to get to this point uh in in my maturity uh as a person to actually say that high performance computing is now more than just those massive tightly coupled applications it's anything that's going to spin or peg some aspect of that hardware at 100 percent and then literally kind of bottleneck based on that that subsystem of your hardware so whether that's cpu whether that's memory io whether that's bandwidth you're going to peg something on your system and that's kind of where i am in terms of the definition of high performance computing it could be a single thread hitting one type one core or even

a a hyper thread on your on your cpu but it's it's pegging something dang it somewhere that's that's where i draw the line i like it efficient utilization of resource that's all i'm hearing i'm not sure if it's efficient in many cases but it's definitely utilization pegging it resource you know whatever gary your definition you've got a fairly large environment to deal with so yeah yeah um you know we uh it's interesting uh when when greg talks about that it brings back a lot of memories i i uh it was exactly the way he said we're looking at really big um usage of the system

but you know we we do take a a kind of a simplistic look at view of like what is hpc we work with a lot of researchers team large team science and then also individual researchers and a lot of individual researchers may have a very powerful workstation they may have a laptop and then at some point you know what they're doing data acquisition or simulation they're going to outgrow it and so they have to move to something and then so what is that something and then for us that that's where we define the hpc we want to move them on to um like you know

instead of coming up with one offer all these people then that's that's our opportunity to move them into an hpc infrastructure and then you know i i think everybody's kind of covered that but i there's enough convergence in the in the field so that now when you describe an hpc infrastructure everybody knows exactly what you're talking about so one thing i just want to follow on that gary said speaking of memories um we did see a lot of people that started off on a laptop then went to a powerful workstation and then we initially when we first started the hpc project at berkeley it was

around this concept of something called mid-range computing and mid-range computing was we we defined as everything between a workstation and a giant facility like nersk and everything in between uh nursk at that point was um limiting people unless they can prove you know a reasonable amount of performance and and efficiency in terms of running on their system they didn't want applications running on that system unless they were efficient on that system because it was such a big expensive resource so one of the one of the big motivations that we had for this group was uh this idea of mid-range and and how do we solve

these mid-range problems and that to me is now turned into what i kind of call sweet spot computing this is where most people are are focused on this is this is for example in high performance computing this is the vendor's bread and butter this is where they're selling the most systems so and that's why we're talking about turnkey hpc honestly because it comes right back to that that's the problem set that we actually really need a solution for yeah that was a that was a huge gap i i think essentially when we did that that early you know we helped to define what is known

as the mid-range computing segment within the doe and i think that actually even articulates how hpc was changing even then gary without us even really kind of coining it or thinking about the the nomenclature we were changing what was hpc even then it started off being those giant systems and then the mid it was moving towards mid-range and it's been still kind of following in that trend in that direction ever since it's very interesting thank you very much gary next we'll move to glenn you have a interesting perspective on this so what is your definition of hpc so so when i started uh in this

field hbc was you know tightly coupled pretty large scale machines right marinette low latency interconnects but when the life sciences started adopting the beowulf architecture it rarely had um those little latency interconnects because it didn't really need it it just needed so they were coining it high throughput computing right on on ethernet and so i read it to people oh it's not really hpc it's just high throughput but you know in my head i was like same architecture just it doesn't have your low latency interconnector like we're doing and it's a different you know type of parallelism um but i still thought that it has

to be something like hpc and then in my head it kind of evolved to like so how do i you know lump those two together so that and then that became research computing so research computing was you know all these folks whether it was uh molecular dynamics or or just you know dna alignment embarrassing the parallel or tightly coupled it was all research computing but then the enterprise was doing it so you know so so now what do i call it um and then to john's point you know i've worked on machines with you know all these gpus they're running a jupyter notebook but you

better have that rapids library and that python tuned optimized really well so they get they get the most out of it and so that if my last employer when we were working on my title which was not vp of freaking sweet ideas unfortunately uh we talked about why not scientific computing you know you're trying to get science done really whether it's you know johnson and johnson trying to figure out you know how to design their next shampoo bottle or if it's you know covered you know um kind of genome analysis so i've come to call everything that we've just talked about scientific computing but if

you hadn't if you had to like nail me down to hpc i'd be thinking of systems that you know try to look like the top 500 right tightly coupled uh the scale of the top 500 um anything you know smaller than that is you know still hpc um i'm not sure my mid-range hpc um redline is but um that's that's how i kind of view the world these days thank you glenn so kind of between what greg talked about and what you talked about i used to be on a mid-range team and mid-range to us just met the solaris team and one of our engineering

stations yeah yeah exactly one of our guys called himself the vp of blinky lights because we had the data center so that's not quite as cool as sweet ideas but pretty cool forest what is your definition of hpc i would i would define hpc kind of surrounding um the efficiency that we're trying to achieve with hpc we have you know hypothetically a single commodity server that we're trying to you know do some traditional hpc workload on whether that be you know simulation or you know just to roll them both into one thing is some type of embarrassingly parallel high throughput type of job you know

if you only have one server you're not going to be able to get the performance that you would need out of that in order to actually um you know efficiently iterate on the work that you're trying to do um so to me hpc comes down to being able to do these really really complex things in an efficient way um regardless of what the system underlying it ends up being whether that's you know a bunch of ec2 instances that's you know a traditional tightly coupled cluster whether that's a cluster that's more meant to provide as many gpus as possible just to you know do mass model

training that type of thing um i think ultimately it comes down less to the comprisement of the system itself but are you building the system to gain performance beyond what you would get with you know the base component of this system in that case i would say you're doing hpc i would also touch on kind of the long tail concept um because uh kind of where i very very first entered this was more so in those tightly coupled jobs but especially at my last institution i started to see a lot of researchers coming out of very what i felt were non-traditional places for hbc um

you know we weren't just having um you know genomics people or simulations people that type of stuff we also had you know education or people from um you know the education department that were trying to do mass data science crunching for their research we had people kind of on the high throughput side we had people that were taking lidar data and doing um huge amounts of analysis on that kind of in a high-throughput way so i definitely think that in addition to it kind of being about efficiency we're definitely seeing you know kind of enterprise and some of these other use cases realizing that they

are trying to do hpc without realizing that they're doing it um so we kind of see uh you know maybe some of the muddying of the waters here as some of these non-traditional uh entities kind of enter the hpc sphere absolutely and that kind of leads into one of my next questions for us and thank you for that is i think we all alluded to it at some point but have the hpc workloads changed over the years and we talked about the infrastructure getting better the machines getting better i mean obviously they're getting far far more and more dense and being able to be more

capable but have the workloads changed over the years and i will throw that out to anybody who wants to take it 100 absolutely the workloads have changed i mean you've got some of the traditional workloads that are still either that are still around in mpi mpi-based ones we touched on this i can't remember if it was in one of our dry runs or or in the actual uh uh meeting last week but we we definitely touched base on the you know who's still using mpi what are you you know what are your percentages of your workloads of different types and that sort of thing but

it was it was definitely uh you know eye-opening i i kind of expected it in some ways but it still was very informative that like that the mixed workload is really here to stay um even in very specific like industries the amount of workloads that yours you know singular cluster or clusters are supposed to to be able to support is massive you're supposed to be able to support you know gpu-based uh you know ai training versus or ml training versus you know being able to to to run you know ultra fast single threaded performance um you know or highly parallelized jobs you're supposed to be

able to run it all in one and you know all of those require different really pain points to overcome in building hpc clusters but i think we're kind of expected to you know as designers and administrators and of those environments we're supposed to be able to provide for all of those in any kind of one size fits all and it just it's difficult to accomplish i i think to some extent um the commoditization of the compute part of this has made maybe hpc simple to do on the compute side but where the actual interesting stuff being done now and probably more so in the future

is on the storage side storage is is actually becoming the critical thing about everything we do regardless was hpci throughput whatever i mean at least for life sciences there a lot of on-prem clusters and uh on-prem infrastructure exists today because you just simply can't move the data sets around like the compute has to be brought to the storage so the compute's more or less secondary to just getting all your data in one place where you can work on it and not only that the way the compute is being orchestrated or designed is quite different in the in the enterprise enterprise is dealing i know of

use cases where they are dealing with um data pipelines at the rate of multiple tens of gigabytes per second constantly coming the that's the data volume that they're dealing with and the workflows and applications they're running they do not have anything mpi no mpi at all nobody in that organization knows anything about mpi so it's uh it's not yeah mpi is not the only hp the data volumes are a lot higher than many organizations deal with but these organizations don't have anything mpi lack of lack of mpi in the enterprise um pushes me into an old man rant moment about a negative connotation of hpc

and that had in the early days of hbc i remember back you know 20 or 30 years ago when i was getting started in this there was an elitism aspect to hpc you know you had to be able to afford a cra to do it it was a very elite club and not everybody got to join that club and that attitude carrying forward over the following 20 years from from when super computing got started led to us having to deal with things today like hadoop and spark and kubernetes and all this just nonsense complicated extra complexity poor performing you know because that's the stuff that

came out of the enterprise when at any point during that we could have just said hey enterprise people how about you help us write a better mpi and let's do this right uh you know efficiently and simply with a simple scheduler i don't think it was that it wasn't offered i think that it wasn't necessarily accepted in amongst industry because so many different corporations were trying to capitalize on that market or well here let me enter my proprietary standard into this this ring let's let's go ahead and and they didn't want to do it open source either generally they wanted to really make it very

tight and and you know hey well now that you've entered into our ecosystem you're stuck and that's you know that's kind of the way that came about i think is it and then of course you had the the competition as you said where now we've got coop and now we've got you know all of these other uh competing not exactly optimal frameworks to put it lightly uh that to do a lot of the same workload that yeah you're right could be could have been much better implemented in hpc so let me ask this pack and john so from an enterprise perspective if i'm looking at

this the way that most of those other languages and other tools came out was to be simple and fast and if i look at mpi it's probably a little more complicated to program for correctly and to get it as efficient as it should be right so was it something that from an enterprise perspective people didn't want to take the time to learn how to code correctly for mpi it was just easier to use some other framework i'm going to throw on java it was a major barrier to entry so because of that barrier to entry whether it be i didn't have the hardware to have

that high speed memory interconnect between the nodes or i didn't have you know ca i'm a you know undergrad i can't afford to get time on a on a super at the lab i couldn't you know or whatever it whatever it was you know uh there's always a barrier to entry or if it's well if it's intel mpi do i have to buy into the entire intel ecosystem do i have to go buy an intel compiler back in the day does that mean that now that i'm running on my amd chips is it going to be slower because intel spiked the compiler i mean you

know whatever it is you know it just it depends upon the the environment but there was always a barrier to entry um and then of course then you had open mpi so now we've got competing standards uh so i i'm gonna i'm gonna push back a little bit because um i think it's very use case specific so the use cases and the applications that we were solving in high performance computing were very different than what enterprise was really even looking at solving now the map reduce um kind of focus there i do i do agree there we could have done better in terms of cross-pollinating

between hadoop and then spark and then then hpc mpi and piio and other things and we're starting to see that also happening in in ai starting to right this is a few years old now at this point but things like how do we do training how do we optimize training how do we start parallelizing training if anybody remembers some of the early training frameworks uh it wasn't running mpi they they created something from scratch almost like mpi didn't exist and they solved very you know it was it was a duplication of effort and to the point patrick i think you made this um it may

have been grisnog but if we would have come together right at the beginning in a better way more open way and i think showed more interest in collaboration i think we would have seen um very different outcomes uh and and but this goes back to i mean i joke about this uh you know in in all of the organizations that i've seen and worked with that have hpc focused people traditional hpc focused people there's i t and then there's hpc even if hpc is under i.t they don't talk they don't work together they don't leverage each other because the technology is both so different but

they're also they're solving such different problems but there's also just this legacy mindset of on both sides it's not until fairly recently in my mind containers was the pandora's box that that really started blurring this it wasn't really until containers did did anyone in hpc even care what the heck was happening in enterprise we took off at that point and on the enterprise side it wasn't until just recently did they start looking at hpc saying how do we do training faster how do we do better analytics how do we get more insights from our data and that to me was the crossover point and now

we need to actually all come together and realize we're solving very similar problems and we need to stop innovating on our own and start back start actually working together and solving problems together we need to be a big happy family again need to be yes seemed like you were you were going to what do you mean what do you mean again um all right i just kind of want to rip off that a little bit um of a side jaunt but back to the topic of how hbc is kind of changing um something that i really see is uh the hardware side of hpc changing

it seems like for a long time you know we had you know infiniband cpus gpus and that was you know kind of the basic compromi are you know the basic components of a cluster as far as you know the compute part we start to see now that there's a lot of companies that are starting to invest in creating new solutions around some of these very specific use cases so you know now we see for ai you know we're only 10 years off of alex uh krishevsky i think i pronounced his right name or his name right there um you know training alex net on two

gtx 580s and now you know 10 years later we already have companies that are trying to replace the gpu for ai training because there's better ways that that can be done than just on gpus um so some way that i really see it changing is on the hardware level i mean we've got um new ways of training ai new uh much faster interconnects than we've ever had available before new storage technologies it seems to me like we're really in kind of a renaissance of hpc hardware um you know the uh the use of pcie as a broader interconnect than just to put you know accelerators

into a cluster and stuff like that uh we're really it seems like kind of in a renaissance as far as hpc hardware goes um and our clusters are really kind of starting to change from just the basic cpu interconnect gpu type of thing with fpgas specialized chips what have you of course i'm glad you brought that up because that kind of reminded me of a question i had for gary last time is he made the statement that his gcp usage has gone up by 3x over the last couple of years a few years now what is driving that and is that something that is more

hardware specific since it is difficult to just go purchase whatever you need because it changes all the time is that kind of what you're seeing is that what's driving that yeah you know i for a lot of people they're working with workflows if somebody's um wants to put together or string together uh an infrastructure with a web server and a database and do something and move some data is pretty easy you know the term i like to use is is frictionless computing because they're able to kind of do it themselves and without having to say wait for a network drop in a data center and

then for somebody to set up the machine and all that so it's a lot faster that that's it and then uh probably the other thing uh is this really easy to prototype things and then the new tools like automl and sagemaker make it really easy for people to get started using machine learning and so um so it's interesting we'll see we'll onboard people onto the cloud we'll kind of like prototype it using auto ml or something like that you know get going and then then sometimes what happens is that they'll figure it's find out that it costs a lot of money and then we'll go

back and now that they see how how efficient it is and how this helps their science uh we'll go back and um transition on onto our on-prem hpc um using uh you know maybe slightly different tools interesting but yeah i was just curious because that's it's interesting to see how that's taking place and i talked to a lot of different customers who are hpc that are trying to like you said try things out in a cloud before they actually make the investment to see if it's gonna to work for their needs or do what they expected to do so thank you i think we have

a question from mr ignite so i before we pop it up i can go ahead and start reading it is nobody's really talked about the vector processing as a part of hpc build out so the the example that's given is a medium scale iot collection and analysis of billions of devices generate tens of bytes per second driving anomaly detection pattern recognition and other applications uh interesting interesting topic and if you guys have any thoughts on that i'd be curious to hear if anybody's ever done anything on a cluster that was in real time and the the closest i've ever got was a group that wanted

to produce a weather forecast every day and so they had data arrive and then they had a deadline to get stuff out but i've i don't i don't recall ever having anybody want to do real-time stream processing as part of a cluster workload um actually krishna and greg and i could probably have an example of where we're currently building a system for a gamma ray detector and this uh the system is going to be the uh the first level trigger of the data coming off the detector and um it's going to be a kubernetes cluster we're going to run different containers depending on what's going

to be analyzed but uh we we are using instead of fpga hardware they figured it's going to be a lot more flexible do with the cluster so that's currently uh being put together engineered and put together right now so to kind of and i kind of figured you know gary and krishna would have some examples here but um yeah there there were multiple different uh examples of people doing detector research and uh they basically you can you can think of a detector as basically you know a very kind of simple idea of one is you know you have some sort of reaction happening within a

a spherical enclosure and then you've got a whole bunch of detectors all the way around the spherical enclosure and it's it's monitoring all of the different types of rays and and what is happening at the center of the sphere and all of these different detectors are uh creating streams of data then these streams of data have to be computed in parallel in real time so you can see and visualize what's happening within that experiment and sometimes these are actually in real time but in many cases it's near real time which basically means now you actually have to um look at it and then decide do

you want to rerun the experiment do you have good data and then feed that back into your into your process flow but it needs to be either real time or near near real time for that and as gary just mentioned if you're running it through kubernetes it's an extraordinarily different architecture than what we've been doing in high performance computing and without even knowing the specifics on that i would say probably it's a bunch of services that are running and doing the equivalent of like an inference on each one of those data streams coming in and then computing on those via gpus so this is like

a performance critical service that's running and kubernetes is a is a good architecture for managing services um i i'd actually i i i'm curious on the efficiency that you're getting because i've actually had really bad results with regards to kubernetes and just to go on a quick tangent real quick uh we actually were talking to some people that were doing something along these lines with kubernetes but because kubernetes is so big and bloated uh it was usually utilizing about 15 to 30 percent of the underlying hardware resources uh just to run kubernetes so now let's scale this up imagine buying a 5 000 node cluster

or even a 500 node cluster and 100 100 nodes out of your 500 nodes is just running your your your scheduling system your your orchestration system right and you're you're basically losing 20 of your system to run just for kubernetes so at a small scale i think kubernetes would do it but as soon as we start scaling bigger we've got to come up with better ways of doing that yeah another another approach i have seen in the industry for these real-time stream computers is a traditional way which is so if you look at a ddn appliance a storage appliance um they usually come in pace

and they come with some fault tolerant resilient designs using basic technologies like ha proxy korosync such that services can flip from one controller to the other controller automatically if if there is a failure in top controller such services can luster services can move on to the bottom controller that's the basic components that you nee you would anyone you would use for fault tolerance um i'm sure this even kubernetes is kind of built on some basic components like that um in the industry i have noticed if you build a real-time streaming computer where your machine goes into a production pipeline and they need real-time feedback of

the quality of the product being being manufactured and if this computer is down and that quality is not being generated depending upon the value of the product you are making it can cost multiple hundred thousands of dollars per day of production loss so that's very costly so this real-time computer giving the feedback on the quality of the product is a very critical component and when you're making such a designing such a computer fault tolerant designs having backup components so critical and things falling back automatically from one server to other server is such a critical thing in your design if you do not have it you

you are costing hundreds of cases of losses to your company to your product um that is something that i have i'm noticing heavily and not everybody in the enterprise is adopting kubernetes they they're just not trying to accomplish that using the ha proxy chorusing services which have existed for a long time uh and trying to put together everything um like we have redundant switches redundant network cables run from uh servers to switches and how services fail over from one switch to the other switch if there is a switch failure or a network cable failure those are those are another tangent an extension to hpc design

that i am seeing in the enterprise and that is very critical and needed when you are making a building a real-time computer for these use cases so from my past iot experience most of what we were trying to do was do as much analyzing of data at the edge as we could so that we weren't sending massive amounts of data back to be processed or analyzed you were looking for that anomaly and then whatever data was driving that is what we were sending back so from a real-time usage i feel like just because we were trying not to send a lot of data that may

have been irrelevant we were doing a lot of pre-processing at the edge and then only moving back what was necessary for the next type of action or the next time event that needed to be taking place but hey that's an interesting question when you're talking about massive data like in greg's example it's not really an edge's case because everything is right there together you're trying to do it in the place that it's taking place from an iot perspective when you have billions of sensors out that amount of data streaming back could be something that would be overwhelming and a lot of that at the end

of the day just gets dropped on the floor and you never even look at it so a lot of different ways to approach it when you look at iot it kind of depends on what you're trying to accomplish in my opinion my history with this was uh back at bell labs they we were monitoring uh numbers of phone switches in real time and in order to do that it required i mean this was it's quite a long time ago this was the fi the error of the 5e which that that should kind of explain how old this was but um and they were quite new

it was one of the first digital phone switches but it was uh required massive amounts of customized hardware to monitor these switches and today i don't see the investment in that type of customized hardware for real-time computing as much i mean it's it's kind of common or i'm sorry kind of uncommon to see um you know really really specifically focused uh hardware for real-time computing like we used to see quite a lot i mean you'd see you know real-time linuxes you'd see real-time um you know that were made for all sorts of different rtc clocks and rt rtc controller cards and things like that to

ensure you know absolute you know peak timing and synchronization and we see some of that today but not nearly that we did maybe 20 years ago so i think there's either because the performance of machines has gotten so much better and we're close to a okay level or our standards have slipped a little bit um i think or maybe kind of maybe a merger of both i'm not sure but that's great thanks patrick we do have a question from sylvie and it's have the programming languages and interfaces changed over time i think it's a great question um there's yes um when i was first getting into this uh gary you you may you may attest to this um we actually had to help a lot of scientists and researchers uh upgrade to fortran 77.

um upgrade two fortran 77. uh because they were running predecessors i forget what fortran was before 77 60 something and that was the year by the way um that the spec was was created uh and yes it has definitely changed and we've seen a lot of that now on parallelization you know we've seen mpi really taking off we've talked a little bit about the different forms of mpi um but the specification has definitely evolved over time but that specification uh has created a lot of compatibility so even though we have many different mpis coding for any of those mpis should be the same no matter

what right you're if you're if you're adhering to the uh to the strict specification um some there's some alternative or optional pieces of the specification but as long as you adhere to the strict specification it should be mpi compatible no matter what now every programming language is going to have its own uh aspect of or interface to mpi so you could have for example uh c bindings for tran bindings for trend 90 bindings and more recently we've seen a lot of interest in higher level programming language like python and python is absolutely taken off in scientific computing uh to the point where uh i actually

think it's probably one of the most dominant programming languages nowadays and i'm kind of curious from from others are you seeing the same thing and what is that like in industry specific areas like life sciences is it similar there too our software stack is almost entirely conda at this point like that there are some there are a few applications in there bow tie sam tools that kind of stuff that that is not in python but for the most part got to be at least 70 or 80 percent of everything is done in python maybe with a mix of r in in that too glenn are

you seeing the same thing in in your area of life sciences yeah things have evolved from from pearl to python python has really done a fantastic job um in uh adding libraries and then and then of course conda um to support uh the researchers it's i think the most popular language um there is a lot of r in there from from folks who come from a statistical side they you know they seem to you know like r a lot but python is is you know this is just my opinion but if you're trying to do a lot of other things than just statistics python is

you know much better you're not you're not bringing up a you know a web server in r so we see python doing a lot um in in the life sciences um the the folks that are using c are the folks in writing the algorithms whether it's uh you know for dna alignment or you know uh drug docking or uh or on the gpus trying to you know get their stuff uh onto the gpus with cuda um those are the folks using the low level languages everybody else is is a little bit higher up at the python level um there seems to be a little bit

of a push to i think i can't remember the end of the project but actually getting some actual typing as opposed to just you know static typing as opposed to just dynamic typing uh with python for some things in uh in research so i'm actually kind of interested to follow that um but yeah i i that's that's what uh the trend has been it's it's funny for me even to consider and think that hpc has changed so much over the years that we're not even focused solely on compiled and and purely efficient languages that we're running across a whole bunch of uh different languages um

i think there's there's even some rust that i've heard of um patrick i think you've mentioned rust as well uh kind of coming into hpc there's there's rust um and also i mean there's still there's still some rubies creeping around in the eda world um and moving moving in there and uh let's see there's but but primarily it's these days it's it's python and and rust and again some some ruby and and of course sadly that pearl still exists um this was awesome you're making greg cry on her no i love the swiss army chainsaw but i also hate it um you know if you've

ever walked into a project where where you've had you know somebody who's they've been in their own pearl world for years and years and years just designing it and designing and designing it and just adding on and then you try to take over that project and figure out what was actually done with this stack of pearl scripts it's um it can be fun and then you rewrite it and fashion like 10 lines and make you hate pearl yes uh dave so we have a new question from dave i've seen so much scripty and numpy usage at scale these days there are a lot of compile

time optimizations to be considered for those libraries alone intel compiler and kl scipy by the way what did i say sorry skippy no sorry i heard skipping maybe i heard it wrong i've just i don't know i don't miss any opportunity to poke at you zane and because there's not many you didn't know whatever i did i see um do you have a question or is it just like to make fun of zane time i'm good with that either way well as i said it doesn't happen often so i have to take every advantage that i can well as on that it happens all the

time because he's good at it too i got my shirt on exactly look down yeah yeah so um yeah there's there's a lot of there's a lot of scientific libraries that are that are also becoming more and more available for python and other higher level programming languages like scipy numpy and and and others i haven't seen and i'm curious if others have seen uh architecture optimize math libraries uh like mkl like intel mkl and whatnot uh are there interfaces for python for these or are we kind of moving away for the for the kind of specific need of high performance and and absolute tuning and

profiling and building the the fastest programs ever in exchange for kind of more general purpose uh hpc applications and and and whatnot i'm kind of curious i know we're kind of at the top but i'm really curious on people's take on that maybe that's where we leave off for our next roundtable it's got to be at least seven or eight years since i built a blaz library to do specific optimizations i think at some point the hardware got fast enough that people certainly at the level that i have to deal with they don't care about getting the last five or ten percent out of anything

right like the next generation of processor will solve that problem for us they focus more on the writing a better algorithm in general rather than trying to get a little bit more compiler optimization out of it when uh when intel came out with their with their uh like intel python and then like mkl and blas and boost and things like that optimize underneath we were really excited to use it but we it just didn't seem to play well with the typical like conda at pip tools uh we so when we try to install it like it just kept blowing up on us and so we

we gave up on it because we thought we we could get a lot of good performance gains because uh everyone was you know using python but just it came down to you know just usability with or just trying to introduce it into already kind of a mature you know python environment that everyone was familiar with hey just try to use these libraries and things like that it just kind of didn't work so uh we we eventually dropped that um but yeah we other than that we haven't built um we haven't built or compiled anything specific with uh for for tuning or a library in it

yeah in a really long time yeah my my most frequently used c flag is m tune generic just because i want it to run everywhere right i don't care if it's fast i just want it to work everywhere i'm going to run it that see that's that's that's what really comes down is that you can make snowflakes that work really really really well one time making something resilient that can run everywhere all the time and reduce your issues is is so is so important to hpc in my opinion because if you're constantly retuning retuning retuning for different architectures and different like that that last two

percent of optimization then how much time are you losing to get that two percent when you know you could just throw more hardware at it for more course throw more memory or you know or throw a cloud at it i mean that's the other aspect is now that you can you know fairly easily scale up it's kind of like for for some of that it's easier to do that than it is to optimize like just down to that little little tiny little bit at the end thank you for that patrick we have one more question before we wrap up uh are there analogs to mkl

for things like the arm ct for architecture so i don't have an answer specifically for the question um i would assume if there's not already that they're going to be coming they're coming soon um but i just wanted to say aydah dave it's been a long time since we chat it's great to see you here and thanks for all the questions dave does anybody else know if there are uh optimized versions of math libraries for um arm dave we're going to probably have to reach out to some people that are running some arm clusters i know sandia has some pretty good-sized arm clusters um and

uh and as well as maybe some arm people and we will hopefully have an answer for that either in our in our in the chat or uh maybe for our next our next webinar i'm taking notes well guys we are a little bit over and we do appreciate your time we know you guys are busy and thank you for joining us again krishna nice to meet you gary john patrick glenn forrest as always craig thank you very much guys go out like subscribe and we will see you next week thank you very much you