Live Demos at SC23!

which these settings will be used to configure werewolf uh and then and then I have some Mountain configurations which will werewolf uh uh I'd be happy to take any questions was uh yeah yeah so uh with your REO definitely set up another uh project to do that so uh how I would how I would probably do it is maybe have like a production uh project and then a Dev project that uh connects to like the dev branch of your automation okay yeah uh so a were so within werewolf uh there is this concept of overlays and basically these are configuration files uh that are sto

stored as a cpio archive and when you're no boots the CPI it will pull the overlay as a cpio archive and unzip it and overlay it on top of the note image yeah yep yep and uh yeah and uh within overlays uh we also have some templating features where you can pull uh for example tags within a werewolf node config so you can really template out these files uh to meet the needs of like your nodes [Music] basically go pulled up so as you can see this is HPC lab compute uh zero uh it's booted uh you can see the uh internal uh network interface

that this is using subnet uh the internal Network that uh this compute node is on and also the compute node's uh Mac address so and I should mention all of this infrastructure is currently deployed on vulture um a cloud provider uh the reason why I'm using vulture at the moment is because uh vulture gives you the ability to picky boot which is uh how werewolf boots your nodes so with that being said let's reboot this node and we can watch this come up and as you can see we are currently doing some [Music] DHCP and we are basically starting the pixie boot process right here

and it's contacting the werewolf controller node and as you can see it's pulling down the node image right now and and now our note is booting now you can see Espin and it starting and all of our services and our compute node is now up cool so uh just to show you on this cluster if I do PBS I think my session may have di that is let me just pull this over just to get this node back up real quick open tab did my session die okay we're back uh so I just lost some network but uh as you can see uh in the

terminal output here uh our compute nodes are now and running and are talking to uh the open PBS server um so uh what I can do here is uh submit a interactive job uh so this job will request two nodes four cores per node and use a total of eight cores and as you can see here my job has started and let's see and I can just run let me just do prun and basically I've compiled uh so if I do prun a.out uh so a.out is basically um a hello world example that uh comes with uh open HPC so uh this is just a

simple hello world program so if I run this within uh this interactive job you can see that uh I'm able to use all eight processes that I've requested and uh yeah this is just a simple hello world program but as you can see you can apply this to you know your cfd workflows if you're running very large MPI jobs uh this is just a very easy way to uh set up uh your cluster and with that being said let me switch back to this and you know CQ ciq is here to help so if you guys are interested in doing a deployment with a sender

werewolf Rocky uh you know you can get in touch with us at info@ c.com we'll be around here to have a conversation if you'd like and yeah thank you very much for uh coming to this demo and I hope you all have a great day thank you [Music] areans Unity of P took six weeks the ex r engaged the rest on I or she was out there we okay we're going to go ahead and get started we're going to start talking about securing your software supply chain so everybody knows about obtainer right everybody knows about building containers and using containers in HPC um so I

decided at this SC I don't want to just do a just a just a a a basic container demo I want to do something I want to do I want to do something a little different and um you know talk about something a little bit uh more more advanced so building containers is great um but how do you how do you build your containers almost everybody builds their containers from existing containers and in fact that's such a common pattern that a lot of times when you start talking about this people don't even realize that there's an people might not even realize that there's an alternative

so um cutting out so okay so but if you're building your containers from a a public container from another container which just on a public repository how do you know what's in your containers okay how H how how do you know there's there's lots of stuff in there them so how do you know what what goes into them so do you know and trust the Upstream authors of your base containers do you know who's building those containers and if you do know who's building them do you trust them um do you trust the administrators of the registry so maybe you're building your container from a

base container up on dockerhub how do you know that the people who are maintaining Docker Hub are doing a good job and have not allowed anybody to you know get into somebody else account or replace containers or do something like that and how do you guard against man- inth the-middle attacks so how do you know when you're downloading these that they're not being altered along the way okay so this method lacks security right you're just sort of you're trusting you're putting your trust in a lot of extra entities when you're building from base containers on public uh public repositories so may maybe as an alternative

you can build your containers from Upstream sources like distribution mirrors so there are methods a lot of people don't realize this but there are methods in place that you can build your containers from an RPM uh repository for instance you can build it from the Upstream mirrors for the operating system all right cool but that process is slow and brittle okay um you need to have specialized tooling on your system like maybe you're on an intu system but you want to buildall on intu which seems like a really bad idea or you might need something like de bootstrap to do this you might need um

so that doing this get get something a little bit different and it also adds a lot of complication to your to your uh definition files there's a lot of reasons why you don't want to do this for every build so this solution that lacks security and on the other hand we have a solution that lacks convenience so I'm going to propose and show you a way today where you can have the best of both worlds okay tools like dnf de bootstrap things like that and it's kind of a you know it's a slow process the first time once you've got that container now you can

push that container back up to a public registry and not worry about whether or not that registry is secure now you can use that container as a secure base and you can verify that container as part of your build process right so now we've got both security and convenience we've got the best of both worlds all right so now I'm going to I've got some like slides and stuff but I'm just going to dive over here into a terminal and I'm just going to you know I haven't really practiced this height and talking to the mic at the the same time so bear with me

if this is a little uh little weird this might be okay I'm going to try like this all right so here about to pull a container and I'm about to pull a container from Docker but I'm using kind of this weird notation here this is oras and registry at storage and the oras protocol allows us to push and pull arbitrary data to to oci Registries like Docker so I've got a sift file now I've got a a container up on the container registry which is not an oci container it's actually a native sift file and I'm about to pull that back and get it back

from dockerhub all right so I've already done this so I've got a cached image but this would work just the same if I didn't have a cached image now let me go ahead and show you I'm going to go ahead and do um another command here I'm just going to verify this container that I just pulled very very simple single command um let's see let me do the container I just pulled standby okay hold on so I just pulled uh Rocky like I said I didn't practice this beforehand so I apologize if I'm having any uh hold on just a second hold it watch thank

you yeah yeah yeah cool all right all right let me just let me just do that PLL again because for some reason I didn't end up with the uh file that I thought I was going to end up with thank you all right so we're going to go ahead and try to pull that file again it's going to use the cache sorry for the uh inconvenience here so this is the file that I just [Applause] pulled oh no I pulled oh I'm sorry I okay I pulled this LOL C Rocky let me back up okay that was the image I thought I was going to

pull my bad Rocky Linux 9.if and it's going to tell me that I've signed this image okay and it's going to give me not only did I sign this image but it's going to give me the fingerprint that I signed it with and that's important right because anybody can sign an image right um thanks Brian anybody can sign an image but if that fingerprint lets you know that it was signed by the entity that you think it was signed by so the fingerprint the question is how do you test and make sure that the fingerprint is the right fingerprint once again now I'm I'm I'm

um I'm back to trusting dockerhub again because somebody could have changed the fingerprint up here but just to illustrate this point uh look at this Rocky Linux one here and just as a convenience I've put this um fingerprint up here on this okay and so I can compare and I can make sure this fingerprint matches the one that I just got all right now once again you don't want to trust dockerhub so you don't want to do that right you want to you want to get it directly from the author or if it's your own that you've signed you can just save that fingerprint and

make sure that it's it's there okay so I just showed you that you can pull a a native sift file and you can check the fingerprint and now I know that this sift file is a bit production of the original one that I created okay well who cares about that why why would you you know that's that's a really kind of a pain in the have here so now you can do this as one of the steps when you're building your container so here I've got uh I'm I'm using um built that Rocky Linux container you did that from an existing container so you're still

trusting somebody else to build containers for you let me go back up to this definition file that I had up here before so you can see I used yum to build this container and I pointed it to the Upstream Rocky mirrors and so because these are secured through gpg Keys now I'm removing dockerhub and whoever builds the rocky Linux containers uh from dockerhub and anybody who admins them and all that kind of stuff I'm removing all that stuff from my chain of trust and I'm basically just trusting whoever admins the mirrors so at this point you might say okay why do you trust the people

who admin the mirrors well if you don't trust the people who admin the mirrors you've got bigger problems because you can't trust your operating system right so this so at at some point you got to trust somebody unless you just you know build your own computers and write all your own code uh but this I would say is about as good as you can get as far as securing your your containers so if we go down a little bit you know further here um we can see once again that I'm using this definition file to build a new container um and this is a container

that you might have seen before this is the one version of the LOL cow container so let me just pop back over here um and I'm going to make sure that I've got this definition file type one hand okay so let's go ahead and build that container and let's see what that looks like when you build from a signed base okay there you go I'm building and now notice this line here it says that it's checking that the bootstrap image verifies with the following fingerprint and so that's happening for you at build time now critically if if your container has been uh hacked if somebody's

gotten onto dockerhub and they've replaced your container with another one or if if if it was made by an untrustworthy author you don't want that container to be downloaded to your system and then check it and then maybe do some stuff inside of it to figure out whether or not it's signed so using this workflow we're not even going to get into the post section of the definition file if that fingerprint doesn't check out it's going to go ahead and reject that out right and we're not even going to get that far okay you can see that my internet here is kind of slow so

it's going to take a little while to build um so maybe while that's building um yeah we can go over here all right so I've got a little demonstration here of what it looks like to sign the image as well so once you've Dem once you've um created your your key within obtainer which is just a another obtainer um key command you can go ahead and sign it like this it's very very simple to do that okay so now I've just built this new container this Rocky Linux 9 LOL cow so this is this is a new version of the LOL Cal container which is

based on Rocky Linux instead of a different base let's just test it to make sure that it works sorry I can't type and hold this hold on okay so it it looks like it does work work so now I'll go ahead and I'll sign this [Music] [Music] container third times a charm okay hopefully everybody was able to clock my password there as I signed that the third time all right so now um I've signed the container and now uh that's not really useful okay so let me go ahead and push this container back up to docker Hub and notice what I'm doing C registry of

storage so I can push arbitrary data blobs in this case it's a CFT file and I'm going to push that up to uh dockerhub and I'm going to give it the tag in this what was confusing me at the beginning of the talk because I tagged it in a weird way but I'm giving it the tag Rocky Linux 9 to show um yeah and I've got to actually give it the right name of the container um to okay here we go we're uploading maybe um one but it's going to take about 40 minutes no big deal you guys have time right so let me go

ahead and go over here turns out that I've already uploaded one obviously because I I tried this a few times several weeks ago when I first put this together so I've got this LOL Cal repo and if we look at the tags was pushed okay so I put this talk together 20 days ago that's why I'm a little rusty on it sorry about that but in any case this is the workflow that I am now advocating I'm I'm security of knowing that your base containers are coming all the way from the Upstream mirrors you're not trusting any intermediary for your base containers but you're also

getting the convenience of container stored all right and we can check back later to make sure that this get got uploaded that's pretty much all I've got for this talk so very much for coming and attending um if you got any questions uh I could pass the mic around and we could ask questions and um go over them so what's the difference or what's modification changes you have made without being political between uh Singularity and obtainer what are the advantage sure I mean I think the best thing I can do is I can just kind of go through the history so um you know there

there's a lot of confusion about this so it' be good to talk about that a little bit so decided to Fork Singularity um essentially because they kind of wanted to get changes in a little bit faster than going through the the normal uh the normal uh poll request review process CE so it's not Singularity Singularity CE but they didn't change things like the commands that you actually type and that caused a lot of confusion um after that the The Singularity Community approached the Linux foundation and said hey would you consider supporting um Singularity as a project to look at Singularity this was to ensure the

open source nature of this project the Linux Foundation had a look at the project and they said we would love to but we'd like you to change the name so we took a vote and we decided to change it to abtain so obtainer does that answer your question great are there any other questions yeah great I'll be doing uh demos of Apper um you know throughout the entire uh program and I can I can definitely do some on demand what testing whoa testing testing all right thank you everyone for coming I'm Chris um my day job is the deputy director at the Massachusetts green high

performance Computing Center and by night I'm a main container for open HPC and as of this coming February will be the 2-year anniversary of me taking over as Project Lead um open HPC is the largest open- source ecosystem for building supercomputers and essentially what we do is provide two things we provide validated bare metal recipes and we provide software right software repos um and before we go on I just want to make everyone aware that tomorrow we are having a happy hour from 6:00 to 8:00 p.m.

um if you've been to Denver before uh for SC you might have been to wind coup Brewing Company it's a pretty cool place about a mile from here um because it is a mile and there and back and everyone's already done a lot of walking we're offering free Uber rides there and back so feel free to scan this OR come by our booth we're booth number 871 right over there uh pick up a card and hope to see you there tomorrow so open HPC is supported through uh commercial sponsors right we have the standard sort of platinum gold silver level of memberships uh academic institutes

and or nonprofit in government Labs can join for free but you know these um companies use open HPC as a basis for them building their commercial offerings um most of these companies have been with us for a long period of time uh last year Huawei joined us um and I'm going to talk about some of the developments that came out from that but basically we've been adding support and testing for their cuming uh arm architecture as well as uh support for their Enterprise Linux open Oiler um the open HPC project is supported by volunteers um and the volunteers make up the tsse the technical steering

committee and several of the tsse members are here in the crowd ree Jeremy please raise your hand thank you very much for all your contributions if you would like to be cool like Reese and Jeremy please consider testing there we go please uh consider contributing to the project we are 100% volunteer effort we are always have way more work to do than we have people who who can do it um and we do tsse nominations uh in June every year so join our mailing lists in early June I'll send out an email looking for tsse nominations and you can self-nominate you don't have to contribute

to the project to be part of the TSC we have several members who are in user site Representatives I don't know why this keeps going out there we go um they provide feedback on changes we propose uh because they use open HPC for their academic centers now open HPC supports multiple different versions simultaneously now the original version was the 1.x branch which was focused on centas 7 and sles 12 and in October of 2020 we released 2.0 which initially focused on Centos support um and by the date of the release and me Xing it out you kind of know what's coming down the line um

in addition to the 2.x recently we released the 3.x branch and with 3.0 we introduced uh support for real 9 and real 9 clones as well as support for uh Susa leap 155 and open Oiler uh the latest LTS release open Oiler is what huawei's open source um it keeps going in and out uh uh open Oiler is huawei's uh open- Source uh uh rail clone now looking at the project the project uh initially released at supercomputing in 20 at 2015 and since then we've seen uh pretty impressive growth snapshots because we're an open source project we don't collect telemetry information and we rely on

people to submit that they're using it on their systems and of course that captures very small fraction of our user base so what we do to look at adoption is the number of visitors per month to our software repos as well as the average downloads per month and as you can see sort of year-over-year we've seen steady increase and for this year we're currently having about 1,000 unique downloads per month uh and around 112 uh terabytes of soft W repos that are downloaded per month now as you may remember I'd said that 2.0 was originally released in October of 2020 and in December of 2020

is where we got the notice from real that Centos would be retired and replaced by Cintas stream and so we didn't there were several different directions we could go based on those changes we could continue to build and test and support on senta stream we could move to real proper and uh use the Enterprise version or we could you know test and build and release on the real clones of course which Rocky is of one and that's why I'm here talking today um instead of making a decision right away for 2 21 22 and 23 we continue building and testing on Centos 8 but for

24 we made some changes before we made those changes we spent a year soliciting Community feedback at all the sort of annual events so at ISC perk and super Computing we sort of pulled the attendees of our bof and we noticed a couple of interesting things for ISC and perk Rocky Linux was by far the most popular choice of what people were going to adopt and at SC it was re which was the most popular with Rocky coming in second so going forward what we decided to do is build on re and test on Rocky so uh Red Hat donated some Community entitlements to us

so our build servers all use ra proper and our test infrastructure for 2.x was tested on is tested on Rocky 8 um now just because it's tested on Rocky 8 doesn't mean it's not going to work on alar Oracle right all those work and we have users using open HPC with Alma Linux as well as um with Oracle Linux and so far this has been working pretty well and of course we continue to support open Susa as well now open HPC I said distributes validated bare metal recipes and a lot of work goes into creating these recipes so here's a snapshot of our CI infrastructure

or CI right cyber infrastructure um we have used OBS to build our packages uh our OBS server runs uh at AWS on an ec2 instance we have Hardware that's been donated by by Huawei um and we have some Intel Hardware that's hosted at Tac and then we have additional Builders at ec2 for both arm and x86 so all the packages are built by OBS and we run some small tests on packages at the single package level but not really integrated test yet then we move the RPMs from the OBS server to a staging area on our re repository server and that is when we invoke

our bare metal testing so we use Jenkin for our testing and what we have is an infrastructure where the sources for our documentation are parsed so our documentation is written in latch and we use Pearl to actually parse the tech sources and take the commands that you run as part of the bare metal recipe and use that to build bare metal clusters so we use latch and pearl because we're old right and these are kind of the tools we used historically ly and so before every release all the text sources of our recipes are parsed by a pearl script and then we deploy bare metal

systems on bare metal systems on x86 and arm for all of our supported dros and then we test both ether we test both ethernet and infin band interconnects and it's only after we build these systems and then run our test Suite that we deploy our test Suite consists of over 1100 individual tests that we run on every linear combination of supported architecture interconnects operating systems and Q systems so there's a lot of testing that we do and that's mostly why the commercial companies you know sponsor open HPC because we do this bare metal validation that few people are willing willing to do all right so

a couple of weeks ago uh on November 9th we released open HPC 2.7 supports real 88 and of course Rocky Alma Oracle whatever uh as well as leap 153 um as you can see we support several other ciq products we use werewolf 3 as our supported provisioner um and we currently have werewolf 4 as a Tech preview you can download and use and install werewolf for as part of open HPC but we don't have recipes yet and we're not doing the testing yet on werewolf 4 um we also support Singularity as one of our container run times um additionally we had several other changes that

I'll talk about in a bit mostly around the Intel 1 API toolkit and then here's all the other various packages that we build for all the linear combinations of architectures operating systems as well as MPI stacks so we support multiple MPI Stacks open MPI impit mvapich Intel MPI so with open HPC 2.7 comes eight different bare metal recipes for all the linear combinations of operating system Rocky and leap architectures x86 and arm and uh Q systems open PBS and slurm and when you install open HPC if you install the docs RPM then those uh bare metal recipes are available on your system and also the

Pearl scripts and the test infrastructure we use to build is available as well in 2.7 we updated the target distributions to 88 and leap 153 we introduced nonmi builds for uh the various uh net CDF packages we upgraded elmod to handle GE EnV so the latest version of elmod um we rebuilt all of our Intel pack packages with the latest release of Intel compiler 20232 we added papy 57 um after we had some bug reports for papy 6 so we still distribute papy 6 but we also distribute papy 57 as well and there are a couple of outstanding issues on 2.7 um we're still using

for 2.7 we're still using the Intel classic compiler right I for ICC ICPC um there's a couple of packages which are having regressions and we were not able to get them to pass our numerical differences and those are plasma towel and super luu disc so for those packages we recommend you still use the G new variant also on 2.7 uh and starting with 2.6 we have a sort of different different integration with the Intel compiler so if you remember previously open HPC supported psxe parallel Studio extreme Edition and with that you downloaded the software from from Intel and you needed a license right and you

would install that separately and then you would install our packages which were compiled against it now when you were installing the parallel Studio compilers from a tarball essentially you could choose the install path well now Intel has released the 1 API compilers they're available via yum and zipper repos and so we can no longer in control the install path so those go in opt Intel right and uh so in our recipe we've changed it so that we also in export up Intel to the compute nodes right now with the Intel 1pi compilers they have their own infrastructure to generate the elmod module files so we

now use that and we use the modules the way Intel does it not the way we used to do it so if you're going from parallel Studio extreme Edition open HPC modules to the new 1 API modules you'll notice many more packages get loaded and that's just the way Intel decided to do it additionally um arm now provides their commercial compiler uh for grus arm now provides their commercial compiler gratus as well um just like one API but it's not Libra and you do the same thing there as well you install their compiler and then you install our packages which have been compiled with the

arm compiler now um in uh in SE l in September we released 3.0 and 3.0 was about a Year's worth of work so 3.0 introduced support for real 9 right Susa 15 Susa leap 155 as well as open Oiler and with 3.0 we also started testing on cening arms uh huawei's arm architecture and then 3.0.1 was released uh October 18th and that includes the patches for the slurm cve that was uh announced uh about a week before that release now the packages we distribute with the 3.x branch look just like the packages we distribute to the 2.x same same p uh same packages same sort

of combinations of architectures uh Q systems some some different operating system versions um but everything else works the same except now with 3.0 instead of eight total recipes we have 16 total recipes so we added support for open Oiler but we also started testing and releasing recipes for Alma as well if you remember during the summer rails sort of changed the way they were releasing their Source RPMs and as such the clones have diverged in the way that they're getting those updates so the rocky team is doing it one way the alma team is doing it another so we're still building on real but we're

now testing on Rocky and Alma as well highlights for the 3.0 uh1 release uh as I said we support three new major OS distributions Enterprise Linux 9 leap 155 and the latest LTS release of open Oiler the 3.0 um targets new distributions so you can't do just an update from 2.0 to 3.0 you have to do a clean install you have to start over um the uh 3.x builds for Susa um are now built against leap instead of sles we used to build against sles and test on leap now we build and test on leap um and the let's see here yeah nine nine builds

are performed against Real tested on Rocky and Alma um and uh the slurm based installations with 3.0 uh re-enabled pimi support a couple of Rec releases ago we had to um disable Pik we had to disable pimi support because of a regression and it's been fixed on the slurm side but there's still issues on the open open PBS side so open MPI on the open PBS side does not come with pinx along with the 3.0 release comes a tech preview of werewolf 4 um and we're currently working on the recipes and the documentation from that on the 3.0 Branch we've retired the Intel classic compiler

and we're now using the clanged back uh compilers um from 1 API we removed adios and replaced it with adios 2 um again we reintroduced pimix support um and introduced uh gnu 13 as a compiler variant as a tech preview so gnu 13 and the MPI Stacks compiled with ganu 13 are there but the independent packages that we use aren't there are a couple of issues right now we've only been testing the alma recipes on arm we don't have the x86 infrastructure for that just yet in vapage 2 one of our MPI Stacks is not supported on open Oiler and open MPI 4 has some

regressions with the Intel 1 API compiler so that's not distributed now along with the release of real 9 red hat made some changes to introduce support for a new microarchitecture so in 2020 AMD Intel red hat and Susa got together and formed a committee to set up three new so x86-64 V2 add some SS streaming uh streaming simd extension support x uh 64 V3 supports avx2 and then V4 supports AVX 512 now historically we've compiled um open HPC packages with the lowest level of optimization to sort of hit as many users as possible um our red hat packages on 3.0 are compiled with V2 but

we're working on the infrastructure to make it really easy as an enduser to recompile with V3 or V4 support to get those microarchitecture uh level optimizations our near-term road map for 3.1 so as I said we plan to make it easier to release with these optimizations V3 and V4 we're working on the werewolf 4 documentation and recipes and we're working on a new documentation framework so as I mentioned we we historically rely a lot on Sphinx and latch and because of that contribute please reach out cu we really could use some help here um and then finally in the near term we're looking to update

our CI to support you know right now we don't have any recipes or documentation for NVIDIA or am open HPC but people would like to see the recipes to make it easier to do all right and with that I will take questions oh okay so the question was why was Singularity crossed off uh testing testing okay so um historically our our the the our approach is to not rebuild packages that have sufficient Upstream versions right that are new enough to meet our needs historically The Virgins and singularity in Enterprise Linux 8 and Sousa leip sort of you know lagged behind what was the current release

version and in particularly they uh you know the ones in the major repos didn't support build rout right now the versions that are in the Upstream repository of all our OS distributions are now new enough that we don't have to inst build it install it directly as part of open HPC instead our meta packages Pull It in from the OS repos so an open HPC system still has Singularity it's just we no longer build it and and and and um provide it other questions ask coming so the question was was how's Debian support coming so this is a question of course we get every year

um and we are actually working on it finally so um right now we're not doing any bare metal testing but we do have a group who is working on porting all of our RPMs to Deb files and so we hope to have sort of um progress of these efforts uh available for testing by middle of next year a like ACH today CV that kind of thing so the question was do we commit to a patching interview for cve um we we do it as as best effort right you know as I said it's an entire volunteer effort when the slurm CV when the slurm cve

was announced we had new packages available less than 24 hours later now those packages were available on our OBS server and we made the the community aware of them via our slack and our mailing lists we weren't able to release as a full as a full update within a day because we have to run extensive CI and all the linear combinations right eight different combinations for 2.x and 16 combinations for 3.0 so that took us an additional week so we were able to release it for both 2.6 and 3.0 for all 24 recipes within 7 days which I think is pretty good for a volunteer

project other questions Speak now or find me later available to contri to how we so the question was if we have Hardware we want to contribute to CI how do we do that that um you can just mail them to no um so typically what we do typ typically what we do is if people want to host it we let them host it and we just need to poke a wall in the fire the poke a hole in their firewall to allow um our servers to connect with it we have to have SSH open um and that's we can do most things through an SSH

tunnel like all of our CI that's hosted in China we have one port open and we do everything through an S Tunnel um and of course it's slow right so um if you want to send us Hardware physically we also have Partners who will host the hardware for us um we we always need Hardware right right now we have no accelerators we have uh x86 pretty well covered on the arm side we have arm Hardware on the arm side we have arm Hardware but we don't have any infiniband cars in arm Hardware so all of our arm testing is ethernet only um and as I

said our only arm systems are in China and the great firewall makes downloading packages slow right so things things take four or five times there just because of the bandwidth available to us other questions all right well thank you everyone for your time oh and one last plug I plugged at the beginning Neil So Neil's our Linux Foundation um program manager uh he's the paid employee on the open HPC project so thank you very much Neil um and once again we are having a happy hour tomorrow 6:00 to 8:00 p.m.

um you can register with that QR code registration is not required it's just nice for us to have a headcount and and then of course we also are providing free Uber rides there and back so hope to see you guys there good afternoon welcome to C Boos and uh we have here to discuss with you a life science platforms uh this is the Integrated Solutions I like to First bring our Cru on the stage is uh Brandon and David harini like I said this is Integrated Solutions in this solutions so we have MD and we have a santian and we have a rocky L if

I spell it wrong correct me and then C Storage are us so move to the agenda so we're going to discuss in this presentation that where we see the challenge is in this life science analytical workload uh we're going to take a look at what are the some of the key challenges and what is the answers we have to address those challenges and the first I'm going to give the floor to Brandon and Brandon C to walk you through those challenges Brandon okay yeah thank you very much Tony um are you you're going to work the slides or do I I don't know okay so

yeah I'm Brendon Gallagher from sanon we're a b matics companies it's a application for um Big Data HPC we we write very efficient and um elegant and accurate algorithms to take advantage of um Hardware like AMD so we're doing we're all a software only solution that then runs on top of all these stacks and what's going on is basically there's been a very big boom in genomics and genomic data um production over the last let's say 10 years the field is really moving into um production where it's impacting people's Healthcare it really if you're here and you're a healthy adult it's not going to change

your life too much immediately there that's more just um you could get some information about your genome but literally what our software does is Define um the sample whatever it is either a human gen um the sample whatever it is either a human genome it's also very important in oncology so in cancer um cancer is basically a genetic disease where um your cells have mutated and so your genome has changed and then they are out of control and dividing and you know consuming your organs or wherever they are and spreading over your body so that's obviously everyone's familiar with that and how important that is

and then also um there's been a lot of progress on I'm kind of rare disease so that's more in children but in with a rare disease um that means that you really don't know um the problem and or you just see that they're sick for some reason and if you analyze their genome that can give you ideas on what kind of treatments to give them and it could be something just like um they're not good at um processing iron so it means they either have a typ like low iron diet or high iron diet and that's the kind of thing that if you're doing these

big data analysis like you're you're turning through all this data and then that's kind of the end output right is that you get to improve a child's life or give someone a better cancer diagnosis or help um in just drug Discovery or whatever like to create more better new drugs that's the whole idea of personalized medicine instead of treating you like um whatever the statistics say and the insurance company says hopefully will'll move to a point where um it's more personalized um and so in that um there's a ton of data gets generated basically your genome is 3 billion a string that's three billion long

and there's two copies of it and so the way we sequence that typically is kind of randomly um enriching and creating a ton of little pieces of that genome and that ends up making about 200 GB of data um and right now so people are like there's big kind of jum factories around the world that are producing this so it's you know terabytes and terabytes of data every day and then that needs to be transformed um from raw data to kind of the definition of the genome and then that's what our software does so we take it from that raw data called a pass Q

file and transform it into a variant call file which is basically the list of um variants that make you unique right like how also how everyone is kind of 99% the same but out of 3 billion right then you have you know 3 or 4 million differences between you and everyone else um so our software really defines that um and I'd say you know it is a big challenge for a lot of people but part of that challenge is just identifying the right software to do it and i' would say our customers combine with a stack um like what we're presenting here take this big

data Challenge and make it um very easy and very production ready and now you know we can process a genome for a little over a dollar basically with AMD and 14 minutes or 12 minutes that's the other one just 12.9 minutes uh so it can be done very it can be done as fast as you want um and also as cost effective and as efficient as you'd like um so maybe is there another slide yeah so this is U my slide I would say so this is where this you know this you take a big stack to kind of um take care of everything there's

also a huge storage component right that's probably why we're in the um Seagate Booth all of this data needs to be stored and it also needs to be um stored for a particular amount of time often especially if it's once it's part of a clinical diagnostic process there's regulations about how long you store the data so that becomes an expense and you really want to think intelligently about how to um store this big data um and then that's where we fit in it's kind of also like a big relay race right where everyone uh you know there's all these different experts around the world come

together to make a really good production solution that then just works right and it's almost like magic but it's because people are kind of doing this relay race from creating the sequencers and making a very high quality sequencing system to companies like me who are doing the secondary analysis which is the very um elegant accurate um compute algorithms and pipelines to do the data processing um then you have all of these regulations right about how who to share the data with why like moving it around um and then also interpreting it uh which is kind of like the biological importance of it right so once

you get a high quality list of variant then you go and have to say what does this mean so that's like what I said before about um there's an iron issue with a child or this cancer is driven by um you know over production of this gene or whatever um and that's all kind of a big um it's kind of a you know a big stack of Technology that's here and it's um really trying to help improve human life basically um across also all kinds of Industries right it's not just um human genomics but it can be um agricultural right um all kinds of things

um there's so anyway um I think this is still ours or this still my uh slide um I think the important thing here is again just if you're already here listening to me um you should check out our software we would welcome all benchmarks or you can just start using it and it'll uh give you very nice results already um but what we do again there's all these very um not esoteric but like just specific kind of applications and we are a big software suite that handles kind of all of these applications cool thank you very much oh hey all right I'm giving this yeah

today so um you guys probably saw this once and BR didn't talk you enough about the analytical world and how you get that data from NS and to the analytical Appliance engines so you can further process the data and get the business intelligence into it and get what you wanted however there's a challenges of doing that if you take a look at this one first one a lot of people want to have a data in the memory type of operations for your life science workload correct anybody here doing that analytical work no okay first one the first level of challenges how to get those tons

of data into a memory so you can f process that the Second Challenge is that how do you get that data into the storage after you process that you got to have some space a room to accommodate those datas of process the datas here comes to us right this is a Integrated Solutions in this solutions we have multiple pieces over here starting from the left you see that's where the andd process servers uh we call it work be uh in the um GPS from the GPS P perspective you will call it work be work note okay over there is all you have is a just

the operating systems uh with your business applications so for example in this case zenti applications running on each every one of those nodes all of those nodes has a 25 gig ease in the front to process to transport the data after the process then in the middle the second columns is your network stats where you actually exchange data so with gpfs primary node or we call it second notes it's a cluster IBM gpf as a clusters that's where your process main process engineers and right on the Right Far Right columns that's us c g and uh what we listed over here is a tiered approach

from uh the hot tier to warm tier to cold tiers and we build this for with the consideration that some of your workload may need a fast access to the data because you can only do enough on your local cashing because you understand as you load up the pile of those D Rams your cost is going to go up right so what we offer here is a tiered approach to dress your statment cost issues to do some cost controls only those datas are hot extremely hot datas goes to the solid state all right and then you have a capacity Place either scale up vertically in

a box gu a with a sayate per box you can scale up to paba of storage once that you used up those PAB capacities you can horizontally scale out to do the works to accommodate works you probably heard about the the numbers from IDC is that with every one terte of processed and electrical datas your capacity for the whole genomic data is going to increase dramatically into something like a path by territory so with the uh architecture Solutions like can you go back can you go back to the previous slide so I thought IBM have their own hardware for the uh gpfs what does Cate

SE in here what's the relationship good questions thank you IBM gpf as it does I call the gpf as because it's just a im better in my mind that now IBM call it IBM storage Spectrum or skill right they do have their Hardware but this is the beauty where SE the ccil what you have a seate we are application of file system agnostic you don't have to worry about the customer loggings right if you want to switch you just switch at will if you want you choose to use IBM Hardware go for it we don't have a problem we don't have a login if you

ason you have a good data migration tools you can just migrate the data us at will we offer you a cost effective Storage Solutions with a building quick fast rebuild don't forget about the seate adaptive technology that feature helps you to reduce the amount of times for your disco drive failure rebuild okay uh you probably heard to build a three terab of a DAT takes like three days that's no longer the case with seates storage we can rebuild 3 terb of stor with like 10 minutes and half an hours okay that's the beauty that you have eight options besides IBM's hardw so good question I

installation for you who does this installation IBM or Cate for uh IBM storage skill so again we can have a conversation to support that um and then work through your preferred reseller or your preferred part Tony I'm G to go ahead and pass it over you to oh I'm sorry her's next well hello everyone I'm from AMD harini Malik and I manage the Strategic business development for healthcare and we talked about the challenge that we have with genomics there is the compute challenge as well as the Big Data challenge the Seagate clusters actually solve for that big data that's getting transferred you know for from

from a genomic standpoint to go solve this the AMD CPUs the fourth generation CPUs solve the computational challenge so with the fourth generation CPU um from AMD we have up to 128 cores in this particular case we are using the 96 core 9654 from AMD and this allows for that core the number of cores actually allow for that computer Ute capability that this problem ask for we have 12 ddr5 memory channels transferring at 6,000 megat transfers per second and this allows for that fast data transfer between the CPU and the memory to help accelerate that genomics code in addition we have over 160 pcie Lanes

so for that fast transfer of data between the CPU to memory to the hard that 165 160 pcie Gen 5 Lanes helps with that that transfer rate as well in addition we have other unique security features that um our processor brings like uh the secure encrypted virtualization as well as encrypted memory to help for that secure data transfer as well so now let's look at what this solution enables I mean so you saw what that reference architecture looked like and from an AMD standpoint this is just a 2p general purpose AMD server with the fourth generation epic processor with the sention software no gpus nothing

else and we can complete this in 14 minutes and 21 seconds and that is about 100 genomes per day this is the 30X genomes that we are processing let's compare this to what is publicly available the Oracle Cloud infrastructure with eight A1 100s guys eight and um it's say it the AMD solution can perform 25 more genomes per day compared to 8 a100 Nvidia Solutions running Nvidia parabricks isn't that an excellent data and by the way I think ction just measured did some recent measurements on an h100 an 8 h100 um solution and we still perform a minute faster than the 8 h100 Solutions and

what was the cost um it was 15 times cheaper so like in Amazon uh the Nvidia um h100 is $96 an hour and the AMD 1 is $7 an change an hour so since we're a minute faster and 96 / 7 and A2 is turns into um about 24 times more cost effective isn't that an excellent way for you to go actually be able to do the genomic sequencing in a much faster fashion with with this architecture that we just outlined thank you thank you her wow uh 100 genomics just on that single note nowadays the popular saying is do more with what with a

less right you don't have to have like expensive gpus of course if you have a deep pocket that's your freedom but we offer here a alternative Solutions and without the front upload tons of monies outlays and get the business going and get to where you want to be right so we are doing more with less money so the takeaway is that if you're in the life