Prioritize Hiring and Retaining Great Employees Over Copycat Layoffs

January 12, 2023

Join us as we discuss recent layoffs in the tech industry and how you can prioritize hiring and retaining great employees during this time. Our panelists bring a wealth of knowledge and are happy to answer your questions during the live stream.

Webinar Synopsis:

How To Prepare For Employment In HPC
Dr. Sill’s Thoughts On Hiring And Retaining Talent
How To Bring On Talented People
How To Encourage Competency In An Employee
When To Let Employees Go
Retaining Talent
Efficacy Of Continued Off-Site Training And Certifications
Mastodon Server, hpc.social, Slack And Discord As HPC Resources
The Future Of Layoffs In The Tech Community
How To Prepare For Employment Opportunities In HPC
Will Innovations LIke chatGPT Lead To Layoffs?
Academia Versus Industry For Experience

Speakers:

Zane Hamilton, VP of Solution Engineering, CIQ
- Linkedin
Jonathon Anderson, Solutions Architect, CIQ
- Linkedin
Forrest Burt, HPC Systems Engineer, CIQ
- Linkedin
Alan Sill, Managing Director, Texas Tech/NSF CAC
- Linkedin
John Hanks, HPC Principle Engineer, CZBioHub
- Linkedin

Note: This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors.

Full Webinar Transcript:

Zane Hamilton:

Good morning, good afternoon, and good evening, wherever you are. Welcome to 2023 and another CIQ webinar. My name is Zane Hamilton and I'm the Vice President of Solutions Engineering here at CIQ. Here at CIQ, we're focused on powering the next generation of software infrastructure, leveraging the capabilities of cloud, hyperscale and HPC. From research to the enterprise, our customers rely on us for the ultimate Rocky Linux, Warewulf, Apptainer support escalation. We provide deep development capabilities and solutions all delivered in the collaborative spirit of open source. Like I said, welcome to 2023. This topic is going to be interesting, not a normal topic that we have typically talked about. It was brought up by one of our panelists, and I'm interested to get into this and see how things go. We're going to be talking about prioritizing, hiring and retaining great employees over copycat layoffs, so we can bring in the panel. Hello guys.

Introduction Of Panel [6:09]

Jonathon Anderson:

Hello.

Zane Hamilton:

How are you?

Forrest Burt:

Hello.

Zane Hamilton:

Welcome to 2023.

Jonathon Anderson:

Happy New Year to you as well.

Zane Hamilton:

I don't know if you guys had an opportunity to go read the article that Fernandez sent over. To me this is something that it hits a little bit close to home. I've been through layoffs from having to lay people off far more times than I've ever cared to do. Reading through this, it brings up some interesting and emotional times for me. I'm curious if we look at things from a hiring and retaining perspective. I know that we've been going through a lot and doing a lot of hiring and then making sure we maintain the talent that we have. There's a lot to talk about just on that side. I'd like to start there and talk about what we should be looking for? What should we be looking for in terms of HPC? Because, we see this in our customers right now as well, if they're going through some interesting times. I'll start with you, Jonathan. That's a lot. I know there is a lot there.

Jonathon Anderson:

Yeah. I personally haven't been through a layoff situation, so I don't have that experience directly. I know that when, for me, I'm always looking to find a place that I'm staying long term. I like to find people that are looking to make a home as well. That's part of the story that we've been telling when we sell CIQ services, especially right now when people have been having a hard time holding onto people, is that we can help them bridge that gap in their staffing by having a steady hand in a services organization of, you'll see these people and they're not going anywhere, that kind of thing. I just look for people who are passionate about what they're doing, and are looking for an organization to support those activities and where those things align. So far, that seems to be what's happening here.

Zane Hamilton:

Absolutely. I'm excited to be here for that same reason. Welcome Dr. Sill good to see you.

Alan Sill:

Greetings.

Zane Hamilton:

Welcome to 2023.

Alan Sill:

Well, yeah, that, well, the light is out there.

Zane Hamilton:

Maybe. Forrest, you have an interesting perspective coming out of school recently. Going through the process of looking for someone and somewhere that you wanted to work, what did that look like for you?

How To Prepare For Employment In HPC [8:33]

Forrest Burt:

It was an interesting process. Just for background, I graduated with my degree in computer science in May of 2021. Before that I had worked for about two and a half years at my university's research computing department as their student system administrator on their HPC systems. I got into that position because I came to college just very interested in supercomputers in general. I already had a background in computing and programming and things like that from high school and before. I was very interested in that kind of stuff. Coming to college, I'd found out about the San Diego Supercomputing Center at some point, but I didn't really know what a supercomputer was until I was working on a research project like I said, about two and a half years into college.

They asked me if I wanted an account on the university's computers. I said, oh, Boise State has a super computer. What? This is incredible news! Within very, very short order, while I was a little bit bored at the office assistant job, which I had at the time, I on whim pinged the research computing department and said, Hey, do you guys happen to have any jobs available for students on your supercomputer? It just so happened that that day they had posted the student position that I ended up getting, just so locally to when I emailed them that it said posted just a few seconds ago on the posting that they sent me.

I was, oh, perfect. I applied. I got in, like I said, I worked for about two and a half years there. There had been discussion for a long time of me converting to a full-time role once I graduated. However, in the middle of the covid with budgets and odd stuff like that money was not coming in. It was the covid. It was an odd period that ended up falling through shortly before I graduated. I wanted to stay in HPC, so I started looking for other positions. I actually applied, amusingly, at where Jonathan was working at the time and got to be interviewed there for a little while. I applied at a couple of other places that were also doing HPC. I applied for some more conventional software engineering type jobs that were more locally to where I was living at the time.

Ultimately, what ended up coming through was I got pinged by Greg on LinkedIn. I'd always kept up a very sophisticated LinkedIn profile. I'd always been very up to date with what I was working on at HPC and stuff like that. I got pinged about CIQ and was asked if I was interested in a research computing type position there. I was extremely interested, much like Jonathan articulates. I'm very passionate about HPC. When I saw some of the things, which we were working on or that CIQ is working on, I was, oh, these are the people that obviously have the passion for HPC to work at the cutting edge with this type of thing. I knew that I was very excited to potentially join them.

Shortly thereafter, I ended up at CIQ. I, like Jonathan said, I think passion is one of the biggest things to look for in people, especially in HPC. It's a very small crowd of people in general that are in this field. Just by the nature of how people get into it, people get out of it. It tends to be a very small field. If you find people that do have that passion for HPC competency in some of these things that we work on is important. But some of these things that we work on are difficult to work on yourself without access to the resources. If you're looking at people that are early in their career, it can be difficult to get onto a supercomputer to get some of that experience. If people have the passion and the interest in supercomputing, and they have the basic tech background to be able to thrive and understand what's going on in that environment. That's one of the strongest things you can look for. It is just that passion for HPC and for seeing things run at that scale.

Zane Hamilton:

I think that's interesting. I know Dr. Still has interesting views on this as well, because he's been going through this for a while, and I know that you have some great people that are on your staff too. I think from your perspective, hiring people and then retaining people, you've obviously done a good job with that because I love Misha and he's been there for a while. I'm interested in your thoughts on this.

Dr. Sill's Thoughts On Hiring And Retaining Talent [13:19]

Alan Sill:

Yeah, I work in basically two roles. One is as managing director for the high performance computing center here at Texas Tech. We run the clusters just shy of 50,000 cores, and just shy of 10 petabytes total storage of a few dozen GPUs. That's my day job. Also, I run a center that's focused on cloud and highly automated computing called the NSF cloud, not a computing center. I see both. In the first role, I'm constantly trying to hire people and in the second role everybody we graduate through that program walks into a six figure job. Now, these two observations are correlated. It's difficult for academic research centers to hire the right type of people because there is a tremendous competition from industry in identifying those.

We look for people like Forrest. I won't openly try to steal him from you on this call. Nobody knows what to call it. Is it high performance computing? Is it academic research computing? Is it research computing and data? All I know is we can't get enough of any of these roles. Is it data science? We can't get enough qualified people. I'm very lucky to have a talented team, but it's a small team and it's heavily worked. I worry about burnout. Burnout is possible at any job, certainly possible in industry jobs. It's also possible in academic support jobs because we have thousands of researchers that depend on this tiny staff of ours to keep and by the way, they expect you to know everything within two sentences, they expect you to know their entire field and why this particular little thing they were trying didn't work.

They were smiling because they knew it's true. But on the other hand, that creates the need for companies like yours. We need people that we can turn to. I guess the point I'm trying to make is it's part of an ecosystem. If you focus on one narrow aspect of it, you'll miss the picture. It's using advanced technology to support academic research, but we use many of the same tools and techniques that you use in industry settings. Ideally as part of that ecosystem, our students go into that ecosystem. It would be a disaster if they all stayed in academia. But then trying to lure people in who have the right talents to support our work is a challenge.

Zane Hamilton:

From bringing people in, once you get people, I think the next part of this is that there are a lot of talented people out there. How do you keep them? And I think, John, welcome. Happy 23. Love the hat. It's good to see you again.

How To Bring On Talented People [16:29]

John Hanks:

Alright, good to be here. My thoughts on retention there are two things I want to touch on. I'll start with the one that I find the most annoying. I've worked in and around a lot of environments where you see the impact of what I call the competency filter. Where you hire a hundred people and randomly. Ten of those people are really great. 90 of those people are awful. Of those 90 people who are awful, this may be the best job they ever get. They're not leaving until you fire them. Those 10 people that are competent really quickly get sick of dealing with those 90 people. They can leave. They can put their resume out there, interview, get another job, they leave. Over time, this filter builds up this layer of cruft and dead wood in an organization that just sits there and doesn't do anything.

If you've ever worked in or around a large IT org, especially in academia, you've seen this because in academia, it's really hard to fire people no matter how incompetent they are. These people build up and they just block everything and they make life miserable for people who actually can do stuff and actually want to get stuff done and want to make progress. That's my gripe about the way people do things. Now, specifically to why that happens, when people look at retention, everybody, the first go-to is money. It's not always money. I mean, the people who really enjoy their job and are passionate about doing something, they want the ability to do the thing they're passionate about. Yeah, they need to get their bills paid, but really they just don't want to waste all day, every day doing stupid processes, checking boxes, whatever they want to do interesting stuff. Too many environments to focus on, are we going to throw more money? Are we going to throw more benefits? How do we retain this good person when really all that person wants is to be left alone, left out of meetings, to just do something interesting. They can be happy at work every day.

Zane Hamilton:

I think you've said a lot of different things and touched on a lot of different topics, and I think keeping people happy is one thing. I do not know if you guys have seen the meme or CEO talking to a CFO and CFO says, well, what if we train them and they leave and the CEO says, what if we don't train them and they stay? I think that's something that goes to what you're saying, John, if you have people maybe that 90% that you were talking about that aren't capable, is there a way to make them capable? I know there's also going to be another subset of that number that possibly don't want to, or they don't have the skill, but is there a percentage of that we could find a way to incentivise from making their job more interesting or giving them something better to do, finding out what they're passionate about, and, and get them to a level where they could benefit.

How To Encourage Competency In An Employee [19:24]

John Hanks:

This is where my view of the world is completely skewed because I hear people talk about work-life balance, and for me, there is no difference between work and life. My cluster is my life. I don't know what you do. For somebody who has a work-life balance, that person, how do you retain a person that has a work-life balance? I don't know. Let them pursue their daily life passion at work. Yeah, man, I'm at a loss there. I look for people who are weird like me for whom this is not a job. This is a way of life. They enjoy doing it. And finding those people once found, when you find one treasure that is a unicorn.

Jonathon Anderson:

Yeah. Somewhere, in one of the CIQ not performance management, but we're doing some form of our values and things like that in different departments within CIQ and someone wrote on one of them, there is no teaching, there is only learning. That has really stuck with me this idea that you want to find people that are themselves, they're eager to learn new things and are learning them themselves. If you've got a big, I'm not quite as, as cynical about it as our friend John here is. But, if you can't make people improve, you want to find people that are improving themselves. Even if you could make someone improve, that isn't a process that scales you would need like a training organization as big as the rest of the workforce to do it. I'm always eager and interested to learn new things, and I look for people who are the same.

Alan Sill:

I want to focus on the part I agree with John on, because there's a lot that I disagree with, I think, but in a friendly way. I agree that you'd keep people by giving them interesting problems, and in fact, you decide who you are going to keep based on the response to that. I disagree in the sense that I think it's part of my job to keep my staff healthy and not burned out. Yes, they all have this instinct that you talk about, which is a characteristic of the employee that you want. We select them for that. I tell them this story about the researcher's expectations, and I tell them that if you're not going to be happy in that setting, you should find another way to be happy. If you like that kind of thing, you'll like it here. But, I actually think that I'm not interested in having them without a work-life balance. I am interested in making sure that they're, let's say, say sane when I work with them and not about to snap the next user's head off.

Zane Hamilton:

Makes sense. Thanks. John, you wanted to say?

John Hanks:

I couldn't really think of a good way to say it though. Yeah, let me ponder a little bit longer.

Zane Hamilton:

Okay.

John Hanks:

Alright.

Zane Hamilton:

You also brought up another notion that is interesting to me that I've lived through multiple times. It is, if you have that 90%, let's just say it's 2% that aren't capable, have we made it too difficult to let people go?

When To Let Employees Go [22:53]

John Hanks:

That's what I was trying to formulate a nice way to say. I guess probably the nicest way to say it is that very often managers are awful. I mean, you all know this. One of the most awful things a manager can do is leave somebody employed in a job, which is making them miserable. In many cases, letting somebody go is doing them a favor. You're giving them the chance to go maybe find something that they would enjoy doing or that they can get passionate about or get behind and that a lot of managers fall short in that. I don't think it would be fun to fire anybody. I've worked with some people who I personally wanted to strangle, but I still would've felt bad firing them. I wanted to go for the choking. Too often people just linger in a position that they're not happy in, the organization's not happy having them in and it just sticks and stays and festers and gets worse and worse.

Zane Hamilton:

That's something that I've witnessed firsthand multiple times. I think there's another aspect of this is people learn that system. They understand that they can do that, and then they can take advantage. So It would be incredibly difficult to help, or even to manage someone out and try to coach them to go find something else that would make them more happy. They know how to play the game and they can make it last for a long time. I've actually seen employees have multiple jobs while they're playing this game, milking the system for multiple paychecks until it gets caught, and then it's ugly for everybody. That is always the part of this that I look at. How do you even go about interviewing around that? How do you find the person so you don't end up in that situation? Because interviewing is one thing, and then actually working is another.

Alan Sill:

You guys are making me feel better and better about my staffing situation. Keep talking. I might expect the day in a good mood.

Retaining Talent [24:50]

Zane Hamilton:

It's not funny, Alan, but it's funny. No, I think so retaining talent. I would like to stay on that one for a little bit. Jonathan you said something that's very interesting and I think too often we do try to teach and make people learn something. Maybe, that's not even what they want to be doing, but we don't really ask. I feel like that's something that a lot of places, not necessarily where we are now, I think we're very different in a lot of ways. I've seen that happen a lot, and I hear about that a lot. I hear of people forcing certifications on their people that are irrelevant to their job. It's something they expect you to do, they expect you to do it after hours. I think there are a lot of ways to go around that. I'm curious about that kind of thing.

Jonathon Anderson:

For training specifically, I've gotten very little value out of formal training and certifications. I've heard good things about some of them, but it certainly hasn't been a part of my career. I like to look for people who are digging into things with side projects, or even if it's always in a professional realm. They're self-motivated to find ways to improve an environment. When I've hired and in the interview process, I like to tell people a little bit about what we're doing and see what they do with it. Also, to see if that ignites something in their mind and their attention that makes them want to be involved because they see work that they want to do in that environment rather than a job that they could do. In this way, I think I'm very similar to Griznog. I'm looking for people who, when placed in that environment, will naturally want to do the things that I would want them to do because people do their best work when it's what they want to be doing.

John Hanks:

Absolutely. It's funny that you said that because I have said it many times. The times when I have learned the most in a short period of time have been when I was sent to a training that required travel, and I set through the training board out of my mind, but after the training was completely free of family, work, or any interruptions and could focus on just learning something for myself. like They could have saved a lot of money by just sending me somewhere to sit in a hotel for a week without the training overhead. Just let me sit alone and leave me alone for a week. That's the best way to learn something.

Zane Hamilton:

That's a great point, John. I know that we send people to training, and I say we, it's not necessarily here, but as an industry. We send people to training and it's not necessarily something that they've done before. We send them off to get a new skill and they come back and they never use it. Was it valuable? Was it a waste of time? Something to just do something and learn something new? How often does that happen?

Efficacy Of Continued Off-Site Training And Certifications [27:39]

John Hanks:

I've attended, I don't know, maybe certainly in the twenties or thirties training over the course of my career. I can't remember a single one that was actually worth going to. For the most part, the thing that I remember being a blocker in some of them was the prerequisites for this training are A, B, and C. And the room was full of people who had never done A, B, and C and me sitting there waiting for those people to catch up. It's super frustrating. You're wasting a whole day just ready to go to the next step and waiting for the rest of the class who didn't prepare to catch up to where they should be.

Alan Sill:

Well, you're helping me understand all those travel requests. Wait, so is it true Forrest? They sit you in a corner and you don't get to travel for training? You need to get in on this program here that John is an outline for you?

Forrest Burt:

I've definitely been around and about at CIQ. I did a little bit of stuff while I was at Boise State. That was pretty fun to get to be a part of. This past year, especially at CIQ, I was not so much for trainings, but I've definitely got to see some different places. I'm on the program.

Mastodon Server, hpc.social, Slack And Discord As HPC Resources [28:58]

Alan Sill:

I'm in the camp that Jonathan mentioned. The variety of what we see is so huge. Just every day at work is a training session and usually dozens of things, maybe not dozens, but as many as you can fit into the day, let's say. I don't think that that's unusual. I think when this gets to be too much and I don't think we retain a whole bunch of bad people in academia. I don't think that's a problem. Certainly, people do move around. I've asked for an opportunity, I'll take it now if that's okay. One thing we've done, which started at Supercomputing. We launched this thing, to be perfectly blunt, it was because of the crash and burn activities at Twitter, which have been mentioned in the comments by the way driving people away who aren't tied to visas and stuff.

Pretty much my entire use of Twitter was to keep in touch with technical colleagues and friends. I kept my political stuff to a minimum. That just became untenable for many of us. We launched a new Mastodon server and we decided to do it in a wholesale way to have a whole thing. We called it hpc.social. The thing has just taken off in the six weeks or so since we launched it. After Supercomputing, we have 338, I think, accounts as if it's today. We, just a couple weeks ago, decided to launch Slack and Discord channels. The Discord crew is distinctly younger than me, let me tell you. They already have almost a hundred accounts that actually had been started at Supercomputing by students.

That sort of accounts for that. We have Slack approaching a hundred people. We have a jobs board, and that's relevant to this topic here. If anybody is curious and thinking, well, maybe I should be looking around, well, hpc.social/jobs and anybody's welcome to post an HPC related job there, it's explained in hpc.social/projects/jobs. Anybody can use that. We've already had in the two or three days since it's been opened, we've had over two dozen jobs posted. This thing is just taking off. We actually also recently integrated, I don't know if folks are familiar with the HPC-Hallway Huddle that was started by Andrew Jones when he was back at NAG. He's at Microsoft now, but he keeps it going.

It's just been a weekly video meeting of people, and they just decided to merge with us. We'll get a card up there pretty soon. The HPC-Hallway huddle, if you've been part of that, is joining the hpc.social set of services. This is all organically happened by community member activity. There's a community map, there's podcasts, there's a blog aggregation. If you write a blog personal or community blog, or even a commercially oriented technical blog, you can get it syndicated through the blogs project there. If you don't like what you see, just go to the GitHub discussions page there and make suggestions. I'm enormously pleased with it. I especially hope the jobs board totally refutes this idea that somehow the tech sector is in decline. I don't see that at all. Everybody I know who wants a job in our area is getting one. If anything, there's more people in my category. I still have two jobs open, which are posted there too. So please look for those.

Zane Hamilton:

Excellent. Congratulations. Jonathan, I know you wanted to talk a little bit about the layoff side of this.

The Future Of Layoffs In The Tech Community [33:12]

Jonathon Anderson:

Yeah.

Zane Hamilton:

The downer, part of it.

Jonathon Anderson:

As a start for this conversation, we circulated this article internally. I'd seen it before and I skimmed it a little bit from Stanford. I don't know how to pronounce the professor's name, so I apologize to him for that.

Zane Hamilton:

Jeffrey Pfeffer.

Jonathon Anderson:

Jeffrey Pfeffer. Putting forward the perspective that the layoffs are mostly a social aberration and that someone started it, and tech companies are just following suit now. I've mentioned at the beginning that I don't really have experience firsthand of being in that situation where I have had a member of the team not leave, or they have had to be fired or laid off or anything like that. As John mentioned, probably because most of my career has been in academia. I'm curious to hear other people's perspectives on that. That's what I want to be true. I read that article and want to believe Alan, as you have mentioned, that there's really no problem here. It's just some companies freaking out. What do other people think when they read that article? Does it ring true? Do we have reason to believe, aside from the word of and the research of one economics professor that that is the case, or is there more there, there?

Zane Hamilton:

I will start because I have been through this multiple times, about seven years worth every year, at least once at a previous life. It is a very difficult thing to go through. Reading through this, it very much is true. But what are you saying? The fact that you are appearing to save money by eliminating them, you're still paying a severance, so you're paying them to go away. Other things increase because of it, it leaves you with a hole. In some of the places that we would eliminate, a role that may not have had something to do that week, that month, and then two months later, you would need that person back, so you would've to go backfill them at a much higher rate. That part was very true. Then the impact that it had on the existing team, especially when it's something that, you know, comes every quarter, there's a list that gets passed around and there's a potential of, Hey, maybe next month, maybe next month.

It very much not only impacts the teams, and at the time, I had about a team of 250 people. You start impacting a large team, and it can be detrimental to productivity, but it also becomes very difficult on the leaders because you fret over having to do this. Because a lot of times, I worked with most of them side by side before I moved into leadership of that team. At that point, you're talking about having to lay off your friends and it's hard. It's a hard conversation to have. It's a hard thing to have to deal with. It's hard to sleep at night. It's not something fun. It absolutely starts to impact not only the health of your team, but the health of leadership, the decisions you make, you make for a different reason. I think it can lead to some really interesting side effects. I very much agree with what's going on here.

I also think, going back to John's earlier point, that in some ways it was so hard to help people realize maybe they were in the wrong role, and they would stick around for far too long. It also became a vehicle to do that, to help them move along, which also wasn't an easy thing, but it became necessary. It is what I was bringing up. It allowed people to know it was coming and behave in such a way that they took advantage of it. There's a lot in the article. It's a fantastic article, but it is such a complex thing, and I do think people jump to this as a solution to problems instead of trying to put people in the right role, they just start talking about laying off. It's just a, Hey, that's what everybody else does. I agree.

John Hanks:

I've only been through one round of layoffs in my entire career. At that time, I just happened to be in a management role and got to see it from the manager's side. It was all budget driven. There was a huge budget cut, and people literally had to go, there was no way around it. The thing that struck me most about the process of deciding who would be let go was how much more social it was than technical. We went through an exercise where they basically threw a picture up of every employee in the organization and around the table discussed whether they should stay, go, whatever. There were people who maybe socially weren't really out there making friends and being part of the right group or whatever, who tended to be pushed toward the let go side, who in my view were very critical technical people who should have been kept.

Then plenty on the other side where everybody likes this person. They got favorable ratings, but in my view, they did nothing actually productive to help. After that, I have always had a very negative view of any layoff process, because I think if you have to go through that if you're forced to go through that regardless, whatever your motivation is. If you're forced to go through that, it's a point at which you can do the right thing and let some of these people out of jobs that they shouldn't be in and retain really good people and maybe reshuffle or reorganize to get people doing the right work. I suspect based on my single data point that I'm extrapolating from that, that's not what happens in these situations. I think people clean house of the people they don't like, the people in power, the people who don't say yes often enough or whatever.

Zane Hamilton:

I think that's probably a big part of it. John. I think you're right. I know that most of ours started off as a numbers game, where it would come in as, here's the list, feel free to change it as you see fit. Most of the time, it was a reduction in some other area, not ours, but it was, it impacted everybody. There was a, Hey, we're reducing this business over here, therefore we have to eliminate this percentage of people. Here's the number you have to meet. I need three names by the end of the month. It was very challenging and a difficult thing to go through.

Alan Sill:

There was an NPR story this morning about layoffs at Goldman Sachs. It put me in mind of a post a couple of years ago that has been widely duplicated and copied. I'll put a link if you want. This young woman observed that whenever she hears the phrase, the economy, she just mentally substitutes rich people's yacht money. Is Covid going to be bad for the economy? Is Covid going to be bad for rich people's yacht money? That's just stuck in my head ever since then. I can't get it out. I hear these stories about the economy. Goldman Sachs might be laying off. I do not see it in the highly technical fields that I work in. Maybe, there are some big companies shedding workforce, but I think it has to do with their management approaches, rich people's yacht money.

Twitter layoffs, those are not indicative of the tech economy. By the way, social media is not tech. Let's just be clear there. They may buy some resources, but they don't do what we do. In the technical fields, bioinformatics, computational chemistry industry, academia, we can't find people fast enough. Every person I know, by the way, every major academic outfit is in the same position as me. Yeah, we pay half what industry pays. Maybe, we aren't seeing the fluctuations that they do, but at the level the tide reaches us, it's undetectable.

John Hanks:

I want to throw in a plug. I guess specifically for Alan, maybe anybody that listens to this, who has never worked in academia and is thinking about it, I've worked in academia and the real world, and I can tell you they both have pros and cons, but for peace of mind and enjoyment of your work, you cannot beat academia. There is, hands down, working in an academic institution is far more soul satisfying, at least for me, than the time I spent out in the real world.

Zane Hamilton:

So we do have a question, and I'll go back to that, John in just a second question for me. Go, we appreciate your question. It is, where do you get trained to work in this field? So from an HPC perspective, where should people go? If you're already passed going into school or coming outta school like Forest did, where would you start?

How To Prepare For Employment Opportunities In HPC [42:09]

John Hanks:

I have a quick one on that one. This is not necessarily an answer to that question, but something that I wish our industry had is apprenticeships. I don't think you train for this job. I think you learn this job by doing this job, and the best way to learn by doing is to jump in. Wouldn't it be nice if a lot of people, at this stage in my career, I should be working with people, not to further my own daily tasks or whatever to say, I did this, but I should be helping other people succeed as they enter this career. There's no, that I know of, there's no structure in our industry to do that.

Jonathon Anderson:

I'm in a hundred percent agreement with that, and I, I have said the same. Really the only way I've seen people get into this field, specifically if we're talking about academic high performance research, computing data, that kind of thing. It was someone who had a skill set that made us go, oh, that person doesn't know that they know this, but because all the terms are wrong, all the vocabulary is wrong. Or maybe there's just a small change in the way that they've tended to construct it. If we just take them and put them in this environment, they will thrive here. Then there's a whole bunch of just making them be a part of that team, making them be a part of that community and a learning curve to getting someone in the door. I can't imagine breaking into this field if I hadn't done it right out of school.

Alan Sill:

Okay, so that's a good thing. Let's pick up on that. And, the thrust of the question, is where is the door? Where is the opening? We've talked a lot about the high powered, high intensity research computing that is represented here.The folks your company supports, that John and I, well, in our different roles work to support. One of the things that I've been doing that we've talked about on other calls is working through our industry university cooperative research center trying to improve the state of automation so the data centers can be run at renewable energy power sources. We've talked about this before, so I won't go into it in a lot of detail, but a lot of my work goes into designing data center automation so that you can put a data center out in a cotton field next to a wind farm.

Trust me, it is not easy when we can go in and out of the machine room a dozen times a day. It's definitely not easy when you have to drive two hours to get to the damn thing. There's a lot of research left to do there, and we're working on that. In the process, I've come to realize that there's this whole other industry sort of underlying ours parallel to it in some ways that is desperate for people. That's the data center industry. Those folks can, they have giant meetings of top level staff trying to figure out how to hire people. I've been working with an organization called iMasons infrastructure masons imasons.org, and that's all they do is sit around and come up with scholarship programs and training programs and try to create entry paths into the data center industry.

Data center industry is huge and has a huge need for people. What I see is the possibility that some of those jobs are out here in the middle of nowhere where I live on an island surrounded by land, as I often say and at wind farms and I want to see data centers there. If you're looking for an associate's degree type career or bachelor's degree type career, you're sort of stuck in the oil field or lately the last dozen years or so, wind farms, you can get a job servicing and building wind farms. I think data centers are right there as a career path. It is that entry level door where you can get started doing. These are serious system administration jobs of hundreds, thousands of clusters where it's up to you how fast do you climb up that ladder? If you want to end up where we are now, there are pathways.

Zane Hamilton:

I agree, Alan, I actually drove through the panhandle Texas last weekend and there are wind farms everywhere. Every time I drive through, it's amazing how much more it has grown and expanded.

Alan Sill:

They're growing data centers. I used to tell people, you should ask yourself, where's the data center? You don't have to ask yourself. Now they're building them. I think there are jobs there, and I think there are careers. Yes, again, it's an ecosystem. How do you get started? There are entry pathways, but I guess the point I'm trying to make is there are people desperate to hire folks. If anybody's curious, just send me an email and I'll be happy to connect you with those folks.

Zane Hamilton:

That's great. THanks you Alan. Forrest, is there something you wanted to say. And John, and I'll come back to you.

Forrest Burt:

I just wanted to riff off of a point really quickly that Alan made previously about the tech industry in general and what we were saying about contagion and things like that. It's important to note that this entire thing began with the Twitter debacle, which is an entirely separate debacle that isn't related to the overall economic condition. It's a unique event that happened, and then there was a cleaning of house, et cetera. I think that there's definitely the component of social contagion here where these other companies are seeing the opportunity to, in the face of maybe a pullback in the overall state of the economy, to try to clean house a little bit.

Then, we seize our things coming in where it becomes popularity contests and things like that. I just wanted to say that, you just like Alan says, social media is not the tech industry. We were just, as we've discussed in this webinar before, a lot of us were just at SuperComputing in Dallas. 11,000 people all gathered for the largest tech industry in supercomputing internationally even. The level of innovation and technology and just new things coming out that we saw, I mean, people developing entirely new chips to do high frequency stock trading, people developing chips to do AI, people developing immersion cooled systems to put more AI chips and stuff like that, and the things to do more AI work. The state of the tech industry could not be in a bigger renaissance at the moment, at least relative to what we're doing in high performance computing.

All I see around me is that HPC is a very niche field, which always needs more people into it, just as we've discussed. When I was at Boise State for a while, there was a period where it was just myself and our chief cis admin managing this entire infrastructure. We supported not only our university, but national labs and stuff like that in our region. While social media and these big tech, these fang companies that a lot of people try to pour into especially right out of school, while those might be having trouble because they're running at such incredible scale and they're going to take advantage of these types of things, the actual operational boots on the ground, the smaller tech companies that are really driving innovation and are really doing incredible work in this field are hiring. Like I said, it from what I can see, could not be a greater renaissance of just incredible work going on and incredible need for competent people and passionate people to do that work.

Zane Hamilton:

That's very true. THanks you Forrest. John, you had something you wanted to add?

John Hanks:

Two points about ways to get into this. One thing I often look for on a resume when somebody puts a resume in front of me is, has this person contributed to a relevant open source project? Anybody can go out and jump into a project they're interested in and start making contributions, and that looks really good on your resume when you're passing it around in front of somebody. The other one is for anyone still in school at a university. Somewhere on your campus, there is a researcher who needs someone who knows all about computers, find that person, help them run their jobs on whatever system they're running them on. When you get access to that system and start running jobs, be a nuisance to the people who run that system. Be the person who talks to them all the time. Be the person who does stuff that they view as interesting on that system, and eventually they may offer you a job, and if not, they may point you to somebody who will offer you a job. I ran HPC clusters for biological research for close to 10 years before I got an official job as an HPC man. I did this for free for nearly a decade before I finally landed my first official HPC title. It's not fun to work for free, but if this is what you want to do, that's one way to get in the door, start doing it, even if you're doing it for yourself. It's easy to set up a home lab.

Will Innovations Like chatGPT Lead To Layoffs? [51:43]

Zane Hamilton:

It's great advice. Appreciate that, John. We do have a question from Tron, a point he would like us to talk about. Perhaps are employers embracing copycat layoffs because of things like chatGPT that can replace laid off folks?

Jonathon Anderson:

I don't think we are there quite yet. If so, it's possible that's in some higher ups mind, but if so, I could think of no more likely scenario of, and now they'll have to hire them back very soon. At the very least, it's jumping the gun. I find it more likely that that is just not going to materialize in that way. These are clearly useful tools, but I expect them to be tools that empower more people to be working, not fewer.

Alan Sill:

I usually refer to them as plagiarism as a service. It's not like we don't start by just going to stack overflow and cut and pasting. There's a role for that. I sort of consider coding with these tools as a fireable offense at the current time. On the other hand I think that the problems have entirely to do with the undefined nature of what people have been asking these tools to do. Write me a poem of course it's going to steal style and maybe substance from something that's been programmed to do so. If you get definite with them for example, APIs, APIs are completely defined if you've done them right. There's a tool called hgpte.io actually you can go to write that hgpte.ai. They've already realized that when you have a completely closed ecosystem, you can in fact phrase natural language queries to APIs and have the thing do something for you. So yes, there'll be a future for these things, but I don't think it's what people think it is.

John Hanks:

I think from a very personal perspective on these I figure a significant part of my day every day is spent trying to sort through Google results to find the answer I need. To the extent these tools can do in three seconds, what will take me three hours of searching to do, they will make me more efficient. As a result, maybe you don't need as many of me.

Alan Sill:

The problem is that they will just make stuff up rather than find it for you.

John Hanks:

I'm thinking from the perspective of pacing in an error from an infiniBand log and getting the exact thing I need to do to fix that error where it takes me a long time to do.

Alan Sill:

The current state of the art is that these things just hallucinate answers and make references that don't exist.

Forrest Burt:

I can discern fact from fiction with regard.

Alan Sill:

That's right. I usually like to point out that we've created a way that computers can now not do math.

John Hanks:

I refer to it as Dunning-Kruger as a service.

Forrest Burt:

I think generally we may be impressed by where the state of the technology goes, because I certainly never thought we would see some of the things that we saw last year come out in AI to the level that they're at. I think in general, we may be impressed by how fast the technology progresses from here, but it will be, well, I'm certain that across exec's minds automation money, I can now automate my software engineers. Things like that are starting to come to people's minds. It will be a little while yet and always look out for your fellow humans out there.

Alan Sill:

The thrust of the question is we have an effect on the job market, and I think probably not for a while, but eventually it will be why did we want humans doing those jobs anyway?

John Hanks:

If I was a lawyer, I'd be worried about this. Yeah. I mean, you're preparing the same legal document over and over again. Just change the name of the plaintiff and the defendant. Lawyers lie anyway, so it doesn't have to be that accurate. These things will be perfect for that.

Alan Sill:

Lots of comments in the comments agreeing with you.

John Hanks:

I love it.

Academia Versus Industry For Experience [56:11]

Forrest Burt:

I just have one more thing that I wanted to riff off what John had to say about hiring out of a university like I touched on at the start. That's my story. I went into that whole position that I started on at Boise State with just basic Linux knowledge. I'd run some VMs at different points. I knew generally about file systems and stuff, but I was very intimidated by their posting, which wanted you to know C and Fortran and all this type of stuff. All these HPC paradigms. I've never even heard of most of this stuff. I went in and I knew, at the time, the GPUs were in short supply, so I was talking to them about issues getting GPUs in for their clusters and stuff in my interview.

I was just in general incredibly passionate about what I was doing there and already had just that little bit of technical background that allowed me to talk to them. If you're out there in college, or whatever and you're looking to get involved with the research computing world, just like what John said it was the same thing for me. I didn't realize for a long time that we actually had a cluster there until I got involved with the research project that was oh, let's get you an account on the cluster. I was so interested in it that just a week later I just straight up emailed the research computing department and said, Hey, do you have any jobs available? Absolutely, just like John says, pester them, be the one doing interesting work, help researchers. Always have an eye for making sure you're getting paid for what you're doing, especially if you're a broke college student. Definitely don't hesitate to reach out to those people. The pipeline of training up students at university research computing departments and that being how a lot of people get into HPC is very real. If you have the opportunity to pursue that pipeline and the interest, it's good to take advantage of that while you happen to be in school and have it easily available there. Because academic centers are one of the easiest and most accessible ways if you're already there to get into HPC immediately.

Zane Hamilton:

Well, I'm assuming, back to what John said earlier, that it's great to be in academia. I'm assuming you get to see probably more in academia than you would if you got stuck somewhere. That's very specific in industry where you're doing one focused thing, you get to see all kinds of stuff.

Forrest Burt:

That was what happened. I got there. For me it was more because we were perpetually understaffed research computing departments. It became more understaffed after someone left shortly after I got there. I had a lot of opportunities for it to just be me and the chiefs sat and working on things. It was more because of that type of normal lack of people. There's all kinds of incredible work

Alan Sill:

Forrest, you said that the job requirements were intimidating, keep in mind that many times those are forced on us by the institution. There's the old rule it definitely applies for historically underrepresented people, women and everything. You definitely apply. You definitely don't get it if you don't apply. If you apply you may find that some of those things were there for institutional reasons.

John Hanks:

The thing that often is overlooked in academia, that I view as a bonus, but most people view as a negative. When I was still running clusters for free, my day job was with the IT department at Utah State. At one point, they decided that we were going to migrate from a mainframe to a regular unit system for the ERP system. There was a meeting and the whole group is in the meeting and they said, we need somebody to be the DBA systems man for this new ERP system. Nobody raised their hand. I naively raised my hand and said, “That sounds like something I'd like to try.” With absolutely no experience, none whatsoever in six months, I was the Oracle DBA system man sole Oracle DBA systems man for a major research one university ERP system for the entire campus. You can't do that in industry. Academia, they'll promote you just that quickly because there's nobody else willing to do it, and there's a shortage of bodies. I recommend to people all the time, start out your career, even if you're going to be an IT person, start it out in an academic environment because they're going to be resource constrained and you're going to be given opportunities that you would never get in another environment.

Alan Sill:

Well, I feel like I should send you guys all checks for recruiting.

Zane Hamilton:

Send your resumes too.

Alan Sill:

The flip side is true for us too, that as I mentioned at the beginning of the call people routinely walk out of here to double the salary elsewhere. That's been my challenge is that yes, that you gain a lot of skills and then people hire you. Then I have a problem. I like it here. I get to do a lot of things that I really want to do. Going back to John's point, you pick what makes you happy and this makes me happy.

John Hanks:

Yeah.

Zane Hamilton:

I am very glad it does, Alan. I enjoy working with you. I enjoy working with all of you guys. It was also good to see you all at SC 22. I keep going back to that, but I'm looking forward to it now. I started getting emails today about SC 23, which seems way premature to me, but looking forward to it. I think we are up on time guys. I really appreciate the conversation. I know this was a little bit different than what we typically talk about, but I'm glad we talked through this. It's always interesting to see everybody's perspective. We really appreciate you guys spending the time with us. Go ahead and like, and subscribe and we will see you next week. Thanks again, panel. Appreciate it.