Ascender Automation: Mastering Rocky 8 to 9 Migration and AAP to AWX Integration

they touch it broke and how that is so necessary now in our a lot of our jobs like we need those people just get in there and tinker and do stuff so that we can continue to iterate on it and yeah Dave Rush nailed it yeah I would love to take a poll and I know where Dave would stand on this one of who were those kids that actually tore stuff apart because I'm pretty sure it's most of us I sure did too yeah that that's exactly my brother was like that and I thought he was insane right I was like why why do you

have to take everything apart like this is you know like you gotta know yeah exactly he wanted to know how it worked and then is like a little bit challenging to put it back together he was like whatever I'll let somebody else do that just buy a new one you might have some extra parts it's okay they weren't necessary oh that's right you open source you get to go tear it apart yeah yeah somebody actually will put it back together so that's the good thing it's already together yes it's a valued skill appliances TVs toys yeah I the phone I was like dude the come

on why do you have to take the phone apart you know it's like back in like you know the days when you actually had a phone you could pick up yes good time I did that too I think what you describe um really is my entire it career is just things break and then I try it and it's still broken I try it it's still broken eventually it starts working and then I move on to the next thing and it's broken and broken and then eventually it works it's kind of trial trial and error is is is probably a good descriptor of it and if

uh because I know anytime I did anything and it worked on the first try I was worried because it's like that means I missed something something's broken I just haven't figured out what it is yet so let me try and sus that out thought you were breaking something that wasn't quite complicated enough it was too easy it just was too easy well it's like um I used to have a a saying no change window goes on pun something's going to go wrong I want it to be big and obvious like big red light right in front of me so I can fix that and then

I'll feel good and I'll move on otherwise like I just wouldn't be able to sleep the rest of the night yes I missed something definitely something was missed yes a neon sign which is clear you appreciate neon signs because you got it going on there in your background that's fantastic broken cool so are we gonna start with Rocky to9 migration is this let's talk about that why why would you do that why would somebody want to migrate from eight to nine caveat all the caveats and warnings from previous well why I think it's it's pretty easy answer eventually um things stop being supported right so

you you eventually have to to move to a new version uh at some point or another um we already talked I mean didn't we didn't we do on one of these we were talking about um migrating from centos7 to Rocky 9 I think we already talked about that once right we did yeah it's it's it never hurts to I guess rehash that centos7 is about to be completely EOL fairly soon so uh if you haven't migrated off those systems hopefully you've got a plan and you're implementing them um if you don't we did a a quick webinar I've got a blog post and a video

that kind of shows using the uh Elevate tool to do that that although you know it's funny that that's not necessarily our recommendation and um I can show you an 8 to n demo as well uh just going from Rocky 8 to Rocky n right because you also have to move between versions here um and it's using the exact same Elevate tool I would show you the demo but it's almost a Ono one of the other one I push the go button and it migrates your Rocky 8 to a nine system using this tool which is really cool it kind of does the leap it

evaluates what's in there and it'll automatically try and fix any dependencies required in there and then it'll go ahead and do the the full migration that's boring but um normally when we tell people like hey you need to migrate from like one version of an OS to another we usually say it's best to build a new one like a if I want to go from Rocky eight to Rocky 9 it's best to build a rocky n and then migrate my apps I mean that's that's your experience too right Z absolutely that that is that is our preferred way of doing it it's the cleanest I've

I've used but the leap tool and it went great and then six months later there were some really weird things going on and I had to rebuild it anyway so it's just it's just cleaner to start from scratch it just is also a really good opportunity to automate things so I know previous life we didn't necessarily automate the installs of applications or how that was going to look or the testing so if you have those Legacy things when you're going through an upgrade from a major version to Major version it's a great time to go back and do some sort of configuration management software automation

installs U it kind of future appr you when you have to do this again so I know we we have a lot of people moving from Cento 7 straight to Rocky 8 and then they're looking at moving to nine so if you automate it you actually kind of make that a little more simple but what Greg's going to show it's really cool if you try it just remember make sure you have backups make sure you've tested that your backups work we have had some um people try things in production and not necessarily having backups so make sure you back it up make sure you test

this in a lower environment I think I've caveat got it as much as I can test yeah yeah for sure and that's that one is a super snoozefest to demo so what I actually wanted to talk about was something I've been working on that hasn't seen the light of day necessarily yet because I'm not completely complete on it but the real core components that are functional and need to be there are there and it's how would I simulate the manual migration right where I I build a new system and I migrate my apps I have kind of done that in a little demo scenario I'm

not saying this is exactly how you would do it in your environment um but it is a way you could do it and so I have kind of a um discover system here right and so what you do is you run the Discover and it's a simple Playbook that you've got all of your migration hosts and it's going to Loop through each one of those hosts you want to migrate and it's looking for all of the RPMs I could have done this using gather facts and anible which is a system where it will go and just pull a whole bunch of stuff variable IED although

I found if you're running against Cento 7 it doesn't return all that information um but the tried andrue RPM command does absolutely work on everything you could be potentially running it so it will go and it'll find all the apps you have installed and then it's going to run another command and it's going to look for what's actually enabled you know what's actually running so maybe I have Apache installed but it's disabled it's not running right so it just happened to be something they put on there I probably don't need to migrate that over right it's just some dead application that somebody put on this

thing so what it does is it Loops through finds all the applications you have and it attempts to back them up one and all this part Works uh just fine let me hide that right there so I can see everything let me zoom in just a little bit so it's going to go through and I have to one I have to know what kind of applications I'm going to be backing up right so it could be engine X could be aache and when you're backing those things up it's going to be kind of different based on those things well I've done it where I'm showing

you how to do it in Apache so you can take this you can copy and paste it make modifications for your various applications but here I'm saying hey if you have Apache installed on the system and it's actually enabled you know it's it's running well then call this task file right and so my task file is right here it's Apache backup so if you want to do one for engine X you'd probably name it engine X backup or um if you're running uh you know Maria DB or whatever it happens to be right your specific one for that well in here I know uh my

config files are going to be here so you know what create a zip file that's got all my config files in there then create a zip file that's got all of my uh data directories in there of rww so basically back all that stuff off and stick it on a backup server it's like nothing too crazy there and then I have a restore playbook for Apache so now the interstitial the middle piece that I'm still working on is I'm using VMware I'm going to clone a template of a new system it'll have it in there once it's up it's going to do the restore over

there so I am about 95% done because I've done all these things in various Fashions but here's the restore it copies them from the backup server and then paste them in the correct directories make sure Apache is installed right here first right and then uh it's going to extract those archives then it's going to start the service and say start boot so really it's going to take old system scan through what applications do you have oh I've got these while I've created a template for uh htpd right so Apache pull all the configs then push them over um Jimmy I was talking to Jimmy super

smart guy he was like why don't you go straight to the server and I was thinking you know that actually does make efficiency uh sense right I stand up a new server and I just transfer it straight over there that could be a really good use case um for me I like stuff being in a backup location so I have it I can look at it I can test with it I can play with it uh before I actually push it over to my production system but essentially you could stand up the new uh and then um you know once you feel comfortable with it

it's up it's operating you can uh in VMware just turn off the old system and then you know set a reminder a month from now go ahead and kill that old system maybe six months from now I don't know how comfortable you are with that Z but I would I would probably wait a little while before I completely kill my uh old systems but the cool part about this is I just have to make these template files like this for the different applications I might be running right so maybe I'm running a mail server or I could potentially be running uh ntp server or a

DNS server all these things whenever I'm hitting this host system that I'm going to migrate I don't necessarily have to know in advance everything that's already going to be installed on there right that compliment because at runtime it's going to look it's going to evaluate what's already installed what am I actually using and then I've already written my template files to back up those specific things so I was trying to think about how to make it as flexible as possible and I think a lot of people and I'm going to say you probably should know everything that's running on your server but does that translate

to reality of the system that's running for four years do I really probably no it does not no so it's it's probably conducive to your mental health if you have something that's a little bit more flexible that will find oh for some reason you running a bootp server on this thing um because it turns out you were bootstrapping some stuff over here and I'm going to go ahead and back that up and then I'm going to push it to the new version of whatever this server is right so it just um it to me in my opinion it like helps my brain feel more comfortable

for it to go ahead and discover all these little things I've forgotten that I've done and uh help me migrate I'm curious what your thoughts on because I haven't actually even talked to anybody about sort of this buildout and this demo like what do you uh what do you think about though I have lots of questions and ideas not ideas I just have questions to me from what you're showing it gives you a really good idea to understand what you do have on your machine because how many times you install the machine and you just let it run through the the default installer and you

don't pay attention to what's on there or what's running I hope people don't do that in production but I know people that do it's a really bad idea I think this is a great great opportunity to identify that stuff that doesn't need to be running like I hate the fact that Bluetooth used to be on by default on a server that just drove me insane or like printer services and things like that yes yes so no I think this is fantastic and being able to see making sure you're going to get what's there and bring it over but being able to actually identify that that's

that would be cool yeah so this I hope will serve as kind of like a little template by which maybe people could grab this and and sort of move Beyond there now keeping that in mind if you did need more help say you're under the gun say I don't know you've got a whole bunch of systems on Centos and you need them migrated you know before it reaches end of life or you know what you have a bunch of RA servers and you want to migrate them over to Rocky 8 or rocky n we can help you with that um I say this is kind

of a template by which you could do these little things uh that amazing engineer I talked about Jimmy has worked out some tools that he's utilized before that actually wraps a gooey around all this stuff and makes it clicky click simple to do a lot of this stuff gives you great dashboards and visualizations and all that stuff and we sciq would be happy to help you migrate all those systems in a very efficient repeatable fashion and uh I used to do some Consulting work and I always said the mark of a good consultant is they work themselves out of a job and so we want

to leave you with an Ascender server with all of the services installed that will allow you to you know continue to uh configuration is code all of your infrastructure because essentially that's what it breaks it down to being you have all of your configurations in a repository somewhere right as code so I want to rebuild a server it's as simple as clicking a button and it will rebuild all that infrastructure we're not just talking about one server we're talking about a hundred or a thousand servers right you push those into test development you know have that move into prod it's entirely up to you so

to me it's a really good Kickstart or uh help if you aren't like fully automated or you just need some quick assistance getting all this stuff stood up uh I'm sure Zayn would love to take your call I would love to take your call I'd also like to ask you another question so on on the stuff not necessary that you're showing here that we're talking about that we can help you with going through a migration that that would get left behind by ciq as a part of this thing running continually in the future that would actually give you the ability to not have to talk

to your internal audit team or your compliance team anymore if I'm not mistaken right like they could actually go see this dashboard and they would never come and bug you again well I mean that's blessing i i well it's good for them because they don't have to come fight with you it's good for you because you don't have to go do the mundane task of I have to go to Every server and figure out what's installed they get to go do it themselves and get whatever they need all right but then uh a win-win to me you have a junior engineer that installs something while

he's troubleshooting and now you get to explain why that's there so you know I mean it's a blessing in a curse no but it it absolutely helps with anybody who's got compliance requirements it's going to be a godsend for all that stuff thank you yeah absolutely just curious like once this is all set up and you have another major migration to do in the future like can you quantify like how much time and energy that will save someone if they do this correctly now yeah it should be um buttloads I think it's a technical term no it's going to save you a lot of time

but also um theoretically it provides you a path to build your infrastructure differently instead of one at a time building build up a server installing all the things now you've got everything truly automated so it becomes click button deployment for this new infrastructure so it can actually be I'm not saying you have to right nothing is a is as a have to here but it gives you the opportunity to shift the way you operate instead of manually doing all this stuff now literally everything is here as well as methods to do backups so it'll pull all your stuff to configuration is code if you want

to I was always a belt in suspenders kind of person so I would um maintain here's the current state of configurations and then you know here's my backup systems that are actually backing up what is truly there on the thing you know it's the difference when you build a house it's got your initial plans and then you've got the as builts like this was what we had coming in this is what we thought was going to happen this is what we really ended up doing uh at the end of the day so I was always um definitely built in suspenders and so this allows you

to kind of do that on an automated fashion or you can schedule it in the system where it will just automatically do these backups for you right these configurations code backups um makes it really simple but as to your point Rose yeah um uh Rocky tin comes out and you want to start migrating hosts over that you get to use this exact same set of automations to do those migrations if you want to not just deployments right like after we do the migration we use them for deployment but then when the next set of migrations come around um those sorts of things are they aren't

available as well is that there is there a limit to the number of applications that you can automate like it seems to me like if you had 10,000 that would maybe make you crazy like is there is is there like people usually do like 10 do they do all of them is it the more the marrier you know even in really large organizations we've um talked to and Zane you've done a lot more Consulting than I have so you're probably better equipped to to answer this question but it usually boils down to you're talking to somebody that's got got like 3,000 virtual and physical servers

they've probably got between 15 and 100 applications total running on all that stuff so you might have you know 200 web servers but they're all pretty much ran in the exact same complement of applications uh in there at least that's my experience what what have you seen out there Zan no absolutely it is as much as people want to call them snowflakes and that goes back to everyone's built differently they're typically doing the same thing so there aren't a lot of variations in the actual base of the server there just might be some different packages that actually run the app so 90% of it on

that 3ou 3,000 servers is probably exactly the same so to Greg's Point there's a lot of overlap there's a lot so you can do massive amounts of servers because typically in the bigger environments there are a lot of it's just it's scale they need more servers to scale than they have more applications now some people have a lot of applications not saying they don't for sure well why would talking about migrations I I guess it was in November late November I completed um another set of playbooks for migration like I've had migration on the brain for for a minute now this one actually took me

about a month to kind of put all the pieces together um but it's the ability to migrate between various awx derivatives right so awx which is the Upstream that Ascender built from right kind of our automation platform here um that includes Ascender right so you have awx Ascender another Downstream is Red Hats AAP anle automation platform and I've written some playbooks using some Community Source uh collections of various things to be able to migrate everything between those systems which I think is really fun so I thought I would do a quick live demo of I've got an Ascender server here you can see my number

of hosts are zeros inventories are low right if I look at my job templates I just have the six that come baked in by default and what is this right here but it is an AAP install in my lab environment how fun is that so I can take my playbooks that I've created migration well without the Slash and these two right here I've got one that exports data and one that Imports data so if I well actually I'm going to go ahead and run this import data so what it's going to do is it's going to take all the data from this red hat AAP

server and it's going to stick it into that Ascender server I'm going to go ahead and run it now because the import process takes about seven and a half eight minutes and mine aren't like super loaded up right the more data it has to transfer the longer it's going to take but I wanted to show you how simple this process really is I have a couple of playbooks you interact with here I've got an export one click on the export one and really all all you do in here is you designate what do I want to export and so all these things are available for

us to export now as Zayn said there are always caveats with everything in existence so we don't get to export absolutely everything so like system settings like your logging settings those don't get exported um rolls aren't exported natively now all that's available in the API all the stuff is available in the API it's just in the tools that exist now you know the community Supply tools those capabilities aren't there but this is version 1.0 and who's to say 1.1 isn't going to start adding some of those things in there so I'm definitely curious what the community would like to see you know especially if you

want to contribute Cod I actually had a couple of people already um contribute to the repository which I think is really cool uh so people are excited and interested about this stuff but essentially what you do is in the export Playbook here I just designate what thing do I want to actually export out and it's going to connect into the uh system and it's going to pull all those out and save them as standard gaml files if you look at the Playbook that's it it's really super duper short so it just kind of Loops through all these things you're looking to export pulls them out

as files once you have that let me go back just a tick I uh put all of my configs in my config folder here and I'll show you I don't know what do you want to see my job templates it's just standard gaml boring formatted files so here's all of my job templates that it spits out in there uh next I run the import Playbook right here it's the same kind of deal anything that I want to import I just have right here anything I don't want to import I comment out and so this is where a lot of the interesting Parts came in for

me and a lot of the time consuming so a lot of the information that's exported to repport back in with the tools you have to form at it in very specific ways you have to pull certain information out and so good for you I've gone through all the trouble of figuring all those things out um I also noticed whenever you um say import your inventories and so inventories big giant list of all the hosts I could potentially operate against uh by default it doesn't pull any of the hosts or groups in because they kind of assume you're going to be dynamically pulling that right so

if you are say using a configuration management data base like uh service now or you're pulling straight from AWS or your VMware environment you're pulling all those hosts in dynamically no problem right it doesn't matter because all that stuff gets populated automatically um but for me I've got a ton of stuff at static or in my lab environment so I actually wanted to import all those so I wrote a portion that uh consumes those files figures it all out and allows you to import it so if you have these set to true on importing of inventories it'll also bring all those hosts in too so

you can get all that stuff in there um after that it's really just some nice to have so you can can filter out what users you don't want to bring in you definitely don't want it to bring in admin if you're using that to import because guess what it overrides the password and then everything fails after for that but then it's just some cleanup stuff if you look at this this is all of me just editing data and fixing it was gonna say this sounds very Ginger toy so it will go in this is still yeah okay now it finally gets to the part where

it's actually pushing things in I use a block and rescue system in here I've noticed if you're importing a whole big giant chunk of data sometimes because my servers in my VMware environment it's a shared environment so I keep my resources really low sometimes the API will time out and so this gives it a chance to oh if it times out it'll just retry really quick and then Bob's your uncle everything generally completes fairly clean so after that you don't have to touch anything and it should pull everything in see I tried to talk kind of slow let's see if we made it uh 7

and a half minutes how far are we in we're at four and a half so it's only got about half of that stuff imported in here but let's do a quick refresh just to see what it's gotten through so it's already pulled in a bunch of the hosts so I've pulled in my inventories I've got my projects in here right all this stuff was sitting at zero I'm not sure if it's hit the template section yeah yep it's starting to pull in templates now we started at 6 it's at 14 it should end in about 36 so uh as you can see it allows you

to take any system you're using out there right pull all that data out you can just keep it as a backup if you wanted to say for example uh but you can also use it to easily push over to other systems so something I'm seeing or it's not seeing I'm envisioning for kind of my next demo using this is you have a main server in your data center so this is kind of a weakness of the awx infrastructure it's not really good at migrating information kind of all over the globe sort of thing so I've got my main server here uh I live in Texas

so it'll be in my my Dallas Data Center and that's where I make most of my changes I put all my infrastructure in there what I can have it do on a schedule or on a reaction is I can have it uh export all this stuff into a g repository and then I can have it push that out to my uh Sydney Australia Data Center and so it will push all of my new configurations in there and it'll push to my London based one and then my one over in Germany right so you can actually use it to push a different environments now it could

be uh a push to you know Dev environment or production environment so it doesn't actually have to be like Geographic separate it doesn't have to be that big and cool you beat me to it it can't be something small and simple as well right like if you're just moving it from one testing environment to another or what have you but it gives you the ability to kind of migrate your infrastructure Whole Hog so if you are using a system and you would love to move over to uh Ascender so you can get our CQ support and work with people with amazing mustaches Andor beards uh

you've got a great mustache T Rose uh then uh it gives you the ability to really succinctly uh do that so I'm curious if you guys have any questions on that yeah you kind of answered my question but the the question I had was in the past when I've seen awx derivatives installed typically the way to do it or most people do it is they have one install for everything Dev QA staging production all of them but I've always been asked I want to have one for each I don't want them all the share I don't want my production stuff talking to Dev test but

it's always been really difficult most of the time to actually move something through a a process of going from Dev to staging QA production this helps me there right yeah absolutely okay and again these playbooks are a jumping off point right now it's set to pull everything again you can like pick and choose in between there and even then you can modify it so that you can squeeze down like um job templates you could say just pull in the job templates that have this in the name or are associated with this inventory or or whatever you could get more granular in there if you want

so you could be very specific if you want pick and choose whatever pieces you like that's very cool because that was always the complaint that I heard was from a development standpoint people would build a complicated workflow and they would test it out on their laptop and then they would try to deploy it somewhere else and they like I have to go recreate it every time it's painful so that's fantastic very cool it's a big pain I know like upgrading between versions of a sinder is super easy now um but if you did want to I'm not saying You' ever do this you have an

install and then you know you don't think about upgrading it for a few years you know this might be a nice safe way of testing upgrading to a completely new system none of us would ever do that obviously we would never do that uh but if you were in that kind of a pickle it's a really nice Safeway because all you're doing is just pulling data out right you're extracting it after that you can do whatever you want with it it's fantastic it's really cool to see yeah what else do we have going um this is something multiple no continue uh something multiple customers have

queried me about is how do we create a uh local Rocky repository right so instead of um having to update all my machines all the time like how can I actually have kind of a a local repo inside in the I think it's the rocky Foundation actually put out a nice little article uh about how here's the steps you can follow although they assume you already have a bunch of stuff in there like they assume you already have a web server installed and and various other pieces and I thought what would it be like if I have just a plain system not doing anything uh

follow those steps and so I created one right here and it's create a local repository so I thought there are probably a couple of reasons so also at the end of this I can either synchronize all that stuff immediately or run on a Cron job I'm tell it to go now I'm going to launch it so this is actually reaching out to just a clean Rocky n system and it's going to do the install on it um so a couple of reasons are uh the one that was a big one for me is bandwidth constraints say I've only got 500 Megs or a gig connection

coming in and I've got I don't know 2,000 servers inside and I need to do updates all 10,000 of those things are going to be hammering my internet connection trying to pull that data in right so if I have a local repository I'll just pull straight out of the network right no big deal only that local repository has to synchronize to the Internet so it's definitely going to save me um uh bandwidth resources there I've also talked to people before where their data centers are like connected through some weird I mean I think uh I think Zan you guys had somebody that was going over

like a T1 or something like that so it is very very much a non-starter if you're trying to push through like 1.54 megabits per second connection at best I guess you could have that local repository that would synchronize overnight and everybody could could just pull from that but also um security concerns that's another big one a lot of people don't let their machines have like direct internet access right it makes a lot of sense uh whether you're pulling in maybe bad packages or data exfiltration whatever it happens to be they just don't want those things talked in the internet so you really have to have

a local reposit anyway uh to satisfy uh those uh security requirements compliance requirements whatever it happens to be you know a local repository can actually be super useful so this just connects into uh the lock the the local Rocky server I'm sorry so you could have a be install it will connect to this server if you don't have a web server guess what it goes ahead and installs it uh configures it it actually connects to your firewall on there you can see it setting up the HTP htps ports it's uh configuring SE Linux as well which all of us know SE Linux is your friend

which is why it says it failed because I have that turned off but it would have done it had I had SD Linux actually running on this system say do as I say not as I do um then it uh installs the uh installs rsync on the system it basically puts absolutely everything you need in there and then it tells it hey go ahead and run and now that it's completed it says go ahead and synchronize now don't wait for the Cron job to hit let me go back over to that web server do a quick reload we're doing it live there you are now

you can see the let me zoom in a little bit it's not that cool uh but you could see all of my repositories starting to synchronize all the various files are in here and available for my local systems to start grabbing so this is good if you have one server you want to have a cluster of servers guess what you just point it towards your cluster of servers and it configures all this stuff stuff for you you don't have to do anything else literally I made zero modifications of the Playbook I just pushed go in it installs everything for you it's fantastic and customer you

want something like this we have Mountain we can actually deliver you repositories custom repositories secured repositories long-term supported repositories all of our other products so yeah the cool thing about Mountain too is it gives you access to um custom kernels if you want us to do any of that stuff if you want long-term support say like 8.6 8.8 any of that stuff all that's going to be available through mountain right very cool it's great Craig thank you than you mentioned um some customers reaching out to you and adding to the Ascender what where did they do that uh they did it right through uh GitHub

so all my stuff is in public G repositories in GitHub so anybody can uh do poll requests on that so they can make modifications submit them to me and I'm not the fastest person about reviewing all those changes but I do get to them pretty much all of them and um happy to have that stuff integrated I think in the migration one somebody added uh in the centos7 migration one somebody added like preps and post steps in there so they did a really good job of hey you know if you've got some pre-files I'll go check to see if they're there if they are I'll

go ahead and Implement them for that server so they did some really cool stuff in there very nice thank you to those out there that are helping contribute and give back that's fantastic it's good to have friends which shows me there's a need for it people are people actually using it and uh absolutely eight to nine stuff's good and I'm curious to see if anybody for kind of the the manual method what people will do I'm sure somebody has uh a way better more efficient idea of doing that and I would be curious to hear it this is just what my brain could comprehend I

was a network guy right so it's like you know my uh my intelligence only goes so far I'd love to see it too I think we have a lot of people going through this right now and if they're not going through it they're thinking about having to go through it I think there's a lot of people getting very anxious because the clock is ticking and they running out of time really fast yeah well not just the 8 to n but the seven to something seven to that's the one I was referring to thank you Rose s to something yeah so officially you have 178 days

11 hours 21 minutes and 25 24 23 2 not much time no we have a countdown going on on the website so I was just looking at that yeah yeah if you're looking for help on that please reach out and let us know we'd love to help um again we're getting this a lot we we are here to make make sure we can make this as easy as possible obviously the complexity of your environment is going to make it more complex and more painful but got to do it you don't have much time yes and just just do it the right way the first time

because then otherwise you're gonna have to call us anyway and we are also happy to help all of those people who reach out and say oopsie true the more people we help migrate the faster we get because we just build up our you know our dictionary of these applications have all these things so we just get faster and more efficient every time we do it absolutely yeah awesome well great work Greg and Jimmy and Michael and Justin and everybody else in the world who has been adding to all of the playbooks and everything to make cender what it is today so it is perfect timing

I mean really this is like some really amazing timing because it is is needed right now again 178 days 11 hours 20 minutes and now five seconds let's go you guys can feel free to reach out to um any of us uh it's easy to get a hold of us at ci.com make sure that you like and share and subscribe this is valuable information so again thank you Greg for coming on and show showing us in real time that's always fun to do things live so again thanks for being here happy New Year we will definitely see you next week thank you bye live next

week in person live you and me oh that's right yeah meet everybody together ah coming live from Reno that's right yeah cool are we doing a webinar next week oh yeah rock and roll man just have the whole the whole game there that's right we will still have live hecklers it'll be cool that be awesome looking forward to it oh amazing all right thanks guys like And subscribe and share and we'll see you next week cheers thank you everybody bye everybody [Music]