Network maintenance - an insight

|   Information

Last week during the night from Wednesday to Thursday, aixit GmbH was scheduled for network maintenance at the Frankfurt data center. With the announcement sent out in advance, the customers were informed and so the aixit employees went well prepared into the maintenance night.

In this blog entry we want to take a look at this maintenance, what the goals were, if everything worked and how you actually sleep best before and after such a nightly operation.

 

Frankfurt, Rebstöckerstraße, 15.09.2021. At 23:15, the aixit employees, led by aixit networker Gregor, arrive at the data center.

In the introductory conversation, the goals of the maintenance are discussed once again and an approximate schedule is outlined.

There are two goals for this maintenance: on the one hand, the two SSDs of one of the core routers are to be replaced, since they are at the end of their lifecycle and have communicated this via alert.

Secondly, the latest operating system version is to be installed for all core routers and edge routers. Although the devices can all remain in operation via an inservice upgrade, the company does not want to take any risks. Therefore, potential short outages have been announced and it is preferred to carry out this work at night.

The work on the data center area can then begin punctually at 0:00. For the first time, two of the total of four aixit trainees are also present during this maintenance, who want to help out that night but of course also learn a lot.

The exchange of the SSDs also goes without a hitch. Although the old disks have to be completely removed with rails and the new disks then mounted back into the rails in a separate office, this only costs a few extra minutes per disk.

The hardware swap is completed around 1:30. Now it's time to update the core routers and edge routers installed at the Frankfurt site. Most of this can be done via console in the offices. Here you have to wait a bit longer, especially for the core routers, because the updates took their time. Despite the late hour and the onset of fatigue, concentration must of course not be allowed to wane.

Trainee Nicolai sums up this time appropriately afterwards: "The last two hours were really exhausting. You have to concentrate very hard, because you do notice the fatigue. And trainee colleague Laris adds with a laugh, "We couldn't contribute much more in this step and our main task was to keep everyone happy."

Except for a multiple network card that wasn't working right away, this part of the maintenance went without any major complications, so that by 4:45 a.m. all systems were up to date and there were no unexpected outages, except for a short swing between the core routers. A look at the ticket system to see if there are any unpleasant surprises from the customer side also shows that everything went well.

The conclusion at the final meeting at 5:00 a.m. is correspondingly positive. This is also the last item on the agenda for the employees that night.

"I should have slept a little more in advance, I was very tired at the end. But it was really exciting." Says Laris at the end of the maintenance night shortly before everyone heads for home.

"Then on the train, everyone was already on their way to work with coffee in hand. So I was glad that my bed wasn't far away anymore" Nicolai summarizes his way home.

And with regard to the cooperation with the trainees, maintenance manager Gregor adds: "The two of them did their job really well and held out extremely well. I think for the two of them in particular it was a very exciting but also exhausting night. In any case, everyone was very happy when they went to bed in the morning."

With a few days to spare, everyone involved is already looking forward to the next maintenance, which is sure to be just as trouble-free as the last one.

back
Network maintanance