110

This is kind of a horror story, with a happing ending. It contains a lot of gore images, and some porn. Very long story.

TL;DR Network upgrade

Once upon a time, there were two companies HA and HP, both owned by HC. Many years went by and the two companies worked along side each one another, but sometimes there were trouble, because they weren't sure who was supposed to bill the client for projects HA and HP had worked on together.

At HA there was an IT guy, an imbecile of such. He's very slow at doing his job, doesn't exactly understand what he's doing, nor security principles.

The IT guy at HA also did some IT work for HP from time to time when needed. But he was not in charge of the infrastructure for HP, that was the jobb for one developer who didn't really know what he was doing either.

Whenever a new server was set up at HP, the developer tried many solutions, until he landed on one, but he never removed the other tested solutions, and the config is scattered all around. And no documentation!!
Same goes with network, when something new was added, the old was never removed or reconfigured to something else.

One dark winter, a knight arrived at HP. He had many skills. Networking, server management, development, design and generally a fucking awesome viking.
This genius would often try to cleanse the network and servers, and begged his boss to let him buy new equipment to replace the old, to no prevail.
Whenever he would look in the server room, he would get shivers down his back.
(Image: https://i.bratteng.xyz/Ie9x3YC33C.j...)

One and a half year later, the powerful owners in HA, HP and HC decided it was finally time to merge HA and HP together to HS. The knight thought this was his moment, he should ask CEO if he could be in charge of migrating the network, and do a complete overhault so they could get 1Gb interwebz speeds.

The knight had to come up with a plan and some price estimates, as the IT guy also would do this.
The IT guy proposed his solution, a Sonicwall gateway to 22 000 NOK, and using a 3rd party company to manage it for 3000 NOK/month.
"This is absurd", said the knight to the CEO and CXO, "I can come up with a better solution that is a complete upgrade. And it will be super easy to manage."
The CEO and CXO gave the knight a thumbs up. The race was on. We're moving in 2 months, I got to have the equipment by then, so I need a plan by the end of the week.
He roamed the wide internet, looked at many solutions, and ended up with going for Ubiquiti's Unifi series. Cheap, reliable and pretty nice to look at.

The CXO had mentioned the WiFi at HA was pretty bad, as there was WLAN for each meeting room, and one for the desks, so the phone would constantly jump between networks.
So the knight ended up with this solution:
2x Unifi Securtiy Gateway Pro 4
2x Unifi 48port
1x Unifi 10G 16port
5x Unifi AP-AC-Lite
12x pairs of 10G unifi fibre modules
All with a price tag around the one Sonicwall for 22 000 NOK, not including patch cables, POE injectors and fibre cables.

The knight presented this to the CXO, whom is not very fond of the IT guy, and the CXO thought this was a great solution.
But the IT guy had to have a say at this too, so he was sent the solution and had 2 weeks to dispute the soltion.
Time went by, CXO started to get tired of the waiting, so he called in a meeting with the knight and the IT guy, this was the IT guys chance to dispute the solution.
All he had to say was he was familiar with the Sonicwall solution, and having a 3rd party company managing it is great.
He was given another 2 weeks to dispute the solution, yet nothing happened.

The CXO gave the thumbs up, and the knight orders the equipment.
At this time, the knight asks the IT guy for access to the server room at HA, and a key (which would take 2 months to get sorted, because IT guys is a slow imbecile)

The horrors, Oh the horrors, the knight had never seen anything like this before.
(Image: https://i.bratteng.xyz/HfptwEh9qT.j...)
(Image: https://i.bratteng.xyz/HfptwEh9qT.j...)
(Image: https://i.bratteng.xyz/hmOE2ZuQuE.j...)
(Image: https://i.bratteng.xyz/4Flmkx6slQ.j...)

What are all these for, why is there a fan ductaped to on of the servers.
WHAT IS THIS!
Why are there cables tied in a knot.
WHY!
These are questions we never will know the answers too.

The knight needs access to the servers, and sonicwall to see how this is configured.
After 1.5 month he gains access to the sonicwall and one of the xserve.
What the knight discovers baffles him.
All ports are open, sonicwall is basically in bridge mode and handing out public IPs to every device connected to it.
No VLANs, everything, just open...

Comments
  • 44
    And the way the xserve storage is setup, is some of the strangest he has ever seen.

    Till this day, the knight has yet to understand how it is all setup. And the knight never got access to the rest of the servers.

    This is the list of servers in there, as explained by the IT guy:

    (Image: https://i.bratteng.xyz/UeC2dkW3f0.j...)

    The knight has spent some time testing the new equipment, and decides to just install it in the rack and let it run there.

    Do some testing of solutions etc., see if the 1G fibre has been installed.

    (Image: https://i.bratteng.xyz/eYxU4Q0YBY.j...)

    During this time, the knights colleague has been working on putting the desktops to the 3D animators in the server room, to reduce noise and heat in their office.

    These machines will be connected to a 10G SAN by fibre. And they have a 25m active HDMI and USB 3 cable connected to them.
  • 42
    During this time, the knights colleague has been working on putting the desktops to the 3D animators in the server room, to reduce noise and heat in their office.

    These machines will be connected to a 10G SAN by fibre. And they have a 25m active HDMI and USB 3 cable connected to them.

    It has turned out pretty good, just some cable managing in the office.

    (Image: https://i.bratteng.xyz/CNILUwCaMo.j...)

    (Image: https://i.bratteng.xyz/QZudrMOnjb.j...)

    (Image: https://i.bratteng.xyz/7zGlIR1fue.j...)

    2 weeks ago:

    1 month goes by, the knights team leader grows tired of waiting. We got all this new equipment but can't deploy it because he the slow imbecile IT guy.

    "Pull the shit down, he (IT GUY) has had such a long time to give you access to this. I'm sick of his excuses.", the team leader told the knight, "If anyone complains on when the new network is up, send them to me, I'll deal with it." (The team leader is a short, angry Portuguese guy, very similar to Kratos in God of War)
  • 40
    Friday:

    The knight shuts down the servers at HP, and moves them to his desk for reconfiguring and set them up on the new network + some testing of firewall solutions.

    And the UPS, which was installed in 2012, needed batteries replaced in 2016. Neglect... Sad.

    (Image: https://i.bratteng.xyz/jGSQYvhFck.j...)

    Yesterday, leading up to today

    The knight has been pulling down the old gateway and some switches, plus rearranged the fibre, ISP router and some other equipment.

    (Image: https://i.bratteng.xyz/Tzor2iEt7L.j...)

    All the old cables from the backside of the rack has been pulled out, switches removed and PDU's moved.

    (Image: https://i.bratteng.xyz/BxDXtAAuWK.j...)

    Switches and patch panels has been moved to the front and ordered in a nice fashion.

    (Image: https://i.bratteng.xyz/ELOlvUr2TN.j...)

    All 72 patch cables á 15cm has been attached

    (Image: https://i.bratteng.xyz/aHEk9TyJuv.j...)

    End of story.
  • 34
    I managed to buy 2x single mode fibre cables, those meant to go between the 10G backbone and the 48port switches, so I'm using ethernet for now.

    Should also buy a shorter fibre cabled (this one is 2m) between the fibre tray and ISPs router.

    Next is to order the fibre cables for the 3D animators, some cable management items. And the insert the 2nd USG between the other USG and the 10G switch.

    The 2nd USG is going to be used as a failover in the unlikely event it would fail.
  • 10
    Amazing rant, 12/10 story telling, nice pictures for context.

    I am not a network or system engineer, but as far as I can see, you did an amazing job! Looking forward to your next rants :)
  • 5
    We Portuguese people are fearless, specially if we're angry. 😁
  • 3
    Such wonderful tale. I'm gonna tell this story to my future children.
  • 3
    For such long storytelling I'd like to put ++ in a loop.
  • 3
    @ChainsawBaby Such terror! I weep for the struggles of the knight!!
  • 2
    We portuguese tend to solve things the pratical, if somewhat blunt, way:
    If someone is clearly in the way for too long, they have to be pushed away.
    And we have little patience for pleasantries when an issue is on the table 😂
Add Comment