My first job: The Mystery of The Powered-Down Server

I paid my way through college by working every-other-semester in the Cooperative-Education Program my school provided. My first job was with a small company (now defunct) which made some of the very first optical-storage robotic storage systems. I honestly forgot what I was "officially" hired for at first, but I quickly moved up into the kernel device-driver team and was quite happy there.

It was primarily a Solaris shop, with a smattering of IBM AIX RS/6000. It was one of these ill-fated RS/6000 machines which (by no fault of its own) plays a major role in this story.

One day, I came to work to find my team-leader in quite a tizzy -- cursing and ranting about our VAR selling us bad equipment; about how IBM just doesn't make good hardware like they did in the good old days; about how back when _he_ was in charge of buying equipment this wouldn't happen, and on and on and on.

Our primary AIX dev server was powered off when he arrived. He booted it up, checked logs and was running self-diagnostics, but absolutely nothing so far indicated why the machine had shut down. We blew a couple of hours trying to figure out what happened, to no avail. Eventually, with other deadlines looming, we just chalked it up be something we'll look into more later.

Several days went by, with the usual day-to-day comings and goings; no surprises.

Then, next week, it happened again.

My team-leader was LIVID. The same server was hard-down again when he came in; no explanation. He opened a ticket with IBM and put in a call to our VAR rep, demanding answers -- how could they sell us bad equipment -- why isn't there any indication of what's failing -- someone must come out here and fix this NOW, and on and on and on.

(As a quick aside, in case it's not clearly coming through between-the-lines, our team leader was always a little bit "over to top" for me. He was the kind of person who "got things done," and as long as you stayed on his good side, you could just watch the fireworks most days - but it became pretty exhausting sometimes).

Back our story -
An IBM CE comes out and does a full on-site hardware diagnostic -- tears the whole server down, runs through everything one part a time. Absolutely. Nothing. Wrong.

I recall, at some point of all this, making the comment "It's almost like someone just pulls the plug on it -- like the power just, poof, goes away."

My team-leader demands the CE replace the power supply, even though it appeared to be operating normally. He does, at our cost, of course.

Another weeks goes by and all is forgotten in the swamp of work we have to do.

Until one day, the next week... Yes, you guessed it... It happens again. The server is down. Heads are exploding (will at least one head we all know by now). With all the screaming going on, the entire office staff should have comped some Advil.

My team-leader demands the facilities team do a full diagnostic on the UPS system and assure we aren't getting drop-outs on the power system. They do the diagnostic. They also review the logs for the power/load distribution to the entire lab and office spaces. Nothing is amiss.

This would also be a good time draw the picture of where this server is -- this particular server is not in the actual server room, it's out in the office area. That's on purpose, since it is connected to a demo robotics cabinet we use for testing and POC work. And customer demos. This will date me, but these were the days when robotic storage was new and VERY exciting to watch...

So, this is basically a couple of big boxes out on the office floor, with power cables running into a special power-drop near the middle of the room. That information might seem superfluous now, but will come into play shortly in our story.

So, we still have no answer to what's causing the server problems, but we all have work to do, so we keep plugging away, hoping for the best.

The team leader is insisting the VAR swap in a new server.

One night, we (the device-driver team) are working late, burning the midnight oil, right there in the office, and we bear witness to something I will never forget.

The cleaning staff came in.

Anxious for a brief distraction from our marathon of debugging, we stopped to watch them set up and start cleaning the office for a bit.

Then, friends, I Am Not Making This Up(tm)... I watched one of the cleaning staff walk right over to that beautiful RS/6000 dev server, dwarfed in shadow beside that huge robotic disc enclosure... and yank the server power cable right out of the dedicated power drop. And plug in their vacuum cleaner. And vacuum the floor.

We each looked at one-another, slowly, in bewilderment... and then went home, after a brief discussion on the way out the door.

You see, our team-leader wasn't with us that night; so before we left, we all agreed to come in late the next day. Very late indeed.

"years of experience" basically means nothing, both for people and organisations. You can work with someone who has 30 years experience who knows nothing, and someone with 1-2 years who's practically an expert.

Joined a large multi-national fresh out of college, that had been around for +90 years. I expected them to know software development inside and out. Didn't expect to see so many failed projects for stupid reasons, so many over sights, so many .... morons, to be honest.

Worked for a startup company where most only had 1 or 2 years more experience than me and learned so much.

Worked for a small company where everyone had 1.5 - 2 times my experience, where I learned the meaning of "bewilderment".

Never feel small, or less valuable because of a number. Theres a good chance you are working with jackasses - practiseSafeHex

Well today I got a fantastic surprise (truthfully). We hired a dev some months ago, who was on 6 months probation and, to put it politely, he was not going to pass it.

*side note: for details of some of the above, read my last 10 or so rants. They are pretty much all him.

Anyway, management put him on an improvement plan to make sure everything was fair, it wasn't working out, but they said we had to finish it to be fair.

So we had another 2 weeks left when he announced last night he's leaving for a new junior role, technical but not a dev.

Months of stress, heartache, bewilderment, late nights and weekends all just came to an end.

The English language fails me to express my overwhelming joy at this moment. The only way I can come close to it is to say that when he made his announcement, a colleague told me I should stop smiling as it could be taken as being rude.

I'd like to take this moment to thank the community for supporting me over the past few difficult months. Without you I probably would have tried to kill him with my dev rant stressball.

Thanks,
practiseSafeHex

Generic-IT
--------------
Client:"So we would like to found a new company and offer IT and network consulting. Would you be able to build our website?"

Me:"Absolutely. What will be the name of your company?"

Client:"The name is going to be 'Generic-IT'. The website is going to be 'generic-IT.com' . We checked that with google."

Me:"I am sorry to tell you that generic.com is already taken by another company. Incidentally that company offers the same services, that you intend to offer. They also seem to be quite big an have businesses in 5 different countries.
Because of this I advise you to pick a different name that does not get you into trouble and makes positioning your own brand easier."

Client:"We want to neglect that problem for now."

Me:"0.0 ..... -_-""""
"Well, listen. Apart from the possible branding and copyright problems imagine how people will find you on the web. ...What will happen if you google 'generic IT'?"

Client:"Yeah well, we want to neglect that. And with SEO you can do something about that."

Me:"..........Welllll, you that SEO is not a cure all, right? The older an bigger company will come up first. Why not avoid that missunderstanding and come up with a unique name?"

Client:"......"

Me:"Please tell me. Doesn't any part of my argument make sense to you?"

Client:"..."

Me:"Well, ok. I will send you the estimate on monday."

___________

Then over a back channel I hear that the client is ...bewildered, why I would not stick to my area of expertise.

There I was now. Left bewildered myself, being the one with the webagency that does frontend design and branding.

So once in school, I went to my CS lab to find my friend cursing under his breath. On asking him, he tells me that he can't figure out how to print his code in BlueJ(our school requires us to use BlueJ). So I told him to download another IDE and use that instead. He had a look of bewilderment in his eyes and asked me what an IDE is. He was absolutely shocked to learn that you could write Java programs elsewhere.

P.S This was our fourth year of Java

So my friend texts me with the message, you're smart can you figure it out?

What he sent me was a picture of a post, probably from Facebook which said "skynet is working on making you the next Terminator" with a bunch of bytes following it.

So I decode the first letter using an ascII table and sent it to him, and I get the response WOAH Really!!!

Lol even though it's super simple, it make see feel like 😎.