Learn More
Resend Email
35
Best office prank on you/from you?
Week 37 Group Rant
shine
5y

One of our newly-joined junior sysadmin left a pre-production server SSH session open. Being the responsible senior (pun intended) to teach them the value of security of production (or near production, for that matter) systems, I typed in sudo rm --recursive --no-preserve-root --force / on the terminal session (I didn't hit the Enter / Return key) and left it there. The person took longer to return and the screen went to sleep. I went back to my desk and took a backup image of the machine just in case the unexpected happened.

On returning from wherever they had gone, the person hits enter / return to wake the system (they didn't even have a password-on-wake policy set up on the machine). The SSH session was stil there, the machine accepted the command and started working. This person didn't even look at the session and just navigated away elsewhere (probably to get back to work on the script they were working on).

Five minutes passes by, I get the first monitoring alert saying the server is not responding. I hoped that this person would be responsible enough to check the monitoring alerts since they had a SSH session on the machine.
Seven minutes : other dependent services on the machine start complaining that the instance is unreachable.

I assign the monitoring alert to the person of the day. They come running to me saying that they can't reach the instance but the instance is listed on the inventory list. I ask them to show me the specific terminal that ran the rm -rf command. They get the beautiful realization of the day. They freak the hell out to the point that they ask me, "Am I fired?". I reply, "You should probably ask your manager".

Lesson learnt the hard-way. I gave them a good understanding on what happened and explained the implications on what would have happened had this exact same scenario happened outside the office giving access to an outsider. I explained about why people in _our_ domain should care about security above all else.

There was a good 30+ minute downtime of the instance before I admitted that I had a backup and restored it (after the whole lecture). It wasn't critical since the environment was not user-facing and didn't have any critical data.

Since then we've been at this together - warning engineers when they leave their machines open and taking security lecture / sessions / workshops for new recruits (anyone who joins engineering).

rant

security

ssh

wk37

rmrf

wk50

production

wk55
Favorite
Ranter
Comments
  • 5
    5y
    You know he screwed up by leaving a ssh connection open but why wasn't there an inactivity timeout setup? While this can be worked around it should be set to prevent this from happening. So as the responsible senior sysadmin you screwed up also unless the dumbass had keepalive setup to bypass it
  • 3
    5y
    @PerfectAsshole Maybe or maybe not.

    * Maybe, the senior sysadmin should have set a shorter inactivity timeout. (We do have a considerably long inactivity timeout for various reasons)

    * Maybe the junior sysadmin returned too quickly to beat the inactivity timeout.

    * Maybe if the senior sysadmin hadn't touched the session in between (resetting the inactivity counter), the session would've timed out.

    * Maybe the junior sysadmin had ServerAliveInterval set (but that's highly unlikely; they had just started with the team; ~ 1 week)

    that's a lot of possibilities of maybes.

    In a cracking scenario, none of this would matter. If the session is open, you're not only compromising your machine, but also another server.

    Lesson to be learnt : never leave an open SSH active if you're going away from your keyboard (AFK). It might be inconvenient to come back and start the SSH all over again and go back to whatever you're doing but that's the safest play you have.
  • 1
    5y
    @shine seeing that you said the screen went to sleep and the newby left ssh open for convenience that would make the assumption that the screen off timeout would be set to atleast 10 minutes a reliable assumption. But i guess it's just my security background being picky
  • 1
    5y
    @PerfectAsshole shouldn't it be shorter actually?

    The screen timeout by default is usually a minute or so. The SSH being active or not shouldn't matter really for the screen timeout to kick in. If I understand correctly, machines monitor the input devices (keyboard and mouse) for activity before kicking the screensaver in. So, if after I left the machine alone, was over a minute, the screensaver should've kicked in.
  • 1
    5y
    @shine i usually see screensavers set for 5 minutes and screen off for 10-20 minutes normally.
  • 4
    5y
    The first time I had to teach someone about computer security was on an 8088. While they were away, I modified the autoexec.bat file to print out "Formatting drive..." and then switch to black on black. That was in the last Century. The last time I taught someone a lesson, they emailed the entire team that they loved everyone so much that they were going to bring donuts and kolaches the following Wednesday. They actually brought the food in as penance for their crime. That was recently.
  • 5
    5y
    What a wanker thing to do. To a colleague and to your employer. Besides, it's illegal to check someone else's phone or computer without their permission.
  • 2
    5y
    @Sabro except if it's a company owned computer. It's sad that people have to be told to secure their system. It should be like breathing; it just gets done.
  • 1
    5y
    @Sabro not if they didn't bother to secure their systems so that I _couldn't_ check it (even if I wanted to).

    If they left their systems open and unsecured, they're inviting trouble. I just happened to be a passer-by who wanted to show them the importance of not leaving your machine unattended AND unsecured.

    And besides it was just not their machine that was left unsecured, there was a SSH session to a server left active and open. That's even worse.
  • 4
    5y
    @iAmNaN Not allowed to check work computer either, as employee can be expected to have some private stuff on there, and possibly confidential data. Nor is your employer allowefd to spy without due reason.
  • 2
    5y
    @Sabro Nobody is spying anyone here. The user consciously decided to leave their machine (and another company asset) unattended and unsecured.

    A passer-by saw the security crisis and exploited the opportunity to teach the user the value / impact of a security breach.
  • 1
    5y
    is that you, satan?
  • 2
    5y
    @Sabro This. I'd have a huge problem if I'd do this at work.

    I'd lock the pc for my colleague. Also, this is very illegal where I'm from (the running a command on someone else's computer part, the second one is not good but not illegal).
  • 1
    5y
    @linuxxx You could be the good Samaritan by locking your colleagues computer for them.

    1. They would never know that you did it for them. They'd just assume that the computer went to sleep on it's own.

    2. A cracker wouldn't do you that favor if you left your machine unattended / unsecured in a public environment (like a cafe, for example; where a lot of people prefer to work out of)

    That's why I decided to be the satan, like @ozeta86 prefers to call me, and did this to them.

    I didn't stop there though. I sat them down and gave them a whole lecture about why security was important (before I admitted I had a backup and restored it). Especially in a public setting. I gave them various scenarios to understand the possible implications of such seemingly trivial inconspicuous actions.
  • 1
    5y
    well, at my workplace I learned the lession of close sessions / locking the scren at my second day of work, as someone put on this page on my browser, in fullscreen https://geekprank.com/win10-update/

    rm -rf / seems a little brutal to me lol
  • 1
    5y
    @shine Oh no I'd report it. It's just that if I'd do what you did and someone would report it, I could face up to four years of prison time.

    It's no fucking joke here.
  • 2
    5y
    @ozeta86 The point is that you learnt it the hard way - brutal or not.

    And it still sticks with you. Maybe, mine was a bit intense, but it worked with this particular person.

    Now we're partners in crime together. \o/
  • 1
    5y
    @linuxxx I was probably lucky / fortunate that this person chose not to. And joined me together in the force. They probably now fully understand the implications of such a breach and decided that it was better to generate more awareness than report me, get me fired or even locked up in jail.
  • 2
    5y
    @shine You think of it what you want man, I'm just saying that I find it a very wrong thing to do.
  • 1
    5y
    While illegal, I do applaud your efforts, as even though it wasn't in use for anything but non-critical internal stuff, in an actual prod situation that could've EASILY lead to a pivot and a massive data breach of pretty much *everything* in the company facility (or even pivotable to other facilities depending on the setup!)

    @Sabro >employer not allowed to spy without due reason
    what fucking place do you live? In the US keyloggers and stealth recorders are legally allowed on ANYTHING handed out by the company (and is 99% of the time there and active 24/7 even without suspicion just because they can)
  • 0
    5y
    Who the fuck wakes computer/screen by pressing enter? How retarded is this guy? C
  • 1
    5y
    I know, right? I'm curious though; what's your preferred key to wake your machine from sleep?
  • 1
    5y
    @shine i use CTRL. It's on the edge of the keyboard and never does anything alone
  • 0
    5y
    Nice, mine is the space because it is the biggest key on the keyboard - easy to spot, big enough not to miss and is loud enough to bang on.
  • 0
    5y
    That might cause problems if you have for example a confirmation window with active selection
  • 1
    5y
    sure, but I always have my login screen for my machine to wake up to, not an unlocked session with a terminal screen with an SSH session to a production machine.
Related Rants
devRant © 2021 Hexical Labs LLC
Privacy Policy  |  Terms of Service
Add Comment