TL;DR: disaster averted!

Story time!

About a year ago, the company I work for merged with another that offered complementary services. As is always the case, both companies had different ways of doing things, and that was true for the keeping of the financial records and history.

As the other company had a much larger financial database, after the merger we moved all the data of both companies on their software.

The said software is closed source, and was deployed on premises on a small server.

Even tho it has a lot of restrictions and missing features, it gets the job done and was stable enough for years.

But here comes the fun part: last week there was a power outage. We had no failsafe, no UPS, no recent backups and of course both the OS and the working database from the server broke.

Everyone was in panic mode, as our whole company needs the software for day to day activity!

Now, don't ask me how, but today we managed to recover all the data, got a new server with 2 RAID HDDs for the working copy of the DB, another pair for backups, and another machine with another dual HDD setup for secondary backups!

We still need a new UPS and another off site backup storage, but for now...disaster averted!

Time for a beer! Or 20...

That is all :)

This is not an interview test just an awkward experience in general regarding interview.

This happened two years ago when I was a fresh university graduate looking for a job in UK as an immigrant (Im EU national).

Went to an interview for a web dev+tech support position. Two fat guys with tshirts met me and started interviewing me for a sysadmin position. Started asking me about disaster recovery and stuff.

Turns out recruiter messed up not only companies but positions as well. Also these two guys didnt bother to check anything.

I pulled out the job ad for which I applied originally, interviewers had a look at it and still proceeded questioning me while knowing that I prepared for completely different position interview.

Needless to say, it went terrible and I didnt get the job. I dont know if its just me or Im unlucky, but I had a lot of encounters in UK with so many incompetent recruiters.

Investigating a "bug"(turned out to be user error) on the production servers, accidently wiped out the permissions for 1,800 users.
Thankfully was able to recover them in under 10min. God bless a solid disaster recovery policy.

"UPDATE table SET field = 1"
Neglected to include "WHERE ClientId = XXX"

1800 users loose access to thier software in the middle of the day

Restored in 10 minutes thanks to disaster recovery policy.

Full stack developer.

I know what it's supposed to mean, but I feel like it gives discredit to the devs who perfect their area (frontend, backend, db, infrastructure). It's, to me, like calling myself a chef because I can cook dinner..

The depth, analysis and customization of the domain to shape an api to a website is never appreciated. The finicle tweaks on the frontend to make those final touches. Then comes a brat who say they are full stack, and can do all those things. Bullshit. 99.9% of them have never done anything but move data through layers and present it.

Throw these wannabes an enterprise system with monoliths and microservices willy nelly, orchestrate that shit with a vertical slice nginx ssi with disaster recovery, horizontal scaling, domain modeling, version management, a busy little bus and events flowing all decimal points of 2pi. Then, if you fully master everything going on there, I believe you are full stack.

Otherwise you just scraped the surface of what complexities software development is about. Everyone who can read a tutorial can scrape together an "in-out" website. But if your db is looking the same as your api, your highest complexity is the alignment of an infobox, I will laugh loud at your full stack.

And if you told me in an interview that you are full stack, you'd better have 10+ years experience and a good list of failed and successful projects before I'd let you stay the next two minutes..

Don't use your senior software engineer title or years of experience as reasons in a debate or argument about software

My manager was asking me what steps needs to be done to perform a disaster recovery for our cluster( on production). I will be honest here, I have not maintained this type of cluster(kubernetes) in production before. However, I have enough understand of the system to answer my boss question. I basically told him there are A, B, C you have to do.

My senior developer jumped in and said "No you should do A,C, B because C is more critical than B. " I then replied to him: "I understand your point, I notice that too, but .." Before I can even finish my sentence, this dude has already rolled his eyes and interrupted me very loudly: "Have you worked with these systems on production before? I did". The asshole knows I haven't maintained Kubernetes on production yet of course.

I got super pissed at him and pretty much shouted back to him and my manager: "Just because I haven't worked on this system on production yet, does not mean my argument is wrong" .

I then dragged my managers, that asshole, and other engineers in a room and settle this out. In the end, people agreed with my steps over that asshole senior engineer dude because I gave them rational reasons.

The conclusion is: Your senior title is given by the company, It doesn't mean anything to me. Also, it doesn't make you more right than another person just because he has a "lower" title than you.

Blizzard coming this weekend. I get to spend the night at work because our disaster recovery plan isn’t in full swing (Yes it’s paid, but I am going to lose my mind). Awesome. Suck my ass, winter.

TL;DR - (almost) childhood trauma due to Wesrern Digital crap products lead to lot of data loss and a plege to not trust or purchase their products for the rest of my life.

....

So, I got my first ever Wester Digital 2TB Mybook, back when 2TB was a really big thing. While in the midst of moving (not copying) a LOT of data to it, the damn disk just.. died. There was no fall, no power outage, no damage, it just stopped working. I was out of words and out of options. Tried yanking out the disk and connecting it directly to a system, but no luck because it looks like it's the HDD mobo that died.

Also stupid young me did not realise back then that, even if a "moved" the data, the original data is still most likely in their original location, and so, never bothered a recovery.

Lots of good stuff lost that day.

And as with a lot of you, my disaster recovery system kicked up 10 fold. Now I got redundant local and cloud backup copies of all critical and otherwise unattainable data.

As you may have guessed, I never bought another Wester Digital product ever again. My internal HDDs are Segate, and external is a suprisingly long lived Toshiba Canvio.

A few months ago I applied for an IT Support role managing computer systems for a smaller manufacturing corporation. Now some back story, I'm a recent college grad looking for work and this hit my radar. I did well in the phone interview and really enjoyed the in person interview as well.

However, if I was offered the role I'd be the only person working on their infrastructure. The person who I interviewed with was leaving and thus his position was available. It was kinda strange to interview with the person you'd be replacing.

I started asking questions about their critical infrastructure and how they manage it. Short answer is they don't know.

I asked about off-site disaster recovery. "Oh we back everything up to a 2TB disk and I take it home every day."
I asked "What if that backup fails?"
Their response was "That would suck."

The company decided to go with a managed IT solution instead of me as I don't have the required experience in their eyes. The previous guy left because they we're stuck in their ways.
Yah, no thank you.

acquiring ownership of the bitchwork disaster recovery nobody wants to do

Who the fick asks for BPMN diagrams, Disaster Recovery diagrams and Business Continuity Plan for a SaaS product that is deployed in cloud (Azure) ??

This is a simple app/dashboard that just showcases some data in fancy charts. It shows the data YOU feed. Why do you need BPMN diagram? What am I supposed to put? "Client loads data" and "user sees data"?

I already linked Azure's page on DR. Do yiu want me to copy-paste the contents from that link?

Clients are too much PITA!!!

No idea what the fuck just happened, but my home router just dropped the internet connection and started demanding that I change the admin (default) password.
Now, I know that default passwords are bad and all that, but why the fuck now? This thing has been sitting there for over a year, and it only decided to complain now?

There have been some weird things going on lately, and I'm starting to worry that some of my systems may have been compromised in some way... but I'm not sure what/how, nor how to look for it...

Any tips for identifying a breach and disaster recovery?

Serious question guys.

How do you deal with stress of f-ing up at work?

I had to upgrade a whole postgresql stack today. Most of the upgrade went fine, but... Restoring a backup by pg_basebackup lead to an unusable database (would not replicate), had to apologize to the client and make last second modifications as disaster recovery, and all the while, ever since the DB didn't start up, to when I eventually went back to work and was no longer alone on the task, I was going through a crippling anxiety...

I... Love the job, but incidents like this... Make me doubt my future as anything more than a mediocre sysadmin...

So I decided to have some funzies over the weekend. Started XCOM 2. Built a team for a mission, spent an hour customising. Then I spent 3 hours going through and not failing miserably even once. The last fucking part of the mission, my favourite sniper dies after getting hammered in the face with a fucking laser beam. Then I lose control of the battlefield and everyone dies. Start again because I'm NOT gonna lose my awesome sniper!

This game is like designing software, planning deployment strategies, and disaster recovery at the same time. Recommended for all devs!

Is disaster recovery capability to meet business requirements and testing it part of your new system implementation roadmap?

Been asking the devops guy to schedule database backups for some of our side services for weeks! Just waiting for the day it all goes to hell and have no disaster recovery plan to fall back on.

Hey if your company refreshing / retiring laptops for the latest greatest - you might want to think about using for disaster recovery. Store them in another location or home. Push out updated image automatically. Recently saved a company 400k recurring expense with this strategy.

When you roll out a new system is disaster recovery testing on your roadmap? For example, do you do failover and have users work through a comprehensive test script and have them sign off before going into production?

When you identify your most critical process and do insufficient mitigation against loss. Bad.

Two new paragraphs in our Contingency and Disaster Recovery Plan.

If you had to evacuate your office due to a fire without your laptop and the office was destroyed could you continue to work? Do you have offsite recovery laptops?

Developers usually have flexibility where they can work from but sometimes do not have backup machines available and configured.

As a side question - would your other critical processes - accounting, HR, sales be able to continue to keep your business going in a disaster - or would they be like deer in headlights?

Try leveraging retired laptops for disaster recovery. Companies can realize resilience and save a fortune doing this. Hey, if everyone evacuated your office right now could you continue to keep your critical processes going?

Microsoft is responsible for protecting the Office 365 physical and virtual infrastructure and ensuring availability. Although Microsoft addresses certain security threats, it cannot prevent all malicious threats. Businesses are responsible for protecting their data. This means that if a business’s Office 365 data is compromised or corrupted, it is not Microsoft’s job to restore the data outside of the Software Licensing Terms. To protect data, businesses need to make sure they have office 365 disaster recovery and recovery plans in place

All day developing a solution for a major incident to be told 'thanks but don't deploy, we only wanted to show we had the capability' FFS I have a project deploying on Monday!! I have shit to do!

CONSULT A REPUTABLE LOST CRYPTO RECOVERY SERVICES // DIGITAL HACK RECOVERY

Moving is one of life’s great stressors, but losing access to your crypto wallet during the chaos? That’s next-level stress. Picture this: I had just moved into a new apartment, ready to start fresh, and during the madness of unpacking, I somehow misplaced my hardware wallet containing a staggering $480,000 in Bitcoin. Yes, you read that right—$480,000! It felt like losing a winning lottery ticket in a sea of boxes.
In a panic, I began tearing through every box, drawer, and closet like a contestant on a home improvement show, only to come up empty-handed. “It has to be here somewhere!” I muttered to myself, desperately rifling through bubble wrap and crumpled packing paper. I was convinced my wallet had sprouted legs and decided to go on an adventure of its own. After a few sleepless nights filled with nightmares of lost funds and dwindling hopes, I knew I needed help.
That’s when I called Digital Hack Recovery. I figured if anyone could help me locate my missing wallet, it was them. Their reputation for expertise in wallet recovery was my only glimmer of hope amidst the chaos. As I explained my predicament complete with an embarrassing recount of my moving blunders—their team remained unfazed by my moving mishap. They assured me that they had dealt with all sorts of wallet dilemmas, from the absurd to the downright tragic. I felt a wave of relief wash over me.
Their team immediately got to work on the case, employing their advanced forensic tools to dive into the digital depths of my lost wallet’s information. As the hours passed, I paced around my new apartment, trying to focus on anything other than the sinking feeling in my stomach. Would they be able to recover my funds, or was I destined to wallow in my own packing misfortune?
To my sheer astonishment, it didn’t take long for them to work their magic. In no time at all, they had recovered my wallet’s information and confirmed that my $480,000 was safe and sound! I felt like I’d been handed a lifeline amidst the turmoil of my move.
Now, I label everything during moves including my wallet! If you ever find yourself in a similarly stressful situation, I can’t recommend Digital Hack Recovery enough . They turned my moving disaster into a success story, saving my sanity and my life savings in the process! For quick assistance talk to Digital Hack Recovery through⁚

WhatsApp +19152151930
Email; digital hack recovery @ techie . com
Website; https : // digital hack recovery . com

BITCOIN FRAUD-RECOVER STOLEN ASSETS HIRE TECHNOCRATE RECOVERY

I decided to explore the world of cryptocurrency, intrigued by the possibilities it offered. With every hard-earned dollar I saved, I ventured into Bitcoin, investing a substantial amount that eventually totaled $370,000. I was excited about the prospect of growing my savings and ensuring my children would have a stable future. But, as fate would have it, disaster struck when I lost access to my Bitcoin wallet. It happened in a flash. I was multitasking—baking, icing cupcakes, and managing customer orders—when I accidentally clicked on a phishing link disguised as an update for my cryptocurrency app. In an instant, my wallet was compromised, and I was locked out. The realization hit me hard: my investment, my dreams, and my family’s future were hanging by a thread. Feeling utterly devastated and betrayed by my own oversight, I knew I had to act fast. I shared my predicament with a close friend who had been through a similar experience, and they recommended TECHNOCRATE RECOVERY. Skeptical but desperate, I reached out to their team. From the moment I contacted them, I felt an overwhelming sense of support. The professionals at TECHNOCRATE RECOVERY Tool were compassionate and knowledgeable, assuring me that they could help me navigate this overwhelming situation. Their team sprang into action, utilizing advanced technology to track the transactions that led to my locked assets. Within days, they uncovered critical information that would lead to the recovery of my funds. Finally, after what felt like an eternity, I received the news I had been praying for: my Bitcoin had been successfully recovered. Thanks to their incredible work, I can continue focusing on my bakery and my family, knowing I have the tools to navigate the digital world securely.

CONTACT BELOW INFO FOR HELP.

Inbox Email: t echnocratrecovery@contractor. net

My story is about a digital disaster. I thought I had $950,000 worth of Bitcoin stored securely on a hard drive. I was confident in my setup—until one day, the hard drive crashed unexpectedly. Suddenly, I found myself locked out of my own money because I had lost access to my private keys. I did have backups, but they were encrypted, and for the life of me, I couldn’t remember the password I had chosen to protect them. It felt like I was staring at a vault filled with cash but had no key to get inside, I spent countless hours searching for recovery tools online, trying everything I could find, but nothing worked. I felt more hopeless with each failed attempt. It was a terrifying experience to think I might lose everything I had worked for all those years. Just when I thought all hope was gone, I came across Digital Web Recovery while browsing online forums. At first, I was unsure if they could truly help me, but desperation pushed me to give them a shot, From the moment I reached out to them, their team was incredibly responsive and professional. They took the time to understand my situation and reassured me that they had handled similar cases before. This was the first time I felt a glimmer of hope since the hard drive crash. They explained their process for decrypting backups and gave me confidence that they could help, Over the next week, Digital Web Recovery worked diligently on my case. They kept me updated throughout the entire process, which eased my anxiety. When I received the message that they had successfully recovered my private keys, I could hardly believe it. The moment I logged into my wallet and saw my Bitcoin balance again was a feeling I’ll never forget. It was like a weight had been lifted off my shoulders. I had come so close to losing everything, but Digital Web Recovery saved the day If you ever find yourself in a similar bind, I highly recommend reaching out to Digital Web Recovery. These guys are the real deal. They know what they’re doing and truly care about helping their clients. Thanks to their expertise, I have my Bitcoin back and can finally breathe easy again Website; https: // digitalwebrecovery. com WhatsApp; ‪+13433003465‬

HIRE A CRYPTO RECOVERY EXPERT; BEST BITCOIN RECOVERY SERVICE VISIT CYBER CONSTABLE INTELLIGENCE

It was a devastating blow - one moment I was the proud owner of a small digital fortune, 8,000 Bitcoin that I had painstakingly accumulated over years of savvy investing and cautious storage. The next, that entire life-changing sum had vanished without a trace, lost to a cruel twist of fate and my own careless misstep. I had fallen victim to a data breach, leaving me helpless to prevent the thieves from making off with my entire cryptocurrency holdings. The anguish was palpable, a sinking feeling of hopelessness and despair as I realized the scale of my loss. Thousands upon thousands of hard-earned Bitcoins, now in the hands of faceless criminals, beyond my reach. But I refused to give in to despair. Determined to recover what was rightfully mine, I sought out the services of Cyber Constable Intelligence, a specialized team of blockchain forensics experts renowned for their ability to track down and reclaim stolen digital assets. Through their meticulous investigative work, leveraging the transparency of the Bitcoin ledger and their deep technical expertise, they were able to painstakingly trace the movement of my stolen funds, identifying the wallet addresses the thieves had used to launder the cryptocurrency. With this critical intelligence in hand, Cyber Constable Intelligence then coordinated with law enforcement to freeze those illicit accounts, preventing the funds from being moved any further. The final step was a delicate negotiation process, with the recovery team using their connections and influence to compel the thieves to return the stolen Bitcoins - a tense and high-stakes affair, but one that ultimately proved successful. After weeks of anxious waiting, I was overjoyed to have my 8,000 BTC safely returned to my control, a true testament to the skill and determination of the Cyber Constable Intelligence team. It was a harrowing ordeal, but one that reinforced the importance of robust security measures and the remarkable capabilities of specialized firms dedicated to combating cryptocurrency crime and restoring rightful ownership. Losing 8,000 BTC was one of the most stressful events of my life, but thanks to Cyber Constable Intelligence, I was able to reclaim what I thought was lost forever. Their professionalism, technical expertise, and commitment to security made all the difference. If you find yourself in a similar situation, I cannot recommend them highly enough. Cyber Constable Intelligence turned a financial disaster into a remarkable recovery, and I will forever be grateful for their role in securing my assets.

CYBER CONSTABLE INTELLIGENCE INFO:
Website: w w  w. cyberconstableintelligence com