You know your week starts great, when you receive a call, that the whole shop database of a company is not working anymore.
Turns out MS Fucking SQL decided to drop all tables, just because the harddisk was full.

A SERIOUS FUCK YOU TO MICROSCHROTT!

It's not a lot of fun, when 15 people have to wait for you about 1 hour, until it is restored.

Why does this have to happen on a monday?

Timezones. So, general rules are:

1. If you don't store timezone, always use and assume UTC. Databases, backends, whatever you use, all time must be kept be in UTC.
2. If you store timezones, ensure you store them everywhere and don't drop them anywhere.
3. It's always better to ignore backend server time in favor of database's `now().` Having a single source of truth makes time consistent (if it's the same database, obviously). If you combine backend time and database time, you likely get a violation of causality.

I've just spent a couple of hours investigating "weird random one-hour time drifts on updates." Guys violated all three rules above:
- they didn't store the timezone;
- their servers had inconsistent timezones. Java was in +XX., while the server itself in UTC. On one host, they forgot to put JVM in the same timezone;
- they dropped the timezone because they thought it was the same everywhere, so there was no point in serializing it.

Thought I dropped the wrong table from a database. Worse 2 minutes of my life!

Worst data loss? Probably back when I was working at gitlab and accidentally dropped a production database....

I sometimes remember the time when I wrote a Email-inbox-exporter-PHP-script-type of application that collects all the emails from an inbox, "copied" it to a database with the attachements and stuff and moves it to a folder..

I just started at the company for like a couple of months, had no privileges to create mailboxes and such and I didn't want to interrupt our programmer to do this for me, so... I decided.. to save time and resources.. to test run it on our global, live 'support' mailbox.. :D Well.. You might guess what happened.. Apparently I mistyped the name of the move-destination folder (because imap-weird-things) that resulted in a completly empty mailbox and an empty database because the inserts failed due to bad encoding and mime-type issues..

The moment I refreshed my Outlook and noticed that all our mails where gone.. I swear, I can't describe that feeling of fear, cold sweat, intense heartbeat... I just stood up, asked if anyone wanted coffee, and just walked out of the office.. When in the hallway, I heard my collegues ask to one another "do you have any issues with outlook, all my mails are gone?". Everyone was stressing out, the chief was stressing out "what happened?!", nobody knew what happened.. :D

They could partially resolve it via one collegue who hadn't refreshed the mailbox and he could forward all the mails back to our support mailbox..

I dropped the project idea and learned to work with dev environments :D A couple of months later, I accidentially forgot a where condition in my SQL UPDATE statement, but that was the last time I seriously f*cked up.. :D Got to learn the hard way I guess.. Now everything I do runs in dev environments, I test everything before publishing,.. When I look back.. I don't even recognize the (inexperienced) guy I was back then ! :D

Ps. No one still knows what happened that day and they blamed it on server issues :D

I've optimised so many things in my time I can't remember most of them.

Most recently, something had to be the equivalent off `"literal" LIKE column` with a million rows to compare. It would take around a second average each literal to lookup for a service that needs to be high load and low latency. This isn't an easy case to optimise, many people would consider it impossible.

It took my a couple of hours to reverse engineer the data and implement a few hundred line implementation that would look it up in 1ms average with the worst possible case being very rare and not too distant from this.

In another case there was a lookup of arbitrary time spans that most people would not bother to cache because the input parameters are too short lived and variable to make a difference. I replaced the 50000+ line application acting as a middle man between the application and database with 500 lines of code that did the look up faster and was able to implement a reasonable caching strategy. This dropped resource consumption by a minimum of factor of ten at least. Misses were cheaper and it was able to cache most cases. It also involved modifying the client library in C to stop it unnecessarily wrapping primitives in objects to the high level language which was causing it to consume excessive amounts of memory when processing huge data streams.

Another system would download a huge data set for every point of sale constantly, then parse and apply it. It had to reflect changes quickly but would download the whole dataset each time containing hundreds of thousands of rows. I whipped up a system so that a single server (barring redundancy) would download it in a loop, parse it using C which was much faster than the traditional interpreted language, then use a custom data differential format, TCP data streaming protocol, binary serialisation and LZMA compression to pipe it down to points of sale. This protocol also used versioning for catchup and differential combination for additional reduction in size. It went from being 30 seconds to a few minutes behind to using able to keep up to with in a second of changes. It was also using so much bandwidth that it would reach the limit on ADSL connections then get throttled. I looked at the traffic stats after and it dropped from dozens of terabytes a month to around a gigabyte or so a month for several hundred machines. The drop in the graphs you'd think all the machines had been turned off as that's what it looked like. It could now happily run over GPRS or 56K.

I was working on a project with a lot of data and noticed these huge tables and horrible queries. The tables were all the results of queries. Someone wrote terrible SQL then to optimise it ran it in the background with all possible variable values then store the results of joins and aggregates into new tables. On top of those tables they wrote more SQL. I wrote some new queries and query generation that wiped out thousands of lines of code immediately and operated on the original tables taking things down from 30GB and rapidly climbing to a couple GB.

Another time a piece of mathematics had to generate all possible permutations and the existing solution was factorial. I worked out how to optimise it to run n*n which believe it or not made the world of difference. Went from hardly handling anything to handling anything thrown at it. It was nice trying to get people to "freeze the system now".

I build my own frontend systems (admittedly rushed) that do what angular/react/vue aim for but with higher (maximum) performance including an in memory data base to back the UI that had layered event driven indexes and could handle referential integrity (overlay on the database only revealing items with valid integrity) or reordering and reposition events very rapidly using a custom AVL tree. You could layer indexes over it (data inheritance) that could be partial and dynamic.

So many times have I optimised things on automatic just cleaning up code normally. Hundreds, thousands of optimisations. It's what makes my clock tick.

This just happened at work, except for the fact the entire database was dropped...

Worst collaboration experience story?

I was not directly involved, it was a Delphi -> C# conversion of our customer returns application.

The dev manager was out to prove waterfall was the only development methodology that could make convert the monolith app to a lean, multi-tier, enterprise-worthy application.

Starting out with a team of 7 (3 devs, 2 dbas, team mgr, and the dev department mgr), they spent around 3 months designing, meetings, and more meetings. Armed with 50+ page specification Word document (not counting the countless Visio workflow diagrams and Microsoft Project timeline/ghantt charts), the team was ready to start coding.
The database design, workflow, and UI design (using Visio), was well done/thought out, but problems started on day one.
- Team mgr and Dev mgr split up the 3 devs, 1 dev wrote the database access library tier, 1 wrote the service tier, the other dev wrote the UI (I'll add this was the dev's first experience with WPF).
- Per the specification, all the layers wouldn't be integrated until all of them met the standards (unit tested, free from errors from VS's code analyzer, etc)
- By the time the devs where ready to code, the DBAs were already tasked with other projects, so the Returns app was prioritized to "when we get around to it"

Fast forward 6 months later, all the devs were 'done' coding, having very little/no communication with one another, then the integration. The service and database layers assumed different design patterns and different database relationships and the UI layer required functionality neither layers anticipated (ex. multi-users and the service maintaining some sort of state between them).

Those issues took about a month to work out, then the app began beta testing with real end users. App didn't make it 10 minutes before users gave up. Numerous UI logic errors, runtime errors, and overall app stability. Because the UI was so bad, the dev mgr brought in one of the web developers (she was pretty good at UI design). You might guess how useful someone is being dropped in on complex project , months after-the-fact and being told "Fix it!".

Couple of months of UI re-design and many other changes, the app was ready for beta testing.

In the mean time, the company hired a new customer service manager. When he saw the application, he rejected the app because he re-designed the entire returns process to be more efficient. The application UI was written to the exact step-by-step old returns process with little/no deviation.

With a tremendous amount of push-back (TL;DR), the dev mgr promised to change the app, but only after it was deployed into production (using "we can fix it later" excuse).

Still plagued with numerous bugs, the app was finally deployed. In attempts to save face, there was a company-wide party to celebrate the 'death' of the "old Delphi returns app" and the birth of the new. Cake, drinks, certificates of achievements for the devs, etc.

By the end of the project, the devs hated each other. Finger pointing, petty squabbles, out-right "FU!"s across the cube walls, etc. All the team members were re-assigned to other teams to separate them, leaving a single new hire to fix all the issues.

Soooo... My senior dev just dropped a very importand table om our production enviroment.
Time to restore the database.. only 50 GB left

Completely got my localhost and live database confused and dropped the whole live server. And there was a power outage so the last backup is from Friday. Luckily not completely live, but still having a stream of people walking in. Also pretty obvious that they are talking about me especially since there's no other shes in the department.

I was testing database migration and there was some issues which I couldn't fix them. So I drop the table just to test if the issues are still there.
"Why nothing have been changed?" After some minutes I realize it was production database which I had dropped 😓

I work on a warehouse dev team. One day this past year, I was trying to deploy a new build to a QA server. Earlier that day I had been looking at the logs on the production server and had left the ssh session open. I had been working for less than a year out of college at this point and shouldn't have had access to deploy to the production server.

Long story short I deployed my QA build to the production server and saw there were problems connection to our production database. Then my heart dropped in my chest as I realized I had just brought down our production server.

I managed to get the server back up by rolling back in about 5 minutes and no one ever knew except some people on my team.

I felt horrible for the longest time. Later in the year another guy that joined my team that has about 20 years of experience under his belt did the exact same thing, but needed help rolling it back. Needless to say, that made me feel a lot better. 😂

Definitely the worst moment of my year.

Today I dropped and updated a database to override it with local changes. Turned out the same dB was used by another project.

I just destroyed a week worth of work and content.

Good thing I took a dump before dropping everything. Got it running in 5 Minutes

I started programming when I was 14, because I was deeply enrooted in MMORPG hacking communities. It gave me an escape from real life, and I felt empowered by the skill to create something from nothing. My first language was Lazarus FPC, followed by VB.NET, C#, C++ ( managed and unmanaged non CLR ). As time went on, I found more ways to turn my "hacks" into software, and finally I began selling subscriptions which required me writing an authentication system.

After weeks of research, I began writing my own REST API in PHP using MySQL as my database. At this point I had an IPB forum up and running for a year, but with my newly acquired knowledge I was able to couple my API with my forum software. To properly distribute my API i had to learn NGINX to route my API to a subdomain.

Soon after I began writing my own portal for my authentication system, at which point I had become entirely enveloped in Web Development. I was 17 when I dropped my forum, I'm now 21 and freelancing web app consulting, day job as a QA automation developer.

I was cleaning up dangling images in docker, and I accidentally removed the production database container as well.

Its not a big issue, I can just up the container back and everything should be fine. But after I up the container and connected to the database, I found out there's no data inside. I thought I fucked up, and sent msg in slack channel that I nuked the db.

Later my friend asked me which compose file I am using and that's when I realized I used the wrong config to up the db. Used the correct config to up the database again and everything goes back to normal.

It's friday evening and if I really dropped the db it would be fucking bad weekend....

A couple of years ago, we decide to migrate our customer's data from one data center to another, this is the story of how it goes well.

The product was a Facebook canvas and mobile game with 200M users, that represent approximately 500Gibi of data to move stored in MySQL and Redis. The source was stored in Dallas, and the target was New York.

Because downtime is responsible for preventing users to spend their money on our "free" game, we decide to avoid it as much as possible.

In our MySQL main table (manually sharded 100 tables) , we had a modification TIMESTAMP column. We decide to use it to check if a user needs to be copied on the new database. The rest of the data consist of a savegame stored as gzipped JSON in a LONGBLOB column.

A program in Go has been developed to continuously track if a user's data needs to be copied again everytime progress has been made on its savegame. The process goes like this: First the JSON was unzipped to detect bot users with no progress that we simply drop, then data was exported in a custom binary file with fast compressed data to reduce the size of the file. Next, the exported file was copied using rsync to the new servers, and a second Go program do the import on the new MySQL instances.

The 1st loop takes 1 week to copy; the 2nd takes 1 day; a couple of hours for the 3rd, and so on. At the end, copying the latest versions of all the savegame takes roughly a couple of minutes.

On the Redis side, some data were cache that we knew can be dropped without impacting the user's experience. Others were big bunch of data and we simply SCAN each Redis instances and produces the same kind of custom binary files. The process was fast enough to launch it once during migration. It takes 15 minutes because we were able to parallelise across the 22 instances.

It takes 6 months of meticulous preparation. The D day, the process goes smoothly, but we shutdowns our service for one long hour because of a typo on a domain name.

Are you even a dev if you haven't dropped a production database at least once?

Welp, this made my night and sorta ruined my night at the same time.

He decided to work on a new gaming community but has limited programming knowledge, but has enough to patch and repair minor issues. He's waiting for an old friend of his to come back to start helping him again, so this leads to me. He needed a custom backend made for his server, which required pulling data from an SQL/API and syncing with the server, and he was falling behind pace and asked for my help. He's a good friend that I've known for a while, and I knew it wouldn't take to long to create this, so I decided to help him. Which lead to an interesting find, and sorta made my night.

It wasn't really difficult, got it done within an hour, took some time to test and fix any bugs with his SQL database. But this is where it get's interesting, at least for me. He had roughly a few hundred people that did beta testing of the server, anyways, once the new backend was hooked in and working, I realized that the other developer he works with had created a 'custom' script to make sure there are no leaks of the database. Well, that 'custom' script actually begins wiping rows/tables (Depends on the sub-table, some get wiped row by row, some just get completely dropped), I just couldn't comprehend what had happened, as rows/tables just slowly started disappearing. It took me a while of checking, before checking his SQL query logs (At least the custom script did that properly and logged every query), to realize it just basically wiped the database.

Welp, after that, it began to restrict the API I was using, and due to this it identified the server as foreign access (Since it wasn't using the same key as his plugin, even though I had an API key created just so it could only access ranks and such, to prevent abuse) and begin responding not with denied, but with a lovely "Fuck you hacker!" This really made my night, I don't know why, but I was genuinely laughing pretty hard at this response.

God, I love his developer. Luckily, I had created a backup earlier, so I patched it and just worked around the plugin/API to get it working. (Hopefully, it's not a clusterfuck to read, writing this at 2 am with less than an hour of sleep, bedtime! Goodnight everyone.)

While doing my undergrad, we had to submit a major project in last sem. We were grouped into teams. Spent whole semester doing nothing and near the deadline, a friend bought a project and i just changed the footer. On submission, we had the discussion with the external lecturer who viewed our project and he asked us to write down the number of tables we used in our project in separate paper each. A friend wrote 5 , I wrote 12, and the friend who has bought the project wrote 17. Turns out there were 11 tables. I said someone must have dropped a table.. And later made up saying i was doing the database and others were doing the rest of the project.
It was one of the lamest excuse we made to escape such uncomfortable situation.

1) Learning little to nothing useful in formal post-secondary and wasting tons of time and money just to have pain and suffering.

"Let's talk about hardware disc sectors divisions in the database course, rather than most of you might find useful for industry."

"Lemme grade based on regurgitating my exact definitions of things, later I'll talk about historical failed network protocols, that have little to no relevance/importance because they fucking lost and we don't use them. Practical networking information? Nah."

"Back in the day we used to put a cup of water on top of our desktops, and if it started to shake a lot that's how you'd know your operating system was working real hard and 'thrashing' "

"Is like differentiation but is like cat looking at crystal ball"

"Not all husbands beat their wives, but statistically...." (this one was confusing and awkward to the point that the memory is mostly dropped)

Streams & lambdas in java, were a few slides in a powerpoint & not really tested. Turns out industry loves 'em.

2) Landed my first student job and get shoved on an old legacy project nobody wants to touch. Am isolated and not being taught or helped much, do poorly. Boss gets pissed at me and is unpleasant to work with and get help from. Gets to the point where I start to wonder if he starts to try and create a show of how much of a nuisance I am. He meddle with some logo I'm fixing, getting fussy about individual pixels and shades, and makes a big deal of knowing how to use GIMP and how he's sitting with me micromanaging. Monthly one on one's were uncomfortable and had him metaphorically jerking off about his lifestory career wise.

But I think I learned in code monkey industry, you gotta be capable of learning and making things happen with effectively no help at all. It's hard as fuck though.

3) Everytime I meet an asshole who knows more and accomplish than I do (that's a lot of people) with higher TC than me (also a lot of people). I despair as I realize I might sound like that without realizing it.

4) Everytime I encounter one of my glaring gaps in my knowledge and I'm ashamed of the fact I have plenty of them. Cargo cult programming.

5) I can't do leetcode hards. Sometimes I suck at white board questions I haven't seen anything like before and anything similar to them before.

6) I also suck at some of the trivia questions in interviews. (Gosh I think I'd look that up in a search engine)

7) Mentorship is nigh non-existent. Gosh I'd love to be taught stuff so I'd know how to make technical design/architecture decisions and knowing tradeoffs between tech stack. So I can go beyond being a codemonkey.

8) Gave up and took an ok job outside of America rather than continuing to grind then try to interview into a high tier American company. Doubtful I'd ever manage to break in now, and TC would be sweet but am unsure if the rest would work out.

9) Assholes and trolls on stackoverflow, it's quite hard to ask questions sometimes it feels and now get closed, marked as dupe, or downvoted without explanation.

I’m either going crazy or someone just dropped a table from a database I’m actively working in. I don’t have a drop script for any tables in my code, not anywhere. One moment it’s right there, the next it’s gone? Luckily it’s pretty much empty, only generic data. On my way to check permissions.

I studied computer game development at a university that had pretty low standards. It was perfect for a slacker like me. I enjoyed it. Maybe it was implied that you'd have to study on your own and that completing the courses wasn't enough to make you a competent developer, but maybe I'm a bit slow or something because that never occurred to me until after we were finished.

I was taught enough programming and database stuff to land me an entry position job at a consulting firm before I was able to complete my thesis. Technically I dropped out, I guess, since I don't have a diploma.

I built a portfolio consisting of different projects/essays I'd completed/wrote for different courses. That, together with my charm and boyish good lucks made me get the job.

Anyways, even though I learned more practical stuff my first year on the job than I did my 3 years of uni it was a very good experience. It helped me understand what I was interested in so that I could pursue that later and some of the people I got to know would help my career later.

I mean, if education wasn't free (except living expenses, books, etc) I'd might say that I had been better off just taking a year of egghead/udemy/Indian Guy On YouTube classes to learn what I needed to land myself a job. But I'd need to know which courses to take so I'd probably find a group of courses that someone else put together. I guess it would be nice to take those classes with other people so that you can work together, learn from each other, and make some friends and connections as well. Oh, that sounds kinda like uni ¯\_(ツ)_/¯

Our owner's other company sells products online (or has the ability to anyways). Their current site is 7+ years old WordPress/Woocommerce and is seriously outdated because the site breaks if you update anything so we've been told to make a new site (finally). They also said they were going to release a whole new line up of products. So the first thing I tried to do was get them to nail down their product line and how shipping was going to be configured. I was told to just use the shipping from the previous site.

Turns out those shipping rates don't use any sort of math or automation at all, there is literally a manually set shipping value for every single product for every single shipping location (30*60) and even values for different quantities. And there's no way to export these rates into a readable table because the plugins they use shove all the data into the postmeta table, I'm forced to go through and put the data into a spreadsheet so that I can attempt to organize it and hopefully find someone way to automate it. Owner claims at one point that he has a similar spreadsheet that's more up to date but for some reason refuses to send it over or put me in touch with the right people in the shipping department.

I've gone through the shipping rates with the old products and the new products and organized them as best I can and each time I've gotten done and shown them the spreadsheet with their products and shipping, they add or change something which requires me to basically wipe the slate clean and start over eating another 50 or so hours of my time, which with everything else really means another month+ to find time to work on it between other projects.

After about a year they finished their products and I finally finished the planning and got approval to build it out for the site. Small victory!!

After about 60 hours plugging these values into the database (only about 1/3 done) I get an email from their head of shipping who tells me the values in my spreadsheet are "terribly inaccurate, in some areas by $100+" and that the data should not be used anywhere.

So after something like a year and a half and 200+ hours of work, the data I've been using to plan all this isn't even accurate. I'm trying not to go crazy here but this kind of shit is unacceptable. When we're done with this I'm going to send the owner an invoice to show him how much money he wasted on this because nothing was planned and he just wanted it built. There's a fucking process for a reason, when you don't follow the process you fuck everything up. If a client had pulled this shit and turned their simple site into this much work they would have been dropped. I get constant emails asking when the new site will be done and every time my answer is "I'm still waiting for x items that I asked for last time you asked where we were." He gets a couple things on the list and sends them back and then goes unresponsive for weeks at a time.

Management has been telling me that I seem more stressed lately but only one of them understands what's going on here when I explain it. The rest say stupid shit like "why don't you automate it" or "make an intern do it." You won't let me hire an intern and even if I did, I'm not sure I could explain how the shipping works now to even trust someone else to do it. I'm hoping when the shipping guy gives me the new sheet that maybe there's some easier solution here because I'm ready to start shooting people.

Finally dropped our old useless in-house legacy content editor (built by another team) today. Going through the database and ripping out all the shit that we had to keep for legacy support is the most liberating feeling I've had in weeks!

Fucking Ruby.

Installed my new job's codebase on my machine and it's fucked everything.

While trying to get the database working, someone's dropped my User table, so I can't log in as 'Josh' anymore.

Now I can't compile scss assets without a fucking gem error.

I'M IN A PYTHON ENVIRONMENT, FUCK OFF!

GRR.

When you work on a project for months, the newly appointed tech lead says "we all (him, the boss and other colleagues not involved in this project) looked at your code and decided that it should be dropped and we are starting from scratch again....now I'm not against code review (which we never did) so I welcome the input but allow me to vent my frustrations about how this is being done. Also to have a review & verdict without me being present?

So I ask what was so wrong:
* You changed the database structure. Valid, I tried to make your db an actual db with relationships, so I added some foreign keys, delete fields that were never used, all because they told me to use an ORM.
* You used to much logic in setters, validation etc, valid again but this would be something we could look at and fix imho.
* You are passing classes in your constructor, valid I wanted to use DI to make unit testing possible. Ohh but I don't like unit testing so I don't see the point and it makes it to complicated was the response.

So not only was the project cancelled, the new iteration is being developed without me, I'm shunned from all meetings. Ohh and from what I see they are now using 5 tables instead of 25 and completely started the db model from scratch...