Learn More
Resend Email
36
Root
6y

Summary: Burnout, and everything's broken.

I don't feel like doing a damn thing today. I look at the code and cringe. I look at Slack and think "ugh. i can't." Mental capitals are even too much work.

(I've started reading "Zen and the Art of Motorcycle Maintenance" to try and combat burnout. I'll write a rant/story about it here if I find it helpful. but all I want to do today is drink tea and read.)

But onto the story:

Heroku is deprecating support for and will automatically upgrade any old verisons of Postgres running on its platform after August something (like five days from now).

I performed the upgrade to PG10 on Sunday (and late into the night), provisioning a new follower, blah blah blah.

However, the version of Rails we're using (4.2.x) doesn't support PG10 sequences, so I manually added in support via a monkeypatch. I did this on our QA servers first, obviously, and everything worked as expected. After half a day of no issues, I did the same on production, and again: everything worked as expected.

But today? I keep hearing about new things that are broken. One specific type of alert doesn't work for one specific person (wat). Can't send [redacted] at all. Can't update merchants! Yet there are magically no errors logged.

That last one (well, two) are just great; let me explain: when there's an error concerning merchants, the error gets caught, isn't logged or recorded anywhere so it just disappears, and the rescue block triggers a json response instead and happily exits. This is for an internal admin tool, so returning a user-friendly error is kinda stupid anyway, but masking what actually happened? fuck that dev with an obelisk made from spikes and solidified pain. That json response is also lovely: it's a 200 OK returning {status: 1, data: "[generic message containing incorrect IT jargon]"}. Doesn't even say "error" anywhere. Bloody everything about this pattern is absolutely wrong. Even the friggin' text.

Fucking hell. I want to pipe the entire codebase into shred and walk out the door.

But I digress. So many things are broken, my motivation is wanning to a sliver, and I have a conference call today where I'll undoubtedly be asked why everything is on smoking and/or on fire, and my huge and overly productive week last week will ofc mean nothing by contrast.

Ugh.

`shred ~/dev/work -zfu -n 32 &; ./brew tea --hot && wine ~/takeabreak.exe`

rant

zen and the art of motorcycle maintenance

postgres

heroku

ship's sinking and the fixer's all fixed out

burnout
Favorite
Ranter
Comments
  • 3
    6y
    I'd ask to take a little break. Even when things are on fire, you can't be the only one who can extinguish it, right? Someone else can take care of it I guess. By the way, can you roll back to the previous version of Postgres? I mean, any decent hoster should at least give control over and allow grace time even on managed solutions, right. If they mindlessly upgrade stuff without taking their customers into account, they aren't a good hoster and I'd use that occasion to migrate to somewhere else that does allow its customers to work at their own schedule and to do whatever the fuck they want on their instances to begin with.
  • 2
    6y
    @Condor

    No, nobody else can fix it. I am literally the only person in the company who can work on the codebase and servers. We have a contractor dev, but he uses almost exclusively .NET and Angular1, and has never touched any of the rest of the stack (Ruby/Rails, Postgres, Heroku, React, ...). It's just me.

    Also: Heroku does perform automated upgrades, but only after a pretty generous window (months). I've had so much on my plate that I sort of forgot about it until last week. I can't rollback the upgrade because it's an entirely different server. I could, however, downgrade the same way I upgraded it. (and if I can't figure out the issues soon, I may have to.)
  • 4
    6y
    You might find yourself asking: "Why not just upgrade the version of Rails to something that supports PostgreSQL v10?"

    That's a lovely question!

    Have a nightmare in answer: A test suite that hasn't been touched in 3-4 years. Three Rails applications running slightly different versions. Upgrading Rails, even minor version bumps, breaks random features, and because of the aforementioned test suite it's impossible to tell which or how many. Likewise with dependency updates.

    `shred` is the only justifiable answer.
  • 3
    6y
    @Root In that case, I'd also shred the whole codebase (or if possible gradually replace components) and start anew. Convoluted mess in a product is never a good thing. Either way, your health is more important than any product you're in charge of. Burn-outs can be far more damaging than a broken feature.. so do take a rest until you're able to get back your motivation :)
    Oh and complain to management/HR about the lack of redundancy in developers who can actually maintain that tech stack.
  • 1
    6y
    Is it possible to migrate to other platform that allows the version of postgres that you need?
  • 2
    6y
    @FrodoSwaggins Best comment is best comment.
  • 1
    6y
    @shellbug Heroku might allow me to remain with 9.6 -- I'm unsure. There's conflicting official documentation.

    I do worry about the future, however, if they decide to force everyone to upgrade to e.g. v11.
  • 2
    6y
    @EvilArcher That is an insane amount of work.

    @Condor I could host my own Postgres database on AWS and point my env vars to that instead. I'd miss out on Heroku features like automated backups, maintenance, etc. but I could use any DB I wanted.
  • 2
    6y
    @Root That'd be a lot better IMO. Backups can easily be done via an rsync cron job.. perhaps tar for compressing them, but I'm not too fond of that to be honest. Just put one or 2 backup servers in service (perhaps coupled with some redundancy by making them live as well) and you're golden. That's how I'm currently maintaining my mailers as well (where I've got only a SSH shell, because even my hoster's web shell doesn't work :') so yeah).
  • 2
    6y
    @Condor I'll consider doing this (and call it "life support") if Heroku forces PG11 on me.
  • 2
    6y
    Please do let us know how you find that book. Its been highly recommended to me by a dear friend.
  • 1
    6y
    @Braed Will do.
  • 1
    6y
    @MrJimmy I don't feel burnout while drunk, so drunk I was. Rum! 🧡

    It increased my productivity from a probable zero to a slightly-higher-than-zero. Fixed some issues in production at least.

    Also, I can't read the book in a day lol
  • 0
    6y
    @Root here's some music to go along with that rum:

    https://youtu.be/f55CqLc6IR0
  • 0
    6y
    Wait where did you get the info that you'll be forced to update to Postgre 10 on Heroku?

    Couldn't find that Postgres 9.6 is deprecated on Heroku anywhere, and didn't receive any mail about that either.
  • 0
    6y
    @Fradow I dont think Heroku deprecated 9.6, but we were using 9.3. As I recall (and I'm quite certain) heroku was to automatically upgrade it to 10 this Friday or so; I don't have access to the original email right now or I'd look.
  • 0
    6y
    @Root Thanks for the explanation, you almost gave me a heart attack.

    Why not just migrate to 9.6 instead of doing 10 right now? Unless you have a very large DB, it's pretty painless and can be done in under 15 min of following the doc (create a new DB, copy from old to new, point your env to new DB, delete old DB).
  • 0
    6y
    @Fradow I was under the impression PG10 was a mandatory upgrade. And no, it didn't take too long. 20 min for QA, ~45 for production.
  • 1
    6y
    @Root it can't be mandatory if you don't use that particular DB anymore :)

    I just checked you can still provision Postgre 9.6 (with CLI), though 10 is now the default.

    Heh, you'll need to bite the bullet at some point, and so do I, about 1 hour is actually very little time for that.
  • 1
    6y
    Some of the bugs fixed themselves today.
    That's not worrying or anything.
  • 0
    6y
    @Nanos Yeah. Me too until I started reading it. If you take that book as a guide to motorcycle maintenance, though, you'll end up with something that looks vaguely like a motorcycle and sounds vaguely like Niagara Falls.
Related Rants
devRant © 2021 Hexical Labs LLC
Privacy Policy  |  Terms of Service
Add Comment