I’m debugging some intermittently failing specs.

The problem is, they only fail when RUNNING THE ENTIRE FUCKING FILE, which, naturally, takes TEN FUCKING YEARS.

And it’s not like they fail when they get run first. No. They only fail when run near the end.


  • 8

    *Oh fuck...*
  • 1
    Is there some shared state that could be modified between tests that put the state of the application in a different one than expected? Is there a connection being closed that should stay open? An IO block?
  • 1
    I uh...

    Man. This sounds tough. Keep pushing though!
  • 2
    @AmyShackles @OmerFlame I resolved it yesterday. The cause ended up being pretty simple: a constant that kept changing.

    But the bloody long run time! Ughhh
  • 1
    @Root somehow sounds like JS...
  • 4

    A constant that keep changing.... wtf.
  • 2
    @N00bPancakes @100110111
    I know right 😂

    But they’re specs. Changing things around to test edge cases is pretty normal.
  • 2
    @Root how the fuck did the CONSTANT change? Isn’t it, like, supposed to be unchangeable?
  • 5
    @OmerFlame Depends on the language.

    I’m feeling chatty, so. Let’s go!

    In some languages, constants aren’t constant. They’re just sugar, and it’s dumb. In Ruby, it’s a bit of both? You can redefine them, but (usually) not directly. It takes some effort, but it’s easier in specs for whatever reason. Also, changing their values almost always generates a warning. (I can think of one instance where it doesn’t — black `eval` magic)

    In this case the constant is a whitelist. Most of the specs could be written to work without changing the whitelist, but that’s how they are 🤷🏻‍♀️. Some, like checking the error handling around when the whitelist itself is empty, require changing the constant.

    The issue was that some of the tests were mutating the whitelist without restoring it afterwards. Slow to find, easy to fix. I also changed the new specs to set the whitelist to what they expect prior to running so they won’t be intermittent again.

    Done and done.
  • 2

    1. I like it when you’re chatty, I love talking to people about weird programming :D

    2. What do you mean by “sugar”?
  • 3
    @OmerFlame As in “syntactic sugar” — variations in syntax that makes the language more pleasant to use or increases readability, but (usually) doesn’t confer any additional functionality.

    if x
    if y
    `foo if x`
    `bar If y`

    Functionally identical, but the latter can make things easier to read. (Pardon if that’s super obvious; just wanted to be clear! 😊)

    I also refer to wrappers as “sugar” because they make things easier and more pleasant, but they usually also confer some additional functionality like error handling.

    In Ruby’s case, constants do add some functionality in the form of exceptions/warnings when changing them, and the interpreter knows it’s a constant simply because of their case. But in the end, they are just vars like any other. That’s why i called them a bit of both.
  • 2

    Shame that devRant doesn’t have code highlighting...
  • 2
    @Root these sort of steo by steo break downs of bug hunts are so enjoyable to read.
  • 2
  • 1
    @Wisecrack I’ve got another for you:

    Smarter updating of Apple and Google wallets. They only update when specific columns change, and merchants can define which ones they care about.

    The code works perfectly on Apple wallets, but not Google wallets. for google, they always update, no matter what.

    The kicker?
    They both use the exact same code. I had three other engineers look through the code, and everything looks solid.

    Works on my machine. Works in specs. I wrote specs specifically for the misbehavior and related cases, and they all pass. Doesn’t work on staging or any QA/test environment, or on the prod clone with test data. We’re all stumped.
  • 1

    That issue...
  • 2
    @N00bPancakes @Wisecrack
    The cause is in some abstracted magic that compares current and previous values to see if a virtual column has changed. The previous values are never getting populated (or maybe persisted). I’ll know more when I delve into it.

    Turns out QA missed this edge case on Apple wallets, and it does in fact happen with both. 🤦🏻‍♀️
Add Comment