I jump on an existing scala project.

git pull && sbt compile test

Tests are failing.

Me: "Hey team, the tests are failing."

Team member: "That cannot be. They were passing for the the last run."

Me: "Did you run them locally?"

Team member: "No, on Jenkins. It was fine."

I check Jenkins.

Me: "What do you mean it's fine. The last successful deployment was on the end of May."

Team member: "The Pull Request checker always went through successfully."

I check how our Jenkins tasks are configured. It's true that the Pull Request Checker runs successfully yet due to a "minor misconfiguration" (aka "major fuckup") the Pull Request Checker only tests a tiny subset of the entire test suite.

Team members were were fine if their Pull Request got the "Success" notification on bitbucket's pull request page. And reviewers trusted that icon as well.

They never checked the master run of the Jenkins task. Where the tests were also failing for over a month.

I'm also highely confused how they did TDD. You know, writing a test first, making it green. (I hope they were just one specific test at a time assuming the others were green. The cynic in me assumes they outsourced running the tests to the Jenkins.)

Gnarf!

Team member having run the tests locally finally realizes: "The tests are broken. Gonna fix them."

Wow. Please, dear fellow developers: It does not kill you to run the entire test suite locally. Just do it. Treat the external test runners as a safety net. Yet always run the test suite locally first.