Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "mutation testing"
-
Has anyone else actually *used* mutation testing at all?
Heard a lot about it recently - it seems all the rage amongst the bloggers, but I'm generally always very sceptical of things touted as the "latest hotness" (my thoughts on blockchain for instance are well known.)
So I went ahead and whacked http://pitest.org/ into one of my more recent pet projects to see if it offered anything decent. Surprisingly, it did - in particular it caught a number of places where switching "<" for "<=" and similar had no affect on the pass / fail rate (indicating the tests should be better.) There were a *few* false positives, and some which were borderline useful, but as a whole I'd say it was a worthwhile addition.
Curious as to if anyone else has had the same experience?1 -
Our pipeline had been running for 45 minutes, I went to investigate
Our mutation testing framework had identified 90 potential mutants and started running through our 342 tests for every single mutant, so 90 * 342 tests
And I had been complaining about the amount of pipeline agents being too low and pipelines blocking up...5 -
I wish everyone would move away from code coverage as a metric and towards some kind of mutation framework.
There seem to be increasing numbers of devs getting themselves off on their "shiny 95% coverage" and patting themselves on the back for covering everything but the 5% of the codebase that actually needs thorough testing.
Oh, and that's ignoring the tests that just assert an exception isn't thrown, or don't assert anything at all. Completely bloody useless, but hey, you just carry on boasting how great all your tests are because you've got a higher coverage than the team next door 😤🙄3