I hired a woman for senior quality assurance two weeks ago. Impressive resume, great interview, but I was met with some pseudo-sexist puzzled looks in the dev team.

Meeting today. Boss: "Why is the database cluster not working properly?"

Team devs: "We've tried diagnosing the problem, but we can't really find it. It keeps being under high load."

New QA: "It might have something to do with the way you developers write queries".

She pulls up a bunch of code examples with dozens of joins and orderings on unindexed columns, explains that you shouldn't call queries from within looping constructs, that it's smart to limit the data with constraints and aggregations, hints at where to actually place indexes, how not to drag the whole DB to the frontend and process it in VueJS, etc...

New QA: "I've already put the tasks for refactoring the queries in Asana"

I'm grinning, because finally... finally I'm not alone in my crusade anymore.

Boss: "Yeah but that's just that code quality nonsense Bittersweet always keeps nagging about. Why is the database not working? Can't we just add more thingies to the cluster? That would be easier than rewriting the code, right?"

Dev team: "Yes... yes. We could try a few more of these aws rds db.m4.10xlarge thingies. That will solve it."

QA looks pissed off, stands up: "No. These queries... they touch the database in so many places, and so violently, that it has to go to therapy. That's why it's down. It just can't take the abuse anymore. You could add more little brothers and sisters to the equation, but damn that would be cruel right? Not to mention that therapy isn't exactly cheap!"

Dev team looks annoyed at me. My boss looks even more annoyed at me. "You hired this one?"

I keep grinning, and I nod.

"I might have offered her a permanent contract"

  • 155
    haba, that's freaking awesome. Great that you found quallified personal :)
    And I especially liked her analogy.
  • 92
    Hah! Nice one!

    I hate those fake developers that do not stick to the best practices and write poor quality code.

    Lately, as education became easier and more popular we got much more of these people.
    People that have no natural talent nor passion towards programming, but do it because it yields high salaries, considered cool and is easier than physical labor.
  • 27
    You made a good choice, apparently. :D
  • 84
    @Noob @plusgut

    With 40 devs who get a bit too giddy about the magic of Laravel + Vue, I just couldn't fight the optimization fight alone anymore.

    She referred to Rich Hickey (creator of Clojure) during her interview about how developers confuse "easiness" with "simplicity", how simplicity is the opposite of the complex mess we encounter when devs promote their desire to keep things easy.

    And that is the problem: Software development was hard and time consuming, with meticulously designed systems. The counter movement brought us awesome tools for agile development, for quickly pooping out working prototypes, but we've lost important skills.

    The love for optimization, the art of reducing a HTTP request to a minimum amount of bytes, the drive to put some design into data models... we have sacrificed that in favor of obfuscating bloated frameworks and inefficient ORMs.

    Super useful if you are building a WordPress clone, frustrating when you create a 10-million-user platform.
  • 33
    @bittersweet your employer is lucky to have you two.
    I completly agree with everything you just said.
  • 19
    @bittersweet 100% true! I keep repeating this every day - easy is not simple!
  • 7
    great story :-)
  • 47
    @bittersweet... You know you have to get her on DevRant now right?

    She sounds like our kind of people.
  • 7
    Love this rant that I actually laughed too hard.
  • 14
    Wait, either a QA with that level of DB skill is a goddam miracle in any gender, or all the QA's I've been working with have been severally under representing the position
  • 6
    Well... m gonna bow down to her highness 😎😎✌🏻✌🏻
    Amazing analogy too 😂😂😂😂😂
  • 4
    @AlexDeLarge word. 64bit word.
  • 6
    In the end, those changes she and you are making are gonna save so much time and headaches for everyone. Good job!
  • 2
    The poor person can get ganged up by your team. Feel sorry for her.
  • 27
    Nah, she's from eastern Ukraine, fled from war. More worried about the team than about her.

    Maybe if you read the original post with the a slavic accent of a 45 year old woman who could bench a fully loaded server rack and knows how to defuse landmines, it makes sense why my boss quickly agreed to the refactoring tasks.
  • 7
    she sounds awesome get her on devRant nooooow
  • 2
    "You shouldn't call queries from within looping structures..." ok, why not? And surely there's plenty of situations where you can't get around it anyway
  • 2
    She's awesome. I love the therepy analogy. Lol
  • 9
    @Inigo call your query and dump it into an array or collection then loop through that. Saves a ton of overhead. Sometimes you can't avoid it but almost always you can.
  • 6
    @Inigo The problem with many ORMs is that they make it too easy to do dirty things like

    foreach (top10users as user) {
    count += user->getComments()->count()

    Which first queries the user table once, and then does 10 roundtrips to the DB for each user's comments, and counts them on the webserver instead of letting the DB do the heavy lifting.

    Ideally, you should always get the most concise answer from the DB in one single query.

    In practice, it's often easier to write multiple queries for the "sub questions" you're asking.

    But the amount of queries needed for a pageview should be constant, not increase with the amount of objects you're retrieving.
  • 2
    I see what you're saying. I guess I'm used to using frameworks where you just don't need to worry about that; I'm think I'm right in saying that Laravel takes care of all that for you behind the scenes
  • 8
    @Inigo The problem is, Laravel really doesn't. It obfuscates a lot through "magic", but it doesn't always do what you'd expect it does. Most people discover this when their application scales up.

    Use barryvdh/debugbar, and keep an eye on the query count.
  • 4
    @bittersweet that's interesting. Thanks, I'll check it out- and do some reading on this. Almost as good as Stackoverflow, this place :D
  • 2
    @Inigo Laracasts, both the paid videos and the forum, can be helpful too at times.

    This optimization problem is commonly called the "n+1" problem, and occurs often when people create repeated view elements (for example table rows) which call methods on the model. But it's also easy to accidentally put a get() in a collection map/filter. Eager loading using with/withCount/load offers a solution, although sometimes you just need to do a raw query.
  • 2
    @bittersweet So this applies to eloquent models as well? For example:

    foreach($entrylist as $entry){

    This will produce as many queries as there are keys in the $entrylist array?

    I thought this kind of thing was pretty standard/accepted practice. I also thought I'd read somewhere that eloquent optimised this automatically in terms of interacting with the database- but perhaps I'm wrong on that.
  • 3
    @Inigo Yeah that wouldn't be optimal. Works fine if there's only a few ids, but can become a problem if the list becomes longer.

    Luckily, you can pass an array to find, or use where:

    Entry::find(collect($entrylist)->pluck('id)) or Entry::whereIn('id', $ids)->get().

    ->toSql() is very useful for checking how the query is constructed. There's also DB::listen(), which can dump all queries, bindings and latencies.
  • 4
    @bittersweet Very good to know! Thanks! Will do some experimentation when I get the chance
  • 3
    She jsut owned the room. I raise my coffee for that level of expertise ☕
  • 1
    @bittersweet please, for the love of H. P. Lovecraft, tell me that querying the db within loops is a foul practice of only few ignoramus 😯
  • 3

    No, the N+1 problem is very common, especially in active record ORMs like those used in Ruby on Rails and Laravel.

    The idea is that you don't want to burden a pure backend developer with knowledge about databases and SQL, so the language/framework wraps it in very convenient methods.

    This is often so magical and opaque that users lose sight of what happens in the background. Just accessing a property on an object can trigger a select query.

    For example, in Ruby you wouldn't directly see that this causes a query on the authors table during each loop iteration:

    @articles.each do |article|
    Title: <%= article.title %>
    Author:<%= article.author.name %>

    So it's not really the developer's fault, but it's still the cause of most scaling problems in RoR and Laravel.

    I feel that by trying to make things easy, they forgot to keep them simple.
  • 1
    I see. Haven't used RoR or any AR ORM in ages. My primary ORM in PHP land has been Doctrine; not sure if it makes any difference in performance with your example, as it can be configured to load "properties" lazily.
  • 1
    @mathewrapid Doctrine has the same problem when used naively, but it's less prevalent.

    When using repositories, most developers snap into the mindset where they write very specific raw-ish queries using doctrine. Doctrine is a bit closer to the SQL underneath, where Eloquent adds many more abstraction layers on top of it.
  • 7
    @illusion466 Maybe I've worded it a bit weirdly.

    She wasn't treated unfairly, but there were a lot of crude remarks about the fact that I hired a woman for a senior dev position, things like "are you sure she isn't just a designer" to "I doubt she has the analytical skills required for the job". Those might be valid questions to ask about a new developer, but not if you only know the person from shaking their hand after an introduction round.

    So during the first meeting she attended, I was happy to see that she slapped some sense into the team — both regarding the quality of their code, and that it's not impossible for a woman to be a good developer.

    I'm not in favor of diversity quota, or minority coder clubs, or artificially cheering at women to "empower" them.

    But recent events did make me realize that a lot of people are still prejudiced, and it's good if those assumptions are challenged a bit. I didn't hire her for that task, but I'm nevertheless happy about it.
  • 4
    I'm sad to see that a woman in our field (and in so many other fields) has prove herself, more than any man, before taken seriously.
  • 1
    @Noob mmh I agree with everything but is easier that physical labor? If you mean that you sweat less (maybe) yeah but easier? My grandpa taught me how to chop wood in 2 days... Our job instead, a degree, almost 10 years on the field and I am still learning!
  • 1
    @donnico of course, our job is much more complicated and difficult to learn.
    But it is much easier than exhausting physical labor every day.
  • 1
  • 0
    Damn... I am soo speechless for this post.

    Btw anyone have good article reference for optimization?
  • 2

    Written all cap to demo the frustration I am feeling when I suggest similar thing at work :(
  • 2
    @gitpush Me too! I'll take one. We need a developer with a rare skill set that only maybe 200 people have. Should be an interesting applicant search.
  • 0
    @vlatkozelka exactly mate. Exactly.
  • 1
    @vlatkozelka I think the point is balancing short and long term.

    A product owner wants new features, fast, and once they're done, they're done. No matter how hacky the implementation.

    As a developer, you need to keep in mind that the business needs to earn money to pay your salary, so delivering fast can be important. But if you cheat too much in the short term, you might run into enormous walls later.

    Many developers love to openly fantasize about awesome features in front of stakeholders ("Oh we could build smart recommendations using neutral networks here") and underestimate work ("I guess it took a bit more than one afternoon").

    So you always need to break up work into tiny bits, communicate often with stakeholders, and plan ample time for refactoring.

    Adding a button to a webpages is NOT "done in five minutes". It sounds so simple, but you might need translations for the label, it needs to look good on all devices, you might add styling for state changes, write tests...
  • 0
    @vlatkozelka I think as devs we're pleasers, we're often too used to saying "of course we can build that, it sounds like an interesting technical challenge" instead of "if we build that, it will cost time and maintenance, and it adds incredible amounts of unnecessary complexity to our service, just to please one of our clients"

    As a dev nerd I often feel disappointed when I shoot down an interesting complex "puzzle", but if I don't bring up counter points I shoot myself in the foot, because I also know deep down that it's unneeded bloat with a very real maintenance cost.
  • 1
    Making code reviews fun again!
  • 1
    As a person in scientific high performance computing, your love for optimization makes me very happy.
  • 1
    *revives thread*
    Yep, those are the kinds of people I want to work with in the future!
Add Comment