Interviewer: (asks technical question)
Me: (answers correctly)
Interviewer: Oh thank God, most of the people we interview fail to answer that. So, on another note do you believe in rubber duck debugging?
Me: yes, ofc
Interviewer: but it's just a toy you know
[I was kinda taken aback]
Me: ...
Me: God is imaginary too you know
Interviewer: (he just laughed)

(So I laughed with him) 😅

(Written March 13th at 2am.)

This morning (yesterday), my computer decided not to boot again: it halts on "cannot find firmware rtl-whatever" every time. (it has booted just fine several times since removing the firmware.) I've had quite the ordeal today trying to fix it, and every freaking step along the way has thrown errors and/or required workarounds and a lot of research.

Let's make a list of everything that went wrong!

1) Live CD: 2yo had been playing with it, and lost it. Not easy to find, and super smudgy.

2) Unencrypt volume: Dolphin reports errors when decrypting the volume. Research reveals the Live CD doesn't incude the cryptsetup packages. First attempts at installing them mysteriously fail.

3) Break for Lunch: automatic powersaving features turned off the displays, and also killed my session.

4) Live CD redux: 25min phonecall from work! yay, more things added to my six-month backlog.

5) Mount encrypted volume: Dolphin doesn't know how, and neither do I. Research ensues. Missing LVM2 package; lvmetad connection failure ad nauseam; had to look up commands to unlock, clone, open, and mount encrypted Luks volume, and how to perform these actions on Debian instead of Ubuntu/Kali. This group of steps took four hours.

6) Chroot into mounted volume group: No DNS! Research reveals how to share the host's resolv with the chroot.

7) `# apt install firmware-realtek`: /boot/initrd.img does not exist. Cannot update.

8) Find and mount /boot, then reinstall firmware: Apt cannot write to its log (minor), listed three install warnings, and initially refused to write to /boot/initrd.img-[...]

9) Reboot!: Volume group not found. Cannot process volume group. Dropping to a shell! oh no..

(Not listed: much research, many repeated attempts with various changes.)

At this point it's been 9 hours. I'm exhausted and frustrated and running out of ideas, so I ask @perfectasshole for help.
He walks me through some debugging steps (most of which i've already done), and we both get frustrated because everything looks correct but isn't working.

10) Thirteenth coming of the Live CD: `update-initramfs -u` within chroot throws warnings about /etc/crypttab and fsck, but everything looks fine with both. Still won't boot. Editing grub config manually to use the new volume group name likewise produces no boots. Nothing is making sense.

11) Rename volume group: doubles -'s for whatever reason; Rebooting gives the same dreaded "dropping to a shell" result.

A huge thank-you to @perfectasshole for spending three hours fighting with this issue with me! I finally fixed it about half an hour after he went to bed.

After renaming the volume group to what it was originally, one of the three recovery modes managed to actually boot and load the volume. From there I was able to run `update-initramfs -u` from the system proper (which completed without issue) and was able to boot normally thereafter.

I've run updates and rebooted twice now.
After twelve+ hours... yay, I have my Debian back!

oof.

Rant:
I start a phone project and get an error during debugging
I look up the error and learn that its a visual studio issue
I try uninstalling visual studio and fail
I try reinstalling visual studio and fail
I format the machine and reinstall everything
I run the project and get the exact same error
I later learn that it was not an error but a system message unrelated to the problem

I’m debugging some intermittently failing specs.

The problem is, they only fail when RUNNING THE ENTIRE FUCKING FILE, which, naturally, takes TEN FUCKING YEARS.

And it’s not like they fail when they get run first. No. They only fail when run near the end.

ASDFHSGSKSHDHDKAJDKF

I got a dayjob in a company. I got an error. I cannot solve it and I am so desperate. So I go to stackoverflow, nobody answers. I post on git issue, but nobody solves the problem. So, I pay someone to solve it, like Hackhands.com to find a mentor. There is no mentor that can help. So I pay more, hired a peer, and finally a development team just to help me. They get paid only if they solved it.

But each of my folks repeat my same steps, asking on stackoverflows or github, and none of these help. So, they end up hiring their own friends and mentors. Their friends also end up paying (pay before problem solved) someone to help them.

their friends pay for friends of friends, then friends of friends of friends

And all of a sudden it becomes a giant MLM scheme.

And those people they paid for actually work for a company behind the scene which I am a founder of 😁

Multi billions startup idea, is it?

Searching for random Linux bug.. finds Gist with exact match (thinking, WOW, thanks Google!)..

It's a 2000+ line log dump. : |

The best way by far to loop a method while debugging......

Debugging a java program for hours only to find that the mistake was a semicolon just after the for loop declaration FML

Its amazing how many people call themselves programmers yet cannot figure out the usefulness of print/log statements. 🤦‍♂️

Debugging your JavaScript Code for 2 hours and trying out every imaginable solution because it works on the demo page but not your site... and then you find out the fking library had a bug and the Readme.md contained a cdn link to an old version and the demo page was up to date... arrgghh

** me setting up GitLab CI **

- run pipeline
- FAIL
- env variable not passed to one of the shell scripts

- set -x, rerun
- FAIL
- same reason. env variable is OK in the `set -x` output

- comment out `set -x`, rerun
- still FAIL
- same reason

- find a `set +x` left in one of the scripts
- comment that out
- rerun
- PASS

- WTF?!?!?!?
- continue on swearing for wasted better half of the day debugging my scripts

Stupid pipeline bullshit.

Yeah i get it, it speeds up development/deployment time, but debugging this shit with secret variables/generated config and only viewable inside kubernetes after everything has been entered into the helm charts through Key Vaults in the pipeline just to see the docker image fail with "no such file found" or similar errors...

This means, a new commit, a new commit message, waiting for the docker build and push to finish, waiting for the release pipeline to trigger, a new helm chart release, waiting for kubernetes deployment and taking a look at the logs...

And another error which shouldn't happen.

Docker, fixes "it runs on my machine"
Kubernetes, fixes "it runs on my docker image"
Helm, fixes "it runs in my kubernetes cluster"

Why is this stuff always so unnecessarily hard to debug?!

I sure hope the devs appreciate my struggle with this... well guess what, they won't.

Anyways, weekend is near and my last day in this company is only four months away.

I'm working on a codebase that is terminally ill. It's split so badly into microservices that no matter what you do, every one of them talks to every one of them over and over. If there's any way they can avoid just invoking a method on a class and send themselves a message or make an HTTP request, they'll do it. One of the services just sends messages to itself for no apparent reason. Except it doesn't even send messages to itself. It sends an HTTP request to a controller in another app, and that controller sends a message which is received by the same class that made the request.

The point is that this application is screwed. The defects pile up and there is literally no one who can understand what it's supposed to do in any scenario. I'm good at this. I can follow confusing code and document it. But not this one. It's overwhelming. It's insanity.

When these defects come in we're told to just run the app from the UI, see what HTTP requests it makes, and start tracing the code manually. Running and debugging it locally would be a nightmare but it's impossible anyway.

They decided that we all need to understand the application better so we can work on it, so we were each given six poorly-define five-hour tasks to "understand" various things. Those things don't make any sense. It's like if someone gave you the source code to Excel and told you to spent five hours understanding columns, five more understanding rows, and five more understanding cells.

Here's the thing: I'm okay with learning and understanding some code. It's part of the job. But I'm not going to abandon my career as a software developer so I can become an expert on debugging their awful code. I didn't make this mess. I'm not going to live with it. I'm moving on as quickly as I possibly can.

I've tried to explain to them that if they want the situation to improve they need to improve the code. They need to learn how to write tests. If your plan is that people will study your code, know it inside and out, and then spend all their time debugging it, that's a plan for failure. Everyone who can will leave and take what they know with them.

These companies just don't get it. They need their software to work, but the types of developers who can help them don't need that software to work. No one capable of doing good work is going to spend several years debugging their awful code unless you pay them a crazy ton of money.

Just don't make a mess in the first place. Hire developers who can do a good job. If you hire the cheapest people you can find you won't be able to get someone else to fix it later. It's not personal but I wish failure on those projects or even those companies. I want them to fail because failure is so expensive. I want them to fail so that others learn from it and don't repeat the same mistakes.

As an industry we're a bunch of genuine idiots. We just keep doing the same things over and over again no matter how much it hurts.

Stop commenting out code blocks!

Either fix your shit or delete it.

I am open to argue what fixing may mean, as it is perfectly fine to make your broken code not reachable, e.g. via feature flags or skipping certain tests. Yet never ever should you comment those blocks!

So you say you want to keep it for historic reasons? You know, that is why we use version control! If you ever need certain functionality back, you can restore that state.

Each decent IDE also offers a local history where you can even restore code blocks that weren't even pushed or committed. So use that!

Commenting out test cases is a really bad habit, as you have no reminder that you shall restore it.

And no, a TODO and a FIXME won't count as a reminder as you have to actively look for them. And we all know how well that goes, don't we? (One time, I found a typo of a `TDO`. So even with a regular lookup for TODO, stuff will slip.)

Each test suite offers you ways to skip tests if there are valid reasons why they should not fail the build temporary and they offer colorful feedback. Yes, that means that your tests won't be green, but guess what: That's a feature! They shouldn't be.

That yellow is a fine reminder, aka warning!, that you should really fix your shit.

Commented code screams: "I DON'T KNOW WHAT I WAS DOING!" and it confuses the hell out of other developers ("Was this commented because of debugging purposes and should be active again or can I safely delete this!?") and adds verbose crap to the code base.

If you find yourself to be in a place that you comment code a lot, I also argue that your workflow is broken.

When you are using a decent debugger, there shouldn't that much of a need to comment in and out a lot of code in order to reason about your code-base.

Spend hours debugging a python script because tests fail. Turns out ftp (the unix binary) adds CR bytes to every byte in a .gz file that looks like a LF on upload (in test setup). Client and server are both linux. 😭

Yes i know switch to binary mode (that fixed it - why not default??). But still WHY CR? Didn't ask for it. No windows in sight.

Lead developer likes premature optimization. Always forces me to remove a if here and there because "bad for the performance".

Now her optimization caused a base component to fail, ppl will have to spend the night debugging.

Best of all, it's my last day ;)

Why the fuck does the Execute Process Task from SSIS in Visual Studio fail when trying to use variables in an expression?!?

I've been debugging this shit for hours and have made absolutely no progress. There's no apparent workaround.

Fuck you Microsoft, for leaving a known bug in VS for over a decade, where the expressions are surrounded in double quotes, negating the entire purpose of using an expression for variables!!! 😡

Spring batch is saying Job Execution failed... But all the steps completed without errors.

So why did it fail?

Googled all afternoon but can't find anything... And I don't usually use Batch.... Just debugging another issue...