I was supporting a legacy CRM app which front end used Visual Basic 6 and almost the entire business logic was written on SQL store procedures.

A "feature" of the product was the open code, anyone with admin access could modify forms, code and store procedures.

We also sold "official" (and expensive) consulting services to modify the code.

A long time customer owned this thing and it was heavily customized. They had hired us to change something, hired a third party to make other changes and decided to modify some stuff themselves because, why not?

Suddenly they came to product support asking to fix a bug. The problem happened on a non customized form.

After reviewing, I realized the form used several of the modified store procedures in the business layer. I tried saying we don't support custom code but my boss was being pushed and said "look into it"

All 3 parties denied responsibility and said their changes were NOT the problem (of course). Neither of them commented or documented their changes.

The customer started to threaten to sue us.

I spent 5 full days following every field on the form through the nested and recurrent SQL store procedures and turns out it was a very simple error. A failed insert statement.

I was puzzled of why the thing didn't throw any error even while debugging. Turns out in SQL 2003 (this was a while ago) someone used a print line statement and SQL stopped throwing errors to the console. I can only assume "printing" in SQL empties the buffered error which would be shown in the console.

I removed the print statement and the error showed up, we fixed it and didn't get sued
:)