devRant - A fun community for developers to connect over code, tech & life as a programmer

I was Just college fresher who completed his Engineering. My first week in the office. And a system was provided to me, since it was support project so I was given direct access to production database.

Fresher + Production Database + Access of Admin credentials = Worst Possible Combination

So it was my night shift, I was told to update new tariff plan for our client (which was one of the largest telecom service in India) .

If someone recharges for more than 200 Rupee, that person will get 10% or 20% extra talk time. Which was only applicable for particular circle (Like Bihar and Rajasthan).

Since I was fresher, I was told to update given query from my senior employee which he shared on the shared folder. Production downtime was in the mid night, so at that time I updated that query on the production database.

Query successfully updated. I completed my night shift, went home and slept.

When I woke up, I saw my mobile it had 200+ missed calls from different locations of India. They were Circle heads of that telecom service provider who contacted me. I realized something unexpected is expecting me.

Then at that moment my team lead called me and he asked me to come office right away.

Reminding you I was a fresher, I was shivering. What have I done there?

When I reached office, I came to know that the query I updated on production bombarded.

Every person who recharged that day (duration from midnight to morning 10 AM) got 10 times or 20 times more talktime.

A part of Query was something like this where error was made:

TalkTime = RechargeAmount + RechargeAmount * 10/100; (Bihar)
or

TalkTime = RechargeAmount + RechargeAmount * 20/100; (Rajasthan)
But instead of this query, I updated below one:

TalkTime = RechargeAmount + RechargeAmount * 10;
or

TalkTime = RechargeAmount + RechargeAmount * 20;
In a span of 10 hours, that telecom service lost revenue of 6.5 crore Rupees. Thanks to recovery team they were able to recover 6 crore but still 50 lakh Rupees were in loss.

One small query, and approx 1 million dollar was on stake.

Aftermath of this incident

My Mistake:

I should have taken those queries on mail. Or, there should have been mail communication regarding this.
Never ever do anything over oral communication. Senior employee who did this denied and said he provided correct query, and I had no proof of communication.
I told them, it was me who executed that query on production. Since I was fresher, and took my responsibility of that incident. My team lead rescued me from that situation.
Lesson Learned:

Always test your query and code multiple times before you execute or Go live it on production.
Always have email communication for every action you take on production.
Power comes with responsibility. If you have admin credentials of production never use it for update/delete/drop until you are sure.
Don’t take your job lightly.
I was not fired from that Job, but I have learnt my lesson very well.