Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "sql client database"
-
I had just started my new job and deleted 3 years of data that the client had spent over £450,000 collecting 😱
another developer used my PC to quickly access the clients database while I was out the room as I had sql management studio open. I went back to my PC thinking I was connected to my local database, did a few truncate tables to test my software and :0 minutes later I get a call asking why there was no data on the server!
Thank god for backups 😓7 -
I was hired as a senior software engineer. During handover I found out I'm actually replacing the CTO.
I queried why he was leaving and got a simple "just want a break from working" which I found odd.
Fast forward and now I also just want a break from work, permanently. This place has followed every bad practise and big no-no out there. Every bit of software is a built in house knockoff janky piece of crap that doesn't work and makes people's jobs 5000 times harder.
The UI looks worse than Windows 3.1, absolutely horrendous code formatting, worst database structure I've ever seen.
The mere mention of using a team communication tool results in being yelled at from the CEO whom communicates purely via email, who then gets annoyed when you don't reply because they sent the email to a client instead of you.
We get handed printed out "tickets" to work instead of the so called "amazing in house ticket system" built using PHP 5 and is literally crammed into an 800x600 IFrame. Yes a F$*#ing IFRAME!
It's not like we have an outdated TFS server that has work items we can use...
Why not push for changes you say. I have, many times, tried to suggest better tools. The only approval I've gotten is using PhpStorm. Everything else is shutdown immediately and you get the silent treatment.
The CEO hired me to do a job, then micromanages like crazy. I can't make UI changes, I can't make database changes, why? They insists they know best, but has admitted multiple times to not knowing SQL and literally uses a drag and drop database table builder.
Every page in the webapps we make are crammed into 800x600 iframes with more iframes inside iframes. And every time it's pointed out we need to do something, be it from internal staff or client suggestions, the CEO goes off about how the UI is industry leading and follows standards.. what in the actual f....
Literally holding on by a thread here. Why hire a CTO under the guise of being a senior developer but then reduce the work that can be done down to the level of a junior?
Sure the paycheck is really nice but no job is worth the stress, harassment and incompetent leadership from the CEO.
They've verbally abused people to the point they resign, best part is that was simply because the CEO made serious legal mistakes, was told about it by the employee then blamed it on others.21 -
What an awful day :(
The server where I host my 4 clients websites crashed.
Unable to reboot from the console.
I contact the support. 15 minutes later: "we'll look at this"
No news for 1 week despite my messages.
Then... 1st ticket escalation... 2nd ticket escalation... 3rd ticket escalation...
Answer: "Sorry, your server is down and cannot be repaired."
Fuck.
I ask "is there any way to get my data back?". Answer: "No, because we would shutdown the whole bay and all our clients would be impacted".
Fuck.
I subscribe to another server, at another provider.
I look at my backups... shit, the last one is 4 month ago!!
I restore the first website: OK
I restore the second website: OK
I restore the third website: My new server is "too recent" and not compatible. with this old Wordpress. Fuck! I'll look at this later...
I restore the fourth website: database is empty!! What??? I look at the SQL backup for this site... it failed...
I lost ALL my 4th client data!!!
I'm sooooo piece of crap!14 -
Worst dev team failure I've experienced?
One of several.
Around 2012, a team of devs were tasked to convert a ASPX service to WCF that had one responsibility, returning product data (description, price, availability, etc...simple stuff)
No complex searching, just pass the ID, you get the response.
I was the original developer of the ASPX service, which API was an XML request and returned an XML response. The 'powers-that-be' decided anything XML was evil and had to be purged from the planet. If this thought bubble popped up over your head "Wait a sec...doesn't WCF transmit everything via SOAP, which is XML?", yes, but in their minds SOAP wasn't XML. That's not the worst WTF of this story.
The team, 3 developers, 2 DBAs, network administrators, several web developers, worked on the conversion for about 9 months using the Waterfall method (3~5 months was mostly in meetings and very basic prototyping) and using a test-first approach (their own flavor of TDD). The 'go live' day was to occur at 3:00AM and mandatory that nearly the entire department be on-sight (including the department VP) and available to help troubleshoot any system issues.
3:00AM - Teams start their deployments
3:05AM - Thousands and thousands of errors from all kinds of sources (web exceptions, database exceptions, server exceptions, etc), site goes down, teams roll everything back.
3:30AM - The primary developer remembered he made a last minute change to a stored procedure parameter that hadn't been pushed to production, which caused a side-affect across several layers of their stack.
4:00AM - The developer found his bug, but the manager decided it would be better if everyone went home and get a fresh look at the problem at 8:00AM (yes, he expected everyone to be back in the office at 8:00AM).
About a month later, the team scheduled another 3:00AM deployment (VP was present again), confident that introducing mocking into their testing pipeline would fix any database related errors.
3:00AM - Team starts their deployments.
3:30AM - No major errors, things seem to be going well. High fives, cheers..manager tells everyone to head home.
3:35AM - Site crashes, like white page, no response from the servers kind of crash. Resetting IIS on the servers works, but only for around 10 minutes or so.
4:00AM - Team rolls back, manager is clearly pissed at this point, "Nobody is going fucking home until we figure this out!!"
6:00AM - Diagnostics found the WCF client was causing the server to run out of resources, with a mix of clogging up server bandwidth, and a sprinkle of N+1 scaling problem. Manager lets everyone go home, but be back in the office at 8:00AM to develop a plan so this *never* happens again.
About 2 months later, a 'real' development+integration environment (previously, any+all integration tests were on the developer's machine) and the team scheduled a 6:00AM deployment, but at a much, much smaller scale with just the 3 development team members.
Why? Because the manager 'froze' changes to the ASPX service, the web team still needed various enhancements, so they bypassed the service (not using the ASPX service at all) and wrote their own SQL scripts that hit the database directly and utilized AppFabric/Velocity caching to allow the site to scale. There were only a couple client application using the ASPX service that needed to be converted, so deploying at 6:00AM gave everyone a couple of hours before users got into the office. Service deployed, worked like a champ.
A week later the VP schedules a celebration for the successful migration to WCF. Pizza, cake, the works. The 3 team members received awards (and a envelope, which probably equaled some $$$) and the entire team received a custom Benchmade pocket knife to remember this project's success. Myself and several others just stared at each other, not knowing what to say.
Later, my manager pulls several of us into a conference room
Me: "What the hell? This is one of the biggest failures I've been apart of. We got rewarded for thousands and thousands of dollars of wasted time."
<others expressed the same and expletive sediments>
Mgr: "I know..I know...but that's the story we have to stick with. If the company realizes what a fucking mess this is, we could all be fired."
Me: "What?!! All of us?!"
Mgr: "Well, shit rolls downhill. Dept-Mgr-John is ready to fire anyone he felt could make him look bad, which is why I pulled you guys in here. The other sheep out there will go along with anything he says and more than happy to throw you under the bus. Keep your head down until this blows over. Say nothing."11 -
*Client phones me at 11pm*
Client: It's not working!!
Me:What's the error you're getting?
Client: "Database connection error"
*Phones system/dB admin*
DB Admin: Yeah we had to change the SQL logins, I've sent you the new ones
*Phones junior dev in charge of dB programming*
Junior Dev: Yeah you'll just have to go and change the credentials. They're in all the places where we're using the dB, just before the statement, in the connection strings...
We make over 470 calls to the DB 😑16 -
Worst legacy experience...
Called in by a client who had had a pen test on their website and it showed up many, many security holes. I was tasked with coming in and implementing the required fixes.
Site turned out to be Classic ASP built on an MS Access database. Due to the nature of the client, everything had to be done on their premises (kind of ironic but there you go). So I'm on-site trying to get access to code and server. My contact was *never* at her desk to approve anything. IT staff "worked" 11am to 3pm on a long day. The code itself was shite beyond belief.
The site was full of forms with no input validation, origin validation and no SQL injection checks. Sensitive data stored in plain text in cookies. Technical errors displayed on certain pages revealing site structure and even DB table names. Server configured to allow directory listing in file stores so that the public could see/access whatever they liked without any permission or authentication checks. I swear this was written by the child of some staff member. No company would have had the balls to charge for this.
Took me about 8 weeks to make and deploy the changes to client's satisfaction. Could have done it in 2 with some support from the actual people I was suppose to be helping!! But it was their money (well, my money as they were government funded!).1 -
Intelligence and ability cannot be measured by education.
I have a client who asked a Master in Computer Science to develop a small system, for querying product title and their code. The guy used python, vanilla js, and... Txt file for the "database". Then my client asked me to integrated this in... WordPress.
This was in 2016. And idiot as I'm, I agreed and adapted his code to use php and a database.
April this year, my client said they are still using the python system to add new products all this time, in parallel. And wanted to update the WordPress with the data.
- No problem! - I said. Just send me the SQL file.
So the Master in CS sent me a SQL coded in ANSI. I asked for the SQL again, but with a more appropriate encoding. He took 1 month to reply back, and said it would be better if I get rid of the database and just use the txt file for querying.
This is outrageous.
I really hate people who are educated but completely useless.5 -
At the data restaurant:
Chef: Our freezer is broken and our pots and pans are rusty. We need to refactor our kitchen.
Manager: Bring me a detailed plan on why we need each equipment, what can we do with each, three price estimates for each item from different vendors, a business case for the technical activities required and an extremely detailed timeline. Oh, and do not stop doing your job while doing all this paperwork.
Chef: ...
Boss: ...
Some time later a customer gets to the restaurant.
Waiter: This VIP wants a burguer.
Boss: Go make the burger!
Chef: Our frying pan is rusty and we do not have most of the ingredients. I told you we need to refactor our kitchen. And that I cannot work while doing that mountain of paperwork you wanted!
Boss: Let's do it like this, fix the tech mumbo jumbo just enough to make this VIP's burguer. Then we can talk about the rest.
The chef then runs to the grocery store and back and prepares to make a health hazard hurried burguer with a rusty pan.
Waiter: We got six more clients waiting.
Boss: They are hungry! Stop whatever useless nonsense you were doing and cook their requests!
Cook: Stop cooking the order of the client who got here first?
Boss: The others are urgent!
Cook: This one had said so as well, but fine. What do they want?
Waiter: Two more burgers, a new kind of modern gaseous dessert, two whole chickens and an eleven seat sofa.
Chef: Why would they even ask for a sofa?!? We are a restaurant!
Boss: They don't care about your Linux techno bullshit! They just want their orders!
Cook: Their orders make no sense!
Boss: You know nothing about the client's needs!
Cook: ...
Boss: ...
That is how I feel every time I have to deal with a boss who can't tell a PostgreSQL database from a robots.txt file.
Or everytime someone assumes we have a pristine SQL table with every single column imaginable.
Or that a couple hundred terabytes of cold storage data must be scanned entirely in a fraction of a second on a shoestring budget.
Or that years of never stored historical data can be retrieved from the limbo.
Or when I'm told that refactoring has no ROI.
Fuck data stack cluelessness.
Fuck clients that lack of basic logical skills.5 -
A few weeks ago a client called me. His application contains a lot of data, including email addresses (local part and domain stored separately in SQL database). The application can filter data based on the domain part of the addresses. He ask me why sub.example.com is not included when he asked the application for example.com. I said: No problem, I can add this feature to the application, but the process will take a longer.
Client: No problem, please add this ASAP.
So, the next day I changed some of the SQL queries to lookup using the LIKE operator.
After a week the client called again: The process is really slow, how can this be?
Me: Well, you asked me to filter the subdomains as well. Before, the application could easily find all the domains (SQL index), but now it has to compare all the domains to check if it ends with the domain you are looking for.
Client: Okay, but why is it a lot slower than before?
Me: Do you have a dictionary in your office?
<Client search for a dictionary, came back with one>
Me: give me the definition of the word "time"
<Client gives definition of time>
Me: Give me the definition of all words ending with "time"
Client: But, ...
Never heard from him again on this issues :-P5 -
I've optimised so many things in my time I can't remember most of them.
Most recently, something had to be the equivalent off `"literal" LIKE column` with a million rows to compare. It would take around a second average each literal to lookup for a service that needs to be high load and low latency. This isn't an easy case to optimise, many people would consider it impossible.
It took my a couple of hours to reverse engineer the data and implement a few hundred line implementation that would look it up in 1ms average with the worst possible case being very rare and not too distant from this.
In another case there was a lookup of arbitrary time spans that most people would not bother to cache because the input parameters are too short lived and variable to make a difference. I replaced the 50000+ line application acting as a middle man between the application and database with 500 lines of code that did the look up faster and was able to implement a reasonable caching strategy. This dropped resource consumption by a minimum of factor of ten at least. Misses were cheaper and it was able to cache most cases. It also involved modifying the client library in C to stop it unnecessarily wrapping primitives in objects to the high level language which was causing it to consume excessive amounts of memory when processing huge data streams.
Another system would download a huge data set for every point of sale constantly, then parse and apply it. It had to reflect changes quickly but would download the whole dataset each time containing hundreds of thousands of rows. I whipped up a system so that a single server (barring redundancy) would download it in a loop, parse it using C which was much faster than the traditional interpreted language, then use a custom data differential format, TCP data streaming protocol, binary serialisation and LZMA compression to pipe it down to points of sale. This protocol also used versioning for catchup and differential combination for additional reduction in size. It went from being 30 seconds to a few minutes behind to using able to keep up to with in a second of changes. It was also using so much bandwidth that it would reach the limit on ADSL connections then get throttled. I looked at the traffic stats after and it dropped from dozens of terabytes a month to around a gigabyte or so a month for several hundred machines. The drop in the graphs you'd think all the machines had been turned off as that's what it looked like. It could now happily run over GPRS or 56K.
I was working on a project with a lot of data and noticed these huge tables and horrible queries. The tables were all the results of queries. Someone wrote terrible SQL then to optimise it ran it in the background with all possible variable values then store the results of joins and aggregates into new tables. On top of those tables they wrote more SQL. I wrote some new queries and query generation that wiped out thousands of lines of code immediately and operated on the original tables taking things down from 30GB and rapidly climbing to a couple GB.
Another time a piece of mathematics had to generate all possible permutations and the existing solution was factorial. I worked out how to optimise it to run n*n which believe it or not made the world of difference. Went from hardly handling anything to handling anything thrown at it. It was nice trying to get people to "freeze the system now".
I build my own frontend systems (admittedly rushed) that do what angular/react/vue aim for but with higher (maximum) performance including an in memory data base to back the UI that had layered event driven indexes and could handle referential integrity (overlay on the database only revealing items with valid integrity) or reordering and reposition events very rapidly using a custom AVL tree. You could layer indexes over it (data inheritance) that could be partial and dynamic.
So many times have I optimised things on automatic just cleaning up code normally. Hundreds, thousands of optimisations. It's what makes my clock tick.4 -
I think I want to quit my first applicantion developer job 6 months in because of just how bad the code and deployment and.. Just everything, is.
I'm a C#/.net developer. Currently I'm working on some asp.net and sql stuff for this company.
We have no code standards. Our project manager is somewhere between useless and determinental. Our clients are unreasonable (its the government, so im a bit stifled on what I can say.) and expect absurd things from us. We have 0 automated tests and before I arrived all our infrastructure wasn't correct to our documentation... And we barely had any documentation to begin with.
The code is another horror story. It's out sourced C# asp.net, js and SQL code.. And to very bad programmers in India, no offense to the good ones, I know you exist. Its all spagheti. And half of it isn't spelled correctly.
We have a single, massive constant class that probably has over 2000 constants, I don't care to count. Our SQL projects are a mess with tons of quick fix scripts to run pre and post publishing. Our folder structure makes no sense (We have root/js and root/js1 to make you cringe.) our javascript is majoritly on the asp.net pages themselves inline, so we don't even have minification most of the time.
It's... God awful. The result of a billion and one quick fixes that nobody documented. The configuration alone has to have the same value put multiple times. And now our senior developer is getting the outsourced department to work on moving every SINGLE NORMAL STRING INTO THE DATABASE. That's right. Rather then putting them into some local resource file or anything sane, our website will now be drawing every single standard string from the database. Our SENIOR DEVELOPER thinks this is a good idea. I don't need to go into detail about how slow this is. Want to do it on boot? Fine. But they do it every time the page loads. It's absurd.
Our sql database design is an absolute atrocity. You have to join several tables together just to get anything done. Half of our SP's are failing all the time because nobody really understands the design. Its gloriously awful its like.. The epitome of failed database designs.
But rather then taking a step back and dealing with all the issues, we keep adding new features and other ones get left in the dust. Hell, we don't even have complete browser support yet. There were things on the website that were still running SILVERLIGHT. In 2019. I don't even know how to feel about it.
I brought up our insane technical debt to our PM who told me that we don't have time to worry about things like technical debt. They also wouldn't spend the time to teach me anything, saying they would rather outsource everything then take the time to teach me. So i did. I learned a huge chunk of it myself.
But calling this a developer job was a sick, twisted joke. All our lives revolve around bugnet. Our work is our BN's. So every issue the client emails about becomes BN's. I haven't developed anything. All I've done is clean up others mess.
Except for the one time they did have me develop something. And I did it right and took my time. And then they told me it took too long, forced me to release before it was ready, even though I had never worked on what I was doing before. And it worked. I did it.
They then told me it likely wouldn't even be used anyway. I wasn't very happy at all.
I then discovered quickly the horrors of wanting to make changes on production. In order to make changes to it, we have to... Get this
Write a huge document explaining why. Not to our management. To the customer. The customer wants us to 'request' to fix our application.
I feel like I am literally against a wall. A huge massive wall. I can't get constent from my PM to fix the shitty code they have as a result of outsourcing. I can't make changes without the customer asking why I would work on something that doesn't add something new for them. And I can't ask for any sort of help, and half of the people I have to ask help from don't even speak english very well so it makes it double hard to understand anything.
But what can I do? If I leave my job it leaves a lasting stain on my record that I am unsure if I can shake off.
... Well, thats my tl;dr rant. Im a junior, so maybe idk what the hell im talking about.rant code application bad project management annoying as hell bad code c++ bad client bad design application development16 -
Coworker: so once the algorithm is done I will append new columns in the sql database and insert the output there
Me: I don't like that, can we put the output in a separate table and link it using a foreign key. Just to avoid touching the original data, you know, to avoid potential corruption.
C: Yes sure.
< Two days later - over text >
C: I finished the algo, i decided to append it to the original data in order to avoid redundancy and save on space. I think this makes more sense.
Me: ahdhxjdjsisudhdhdbdbkekdh
No. Learn this principal:
" The original data generated by the client, should be treated like the god damn Bible! DO NOT EVER CHANGE ITS SCHEMA FOR A 3RD PARTY CALCULATION! "
Put simply: D.F.T.T.O
Don't. Fucking. Touch. The. Origin!5 -
Worst one I’ve seen so far is when I was working for my previous community another developer joined to help me, without the permission of me or the other lead developer he pushed a client-side update. We didn’t think it was a big deal, but once we began reviewing the code it became a big deal... he had placed our SQL credentials into that file that every client downloads. All the person had to do was open the file and could connect to our SQL which contained 50k+ players info, primarily all in-game stuff except IPs which we want to protect at all costs.
Issue becomes, what he was trying to do required the games local database on the client-side, but instead he tried connecting to it as an external database so he decided to copy server-side code and used on the client.
Anyways, the database had a firewall that blocked all connections except the server and the other lead dev and myself. We managed to change the credentials and pull the file away before any harm was done to it, about 300 people had downloaded the file within an hours period, but nothing happened luckily. IP to the DB, username, password, etc, were all changed just to keep it protected.
So far this is the worst, hopefully it doesn’t get worse than this :/1 -
<just got out of this meeting>
Mgr: “Can we log the messages coming from the services?”
Me: “Absolutely, but it could be a lot of network traffic and create a lot of noise. I’m not sure if our current logging infrastructure is the right fit for this.”
Senior Dev: “We could use Log4Net. That will take care of the logging.”
Mgr: “Log4Net?…Yea…I’ve heard of it…Great, make it happen.”
Me: “Um…Log4Net is just the client library, I’m talking about the back-end, where the data is logged. For this issue, we want to make sure the data we’re logging is as concise as possible. We don’t want to cause a bottleneck inside the service logging informational messages.”
Mgr: “Oh, no, absolutely not, but I don’t know the right answer, which is why I’ll let you two figure it out.”
Senior Dev: “Log4Net will take care of any threading issues we have with logging. It’ll work.”
Me: “Um..I’m sure…but we need to figure out what we need to log before we decide how we’re logging it.”
Senior Dev: “Yea, but if we log to SQL database, it will scale just fine.”
Mgr: “A SQL database? For logging? That seems excessive.”
Senior Dev: “No, not really. Log4Net takes care of all the details.”
Me: “That’s not going to happen. We’re not going to set up an entire sql database infrastructure to log data.”
Senior Dev: “Yea…probably right. We could use ElasticSearch or even Redis. Those are lightweight.”
Mgr: “Oh..yea…I’ve heard good things about Redis.”
Senior Dev: “Yea, and it runs on Linux and Linux is free.”
Mgr: “I like free, but I’m late for another meeting…you guys figure it out and let me know.”
<mgr leaves>
Me: “So..Linux…um…know anything about administrating Redis on Linux?”
Senior Dev: ”Oh no…not a clue.”
It was all I could do from doing physical harm to another human being.
I really hate people playing buzzword bingo with projects I’m responsible for.
Only good piece is he’s not changing any of the code.3 -
Had to implement a search feature for a client so I did and told him to check it out (it was a LIKE SQL statement)
He tested it out by typing in iPhone and Phone (there was an item called iphone in the database) and he was amazed about the fact it worked for some reason.
A real answer from him:
"it's working.. how is it working?
each item is tagged to multiple keywords without the user doing anything"
I said it's just a text search but I guess I should've said I'm using an advanced AI to extract all possible terms related to the item title.1 -
I had spent the last year working on a online store power by woocommerce with over 100k products from various suppliers. This online store utilized a custom API that would take the various formats that suppliers offer their inventory in and made them consistent. Now everything was going swimmingly initially, but then I began adding more and more products using a plug-in called WP all import. I reached around 100k products and the site would take up to an entire minute to load sometimes timing out. I got desperate so I installed several caching plugins, but to no avail this did not help me. The site was originally only supposed to take three to four months but ended up taking an entire year. Then, just yesterday I found out what went wrong and why this woocommerce website with all of these optimizations was still taking anywhere from 60 to 90 seconds to load, or just timing out entirely. I had initially thought that I needed a beefier server so I moved it to a high CPU digitalocean VM. While this did help a little bit, the site was still very slow and now I had very high CPU usage RAM usage and high disk IO. I was seriously stumped the Apache process was using a high amount of CPU and IO along with MYSQL as well. It wasn't until I started digging deeper into the database that I actually found out what the issue was. As I was loading the site I would run 'show process list' in the SQL terminal, I began to notice a very significant load time for one of the tables, so I went to go and check it out. What I did was I ran a select all query on that particular table just to see how full it was and SQL returned a error saying that I had exceeded the maximum packet size. So I was like okay what the fuck...
So I exited my SQL and re-entered it this time with a higher packet size. I ran a query that would count how many rows were in this particular table and the number came out to being in the millions. I was surprised, and what's worse is that this table belong to a plugin that I had attempted to use early in the development process to cache the site. The plugin was deactivated but apparently it had left PHP files within the wp content directory outside of the actual plugin directory, so it's still executing scripts even though the plugin itself was disabled. Basically every time I would change anything on the site, it would recache the whole thing, and it didn't delete any old records. So 100k+ products caching on saves with no garbage collection... You do the math, it's gonna be a heavy ass database. Not only that but it was serialized data, so when it did pull this metric shit ton of spaghetti from the database, PHP then had to deserialize it. Hence the high ass CPU load. I had caching enabled on the MySQL end of things so that ate the ram. I was really desperate to get this thing running.
Honest to God the main reason why this website took so long was because the load times made it miserable to work on. I just thought that the hardware that I had the site on was inadequate. I had initially started the development on a small Linux VM which apparently wasn't enough, which is why I moved it to digitalocean which also seemed to not be enough, so from there I moved to a dedicated server which still didn't seem to be enough. I was probably a few more 60-second wait times or timeouts from recommending a server cluster to my client who I know would not be willing to purchase it. The client who I promised this site to have completed in 3 months and has waited a year. Seriously, I would tell people the struggles that I would go through with this particular site and they would just tell me to just drop the site; just take the money, just take the loss. I refused to, this was really the only thing that was kicking my ass. I present myself as this high-and-mighty developer like I'm just really good at what I do but then I have this WordPress site that's just beating the shit out of me for a year. It was a very big learning experience and it was also very humbling as well, it made me realize that I really don't know as much as I think I might. It was evidence that there is still so much more to learn out there, I did learn a lot from that experience especially about optimizing websites the different types of methods to do that particular lonely on the server side and I'll be able to utilize this knowledge in the future.
I guess the moral of the story is, never really give up. Ultimately things might get so bad that you're running on hopes and dreams. Those experiences are generally the most humbling. Now I can finally present the site that I am basically a year late on to the client who will be so happy that I did not give up on the project entirely. I'll have experienced this feeling of pure euphoria, and help the small business significantly grow their revenue. Helping others is very fulfilling for me, even at my own expense.
Anyways, gonna stop ranting. Running out of characters. If you're still here... Ty for reading :')7 -
I promised a friend to have a look over his dads website to add a small blog. No big deal, I've got it on my drive, can reuse it just need to adapt it to the environment.
I take a look at what I'm working with and I see the most terrifying piece of "Please, take my data" code I could possibly imagine (And I've seen passwords, in plain text in a script tag). I quote "function queryDB(mode, val) {
var query=" ";
if(mode==="findProd")
query="Select * from Products where ProdNam=" +val;
... (same shit for different cases)
sendQuery(query) ;
}
He literally built the query on the client side sent it to a php script (without validation) and inserted it into the database.
You could literally call window.sendQuery with any sql query and get the result printed into the console.
And other than the plain text passwords guy that wasn't some kid someone knew, this was a "Webdesign" Agency.
Now I took the entire thing offline, called my friends dad, explained it to him and try to sort this out. I would not charge a good friends father but that hack will get a quite hefty bill since my hourly rate just tripled.
And the worst thing : If I publicly name that asshole or warn the people in his portfolio I can, according to Google, be sued. (But, and I assume thats vague enough not to count as bad mouthing, if anyone of you has a customer from Rheinland-Pfalz, Germany with a preexisting page, please have a look at the database interface)
I will call that agency tomorrow, ask for a detailed explanation for why they apparently let trained monkeys write their code and anonymously warn everyone in their portfolio about those flaws...
I don't know if I'm cursed or if there are just that many bad devs but it seems that once a year I have to stumble over some "mistakes" that make me question my sanity.4 -
Beware: Here lies a cautionary tale about shared hosting, backups, and -goes without saying- WordPress.
1. Got a call from a client saying their site presented an issue with a third-party add-on. The vendor asked us to grant him access to our staging copy.
2. Their staging copy, apparently, never got duplicated correctly because, for security reasons, their in-house dev changed the name of the wp-content folder. That broke their staging algo. So no staging site.
3. In order to recreate the staging site, we had to reset everything back to WP defaults. Including, for some reason, absolute paths inside the database. A huge fucking database. Because WordPress.
4. Made the changes directly in a downloaded sql file. Shared hosting, obviously, had an upload limit smaller to the actual database.
5. Spent half an hour trying to upload table by table to no avail.
6. In-house uploads a new, fixed database with the help of the shared hosting provider.
7. Database has the wrong path. Again.
8. In-house performs massive Find and Replace through phpMyAdmin on the production server.
9. Obviously, MySQL crashes instantly and the site gets blocked for over 3 hours for exceeding shared hosting limits.
10. Hosting provider refuses to accept this was caused by such a stupid act and says site needs to be checked because queries are too slow.
11. We are gouging our eyeballs as we see an in-house vs. hosting fight unfold. So we decide to watch a whole Netflix documentary in between.
12. Finally, the hosting folds and enables access to the site, which is obvi not working because, you know, wrong paths.
13. Documentary finishes. We log in again, click restore from backup. Go to bed. Client phones to bless us. Client’s in-house dev probably looking for a cardboard box to pack his stuff first thing in the morning. \_(ツ)_/¯ -
I just found a vulnerability in my companies software.
Anyone who can edit a specific config file could implant some SQL there, which would later be executed by another (unknowing) user from within the software.
The software in question is B2B and has a server-client model, but with the client directly connecting to the database for most operations - but what you can do should be regulated by the software. With this cute little exploit I managed to drop a table from my test environment - or worse: I could manipulate data, so when you realize it it's too late to simply restore a DB backup because there might have been small changes for who knows how long. If someone was to use this maliciously the damages could be easily several million Euros for some of our customers (think about a few hundred thousand orders per day being deleted/changed).
It could also potentially be used for data exfiltration by changing protection flags, though if we're talking industry espionage they would probably find other ways and exploit the OS or DB directly, given that this attack requires specific knowledge of the software. Also we don't promise to safely store your crabby patty recipe (or other super secret secrets).
The good thing is that an attack would only possible for someone with both write access to that file and insider knowledge (though that can be gained by user of the software fairly easily with some knowledge of SQL).
Well, so much for logging off early on Friday.5 -
I am a Technical Lead in the department in my company that writes code for our clients that have money but doesn't have the technical expertise to handle the complexities of our own software.
Part of my tasks involve taking care of a few projects written by employees that have left after using third-party tools rather than using our own software. No one else in this department knows these third-party tools, they only know our own, and my *still limited* web development experience means I get dumped these things in my lap.
And I'm SO pissed at these projects and their authors and the manager that let these ex-employees write these things. There is this one project that was managed by two different "developers" (I don't know they deserve this title) at two different times, and it is so riddled with different technologies it makes me want to throw up almost daily.
Don't believe me? Here is a complete list of the dependencies listed in the package.json of this project: babel-polyfill, body-parser, cookie-parser, debug, edge, edge-sql, excel-to-json, exceljs, express, html-inline, jade, morgan, mssql, mysql, pug, ramda, request, rotating-file-stream, serve-favicon, webpack, xlsx, xml2js
What this doesn't even show, is that one part of this project (literally one page) is made using react, react-dom, react-redux, and jade. The other part (again literally one page) is made using Angular and Pug. In case you missed it while picking up your jaw, there's also mssql, mysql, edge and edge-sql. excel-to-json, exceljs, xlsx.
Oh you want *more* juicy details? This project takes the entire data object used by the front-end, stringifies it into JSON, and shoves it into the database *as a single field*. And instead of doing WHERE clauses in the SQL queries, it grabs the entire table, loops, parses the json, and does a condition on it. If even one of those JSON entries gets corrupted, the entire solution breaks because these "developers" don't know what try/catch is.
The client asked for a very simple change in their app, which was to add a button that queries the back-end for a URL, shows it in a modal dialog, after which a button is clicked to verify the link by doing a second query to the back-end before modifying a couple of fields in the page.
This. Took. Me. Two. Months*. Save me. Please, save me.
*between constant context switches between this and other projects that were continuously failing because of their mistakes.4 -
It was the last year of high school.
We had to submit our final CS homework, so it gets reviewed by someone from the ministry of education and grade it. (think of it as GPA or whatever that is in your country).
Now being me, I really didn’t do much during the whole year, All I did was learning more about C#, more about SQL, and learn from the OGs like thenewboston, derek banas, and of course kudvenkat. (Plus more)
The homework was a C# webform website of whatever theme you like (mostly a web store) that uses MS Access as DB and a C# web service in SOAP. (Don’t ask.)
Part 1/2:
Months have passed, and only had 2 days left to deadline, with nothing on my hand but website sketches, sample projects for ideas, and table schematics.
I went ahead and started to work on it, for 48 hours STRAIGHT.
No breaks, barely ate, family visited and I barely noticed, I was just disconnected from reality.
48 hours passed and finished the project, I was quite satisfied with my it, I followed the right standards from encrypting passwords to verifying emails to implementing SQL queries without the risk of SQL injection, while everyone else followed foot as the teacher taught with plain text passwords and… do I need to continue? You know what I mean here.
Anyway, I went ahead and was like, Ok, lets do one last test run, And proceeded into deleting an Item from my webstore (it was something similar to shopify).
I refreshed. Nothing. Blank page. Just nothing. Nothing is working, at all.
Went ahead to debug almost everywhere, nothing, I’ve gone mad, like REALLY mad and almost lose it, then an hour later of failed debugging attempts I decided to rewrite the whole project from scratch from rebuilding the db, to rewriting the client/backend code and ui, and whatever works just go with it.
Then I noticed a loop block that was going infinite.
NEVER WAIT FOR A DATABASE TO HAVE MINIMUM NUMBER OF ROWS, ALWAYS ASSUME THAT IT HAS NO VALUES. (and if your CPU is 100%, its an infinite loop, a hard lesson learned)
The issue was that I requested 4 or more items from a table, and if it was less it would just loop.
So I went ahead, fixed that and went to sleep.
Part 2/2:
The day has come, the guy from the ministry came in and started reviewing each one of the students homeworks, and of course, some of the projects crashed last minute and straight up stopped working, it's like watching people burning alive.
My turn was up, he came and sat next to me and was like:
Him: Alright make me an account with an email of asd@123.com with a password 123456
Me: … that won't work, got a real email?
Him: What do you mean?
Me: I implemented an email verification system.
Him: … ok … just show me the website.
Me: Alright as you can see here first of all I used mailgun service on a .tk domain in order to send verification emails you know like every single website does, encrypted passwords etc… As you can see this website allows you to sign up as a customer or as a merc…
Him: Good job.
He stood up and moved on.
YOU MOTHERFUCKER.
I WENT THROUGH HELL IN THE PAST 48 HOURS.
AND YOU JUST SAT THERE FOR A MINUTE AND GAVE UP ON REVIEWING MY ENTIRE MASTERPIECE? GO SWIM IN A POOL FULL OF BURNING OIL YOU COUNTLESS PIECE OF SHIT
I got 100/100 in the end, and I kinda feel like shit for going thought all that trouble for just one minute of project review, but hey at least it helped me practice common standards.2 -
Still using a database from 90' - Enea Polyhedra:
- no decent visual sql client
- utterly limited scripting language
- weird communications protocol
- no redundancy beyond master-replica
- no encryption of communication protocols
- etc. -
The company I work in had to build a software that establishes a connection to a MySQL database running on an external server. It doesn't work for the client company because the firewall is very restrictive and only lets through connections on port 80, so we had to build a fucking http server that forwards SQL queries to the MySQL server and returns the result. This is so horrible!
(Running MySQL on port 80 isn't an option as any other connection type than http is blocked by the firewall)8 -
My work product: Or why I learned to get twitchy around Java...
I maintain a Java based test system, that tests a raster image processor. The client is a Java swing project that contains CORBA bindings to the internal API of the raster image processor. It also has custom written UI elements and duplicated functionality that became available in later versions of Java, but because some of the third party tools we use don't work with later versions of Java for some reason, it's not possible to upgrade Java to gain things as simple as recursive directory deletion, yes the version of Java we have to use does not support something as simple as that and custom code had to be written to support it.
Because of the requirement to build the API bindings along with the client the whole application must be built with the raster image processor build chain, which is a heavily customised jam build system. So an ant task calls out to execute a jam task and jam does about 90% of the heavy lifting.
In addition to the Java code there's code for interpreting PostScript files, as these can be used to alter the behaviour of the raster image processor during testing.
As if that weren't enough, there's a beanshell interface to allow users to script the test system, but none of the users know Java well enough to feel confident writing interpreted Java scripts (and that's too close to JavaScript for my comfort). I once tried swapping this out for the Rhino JavaScript interpreter and got all the verbal support in the world but no developer time to design an API that'd work for all the departments.
The server isn't much better though. It's a tomcat based application that was written by someone who had never built a tomcat application before, or any web application for that matter and uses raw SQL strings instead of an orm, it doesn't use MVC in any way, and insane amount of functionality is dumped into the jsp files.
It too interacts with a raster image processor to create difference masks of the output, running PostScript as needed. It spawns off multiple threads and can spend days processing hundreds of gigabytes of image output (depending on the size of the tests).
We're stuck on Tomcat seven because we can't upgrade beyond Java 6, which brings a whole manner of security issues, but that eager little Java updated will break the tool chain if it gets its way.
Between these two components we have the Java RMI server (sometimes) working to help generate image data on the client side before all images are pulled across a UNC network path onto the server that processes test jobs (in PDF format), by reading into the xref table of said PDF, finding the embedded image data (for our server consumed test files are just flate encoded TIFF files wrapped around just enough PDF to make them valid) and uses a tool to create a difference mask of two images.
This tool is very error prone, it can't difference images of different sizes, colour spaces, orientations or pixel depths, but it's the best we have.
The tool is installed in both the client and server if the client can generate images it'll query from the server which ones it needs to and if it can't the server will use the tool itself.
Our shells have custom profiles for linking to a whole manner of third party tools and libraries, including a link to visual studio 2005 (more indirectly related build dependencies), the whole profile has to ensure that absolutely no operating system pollution gets into the shell, most of our apps are installed in our home directories and we have to ensure our paths are correct for every single application we add.
And... Fucking and!
Most of the tools are stored as source bundles in a version control system... Not got or mercurial, not perforce or svn, not even CVS... They use a custom built version control system that is built on top of RCS, it keeps a central database of locked files (using soft and hard locks along with write protecting the files in the file system) to ensure users can't get merge conflicts by preventing other users from writing to the files at all.
Branching is heavy weight and can take the best part of a day to create a new branch and populate the history.
Gathering the tools alone to build the Dev environment to build my project takes the best part of a week.
What should be a joy come hardware refresh year becomes a curse ("Well fuck, now I loose a week spending it setting up the Dev environment on ANOTHER machine").
Needless to say, I enjoy NOT working with Java. A lot of this isn't Javas fault, but there's a lot of things that Java (specifically the Java 6 version we're stuck on) does not make easy.
This is why I prefer to build my web apps in python or node, hell, I'd even take Lua... Just... Compiling web pages into executable Java classes, why? I mean I understand the implementation of how this happens, but why did my predecessor have to choose this? Why?2 -
So.. I'm migrating a physical server to a virtual (Hyper-V) one.
The physical server is running Windows Server 2008 R2 with IIS6 and Windows SQL Server 2012.
I've set up a VM with Windows Server 2016, IIS10 and Windows SQL Server 2017.
I'm testing with just moving 1 db at a time (we have about 20, 1 per client running this software and a few others) and I've already imported all of the IIS sites.
So the database import and IIS import went smoothly and was surprisingly without hassle but now I'm trying to run the website that I imported the database for and it is throwing 503 Errors at me.
I've been trying to find out the cause but for some reason IIS isn't making any logs.
It's not any 64/32bit system problems (they're both x64) and I can't seem to find anything wrong with the imported config.
Anyone got any ideas?14 -
Sooooo I came in to work yesterday and the first thing I see is that our client can't log on to the cms I set up for her a month ago. I go log in with my admin credentials and check the audit logs.
It says the last person to access it was me, the date and time exactly when we first deployed it to production.
One month ago.
I fired a calm email to our project managers (who've yet to even read the client complaint!) to check with ops if the cms production database had been touched by the ops team responsible for the sql servers. Because it was definitely not a code issue, and the audit logs never lie.
Later in the day, the audit log updated itself with additional entries - apparently someone in ops had the foresight to back up the database - but it was still missing a good couple weeks of content, meaning the backup db was not recent.
Fucking idiots. -
// !rant
Need some assistance with Drupal and Dreamfactory.
Dreamfactory is an amazing piece of software that basically turns any database into a REST API. I mean any DB from SQL Server to MySQL and all kinds of others. For a connection to the API it uses JWT (JSON Web Tokens) which expire momentarily.
On Drupal, there's wsdata and rest client modules. Restclient is a module where you configure a connection via OAuth or HybridAuth to a rest server. The problem is that the rest server for dreamfactory uses JWT and i'm not sure how to get Drupal and restclient to connect that way. -
I am in need of a good web host for my personal website... the one I use now is free and therefore sucks. I can't even get external access to the MySQL database and their SQL client sucks.9
-
! rant, I hope😗
To anybody with MySQL/Mariadb experience, how different are these two as I need to pick one for a database application I'm building.
Also, and God will I sound like an amature for asking, is it best practice to debug and deploy the SQL database on an actual hosted server or to just do it on the client PC? I have both and I want to work with whichever one will give me less of a headache.😣
Any feedback would be helpful! The server is Debian and the client is Arch Linux.8 -
I got situation here,
I am getting 524 error from cloud fare. I sent some data using AJAX, process it and then return the result. Since the data is large and have some SQL manipulation on it so it take a lot of time. I put the process in back end. But still even for 10k records it took 4-5 minutes to process, Issue is everything works fine but since cloud fare response time is 1-2 minute so it through 524 error (as it does not getting any response within its time frame). How I am suppose to tackle this. May be using job scheduler now ? My client simply refuse to send small data. My Friend is suggesting don't use ajax, simply reload the page. But again data is too much so page loading will also through 524 error. Kindaa stuck here. Any idea/suggestion how I can proceed.
Language I am using PHP. Database, MySQL and SQL.
Hmm Here is some more explanation
https://github.com/marcialpaulg/...
But not working
Here is also something
https://stackoverflow.com/questions...
But I am thinking why redirecting ? It doesn't make sense to me7 -
A question guys, I'm looking for a DB client. We are using SQL Management Server Studio, but its way way overkill for a frontend guy like me.
Min Requirements:
- Needs to be simple
- Must support MSSQL
- Plugin support and/or dark mode
- Free and/or not too buggy evaluation (like Sublime or Winrar)
- GUI should be reasonably modern
- Should also be native. Our database is a denormalized mess.6 -
So, I am in the last stages of development of a really big project and I need to figure out a way to package future patches and updates for the client in order for them to manually update the project on prod server.
For reasons I cannot specify here, they will not use any automated process, and we need to provide regular patches and updates for the next year.
So I was thinking of using git archive to package changed files from our repo for every new commit, or series of commits, and just give them that, along with any database schema updates as sql files (again, no automation can be used).
We are talking about a large PHP + MySQL app, and cannot use automated deployment strategies.
I feel there must be a better way to do this, but this is the best I could come up with so far.
What do you people think?
Any ideeas? -
I did some of the front-end and whole backend. build and manage the SQL + elasticsearch database. After all of this, only 17 lines of mother fu**er code ruined my life. The client is asking for code. And.... And... Can't say anymore.
input {
file {
path => "/home/rsa-key-20200528 /aslogger.log"
type => "java"
start_position => "beginning"
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["localhost:9200"]
index => "aslogger"
}
}