Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
--- HTTP/3 is coming! And it won't use TCP! ---
A recent announcement reveals that HTTP - the protocol used by browsers to communicate with web servers - will get a major change in version 3!
Before, the HTTP protocols (version 1.0, 1.1 and 2.2) were all layered on top of TCP (Transmission Control Protocol).
TCP provides reliable, ordered, and error-checked delivery of data over an IP network.
It can handle hardware failures, timeouts, etc. and makes sure the data is received in the order it was transmitted in.
Also you can easily detect if any corruption during transmission has occurred.
All these features are necessary for a protocol such as HTTP, but TCP wasn't originally designed for HTTP!
It's a "one-size-fits-all" solution, suitable for *any* application that needs this kind of reliability.
TCP does a lot of round trips between the client and the server to make sure everybody receives their data. Especially if you're using SSL. This results in a high network latency.
So if we had a protocol which is basically designed for HTTP, it could help a lot at fixing all these problems.
This is the idea behind "QUIC", an experimental network protocol, originally created by Google, using UDP.
Now we all know how unreliable UDP is: You don't know if the data you sent was received nor does the receiver know if there is anything missing. Also, data is unordered, so if anything takes longer to send, it will most likely mix up with the other pieces of data. The only good part of UDP is its simplicity.
So why use this crappy thing for such an important protocol as HTTP?
Well, QUIC fixes all these problems UDP has, and provides the reliability of TCP but without introducing lots of round trips and a high latency! (How cool is that?)
The Internet Engineering Task Force (IETF) has been working (or is still working) on a standardized version of QUIC, although it's very different from Google's original proposal.
The IETF also wants to create a version of HTTP that uses QUIC, previously referred to as HTTP-over-QUIC. HTTP-over-QUIC isn't, however, HTTP/2 over QUIC.
It's a new, updated version of HTTP built for QUIC.
Now, the chairman of both the HTTP working group and the QUIC working group for IETF, Mark Nottingham, wanted to rename HTTP-over-QUIC to HTTP/3, and it seems like his proposal got accepted!
So version 3 of HTTP will have QUIC as an essential, integral feature, and we can expect that it no longer uses TCP as its network protocol.
We will see how it turns out in the end, but I'm sure we will have to wait a couple more years for HTTP/3, when it has been thoroughly tested and integrated.
Thank you for reading!27 -
"Arch Linux is actually not that difficult".
I ssh'ed into my home server yesterday.
I was greeted by a message from an ext3 disk about needing fsck. Fine, "I haven't been in here for a while, might as well do some maintenance". fsck /dev/sda6, let's go!
This nicely "repaired" the sshd service (i.e. cleared the sectors), I cursed at myself for pressing enter at "repair (y)" right before the connection broke.
So I connected a display and keyboard... ok so let's just pacman -Sy sshd or whatever. We can do this! Just check the wiki, shouldn't be that hard!
Wait... pacman has not run since 2010? WAIT IT'S ACTUAL UPTIME IS 9 YEARS??? I guess we know why I'm a DB admin and not devops...
Hmm all the mirrors give timeouts? Oh. The i686 processor architecture isn't even supported anymore...?
4 hours, 11 glasses of cognac, 73 Arch32 wiki/forum pages, 2 attempts at compiling glibc, and 4 kernel panics later: "I think I'll buy a new server".16 -
I realize I've ranted about this before, but...
Fuck APIs.
First the fact that external services can throw back 500 errors or timeouts when their maintainer did a drunk deploy (but you properly handled that using caching, workers, retry handlers, etc, right? RIGHT?)...
Then the fact that they all speak a variety of languages and dialects (Oh fuck why does that endpoint return a JSON object with int keys instead of a simple array... wait the params are separated with pipe characters? And the other endpoint uses SOAP? Fuck I need to write another wrapper class around the client...)
But the worst thing: It makes developers live in this happy imaginary universe where "malicious" is not a word.
"I found this cloud service which checks our code style" — hmm ok, they seem trustworthy. Hope they don't sell our code, but whatever.
"And look at this thing, it automatically makes database backups, just have to connect to it to DigitalOcean" — uhhh wait...
"And I just built this API client which sends these forms to be OCR processed" — Fuck... stop it... there are bank accounts numbers on those forms... Where's that API even located? What company?
* read their privacy policy *
"We can not guarantee the safety of your personal data, use at your own risk [...] we are located in Russia".
I fucking hate these millennial devs who literally fail to get their head out of the cloud.
Somehow they think it's easier to write all these NodeJS handlers and layers around some API, which probably just calls ImageMagick + Tesseract on the other side.
If I wasn't so fucking exhausted, I'd chop of their heads... but they're like hydra, you seal one privacy breach and another is waiting to be merged, these kids just keep spewing their crap into easy packages, they keep deploying shitty heroku apps... ugh.
😖8 -
Boss comes in and gives me some js code for syncing data (he hacked it together the other day, really messy with like 5 callback lamdas stacked into each other)
Boss: Make it faster and more reliable and add some progress indicator
So i look at the code and he literally pulls all the data as one json (20+ MiB). Server needs multiple minutes to generate the response (lots of querys), sometimes even causing timeouts....
So i do what everyone would do and clean up the code, split the request into multiple ones, only fetching the necessary data and send the code back to my boss.
He comes in and asks me what all this complexity is about. And why i need 5 functions to do what he did in one. (He didn't -.-). He says he only told me to "make it faster and show progress" not "to split everything up".
So I ask him how he wants to do this over HTTP with just one request...
His response: "I don't care make it work!".
Sometimes i hate my job -.-11 -
"I couldn't fix the test so I commented it out."
"I removed build timeouts because our jobs started taking that long."
Next I'm waiting for "Compiling the code is good enough we don't need tests" before I lose it...1 -
I had spent the last year working on a online store power by woocommerce with over 100k products from various suppliers. This online store utilized a custom API that would take the various formats that suppliers offer their inventory in and made them consistent. Now everything was going swimmingly initially, but then I began adding more and more products using a plug-in called WP all import. I reached around 100k products and the site would take up to an entire minute to load sometimes timing out. I got desperate so I installed several caching plugins, but to no avail this did not help me. The site was originally only supposed to take three to four months but ended up taking an entire year. Then, just yesterday I found out what went wrong and why this woocommerce website with all of these optimizations was still taking anywhere from 60 to 90 seconds to load, or just timing out entirely. I had initially thought that I needed a beefier server so I moved it to a high CPU digitalocean VM. While this did help a little bit, the site was still very slow and now I had very high CPU usage RAM usage and high disk IO. I was seriously stumped the Apache process was using a high amount of CPU and IO along with MYSQL as well. It wasn't until I started digging deeper into the database that I actually found out what the issue was. As I was loading the site I would run 'show process list' in the SQL terminal, I began to notice a very significant load time for one of the tables, so I went to go and check it out. What I did was I ran a select all query on that particular table just to see how full it was and SQL returned a error saying that I had exceeded the maximum packet size. So I was like okay what the fuck...
So I exited my SQL and re-entered it this time with a higher packet size. I ran a query that would count how many rows were in this particular table and the number came out to being in the millions. I was surprised, and what's worse is that this table belong to a plugin that I had attempted to use early in the development process to cache the site. The plugin was deactivated but apparently it had left PHP files within the wp content directory outside of the actual plugin directory, so it's still executing scripts even though the plugin itself was disabled. Basically every time I would change anything on the site, it would recache the whole thing, and it didn't delete any old records. So 100k+ products caching on saves with no garbage collection... You do the math, it's gonna be a heavy ass database. Not only that but it was serialized data, so when it did pull this metric shit ton of spaghetti from the database, PHP then had to deserialize it. Hence the high ass CPU load. I had caching enabled on the MySQL end of things so that ate the ram. I was really desperate to get this thing running.
Honest to God the main reason why this website took so long was because the load times made it miserable to work on. I just thought that the hardware that I had the site on was inadequate. I had initially started the development on a small Linux VM which apparently wasn't enough, which is why I moved it to digitalocean which also seemed to not be enough, so from there I moved to a dedicated server which still didn't seem to be enough. I was probably a few more 60-second wait times or timeouts from recommending a server cluster to my client who I know would not be willing to purchase it. The client who I promised this site to have completed in 3 months and has waited a year. Seriously, I would tell people the struggles that I would go through with this particular site and they would just tell me to just drop the site; just take the money, just take the loss. I refused to, this was really the only thing that was kicking my ass. I present myself as this high-and-mighty developer like I'm just really good at what I do but then I have this WordPress site that's just beating the shit out of me for a year. It was a very big learning experience and it was also very humbling as well, it made me realize that I really don't know as much as I think I might. It was evidence that there is still so much more to learn out there, I did learn a lot from that experience especially about optimizing websites the different types of methods to do that particular lonely on the server side and I'll be able to utilize this knowledge in the future.
I guess the moral of the story is, never really give up. Ultimately things might get so bad that you're running on hopes and dreams. Those experiences are generally the most humbling. Now I can finally present the site that I am basically a year late on to the client who will be so happy that I did not give up on the project entirely. I'll have experienced this feeling of pure euphoria, and help the small business significantly grow their revenue. Helping others is very fulfilling for me, even at my own expense.
Anyways, gonna stop ranting. Running out of characters. If you're still here... Ty for reading :')7 -
Want to make someone's life a misery? Here's how.
Don't base your tech stack on any prior knowledge or what's relevant to the problem.
Instead design it around all the latest trends and badges you want to put on your resume because they're frequent key words on job postings.
Once your data goes in, you'll never get it out again. At best you'll be teased with little crumbs of data but never the whole.
I know, here's a genius idea, instead of putting data into a normal data base then using a cache, lets put it all into the cache and by the way it's a volatile cache.
Here's an idea. For something as simple as a single log lets make it use a queue that goes into a queue that goes into another queue that goes into another queue all of which are black boxes. No rhyme of reason, queues are all the rage.
Have you tried: Lets use a new fangled tangle, trust me it's safe, INSERT BIG NAME HERE uses it.
Finally it all gets flushed down into this subterranean cunt of a sewerage system and good luck getting it all out again. It's like hell except it's all shitty instead of all fiery.
All I want is to export one table, a simple log table with a few GB to CSV or heck whatever generic format it supports, that's it.
So I run the export table to file command and off it goes only less than a minute later for timeout commands to start piling up until it aborts. WTF. So then I set the most obvious timeout setting in the client, no change, then another timeout setting on the client, no change, then i try to put it in the client configuration file, no change, then I set the timeout on the export query, no change, then finally I bump the timeouts in the server config, no change, then I find someone has downloaded it from both tucows and apt, but they're using the tucows version so its real config is in /dev/database.xml (don't even ask). I increase that from seconds to a minute, it's still timing out after a minute.
In the end I have to make my own and this involves working out how to parse non-standard binary formatted data structures. It's the umpteenth time I have had to do this.
These aren't some no name solutions and it really terrifies me. All this is doing is taking some access logs, store them in one place then index by timestamp. These things are all meant to be blazing fast but grep is often faster. How the hell is such a trivial thing turned into a series of one nightmare after another? Things that should take a few minutes take days of screwing around. I don't have access logs any more because I can't access them anymore.
The terror of this isn't that it's so awful, it's that all the little kiddies doing all this jazz for the first time and using all these shit wipe buzzword driven approaches have no fucking clue it's not meant to be this difficult. I'm replacing entire tens of thousands to million line enterprise systems with a few hundred lines of code that's faster, more reliable and better in virtually every measurable way time and time again.
This is constant. It's not one offender, it's not one project, it's not one company, it's not one developer, it's the industry standard. It's all over open source software and all over dev shops. Everything is exponentially becoming more bloated and difficult than it needs to be. I'm seeing people pull up a hundred cloud instances for things that'll be happy at home with a few minutes to a week's optimisation efforts. Queries that are N*N and only take a few minutes to turn to LOG(N) but instead people renting out a fucking off huge ass SQL cluster instead that not only costs gobs of money but takes a ton of time maintaining and configuring which isn't going to be done right either.
I think most people are bullshitting when they say they have impostor syndrome but when the trend in technology is to make every fucking little trivial thing a thousand times more complex than it has to be I can see how they'd feel that way. There's so bloody much you need to do that you don't need to do these days that you either can't get anything done right or the smallest thing takes an age.
I have no idea why some people put up with some of these appliances. If you bought a dish washer that made washing dishes even harder than it was before you'd return it to the store.
Every time I see the terms enterprise, fast, big data, scalable, cloud or anything of the like I bang my head on the table. One of these days I'm going to lose my fucking tits.10 -
We have to use a 20 year old API that is half assed and doesn't even work right every time.
Every three months the same discussion comes up why something doesn't work that relies on that API. I have to explain the situation over and over again... And then my boss starts to give 'solutions' which we already use or are utterly stupid... >.<
In case someone is wondering: SOAP API on a Windows Server 2003 with timeouts every few minutes and XML output in a language that is not English (even the tags!).3 -
Why would anyone use CPU ticks for connection timeouts? Everytime that software is used in a new CPU it gets messed up. Why not just milliseconds?
-
The system I'm developing is depending on an image generating application, that is generating thousands of unique images per day. The kicker? The application is a single thread wpf application hosted on two desktop computers, behind a load balancer, in our server room... because 'it can't be converted to a service'...1
-
120 people on a conference call can't figure out why an half of the country is having connection timeouts3
-
always put timeouts on your connection code..number of times I see shit blocked forever ..and we know it’s not good to have your shit blocked5
-
/** Null until this web socket is connected. Used for writes, pings, and close timeouts. */
private ScheduledExecutorService executor;
Dear boys and girls.
If you ever do this again and release this as a public library (even better - an official client of your solution, e.g. kuber-fucking-netes), I will get my way into dR's gateway servers, trace down your IP in nginx's logs, find your location, probably use some means to get your first and last name (you prolly have a domain registered under your IP anyways...), buy a ticket to your town, get to your home and wait for night to fall. Once it's dark and you're asleep, I'll make sure to leave a real nice, warm and extraordinarily smelly turd on your doorstep (I'll also make sure the process of manufacturing that gem is as noisy as it gets - you just have to bend the right way, and....).
Gents. If you really, REALLY want to make writes asynchronous, at least provide a way to either get a notification once the write is synchronized, or allow the user to handle the threads/executors himself!
https://youtube.com/watch/...
5 -
working against a database over a bugged VPN connection, which gives you periodically timeouts, after the request takes 15 minutes 😑
-
!rant (Feature idea)
Rant Rooms!
Each room has a topic as it's title, and this room acts like a group chat. Individuals can join public rooms and rant about the specified topic. Or, private rooms can be created and the creator can invite people to this room.
Perhaps timeouts could be set so that after a specified amount of time without activity, the room expires and disappears.
Probably a shit idea but figured I'd mention it 😂4 -
While waiting for internet to load on crappy bandwidth, the wait can burn you out just as easily. Including response timeouts...
-
Anyone else getting 504 timeouts on devRant or It just me.....
Or maybe it's that giant image in the feed taking forever to load 🧐4 -
Not a productive day. I wanted to restore part of database from test server but downloading data using reat api was getting timeouts.
I made simple app that dumps data and I launched process in screen and started watching Dragon ball
I like those old first series with young Songo looking for his grandpa.
After about 2 hours and partial backup still not finished I started to cut crap data from backup script will finish it tomorrow.
Looks like my ux is still ok and views got almost all approved.
Need to fix some shit on backend but I need those backups to work for more complicated customers.
Luckily frontend developer is back so he will handle visual parts.
Maybe we will release new features on the beginning of February.3 -
Keep going into application level timeouts and insufficient number of xml messages frames transmitted through tcp sockets.
Found that a "custom" C-written xml parser needs 10 seconds to find the core data within 2500 chars of that xml.
I knew that C is blazing fast but that's until it's used by some dick head.2 -
After three months of development, my first contribution to the client is going live on their servers in less than 12 hours. And let me say, I shall never again be doing that much programming in one go, because the last week and a half has been a nightmare... Where to begin...
So last Monday, my code passed to our testing servers, for QA to review and give its seal of approval. But the server was acting up and wouldn't let us do much, giving us tons of timeouts and other errors, so we reported it to the sysadmin and had to put off the testing.
Now that's all fine and dandy, but last Wednesday we had to prepare the release for 4 days of regression testing on our staging servers, which meant that by Wednesday night the code had to be greenlight by QA. Tuesday the sysadmin was unable to check the problem on our testing servers, so we had to wait to Wednesday.
Wednesday comes along, I'm patching a couple things I saw, and around lunch time we deploy to the testing servers. I launch our fancy new Postman tests which pass in local, and I get a bunch of errors. Partially my codes fault, partially the testing env manipulating server responses and systems failing.
Fifteen minutes before I leave work on the day we have to leave everything ready to pass to staging, I find another bug, which is not really something I can ignore. My typing skills go to work as I'm hammering line after line of code out, trying to get it finished so we can deploy and test when I get home. Done just in time to catch the bus home...
So I get home. Run the tests. Still a couple failures due to the bug I tried to resolve. We ask for an extension till the following morning, thus delaying our deployment to staging. Eight hours later, at 1AM, after working a full 8 hours before, I push my code and leave it ready for deployment the following morning. Finally, everything works and we can get our code up to staging. Tests had to be modified to accommodate the shitty testing environment, but I'm happy that we're finally done there.
Staging server shits itself for half a day, so we end up doing regression tests a full day late, without a change in date for our upload to production (yay...).
We get to staging, I run my tests, all green, all working, so happy. I keep on working on other stuff, and the day that we were slated to upload to production, my coworkers find that throughout the development (which included a huge migration), code was removed which should not have. Team panics. Everyone is reviewing my commits (over a hundred commits) trying to see what we're missing that is required (especially legal requirements). Upload to production is delayed one day because of this. Ended up being one class missing, and a couple lines of code, which is my bad (but seriously, not bad considering I'm a Junior who was handed this project as his first task at his first job).
I swear to God, from here on out, one feature per branch and merge request. Never again shall I let this happen. I don't even know why it was allowed to happen, it breaks our branch policies. But ohel... I will now personally oppose crap like this too...
Now if you'll excuse me... I'm going to be highly unproductive and rest, because I might start balding otherwise after these weeks... -
Is HackerRank a good site to prepare for technical interviews? And I guess the general question is how to best prepare?
The problems are interesting but it seems most of the Medium+ ones require knowledge of a specific approach or the large test cases will terminate with timeouts or out of memory.
Been sitting on this for a week. Just implemented the recursive versionwhich is better but now times out.
https://hackerrank.com/challenges/...6 -
Compare and harmonize the web configs
Oh no someone set execution timeouts to 14 days
Fuck fuck fuckity duck
Hey compare all the web configs of all environments and harmonize them all wtf cmon bruh do your job as a developer
Take them and back them up into svn. What do you mean svn isn't a back up system of course it is well its the only thing we have fuck
What do you mean we have shit logging where people will catch an exception and only print the word exception in the log you can figure it out can't you we have live produxtion issues that hace to be solved now what the fuck
How dare you make a. Mistake copying our shitload of a bloated codebase and configuring our 100s of different options all by fukcing hand what the fuck dude do yoh write anyrhing down?
Please catalogue all the exception mails we are getting but we have no db or error reporting system so they all just plop into tue inbox and thats all ypur fuckjng data figure it out kid
This is a rewarding, fulfilling job whwrw you can be both dev ops and a developer and manage all of our fucking environments of which there are about 15 of all your own with no sort of tool or software to aid you because haha what the fuck we wouldn't make your life easy
Whata that you want to spend time to write stuff or change stuff that will nake it easier fot you fuxk that bruh get back to your biklable tasks like holy shit you thjnk this is a charity ofr aomw shit
Live production issues
Live production issues
Produxtion issues. A ghost in the machine. Find it fix if find it fix it find it fix it cmon why can't you fix it I expect you to spend your day hopelessly pretending to try to solve something you fucker
One of the only peopel able to help you sometimes though hes a bit of an old laxky, yeah hea fucking leaving see ya seeya kid and now we're not hirinf anyone to fuckjng help you no no no managing and monitoring the environments its your jov alll fof them every sngle on do you knkw all the xonfiguraiton values for them yet??
Instead we are hiring a new sales person to fucking make us some more money and we don't need naother seceloper to help you infqct lets have you use this mid end retail computer from 2014 to develop on yeah yeah oh but all our shitty code and visual studip will destry your memory but too bad!! Hahahahahdhsj
Go lice is all you, why sare you so slow
How long will it take
How long will it take
How long will it take
How long witll it tqk2
How long will it take holy shit
Give time estimate for sonethign that I don't fucking know how about it will tqke till fuxk you oxloxk4 -
Follow up sorta...
So I got pulled into a support issue on a day off. Some system was facing timeouts on our servers so had to investigate.
Over the weekend as part of the release, I released the ELK stack I built and today I used that to help.
Pretty much immediately pinpointed which machine was hanging though still had to investigate and confirm so split between KQL and checking the server logs.
One thing I've always griped about is how no one created schema docs for it mongo collections so can't easily figure out what they do or your to get the document needed.
Well guess it's my turn.... Because only I know the schema :) -
I vaguely remember some joke about how difficult networking is and how some Jeff Atwood blog post I think makes the comparison about analogy of sending actors in a taxi to somewhere being compared to a packet, anything can happen inbetween but you will get the packet or something indistinguishable from the original at the other end if all goes well.
Are occasional/intermittent 503 service unavailable or 504 gateway timeouts unavoidable for microservices calling another external microservice?
Like at that point isn't receiving a 503 or 504 from something else, somewhat outside my jurisdriction, albeit I am fucked if I am depending on them and need to fail gracefully.3 -
The application I work on starts throwing timeout errors for about every third user. Lead developer cannot figure out what happened. DBA is out of town and cannot be reached. I do a quick Google search and run the stored procedure sp_updatestats. Timeouts stop and there is a big performance boost on the application. Everyone congratulates me on fixing the problem, and now I'm reading up on MS SQL Server Statistics and wondering about what other magical tools everyone else knows about that me and my team are clueless on...
Top Tags
Weekly Rant
View