Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "service is unavailable"
-
This just in everyone...
Android Dev: *sent and email to network admin* can you please unblock github for a few mins.
Network admin: *Replied* Can you take a screenshot whats the error your getting.
Android Dev: *Replied with screenshot* "Failed to load resource: the server responded with a status of 503 (Service Unavailable)"
Network admin: that is a known issue. *Replied with Wordpress Links.
Android Dev: why is github working outside our network then?
Network admin: there must be a problem with your code that needs to be tweaked.
Team: *FACE PALM*5 -
Most times I've my phone's location switched off. Every time I open some random app, which doesn't require any location service, a toast pops up saying location unavailable. This is simply bonkers.3
-
Issue or Error? Rant story time!
I was working on a windows desktop app, and everything was ok, you know, tests completed succesfully, all in time, etc. The problem was when we showed the demo to one of our clients. He saw several screens and we explained all the features to him.
Client: *Sees a Error pop-up indicating that a remote service is temporarly unavailable (what it has to happen in order to show him how the system would warn him when an external service is out of service)
Client: What’s that?
Lead dev: What do you mean?
Client: Yes. That’s an error pop-up
Lead dev: Yes, it’s a message that tells you that there was an issue connecting to the bank service
C: No, no and no. Please change it
L: Why? Don’t you want the system to tell you when there is a connection issue and why is that happening?
C: Yes, but my employees could lost their minds because of this class of messages!
L: So...?
C: You have to change it
L: Ok. What do you want to change?
C: First of all, don’t put an “Error” icon, put an “Warning” icon, and instead of “Error” title, put... “Issue”
L: “Issue?”
C: Yeah. Don’t put the “E” word, if the users see an “Error” message, they could think that the program doesn’t work, even if it does work.
We all though “WTF?!”
To make the story shorter, we changed all the pop-ups. That took two days.
Is that correct? I know that “Error” sounds hard but, seriously? “Issue: The remote service is not available, contact your bank?”rant wtf brain software development wtf is going on wtf? story time windows problems wtf wtf are you doing!6 -
I've just noticed something when reading the EU copyright reform. It actually all sounds pretty reasonable. Now, hear me out, I swear that this will make sense in the end.
Article 17p4 states the following:
If no authorisation [by rightholders] is granted, online content-sharing service providers shall be liable for unauthorised acts of communication to the public, including making available to the public, of copyright-protected works and other subject matter, unless the service providers demonstrate that they have:
(a) made best efforts to obtain an authorisation, and
(b) made, in accordance with high industry standards of professional diligence, best efforts to ensure the unavailability of specific works and other subject matter for which the rightholders have provided the service providers with the relevant and necessary information; and in any event
(c) acted expeditiously, upon receiving a sufficiently substantiated notice from the rightholders, to disable access to, or to remove from, their websites the
notified works or other subject matter, and made best efforts to prevent their future uploads in accordance with point (b).
Article 17p5 states the following:
In determining whether the service provider has complied with its obligations under paragraph 4, and in light of the principle of proportionality, the following elements, among others, shall be taken into account:
(a) the type, the audience and the size of the service and the type of works or other subject matter uploaded by the users of the service; and
(b) the availability of suitable and effective means and their cost for service providers.
That actually does leave a lot of room for interpretation, and not on the lawmakers' part.. rather, on the implementer's part. Say for example devRant, there's no way in hell that dfox and trogus are going to want to be tasked with upload filters. But they don't have to.
See, the law takes into account due diligence (i.e. they must give a damn), industry standards (so.. don't half-ass it), and cost considerations (so no need to spend a fortune on it). Additionally, asking for permission doesn't need to be much more than coming to an agreement with the rightsholder when they make a claim to their content. It's pretty common on YouTube mixes already, often in the description there's a disclaimer stating something like "I don't own this content. If you want part of it to be removed, get in touch at $email." Which actually seems to work really well.
So say for example, I've had this issue with someone here on devRant who copypasted a work of mine into the cancer pit called joke/meme. I mentioned it to dfox, didn't get removed. So what this law essentially states is that when I made a notice of "this here is my content, I'd like you to remove this", they're obligated to remove it. And due diligence to keep it unavailable.. maybe make a hash of it or whatever to compare against.
It also mentions that there needs to be a source to compare against, which invalidates e.g. GitHub's iBoot argument (there's no source to compare against!). If there's no source to compare against, there's no issue. That includes my work as freebooted by that devRant user. I can't prove my ownership due to me removing the original I posted on Facebook as part of a yearly cleanup.
But yeah.. content providers are responsible as they should be, it's been a huge issue on the likes of Facebook, and really needs to be fixed. Is this a doomsday scenario? After reading the law paper, honestly I don't think it is.
Have a read, I highly recommend it.
http://europarl.europa.eu/doceo/...13 -
Stakeholder: Users are unable to buy tickets on the website. IT says Azure’s health check is showing an unhealthy status.
[It’s Sunday. Web Engineering is not on call so no one sees this right away.]
Stakeholder: IT restarted the Azure website twice, but users still can’t place orders.
Me: There was never an issue with the Azure site. That health check is inaccurate. There is a rewrite rule that sends the Azure supplied domain to our custom domain. The Azure health check doesn’t like that so it returns an unhealthy status. The problem is the ticketing server that the website has to communicate with. The ticketing server is overwhelmed and can’t handle more requests. IT should have checked the ticketing server’s logs. This has happened before and it’s never been an Azure issue. It’s a ticketing server issue.
Stakeholder and IT: Oops 😅
—-
JFC. Stop trying to make this web engineering’s problem. Stop trying to make it look like engineering dropped the ball. The ticketing server has experienced this issue multiple times. The ticketing server is maintained by a different team. The website’s symptoms are always the same and there are steps you need to take before you make the decision to restart the website, which will cause the website to show a blue screen of death that says 503 service unavailable for a few minutes. And we have a switch to shut off all transactions. Why do you not want to use it when it’s clear the website can’t process transactions???3 -
I've read the docs but my tired brain overrided an important detail.
https://haproxy.com/documentation/...
"By default, HAProxy Enterprise will serve these pages only if it initiated the error itself. For example, it will return the page for a 503 Service Unavailable error if it can't reach any backend servers."
I had _the_ return part for interception of the error page from the backend added, not the default override for the error page of HAPRoxy itself.
Took me 4 hours, crying, madness and screaming to realize it.
This week is really wringing the last bits of the gooey slime what should be my brain out...
-.-
Another fun part is that I mistakenly thought the delimiter for multiple strings to an ACL comparison is a comma... It's a whitespace.
acl is_evil hdr(host) -i one,two is wrong.
acl is_evil hdr(host) -i one two is right.
I used to write HAPRoxy configurations blindly, today it was more like writing two lines of codes 100000000 times and still doing it wrong TM.
I need new brain.
Anyone got an offer?3 -
In last episode of "How SystemD screwed me over", we talked about Systemd's PrivateTMP and how it stopped me from generating SSL certificates.
In today's episode - SystemD vs CGroups!
Mister Pottering and his team apparently felt that CGroups are underused (As they can be quite difficult to set up), and so decided to integrate them into SystemD by default. As well as to provide a friendlier interface to control their values.
One can read about these interactions in the manual page "systemd.resource-control"
All is cool so far. So what happened to me today?
Imagine you did a major system release upgrade of a production server, previously tested on a standalone server. This upgrade doesn't only upgrade the distribution however, it also includes the switch from SysVInit to SystemD. Still, everything went smooth before, nothing to worry now then, right? Wrong.
The test server was never properly stress-tested. This would prove to be an issue.
When the upgrade finishes, it is 4 AM. I am happy to go to bed at last. At 6 AM, however, I am woken up again as the server's webservices are unavailable, and the machine is under 100% CPU load. Weird, I check htop and see that Apache now eats up all 32 virtual cores. So I restart it, casting it off to some weird bug or something as the load returns to normal.
2 hours later, however, the same situation occurs. This time, I scour all the logs I can, and find something weird - Many mentions that Apache couldn't create a worker thread? That's weird.
Several hours of research and tinkering later, I found out the following:
1 - By default, all processes of a system that runs SystemD are part of several CGroups. One of these CGroups is the PID CGroup, meant to stop a runaway process from exhausting all PIDs/TIDs of a system.
This limit is, by default, set to a certain amount of the total available PIDs. If a process exhausts this limit, it can no longer perform operations like fork().
So now, I know the how and why, but how should I solve this? The sanest option would be to get a rough estimate of just how many threads the Apache webserver might need. This option, though, is harder, than apparent. I cannot just take the MaxRequestsWorkers number... The instance has roughly double the amount of threads already. The cause being, as I found out, the HTTP/2 module, which spawns additional threads that do not count towards this limit. So I have no idea what limit to set.
Or I could... Disable the limit for just the webserver via the TasksAccounting switch. I thought this would work. And it did seem to... Until I ran out of TIDs again - Although systemctl status apache2.service no longer reported the number of tasks or a task limit of the process, the PID CGroup stayed set to the previous limit. Later I found out that I can only really disable the Task Accounting for all the units of a given slice and its parents.
This, though, systemctl somewhat didn't make apparent (And I skimmed the manual, that part was my fault)
So... The only remaining option I had was to... Just set the limit to infinite. And that worked, at last.
It took me several hours to debug this issue. And I once again feel like uninstalling systemd again, in favor of sysvinit.
What did I learn? RTFM, carefully, everything is important, it is not enough to read *half* the paragraph of a given configuration option...
Oh, and apache + http/2 = huge TID sink. -
anyone else get second thoughts about their bank when they see this kind of message after logging in?
"Online Banking is unavailable
We are really sorry we are preventing you from completing your banking today. Our technical teams are aware and actively investigating. Please know they are working hard to restore service as soon as possible.
Please call us at 123-456-7890 for banking assistance or information."
er... what?2 -
I'm following this fucking tutorial (https://blog.ssdnodes.com/blog/...) and everything goes well, I have docker running, docker compose installed properly, but when I start trying to create the docker-compose.yml and accessing the stupid site using the virtual host domain i set I can't it keep getting "503 Service Temporarily Unavailable" or "502 Bad gateway" what the hell am i doing wrong, I just want to get this working in my VM so i can move it to my damn server and have my own fucking cloud. This damn bullshit is exactly why i went into programming rather than dealing with configuring servers and bullshit like this i know it's outside my level of understanding but I really fucking want my own cloud system but I want it containerized for both isolation and learning purposes.
I have no idea what the hell i'm doing wrong and all the damn articles and links i'm reading aren't helping at all with my level of stupid not allowing me to understand what i'm doing wrong1 -
3 weeks into getting dragged into another MS-stack project, I have already been repeatedly reminded why I decided to invest so much energy in moving my career as far away from Microsoft dependency as possible. Even something as simple as reviewing settings on Azure App Service is a hit-or-miss affair, being completely unavailable for hours at a time at least once a week. Azure Functions are consistently unavailable for at least a few minutes every day. Don't even get me started on "Azure DevOps".
Why the fuck do people still place their trust (and critical infrastructure) in the hands of Microsoft?4 -
So the saga of broken fucking everything continues at work, and I'm managing it, effectively, and doing it correctly on the first go-round. It's a long process though, because the two retards who preceded me were equally inept for completely different, yet equally disruptive and destructive reasons. The first dude was just plain psychotic, probably still is. I'd post some of his code, but I don't want anyone's face to melt off like those Nazi dudes at the end of Raiders of the Lost Ark. I can handle it because I'm constantly inebriated, which is not as fun as it sounds. If you have to ask yourself if you can handle it, you probably aren't, unless you've had to Uber to/from work due to still being fucking drunk. Anyway, enough about that, and it was only like twice. The rest of the times, I was more blazed than Jerry Garcia at a weed smoking contest. Moving along.
UPS shipping labels broke two weeks ago, I fixed it, but these fucking 10xers jointly decided to not only never implement anything resembling error handling, other than EMPTY GOD DAMN "try/catch"es (empty catch, wow so efficient), and instead of using COMMENTS, which I know are a new thing, they'd wrap blocks of code in something like: if 1 = 0 {} FUCK YOU DICKFACES. As I was saying before I got emotional again, they tied the success to all kinds of unrelated, irrelevant shit. I'm literally needle/haystacking my way through the entire 200GB codebase, ALONE, trying to find all the borked things. Helpfully, my phone is ringing all the time from customer service, complaining about things that are either nothing to do with the site, or due to user stupidity, 75% of the time.
A certain department at my company relies on some pretty specific documents to do their job, and these documents are/were generated from data in the database. So until I can find and fix all of the things, I've diverted my own attention as much as possible to the rapid implementation of a report generation microservice so that no one elses work is further disrupted while I continue my cursed easter egg hunt from fucking hell.
After a little more than two days, I'm about to lauch a standalone MS to handle the reports, and it's unfortunately more complicated than I'd like, because it requires a certain library that isn't available on Winblows, so I've dockerized the application. Anyway, just after lunch, I've finished my final round of tests, and I'm about ready to begin migrating it to the server and setting up (shitty fucking shit) IIS to serve it appropriately. At this point, this particular report has been unavailable by web for about 8 days.
A little after lunch, and with no forewarning of any kind, the manager of managers runs upstairs and screams at me to "work faster" and that "this needs to be back online RIGHT NOW", but I also know that this individual is going to throw a fit if things on this pdf aren't a pixel perfect match. So I just say "that's some amazing advice, I wish I'd had the foresight to just do it better and work faster". Silence for a good five seconds, then I follow up with "please leave and let me get back to my work". At that moment from around the corner, my "supervisor" suddenly, magically even, remembers that he has had the ability to print this crucial, amazingly super fucking important document all along, despite me directly asking him a week ago, and he prints it and takes it where it needs to go. In the time that it takes him to go to that other department and return, I deploy my service.
I spent the rest of the day browsing indeed and linkedin jobs, but damn this market is kinda weird right now, yeah?2 -
i am feeling angry and frustrated. not sure if it's a person ,or codebase or this bloody job. i have been into the company for 8 months and i feel like someone taking a lot of load while not getting enough team support to do it or any appreciation if i do it right.
i am not a senior by designation, but i do think my manager and my seniors have got their work easy when they see my work . like for eg, if on first release, they told me that i have to update unit tests and documentation, then on every subsequent release i did them by default and mentioning that with a small tick .
but they sure as hell don't make my work easy for me. their codebase is shitty and they don't give me KT, rather expect me to read everything on my own, understand on my own and then do everything on my own, then raise a pr , then merge that pr (once reviewed) , then create a release, then update the docs and finally publish the release and send the notification to the team
well fine, as a beginner dev, i think that's a good exercise, but if not in the coding step, their intervention would be needed in other steps like reviewing merging and releasing. but for those steps they again cause unnecessary delay. my senior is so shitty guy, he will just reply to any of my message after 2-3 hours
and his pr review process is also frustrating. he will keep me on call while reviewing each and every file of my pr and then suggest changes. that's good i guess, but why tf do you need to suggest something every fucking time? if i am doing such a shitty coding that you want me to redo some approach that i thought was correct , why don't you intervene beforehand? when i was messaging you for advice and when you ignored me for 3 hours? another eg : check my comment on root's rant https://devrant.com/rants/5845126/ (am talking about my tl there but he's also similar)
the tasks they give are also very frustrating. i am an android dev by profession, my previous company was a b2c edtech app that used kotlin, java11, a proper hierarchy and other latest Android advancements.
this company's main Android product is a java sdk that other android apps uses. the java code is verbose , repetitive and with a messed up architecture. for one api, the client is able to attach a listener to some service that is 4 layers down the hierarchy , while got other api, the client provides a listener which is kept as a weak reference while internal listeners come back with the values and update this weak reference . neither my team lead nor my seniors have been able to answer about logic for seperation among various files/classes/internal classes and unnecessary division of code makes me puke.
so by now you might have an idea of my situation: ugly codebase, unavailable/ignorant codeowners (my sr and TL) and tight deadlines.
but i haven't told you about the tasks, coz they get even more shittier
- in addition to adding features/ maintaining this horrible codebase , i would sometimes get task to fix queries by client . note that we have tons of customer representatives that would easily get those stupid queries resolced if they did their job correctly
- we also have hybrid and 3rd party sdks like react, flutter etc in total 7 hybrid sdks which uses this Android library as a dependency and have a wrapper written on its public facing apis in an equally horrible code style. that i have to maintain. i did not got much time/kt to learn these techs, but once my sr. half heartedly explained the code and now every thing about those awful sdls is my responsibility. thank god they don't give me the ios and web SDK too
- the worst is the shitty user side docs. I don't know what shit is going there, but we got like 4 people in the docs team and they are supposed to maintain the documentation of sdk, client side. however they have rasied 20 tickets about 20 pages for me to add more stuff there. like what are you guys supposed to do? we create the changelog, release notes , comments in pr , comments in codebase , test cases, test scenarios, fucking working sample apps and their code bases... then why tf are we supposed to do the documentation on an html based website too?? can't you just have a basic knowledge of running the sample, reading the docs and understand what is going around? do i need to be a master of english too in addition to being a frustrated coder?
just.... fml -
Just deployed my Nodejs app but some routes are returning *503 Service Unavailable*.
And others are working just fine
Is there a special way to name routes when on production?
Because everything was working fine on localhost
Thanks for your help1 -
I vaguely remember some joke about how difficult networking is and how some Jeff Atwood blog post I think makes the comparison about analogy of sending actors in a taxi to somewhere being compared to a packet, anything can happen inbetween but you will get the packet or something indistinguishable from the original at the other end if all goes well.
Are occasional/intermittent 503 service unavailable or 504 gateway timeouts unavoidable for microservices calling another external microservice?
Like at that point isn't receiving a 503 or 504 from something else, somewhat outside my jurisdriction, albeit I am fucked if I am depending on them and need to fail gracefully.3