Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "scaling"
-
Good news: Today my app reached 1 million daily users. 😃😃😃💰
Bad news: It started out as a side project and my shitty code is not scaling well at all. 🙃19 -
Angry customer (😤): Your software is still too slow!!!
Me (🙄): It's running good all in all. Let's divide this into smaller aspects. Which steps do you need to perform faster?
😤: Every step needs to be as fast as google!
🙄: But our software ain't google, not even to mention your infrastructure
😤: Everything needs to run in 2 seconds!!!
🙄: You aren't helping in any way. We need something to grasp...
😤: It's all your fault.
📴10 -
"Let's go for the low hanging fruit first" 🤢
"I think we should do some market research" 🤢
"Yeah that is also on my radar" 🤢
OKAY YOU FUCKING CUNT, STOP WITH YOUR PATRONIZING SHIT.
FIRST OF ALL, ARE YOU REALLY SO SMALL MINDED THAT YOU CAN'T REACH ANY HIGHER? THIS TREE IS FUCKING RAW AMD BARE ON THE BOTTOM, WITH YOUR FILTHY CLAWS GRASPING FOR ALL THOSE EASY NARROW FIXES.
SECONDLY, A FUCKING EMAIL SURVEY WITH BIASED QUESTIONS ANSWERED BY 3 HOBOS IS NOT BLOODY MARKET RESEARCH.
THIRDLY, IF THIS NUCLEAR ICBM OF AN INFRASTRUCTURE PROBLEM IS ON YOUR RADAR, MAYBE STOP FONDLING YOUR SWEATY BALLS FOR A MINUTE AND TAKE ACTION.
"Okay lets peel this onion, so we hit the ground running" 😩🤢😞
NO, LET'S NOT "HIT THE GROUND RUNNING", YOU'RE GOING TO FUCKING TRIP AND MESS UP YOUR FACE EVEN MORE. HOW ABOUT GET YOUR PILE OF SHIT IN ORDER FOR ONCE, PREPARE FOR A MEETING? HOW ABOUT THOUGHTFUL ACTION, SOME FEATURE DESIGN?
"No, just implement it quick and dirty" 🤢😡👿
OH YOU WANT IT QUICK AND DIRTY? IS THAT HOW YOU FUCK YOUR DAD AS WELL?
"Let's evaluate the fix in a few weeks. We really had good synergy here team" 🤢😫
YEAH SURE, LET'S EVALUATE THIS.... BUT LET'S EVALUATE IT RIGHT NOW: 😡
"Ahem....
1. You're always late for meetings.
2. After 6 months, you still barely know what we do as a company, you still don't know the teams, and you still don't know the product.
3. You do not listen to engineers flooding you with red flags, requiring time for a redesign to fix serious scaling issues.
4. Everything must be a quickfix, nothing is allowed to require thought, because you CAN ABSOLUTELY NOT think ahead for more than 30 seconds.
OH AND IF YOU EVER AGAIN COVER UP ONE OF YOUR MANY SHORTCOMINGS WITH THAT FUCKING SLIMEY DOUCHEBAG MANAGER VOCABULARY OF YOURS, LET'S SEE HOW MUCH SYNERGY YOU FEEL WHEN YOU'RE DEEPTHROATHING A CACTUS."18 -
Me: Optimize a sort & match method in backend because users complain it's a bit slow.
Coworker: These algorithms are both O(n), so they're identical *closes PR*
Me: *start zoom call* "Heeeeeeeeeey Iiiiiiiiiii wouuuuuuuld liiiiiiiiike toooooo diiiiiisscuuuuus thaaaaaaaat puuuuuuulllll reeeeeequuuueeest yooooouuuuu cloooooossseeeed"
Coworker: "wtf are you doing, why are you talking so slow"
Me: "No matter whether I talk fast or slow, the information still reaches you in O(n) time, so why are you complaining"
I fucking hate it when people misunderstand the purpose of (or abuse) big O notation. It's an estimate of how an algorithm SCALES once the set increases in size, in which case you leave out both less significant terms and constant factors.
But those terms and factors are important when you're talking about the DIRECT PERFORMANCE of the algorithm on fixed-size sets, instead of SCALING to larger sets.
1n and 10n are both O(n), but 10x performance on a job that used to take 10 minutes is still significant.19 -
Biggest scaling challenge?
The imaginary scaling issues from clients.
Client : How do you cope with data that's a billion times bigger than our current data set? Can you handle that? How much longer will it take to access some data then?
I could then give a speech about optimizing internal data structures and access algorithms that work with O(log n) complexity, but that wouldn't help, non-tech people will not understand that.
And telling someone, the system will be outdated and hopefully been replaced when that amount of data is reached, would be misinterpreted as "Our system can not handle it".
So the usual answer is: "No problem, our algorithms are optimized so they can handle any amount of data"6 -
So i heard that docker is now a thing and decided to put my servers into containers as well (see image)...but somehow I still don't understand how is this supposed to simplify deployment, vertical scaling and so on...? :D6
-
Biggest scaling challenge I've faced?
Around 2006~2007 the business was in double-digit growth thanks to the eCommerce boom and we were struggling to keep up with the demand.
Upper IT management being more hardware focused and always threw more hardware at the problem. At its worst, we had over 25 web servers (back then, those physical tall-rectangle boxes..no rack system yet) and corresponding SQL server for each (replicated from our main sql server)
Then business boomed again and projected the need for 40 servers (20 web servers, 20 sql servers) over the next 5 years. Hardware+software costs (they were going to have to tear down a wall in order to expand the server room) were going to be in the $$ millions.
Even though we were making money, the folks spending it didn't seem to care, but I knew this trajectory was not sustainable, so I started utilizing (this was 2007) WCF services and Microsoft's caching framework Velocity. Started out small, product lookup data (description, price, the simple stuff) and within a month, I was able to demonstrate the web site could scale with less than half of our current hardware infrastructure.
After many political battles (I've ranted about a few of those), the $$ won and even with the current load, we were able to scale back to 5 web servers and 2 sql servers. When the business increased in the double-digits again, and again...we were still the same hardware for almost 5 years. We only had to add another service server when the international side of the business started taking off.
Challenge wasn't the scaling issue, the challenge was dealing with individuals who resisted change.3 -
Worst dev team failure I've experienced?
One of several.
Around 2012, a team of devs were tasked to convert a ASPX service to WCF that had one responsibility, returning product data (description, price, availability, etc...simple stuff)
No complex searching, just pass the ID, you get the response.
I was the original developer of the ASPX service, which API was an XML request and returned an XML response. The 'powers-that-be' decided anything XML was evil and had to be purged from the planet. If this thought bubble popped up over your head "Wait a sec...doesn't WCF transmit everything via SOAP, which is XML?", yes, but in their minds SOAP wasn't XML. That's not the worst WTF of this story.
The team, 3 developers, 2 DBAs, network administrators, several web developers, worked on the conversion for about 9 months using the Waterfall method (3~5 months was mostly in meetings and very basic prototyping) and using a test-first approach (their own flavor of TDD). The 'go live' day was to occur at 3:00AM and mandatory that nearly the entire department be on-sight (including the department VP) and available to help troubleshoot any system issues.
3:00AM - Teams start their deployments
3:05AM - Thousands and thousands of errors from all kinds of sources (web exceptions, database exceptions, server exceptions, etc), site goes down, teams roll everything back.
3:30AM - The primary developer remembered he made a last minute change to a stored procedure parameter that hadn't been pushed to production, which caused a side-affect across several layers of their stack.
4:00AM - The developer found his bug, but the manager decided it would be better if everyone went home and get a fresh look at the problem at 8:00AM (yes, he expected everyone to be back in the office at 8:00AM).
About a month later, the team scheduled another 3:00AM deployment (VP was present again), confident that introducing mocking into their testing pipeline would fix any database related errors.
3:00AM - Team starts their deployments.
3:30AM - No major errors, things seem to be going well. High fives, cheers..manager tells everyone to head home.
3:35AM - Site crashes, like white page, no response from the servers kind of crash. Resetting IIS on the servers works, but only for around 10 minutes or so.
4:00AM - Team rolls back, manager is clearly pissed at this point, "Nobody is going fucking home until we figure this out!!"
6:00AM - Diagnostics found the WCF client was causing the server to run out of resources, with a mix of clogging up server bandwidth, and a sprinkle of N+1 scaling problem. Manager lets everyone go home, but be back in the office at 8:00AM to develop a plan so this *never* happens again.
About 2 months later, a 'real' development+integration environment (previously, any+all integration tests were on the developer's machine) and the team scheduled a 6:00AM deployment, but at a much, much smaller scale with just the 3 development team members.
Why? Because the manager 'froze' changes to the ASPX service, the web team still needed various enhancements, so they bypassed the service (not using the ASPX service at all) and wrote their own SQL scripts that hit the database directly and utilized AppFabric/Velocity caching to allow the site to scale. There were only a couple client application using the ASPX service that needed to be converted, so deploying at 6:00AM gave everyone a couple of hours before users got into the office. Service deployed, worked like a champ.
A week later the VP schedules a celebration for the successful migration to WCF. Pizza, cake, the works. The 3 team members received awards (and a envelope, which probably equaled some $$$) and the entire team received a custom Benchmade pocket knife to remember this project's success. Myself and several others just stared at each other, not knowing what to say.
Later, my manager pulls several of us into a conference room
Me: "What the hell? This is one of the biggest failures I've been apart of. We got rewarded for thousands and thousands of dollars of wasted time."
<others expressed the same and expletive sediments>
Mgr: "I know..I know...but that's the story we have to stick with. If the company realizes what a fucking mess this is, we could all be fired."
Me: "What?!! All of us?!"
Mgr: "Well, shit rolls downhill. Dept-Mgr-John is ready to fire anyone he felt could make him look bad, which is why I pulled you guys in here. The other sheep out there will go along with anything he says and more than happy to throw you under the bus. Keep your head down until this blows over. Say nothing."11 -
My team handles infrastructure deployment and automation in the cloud for our company, so we don't exactly develop applications ourselves, but we're responsible for building deployment pipelines, provisioning cloud resources, automating their deployments, etc.
I've ranted about this before, but it fits the weekly rant so I'll do it again.
Someone deployed an autoscaling application into our production AWS account, but they set the maximum instance count to 300. The account limit was less than that. So, of course, their application gets stuck and starts scaling out infinitely. Two hundred new servers spun up in an hour before hitting the limit and then throwing errors all over the place. They send me a ticket and I login to AWS to investigate. Not only have they broken their own application, but they've also made it impossible to deploy anything else into prod. Every other autoscaling group is now unable to scale out at all. We had to submit an emergency limit increase request to AWS, spent thousands of dollars on those stupidly-large instances, and yelled at the dev team responsible. Two weeks later, THEY INCREASED THE MAX COUNT TO 500 AND IT HAPPENED AGAIN!
And the whole thing happened because a database filled up the hard drive, so it would spin up a new server, whose hard drive would be full already and thus spin up a new server, and so on into infinity.
Thats probably the only WTF moment that resulted in me actually saying "WTF?!" out loud to the person responsible, but I've had others. One dev team had their code logging to a location they couldn't access, so we got daily requests for two weeks to download and email log files to them. Another dev team refused to believe their server was crashing due to their bad code even after we showed them the logs that demonstrated their application had a massive memory leak. Another team arbitrarily decided that they were going to deploy their code at 4 AM on a Saturday and they wanted a member of my team to be available in case something went wrong. We aren't 24/7 support. We aren't even weekend support. Or any support, technically. Another team told us we had one day to do three weeks' worth of work to deploy their application because they had set a hard deadline and then didn't tell us about it until the day before. We gave them a flat "No" for that request.
I could probably keep going, but you get the gist of it.4 -
Oh you're a frontend guy? Good, we need one of those.
Oh you're a backend guy too? Good, we need one of those.
Oh you're a security guy too? Good, we need one of those.
Oh you're a devops guy too? Good, we need one of those.
Oh you're a QA guy too? Good, we need one of those.
Oh you're an SEO guy too? Good, we need one of those.
"Well, sorry to say fullStackCraft, but we found your cloud architecture skills just a little too lacking for this position. We really need someone who can do frontend, backend, security audits, QA assessments, SEO, AND build scaling cloud architecture. Oh and while you're at it, can you turn fucking water into gold? We need that at our company too. You didn't get the position, but it'd be great if you could refer us to someone who is very advanced in fucking alchemy. Thanks!"
Absolutely toxic the way software people are treated I swear. The money may be the only good thing that is left.19 -
Being a sysadmin, it's pretty difficult to get around the whole development of front-end stuff.. positioning, scaling, and everything... I hate it. So many ideas but only the ability to make the back-end and if it involves electronics that as well as networking. But building a pretty UI is beyond me... I love hating on all the frameworks and Node, but in all honesty.. front-end people, I kind of envy you 😅6
-
I was tasked with maintaining a handful of chemistry simulations for the web. First things first, replace these outdated links and turn them into Bootstrap buttons that work on mobile. Wait, why do all of these look so horrible on mobile? nothing's scaling properly. *looks at fucking 1,000-line CSS file* The old developers used position:absolute for absolutely EVERYTHING. And I counted one div on the entire page. This is why you don't trust chem engineers to do your "web design" #FuckDreamweaver7
-
A Developer is desperate: his java application servers are unresponsive, thousand of dead zombie threads are sucking all cpus, memory is leaking everywhere, garbage collector has gone crazy, the cluster sessions are fucked....
The Developer goes to the closest bridge, ties a stone to his neck and gets ready to jump.
Suddenly a bearded old man with a fiery look runs toward him, yelling:
- stop stop!!!! Your application is not scaling and misconfigured, your servers are melting, cpu usage is not sustainable anymore, but don't despair
The Developer, puzzled, looks at him:
-I've never seen you...how do you know...
- Hey, man, I'm the Devil. I know everything. All your problems are solved. I'll give you magic functions. They are called Lambda.
You'll never have to worry about your servers, scalability, security, configuration and shit.
The Developer seems astonished but relieved:
- Ok, sounds great! let's try it - suddenly suspicion creeps in - hmmmm but you are the Devil....so...you want something back, don't you?
(the Devil nods lightly with a diabolic smile)
- ...and...you want my soul, I guess...
- your soul??? come on!!! - the Devil burst in a laugh - we are in 2019. I don't care about your soul. I want your ass.
- What!???!!!?
- yes, I want to fuck your ass
The Developer, evaluates quickly the situation.
Few moments of pain or slight discomfort (?) in exchange for magic lambda. It could be worth. He accepts.
After a while of rough anal fucking, the devil asks
- Hey, how old are you anyway?
- 45, why?
- Oh jeeez...45!!!??? and you still believe in the devil?5 -
Scaling a badly-written code which is not designed to scale.
Ended up re-writing the entire shit from scratch.4 -
Well, it happened!
After decades of waiting, it finally happened...
Very early on in my career, I was told that one day, one of your client will ask you to make the logo bigger.
Well, today was that day.
My client didn’t really understand just how much of a negative impact it would have on his site’s navigation by making his always obnoxiously larger ugly icon an additional 10% bigger.
He also failed to understand aspect ratio and forced me to stretch his low/res raster image without scaling it in all 2 dimensions.
He actually used the line from the infamous song “...I don’t want to tel you how to do your job by can you make the logo bigger?”
https://youtube.com/watch/...3 -
1. Fucking MySQL database clusters.
There's nothing fun about MySQL clusters. Sometimes they start producing deadlock errors for no apparent reason... well, there's probably a reason, but it's never a transparent easy to find reason.
What was even less fun is that those errors took down a Sentry server. When your error log server goes down through ddos from your database messages, it's time to rethink your setup.
2. Wiring up a large factory with $2 arduino clones, each with a $2 esp8266 wifi chip, with various sensors for measuring flow of chemical solutions (I wanted cheap real time monitoring as an early warning system next to periodic sampling).
The scaling issue was getting over 500 streaming wifi signals to work in a 55c moist slightly corrosive atmosphere with concrete and steel everywhere, and getting it all into a single InfluxDB instance for analysis.12 -
"We should use ambiguity-driven development using co-domain model sockets, anfractuous horizontal scaling, and recursive async testing"
"What even is that"
"I found some words I like in the dictionary, and used it as the title for a medium article. I thought that's how we play the game as devs these days"6 -
Had to implement an API reporting on exabytes of data, scaling up eventually to a zettabyte.
I'd never even heard of these words until I started on the project.6 -
So I ve been clinically depressed for about 10 years now. Been really great at hiding it. My illness and loneliness was so severe that i made up imaginary friends and that got so severe i couldn't tell what s real and what s not. Then about 5 years ago, i met a girl. As the cliche goes, everything felt better. Sunshine and stuff. I opened up to her. Shared stuff. I started becoming normal. The pain became bearable and manageable. Turned to entrepreneurship. Had goals and stuff. Had 7 failed startups but kept on going. Raised investment for an 8th. It went better than anyother. Was going to become the next big thing bla bla. She became the reason i turned from being a loner weirdo to someone awesome. Anyway, as nothing tends to last, my best friend who had been through thick and thin in my work, quit last year in October. He messed up some work from big client nd we had a fight. He left. In the meantime i scored a big multinational company. I was gonna propose to my girlfriend in March this year. But instead she decided to leave for someone better who left her in 3 weeks lol. Anyways, we broke up. During that time, my second friend decided to fuck up my work with the big company so hard that they were about to blacklist my company. And then he left too. I had a small team. 4 5 people doing their best. By that time, i was the only one left. On 28th feb i had my breakup, on 1st march i was sitting 700 km away from home in an office trying to talk the company out of blacklisting us. It took me around 20 days to make that happen. All the while dealing with the obvious, my depression getting stronger than ever. My imaginations taking shape and fucking up my reality. The voices in my head getting stronget and stronger. 4 months now since she left. I dont think i miss her anymore. She tried coming back once but i didn't let her. In the 4 months, i m at my worst. I am getting government contracts now. But i have no desire to do anything. The pain is unbearable. So much that on its good days it sucks the life right out of me. So much that when it gets severe the urge to harm myself in any way goes of the charts. My best friend and i, we became friends again after my ex left. He s been helping me as much as he can. I have all the good oppurtunities and chances that any entrepreneur who has been busting his ass for 5 years straight would kill to have. But i cant do anything. I m the only one left on my team. I have to handle the business, dev, marketing etc etc ends on my own. I tried hiring and scaling up but i messed that up because of obvious reasons. And now my company has 2 months of runway left. And i know if i bust my ass i can make it to 8 months more and even raise a round a. But its really hard to do when either you re sleeping 20 hrs a day or you re sleeping 3 4 hrs because you re afraid of the nightmares. Or when even you ve had a good day, the pain becomes so much that you lay on the floor having a breakdown. Yeah, i m trying professional help. I m hoping it helps me. Because right now, i dont care about being happy. I just want my sanity. Something i m clinging to with every fiber of my being. Something that s burning out like a candle burning from both ends. I cant give up my work. I dont want to. That s all i have. That s all what i love doing and now i cant even do that. I just want this to end somehow. Either i get better and the pain and the void and silence and everything else goes away, or i do. I dont know what will happen first. And i dont care. I just want to be normal. But i guess that s too much to ask.8
-
Okay, story time.
Back during 2016, I decided to do a little experiment to test the viability of multithreading in a JavaScript server stack, and I'm not talking about the Node.js way of queuing I/O on background threads, or about WebWorkers that box and convert your arguments to JSON and back during a simple call across two JS contexts.
I'm talking about JavaScript code running concurrently on all cores. I'm talking about replacing the god-awful single-threaded event loop of ECMAScript – the biggest bottleneck in software history – with an honest-to-god, lock-free thread-pool scheduler that executes JS code in parallel, on all cores.
I'm talking about concurrent access to shared mutable state – a big, rightfully-hated mess when done badly – in JavaScript.
This rant is about the many mistakes I made at the time, specifically the biggest – but not the first – of which: publishing some preliminary results very early on.
Every time I showed my work to a JavaScript developer, I'd get negative feedback. Like, unjustified hatred and immediate denial, or outright rejection of the entire concept. Some were even adamantly trying to discourage me from this project.
So I posted a sarcastic question to the Software Engineering Stack Exchange, which was originally worded differently to reflect my frustration, but was later edited by mods to be more serious.
You can see the responses for yourself here: https://goo.gl/poHKpK
Most of the serious answers were along the lines of "multithreading is hard". The top voted response started with this statement: "1) Multithreading is extremely hard, and unfortunately the way you've presented this idea so far implies you're severely underestimating how hard it is."
While I'll admit that my presentation was initially lacking, I later made an entire page to explain the synchronisation mechanism in place, and you can read more about it here, if you're interested:
http://nexusjs.com/architecture/
But what really shocked me was that I had never understood the mindset that all the naysayers adopted until I read that response.
Because the bottom-line of that entire response is an argument: an argument against change.
The average JavaScript developer doesn't want a multithreaded server platform for JavaScript because it means a change of the status quo.
And this is exactly why I started this project. I wanted a highly performant JavaScript platform for servers that's more suitable for real-time applications like transcoding, video streaming, and machine learning.
Nexus does not and will not hold your hand. It will not repeat Node's mistakes and give you nice ways to shoot yourself in the foot later, like `process.on('uncaughtException', ...)` for a catch-all global error handling solution.
No, an uncaught exception will be dealt with like any other self-respecting language: by not ignoring the problem and pretending it doesn't exist. If you write bad code, your program will crash, and you can't rectify a bug in your code by ignoring its presence entirely and using duct tape to scrape something together.
Back on the topic of multithreading, though. Multithreading is known to be hard, that's true. But how do you deal with a difficult solution? You simplify it and break it down, not just disregard it completely; because multithreading has its great advantages, too.
Like, how about we talk performance?
How about distributed algorithms that don't waste 40% of their computing power on agent communication and pointless overhead (like the serialisation/deserialisation of messages across the execution boundary for every single call)?
How about vertical scaling without forking the entire address space (and thus multiplying your application's memory consumption by the number of cores you wish to use)?
How about utilising logical CPUs to the fullest extent, and allowing them to execute JavaScript? Something that isn't even possible with the current model implemented by Node?
Some will say that the performance gains aren't worth the risk. That the possibility of race conditions and deadlocks aren't worth it.
That's the point of cooperative multithreading. It is a way to smartly work around these issues.
If you use promises, they will execute in parallel, to the best of the scheduler's abilities, and if you chain them then they will run consecutively as planned according to their dependency graph.
If your code doesn't access global variables or shared closure variables, or your promises only deal with their provided inputs without side-effects, then no contention will *ever* occur.
If you only read and never modify globals, no contention will ever occur.
Are you seeing the same trend I'm seeing?
Good JavaScript programming practices miraculously coincide with the best practices of thread-safety.
When someone says we shouldn't use multithreading because it's hard, do you know what I like to say to that?
"To multithread, you need a pair."18 -
The biggest scaling challenge...
Aha, when I joined my first (startup) company as an IT guy, they had 2 rooms in a small corner of a commercial building.
When I left the company after 2 years, they had two floors of that building with 40 rooms, had 5 different websites running in AWS, was using managed GSuite and a lot more.
So yeah, keeping up with all those was my biggest challenge.1 -
Our client decided to save some $$. At the end of each business day teams downscale their environments before leaving and the next day scale them up in the morning to start working.
The idea is not bad, but they are a bit too ignorant to the fact that some environments are exceeding AWS APIs limits already (huge, HUGE accounts, huge environments, each env easily exceeding /26 netmask, not even taking containers into account). Sooo... scaling up might take a while. Take today for example:
- come in to the office at 7
- start scaling up
- have lunch
- ~15:00 scaleup has finished
- one component is not working, escalating respective folks to fix them
- ~17:00 env is ready for work
- 17:01 initiate scaledown process and go home
Sounds like a hell of a productive day!!! -
"Let's create a docker-swarm cluster thingy for this application with horizontal scaling to learn docker and run this application better!"
This stuff is so overwhelming and I don't understand half of how to possibly set this up 😅32 -
Hey all - I made a big update to in app "x replied to a rant you commented on" in app notif to try to fix some bugs/scaling issue.
Please let me know if you notice any weirdness/notif issues. Thanks.5 -
I switched to using Ubuntu as my primary development environment about a month ago from Windows and I couldn't be happier! The initial setup was a pain in the neck (esp. display scaling issues, which Windows 10 is much better at than Ubuntu), but now that it's all set up, I'm lovin' it!6
-
Black Friday fail.
Local online store totally failed last year on BF, so they spent the whole week on news, radio, print ads, newsletter raving about how they upgraded the systems for BF and are expecting double the traffic of last year and are super ready. 60 seconds into BF, site went totally tits up, was not till 01:45 that anyone could access it again, and according to uptimerobot, they were tits up for about 55% of the day.1 -
Sysadmin: Apps on containers and kube is mandatory from now on, scaling is mandatory!
Devs: The systems weren’t designed for containers, we haven’t prepared shirt for scaling!
Sysadmin: Hold my beer! -
Windows 10 as a development environment... Oh freakin' goody.
📍Multi-montor is finicky
📍Text scaling is still horribly broken
📍It takes longer to resume from standby than my old laptop takes to boot up from a cold start
📍Color matching LCD monitors is pants-on-head retarded
📍32GB of RAM isn't going to be enough...
This is going to be a long summer in corporate land.20 -
Job posts that look for experience in everything! Experience in large scale enterprise kubernetes bullshit! What the fuck is kubernetes, a Greek god?? 4 plus years experience in aws! 5 years experience in cloud infrastructure scaling! 5 years experience in working with stakeholders and collaborating UX design! 5 years experience in React Native! 5 years experience in noSQL! 5 years experience in firebase! 5 years experience in graphics design! 5 years experience in node CSS! And every javascript known to mankind! I would love to meet this legendary developer that every company seems to want! Sick of these ads that ask for god level experience in every development role or tech. It’s like they’re hiring one developer to write their entire system from scratch which would obviously require godly expertise in front back and every fucking end there is to fucking build10
-
Oh dear, a scaling problem I solved was replacing some Regex matching with simpler string functions. While I'm a huge fan of Regex, it's unreal how much performance they can suck out of some high-n loops...
I got about 120x out of some critical code thus making a CPU upgrade unnecessary.8 -
how to sound smart in a meeting: pretend to listen carefully for the new idea being said, and then ask one question: "does it scale?"1
-
Woohoo!!! I made it to 1000++s :) Now I feel less newbie-like around here :)
So... I don't want to shit-post, so in gratitude to all you guys for this awesome community you've built, specially @trogus and @dfox, I'll post here a list of my ideas/projects for the future, so you guys can have something to talk about or at least laugh at.
Here we go!
Current Project: Ensayador.
It's a webapp that intends to ease and help students write essays. I'm making it with history students in mind, but it should also help in other discipline's essay production. It will store the thesis, arguments, keywords and bibliography so students can create a guideline before the moment of writting. It will also let students catalogue their reads with the same fields they'd use for an essay: that is thesis, arguments, keywords and bibliography, for their further use in other essays. The bibliography field will consist on foreign keys to reads catalogued. The idea is to build upon the models natural/logical relations.
Apps: All the apps that will come next could be integrated in just one big app that I would call "ChatPo" ("Po" is a contextual word we use in my country when we end sentences, I think it derived from "Pues"). But I guess it's better to think about them as different apps, just so I don't find myself lost in a neverending side-project.
A subchat(similar to a subreddit)-based chat app:
An app where people can join/create sub-chats where they can talk about things they are interested in. In my country, this is normally done by facebook groups making a whatsapp group and posting the link in the group, but I think that an integrated app would let people find/create/join groups more easily. I'm not sure if this should work with nicknames or real names and phone numbers, but let's save that for the future.
A slack clone:
Yes, you read it right. I want to make a slack clone. You see, in my country, enterprise communications are shitty as hell: everything consists in emails and informal whatsapp groups. Slack solves all these problems, but nobody even knows what it is over here. I think a more localized solution would be perfect to fill this void, and it would be cool to make it myself (with a team of friends of course), and hopefully profit out of it.
A labour chat-app marketplace:
This is a big hybrid I'd like to make based on the premise of contracting services on a reliable manner and paying through the app. "Are you in need of a plumber, but don't know where to find a reliable one? Maybe you want a new look on your wall, but don't want to paint it yourself? Don't worry, we got you covered. In <Insert app name> you can find a professional perfect to suit your needs. Payment? It's just a tap away!". I guess you get the idea. I think wechat made something like this, I wonder how it worked out.
* Why so many chat apps? Well... I want to learn Erlang, it is something close to mythical to me, and it's perfect for the backend of a comms app. So I want to learn it and put it in practice in any of these ideas.*
Videogames:
Flat-land arena: A top down arena game based on the book "flat land". Different symmetrical shapes will fight on a 2d plane of existence, having different rotating and moving speeds, and attack mechanics. For example, the triangle could have a "lance" on the front, making it agressive but leaving the rest defenseless. The field of view will be small, but there'll be a 2d POV all around the screen, which will consist on a line that fills with the colors of surrounding objects, scaling from dark colors to lighter colors to give a sense of distance.
This read could help understand the concept better:
http://eldritchpress.org/eaa/...
A 2D darksouls-like class based adventure: I've thought very little about this, but it's a project I'm considering to build with my brothers. I hope we can make it.
Imposible/distant future projects:
History-reading AI: History is best teached when you start from a linguistic approach. That is, you first teach both the disciplinar vocabulary and the propper keywords, and from that you build on causality's logic. It would be cool to make an AI recognize keywords and disciplinary vocabulary to make sense of historical texts and maybe reformat them into another text/platform/database. (this is very close to the next idea)
Extensive Historical DB: A database containing the most historical phenomena posible, which is crazy, I know. It would be a neverending iterative software in which, through historical documents, it would store historical process, events, dates, figures, etc. All this would then be presented in a webapp in which you could query historical data and it would return it in a wikipedia like manner, but much more concize and prioritized, with links to documents about the data requested. This could be automated to an extent by History-reading AI.
I'm out of characters, but this was fun. Plus, I don't want this to be any more cringy than it already is.12 -
Theoretically I can build multicloud, geo reudant, load balanced, auto scaling infrastructure. But most of my workday I spent in outlook writing E-mails and arranging up meetings.1
-
Best code performance incr. I made?
Many, many years ago our scaling strategy was to throw hardware at performance problems. Hardware consisted of dedicated web server and backing SQL server box, so each site instance had two servers (and data replication processes in place)
Two servers turned into 4, 4 to 8, 8 to around 16 (don't remember exactly what we ended up with). With Window's server and SQL Server licenses getting into the hundreds of thousands of dollars, the 'powers-that-be' were becoming very concerned with our IT budget. With our IT-VP and other web mgrs being hardware-centric, they simply shrugged and told the company that's just the way it is.
Taking it upon myself, started looking into utilizing web services, caching data (Microsoft's Velocity at the time), and a service that returned product data, the bottleneck for most of the performance issues. Description, price, simple stuff. Testing the scaling with our dev environment, single web server and single backing sql server, the service was able to handle 10x the traffic with much better performance.
Since the majority of the IT mgmt were hardware centric, they blew off the results saying my tests were contrived and my solution wouldn't work in 'the real world'. Not 100% wrong, I had no idea what would happen when real traffic would hit the site.
With our other hardware guys concerned the web hardware budget was tearing into everything else, they helped convince the 'powers-that-be' to give my idea a shot.
Fast forward a couple of months (lots of web code changes), early one morning we started slowly turning on the new framework (3 load balanced web service servers, 3 web servers, one sql server). 5 minutes...no issues, 10 minutes...no issues,an hour...everything is looking great. Then (A is a network admin)...
A: "Umm...guys...hardly any of the other web servers are being hit. The new servers are handling almost 100% of the traffic."
VP: "That can't be right. Something must be wrong with the load balancers. Rollback!"
A:"No, everything is fine. Load balancer is working and the performance spikes are coming from the old servers, not the new ones. Wow!, this is awesome!"
<Web manager 'Stacey'>
Stacey: "We probably still need to rollback. We'll need to do a full analysis to why the performance improved and apply it the current hardware setup."
A: "Page load times are now under 100 milliseconds from almost 3 seconds. Lets not rollback and see what happens."
Stacey:"I don't know, customers aren't used to such fast load times. They'll think something is wrong and go to a competitor. Rollback."
VP: "Agreed. We don't why this so fast. We'll need to replicate what is going on to the current architecture. Good try guys."
<later that day>
VP: "We've received hundreds of emails complementing us on the web site performance this morning and upset that the site suddenly slowed down again. CEO got wind of these emails and instructed us to move forward with the new framework."
After full implementation, we were able to scale back to only a few web servers and a single sql server, saving an initial $300,000 and a potential future savings of over $500,000. Budget analysis considering other factors, over the next 7 years, this would save the company over a million dollars.
At the semi-annual company wide meeting, our VP made a speech.
VP: "I'd like to thank everyone for this hard fought journey to get our web site up to industry standards for the benefit of our customers and stakeholders. Most of all, I'd like to thank Stacey for all her effort in designing and implementation of the scaling solution. Great job Stacy!"
<hands her a blank white envelope, hmmm...wonder what was in it?>
A few devs who sat in front of me turn around, network guys to the right, all look at me with puzzled looks with one mouth-ing "WTF?"9 -
I am sick and tired of big companies trying to shove their technologies down developer's throat in the name of developer advocacy. Last week I attended one of the IBM workshops which was supposed to be about ML and AI techniques but ended being solely about IBM Cloud (Bluemix), click here, click there, purchase it. I am not against developer advocacy and them trying to advertise their product but they should always keep in mind that developers won't get interested if they aren't learning any transferable core skills.
I was checking a course on Udacity about building scalable java apps. It turned out to be about Google Cloud Platform, auto scaling and nothing much. How deceiving is that?4 -
css quick maffs
so you did this:
.foo:hover {
transform: scale(1.1);
}
and now ugly scrollbar is there when the element is scaled.
No, don't do overflow: hidden. There's a better way. Instead, do this:
.container {
padding: 1rem;
box-sizing: border-box;
}
the element total width is calculated based on the width of its content. That's true unless you specified width and height explicitly (if you did so, you're a doofus, I'm sorry).
Scaling makes content somewhat larger. With border-box, paddings work differently with the total width.
By default, if you set width to say 100px, and paddings are 20px, total width will be 140px — it's your 100px of content plus two paddings of 20px. width property set the width of the content, not the total width.
With border-box, width property sets the total width. So if you set width to 100px and paddings to 20px, total width would be 100px, just like you set it, and content will be 60px wide — it's 100px minus 20px times two.
The key part is it doesn't end with explicit width. The algorithm remains. When some node is rendered, its total width is calculated. When you use border-box, the total width will stay the same even if your content grows by some value that is less than your paddings. So, your content was 100px, you scaled it, and it became 110px. Well, then that extra 10px will be subtracted from your paddings, and they will be 15px each instead of 20px.
No more ugly scroll bar. Yaaay!
aight bye8 -
This begs for a rant... [too bad I can't post actual screenshots :/ ]
Me: He k8s team! We're having trouble with our k8s cluster. After scaling up and running h/c and Sanity tests environment was confirmed as Healthy and Stable. But once we'd started our load tests k8s cluster went out for a walk: most of the replicas got stoped and restarted and I cannot find in events' log WHY that happened. Could you please have a look?
k8s team [india]: Hello, thank you for reaching out to k8s support. We will check and let you know.
Me: Oh, you're welcome! I'll be just sitting here quietly and eagerly waiting for your reply. TIA! :slightly_smiling_face:
<5 minutes later>
k8s team India: Hi. Could you give me a list of replicas that were failing?
Me: I gave you a Grafana link with a timeframe filter. Look there -- almost all apps show instability at k8s layer. For instance APP_1 and APP_2 were OK. But APP_3, APP_4 and APP_5 were crashing all over the place
k8s team India: ok I will check.
<My shift has ended. k8s team works in different timezone. I've opened up Slack this morning>
k8s team India: HI. APP_1 and APP_2 are fine. I don't even see any errors from logs, no restarts. All response codes are 200.
Me: 🤦♂️ .... Man, isn't that what I've said? ... 🤦♂️5 -
Worst thing you've seen another dev do? Here is another.
Early into our eCommerce venture, we experienced the normal growing pains.
Part of the learning process was realizing in web development, you should only access data resources on an as-needed basis.
One business object on it's creation would populate db lookups, initialize business rule engines (calling the db), etc.
Initially, this design was fine, no one noticed anything until business started to grow and started to cause problems in other systems (classic scaling problems)
VP wanted a review of the code and recommendations before throwing hardware at the problem (which they already started to do).
Over a month, I started making some aggressive changes by streamlining SQL, moving initialization, and refactoring like a mad man.
Over all page loads were not really affected, but the back-end resources were almost back to pre-eCommerce levels.
The main web developer at the time was not amused and fought my changes as much as she could.
Couple months later the CEO was speaking to everyone about his experience at a trade show when another CEO was complementing him on the changes to our web site.
The site was must faster, pages loaded without any glitches, checkout actually worked the first time, etc.
CEO wanted to thank everyone involved etc..and so on.
About a week later the VP handed out 'Thank You' certificates for the entire web team (only 4 at the time, I was on another team). I was noticeably excluded (not that I cared about a stupid piece of paper, but they also got a pizza lunch...I was much more pissed about that). My boss went to find out what was going on.
MyBoss: "Well, turned out 'Sally' did make all the web site performance improvements."
Me: "Where have you been the past 3 months? 'Sally' is the one who fought all my improvements. All my improvements are still in the production code."
MyBoss: "I'm just the messenger. What would you like me to do? I can buy you a pizza if you want. The team already reviewed the code and they are the ones who gave her the credit."
Me: "That's crap. My comments are all over that code base. I put my initials, date, what I did, why, and what was improved. I put the actual performance improvement numbers in the code!"
MyBoss: "Yea? Weird. That is what 'Tom' said why 'Sally' was put in for a promotion. For her due diligence for documenting the improvements."
Me:"What!? No. Look...lets look at the code"
Open up the file...there it was...*her* initials...the date, what changed, performance improvement numbers, etc.
WTF!
I opened version control and saw that she made one change, the day *after* the CEO thanked everyone and replaced my initials with hers.
She knew the other devs would only look at the current code to see who made the improvements (not bother to look at the code-differences)
MyBoss: "Wow...that's dirty. Best to move on and forget about it. Let them have their little party. Let us grown ups keeping doing the important things."8 -
Is asking for a nice email client in Linux too much?
In thunderbird, I found no way to customize the "messages pane". Title, sender, time all in one line.
In KMail, the fonts and style of HTML signature is not working as expected.
In Evolution, the fonts scaling doesn't work correctly.
Fuck me!11 -
Full stack developer.
I know what it's supposed to mean, but I feel like it gives discredit to the devs who perfect their area (frontend, backend, db, infrastructure). It's, to me, like calling myself a chef because I can cook dinner..
The depth, analysis and customization of the domain to shape an api to a website is never appreciated. The finicle tweaks on the frontend to make those final touches. Then comes a brat who say they are full stack, and can do all those things. Bullshit. 99.9% of them have never done anything but move data through layers and present it.
Throw these wannabes an enterprise system with monoliths and microservices willy nelly, orchestrate that shit with a vertical slice nginx ssi with disaster recovery, horizontal scaling, domain modeling, version management, a busy little bus and events flowing all decimal points of 2pi. Then, if you fully master everything going on there, I believe you are full stack.
Otherwise you just scraped the surface of what complexities software development is about. Everyone who can read a tutorial can scrape together an "in-out" website. But if your db is looking the same as your api, your highest complexity is the alignment of an infobox, I will laugh loud at your full stack.
And if you told me in an interview that you are full stack, you'd better have 10+ years experience and a good list of failed and successful projects before I'd let you stay the next two minutes..1 -
It is time... to rant about macs!
No, seriously - I had such a different experience about which not many talk in real life or pretend that it never happens....
Model: 2015 mid MBP 15" with second to highest specs (don't have dedicated gpu).
Rattling fucking toy.... Yea, it rattles! If you shake/move ir sit in trait/bus - it non-stop rattles as a fucking toy. Worst part? It's confirmed issue by apple and it manifacturing issue that they are not keen on fixing!!!! WTF? We have 4 macs in our office - all of them fucking rattles... God help me how annoying that is. (Lose LCD control panel that unsticks from glue. Replacing it solves the issue for 1 month if you carry it anywhere).
Constant fucking crashing/updates.... Every morning I wake up and don't have an app that requires confirmation for restart - it's restarted. YAY, turning on all apps once again.... Why you may ask? Well, because if you tinker with software in any way - it fails to update it and hell breaks lose. It's been a long time since High-Sierra came around and the issue is still there (not running Mojave as it conflicts with soft I have... Woo!). Tried few times - updates fail. Resolution? Reinstall OS!
OS conflicts with applications - damn... People told me it works out of the box.... Yeah, as long as you don't upgrade the OS - then it breaks. Why? Well, because.
Piece of shit power supply. With 4 of our office power supplies - 2 of them failed twice withing warranty and once afterwards... Really? Not to mention that all 4 are starting to shear the sleeve or already did (mine is just wrapped with white electrical tape to give it a support... lol).
Bluetooth - who the hell needs that in mac, right? Well, people do. To start with - it conflicts with 2.4GHz wireless network - you might have one of those and not both at the same time. Next thing is using a device that needs constant connection (mouse, headphones, keyboard - non apple branded) - shit... They can't stay connected for more than an hour without any issues... Constant battle to re-connect it, to re-pair the device and all due to smart apple bluetooth settings. Hell, my mouse (logitech MX master) was even printing random symbols in some applications if moved. All of the issues went away after using a bluetooth dongle... WOO!!!!
Xcode... Ahh, you may never prepare your mac if you don't download 17GB of fucking xCode libraries that enables some tools to be installed/runned as you can NOT get them in any other way and you have to install full xCode software in order to get them... YAY! 17GB wasted on my 256GB SSD that I can't upgrade. GREAT!
OsX applications - ah, don't get offended but if you are using them and you are fine with them - you are probably a monkey that loves being told what to do. You can't customise any actions, you can't configure it the way you like - either you accept their default workflow or go kill yourself. Yep... Had issues with calendar, mail, iMessages, safari... None of them fit my needs :)
Resolution scaling... Fucking hell, the display is 2880 x 1800 but all you let me to use is 1440x900 without scaling? Am I blind to you? Scaling the resolution means that you are fucked if some applications don't support scaling very well. Looking at you Jetbrains - your IDES suck at scaling and slows down the pc to a potato....
Now the pros - keyboard is way better than the new ones, trackpad is GREAT - no need for mouse (using it on external 4k displays only), the battery life is great - getting around 6h of continues development time, 8 if using sublime instead of phpStorm and well, that's about it...
To clarify:
I've bought this device due to the fact that at that time mac and windows pc's with similiar specs costed the same while windows pc sucked with their quality of the device and trackpad... Now the situation is better and when time comes for a next upgrade - it's going to be one of these:
Razer Blade 15, Dell XPS 15, Lenovo Carbon X1 series.
And of course - LINUX. I've had enough issues with windows, and had enough of retardness of apple ecosystem, so switching it is a must for me.
Disclaimer: I might be an unhappy customer, a bit picky but I'd like my device to be setted up as I like and continue to have that until I don't like, not until the company decides to break it. Not to mention that paying almost a yearly salary in my country for one device - I'd expect it to be at least reliable and work without issues....
Rant over.
ps. You can disagree with me, this is my personal experience with MBP over the last 3 years :)8 -
Installing my company's microsystems architecture to run locally is a pita because it is 60 GB of docker containers. With my 256 GB Macbook, that's a scaling problem for the years to come.6
-
Top gripes about getting older as I'm about to turn 40:
5. Actually starting to have moments at home after work where I'm contemplating saying 'Hey babe, wanna bang?' but before I can get the words out my body pipes in with 'Dude, cool your jets, we're wiped out today; check back tomorrow.' Women say they like older guys because <insert character trait here> but I'm now convinced it's just because they know there's less work involved. =/
4. Friends with young children. I hardly ever see them anymore, and when I do, all they talk about are their kids and their shitty relationship with their co-parent. The circle continues to get smaller...
3. Having to go get glasses in order to renew my driver's license. How do we not have a heads-up display in every vehicle by now that shows the street numbers of buildings as I'm perpendicular to them as well as the names of upcoming cross streets? That way I'd fix the problem the way I do for everything else: notch up the font scaling on my display a point or two. Elon, you're slipping...
2. Realizing that the "American Dream" isn't worth the paper it was printed on. (Anyone else remember paying 97¢ for a gallon of gas or $2 for a pack of Marlboros?) Concurrent realization: It's not easy to find work in another country without moving there first, even if you speak the language. Any devs in Portugal that read this, ligue-me.
1. Being too busy to just chat with new people I meet except on rare occasion. Mostly referring to work time here, when it seems I'm always needing to find the shortest route to the objectif du jour. If I could tell my teenage self just one piece of advice, it'd probably be "start your career in Europe, not the USA" but I really want it to be "treasure the time you spend on IRC talking about anything and everything with people that always have time for you and vice versa, because it's going to be over before you know it." -
I got an interview with a big multinational software company as a senior dev - the kind of place I never thought I would be privileged or knowledgeable enough to work for and wasnt expecting to get In to...
I aced it. They gave me an offer but - FOR DEVOPS 😬
basically my skills fit in perfectly with the server/ scaling issues they have and are far more valuable there. I know they do, I also know I can fix the issue and will have alot of fun coding it - I just dont think I want to monitor it or anything else.
I mean I do devops stuff all the time in aid of anything I code but their stack is a full time job- im scared that once the toolchain is automated ill be pulled towards sys admin like duties and lose touch with my craft... what do you guys think? Anyone shifted from dev to devops?9 -
Monthly Update call
Me : I need 2 undivided weeks in the next 3 months to make some refactoring to our codebase, so we dont have issues in scaling.
Senior Manager (with years of tech experience almost equal to my age) :
Why would we need that, We are already hosted on AWS so we can just spin up more instances.
Me ( trying to hold the rage inside) : No it doesnt work that way, that will only hurt us more than help.
Him : What if we try with a bigger instance size. Its AWS, sure they will have a bigger instance.
Me : No, tha....
Him : (cuts me off) How about we use autoscaling and let AWS decide
Me : (Silently praying for a gun that shots across video calls.)8 -
So our team is scaling and we've been handling increased workload and I've been wanting to hire two juniors ever since. Finally narrowed down the candidates to three, but I couldn't decide between them so after some careful considerations, I've decided to hire all three. Fingers crossed I have enough time to mentor and manage the three of them and I don't fuck them up.1
-
Years ago, we were setting up an architecture where we fetch certain data as-is and throw it in CosmosDb. Then we run a daily background job to aggregate and store it as structured data.
The problem is the volume. The calculation step is so intense that it will bring down the host machine, and the insert step will bring down the database in a manner where it takes 30 min or more to become accessible again.
Accommodating for this would need a fundamental change in our setup. Maybe rewriting the queries, data structure, containerizing it for auto scaling, whatever. Back then, this wasn't on the table due to time constraints and, nobody wanted to be the person to open that Pandora's box of turning things upside down when it "basically works".
So the hotfix was to do a 1 second threadsleep for every iteration where needed. It makes the job take upwards of 12 hours where - if the system could endure it - it would normally take a couple minutes.
The solution has grown around this behavior ever since, making it even harder to properly fix now. Whenever there is a new team member there is this ritual of explaining this behavior to them, then discussing solutions until they realize how big of a change it would be, and concluding that it needs to be done, but...
not right now.2 -
Dev: The server is completely down right now. Nobody can access the application, we need to divert some resources to horizontally scaling our app.
Manager: Hm, this was not in the schedule. I need to consult senior leadership on what to do about this. I don’t want to be held accountable for making a decision on this complex and highly nuanced situation.
Dev: No need, I have a solution. Just need a week to build/test/deploy the ability to horizontally scale.
Manager: But that will cause delays to new features.
Dev: New features don’t matter if the app can’t even load.
Manager: Ok you can implement your solution but it can’t take any time. I need those new features out.
Dev: ???????5 -
Today I had to present my final year project called segmentation and detection of glioma using deep learning.
It took 15 minutes for the evaluators to understand what an mri image dataset (BRaTs) dataset looks like (they are voxels and not pixels). My god, these peasants...
And I was there expecting them to understand down sampling convolutions and up scaling convolutions of U Net model 😂. Life is so convoluted right now!2 -
Hey, we need a service to resize some images. Oh, it’ll also need a globally diverse cache, with cache purging capabilities, only cache certain images in the United States, support auto scaling, handle half a petabyte of data , but we don’t know when it’ll be needed, so just plan on all of it being needed at once. It has to support a robust security profile using only basic HTTP auth, be written in Java, hosted on-prem, and be fully protected from ddos attacks. It must be backwards compatible with the previous API we use, but that’s poorly documented, you’ll figure it out. Also, it must support being rolled out 20% of the way so we can test it, and forget about it, and leave two copies of our app in production.
You can re-use the code we already have for image thumbnails even though it’s written in Python, caches nothing and is hosted in the cloud. It should be easy. This guy can show you how it all works.2 -
On the nth day of Christmas, my true love gave to me:
1 misconfigured autoscaling group
200 unnecessary servers
29 urgent emails
3 support teams that would have fixed the problem in an hour of anybody had bothered going to them first
Ugh. Idiots. Somehow the whole issue was caused by a single full hard disk, which caused database transactions to fail, which caused the group to scale out and spin up new instances, which didn't actually fix the problem so it kept scaling until it hit the limit and then continuously failed to create new instances for several hours straight, generating loads of notification emails and generally causing problems for everyone involved. -
Spent 5 hours working on a solution for a hash difficulty comparison/scaling algorithm. after a bunch of different iterations and approaches, I find that my problem can be solved by the attached equation. Its such a simple answer but no way in hell would you be able to discern the amount of time and brainpower that was put into it. The git commit is literally 10 lines of code total, but I guess its not about the amount of code, but the time spent thinking about it thay counts?6
-
As a dev, I never want to personally talk to Clients anymore.
I had this Client for whom I developed a website for a tool he was making. I did the front-end work, did the backend, added the shop and everything he asked for. I showed him everything so he could see for himself what was done and how it all works. He went ahead and tested the finished product to see if there were any bugs.
He came back to me and wanted a call on skype and I joined the call. He shared his screen to show me that when he resizes the browser, the content of it gets bigger and smaller according to scale. I worked so hard to get it responsive and whatnot, and he tells me that it is doing what it is supposed to do. I die inside every time I think about it2 -
I'm in a company with no senior devs I can look to for mentoring. How do you go about scaling with the company without a developer more senior to guide you during development?
I feel like I'm always second guessing decisions.14 -
Crazy... Hm, that could qualify for a *lot*.
Craziest. Probably misusage or rather "brain damaged" knowledge about HTTP.
I've seen a lot of wild things when devs start poking standards, but the tip of the iceberg was someone trying to use UTF-8 in headers...
You might have guessed it - German umlauts. :(
Coz yeah. Fucktard loved writing everything in german, so why not write custom header names in german.
The fun thing is: It *can* work, though the usual sane thing is to keep it in ASCII range for the obvious reason that using UTF-8 (or ISO-8859-1, which is *not* ASCII) is a gamble you gonna loose.
The fun game was that after putting in a much needed load balancer between services for monitoring / scaling etc suddenly *something* seemed off.
It took me 2 days and a lot of Wireshark hoola hooping to find out why, cause the header was used for device detection aka wether it's a bot or not. Or in the german term the dev used: "Geräte-Art".
As the fallback was to assume a bot, but only rate limit based on IP, only few managed to achieve the necessary rate limit to get blocked.
So when I say *something* seemed off, I really mean a spooky kind of "sometimes IP blocked for seemingly no reason at all".
Fun stuff. The dev btw germanized everything. Untangling the code base was a lot of non fun. -.-6 -
I recently got a project for scaling multiple wordpress sites for around 5k req/s
The biggest mistake I made is that even though it's wordpress I accepted it, fml!
Even aws load balancer gave up.5 -
Pulled into an 'emergency' meeting with a group of web designers deeply concerned a particular service wasn't going to meet all their requirements.
DevA: "For each page, Its going to be A LOT of work to retrieve all the data and store it's state. Every page load will require a round trip to the service."
DevB: "Yes, we aren't sure how the service should be changed to do what we need."
Mgr: "What is it not doing now? Doesn't the service already returns all the necessary data?"
DevA: "Well...um...its all the boolean fields. Some may be defaulted from the database or false because the user unchecked the box. We have to know which is which"
Me: "Why? Are you doing anything different in the UI? Checkbox will be true or false. What or who set that value is irrelevant"
DevC: "Well, it matters if the user didn't fill out all other other values."
Me: "How so?"
DevA: "Its matters because the values in the other fields. Its going to be A TON of work to figure out."
<mgr goes to the white board>
Mgr: "Lets map this out...what fields are you needing to trigger the state on?"
DevA: "Um...uh...the 'Approved' field...and um...'OK to Contact' field"
Mgr: "Just those two?"
DevA: "Yea..um...there are other fields, but whether or not to show the edit box depends on those two."
Me: "The service already returns data, you only have two fields to check? I don't see a need to change the service at all."
DevA: "Returning all that data, we are going have a serious scaling problem. We'll be hitting the service A LOT. All that javascript could cause performance problems too"
Me: "How much data are we talking about? Name, address, couple of booleans?"
DevA: "I have to serialize the data. All that logic is going to be reeeaaallly complicated. It might be better if the service returned only the data I need."
Me: "$64,000 question, how often is this feature going to be used on the web site? Maybe once? Few hundred a week?"
Mgr: "We have no idea. A lot of the data will be pre-populated and we're only sending out a few thousand invitations. More around the holidays...but honestly, not very many."
Me: "Changing that service only for this particular area of the web site isn't going to happen. Changing the UI is the only course of action."
DevA: "Oh frack I can't wait until this project is over."
DevA...how the funck do still have a job here? You wasted about half-hour of my time with your cry-baby crap. Where is my hammer...no...no..don't go there...ahhh...thanks devrant. Prison sentence diverted.2 -
Is your code green?
I've been thinking a lot about this for the past year. There was recently an article on this on slashdot.
I like optimising things to a reasonable degree and avoid bloat. What are some signs of code that isn't green?
* Use of technology that says its fast without real expert review and measurement. Lots of tech out their claims to be fast but actually isn't or is doing so by saturation resources while being inefficient.
* It uses caching. Many might find that counter intuitive. In technology it is surprisingly common to see people scale or cache rather than directly fixing the thing that's watt expensive which is compounded when the cache has weak coverage.
* It uses scaling. Originally scaling was a last resort. The reason is simple, it introduces excessive complexity. Today it's common to see people scale things rather than make them efficient. You end up needing ten instances when a bit of skill could bring you down to one which could scale as well but likely wont need to.
* It uses a non-trivial framework. Frameworks are rarely fast. Most will fall in the range of ten to a thousand times slower in terms of CPU usage. Memory bloat may also force the need for more instances. Frameworks written on already slow high level languages may be especially bad.
* Lacks optimisations for obvious bottlenecks.
* It runs slowly.
* It lacks even basic resource usage measurement.
Unfortunately smells are not enough on their own but are a start. Real measurement and expert review is always the only way to get an idea of if your code is reasonably green.
I find it not uncommon to see things require tens to hundreds to thousands of resources than needed if not more.
In terms of cycles that can be the difference between needing a single core and a thousand cores.
This is common in the industry but it's not because people didn't write everything in assembly. It's usually leaning toward the extreme opposite.
Optimisations are often easy and don't require writing code in binary. In fact the resulting code is often simpler. Excess complexity and inefficient code tend to go hand in hand. Sometimes a code cleaning service is all you need to enhance your green.
I once rewrote a data parsing library that had to parse a hundred MB and was a performance hotspot into C from an interpreted language. I measured it and the results were good. It had been optimised as much as possible in the interpreted version but way still 50 times faster minimum in C.
I recently stumbled upon someone's attempt to do the same and I was able to optimise the interpreted version in five minutes to be twice as fast as the C++ version.
I see opportunity to optimise everywhere in software. A billion KG CO2 could be saved easy if a few green code shops popped up. It's also often a net win. Faster software, lower costs, lower management burden... I'm thinking of starting a consultancy.
The problem is after witnessing the likes of Greta Thunberg then if that's what the next generation has in store then as far as I'm concerned the world can fucking burn and her generation along with it.6 -
The person who claims we should be "in control" of our stack and not use any of this pre-made, one size fits all, BaaS bullshit.
Even though none of our projects are even remotely close to needing scaling and whatnot.
Then proceeds to argue all we need is a step by step guide on how to deploy things very easily using Cloud Formation.
Spends the next two weeks trying to figure out why config files and scripts don't work.
And now that we have that "recipe", we're fully capable of deploying our own pre-made, one size fits all backend.1 -
Just need to vent, so here goes:
Fuck doing cutting edge projects for great glory, low budgets and tight deadlines. I'm tired, burnt out and just don't give a shit anymore.
I got promoted to lead dev and thought my fortune was made but what it really meant was just: Here solve all these bullshit bugs that the rest of the team can't figure out and oh we are also taking this single app you guys made and scaling it globally. You have half a year to figure that out. You handle the devops.... sigh
Fuck that noise.
Honestly i just feel like quitting and finding a nice specialist place, with a cap of at max Senõr developer, no more being the one making the big decisions for me, rather just diving into certain areas and coding the fuck out of that. Maybe some teaching too, i like that.
Anyway, won't happen right now, i need the salary. My wife just graduated and can't find a job what with a certain flu fucking over the economy, so I am stuck here for now.2 -
I'm so sick of microservice architecture... in theory it was going to make scaling elastic and deployment easier. In reality it seems to slow productivity to a 🐌 pace.
Anyone have any brilliant suggestions on how to herd these cats in production?10 -
When a marketing / sales person says in a meeting with executives that our current infrastructure can handle 300.000 new customers easily. (We're at 90 ATM).
Mfw I'm the techie in the room and aware of our non-scaling mysql DB. -
I know this really isn't the place for this, but I REALLY need help..
I need to add app scaling inside my game.
To do this, I need to listen for a MouseDragEvent (or similar) on a JavaFX object like a Label or Image.
Here is my SO post:
http://stackoverflow.com/questions/...
Thanks in advance!22 -
I'm going to replace Designer with Architect.
Every single day: WHAT MONKEY DECIDED TO BUILD THIS SYSTEM THAT WE NOW HAVE TO CONTINUE DEVELOPING?
HOW THE FUCK DID HE THINK TO SCALE BY BUILDING A SYSTEM THAT TYING EVERYTHING TO A SINGLE DB?2 -
Thoughts on forced emergency support?
I am with a company I generally like a lot but there are some things I generally despise about it. Like forced emergency support.
I am not good at it, I don't claim to be.. I generally struggle with anxiety, stress and depression, I specifically avoid roles that require on-call service .. I'm a senior level software engineer.
I find it very frustrating to be expected to be on-call from 7-7 in support of infrastructure I did not architect, did not code and basically know nothing about. They provided me with a ten minute discussion about ops genie and where to find internal support articles for my training and that's about it.
Last night I received an ops genie alarm and acked it as I was instructed to do, I went around the system looking for the alarm cause and basically had no idea what to do except watch our metrics graphing praying there wouldn't be an outage. Fortunately the alarm was for our load balancer scaling operation, it was taking a bit longer than usual ... Sigh of relief. Stay up til 6am and fall asleep..
Wake up to a few messages from various people asking why I didn't do this and that and it took me every inkling of my being to remain cordial and polite but I really just wanted to scream and say a bunch of shit that would probably get me fired.
What the actual fuck?
Why expect someone that has no god damn clue what they are doing to do something like this? Fuckin shit training and no leadership to mentor me and help me get better at this role, no shadowing, no regiment ..
#confused and #annoyed
Thoughts? Am I a bitch? Is it unreasonable for me to expect my job duties stay in line with what I'm actually good at!?
Thanks.15 -
Recently had to implement some microbenchmarks for a project. First time I ran them, I thought they had gone stuck. It turned out they weren't, but the processing took forever.
Some smaller benchmarks revealed that runtime was not scaling linearly with the input size as one would expect for the problem. It turned out that code was iterating over the input to find corresponding entries in the output.
We changed the processing so that it creates the output in the same order as the input and just compare entries at the same position. With that we were able to cut runtime from a few hours to a few seconds. -
What game engine would you recommend to an indie developer? The type who can't afford a fucking server to run the bloated and buggy unity editor but is actually a developer so isn't afraid of typing.
I've had enough of the improper sandboxing (will crash bc of game scripts), tempfile-based crash-unaware instance tracking (won't restart afterwards) and lack of UI scaling (seriously, that's like accessibility/retina support basics) that is the unity editor. If they had command line tools I'd use them happily.10 -
Since I bought my laptop the screen was scaled to 125% (1080p display) so that it looks like a ~768p display.
I like it, it looks good, but now I'm trying it at 100% scaling (15" screen).
Yes, everything looks so small now, and my eyes cried for the first 10 minutes, but I'm starting to like it.
Do you have any similar experience? What do you prefer, 100%, 125% or something else? Why?16 -
docker service scale serviceID=5
If only scaling my bank balance were as simple as docker swarm scaling !1 -
Since I'm back to working for myself again and haven't been able to find a reliable hire, I'm alone. In this bubble, no one cares/sees/appreciates my backend code and I just realized that's why I've been slacking so bad on this ETL process. No one gives a shit about it but me. If I build an interface, I get kudos and everyone celebrates, but working on a three server process with layers of abstraction, auto-scaling, etc...and people just wonder if I'm jerking off all day.
Sometimes it sucks to be a lone ranger.1 -
Software engineering doesn't evolving the way you think of it.
There are no new big patterns. There are no new big concepts and ideas to bring that evolution to us. Rob Pike thinks that the concepts he used twenty years ago are the best possible way of implementing everything and he creates Golang.
The evolution of software engineering, and maybe the whole evolution as a concept is a tick-tock. Software engineering had its latest tick at nineties, when the concepts we call modern were developed. And the latest tock was the rise of the internet, and it given the single-computer-centered Von Neumann architecture really hard challenges. I mean ticks are theoretical inventions and patterns and ideas and etc, while tock is more of some practical, business-oriented implementations.
PHP is still in use. We have troubles with scaling and deployment. Banking systems still run old Java, Windows XP and even COBOL. We had persistence really, really long time ago, and now frontenders reinvent it and call it 'immutability'!
We had our tick many, many years ago. It's time for tock. With not only scientific but commercial use of things such as Clojure, CRDTs and maybe Rust lang, we are heading straight to our new big tock, which'll bring us new great problems to solve.
That's how any evolution goes.rant rust lang paradigms rob pike evolution golang ideas rust wk127 clojure patterns software engineering -
Node: The most passive aggressive language I've had the displeasure of programming in.
Reference an undefined variable in a module? Prepare to waste your time hunting for it, because the runtime won't tell you about it until you reference a property or method on the quietly undefined module object.
Think you know how promises work? As a hiring manager, I've found that less than 5% of otherwise well-experienced devs are out of the Dunning Kruger danger zone.
Async causes edge cases and extra dev effort that add to the effort required to make a quality product.
Got a bug in one of your modules? Prepare yourself for some downtime because a single misplaced parentheses can take out the entire Node process, killing unrelated pages and even static file hosting.
All this makes for a programming experience that demands much higher cognitive load, creates more categories of bugs, and leads to code bloat/smell much more quickly than other commonly substituted languages.
From a business perspective, the money you save on scaling (assuming your app is more compute efficient under Node) is wasted on salaries and opportunity costs stemming from longer dev time, more QA, and more frequent outages.
IMO, Node is an awesome experiment, a fun language, a great tool for specific use cases, and a terrible fucking choice for an entire website.8 -
At what point do you stop optimizing queries and realize it's a database architecture, scaling problem?
We've been having production issues this week because a lot more users with more demands, and I'm going we need more servers... We can't just have one db, we need to parallelize like Hadoop...
Everyone else is going, how do we optimize queries, indexes, reduce the load...11 -
As much as I like Windows (now) the scaling on high res display is sooo awful, so now I am using a resolution of 1440 x 900 on my 2560x1600 retina display o_O1
-
Multi-continent low-latency auto-scaling eventually-consistent kubernetes-orchestrated and spark-powered multi-cloud data-plarform.
(Note to self: why do jargon words always come in twos?)
But seriously, the engine ELT's naval and logistical data from every continent and ocean and feeds a global analytics platform for less then 0.25 USD per ingested Gb across all systems.
And sometimes the PODs are even onboard en-route ships! Edge computing, y'all!
Tech project I'm most proud of.2 -
We use at our company one of the largest Python ORM and dont code ourselfs on it, event tough I can code. Its some special contract which our General Manager made, before we as Devs where in the Project and everything is provided from the external Company as Service. The Servers are in our own Datacenter, but we dont have access.
We have our Consultants (Project Manager) as payd hires and they got their own Devs.
Im in lead of Code Reviews and Interfaces. Also Im in the "Run" Team, which observes, debuggs and keeps the System alive as 3rd-Level (Application Managers).
What Im trying to achieve is going away from legacy .csv/sftp connections to RestAPI and on large Datasets GraphQL. Before I was on the Project, they build really crappy Interfaces.
Before I joined the Project in my Company, I was a Dev for a couple of Finance Applications and Webservices, where I also did coding on Business critical Applications with high demand Scaling.
So forth, I was moved by my Boss over to the Project because it wasn't doing so well and they needed our own Devs on it.
Alot of Issues/Mistakes I identified in the Software:
- Lots of Code Bugs
- Missing Process Logic
- No Lifecycle
- Very fast growing Database
- A lot of Bad Practices
Since my switch I fixed alot of bugs, was the man of the hour for fixing major Incidents and so on so forth. A lot of improvements have been made. Also the Team Spirit of 15+ People inside the Project became better, because they could consult me for solutions/problems.
But damn I hate our Consultants. We pay them and I need to sketch the concepts, they are to dumb for it. They dont understand Rest or APIs in general, I need to teach them alot about Best Practices and how to Code an API. Then they question everything and bring out a crooked flawed prototype back to me.
WE F* PAY THEM FOR BULLCRAP! THEY DONT EVEN WRITE DOCUMENTATION, THEY ARE SO LAZY!
I even had a Meeting with the main Consultant about Performance Problems and how we should approach it from a technical side and Process side. The Software is Core Business relevant and its running over 3 Years. He just argumented around the Problem and didnt provide solutions.
I confronted our General Manager a couple of times with this, but since 3 Years its going on and on.
Im happy with my Team and Boss, they have my back and I love my Job, but dealing with these Nutjobs of Consultants is draining my nerves/energy.
Im really am at my wits end how to deal with this anymore? Been pulling trough since 1 year. I wanna stay at my company because everything else besides the Nutjob Consultants is great.
I told my Boss about it a couple of times and she agrees with me, but the General Manager doesnt let go of these Consultants.
Even when they fuck up hard and crash production, they fucking Bill us... It's their fault :(3 -
One thing that I'm not that big a fan of in Kubuntu is how I can't just scale one of my screens. I need to scale my 4k monitor to 1.5 the size otherwise it will all be too small. However my 1080p monitor also gets scaled, which is really doesn't since it makes everything wayy too big.
However, when I use display scaling my apps actually fucking scale. Meanwhile on fucking Windows they haven't figured that out yet
Bigger image: https://imgur.com/a/3qBo31o4 -
So we started a new Unity video game project for mobile in June 2021. Hooray!
Being a mobile project, one of the earliest things we think about is scaling the interface across all sorts of device screen resolutions and aspect ratios, right? Well, to preemptively solve this problem early on, I decided to letterbox the game view - just choose one aspect ratio for the game and pad black bars to the sides of the screen. Simple, solves the game's world space problem without trying too hard, and it automatically adapts to Android's split-screen mode.
I showed the early builds to management as well as game design team and they gave me some general nods. Sounds like green light ahead. I spent the next few months building the game logic and scale the UI around a consistent letterboxed game view. If you had experience scaling Unity UI to a letterboxed area, you should already knew that it takes a whole paradigm of its own that's kinda hard to break out of, but the fact that it stays consistent across all screen aspect ratios is so worth it. Regardless, the biggeer benefit of letterboxing is simpler world space setup. You don't worry about whether this particular area will be overflowed horizontally or vertically in a particular device or not. You have a 9:16 window to view the world through, nothing needs to move at runtime and that's about it.
Fast-forward to early September 2021 and 40+ builds later, the GD started having concern that the playing area is not filling up his phone screen and that the letterboxes are bothering him. He wants to get rid of the letterboxes and wants the game world as well as UI to fill up his screen.
Yes. After 40+ builds, for all of which the letterbox was present, nobody in the project raised a concern about the letterbox. It's only NOW that they all of the sudden side with the GD and demand the removal of the letterbox. I feel like almost half of my effort on this game has been wasted. These clueless guys didn't spend one second looking at the early builds thinking of the possibility that the black bars at the top and bottom of their phone screens (which I repeat: has been around since the very first build) is gonna bother them? Somebody must be playing a cruel joke at this company. They had all the chances to bring this up as a potential issue and TODAY is the first time I hear of it.
See, designers. You waste our time and your time by doing this kind of thing. Please raise your issues early. Complain to us ASAP. If you wait for so long before raising an issue that has been in-your-face the whole time, I can't fault any developer for assuming you're trying to play a long prank. I can tell designers right now: it's not funny.1 -
Turns out you can treat a a function mapping parameters to outputs as a product that acts as a *scaling* of continuous inputs to outputs, and that this sits somewhere between neural nets and regression trees.
Well thats what I did, and the MAE (or error) of this works out to about ~0.5%, half a percentage point. Did training and a little validation, but the training set is only 2.5k samples, so it may just be overfitting.
The idea is you have X, y, and z.
z is your parameters. And for every row in y, you have an entry in z. You then try to find a set of z such that the product, multiplied by the value of yi, yields the corresponding value at Xi.
Naturally I gave it the ridiculous name of a 'zcombiner'.
Well, fucking turns out, this beautiful bastard of a paper just dropped in my lap, and its been around since 2020:
https://mimuw.edu.pl/~bojan/papers/...
which does the exact god damn thing.
I mean they did't realize it applies to ML, but its the same fucking math I did.
z is the monoid that finds some identity that creates an isomorphism between all the elements of all the rows of y, and all the elements of all the indexes of X.
And I just got to say it feels good. -
Does anyone enjoy their Windows development laptop? I fight with it so much. I would be happy to smack it hard with a hammer for all of the embarrassing things it has done in front of customers.
They are trusting me to build a multi million dollar system yet for some reason I can't join a Skype meeting, or my mic doesn't work, or the scaling on screen makes everything look oversized.
What am I supposed to say? "Trust me to build your system. I swear I'm not as retarded as I appear."8 -
Although my profile is of full stack dev but at my company we barely do anything to scale anything.
Whenever I go for any interview, everyone asks me about scaling the application. What should I do?2 -
Anyone else think the emphasis on scaling is misplaced?
I think we can have more innovative products that solve real problems in specific ways if people focused on solving specific problems rather than targeting wider audience in attempt to sell to as many as possible.24 -
I've just used https://gtmetrix.com to see how devRant fares. Pretty well actually, with one major gotcha that should be easy to fix. There are a lot of static resources without browser cache expiration date.
A little image optimisation could also be done, see the PageSpeed tab. And scaling down images in CSS could also be replaced by proper scaling of the image itself.
The YSlow tab shows that a little JS minification is missing, and maybe 4 external scripts could be combined into one.1 -
This is a question and a rant about my frustrations with an API description. First take a look at this usage information for a library I have been playing with:
https://github.com/avaneev/avir/...
Now my first question: Is it clear as to you what the formats are for the InBuf and OutBuf parameters are?
Now, read his response to people (including me) struggling with determining the format of the buffers for this resize function:
https://github.com/avaneev/avir/...
Does the guy come across as condescending as hell? Am I reading into this? It is like the guy takes every opportunity to find fault with people not understanding the details he didn't put in his API description. I would find it difficult to have to work with someone like this.
The irony about this is the description of his code makes me think it is going to magically create this wonderfully rescaled image preserving details. The result is that a standard bilinear filtering scaling function looks practically the same. I saw no real perceptible improvements of his code over the scaler I tested against. When I adjusted parameters using presets he provided it didn't improve the results and added artifacts I could not accept. His scaler is also at least a magnitude slower than the bilinear version. So the code is pretty much a non-starter for my current project.
Ah well, I appreciate him posting the code and making it a very permissive license. That part is really cool.16 -
!rant
I've tried Ionic in the past, and put off development with it because I couldn't get it to be performant without crosswalk.
In 2015, React Native put an end to the 'Are hybrid apps viable?' argument. It had a much smaller compile size, large component library, and is very reactive.
Ionic recently released news on scaling back their tools to focus on core offerings. I can't help but feel they're flogging a dead horse.
I'm sure the Ionic team has very smart people on board. Can't they see they're about to be 'Parse-ed'?1 -
Any of you guys have experience on scaling a node.js app with socket.io past 10000+ concurrent connections? 🙂7
-
Alright, my very first post here was about this project and I am thinking it out loud again.
I see a problem and I am struggling to find a solution.
Now what I am thinking of is to articulate the problem well and state WHY I believe it needs to be solved. There are some reasons which must be presented in a capitalist way.
Furthermore, I am thinking of doing a market research to understand various demographics, validate the idea, and figure out the product-market fit.
Now, this qualitative research and quantitative data will help me decide whether it is worth putting in the efforts to solve the problem or not.
And since, we have an MVP already (funnily yes, we built it before all of the above), that will help me validate the tangible solution.
Once we get a confidence boost, then it will be time to get that single transaction which has net positive cash flow.
Start scaling to 'next billion users', so a billion transaction with net positive cash flow.
I won't be branching out into multiple verticals before be able to sustainably scale the core USP.
And while the second half sounds like, 'I have a million dollar idea', I am trying to be more and more realistic and rationale instead of falling in love with my idea.
I don't even have an idea (read solution) to fall in love with. Rather I have a problem that is bothering me.
So, yes, I am continuing this journey to solve the problem which started in second year of my hostel room and has evolved over 10 years. -
Just completed a 24 hour hackathon at my school in which the 'best software' winner purely had mock ups of yet another mobile app and had no proof of concept. Meanwhile my team developed a scaling platform online that adapts to groups of user's trends to create optimal results.
I guess I keep misreading the definition of 'software' in the dictionary each morning. (RULE #8.2 - Software Engineers shall read the definition of the following phrases each morning excluding Saturday: software, heap, ego, scrum, algorithm, the documentation of C)4 -
Scaling a 3D STL File so the Tolerances are still the same but the Model is Like 300% scaled... Fucking hard...1
-
You know your night is off to a great start when it takes you 30 minutes to remember VM's exist and can be really good for testing out display scaling... Well done Alex... Well done1
-
As we are all aware, no two programmers are identical with regard to personal preferences, pet peeves, coding style, indenting with spaces or tabs, etc.
Confession:
I have a somewhat strong fascination with SVG files/elements. Particularly icons, logos, illustrations, animations, etc. The main points of intrigue for me are the most obvious: lossless quality when scaling and usage versatility, however, it goes beyond simply appreciating the format and using it frequently. I will sit at my PC for a few hours sometimes, just "harvesting" SVG elements from websites that are rich with vector icons, et al. There is just something about SVG that gets my blood and creativity flowing. I have thousands of various SVG files from all over the web and I thoroughly enjoy using Figma to inspect and/or modify them, and to create my own designs, icons, mockups, etc.
Unrelated to SVG, but I also find myself formatting code by hand every now and then. Not like massive, obfuscated WordPress bundle/chunk files and whatnot, but just a smaller HTML page I'm working on, JSON export data, etc. I only do it until it becomes more consciously tedious, but up to that point, I find it quite therapeutic.
Question:
So, I'm just curious if there are others out there who have any similar interests, fascinations or urges, behaviours, etc.
*** NOTE: I am not a professional programmer/developer, as I do not do it for a living, but because it is my primary hobby and I am very passionate about it. So, for those who may be speculating on just what kind of a shitty abomination of a coworker I must be, fret not. Haha.
Also, if anyone happens to have knowledge of more "bare-bones" methods of scraping SVG elements from web pages, apps, etc. and feels inclined to share said knowledge, I would love to hear your thoughts about it. Thank you! :)2 -
does recursion have any practical use outside of being a cute/elegant solution under constraints where stack overflow isn't a concern due to small input size, and leetcode?
im having trouble thinking of anywhere you could justify using recursion in industry outside of leetcoding people
i assume the iterative approach would be preferred in scenarios where scaling matters18 -
In the past, apps I've written have used a flat file backend. It's very fast, but obviously clunky to have a big structure of flat files for an app. It ran circles around framework-based RDBMS backends, as performance is concerned, but again, it was clunky. Managing backups and permissions on tens or hundreds of thousands of small files was no fun. Optimizing code for scaling was fun- generating indexes, making shortcuts -but something was still missing. Early in 2017 I discovered redis. A nosql backend that just stores variables and lives almost entirely in memory. Excellent modules and frameworks for every language. It was EXACTLY what I'd needed, even though I didn't know I did. I spent a good deal of time in 2017 converting apps from flat files to redis, and cackled with glee as they became the apps I wanted them to be. Earlier this week, I started building my first app that started with redis, instead of flat files, and I can't stop gushing to anyone who will listen. Redis for president!
-
I understand that front end frameworks are important when scaling an app/project but it’s not a necessity for small student projects.
Can someone make me understand how a student can make use of the frontend frameworks not just for the sake of learning it but the framework actually being a necessity4 -
Received email from Docker that they’re limiting anonymous container pulling to 100 requests per 6 hours.
It’s time to die for this company, that’s their last breath.
More about this on their blog here:
https://docker.com/blog/...2 -
I’ve been very unlucky, undertalented, or underfunded with scaling issues. Can’t think of a time I ever successfully scaled a website or application to a faster performance because of any or all of those three factors.1
-
Question for my fellow devs:
Do you feel like you are spending too much time on maintaining ur devops/infrastructure rather than focusing on the actual product?
Do you think your company would be willing to spend a bit of money to outsource scaling problems to someone else and just focus on the product?
Ik we got lots of fancy new CI platforms like Circle CI GH actions etc but like I personally feel like I’m doing certain infrastructure tasks twice when I look at the two different codebases I work on.8 -
I don't know how to begin.
But i really appreciate you and your efforts if you do the whole devops job through terminal.
I mean creating, upscaling, downscaling, configure, manage an instance just through terminal.
Big d*ck energy6 -
Does anybody have some good suggestion for horizontal scaling solution of relational database (MySQL) that's not commercial? Both read and write scalings are of relevance btw.23
-
Setting up active/dr site that is not allowed to subscribe to any “cloud” services to facilitate scaling/auto failover. Ive resorted to use DNS-based failover which updates the ip attached to the host and re-propogate dns records which took 2minutes to come back online... this shoulve been better if we’re allowed to use cloud-based load balancers
-
Finally, after days through hell I finally made it to scale the wordpress sites at 6.5k req/sec even at a reasonable price.
Through, the data transfer rate will be sky rocketing 😂2 -
AWS Contractor
I've been putting a web application together that I'm looking to have published on AWS. Not having too much experience with AWS, I am looking to hire a contractor. I've had a number of quotes from different AWS admin's ranging from $40 an hour to $200 an hour, from 1-days worth of work to 2-months worth of work!
I'm not really sure what to make of it or to whom to trust. I believe they’re using my ignorance to overcharge me. I've listed my requirements below, could you guys use your professional experiences to let me know what you think is reasonable charge and where best I could find someone to help me.
My application is a US shopping website where people can set up an online shop and upload their products and maintain an inventory of the items.
This is what I’m looking for setup and configuration with the following two areas:
1) AWS SYSTEMS…
* AIM - Set up my server admin users.
* EC2 - Web Hosting.
* RDS - Fast DB.
* SES - To send emails.
* S3 Buckets - Uploaded image hosting.
Route 53 - I don’t know but someone said I should have this.
* Elastic Load Balancing - For, well, load balancing.
2) SCRIPTS…
* A script that would back up the database once a day and save it to a private S3 Bucket.
* A script that will run once a day that calls an internal API, and POST a query to it.
* A script that runs once every 90 days, to refresh the SSL using ZeroSSL.com
Is there anything that I've missed such as security systems, firewalls, auto scaling and CDNs?
The quotes that I've received arranged from $320 to $64,000. I know I am being abused because of my ignorance. I would never overcharge someone because the customer doesn't know the efforts of the work. I hope someone here can help to understand the efforts needed and can tell me the true cost.
Thank you6 -
"my greatest fear in life is my best not being good enough."
Currently, I am building my second business around blockchain.
I am stacking on using the popularity of cryptocurrency and it's novelty to push the product universal.
My limitation (what I think):
1. My environment - unfortunately I live in a third world country
2. Naivety: I have never scaled a business, failed in my first attempt(this is my second).
3. Lack of fund: my budget is pretty low, and no I dont have a family support to raise any for marekting and promoting the business, so I am let with option of scaling it organically ( what "organically" means is spamming social media, forum's comments section to grow customers
4. Really the only option present: most folks here wont know what it means to be in a state of "survival", failing will cause you suffering.
5. Poor network: My friends, or the people around dont understand, cant comprehen what this means.
Cons:
1. I get to know what it means to carry your idea to the world again, this I hope will improve my knowledge base on business and make me less naive.
2. Portfolio boost: "wow!" that should be people's reaction when I tell them about the project.
3. If I succed, I hope the incentive will take me out of this shit hole.
4. I really want to get out of this shit hole - this should work!2 -
A question or more discussion / looking for feedback.
Let's say we have
Application Firewall
- Application Routing
-- API Gateway
Then lots of containers.
Would you use host based routing in the containers?
It seems wrong to me, as in my opinion it breaks the boundary the API gateway provides.
Maybe someone can share his experience, I'd be delighted.5 -
I ranted about my new laptop and linux mint on it https://devrant.com/rants/1919501 and I said there will be a rant about the OSs I tried
So my new laptop is the Xiaomi notebook pro, with the highest config: i7/16g/256g/mx150 gpu/alu body/10h battery/perfect keyboard/great screen. Its Chinese, but Xiaomi... you kinda expect flaws, problems, but i watched all the reviews and knew about all the things, and the price was 35% down (836 + taxes = 997EUR) for a macbook pro clone? its a no brainer.. but i had a rattling vent (fixed with shoe glue lol) now its just loud in windows but not in linux, strange
I changed the Chinese windows on it to EN... worked perfect... but... It has 2 slots for NVMe ssd so i bought a 500gb one for the second slot, I put windows on that (because games, occasional insta story video edit, big files, anyway...) and put Ubuntu on the 256gb original ssd.. (to develop on that) and it was slow as fuck, I got errors all over the places, problems I never had before with ubuntu.. and mind you Windows had over 3000 MB/s for read and almost 2000 MB/s for write speeds on that disk... I was disappointed af. MIND YOU all my life I had Ubuntu on secondary old/slow laptops/pcs working JUST FINE... I still don't know what the fuck happened.. the ui was choppy to say the least and I just was not ready to accept that on this HW while windows worked like a charm (yuck)
Then I went with Manjaro (based on arch, here on devrant people like that stuff, must be great)... well after I installed it, it booted up to the login page and black screen... something with the MX150 GPU according to the interwebs... by this time I was so frustrated and in time stress because of my flight home for xmas that I decided not to fix Manjaro but to go with another flavour
Linux Mint it is... everything kinda works out of the box, like they say... it has dark mode everywhere in the settings without downloading some bloated theme or plugin like on other flavours. So I sticked with Linux Mint. Im not saying its perfect, but I have it for like a month now and all its flaws are these small irrelevant settings not working, utilities like the battery showing funny numbers in the post I linked in the beginning.
Other than this I want to ask you guys. In all 3 distros I tried, they all had text scaling issues everywhere (os, apps, web). I think I have a regular fullHD display, its sharp, but I mean... I never expected resolution or scaling issues or things like that. On Windows I never had those scaling issues... other than the famous win10 "blurry apps"3 -
PHP dev help/advice needed!
We have problems with mysql. Still stuck with mariaDB, I'm using indexes (correct ones) and we have problems with scaling. we have a few tables with over 100mil rows, 1 of them is being read every morning with a subselect that counts unique rows, and fails every time because of timeout/lock, the temp table size was increased and helped for a little while but as time goes on the table grows and the problem reappears. I'm reading from a slave server that was purposely created for read only, yet we still have problems. We're using managed dedicated servers for out hosting and they aren't willing to optimise the database configs for our needs. What are the easiest options for scaling at this point? Going fully dedicated server and perconaDB? NOsql? Sharding the server? Anyone got any good blogposts or something to read about this? your own experience?11 -
How is this for competence...
To push new features or fixes on one of our clients production sites that runs on AWS with auto scaling instances, we have to manually change the files in each instance that is running by ftp, and the template in S3. -
Why must pixel perfect scaling be so fucking hard... It all works perfectly but I'm limited to a max of 540p for the games camera view but as soon as you drop your resolution bellow 1080p you get stuck with 300p or lower... Ughhhhhhhh
Why did I choose to do pixel art... Wonder if converting everything to vectors would help, wonder is smashing my head into a fucking wall would help... -
This *is* a question you silly wrong tagging mother fucker, how dare you doubt me?
Alright, no more disclaimer: I like dungeons and dragons, but it's too fucking much in terms of rules and systems and shit, as in just *making* a character can take a long ass while.
And if that's the highest level of all your ANAL preferences then OK, but I'm not you and things only come OUT of my ass, not inwards, I swear.
Anyhoo, I got fed up with it and wrote my own ruleset and setting as a last fuck you to everyone. It's very simple: if you want to be some kinky magical alien hermaphrodite royal prostitute half sewer dragon princess and three quarters bearded female incest child of demons and fairies then FINE, but you get no bonuses for that shit.
Get it? No complex racial level scaling bullshit, FUCK YOU, race and background is just for vibes, end of story.
You get no attribute or skills or shit to distribute on level one. All you get is a prompt: pick three actions, that's it. You wanna be sexy? Pick "seduce". You wanna set turds on fire? Pick "ignite". Are you an edge lord? Pick "summon". Would you be my wife? Pick "heal", "buff" and "smite".
The game is turn based, and each action you can take is effectively a spell. Everyone can cast a basic spell like walk, attack, talk, crouch, etcetera -- that costs no mana. Special crap like flying and firing fucking electricity costs mana, and you can only do those if you either picked the spell on level one or learnt it later from a book/tutor/demonic bargain/whatever.
Which spells are valid for taking at level one is up to the game master; I just tell people to pick three verbs or short sentences, and if they choose something that's too broken like "split the Red Sea" I'm like nah you're not Moses, try again.
Still with me? Good. You get eight points of health, four points of mana, and one point of stamina. They're all energy, and you can use it to power your magery, but spending all your health means you fucking die.
Stamina recharges fully every turn, and is used for the aforementioned basic actions. All of these cost one point of stamina each. If you run out of stamina, you can use mana. Or your BLOOD.
Level one spells cost one mana, level two cost two and so on. You get back one point of mana each turn, and you can fire all the spells you want during it, long as you have mana. Or BLOOD.
That's good and all, but if you spend anywhere over eleven combined points of energy in one go, you spontaneously combust and die, erasing all signs of life in a twenty-meter radius. This is called incineration, and it *will* leave behind a blackened crater from which the dark servants of the Horror Immemorial may or may not crawl out of.
In case you didn't guess by now, your blood doesn't fucking come back unless you eat, sleep or see a healer.
But anyway, the more points you spend into casting a spell -- and remember, basic attack counts as a spell -- the more powerful it is, so the bigger your diceroll can get. My rule is I add one dice for every fourth point of energy spent, so (1d4), (1d4 + 1d6), (1d4 + 1d6 + 1d8), incineration.
Additionally, for every three points of energy spent, your spell can hit one more target. That's right, you like AoE? Then spend more mana, bitch. Oh, and if you're using shit like poison it lasts one more turn for every two points of energy spent.
How do we calculate damage? Diceroll over two and fuck your mother. Armor class? Resistances? Out of my face with that shit. Damage reduction is called "tyranny" and is for dungeon bosses only.
If you live long enough to get to level two, you *do* get attributes. Pick:
- Grit: +2 health, +1 to fighter shit type rolls.
- Cunning: +2 mana, +1 to rogue shit type rolls.
- Allure: +1 stamina, +2 to wizard shit type rolls.
- Spirit: +1 to elemental shit type spells.
- Faith: +1 to benefactor paragon asshole shit type spells.
- Hatred: +1 to demonic murder hobo destructive shit type spells.
On second level, you can pick one of the spells you know to get +1 to it, specifically. Eh, "+1" just means you get a bonus to some diceroll, no time to explain I'm running out of characters what the fuck.
On level three, the cycle repeats. Pick attr, pick spell. DONE.
Oh right, and weapons. Mostly just vibes, pick your fancy and fuck off. Normally, you can hit things one tile away; if you have a BIG melee weapon you can hit from *two* tiles away, and if you have a ranged weapon you can shoot anyone in sight, but you need to spend one point of energy to reload.
And there, all bases covered in less that 5000 characters with some flair to spare, now suck my fucking cock Hasbro.
What was the question? Oh yeah right, I'm gonna GPL this shit and put it in browsers. I think I'm going to write it in Kotlin but I'm open to suggestions. Would you guys like to play it/contribute to it's development for shits and giggles?8 -
I might give up installing Manjaro... I'm... Too weak... Managed to install it but now my 4k screen is a bit laggy and I'm having some scaling issues with my 1080p second monitor. Idk how to solve those and, since Manjaro has a rolling release model I'm assuming doing anything in this will be frustrating. Not because the OS sucks, but because I lack the knowledge to make it all work smoothly.17
-
Is anyone using Wayland with KDE-Plasma environment already? I'm still waiting for this to work on my laptop. X doesn't support multi-monitor scaling which I desperately need
-
After one year working part time (10%) as new consultant in the company, the consultant calls the technical director in the evening and asks: "hey, how much do you think it will cost to rewrite a SaaS platform somewhat similar to ours from scratch in outsourcing and scaling from the actual 500 customers to 500 thousands? I need to know by tomorrow morning, as we have a pitch with investors. Do you think it's ok to ask for 500K$? Actually, I can only ask for 500K$."3
-
Oh there are lots of good tools to handle different screen sizes like containers, anchors, scaling and all that good stuff huh ?
.
.
.
Sets it to stretch mode instead -
Using AWS DocumentDB elastic cluster for a sharded managed MongoDB
It's ass is so heavy!! More than 15min for scaling 2 -> 3 shards!
Am I taking a bad decision?7 -
For the guys who uses pop! os ( apparently ubuntu breaks on rtx 3070 and don’t have the patience to find the proper drivers ) and i3wm how do you manage the scaling / dpi on different monitors ( i have 4 of them: 4k, 2k, 1920 and ultrawide 2560x1080 ) and apparently on 3 of them the cursor and font is fucking huge and on the 4k is too small. Tried scaling them manually using xrandr but it still doesn’t look right .6
-
Google being Google I can't find a bloody answer, how do I decide to what PHP framework to use for a project? It's an ecommerce platform, not much need for scaling, but I can't bloody figure out what framework to use, either way it will mean learning it from scratch4
-
What’s the best way to manage third party libraries in C++ especially when you’re not just dealing with software but several hardware?
I usually just store each library in its own sub module that gets rebuilt each update/pull, but this is started to get crazy as my project gets larger that it is not scaling.2 -
On a low motivation ride today again.
So, the thing is, mujoco. That's it. Mujoco is very annoying.
Current problem is that I have an xml as a robot description input from elsewhere, but it is not scaling well when I use it as input in another file. And it seems I can't scale everything else down, so I have to scale up the robot. And as I haven't found a way to scale up the entire thing at once, I might have to modify the xml for the arm. And that's annoying because I have to change every single joint and site and other stuff for it to work well...
Fuuuuuuuuck. 😑 -
Hire Shopify Developer
Keeping the equipment static and expanding or diminishing the amount of clients to learn if the presentation is affected relatively to comprehend the consistency of the framework is named unsurprising adaptability.
At the point when a worker isn't skilled handle the current responsibility, the strategy for acquiring new worker to share the responsibility close by the current worker is named scaling out or even scaling. -
Why did the developer refuse to date their AWS bill?
Because it kept scaling their expenses, and their heart just couldn't handle the load! 💔😂5 -
i want to deploy websockets on multiple servers with horizontal scaling. i don't know what to use. redis pub/sub? haproxy? i wanna know your opinions. ❤️6
-
Trying to make a nodejs backend is pure hell. It doesn't contain much builtin functionality in the first place and so you are forced to get a sea of smaller packages to make something that should be already baked in to happen. Momentjs and dayjs has thought nodejs devs nothing about the fact node runtime must not be as restrained as a browser js runtime. Now we are getting temporal api in browser js runtime and hopefully we can finally handle timezone hell without going insane. But this highlights the issue with node. Why wait for it to be included in js standard to finally be a thing. develop it beforehand. why are you beholden to Ecma standard. They write standards for web browser not node backend for god sake.
Also, authentication shouldn't be that complicated. I shouldn't be forced to create my own auth. In laravel scaffolding is already there and is asking you to get it going. In nodejs you have to get jwt working. I understand that you can get such scaffolding online with git clone but why? why express doesn't provide buildtin functions for authentication? Why for gods sake, you "npm install bcrypt"? I have to hash my own password before hand. I mean, realistically speaking nodejs is builtin with cryptography libraries. Hashmap literally uses hashing. Why can't it be builtin. I supposed any API needed auth. Instead I have to sign and verfiy my token and create middlewares for the job of making sure routes are protected.
I like the concept of bidirectional communication of node and the ugly thing, it's not impressive. any goddamn programming language used for web dev should realistically sustain two-way communication. It just a question of scaling, but if you have a backend that leverages usockets you can never go wrong. Because it's written in c. Just keep server running and sending data packets and responding to them, and don't finalize request and clean up after you serve it just keep waiting for new event.
Anyway, I hope out of this confused mess we call nodejs backend comes clean solutions just like Laravel came to clean the mess that was PHP backend back then.
Express is overrated by the way, and mongodb feels like a really ludicrous idea. we now need graphql in goddamn backend because of mongodb and it's cousins of nosql databases.7 -
So, as per usual, I am not sure what I am doing.
I want to make multiplayer games on the web using web sockets.
But of course I want there to be multiple game servers for horizontal scaling (I'm gonna hit it big)
Specifically for either Node.JS or ASP.Net (or both) how could I manage such a thing where there are 2 servers and 2 users. User A is assigned to server A by load balancer, and user B is assigned to server B. But they play in the same game?
Best I know of so far is to connect both game servers to a redis backing. But this seems like a convoluted way to communicate. I would rather have them both route to the game server (Whichever server the game starts on)17 -
Image implementing angular universal which for it self is quite painful... Timeinvestment 8hours
Testing local 12 hours by the team
.. deploying to Google app engine because we need a nodejs server and we do don't have one yet ... Server crashing 24/7 with random errors most are memory related spend 3 days almost rewriting everything ... trying to find the memory leak
Then when I was about to give up stumble over a GitHub issue where someone is saying something about tiers on app engine.
Me going wtf there are tiers everywhere it just says automatically scaling instances ...
Googling .. setting to highest tier .. app works. Apparently I was in lowest tier which only has 156 MB ram app needs 150- 250 MB. Me now crying in corner about my wasted 4 days. -
I really would love to buy a microsoft surface pro, went to the store yesterday to try it out ... and I leaved without buying. I hope windows will some day get some nice scaling on high dpi displays like mac OS but till now it looks terrible IMHO!
Even the Icons in the file explorer look better on fullhd then highdpi -.-