Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Get a devDuck
Rubber duck debugging has never been so cute! Get your favorite coding language devDuckBuy Now
Search - "caching"
After listening to two of our senior devs play ping pong with a new member of our team for TWO DAYS!
DevA: "Try this.."
Junior: "Didn't work"
DevB: "Try that .."
Junior: "Still not working"
Me:"What is the problem?"
Few ums...uhs..awkward seconds of silence
Junior: "App is really slow. Takes several seconds to launch and searching either crashes or takes a really long time."
DevA: "We've isolated the issue with Entity Framework. That application was written back when we used VS2010. Since that application isn't used very often, no one has had to update it since."
DevB: "Weird part is the app takes up over 3 gigs of ram. Its obviously a caching issue. We might have to open up a ticket with Microsoft."
Me: "Or remove EF and use ADO."
DevB: "That would be way too much work. The app is supposed to be fully deprecated and replaced this year."
Me: "Three of you for the past two days seems like a lot of work. If EF is the problem, you remove EF."
DevA: "The solution is way too complicated for that. There are 5 projects and 3 of those have circular dependencies. Its a mess."
DevB: "No fracking kidding...if it were written correctly the first time. There aren't even any fracking tests."
Me:"Pretty sure there are only two tables involved, maybe 3 stored procedures. A simple CRUD app like this should be fairly straight forward."
DevB: "Can't re-write the application, company won't allow it. A redesign of this magnitute could take months. If we can't fix the LINQ query, we'll going to have the DBAs change the structures to make the application faster. I don't see any other way."
Holy frack...he didn't just say that.
Over my lunch hour, I strip down the WPF application to the basics (too much to write about, but the included projects only had one or two files), and created an integration test for refactoring the data access to use ADO. After all the tests and EF removed, the app starts up instantly and searches are also instant. Didn't click through all the UI, but the basics worked.
Sat with Junior, pointed out my changes (the 'why' behind the 'what') ...and he how he could write unit tests around the ViewModel behavior in the UI (and making any changes to the data access as needed).
Junior: "Employee app is fixed. Had some help removing Entity Framework and how it starts up fast and and searches are instant. Going to write unit tests today to verify the UI behaivor. I'll be able to deploy the application tomorrow."
DevA: "What?! No way! You did all that yesterday?"
Me: "I removed the Entity Framework over my lunch hour. Like I said, its basic CRUD and mostly in stored procedures. All the data points are covered by integration tests, but didn't have time for the unit tests. It's likely I broke some UI behavior, but the unit tests should catch those."
DevB: "I was going to do that today. I knew taking out Entity Framework wouldn't be a big deal."
Holy fracking frack. You fracking lying SOB. Deeeep breath...ahhh...thanks devRant. Flame thrower event diverted.13
Oh, man, I just realized I haven't ranted one of my best stories on here!
So, here goes!
A few years back the company I work for was contacted by an older client regarding a new project.
The guy was now pitching to build the website for the Parliament of another country (not gonna name it, NDAs and stuff), and was planning on outsourcing the development, as he had no team and he was only aiming on taking care of the client service/project management side of the project.
Out of principle (and also to preserve our mental integrity), we have purposely avoided working with government bodies of any kind, in any country, but he was a friend of our CEO and pleaded until we singed on board.
Now, the project itself was way bigger than we expected, as the wanted more of an internal CRM, centralized document archive, event management, internal planning, multiple interfaced, role based access restricted monster of an administration interface, complete with regular user website, also packed with all kind of features, dashboards and so on.
Long story short, a lot bigger than what we were expecting based on the initial brief.
The development period was hell. New features were coming in on a weekly basis. Already implemented functionality was constantly being changed or redefined. No requests we ever made about clarifications and/or materials or information were ever answered on time.
They also somehow bullied the guy that brought us the project into also including the data migration from the old website into the new one we were building and we somehow ended up having to extract meaningful, formatted, sanitized content parsing static HTML files and connecting them to download-able files (almost every page in the old website had files available to download) we needed to also include in a sane way.
Now, don't think the files were simple URL paths we can trace to a folder/file path, oh no!!! The links were some form of hash combination that had to be exploded and tested against some king of database relationship tables that only had hashed indexes relating to other tables, that also only had hashed indexes relating to some other tables that kept a database of the website pages HTML file naming. So what we had to do is identify the files based on a combination of hashed indexes and re-hashed HTML file names that in the end would give us a filename for a real file that we had to then search for inside a list of over 20 folders not related to one another.
So we did this. Created a script that processed the hell out of over 10000 HTML files, database entries and files and re-indexed and re-named all this shit into a meaningful database of sane data and well organized files.
So, with this we were nearing the finish line for the project, which by now exceeded the estimated time by over to times.
We test everything, retest it all again for good measure, pack everything up for deployment, simulate on a staging environment, give the final client access to the staging version, get them to accept that all requirements are met, finish writing the documentation for the codebase, write detailed deployment procedure, include some automation and testing tools also for good measure, recommend production setup, hardware specs, software versions, server side optimization like caching, load balancing and all that we could think would ever be useful, all with more documentation and instructions.
As the project was built on PHP/MySQL (as requested), we recommended a Linux environment for production. Oh, I forgot to tell you that over the development period they kept asking us to also include steps for Windows procedures along with our regular documentation. Was a bit strange, but we added it in there just so we can finish and close the damn project.
So, we send them all the above and go get drunk as fuck in celebration of getting rid of them once and for all...
Next day: hung over, I get to the office, open my laptop and see on new email. I only had the one new mail, so I open it to see what it's about.
Lo and behold! The fuckers over in the other country that called themselves "IT guys", and were the ones making all the changes and additions to our requirements, were not capable enough to follow step by step instructions in order to deploy the project on their servers!!!
[Continues in the comments]26
A typical demo...
Me: We added validation, server communication, caching....
Me: We fixed bugs, sped up queries, implemented X features.
Me: We surpassed the speed of light, transcended to another plane of reality, cured cancer, brought peace to galaxy.
UI Designer: I prepared these sketches for the UI
Customer: Wow, so innovative, look at that beautiful transitions, even mobile design, just wow
Me: * dies *11
"it's not a bug it's a feature",
"it worked on my machine",
"i tested it and it worked",
"its production ready",
"your browser must be caching the old content",
"that error means it was successful",
"the client fucked it up",
"the systems crashed and the code got lost" ,
"this code wont go into the final version",
"It's a compiler issue",
"it's only a minor issue",
"this will take two weeks max",
"my code is flawless must be someone else's mistake",
"it worked a minute ago",
"that was not in the original specification",
"i will fix this",
"I was told to stop working on that when something important came up",
"You must have the wrong version",
"that's way beyond my pay grade",
"that's just an unlucky coincidence",
"i saw the new guy screw around with the systems",
"our servers must've been hacked",
"i wasn't given enough time",
"its the designers fault",
"it probably won't happen again",
"your expectations were unrealistic",
"everything's great on my end",
"that's not my code",
"it's a hardware problem",
"it's a firewall issue",
"it's a character encoding issue",
"a third party API isn't responding",
"that was only supposed to be a placeholder",
"The third party documentation is wrong",
"that was just a temporary fix.",
"We outsourced that months ago.","
"that value is only wrong half of the time.",
"the person responsible for that does not work here anymore",
"That was literally a one in a million error",
"our servers couldn't handle the traffic the app was receiving",
"your machines processors must be too slow",
"your pc is too outdated",
"that is a known issue with the programming language",
"it would take too much time and resources to rebuild from scratch",
"this is historically grown",
"users will hardly notice that",
"i will fix it" };11
So a user reported they couldn't login to our site, so I reset their password to:
And sent them back an email with the updated password. A few minutes later, they replied and said that password didn't work. They even tried a different web browser, etc. I tried it myself, and sure enough, it didn't work.
I spent the next several hours trying to figure out why the password didn't save properly, or why the logic didn't compare them correctly. Perhaps it was some sort of caching issue? Oh the horror.
As it turns out, the problem was a maxlength of 28 on the login form field:
<input type="password" name="password" value="" maxlength="28"/>
I don't know who wrote that code, but it sure wasn't me.23
Biggest scaling challenge I've faced?
Around 2006~2007 the business was in double-digit growth thanks to the eCommerce boom and we were struggling to keep up with the demand.
Upper IT management being more hardware focused and always threw more hardware at the problem. At its worst, we had over 25 web servers (back then, those physical tall-rectangle boxes..no rack system yet) and corresponding SQL server for each (replicated from our main sql server)
Then business boomed again and projected the need for 40 servers (20 web servers, 20 sql servers) over the next 5 years. Hardware+software costs (they were going to have to tear down a wall in order to expand the server room) were going to be in the $$ millions.
Even though we were making money, the folks spending it didn't seem to care, but I knew this trajectory was not sustainable, so I started utilizing (this was 2007) WCF services and Microsoft's caching framework Velocity. Started out small, product lookup data (description, price, the simple stuff) and within a month, I was able to demonstrate the web site could scale with less than half of our current hardware infrastructure.
After many political battles (I've ranted about a few of those), the $$ won and even with the current load, we were able to scale back to 5 web servers and 2 sql servers. When the business increased in the double-digits again, and again...we were still the same hardware for almost 5 years. We only had to add another service server when the international side of the business started taking off.
Challenge wasn't the scaling issue, the challenge was dealing with individuals who resisted change.2
Had to wirte and optimize a C++ program that finds for 1000x1000x1000 grid points the 100 nearest points for each (with an additional factor to make it more complicated).
It had to run in under 18 minutes to pass. No matter what I did I couldn't get it fast enough. I tried kd-trees, caching of certain points, optimizing distamce calculations by ommiting any irrelevant factor, saving points' calculated squares etc etc. When Ibwas down to 20 minutes, I realized, that my makefile had an error and ignored the - O3 flag...
Well, it actually ran 5 minutes with -O3.10
(Interview for sde-3 position)
(continuation of https://devrant.com/rants/2132431/... )
Interviewer - *opens laptop. Gives a question.* solve this.
Me - *a bit surprised that such questions were being asked on a sde-3 level*
this is the 4th or 5th question from geeksforgeeks, isn't it? I know the answer to this. Do u still want me to solve it?
Interviewer - *not believing me* Yes
Me - okay. Well this *writing down the original solution mentioned on the site* is the verbatim code mentioned on the website, with complexity O(n^2).
However I feel this is not the optimal solution. Let me write a better solution.
*I provide a better solution*
This has a complexity of O(n log n) . What do you think?
Interviewer - Nope. This could be a lot better.
Me - okay. Let me see. Did some minor changes, added some caching (obviously this will have no effect on the base algorithm) etc
How about now?
Interviewer - nope. Still not good.
Me - okay. Can you tell me how to improve it?
Interviewer - no we are not allowed to solve problems for you. It is not our interview, it is yours.
Me - that makes no sense. Interviews are a two way street. I'd very much like to know the optimal answer to this.
Interviewer - okay
*copies down the answer from geeksforgeeks*
This is good
Me - *at first I thought this was a prank or something. *
I just mentioned this answer here.
Then I spent the next 10 minutes providing a BETTER solution.
May I know how yours is better?
Interviewer - this solution has 2-3 loops. Yours has a function calling itself.
Me - that's called divide and conquer using recursion mf!
Anyways let's take an example and do a dry run.
Interviewer - okay
*we do dry run*
Interviewer - oh yes. Yours ran faster. But it will run fast only sometimes.
Me - yes. Each time the algorithm rolls a dice to decide if it should run fast or slow. You have one goddamn awesome weed dealer man.
I got to go. Thank you for meeting me.16
Google: “Your websites must load the first byte in under 500ms and be fully loaded with no render blocking and local caching of all external site callouts to even begin to rank in Google searches.”
Me: “Ok, Google. Your wish is my command.”
*Looks at Chrome’s memory usage to load a blank page*11
Boss: we have to cache everything.
Me: but some parts of the page are dynamic, we can not cache them
Few days later...
Boss: This part of the page displays the wring content!
Well duh. If content changes with every request caching is probably not a good idea...9
Sometimes I stare at the screen with my void eyes, questioning my abilities and say with a shivery voice "WTF" and refresh the website.
Then it works again.3
I realize I've ranted about this before, but...
First the fact that external services can throw back 500 errors or timeouts when their maintainer did a drunk deploy (but you properly handled that using caching, workers, retry handlers, etc, right? RIGHT?)...
Then the fact that they all speak a variety of languages and dialects (Oh fuck why does that endpoint return a JSON object with int keys instead of a simple array... wait the params are separated with pipe characters? And the other endpoint uses SOAP? Fuck I need to write another wrapper class around the client...)
But the worst thing: It makes developers live in this happy imaginary universe where "malicious" is not a word.
"I found this cloud service which checks our code style" — hmm ok, they seem trustworthy. Hope they don't sell our code, but whatever.
"And look at this thing, it automatically makes database backups, just have to connect to it to DigitalOcean" — uhhh wait...
"And I just built this API client which sends these forms to be OCR processed" — Fuck... stop it... there are bank accounts numbers on those forms... Where's that API even located? What company?
"We can not guarantee the safety of your personal data, use at your own risk [...] we are located in Russia".
I fucking hate these millennial devs who literally fail to get their head out of the cloud.
Somehow they think it's easier to write all these NodeJS handlers and layers around some API, which probably just calls ImageMagick + Tesseract on the other side.
If I wasn't so fucking exhausted, I'd chop of their heads... but they're like hydra, you seal one privacy breach and another is waiting to be merged, these kids just keep spewing their crap into easy packages, they keep deploying shitty heroku apps... ugh.
Weekend so far:
Chrome Update FUCKED UP my website.
Tried to update my server to Ubuntu 16.04. That FUCKED UP in the middle and I didn't have any recent backup.
Went back to old backup. But didn't see any changes in the website. Was wondering that for 1 hour.
Forgot that my website was using cloudflare caching. In the meantime I have changed my DNS settings.
Out of frustration removed website from cloudflare. That FUCKED UP the DNS further.
Now I have no idea how long it will take the DNS to update.
FUCKING F M L8
Spent half an hour trying to figure out why the css isn't working only to realise chrome was caching it the whole time11
Writing more infrastructure than product.
Look, my application requests and transforms data from a single external API endpoint, it's just one GET request...
But I made an intelligent response caching middleware to prevent downtime when the parent API goes down, I made mocks and tests for everything, the documentation is directly generated from the code and automatically hosted for every git branch using hooks, responses are translated into JSONschema notation which automatically generate integration tests on commit, and the transformations are set up as a modular collection of composable higher order lenses!
Boss: Please use less amphetamine.5
Arrived today, hyped!
I have real-world experience with MongoDB, MySQL, Firebase, and caching with Redis.
A colleague in ops recommended this book a while back and I thought I'd give it a whirl to better understand what other options I have available.7
Our web department was deploying a fairly large sales campaign (equivalent to a ‘Black Friday’ for us), and the day before, at 4:00PM, one of the devs emails us and asks “Hey, just a heads up, the main sales page takes almost 30 seconds to load. Any chance you could find out why? Thanks!”
We click the URL they sent, and sure enough, 30 seconds on the dot.
Our department manager almost fell out of his chair (a few ‘F’ bombs were thrown).
DBAs sit next door, so he shouts…
Mgr: ”Hey, did you know the new sales page is taking 30 seconds to open!?”
DBA: “Yea, but it’s not the database. Are you just now hearing about this? They have had performance problems for over week now. Our traces show it’s something on their end.”
Mgr: “-bleep- no!”
Mgr tries to get a hold of anyone …no one is answering the phone..so he leaves to find someone…anyone with authority.
4:15 he comes back..
Mgr: “-beep- All the web managers were in a meeting. I had to interrupt and ask if they knew about the performance problem.”
Me: “Oh crap. I assume they didn’t know or they wouldn’t be in a meeting.”
Mgr: “-bleep- no! No one knew. Apparently the only ones who knew were the 3 developers and the DBA!”
Me: “Uh…what exactly do they want us to do?”
Mgr: “The –bleep- if I know!”
Me: “Are there any load tests we could use for the staging servers? Maybe it’s only the developer servers.”
DBA: “No, just those 3 developers testing. They could reproduce the slowness on staging, so no need for the load tests.”
Mgr: “Oh my –bleep-ing God!”
4:30 ..one of the vice presidents comes into our area…
VP: “So, do we know what the problem is? John tells me you guys are fixing the problem.”
Mgr: “No, we just heard about the problem half hour ago. DBAs said the database side is fine and the traces look like the bottleneck is on web side of things.”
VP: “Hmm, no, John said the problem is the caching. Aren’t you responsible for that?”
Mgr: “Uh…um…yea, but I don’t think anyone knows what the problem is yet.”
VP: “Well, get the caching problem fixed as soon as possible. Our sales numbers this year hinge on the deployment tomorrow.”
- VP leaves -
Mgr: “I was thinking the same thing. I’ll have networking run a trace.”
4:45 Networking run their trace, and sure enough, there was some relative path of ‘something’ pointing to a local resource not on development, it was waiting/timing out after 30 seconds. Fixed the path and page loaded instantaneously. Network admin walks over..
NetworkAdmin: “We had no idea they were having problems. If they told us last week, we could have identified the issue. Did anyone else think 30 second load time was a bit suspicious?”
4:50 VP walks in (“John” is the web team manager)..
VP: “John said the caching issue is fixed. Great job everyone.”
VP: “But the caching is fixed? Right? John said it was caching. Anyway, great job everyone. We’re going to have a great day tomorrow!”
NetworkAdmin: “Ouch…you feel that?”
Me: “Feel what?”
NetworkAdmin: “That bus John just threw us under.”
Mgr: “Yea, but I think John just saved 3 jobs. Remember that.”4
The answer to all your computer issues (now including the caching issues on websites): Restart your computer5
Favorite co-worker conversations:
Guy 1: PHP can be plenty fast! Just put in APC, Memcached, and Varnish and you can handle just about any load.
Guy 2: So you're saying PHP is fast when it doesn't run.1
So, I'm using a new MacBook Air (running Sierra), and while I'm still getting used to it (especially the different Sublime hotkeys), overall it really is quite wonderful. I particularly love the magic touchpad and ease of scrolling/swiping between desktops.
However, I ran into an issue this morning that gave me pause: apparent file caching.
My webpack setup auto-compiles my project when files change, and I noticed something was causing errors -- not really surprising since I was in the middle of fixing the project last night. However, the error it displayed wasn't something I was expecting, and referenced a line I was positive I had removed several hours before calling it a night. Whatever, I was probably mistaken, so I went to remove it.
... It wasn't there.
I double checked that I was looking at the right file. Yep, src/styles/header.scss -- that's the correct file. Figuring webpack was acting up, I killed and restarted it.
So whatever, maybe Sublime cached it. Rather unexpected, but possible, and I am on a mac now... so maybe. So, I closed the file and reopened it. The line wasn't there. I did this twice more. It STILL wasn't there. Maybe I'm going crazy...? I checked the file with cat. The line was there. I checked with vim. The line was still there.
OKAY. I've seen a lot of people with beef with Sublime, and I often defended it. but maybe they're actually right. maybe Sublime really isn't the way to go. :( So, I killed and reopened Sublime, and I checked the file again.
The line STILL ISN'T THERE.
Maybe I'm going crazy? I double, triple, quadruple checked the path. all correct.
Alright; let's try again and make sure I do it properly. I closed everything I had open in sublime (two projects), and quit. I reopened Sublime, navigated to the correct path, and reopened the file...
The offending line STILL wasn't there.
I'm angry at this point and just mash the keyboard. I save the resulting garbage, and cat the file again. No visible changes.
KAJSFLK STUPID PIECE OF <redacted>
okay, whatever. Reboots fix everything, right? So I reboot, and keep the option to re-open everything again ticked.
The terminal comes back up, along with half(?) my browsers, but Sublime doesn't. grrrrrrr.
so I cat the damn thing.
THE GARBAGE IS THERE.
Sublime was doing its job. BUT EVERYTHING ELSE FAILED.
(Oh Sublime, why did I ever question you? 💚)
... but seriously, what the fuck could have caused that? Was the OS caching the file for some programs, but not others? Now I'm questioning the macbook...23
I've found and fixed any kind of "bad bug" I can think of over my career from allowing negative financial transfers to weird platform specific behaviour, here are a few of the more interesting ones that come to mind...
#1 - Most expensive lesson learned
Almost 10 years ago (while learning to code) I wrote a loyalty card system that ended up going national. Fast forward 2 years and by some miracle the system still worked and had services running on 500+ POS servers in large retail stores uploading thousands of transactions each second - due to this increased traffic to stay ahead of any trouble we decided to add a loadbalancer to our backend.
This was simply a matter of re-assigning the IP and would cause 10-15 minutes of downtime (for the first time ever), we made the switch and everything seemed perfect. Too perfect...
After 10 minutes every phone in the office started going beserk - calls where coming in about store servers irreparably crashing all over the country taking all the tills offline and forcing them to close doors midday. It was bad and we couldn't conceive how it could possibly be us or our software to blame.
Turns out we made the local service write any web service errors to a log file upon failure for debugging purposes before retrying - a perfectly sensible thing to do if I hadn't forgotten to check the size of or clear the log file. In about 15 minutes of downtime each stores error log proceeded to grow and consume every available byte of HD space before crashing windows.
#2 - Hardest to find
This was a true "Nessie" bug.. We had a single codebase powering a few hundred sites. Every now and then at some point the web server would spontaneously die and vommit a bunch of sql statements and sensitive data back to the user causing huge concern but I could never remotely replicate the behaviour - until 4 years later it happened to one of our support staff and I could pull out their network & session info.
Turns out years back when the server was first setup each domain was added as an individual "Site" on IIS but shared the same root directory and hence the same session path. It would have remained unnoticed if we had not grown but as our traffic increased ever so often 2 users of different sites would end up sharing a session id causing the server to promptly implode on itself.
#3 - Most elegant fix
Same bastard IIS server as #2. Codebase was the most unsecure unstable travesty I've ever worked with - sql injection vuns in EVERY URL, sql statements stored in COOKIES... this thing was irreparably fucked up but had to stay online until it could be replaced. Basically every other day it got hit by bots ended up sending bluepill spam or mining shitcoin and I would simply delete the instance and recreate it in a semi un-compromised state which was an acceptable solution for the business for uptime... until we we're DDOS'ed for 5 days straight.
My hands were tied and there was no way to mitigate it except for stopping individual sites as they came under attack and starting them after it subsided... (for some reason they seemed to be targeting by domain instead of ip). After 3 days of doing this manually I was given the go ahead to use any resources necessary to make it stop and especially since it was IIS6 I had no fucking clue where to start.
So I stuck to what I knew and deployed a $5 vm running an Nginx reverse proxy with heavy caching and rate limiting linked to a custom fail2ban plugin in in front of the insecure server. The attacks died instantly, the server sped up 10x and was never compromised by bots again (presumably since they got back a linux user agent). To this day I marvel at this miracle $5 fix.1
Unable to fetfh data..
This codebase is my NIGHTMARE. Abandon all your common sense and throw away your education. I thought I've spoken enough when the other developer asked what the problem was and I said, "The code looks like a maze, to be honest." But I spoke too fast. Not only is someone's project structure so avant-garde that it can't be understood in a three dimensional world, it's also so damn hopeless, not even Jesus can save it.
Imagine debugging this fecal matter and before you even go through an in-depth investigation, you are halted by something as simple as searching for a variable. Ah, you want to know how they're fetching (sorry, "fetfhing") the country code? Simple enough, search for "country". No results? You dummy, it's "coutry"! Come on, how can you not know that?!
Oh, you want to correct all the trash? Too bad I stuck with the misspellings and dragged it all the way to my test cases. You want to proceed anyway? Well, I don't know. Do you have access to the other repositories that also use the APIs in this code that return those fucked up responses and will probably fail because I spawned this monster of child and I'm sticking to it up to the very end? Are you sure you want to do that, you little bitch?
I'm seeing red. I think my eyes are bleeding.10
I showed a friend of mine a project I made in two days in Docker and Symfony php. It is a rather simple app, but it did involve my usual setup: Nginx with gzip/cache/security headers/ssl + redis caching db + php-fpm for symfony. I also used php7.4 for the lolz
He complained that he didn't like using Docker and would rather install dependencies with composer install and then run it with a Laravel command. He insisted that he wanted a non-docker installation manual.
I advised him to first install Nginx and generate some self-signed certificates, then copy all the config files and replace any environment-injected values (I use a self-made shell script for this) with the environment values in the docker-compose files.
Then I told him to download php-fpm with php 7.4 alpha, install and configure all the extensions needed, download and set up a local Redis database and at last re-implement a .env file since I removed those to replace them with a container environment.
He sent an angry emoji back (in a funny way)
God bless containerized applications, so easy to spin up entire applications (either custom or vendor like redis/mysql) and throw them away after having played with them. No need to clutter up your own pc with runtime environments.
I wonder if he relents :p9
Like most people I needed some extra cash during uni, so I proceeded to learn CSS + Photoshop (yeah, I know). Followed by PHP and WordPress.
It can be a very shitty platform until you realize that you can stop combining plug-ins from all over the place with dubious code quality and roll your own.
Anyhow I kept at it until I was able to join a niche company doing a quite popular caching plug-in for WP (yeah, W3 Total) when I suddenly became *very* interested in anything and everything performance.
This landed me a very cozy consulting gig in the Nordics - they were using WP for an elephant-traffic website and had run into a myriad of perf issues.
Fixing them and breaking the monolith awarded me with skills in nodejs, linux, asynchronous caching among others.
I was soon in charge with managing the dev boxes for the entire team, and when the main operations dude left, I was promoted to owning the entire platform. (!) Tinkering with Linux for most of my life really came in handy here. (remember Debian potato?)
Used saltstack + aws cloudformation to achieve full parity between all environments. Learned myself some python and all various tips and tricks which in the end amounted to 90% reduction in time-to-first-byte and considerable cost savings.
By the end of the 2yr contract I had turned myself into a fullstack systems engineer and never looked back.
Lawyers not getting along resulted in us having to abandon NewRelic, so I got to learn and deploy the ELK stack as a homegrown replacement, which was super-fun.
Now I work in the engineering effectiveness department of a Swedish fintech unicorn where all languages under the Sun are an option (tho we prefer Python), so the tech stack is unlimited. Infinite tools and technologies, but with strong governing principles and with performance always in mind so as to pick the right tool for the job.
It's like that childhood feeling when you've just dumped a ton of Lego on the floor and are about to build something massive.
I guess the morale here is however disappointed you feel by your current stack - don't. Always strive to make things better, faster, more decoupled, easier to test, etc. and always challenge yourself to go outside the comfort zone.6
I am much too tired to go into details, probably because I left the office at 11:15pm, but I finally finished a feature. It doesn't even sound like a particularly large or complicated feature. It sounds like a simple, 1-2 day feature until you look at it closely.
It took me an entire fucking week. and all the while I was coaching a junior dev who had just picked up Rails and was building something very similar.
It's the model, controller, and UI for creating a parent object along with 0-n child objects, with default children suggestions, a fancy ui including the ability to dynamically add/remove children via buttons. and have the entire happy family save nicely and atomically on the backend. Plus a detailed-but-simple listing for non-technicals including some absolutely nontrivial css acrobatics.
After getting about 90% of everything built and working and beautiful, I learned that Rails does quite a bit of this for you, through `accepts_nested_params_for :collection`. But that requires very specific form input namespacing, and building that out correctly is flipping difficult. It's not like I could find good examples anywhere, either. I looked for hours. I finally found a rails tutorial vide linked from a comment on a SO answer from five years ago, and mashed its oversimplified and dated examples with the newer documentation, and worked around the issues that of course arose from that disasterous paring.
So I ended up storing the markup (rendered from a rails partial) in an html comment of all things, and pulling the markup out of the comment and gsubbing its IDs on document load. This has the annoying effect of preventing me from using html comments in that partial (not that i really use them anyway, but.)
Every step of the way on building this was another mountain climb.
* singular vs plural naming and routing, and named routes. and dealing with issues arising from existing incorrect pluralization.
* reverse polymorphic relation (child -> x parent)
* The testing suite is incompatible with the new rails6. There is no fix. None. I checked. Nope. Not happening.
* Rails6 randomly and constantly crashes and/or caches random things (including arbitrary code changes) in development mode (and only development mode) when working with multiple databases.
* nested form builders
* styling a fucking checkbox
* Making that checkbox (rather, its label and container div) into a sexy animated slider
* passing data and locals to and between partials
* misleading documentation
* building the partials to be self-contained and reusable
* coercing form builders into namespacing nested html inputs the way Rails expects
* input namespacing redux, now with nested form builders too!
* Figuring out how to generate markup for an empty child when I'm no longer rendering the children myself
* Figuring out where the fuck to put the blank child template markup so it's accessible, has the right namespacing, and is not submitted with everything else
* Figuring out how the fuck to read an html comment with JS
* nested strong params
* nested strong params
* nested fucking strong params
* caching parsed children's data on parent when the whole thing is bloody atomic.
* Converting datetimes from/to milliseconds on save/load
* CSS and bootstrap collisions
* CSS and bootstrap stupidity
* Reinventing the entire multi-child / nested params / atomic creating/updating/deleting feature on my own before discovering Rails can do that for you.
I am so glad it's working.
I don't even feel relieved. I just feel exhausted.
But it's done.
and it's done well. It's all self-contained and reusable, it's easy to read, has separate styling and reusable partials, etc. It's a two line copy/paste drop-in for any other model that needs it. Two lines and it just works, and even tells you if you screwed up.
I'm incredibly proud of everything that went into this.
But mostly I'm just incredibly tired.
Time for some well-deserved sleep.8
Microsoft Office Sharepoint Server.
There is no technology on Earth that speaks worse of Microsoft than is this crap. Nothing they ever made (not even Comic Sans) is as bad as Sharepoint.
No proper editor. Everything is slooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooow. To run it you need a state-of-the-art server. There is no way to make the UI modern, as Sharepoint itself is built upon 1995 era HTML. Tables in tables in tables in tables in tables. And even if you do a web part that's readable, it will be wrapped in shit and presented to the client anyway.
It's so easy to break too. Most of the time I was just watching why the fuck it didn't work. Huge problem with caching as well. Deploying any change requires 10 minutes of manual labor.
I get why companies want to use it. Out of the box it's got quite a few very nice features, and aside from the problems setting it up, and hardware requirements, it works decently well.
But I won't come near it unless I'm paid 100$ per hour or starving to death.10
I've optimised so many things in my time I can't remember most of them.
Most recently, something had to be the equivalent off `"literal" LIKE column` with a million rows to compare. It would take around a second average each literal to lookup for a service that needs to be high load and low latency. This isn't an easy case to optimise, many people would consider it impossible.
It took my a couple of hours to reverse engineer the data and implement a few hundred line implementation that would look it up in 1ms average with the worst possible case being very rare and not too distant from this.
In another case there was a lookup of arbitrary time spans that most people would not bother to cache because the input parameters are too short lived and variable to make a difference. I replaced the 50000+ line application acting as a middle man between the application and database with 500 lines of code that did the look up faster and was able to implement a reasonable caching strategy. This dropped resource consumption by a minimum of factor of ten at least. Misses were cheaper and it was able to cache most cases. It also involved modifying the client library in C to stop it unnecessarily wrapping primitives in objects to the high level language which was causing it to consume excessive amounts of memory when processing huge data streams.
Another system would download a huge data set for every point of sale constantly, then parse and apply it. It had to reflect changes quickly but would download the whole dataset each time containing hundreds of thousands of rows. I whipped up a system so that a single server (barring redundancy) would download it in a loop, parse it using C which was much faster than the traditional interpreted language, then use a custom data differential format, TCP data streaming protocol, binary serialisation and LZMA compression to pipe it down to points of sale. This protocol also used versioning for catchup and differential combination for additional reduction in size. It went from being 30 seconds to a few minutes behind to using able to keep up to with in a second of changes. It was also using so much bandwidth that it would reach the limit on ADSL connections then get throttled. I looked at the traffic stats after and it dropped from dozens of terabytes a month to around a gigabyte or so a month for several hundred machines. The drop in the graphs you'd think all the machines had been turned off as that's what it looked like. It could now happily run over GPRS or 56K.
I was working on a project with a lot of data and noticed these huge tables and horrible queries. The tables were all the results of queries. Someone wrote terrible SQL then to optimise it ran it in the background with all possible variable values then store the results of joins and aggregates into new tables. On top of those tables they wrote more SQL. I wrote some new queries and query generation that wiped out thousands of lines of code immediately and operated on the original tables taking things down from 30GB and rapidly climbing to a couple GB.
Another time a piece of mathematics had to generate all possible permutations and the existing solution was factorial. I worked out how to optimise it to run n*n which believe it or not made the world of difference. Went from hardly handling anything to handling anything thrown at it. It was nice trying to get people to "freeze the system now".
I build my own frontend systems (admittedly rushed) that do what angular/react/vue aim for but with higher (maximum) performance including an in memory data base to back the UI that had layered event driven indexes and could handle referential integrity (overlay on the database only revealing items with valid integrity) or reordering and reposition events very rapidly using a custom AVL tree. You could layer indexes over it (data inheritance) that could be partial and dynamic.
So many times have I optimised things on automatic just cleaning up code normally. Hundreds, thousands of optimisations. It's what makes my clock tick.4
"We need high performance so cache basically everything, but we also need to see the newest data at all times that's not cached"
Wut? What the fuck are you smoking?!...
Can I have some?3
TL;DR age != competence
My boss is a fucking computer illiterate self taught programmer.
Don't get me wrong, he can do shit, pretty shitty but it gets done...
But the dude has 38 fucking years old and somehow still searches for keys on the fucking keyboard and struggles to touch type anything...
I sometimes crying the fuck out when I have to help him with something...
I'm having a mini fucking panic attack right now just thinking of it... Fuck
He is our "manager" but doesn't even have the fucking balls to confront his own subordinates when they need to be confronted... Everyone is aware of this and everyone is fucking around... And no one sees any consequences... I wonder why deadlines are always missed...
He is so passive that every fucking thing someone asks he goes and says it is OK...
I was studying same psychology about ignorance and I think he lacks the understanding that shit is hard to do...
We literary had a conversation the other day something like that:
Boss: so, what do you think? One call to the api for it to return all data or multiple calls to return smaller ones?
Me: well... It takes ~180ms just for latency to the server for one call, if you have 10 calls it will take 180*10ms, it is better if we have one call and cache it if necessary on the backend.
( he has no fucking clue wtf caching is, besides browser cache)
Boss: (looking confuse AS FUCK!!) Well, I don't get it... Maybe I'll test it later.
Me thinking: test how you dumb motherfucker? On you fucking workstation with no fucking latency?
There is no fucking test. I'm stating it. IT IS A FUCKING FACT!
Me: well, it takes that for the call to go to the api and come back , its simple math. 1 == 180, 10 == 1800.
Believe it or not git is quickly becoming a piece of software that I hate as much as windows. It’s really disappointing to think that a shit idea like a global .gitconfig came from the same mind that brought us Linux.
The number of times that I’ve committed to a repository several hundred times over the course of several months only to then discover six weeks later that my last thousand commits were using an email that I don’t want associated with that project, or for that matter an email at all when I want to remain anonymous is getting fucking ridiculous.
And of course it’s damn near impossible to go back and fix old commits without major surgery and being the admin of the repo.
You know code is bad when it’s GPL licensed and it’s still threatening your privacy on a large scale multiple times. How does one fuck up version control that badly? God damn it I have to authenticate to push, just mark my commits with that user tag. It’s arbitrary and nice.
If that doesn’t work with the local and remote schema then maybe you should have fucking thought of that before you designed it. News flash... I have never found that useful at all other than caching commits offline when I don’t have internet.
Im so enraged over this that I can’t sleep21
What features would you want in a logger?
Here's what I'm planning so far:
- Tagged entries for easy scanning of log file
- Support for indenting to group similar sequential entries
- Multiple entry types (normal, info, event, warning, error, fatal, debug, verbose)
- Meta entries, so the logger logging about itself, e.g. disk i/o failures.
- Ability to add custom entry types, including tag, log-level, etc.
- Customizable timestamp function
- Support for JS's async nature -- this equates to passing a unique key per 'thread'; the logger will re-write all the parent blocks for context, if necessary. if that sounds confusing, it's okay; just trust that it makes sense.
- Caching, retries, etc. in the event of disk i/o issues.
- Support for custom writers, allowing you to e.g. write logs to an API rather than console or disk.
How about these features?
- Multiple (named) logs with separate writers (console, disk, etc.)
- Ability to individually enable/disable writing of specific entry types. (want verbose but not info? sure thing, weirdo!)
- Multiple writers per log. Combined with the above, this would allow you to write specific entry types (e.g. error, warning, fatal) to stderr instead of stdout, or to different apis.
- Ability to write the same log entry to multiple logs simultaneously
What do you think of these features?
What other features would you want?
I'm open to suggestions!19
Recently for a project I needed to read/write ID3 tags from MP3 files. And after a long search, I found this bloated, monolithic but quite stable library, "getID3".
So, I was looking through the code-base and I found this. This guy literally storing the key value based data embedded as comments within the class file. Then wrote a method to parse the data and even used caching to ensure maximum speed! And such usage is repeated all over the code-base.
So, this is what people used do before arrays were invented :314
Inmates are trying to take over the asylum again.
Got a message from the web team manager deeply concerned because since switching to the new logging framework, the site is significantly slower.
She provided no proof or any data to what 'significantly slower' means.
#1 The 'new logging' has been in place and logging for 5 years. We only recently depreciated the ILogger interface ('new' ILogger interface only has 1 method instead of 5)
#2 The 'old logging' was modified 5 years ago, so even if you were using the 'old' interface, the underlying implementation is still the same.
She tried to push the 'it wasn't this slow before' argument, so I decided to do some fact based analysis.
Knowing they deployed their logging changes couple of weeks ago, I opened up AppDynamics, looked at the average call time to Splunk (along with a few other http calls they are doing)
- caching services - 5ms
- splunk - 30ms
- Order Service - 350ms
- Product Data Service -525ms
Then I look at the data they are logging, for the month of June, over 5 million messages. At 30ms each, that's almost 42 hours spent logging errors...yes errors. Null reference exceptions, Argument exceptions, easily fixable stuff.
So far for the month of July (using the 'new' logging), almost 2.5 million errors. Pretty close so far with June's numbers.
My only suggestion was to fix the bugs in their code so they don't log so many errors.
Her response.."Can we have one of our developers review your logging code? We believe we can find ways to optimize the http requests"
Oh good Lord. I'm not a drinking man..but ...I might start.2
I hate time.
Yes, that dimension which unidirectionally rushes by and makes us miss deadlines.
Also yes, that object in most programming languages which chokes to death on formatting conversions, timezones, DST transitions and leap seconds.
But above all, I hate doing chronological things from the point of view of code, because it always involves scheduling and polling of some kind, through cron jobs and queues with workers.
When the web of actions dependent on predicted future and passed past events becomes complicated, the queries become heavy... and with slow queries, queues might lock or get delayed just a little bit...
So you start caching things in faster places, figure out ways to predict worker/thread priorities and improve scheduling algorithms.
But then you start worrying about cache warming and cascading, about hashing results and flushing data, about keeping all those truths in sync...
I had a nightmare last night.
I was a watchmaker, and I had to fix a giant ticking watch, forced to run like a mouse while poking at gears.
I fucking need a break. But time ticks on...2
I had spent the last year working on a online store power by woocommerce with over 100k products from various suppliers. This online store utilized a custom API that would take the various formats that suppliers offer their inventory in and made them consistent. Now everything was going swimmingly initially, but then I began adding more and more products using a plug-in called WP all import. I reached around 100k products and the site would take up to an entire minute to load sometimes timing out. I got desperate so I installed several caching plugins, but to no avail this did not help me. The site was originally only supposed to take three to four months but ended up taking an entire year. Then, just yesterday I found out what went wrong and why this woocommerce website with all of these optimizations was still taking anywhere from 60 to 90 seconds to load, or just timing out entirely. I had initially thought that I needed a beefier server so I moved it to a high CPU digitalocean VM. While this did help a little bit, the site was still very slow and now I had very high CPU usage RAM usage and high disk IO. I was seriously stumped the Apache process was using a high amount of CPU and IO along with MYSQL as well. It wasn't until I started digging deeper into the database that I actually found out what the issue was. As I was loading the site I would run 'show process list' in the SQL terminal, I began to notice a very significant load time for one of the tables, so I went to go and check it out. What I did was I ran a select all query on that particular table just to see how full it was and SQL returned a error saying that I had exceeded the maximum packet size. So I was like okay what the fuck...
So I exited my SQL and re-entered it this time with a higher packet size. I ran a query that would count how many rows were in this particular table and the number came out to being in the millions. I was surprised, and what's worse is that this table belong to a plugin that I had attempted to use early in the development process to cache the site. The plugin was deactivated but apparently it had left PHP files within the wp content directory outside of the actual plugin directory, so it's still executing scripts even though the plugin itself was disabled. Basically every time I would change anything on the site, it would recache the whole thing, and it didn't delete any old records. So 100k+ products caching on saves with no garbage collection... You do the math, it's gonna be a heavy ass database. Not only that but it was serialized data, so when it did pull this metric shit ton of spaghetti from the database, PHP then had to deserialize it. Hence the high ass CPU load. I had caching enabled on the MySQL end of things so that ate the ram. I was really desperate to get this thing running.
Honest to God the main reason why this website took so long was because the load times made it miserable to work on. I just thought that the hardware that I had the site on was inadequate. I had initially started the development on a small Linux VM which apparently wasn't enough, which is why I moved it to digitalocean which also seemed to not be enough, so from there I moved to a dedicated server which still didn't seem to be enough. I was probably a few more 60-second wait times or timeouts from recommending a server cluster to my client who I know would not be willing to purchase it. The client who I promised this site to have completed in 3 months and has waited a year. Seriously, I would tell people the struggles that I would go through with this particular site and they would just tell me to just drop the site; just take the money, just take the loss. I refused to, this was really the only thing that was kicking my ass. I present myself as this high-and-mighty developer like I'm just really good at what I do but then I have this WordPress site that's just beating the shit out of me for a year. It was a very big learning experience and it was also very humbling as well, it made me realize that I really don't know as much as I think I might. It was evidence that there is still so much more to learn out there, I did learn a lot from that experience especially about optimizing websites the different types of methods to do that particular lonely on the server side and I'll be able to utilize this knowledge in the future.
I guess the moral of the story is, never really give up. Ultimately things might get so bad that you're running on hopes and dreams. Those experiences are generally the most humbling. Now I can finally present the site that I am basically a year late on to the client who will be so happy that I did not give up on the project entirely. I'll have experienced this feeling of pure euphoria, and help the small business significantly grow their revenue. Helping others is very fulfilling for me, even at my own expense.
Anyways, gonna stop ranting. Running out of characters. If you're still here... Ty for reading :')8
I don't know why, but each time I have the chance to create a caching system with redis to e.g. cache requests to APIs I get all excited about it.5
Fucking hate my job 😡
I joined as nodejs dev at a mnc 3months ago involved in banking software in which i dont have any domain knowledge.. first 10 days I was told to go through fucking udemy nodejs and graphql tutorial (wtf) which i already have experience with before joining.. after that my reporting manager gives me task to resolve fields and gave me shitty jira story link to read.. that shit story link had no explanation about the fields and what the database it is, then she says to use some shitty sdk which is built internally by shiity devloper which had no documentation and have to follow other module which was again written by that sr. Dev... They hav fucked up the graphql and nodejs and entire stack and also till date no one has ever given any explanation about the domain and the fields and database schema.. this manager refuses to share knowledge about the domain now how the fuck i resolve the graphql schema which was again written by non technical b.a.. all they have used is latest technology in a shitty way with no standards to to follow .. no dataloading no caching no batching.. use shitty sdk which does not give access to dbconn and fucking tightly coupling expressjs which when i start consumes crazy 400Mb of memory .. these fucking seniors devs + the fucking b.a having 12+. Yrs exp each have fucked the entire codebase... Each day killing my passion for app development.. fuckkk ... Dunno what to do now7
Many years ago I was told by a senior dev that caching is one of the hardest things I'll ever come across. At the time I didn't know what he meant, but these past few days I'm starting to understand what he meant. It's not using the cache itself, it's the cache management that is hard. Determining what needs to be cached, when a cache should be invalidated, how long should something be cached etc. It's pretty insane when you start having to compare it with the requirements.1
When I think how big companies like Google, Amazon, Netflix manage all their services so well (all those load balancings, caching, etc. ), I feel so bad I don't know anything about it. I don't even know how people decide which technology to use. E.g., for a scalable Web app, one can use Node.js or maybe Django (but not definitely Flask, I suppose). Also, which DBMS to use, how to write flawless APIs. You see, I am just beginning my career. Any recommendations (books, videos, etc.) that teach these things? Please help.4
Is your code green?
I've been thinking a lot about this for the past year. There was recently an article on this on slashdot.
I like optimising things to a reasonable degree and avoid bloat. What are some signs of code that isn't green?
* Use of technology that says its fast without real expert review and measurement. Lots of tech out their claims to be fast but actually isn't or is doing so by saturation resources while being inefficient.
* It uses caching. Many might find that counter intuitive. In technology it is surprisingly common to see people scale or cache rather than directly fixing the thing that's watt expensive which is compounded when the cache has weak coverage.
* It uses scaling. Originally scaling was a last resort. The reason is simple, it introduces excessive complexity. Today it's common to see people scale things rather than make them efficient. You end up needing ten instances when a bit of skill could bring you down to one which could scale as well but likely wont need to.
* It uses a non-trivial framework. Frameworks are rarely fast. Most will fall in the range of ten to a thousand times slower in terms of CPU usage. Memory bloat may also force the need for more instances. Frameworks written on already slow high level languages may be especially bad.
* Lacks optimisations for obvious bottlenecks.
* It runs slowly.
* It lacks even basic resource usage measurement.
Unfortunately smells are not enough on their own but are a start. Real measurement and expert review is always the only way to get an idea of if your code is reasonably green.
I find it not uncommon to see things require tens to hundreds to thousands of resources than needed if not more.
In terms of cycles that can be the difference between needing a single core and a thousand cores.
This is common in the industry but it's not because people didn't write everything in assembly. It's usually leaning toward the extreme opposite.
Optimisations are often easy and don't require writing code in binary. In fact the resulting code is often simpler. Excess complexity and inefficient code tend to go hand in hand. Sometimes a code cleaning service is all you need to enhance your green.
I once rewrote a data parsing library that had to parse a hundred MB and was a performance hotspot into C from an interpreted language. I measured it and the results were good. It had been optimised as much as possible in the interpreted version but way still 50 times faster minimum in C.
I recently stumbled upon someone's attempt to do the same and I was able to optimise the interpreted version in five minutes to be twice as fast as the C++ version.
I see opportunity to optimise everywhere in software. A billion KG CO2 could be saved easy if a few green code shops popped up. It's also often a net win. Faster software, lower costs, lower management burden... I'm thinking of starting a consultancy.
The problem is after witnessing the likes of Greta Thunberg then if that's what the next generation has in store then as far as I'm concerned the world can fucking burn and her generation along with it.6
Why the hell do so many Android games require internet? I want to play it in undergound when I have nothing better to do! If I had Internet I could be doing something usefull instead of playing stupid games! Even devRant survives by caching in the stations and doesn't f***ing close whenever I'm offline!5
Finding the right balance between well written, need-one-week, maintainable software, and fast-written, ready-in-2-hours-and-never-look-at-it-again software.
Last time it took me 20 minutes to integrate with a new API. I had a script that did everything you needed. I then spent 2 weeks on handling error responses, unexpected responses, exceptions, intelligent retries, logging, unit tests, integration tests, caching, documentation, etc.
I've seen some rants about people complaining about websites using the 'www' subdomain, so I'd like to take this opportunity to try to explain my opinion about why sites might use it.
I use to feel the same way about not having the www subdomain. It felt like an outdated standard that serves no purpose. But I have changed my option...
Sometimes certain servers have other services running other than just the website, such as ssh, ftp, sql, etc., running on different ports. What if you want to use a web proxy and caching service similar to cloudflare or a cdn? We'll you can't, because they won't allow traffic to flow through to your other ports.
That's where the www subdomain comes in. Enable your caching and cdn on your www subdomain, and slap a 301 redirect from your primary domain on port 80 or 443 to the www subdomain. This still allows you to access your other services via the domain name while still gaining the benefits of using a cdn.
Now I know you could use an 'ftp' subdomain or the like, but to each their own in that regard.7
Why WordPress is not very good:
I wrote a quick 230 line python script that uses the power of urllib, ebooklib and 12 regular expressions that would make any italian proud to download webnovels from virlyce.com and turn them into .epub files for me.
The chapters are all individual WordPress pages, and after sequentially downloading only 202 of them I got an internal server error.
Of course, I saw this coming and put mitmproxy to good use caching everything, so even though my python script with terrible error handling crashed I don't have to do it all again (yay)4
So I was assigned to improve an existing internal CMS application where they wanted the ability to add extra form applications and restricting them based on people from different departments. As well as include some other improvements like speed as they mentioned that it was slow in some instances.
What I found was the original developer decided to not use any kind of framework and decided to be creative by creating his own MVC framework. With about 300 users in this system and utilising no caching of queries, views, not even using PHP OpCache, even quite a few security holes, I was damn surprised at how this thing was running. I asked the original developer why he didn't use an open source framework and he said that he thought that he'd create something and be the next Facebook.
It was a mammoth task to "improve" this system but the main thing was that I took custody of this project and that I prevented him from trying to make a bigger mess of things for this project.
WHY DOES GOOGLE CHROME CACHE THIS SHIT AND WON'T LOAD IT AGAIN. I THOUGHT I DIDN'T FIX THE BUG BUT GOOGLE CHROME IS THE BUG. THIS FLYING FUCK9
While styling some frontend stuff with LESS, I experienced that on one page template the <header> was not displaying the given line-height eventhough the whole fscking code was 1:1 identical with the other template in which everything was fine. I checked EVERYTHING... caching, URL, source, classes, open / wrong tags, HEAD, ... I even did a diff compare. NO FSCKING DIFFERENCE!
After one hour of pulling out hair I suddenly saw that in the faulty template file 2 lines were missing:
WHOEVER DID THIS: YOU ARE FSCKING STUPID!!! (it was me...)7
Caching is a subject, misunderstood by a lot of developers. Optimize then cache, don't cache to optimize.2
Other Team: "our builds don't clean up properly"
*docker rm -f $(docker ps -a)*
Other Team: "our builds keep filling up machines"
*docker rmi $(docker images -q)*
Random Team Member: "My builds keep failing on service foo randomly exiting"
Other Random Team Member: "Why is there no caching on our builds"
team panics thinking it's their fault as our main job still passes, as it's on another machine.
When we find out after tracking build history
I'm really sick of elitist JS/front-end devs acting like these front-end heavy sites are any better than a traditional site using SSR (server-side rendering). Single page apps (SPAs) have 1 large benefit over an app with full page requests: the web dependencies (CSS,JS,etc) don't have to be looked up and downloaded on every page load. With optimized caching headers and HTTP2, this is not a problem. I agree with every point this guy makes: https://blog.usejournal.com/spa-or-...7
After this weekend's work with React and Material UI, I started thinking about the downsides to going that route (particularly social media sharing & SEO). The obvious solution is to render server-side, but then you have to ask yourself...is there really a benefit to all of this, or are we just coming full-circle and pretending this is better? I'm all for something that improves maintainability, performance, and reliability, but I feel the more I try to keep up with the latest fads, the slower I work, the more compiling and caching b.s. I have to deal with, and the less I trust the final product to "just work". Anyone else feel this way?4
Of course the shouting episodes all happened during the era I was doing WordPress dev.
So we were a team of consultants working on this elephant-traffic website. There were a couple of systems for managing content on a more modular level, the "best" being one dubbed MF, a spaghettified monstrosity that the 2 people who joined before me had developed.
We were about to launch that shit into production, so I was watching their AWS account, being the only dev who had operational experience (and not afraid to wipe out that macos piece of shit and dev on a real os).
Anyhow, we enable the thing, and the average number of queries per page load instantly jumps from ~30 (even vanilla WP is horrible) to 1000+. Instances are overloaded and the ASG group goes up from 4 to 22. That just moves the problem elsewhere as now the database server is overwhelmed.
Me: we have to enable database caching for this thing *NOW*
Shitty authors of the monstrosity (SAM): no, our code cannot be responsible for that, it's the platform that can't handle the transition.
Me: we literally flipped a single switch here and look at the jump in all these graphs.
SAM: nono, it's fine, just add more instances
Me: ARE YOU FUCKIN SERIOUS?
Me: - goes and enables database caching without any approvals to do so, explaining to mgmt. that failure to do so would impair business revenue due to huge loading times, so they have to live with some data staleness -
SAM: Noooo, we'll show you it's not our code.
SAM: - pushes a new release of the monstrosity that makes DB queries go above 2k / page load -
Tho on the bright side, from that point on I focused exclusively on performance, was building a nice fragment caching framework which made the site fly regardless of what shitty code was powering it, tuned the stack to no end and learned a ton of stuff in the process which allowed me to graduate from the tar pit of WP development.5
I remember when I had my first "website job" and I put it on a test domain to show it to the client.
But when I changed things up in the css or something, the client wouldn't see the updates.
It took me a whole bunch of time to figure out it was caching. So I told him to use an anonymous tab. Fun time, back then..
How do professionals manage these things? :D6
I've implemented an in memory caching system for database queries with Redis in one of the blogs I manage.
Will it work well? Or do you think it will produce issues? I have no experience with Redis yet.14
So I had been developing a real estate website and developing a MLS feed parser. I had only 1 year experience at that time and parsing a XML feed was already complex enough. On top of it, the client wanted to automate feed download from the MLS provider through HTTP authentication. Managed to do it. Everything worked for 15 days and on 16th day the property location markers stopped appearing on Google maps. Turned out that address to lat-long reverse geocoding was failing because API limit exhausted. My bad, I coded it on view instead of caching the lat-long in database. Fixed it in a day and viola!
The moment when your CEO freaks out because he can’t login and starts insulting your architecture because he thinks its a caching issue even though it is a cookie issue because his CTO changed the domain on your cookie for no reason.2
Ooooh MOTHERFUCKER. God fucking dammit. Jesus FUCKING christ. Motherfucking local caching on firefox and chrome. Reload the MOTHERFUCKING PAGE, it's why I pressed CNTRL R you fucking blighted cunts.
Some days I wish I had a brick to toss at the fucking head of the nearest chrome/firefox developer.
Fucking assholes. Eat shit and die alone of cancer fucksticks.10
** this means words are muted **
I send a mail the client a Google doc with elaborate details about evaluation of an Android tablet from a Chinese manufacturer.
The client is upset, he says "You say there is no GPS chip on the tablet while the manufacturer says otherwise"
Me- "I have clearly mentioned that it has a GPS chip"
Client- Opens the Google doc, points to a sentence. Looks at me like I did something horrible.
Me - **This guys is either word blind or something else is wrong with him, the line reads 'GPS chip available'**
Me- "Look, it says 'GPS chip available'.
Client- **Blinks n blinks again** "Alright, but why did you share a Google document, why not PDF, docx"
Me-**Politely** "You can download the document in any format, look I will show you..."
Client- "It should have been in the mail itself ideally"
Me- **WTH** "We normally maintain a document for such things to keep everything organised, but if you want I will put everything in mail itself"
Client- "Hmm.. do both from next time"
Me- "Alright" **BS**
Client- "Why is the new feature taking so much time"
Me- "As planned earlier, we going to deliver it tomorrow"
Client- "Why not today??" **Gives a strange look.**
Me thinking - **Enough**
Me- "See, I am trying to integrate a smarten with a socket connection, reading it's data via exposed APIs that are hardly documented, we need faster performance so I need to implement caching, multi threading, offline handling, multiple processes to avoid memory fluctuations, sync adapter to sync data...."
Client- "Ok ok ok, it's fine if you give working build tomorrow"
Me- "Ok, fine"
Fuck XCode! -
Yesterday I had the stupid idea to rename an icon file. Checked that XCode was building the application still fine. Ran it over the build server: Failed, complaining about the old missing icon file! Checked again and again, but there was no friggin' reference to the old file in the whole repo.
Log in to the machine clear the build folder and try to build the component again. Bang still same error and the references to no longer existing files reappear.
Turns out XCode was caching those references somewhere in the home directory as "DerivedData" and after deleting those, I could build again... but why on earth are you building a cache if you cannot properly invalidate it? Just to waste our time?
Goddamnit! Why is the app not picking up the new maintenance status from the server! Oh fuck... Caching....
We released a website for a client 10 days ago. The site was up and running and everything seemed fine.
But it turns out that the site used 15GB of bandwidth in 5 days ( WTF???).
So now I need to go and examine my code to make sure I didn't forget something and implement a new caching methods to try and reduce the amount of bandwidth being used.
But I still don't understand how a small "newspaper" website with a max upload size of 5MB could of used so much in so little time.
I also added a screenshot showing the number of visits from an addthis dashboard5
It sometimes really sucks to see how many developers, mostly even much better than me, are too lazy to implement a function to its full UX finish.
Like how can you not implement pagination if you know there's going to be fuck ton of content, how can you not allow deleting entries, how can there be no proper search of content, but instead some google custom search, how can you not implement infinite loading everywhere, but only in parts of your application, how can you not caching a rendered version or improve the page, that loads EVERY SINGLE ENTRY in your database with 11k entries, by just adding a filter and loading only chunks of it.
I know sometimes you need to cut corners, but there's rarely any excuse with modern toolsets to just write 3 lines more and have it ready for such basic things.
I sometimes just wake up during the night or before falling asleep and think "oh, what if in the future he might want to manage that, it's just another view and another function handling a resource, laravel makes that very easy anyway", write in on my list and do it in a blink the next day, if there's nothing else like a major bug.
I have such high standard of delivery for myself, that it feels so weird, how somebody can just deliver such a shitty codebase (e.g. filled with "quick/temporary implementations"), not think of the future of the application or the complete user and or admin experience.
Especially it almost hurts seeing somebody so much more versatile in so many areas than me do it, like you perfectly fine know how to cache it in redis, you probably know a fuck ton of other ways I don't even know of yet to do it, yet you decide to make it such a fucking piece of shit and call it finished.3
Delivered a website to a client, pagespeed results 99 on both mobile and desktop (just 1 minus point for implementing Google analytics- since it has a too low caching time). Client insists on having a 100 pagespeed result even it doesn't change real pagespeed ... -.-2
Firefox is getting better and better!
There are even some nice new features like the screenshot tool (in 58 available for the private mode, too!) that are really exciting.
I am looking forward that Firefox could maybe become again what it was back then! It's so cool!6
In Django code, looking at a class for caching REST calls. The cache is using Redis via Django's cache layer. In order to store different sets of parameters, each endpoint gets a "master" cache, that lists the other Redis keys, so they can be deleted when evicting the cache. Something isn't right, though. The cache has steadily increased in size and slowed down since 2014 even though many events clear the whole thing!
... And then it hit me. Nothing empties the list of cache keys. Nothing. So it has been growing endlessly since 2014. And everytime it grows, cache eviction gets a little more expensive, network traffic increases a little more, and cache evictions get a little slower.
Fixing this bug took things that were taking routinely an entire minute to complete and made them take a couple seconds.
CTO at my previous company think that wordpress based website is took a long time to load.
I suggest to use caching and fix ton of abusive query, He refused. He spun up more VM, upgrade the ec2 instance level to the max. Said that he resolved the problem. But the problem still persist actually.
Blame me for slow response website, blame me for late of deployment because data is not ready ( there's a lot of spam in there, we need to clean it before )
I left the company, Coworker said that he just install a bunch of caching plugin,
He made the website down for entire day and don't understand what is happening. Ask other developer to fix it quickly, to do unpaid overime
The site is back to bussiness, said to all team that he already fixed it.
Everything good happened, he claimed that it was his idea.
And the best part is : he put 'ssh' as skill list in his personal site1
I just setup an apt-cache on my Macbook. Docker no longer takes 10 years to `apt-get install` when I'm at the coffee shop. This coffee shop is going to loose so much money now that my work is done faster.
well, there was this time I was facing a bug in production, http requests kept receiving partial data randomly (so downloaded files were always incomplete), but it was working well on other platforms, turned out after 9 hours of research, that I forgot to disable caching
The day my co-worker wraps his mind around Varnish and/or other caching mechanisms will be the day I rejoice unto the lords of the interwebz...2
You're asking why I am so fucking angry at this piece of software? Well because I was awake at night for 3 hours reinstalling my mail system because I thought the Web UI was broken due to a corrupted database. Guess what - the caching of chrome caused the buttons to silently hide beneath the header of the UI. Hahahahahakillmehahah
To be fair, this could have happened with every browser. But since everyone is on the "anti-google" trip anyway, I'm gonna switch to Firefox 🙃1
Oh, you've found a work around for your browser caching? No problem, here's DNS caching so you can fuck with your code again... and not know what the issue is
~ Sincerely, ISP4
I'd been with the company for maybe two weeks, pushed some changes and updates to a client's site on a Friday afternoon as instructed by my boss, checked everything over and it's all fine.
Come Monday morning and this client is seriously miffed, not all of the changes had applied and the site was a mess all weekend. Turns out a bug with the caching plugin meant what we were getting in the office was different to outside.
Meetings were held and a new QA procedure was put in place.
Chrome has failed me. At least, I was disappointed.
So, I have been working with an animation studio to make some changes to their Website, typical WordPress website.
Nothing wrong there, I have a copy of their WP site running on a localhost so I can make changes & tests before pushing to bitbucket (then to be deployed). Now, a lot of the changes I have been making are minor css, html & js changes. Mostly FrontEnd changes.
The frustration came when working on a couple JS sheets; I would change some CSS and JS, save the files then go over to Chrome to test them out.
Open the localhost and test the changes, CSS changes worked! Looks good, but for what ever reason the JS functionality would not change. 2 ish hours of frustration, seeing only half of these changes working I decide to step out for a coffee break. Then I remembered; Chrome has a nasty habit of caching files it has used before for later use. Turns out it was using some older versions of the files that it had cached.
Thankfully I remembered this; only ended up being 2 hours of frustration. For anyone else using Chrome for development; keep this in mind.1
This morning me and my colleague had huge debate about using GraphQL or REST. While I was in total favour of GraphQL, that guy was more on REST side because he read some random articles on dev.to and medium and was highly motivated to use REST instead of GraphQL.
The problem is, some people write anything on blogging websites without even doing a proper research.
Since, I have worked on GraphQL, I knew it's pros and cons very clearly and what are the things that can be done to solve them.
The guys said that we can't do native caching in GraphQL at which the lava from my head just got burst out.
I showed him the official GraphQL docs where it was clearly mentioned that we can do caching in GraphQL.
Poor guy couldn't say anything after that.
P.s: We are still going to use old school REST APIs but I am happy that I could prove my point. I'll use GraphQL in my side projects anyway, loss for him if he's not exploring something new.7
Let's see what's on the menu today:
* Web Application Catastrophe Special *
Includes, but not limited to:
- Orphaned server processes in the configuration management cluster
- Microservice back-end architecture with no API documentation
- Poorly implemented cache microservice with no documentation
- Stale data causing everything to be shown as down in production, despite everything running fine
Cost: 1 developer's sanity
Today I have created a server application on Python Tornado which can forward TCP Packets directly to HTTP request queue without any intermediate caching.
Our remote IOT devices (microcontrollers with sensors attached) send sensor reading over TCP Socket to our server and all the connected web applications can show the data instantly using long polling and the above mentioned technique.1
Ugh, I feel like fucking crying every time I have to explain to the other developers and network sysadmins that we can't have the same Redis server for both session and caching data in the current project we're working on. I wrote all this down in the specifications which NO ONE reads!!
There are times like these I wish I just had access to the AWS account so I can do all of this myself.3
I just love optimizing stuff and here is my story I think somehow is very cool as you can see the progress just looking at two values.
//NOTE: This api relies on another api which is slow af. The requests are done here directly but later will be carried out async and in the background by using some ticket system.
1) First PoC
2) Used caching of requests and improved method to filter out unnecessary stuff
3) Optimized response formatting
NOTE to myself: Things are going too good...4
One of the most annoying thing to explain to clients is caching and why their site isnt updated......,1
My first software boss who didn't like sharing the source code...
OR using built in classes
and insisted on using frames because "they prevent caching" 😂2
My biggest hurdle has been having to explain every aspect of development to every non-technical MD I work with because my is non-technical as well.
Just this morning I've had to explain WSGI, caching, nginx and why different browsers render things differently.
Optimizing my RESTful API plugin, used in a CMS to make it headless, was extremely satisfying.
Thanks to caching and alorithm improvements, I brought the response times down by a factor of 150.
The caching also includes dependency tags thus it will automatically invalidate on changes in the dependent element.1
Google PageSpeed Insights can kill my motivation. Few days ago I launched a site and everything is awesome, page loads around 0.535 seconds, caching enabled, images optimized. Than my boss run it on Google PageSpeed Insights result was 85/100 then my boss said why I can't get 100/100.3
When it comes to config files for any kind of application, I tend to make sure that I understand what each config does, and that for each environment, they have the correct settings.
However some of my co-workers don't bother understanding what the configs mean and so I have people copying and pasting config details from development environments into staging and production configs. They think that just changing hostnames is more than enough.
So they ended up wasting a good 3 hours trying to figure out why sessions were always invalidating and cached objects were not caching. I had a look at the config and realised their mistake in like 3 mins.
At my new internship I am have to work in Magento. I come to FUCKING hate it.
From the phtml files, the choice between caching or having to wait 20 fucking seconds for a page reload to the huge file structure and the "documentation".
The whole fucking thing is a mess with a shit load of bugs and confusing git tickets that never seem to be added as updates!!!
Fucking hate this shit
After spending lots of money on a Mac mini 2014, ngrox, and other dev tools to get jenkins working for android, iOS, and node.js, I'm giving up and switching to bitbucket pipelines. Recently, my web builds having been taking +25 mins just downloading node modules. Let's see how fast bitbucket pipelines is. Especially with caching node modules.1
Okay, if I understand correctly, if you want your website to be RGPD compliant, you must wait for user opt-in before storing anything to their device.
Maybe I'm asking myself too much questions but, how exactly does this work for a PWA ? Should you ask user for permission before starting a service worker and/or before caching any content ? If so, what if the user refuses the authorization ? The app is broken ? Or it just fallback to good old http browsing if it's server-rendered ?3
Recovering a Raid 1 EXT4 disk to a ZFS disk. Thank god for Adobe programs and their multi million file-based caching files.
Sat here for 30 minutes trying to find an error a client saw that I fixed earlier (mobile error). After going through everything I could think of, it turns out his browser was just caching... Fml2
myAddFunc(1,2); // output: 3
myAddFunc(4,5); // output: 3 + 9=12
True Fucking Story :(1
My manager wants me to add a caching layer on top of an API in 30 mins. Even to think about which part of the fucked-up system to cache will take more than 2 hours. How is anyone supposed to do it in 30 mins?
I just gave up after about 4 hours. Gonna sleep and start fresh tomorrow. Pretty sure, I'm not gonna finish it tomorrow either.4
All those mine WTF moments are somehow related with caching which i keep on forgetting... the most fresh one was last week, i had some GIGANTIC mySQL query, and for the sake of response time I immediately made a cache function that kept Redis cache for a day or so... so last week i had to change something (good ol' client and his visions for app). So there i was with the query that returned same god damned results every time, i copy the query in some mySQL manager and it goes fine, but in the app it doesn't... what the actual FUCK!!! i was questioning my career until i figured it out, i was planning to buy some sheeps and a fife and to hell with this, a loud facepalm was echoed through the office that day...3
Aaaaaaaarg GCC! Stop caching my failures. I do correct my code, stop pulling the old copy from /tmp you bloated piece of C3
Hey remote workers.
What would be your advice for someone with experience that's interested in exploring remote work.
I'd like to target this question to remote workers that live outside USA/EU/UK. Say South America, South Asia.
A little introduction.
I'm a full stack engineer, did one project in embedded systems with QT/C++/RPI can do backend in Python, Node, Java, C#. I have some experience with React Native (just 2 apps)
I currently I do full stack with Node, React, postgres and caching with couchdb.
I gather requirements, write the projects, proposals and then I do the implementation. (Really full stack, I kinda like it though, when I'm bored with code I pick up an issue and contact the client to socialize/get answers. I found out that nondevs like to feel they talk to a human not a robot)
I'm making about 600usd/month (dev in a poor country) working 30hrs /week. I'd like to ramp up my income, working remote part time to fill up about 50hr week.
What can I expect?
Where do I start?
Are there part time opportunities for working remote?
What kind of roles are in demand?9
*attempts to leverage browser caching to raise site speed
*Site speed suddenly jumps to 100
*Very happy for 2 seconds
*Realise it's because I've knocked the site over
Was having problems on a VPN where my URL was constantly redirecting to https, after https was disabled, spent ages reconfiguring nginx, removing and adding nginx again with no luck. Eventually said fuck it, backed up everything of importance, destroyed the droplet and spun up a new one. Installed nginx and redone the DNS for the domain only for the same thing to happen. It was at that moment I discovered it was chrome caching the HSTS domain. I now have a long night ahead of me configuring the new droplet and restoring the backup data.
Do anyone of you use a npm registry server like verdaccio for caching of packages from npmjs.org?
Today I tried verdaccio within a local docker container.
I successfully connected via npm --registry <registry-url> install
There where no errors, but verdaccio kept delivering packages with 200.
Shouldn't it be 304 since the packages already exist in the storage folder of verdacio?14
Is GraphQL worth it? It promisses to keep some load away from backend programmers but what about this scenario:
There's a list of items with scroll load/infinite scroll. There can be several filters as well as the Option to change the ordering of the list items.
With "traditional" REST, I'll hit the DB with one request, get the data in the right order to the backend, might itterate over it once to add additional information, cache the result in Redis/Memcache and send it off.
Using GraphQL, the frontend has to load all entries first, sort them in JS (which probably is slow on mobile devices), and then display it. No matter how "expensive" the query is, there's no caching.
Is that about it? Did I get something backwards?3
I've been developping some software so an entire debian OS gets bootstrapped and installed with all the desired software with the help of puppetlabs software...i need to prepare a server that can handle virtualization and be fast at it. So all the goodies a decent server needs, the apt, caching, networking, firewall, everything checks out... I want to test kvm virtualization... Doesn't work. Wtf? Spend a decent amount of time figuring out what the hell is wrong... I finally dzcide to think 'what if my buddy accidentally gave me a bad mobo'...
$ grep -e (vmx|ssm) /proc/cpuinfo
I feel so stupid to not check the mobo virtualization capabilities.3
To be a Java (or other business popular language) developer
* Java 6, 8 and features up to 14
* SQL + nosql
* Logging eg log4j2,
* Searching eg elastic stack
* Framework (at least 1, but hey, knowing 1 is lame..)
* Networking or at least base http knowledge
* Tomcat, jboss or other shit
* Aws, heroku, GCE or other SAAS/paas
* Rest, RPC, soap
* Business Hello World example
* Hexagonal Architecture
* 12 app factor
* Security, oauth2
* Eureka or consul as service Discovery
* Config server
* Hazel cast
* Endless story ...
Then we can start hello word app2
Western Digital takes forever to RMA their hard drive under warranty. Had to buy a spare just to get my NAS back up and running before I lose everything (I only have a 1 drive fault tolerance RAID array). Too bad I only have 5 drive slots. 4 4TB WD Reds and an SSD for caching = 10TB usable space in this configuration. Actually this kinda pisses me off....I pay for 4TB drives and I get 3.64TB. For the 4 drives, that's a loss of 1.44TB.2
Reading some documentation about how to use the caching method to store data with the framework, and it has an example
The best part of it is that it's doesn't store the data so I don't see the point of this1
TL;DR When talking about caching, is it even worth considering try and br as memory efficient as possible?
I recently chatted with a developer who wanted to improve a frameworks memory usage. It's a framework creating discord bots, providing hooks to events such as message creation. He compared it too 2 other frameworks, where is ranked last with 240mb memory usage for a bot with around 10.5k users iirc. The best framework memory wise used around 120mb, all running on the same amount of users.
So he set out to reduce the memory consumption of that framework. He alone reduced the memory usage by quite some bit. Then he wanted to try out ttl for the cache or rather cache with expirations times, adding no overhead, besides checking every interval of there are so few records that should be deleted. (Somebody in the chat called that sort of cache a meme. Would be happy , if you coukd also explain why that is so😅).
Afterwards the memory usage droped down to 100mb after a Around 3-5 minutes.
The maintainer of the package won't merge his changes, because sone of them really introduce some stuff that might be troublesome later on, such as modifying the default argument for processes, something along these lines. Haven't looked at these changes.
So I'm asking myself whether it's worth saving that much memory. Because at the end of the day, it's cache. Imo cache can be as big as it wants to be, but should stay within borders and of course return memory of needed. Otherwise there should be no problem.
But maybe I just need other people point of view to consider. The other devs reasoning was simple because "it shouldn't consume that much memory", which doesn't really help, so I'm seeking you guys out😁
It would be great if it had Angular, Node.js being used on EC2 instances with a ELB and a RDS (master and slave) instances, some caching etc.2
If you develop something with bulk-internet-traffic (caching, backup and so on) buffer that long that OpenVPN can connect without interruption of a download, stream or so...
!rant, so I'm trying to decide on what caching system I should use. It's for a PHP app, using Symfony as framework, tlgether with Doctrine for DB. The caches in Question are memcached, APCu or redis.
The goal: speed shit up.
The app currently uses Symfony 2.8 and is hosted on a single server (so no distributed system is needed). I'd currently opt for APCu, but more since it's not distributed, there won't be an overhead from that. A nice thing about memcached would be the abillity to store user seeions, even if we would decide to have multiple servers in the future.
What would you reccomend and why?3
Not using front-end caching and etags because holy ncs database calculates global etags. A senior engineer fixed it but management told him not to spend time on such a problem.
So I did this little experiment PWA caching and service worker which caches entire website (js/css/html) [It's a small website]. Now I do not understand the concept of PWA caching entirely because when I 'Add it to Home screen' it becomes an app as expected but when I turn off the internet and then open the app it requires the internet.
What in fucking ass, why won't you just render those html pages which are cached ?16
I need help speeding up my website. It's hosted on GoDaddy shared and uses Managed WordPress. I've run tests on Pingdom, PageInsights, GTMetrix and load times vary from 1.3s to 5s depending on the server location. Apparently Managed WP GoDaddy uses a CDN and their own caching plugin (Varnish). I don't know if that's good or bad.
PageInsights says I need to optimize images. I'm using WP Smush but clearly it isn't helping too much. I always manually resize all my images to <=100kb.
Also, the age old leverage browser caching problem... How do I fix that?
Thanks a lot for taking the time to go through this :)4
Well, a combination of DXVK, wine and dxvk-cache-pool was used to try and play Path of Exile. The problem seems to be that I can't have any pre-built caches due to them not existing. Seems like a GTX 660 isn't really used anymore and if I want to play a game I will have to have DXVK build its own cache.
Until then, I'm stuck with a stuttery mess of a game due to Path of Exile having a rather many levels. A full playthrough will be necessary until it starts working smoothly.7
So I just tried to restart my dns caching service while conversing with someone. I typed 'sudo shutdown -r now'. 🙊
A question to all the guys and girls that launched a startup: How powerful was your infrastructure at the beginning? How many requests per seconds did you encounter after the first few weeks after the launch? Did you distribute the workload to different systems in the first place or was that something that was done later?
I am currently working hard in my freetime to get my first project done. As it's still a side project, that I am working on in my freetime, I want to make the launch as smooth as possible. I imagine that it's really hard to make serious changes to the whole design, just because the initial approach doesn't scale well enough. So I am currently in the process of stresstesting the whole infrastructure. But during the stresstest I realized that I don't really know what I should aim for.
What I also want to avoid is, that I am wasting my time on creating a large infrastructure of database servers, caching instances and load balancers that isn't really necessary for the initial launch.
Would really love to hear your experiences on that.3
Why does a site have two caching plugins? Whoever made this site (that I'm working on improving) needs to get new and updated training.
And why is this site using godaddy shared hosting? I couldn't even update more than two plugins at a time.2
Fuck Ubuntu and it's caching. What's that? Oh you have 180 Gigs of free space? Let me take care of that for you in a few minutes. Say goodbye to your next boot and good luck figuring that shit out.5
Still on this : https://devrant.com/rants/1430952/...
So I understand that on the framework of the company, to store data in your cache, you use a method, called
So yeah, that seems kinda okay somehow. But this method is called by another one called GetOrLoad, that will get or load the data.
Is my english bad or is it really ambiguous given that context ?2
Best eays to improve database performance?
I have a huge database table with a lot of data, 10+columns and around 5-6million rows and growing. We are thinking about optimisation methods, the data will be frequently accessed, but not in the same ways. even though caching is helping, some initial requests take 6-20secs. What are good ways to improve a relational db performance? The data is date and item based, every item can have some data back from 2016 or older and up to 1000 related rows in each day. Every day new data would be added. Right now working with php.9
I’m trying to add caching functionality in my scalable spark cluster on Docker. I am able to add redis to my Docker container, but that doesn’t add redis-CLI or any other useful tools that I need. And there are some redis-cache projects on PyPi, but they are very old and not compatible with python3.5
Setting Cache-Control headers that are aware of future changes on a given page is freakishly complex. Too bad the Expires header doesn't overrule Cache-Control.1
Trying to setup drupal from scratch on main windows desktop, 2 days in fiddling with phpa and apache server to get clean URLs and OPCODE caching working. To finally finish the installation.
Python working non-deterministic on VSCode? That just happened ... somehow, wtf??
Getting a list from a method and comparing its last element with an int value. Always worked before like a charm, didn't change a thing. All of a sudden TypeError, cannot run anymore? Restart VSCode, run again, still not running ... ?? Retry and print the element, in case I've surprisingly actually been an idiot all along ... nope, value looks in print as expected. Continue execution, suddenly condition works again. WTF just happened??? Caching, python extension bug, anything like that to blame?1
Most satisfying was reducing the time my ci/cd did to build,test,verify complance and deploy of virtually anything i want in lrss then 10 minutes. From code to running appliance fully configured and being absolute certain it will work without any other modificatio . it used to be an hour.
Achieved this to do lots of caching and parallell test runs.
The downside is that my development server is feeling like a unvoluntary black person from ghana moving to the newfound united states 400 years ago...
Here I am sitting again to explain to some nice people that it is not my service causing the havoc but the infrastructure.
Always getting cached content when using the public route (private is fine). When I firstly hit something it should cache then I get that content for every url I hit on that service (e.g. Getting the favicon when fetching the html)
Even when I stop the service the public route does still return content. Let's see if they accept that there may be a caching issue 😥😥😥 ah and the service is running in 2 other environments - must be an application problem
Idea - possibly a bad one - visual studio extension that hits a database to display possible values to supply as a method argument. With caching of course..
I'm thinking along the lines of permissions in a database that at some point have to be hard coded against code to enforce them. Stuff like that.
Possible or beyond stupid?7
Caching is cool. But damn it! When it comes to testing it's just a hell of a pain to search and disable every caching found to ensure the result is repeatable.
npm cache, /tmp, ~/.cache, and more, and more!