Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Get a devDuck
Rubber duck debugging has never been so cute! Get your favorite coding language devDuckBuy Now
Search - "memory limit"
My team handles infrastructure deployment and automation in the cloud for our company, so we don't exactly develop applications ourselves, but we're responsible for building deployment pipelines, provisioning cloud resources, automating their deployments, etc.
I've ranted about this before, but it fits the weekly rant so I'll do it again.
Someone deployed an autoscaling application into our production AWS account, but they set the maximum instance count to 300. The account limit was less than that. So, of course, their application gets stuck and starts scaling out infinitely. Two hundred new servers spun up in an hour before hitting the limit and then throwing errors all over the place. They send me a ticket and I login to AWS to investigate. Not only have they broken their own application, but they've also made it impossible to deploy anything else into prod. Every other autoscaling group is now unable to scale out at all. We had to submit an emergency limit increase request to AWS, spent thousands of dollars on those stupidly-large instances, and yelled at the dev team responsible. Two weeks later, THEY INCREASED THE MAX COUNT TO 500 AND IT HAPPENED AGAIN!
And the whole thing happened because a database filled up the hard drive, so it would spin up a new server, whose hard drive would be full already and thus spin up a new server, and so on into infinity.
Thats probably the only WTF moment that resulted in me actually saying "WTF?!" out loud to the person responsible, but I've had others. One dev team had their code logging to a location they couldn't access, so we got daily requests for two weeks to download and email log files to them. Another dev team refused to believe their server was crashing due to their bad code even after we showed them the logs that demonstrated their application had a massive memory leak. Another team arbitrarily decided that they were going to deploy their code at 4 AM on a Saturday and they wanted a member of my team to be available in case something went wrong. We aren't 24/7 support. We aren't even weekend support. Or any support, technically. Another team told us we had one day to do three weeks' worth of work to deploy their application because they had set a hard deadline and then didn't tell us about it until the day before. We gave them a flat "No" for that request.
I could probably keep going, but you get the gist of it.7
Woohoo! 32k achieved!!! Finally I can post some new rant without risking some sudden overshoot 😁
So putting celebrations aside for a minute, a while ago I've noticed a tingle when I stroke my finger across metal areas of my tablet, or the sides of my phone (which probably has metal near it too) while it's charging. And it's been bugging me ever since.
Now, some things to note are that it only happens when my feet are touching the ground though slippers, and that the frequency is so low that I can actually feel the tingle when I slide my finger across the material. This to me at least seems like electricity flows through me into ground, and touching the ground directly provides a path so easy for the electrons to run away that I don't feel it at all. But if I lift my feet off the ground entirely, I just get charged up and after that, nothing else happens.
So those are my ideas. The answers on the subject on the other hand.. absolute cancer. Unsurprisingly, most of them came from Apple users. Here's some of them.
- I've not noticed it, but if you're concerned bring the phone to Apple for evaluation.
- Me too facing same problem.. did u visit apple care?
And one good answer at least...
- google emf sensitivity, its real. You are right, there is a small current flowing through your body, try to limit your usage. The problem with this issue is those who aren't affected (lucky ones for now) will tell you these products are 100% safe. To a degree they are, i used my ipod touch for about 2 years straight vwith virtually no symptoms. then the tingling started and it gets worse.You will get more sensitive to progressively less powerful things. I dont want to scare you but just limit your usage like i didnt do 🙂
Overall that discussion was pretty good actually, aside from "bring it to the Genius Bar, they'll know for sure and not just sell you another unit". But then there's Reddit.
- Ok, real reason is probably that the extension cord and/or outlet is probably not grounded correctly. Either that or you are using a cheap knockoff charger.
Either use a surge protector and/or use the authentic Apple Charger.
- It's not the volts that hurt you, it's the amps
- I think you are in deep love with your phone. That tingling sensation is usually referred to as "love" in human language.
- Do less acid, I would advise.
Okay, so that's the real cancer. Grounding issue sounds reasonable despite it being wrong. Grounding is actually not needed when your charging appliance doesn't have any exposed metal parts. And isolation from high voltage to low voltage side actually happens through things like routering holes into the PCB, creating spark gaps, and using galvanic isolation through things like optocouplers. As for a surge protector? I'm using them to protect my PC and my servers, but the only purpose they serve is to protect from.. you guessed it.. voltage surges, like lightning bolts hitting the grid. They don't do shit for grounding or reducing this tingle! What a fucking tool.
It's not the volts that kill, it's the amps.. yeah I'm sure that the debunking of that is easy to find. Not gonna explain that here. And the rest of it.. yeah it's just fucking cancer.
Now what's the real issue with this tingle? It's actually a Class-Y rated (i.e. kV rated) capacitor that's on the transformer of any switch-mode power supply, including phone chargers. If memory serves me right, it helps with decoupling the switching noise and so on. But as it's connected to the primary side of the transformer, if the cap is sufficiently large and you are sufficiently sensitive, it can actually cause that tingle by passing a fraction of the mains electricity into your body. It's totally safe though, as the power that these caps pass is very small. But to some, it's noticeable.
Hope you found this interesting! And thanks a lot for bringing me to 2^15. I really appreciate it ♥️24
Here are the reasons why I don't like IPv6.
Now I'll be honest, I hate IPv6 with all my heart. So I'm not supporting it until inevitably it becomes the de facto standard of the internet. In home networks on the other hand.. huehue...
The main reason why I hate it is because it looks in every way overengineered. Or rather, poorly engineered. IPv4 has 32 bits worth, which translates to about 4 billion addresses. IPv6 on the other hand has 128 bits worth of addresses.. which translates to.. some obscenely huge number that I don't even want to start translating.
That's the problem. It's too big. Anyone who's worked on the internet for any amount of time knows that the internet on this planet will likely not exceed an amount of machines equal to about 1 or 2 extra bits (8.5B and 17.1B respectively). Now of course 33 or 34 bits in total is unwieldy, it doesn't go well with electronics. From 32 you essentially have to go up to 64 straight away. That's why 64-bit processors are.. well, 64 bits. The memory grew larger than the 4GB that a 32-bit processor could support, so that's what happened.
The internet could've grown that way too. Heck it probably could've become 64 bits in total of which 34 are assigned to the internet and the remaining bits are for whatever purposes large IP consumers would like to use the remainder for.
Whoever designed IPv6 however.. nope! Let's give everyone a /64 range, and give them quite literally an IP pool far, FAR larger than the entire current internet. What's the fucking point!?
The IPv6 standard is far larger than it should've been. It should've been 64 bits instead of 128, and it should've been separated differently. What were they thinking? A bazillion colonized planets' internetworks that would join the main internet as well? Yeah that's clearly something that the internet will develop into. The internet which is effectively just a big network that everyone leases and controls a little bit of. Just like a home network but scaled up. Imagine or even just look at the engineering challenges that interplanetary communications present. That is not going to be feasible for connecting multiple planets' internets. You can engineer however you want but you can't engineer around the hard limit of light speed. Besides, are our satellites internet-connected? Well yes but try using one. And those whizz only a couple of km above sea level. The latency involved makes it barely usable. Imagine communicating to the ISS, the moon or Mars. That is not going to happen at an internet scale. Not even close. And those are only the closest celestial objects out there.
So why was IPv6 engineered with hundreds of years of development and likely at least a stage 4 civilization in mind? No idea. Future-proofing or poor engineering? I honestly don't know. But as a stage 0 or maybe stage 1 person, I don't think that I or civilization for that matter is ready for a 128-bit internet. And we aren't even close to needing so many bits.
Going back to 64-bit processors and memory. We've passed 32 bit address width about a decade ago. But even now, we're only at about twice that size on average. We're not even close to saturating 64-bit address width, and that will likely take at least a few hundred years as well. I'd say that's more than sufficient. The internet should've really become a 64-bit internet too.36
I've optimised so many things in my time I can't remember most of them.
Most recently, something had to be the equivalent off `"literal" LIKE column` with a million rows to compare. It would take around a second average each literal to lookup for a service that needs to be high load and low latency. This isn't an easy case to optimise, many people would consider it impossible.
It took my a couple of hours to reverse engineer the data and implement a few hundred line implementation that would look it up in 1ms average with the worst possible case being very rare and not too distant from this.
In another case there was a lookup of arbitrary time spans that most people would not bother to cache because the input parameters are too short lived and variable to make a difference. I replaced the 50000+ line application acting as a middle man between the application and database with 500 lines of code that did the look up faster and was able to implement a reasonable caching strategy. This dropped resource consumption by a minimum of factor of ten at least. Misses were cheaper and it was able to cache most cases. It also involved modifying the client library in C to stop it unnecessarily wrapping primitives in objects to the high level language which was causing it to consume excessive amounts of memory when processing huge data streams.
Another system would download a huge data set for every point of sale constantly, then parse and apply it. It had to reflect changes quickly but would download the whole dataset each time containing hundreds of thousands of rows. I whipped up a system so that a single server (barring redundancy) would download it in a loop, parse it using C which was much faster than the traditional interpreted language, then use a custom data differential format, TCP data streaming protocol, binary serialisation and LZMA compression to pipe it down to points of sale. This protocol also used versioning for catchup and differential combination for additional reduction in size. It went from being 30 seconds to a few minutes behind to using able to keep up to with in a second of changes. It was also using so much bandwidth that it would reach the limit on ADSL connections then get throttled. I looked at the traffic stats after and it dropped from dozens of terabytes a month to around a gigabyte or so a month for several hundred machines. The drop in the graphs you'd think all the machines had been turned off as that's what it looked like. It could now happily run over GPRS or 56K.
I was working on a project with a lot of data and noticed these huge tables and horrible queries. The tables were all the results of queries. Someone wrote terrible SQL then to optimise it ran it in the background with all possible variable values then store the results of joins and aggregates into new tables. On top of those tables they wrote more SQL. I wrote some new queries and query generation that wiped out thousands of lines of code immediately and operated on the original tables taking things down from 30GB and rapidly climbing to a couple GB.
Another time a piece of mathematics had to generate all possible permutations and the existing solution was factorial. I worked out how to optimise it to run n*n which believe it or not made the world of difference. Went from hardly handling anything to handling anything thrown at it. It was nice trying to get people to "freeze the system now".
I build my own frontend systems (admittedly rushed) that do what angular/react/vue aim for but with higher (maximum) performance including an in memory data base to back the UI that had layered event driven indexes and could handle referential integrity (overlay on the database only revealing items with valid integrity) or reordering and reposition events very rapidly using a custom AVL tree. You could layer indexes over it (data inheritance) that could be partial and dynamic.
So many times have I optimised things on automatic just cleaning up code normally. Hundreds, thousands of optimisations. It's what makes my clock tick.4
//Proper Haxk20 tech rant
So uhmmm this is interesting.
Apparently Acer decided to not make proper vBIOS for Nitro 5 RX560X and they left some stuff well just empty or to stock.
For example they set the maximum OC clocks to stock so even if you want to overclock you cant exactly on Linux.
OK but why can we OC on windows ?
Well to no ones suprise windows stuff doesnt exactly follow specs published and well they just ignore some stuff so if you want you can OC on windows even past the max OC clocks because oh safety on windows isnt a fucking thing.
OK so how do we fix it on Linux ?
Well we have 2 ways.
1. Is to dump vBIOS. Edit it to contain the proper max limit and load it on startup because flashing it isnt great to do on laptop.
This will get you OC support but takes ages because again this isnt normal GPU. Its desktop GPU in laptop and Acer had to do something to it because oh well. So you will get lot of shit in the way.
Second way you can OC is to lets just say disable the security feature on Linux.
Hmmmm but AMDGPU driver doesnt exactly support that.
What do we do ?
Well if you look at AMDGPU code there is code for overdrive which is basically driver for overclocking support. When you look at the code for it you will be blown away first at what you are looking at but then Oh boom. variables for max clock on memory and core. Bingo.
OK set it to the values and boot the patched kernel. Hmmmmmm why is it so slow ?
Why is it running at minimum clocks ?
Look at the code again and you forgot that drivers use Hz instead of MHz. Yeahhhhhh. OK multiply the value to match Hz and its at stock. But no worries. Now when i tell the GPU to run at clocks past its limit it wont yell at me that i cant do that.
OK now comes a little weird part. I didnt find docs about boot parameters in linux and i didnt find a way to implement them. So if anybody has a way of implementing them please comment here a link to it. I will possibly even make this a patch and send it to kernel since well manual patching is shit.2
Researchers were able to store 3 bits of data per cell in phase-change memory, beating the previous limit of 1 bit per cell, IBM said. PCM competes with dynamic random-access memory.
My boss just now: "In a 32 bit machine the memory limit is 1.6 GB. After that programs routinely crash."
What really happens IMO? He writes programs that crash when they reach 1.6 GB allocated and the architecture is 32 bit. But it's a limit of his software, not one of the OS.8
HOW COCKSUCKING DIFFICULT CAN IT BE TO CHANGE THE FUCKING MEMORY LIMIT FOR NPM PACKAGES?!
HOLY MOTHERFUCKING SHIT.6
** this means words are muted **
I send a mail the client a Google doc with elaborate details about evaluation of an Android tablet from a Chinese manufacturer.
The client is upset, he says "You say there is no GPS chip on the tablet while the manufacturer says otherwise"
Me- "I have clearly mentioned that it has a GPS chip"
Client- Opens the Google doc, points to a sentence. Looks at me like I did something horrible.
Me - **This guys is either word blind or something else is wrong with him, the line reads 'GPS chip available'**
Me- "Look, it says 'GPS chip available'.
Client- **Blinks n blinks again** "Alright, but why did you share a Google document, why not PDF, docx"
Me-**Politely** "You can download the document in any format, look I will show you..."
Client- "It should have been in the mail itself ideally"
Me- **WTH** "We normally maintain a document for such things to keep everything organised, but if you want I will put everything in mail itself"
Client- "Hmm.. do both from next time"
Me- "Alright" **BS**
Client- "Why is the new feature taking so much time"
Me- "As planned earlier, we going to deliver it tomorrow"
Client- "Why not today??" **Gives a strange look.**
Me thinking - **Enough**
Me- "See, I am trying to integrate a smarten with a socket connection, reading it's data via exposed APIs that are hardly documented, we need faster performance so I need to implement caching, multi threading, offline handling, multiple processes to avoid memory fluctuations, sync adapter to sync data...."
Client- "Ok ok ok, it's fine if you give working build tomorrow"
Me- "Ok, fine"
Okey, so the recruiters are getting smarter, I just clicked how well do you know WordPress quiz (I know it's from a recruiter, already entered a php quiz An might win a drone)
So the question is how to solve this issue:
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2348617 bytes) in /home4/xxx/public_html/wp-includes/plugin.php on line xxx
A set memory limit to 256
B set memory limit to Max
C set memory limit to 256 in htaccess
D restart server
These all seem like bad answers to me.
I vote E don't use the plug-in, or the answer that trumps the rest, F don't use WordPress4
Well as it seems Arduino isnt the best option to send large data over BT because it writes it into memory and when you exceed that shit then random characters start falling onto your terminal.
Also Uno isnt the best option too since it has only 1 serial port and that is used with PC too so uploading is impossible if the device also receives from those pins.
I might switch to MEGA but well that wont arrive in time and then so this is kind of fucked up as i pretty much have to limit the characters too much now but i can do that it will be just impractical.
And i was wondering why the fuck i get random characters if i write too much from MEGA to Uno. Its fucking memory overflow.13
Before anyone jumps into comments. Yes Haxk20 overclocking the shit out of GPU again.
So the new Acer nitro 5 AN515-43 has the same GPU as me. At start i was wondering how are they getting so much better performance then me.
Then i looked at the clocks. HOLY SHIT mine is 1450 on memory and that model is running 1750.
Thats 300Mhz boost on there.
OK so after fixing OverDrive on Linux. Better say hardcoding the limit because VBIOS doesnt have it.
I got near the same performace as he did on my old model. Ouuuu yeah thats what im talking about.
From 30FPS to near 42. Thats HUGE.
And thats only on 1700Mhz and im experiencing no issues at all.
So IDK but it seems that the GPU in those newer models is overclocked out of the box to i suppose make it better because its not the best GPU in the world. And that performance boost is indeed NICE.4
So, vs2015 is crashing when the process gets to 2gb .... 32bit .net process memory limit strikes again!!! 80 projects in the solution & what looks a run away extension is taking the memory !!!
Come on M$ it should be 64bit on a 64bit system!!
Now the hunt for the extension that's causing it!!
"Our supplier asks that you double the number of php child processes for this fpm pool"
"Are you aware, that that would lead to about 100% of memory overcommit, taken the current limit of 128MB/child, and that if a lot of them started at once, the system would probably go for OOM-Kill, which would most probably kill your database, that still runs on 100% MyISAM tables that do not support transactions, and you'd have to kiss your data integrity goodbye, right?"
"Uh... Nevermind then"
I get that some people are not IT-versed, but really... Hire someone who knows what they are doing and doesn't live 20 years in the past, god damn it!
The feeling when you set a kernel parameter as a test but later yo forget to remove it and it causes your GPU to not enter higher memory clock speeds. Yeah been stuck at 625mhz with poor FPS when compared to 1450mhz.
Waiting for kernel 5.0 or 4.21 whatever it will be called (FFS call it 5.0 finally) that will add the ability to allow my GPU to use more power if i set the limit higher. Then lets fucking overclock it to hell.
And lets mod skyrim to hell right now. ENB and all that cool shit i coudlnt do with my old hardware.
I can already hear the fans going strong. Hmmmmm
Maybe i should apply some better thermal paste because it seems that this one is kind of bad. And oh yeah 5.0 adds ability to control the fans properly. So i can even "Overclock them even tho i wont because they were designed to run like this"
Some acer shitty tech that just applies more voltage to fans then normally. Im not able to do that on linux yet but this is going to change on 5.0
I cant wait.Tried the next branch and damn it feels nice. Fans going strong in games and i can unlimit the GPU power limit finally from 60W. But well its unstable as fuck because its next. So lets just wait for RC.
Running NPM install on an average size project on a free tier Cloud9 instance.
Didn't go too well kept hitting memory limit which killed the NPM process.
Upgrade it is..
Customer: your app is not returning all the objects in my bucket
Support: check console log 500 server error, ssh into box check logs exhausted memory limit.
Sudo vim /etc/php.ini search memory limit
Update to a high number restart Apache sit back and think fuck did I set it to high will it blow up my server.
Only time will tell!!! Sorted out the issue until the next user with millions of objects in their buckets
Visual Studio Code and xdebug: my saviors to finally not have to put up with NuSphere PhpEd constantly shitting itself over memory limit issues in my 3,704 file PHP project.
Pff today I start 400 threads in python initialized in a for loop, one thread per record, each record initialize a heavy process, Memory Limit Exceeded, I'm frustrated with this shit!
How to run 400 records doing a heavy task preventing the CPU to close the task executed by timeout limit exceeded, or how to run multiple threads to end before reach timeout limit each task without exceeding the memory limit.
[Prestashop question / rant]
Yes, it's not StackOverflow here, neither is it prestashop support forum - but I trust u people most :)
Proper solution for working with big(?) import of products from XML (2,5MB, ~8600 items) to MySQL(InnoDB) within prestashop backoffice module (OR standalone cronjob)
"solutions" I read about so far:
- Up php's max execution time/max memory limit to infinity and hope it's enough
- Run import as a cronjob
- Use MySQL XML parsing procedure and just supply raw xml file to it
- Convert to CSV and use prestashop import functionality (most unreliable so far)
- Instead of using ObjectModel, assemble raw sql queries for chunks of items
- Buy a pre-made module to just handle import (meh)
Maybe an expert on the topic could recommend something?3