Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
Long rant ahead, but it's worth it.
I used to work with a professor (let's call him Dr. X) and developed a backend + acted as sysadmin for our team's research project. Two semesters ago, they wanted to revamp the front end + do some data visualization, so a girl (let's call her W) joined the team and did all that. We wanted to merge the two sites and host on azure, but due to issues and impeding conferences that require our data to be online, we kept postponing. I graduate this semester and haven't worked with the team for a while, so they have a new guy in charge of the azure server (let's call him H), and yesterday my professor sends me (let's call me M), H and W an email telling us to coordinate to have the merge up on azure in 2-3 days, max. The following convo was what I had with H:
M: Hi, if you just give me access to azure I'll be able to set everything up myself, also I'll need a db set up, and just send me the connection string.
H: Hi, we won't have dbs because that is extra costs involved since we don't have dynamic content. Also I can't give you access, instead push everything on git and set up the site on a test azure server and I will take it from there.
M: There is proprietary data on the site...
H: Oh really? I don't know what's on it.
<and yet he knows we have no dynamic data>
M: Fine, I'll load the data some other way, but I have access to all the data anyway, just talk to Dr. X and you'll see you can give me access. Delete my access after if you want.
H: No, just do what I said: git then upload to test azure account.
Fine, he's a complete tool, but I like Dr. X, so I message W and tell her we have to merge, she tells me that it's not that easy to set it up on github as she's using wordpress. She sends me instructions on what to do, and, lo and behold, there's a db in her solution. Ok, I go back to talking to H:
M: W is using a db. Talk to her so we can figure out whether we need a database or not.
H: We can't use a database because we want to decrease costs.
M: Yes I know that, so talk to her because that probably means she has to re-do some stuff, which might take some time. Also there might be dynamic content in what she's doing.
H: This is your project, you talk to her.
<I'm starting to get mad right now>
M: I don't know what they had her do apart from how it interfaces with what I've done.
H: We still can't have databases.
M: Listen, I don't do wordpress, and I'm not gonna mess with it, you talk to her
H: I won't do any development
<So you won't do any dev, but you won't give me access to do it either?>
M: Man, the bottleneck isn't the merging right now, it's the fact that W needs a db
H: I know, so talk to her
M: THE RESTRICTION TO NOT HAVE DATABASES IS NOT MINE, IT'S YOURS, YOU TALK TO HER. I can't evaluate whether it's a reasonable enough reason or not since I don't know the requirements or what they're willing to spend.
H: It's your project.
M: Then give me fucking access to azure and I'll handle it, you know you'll have to set up wordpress again regardless whether we set it up the first time.
H: Man just do your job.
At this point I lost it. WHAT A FUCKING TOOL. He doesn't wanna do dev work, wants me to go through the trouble of setting up on a test subscription first, and doesn't want to give me access to azure. What's more, he did shit all and doesn't want to anything else. Well fuck you. I googled him, to see if he's anyone important, if he's done anything notable which is why he's being so God damn condescending. MY INTERNSHIP ALONE ECLIPSES HIS ENTIRE CV. Then what the fuck?
There's also this that happened sometime during our talk:
M: You'll have to take to Dr. Y so he'll change the DNS to point to the azure subscription instead of my server.
H: Yea don't worry, too early for that.
M: DNS propagation takes 24 hours...
H: Yea don't worry.
DNS propagation allows the entire web to know that your website is hosted on a different server so it can change where it's pointing to. We have to do this in 2-3 days. Why do work in parallel? Nah let's wait.
I went over his head and talked to the professor directly, and despite wanting to tell him that he was both drunk and high the day he hired that guy, I kept it professional. He hasn't replied yet, but this fucker's pompous attitude is just too much for me alone, so I had to share.
PS: I named his contact as Annoying Prick 4 minutes into our chat. Gonna rename him cz that seems tooooooo soft a name right now.46 -
Each month my department compiles a 4M row 150 column data table for compliance with a federal agency. Before submitting, we check it against about 400 rules.
The existing system was simply 400 queries that ran in sequence, table-scanning 4M rows each time, taking upwards of 6 hours, which is a huge bottleneck, especially if you have to make changes and rerun. Plus the output was rather one-dimensional.
I built a proper normalized database and created a sort of rules engine, running all 400 rules in one table scan. Not only does it complete in 30 minutes, but the reports generate automatically, and the results can be filtered on several dimensions to aid with root-cause analysis.
Management was pleased.4 -
The awkward moment when your ethernnet cable is the bottleneck in your Internet connection.
Never thought I'd see such speeds in India. Wow moment!
Side note: I was subscribed to a 50mbps plan. Last night my subscriber upgraded their network and just gave me a upgrade FREE of cost.
23 -
Our web department was deploying a fairly large sales campaign (equivalent to a ‘Black Friday’ for us), and the day before, at 4:00PM, one of the devs emails us and asks “Hey, just a heads up, the main sales page takes almost 30 seconds to load. Any chance you could find out why? Thanks!”
We click the URL they sent, and sure enough, 30 seconds on the dot.
Our department manager almost fell out of his chair (a few ‘F’ bombs were thrown).
DBAs sit next door, so he shouts…
Mgr: ”Hey, did you know the new sales page is taking 30 seconds to open!?”
DBA: “Yea, but it’s not the database. Are you just now hearing about this? They have had performance problems for over week now. Our traces show it’s something on their end.”
Mgr: “-bleep- no!”
Mgr tries to get a hold of anyone …no one is answering the phone..so he leaves to find someone…anyone with authority.
4:15 he comes back..
Mgr: “-beep- All the web managers were in a meeting. I had to interrupt and ask if they knew about the performance problem.”
Me: “Oh crap. I assume they didn’t know or they wouldn’t be in a meeting.”
Mgr: “-bleep- no! No one knew. Apparently the only ones who knew were the 3 developers and the DBA!”
Me: “Uh…what exactly do they want us to do?”
Mgr: “The –bleep- if I know!”
Me: “Are there any load tests we could use for the staging servers? Maybe it’s only the developer servers.”
DBA: “No, just those 3 developers testing. They could reproduce the slowness on staging, so no need for the load tests.”
Mgr: “Oh my –bleep-ing God!”
4:30 ..one of the vice presidents comes into our area…
VP: “So, do we know what the problem is? John tells me you guys are fixing the problem.”
Mgr: “No, we just heard about the problem half hour ago. DBAs said the database side is fine and the traces look like the bottleneck is on web side of things.”
VP: “Hmm, no, John said the problem is the caching. Aren’t you responsible for that?”
Mgr: “Uh…um…yea, but I don’t think anyone knows what the problem is yet.”
VP: “Well, get the caching problem fixed as soon as possible. Our sales numbers this year hinge on the deployment tomorrow.”
- VP leaves -
Me: “I looked at the cache, it’s fine. Their traffic is barely a blip. How much do you want to bet they have a bug or a mistyped url in their javascript? A consistent 30 second load time is suspiciously indicative of a timeout somewhere.”
Mgr: “I was thinking the same thing. I’ll have networking run a trace.”
4:45 Networking run their trace, and sure enough, there was some relative path of ‘something’ pointing to a local resource not on development, it was waiting/timing out after 30 seconds. Fixed the path and page loaded instantaneously. Network admin walks over..
NetworkAdmin: “We had no idea they were having problems. If they told us last week, we could have identified the issue. Did anyone else think 30 second load time was a bit suspicious?”
4:50 VP walks in (“John” is the web team manager)..
VP: “John said the caching issue is fixed. Great job everyone.”
Mgr: “It wasn’t the caching, it was a mistyped resource or something in a javascript file.”
VP: “But the caching is fixed? Right? John said it was caching. Anyway, great job everyone. We’re going to have a great day tomorrow!”
VP leaves
NetworkAdmin: “Ouch…you feel that?”
Me: “Feel what?”
NetworkAdmin: “That bus John just threw us under.”
Mgr: “Yea, but I think John just saved 3 jobs. Remember that.”4 -
Okay, story time.
Back during 2016, I decided to do a little experiment to test the viability of multithreading in a JavaScript server stack, and I'm not talking about the Node.js way of queuing I/O on background threads, or about WebWorkers that box and convert your arguments to JSON and back during a simple call across two JS contexts.
I'm talking about JavaScript code running concurrently on all cores. I'm talking about replacing the god-awful single-threaded event loop of ECMAScript – the biggest bottleneck in software history – with an honest-to-god, lock-free thread-pool scheduler that executes JS code in parallel, on all cores.
I'm talking about concurrent access to shared mutable state – a big, rightfully-hated mess when done badly – in JavaScript.
This rant is about the many mistakes I made at the time, specifically the biggest – but not the first – of which: publishing some preliminary results very early on.
Every time I showed my work to a JavaScript developer, I'd get negative feedback. Like, unjustified hatred and immediate denial, or outright rejection of the entire concept. Some were even adamantly trying to discourage me from this project.
So I posted a sarcastic question to the Software Engineering Stack Exchange, which was originally worded differently to reflect my frustration, but was later edited by mods to be more serious.
You can see the responses for yourself here: https://goo.gl/poHKpK
Most of the serious answers were along the lines of "multithreading is hard". The top voted response started with this statement: "1) Multithreading is extremely hard, and unfortunately the way you've presented this idea so far implies you're severely underestimating how hard it is."
While I'll admit that my presentation was initially lacking, I later made an entire page to explain the synchronisation mechanism in place, and you can read more about it here, if you're interested:
http://nexusjs.com/architecture/
But what really shocked me was that I had never understood the mindset that all the naysayers adopted until I read that response.
Because the bottom-line of that entire response is an argument: an argument against change.
The average JavaScript developer doesn't want a multithreaded server platform for JavaScript because it means a change of the status quo.
And this is exactly why I started this project. I wanted a highly performant JavaScript platform for servers that's more suitable for real-time applications like transcoding, video streaming, and machine learning.
Nexus does not and will not hold your hand. It will not repeat Node's mistakes and give you nice ways to shoot yourself in the foot later, like `process.on('uncaughtException', ...)` for a catch-all global error handling solution.
No, an uncaught exception will be dealt with like any other self-respecting language: by not ignoring the problem and pretending it doesn't exist. If you write bad code, your program will crash, and you can't rectify a bug in your code by ignoring its presence entirely and using duct tape to scrape something together.
Back on the topic of multithreading, though. Multithreading is known to be hard, that's true. But how do you deal with a difficult solution? You simplify it and break it down, not just disregard it completely; because multithreading has its great advantages, too.
Like, how about we talk performance?
How about distributed algorithms that don't waste 40% of their computing power on agent communication and pointless overhead (like the serialisation/deserialisation of messages across the execution boundary for every single call)?
How about vertical scaling without forking the entire address space (and thus multiplying your application's memory consumption by the number of cores you wish to use)?
How about utilising logical CPUs to the fullest extent, and allowing them to execute JavaScript? Something that isn't even possible with the current model implemented by Node?
Some will say that the performance gains aren't worth the risk. That the possibility of race conditions and deadlocks aren't worth it.
That's the point of cooperative multithreading. It is a way to smartly work around these issues.
If you use promises, they will execute in parallel, to the best of the scheduler's abilities, and if you chain them then they will run consecutively as planned according to their dependency graph.
If your code doesn't access global variables or shared closure variables, or your promises only deal with their provided inputs without side-effects, then no contention will *ever* occur.
If you only read and never modify globals, no contention will ever occur.
Are you seeing the same trend I'm seeing?
Good JavaScript programming practices miraculously coincide with the best practices of thread-safety.
When someone says we shouldn't use multithreading because it's hard, do you know what I like to say to that?
"To multithread, you need a pair."18 -
Just optimized a bottleneck function in a program down from 160ms to 3ms. First time I've ever used a profiler, and it feels awesome.1
-
- be the most productive member in the team and everyone dumps their tasks on you.
- be the laziest person in the team and everyone sees you as a bottleneck.
- be the senior + the laziest person in the team and you can complain about others being a bottleneck and dump your tasks on the most productive member.
fml ( ︶︿︶)_╭∩╮3 -
Customer: We don't know what we want, but we want it yesterday.
Sales: No problem.
Me: WTF?
I'm not even mad or stressed anymore. You didn't ask me before, so why should I bend and break, so that you don't have to deal with staving the customer. Most of the time, he changes his requirements or is the bottleneck himself, anyway.2 -
Biggest challenge I overcame as dev? One of many.
Avoiding a life sentence when the 'powers that be' targeted one of my libraries for the root cause of system performance issues and I didn't correct that accusation with a flame thrower.
What the accusation? What I named the library. Yep. The *name* was causing every single problem in the system.
Panorama (very, very expensive APM system at the time) identified my library in it's analysis, the calls to/from SQLServer was the bottleneck
We had one of Panorama's engineers on-site and he asked what (not the actual name) MyLibrary was and (I'll preface I did not know or involved in any of the so-called 'research') a crack team of developers+managers researched the system thoroughly and found MyLibrary was used in just about every project. I wrote the .Net 1.1 MyLibrary as a mini-ORM to simplify the execution of database code (stored procs, etc) and gracefully handle+log database exceptions (auto-logged details such as the target db, stored procedure name, parameter values, etc, everything you'd need to troubleshoot database errors). This was before Dapper and the other fancy tools used by kids these days.
By the time the news got to me, there was a team cobbled together who's only focus was to remove any/every trace of MyLibrary from the code base. Using Waterfall, they calculated it would take at least a year to remove+replace MyLibrary with the equivalent ADO.Net plumbing.
In a department wide meeting:
DeptMgr: "This day forward, no one is to use MyLibrary to access the database! It's slow, unprofessionally named, and the root cause of all the database issues."
Me: "What about MyLibrary is slow? It's excecuting standard the ADO.Net code. Only extra bit of code is the exception handling to capture the details when the exception is logged."
DeptMgr: "We've spent the last 6 weeks with the Panorama engineer and he's identified MyLibrary as the cause. Company has spent over $100,000 on this software and we have to make fact based decisions. Look at this slide ... "
<DeptMgr shows a histogram of the stacktrace, showing MyLibrary as the slowest>
Me: "You do realize that the execution time is the database call itself, not the code. In that example, the invoice call, it's the stored procedure that taking 5 seconds, not MyLibrary."
<at this point, DeptMgr is getting red-face mad>
AreaMgr: "Yes...yes...but if we stopped using MyLibrary, removing the unnecessary layers, will make the code run faster."
<typical headknodd-ers knod their heads in agreement>
Dev01: "The loading of MyLibrary takes CPU cycles away from code that supports our customers. Every CPU cycle counts."
<headknod-ding continues>
Me: "I'm really confused. Maybe I'm looking at the data wrong. On the slide where you highlighted all the bottlenecks, the histogram shows the latency is the database, I mean...it's right there, in red. Am I looking at it wrong?"
<this was meeting with 20+ other devs, mgrs, a VP, the Panorama engineer>
DeptMgr: "Yes you are! I know MyLibrary is your baby. You need to check your ego at the door and face the facts. Your MyLibrary is a failed experiment and needs to be exterminated from this system!"
Fast forward 9 months, maybe 50% of the projects updated, come across the documentation left from the Panorama. Even after the removal of MyLibrary, there was zero increases in performance. The engineer recommended DBAs start optimizing their indexes and other N+1 problems discovered. I decide to ask the developer who lead the re-write.
Me: "I see that removing MyLibrary did nothing to improve performance."
Dev: "Yes, DeptMgr was pissed. He was ready to throw the Panorama engineer out a window when he said the problems were in the database all along. Didn't you say that?"
Me: "Um, so is this re-write project dead?"
Dev: "No. Removing MyLibrary introduced all kinds of bugs. All the boilerplate ADO.Net code caused a lot of unhandled exceptions, then we had to go back and write exception handling code."
Me: "What a failure. What dipshit would think writing more code leads to less bugs?"
Dev: "I know, I know. We're so far behind schedule. We had to come up with something. I ended up writing a library to make replacing MyLibrary easier. I called it KnightRider. Like the TV show. Everyone is excited to speed up their code with KnightRider. Same method names, same exception handling. All we have to do is replace MyLibrary with KnightRider and we're done."
Me: "Won't the bottlenecks then point to KnightRider?"
Dev: "Meh, not my problem. Panorama meets primarily with the DBAs and the networking team now. I doubt we ever use Panorama to look at our C# code."
Needless to say, I was (still) pissed that they had used MyLibrary as dirty word and a scapegoat for months when they *knew* where the problems were. Pissed enough for a flamethrower? Maybe.6 -
Oh dear, a scaling problem I solved was replacing some Regex matching with simpler string functions. While I'm a huge fan of Regex, it's unreal how much performance they can suck out of some high-n loops...
I got about 120x out of some critical code thus making a CPU upgrade unnecessary.8 -
[long]
When searching for internship via school I found this small startup with this cute project of building a teaching tool for programming. There were back then 2 programmers: the founder and the co-founder.
Then like 1 week before the internship started, the co-founder had a burnout and had to get off the project, while the company was so low on budget the founder, aka my new b0ss, had to work separate jobs to keep the company alive. (quite metal tbh)
It's funny because I'm a junior developer, 100%. I've been coding as a hobby for around 8 years now but I've never worked in a big company before. (No exception to this workplace either)
First project I get: rewrite the compiler. The Python compiler.
"But wait, why not just embed a real compiler from the first case?"
-nanananana it's never simple, as you probably know from your own projects.
The new compiler, as compared to existing embedded compiler solutions out there, needed these prime features:
- Walk through the code (debugger style), but programmatically.
- Show custom exceptions (ex: "A colon is needed at the end of an if-statement" instead of "Syntax error line 3")
- Have a "Did-you-mean this variable?" error for usage of unassigned variables.
- Be able to be embedded in Unity's WebGL build target
All for the use case of being a friendly compiler.
The last dash in the list is actually the biggest bottleneck which excluded all existing open-source projects (i could find). Compliant with WebAssembly I can't use threads among other things, IL2CPP has lots of restrictions, Unity has some as well...
Oh and it should of course be built using test-driven development.
"Good luck!" - said the founder, first day of work as she then traveled to USA for **3 weeks**, leaving me solo with the to-be-made codebase and humongous list of requirements.
---
I just finished the 6th week of internship, boss has been at "HQ" for 3 weeks now, and I just hit the biggest milestone yet for this project.
Yes I've been succeeding! This project has gone so well, and I'm surprising myself how much code I've been pumping out during these weeks.
I'm up now at almost 40'000 lines of source and 30'000 lines of code. ‼
( Biggest project I've ever worked on previously was at 8'000 lines of code )
The milestone (that I finished today) was for loops! As been trying to showcase in the GIF.
---
It's such a giant project and I can honestly say I've done some good work here. Self-five. Over-performing is a thing.
The things that makes me shiver though is that most that use this application will never know the intricates of it's insides, and the brain work put into it.
The project is probably over-engineered. A lot. Having a home-made compiler gives us a lot of flexibility for our product as we're trying to make more of a "pedagogic IDE". But no matter that I reinvented the wheel for the 105Gth time, it's still the most fun I've had with a project to date.
---
Also btw if anyone wants to see source code, please give me good reasons as I'm actively trying to convince my boss to make the compiler open-source.
Cheers!
4 -
<just got out of this meeting>
Mgr: “Can we log the messages coming from the services?”
Me: “Absolutely, but it could be a lot of network traffic and create a lot of noise. I’m not sure if our current logging infrastructure is the right fit for this.”
Senior Dev: “We could use Log4Net. That will take care of the logging.”
Mgr: “Log4Net?…Yea…I’ve heard of it…Great, make it happen.”
Me: “Um…Log4Net is just the client library, I’m talking about the back-end, where the data is logged. For this issue, we want to make sure the data we’re logging is as concise as possible. We don’t want to cause a bottleneck inside the service logging informational messages.”
Mgr: “Oh, no, absolutely not, but I don’t know the right answer, which is why I’ll let you two figure it out.”
Senior Dev: “Log4Net will take care of any threading issues we have with logging. It’ll work.”
Me: “Um..I’m sure…but we need to figure out what we need to log before we decide how we’re logging it.”
Senior Dev: “Yea, but if we log to SQL database, it will scale just fine.”
Mgr: “A SQL database? For logging? That seems excessive.”
Senior Dev: “No, not really. Log4Net takes care of all the details.”
Me: “That’s not going to happen. We’re not going to set up an entire sql database infrastructure to log data.”
Senior Dev: “Yea…probably right. We could use ElasticSearch or even Redis. Those are lightweight.”
Mgr: “Oh..yea…I’ve heard good things about Redis.”
Senior Dev: “Yea, and it runs on Linux and Linux is free.”
Mgr: “I like free, but I’m late for another meeting…you guys figure it out and let me know.”
<mgr leaves>
Me: “So..Linux…um…know anything about administrating Redis on Linux?”
Senior Dev: ”Oh no…not a clue.”
It was all I could do from doing physical harm to another human being.
I really hate people playing buzzword bingo with projects I’m responsible for.
Only good piece is he’s not changing any of the code.3 -
The worst boss and human being so far, still wondering how he keeps the company afloat. This was my first longterm developer job almost a decade ago and I was a student at that time. The application was an outlook plug in for a document management system.
Scene 1:
Boss: The processing is too slow. Make it faster.
Me: After analysis and profiling I can prove that the core (developed in VB6 by a physicist and autoconverted to VB.NET) is the bottleneck.
Boss: I don't care. Make it faster and don't touch the core.
Scene 2:
Boss: I want the app to behave in that way.
Me: This is not what we specified previously. Look here. Nonetheless, I would have to rewrite half of the plugin. Mind that it is an outlook plug in and we are restricted by outlook. If you want that, it would take XX days and we do not have enough time until release.
Boss: I don't care. Do it. And the deadline stays as it is.
Boss 2 weeks later: I don't like it.
Scene 3:
Me: To release in time I need more resources. I need at least one tester and another developer would be a huge plus. Also, I need a second PC for testing.
Boss: No.
2 weeks later:
Boss: why does it not work properly in outlook 2010? Didn't you test it?
Me: I could not. I have only outlook 2007. I asked for more resources and did not get them.
Boss: it's your fault. Bad work.
Scene 4:
*Me having failed multiple exams, stress at work, started to drink*
Boss: Don't you like working here?
Me: ...
Finale:
*Me getting written sick with severe depression*
Boss: fires me.
Me: Loses flat. Quits uni. Unemployed for 6 Months, one rejection after another (boss was phoned, that's sure). Moving back to parents. Sues boss. Gets money.
I still hate him and wish him the most painful experiences in life. Such people belong behind bars. But the justice isn't always served. One has to move forward and improve himself.3 -
Best code performance incr. I made?
Many, many years ago our scaling strategy was to throw hardware at performance problems. Hardware consisted of dedicated web server and backing SQL server box, so each site instance had two servers (and data replication processes in place)
Two servers turned into 4, 4 to 8, 8 to around 16 (don't remember exactly what we ended up with). With Window's server and SQL Server licenses getting into the hundreds of thousands of dollars, the 'powers-that-be' were becoming very concerned with our IT budget. With our IT-VP and other web mgrs being hardware-centric, they simply shrugged and told the company that's just the way it is.
Taking it upon myself, started looking into utilizing web services, caching data (Microsoft's Velocity at the time), and a service that returned product data, the bottleneck for most of the performance issues. Description, price, simple stuff. Testing the scaling with our dev environment, single web server and single backing sql server, the service was able to handle 10x the traffic with much better performance.
Since the majority of the IT mgmt were hardware centric, they blew off the results saying my tests were contrived and my solution wouldn't work in 'the real world'. Not 100% wrong, I had no idea what would happen when real traffic would hit the site.
With our other hardware guys concerned the web hardware budget was tearing into everything else, they helped convince the 'powers-that-be' to give my idea a shot.
Fast forward a couple of months (lots of web code changes), early one morning we started slowly turning on the new framework (3 load balanced web service servers, 3 web servers, one sql server). 5 minutes...no issues, 10 minutes...no issues,an hour...everything is looking great. Then (A is a network admin)...
A: "Umm...guys...hardly any of the other web servers are being hit. The new servers are handling almost 100% of the traffic."
VP: "That can't be right. Something must be wrong with the load balancers. Rollback!"
A:"No, everything is fine. Load balancer is working and the performance spikes are coming from the old servers, not the new ones. Wow!, this is awesome!"
<Web manager 'Stacey'>
Stacey: "We probably still need to rollback. We'll need to do a full analysis to why the performance improved and apply it the current hardware setup."
A: "Page load times are now under 100 milliseconds from almost 3 seconds. Lets not rollback and see what happens."
Stacey:"I don't know, customers aren't used to such fast load times. They'll think something is wrong and go to a competitor. Rollback."
VP: "Agreed. We don't why this so fast. We'll need to replicate what is going on to the current architecture. Good try guys."
<later that day>
VP: "We've received hundreds of emails complementing us on the web site performance this morning and upset that the site suddenly slowed down again. CEO got wind of these emails and instructed us to move forward with the new framework."
After full implementation, we were able to scale back to only a few web servers and a single sql server, saving an initial $300,000 and a potential future savings of over $500,000. Budget analysis considering other factors, over the next 7 years, this would save the company over a million dollars.
At the semi-annual company wide meeting, our VP made a speech.
VP: "I'd like to thank everyone for this hard fought journey to get our web site up to industry standards for the benefit of our customers and stakeholders. Most of all, I'd like to thank Stacey for all her effort in designing and implementation of the scaling solution. Great job Stacy!"
<hands her a blank white envelope, hmmm...wonder what was in it?>
A few devs who sat in front of me turn around, network guys to the right, all look at me with puzzled looks with one mouth-ing "WTF?"9 -
Once upon a time, one or two jobs ago, a really awesome engineer specced out a distributed search application in response to a business need. This company was managed pretty oldschool and required a ton of paperwork and approvals.
The engineer spent many weeks running tests and optimizing the hell out of this app cluster. It flew, and he had the data to prove it could handle production workloads (think hundreds of terabytes of data being processed every single day)
Part of the way he achieved this was having RAID0 on all of the servers to maximize I/O throughput. He didn't care much about data loss, since the application itself was fault tolerant on a much more granular level.
Management, hearing about this, absolutely flipped their shit and demanded RAID6 instead. This despite the conclusive data that the engineer had that proved RAID6 couldn't keep up.
He more or less got told to STFU.
Even this despite the fact that a RAID restripe would actually take many times longer than rebuilding the failed node from scratch (a process that took about 30 minutes by hand, and could probably be automated to be done in less than five), causing a longer exposure to actual data loss throughout the length of the days-long array rebuild time.
The ill-thought-out requirement added about 50% to the cost of the project (*many* more hard drives now required), beyond the original budget, and the subsequent bureaucratic wrangling resulted in a late product launch.
6 months or so later, after real customers were using this product, the app was buckling under around half of its expected workload. A friend of the engineer suggested to management to try RAID0. Sure enough, that resolved the I/O bottleneck.
This rage-inducing story has a happy ending, though! Said engineer left the company not long after this incident, citing it as a reason for his departure. He was immediately hired by another company, making integer multiples of his prior salary.
The product the company botched the launch of by ignoring his spec? It died a few months later. Maybe the poor customer experience was to blame? Maybe the late launch? Maybe it was another reason entirely.
Either way, millions of dollars of hardware now sat fallow. This was a black eye on the company all the way up to the C-level.
tl;dr: Listen to your engineers. You hired them for their expertise.5 -
Suddenly there's this tight deadline, everyone's pumping hours in, I am the one that has to discuss with everyone and integrate their work into mine. So I schedule an early morning meeting with a colleague, whose work is crucial in order to continue integrating the others' modules.
30mins into the meeting, he's not there yet. I reach him
"Oh, sorry, I forgot to mention, I'm actually not available today, and until 3 days before the deadline"
Well isn't this great.5 -
If your startups CEO is the major stakeholder, that’s great.
If your CEO is the major bottleneck, that’s a problem.
Don’t handcuff your employees. Instead enable them to work efficiently and effectively. -
A gigantic codebase of several tens of millions of loc [prolly hundreds of mill's as we don't see all of it] in java EE.
Very complex business logic where bells have their own whistles with their own bells with .... You get it.
Very fine-tuned performance to make app so performant that the only bottleneck becomes the db. The beefiest rds instance becomes too laggy [orm, so sqls are immutable]
Client moves to rewrite the whole thing in PHP. Motivation: lower TTM
:facepalm:11 -
Sorry to keep whining about my stupid fucking job, but y'all, I think I'm nearing my limit.
There's some good...I am pretty much free to resolve issues any way I want to, as the only other person in the company who "codes" only knows one old ass language that doesn't apply to 90% of the rest of the tech stack at all, and some SQL - all of that to say, we may disagree, but ultimately, these matters are always deferred to me at the end of the day, insofar as the actual implementation goes (which is to say I am not micromanaged). At least as far as non-visuals are concerned, because those of course, are the most important things. Button colors and shit, woo hoo**. That's what we should focus on as we're bringing in potentially millions of dollars per month - the god damn button color and collapsible accordions based on data type over the shit ass DB performance bottleneck, the lack of redundancy or backups (aside from the one I made soon after I started -- literally saved everyone today because of that. My thanks? None, and more bullshit tasks) or the 300GB+ spaghetti code nightmare that is the literal circulatory system of the FUCKING COMPANY. Hundreds of people depend on it for their livelihoods, and those of their families, but fuck me in the face, right? I'm just a god damn nerd who has worked for the federal government, a handful of fortune 500's, a couple of fortune 100's, some startups, etc. But the fuck do I know about the lifecycle of companies?
I could continue ranting, but what's the point? I've got a nice little adage that I've started to live by, and y'all might appreciate it: "If everything is a priority/is important, nothing is". These folks just don't fucking get it. I'm torn because, on the one hand, they waste my time and kinda underpay me, in addition to forcing me to be onsite for 50 hours a week. They don't listen to me, couldn't give a flying shit about my experientially based opinions. I'm just a fucking chimp with a typewriter, there to take commands like a fucking waiter. But there's a lot of job security, assuming I don't fucking snap one day, and the job market for devs (I'm sure I don't need to tell you) is hostile atm. I'm also drinking far more than usual, and I really need to do something about that. It's only wednesday - I think...not 100% on that truth be told, and I logged my fourth trip to the liquor store this week already.
**Dear backenders - don't ever learn front end, or if you do, just lie about it to avoid being designated full stack. It's not worth it.5 -
Solved a major scalability issue today.
I'm starting to think I might actually be as good as what I told the recruiters I was.
This hadoop-ecosystem job used to take about 2h40min and cost about USD 1.00/GB.
Now it takes 36 mins. At about 0.85/GB.
Fixed some over shuffling, restructured some bottleneck serial stages, used lots of weird words.
Folks in this company I just entered were struggling with this formerly unwieldy process for a year.
Now it's nimble enough to run every hour.
Maybe that whole "experience" thing people were always yammering about wasn't completely bullshit.4 -
PM: To achieve this, do A
Me: That's unnecessarily complex, can't we do B
PM: Ooh, so it's too complicated for you?
Me: No, it's complex, it will bottleneck the system
PM: We've done A in 5 different websites, so you should do the same
Me: ......5 -
I'm so fucking sick of pouring hours of work into providing application code for someone who could give two shits about what I've done -- instead he completely fixates on what's missing or broken -- nevermind that I completely eliminated a bad UI thread bottleneck.
Sometimes I swear that coding is a thankless job and people just expect miracles.1 -
Interviewing candidates for a middle/senior dev position:
Me: Imagine you have this button, but whatever it's doing when you click it, it's taking too long to load. How would you improve the speed performance?
Candidate: Redis!
Me: Okay... but how would you find where the bottleneck is?
C: Redis!
Me: How abo-
C: REDIS!3 -
can we all take a moment to appreciate the developers of flutter. they're smart, and they took the time to make flutter the *right* way.
they used an easy to learn language that's ideal for mobile development, which means hot reload/restart is possible (because dart supports aot and jit compilation)
the way it's designed is beautiful. everything is a widget, and it's easy to customize them via named parameters.
the community is great. it's not large, but it's supportive, with two active subreddits. yesterday i asked a question on r/flutterdev, and a member of the flutter team at google answered the question with a comprehensive answer.
flutter is very consistent across platforms. if it works on android or ios, you can bet it'll work on the other just as well, with the exception of platform-specific code.
it is VERY performant. unless you write a major bottleneck, 60fps is easy to achieve.
animations are EASY. define a tween and animation controller and then write a callback function. not to mention it's straightforward, and complex/combined animations are easy, too.
you can get almost direct access to the canvas, should you need it, with custompainter.
oh my god, this is revolutionary in the programming world. development is quicker than it is with native android alone, and for people who have no access to a mac, like me, i can develop for ios and compile via code magic. if you haven't checked it out and you develop for mobile, check it out.
oh yeah, did i mention it's not just mobile. hummingbird - flutter compiled to web - is already in experimental public betas, and will likely be released by the end of the year. there's also experimental desktop support, which is amazing, and much better than electron. not to mention flutter is the future, as it will be the primary way to make apps on fuchsia os.13 -
Freelance Headhunters...
No idea where they got my name and the number of my work phone...
- they do not give a damn and call me 1045 am to discuss "EXCITING BUSINESS OPPORTUNITIES™"
- they are sure I am having a case of stockholm syndrome, because I could NEVER be satisfied in my current employment
- they call every other day with a new number and start the same process again
How fucking dense can people get?11 -
In 2015 I sent an email to Google labs describing how pareidolia could be implemented algorithmically.
The basis is that a noise function put through a discriminator, could be used to train a generative function.
And now we have transformers.
I also told them if they looked back at the research they would very likely discover that dendrites were analog hubs, not just individual switches. Thats turned out to be true to.
I wrote to them in an email as far back as 2009 that attention was an under-researched topic. In 2017 someone finally got around to writing "attention is all you need."
I wrote that there were very likely basic correlates in the human brain for things like numbers, and simple concepts like color, shape, and basic relationships, that the brain used to bootstrap learning. We found out years later based on research, that this is the case.
I wrote almost a decade ago that personality systems were a means that genes could use to value-seek for efficient behaviors in unknowable environments, a form of adaption. We later found out that is probably true as well.
I came up with the "winning lottery ticket" hypothesis back in 2011, for why certain subgraphs of networks seemed to naturally learn faster than others. I didn't call it that though, it was just a question that arose because of all the "architecture thrashing" I saw in the research, why there were apparent large or marginal gains in slightly different architectures, when we had an explosion of different approaches. It seemed to me the most important difference between countless architectures, was initialization.
This thinking flowed naturally from some ideas about network sparsity (namely that it made no sense that networks should be fully connected, and we could probably train networks by intentionally dropping connections).
All the way back in 2007 I thought this was comparable to masking inputs in training, or a bottleneck architecture, though I didn't think to put an encoder and decoder back to back.
Nevertheless it goes to show, if you follow research real closely, how much low hanging fruit is actually out there to be discovered and worked on.
And to this day, google never fucking once got back to me.
I wonder if anyone ever actually read those emails...
Wait till they figure out "attention is all you need" isn't actually all you need.
p.s. something I read recently got me thinking. Decoders can also be viewed as resolving a manifold closer to an ideal form for some joint distribution. Think of it like your data as points on a balloon (the output of the bottleneck), and decoding as the process of expanding the balloon. In absolute terms, as the balloon expands, your points grow apart, but as long as the datapoints are not uniformly distributed, then *some* points will grow closer together *relatively* even as the surface expands and pushes points apart in the absolute.
In other words, for some symmetry, the encoder and bottleneck introduces an isotropy, and this step also happens to tease out anisotropy, information that was missed or produced by the encoder, which is distortions introduced by the architecture/approach, features of the data that got passed on through the bottleneck, or essentially hidden features.4 -
For two projects, I have been in a solo work pattern, been a time bottleneck, and been irreplaceable on the projects. Four months ago I told management, "If anything happens to me these projects will be in trouble. I want to train a backup. I can't sustain this momentum. It isn't good for me, or for the success of these projects."
Four months later I still have no backup. They decided to diversity hire some new developers in the wrong area and now there is no money for a backup for me. I can't do all the work on both projects as a solo developer. I could have if I wasn't pushed into doing trial and error development on a poorly defined MS Dynamics API. Since the projects were behind schedule the customers lost confidence in the company to deliver. So the executives railroaded both project managers to save face.
Instead of addressing the development issues they did a bunch of other silly things. I got a job offer lined up and issued my resignation. That news absolutely exploded. After resigning my executive decided to say how awful I am in front of the customer in an attempt to save face for the company. The customer contacted the recently railroaded project manager and asks why. Former project manager tells customer, "You noticed how much faster the development of that part of the application went when he joined. You noticed how much better the quality of the project was. What do you think is happening? Do you think that a very good developer and an experienced project manager are to blame for the failures here?" So the executive is 13/10 pissed off because I may have accidentally struck a death blow for millions of dollars of business. I committed to taking care of the handover to the customer, and the company can't afford to get rid of me without completely losing confidence of the customer. The developers that I work with don't blame me at all and they are disgruntled that executive tried to character assassinate me and realize that it could have been them. I sense that I also may have initiated a developer mass-exodus. So the last few days have been the most stressful of my career but none of it is sticking to me because I followed all of the correct process.
You play stupid games you win stupid prizes.4 -
!dev
ISP: "We are going to give you this piece of garbage Puma6 modem that will cause your internet to randomly die because we want to save a few bucks instead of actually having it function properly"
Me: "Can I replace it with my own?"
ISP: "No?"
I love not having fibre.
So either I shoot myself in the foot by going to another ISP (meaning I'll get *at best* 1/5th of the speed, while my current speed already is a severe bottleneck) or I have to keep content with this fucking piece of garbage internet that decides to commit sudoku over nothing...7 -
Been bughunting the last week or so on this import job that is suddenly running so slowly that it takes more than 24h and is restarting on top of itself.
It used to run in anywhere from 4 to 8 hours, which was bad enough. You see, it ran on timer scheduled by our main site. So our deployment window was determined by when this job finished, and if it didn't finish during work hours, then no deployment that day.
So we got the idea to move it to a separate service to eliminate that deployment window bottleneck. And now, seemingly unrelated, it is just running slow as shit.
There is a lot of bad design in the code, and we know we want to build a completely new solution. But we also absolutely need this import to run every day until a better solution can take over.
We've taken care of some of the most obvious problems that could cause the poor performance, but it's unclear whether it's going to be enough. And with a runtime of about a day and wild variations of the most atomic partial imports, it's extremely tedious to test as well...3 -
TL;DR: A new "process" for collaboration between teams was created in order to stonewall requests from my team.
A couple months ago, we created a new Dev team that specializes in writing internal tools. This team was staffed with internal developers, and got a separate manager. The whole point of this team was to collaborate with my dev team so we can both help each other develop tools that the company needs.
One of the developers that was on my team went over to this team while he and I were still working on a big application. For a few weeks, he still worked on this application as he normally would, and we'd sit with each other and work through features together whenever we needed a fresh set of eyes.
Well, eventually his new team got protective of him and created a new "process" for our teams to request assistance from one another. So now instead of just popping over to someone's desk to ask a quick question, you have to send an email to the team and request that you can borrow that particular developer for a question, and then the entire team sits down and discusses whether or not they're going to allow that person to answer your question. Then after a week of discussion, if they decide to allow it, they schedule a meeting for a week later, in which you will get the question answered.
So instead of just spending 2 minutes to ask and answer the question, you have to spend weeks in order to request assistance, and then schedule a meeting.
It's ridiculous, and it's all because his team got protective that he was working with another Dev team. Dev teams collaborate all the time, and work together. My team is constantly helping other teams, and we don't have this ridiculous process. We get asked a question, and we answer it. Simple as that.
Last week, I sent an email for assistance in completing a feature, and didn't hear back. I talked to the Product Owner for the team, and he said "Just send an email," to which I responded that I did and hadn't got a response. He said "Oh....." I then told my boss that this is an enormous bottleneck, and he seemed surprised hearing that this is a bottleneck.
A week passed and today I still hadn't got a response, so my boss reached out to the Product Owner to push him. Finally, I got a response and they scheduled a meeting to answer my question 3 days down the road. So it's going on 2 weeks to get this simple question answered.
Normally I'd just have the other developer come over and help, but apparently they yelled at him the last time he did that.
The issue is that the process was created with the assistance of our "senior" developers, who never work with this other team in this capacity, so they just nodded and smiled and let them put this ridiculous process in place.
Like, get off your high horses. You don't "own" him, he's allowed to collaborate with other teams. This question would've taken literally 10 minutes, but because of your new "process" you've turned it into a 2 week debacle and you've effectively delayed the app launch with your pettiness.
They say that this process isn't intended to prevent us from getting assistance, and that might not have been the original intention of the Product Owner/manager, but it's very clear that the developers on the other team are taking advantage of it and using it as a big stonewall so they can beat around the bush and avoid providing assistance when it's needed.
If this becomes a trend, I'm going to schedule a meeting (which apparently they love to do,) and we're going re-work this entire process, because it's extremely counterproductive and seems to only exist in order to create red tape.3 -
An application requiring lots of servers to handle the workload is not the problem.
The problem is an application that cannot handle workload no matter how many servers you throw at it.2 -
my hand is becoming a bottleneck.
it used to be slow device. upgraded.
it used to be mouse & IDE. upgraded to nvim.
now. my wrist gets sore.
waiting for Elons brain computer interface.9 -
I'm moving to PHP.
No, seriously. PHP devs were treated like “you're the tech guy, I don't care, make it work” for so long that PHP deps library has everything. If you need to do an unusual task like slowing download speed to 64 kbps, there is a lib for that. Caching is one lib away. Yes, libs themselves are subpar, but they do the job.
Performance? I never had any perf issues in my apps. DB is always the bottleneck, and I know databases.
Frameworks? I don't care about them.
Also, I'll always find PHP devs on the market.
Shut the fuck up with your elitist rust crap. PHP is a nuclear-resistant cockroach that will outlive you, your stupid language and everything you wrote in it. My PHP code will be running fine after every line of code you ever wrote in rust/python/java/scala/whatever fancy language you like is no longer in use.
Yes, I talked shit about PHP in the past. I was neither pragmatic nor mature. Many things changed since. For starters, I'm a CTO now. Hating PHP was easy and socially acceptable. Talk shit about PHP, get internet points — that's how it always worked.
No more. PHP is the king.9 -
Under pressure for a big feature that had to be merged into develop like one month ago. But I couldn't because of issues I discover every single fucking day.
Today's issue is that a Cucumber test fails. I try reproducing it on my machine, it fails with a different error. Apparently I need to download some 10GB database file from some company server.
Alright, let's download it. But it's damn too slow. Well, let's have lunch in the meantime.
I come back, the download timed out at basically the same point I left it at.
I don't wanna try again. Not without trying to improve things. Download speed is ridiculous. Switching from Wi-Fi to Ethernet definitely helps, I thought.
The cable doesn't work. The port LEDs are both off. Is that cable even connected to something? So I follow that damn cable throughout my colleagues' desks. I'm now doing things without even remembering why.
I finally find the other end. It is plugged to the wall. I try another plug, but that fucking LED is still off. A colleague tells me: not all the sockets are actually connected to the switch, you have to call IT to have yours patched. Stay calm, stay caaaaalm...
A small lamp turns on in my head. Maybe something in my laptop is broken. So I try with a colleague's ethernet. That fucking LED is still off. A-ha.
Turns out, the shitty macbook adapter has this Ethernet port that DOESN'T work out of the box. It needs a driver to even realize there's a port. I look for it, I find it. I finally have wired connection. It's like having drinking water again.
I turn off WiFi, I re-try downloading that fucking database.
Nope, it's still stupidly slow. The bottleneck was in the dumbfuck internal server.
FUCK.
At least I have Ethernet now.1 -
Today spent 20min in a senior android dev interview debating an ex backender CTO about the importance of final classes where he tried to pull out some sort of perfect answer from me about it. Ironically this is the same CTO who failed managing a previous android contractor who was supposed to rewrite old app and ended up with an even shittier new app in 6 months of time. Now they are insecure and are looking for a new contractor who will be micromanaged this time.
But hey I guess he knows the importance of final classes. Some CTO's need a reality check and at least some business training, because your perfectly written app is useless if it doesnt fulfill business needs.
Their app is based on heresdk and built around navigation. The biggest bottleneck is that it works shitty on low end devices so their competition solved this problem by using a whitelabel rooted tables with a custom ROM wher u have full control over hardware, permissions and battery management. However this startup thinks they can build a perfect navigation app which will work perfectly on all devices while at the same time while also relying on a poorly optimized navigation sdk. Poor initial strategy I'd say and they didnt learn from previous 2 failures, now they are searching for the next savior android contractor who will have to solely implement evrything. -
To be honest with you, I’ve never had a bad experience with PHP.
Yes, it’s “dirty” compared to something like Haskell, but it’s not a bad thing. Dirty things usually bring simplicity and allow implementing the intended case super quickly, at the cost of breaking apart at scale. There are no bad tools, there are wrong tools for the job.
Premature optimization is the root of all evil. The more I launch new projects for me/other companies, the more I come to the realization that the vast majority of the projects out there will never see scale. They will be proven non-viable/impractical and deemed obsolete way before they outgrow the $20 VPS they were hosted on.
Sometimes (all the time, really) launching quickly like there is no tomorrow is the most viable business strategy. If (yes, “if”, not “when”) your project outgrows PHP and gets to the point when PHPs abstraction model is the bottleneck, you’ll have the money to rewrite the project in any language out there, trust me.
As someone said on biking subreddit to a person that asked how to buy the newest super-aero helmet, “if the aerodynamics of the old helmet is what holds you back, someone will be sending you the new one for free”.6 -
Load tests:
I'm used to do load tests in Visual Studio where it gives which line is exactly your bottleneck. But now I'm using VS Code (visual studio requires enterprise license for load tests :\ no longer have one)
Anyways long story short, what are the best practices for load tests? For me what I'm testing is how much can a given hardware specs handle and when test fails I go back and check if code can be optimized, is this the correct way to do this?7 -
!rant
27 days ago I asked here for advice on how to mentor software engineer student that was terrible at coding.
So, we are in the middle of the mentoring, my approach is for her to get used to normal engineering tools, in this occasion she is learning Git and "kanban" (basically we are using Clubhouse for this one) and Github PR submission and approval (I'm the one who approves them, naturally) by doing.
With git, things are hard because we cannot share a terminal session (via upterm) due to her using Windows on her laptop (WSL is an option for using upterm but her internet is so damn slow doing the configuration takes way too long), otherwise teaching her use git would be smoother than it is currently, with the other tools she is gaining a good grasp of them, it pleases me that the bottleneck is with Git itself.
She is working on a hangman game with Python, nothing fancy just the terminal. I made the stories with the requirements in Clubhouse for her to work on each as a unit removing some "thought process" of reading requirements and implementing solutions (at Uni it seems the professor writes a document of several pages detailing the background of the project and the requirements, I can see how it can become confusing for some students like her).
She will start Uni again this August 10th, there is a chance that our first "session" at this will end by then, my fear is that she forgets how to use the tools she learned, so I need to find a way to encourage her to keep using them somehow.3 -
The one in which I am rn is the reason why so many people dislike php, jquery and Java on the server.
Then previous to this one, classic ASP for the web interface and our desktop components were delphi (OLD ass delphi)
Mind you, these are all tech stacks that I do like (php, java and O Pascal in particular) but really dislike in:
php: we have just your standard procedural spaghetti php on some old ass shit.
Classic ASP: Same as with php, no proper structure, made more apparent by the intense limitations of VBScript, I did enjoy the language tho, had it evolved better It would have been more tolerable, but the hoops i had to take to build a propee API in it....boooooy that shit was an eye opener.
Delphi: Not bad in itself, but the original dev had a shit notion about how architecture should work.....or what architecture is for that matter.
The Java one: this shit was coded when Spring was already an alternative to just fucking around with JSP, or any other framework for that fucking matter. Dude tried....TRIED to implement design patterns in it and it failed on every single fucking component. Worst of all, it was coded in such a shit way that during certain...err...conditions, the bottleneck proved too massive of an ubdertaking and the app chokes and needs to be restarted ... constantly
their use cases for jquery are not bad, but loading all of jquery for the shit they mostly do could have been easily done with just standard vanilla JS.
I got more, but thede are just from the top of my head
I love php, mind you, but shit like this makes me see why some people GREATLY dislikes it.
I alsp have some old web forms in c# and vb net that I loathe, funny enough the code for thise in vb.net is more elegant, almost as if it were from a different developer.3 -
Leetcode.
It doesn't matter if you've done multiple projects with different tools, languages, team sizes and requirements for ANY company / org etc.
You will feel fucking stupid while taking too long on some of these questions.
I know interview questions are mostly to test your critical thinking skills but fuck I feel so bad after 2 evenings of doing this shit.
It is addictive though...2 -
Microservices in real life:
You have 4 different project opened in IDE simultaneously editing all of them to make something to work.
All projects share same db.
Basically this is monolith but splitter to different git repos.
Communicating one another via http calls or directly altering shared db tables.
Everyone is happy until solution goes to production then :
Solution doesn't scale because db is a bottleneck.
Services can be unavailable for 1-2s so some processes are left in unfinished state forever, because everyone assume 99,9% SLA means it always available and http calls will never fail.
Http calls take time.2 -
Hey guys,
I'm wondering if anyone here has experience with Outlook clients with close to 30 exchange mailboxes.
The software is incredibly unstable and keeps freezing.
I've tried setting the cache period to 1 week to no avail and it's starting to piss off my coworkers...
If anyone has any tips on improving the performance please share them.
Currently running SSD's (850 Evo)
I5-6600 and 8GB of ram.
The bottleneck seems to be the CPU spiking to 80%...
Any help is greatly appreciated.1 -
1) Had to fix severe bugs in a dynamic UI (configuration-driven forms) component.
Recognized undocumented Copy/Paste/Modify/FuckUp driven variations of the same component all over the project. Unsurprisingly, the implementations covered 99% of the antipattern catalog on wiki.c2.com and could compete with brainfuck in regard to human-readable code.
Escalated the issue, proposed a redesign using a new approach, got it approved.
Designed, Implemented, tested and verified the new shared and generic component. Integrated into the main product in the experimental branch. Presented to tech lead/management. Everyone was happy and my solution opened even more possibilities.
Now the WTF moment: the product with the updated dynamic UI solution never has been completely tested by a QA engineer despite my multiple requests and reminders.
It never got merged into baseline.
New initiatives to fix the dynamic UI issues have been made by other developers. Basically looking up my implementation. Removing parts they do not understand and wondering why the data validation does not work. And of course taking the credit.
2) back in 2013, boss wanted me to optimize batch processing performance in the product I developed. Profiling proved that the bottleneck ist not my code, but the "core" I had to use and which I must never ever touch. Reported back to him. He said he does not care and the processing has to get faster. And I must not touch the "core".
(FYI: the "core" was auto-generated from VB6 to VB.Net. Stored in SourceSafe. Unmaintainable, distributed about a bunch of 5000+ LoC files, eye-cancer inducing singlethreaded something, which had naive raw database queries causing the low performance.) -
!rant, more of an incredulous/cruelly amused "you had ONE job..."
so: biggest IT/PC/electronics store in my (and neighboring) country. their webpage, of course with the function to buy online, because of course.
the big green "Buy" button does nothing. doesn't work. doesn't react. I keep clicking it multiple times, shorter, longer, etc, because maybe their JS scripts are just shit so they slow.
nope.
okay. open devtools, JS console.
hover over the button: "Error: isMobile is not a function".
click the button: "Error: isMobile is not a function"
WAT.
search for isMobile in the script.
173 occurences.
fuck this.
console: isMobile = function(){return false;}
because I'm not on my phone.
click the "Buy" button.
works flawlessly.
...HOW?
THE WHOLE PAGE IS AN ESHOP YOU COMIC RELIEF INCOMPETENTS! =D
173 uses of non-existing function that blocks business-critical feature, THE ONLY CORE FEATURE FOR WHICH YOUR SITE EVEN EXISTS, and NOBODY, not the dev who fucked it up, NOT EVEN QA, noticed it??? =D =D
if I was the boss of the devs, or even boss of the whole company...
git blame
...and then i'd go the whole chain from the dev who caused the bug, through all of the QA people who "tested" that version before deploy, and I would personally, on the spot, fire each and every single one of them.
mainly because of who knows how much money this stupid not even a proper bug lost them.
but secondarily, because clearly none of those people give a single shit (n)or have an idea how to do their jobs.
=D =D
yeah but I was a good guy, filed a bug report in the "Complaints" section of their Contact form.
it goes to some call-center-like peon, so it starts with a sentence "forward this to your site's dev people outright to file as a bug, thank you".
but... HOW.... =D
HOW can you let something like this through? =D
the bottleneck of your whole user interaction, which forms first of the three steps OF THE MAIN AND MOST IMPORTANT FUNCTION of your whole business... =D
...I...
...does not compute =D
...BUT THEY USING ANGULAR, SO THEY ALL MODERN AND HIGH-TECH AND EVERYTHING'S FINE!!! =D =D1 -
Anybody else experienced this massive of a memory bottleneck when using Google Maps' iOS SDK?
Seriously Google you already own 90% of my RAM.. Now you wanna own 98% of the entire size of my app?
I didn't even know Github dropped files larger than 100MB..
1 -
Is anyone on here a proud owner of one of the current gen enthusiast platforms? (Threadripper or i9)
I am seriously wondering how much impact that brute power has on my build times. I just tried a ramdisk for my maven build and it seems that the faster storage did not increase build times much. So I am assuming there is a cpu bottleneck?3 -
Lesson learned from my previous rant:
https://devrant.com/rants/2059047/...
CPU bottleneck spotted. Time to fix some shit.
nvidia-docker vs native code execution brings around 10% performance decrease so far.
-
Consider an API that uses the HTTP path to represent position in a tree that literally represents a file tree with minimal constraints, and GET/PUT/DELETE methods to read, write and destroy the nodes. How would you encode read/write operations to per-node metadata? The kinds of metadata are static and around 4, so inventing HTTP verbs for each of them is infeasible but filtering is not necessary.
Options considered so far:
- toplevel resources alongside a namespaced /data such as /acl, /lock
- magic keywords to the Range header (this is apparently compliant)
- mimetypes such as text/plain+acl
- SETPROP / PROP methods in the spirit of WebDAV
- headers (I worry this may become an immitigable bottleneck really fast)
I'm looking for any kind of suggestion or insight, not perfect answers.
I read the WebDAV specification and I won't even suggest that I'm trying to align with it, the only protocol I'd seen in the past with comparable scope bloat is WebRTC.22 -
Allright, to settle a debate what is the more incorrect way of getting the last elememt of a std::vector?
Is it this:
auto i = myVec.at(myVec.begin() + myVec.size() - 1);
or:
auto i = *(&myVec.at(0) + (myVec.size() - 1));
I think the second method is better(worse) because it has more brackets and only works on Implementations where the vector's data is laid out concurrently (that is in the c++11 spec, but I've seen some shit)
My friend argues for the first method because it takes more time. (If IO is not a bottleneck)1 -
This question might make you lose a brain cell because of stupidity in the question. Read with caution
Is there a way to compile a game for Windows from Linux in Unreal engine? I did google some posts but the answer was either use a Virtual machine which will not be done or use the the theoretical method of using mingw but the forum posts state that it will be tricky business or use a windows machine. I have dual booted windows with linux on my machine.
However since the machine has a 512 gb ssd most of the storage space is devoted to unreal engine which takes 47 gigs in itself and have a lot of programs installed I have a usable 20 gigs left out of 145 gig partition. Windows has around 318 gigs of storage to it but I have 100 gigs free at most. So after installing the windows sdk, visual studio with extensions, unreal engine and some other stuff I don't have much space left for myself. I need that much space since I install a lot of games to my ssd. So now I cant load my bigger projects for playing on my windows. I could use my hdd which is mostly used for backups and 100+ gig stuff. Though the hdd's are of course far slower than ssd's which shouldn't be a problem however last time I used visual studio it ate more than 2 gigs of ram for a solution meaning that the compiler has very low memory for itself to actually compile so for any large files the hdd has more of a bottleneck.
Oh and I can't upgrade my ssd's or ram because I don't have enough money.
Thanks for the answers in advance4
Top Tags
Weekly Rant
View