I have to let it out. It's been brewing for years now.

Why does MySQL still exist?

Really, WHY?!
It was lousy as hell 8 years ago, and since then it hasn't changed one bit. Why do people use it?

First off, it doesn't conform to standards, allowing you to aggregate without explicitly grouping, in which case you get god knows what type of shit in there, and then everybody asks why the numbers are so weird.

Second... it's $(CURRENT_YEAR) for fucks sake! This is the time of large data sets and complex requirements from those data sets. Just an hour through SO will show you dozens of poor people trying to do with MySQL what MySQL just can't do because it's stupid.

Recursion? 4 lines in any other large RDBMS, and tough luck in MySQL. So what next? Are you supposed to use Lemograph alongside MySQL just because you don't know that PostgreSQL is free and super fast?

Window functions to mix rows and do neat stuff? Naaah, who the hell needs that, right? Who needs to find the products ordered by the customer with the biggest order anyway? Oh you need that actually? Well you should write 3-4 queries, nest them in an incredibly fucked up way, summon a demon and feed it the first menstrual blood of your virgin daughter.

There used to be some excuses in the past "but but but, shared hosting only has MySQL". Which was wrong by the way. This was true only for big hosting names, and for people who didn't bother searching for alternatives. And now it's even better, since VPS and PaaS solutions are now available at prices lower than shared hosting, which give you better speed, performance and stability than shared hosting ever did.

"But but but Wordpress uses MySQL" - well then kill it! There are other platforms out there, that aren't just outrageously horrible on the inside and outside. Wordpress is crap, and work on it pays crap. Learn Laravel, Symfony, Zend, or even Drupal. You'll be able to create much more value than those shitty Wordpress sites that nobody ever visits or pay money on.

"But but but my client wants some static pages presented beside their online shop" - so why use Wordpress then? Static pages are static pages. Whip up a basic MVC set-up in literally any framework out there, avoid MySQL, include a basic ACL package for that framework, create a controller where you add a CKEditor to edit page content, and stick a nice template from themeforest for that page and be done with that shit! Save the mock-up for later use if you do that stuff often. Or if you're lazy to even do that, then take up Drupal.

But sure, this is going a bit over the scope. I actually don't care where you insert content for your few pages. It can be a JSON file for all I care. But if I catch you doing an e-commerce solution, or anything else than just text storage, on MySQL, I'll literally start re-assessing your ability to think rationally.

1. Fucking MySQL database clusters.

There's nothing fun about MySQL clusters. Sometimes they start producing deadlock errors for no apparent reason... well, there's probably a reason, but it's never a transparent easy to find reason.

What was even less fun is that those errors took down a Sentry server. When your error log server goes down through ddos from your database messages, it's time to rethink your setup.

2. Wiring up a large factory with $2 arduino clones, each with a $2 esp8266 wifi chip, with various sensors for measuring flow of chemical solutions (I wanted cheap real time monitoring as an early warning system next to periodic sampling).

The scaling issue was getting over 500 streaming wifi signals to work in a 55c moist slightly corrosive atmosphere with concrete and steel everywhere, and getting it all into a single InfluxDB instance for analysis.

Help.

I'm a hardware guy. If I do software, it's bare-metal (almost always). I need to fully understand my build system and tweak it exactly to my needs. I'm the sorta guy that needs memory alignment and bitwise operations on a daily basis. I'm always cautious about processor cycles, memory allocation, and power consumption. I think twice if I really need to use a float there and I consider exactly what cost the abstraction layers I build come at.

I had done some web design and development, but that was back in the day when you knew all the workarounds for IE 5-7 by heart and when people were disappointed there wasn't going to be a XHTML 2.0. I didn't build anything large until recently.

Since that time, a lot has happened. Web development has evolved in a way I didn't really fancy, to say the least. Client-side rendering for everything the server could easily do? Of course. Wasting precious energy on mobile devices because it works well enough? Naturally. Solving the simplest problems with a gigantic mess of dependencies you don't even bother to inspect? Well, how else are you going to handle all your sensitive data?

I was going to compare this to the Arduino culture of using modules you don't understand in code you don't understand. But then again, you don't see consumer products or customer-specific electronics powered by an Arduino (at least not that I'm aware of).

I'm just not fit for that shooting-drills-at-walls methodology for getting holes. I'm not against neither easy nor pretty-to-look-at solutions, but it just comes across as wasteful for me nowadays.

So, after my hiatus from web development, I've now been in a sort of internet platform project for a few months. I'm now directly confronted with all that you guys love and hate, frontend frameworks and Node for the backend and whatever. I deliberately didn't voice my opinion when the stack was chosen, because I didn't want to interfere with the modern ways and instead get some experience out of it (and I am).

And now, I'm slowly starting to feel like it was OKAY to work like this.

Is there a lot of people in the same boat as me?

I'm a self taught guy. Never in my life had I a senior developer i could bug for answers. Every little bug and inconveniece i have ever experienced - left alone to cope and find solutions. I just feel like sooo burned out. I have some large complex system questions building up and googling doesnt give me the answers anymore. This is frustrating. I'm supposed to be a mid level developer, but I'm acting as a senior to one of my colleagues even though I have so many questions and doubts in my mind. I think I developed a lot of plot holes in my knowledge and I have no real way to know which are which. I feel I dont know so much. Fuck. Where do I go from here?

A couple of months back we were discussing sh with a third party vendor for a very large ass fuck system that another department uses. I had been called into the meeting because the entire I.T department counts on me to at least act as an assessor to the many issues that other departments might have.

the department for which i was working with manages the databases that our institution uses, and in this particular question the DBA (my best friend mind you) was part of the meeting.

Mind you, issues that the third party vendor were having were all fixed by our DBA, and he had documented and mentioned these items to me as I provided assistance to him through the 3 weeks prior to this meetings. Once such case was that we needed a transitioning as well as intermediary system for some processes to happen from one DB to the other and a lot of other technical babble. Well, the DBA used to be an excellent (fuck you) VB developer who recently re-learned the language into .net. He had shown me many of his old programs and even by the limitations of the language they were elegant and fascinating. They really are and ya'll devrant fam know that I ain't one to hate on tech at all.

When the DBA explained how he went around some of the issues by generating programs that could assist him, he mentioned the tech stack, I had coached him into knowing that being descriptive about the tools he used would be beneficial to everyone else. While he mentioned VB.NET the vendor snickered and my boy got quiet.

Then I broke the silence, fuck you. "what was that?" and the dude said "nothing, sorry"
So I said "no no, I want to know, I am not going past this point until you, the dude getting paid over $100 an hour for something YOU couldn't fix explain to me the little hehe moment you had"

The mfker went silent. then explained how he was aware that people were moving past vb.net and shit like that, me "imagine that, someone used a tech stack that your ignorance thought obsolete to fix something you could not solve, even though we are paying you for it, were it me or in my hands, and mind you i have direct access to the VP so this foolishness might change, I would have cut you and your little sect loose months ago, I have no patience, or appreciation from leeches like you or the rest of the "professionals" that work for your company or other similar entities, much less, as you can see, my patience runs even less when you people snicker at the solutions that our staff has to take when you all slack"

The entire meeting was uncomfortable as high heaven.

Fuck you, if someone I know manages to run shit on fucking liberty basic then so fucking be it. I will slap you 10 fucking times over, and then fuck your girl, if you try to put someone else down for the tech stacks you use.

I hate neck beards, BUT I hate fake ass neckbeards ever more

*Colin Farrell in true detective mode: FUCK....YOU

A misconception that software engineers just sit in front of their laptops and code 40 hours a week, with no social interaction.

A software engineer’s job is actually pretty social. Personally, I probably spend around half of my time interacting with people. This could be partially due to 1:1, team, and other meetings. But a large part of it is spent in bouncing off ideas about your project with your project mates (especially during the planning phase), chiming in the conversations about some recent or urgent problems to help find or propose solutions, answering others’ questions, organizing some events, etc.

Of course, I do need some dedicated uninterrupted time to focus on programming and to get into the zone, but it’s certainly not the only activity I do at work. The main point to understand is that the software engineering is not a solitary, but a social job.

Overall, I’m very happy with my profession. The enjoyment I get out of my work vastly outweighs all of these points combined.

There was a sales manager who was raked with overseeing me and another dev finish a last minute request project. He said at one point to the other dev that he was mad at developers because we understood something that he would never understand.

This same manager would often sit in on estimation meetings and constantly say that we were estimating too high and needed to come up with faster solutions. When we would offer him with caveats of possible technical debt or unintended side-effects/performance issues, he'd want us to go with that solution. He would then complain that we were always wanting to work on technical debt and that our application was slow. He would also ask for very high level estimates for large, unscoped features/apps without any meaningful level of detail, then hold us to the high-level estimated date even after revealing additional features previously unmentioned.

We learned to never compromise on the right solution and to push back hard on dates without proper scoping. They didn't learn, so I and most of the good devs left.

This would be my first official post.

Been a IT Technician for a managed service provider for the past 9 years up until last year August. Managing director pulls me in with a movement to App Development after coming across some personal hobby projects I have done in the past.

Started in the new position in November as Junior Developer and workloads get dumped on me and left to figure it out. 4 weeks of running through code without documentation and the solutions started to make sense.

Started a new solution for a Large remote customer with documentation and timelines in December and I get pulled in again for a second time in front of the MD.

Good News:With effect in January I have been promoted to Head of Application development.

Bad News: The existing department head is leaving end of the month and I am to go 900km from home to hand over all responsibilities for the next 3 weeks.

Better News: Department has started shifting to DevOps and it is up to me to set the policies and work flows to how I see fit.

Worse news: it starts by expanding the team asap as 10 projects accounting to 4000 man hours with deadlines in Q3.

Wish me luck. It's going to be twisted Rollercoaster ride...

Not a rant but I'm not sure if I made the right choice. I got really curious this year in cloud infrastructure and started studying for my AWS Solutions Architect Exam. The company I work for (large company) allows me to switch roles/teams. Last week I was so bored with frontend I requested to switch to the Platform engineering team. The Engineer manager slacked me this morning....

PE Manager: I'll see you at standup this morning.

Me: Cool ill be there at 9, where is it? By the cafe?

PE Manager: No, in hell. Glad you joined the team 😃

What did I get myself into.

If you've ever tried using Go plugins raise your hand.
If you've ever tried doing plugins in Go, raise your hand.
If you think that the following rant will be interesting, raise your hand.
If you raised your hand, press [Read More]:

This is a tale of pain and sorrow, the sorrow of discovering that what could be a wonderful feature is woefully incomplete, and won't be for a very long time...

Go plugins are a cool feature: dynamically load pre-compiled code, and interact with it in a useful and relatively performant way (e.g. for dynamically extending the capabilities of your program). So far it sounds great, I know right?

Now let me list off some issues (in order of me remembering them):
1. You can't unload them (due to some bs about dlopen), so you need to restart the application...
2. They bundle the stdlib like a regular Go binary, despite the fact that they're meant to be dynamic!
3. #2 wouldn't be so bad if they didn't also require identical versions of all dependencies in both binaries (meaning you'd need to vendor the dependencies, and also hope you are using the right Go version).
4. You need to use -trimpath or everything dies...

All in all, they are broken and no one is rushing to fix it (literally, the Go team said they aren't really supporting it currently...).

So what other options are there for making plugins in Go?

There's the Hashicorp method of using RPC, where you have two separate applications one the plugin, one the plugin server, and they communicate over RPC. I don't like it. Why? Because it feels like a hack, it's not really efficient and it carries a fear of a limitation that I don't like...

Then we come to a somewhat more clever approach: using Lua (or any other scripting language), it's well known, it's what everyone uses (at least in games...). But, it simply is too hard to use, all the Go Lua VMs I could find were simply too hard to set up...

Now we come to the most creative option I've seen yet: WASM. Now you ask "WASM!? But that's a web thing, how are you gonna make that work?" Indeed, my son, it is a web thing, but that doesn't mean I can't use it! Someone made a WASM VM for Go, and the pros are that you can use any WASM supporting language (i.e. any/all of them). Problem inefficient, PITA to use, and also suffers from the same issues that were preventing me from using Lua.

Enter Yaegi, a Go interpreter created by the same guys who made (and named) Traefik. Yes, you heard me right, an INTERPRETER (i.e. like python) so while it's not super performant (and possibly suffering from large inefficiency issues), it's very easy to set up, and it means that my plugins can still be written in Go (yay)! However, don't think this method doesn't have its own issues, there's still the problem of effectively abstracting different types of plugins without requiring too much boilerplate (a hard problem that I'm actively working on, commits coming soon). However, this still feels to be the best option.

As you can see, doing plugins in Go is a very hard problem. In the coming weeks (hopefully), I'm going to (attempt to at least) benchmark all the different options, as well as publish a library that should help make using Yaegi based plugins easier. All of this stuff will go (see what I did there 😉) in a nice blog post that better explains the issues and solutions. But until then I have some coding to do...

Have a good night(/day)!

PouchDB.

It promised full-blown CRDT functionality. So I decided to adopt it.

Disappointment number one: you have to use CouchDB, so your data model is under strict regulations now. Okay.

Disappointment number two: absolutely messed up hack required to restrict users from accessing other users’ data, otherwise you have to store all the user data in single collection. Not the most performant solution.

Disappointment number three: pagination is utter mess. Server-side timestamps are utter mess. ANY server-side logic is utter mess.

Just to set it to work, you need PouchDB itself, websocket adapter (otherwise only three simultaneous syncs), auth adapter (doesn’t work via sockets), which came out fucking large pile of bullshit at the frontend.

Disappointment number four, the final one: auth somehow works but it doesn’t set cookie. I don’t know how to get access.

GitHub user named Wohali, number one CouchDB specialist over there, doesn’t know that either.

It also doesn’t work at Incognito mode, doesn’t work at Firefox at all.

So, if you want to use PouchDB, bear that in mind:
1. CouchDB only
2. No server-side logic
3. Authorization is a mess
4. Error logs are mess too: “ERROR 83929629 broken pipe” means “out of disk space” in Erlang, the CouchDB language.
5. No hosting solutions. No backup solutions, no infrastructure around that at all. You are tied to bare metal VPS and Ansible.
6. Huge pile of bullshit at frontend. Doesn’t work at Incognito mode, doesn’t work at Firefox.

I work in a large organization that previously didn't have it's own development team. Therefore various business areas have built their own solutions to solve problems which mostly involve Access and Excel.

Many of these applications still exist and we are expected to resolve any issues with them and update them when necessary performing this support role while still expected to meet our (very tight) development timelines.

I can't tell you how much of a pain in the tits it is to be trying to power through a priority development only to be interrupted with an urgent instruction to fix a 17 year old Access database that's running slow.

Of course it's pissing running slow, it's 17 years old, has nearly a million records and you have multiple users accessing it across the country!! I think it's time to peacefully let it die.

Got a new job on a big brand bank in the financial district in NYC, went through multiple code interviews, 2 hours of in person interviews asking me about architectures, design patterns, solutions to imaginary complex problems(which I enjoyed thinking about), finally got accepted, background checks needed before starting (previous job check, credit, drugs, etc..) so I waited 2 months, 1st day at work, the building is huge and cool, biggest spaces I've ever seen, amazingly insane large monitors and people working on a great variety of new technologies.

I was assigned to a corner far away from the open spaces, trying to understand a project that I will maintain who works with java 5, struts and jsp(for fucks sake, JSP!!!)

Why life laughs on my face? why?

We use at our company one of the largest Python ORM and dont code ourselfs on it, event tough I can code. Its some special contract which our General Manager made, before we as Devs where in the Project and everything is provided from the external Company as Service. The Servers are in our own Datacenter, but we dont have access.

We have our Consultants (Project Manager) as payd hires and they got their own Devs.

Im in lead of Code Reviews and Interfaces. Also Im in the "Run" Team, which observes, debuggs and keeps the System alive as 3rd-Level (Application Managers).

What Im trying to achieve is going away from legacy .csv/sftp connections to RestAPI and on large Datasets GraphQL. Before I was on the Project, they build really crappy Interfaces.

Before I joined the Project in my Company, I was a Dev for a couple of Finance Applications and Webservices, where I also did coding on Business critical Applications with high demand Scaling.

So forth, I was moved by my Boss over to the Project because it wasn't doing so well and they needed our own Devs on it.

Alot of Issues/Mistakes I identified in the Software:
- Lots of Code Bugs
- Missing Process Logic
- No Lifecycle
- Very fast growing Database
- A lot of Bad Practices

Since my switch I fixed alot of bugs, was the man of the hour for fixing major Incidents and so on so forth. A lot of improvements have been made. Also the Team Spirit of 15+ People inside the Project became better, because they could consult me for solutions/problems.

But damn I hate our Consultants. We pay them and I need to sketch the concepts, they are to dumb for it. They dont understand Rest or APIs in general, I need to teach them alot about Best Practices and how to Code an API. Then they question everything and bring out a crooked flawed prototype back to me.

WE F* PAY THEM FOR BULLCRAP! THEY DONT EVEN WRITE DOCUMENTATION, THEY ARE SO LAZY!

I even had a Meeting with the main Consultant about Performance Problems and how we should approach it from a technical side and Process side. The Software is Core Business relevant and its running over 3 Years. He just argumented around the Problem and didnt provide solutions.

I confronted our General Manager a couple of times with this, but since 3 Years its going on and on.

Im happy with my Team and Boss, they have my back and I love my Job, but dealing with these Nutjobs of Consultants is draining my nerves/energy.

Im really am at my wits end how to deal with this anymore? Been pulling trough since 1 year. I wanna stay at my company because everything else besides the Nutjob Consultants is great.

I told my Boss about it a couple of times and she agrees with me, but the General Manager doesnt let go of these Consultants.

Even when they fuck up hard and crash production, they fucking Bill us... It's their fault :(

At work I am "the" programmer and is the first time in which I actually enjoy showing different solutions to problems without having a fear of implementing large things without having any form of recognition.

Seeing someone get happy because of something you created is a great feeling and even tho most of us are misantrophic af we can still appreciate bringing happiness through code.

To me, software engineering is the closest thing to magic and I really believe that.

Two days ago I showed my manager a little utility to build small portions of the site we are building and make changes to it in real time without browser refreshes for whatever change she would like to do. She was super happy and excited and it made me feel real happy.

Such great feeling man. Nothing but good vibes brother!!

Situation: I have a love hate relationship with python due to the lack of types as I have in more established languages such as C#, Java and shit even TypeScript

Situation (cont): A rather large codebase that i have developed for multiple processes at work run on Python.

I don't hate it, I just don't absolutely love it, there is a lot of things to like about Python, but man I do have some conflicts with it, I have been facing out to use other solutions that feel scripty, such as the newer versions of C# with .net, but I would say that about 80% of our codebase runs on Python, the rest is PHP.

I am somewhat traditional in the way my programs run, I started with C++ and Java, then for whatever reason (I blame codecademy at the time) switched over to Ruby and Javascript, mostly Javascript. I do not remember how I found Python, I do remember learning it with an online tutorial, shit was easy to get started with.

My codebase running on Python is huge, and they do a lot from automation scripts, to data gathering and database management, never had I been bitten with the "oh noes is so slow" bug since my code is not Google level big, for everything else Python seems rather fast imho

I dunno, big time love hate relationship

When you are in Visual studio and you want to find something and you do ctrl+f and type in your search and hit enter and realize that the search location was set to "Entire Solution" 😖😖😖

I hate the elasticsearch backup api.

From beginning to end it's an painful experience.

I try to explain it, but I don't think I will be able to cover it all.

The core concept is:
- repository (storage for snapshots)
- snapshots (actual backup)

The first design flaw is that every backup in an repository is incremental. ES creates an incremental filesystem tree.

Some reasons why this is a bad idea:
- deletion of (older) backups is slow, as newer backups need to be checked for integrity
- you simply have to trust ES that it does the right thing (given the bugs it has... It seems like a very bad idea TM)
- you have no possibility of verification of snapshots

Workaround... Create many repositories as each new repository forces an full backup.........

The second thing: ES scales. Many nodes / es instances form a cluster.

Usually backup APIs incorporate these in their design. ES does not.

If an index spans 12 nodes and u use an network storage, yes: a maximum of 12 nodes will open an eg NFS connection and start backuping.

It might sound not so bad with 12 nodes and one index...

But it get's pretty bad with 100s of indexes and several dozen nodes...

And there is no real limiting in ES. You can plug a few holes, but all in all, when you don't plan carefully your backups, you'll get a pretty f*cked up network congestion.

So traffic shaping must be manually added. Yay...

The last thing is the API itself.

It's a... very fragile thing.

Especially in older ES releases, the documentation is like handing you a flex instead of toilet paper for a wipe.

Documentation != API != Reality.

Especially the fault handling left me more than once speechless...

Eg:
/_snapshot/storage/backup
gives you a state PARTIAL
/_snapshot/storage/backup/_status
gives you a state SUCCESS

Why? The first one is blocking and refers to the backup status itself. The second one shouldn't be blocking and refers to the backup operation.

And yes. The backup operation state is SUCCESS, while the backup state might be PARTIAL (hence no full backup was made, there were errors).

So we have now an additional API that we query that then wraps the API of elasticsearch. With all these shiny scary workarounds like polling, since some APIs are blocking which might lead to a gateway timeout...

Gateway timeout? Yes. Since some operations can run a LONG (multiple hours) time and you don't want to have a ton of open connections hogging resources... You let the loadbalancer kill it. Most operations simply run in ES in the background, while the connection was killed.

So much joy and fun, isn't it?

Now add the latest SMR scandal and a few faulty (as in SMR instead of CMD) hdds in a hundred terabyte ZFS pool and you'll get my frustration level.

PS: The cluster has several dozen terabyte and a lot od nodes. If you have good advice, you're welcome - but please think carefully about this fact.

I might have accidentially vaporized people sending me links with solutions that don't work on large scale TM.

I am starting my web development business and i intend to focus on building mainly custom ecommerce solutions to small and medium businesses or large ones too. I am just about to launch my first project and my client has been great.
Any advice from the pros in the house who have been there done that will be worth more than diamond right now to me.
I like advice mainly on how to find new clients.

Anyone out there ever had to deal with WordPress in Azure? Do you have any horror stories to share? I’m working on convincing a potential BIG client to switch away from Azure and need some good examples of why it’s not as suitable for large-scale hosting and management of multiple WordPress installs as other solutions available.

I had mistakenly added a large file to a commit, and I'm now spending 3 hours of my life just to remove it and being able to push.

I've deleted the file from tracking, but it remained in history so when I try to push, github rejects to continue.

And, still worse, trying various solutions on StackOverflow I've done a mess on the history which now looks unrelated to the remote one, and I think it's a never-end catastrophe.

It's absurd how badly designed is git, and how hard it is to use besides the 3 commands that you learnt by heart

Need an advice on syncing large amounts of small files. For starters I have 2 million of ~20-25kb sized files in dropbox. Syncing it with my local machine is a pain in the ass (API limits syncing to 4-5 files per second, so I will need at least 96 hours just to take out everything from dropbox).

Recently I started looking into self hosted cloud solutions. I fiddled around with nextcloud but it's client limits syncing to 5-6 files per second as well. Accessing nextcloud via webdav from a rclone client I was able to up the syncing speed to 15-25 files per second but that's all.

Any solutions where I should host 2mil of files and would be able to sync them fast to my own machine with some cloud client when I need it to?