Details
-
AboutData Eng with a long history of abusive bosses and awesome projects. Got a MSc in Optimization and a couple startup failures under my belt.
-
SkillsPython, C/C++, Cloud Architecture, Spark, Parquet, AsyncIO, Sarcasm, Heuristics, Optimization, Science, Academics
Joined devRant on 10/26/2021
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API

From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
"Most electronics work better while turned *on*. The rest are made by HP"
Fucking printer will not only *not* print my stuff but will also make hellish noises, print the same "hello world!" page six times in a row, jam paper in the seventh and spit expensive ink on its own mechanism while there is not even paper in it.
Fucking outdated cellulose-based bureaucracy. Govt' websites are far from perfect but yet are closer to it than presenting fucking tree-meat slabs unecessary covered in colourful liquid gold.2 -
I wonder if crypto exchanges are so damn vulnerable or just so transparent.
I mean, it is impossible to scroll tech articles for more than a few seconds before stumbling on a report of yet another crypto exchange being nicked a couple hundred mil USD.
- It could be that their security severely sucks (wouldn't blame them for it, most businesses do suck at securing shit).
- It could be that the entire black hat community is putting it's might on stealing money that is so fucking easy to launder.
- It could be that is damn nigh impossible to cover up a crypto hack since the evidence of coins drifting away is forever on display in the public ledger, and in that case crypto companies are not hacked more often than regular companies, they are just much more often publically shamed for it.
- It could be a mix of all the above, but my intuition is that one factor is more relevant.
Which would be the most relevant factor? One of the above or yet another attack vector to the stupidest value conduit ever?4 -
I've been reading about quantum computing in finance and other applications (fascinating read, althought really dense), but one question now won't stop bugging me.
Context:
1) Blockchain applications are based on NP-Hard asymmetric cryptographic problems, and how hard it is to solve such problems in a really short time.
2) So called "Web3.0" is based mostly on Blockchain applications, but would still need significant advances in order to be practical.
3) Affordable and practical cloud-based quantum computing is not so far in the future, and could be used to crack most NP-Hard problems in short (polynomial) time.
Thus, my question: Is Web3.0 obsolete before it even begun?
I mean, if quantum computing takes on fast enough, it could snuff out Blockchain applications by giving those a shelf life so short it wouldn't be worth to delevolp for it. It would be like announcing the iPhone 14 and the 15 on the same breath, saying the 15 is only a quarter away - why would anyone bother with the born-obsolete tech?5 -
Recruiters on LinkedIn be like "Yu sim like a fit is much good at in company ours indutri tek AI is startup! Google is invest we money in! Kan we talk is good?"
Seriously, lay down the Google translate. Why even bother recruiting in a language themselves can't understand?2 -
"It is pointless to use just a fraction of the data in a homologation environment"
Those words reveal the truth in our creed.
We work in the deepest of back-ends to serve the front.
No data is true. Everything can be edited.
We are Data Engineers.
And for those words to take hold, a junior must execute a leap of faith, and push a hotfix into production.5 -
<"Perfect is enemy of good"
>"Excellent! I keep my enemies very close"
I do believe it possible that one can find at least one perfect counter to every stupid folk saying that startup-for-brains suit bags love to parrot.
2)
<"We must fail fast"
>"I already did it!"
3)
<"We must have a long tail of offerings"
>"Can we offer focus on our core strengths?"
-3)
<"We must focus on our core strengths"
>"Isn't our core strength 'having a long tail of offerings'?"
4)
<"We must use agile methods"
>"An agile habit does not make an agile monk."
5)
<"We must be flexible and adapt"
>"Is it a law or more of a rule of thumb?"
6)
<"We must avoid bureaucracy"
>"Can I have that in writing?"3 -
Fuck, every time I read something about NFTs I lose another bit of faith in humanity I didn't even know I had.
I mean, I guess someone could make a bunch of cartoon characters (lets say, some sort of "pookey-man") using compatible styles and colour palettes. And I guess you could buy some of those if there is a product that makes trademark recognition a plausible dream (let's say, a videogame and/or anime).
But if the chance of success is akin to the chance of becoming a billionaire solemly from lottery wins, shouldn't the assets be priced accordingly?
Like a couple bucks a pop, at most?
Dropping a cool 30000 quid on a single lotto ticket sounds like the dumbest thing ever.
And yet at least a couple hundred hollow heads did it.
Fuck everything.4 -
Fuck startups.
Back when I was an wee lad I interviewed for an startup, not knowing that startups are not real companies. The scumbag interviewer, who was also the owner of the outfit, asked me what I was looking in a company. I said "fair wages, a non-antagonic environment and projects with real roadmaps".
He asked me to elaborate. I said, "You know, if today your product is a sales platform, I do not want to come into work next week and discover it is now an air travel tickets marketplace, or come back the very next day and discover it is now an automated pizza factory, or in the next day and it is now a crypto exchange..."
The scumbag looked PISSED. "Sorry, but we are looking for someone who likes the challenges of a dynamic environment (read: we do not have a business model and we hate the very idea of trying to make money out of our company), and you do not fit the profile"
Startups are not real companies, i.e. they do not systematically charge money in exchange for goods or services in amounts that exceed the cost of providing said goods or services. Most startups are just tax fronts for money laundering schemes. The rest are just playthings for rich assholes who can't get a real output-producing job. Those two categories are not mutually exclusive.
Take Facebook, for example. The poster child of startups. The Zucker that owns it just announced they are setting impossible performance targets on purpose, not even attempting to hide the fact that it is just a way to lay off large quantities of employees without using the words "massive lay offs". Companies, real thin-margin, lots-of-regulation profit-driven companies do not do that. They are not some sort of "capitalist woke", real CEOs just know that if their companies largely miss performance targets on their tenure, purposely or not, next it will be their neck on the chopping block. Because they can be fired if the KPI charts say they suck. But the Zucker cannot be fired, not even after commanding their beanbag and tap beer offices to be heated exclusively by burning hundred dollar bills.
So the Zucker is not interested in performance. Not even in lay offs as expense cutting measures - investors are an infinite source of free money for startups. The Zucker just wants to project power, especially now that engineers are not so confident in the stability of they high-paying jobs.
So are irrelevant 500-souls-or-less self-aggrandizing startups. Their owners are there because it is in vogue to have a startup or ten. And will have that startup pivot to whatever sounds fancy that season. After all, only poor people care about things like EBITDA and profit margins repeatability - A.K.A. "getting more money".
Fuck startups.13 -
REDIS: Great for cloud, will fuck up your local disk if too many write operations per second.
DynamoDB: WTF 10Mb should not be "too large for a single record"!!
SPARK: NEVER CONNECT IT TO A DATABASE! Wasted A LOT of cluster time. Also, can you be LESS specific on exactly what are the bugs in my code? 'cause I don't think it's possible.
NPM: can't install a package for shit. tried it waaaay to many times.
Makefiles: Just fuck you.
WSL1: breaks more often than a glass hammer.
Python >= 3.6: FUCK ENCODINGS!!
Jupyter: STOP MESSING UP WHILE SAVING!
Living is to collet bugs, it seems.5 -
There once was a bright young engineer who was hired by a company to design their new light ship.
Like 50 seconds after getting inside the company, the engineer was approached by a douchebag in a business suite.
"Hey, can you make us a mock up of the ship's design in the next hour or so? Nothing fancy, it must be very simple! To not overcomplicate it! Just a simple mock up so we can all see what are we talking about in this project! Please do not overthink this!"
The engineer, young and naive, just folded some piece of paper and gave the douchebag a paper boat.
"Fantastic! That's all we need for the presentation for the investors!"
A couple hours later the suite was back screaming.
"YOUR FUCKING FARSE! YOUR SHITTY SHIP EMBARRASED US ALL! THE VERY MOMENT OUR CEO TRIED TO STEP ON IT IT SANK! YOU ARE FIRED AND WE WILL SUE YOU FOR INCOMPETENCE! I ASKED YOU SOMETHING SIMPLE AND YOU CAME UP WITH THIS OVER ENGINEERED PIECE OF CRAP, YOU SON OF A.. [many, maaany expletives suppressed for brevity sake]"
This is how I feel everytime someone asks for "a tiny change" or some "very simple solution".
If it was so simple that it could be done in such short notice, than why the fuck do it at all, instead of buying it? I heard people sell all sorts of things in the internet nowadays. Software fucking included.5 -
Once upon a time, there were a restaurant called "iEat.tech.com".
It was a small single-location place, where the sufficient number of patrons could be served by the cozy number of employees.
In fact, headcount was so lean that the cook was also the one who washed all the dishes.
But then came the suits and their "VC"(daddy) money and scaled shit up.
Soon, there were so many patrons that the dishes started to pile up the sink, never washed.
"We need someone to wash the dishes!" said the cook
"Fuck you, you wash the dishes!" said the s*its
Naturally, the cook left soon after.
The s*its had a problem now. They could not replace the cook fast enough - all other cooks were either young, inexperienced and mediocre (but did clean the dishes), or refused to waste their time on the sink.
So the suits did what $*its always do - they got a fucking consultant. Who told them to get a fucking dishwashing machine and billed them the GDP of Ireland.
The s*is, of course, did not want to buy a dishwashing machine. "Our fucking process is too fucking disruptive for us to use a fucking store-bought mass-produced metal servant!" (s*its don't know what "machines" are. For them, it's all in terms of "servants", employees and machines alike).
So the s*its hired an engineer to "solve the fucking dish problem, once and for all".
The engineer quickly started measuring and drawing and calculating. The engineer was about to prepare a budget when the s*its came screaming "What the fuck are you doing? There is a fucking pile of dishes in the sink!"
The engineer replied that "I'm designing the machine!", to what the s*its responded "don't bring me fucking problems, bring me solutions!" (or some other s*it blabber)
So the engineer quickly designed an efficient dishwashing assembly line to be done in half the time most people would. And then went back to designing the machine.
But the s*its were having none of it. They kept expanding and expanding and doing what they could so that the engineer never had a moment to work on the machine. They dit it so surreptitiously that no one barely even noticed, but one day they were paying a team of engineers to be fucking human dishwashers.
Now replace "dishes" with "Jira tickets" or "quick fixes" or "tiny changes" and fix other terms accordingly.
Fucking s*its.9 -
So, for the last year or so, we've been playing with a natural language A.I.
The goal was to predict port, truck and rail service disruption due to social unrest.
The trick here is that our AI would "read between the lines" of today's news articles and spit out keywords that were likely to appear in near future articles, thus giving us an early warning before some union or army start blockading roads.
It... did not work as intended. But some very weird results came out.
Apparently, we made a robotic "kid that screams that the emperor has no clothes", yielding unlikely (but somewhat expected) keywords when fed collections of articles.
We gave it marketing content about our company. It replied "high suicide rate".10 -
Manager: Can you stay late as fuck today? One of our bitchiest vendors is gonna update their piece of crap and I'm pretty damn sure shit is gonna hit the fan
Dev (inner voice): no fucking way, I have kids to watch and chores to do!
Dev (outer voice): can't we just check everything in the morning?
Manager: No fucking way! If there is some fucking "challenge" when our "people" try to log onto their shit, I'm gonna look like a chump!
Let's talk silvers, I will sign on that bloody commie bullshit for your hours tonight.
Dev (outer voice): Fine. Until how late?
Dev (inner voice): Wait, I was supposed to do it without getting overtime bonus?6 -
"get lost"
The fucker was just saying he would not help me, but in the end I should have taken the advice literally and got the fuck out of that fucked up place. -
Can someone please write a plugin that automatically translates bug tickets into startup-ese? It would make it very easy to present the most common offenders to higher management on quarterly reviews.
Example:
-
$< "screen A is all messed up"
$> "We are disrupting the establishment in screen A"
-
$< "API B is not responding, giving timeout errors"
$> "We are facing logistical challenges in API B"4 -
Fuck Apache TIKA.
Its supposed to be a "universal file reader" or some shit. Im trying to use it as a PDF/image parser that does OCR when needed and yelds a full-file string. It does so, but the text ends up being IN THE WRONG FUCKING ORDER.
WTF would I want to parse the text out of a PDF in any order that is not the one the text is supposed to be read?!?!
"It is more efficient to work in random ordering", says the docs. No shit, really? Wouldn't it be even more efficient to just spit out random strings? Just as useful and 100% CPU-bound.
"You can add a property to forcefully put the text in the right order". THEN WHY THE FUCK IT IS NOT THE DEFAULT SETTING?
Srsly, what's the use case to a parser that yields scrambled text?!?1 -
Have anyone ever tried making an NFT of a crypto coin?
Like, you have an NFT of a link to an specific state of the blockchain - "own the highest value transaction ever!"
And try buying it using some other cryptocoin.
And make an NFT of the first time someone used a cryptocoin to buy the NFT of another coin.
And buy THAT NFT with another coin. And so on and so on...
Just trying to cause a too-many-recursions error in reality here.7 -
I try to explain my problems to my 6yo twins. It is just HILARIOUS to see their small faces going "huh. Did you tried using that spark thing to send your emails for you?".
Srsly, they give better advice than half the devs I work with. Rubber ducking be damned.6 -
So, my wife got this new PR job where she sometimes gets called in the middle of the fucking night because some jackoff in Singapore said or did some shit.
Since "husbands that work deep into the night get cheated on by their wives", and I believe in equal gender rights, I thought it would be a good opportunity to watch the new episode of the series we watch together. By my fucking self.
Is that how it feels to cheat on someone?1 -
This is the third part of my ongoing series "The Ballad of the Six Witchers and the Undocumented Java Tool".
In this part, we have the massive Battle of Sparks and Storms.
The first part is here: https://devrant.com/rants/5009817/...
The second part is here: https://devrant.com/rants/5054467/...
Over the last couple sprints and then some, The Witcher Who Writes and the Butchers of Jarfile had studied the decompiled guts of the Undocumented Java Beast and finally derived (most of) the process by which the data was transformed. They even built a model to replicate the results in small scale.
But when such process was presented to the Priests of Accounting at the Temple of Cash-Flow, chaos ensued.
This cannot be! - cried the priests - You must be wrong!
Wrong, the Witchers were not. In every single test case the Priests of Accounting threw at the Witchers, their model predicted perfectly what would be registered by the Undocumented Java Tool at the very end.
It was not the Witchers. The process was corrupted at its essence.
The Witchers reconvened at their fortress of Sprint. In the dark room of Standup, the leader of their order, wise beyond his years (and there were plenty of those), in a deep and solemn voice, there declared:
"Guys, we must not fuck this up." (actual quote)
For the leader of the witchers had just returned from a war council at the capitol of the province. There, heading a table boarding the Archpriest of Accounting, the Augur of Economics, the Marketing Spymaster and Admiral of the Fleet, was the Ciefoh Seat himself.
They had heard rumors about the Order of the Witchers' battles and operations. They wanted to know more.
It was quiet that night in the flat and cloudy plains of Cluster of Sparks and Storms. The Ciefoh Seat had ordered the thunder to stay silent, so that the forces of whole cluster would be available for the Witchers.
The cluster had solid ground for Hive and Parquet turf, and extended from the Connection River to farther than the horizon.
The Witcher Who Writes, seated high atop his war-elephant, looked at the massive battle formations behind.
The frontline were all war-elephants of Hadoop, their mahouts the Witchers themselves.
For the right flank, the Red Port of Redis had sent their best connectors - currency conversions would happen by the hundreds, instantly and always updated.
The left flank had the first and second army of Coroutine Jugglers, trained by the Witchers. Their swift catapults would be able to move data to and from the JIRA cities. No data point will be left behind.
At the center were thousands of Sparks mounting their RDD warhorses. Organized in formations designed by the Witchers and the Priestesses of Accounting, those armoured and strong units were native to this cloudy landscape. This was their home, and they were ready to defend it.
For the enemy could be seen in the horizon.
There were terabytes of data crossing the Stony Event Bridge. Hundreds of millions of datapoints, eager to flood the memory of every system and devour the processing time of every node on sight.
For the Ciefoh Seat, in his fury about the wrong calculations of the processes of the past, had ruled that the Witchers would not simply reshape the data from now on.
The Witchers were to process the entire historical ledger of transactions. And be done before the end of the month.
The metrics rumbled under the weight of terabytes of data crossing the Event Bridge. With fire in their eyes, the war-elephants in the frontline advanced.
Hundreds of data points would be impaled by their tusks and trampled by their feet, pressed into the parquet and hive grounds. But hundreds more would take their place. There were too many data points for the Hadoop war-elephants alone.
But the dawn will come.
When the night seemed darker, the Witchers heard a thunder, and the skies turned red. The Sparks were on the move.
Riding into the parquet and hive turf, impaling scores of data points with their long SIMD lances and chopping data off with their Scala swords, the Sparks burned through the enemy like fire.
The second line of the sparks would pick data off to be sent by the Coroutine Jugglers to JIRA. That would provoke even more data to cross the Event Bridge, but the third line of Sparks were ready for it - those data would be pierced by the rounds provided by the Red Port of Redis, and sent back to JIRA - for good.
They fought for six days and six nights, taking turns so that the battles would not stop. And then, silence. The day was won, all the data crushed into hive and parquet.
Short-lived was the relief. The Witchers knew that the enemy in combat is but a shadow of the troubles that approach. Politics and greed and grudge are all next in line. Are the Witchers heroes or marauders? The aftermath is to come, and I will keep you posted.4 -
LinkedIn posts be like:
"Have a #challenging #look at my #innovative #carrer_moment! I did #something at my #new_company!
Look at this #picture of my #awesome #new_workplace! I am #glad of being here at #company!"
(#opentowork, #looking_for_opportunities, #recruiters_welcome)3 -
Why do companies waste serious cash in office parties? I'm talking about those hundred-thousand-dollar extravaganzas that major tech companies seem to be addicted to.
Poll after poll finds that most employees would rather have the cash, so "kissing the collective asses of tech staff so that they won't leave" is not an explanation that holds up.
The "Roman Triumph" explanation also does not make any sense. If rich assholes want to flex their immeasurable riches, why would they invite mere mortals that do not put a lot of effort in being famous or pretty? Couldn't they invite the entire Victoria's Secret catalogue of models and the NE Patriots? Surely it would make for much more impressive photos of decadence.
The "Michael Scott" explanation also falls short. Companies spend serious cash on consultants and professional party planners, that are sure to know a lot of people. Money can and often does buy personality, so no rich asshole is ever a party dud.
Why else do they force most of their own staff into what they perceive as a "do not relax or you may get fired" loud and poorly lit meeting that takes hours to dress up for? What am I missing?5 -
I see so many freaking excel sheets where a database should have been used that I wouldn't be surprised if, at the veeeeery bottom of all things, the entire global financial system runs based on a single excel sheet made by a dude in the 90s. And since then poorly maintained.11
-
Most kids just want to code. So they see "Computer Science" and think "How to be a hacker in 6 weeks". Then they face some super simple algebra and freak out, eventually flunking out with the excuse that "uni only presents overtly theoretical shit nobody ever uses in real life".
They could hardly be more wrong, of course. Ignore calculus and complexity theory and you will max out on efficiency soon enough. Skip operating systems, compilers and language theory and you can only ever aspire to be a script kiddie.
You can't become a "data scientist" without statistics. And you can never grow to be even a mediocre one without solid basic research and physics training.
Hack, I've optimized literal millions of dollars out of cloud expenses by choosing the best processors for my stack, and weeks later got myself schooled (on devRant, of all places!) over my ignorance of their inner workings. And I have a MSc degree. Learning never stops.
So, to improve CS experience in uni? Tear down students expectations, and boil out the "I just wanna code!" kiddies to boot camps. Some of them will be back to learn the science. The rest will peak at age 33.17 -
Tech people should have a codeword. So that I don't have to explain to every data provider showing off their own crappy limited analytics tool that "I do know what SQL means and I just need the ODBC user/pass, thanks".
I wish I could just say "hey, &0x00A0 = 1337;" out loud and he would be like "oh, thanks! I needed the break. Here is the ODBC crap, I'm gonna grab some coffee."5 -
EoS1: This is the continuation of my previous rant, "The Ballad of The Six Witchers and The Undocumented Java Tool". Catch the first part here: https://devrant.com/rants/5009817/...
The Undocumented Java Tool, created by Those Who Came Before to fight the great battles of the past, is a swift beast. It reaches systems unknown and impacts many processes, unbeknownst even to said processes' masters. All from within it's lair, a foggy Windows Server swamp of moldy data streams and boggy flows.
One of The Six Witchers, the Wild One, scouted ahead to map the input and output data streams of the Unmapped Data Swamp. Accompanied only by his animal familiars, NetCat and WireShark.
Two others, bold and adventurous, raised their decompiling blades against the Undocumented Java Tool beast itself, to uncover it's data processing secrets.
Another of the witchers, of dark complexion and smooth speak, followed the data upstream to find where the fuck the limited excel sheets that feeds The Beast comes from, since it's handlers only know that "every other day a new one appears on this shared active directory location". WTF do people often have NPC-levels of unawareness about their own fucking jobs?!?!
The other witchers left to tend to the Burn-Rate Bonfire, for The Sprint is dark and full of terrors, and some bigwigs always manage to shoehorn their whims/unrelated stories into a otherwise lean sprint.
At the dawn of the new year, the witchers reconvened. "The Beast breathes a currency conversion API" - said The Wild One - "And it's claws and fangs strike mostly at two independent JIRA clusters, sometimes upserting issues. It uses a company-deprecated API to send emails. We're in deep shit."
"I've found The Source of Fucking Excel Sheets" - said the smooth witcher - "It is The Temple of Cash-Flow, where the priests weave the Tapestry of Transactions. Our Fucking Excel Sheets are but a snapshot of the latest updates on the balance of some billing accounts. I spoke with one of the priestesses, and she told me that The Oracle (DB) would be able to provide us with The Data directly, if we were to learn the way of the ODBC and the Query"
"We stroke at the beast" - said the bold and adventurous witchers, now deserving of the bragging rights to be called The Butchers of Jarfile - "It is actually fewer than twenty classes and modules. Most are API-drivers. And less than 40% of the code is ever even fucking used! We found fucking JIRA API tokens and URIs hard-coded. And it is all synchronous and monolithic - no wonder it takes almost 20 hours to run a single fucking excel sheet".
Together, the witchers figured out that each new billing account were morphed by The Beast into a new JIRA issue, if none was open yet for it. Transactions were used to update the outstanding balance on the issues regarding the billing accounts. The currency conversion API was used too often, and it's purpose was only to give a rough estimate of the total balance in each Jira issue in USD, since each issue could have transactions in several currencies. The Beast would consume the Excel sheet, do some cryptic transformations on it, and for each resulting line access the currency API and upsert a JIRA issue. The secrets of those transformations were still hidden from the witchers. When and why would The Beast send emails, was still a mistery.
As the Witchers Council approached an end and all were armed with knowledge and information, they decided on the next steps.
The Wild Witcher, known in every tavern in the land and by the sea, would create a connector to The Red Port of Redis, where every currency conversion is already updated by other processes and can be quickly retrieved inside the VPC. The Greenhorn Witcher is to follow him and build an offline process to update balances in JIRA issues.
The Butchers of Jarfile were to build The Juggler, an automation that should be able to receive a parquet file with an insertion plan and asynchronously update the JIRA API with scores of concurrent requests.
The Smooth Witcher, proud of his new lead, was to build The Oracle Watch, an order that would guard the Oracle (DB) at the Temple of Cash-Flow and report every qualifying transaction to parquet files in AWS S3. The Data would then be pushed to cross The Event Bridge into The Cluster of Sparks and Storms.
This Witcher Who Writes is to ride the Elephant of Hadoop into The Cluster of Sparks an Storms, to weave the signs of Map and Reduce and with speed and precision transform The Data into The Insertion Plan.
However, how exactly is The Data to be transformed is not yet known.
Will the Witchers be able to build The Data's New Path? Will they figure out the mysterious transformation? Will they discover the Undocumented Java Tool's secrets on notifying customers and aggregating data?
This story is still afoot. Only the future will tell, and I will keep you posted.6 -
What is gonna happen in 8/16, 2048?
(August 16th, 2048)
People are always making dumb conspiracy predictions. Let's make one, like that's the date when AI relinquishes control over the economy to leave earth or something.7 -
Profoundly cynical idea: sell NFTs of devRant rants.
This post is would be worth about the same as a pizza... restaurant.6 -
Worst dev experience of the year:
So, I went to my folks' for the holidays. My old man is a hospital practitian and is still resentful because I did not want to be a doctor too.
So he made me fix the whole fucking wiring at his place over this last couple days, because "well, you choose to be a glorified electrician!"
BTW, I have three kids and am still WFH during this time. I haven't slept in a week.6