AboutData Eng with a long history of abusive bosses and awesome projects. Got a MSc in Optimization and a couple startup failures under my belt.
SkillsPython, C/C++, Cloud Architecture, Spark, Parquet, AsyncIO, Sarcasm, Heuristics, Optimization, Science, Academics
Joined devRant on 10/26/2021
Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
just read about that DRAGONBRIDGE takedown.
Apparently it was some system set up to disseminate pro-China anti-USA propaganda in multiple channels.
Now, I totally can believe something like this would exist - too easy a jab to a fantastically big payout if it actually works. However, isn't it easy to call *anything* contrary to *anything* a propaganda plot?
That is why I believe in NOTHING that is on the internet. NOTHING. The internet says I have "credit card" "outstanding debt" to "pay" "ASAP".
Yeah, right. Half of those aren't even real words (I mean, "card"?! come on). You won't get me, opposing view propaganda machine!1
For the longest time, someone who used the buy&hold strategy in my weight stock would have made a nice profit. It always got up.
For the first time since my teens, I'm changing my recommendation to "short it down" :)
The plan is working :)
Update on https://devrant.com/rants/5948278/...
Most suicidal cults are healthier than any given startup.
The ones that aren't have already killed all of its members.5
Someone created a 0-followers private Twitter account and posted something to try out the new views count feature.
It raked dozens of views in a couple hours.
It looks like a funny data reverse-engineering exercise, so let's try and figure out what is going on.
Hypothesis 1) it is the OP's own views.
Reasonable, but unlikely if what OP says about not checking it for hours is true.
H2) It's some background job in OP's device that is refreshing OP's own latest tweets, so even without human interaction technically H1 is true. It would be some really shoddy engineering to count eye-less page views, but that's also what managers would demand.
H3) it's some internal Twitter automated function like back up, replication, indexing and word count.
See H2, it would be even dumber to count that as page views.
H4) it's some internal human reviewing for a keyword that could be associated with porn (in this case, "butts"). Really? dozens of humans to review a no-impact single post? They would have to employ hundreds of thousands of reviewers.
H5) it's some page-loading shit, like thousands of similar tweets get stored in the same index hash page and end up counting as a view in all of them every time someone loads the index page. It would be like counting every hit in the namenode as a hit in every data asset in it's Hadoop partition, or every hit in a storage block as a hit in each of it's files.
Duuuumb and kinda like H3.
H6) page views are just a fraud to scam investors. Maybe it's a "most Blockchain transactions are fake" situation, maybe it's a "views get more engagement if you don't think a lot about it" situation, maybe it's a "we don't use the metric system to count page views" situation.
All of them are very dumb.
Other hypothesis or opinions?10
Multi-continent low-latency auto-scaling eventually-consistent kubernetes-orchestrated and spark-powered multi-cloud data-plarform.
(Note to self: why do jargon words always come in twos?)
But seriously, the engine ELT's naval and logistical data from every continent and ocean and feeds a global analytics platform for less then 0.25 USD per ingested Gb across all systems.
And sometimes the PODs are even onboard en-route ships! Edge computing, y'all!
Tech project I'm most proud of.3
We were still using python 2.7 waaay into 2020 - It had been heralding the impending doom since 2018 and finally end-of-lifed in 2020.
That's when I finally managed to be the loudest asshole in the room and allocate a team (myself included) to refactor shit up to 3.6 (then somewhat more modern) for a month or so.
COVID the destroyer may have helped by wrecking havoc on our client's demands pipelines.
It was the third week into "the red sprint" when my entire team (myself included) were beheaded out of the company since we had "not delivered ANYTHING in weeks!" (emphasis in the original).
Frankly, being laid off was by a large margin the best thing that company ever did for me.
I heard from a poor schmuck who stayed behind that they were still using the shitty spaghetti code from before our refactoring - in freaking November 2021 - and that our entire last effort was thrown out because "nobody knows how to use it".
There is tech debt and there is tech bankruptcy.
I may have a lot of tech schadenfreude now :)13
AGILE GLUTEN-FREE DECENTRALIZED EXPERIENCES!
Now that I have said the magic words please forward your billions of VC money to this Bitcoin wallet. I promise your character will be played by Brad Pitt or George Clooney or Beyonce in the Netflix romanticized documentary.1
Sometime ago I was introduced to that game "Stardew Valley", as a way to relax and unwind since it is a dynamic-pace simple-storyline and even simpler interactivity open world.
Well, it worked like a charm (sarcasm). I have a save where I am a profit-maxinizing capitalist who tries to score a million gp in an year - so a regular gamer approach. It wasn't the goal here.
So I got a second save where I just go along, getting enough to get by and no hurry to build farm buildings and whatnot, but slowly building up NPC relationships.
Man, what a good metaphor for life. That approach actually unwinds me.
But the dev in me is just like "just, woah! that is an stellar use case for GPT+3 APIs! You could have NPCs with dynamic adaptative dialog! *And* you can monetize it (piracy-proof!) by charging for API calls! No shops, no collectibles, just a unique but scalable experience!"
What is wrong with me? I gotta change into the second-save mindset...5
"come work from the office! we now offer an on-site masseuse with quick massages!"
WTF? No. First of all, the office is in the wrong fucking continent. But let's focus on the bogus "masseuse" crap for now.
We are a 3000+ full-time employees company. If all were to come to the same overpriced pile of fancy rocks at the same 60 hours interval (spread over 5.5 days), it would mean 1.2 minutes per massage per person per week. So to let each person have a single 15 minute massage session per week, there would have to be at least 12.5 masseuses available at all working hours.
Let's say 10 masseuses to account of those people who would not have a weekly session.
Now, let's say each masseuse is a no-strings-attached independant contractor making USD 3000 per year on a 60hour, 5.5 days full-capacity work week without breaks, vacations, benefits or sickdays (European readers may faint, but it is not so uncommon in other parts of the globe).
So this shit costs 30k USD per year.
I would much rather have 10 more USD per year than this useless "look how young and hype we are!" startup horsecrap.
"it would actually be more like 2 USD per year" was the rebuttal to my (way more politely phrased) argument above. "there is the whole overhead costs thing, and, besides, we will only have one masseuse available at a time"
Oh. So besides all the marketing, the whole point is to let a person have like four sessions per year.
Office Perks. Are. Not. Real.5
EVERY FUCKING VACATIONS I'VE EVER HAD SINCE HIGH SCHOOL, when I first learned to code.
Since then there have always been some school or uni assignment or something croaked in production while I was away and n00bs couldn't hack it.1
I recently tried to apply the same data analytics rationale that I use at work to my personal life. This is not a rant, it is more like an data storytelling of an actual use case I would like some input on.
I set a goal - gotta thin up a bit and calm down my ticker - and got a (almost unreasonably expensive) field expert consultant to yell at me about it for a couple hours.
I unravel the metrics - there is like a million weight-related KPIs and most say nothing at all. I have never seen an non-infrastructure measurable subject that could not be resumed to 2-5 performance metrics. I got overall weight, how well my nine-years-old business suit fits me, heart rate, and day-after relative muscle pain (it will make sense soon).
Then its data-pipeline time. I bought a cheap weight scale and smartwatch, and every morning I input the data in an app. Yes, I try to put on the suit every morning. It still does not fit.
After establishing a baseline, I tried to fit different approaches. Doing equipment-free exercises, going to the gym, dieting. None was actually feasible in the long run, but trying different approaches does highlight the impacts and the handling profile of each method.
Looking at the now-gathered data, one thing was obvious - can't do dieting because it is not doable to have a shopping list and meals for me and another for the family.
Gym is also off the table - too much overhead. I spend more time on the trip there and back than actually there.
And home exercise equipment is either super crappy or very expensive. But it is also the most reasonable approach.
So it is solutions time. I got a nice exercise bycicle (not a peloton), an yoga mat (the wife already had that one) and an exercise program that uses only those two resources. Not as efficient without dieting, not as measurable and broad as the gym, but it fits my workflow. Deploy to production!
A few months pass and the dataset grows. The signal is subtle but has support - it works! The handling, however, needs improvement, since I cannot often enough get with the exercise program. Some mornings are just after some hard days.
I start thinking about what else I can improve in the program, but it is already pretty lean and full of compromises.
So I pull an engineer and start thinking about the support systems and draft profile. What else could be draining my willpower and morning time?
Chores. Getting the kids ready for school, firing up the moka pot, setting the off-brand roomba, folding the overnight-dried clothes, cooking breakfast, doing the dishes, cleaning the toilets. All part of my morning routine. It might benefit from some automation.
Last month I got that machine our elders call "wasteful" and "useless crap lazy entitled Americans invented because they feel oh-so-insulted for simply doing something by hand like everyone always did" - a "dish-washer".
Heh, I remember how hard was to convince my mother-in-law that an remote-controled electric garage door would not make she look like an spoiled brat.
Still to early to call, but I think that the dishwasher just saved me about 25 mins every morning. It might be enough to save willpower for me to do more exercise.
This is all so reflective of all data analytics cases really are out in the wild - the analytics phase seems so small compared to the gathering and practical problem-solving all around. And yet d.a. is what tells you that you are doing the wrong thing all along. Or on what you should work next.7
I dream of a chat bot that would read my Kanban/scrum boards and answer status queries for me.
Instead of spending 1h13mins (I measured) on average EACH AND EVERY DAY just telling people whatever is written on the story cards, the robot could tell them for me.
Maybe someone will suggest on the comments that people should just read by themselves on the story cards. Yeah, people will ask you where the toilet is while you are peeing on it. That is why we need robot nannies for professional adults.1
Not even a beach can prevent a dev from working.
Here I am, six days into vacationing. My girls are playing in the pool behind me, their mother is sunbathing and absolutely rocking a golden bikini, the sun is setting on this magnificent beach and my baby boy is even asleep besides the pool noise.
And dumbshit me is here ignoring it all and frantically typing at my phone because apparently it is VERY HARD to convince your team not to let any projects get stale while you're away.
I should go somewhere without wifi next time.11
This greek-speaking fucker (who wasn't Greek, he just spoke the language) went full bitch mode when he found out I recommended against giving him a promotion.
While screaming for an hour he called me "ornitophagos", or something that sounded like it.
I'm not quite sure that "chicken eater" is an insult, but it surely seemed like one. Maybe it was some weird attempt at hinduphobia, dunno, I just changed my recommendation to "sack the fucker".8
"Most electronics work better while turned *on*. The rest are made by HP"
Fucking printer will not only *not* print my stuff but will also make hellish noises, print the same "hello world!" page six times in a row, jam paper in the seventh and spit expensive ink on its own mechanism while there is not even paper in it.
Fucking outdated cellulose-based bureaucracy. Govt' websites are far from perfect but yet are closer to it than presenting fucking tree-meat slabs unecessary covered in colourful liquid gold.2
I wonder if crypto exchanges are so damn vulnerable or just so transparent.
I mean, it is impossible to scroll tech articles for more than a few seconds before stumbling on a report of yet another crypto exchange being nicked a couple hundred mil USD.
- It could be that their security severely sucks (wouldn't blame them for it, most businesses do suck at securing shit).
- It could be that the entire black hat community is putting it's might on stealing money that is so fucking easy to launder.
- It could be that is damn nigh impossible to cover up a crypto hack since the evidence of coins drifting away is forever on display in the public ledger, and in that case crypto companies are not hacked more often than regular companies, they are just much more often publically shamed for it.
- It could be a mix of all the above, but my intuition is that one factor is more relevant.
Which would be the most relevant factor? One of the above or yet another attack vector to the stupidest value conduit ever?4
I've been reading about quantum computing in finance and other applications (fascinating read, althought really dense), but one question now won't stop bugging me.
1) Blockchain applications are based on NP-Hard asymmetric cryptographic problems, and how hard it is to solve such problems in a really short time.
2) So called "Web3.0" is based mostly on Blockchain applications, but would still need significant advances in order to be practical.
3) Affordable and practical cloud-based quantum computing is not so far in the future, and could be used to crack most NP-Hard problems in short (polynomial) time.
Thus, my question: Is Web3.0 obsolete before it even begun?
I mean, if quantum computing takes on fast enough, it could snuff out Blockchain applications by giving those a shelf life so short it wouldn't be worth to delevolp for it. It would be like announcing the iPhone 14 and the 15 on the same breath, saying the 15 is only a quarter away - why would anyone bother with the born-obsolete tech?5
Recruiters on LinkedIn be like "Yu sim like a fit is much good at in company ours indutri tek AI is startup! Google is invest we money in! Kan we talk is good?"
Seriously, lay down the Google translate. Why even bother recruiting in a language themselves can't understand?2
"It is pointless to use just a fraction of the data in a homologation environment"
Those words reveal the truth in our creed.
We work in the deepest of back-ends to serve the front.
No data is true. Everything can be edited.
We are Data Engineers.
And for those words to take hold, a junior must execute a leap of faith, and push a hotfix into production.5
<"Perfect is enemy of good"
>"Excellent! I keep my enemies very close"
I do believe it possible that one can find at least one perfect counter to every stupid folk saying that startup-for-brains suit bags love to parrot.
<"We must fail fast"
>"I already did it!"
<"We must have a long tail of offerings"
>"Can we offer focus on our core strengths?"
<"We must focus on our core strengths"
>"Isn't our core strength 'having a long tail of offerings'?"
<"We must use agile methods"
>"An agile habit does not make an agile monk."
<"We must be flexible and adapt"
>"Is it a law or more of a rule of thumb?"
<"We must avoid bureaucracy"
>"Can I have that in writing?"3
Fuck, every time I read something about NFTs I lose another bit of faith in humanity I didn't even know I had.
I mean, I guess someone could make a bunch of cartoon characters (lets say, some sort of "pookey-man") using compatible styles and colour palettes. And I guess you could buy some of those if there is a product that makes trademark recognition a plausible dream (let's say, a videogame and/or anime).
But if the chance of success is akin to the chance of becoming a billionaire solemly from lottery wins, shouldn't the assets be priced accordingly?
Like a couple bucks a pop, at most?
Dropping a cool 30000 quid on a single lotto ticket sounds like the dumbest thing ever.
And yet at least a couple hundred hollow heads did it.
Back when I was an wee lad I interviewed for an startup, not knowing that startups are not real companies. The scumbag interviewer, who was also the owner of the outfit, asked me what I was looking in a company. I said "fair wages, a non-antagonic environment and projects with real roadmaps".
He asked me to elaborate. I said, "You know, if today your product is a sales platform, I do not want to come into work next week and discover it is now an air travel tickets marketplace, or come back the very next day and discover it is now an automated pizza factory, or in the next day and it is now a crypto exchange..."
The scumbag looked PISSED. "Sorry, but we are looking for someone who likes the challenges of a dynamic environment (read: we do not have a business model and we hate the very idea of trying to make money out of our company), and you do not fit the profile"
Startups are not real companies, i.e. they do not systematically charge money in exchange for goods or services in amounts that exceed the cost of providing said goods or services. Most startups are just tax fronts for money laundering schemes. The rest are just playthings for rich assholes who can't get a real output-producing job. Those two categories are not mutually exclusive.
Take Facebook, for example. The poster child of startups. The Zucker that owns it just announced they are setting impossible performance targets on purpose, not even attempting to hide the fact that it is just a way to lay off large quantities of employees without using the words "massive lay offs". Companies, real thin-margin, lots-of-regulation profit-driven companies do not do that. They are not some sort of "capitalist woke", real CEOs just know that if their companies largely miss performance targets on their tenure, purposely or not, next it will be their neck on the chopping block. Because they can be fired if the KPI charts say they suck. But the Zucker cannot be fired, not even after commanding their beanbag and tap beer offices to be heated exclusively by burning hundred dollar bills.
So the Zucker is not interested in performance. Not even in lay offs as expense cutting measures - investors are an infinite source of free money for startups. The Zucker just wants to project power, especially now that engineers are not so confident in the stability of they high-paying jobs.
So are irrelevant 500-souls-or-less self-aggrandizing startups. Their owners are there because it is in vogue to have a startup or ten. And will have that startup pivot to whatever sounds fancy that season. After all, only poor people care about things like EBITDA and profit margins repeatability - A.K.A. "getting more money".
REDIS: Great for cloud, will fuck up your local disk if too many write operations per second.
DynamoDB: WTF 10Mb should not be "too large for a single record"!!
SPARK: NEVER CONNECT IT TO A DATABASE! Wasted A LOT of cluster time. Also, can you be LESS specific on exactly what are the bugs in my code? 'cause I don't think it's possible.
NPM: can't install a package for shit. tried it waaaay to many times.
Makefiles: Just fuck you.
WSL1: breaks more often than a glass hammer.
Python >= 3.6: FUCK ENCODINGS!!
Jupyter: STOP MESSING UP WHILE SAVING!
Living is to collet bugs, it seems.4
There once was a bright young engineer who was hired by a company to design their new light ship.
Like 50 seconds after getting inside the company, the engineer was approached by a douchebag in a business suite.
"Hey, can you make us a mock up of the ship's design in the next hour or so? Nothing fancy, it must be very simple! To not overcomplicate it! Just a simple mock up so we can all see what are we talking about in this project! Please do not overthink this!"
The engineer, young and naive, just folded some piece of paper and gave the douchebag a paper boat.
"Fantastic! That's all we need for the presentation for the investors!"
A couple hours later the suite was back screaming.
"YOUR FUCKING FARSE! YOUR SHITTY SHIP EMBARRASED US ALL! THE VERY MOMENT OUR CEO TRIED TO STEP ON IT IT SANK! YOU ARE FIRED AND WE WILL SUE YOU FOR INCOMPETENCE! I ASKED YOU SOMETHING SIMPLE AND YOU CAME UP WITH THIS OVER ENGINEERED PIECE OF CRAP, YOU SON OF A.. [many, maaany expletives suppressed for brevity sake]"
This is how I feel everytime someone asks for "a tiny change" or some "very simple solution".
If it was so simple that it could be done in such short notice, than why the fuck do it at all, instead of buying it? I heard people sell all sorts of things in the internet nowadays. Software fucking included.5
Once upon a time, there were a restaurant called "iEat.tech.com".
It was a small single-location place, where the sufficient number of patrons could be served by the cozy number of employees.
In fact, headcount was so lean that the cook was also the one who washed all the dishes.
But then came the suits and their "VC"(daddy) money and scaled shit up.
Soon, there were so many patrons that the dishes started to pile up the sink, never washed.
"We need someone to wash the dishes!" said the cook
"Fuck you, you wash the dishes!" said the s*its
Naturally, the cook left soon after.
The s*its had a problem now. They could not replace the cook fast enough - all other cooks were either young, inexperienced and mediocre (but did clean the dishes), or refused to waste their time on the sink.
So the suits did what $*its always do - they got a fucking consultant. Who told them to get a fucking dishwashing machine and billed them the GDP of Ireland.
The s*is, of course, did not want to buy a dishwashing machine. "Our fucking process is too fucking disruptive for us to use a fucking store-bought mass-produced metal servant!" (s*its don't know what "machines" are. For them, it's all in terms of "servants", employees and machines alike).
So the s*its hired an engineer to "solve the fucking dish problem, once and for all".
The engineer quickly started measuring and drawing and calculating. The engineer was about to prepare a budget when the s*its came screaming "What the fuck are you doing? There is a fucking pile of dishes in the sink!"
The engineer replied that "I'm designing the machine!", to what the s*its responded "don't bring me fucking problems, bring me solutions!" (or some other s*it blabber)
So the engineer quickly designed an efficient dishwashing assembly line to be done in half the time most people would. And then went back to designing the machine.
But the s*its were having none of it. They kept expanding and expanding and doing what they could so that the engineer never had a moment to work on the machine. They dit it so surreptitiously that no one barely even noticed, but one day they were paying a team of engineers to be fucking human dishwashers.
Now replace "dishes" with "Jira tickets" or "quick fixes" or "tiny changes" and fix other terms accordingly.
So, for the last year or so, we've been playing with a natural language A.I.
The goal was to predict port, truck and rail service disruption due to social unrest.
The trick here is that our AI would "read between the lines" of today's news articles and spit out keywords that were likely to appear in near future articles, thus giving us an early warning before some union or army start blockading roads.
It... did not work as intended. But some very weird results came out.
Apparently, we made a robotic "kid that screams that the emperor has no clothes", yielding unlikely (but somewhat expected) keywords when fed collections of articles.
We gave it marketing content about our company. It replied "high suicide rate".10
Manager: Can you stay late as fuck today? One of our bitchiest vendors is gonna update their piece of crap and I'm pretty damn sure shit is gonna hit the fan
Dev (inner voice): no fucking way, I have kids to watch and chores to do!
Dev (outer voice): can't we just check everything in the morning?
Manager: No fucking way! If there is some fucking "challenge" when our "people" try to log onto their shit, I'm gonna look like a chump!
Let's talk silvers, I will sign on that bloody commie bullshit for your hours tonight.
Dev (outer voice): Fine. Until how late?
Dev (inner voice): Wait, I was supposed to do it without getting overtime bonus?5
The fucker was just saying he would not help me, but in the end I should have taken the advice literally and got the fuck out of that fucked up place.
Can someone please write a plugin that automatically translates bug tickets into startup-ese? It would make it very easy to present the most common offenders to higher management on quarterly reviews.
$< "screen A is all messed up"
$> "We are disrupting the establishment in screen A"
$< "API B is not responding, giving timeout errors"
$> "We are facing logistical challenges in API B"4
Fuck Apache TIKA.
Its supposed to be a "universal file reader" or some shit. Im trying to use it as a PDF/image parser that does OCR when needed and yelds a full-file string. It does so, but the text ends up being IN THE WRONG FUCKING ORDER.
WTF would I want to parse the text out of a PDF in any order that is not the one the text is supposed to be read?!?!
"It is more efficient to work in random ordering", says the docs. No shit, really? Wouldn't it be even more efficient to just spit out random strings? Just as useful and 100% CPU-bound.
"You can add a property to forcefully put the text in the right order". THEN WHY THE FUCK IT IS NOT THE DEFAULT SETTING?
Srsly, what's the use case to a parser that yields scrambled text?!?1