Episode 2 of this rant: https://devrant.io/rants/851636/...

PM went to meet with the provider and came back, so I had a talk with the PM and it went like this:
Me: so how did the meeting go? Hope they corporate with you
PM: *in a sarcastic voice* Bro, you need be a good dev, they told me only a good dev can implement Master Card API calls, and its all in the docs. That's all they told me, so you should know how the meeting went
Me: *there is no wifi in jail, there is no wifi in jail, there is no wifi in jail, there is no wifi in jail* Fine bro, I'll go back to school and come back after 12 years.

Seriously when I send the provider the request and the body and all they tell me is: You have an error, Id must be unique but they fail to point out the exact error or at least send me a valid ID to test, how does this make me the bad dev and him the genius that can run Apple, Google, MS all together at once!

What is even worse, when I called the provider he told me: Man transaction ID must always be unique, how could you not know this, how is Mastercard supposed to differentiate between transactions!!

But hey, no one told me transaction id must be unique, primary keys are never unique, like DUH!

EoS1: This is the continuation of my previous rant, "The Ballad of The Six Witchers and The Undocumented Java Tool". Catch the first part here: https://devrant.com/rants/5009817/...

The Undocumented Java Tool, created by Those Who Came Before to fight the great battles of the past, is a swift beast. It reaches systems unknown and impacts many processes, unbeknownst even to said processes' masters. All from within it's lair, a foggy Windows Server swamp of moldy data streams and boggy flows.

One of The Six Witchers, the Wild One, scouted ahead to map the input and output data streams of the Unmapped Data Swamp. Accompanied only by his animal familiars, NetCat and WireShark.

Two others, bold and adventurous, raised their decompiling blades against the Undocumented Java Tool beast itself, to uncover it's data processing secrets.

Another of the witchers, of dark complexion and smooth speak, followed the data upstream to find where the fuck the limited excel sheets that feeds The Beast comes from, since it's handlers only know that "every other day a new one appears on this shared active directory location". WTF do people often have NPC-levels of unawareness about their own fucking jobs?!?!

The other witchers left to tend to the Burn-Rate Bonfire, for The Sprint is dark and full of terrors, and some bigwigs always manage to shoehorn their whims/unrelated stories into a otherwise lean sprint.

At the dawn of the new year, the witchers reconvened. "The Beast breathes a currency conversion API" - said The Wild One - "And it's claws and fangs strike mostly at two independent JIRA clusters, sometimes upserting issues. It uses a company-deprecated API to send emails. We're in deep shit."

"I've found The Source of Fucking Excel Sheets" - said the smooth witcher - "It is The Temple of Cash-Flow, where the priests weave the Tapestry of Transactions. Our Fucking Excel Sheets are but a snapshot of the latest updates on the balance of some billing accounts. I spoke with one of the priestesses, and she told me that The Oracle (DB) would be able to provide us with The Data directly, if we were to learn the way of the ODBC and the Query"

"We stroke at the beast" - said the bold and adventurous witchers, now deserving of the bragging rights to be called The Butchers of Jarfile - "It is actually fewer than twenty classes and modules. Most are API-drivers. And less than 40% of the code is ever even fucking used! We found fucking JIRA API tokens and URIs hard-coded. And it is all synchronous and monolithic - no wonder it takes almost 20 hours to run a single fucking excel sheet".

Together, the witchers figured out that each new billing account were morphed by The Beast into a new JIRA issue, if none was open yet for it. Transactions were used to update the outstanding balance on the issues regarding the billing accounts. The currency conversion API was used too often, and it's purpose was only to give a rough estimate of the total balance in each Jira issue in USD, since each issue could have transactions in several currencies. The Beast would consume the Excel sheet, do some cryptic transformations on it, and for each resulting line access the currency API and upsert a JIRA issue. The secrets of those transformations were still hidden from the witchers. When and why would The Beast send emails, was still a mistery.

As the Witchers Council approached an end and all were armed with knowledge and information, they decided on the next steps.
The Wild Witcher, known in every tavern in the land and by the sea, would create a connector to The Red Port of Redis, where every currency conversion is already updated by other processes and can be quickly retrieved inside the VPC. The Greenhorn Witcher is to follow him and build an offline process to update balances in JIRA issues.
The Butchers of Jarfile were to build The Juggler, an automation that should be able to receive a parquet file with an insertion plan and asynchronously update the JIRA API with scores of concurrent requests.
The Smooth Witcher, proud of his new lead, was to build The Oracle Watch, an order that would guard the Oracle (DB) at the Temple of Cash-Flow and report every qualifying transaction to parquet files in AWS S3. The Data would then be pushed to cross The Event Bridge into The Cluster of Sparks and Storms.
This Witcher Who Writes is to ride the Elephant of Hadoop into The Cluster of Sparks an Storms, to weave the signs of Map and Reduce and with speed and precision transform The Data into The Insertion Plan.
However, how exactly is The Data to be transformed is not yet known.

Will the Witchers be able to build The Data's New Path? Will they figure out the mysterious transformation? Will they discover the Undocumented Java Tool's secrets on notifying customers and aggregating data?

This story is still afoot. Only the future will tell, and I will keep you posted.

This is the third part of my ongoing series "The Ballad of the Six Witchers and the Undocumented Java Tool".
In this part, we have the massive Battle of Sparks and Storms.

The first part is here: https://devrant.com/rants/5009817/...

The second part is here: https://devrant.com/rants/5054467/...

Over the last couple sprints and then some, The Witcher Who Writes and the Butchers of Jarfile had studied the decompiled guts of the Undocumented Java Beast and finally derived (most of) the process by which the data was transformed. They even built a model to replicate the results in small scale.
But when such process was presented to the Priests of Accounting at the Temple of Cash-Flow, chaos ensued.
This cannot be! - cried the priests - You must be wrong!
Wrong, the Witchers were not. In every single test case the Priests of Accounting threw at the Witchers, their model predicted perfectly what would be registered by the Undocumented Java Tool at the very end.
It was not the Witchers. The process was corrupted at its essence.

The Witchers reconvened at their fortress of Sprint. In the dark room of Standup, the leader of their order, wise beyond his years (and there were plenty of those), in a deep and solemn voice, there declared:
"Guys, we must not fuck this up." (actual quote)
For the leader of the witchers had just returned from a war council at the capitol of the province. There, heading a table boarding the Archpriest of Accounting, the Augur of Economics, the Marketing Spymaster and Admiral of the Fleet, was the Ciefoh Seat himself.
They had heard rumors about the Order of the Witchers' battles and operations. They wanted to know more.

It was quiet that night in the flat and cloudy plains of Cluster of Sparks and Storms. The Ciefoh Seat had ordered the thunder to stay silent, so that the forces of whole cluster would be available for the Witchers.
The cluster had solid ground for Hive and Parquet turf, and extended from the Connection River to farther than the horizon.
The Witcher Who Writes, seated high atop his war-elephant, looked at the massive battle formations behind.
The frontline were all war-elephants of Hadoop, their mahouts the Witchers themselves.
For the right flank, the Red Port of Redis had sent their best connectors - currency conversions would happen by the hundreds, instantly and always updated.
The left flank had the first and second army of Coroutine Jugglers, trained by the Witchers. Their swift catapults would be able to move data to and from the JIRA cities. No data point will be left behind.
At the center were thousands of Sparks mounting their RDD warhorses. Organized in formations designed by the Witchers and the Priestesses of Accounting, those armoured and strong units were native to this cloudy landscape. This was their home, and they were ready to defend it.
For the enemy could be seen in the horizon.

There were terabytes of data crossing the Stony Event Bridge. Hundreds of millions of datapoints, eager to flood the memory of every system and devour the processing time of every node on sight.
For the Ciefoh Seat, in his fury about the wrong calculations of the processes of the past, had ruled that the Witchers would not simply reshape the data from now on.
The Witchers were to process the entire historical ledger of transactions. And be done before the end of the month.

The metrics rumbled under the weight of terabytes of data crossing the Event Bridge. With fire in their eyes, the war-elephants in the frontline advanced.
Hundreds of data points would be impaled by their tusks and trampled by their feet, pressed into the parquet and hive grounds. But hundreds more would take their place. There were too many data points for the Hadoop war-elephants alone.
But the dawn will come.
When the night seemed darker, the Witchers heard a thunder, and the skies turned red. The Sparks were on the move.
Riding into the parquet and hive turf, impaling scores of data points with their long SIMD lances and chopping data off with their Scala swords, the Sparks burned through the enemy like fire.
The second line of the sparks would pick data off to be sent by the Coroutine Jugglers to JIRA. That would provoke even more data to cross the Event Bridge, but the third line of Sparks were ready for it - those data would be pierced by the rounds provided by the Red Port of Redis, and sent back to JIRA - for good.

They fought for six days and six nights, taking turns so that the battles would not stop. And then, silence. The day was won, all the data crushed into hive and parquet.

Short-lived was the relief. The Witchers knew that the enemy in combat is but a shadow of the troubles that approach. Politics and greed and grudge are all next in line. Are the Witchers heroes or marauders? The aftermath is to come, and I will keep you posted.

Really glad the privacy community can make some impact. I still don't understand why venmo has to has any public transaction feed and I don't think they have public off by default still, but at least they are explaining things to users instead of being sneaky with their dumb social-network-ish-thingy that made ALL OF YOUR FINANCIAL TRANSACTIONS PUBLIC!!????

I don't understand why they hid this whole public feed thing. I'm pretty privacy focused and I didn't even know that my transactions where public untill I saw https://publicbydefault.fyi/.

Someone created a 0-followers private Twitter account and posted something to try out the new views count feature.
It raked dozens of views in a couple hours.
HOW?!?

Source: https://twitter.com/briggityboppity...

It looks like a funny data reverse-engineering exercise, so let's try and figure out what is going on.

Hypothesis 1) it is the OP's own views.
Reasonable, but unlikely if what OP says about not checking it for hours is true.

H2) It's some background job in OP's device that is refreshing OP's own latest tweets, so even without human interaction technically H1 is true. It would be some really shoddy engineering to count eye-less page views, but that's also what managers would demand.

H3) it's some internal Twitter automated function like back up, replication, indexing and word count.
See H2, it would be even dumber to count that as page views.

H4) it's some internal human reviewing for a keyword that could be associated with porn (in this case, "butts"). Really? dozens of humans to review a no-impact single post? They would have to employ hundreds of thousands of reviewers.

H5) it's some page-loading shit, like thousands of similar tweets get stored in the same index hash page and end up counting as a view in all of them every time someone loads the index page. It would be like counting every hit in the namenode as a hit in every data asset in it's Hadoop partition, or every hit in a storage block as a hit in each of it's files.
Duuuumb and kinda like H3.

H6) page views are just a fraud to scam investors. Maybe it's a "most Blockchain transactions are fake" situation, maybe it's a "views get more engagement if you don't think a lot about it" situation, maybe it's a "we don't use the metric system to count page views" situation.
All of them are very dumb.

Other hypothesis or opinions?

I just watched https://youtube.com/watch/... - towards the (very) end he's talking about how software developers rule the world... and I just realized something.

A while back, I was working on an accounting sub system for a SaaS product. We managed some of the revenue of our customers and had the accounting for that part as well. Revenue + Payments (with all the VAT / sales tax / ... that you need to have). BUT no expenses.

One day, the head accountant of a customer, angrily demanded that we immediately implement a new payment method, called commission.

You don't need to be an accounting expert for knowing, that a commission is an expense you have because somebody else marketed / sold your product / service for you. Making it a payment method is probably wrong. With a bit more knowledge you'd know that the taxes which are around expenses are completely different to revenue or payments. (btw payments didn't even have any taxes in those countries that we covered at that time at least).

So there I was standing, a software developer, trying to explain the product manager and the head accountant of our customer, that the idea is beyond stupid, and the fact that it comes from an accountant is super scary to me. (he was usually extremely picky about everything we did.)

Luckily, it was easy to convince the manager. He tried to explain it to the accountant but that person just didn't get it.

as if designing resilient distributed systems, which have 99,99% up time weren't hard enough, we also need to be experts in every domain that we have to deal with? And if there is a tiny bug and one out of 10s of thousands of transactions is screwed up, people start panicking and "loose trust in the product"? - what the hell is wrong with them?

Luckily it's a minority of customers only, but each of them is such a pain. Do you also have customers like that? who should know better, but somehow you are the expert in their domain?

"What language should I learn?" Wellll.

[0]
43 PERCENT Of banking systems are built on COBOL

80 PERCENT Of in-person transactions use COBOL

95 PERCENT Of ATM swipes rely on COBOL code

220 BILLION Lines of COBOL in use today

"Experienced COBOL programmers can earn more than $100 an hour when they get called in to patch up glitches, rewrite coding manuals or make new systems work with old." [1]

Found this pretty interesting/crazy.

Source:
[0] http://tmsnrt.rs/2nMf18G
[1] http://reuters.com/article/...

On page 19 of the ethereum yellow paper: http://gavwood.com/paper.pdf it defines the address of a wallet as: A(pr) = Β[96...255](KEC(ECDSAPUBKEY(Pr))), which is the right most 160-bits of the Keccak hash of the public key generated by the private key.

If I expose the public key to the network, do I allow people to impersonate me and sign transactions even though they don't have Tr, Ts, or Tw? My understanding is no, that without those values you can't hash the transaction.

Subsequently, this leads me to wonder if the purpose of the address (besides identifying users) to obscure the public key so that the private key is at a less risk of being reversed? Or can the public key of any address be captured?

If the public key of any address can be identified on the network or off network (without actually being the owner of that key), how do you do that? Are there any resources on how to perform this that are computationally easy?