Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "incremental"
-
So, some time ago, I was working for a complete puckered anus of a cosmetics company on their ecommerce product. Won't name names, but they're shitty and known for MLM. If you're clever, go you ;)
Anyways, over the course of years they brought in a competent firm to implement their service layer. I'd even worked with them in the past and it was designed to handle a frankly ridiculous-scale load. After they got the 1.0 released, the manager was replaced with some absolutely talentless, chauvinist cuntrag from a phone company that is well known for having 99% indian devs and not being able to heard now. He of course brought in his number two, worked on making life miserable and running everyone on the team off; inside of a year the entire team was ex-said-phone-company.
Watching the decay of this product was a sheer joy. They cratered the database numerous times during peak-load periods, caused $20M in redis-cluster cost overrun, ended up submitting hundreds of erroneous and duplicate orders, and mailed almost $40K worth of product to a random guy in outer mongolia who is , we can only hope, now enjoying his new life as an instagram influencer. They even terminally broke the automatic metadata, and hired THIRTY PEOPLE to sit there and do nothing but edit swagger. And it was still both wrong and unusable.
Over the course of two years, I ended up rewriting large portions of their infra surrounding the centralized service cancer to do things like, "implement security," as well as cut memory usage and runtimes down by quite literally 100x in the worst cases.
It was during this time I discovered a rather critical flaw. This is the story of what, how and how can you fucking even be that stupid. The issue relates to users and their reports and their ability to order.
I first found this issue looking at some erroneous data for a low value order and went, "There's no fucking way, they're fucking stupid, but this is borderline criminal." It was easy to miss, but someone in a top down reporting chain had submitted an order for someone else in a different org. Shouldn't be possible, but here was that order staring me in the face.
So I set to work seeing if we'd pwned ourselves as an org. I spend a few hours poring over logs from the log service and dynatrace trying to recreate what happened. I first tested to see if I could get a user, not something that was usually done because auth identity was pervasive. I discover the users are INCREMENTAL int values they used for ids in the database when requesting from the API, so naturally I have a full list of users and their title and relative position, as well as reports and descendants in about 10 minutes.
I try the happy path of setting values for random, known payment methods and org structures similar to the impossible order, and submitting as a normal user, no dice. Several more tries and I'm confident this isn't the vector.
Exhausting that option, I look at the protocol for a type of order in the system that allowed higher level people to impersonate people below them and use their own payment info for descendant report orders. I see that all of the data for this transaction is stored in a cookie. Few tests later, I discover the UI has no forgery checks, hashing, etc, and just fucking trusts whatever is present in that cookie.
An hour of tweaking later, I'm impersonating a director as a bottom rung employee. Score. So I fill a cart with a bunch of test items and proceed to checkout. There, in all its glory are the director's payment options. I select one and am presented with:
"please reenter card number to validate."
Bupkiss. Dead end.
OR SO YOU WOULD THINK.
One unimportant detail I noticed during my log investigations that the shit slinging GUI monkeys who butchered the system didn't was, on a failed attempt to submit payment in the DB, the logs were filled with messages like:
"Failed to submit order for [userid] with credit card id [id], number [FULL CREDIT CARD NUMBER]"
One submit click later and the user's credit card number drops into lnav like a gatcha prize. I dutifully rerun the checkout and got an email send notification in the logs for successful transfer to fulfillment. Order placed. Some continued experimentation later and the truth is evident:
With an authenticated user or any privilege, you could place any order, as anyone, using anyon's payment methods and have it sent anywhere.
So naturally, I pack the crucifixion-worthy body of evidence up and walk it into the IT director's office. I show him the defect, and he turns sheet fucking white. He knows there's no recovering from it, and there's no way his shitstick service team can handle fixing it. Somewhere in his tiny little grinchly manager's heart he knew they'd caused it, and he was to blame for being a shit captain to the SS Failboat. He replies quietly, "You will never speak of this to anyone, fix this discretely." Straight up hitler's bunker meme rage.13 -
- How can we make sure the age input is an integer?
- Let's add some incremental buttons for edit it.
- Nice, sounds pretty safe and convenient6 -
Holy shit. Didn't know I had to vent this out before I had revisited this shit.
Storytime!
Back in May last year, I started working on a dream project (call project X) of mine. Surprisingly it's still a novel idea and shit like this doesn't exist. Made some huge incremental changes. Added all the necessary automation pipeline stuff. Added some sick ass readme with screenshots/badges/glitz/glam.
Worked my ass of for about a month or so until I got distracted by other pending projects in need of clearances. Somewhere partway in that clearance period, I receive a mail from this "GitHub user" asking me why the development of project X had suddenly stopped.
I was a bit taken aback. Firstly because my project had ZERO stars and NO user interaction. Secondly because I hadn't encountered someone with confrontation like this since my middle-school teacher asking me for my homework.
Being the good, responsible child I am, I informed them on my situation and asked them to contribute according to the guidelines and I'd be more than happy to see this becoming a joint effort by the community.
Apparently, they were quite ecstatic to learn that my development was halted. They didn't have plans to contribute. Instead they wanted me to take down the project and stop working on it entirely.
Tough luck fucko.
Their organization had been working on something similar for longer than a couple of years. A similar open-sourced project will *apparently* ruin their market impact and I can *apparently* be sued for it.
I don't know much about open-source "laws" (and I've seen laws fuck people over) but this just seems retarded. At the moment, I'm not quite sure how to continue with the project. I'll still work on it but the fact being that I started receiving threats before stars makes me question the gatekeeping capacity of toxic market conditions (I still don't blame the person entirely. It's just really hard to keep your head above the water)
This is a one off thing but somehow it has definitely hampered my drive to work on the project (combined with the sheer amount of pending project that I've dug my grave with).
On the brighter side I've got 10 anonymous stars with zero promotion. 2 new message threads with productive insights and a person who says "I'm relying on this to work out". So not everything has gone to shit.5 -
When you live in a 3rd world country, get a relatively expensive 16mbps connection (that's still very unreliable), and try to clone WebKit… why the hell is it so large even with `--depth 1` and `--single-branch`? Why doesn't `git clone` support resuming/incremental cloning? Is this even 2018?
I want to code but life is actively fighting me right now. I hate this.
/rant26 -
I wrote a prototype for a program to do some basic data cleaning tasks in Go. The idea is to just distribute the files with the executable on our shared network to our team (since it is small enough, no github bullshit needed for this) and they can go from there.
Felt experimental, so I decided to try out F# since I have always been interested with it and for some reason Microsoft adopted it into their core net framework.
I shit you not, from 185 lines of Go code, separated into proper modules etc not to mention the additional packages I downloaded (simple things for CSV reading bla bla)
To fucking 30 lines of F# that could probably be condensed more if I knew how to do PROPER functional programming. The actual code is very much procedural with very basic functional composition, so it could probably be even less, just more "dense"
I am amazed really. I do not like that namespace pollution happens all over F# since importing System.IO gives you a bunch of shit that you wouldn't know where it is coming from unless you fuck enough with Ionide and the docs. But man.....
No need for dotnet run to test this bitch, just highlight it on the IDE, alt enter and WHAM you have the repl in front of you, incremental quasi like Lisp changes on the code can be REPL changed this way, plethora of .NET BCL wonders in it, and a single point of documentation as long as you stay in standard .net
I am amazed and in love, plus finding what I wanted to do was a fucking cakewalk.
Downside: I work in a place in which Python is seen as magic and PHP, VB.NEt and C# is the end all be all of languages. If me goes away or dies there will be no one else in this side of the state to fuck with F#
This language needs to be studied more. Shit can be so compact, but I do feel that one needs to really know enough of functional programming to be good at it. It is really not a pure language like Haskell (then again, haskell is the only "mainstream" pure functional language ain't it not?) but still, shit is really nice and I really dig what Microhard is doing in terms of the .net framework.
Will provide later findings. My entire team is on the Microsoft space, we do have Linux servers, but porting the code to generate the necessary executables for those servers if needed should be a walk in the park. I am just really intrigued by how many lines of code I was able to cut down from the Go application.
Please note that this could also mean that I am a shit Golang dev, but the cut down of nil err checkings do come somewhere.9 -
Fuck Apple right in the eye hole for trying (and probably succeeding) to normalize phone pricing at hundreds of dollars above the current market. This is going to make Android OEMs follow suit, and the world gets a little shittier for everyone.4
-
Found an old laptop in the apartment I was staying in and wrote a shitty incremental game in HTML and JavaScript with no libraries because there was no interwebs. At least alcohol is mega cheap in eastern Europe.3
-
!rant
Arduino CNC
Hey guys.
Since I mostly see frameworks to use with G-Code in Arduino CNCs I'm gonna make my own framework, where you don't need to know G-Code and the code is executed by Arduino code.
The code would include a template to define steppers steps and such.
Would include a library to work with different stepper shields.
Would this interest to anyone?
I'll provide a full example with stuff to learn for any amateur working with CNCs or that want to work with one. If you're not interested, thank you for reading, you can stop here.
Ex:
X(10);
Y(-5.5);
XY(6,7.5);
Z(-10);
This framework would only use incremental coordinates and will work for basic forms, drilling and such.
<Tutorial>
Coordinates.
Coordinates can be relative/incremental or absolute.
Lets say you have a square with 10mm, (top coordinates: (X=0,Y=0) to (X=10,Y=10).
think your drawing this square.
First line:
X0, Y0
Absolute: x10,y0
Relative: X+10
Second line:
A: x10,y10
R: Y+10
Third Line (...)
Absolute is a fixed point (coordinate)
Relative is a distance to move (not a coordinate but the distance and direction)
</Tutorial>
So, to cut a square with a TR10 (end mill with radius=5, diameter=10)
<code>
// You don't place + in positive values
// The tool always cut in the direction of the tool rotation, meaning on the left of the material.
Z(10); // Security Distance
XY(-5,0); //Compensate the diameter of the tool in radius
Z(-1); // Z=0 is the top of the block to mill, in this case. Z=0 can also be in the bottom
Y(15); //Second Point
X(15); // Third Point
Y(-15); // Forth point
X-15; // Fifth Point
(repeat)
</code>
Now we have a block with 1mm depth. If you use a while or for you can repeat the sequence for x=n passages, change the value to Z for the depth and your done.31 -
When the work isn't as interesting as I'd like, sometimes I accomplish the easiest, smallest incremental task and waste time the rest of the day.
I guess because it feels like work, I'd rather apply minimal effort. It's a bad habit, and one I'm trying to break. -
I see articles going -> "Here's the future of gaming. blah...blah...blah..."
I already know the future of gaming is trash (no matter how many VR glasses u throw at it), because the current state of gaming is a flaming pile of shit.
I'm still hurt by what Cyberpunk 2077 did to the gaming industry. They relayed the message across like -> "Hey you can release any pile of shit mid-development 'game', charge full price of $60 for it and just promise incremental updates over the years."9 -
About slightly more than a year ago I started volunteering at the local general students committee. They desperately searched for someone playing the role of both political head of division as well as the system administrator, for around half a year before I took the job.
When I started the data center was mostly abandoned with most of the computational power and resources just laying around unused. They already ran some kvm-hosts with around 6 virtual machines, including a cloud service, internally used shared storage, a user directory and also 10 workstations and a WiFi-Network. Everything except one virtual machine ran on GNU/Linux-systems and was built on open source technology. The administration was done through shared passwords, bash-scripts and instructions in an extensive MediaWiki instance.
My introduction into this whole eco-system was basically this:
"Ever did something with linux before? Here you have the logins - have fun. Oh, and please don't break stuff. Thank you!"
Since I had only managed a small personal server before and learned stuff about networking, it-sec and administration only from courses in university I quickly shaped a small team eager to build great things which would bring in the knowledge necessary to create something awesome. We had a lot of fun diving into modern technologies, discussing the future of this infrastructure and simply try out and fail hard while implementing those ideas.
Today, a year and a half later, we look at around 40 virtual machines spiced with a lot of magic. We host several internal and external services like cloud, chat, ticket-system, websites, blog, notepad, DNS, DHCP, VPN, firewall, confluence, freifunk (free network mesh), ubuntu mirror etc. Everything is managed through a central puppet-configuration infrastructure. Changes in configuration are deployed in minutes across all servers. We utilize docker for application deployment and gitlab for code management. We provide incremental, distributed backups, a central database and a distributed network across the campus. We created a desktop workstation environment based on Ubuntu Server for deployment on bare-metal machines through the foreman project. Almost everything free and open source.
The whole system now is easily configurable, allows updating, maintenance and deployment of old and new services. We reached our main goal for this year which was the creation of a documented environment which is maintainable by one administrator.
Although we did this in our free-time without any payment it was a great year with a lot of experience which pays off now. -
Anecdote from when I used to be an apprentice:
Setting: Small company, number of employees -> ε, direct superior is the founder and owner, no tech background
Boss: I've looked at this backup directory of the ERP /* why is he even going through that stuff?*/, it looks messy as hell, I want you to tidy that up
Me: Those are incremental backups, I can't just go and delete some of them, the application manages them by itself
Boss: Get it tidy!8 -
That moment when something goes wrong in the server for the first time at 3AM, your last manual backup was 5 hours ago and you stay up until 7AM to rollback and also setup incremental backups every 15 minutes... FML xD15
-
Just had the worst exam of my life today in system development at my university. This cock sucking bitch of a sensor claimed I was wrong in various assumptions about Extreme Programming. Such as: saying XP is an incremental process and not iterative. Claiming UP is more iterative than XP and that various analogies about what iterative means compared to incremental was wrong and even disrupting me while I was talking. Mind you I've been studying these subjects closely the last week and have been reading most of The Pragmatic Programmer to verify various things she disagreed upon. Result grade? In the middle of the fucking scale. Fuck this shit. I'm just glad the grade won't appear on my final graduation papers. And yes, I'm a perfectionist when it comes to this and programming, so if I'm in the wrong please correct me.1
-
I'm working on a report, i have been working on it for three months with incremental feedback meeting sessions. 'It looks great'! 'What we wanted', with detailed notes at each session. So confirmed as complete last fortnight. Today - email out of the blue - we need these major things fixed, why is it doing that, the report has to go to the printers tomorrow! Argh! It is doing that because that's how you asked it to be! And if it was so important, why are you only just looking at it now?3
-
I feel sad to say this but...
I'm hyped for the pixel 4, currently rocking my 6th pixel 2 XL (fuck these screens and camera units) and the pixel 3 was just all levels of no from the notch, incremental upgrade status and just a phone following the trends.
Looking at the leaks and official confirmations of the pixel 4 actually have me keen for photography (I do prefer a DLSR or straight up film unit but a good in the moment camera is amazing), performance and lack of notch just have me keen...
Forgive me father for I have sinned4 -
When you have to stay awake until 4:30am uploading and importing 134 .sql files split up from a larger single exported file because for some stupid reason MySQL can only handle importing the data at this insane level of incremental push.4
-
Fuck me...
Literally spent all day trying to figure why I'm getting a 500.30 error on my application. The damn fucker won't let me get THROUGH. I know it's possible to do because I did it with another application, but this one is being a little shit.
I feel like the very definition of insane right now because I've been doing little incremental changes but getting the same results.
I just want to rip the entire thing apart.7 -
What is it with people revealing their support requests like some sort of incremental escape room riddle?
Internal operations escalates an issue to development regarding an error importing a binary file format.
Confusion ahoy and blows out to 5 developers (3 senior) before the OP originally comes back 24h later to note that the client requesting this also added a note to say that the software that produces this binary may have changed formats. But they didn't think seem to think it was relevant enough to include.
Honestly unsure what measure of this is lacking basic common sense or basic human decency. And further astounding that for once the client did the right thing and this was occluded internally.1 -
I hate the elasticsearch backup api.
From beginning to end it's an painful experience.
I try to explain it, but I don't think I will be able to cover it all.
The core concept is:
- repository (storage for snapshots)
- snapshots (actual backup)
The first design flaw is that every backup in an repository is incremental. ES creates an incremental filesystem tree.
Some reasons why this is a bad idea:
- deletion of (older) backups is slow, as newer backups need to be checked for integrity
- you simply have to trust ES that it does the right thing (given the bugs it has... It seems like a very bad idea TM)
- you have no possibility of verification of snapshots
Workaround... Create many repositories as each new repository forces an full backup.........
The second thing: ES scales. Many nodes / es instances form a cluster.
Usually backup APIs incorporate these in their design. ES does not.
If an index spans 12 nodes and u use an network storage, yes: a maximum of 12 nodes will open an eg NFS connection and start backuping.
It might sound not so bad with 12 nodes and one index...
But it get's pretty bad with 100s of indexes and several dozen nodes...
And there is no real limiting in ES. You can plug a few holes, but all in all, when you don't plan carefully your backups, you'll get a pretty f*cked up network congestion.
So traffic shaping must be manually added. Yay...
The last thing is the API itself.
It's a... very fragile thing.
Especially in older ES releases, the documentation is like handing you a flex instead of toilet paper for a wipe.
Documentation != API != Reality.
Especially the fault handling left me more than once speechless...
Eg:
/_snapshot/storage/backup
gives you a state PARTIAL
/_snapshot/storage/backup/_status
gives you a state SUCCESS
Why? The first one is blocking and refers to the backup status itself. The second one shouldn't be blocking and refers to the backup operation.
And yes. The backup operation state is SUCCESS, while the backup state might be PARTIAL (hence no full backup was made, there were errors).
So we have now an additional API that we query that then wraps the API of elasticsearch. With all these shiny scary workarounds like polling, since some APIs are blocking which might lead to a gateway timeout...
Gateway timeout? Yes. Since some operations can run a LONG (multiple hours) time and you don't want to have a ton of open connections hogging resources... You let the loadbalancer kill it. Most operations simply run in ES in the background, while the connection was killed.
So much joy and fun, isn't it?
Now add the latest SMR scandal and a few faulty (as in SMR instead of CMD) hdds in a hundred terabyte ZFS pool and you'll get my frustration level.
PS: The cluster has several dozen terabyte and a lot od nodes. If you have good advice, you're welcome - but please think carefully about this fact.
I might have accidentially vaporized people sending me links with solutions that don't work on large scale TM.2 -
A new Android app I made went from 100-ish users to 0 due to a mistake in ProGuard config causing crashes upon an incremental update... I'm losing motivation.. devRant, what should I do to gain it back? :(9
-
At the place I intern, we're managing a project that's quite old (about 8 years). There's a specification document that discusses features but the mentioned stuff has been subject to incremental changes over the years so that these documents no longer fully discuss exact specifications of a particular feature. I'm curious on how you keep track of such incremental changes to a feature? Do you update the specification document with the new requirement document or is there some other way to do it?1
-
Logic of my company:
Why have an auto incremental ID for the table when you can have another table where you have to retrieve the last registered id, add 1 to it, then update the register of the last ID, then retrieve the id and finally create the register you intended.
Justification: “It’s a good practice”
FML1 -
The best part about being a developer, is building something cool, and other developers telling you how cool it is (and suggesting improvements)
-
Could it be, that we use i for incremental loops because it's an abbreviation for iterator (or increment)? 🤔 Why haven't I though of this sooner?!8
-
I played around with Git Rebase today to learn a bit about it, and it was fun, but in the process I completely obscured my process and erased a commit that has a published artifact associated with it. What remains is a few incremental preparation commits hoisted up from today's cleanup, then a pair of commits on two projects both of which only compile against the version of the other repo built from the other commit.6