Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "operations system"
-
WASM was a mistake. I just wanted to learn C++ and have fast code on the web. Everyone praised it. No one mentioned that it would double or quadruple my development time. That it would cause me to curse repeatedly at the screen until I wanted to harm myself.
The problem was never C++, which was a respectable if long-winded language. No no no. The problem was the lack of support for 'objects' or 'arrays' as parameters or return types. Anything of any complexity lives on one giant Float32Array which must surely bring a look of disgust from every programmer on this muddy rock. That is, one single array variable that you re-use for EVERYTHING.
Have a color? Throw it on the array. 10 floats in an object? Push it on the array - and split off the two bools via dependency injection (why do I have 3-4 line function parameter lists?!). Have an image with 1,000,000 floats? Drop it in the array. Want to return an array? Provide a malloc ptr into the code and write to it, then read from that location in JS after running the function, modifying the array as a side effect.
My- hahaha, my web worker has two images it's working with, calculations for all the planets, sun and moon in the solar system, and bunch of other calculations I wanted offloaded from the main thread... they all live in ONE GIANT ARRAY. LMFAO.If I want to find an element? I have to know exactly where to look or else, good luck finding it among the millions of numbers on that thing.
And of course, if you work with these, you put them in loops. Then you can have the joys of off-by-one errors that not only result in bad results in the returned array, but inexplicable errors in which code you haven't even touched suddenly has bad values. I've had entire functions suddenly explode with random errors because I accidentally overwrote the wrong section of that float array. Not like, the variable the function was using was wrong. No. WASM acted like the function didn't even exist and it didn't know why. Because, somehow, the function ALSO lived on that Float32Array.
And because you're using WASM to be fast, you're typically trying to overwrite things that do O(N) operations or more. NO ONE is going to use this return a + b. One off functions just aren't worth programming in WASM. Worst of all, debugging this is often a matter of writing print and console.log statements everywhere, to try and 'eat' the whole array at once to find out what portion got corrupted or is broke. Or comment out your code line by line to see what in forsaken 9 circles of coding hell caused your problem. It's like debugging blind in a strange and overgrown forest of code that you don't even recognize because most of it is there to satisfy the needs of WASM.
And because it takes so long to debug, it takes a massively long time to create things, and by the time you're done, the dependent package you're building for has 'moved on' and find you suddenly need to update a bunch of crap when you're not even finished. All of this, purely because of a horribly designed technology.
And do they have sympathy for you for forcing you to update all this stuff? No. They don't owe you sympathy, and god forbid they give you any. You are a developer and so it is your duty to suffer - for some kind of karma.
I wanted to love WASM, but screw that thing, it's horrible errors and most of all, the WASM heap32.7 -
Help.
I'm a hardware guy. If I do software, it's bare-metal (almost always). I need to fully understand my build system and tweak it exactly to my needs. I'm the sorta guy that needs memory alignment and bitwise operations on a daily basis. I'm always cautious about processor cycles, memory allocation, and power consumption. I think twice if I really need to use a float there and I consider exactly what cost the abstraction layers I build come at.
I had done some web design and development, but that was back in the day when you knew all the workarounds for IE 5-7 by heart and when people were disappointed there wasn't going to be a XHTML 2.0. I didn't build anything large until recently.
Since that time, a lot has happened. Web development has evolved in a way I didn't really fancy, to say the least. Client-side rendering for everything the server could easily do? Of course. Wasting precious energy on mobile devices because it works well enough? Naturally. Solving the simplest problems with a gigantic mess of dependencies you don't even bother to inspect? Well, how else are you going to handle all your sensitive data?
I was going to compare this to the Arduino culture of using modules you don't understand in code you don't understand. But then again, you don't see consumer products or customer-specific electronics powered by an Arduino (at least not that I'm aware of).
I'm just not fit for that shooting-drills-at-walls methodology for getting holes. I'm not against neither easy nor pretty-to-look-at solutions, but it just comes across as wasteful for me nowadays.
So, after my hiatus from web development, I've now been in a sort of internet platform project for a few months. I'm now directly confronted with all that you guys love and hate, frontend frameworks and Node for the backend and whatever. I deliberately didn't voice my opinion when the stack was chosen, because I didn't want to interfere with the modern ways and instead get some experience out of it (and I am).
And now, I'm slowly starting to feel like it was OKAY to work like this.7 -
Rant about a german problem in english
I think we as the people should just sue the german government for neglect of progress and neglect of the education system. If your not familiar with the state of german IT we have worse internetspeeds than uganda or the notoriously shitty australia, our neighbourstates look at us in disbelief while laughing in optic fibre. Our school system seperates all students after 4th grade in 3 tiers, the lowest one gives you the future perspective as a social security case. The second and highest tier require masses of useless knowledge, so called "competences"(Kompetenzen) which are totally useless skills with no real world application because they are derived from real skills, a median ground between all possible applications of that skill. And while doing that they terribly insist on doing everything the "proper" way, meaning handwritten. Most people you would expect to have basic computer literacy, meaning age 40 and below, are incapable of using basic functions of a non-smartphone computer and do not understand the slightest of what they are actually doing or supposed to do. And I mean nothing technical. Germans are the reasons they still put word as a job requirement for devjobs because this disqualifies half of our population. This leads to many people having the archaeic "we versus the machines" mentality, thinking that if they ever let the computer do parts of the job, they will then lose all of it to the machines. Thats why you never strive past basic mathematical principles in mathematics, which is a big misnomer because you never do actual mathematics, only calculating and basic calculus and statics. If you get to use your calculator, its some basic casio with no actual functionality then standard operations. And even using that is shunned upon. How is this country ever supposed to become something more than it was in the 90's, if we teach people nothing of use and kill all progress in its root.14 -
The situation right now:
Our client: full of legacy desktop solutions that always ran inside a VPN, but wanting to modernize the system and migrate to be hosted in the cloud.
Our first project with them: Frontend built with Angular, backend in a serverless model, all with GraphQL and heavily tested to assure quality. The system is mostly an internal software for management, but the backed may receive data from an App.
The problem: all management users have weak passwords (like "12345", "password", or their first name).
The solution: restrict our system to be accessible only inside the VPN
The new problem: how the mobile app will send data to our backend?
The new solution: Let's duplicate the backend, one public and the other private. The public one will accept only a few GraphQL operations.
------
This could be avoided if the passwords weren't so easily deductible12 -
math be like:
"Addition (often signified by the plus symbol "+") is one of the four basic operations of arithmetic; the others are subtraction, multiplication and division. The addition of two whole numbers is the total amount of those values combined. For example, in the adjacent picture, there is a combination of three apples and two apples together, making a total of five apples. This observation is equivalent to the mathematical expression "3 + 2 = 5" i.e., "3 add 2 is equal to 5".
Besides counting items, addition can also be defined on other types of numbers, such as integers, real numbers and complex numbers. This is part of arithmetic, a branch of mathematics. In algebra, another area of mathematics, addition can be performed on abstract objects such as vectors and matrices.
Addition has several important properties. It is commutative, meaning that order does not matter, and it is associative, meaning that when one adds more than two numbers, the order in which addition is performed does not matter (see Summation). Repeated addition of 1 is the same as counting; addition of 0 does not change a number. Addition also obeys predictable rules concerning related operations such as subtraction and multiplication.
Performing addition is one of the simplest numerical tasks. Addition of very small numbers is accessible to toddlers; the most basic task, 1 + 1, can be performed by infants as young as five months and even some members of other animal species. In primary education, students are taught to add numbers in the decimal system, starting with single digits and progressively tackling more difficult problems. Mechanical aids range from the ancient abacus to the modern computer, where research on the most efficient implementations of addition continues to this day."
And you think like .. easy, but then you turn the page:15 -
I think the worst work culture you can experience is nepotism and corruption in hierarchy. What do I mean? Well, this happened (and I think is still happening) in my last job. It was a huge logistic/delivery company. I was an intern, working as assistant developer of the only developer of the site. There was also a guy that was the technician, his assistant, a DBA and that's it.
Well, my partner and I were working on a system that managed almost all the operations of the company in this city.
Well, I supplied the dev two weeks when he was on vacation. I knew almost all the system. what happened? the manager from other city came with another Dev, and I'm not saying that I was an expert or something like that, but that dev from the other city was an incompetent. He couldn't even make a small GUI change without messing it all...
Guess what? The company paid him weekly round tickets to come and go from his city to ours (two hours of flight).
I was too disappointed I started searching another job. A week after getting my degree, I left my job and started in the one I am now. Before leaving, I asked my boss if there was a realistic chance to grow up. He answered no. To be honest, that didn't surprised me :/
The thing that makes me angry about this is that a lot of companies give chances to people that come from other cities, even if they don't know anything >:v
Oh, I almost forgot it: The last five months I was working there, they quit our office and send us to trailer-offices :/1 -
I propose that the study of Rust and therefore the application of said programming language and all of the technology that compromises it should be made because the language is actually really fucking good. Reading and studying how it manages to manipulate and otherwise use memory without a garbage collector is something to be admired, illuminating in its own accord.
BUT going for it because it is a "beTter C++" should not constitute a basis for it's study.
Let me expand through anecdotal evidence, which is really not to be taken seriously, but at the same time what I am using for my reasoning behind this, please feel free to correct me if I am wrong, for I am a software engineer yes, I do have academic training through a B.S in Computer Science yes, BUT my professional life has been solely dedicated to web development, which admittedly I do not go on about technical details of it with you all because: I am not allowed to(1) and (2)it is better for me to bitch and shit over other petty development related details.
Anecdotal and otherwise non statistically supported evidence: I have seen many motherfuckers doing shit in both C and C++ that ADMIT not covering their mistakes through the use of a debugger. Mostly because (A) using a debugger and proper IDE is for pendejos and debugging is for putos GDB is too hard and the VS IDE is waaaaaa "I onlLy NeeD Vim" and (B) "If an error would have registered then it would not have compiled no?", thus giving me the idea that the most common occurrences of issues through the use of the C father/son languages come from user error, non formal training in the language and a nice cusp of "fuck it it runs" while leaving all sorts of issues that come from manipulating the realm of the Gods "memory".
EVERY manual, book, coming all the way back to the K&C book talks about memory and the way in which developers of these 2 languages are able to manipulate and work on it. EVERY new standard of the ISO implementation of these languages deals, through community effort or standard documentation about the new items excised through features concerning MODERN (meaning, no, the shit you learned 20 years ago won't fucking cut it) will not cut it.
THUS if your ass is not constantly checking what the scalpel of electrical/circuitry/computational representation of algorithms CONDONES in what you are doing then YOU are the fucking problem.
Rust is thus no different from the original ideas of the developers behind Go when stating that their developers are not efficient enough to deal with X language, Rust protects you, because it knows that you are a fucking moron, so the compiler, advanced, and well made as it is, will give you warnings of your own idiotic tendencies, which would not have been required have you not been.....well....a fucking idiot.
Rust is a good language, but I feel one that came out from the necessity of people writing system level software as a bunch of fucking morons.
This speaks a lot more of our academic endeavors and current documentation than anything else. But to me DEALING with the idea of adapting Rust as a better C++ should come from a different point of view.
Do I agree with Linus's point of view of C++? fuck no, I do not, he is a kernel engineer, a damn good one at that regardless of what Dr. Tanenbaum believes(ed) but not everyone writes kernels, and sometimes that everyone requires OOP and additions to the language that they use. Else I would be a fucking moron for dabbling in the dictionary of languages that I use professionally.
BUT in terms of C++ being unsafe and unsecured and a horrible alternative to Rust I personaly do not believe so. I see it as a powerful white canvas, in which you are able to paint software to the best of your ability WHICH then requires thorough scrutiny from the entire team. NOT a quick replacement for something that protects your from your own stupidity BY impending the use of what are otherwise unknown "safe" features.
To be clear: I am not diminishing Rust as the powerhouse of a language that it is, myself I am quite invested in the language. But instead do not feel the reason/need before articles claiming it as the C++ killer.
I am currently heavily invested in C++ since I am trying a lot of different things for a lot of projects, and have been able to discern multiple pain points and unsafe features. Mainly the reason for this is documentation (your mother knows C++) and tooling, ide support, debugging operations, plethora of resources come from it and I have been able to push out to my secret project a lot of good dealings. WHICH I will eventually replicate with Rust to see the main differences.
Online articles stating that one will delimit or otherwise kill the other is well....wrong to me. And not the proper approach.
Anyways, I like big tits and small waists.14 -
Long time ago had a really painful client. Everyone had an attempt at training the said client. However, the client just didn't get the system at all and kept asking lots of questions.
It got to the point where one poor developer had to make a set of screencasts of him using the system and performing basic operations.
I recall how absolutely insane it was that in addition to the hours of trading time we had provided we were sending him links to video files we'd created.
It was literally face palms all around and so bad it was sort of funny.2 -
[See image]
This guy is wrong in so many ways.
"Windows/macOS is the best choice for the average user. Prove me wrong."
There are actually many Gnu/Linux based operating systems that's really easy to install and use. For example Debian/any Debian based OS.
There are avarage users that use a Gnu/Linux based operating system because guess what. They think its better and it is.
Lets do a little comparision shall we.
- - - - - Windows 10 - - Debian
Cost $139 Free
Spyware Yes. No
Freedom Limited. A lot
"[Windows] It's easy to set up, easy to use and has all the software you could possibly want. And it gets the job done. What more do you need? I don't see any reason for the average joe to use it. [Linux]"
Well as I said earlier, there are Gnu/Linux based operating systems thats easy to set up too.
And by "[Windows] has all the software you could possibly want." I guess you mean that you can download all software you could possibly want because having every single piece of software (even the ones you dont need or use) on your computer is extremely space inefficient.
"Linux is far from being mainstream, I doubt it's ever gonna happen, in fact"
Yes, Linux isn't mainstream but by the increasing number of people getting to know about Linux it eventually will be mainstream.
"[Linux is] Unusable for non-developers, non-geeks.
Depends heavily on what Gnu/Linux based operating system youre on. If youre on Ubuntu, no. If youre on Arch, yes. Just dont blame Linux for it.
"Lots of usability problems, lots of elitism, lots of deniers ("works for me", "you just don't use it right", "Just git-pull the -latest branch, recompile, mess with 12 conf files and it should work")"
That depends totally on what you're trying to. As the many in the Linux community is open source contributors, the support around open source software is huge and if you have a problem then you can get a genuine answer from someone.
"Linux is a hobby OS because you literally need to make it your 'hobby' to just to figure out how the damn thing works."
First of all, Linux isnt a OS, its a kernel. Second, no you dont. You dont have to know how it works. If you do, yes it can take a while but you dont have to.
"Linux sucks and will never break into the computer market because Linux still struggles with very basic tasks."
Ever heard of System76? What basic tasks does Linux struggle with? I call bullshit.
"It should be possible to configure pretty much everything via GUI (in the end Windows and macOS allow this) which is still not a case for some situations and operations."
Most things is possible to configure via a GUI and if it isnt, use the terminal. Its not so hard
https://boards.4chan.org/g/thread/...21 -
Since we're limiting this to things on my desk I can't do any more deep cuts out of my calculator collection, but this one is still somewhat interesting.
The HP 32S was my friend throughout university, it replaced the 15c I used before which does not live on my desk. The notable thing about the 32s is the fact it's an RPN calculator. RPN calculators are the best way to have friends never ask to borrow your calculator. The exchange will start by them asking to use it, you saying sure, and them handing it back a few minutes later without saying a word.
There's two kinds of people in this world. People who go "wtf" in an interview when asked to create a calculator program using a stack, and people who were oddballs and for whatever reason used reverse polish notation devices.
For those not familiar, rather than entering values into the calculator in "10+10" fashion, you instead provide it a compositional set of values until an operation is provided (10,10,+) at which point it executes. The why is, this type of operation allows the calculator to more naturally process operations, and eliminates the need for parenthesis which makes the operations less error prone in practice and easier to track.
The 32s had a 4 year run before being replaced by the 32SII. In the same way using a Curta will give you a significant understanding of how radix computations and floating points work. Using an HP 32s (or any of its predecessors) will do the same for algebraic functions, because you had to program them yourself using a basic label address system that also had subroutine support.
Kids who grew up with graphing calculators don't know how good you had it 😋4 -
This is the third part of my ongoing series "The Ballad of the Six Witchers and the Undocumented Java Tool".
In this part, we have the massive Battle of Sparks and Storms.
The first part is here: https://devrant.com/rants/5009817/...
The second part is here: https://devrant.com/rants/5054467/...
Over the last couple sprints and then some, The Witcher Who Writes and the Butchers of Jarfile had studied the decompiled guts of the Undocumented Java Beast and finally derived (most of) the process by which the data was transformed. They even built a model to replicate the results in small scale.
But when such process was presented to the Priests of Accounting at the Temple of Cash-Flow, chaos ensued.
This cannot be! - cried the priests - You must be wrong!
Wrong, the Witchers were not. In every single test case the Priests of Accounting threw at the Witchers, their model predicted perfectly what would be registered by the Undocumented Java Tool at the very end.
It was not the Witchers. The process was corrupted at its essence.
The Witchers reconvened at their fortress of Sprint. In the dark room of Standup, the leader of their order, wise beyond his years (and there were plenty of those), in a deep and solemn voice, there declared:
"Guys, we must not fuck this up." (actual quote)
For the leader of the witchers had just returned from a war council at the capitol of the province. There, heading a table boarding the Archpriest of Accounting, the Augur of Economics, the Marketing Spymaster and Admiral of the Fleet, was the Ciefoh Seat himself.
They had heard rumors about the Order of the Witchers' battles and operations. They wanted to know more.
It was quiet that night in the flat and cloudy plains of Cluster of Sparks and Storms. The Ciefoh Seat had ordered the thunder to stay silent, so that the forces of whole cluster would be available for the Witchers.
The cluster had solid ground for Hive and Parquet turf, and extended from the Connection River to farther than the horizon.
The Witcher Who Writes, seated high atop his war-elephant, looked at the massive battle formations behind.
The frontline were all war-elephants of Hadoop, their mahouts the Witchers themselves.
For the right flank, the Red Port of Redis had sent their best connectors - currency conversions would happen by the hundreds, instantly and always updated.
The left flank had the first and second army of Coroutine Jugglers, trained by the Witchers. Their swift catapults would be able to move data to and from the JIRA cities. No data point will be left behind.
At the center were thousands of Sparks mounting their RDD warhorses. Organized in formations designed by the Witchers and the Priestesses of Accounting, those armoured and strong units were native to this cloudy landscape. This was their home, and they were ready to defend it.
For the enemy could be seen in the horizon.
There were terabytes of data crossing the Stony Event Bridge. Hundreds of millions of datapoints, eager to flood the memory of every system and devour the processing time of every node on sight.
For the Ciefoh Seat, in his fury about the wrong calculations of the processes of the past, had ruled that the Witchers would not simply reshape the data from now on.
The Witchers were to process the entire historical ledger of transactions. And be done before the end of the month.
The metrics rumbled under the weight of terabytes of data crossing the Event Bridge. With fire in their eyes, the war-elephants in the frontline advanced.
Hundreds of data points would be impaled by their tusks and trampled by their feet, pressed into the parquet and hive grounds. But hundreds more would take their place. There were too many data points for the Hadoop war-elephants alone.
But the dawn will come.
When the night seemed darker, the Witchers heard a thunder, and the skies turned red. The Sparks were on the move.
Riding into the parquet and hive turf, impaling scores of data points with their long SIMD lances and chopping data off with their Scala swords, the Sparks burned through the enemy like fire.
The second line of the sparks would pick data off to be sent by the Coroutine Jugglers to JIRA. That would provoke even more data to cross the Event Bridge, but the third line of Sparks were ready for it - those data would be pierced by the rounds provided by the Red Port of Redis, and sent back to JIRA - for good.
They fought for six days and six nights, taking turns so that the battles would not stop. And then, silence. The day was won, all the data crushed into hive and parquet.
Short-lived was the relief. The Witchers knew that the enemy in combat is but a shadow of the troubles that approach. Politics and greed and grudge are all next in line. Are the Witchers heroes or marauders? The aftermath is to come, and I will keep you posted.4 -
python machine learning tutorials:
- import preprocessed dataset in perfect format specially crafted to match the model instead of reading from file like an actual real life would work
- use images data for recurrent neural network and see no problem
- use Conv1D for 2d input data like images
- use two letter variable names that only tutorial creator knows what they mean.
- do 10 data transformation in 1 line with no explanation of what is going on
- just enter these magic words
- okey guys thanks for watching make sure to hit that subscribe button
ehh, the machine learning ecosystem is burning pile of shit let me give you some examples:
- thanks to years of object oriented programming research and most wonderful abstractions we have "loss.backward()" which have no apparent connection to model but it affects the model, good to know
- cannot install the python packages because python must be >= 3.9 and at the same time < 3.9
- runtime error with bullshit cryptic message
- python having no data types but pytorch forces you to specify float32
- lets throw away the module name of a function with these simple tricks:
"import torch.nn.functional as F"
"import torch_geometric.transforms as T"
- tensor.detach().cpu().numpy() ???
- class NeuralNetwork(torch.nn.Module):
def __init__(self):
super(NeuralNetwork, self).__init__() ????
- lets call a function that switches on the tracking of math operations on tensors "model.train()" instead of something more indicative of the function actual effect like "model.set_mode_to_train()"
- what the fuck is ".iloc" ?
- solving environment -/- brings back memories when you could make a breakfast while the computer was turning on
- hey lets choose the slowest, most sloppy and inconsistent language ever created for high performance computing task called "data sCieNcE". but.. but. you can use numpy! I DONT GIVE A SHIT about numpy why don't you motherfuckers create a language that is inherently performant instead of calling some convoluted c++ library that requires 10s of dependencies? Why don't you create a package management system that works without me having to try random bullshit for 3 hours???
- lets set as industry standard a jupyter notebook which is not git compatible and have either 2 second latency of tab completion, no tab completion, no documentation on hover or useless documentation on hover, no way to easily redo the changes, no autosave, no error highlighting and possibility to use variable defined in a cell below in the cell above it
- lets use inconsistent variable names like "read_csv" and "isfile"
- lets pass a boolean variable as a string "true"
- lets contribute to tech enabled authoritarianism and create a face recognition and object detection models that china uses to destroy uyghur minority
- lets create a license plate computer vision system that will help government surveillance everyone, guys what a great idea
I don't want to deal with this bullshit language, bullshit ecosystem and bullshit unethical tech anymore.11 -
We want one system for all our branches with booking and (inserts 73 huge features crucial to business operations)
- okay, *sends quotation*
You know, we just want a custom made website with (insert 23 major features)
- okay, *sends quotation*
You know, we want a wordpress site, we can expand late
- ...2 -
We had a meeting with the top IT folks in the company (top manager was in the meeting). A mix between operations (customer), engineers, and IT people were at this meeting.
The IT group was not happy that my customer had asked me to do some server work. IT wanted to control that. They wanted to shut down a server without regard to the customer using this resource constantly.
IT had heartburn about the system running Ubuntu rather than Redhat Linux. The top manager during the meeting says, "Why are you running Ubuntu? That is a gaming OS." All the other IT people who worked for them just looked at each other and us with that 'WTF? Dear caught in headlights look.'
My manager who was at same level as head of IT made a few comments. We got to keep our server and IT backed down. -
my company is "pivoting" way too goddamn fast; they are pulling devs from other projects and throwing them into something that is a fragile system (was supposed to be replaced already) and is using a completely different stack than most of them usually work with. they keep promising 3rd parties that we will knock out their requirements within a week or two, and as such they are pushing haphazardly merged feature branches into production with absolutely no regression testing.
then when shit explodes and operations grinds to a halt, they tell the half of the team that actually knows how the system works to drop everything and fix it, and leave the diverted devs to continue to develop shit based on requirements drawn on a cocktail napkin, and then they fucking push both the hotfixes and the newest features at the same time.
I probably have the most tribal knowledge at this company, and they are paying me ok, so it's enough for me to just pour some rye and suck it up for the time being and milk the gig. but this can't be sustainable, right? i'm passively looking around for other work, as I've already had enough being here for over 3 years, but i'm finding most places to be slow on the application/hiring process lately due to covid.
Edit: i think i used the wrong tag here, but what the fuck ever. i haven't figured out how to tag shit properly on any platform3 -
A client's representative (was an operations manager maybe; non technical guy) was explaining his legacy project (a knowledge transfer session), mentioned about using Azure for their new system.
One of the senior .net developer in the room interrupts this guy and asks "can you explain what do you mean by azure?"
I was like "what the fuck! did I hear it wrong" -
Good evening programmers, IT's, devranters and memeians.
I would like to use a little bit of your collective conciousness - the hive mind if you will.
I've been working on my automation system for quite a while and I've received some exposure from non-programmers - which resulted in more questions than suggestions.
I would like to ask you guys to give me some suggestions as to what I could add to my system.. that is, if you have time..
The program in short (if you don't want to read the readme file) is an automation system scriptable in pure Lua.
It utilizes Selenium for web automations, NAudio for audio operations and Moonsharp as an interpreter.
While my tester friends say that they use it for the actual testing, I myself found it very useful in writting bots (for browser games for example).
Here's the github link: https://bit.ly/2GDu92g
Thanks a ton!
PS. Here's an unrelated image to draw your attention.6 -
When the CTO/CEO of your "startup" is always AFK and it takes weeks to get anything approved by them (or even secure a meeting with them) and they have almost-exclusive access to production and the admin account for all third party services.
Want to create a new messaging channel? Too bad! What about a new repository for that cool idea you had, or that new microservice you're expected to build. Expect to be blocked for at least a week.
When they also hold themselves solely responsible for security and operations, they've built their own proprietary framework that handles all the authentication, database models and microservice communications.
Speaking of which, there's more than six microservices per developer!
Oh there's a bug or limitation in the framework? Too bad. It's a black box that nobody else in the company can touch. Good luck with the two week lead time on getting anything changed there. Oh and there's no dedicated issue tracker. Have you heard of email?
When the systems and processes in place were designed for "consistency" and "scalability" in mind you can be certain that everything is consistently broken at scale. Each microservice offers:
1. Anemic & non-idempotent CRUD APIs (Can't believe it's not a Database Table™) because the consumer should do all the work.
2. Race Conditions, because transactions are "not portable" (but not to worry, all the code is written as if it were running single threaded on a single machine).
3. Fault Intolerance, just a single failure in a chain of layered microservice calls will leave the requested operation in a partially applied and corrupted state. Ger ready for manual intervention.
4. Completely Redundant Documentation, our web documentation is automatically generated and is always of the form //[FieldName] of the [ObjectName].
5. Happy Path Support, only the intended use cases and fields work, we added a bunch of others because YouAreGoingToNeedIt™ but it won't work when you do need it. The only record of this happy path is the code itself.
Consider this, you're been building a new microservice, you've carefully followed all the unwritten highly specific technical implementation standards enforced by the CTO/CEO (that your aware of). You've decided to write some unit tests, well um.. didn't you know? There's nothing scalable and consistent about running the system locally! That's not built-in to the framework. So just use curl to test your service whilst it is deployed or connected to the development environment. Then you can open a PR and once it has been approved it will be included in the next full deployment (at least a week later).
Most new 'services' feel like the are about one to five days of writing straightforward code followed by weeks to months of integration hell, testing and blocked dependencies.
When confronted/advised about these issues the response from the CTO/CEO
varies:
(A) "yes but it's an edge case, the cloud is highly available and reliable, our software doesn't crash frequently".
(B) "yes, that's why I'm thinking about adding [idempotency] to the framework to address that when I'm not so busy" two weeks go by...
(C) "yes, but we are still doing better than all of our competitors".
(D) "oh, but you can just [highly specific sequence of undocumented steps, that probably won't work when you try it].
(E) "yes, let's setup a meeting to go through this in more detail" *doesn't show up to the meeting*.
(F) "oh, but our customers are really happy with our level of [Documentation]".
Sometimes it can feel like a bit of a cult, as all of the project managers (and some of the developers) see the CTO/CEO as a sort of 'programming god' because they are never blocked on anything they work on, they're able to bypass all the limitations and obstacles they've placed in front of the 'ordinary' developers.
There's been several instances where the CTO/CEO will suddenly make widespread changes to the codebase (to enforce some 'standard') without having to go through the same review process as everybody else, these changes will usually break something like the automatic build process or something in the dev environment and its up to the developers to pick up the pieces. I think developers find it intimidating to identify issues in the CTO/CEO's code because it's implicitly defined due to their status as the "gold standard".
It's certainly frustrating but I hope this story serves as a bit of a foil to those who wish they had a more technical CTO/CEO in their organisation. Does anybody else have a similar experience or is this situation an absolute one of a kind?2 -
I just spent 2-3 hours writing a python script to make it easier to format solving system of equations through Gaussian elimination easier. It takes a matrix, then allows the user to perform row operations on it, and then it formats every operation done into LaTeX.
Alternatively, I could have spent that time actually working on the assignment it was meant to help me with, and probably finished 3/4 of it.1 -
I think another intriguing job asides programming is engineering (*for some*). A week has past and I've been on the hike assisting my beloved brother on his contracted engineering job while I am less occupied. The job is based on 🗼Tower analysis and It's quite risky as you'd have to climb up to 56 meters high just to take readings of antennas, and fix some other stuffs. The only thing I find intriguing about this job is his love for it, funny enough he also thinks I love the job too and I guess I'm guilty for his thoughts (*Sorry bro, I love the job for you not me*).
With my little experience so far on my *new brotherly job* I noticed the most hectic task isn't going up and down the tower taking readings but at the end of all operations, he'll have to gather the values and snapshots he took while on the tower to prepare reports on msword & excel for the other buttwags at the office (or home I guess)
then archive and sends via mail. Seeing this lengthy process I was forced to ask why he wasn't using any reporting tool like Jotforms or any other equivalent and I was willing to look up some recommendations for him, his reply was: "I'm already used to this form of reporting, its what I was trained with and what the company provided, nevertheless a friend of mine suggested something of such weeks back but I would have to pay monthly fee for its usage which is quite on the high side and I don't think I'd prefer that."
Sounds convincing but not enough, okay here is another deal: You use an android phone right? and at my office we work on system automation (*basically does not know what I do for a living probably thinks I'm a hacker the illegal one*), how about i design you an android app for you to capture the tower data and a PC software for you to auto generate the msword & excel reports, I can get this ready for you in less than 5 nights (*I've got less task on my desk, and was willing to take the timeout to prepare the solution that he needed, all I needed to hear for a kick start was an "Okay" just to be sure he wants it*) I suggested and re-assured but up to this point he still declined my offer and is willing to stick with his current reporting pattern (*Me died*).1 -
(Part 1/2?)
Ohhh my god am I furious and this one's a gem.
Also I'm gonna namespoil all of the entities in my post. If this is against rant rules I'll reframe it.
So the story starts over an year ago. Me, being in a bad place, where I couldn't do a job due to external issues, wanted to try out an internship. Thought I could pull off a 5 hour shift and then attend to my problems.
THE INTERNSHALA ARC:
I apply to a bunch of applications on Angel, Internshala and Indeed.
I was contacted by a few handful of these places. One of them was called "ARCHITECTA SOFTWARE SOLUTIONS". These guys had arranged an online aptitude test for me which I promptly took.
I looked up this company and they seemed like a pretty okay big firm from the outset but didn't have many reviews on Glassdoor and likes of such. (first red flag). Post aptitude test, I was quite sure I fucked up and wouldn't get further contact. Surprisingly, a person from the company sends me his Whatsapp number over chat and asks me to save it. The message is worded like a bulk email (Starting with Hello everyone!!) which I thought was quite odd since the interaction from these platforms has always been a person-to-person contact for me. Since Internshala showed that only around 40 people applied for the position I was quite intrigued but attributed this to my lack of exp in internship operations.
THE WHATSAPP ARC:
I was contacted by the number on WhatsApp saying that they'd be interested in moving forward and I gave them my work experience details.
The person sends me over a development assignment to complete within a few days. The assignment consists of massive scope of details. I'm talking production level concept and implementation. Asks to me implement a custom emotion detection CV model (worded as "emotion camera" lmao), generate a 3d model (specified nowhere and expects to implement a mono-ocular system for the curious) and deploy it over AWS with a website to go along with it and also host that. The website should contain a VR ("360 rotatable") view that can explore the depth-map ("not worded as depth-map") of the face. My first assumption was that they had picked this work up for outsourcing and didn't bother to chip off parts so as to create an assignment out of it (I know very optimistic).
So I shoot it at him on WhatsApp asking which parts of the assignment should I do?
Him: So, which parts CAN you do?
I thought of it as an HR thing.
Me: I could do most of it but given the time-frame of the assignment and my applied position as a web developer it is perhaps out of scope for my application.
Him: Don't worry about the assignment. You can submit when you complete the whole assignment.
I was visibly angry over the stupidity of this man.
Me: This task is a Full-Stack + CV + VR task. It will take over two months to get working. Am I supposed to work on it for that long for an assignment?
Him: Okay just do the basic functionalities like add to cart. But also try to do the camera thing before next week.
At this point I'm sure that they are having trouble handling an eager client and they're offloading work to interns. So I do only the backend and minimal frontend and submit the assignment (a 2 day job done over a weekend).
Nothing. Empty. No messages since then. I tried sending in a Whatsapp message on the application and how to proceed. Then, if I could get to know if I have been rejected. Nothing.
And all this time I can clearly see the account is active as it pushes pretentious motivational quotes over it's Whatsapp status.3 -
I've been noticing that without any specialization development can become repetitive, as in, a bunch of fancy ways of doing the same CURD operations.
Something that just calls me is low level development, wether it is embedded development (microcontrollers and such) or Linux Kernel and device drivers. I've been polishing my C skills for a while now and started to look into kernel development and uff, is it overwhelming!
I just wanted to see if some of you guys are or had experience in system development and how you got there. Thx!3 -
In last episode of "How SystemD screwed me over", we talked about Systemd's PrivateTMP and how it stopped me from generating SSL certificates.
In today's episode - SystemD vs CGroups!
Mister Pottering and his team apparently felt that CGroups are underused (As they can be quite difficult to set up), and so decided to integrate them into SystemD by default. As well as to provide a friendlier interface to control their values.
One can read about these interactions in the manual page "systemd.resource-control"
All is cool so far. So what happened to me today?
Imagine you did a major system release upgrade of a production server, previously tested on a standalone server. This upgrade doesn't only upgrade the distribution however, it also includes the switch from SysVInit to SystemD. Still, everything went smooth before, nothing to worry now then, right? Wrong.
The test server was never properly stress-tested. This would prove to be an issue.
When the upgrade finishes, it is 4 AM. I am happy to go to bed at last. At 6 AM, however, I am woken up again as the server's webservices are unavailable, and the machine is under 100% CPU load. Weird, I check htop and see that Apache now eats up all 32 virtual cores. So I restart it, casting it off to some weird bug or something as the load returns to normal.
2 hours later, however, the same situation occurs. This time, I scour all the logs I can, and find something weird - Many mentions that Apache couldn't create a worker thread? That's weird.
Several hours of research and tinkering later, I found out the following:
1 - By default, all processes of a system that runs SystemD are part of several CGroups. One of these CGroups is the PID CGroup, meant to stop a runaway process from exhausting all PIDs/TIDs of a system.
This limit is, by default, set to a certain amount of the total available PIDs. If a process exhausts this limit, it can no longer perform operations like fork().
So now, I know the how and why, but how should I solve this? The sanest option would be to get a rough estimate of just how many threads the Apache webserver might need. This option, though, is harder, than apparent. I cannot just take the MaxRequestsWorkers number... The instance has roughly double the amount of threads already. The cause being, as I found out, the HTTP/2 module, which spawns additional threads that do not count towards this limit. So I have no idea what limit to set.
Or I could... Disable the limit for just the webserver via the TasksAccounting switch. I thought this would work. And it did seem to... Until I ran out of TIDs again - Although systemctl status apache2.service no longer reported the number of tasks or a task limit of the process, the PID CGroup stayed set to the previous limit. Later I found out that I can only really disable the Task Accounting for all the units of a given slice and its parents.
This, though, systemctl somewhat didn't make apparent (And I skimmed the manual, that part was my fault)
So... The only remaining option I had was to... Just set the limit to infinite. And that worked, at last.
It took me several hours to debug this issue. And I once again feel like uninstalling systemd again, in favor of sysvinit.
What did I learn? RTFM, carefully, everything is important, it is not enough to read *half* the paragraph of a given configuration option...
Oh, and apache + http/2 = huge TID sink. -
!!rant life toptags bottags
My tags seem to be okay. Let's go.
I'm 14. I live in a place where nobody smart lives, and the school I go to has no coders.
Last year, all my friends moved. The only friend I had left now hates me, simply because they yelled at me everyday and I yelled at them once.
I am in the middle of my exams. I also have the flu, but thankfully it's not the e-flu, otherwise you guys should prepare for 24/7 headaches.
Due to the medications I am taking, I'm half-asleep all the time, and I probably am messing up all of my grades.
My entire extended family is in India, and I go there 2 times a year. I miss them so much right now :(.
At the same as doing exams, I am trying to keep my laptop (primary) and PC (secondary, desk) configuration and setup approximately synchronized. In order to do that, I am setting up my dotfiles repository.
Except that all my laptop config (which works) is written horribly, and I need to rewrite it all.
At the same time, I have 3 other projects going on: An OS written in D, a source-based package management system written in D, a small website (not online), and a whatever's cooking in my mind at this moment.
Right now, I'm supposed to be studying for my French exam.
Instead, I'm here, typing this out on my phone.
I have a classmate in school who can type QWERTY at 80WPM. I'm learning Dvorak (Programmer's!) and my current speed is 33WPM, after about 2 months of half-hearted practise during work time and at school.
Sometimes, I look at the world we have here, and what we're doing to it, and I wish that sometimes we could simply be content with life. Let's just live, for once.
I find ~60 random songs in one go, simply by finding a song I know on YouTube and going to the 'Mix - <song>' playlist. I download them all (youtube-dl), and I listen to them. Sometimes, I find this little part in a song (Mackelmore & Ryan Lewis - Can't Hold Us beginning instrumentals, or Safe and Sound chorus instrumentals) that make me feel so happy I feel like all's good in the world. Then the song moves on and with it, my happiness.
I look at Wayland, and X, and I think - Why can't we have one way of doing things - a fixed interface to express anything, so that one common API exists for everything of that type? And I realise it's because they feel that they're missing something from the others. Perhaps it's a bug nobody's solved or functionality that's missing, and they think that they can do better than that. And I think - Well, that's stupid. Submit a fucking bug report or pull request instead of reinventing the wheel. And then I realise that all the programming I've ever done in my life IS simply reinventing the wheel. And some might say, "Well, that guy designed it with spokes and wood. I designed it with rubber and steel," but that doesn't work, because no matter what how you make it, it's just a wheel. They both do the same thing. Both have advantages and disadvantages, because nothing's perfect. We're not perfect because we all have agendas and wants and likes and dislikes and hates and disgusts and all kinds of other crap, and our DNA's not perfect because it manages to corrupt copy operations (which is basically why we die of old age, I think).
And now I've lost my train of thought and this is too large to scroll over so I'm just going to move on to the next topic. At this point (.), I have 1633 letters left.
I hate the fact that the world's become so used to QWERTY because of stuff that happened 100 years ago that Dvorak is enough of a security to stop most people from being able to physically use my laptop.
I don't understand why huge companies like Google want to know about me. What would you do with this information? Know how to take over my stuff when the corporation-opocalypse comes around? Why can't they leave me alone? Why do I have to flash a ROM onto my phone so that Google cannot track me? What do you want, Google?
I don't give a shit any more, so there's my megarant.
Before anybody else (aside from myself) tells me that this is too big, all these topics are related simply because my train of thought went this way. There's a connection between each of these things, but I just don't know what it is.
Goodnight, world. 666 is the number of characters I have left. So is 42, for that matter (thanks, Douglas Adams!). Goodbye.rant life story current project ugh megarant why are you doing this to me life schrodinger's tags 🐈 life3 -
RANT!
Had to do one of those at-home tasks instead of a technical interview as part of applying for a (junior) positon with this startup that is using a blockchain for medical records. The task is build the api to interface with the records. Both for searching and crud operations, (Using a json array of records in local file for mock db) in 2 hours.
Ok fine, doesnt sound totally unreasonable, so I did what I could (which is all but tests, it worked at least)
But thats like 2/3 of what their actual production system is, built in 2 hours, for free. Theres 6 hr + in a work day, and the position is a 24mos contract....
Maybe its just me cause this is the first one of these Ive ever done, but it seems unreasonable that in order to qualify I need to do in 2 hrs what an entire team did in weeks.
I get they want to see if an applicant wasn't lieing on their cv, but damn...
Thats like saying In order to show your good enough for an entry level poistion on the Facebook team, you need to build Facebook; before lunchtime, its 7am. GOGOGO! lol1 -
research 10.09.2024
I successfully wrote a model verifier for xor. So now I know it is in fact working, and the thing is doing what was previously deemed impossible, calculating xor on a single hidden layer.
Also made it generalized, so I can verify it for any type of binary function.
The next step would be to see if I can either train for combinations of logical operators (or+xor, and+not, or+not, xor+and+..., etc) or chain the verifiers.
If I can it means I can train models that perform combinations of logical operations with only one hidden layer.
Also wrote a version that can sum a binary vector every time but I still have
to write a verification table for that.
If chaining verifiers or training a model to perform compound functions of multiple operations is possible, I want to see about writing models that can do neighborhood max pooling themselves in the hidden layer, or other nontrivial operations.
Lastly I need to adapt the algorithm to work with values other than binary, so that means divorcing the clamp function from the entire system. In fact I want to turn the clamp and activation into a type of bias, so a network
that can learn to do binary operations can also automatically learn to do non-binary functions as well.10 -
First let me start this rant by saying: Don't use SharePoint lists as your primary data store if you can avoid it. You're gonna have a bad time.
My coworkers and I work on a system where we need to pull tons of data down from a SharePoint site and run various algorithms and operations on it. Generate reports, that sort of thing. This is all done in the browser using a Typescript React SPFX webpart. Basically using SharePoint as a DB/DAL.
Because of the sheer amount of data we end up pulling down (our system in production is the single source of truth for one of the largest companies in Canada, and they're currently building a pipeline as we speak), in order to maintain a reasonable speed while using it, we have some pretty intense caching logic implemented, logic that ensures we get new items when new items are detected, and merges changes to already exisiting objects. It's pretty brilliant, and that's before we even consider the custom paging that my coworker implemented in order to get around the IndexedDB max size of 100MB.
Well that's all well and good, and works great in production, but it is a horror to work with. Because EVERYTHING we touch on the server is cached locally, it can be IMPOSSIBLE to detect data anomalies, be they local or server side -.- You don't know how many hours I have completely WASTED fixing a "bug" that didn't really exist... Just incorrect data in the cache12 -
A non-relationship database system with drag and drop + business process flows. It was fun at first, but like my Android phone, it slowed the **** down so much that it affected business operations. Must have been their evil plan all along.rant slow down like an ios update business process flows non-relational database management system evil plan drag and drop also my old android phone
-
Upgraded the company operations system system from AIX to Red Hat EL with near zero issues and no business interruptions. Platform and OS all in one. Been trying to repeat that with. NET 4 years into it and only 30% done with nothing but problems until we implemented CI & CD.6
-
How much time do you take usually to code a backend with simple crud operations and notification system?
It's taking me some time, but I feel I'm going too slow...7 -
Okay, so I am learning Python and I have to say it's a very interesting language but I have some questions about how the language is built under the hood as the documentation I can find by Guido doesn't give away all the secrets.
So for the question I am referencing this documentation:
https://python.org/download/...__
So, what does __new__ actually look like inside? Is there a way to see how python itself implements __new__?
I know that the mechanism for C++ malloc and new are well known definitions within that space, but I am having issues understanding exactly what the default __new__ is doing on the machine level.
The documentation I found is great for explaining how to use and override __new__ but it doesn't show what python does it once you hand off operations back to the system.
Any help is greatly appreciated!3 -
going from IT Operations to System Developer.... good progression as one knows alot about the hardware that you are compiling on.
-
If we can transform the search space or properties of a product into a graph problem
we could possibly use Kirchhoff's theorem to reveal products which are 'low complexity'
in particular search spaces, yeah?
Now according to
https://en.wikipedia.org/wiki/...
"n Cycle Space, A family of sets closed under the symmetric difference operation can be described algebraically as a vector space over the two-element finite field Z 2 {\displaystyle \mathbb {Z} _{2}} \mathbb{Z } _{2}.[4] This field has two elements, 0 and 1, and its addition and multiplication operations can be described as the familiar addition and multiplication of integers, taken modulo 2"
Wouldn't this relate to pollards algorithm, because it involves looking for factors of coprimes modulo N or am I mistaken?
Now, according to wikipedia, "in a group, the additive identity is the identity element of the group, is often denoted 0, and is unique."
If we make the multiplicative identity of our ring or field a tuple of the ratio of a/b for some product p, or a (and a/w, where w is the square root of p), or any other set such that n*m allows us to derive a or b, we could reduce the additive identity to the multiplicative identity, making the ring trivial. Solving for p would then mean finding a function from R to R, mapping every number to 0, i.e. finding the additive identity.
Now in a system with a multiplication operation the distributes over addition, the "additive
identity annihilates ring elements", so naturally, the function that maps to 0, gives us
our additive identity, we need only find the subset, no?
Forgive me if I'm wrong, but shouldn't this be convertible to a graph search?
I'm WAY out of my depth here so if anyone is familiar and can enlighten me I'd be grateful.
It's all unknown unknowns to me. -
Seeing just how well ansible works to set up Arch is kinda scary, i wrote my playbook almost a year ago and it still works perfectly (after i renamed some packages)
And this is not even how you are *supposed* to use ansible, i am kinda abusing the system by only running it locally...
This is just reinforcing my Bash-hating bias even more: bash scripts are a terrible solution to 99% of problems, and the language is frankly shit.
Back to Ansible.
The fact that operations are idempotent is *such* a game changer, too: I can just write some extra roles to automate other stuff i can never remember, like setting up those darned wine/Lutris dependencies -
#Suphle Rant 6: Deptrac, phparkitect
This entry isn't necessarily a rant but a tale of victory. I'm no more as sad as I used to be. I don't work as hard as I used to, so lesser challenges to frustrate my life. On top of that, I'm not bitter about the pace of progress. I'm at a state of contentment regarding Suphle's release
An opportunity to gain publicity presented itself last month when cfp for a php event was announced last month. I submitted and reviewed a post introducing suphle to the community. In the post, I assured readers that I won't be changing anything soon ie the apis are cast in stone. Then php 7.4 officially "went out of circulation". It hit me that even though the code supports php 8 on paper, it's kind of a red herring that decorators don't use php 8 attributes. So I doubled down, suspending documentation.
The container won't support union and intersection types cuz I dislike the ambiguity. Enums can't be hydrated. So I refactored implementation and usages of decorators from interfaces to native attributes. Tried automating typing for all class properties but psalm is using docblocks instead of native typing. So I disabled it and am doing it by hand whenever something takes me to an unfixed class (difficulty: 1). But the good news is, we are php 8 compliant as anybody can ask for!
I decided to ride that wave and implement other things that have been bothering me:
1) 2 commands for automating project setup for collaborators and user facing developers (CHECK)
2) transferring some operations from runtime to compile/build TIME (CHECK)
3) re-attempt implementing container scopes
I tried automating Deptrac usage ie adding the newly created module to the list of regulated architectural layers but their config is in yaml, so I moved to phparkitect which uses php to set the rules. I still can't find a library for programmatically updating php filed/classes but this is more dynamic for me than yaml. I set out to implement their library, turns out the entire logic is dumped into the command class, so I can neither control it without the cli or automate tests to it. I take the command apart, connect it to suphle and run. Guess what, it detects class parents as violations to the rule. Wtflyingfuck?!
As if that's not bad enough, roadrunner (that old biatch!) server setup doesn't fail if an initialization script fails. If initialization script is moved to the application code itself, server setup crumbles and takes the your initialization stuff down with it. I ping the maintainer, rustacian (god bless his soul), who informs me point blank that what I'm trying to do is not possible. Fuck it. I have to write a wrapper command for sequentially starting the server (or not starting if initialization operations don't all succeed).
Legitimate case to reinvent the wheel. I restored my deleted decorators that did dependency sanitation for me at runtime. The remaining piece of the puzzle was a recursive film iterator to feed the decorators. I checked my file system reader for clues on how to implement one and boom! The one I'd written for two other features was compatible. All I had to do was refactor decorators into dependency rules, give them fancy interfaces for customising and filtering what classes each rule should actually evaluate. In a night's work (if you're discrediting how long writing the original sanitization decorators and directory iterator), I coupled the Deptrac/phparkitect library of my dreams. This is one of the those few times I feel like a supreme deity
Hope I can eat better and get some sleep. This meme is me after getting bounced by those three library rejections -
Was working a record keeping system for the Airport for tracking departures and arrivals and some COVID-19 data
ended up realizing that the stack i had gone with wasn't gonna cut it
Had to port the whole thing to a new web framework realizing that the one i had gone with made some operations a bit complicated