Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "data architecture"
-
Got assigned an intern to mentor him, with an explicit order not to do any of the legwork for him.
We start out with some fuzzy requirements. Intern starts overengineering a generic solution, so I make out a best architecture that conforms to the business requirements and I explain it to the intern why are we going to use such approach and tell him how we are going to do it in three phases.
I explain the intern the first phase, break it down in small tasks for him and return to my projects...
After a couple of days of no words from the intern, I decide to check up on him to see how is he progressing, only to hear him complaining the task is boring. So, instead of doing the assigned tasks, he decided he should do a "design" for a feature I told him explicitly not to do, since it is going to be designed by the design team later on.
I explain it to the intern that we have to do the boring task first because we can't proceed with the next phase of the implementation without the necessary data from the phase one.
Intern says okay and assures me he got it now. Few days later, I check up on him, and he tells me he feels he is doing all the work and that I don't contribute to the project. I call up my boss and tell him intern wants a meeting. Since I was working from home, I quickly pack my things and head to the office. Boss talks to the intern before I managed to get to the office. Once I got there, I meet the intern, and he tells me everything is okay. I ask what did the boss say to make things okay all of a sudden, and he tells me he said we are a team now. Our company has a flat hierarchy model, so he tells me he doesn't feel he needs a mentor, that we are both equal, and that I have no idea how to work in a team, and then proceeds to comfort me on how human interaction is hard and that I will learn it one day... I was like wtf?
I tell him to finish the phase one of the project and start with the phase two, and I leave home again.
I call up my boss and ask him what did he say to the intern, and he says: "nothing much, just explained the project a little bit and how it fits in the grand scheme of things.". I ask about the equal team members thing, and me not being a mentor any longer, the boss goes wtf, saying he never said anything about that to him.
So the kid can't focus on a single task, over-engineers everything and doesn't feel he can learn anything from developers with more experience, doesn't want to obey commands, and also likes to lie to manipulate others.
Tomorrow we'll decide what to do with him...
Sorry for the long rant, it was a long stressful day.86 -
Hi, I am a Javascript apprentice. Can you help me with my project?
- Sure! What do you need?
Oh, it’s very simple, I just want to make a static webpage that shows a clock with the real time.
- Wait, why static? Why not dynamic?
I don’t know, I guess it’ll be easier.
- Well, maybe, but that’s boring, and if that’s boring you are not going to put in time, and if you’re not going to put in time, it’s going to be harder; so it’s better to start with something harder in order to make it easier.
You know that doesn’t make sense right?
- When you learn Javascript you’ll get it.
Okay, so I want to parse this date first to make the clock be universal for all the regions.
- You’re not going to do that by yourself right? You know what they say, don’t repeat yourself!
But it’s just two lines.
- Don’t reinvent the wheel!
Literally, Javascript has a built in library for t...
- One component per file!
I’m lost.
- It happens, and you’ll get lost managing your files as well. You should use Webpack or Browserify for managing your modules.
Doesn’t Javascript include that already?
- Yes, but some people still have previous versions of ECMAScript, so it wouldn’t be compatible.
What’s ECMAScript?
- Javascript
Why is it called ECMAScript then?
- It’s called both ways. Anyways, after you install Webpack to manage your modules, you still need a module and dependency manager, such as bower, or node package manager or yarn.
What does that have to do with my page?
- So you can install AngularJS.
What’s AngularJS?
- A Javascript framework that allows you to do complex stuff easily, such as two way data binding!
Oh, that’s great, so if I modify one sentence on a part of the page, it will automatically refresh the other part of the page which is related to the first one and viceversa?
- Exactly! Except two way data binding is not recommended, since you don’t want child components to edit the parent components of your app.
Then why make two way data binding in the first place?
- It’s backed up by Google. You just don’t get it do you?
I have installed AngularJS now, but it seems I have to redefine something called a... directive?
- AngularJS is old now, you should start using Angular, aka Angular 2.
But it’s the same name... wtf! Only 3 minutes have passed since we started talking, how are they in Angular 2 already?
- You mean 3.
2.
- 3.
4?
- 5.
6?
- Exactly.
Okay, I now know Angular 6.0, and use a component based architecture using only a one way data binding, I have read and started using the Design Patterns already described to solve my problem without reinventing the wheel using libraries such as lodash and D3 for a world map visualization of my clock as well as moment to parse the dates correctly. I also used ECMAScript 6 with Babel to secure backwards compatibility.
- That’s good.
Really?
- Yes, except you didn’t concatenate your html into templates that can be under a super Javascript file which can, then, be concatenated along all your Javascript files and finally be minimized in order to reduce latency. And automate all that process using Gulp while testing every single unit of your code using Jasmine or protractor or just the Angular built in unit tester.
I did.
- But did you use TypeScript?37 -
Good Morning!, its time for practiseSafeHex's most incompetent co-worker!
Todays contestant is a very special one.
*sitcom audience: WHY?*
Glad you asked, you see if you were to look at his linkedin profile, you would see a job title unlike any you've seen before.
*sitcom audience oooooooohhhhhh*
were not talking software developer, engineer, tech lead, designer, CTO, CEO or anything like that, No No our new entrant "G" surpasses all of those with the title ..... "Software extraordinaire".
*sitcom audience laughs hysterically*
I KNOW!, wtf does that even mean! as a previous dev-ranter pointed out does this mean he IS quality code? I'd say he's more like a trash can ... where his code belongs
*ba dum tsssss*
Ok ok, lets get on with the show, heres some reasons why "G" is on the show:
One of G's tasks was to build an analytics gathering library for iOS, similar to google analytics where you track pages and events (we couldn't use google's). G was SO good at this job he implemented 2 features we didn't even ask for:
- If the library was unable to load its config file (for any reason) it would throw an uncatchable system integrity error, crashing the app.
- If anything was passed into any of the functions that wasn't expected (null, empty array etc.) it would crash the app as it was "more efficient" to not do any sanity checks inside the library.
This caused a lot of issues as some of the data needed to come from the clients server. The day we launched the app, within the first 3 hours we had over 40k crash logs and a VERY angry client.
Now, what makes this story important is not the bugs themselves, come on how many times have we all done something stupid? No the issue here was G defended all of this as the right thing to do!
.. and no he wasn't stoned or drunk!
G claimed if he couldn't get the right settings / params he wouldn't be able to track the event and then our CEO wouldn't have our usage data. To which I replied:
"So your solution was to not give the client an app instead? ... which also doesn't give the CEO his data".
He got very angry and asked me "what would you do then?". I offered a solution something like why not have a default tag for "error" or "unknown" where if theres an issue, we send up whatever we have, plus the file name and store it somewhere else. I was told I was being ridiculous as it wasn't built to track anything like that and that would never work ... his solution? ... pull the library out of the app and forget it.
... once again giving everyone no data.
G later moved onto another cross-platform style project. Backend team were particularly unhappy as they got no spec of what needed to be done. All they knew was it was a single endpoint dealing with very complex model. There was no Java classes, super classes, abstract classes or even interfaces, just this huge chunk of mocked data. So myself and the lead sat down with him, and asked where the interfaces for the backend where, or designs / architecture for them etc.
His response, to this day frightens me ... not makes me angry, not bewilders me ... scares the living shit out of me that people like this exist in the world and have successful careers.
G: "hhhmmm, I know how to build an interface, but i've never understood them ... Like lets say I have an interface, what now? how does that help me in any way? I can't physically use it, does it not just use up time building it for no reason?"
us: "... ... how are the backend team suppose to understand the model, its types, integrate it into the other systems?"
G: "Can I not just tell them and they can write it down?"
**
I'll just pause here for a moment, as you'll likely need to read that again out of sheer disbelief
**
I've never seen someone die inside the way the lead did. He started a syllable and his face just dropped, eyes glazed over and he instantly lost all the will to live. He replied:
" wel ............... it doesn't matter ... its not important ... I have to go, good luck with the project"
*killed the screen share and left the room*
now I know you are all dying in suspense to know what happened to that project, I can drop the shocking bombshell that it was in fact cancelled. Thankfully only ~350 man hours were spent on it
... yep, not a typo.
G's crowning achievement however will go down in history. VERY long story short, backend got deployed to the server and EVERYTHING broke. Lead investigated, found mistakes and config issues on every second line, load balancer wasn't even starting up. When asked had this been tested before it was deployed:
G: "Yeah I tested it on my machine, it worked fine"
lead: "... and on the server?"
G: "no, my machine will do the same thing"
lead: "do you have a load balancer and multiple VM's?"
G: "no, but Java is Java"
... and with that its time to end todays episode. Will G be our most incompetent? ... maybe.
Tune in later for more practiceSafeHex's most incompetent co-worker!!!31 -
So, you start with a PHP website.
Nah, no hating on PHP here, this is not about language design or performance or strict type systems...
This is about architecture.
No backend web framework, just "plain PHP".
Well, I can deal with that. As long as there is some consistency, I wouldn't even mind maintaining a PHP4 site with Y2K-era HTML4 and zero Javascript.
That sounds like fucking paradise to me right now. 😍
But no, of course it was updated to PHP7, using Laravel, and a main.js file was created. GREAT.... right? Yes. Sure. Totally cool. Gotta stay with the times. But there's still remnants of that ancient framework-less website underneath. So we enter an era of Laravel + Blade templates, with a little sprinkle of raw imported PHP files here and there.
Fine. Ancient PHP + Laravel + Blade + main.js + bootstrap.css. Whatever. I can still handle this. 🤨
But then the Frontend hipsters swoosh back their shawls, sip from their caramel lattes, and start whining: "We want React! We want SPA! No more BootstrapCSS, we're going to launch our own suite of SASS styles! IT'S BETTER".
OK, so we create REST endpoints, and the little monkeys who spend their time animating spinners to cover up all the XHR fuckups are satisfied. But they only care about the top most visited pages, so we ALSO need to keep our Blade templated HTML. We now have about 200 SPA/REST routes, and about 350 classic PHP/Blade pages.
So we enter the Era of Ancient PHP + Laravel + Blade + main.js + bootstrap.css + hipster.sass + REST + React + SPA 😑
Now the Backend grizzlies wake from their hibernation, growling: We have nearly 25 million lines of PHP! Monoliths are evil! Did you know Netflix uses microservices? If we break everything into tiny chunks of code, all our problems will be solved! Let's use DDD! Let's use messaging pipelines! Let's use caching! Let's use big data! Let's use search indexes!... Good right? Sure. Whatever.
OK, so we enter the Era of Ancient PHP + Laravel + Blade + main.js + bootstrap.css + hipster.sass + REST + React + SPA + Redis + RabbitMQ + Cassandra + Elastic 😫
Our monolith starts pooping out little microservices. Some polished pieces turn into pretty little gems... but the obese monolith keeps swelling as well, while simultaneously pooping out more and more little ugly turds at an ever faster rate.
Management rushes in: "Forget about frontend and microservices! We need a desktop app! We need mobile apps! I read in a magazine that the era of the web is over!"
OK, so we enter the Era of Ancient PHP + Laravel + Blade + main.js + bootstrap.css + hipster.sass + REST + GraphQL + React + SPA + Redis + RabbitMQ + Google pub/sub + Neo4J + Cassandra + Elastic + UWP + Android + iOS 😠
"Do you have a monolith or microservices" -- "Yes"
"Which database do you use" -- "Yes"
"Which API standard do you follow" -- "Yes"
"Do you use a CI/building service?" -- "Yes, 3"
"Which Laravel version do you use?" -- "Nine" -- "What, Laravel 9, that isn't even out yet?" -- "No, nine different versions, depends on the services"
"Besides PHP, do you use any Python, Ruby, NodeJS, C#, Golang, or Java?" -- "Not OR, AND. So that's a yes. And bash. Oh and Perl. Oh... and a bit of LUA I think?"
2% of pages are still served by raw, framework-less PHP.32 -
So I have been temporarily assigned to new team .. moving from mainly backend.. to help the Web team ..
Me .: Aight guys .. what we working with ?
Team: MVC .net
Me: awesome ..
Team: but we have our own version of MVC .
Me : 🤔 your own MVC ?
Team: yeh we only buse controllers.. but no models at all ?
Me.: 😲 So where does the view gets its data from ?
Team : from Azure functions apps.
M: how ?
T: ( in very proud tone ) .. we use js to call all functions.
M: so why not just use HTML pages . Why MVC then !
T: coz MVC is modern architecture design.
M: but you not using it and all of calles to the functions are exposed publicly.
T: 🧐 THIS IS MODERN DESIGN !!
M: 🤪 My bad .. what the hell do I know ! I only been developing MVC applications for 7 years !!
Please tell me more about your " Modern Design "
🤮🤮🤮25 -
"full stack" means "you'll be doing everything from gathering client requirements through data architecture up to the UI design and of course implementing all of it"
"backend" means "you'll be coding everything from database through server-side code and client-side code including html and css"
"we need you on-site all day every day" means "we have no idea how and why we should use repositories with remote access despite being a company developing an internet app, and we don't trust that you would be working anyway"
"interesting challenging projects" means "the same boring crap as every other company, running on an incredibly botched and dezorganized codebase".
"competitive pay" means "actual pay is around 1.5 times the minimum allowed pay, and everything else is being siphoned off into (stupid and useless) 'benefits' like massage and fitness discount coupons"
"friendly collective having fun at numerous company events each years" means "it is mandatory for you to participate on our weekend drinking retreats but you'll only find out when we fire you because you're 'not a team player' after you refused to participate on those"9 -
Me: Oh I see were using a non-standard architecture on this app. I like this bit but what is this doing? never seen it before.
Him: Ah we use that to abstract the navigation layer.
Me: oh ok, interesting idea, but that means we need an extra file per screen + 1 per module. We also can't use this inbuilt control, which I really like, and we've to write a tonne of code to avoid that.
Him: Yeah we wanted to take a new approach to fix X, this is what we came up with. Were not 100% happy with it. Do you have any ideas?
**
Queue really long, multi-day architecture discussion. Lots of interesting points, neither side being precious or childish in anyway. Was honestly fantastic.
**
Me: So after researching your last email a bit, I think I found a happy middle ground. If we turn X into a singleton, we can store the state its generating inside itself. We can go back to using the in-built navigation control and have the data being fetched like Y. If you want to keep your dependency injection stuff, we can copy the Angular services approach and inject the singletons instead of all of these things. That means we can delete the entire layer Z.
Even with the app only having 25% of the screens, we could delete like 30+ files, and still have the architecture, at a high level, identical and textbook MVVM.
Him: singleton? no I don't like those, best off keeping it the way it is.
... are you fucking kidding me? You've reinvented probably 3 wheels, doubled the code in the app and forced us to take ownership of something the system handles ... but a singleton is a bad idea? ... based off no concrete evidence or facts, but a personal opinion.
... your face is a bad idea15 -
Smart India Hackathon: Horrible experience
Background:- Our task was to do load forecasting for a given area. Hourly energy consumption data for past 5 years was given to us.
One government official asks the following questions:-
1. Why are you using deep learning for the project? Why are you not doing data analysis?
2. Which neural network "algorithm" you are using? He wanted to ask which model we are using, but he didn't have a single clue about Neural Networks.
3. Why are you using libraries? Why not your own code?
Here comes the biggest one,
4. Why haven't you developed your own "algorithm" (again, he meant model)? All you have done is used sone library. Where is "novelty" in your project?
I just want to say that if you don't know anything about ML/AI, then don't comment anything about it. And worst thing was, he was not ready to accept the fact that for capturing temporal dependencies where underlying probability distribution ia unknown, deep learning performs much better than traditional data analysis techniques.
After hearing his first question, second one was not a surprise for us. We were expecting something like that. For a few moments, we were speechless. Then one of us started by showing neural network architecture. But after some time, he rudely repeated the same question, "where is the algorithm". We told him every fucking thing used in the project, ranging from RMSprop optimizer to Backpropagation through time algorithm to mean squared loss error function.
Then very calmly, he asked third question, why are you using libraries? That moron wanted us to write a whole fucking optimized library. We were speechless at this question. Finally, one of us told him the "obvious" answer. We were completely demotivated. But it didnt end here. The real question was waiting. At the end, after listening to all of us, he dropped the final bomb, WHY HAVE YOU USED A NEURAL NETWORK "ALGORITHM" WHICH HAS ALREADY BEEN IMPLEMENTED? WHY DIDN'T YOU MAKE YOU OWN "ALGORITHM"? We again stated the obvious answer that it takes atleast an year or two of continuous hardwork to develop a state of art algorithm, that too when gou build it on top of some existing "algorithm". After listening to this, he left. His final response was "Try to make a new "algorithm"".
Needless to say, we were completely demotivated after this evaluation. We all had worked too hard for this. And we had ability to explain each and every part of the project intuitively and mathematically, but he was not even ready to listen.
Now, all of us are sitting aimlessly, waiting for Hackathon to end.😢😢😢😢😢25 -
!rant
So this year I had a subject at university called "Linux internal architecture", and for the last assignment I had to write a kernel module and interact with it with a separate program written in C.
Once I had finished and tested the driver, I went on to write the other program, which was supposed to use system calls to read and write data to the module. While debugging this program (~500 lines of code) I reached the level of frustration where you just start printing absurd messages everywhere in your code to see what's wrong. So for example instead of printing "This error happened in this function", my error messages were more like "Fuck this fucking function it doesn't fucking work".
Guess who forgot to delete all those messages before sending the code to the teacher...
Also, if a specific mode is selected, the program enters a while(1) that, apart from doing what it's expected to do, also creates a file in the user's home directory called something like 'motherfucker' and appends the words 'fuck this shit' to it. INFINITELY.
I really really hope this teacher doesn't try to run the program in his own computer, or he's in for a big surprise.8 -
Short version:
Dear devRantdairy,
today I was stupid.
The End.
Full version:
I am working on some messaging system, trying to use less as possible overhead sending data. Therefore there of course are asynchronous calls and some templating. But that's just the setting of the rant: I designed an architecture to save conversations in a database. Working with transactions in pdo I wrote a query wich in my eyes should have worked well. But the result just didn't appear in the table. So I started debugging data. Recreated the table. Rewrote the query. Went to bed. Woke up. Further tryed to make this work. And in the end I realized I just forgot to commit the transaction.
How dumb can you be? There's way too much time gone for that mistake. Is there a hole? I want do dig myself.9 -
One step through the door my wife whips around, a look so disgusted she barely seems human. "What's that smell?" she cries. "It's you! You smell like...like bad code!"
Indeed, I am covered with the scent of the forbidden love child of a man who read half a chapter on if-then statements and then pushed out into the world, earthworm-like, a mangled misshapened gelatinous mass that my employer gave the title of line-of-business application purely out of pity.
For more days than I'd like to count I have been porting a ColdFusion 5 application to .NET. Initially written in 2000 and last touched in 2006, it has a data architecture comparable to Dresden after the second world war. It features a table solely comprised of seven columns of IDs so that joins can be made between other tables lacking a common key. Columns that should be contained within a single table spread out among multiple tables. Single columns containing data that should be multiple columns (with handy flags to separate the subsets). A view with 14 joins that playfully displays unintended results. And so much more spread out over almost 200 stored procedures, views, triggers, and tables on the SQL server, and dozens of additional ADO-like SQL statements within the ColdFusion itself. Fortunately, the application overcomes these issues by having absolutely no data validation while allowing nulls pretty much everywhere.
When I am done this will be a very nice ASP.NET MVC app with at least 150 less stored procs, views, and tables. Auto-generated duplicate entries will be a thing of the past. Pop-up windows that inexplicably refresh the underlying screen to display a different part of the program than the one the user wants will be eliminated. And a UI based on the colors of a Rubik's Cube with usability that Mr. Rubik would find challenging will disappear with only the trauma of using it left behind.
Sadly, this is not my worse legacy code experience. Just the most recent. Just the most recent stench added to a lifetime of bathing in code rot.3 -
!rant
We just did a massive update to our prod db environment that would implicate damn near all system in our servers....on a friday.
Luckily for us, our DB is a badass rockstar mfking hero that was planning this shit for a little over a year with the assistance of yours truly as backup following the man's lead...and even then I didn't do SHIT
My boy did great, tested everything and the switch was effortless, fast (considering that it went on during working hours) and painless.
I salute my mfking dude, if i make my own company I am stealing this mfker. Homie speaks in SQL, homie was prolly there when SQL was invented and was already speaking in sql before shit was even set in spec, homie can take a glance at a huge db and already cast his opinion before looking at the design and architecture, homie was Data Science before data science was a thing.
Homie is my man crush on the number one spot putting mfking henry cavill on second place.
Homie wakes up and pisses greatness.
Homie is the man. Hope yall have the same mfking homie as I do5 -
Our Service Oriented Architecture team is writing very next-level things, such as JSON services that pass data like this:
<JSON>
<Data>
...
</Data>
</JSON>23 -
A lot of engineering fads go in circle.
Architecture in the 80s: Mainframe and clients.
Architecture in the 90s: Software systems connected by an ESB.
Architecture in the 2000s: Big central service and everyone connects to it for everything
Architecture in the 2010s: Decentralized microservices that communicate with queues.
Current: RabbitMQ and Kafka.
... Can't we just go back to the 90s?
I hate fads.
I hate when I have to get some data, and it's scattered on 20 different servers, and to load a fucking account page, a convoluted network of 40 apps have to be activated, some in PHP, others in JS, others on Java, that are developed by different teams, connected to different tiny ass DBs, all on huge clusters of tiny ass virtual machines that get 30% load at peak hours, 90% of which comes from serializing and parsing messages. 40 people maintaining this nightmare, that could've been just 7 people making a small monolithic system that easily handles this workload on a 4-core server with 32GB of RAM.
Tripple it, put it behind a load balancer, proper DB replication (use fucking CockroachDB if you really want survivability), and you've got zero downtime at a fraction of the cost.
Just because something's cool now, doesn't mean that everybody has to blindly follow it for fucks sake!
Same rant goes for functional vs OOP and all that crap. Going blindly with any of these is just a stupid fad, and the main reason why companies need refactoring of legacy code.12 -
The IT head of my Client's company : You need to explain me what exactly you are doing in the backend and how the IOT devices are connected to the server. And the security protocol too.
Me : But it's already there in the design documents.
IT Head : I know, but I need more details as I need to give a presentation.
Me : (That's the point! You want me to be your teacher!) Okay. I will try.
IT Head : You have to.
Me : (Fuck you) Well, there are four separate servers - cache, db, socket and web. Each of the servers can be configured in a distributed way. You can put some load balancers and connect multiple servers of the same type to a particular load balancer. The database and cache servers need to replicated. The socket and http servers will subscribe to the cache server's updates. The IOT devices will be connected to the socket server via SSL and will publish the updates to a particular topic. The socket server will update the cache server and the http servers which are subscribed to that channel will receive the update notification. Then http server will forward the data to the web portals via web socket. The websockets will also work on SSL to provide security. The cache server also updates the database after a fixed interval.
This is how it works.
IT Head : Can you please give the presentation?
Me : (Fuck you asshole! Now die thinking about this architecture) Nope. I am really busy.11 -
So, I was gonna rant about how it can be difficult to design event-based Microservices.
I was gonna say some shit about gateways APIs and some other stuff about data aggregation and keeping things idempotent.
I was going to do all this but then as I was stretching out the old ranting fingers I decided to draw a diagram to maybe go along with the rant.
Now I’m not here to really rant about all that Jazz...
I’m here to give you all a first class opportunity to tear apart my architecture!
A few things to note:
Using a gateway API (Kong) to separate the mobile from the desktop.
This traffic is directed through to an in intermediate API. This way the same microservices can provide different data, and even functionality for each device.
Most Microservices currently built in golang.
All services are event based, and all data is built on-the-fly by events generated and handled by each Microservices.
RabbitMQ used as a message broker.
And finally, it is hosted in Google Cloud Platform.
The currently hosted form is built with Microservices but this will be the update version of things.
So, feel free to rip it apart or add anything you think should change.
Also, feel free to tell me to fuck right off if that’s your cup of tea as well.
Peace ✌🏼19 -
!rant but history
I found this old micro controller: The TMS 1000 (from 1974). The specs: 100-400kHz clock speed, 4-bit architecture, 1kB ROM and 32 bytes (!) RAM. According to data sheet, you sent the program to TI and they gave you a programmed controller back - updates to the once upload program were impossible, but an external memory chip was possible.
I'm glad we have computers with more processing power and storage (and other languages than assembler) - on the other hand it enforced good debugging before deployment and and efficient code.
Data sheet: http://bitsavers.org/components/ti/...6 -
One thing I've noticed about devRant is the ratio of web dev/mobile dev posts to database/architecture/big data dev posts. There's A LOT of you web peeps out there, and not enough data dudes, which I guess justifies my constant demand, salary and lack of competition. Just an observation.9
-
At work, an idiot who has never worked on machine learning before and understands nothing about it: "You know what, machine learning isn't actually hard. It is just basic statistics and then you download the model online and that's it! There's nothing else you are doing!"
STFU, you moron! Do you think just any model can work for your use case? Do you fucking think it is easy to come up with new architecture for a very specific use case and test it accurately? Do you think it's easy to effectively train a model and do hyper-parameter optimization?Do you fucking think it is easy to retrieve the right data for your use case? Do you think it's easy to keep up with research papers on arXiv being released daily? Do you think this is fucking javascript and there's a framework for everything? Stfu!
Honestly, i hate ignorant morons who generalize stuff they don't clearly understand.5 -
Imagine yourself being a CTO back then.
Brand new Acura NSX. No MacBooks, ThinkPads are hot. Your company has its own skyscraper. CASE tools are just introduced and they’re hotter than blockchain now. You do software architecture in IBM Rational Rose, typing on your Model M and thinking really hard about Java OOP which is very hot right now.
You have Erlang servers at your own data center. You laugh at people writing in COBOL. You excited about aspect-oriented programming.
What a wonderful time.3 -
Best code performance incr. I made?
Many, many years ago our scaling strategy was to throw hardware at performance problems. Hardware consisted of dedicated web server and backing SQL server box, so each site instance had two servers (and data replication processes in place)
Two servers turned into 4, 4 to 8, 8 to around 16 (don't remember exactly what we ended up with). With Window's server and SQL Server licenses getting into the hundreds of thousands of dollars, the 'powers-that-be' were becoming very concerned with our IT budget. With our IT-VP and other web mgrs being hardware-centric, they simply shrugged and told the company that's just the way it is.
Taking it upon myself, started looking into utilizing web services, caching data (Microsoft's Velocity at the time), and a service that returned product data, the bottleneck for most of the performance issues. Description, price, simple stuff. Testing the scaling with our dev environment, single web server and single backing sql server, the service was able to handle 10x the traffic with much better performance.
Since the majority of the IT mgmt were hardware centric, they blew off the results saying my tests were contrived and my solution wouldn't work in 'the real world'. Not 100% wrong, I had no idea what would happen when real traffic would hit the site.
With our other hardware guys concerned the web hardware budget was tearing into everything else, they helped convince the 'powers-that-be' to give my idea a shot.
Fast forward a couple of months (lots of web code changes), early one morning we started slowly turning on the new framework (3 load balanced web service servers, 3 web servers, one sql server). 5 minutes...no issues, 10 minutes...no issues,an hour...everything is looking great. Then (A is a network admin)...
A: "Umm...guys...hardly any of the other web servers are being hit. The new servers are handling almost 100% of the traffic."
VP: "That can't be right. Something must be wrong with the load balancers. Rollback!"
A:"No, everything is fine. Load balancer is working and the performance spikes are coming from the old servers, not the new ones. Wow!, this is awesome!"
<Web manager 'Stacey'>
Stacey: "We probably still need to rollback. We'll need to do a full analysis to why the performance improved and apply it the current hardware setup."
A: "Page load times are now under 100 milliseconds from almost 3 seconds. Lets not rollback and see what happens."
Stacey:"I don't know, customers aren't used to such fast load times. They'll think something is wrong and go to a competitor. Rollback."
VP: "Agreed. We don't why this so fast. We'll need to replicate what is going on to the current architecture. Good try guys."
<later that day>
VP: "We've received hundreds of emails complementing us on the web site performance this morning and upset that the site suddenly slowed down again. CEO got wind of these emails and instructed us to move forward with the new framework."
After full implementation, we were able to scale back to only a few web servers and a single sql server, saving an initial $300,000 and a potential future savings of over $500,000. Budget analysis considering other factors, over the next 7 years, this would save the company over a million dollars.
At the semi-annual company wide meeting, our VP made a speech.
VP: "I'd like to thank everyone for this hard fought journey to get our web site up to industry standards for the benefit of our customers and stakeholders. Most of all, I'd like to thank Stacey for all her effort in designing and implementation of the scaling solution. Great job Stacy!"
<hands her a blank white envelope, hmmm...wonder what was in it?>
A few devs who sat in front of me turn around, network guys to the right, all look at me with puzzled looks with one mouth-ing "WTF?"9 -
How to get investors wet:
“My latest project utilizes the microservices architecture and is a mobile first, artificially intelligent blockchain making use of quantum computing, serverless architecture and uses coding and algorithms with big data. also devOps, continuous integration, IoT, Cybersecurity and Virtual Reality”
Doesn’t even need to make sense11 -
I started working for a company something around 1-2 months ago, they said because I don't have any experience with their stack, my salary will be lower than other team members. I said there is no problem and started my work. My first task was refactoring codes that their experienced programmers have wrote. My second task was extracting data layer from views. (They use Laravel and MVC architecture and they get data directly in views, not controllers). So, by end of the month when I talked with my boss I said I should get more money because I was better than your experienced programmers. He refused my request so I said I will not work with your team anymore :)
Anyway, never accept a job if you know you deserve more money than what they say will give you.
P.S: Sorry for my bad English. English is not my native language5 -
TL;DR: Stop using React for EVERYTHING. It's not the end-all solution to every application need.
My team is staffed about 50/50 with tenured devs, and junior devs who have never written a full application and don't understand the specific benefits of different libraries/framworks. As a result, most of these junior devs have jumped on the React train, and they're under the impression that React is the end-all answer to any possible application need. Doesn't matter what type of app is, what kind of data is going to be flowing through the app, data scale, etc. In their eyes, React is always the answer. Now, while I'm not a big fan of React myself, I will say that it does its job when its tasked with a data-heavy application that needs to be refreshed/re-rendered dynamically and frequently (like Facebook.) However, my main gripe is that some people insist on using it for EVERYTHING. They refuse to acknowledge that there can be better library/framework choices (Angular, Vue, or even straight jQuery,) and they refuse to learn any other frameworks. You can hit them with countless technical reasons as to why React isn't a good choice for a particular application, and they'll just spout off the same tidbits from the "ReactJS Makes My Nips Hard 101" handbook: "React is the future," "Component-based web architecture is the future," (I'm not arguing with that last one) "But...JSX bro.," "Facebook and Netflix use it, so that's how you know it's amazing." They'll use React for a simple app, and make it overly-complex, and take months to write something that should have taken them a week. For example, we have one dev who has never used any other frameworks/libraries apart from React, and he used React (via create-react-app) to write what is effectively a single form and a content widget inside of a bootstrap template. It took him 4 MONTHS to write this, and it still isn't fully functioning. The search functionality doesn't really work (in fact, it's just array filtering,) and wont return any results if you search for the first word in an entry. His repo is a mess, filled with a bunch of useless files that were bootstrap'd in via create-react-app. We've built apps like this in a week in the past using different libraries/frameworks, and he could have done the same if he didn't overly-complicate the project by insisting on using React. If your app is essentially a dynamic form, you don’t need a freaking virtual DOM.
This happens every time a big new framework hits the scene. New young developers get sucked into it, because it's the cool hip new framework (or in React's case, library.) and they use it for everything, even when it's not the best choice. It happened with Angular, Rails, and now it's happening with React.
React has its benefits, but please please please consider which library/framework is the best choice from a technical standpoint before immediately jumping on the React train because "Facebook uses it bro."2 -
So ok here it is, as asked in the comments.
Setting: customer (huge electronics chain) wants a huge migration from custom software to SAP erp, hybris commere for b2b and ... azure cloud
Timeframe: ~10 months….
My colleague and me had the glorious task to make the evaluation result of the B2B approval process (like you can only buy up till € 1000, then someone has to approve) available in the cart view, not just the end of the checkout. Well I though, easy, we have the results, just put them in the cart … hmm :-\
The whole thing is that the the storefront - called accelerator (although it should rather be called decelerator) is a 10-year old (looking) buggy interface, that promises to the customers, that it solves all their problems and just needs some minor customization. Fact is, it’s an abomination, which makes us spend 2 months in every project to „ripp it apart“ and fix/repair/rebuild major functionality (which changes every 6 months because of „updates“.
After a week of reading the scarce (aka non-existing) docs and decompiling and debugging hybris code, we found out (besides dozends of bugs) that this is not going to be easy. The domain model is fucked up - both CartModel and OrderModel extend AbstractOrderModel. Though we only need functionality that is in the AbstractOrderModel, the hybris guys decided (for an unknown reason) to use OrderModel in every single fucking method (about 30 nested calls ….). So what shall we do, we don’t have an order yet, only a cart. Fuck lets fake an order, push it through use the results and dismiss the order … good idea!? BAD IDEA (don’t ask …). So after a week or two we changed our strategy: create duplicate interface for nearly all (spring) services with changed method signatures that override the hybris beans and allow to use CartModels (which is possible, because within the super methods, they actually „cast" it to AbstractOrderModel *facepalm*).
After about 2 months (2 people full time) we have a working „prototype“. It works with the default-sample-accelerator data. Unfortunately the customer wanted to have it’s own dateset in the system (what a shock). Well you guess it … everything collapsed. The way the customer wanted to "have it working“ was just incompatible with the way hybris wants it (yeah yeah SAP, hybris is sooo customizable …). Well we basically had to rewrite everything again.
Just in case your wondering … the requirements were clear in the beginning (stick to the standard! [configuration/functinonality]). Well, then the customer found out that this is shit … and well …
So some months later, next big thing. I was appointed technical sublead (is that a word)/sub pm for the topics‚delivery service‘ (cart, delivery time calculation, u name it) and customerregistration - a reward for my great work with the b2b approval process???
Customer's office: 20+ people, mostly SAP related, a few c# guys, and drumrole .... the main (external) overall superhero ‚im the greates and ur shit‘ architect.
Aberage age 45+, me - the ‚hybris guy’ (he really just called me that all the time), age 32.
He powerpoints his „ tables" and other weird out of this world stuff on the wall, talks and talks. Everyone is in awe (or fear?). Everything he says is just bullshit and I see it in the eyes of the others. Finally the hybris guy interrups him, as he explains the overall architecture (which is just wrong) and points out how it should be (according to my docs which very more up to date. From now on he didn't just "not like" me anymore. (good first day)
I remember the looks of the other guys - they were releaved that someone pointed that out - saved the weeks of useless work ...
Instead of talking the customer's tongue he just spoke gibberish SAP … arg (common in SAP land as I had to learn the hard way).
Outcome of about (useless) 5 meetings later: we are going to blow out data from informatica to sap to azure to datahub to hybris ... hmpf needless to say its fucking super slow.
But who cares, I‘ll get my own rest endpoint that‘ll do all I need.
First try: error 500, 2. try: 20 seconds later, error message in html, content type json, a few days later the c# guy manages to deliver a kinda working still slow service, only the results are wrong, customer blames the hybris team, hmm we r just using their fucking results ...
The sap guys (customer service) just don't seem to be able to activate/configure the OOTB odata service, so I was told)
Several email rounds, meetings later, about 2 months, still no working hybris integration (all my emails with detailed checklists for every participent and deadlines were unanswered/ignored or answered with unrelated stuff). Customer pissed at us (god knows why, I tried, I really did!). So I decide to fly up there to handle it all by myself16 -
What you are expected to learn in 3 years:
power electronics,
analogue signal,
digital signal processing,
VDHL development,
VLSI debelopment,
antenna design,
optical communication,
networking,
digital storage,
electromagnetic,
ARM ISA,
x86 ISA,
signal and control system,
robotics,
computer vision,
NLP, data algorithm,
Java, C++, Python,
javascript frameworks,
ASP.NET web development,
cloud computing,
computer security ,
Information coding,
ethical hacking,
statistics,
machine learning,
data mining,
data analysis,
cloud computing,
Matlab,
Android app development,
IOS app development,
Computer architecture,
Computer network,
discrete structure,
3D game development,
operating system,
introduction to DevOps,
how-to -fix- computer,
system administration,
Project of being entrepreneur,
and 24 random unrelated subjects of your choices
This is a major called "computer engineering"4 -
I've kinda ghosted DevRant so here's an update:
VueJS is pretty good and I'm happy using it, but it seems I need to start with React soon to gain more business partnerships :( I'm down to learn React, but I'd rather jump into Typescript or stick with Vue.
Webpack is cool and I like it more than my previous Gulp implementation.
Docker has become much more usable in the last 2 years, but it's still garbage on Windows/Mac when running an application that runs on Symfony...without docker-sync. File interactions are just too slow for some of my enterprise apps. docker-sync was a life-saver.
I wish I had swapped ALL links to XHR requests long ago. This pseudo-SPA architecture that I've got now (still server-side rendered) is pretty good. It allows my server to do what servers do best, while eliminating the overhead of reloading CSS/JS on every request. I wrote an ES6 component for this: https://github.com/HTMLGuyLLC/... - Frankly, I could give a shit if you think it's dumb or hate it or think I'm dumb, but I'd love to hear any ideas for improving it (it's open source for a reason). I've been told my script is super helpful for people who have Shopify sites and can't change the backend. I use it to modernize older apps.
ContentBuilder.js has improved a ton in the last year and they're having a sale that ends today if you have a need for something like that, take a look: https://innovastudio.com/content-bu...
I bought and returned a 2019 Macbook pro with i9. I'll stick with my 2015 until we see what's in store for 2020. Apple has really stopped making great products ever since Jobs died, and I can't imagine that he was THAT important to the company. Any idiot on the street can you tell you several ways they could improve the latest models...for instance, how about feedback when you click buttons in the touchbar? How about a skinnier trackpad so your wrists aren't constantly on it? How about always-available audio and brightness buttons? How about better ports...How about a bezel-less screen? How about better arrow keys so you can easily click the up arrow without hitting shift all the time? How about a keyboard that doesn't suck? I did love touch ID though, and the laptop was much lighter.
The Logitech MX Master 3 mouse was just released. I love my 2s, so I just ordered it. We'll see how it is!
PHPStorm still hasn't fixed a couple things that are bothering me with the terminal: can't reorder tabs with drag and drop, tabs are saved but don't reconnect to the server so the title is wrong if you reopen a project and forget that the terminal tabs are from your last session and no longer connected. I've accidentally tried to run scripts locally that were meant for the server more than once...
I just found out this exists: https://caniuse.email/
I'm going to be looking into Kubernetes soon. I keep seeing the name (docker for mac, digitalocean) so I'm curious.
AWS S3 Glacier is still a bitch to work with in 2019...wtf? Having to setup a Python script with a bunch of dependencies in order to remove all items in a vault before you can delete it is dumb. It's like they said "how can we make it difficult for people to remove shit so we can keep charging them forever?". I finally removed almost 2TB of data, but my computer had to run that script for a day....so dumb...6 -
I once agreed to maintain and develop an application used in a different section of the school to keep inventory and make sure everything is where it is supposed to be.
At first there was enthusiasm, together with 2 of my classmates we agreed and git clone-d the .NET application that now graduated students built and maintained for the past few years. What could go wrong right?!
It became clear that the original students that worked on it followed an older curriculum, meaning they still got taught .NET instead of the core variant that we get now, not only that but it also seemed that they either did not fully grasp the Clean/Onion architecture or didn't get it in class since there were infrastructure components in the 'Domain' project of the solution. Think of 2 DBContexts in the domain model, yep.
One of us bailed in the first week, the other one and I felt bad for the people using the app so we went on and tried to work on the first bugs that were described in a document. One of these bugs was 'whenever I filter on something in the list, everybody gets to see that filter on their screen instead of only me'. Woah that's weird! Let's see how they put that together!
Oh god, they are using a _static_ variable to store filters, no wonder that it doesn't work properly. Ever heard of sessions?!
Second bug: Sometimes people can't create an account when we sign them up from the admin panel. Alright that is weird, let's figure that one out! Wait a second it seems to work in development? What's this about.
Oh wait I can't create an account on production either? Oh that's weird, wait a second... Why do I have to put my e-mail in a form that was sent to me through e-mail? Why is my address not filled in already? OOH, if someone types in the wrong e-mail address (which is easy since our school has 4 variants of the same f*cking e-mail address) it won't work since it can't recognize the user! Brilliant! Remove e-mail input box and make a token/queryparam determine the user account.
Ah that seems good, it's a mess but it seems a tiny bit better now, great! We're making progress and some sweet buck.
Next bug, trillions of 50x errors on random pages, that's a weird one.
Hm everything works in development, that's odd. Is the production data corrupted?
DID I MENTION that in order to get into the system in development we have to load in a f*cking production database backup ON OUR DEVELOPMENT MACHINE and then ask one of the users' password to login to it and create an account for ourselves? Seeding? What's that, right?!
Anyway, back to bug fixing. I e-mail the the people responsible for the app and get a production admin account, oh I also can't ssh into it because of policies so I have to do everything over e-mail and figure out what's causing the errors. I somehow also wonder if they have any kind of virtualization in place, giving students a VM to do that stuff in doesn't seem so weird does it ? Even with school policies?
Oh btw, 'deploying' means sending a .zip file to a guy in another building and telling him how to configure it, apparently this resulted in a missing folder that the application needed to work and couldn't make on its own. This after 2 weeks of e-mailing back and forth.
After 3 months i quit out of despair and sadness, and due to the fact that I just couldn't do it anymore. I separated everything into logical subprojects and let the last guy handle it, he was OK with that and understood why I left.
Luckily, around that time I already had an actual job at a software development company :)3 -
Years ago, we were setting up an architecture where we fetch certain data as-is and throw it in CosmosDb. Then we run a daily background job to aggregate and store it as structured data.
The problem is the volume. The calculation step is so intense that it will bring down the host machine, and the insert step will bring down the database in a manner where it takes 30 min or more to become accessible again.
Accommodating for this would need a fundamental change in our setup. Maybe rewriting the queries, data structure, containerizing it for auto scaling, whatever. Back then, this wasn't on the table due to time constraints and, nobody wanted to be the person to open that Pandora's box of turning things upside down when it "basically works".
So the hotfix was to do a 1 second threadsleep for every iteration where needed. It makes the job take upwards of 12 hours where - if the system could endure it - it would normally take a couple minutes.
The solution has grown around this behavior ever since, making it even harder to properly fix now. Whenever there is a new team member there is this ritual of explaining this behavior to them, then discussing solutions until they realize how big of a change it would be, and concluding that it needs to be done, but...
not right now.2 -
That’s it I’m done with writing documents like Software Product Specifications and Software Requirements Documents and Software Architecture Documents, manuals, data sheets and more in MS word..
I’m doing it all form this point forward in LaTeX... I can stay in my editor, it works beautifully with version control because it’s just text... I can split it amung multiple files.. it looks damn sexy. I can focus on the content rather than being distracted by formatting and spelling issues and the rest of that shit.. ALSO.. it doesn’t crash or get corrupted.. well at-least I’ve never had a text editor crash or corrupt my files.
Idk why I didn’t learn latex sooner and do the switch.6 -
Few years ago as a junior android dev with couple years of self taught experience of working in startups I submitted a simple android app assignment for a junior android dev role. Assignment had only like 8 requirements so I followed them to the letter. That didn't end well.
App was simple just 3 screens. Login screen with username and password input fields, login button.
Had to call a login endpoint after login button was clicked, redirecting to home screen, calling items endpoint, displaying a list of items and when an item was clicked passing item data and redirect to item details screen.
Needless to say big swinging dick senior was not impressed. UI was not perfect, I forgot to display a loading animation when fetching data, didnt handle back button properly.
I agreed with some points but other comments were clearly just nitpicking: his preferred variable naming conventions, his opinions on architecture that was not up to his standard (official google arch at the time was not up to his standard).
He also was mad that app wasn't prepared for release to googleplay (another out of the ass requirement). Like I would prepare a 3 screen app for prod release that he will forget ever existed after 20min of his review.
Lots more of nitpicking, encapsulation this encapsulation that, omg now hes shocked that there are a few warnings after the project is built.
Regardless my self confidence was destroyed at that point and after few more negative experiences I dropped android dev alltogether for a couple years and switched to game dev.
After game dev ran its course I went back to android dev and found a supportive place where I could grow.
Looking back, they were actually hiring atleast a mid level for a junior position but I was grilled as a senior. The guy literally didnt wrote any single positive thing in that review about my code even tho my senior peers said my project was decent back then, its just that I didnt handle a few edge cases and that's all.
I looked up the guy in linkedin, turns out hes a uni dropout who posts all books that he red about software dev in his education section of his linkedin profile. Found a bunch of other narcissistic stuff on his profile. Guy was a fucking idiot. Even if I worked under him it would have probably sucked.
Learned some important lessons I guess. Always get a second, 3rd and 4th opinion and dont take criticism too seriously. Always check what kind of person is providing feedback.4 -
I would like to rant one more time about my internship.
I began in July, the first. That's my sister who helped me to find this internship and I was a little scared about how bad it could be.
I came at the office, my boss told me that I would work in an "Innovation lab", an apartment where people works on projects that are less corporate than the enterprise's ones.
To me, it was amazing. So I came in this apartment, it was like a dream. I didn't know that I would have such luck to be in this environment : kitchen, sofas, beds, many decorations for all political ideologies, ideas. There was some decorations that were about weed and many cool things for the young guy I am.
The lab's leader told me that it was a very free environment and all the awesome stuff I could use.
Then they showed me where I would work.
We were two interns employed as web developers. We had a complete room for us.
Then we began to work there, and I was presented to my internship tutor.
He gave me some instructions but told me that I had a week before the project begin.
Here began the troubles.
We waited a complete week without having any instructions. Then we began to build something in PHP with our knowledge and the informations someone from the lab gave us.
When finally we had news from the project, two weeks later, we learned that the project would be built with ASP. NET.
Here we go, I learn ASP. NET alone. I have many problems and nobody helps (even if the problem comes from enterprise's API/Framework). I finally make something usable with no help, after I discovered that my mate wasn't developer at all and just took an option for her classes which forced her to get an internship.
She had 3 month left, I had 6.
Then when the project really began, nobody came to verify what I was doing and on a meeting, they said that I was doing nothing.
The boss even became mad on us because he couldn't see what we were doing (we're back end developers).
I asked for help to the developers of the enterprise and someone came, sad to have to help an internship, and learned some tricks but nothing else.
To have a concrete explanation of what DDD was, I had to ask 4 times for help.
Finally I had something that could receive data from the connected hives we are working on and store them into a database in the architecture of the enterprise.
Then, they wanted me to try an API for them. I tried, and it wasn't working at all. So they make me still wait to change my whole architecture when the API will be released.
Recently, I was told that I would never do the front-end of the project (which was an horror because of the fantasm of the lab leader). Then they realized that my late wasn't a programmer. So they asked me to make a prototype for the front-end. I did for a presentation.
Then they didn't tell me the device they would use for the presentation and it was an iPhone 7. Idk why, safari couldn't display what IE can.
They blamed me for having done a bad work. It wasn't my job. I did it to help because they can't find a fucking front-end developer with a little more experience than me.
Actually, I am an alone developer since my mate is gone and the lab leader don't want me to show up because she considers me as a shame.
I asked to be moved back in the office of the enterprise, they agreed and said it was a 2-weeks delay. It's the Thursday of the second week and I have no news. I send mails to my tutor, even SMS, he doesn't answer me. They didn't call me to give me my pay with a week late. And the person who is responsible doesn't answer me neither. I came to see her, but she wasn't available. I'm now alone in a desk, waiting the time to pass.
Fucking this shit.
I'm in France.
EDIT : I forgot to say that I can't use the sofas or bed because I'm allergic to cats and there were 3 cats. Now there is still one and this beast vomits and poos everywhere in the house...7 -
So we had this legacy Objective-C codebase for a mobile app that was actually pretty good: I'd inherited the codebase and spent the past several years gradually improving it and I was actually quite proud of the work I put into it. So of course management decides to scrap it (with NO consultation from the engineers) and outsource a complete rewrite of the app in C# for Windows Universal.
Let me tell you. That code was without a doubt and without exaggeration the *worst* code I've seen in my close to 30 years of experience as a developer. I mean they broke every rule in the book, I'm talking rookie mistakes. Copypasta everywhere, no consistent separation of concerns, and yet way too many layers. Unnecessary layers. Layers for the sake of layers. There was en entire abstraction layer complete with a replicated version of every single data class *just* to map properties in pascal case to the same property in camel case. Adding a new field to a payload in the API amounted to hours of work and about eight different files that needed to be modified. It was a complete nightmare. This was supposed to be a thin client, yet it had a complete client-side Sqlite database with its own custom schema (oh and of course a layer for that!) completely unrelated to the serverside schema, just for kicks. The project was broken up into about eight or nine different subprojects, each having their own specific dependencies on various of the other subprojects in such a tightly-knit way that it made gradual refactoring almost impossible. This architecture was so impressively bad, it was actually self-preserving!
Suffice it to say it was a complete nightmare, and was one of the main reasons I ended up leaving that company. So just sayin', legacy code isn't always bad. :) -
Two months ago I started working at a new company, who's system is a huge monolith. The company is a bit over one year old, and the code base is huge. The desire to move to more of a microservices architecture is on the radar, but one of the biggest issues in moving towards it is how we should keep our models. The stack is basically Node.js and Mongoose, where there's about a few dozen mongoose models that the whole system uses, and the issue is that, if we moved to a microservices architecture, how could we keep the models in sync. One idea I had was to keep the models in a separate (node) package that would be shared across all microservices, but then there's the issue that if one model needs changes, all microservices that use that model will need to be updated. Another idea we had was to not share models, but instead let every microservice be in charge of everything to do with a certain type of data (eg. Users are only directly accessed by one microservice, companies by another, and no two microservices share responsibility over data), but that might bring problems when one microservice depends on a certain set of data from another microservice. How do you guys manage all that? Any ideas or tips? Thanks ^^14
-
So.. how does being in a company makes you a better engineer?
For eg, this year my team shipped these 2 interesting features (among other things) that were never part of our product.
these were crazy ideas, that got liked by our users, and might have impacted the finances of company positively.
However the engineers (not me, i worked on them even less) that worked on it, did they got their knowledge growth? they just analysed the old codebase, its shitty architecture and drawbacks, and added the feature around it in a similar manner. so basically it was building debris over the debris
but is this growth? if that engg was me, would my experience in dealing with debris and building debris over it (that somehow works) be helpful for any other organisation?
And number (2) : is my organisation even a good one if its allowing to build debris over debris and not leaving a space for discussion or cleaning the mess or thinking about better architectures, data structures for scalability and robustness?
An engineer's growth should be made by giving them a chance to explore new and best solutions and not the best hacks i guess1 -
This was originally a reply to a rant about the excessive complexity of webdev.
The complexity in webdev is mostly necessary to deal with Javascript and the browser APIs, coupled with the general difficulty of the task at hand, namely to let the user interact with amounts of data far beyond network capacity. The solution isn't to reject progress but to pick your libraries wisely and manage your complexity with tools like type safe languages, unit tests and good architecture.
When webdev was simple, it was normal to have the user redownload the whole page everytime you wanted to change something. It was also normal to have the server query the database everytime a new user requested the same page even though nothing could have changed. It was an inefficient sloppy mess that only passed because we had nothing better and because most webpages were built by amateurs.
Today webpages are built like actual programs, with executables downloaded from a static file server and variable data obtained through an API that's preferably stateless by design and has a clever stateful cache. Client side caches are programmable and invalidations can be delivered through any of three widely supported server-client message protocols. It's not to look smart, it's engineering. Although 5G gets a lot of media coverage, most mobile traffic still flows through slow and expensive connections to devices with tiny batteries, and the only reason our ever increasing traffic doesn't break everything is the insanely sophisticated infrastructure we designed to make things as efficient as humanly possible.11 -
TLDR;
How much do you earn for your skill set in your country vs your cost of living?
BONUS;
See how much I & others earn.
Recently I became aware of just how massive the gap in developers earnings are between countries. I'd love to calculate a fixed score for income vs cost of living.
I know this stuff is sensitive to some so if you prefer just post your score (avg income p/m after tax / cost of living).
I'm not shy so I'll go first:
MY RATES
Normal Rate (Long term): $23
Consulting / Short term: $30-$74
Pen Test: $1500 once off.
Pen Test Fixes: consulting rate.
Simple work/websites: min $400+
Family & Friends: Dev friends are usually free (when mutually beneficial). Family and others can fuck off, even if they can pay (I pass their info to dev friends with fair warning).
GENERAL INFO
Experience: 9 years
Country: South Africa
Developer rareness in country: Very Rare (+-90 job openings per job seeker).
Middle class wage in country: $1550 p/m (can afford a new car, decent apartment & some luxuries like beer/eating out).
Employment type: Permanent though I can and do freelance occasionally.
Client Locality: Mostly local.
Developer Type: Web Developer (True web dev - I do anything web related from custom HTTP servers to sockets, services, advanced browser api's, apps & more).
STACKS / SKILLSETS
I'M PROFICIENT IN:
python, JavaScript, ASP classic, bash, php, html, css, sql, msql, elastic search, REST, SOAP, DOM, IIS, apache
I DABBLE WITH:
ASP.net, C++, ruby, GO, nginx, tesseract
MY SPECIALTIES:
application architecture, automation, integrations, db's, real time data, advanced browser apps/extensions (webRTC, canvas etc).
SUMMARY
Avg income p/m after tax: $2250
Cost of living (car+rent+food): $1200
Score: 1.85
*Note: For integrity when calculating my cost of living I excluded debt repayments and only kept my necessities which are transport, food & shelter.
I really hope you guy's post your results, it would be great to get an idea of which is really the worst / best country to be a developer in.20 -
A couple of years ago I was working on a fairly large system with a complex (by necessity) access control architecture.
As is usually the case with those projects, it's awkward for developers to repro bugs that have to do with a user's accesses in production when we are not allowed to replicate production data in test, let alone locally.
We had a bug where I ended up making myself a new row in the production database for a thing I could have access to without affecting real data to repro it safely. I identified the bug so I could repro it in dev/test and removed the row and ensured everything worked normally, whew scary.
Have you ever walked into the office one day, and everyone is hunched over in a semicircle around one person's workstation, before one turns around to look at you and says - after a pause - "... ltlian?.."
Turns out I had basically "poisoned the well" with my dummy entity in a way where production now threw 500 for everyone BUT me who had transitive access to this post-non-entity. Due to the scope of the system, it had taken about a day for this to gradually propagate in terms of caching and eventual consistencies; new entities coming in was expected, but not that they disappear.
Luckily I had a decent track record for this to be a one-off. I sometimes think about how I would explain testing in prod and making it faceplant before going home for the day, other than "I assumed it would be fine". I would fire me.3 -
Worst architecture I've seen?
The worst (working here) follow the academic pattern of trying to be perfect when the only measure of 'perfect' should be the user saying "Thank you" or one that no one knows about (the 'it just works' architectural pattern).
A senior developer with a masters degree in software engineering developed a class/object architecture for representing an Invoice in our system. Took almost 3 months to come up with ..
- Contained over 50 interfaces (IInvoice, IOrder, IProduct, etc. mostly just data bags)
- Abstract classes that implemented the interfaces
- Concrete classes that injected behavior via the abstract classes (constructors, Copy methods, converter functions, etc)
- Various data access (SQL server/WCF services) factories
During code reviews I kept saying this design was too complex and too brittle for the changes everyone knew were coming. The web team that would ultimately be using the framework had, at best, vague requirements. Because he had a masters degree, he knew best.
He was proud of nearly perfect academic design (almost 100% test code coverage, very nice class diagrams, lines and boxes, auto-generated documentation, etc), until the DBAs changed table relationships (1:1 turned into 1:M and M:M), field names, etc, and users changed business requirements (ex. concept of an invoice fee changed the total amount due calculation, which broke nearly everything).
That change caused a ripple affect that resulted in a major delay in the web site feature release.
By the time the developer fixed all the issues, the web team wrote their framework and hit the database directly (Dapper+simple DTOs) and his library was never used.1 -
In 2015 I sent an email to Google labs describing how pareidolia could be implemented algorithmically.
The basis is that a noise function put through a discriminator, could be used to train a generative function.
And now we have transformers.
I also told them if they looked back at the research they would very likely discover that dendrites were analog hubs, not just individual switches. Thats turned out to be true to.
I wrote to them in an email as far back as 2009 that attention was an under-researched topic. In 2017 someone finally got around to writing "attention is all you need."
I wrote that there were very likely basic correlates in the human brain for things like numbers, and simple concepts like color, shape, and basic relationships, that the brain used to bootstrap learning. We found out years later based on research, that this is the case.
I wrote almost a decade ago that personality systems were a means that genes could use to value-seek for efficient behaviors in unknowable environments, a form of adaption. We later found out that is probably true as well.
I came up with the "winning lottery ticket" hypothesis back in 2011, for why certain subgraphs of networks seemed to naturally learn faster than others. I didn't call it that though, it was just a question that arose because of all the "architecture thrashing" I saw in the research, why there were apparent large or marginal gains in slightly different architectures, when we had an explosion of different approaches. It seemed to me the most important difference between countless architectures, was initialization.
This thinking flowed naturally from some ideas about network sparsity (namely that it made no sense that networks should be fully connected, and we could probably train networks by intentionally dropping connections).
All the way back in 2007 I thought this was comparable to masking inputs in training, or a bottleneck architecture, though I didn't think to put an encoder and decoder back to back.
Nevertheless it goes to show, if you follow research real closely, how much low hanging fruit is actually out there to be discovered and worked on.
And to this day, google never fucking once got back to me.
I wonder if anyone ever actually read those emails...
Wait till they figure out "attention is all you need" isn't actually all you need.
p.s. something I read recently got me thinking. Decoders can also be viewed as resolving a manifold closer to an ideal form for some joint distribution. Think of it like your data as points on a balloon (the output of the bottleneck), and decoding as the process of expanding the balloon. In absolute terms, as the balloon expands, your points grow apart, but as long as the datapoints are not uniformly distributed, then *some* points will grow closer together *relatively* even as the surface expands and pushes points apart in the absolute.
In other words, for some symmetry, the encoder and bottleneck introduces an isotropy, and this step also happens to tease out anisotropy, information that was missed or produced by the encoder, which is distortions introduced by the architecture/approach, features of the data that got passed on through the bottleneck, or essentially hidden features.4 -
So... Yesterday I ordered a meal and it had whole jalapenos in it. I didn't order jalapeños. I love the taste but I hate toilet visits after. Hence, was putting them aside. But then I got into that new code, jumping around this new project I'll be working on. We were getting intimate. I liked the architecture, I liked it a lot - it was using event sourcing and respected CQRS. Suddenly I realised I ate everything. Including jalapeños. And the only reason I noticed is because I was eating with my hands. And my eye got watery. And I wiped it.
So, yeah. Yesterday for the first time in my life I was pouring milk into my eyes. Does this count as a proper dev rant? I don't know. Fuck the protein interface that can't process simple food orders, though.6 -
Android development is unnecessarily complex. To the core. Rant #371651681324
Only being few months in to android development, I can say that using too many functions and classes for doing something silly is very illogical. I mean -
Livedata, think about it, if some data is shown on the UI, it must mean the UI be updated instantly on data change, but the concept of Livedata was introduced only few years back.Which again, needs an observer and we actually have to ask it to observe changes.
Android development is a mess. More and more updates to the architecture and code is to cover up the problems that shouldn't have existed in the first place. A simple Spinner with custom object will require more time in stackoverflow rather than actual coding. Very counter-intuitive, inefficient, time-consuming and exhaustive.4 -
I don't know why they made so many algorithms, data structures and big O questions during interview, when all they wanted me to do was to maintain some legacy, tight coupled, spaghetti code with no architecture, documentation, tests nor any kind of engineering behind :/1
-
My image of dream career through different times of my life:
- frontend specs prodigy, css enlightenment, a member of w3c or a similar committee
- indie hacker and entrepreneur, leader of a startup community
- architecture prodigy, expert in scalability
- transsexual evangelist, popular article writer and a rockstar
- hardware engineer: Linux, C, chip and dale’s Gadget-like girlfriends, xkcd, latex, assembly, buying a radio station and a telescope
- scientist like NickyBones, papers, data, more data
- art expert
Though achieving one of this would take the entire life, I had a chance to grasp all of this. WHY does they feel so incompatible? Why do I have to choose?
Why do I feel so sad? Why do I feel like I haven’t achieved anything even though I objectively achieved what I dreamed of like five years ago?
Is it true that it’s in my nature to always seek an environment to feel like a junior in? Is feeling like a junior only pleasant to me because it reminds me of old times when I wasn’t actually this mentally ill and was still happy?
Why do I feel like that arduino and C shit is the equivalent of a red corvette?6 -
FUCK YOU TO GODDAMN MICROSERVICE ARCHITECTURE!
I just want to be able to extensively test stuff on my machine before shipping it instead of being able to test it only partially because shit depends of tons of stuff unavailable locally, get dozens of messages from teammates when unforseeable circumstances (bad data items on the shared noSQL DB created by other services which makes mine fail, cloud issues...) makes my service return 500 and then struggle in tracing the problem because there they're just too many layers of shit to manually inspect.
I can't wait to move towards iOS or desktop development.7 -
Agile is stupid.
You’re trying to build an application based on data that doesn’t exist yet, and that changes weekly; surely it makes more sense for the API to be built and in place before someone tries to create a front end?
The client decides they want an extra feature 1/2 way through which changes the way it should work yet again.
What you end up with is some rigid and poorly structured architecture, that might work - but how you have no time to refactor it to make it good, an account manager that doesn’t understand what the problem is, and a team of developers wondering wtf you were thinking when you wrote this pos4 -
So this web company i joined had a page load time in minutes. The free text search (inverted index search, based on elasticsearch) queries would return results in 10-45 seconds (should be milliseconds always). The indexes had no schema. And they would crawl data and feed into mssql db, which had a 2 gb/db limit on the free version. So everytime the db hit the limit, a new db was created and the name was incremented by one.
Had a very tough time cleaning up that mess. Plus the architect who had made this architecture was on his way out and unhelpful to the core.
What was worse was that most of the changes i did were very simple changes that should have been done long back. Basic sanity changes.4 -
Me: Ok, we'll implement that message tech. But since the clients are servers in that architecture and can't speak IPv6 we've to use a dedicated VPN so the endpoint is able to connect to the servers (clients). Since we have limited network resources we should use VPN cert-encryption and send the actual data plain to save at least some overhead.
Boss: Ok! Let's do it!
Next day.
Boss: Hey! I talked to a guy from that message tech. Their encryption is certified. We should use that instead and get rid of the VPN to save the overhead!
Me: *unable to say a word*
What in "VPN in that architecture is mandatory" is unclear?
Well, I assume we'll kill the architecture then... Fun Time! -
I don't care about market cap. Stick your hype-driven business practices up your ass. Infinite growth doesn't exist. I won't read your fucking books and attend your fucking bootcamps and MBAs. You don't have a business model. Selling data is not a business model. Fuck your quick-flip venture capital schemes, and especially fuck your “ethics”.
I will be the first alt-tech CEO. I only care about revenue. The real money, not capitalization bubble vaporware. You don't need a huge fleet of engineers if you're smart about your technology, know how to do architecture, and you're not a feature creep. You don't need venture capital if you don't need a huge fleet of engineers. You don't need to sell data if you don't need venture capital. See? See the pattern here?
My experience allows me to build products on entirely my own. I am fully aware of the limitations of being alone, and they only inspire lean thinking and great architectural decisions. If you know throwing capacity at a problem is not an option, you start thinking differently. And if you don't need to hire anyone, it is very easy to turn a profit and make it sustainable.
If you don't follow the path of tech vaporware, you won't have the problems of tech vaporware, namely distrust of your user base, shitty updates that break everything, and of course “oops, they raised capital, time to leave before things go south”.
A friend of mine went the path I'm talking about, developed a product over the course of four years all alone, reached $10k MRR and sold for $0.8M. But I won't sell. I only care about revenue. If I get to $10k MRR, I will most likely stop doing new features and focus on fixing all the bugs there are and improving performance. This and security patches. Maybe an occasional facelift. That's it. Some products are valued because they don't change, like Sublime Text. The utility tool you can rely on. This is my scheme, this is what I want to do in life. A best-kept secret.
Imagine 100 million users that hate my product but use it because there are no alternatives, 100 people in data enrichment department alone, a billion dollars of evaluation (without being profitable), 10 million twitter followers, and ten VC firms telling me what to do and what data to sell.
Fuck that. I'd rather have one thousand loyal customers and $10k MRR. I'm different, some call it a mental illness, but the bottom line is, my goals are beyond their understanding. They call me crazy. I won't say it was never about the money, of course it was, but inflating your evaluation is not “money”. But the only thing they have is their terrible hustle culture lives and some VC street wisdom, meanwhile I HAVE products, it is on record on my PH. I have POTDs, I have a fucking Golden Kitty nomination on health and fitness for a product I made in one day. Fuck you.7 -
To me this is one of the most interesting topics. I always dream about creating the perfect programming class (not aimed at absolute beginners though, in the end there should be some usable software artifact), because I had to teach myself at least half of the skills I need everyday.
The goal of the class, which has at least to be a semester long, is to be able to create industry-ready software projects with a distributed architecture (i.e. client-server).
The important thing is to have a central theme over the whole class. Which means you should go through the software lifecycle at least once.
Let's say the class consists of 10 Units à ~3 hours (with breaks ofc) and takes place once a week, because that is the absolute minimum time to enable the students to do their homework.
1. Project setup, explanation of the whole toolchain. Init repositories, create SSH keys for github/bitbucket, git crash course (provide a cheat sheet).
Create a hello world web app with $framework. Run the web server, let the students poke around with it. Let them push their projects to their repositories.
The remainder of the lesson is for Q&A, technical problems and so on.
Homework: Read the docs of $framework. Do some commits, just alter the HTML & CSS a bit, give them your personal touch.
For the homework, provide a $chat channel/forum/mailing list or whatever for questions where not only the the teacher should help, but also the students help each other.
2. Setup of CI/Build automation. This is one of the hardest parts for the teacher/uni because the university must provide the necessary hardware for it, which costs money. But the students faces when they see that a push to master automatically triggers a build and deploys it to the right place where they can reach it from the web is priceless.
This is one recurring point over the whole course, as there will be more software artifacts beside the web app, which need to be added to the build process. I do not want to go deeper here, whether you use Jenkins, or Travis or whatev and Ansible or Puppet or whatev for automation. You probably have some docker container set up for this, because this is a very tedious task for initial setup, probably way out of proportion. But in the end there needs to be a running web service for every student which they can reach over a personal URL. Depending on the students interest on the topic it may be also better to setup this already before the first class starts and only introduce them to all the concepts in a theory block and do some more coding in the second half.
Homework: Use $framework to extend your web app. Make it a bit more user interactive with buttons, forms or the like. As we still have no backend here, you can output to alert or something.
3. Create a minimal backend with $backendFramework. Only to have something which speaks with the frontend so you can create API calls going back and forth. Also create a DB, relational or not. Discuss DB schema/model and answer student questions.
Homework: Create a form which gets transformed into JSON and sent to the backend, backend stores the user information in the DB and should also provide a query to view the entry.
4. Introduce mobile apps. As it would probably too much to introduce them both to iOS and Android, something like React Native (or whatever the most popular platform-agnostic framework is then) may come in handy. Do the same as with the minimal web app and add the build artifacts to CI. Also talk about getting software to the app/play store (a common question) and signing apps.
Homework: Use the view API call from the backend to show the data on the mobile. Play around with the mobile project to display it in a nice way.
5. Introduction to refactoring (yes, really), if we are really talking about JS here, mention things like typescript, flow, elm, reason and everything with types which compiles to JS. Types make it so much easier to refactor growing codebases and imho everybody should use it.
Flowtype would make it probably easier to get gradually introduced in the already existing codebase (and it plays nice with react native) but I want to be abstract here, so that is just a suggestion (and 100% typed languages such as ELM or Reason have so much nicer errors).
Also discuss other helpful tools like linters, formatters.
Homework: Introduce types to all your API calls and some important functions.
6. Introduction to (unit) tests. Similar as above.
Homework: Write a unit test for your form.
(TBC)4 -
You've got the perfect architecture lined up, a great idea about how to transition and you're about to start refactoring. Then it hits you. You've got 10 clients running your code in production and you'll have to migrate their data to the new structures, from the blob to each new microservice datastore. For each release. FML.1
-
Let's see what's on the menu today:
* Web Application Catastrophe Special *
Includes, but not limited to:
- Orphaned server processes in the configuration management cluster
- Microservice back-end architecture with no API documentation
- Poorly implemented cache microservice with no documentation
- Stale data causing everything to be shown as down in production, despite everything running fine
Cost: 1 developer's sanity -
Not so much a problem with the way CS is taught, but I think it's a problem that a lot of people put emphasis ONLY on programming (and maybe data structures and algorithms) and ignore things like Computer Architecture or Theory of Computation.
Most of the CS syllabi I've seen are built very well, but many students (and some teachers) seem to ignore a bunch of subjects because they don't contribute to making them "hireable". -
My boss is being a stupid cunt. To give you a background we were facing issues with our Collections system. First week December 2019, I and a colleague of mine came up with a new efficient collections architecture. My colleague and I started to Code and create automation scripts mid December and completed it in First week of Jan 2020. This PoC version was supposed to be just between the Dev team(App Dev and Back end, also one from the Ops side to verify the data). I did not receive any feedback on the actual collections system and the data integrity but during this time all they’ve done is take meetings with no real outcome. I raised this and the only email I got is data is looking fine when I know it is not.Now in First week of Feb, he is stressing us to go ahead and deploy the architecture in Production and we have not done any Code Review, Static Code analysis, any real tests on Code and deployment scripts. Have not discussed any metrics for our dashboard and alerting. I have no idea how to handle this cunt. I have even asked for resources to atleast productionalize the code and move ahead the deployment and still no out come. I’ll go in a meeting with him in an hour, I will be very blunt and tell him that whatever he is doing is a foolish way and maybe resign in couple of weeks6
-
our team are responsible to build backend restful API for other team to look up data in DB.
the consumer team just sit beside us.
the interface definition came from our pm in a different time zone. btw he did not have any programming background.
and he insisted that just build what he said and ignore the noise from the consumer team. because each interface change should be considered as new features and need him to prioritize and create user story and he will review the schema with the pm from consumer team and so called architecture who did not coding real shit for years.
we ended up with building shit code not useable by our real consumer.
yes he do manage to keep our team busy building worthless shit and accomplishmented lots of jira items to show we have value to change a useless shit into very hard to use shit1 -
FUCK YOU PHP, FUCK YOU SYMFONY AND DEFINITELY FUCK YOU SHOPWARE.
Don't get me wrong, PHP has evolved a lot, but the stuff people are building with it is just the biggest load of fucking shit I have ever seen: Shopware. Shopware is the most ass-sucking abomination to extend. It's nearly impossible to develop anything beyond "use the standard features and shut the fuck up" that is more sophisticated than a fucking calculator.
The architecture of this pile of crap is the worst bullshit ever. A mix of OOP, randomly making use of non OOP concepts and features together with the unnecessarily HUGE amount of useless interfaces and classes. Sometimes I feel like it's 90% fucking shitty boilerplate shit.
And don't get me started with TWIG. It's a nice thought, but WHY THE BLOODY FUCK WOULD YOU NOT USE VUE IF YOU ARE ALREADY USING IT FOR A DIFFERENT PART OF SHOPWARE. This makes no fucking sense whatsoever and makes development of new features a huge pain in the ass. I can't comprehend how people actually like using this shit.
OH AND THE DATABASE. OH MY FUCKING GOD. This one is bad. Ever tried to figure anything out in a database where random strings (yes MySQL "relational" - you might think) that are stored as text in a JSON format make up some object or relations during runtime?? Why the fuck do you have foreign and primary keys if you don't use them properly??
Seriously you can't even figure out which data belongs to what because the architecture just sucks fucking ass. FUCK YOU Shopware wankers, you suck, your product sucks, your support sucks, your architecture sucks and you keep releasing new versions that regularly break shit even in minor versions.
I used to like PHP, but not in projects like these.6 -
FUCK YOU PYTHON. Why you do that to me, uh?
I was using a CNN to classify hand poses and the prediction was not working at all, one class was given 100% all the time. After much investigation, I found the culprit... A FUCKING INDENT was messing my data. Normalization was inside the loop and not outside, so my pixel values were wayyyyy too small...
Also, I'm really dumb, I should have started with making sure everything was right before trying to fiddle with my architecture..
Anyway, it is working now, you can it out here if you want! https://github.com/MrEliptik/...13 -
We have been at a university of applied sciences today with our class.
It was kind of ok. I did expect more surprising things there. The whole building was smaller than our college (not the same as in the US). The rooms, where profs tell you things with a series of rows of seats, were dirty and pretty much used to the point that the seats are about to break easily.
I was expecting the university to be kind of the same as the universities you see in the movies lol.
It could have at least been bigger than our college and more "modern" than our school.
[...]
Anyways, let us get to the point here.
We were first in the foyer and afterwards in their main lecture hall.
We were introduced to the day's plans by a team of engaged students from different study programs and the president of the professors. Yada yada yada.
We got the full program in each room and each individual time span filled with study programs on a sheet of paper.
I did select pharmacy, media production, architecture, data science, applied computer science, computer engineering, mechanical engineering and future energies.
Pharmacy and data science were the most interesting study programs to me. I have asked one of the professors if deep learning was a topic for bachelor students, as well.
He said that that is only the usual case for people who got a promotion.
As an example he told me that yesterday he was at a conference hall with 10.000 people in which he gave a talk about deep learning. "Most of them were professors" he said. "Since this study program is new, it might change in a few years" he added to his conversation.
It is quite hard having to decide now.
Geo informatics and Aerospace Engineering did sound interesting, too.
There are a lot of things I would like to study at the same time haha.
Idk if I should just pick mechanical engineering first and add one or two after it to it. But that would take a lot of time. Geez.7 -
I had the idea that part of the problem of NN and ML research is we all use the same standard loss and nonlinear functions. In theory most NN architectures are universal aproximators. But theres a big gap between symbolic and numeric computation.
But some of our bigger leaps in improvement weren't just from new architectures, but entire new approaches to how data is transformed, and how we calculate loss, for example KL divergence.
And it occured to me all we really need is training/test/validation data and with the right approach we can let the system discover the architecture (been done before), but also the nonlinear and loss functions itself, and see what pops out the other side as a result.
If a network can instrument its own code as it were, maybe it'd find new and useful nonlinear functions and losses. Networks wouldn't just specificy a conv layer here, or a maxpool there, but derive implementations of these all on their own.
More importantly with a little pruning, we could even use successful examples for bootstrapping smaller more efficient algorithms, all within the graph itself, and use genetic algorithms to mix and match nodes at training time to discover what works or doesn't, or do training, testing, and validation in batches, to anneal a network in the correct direction.
By generating variations of successful nodes and graphs, and using substitution, we can use comparison to minimize error (for some measure of error over accuracy and precision), and select the best graph variations, without strictly having to do much point mutation within any given node, minimizing deleterious effects, sort of like how gene expression leads to unexpected but fitness-improving results for an entire organism, while point-mutations typically cause disease.
It might seem like this wouldn't work out the gate, just on the basis of intuition, but I think the benefit of working through node substitutions or entire subgraph substitution, is that we can check test/validation loss before training is even complete.
If we train a network to specify a known loss, we can even have that evaluate the networks themselves, and run variations on our network loss node to find better losses during training time, and at some point let nodes refer to these same loss calculation graphs, within themselves, switching between them dynamically..via variation and substitution.
I could even invision probabilistic lists of jump addresses, or mappings of value ranges to jump addresses, or having await() style opcodes on some nodes that upon being encountered, queue-up ticks from upstream nodes whose calculations the await()ed node relies on, to do things like emergent convolution.
I've written all the classes and started on the interpreter itself, just a few things that need fleshed out now.
Heres my shitty little partial sketch of the opcodes and ideas.
https://pastebin.com/5yDTaApS
I think I'll teach it to do convolution, color recognition, maybe try mnist, or teach it step by step how to do sequence masking and prediction, dunno yet.6 -
Hey DevRant!
Not really a rant but a question:
I just got accepted into a coding bootcamp. Have any of you been involved in one? How was it? What would you do to make it a better experience throughout? Any advice or suggestions?
It's full time, six months long and I start in October and I want to make sure I make the most of the experience and absorb as much as possible.
I'm super happy that the course appears to be less just learning JavaScript and more involved in the Computer Science side of things, even including bits about C/C++, distributed systems, algorithms and data structures, software design/testing, cryptography, database management, and computer architecture. It also, of course, covers tons of resume work, interview practice, and networking.
Thanks!5 -
I was studying a lot the last year, i learned a lot about Machine Learning/Deep Learning, Data Gathering, Data Analysis, ETL, Model Architecture Design, Training, Fine Tuning, Backend Development, DataBases, API Development, ORMs, Rest, GraphQL, OAuth, CI/CD, Docker, Deployment to Production environments like Heroku, Git and more stuff i dont remember while writing this. I built and keep adding stuff to my Github Portafolio.
Im not able to get a job. I started looking for jobs as Data Scientists, no response never. I take a look at freelancer sites, nothing seems to fit my skills. And when there is a minimal fit, they always want a Full Stack Web Developer, i dont know Frontend Development, i dont like do it.
Dont know what to do or how to land any job.
My options aeems to be:
1.Learn Frontend Dev and work as Full Stack in underpaying freelance jobs
2.Keep applying to Remote-Only startups, but they still wants people with 3+ years of experience.
i cant work in my city, here are not any company startup hiring no one, we are 30 years in the past here.
What you do in my place?10 -
My job is decent, but now I've got one developer who's been there a few months longer than me who pushes back on stuff that's considered standard, good practice.
We have a domain with lots of business rules. He's opposed to any sort of domain-centric architecture that puts business logic in one place. He doesn't give any coherent reasons. He can't describe his alternative clearly. He just wants to put stuff all over the place.
If I don't cite any references he says it's just my opinion. If I do, I'm talking down to him.
Then he decided that the database shouldn't have concurrency checks. His reasoning is that as the application grows we'll have more and more concurrent updates, and they all have to succeed.
What if that corrupts our data? He mentions "eventual consistency." which has nothing to do with what we're talking about.
The idea that our code should carefully ensure that our data is correct is "extreme." What are we going to tell people when bugs happen? That expecting the application to work correctly is extreme?
He's not a terrible developer. He's an advanced Expert Beginner. He's convinced himself that whatever he doesn't already know isn't worth knowing. That's fine if he wants to stop learning, but this affects the whole team. He makes such a fuss that it everyone gets stressed out and eventually I have to back off.
The problem is that someone with a reasonable degree of competence can pass off his experience as superior to all knowledge from outside sources.
I've been doing this as long as him. I don't claim to be a rock star genius, but I do keep learning. I don't tell myself that I've reached the pinnacle. But all of that learning goes to waste if I can't use it.3 -
What are your plans for Christmas?!?!!??
I normally won't engage in societal tropes like pointless, generic, smalltalk or those questions people ask for lack of independent thought/societal trope-isms....
Here's my templated answer this year:
Background = ~2k$ in piles of tech... server upgrades components, apparently the only managed switch left in business/non-custom enterprise networking in the country/indexed for sale
(2k in what I would pay.... my tech sourcing is more base level and +4 years pro exp(yea... since age 8... really))
Foreground.... a shiny ✨️ new, wonderfully discounted for dumb reasons that i appreciate... 10Tb LFF HDD! 🥹🥲🤩
I really like raw data... enough raw data and proper context relevant high-level, custom, precise algorithms and i genuinely believe literally any questions or problems can be quantified and solved for
So... I just keep getting data, life, sourcing, stats on human behaviour... i factor everything
Yes i realise im very odd
//initial context plus curiousities
As parsed out to somewhat tangential commentary below... i cant keep making people go away for societally viewed polite engagement. Therefore, when asked again by factory sales rep who enjoys verbosity and apparent finds me extremely worth his intrigue/personal time
// additional context (and my attempt to be more parse and comment conscious)
With a bunch of initial reveals and launches startjng in a week and technically being the "owner/boss"(cringy to me so Ive officially made my title (anywhere with custom input fields) DragonOverlord...dragons being a tied in theme to all sects and no one can say DragonOverlord isn't a position... as it's clearly a class... unless you find a human more style code ignorant, comment inept, and in need of a very multilingual scribe to create a lexicon 2 steps before my code would be even follow-able without a likely, bad, headache and davinci code like adventure including the improbably well placed wise scholars that just happen to have significant unique and vital information they are willing to freely share with strangers.rant christmas data architecture motivational societal tropes temptation so i can build my database structure loathing python raw data data misanthropy databases49 -
Is a masters in statistics worth it?
A bit of background:
I got my bachelor in actuarial math (statistics for insurance risk), then found machine learning and got a couple of gigs in software development and data engineering. I became my previous employers the go to guy for questions about data integrity and structure.
Now I am heading to a new job that specializes in ML for gambling. And while I love the math, I really see myself doing more software development and system architecture work (with some analysis). I already started this masters program, so I got less than a year to finish, but starting to feel like its a waste of my time, but also, I dont want to just quit it. -
ok this may look like a lazy ass beginner crying out for spoon feeding( which it kinda is), but i want some real industrial training in non documented Android coding.
For last 2 years i have been reading tons of Android articles and documentation on "how to use this library", "how to add this feature", "what this function of this class does", but not much about how to use it efficiently, like the way its used in industry.
When I interned with a startup, all they wanted from me was to push new design changes, fix layout bugs and work as fastly as i could. I had no time to understand their core code, which had so many things that i could have learned : those mvp/mvvm design/architecture patterns, dependency injections, kotlin , coroutines, state management designs, data bindings, eventbuses and handling, and VIPER,RIBS (I mean, not everything was particularly in their code, i picked up a few keywords from here n there)... a lot of stuff that is used by many apps for their codebase.
I can read up these stuff by myself, but i always end up feeling bored coz frankly, i got no big/valuable project to implement it upon and feel excited about it. I feel that open source projects from OSS companies could be my window, but their chat spaces are also mostly empty to discuss/get some guidance.
I want some specific training about these. Can you guys provide any online/offline course/company training/books in this subject, the best practices?1 -
Anyone know of any good reference material that teaches you how to implement and train your own Yolo object localization neural network? (Preferably for tensorflow) I'm not looking for pretrained models that you just downloaded and run, rather a tutorial that walks you step by step through the implementation of the network, the reasoning behind the architecture, and examples of the training data used for the training as well as the process of training?
I know it's a lot to ask but it's really frustrating when ever example is just "clone this repo, hit run and use the pretrained models" sure it might get you up and running quickly but it doesn't really teach you anything...
P.s. - seems like every educational post goes from super simple to super complex without any middle ground and the super complex stuff doesn't tell you why its used the way way it is.5 -
What do you think of Elixir + Phoenix to build API’s? Is it a better choice than a more established language like Python or something more new like Scala or Clojure?
At my company we're going through a watershed moment where we're starting to discuss and think about re-building our digital foundations and nothing is off limits. I'm leading the discussion about our architecture where everyone can have their say into what the future looks like for our applications. We're currently on a Drupal (CMS) + PHP7/Symfony (Backend Content Repository) + Symfony Twig templates (Frontend)
Even though I have been developing in PHP most of my career, I personally love Elixir and spend a lot of my time away from work learning it but many of my reasons feels subjective like pattern matching, it's actor concurrency model, immutable data and not having to deal with classes/objects, and I'm not entirely sure how that translates to business value, advocating successfully for a tech stack change requires solid reasoning and good answers to challenges like how do we find Elixir developers when existing devs leave, how easy is it to build a CI/CD pipeline for Elixir/Phoenix, etc.4 -
I really wanna get a keg of rum and start sailing across the globe...
Just to spank some devs / managers arses.
The last years were... very demanding regarding security and upgrades.
It hasn't gotten better.
Microsoft leaked it's security key thx to internal debugging and the tool to secure the debugging process so secure data gets filtered was buggy...
I'd guess I already have carpal tunnel after Redmond.
But the really really sad story is: This has become the gold standard.
https://lwn.net/Articles/943969/
Chrome selling the privacy mode for Ads, long topic ongoing for years... yeah they did it.
Apple... oh boy. I could write a Silmarillion about it and would still need an additional trilogy.
Amazon realizing that a Microservice architecture needs planning, cause yeah... just potting services in a data center doesn't end well.
It goes on and on and on....
Don't even get me started on the plethora of firmware / microcode updates cause there was either yet another CPU bug or another device pooped their pants cause the firmware is a mess and needed some dubious update without any background at all...
Serious question: Am I becoming a pepperidge farm uncle threatening to shoot everyone cause I'm getting old and cranky ....
Or is really everything in IT going down the drain the last few years?
It feels like every week is just another "we fucked it up" event.3 -
Waiting 3 days for your graphics card to get trough all training data to see if you wasted your life with bad architecture.13
-
Can anyone help me with this theory about microprocessor, cpu and computers in general?
( I used to love programming when during school days when it was just basic searching/sorting and oop. Even in college , when it advanced to language details , compilers and data structures, i was fine. But subjects like coa and microprocessors, which kind of explains the working of hardware behind the brain that is a computer is so difficult to understand for me 😭😭😭)
How a computer works? All i knew was that when a bulb gets connected to a battery via wires, some metal inside it starts glowing and we see light. No magics involved till now.
Then came the von Neumann architecture which says a computer consists of 4 things : i/o devices, system bus ,memory and cpu. I/0 and memory interact with system bus, which is controlled by cpu . Thus cpu controls everything and that's how computer works.
Wait, what?
Let's take an easy example of calc. i pressed 1+2= on keyboard, it showed me '1+2=' and then '3'. How the hell that hapenned ?
Then some video told me this : every key in your keyboard is connected to a multiplexer which gives a special "code" to the processer regarding the key press.
The "control unit" of cpu commands the ram to store every character until '=' is pressed (which is a kind of interrupt telling the cpu to start processing) . RAM is simply a bunch of storage circuits (which can store some 1s) along with another bunch of circuits which can retrieve these data.
Up till now, the control unit knows that memory has (for eg):
Value 1 stored as 0001 at some address 34A
Value + stored as 11001101 at some address 34B
Value 2 stored as 0010 at some Address 23B
On recieving code for '=' press, the "control unit" commands the "alu" unit of cpu to fectch data from memory , understand it and calculate the result(i e the "fetch, decode and execute" cycle)
Alu fetches the "codes" from the memory, which translates to ADD 34A,23B i.e add the data stored at addresses 34a , 23b. The alu retrieves values present at given addresses, passes them through its adder circuit and puts the result at some new address 21H.
The control unit then fetches this result from new address and via, system busses, sends this new value to display's memory loaded at some memory port 4044.
The display picks it up and instantly shows it.
My problems:
1. Is this all correct? Does this only happens?
2. Please expand this more.
How is this system bus, alu, cpu , working?
What are the registers, accumulators , flip flops in the memory?
What are the machine cycles?
What are instructions cycles , opcodes, instruction codes ?
Where does assembly language comes in?
How does cpu manipulates memory?
This data bus , control bus, what are they?
I have come across so many weird words i dont understand dma, interrupts , memory mapped i/o devices, etc. Somebody please explain.
Ps : am learning about the fucking 8085 microprocessor in class and i can't even relate to basic computer architecture. I had flunked the coa paper which i now realise why, coz its so confusing. :'''(14 -
Disclaimer: I should know what I'm doing but I don't. 😢
I'm a very experienced full stack dev (15+ years), but I don't know the more modern JS frameworks. I'm trying to learn React and I have a little project I'd like to do.
I have database (in both SQLite form and JSON form). I'd like to read from it, parse it and run various displays in a shared hosting environment (that doesn't have node). So webpack. And either an API to get the data or a React compatible SQL component.
But dagnamit, I cannot find a tutorial or example with this kind of set up and I can't figure it out. What packages do I need and what kind of config?
I genuinely thought this would be a traditional and simple architecture but I'm obviously mistaken. And I'm about to turn in my developer card because I'm clearly a stupid twonk.
Has anyone done this? Do you know of any tutorials or examples of this kind of thing? Is there somewhere else I should ask this question? Thanks anyway...5 -
Heres some research into a new LLM architecture I recently built and have had actual success with.
The idea is simple, you do the standard thing of generating random vectors for your dictionary of tokens, we'll call these numbers your 'weights'. Then, for whatever sentence you want to use as input, you generate a context embedding by looking up those tokens, and putting them into a list.
Next, you do the same for the output you want to map to, lets call it the decoder embedding.
You then loop, and generate a 'noise embedding', for each vector or individual token in the context embedding, you then subtract that token's noise value from that token's embedding value or specific weight.
You find the weight index in the weight dictionary (one entry per word or token in your token dictionary) thats closest to this embedding. You use a version of cuckoo hashing where similar values are stored near each other, and the canonical weight values are actually the key of each key:value pair in your token dictionary. When doing this you align all random numbered keys in the dictionary (a uniform sample from 0 to 1), and look at hamming distance between the context embedding+noise embedding (called the encoder embedding) versus the canonical keys, with each digit from left to right being penalized by some factor f (because numbers further left are larger magnitudes), and then penalize or reward based on the numeric closeness of any given individual digit of the encoder embedding at the same index of any given weight i.
You then substitute the canonical weight in place of this encoder embedding, look up that weights index in my earliest version, and then use that index to lookup the word|token in the token dictionary and compare it to the word at the current index of the training output to match against.
Of course by switching to the hash version the lookup is significantly faster, but I digress.
That introduces a problem.
If each input token matches one output token how do we get variable length outputs, how do we do n-to-m mappings of input and output?
One of the things I explored was using pseudo-markovian processes, where theres one node, A, with two links to itself, B, and C.
B is a transition matrix, and A holds its own state. At any given timestep, A may use either the default transition matrix (training data encoder embeddings) with B, or it may generate new ones, using C and a context window of A's prior states.
C can be used to modify A, or it can be used to as a noise embedding to modify B.
A can take on the state of both A and C or A and B. In fact we do both, and measure which is closest to the correct output during training.
What this *doesn't* do is give us variable length encodings or decodings.
So I thought a while and said, if we're using noise embeddings, why can't we use multiple?
And if we're doing multiple, what if we used a middle layer, lets call it the 'key', and took its mean
over *many* training examples, and used it to map from the variance of an input (query) to the variance and mean of
a training or inference output (value).
But how does that tell us when to stop or continue generating tokens for the output?
Posted on pastebin if you want to read the whole thing (DR wouldn't post for some reason).
In any case I wasn't sure if I was dreaming or if I was off in left field, so I went and built the damn thing, the autoencoder part, wasn't even sure I could, but I did, and it just works. I'm still scratching my head.
https://pastebin.com/xAHRhmfH25 -
I needed to align instruction execution to a 64bit boundary, for custom CPU architecture that I'm building. Basically the ISA had 3 types of instruction lengths; 16bit, 32bit, and 48bit. The core did 64bit fetch from the Instruction Cache, the issue was that if an instruction was in between two blocks of data I needed to fetch two blocks. That would impact performance a bit. So I had to modify GCC sources of the ISA that I'm using. So instead of doing it the right way, I just did it the lazy way and modified the GCC backend to print
.p2alignw 3, 0x00ff, 4
.p2alignw 3, 0x00ff, 2
On each 48bit and 32bit instruction to generate NOPs. And it did work lol. -
Was working in an n-tier website, standard Web forms, BAL, DAL, database architecture. Validation and processing of data done in the BAL. Not the best idea, but whatever. Well apparently some developer thought it was too much work to pass his data through the BAL, so he directly accessed the DAL, performing zero validation on the data being passed in. Luckily, this was in a non-critical part of the site but the PM at the time nearly had a heart attack when I told him.
-
I could use some advice from some tenured developers... (or anyone with some thoughts)
Long story short, I went to school for business (Trust me... business people bug me too now), but in the last six months of college I didn’t like what I was doing (finance/marketing) so I dove into data analytics.
After graduating I was lucky enough to get a job at a great company doing a little data architecture work, writing lots of SQL stored procedures, managing client databases, cubes, etc... I really enjoy my work, but I recently discovered... Python...
After being introduced to Python from people at work as well as my Roomate, I’ve been trying to dig in as much as possible. I try to read/code at least an hour before work everyday and some when I get home. I love it.
So here’s where I need advice...
What do I need to do/learn to get a job writing Python all day? (Or a majority of my day)
What particular skills may I be missing that I should learn?
What do I need to do to make this happen?! (I love SQL, but damn python is amazing)1 -
I like the people I work with although they are very shit, I get paid a lot and I mostly enjoy the company but..
Our scrum implementation is incredibly fucked so much so that it is not even close to scrum but our scrum master doesn't know scrum and no one else cares so we do everything fucked.
Our prs are roughly 60 file hangers at a time, we only complete 50% of our work each sprint because the stories are so fucked up, we have no testers at all, team lead insists on creating sql table designs but doesn't understand normalisation so our tables often hold 3 or 4 sets of data types just jammed in.
Our software sits broken for months on end until someone notices (pre release), our architecture is garbage or practically non existent. Our front end apps that only I know the technology have approaches dictated by team lead that has no clue of the language or framework.
Our front end app is now about 50% tech debt because project management is so ineffectual and approaches are constantly changing. For instance we used to use view models for domain transfer objects... Now we use database entities, so there is no commonality between models but the system used to have shared features relying on that..sour roles and permissions are fucked since a role is a page regardless of the pages functionality so there is no ability to toggle features, but even though I know the design is fucked I still had to implement after hours of trying to convince team lead of it. Fast forward a few months and it's a huge cluster fuck to enforce.
We have no automated testing of any sort or manual testing in place.
I know of a few security vulnerabilities I can nuke our databases with but it got ignored.
Pr reviews are obviously a nightmare since they're so big.
I just tried to talk to scrum master again about story creation since any story involving front end ui as an aspect of it is crammed in under one pointed story as sub tasks, essentially throwing away any ability to calculate velocity. Been here a year now and the scrum master doesn't know what I mean by velocity... Her entire job is scrum master.
So anyway I am thinking about leaving because I like being a developer and it is slowly making me give up on doing things to a high standard and I have no chance of improving things, but at the same time the pay is great and I like the people. -
My friend Just completed his notice period in my company, but now he's saying that I forgot to copy some code for reference like their architecture and authentication code, so, he's asking me to share the repo code of my company. The company has deleted all his data while he was leaving. So what should I do? should I share the code without including .env files or what should I do?10
-
👇 Many people argue whether software engineers need to be good at concepts like data structures, algorithms, and system design.
Also, they think that companies should stop making their hiring decisions based on testing these concepts.
I think a basic understanding of all of the above is really necessary if you want to be a good engineer.
There can be a discussion on whether an engineer needs a mastery of the advanced topics or not.
However, a basic understanding of data structures, algorithms, and system design is essential for engineers.
Why is that?
I think overall; every software application has two parts:
◉ Data (Micro-level)
◉ Design (Macro-level)
Almost every engineer deals with both of these, depending on their role in the team.
If you're a junior engineer, you may not do that much on the design side. However, your most work would be on the micro-level, i.e., dealing with data.
If you're a senior engineer, you may work more on the macro-level, like designing the architecture, structure, arrangement of different parts, and other related stuff.
A good understanding of data structures and algorithms enables you to be good at manipulating data. So it will help you to deal with data efficiently, and you'd be able to make good decisions at the micro-level.
However, to be good at designing the architecture, you'd need to be good at dealing with different parts of the system on the macro-level. This is where system design principles help you.
This is why you need to understand the basics of both.
👉 Do you think engineers need to be good at data structures and algorithms alongside system design?3 -
I’ve been interviewing at a few companies lately. I’m a dev with ~6 years of experience with a specific language. Most of the experience comes from working in companies that developed their own software, not talking about cms stuff. Analytical, data tracking systems. Now working at a fintech. I’ve got an offer to work as a senior developer in a smaller tech team, with more salary. I’ve approached the current company about the offer and they told me that they don’t think I’m a senior dev and rather a strong mid level dev. The Hr also told me to think about if I’m really a senior and if the other companies expectations would be met. They would increase my salary, but not quite match it. It’s not too far off though. Their reasoning for this was that you need a lot of experience with their product (which does not correlate with seniorness of a developer, only the worth of specific employees for a company IMHO) and system architecture design. The problem is that we don’t see any tasks that could implement any system design for as log as I’ve worked here, so I don’t see how I could work into a senior role at this company. Of course imposter syndrome kicked in and I’m triple guessing myself if I should join the other company as a senior now. How should I aproach this? The current company is stressful to work at because of big workload, a lot of my coworkers think the same thing about the workload.11
-
Weekend ruined supporting legacy and poorly designed services coupled with poor architecture.
But "no project bandwidth" to refactor said services.
5 hours of data loss should now hopefully inspire a backlog re-shuffle. -
A year ago I built my first todo, not from a tutorial, but using basic libraries and nw.js, and doing basic dom manipulations.
It had drag n drop, icons, and basic saving and loading. And I was satisfied.
Since then I've been working odd jobs.
And today I've decided to stretch out a bit, and build a basic airtable clone, because I think I can.
And also because I hate anything without an offline option.
First thing I realized was I wasn't about to duplicate all the features of a spreadsheet from scratch. I'd need a base to work from.
I spent about an hour looking.
Core features needed would be trivial serialization or saving/loading.
Proper event support for when a cell, row, or column changed, or was selected. Necessary for triggering validation and serialization/saving.
Custom column types.
Embedding html in cells.
Reorderable columns
Optional but nice to have:
Changeable column width and row height.
Drag and drop on rows and columns.
Right click menu support out of the box.
After that hour I had a few I wanted to test.
And started looking at frameworks to support the SPA aspects.
Both mithril and riot have minimal router support. But theres also a ton of other leightweight frameworks and libraries worthy of prototyping in, solid, marko, svelte, etc.
I didn't want to futz with lots of overhead, babeling/gulping/grunting/webpacking or any complex configuration-over-convention.
Didn't care for dom vs shadow dom. Its a prototype not a startup.
And I didn't care to do it the "right way". Learning curve here was antithesis to experimenting. I was trying to get away from plugin, configuration-over-convention, astronaut architecture, monolithic frameworks, the works.
Could I import the library without five dozen dependancies and learning four different tools before getting to hello world?
"But if you know IJK then its quick to get started!", except I don't, so it won't. I didn't want that.
Could I get cheap component-oriented designs?
Was I managing complex state embedded in a monolith that took over the entire layout and conventions of my code, like the world balanced on the back of a turtle?
Did it obscure the dom and state, and the standard way of doing things or *compliment* those?
As for validation, theres a number of vanilla libraries, one of which treats validation similar to unit testing, which seems kinda novel.
For presentation and backend I could do NW.JS, which would remove some of the complications, by putting everything in one script. Or if I wanted to make it a web backend, and avoid writing it in something that ran like a potato strapped to a nuclear rocket (visual studio), I could skip TS and go with python and quart, an async variation of flask.
This has the advantage that using something thats *not* JS, namely python, for interacting with a proper database, and would allow self-hosting or putting it online so people can share data and access in real time with others.
And because I'm horrible, and do things the wrong way for convenience, I could use tailwind.
Because it pisses people off.
How easy (or hard) would it be to recreate a basic functional clone of the core of airtable?
I don't know, but I have feeling I'm going to find out!1 -
This nonsense gave me an idea. Now I want to start building a big, organised, db for good bad examples. I can think of so many uses for it if everything is tagged/categorised well.
Thank you rando LinkedIn reject, you gave me the best birthday gift I could hope for... another potential branch of my data architecture to play with new data in new and to be discovered ways!
The site of the rando is athensnexus.com5 -
Architecture for Java REST API going to build/port from existing NodeJS one.
So Spring Boot + *
Lots of concurrent requests and large MongoDB calls. Current APIs use like 4GB memory for each instance because they don't use stream/pipe the response. Hold all data in memory and then return it all at once to user.
And well we expect more load in the future, so want to do this the right way.
So my understanding since this morning, is there's the blocking? MongoClient, (find* returns List) and now a Reactive MongoClient which is very async and like JS promises. Based on Pub, Sub model.
But the downside of JS promises was callback hell.
So actually 2 questions.
1. For each request, the db call done using the same MongoClient/db connection such that if there are 2 requests one would block the other?
2. Reactive Mongo would be non-blocking by design so would be better to support streamed responses?8 -
I starting developing my skills to a pro level from 1 year and half from now. My skillset is focused on Backend Development + Data Science(Specially Deep Learning), some sort of Machine Learning Engineer. I fill my github with personal projects the last 5 months, and im currently working on a very exciting project that involves all of my skills, its about Developing and deploy a Deep Learning Model for Image Deblurring.
I started to look for work two months to now. I applied to dozens of jobs at startups, no response. I changed my strategy a bit, focusing on early stage startups that dont have infinite money for pay all that senior devs, nothing, not even that startups wish to have me in their teams. I even applied to 2 or 3 and claim to do the job for little payment, arguing im not going for money but experience, nothing. I never got a reply back, not an interview, the few that reach back(like 3, from 3 or 4 dozen of startups), was just for say their are not interested on me.
This is frustrating, what i do on my days is just push forward my personal projects without rest. I will be broke in a few months from now if i dont get a job, im still young, i have 21 years, but i dont have economic support from parents anymore(they are already broke). Truly dont know what to do. Currently my brother is helping me with the money, but he will broke in few months as i say.
The worst of all this case is that i feel capable of get things done, i have skills and i trust in myself. This is not about me having doubts about my skills, but about startups that dont care, they are not interested in me, and the other worst thing is that my profile is in high demand, at least on startups, they always seek for backend devs with Machine Learning knowledge. Im nothing for them, i only want to land that first job, but seems to be impossible.
For add to this situation, im from south america, Venezuela, and im only able to get a remote job, because in my country basically has no Tech Industry, just Agencies everywhere underpaying devs, that as extent, dont care about my profile too!!! this is ridiculous, not even that almost dead Agencies that contract devs for very little payment in my country are interested in me! As extra, my economic situation dont allows me to reallocate, i simple cant afford that. planning to do it, but after land some job for a few months. Anyways coronavirus seems to finally set remote work as the default, maybe this is not a huge factor right now.
I try to find job as freelancer, i check the freelancer sites(Freelancer, Guru and so on) every week more or less, but at least from what i see, there is no Backend-Only gigs for Python Devs, They always ask for Fullstack developers, and Machine Learning gigs i dont even mention them.
Maybe im missing something obvious, but feel incredible that someone that has skills is not capable of land even a freelancer job. Maybe im blind, or maybe im asking too much(I feel the latter is not the case). Or maybe im overestimating my self? i think around that time to time, but is not possible, i have knowledge of Rest/GraphQL APIs Development using frameworks like Flask or DJango(But i like Flask more than DJango, i feel awesome with its microframework approach). Familiarized with containerization and Docker. I can mention knowledge about SQL and DBs(PostgreSQL), ORMs(SQLAlchemy), Open Auth, CI/CD, Unit Testing, Git, Soft DevOps Skills, Design Patterns like MVC or MTV, Serverless Environments, Deep Learning Solutions, end to end: Data Gathering, Preprocessing, Data Analysis, Model Architecture Design, Training and Finetunning. Im familiarized with SotA techniques widely used now days, GANs, Transformers, Residual Networks, U-Nets, Sequence Data, Image Data or high Dimensional Data, Data Augmentation, Regularization, Dropout, All kind of loss functions and Non Linear functions. My toolset is based around Python, with Tensorflow as the main framework, supported by other libraries like pandas, numpy and other Data Science oriented utils.
I know lot of stuff, is not that enough for get a Junior Level underpaid job? truly dont get it, what is required for get a job? not even enough for get an interview?
I have some dev friends and everyone seems to be able to land jobs, why im not landing even an interview?
I will keep pushing my Dev career, is that or starve to death. But i will love to read your suggestions! how i can approach this?
i will leave here my relevant social presence:
https://linkedin.com/in/...
https://github.com/ElPapi42
Thanks in advance!9 -
Hi Guys if you can share your opinion/experience in what I wrote below it would help me a lot, thanks !
Im a full-stack developer with 4 years of experience, worked with different technologies in backend, frontend, mobile etc.. so I have general knowdgele of how systems works and how they should be built.
So I work as CTO in a startup, Im for almost 2 years here I started here with minimum salary (I decided that, because they said to me we are startup and such things so I wanted to help) 2.2k Euros and it has been almost 2 years without pay rise, so last month I asked for pay rise, but they said to me that they dont have money and sent me +300 euros as gift.
One week ago I wrote to them again (co-founders) that I have a lot of pressure and I dont know if I can handle all of that for much time he told me that I got +300 euro pay rise (which it was gift from them in first place, I refused them to sent this to me), but TODAY CEO and Co-Founder wrote to again me asking if I accept +300 euro pay rise because they can afford to pay me 2.5k or if I dont accept this they can sent me 2.2k again (they think that 2.5k is maximum that they can pay me right now and that this is enough for me).
I want to ask you guys what would you do, would you accepting something like this, considering that right now Im only dev here (yes Im only dev) and Im taking care of these(yes all of these) :
1. Company Website (react js)
2. Web Admin Panel (that clients use to manage their data)(react js)
3. Web Application (that visitors use to see client data)(react js)
4. Widgets (some code that is integrated into clients websites it's same as application, but integrated directly to client website)(react js)
5. Backend of all 3 apps mentioned above (asp.net core)
6. AWS Architecture( some of services : Cognito,Lambda,RDS,API Gateway,CloudFront,S3)
7. DevOps Role
Also consider that I didnt take holidays for 1 year now working on weekends too :)3 -
What the hell am I!? I wonder if you guys can help me...
I've been programming most of my life but I've never actually been a developer by title or job role. I thought maybe if I list what I do and have done someone here could help? I'm sure there are more of you in a similar boat.
- C# and VB dev for some quick DBMS projects to help me understand and mine databases and create a nice simple view for project teams to show findings from the data to help make certain decisions.
- Automating a lot of my colleagues work with Python and if very restricted then just VBA macros in Excel and MSP. This did also include creating tools to gather data during workshops and converting the data for input into other systems.
- Brought Linux to the office with most team members now moving over to Linux with the peace of mind to know that though they do need to try solve their own problems, I can help if need be.
- Had to learn AWS and then implement an autoscaling and load balanced data center installation of a few Atlassian toolsets.
- Creating the architecture diagrams documentation needed for things like the above point.
- Having said that, also have ended up setting up all the Jira/Confluence etc. servers we use and have implemented so far whether cloud (Azure/AWS) or on prem and set up scripts to automate where possible.
- Implemented an automated workflow view in SharePoint based on SP list data and though in an ASPX page, primarily built in JS.
- Building test systems in PHP/JS with Laravel and Angular to help manage integration between systems. Having quite a time right looking into how to build middleware to connect between SOAP and REST API's, the trouble caused more by the systems and their reliance on frameworks we're trying to cut out of the picture.
- Working on BI and MI and training a team to help on the report creation so that I can do the fun creative stuff and then set them to work on the detail :)
Actually it seems safe to say that it seems that though I've finally moved into a dev office (beforehand being the only developer around) I seem to be the one they go to when a strategic solution is needed ASAP and the normal processes can't be followed (fun for someone with a CompSci degree and a number of project management courses under the belt... though I honestly do enjoy the challenges)
But I always end up Jack of all but master of, well hopefully some at least. let's not even get started on the tech related hobbies from circuit design and IoT to Andoid / iOS and game dev and enjoying a bit of pen testing to make sure we're all safe at work and at home.
As much as I don't like boxes, I'm interested to know if there is in fact a box for me? By the way, the above is just a snapshot of my last two years minus the project management work...2 -
Well I'm back on this stupid project with this stupid Product Owner and I really hate this, it really demotivates me.
I was assigned to this project (data analytics) for like 6 months, working alone with this stupid PO that knows nothing about team management or project management.
The guy had a "methodology" where he established all task to be done daily and would not tell me what we have to do in the entire project but instead would tell me day by day all the tasks to be done in each day. This means that HE was the one making the time estimation which is plain wrong!.
Anyways, I talked to him and told him that I need to have a wide overview of the project in order to be able to make a good time estimation, and it kind of worked.
But the guy is a pain in the ass, calls me every 4 hours to "talk" about the project and texts me every hour to check "how are we doing?".
This project was killing me, I had no motivation to work on it, I hated every minute of it, I didn't like it at all to the point my boss (not him) talked to me and asked me what was wrong with me. I told him: This is not the project for me. He told me: Ok let's try to move you to another project.
After six months of agony, the project was stale (customer approval, paperwork, blah, blah) I was assigned to two other projects that I liked, more software architecture and development, not data analytics.
And last week my boss came back to me with "well, the project was approved so we need you back at it".
WHAT PART OF I'M NOT THE RIGHT GUY FOR THIS PROJECT DIDN'T YOU GET?
Now I'm again with this dude, calling me, texting me, sending me infinite emails, asking for minutely updates...
I really don't want to be working on this project. -
#Suphle Rant 4: Laravel closing the gap II
I had expected rant 4 to come at least, some days later. Apparently, I'd miscalculated how fast things work in this wonderful world of software. In an earlier rant, I wrote about how dismayed I was to learn laravel had implemented one suphle feature I'm very proud about. They call it Premonition. Idk if it's officially rolled out yet but you can do a search among accepted pull requests for what it's all about
Well, today, I've just seen a draft from one of their maintainers showing one of the things suphle was designed to do: https://twitter.com/enunomaduro/.... They can't integrate it with this pattern since php doesn't have generics, so it'll either get trashed or with plastered as some band aid. In suphle docs, I explicitly indicated the data structure/typing for that feature is a polyfill for the absence of generics
I think I can get away with it because of where I'm using it (model authorization instead of custom exceptions/throwable operations, in general, like theirs)
I don't feel as distraught as I did on finding the Premonition thingy. Am I impressed with these things dawning on them? Ffs Laravel was invented in 2011. It's incredulous to think it gave me hell for years. Waited ~2 years for me to fix all issues in a brand new framework, only to magically gain iq points and start improving their work
It's weird and brutal. If they keep figuring stuff out, it may not be long before there are no features unique to suphle. Then, my worst nightmares will come to life. I will argue there's one thing nobody will ever copy, not without rethinking the mvc architecture in its entirety.2 -
I just thought of an elegant solution to a problem in prod. Basically need to run 2 different versions of a app.
Just deploy it to uat and point it to prod db for the data.
I guess if we were following microservices architecture though we could deploy both versions to prod but we don't....3 -
i was watching a video on how whatsapp can't make enough profits coz its free and even though its a clear lie (the cartel money made by selling user data will obviously not show up in legal books), i had a thought. can any good consumer software be ever kept free for usage?
Say i made a very awesome chat app. it has 0 bugs, it does the basic tasks of sending /receiving data and media correctly and do not require any maintenence .It also is optimising a lot of cloud cost by keeping user data in their own devices and only transmitting data on triggers.
i still would require a server to keep the trigger architecture alive. and all the servers in the world are maintained by for profit corporates which will charge a premium for their services. so free products are a fallacy as someone is paying for it. it will be an investor, a different business or we the consumer (either directly as subscription , or indirectly via ads or personal data)
So i guess this realisation is going to hit soon to a lot of tiktok and insta influenza kids5 -
I started to use the MVP architecture for Android App development a year ago with a project that wasn't all that complex. My project manager liked the idea but despite that we continued with an MVC'ish approach for other projects (corporate bullshit restrictions). Yesterday I reopened the project after 8 months and Iam wondering why the fuck didn't we switch to MVP I just created 6+ screens with little to no effort including data feeding from an internal API. I would like to hear your experience with MVP architecture.
-
When you do some group programming and let yourself get led wasting an entire day into writing 6 out of ~12 tedious higher level unit tests with lots of data setup and jerry rigging, that turn out to not even test the code changes you made on a ticket that another team is depending on.
But thank you to your tech lead for helping rope you out of that stupid shit with knowledge and clout.
Unfortunately the ticket has your name on it and everybody except the goon squad probably thinks you're a retard for going down that adventure (which was not your idea or desire).
I need to learn how to articulate no this isn't worth it, the complicated monolith software architecture with many different moving parts, among many other things. -
Frustration is starting a very large data object with multiple child objects while a project wide refactor is underway changing the MVC architecture and introducing serialization.
-
In the dynamic realm of software development, where the user interface meets the complex machinery behind the scenes, Back-End Expertise https://sombrainc.com/expertise/... emerges as the unsung hero. As businesses increasingly rely on digital platforms to connect, engage, and transact with their audience, the prowess of back-end development becomes paramount.
At its core, Back-End Expertise refers to the specialized knowledge and skills required to architect, build, and maintain the server-side of applications. While the front end dazzles users with intuitive interfaces and captivating designs, the back end silently weaves the intricate tapestry that ensures seamless functionality, robust security, and optimal performance.
The Back-End Symphony: Orchestrating Digital Harmony
Imagine a symphony where each instrument plays its part to perfection, creating a harmonious melody. Similarly, in the world of software, the back end orchestrates a symphony of databases, servers, and frameworks, ensuring that data flows smoothly, operations execute seamlessly, and applications respond promptly to user commands.
Back-End Experts are the virtuosos who write the code that makes applications tick. They delve into the intricacies of databases, crafting queries that retrieve and store data efficiently. They architect server-side logic, meticulously designing algorithms that power functionalities ranging from user authentication to complex business processes.
Security as the Forte: Safeguarding the Digital Fortress
In an era where data breaches loom as potential threats, Back-End Expertise becomes a formidable fortress. These experts implement robust security measures, safeguarding sensitive information and ensuring the integrity of digital ecosystems. Encryption, authentication protocols, and secure API integrations are the tools of their trade as they create digital bastions against cyber threats.
Optimizing Performance: The Need for Speed
User experience hinges on speed, and Back-End Experts understand the importance of optimizing performance. Through efficient coding practices, load balancing, and server-side optimizations, they strive to minimize latency and ensure that applications respond swiftly, even under heavy user loads.
Future Trends: Back-End Evolution
As technology evolves, so does the landscape of back-end development. Cloud computing, serverless architectures, and microservices are shaping the future of back-end expertise. Back-End Experts must adapt to these trends, embracing new tools and methodologies to stay at the forefront of innovation.
In conclusion, Back-End Expertise is the backbone of digital experiences. While users interact with the front end, the magic unfolds behind the scenes, where Back-End Experts craft the architecture that defines the reliability, security, and performance of applications. Their alchemy transforms lines of code into seamless digital experiences, leaving an indelible mark on the ever-evolving landscape of software development.1