Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Search - "microservice"
CTO: I heard about this great architecture of microservices. What is it exactly?
Me: it's building many servers with one responsbility each.
CTO: That sound good. But let keep it simple and build only one big microservice.
(True story from a comapny a worked for)9
"Coding is solving puzzles".
I think everyone has heard that platitude. But it's not exactly right.
So I grew up in a very poor environment, a moldy building full of jobless addicts.
And in my town there was this shop where super poor parents could take their kids to borrow free toys and stuff.
So as a kid I remember being frustrated by these second hand jigsaw puzzles, because there were always a few pieces which had been teared up or chewed on, or were even completely missing.
That is what development is.
You pull in this seemingly awesome composer package, and that one super useful method is declared private, so you need to fork the whole thing.
Your coworker has built this great microservice in python, but instead of returning 404 not found, it returns 200 with json key/value saying "error": "not found".
There's a shitload of nicely designed templates for the company website, but half of them have container divs inside the components, the other half expect to be wrapped in container divs when included.
You're solving puzzles, but your peers are all brainless jigsaw-piece-chewers. They tried to mend a problem, but half way through got distracted, hungry and angry, started drooling over the task and used a hammer to fit in the remaining stuff.11
To be a good developer, you must thrive in chaos, and have an insatiable desire to turn it into order.
All user input, both work tasks and actual application input, is pure fucking chaos.
The only way to turn that input into anything usable, is to interpret, structure and categorize it, to describe the rules for transformation as adequately as you can.
Sometimes companies create semi-helpful roles to assist you with this process. Often, these people are so unaware of the delicacy of the existing chaos, that any decision they make just ripples out in waves leaving nearly irreparable confusion and destruction in its path.
So applications themselves also slowly wear down into chaos under pressure of chaotic steak-holders which never seem to be able to choose between peppercorn or bernaise sauce for their steaks.
Features are added, data is migrated between formats, rules become unclear. Is ketchup even fucking valid, as a steak sauce?
The only way to preserve an application long term, is refactoring chaos into order.
But... the ocean of chaos will never end.
You must learn to swim in it.
All you can hope to do is create little pools of clarity where new creative ideas can freely spawn.
Ideas which will no doubt end up polluting their own environment, but that's a problem for tomorrow.
So you must learn to deal with the infinite stream of perplexed reactions from those who can't attach screenshots to issue reports.
You must deflect dragging conversations from those who never quite manage to translate gut feeling into rational sentences.
You must learn to deal with the fact that in reality there are no true microservice backends. There are no clean React frontends. There are no normalized databases. Full test coverage, well-executed retrospectives, finished sprints -- they are all as real as spherical cows in a vacuum.
There is no such thing as clean code.
There is only "relatively cleaner code", and even then there are arguments as to why it would be "subjectively relatively cleaner code".
Every repository, every product, every team and every company is an amalgamation of half-implemented ideals, well-intended tug of war games, and brilliantly shattered dreams.
You will encounter fragmented shards of perfect APIs, miles of tangled barbed documentation, beheaded validator classes, bloody mangled corpses of analytical dashboards, crumbled concrete databases.
You must be able to breathe in those thick toxic clouds of rotting technical and procedural debt, look at your reflection in the locker room mirror while you struggle yourself into a hazmat suit, and think:
"Fuck yes, I was born for this job".26
My team was sharing an API key to our company's microservice containing all our customer data.
I say "was" because one team member accidentally published the key online so the security team disabled our key and won't give us a new one.
I don't know whether to laugh or cry4
Still trying to get good.
The requirements are forever shifting, and so do the applied paradigms.
I think the first layer is learning about each paradigm.
You learn 5-10 languages/technologies, get a feeling for procedural/functional/OOP programming. You mess around with some electronics engineering, write a bit of assembly. You write an ugly GTK program, an Android todo app, check how OpenGL works. You learn about relational models, about graph databases, time series storage and key value caches. You learn about networking and protocols. You void the warranty of all the devices in your house at some point. You develop preferences for languages and systems. For certain periods of time, you even become an insufferable fanboy who claims that all databases should be replaced by MongoDB, or all applications should be written in C# -- no exceptions in your mind are possible, because you found the Perfect Thing. Temporarily.
Eventually, you get to the second layer: Instead of being a champion for a single cause, you start to see patterns of applicability.
You might have grown to prefer serverless microservice architectures driven by pub/sub event busses, but realize that some MVC framework is probably more suitable for a 5-employee company. You realize that development is not just about picking the best language and best architecture -- It's about pros and cons for every situation. You start to value consistency over hard rules. You realize that even respected books about computer science can sometimes contain lies -- or represent solutions which are only applicable to "spherical cows in a vacuum".
Then you get to the third layer: Which is about orchestrating migrations between paradigms without creating a bigger mess.
Your company started with a tiny MVC webshop written in PHP. There are now 300 employees and a few million lines of code, the framework more often gets in the way than it helps, the database is terribly strained. Big rewrite? Gradual refactor? Introduce new languages within the company or stick with what people know? Educate people about paradigms which might be more suitable, but which will feel unfamiliar? What leads to a better product, someone who is experienced with PHP, or someone just learning to use Typescript?
All that theoretical knowledge about superior paradigms won't help you now -- No clean slates! You have to build a skyscraper city to replace a swamp village while keeping the economy running, together with builders who have no clue what concrete even looks like. You might think "I'll throw my superior engineering against this, no harm done if it doesn't stick", but 9 out of 10 times that will just end in a mix of concrete rubble, corpses and mud.
I think I'm somewhere between 2 and 3.
I think I have most of the important knowledge about a wide array of languages, technologies and architectures.
I think I know how to come to a conclusion about what to use in which scenario -- most of the time.
But dealing with a giant legacy mess, transforming things into something better, without creating an ugly amalgamation of old and new systems blended together into an even bigger abomination? Nah, I don't think I'm fully there yet.8
The newest *DD development trend: Panic Driven Development.
When your hexagonal domain driven serverless microservice architecture explodes in production and you don't know which one of the 50k components is failing.5
"Your XML is malformed by these strange tags CDATA. I've cleaned it up ;)"
Fuckity Fuck my boy, now the whole website is down. All hail the magic push to prod button.
(there were snippets of user provided HTML that was passed over XML by a microservice)7
So we started looking into docker. As always I needed to do the research and I was fine with it.
We have 4 projects that are sold into one suite so logically I follow the microservices build structure.
3 months later after everything has been set up, we get called into a meeting. The whole suite should be a monolith as microservices doesn't make sense to the people planning everything.
Ok pulled my current plans out abd made everything a monolith. Just note I also get pulled away to other Business Units to do work for them.
Get pulled into another meeting 2 months later. Why isn't the docker containers in microservices!? It is stupid running as a monolith and we should've done our jobs better etc...
After the meeting my manager and I just sighed and walked to the office. So basically 5 months doing the the exact same thing we did in 3 weeks.
Now they want to develop other services and want to strip every method into a microservice and bundle it together.
Life of a DevOps engineer right!1
So... Manager pulls us in. Meeting in 10 minutes guys. I know it's unplanned, but it's important.
Not only is it the 10th time he's interrupted my workflow, but it's almost time to go home. And I was getting some important shit done.
Anyways, come the meeting: we are going to abandon all the work we've done on our microservice platform (2yrs+ in the making) and make it a monolith. Oh, and we have to do it in 4 weeks, because a client is asking for it. Oh, and you'll probably have to do overtime.
I miss old times rants...So i guess, here it goes mine:
Tomorrow is the day of the first demo to our client of a "forward-looking project" which is totally fucked up, because our "Technical Quality Assurance" - basically a developer from the '90-s, who gained the position by "he is a good guy from my last company where we worked together on sum old legacy project...".
He fucked up our marvellous, loose coupling, publish/subscribe microservice architecture, which was meant to replace an old, un-maintainable enormous monolitch app. Basically we have to replace some old-ass db stored functions.
Everyone was on our side, even the sysadmins were on our side, and he just walked in the conversation, and said: No, i don't like it, 'cause it's not clear how it would even work... Make it an RPC without loose coupling with the good-old common lib pattern, which made it now (it's the 4th 2 week/sprint, and it is a dependency hell). I could go on day and night about his "awesome ideas", and all the lovely e-mails and pull request comments... But back to business
So tomorrow is the demo. The client side project manager accidentally invited EVERYONE to this, even fucking CIO, legal department, all the designers... so yeah... pretty nice couple of swallowed company...
Today was a day, when my lead colleague just simply stayed home, to be more productive, our companys project manager had to work on other prjects, and can't help, and all the 3 other prject members were thinking it is important to interrupt me frequently...
I have to install our projects which is not even had a heart beat... not even on developer machines. Ok it is not a reeeeaaally big thing, but it is 6 MS from which 2 not even building because of tight coupling fucktard bitch..., But ok, i mean, i do my best, and make it work for the first time ever... I worked like 10 ours, just on the first fucking app to build, and deploy, run on the server, connect to db and rabbit mq... 10 FUCKING HOURS!!! (sorry, i mean) and it all was about 1, i mean ONE FUCKING LINE!
Let me explain: spring boot amqp with SSL was never tested before this time. I searched everything i could tought about, what could cause "Connection reset"... Yeah... not so helpful error message... I even have to "hack" into the demo server to test the keystore-truststore at localhost... and all the fucking configs, user names, urls, everything was correct... But one fucking line was missing...
EXCEPT ONE FUCKING LINE:
spring.rabbitmq.ssl.enabled=false # Whether to enable SSL support.
This little bitch took me 6 hours to figure out...so please guys, learn from my fault and check the spring boot appendix for default application properties, if everything is correct, but it is not working...
And of course, if you want SSL then ENABLE it...
BTW i really miss those old rants from angry devs, and i hope someone will smile on my fucking torture5
"This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface." -Doug McIlroy
In today's context we can draw parallels with the microservice approach towards building software.2
Don't get me wrong, I am a huge fan of disparate microservices when building applications. I started 4 years ago building monolithic applications on symfony, laravel and .net.
SPAs have not only made my applications more flexible but have allowed me to build more flexible front ends. But I almost ALWAYS have this dirty, unorganized feeling when working in this microserviced approach.
My data always feels scattered. Because my backend doesn't have as tight of a grip on what view is being rendered and what data it gets, I am always left with a sense of disorganization and lack of security. I have more anxiety that my application will unravel at any given moment then I've ever had and that keeps me up at night.
Does anyone else feel this way when working with disparate systems?
And dont even get me started on mongo. Again, I love it, but the feeling of disorganization is intense when using it.4
Finished writing a microservice in NodeJS. Wrote tests, had clever optimisations, did profiling, the works. Lead dev says to me on a Friday evening to port my code to Java in 2 days. (Reason: to standardize everything) #FML3
Two months ago I started working at a new company, who's system is a huge monolith. The company is a bit over one year old, and the code base is huge. The desire to move to more of a microservices architecture is on the radar, but one of the biggest issues in moving towards it is how we should keep our models. The stack is basically Node.js and Mongoose, where there's about a few dozen mongoose models that the whole system uses, and the issue is that, if we moved to a microservices architecture, how could we keep the models in sync. One idea I had was to keep the models in a separate (node) package that would be shared across all microservices, but then there's the issue that if one model needs changes, all microservices that use that model will need to be updated. Another idea we had was to not share models, but instead let every microservice be in charge of everything to do with a certain type of data (eg. Users are only directly accessed by one microservice, companies by another, and no two microservices share responsibility over data), but that might bring problems when one microservice depends on a certain set of data from another microservice. How do you guys manage all that? Any ideas or tips? Thanks ^^14
Our team - if ever existed - is falling apart. Pressure raising. Release deadline probably failing. No release ready for Big Sur.
Almost seemed we were getting somewhere: More focus on code quality, unit tests, proper design, smaller classes. But somehow we now ended up in "microservice" hell; a gazillion classes, mostly tested in isolation, but together they just fail to do their job. A cheap and dirty proof of concept from March is still more capable than this pile. I really start to doubt all that "Clean code", TDD, Agility rhetorics. What does it help you, if nobody cares for the end result? It's like a month I try to hammer down that message: we have to have testable artifacts, we have to ensure code signing works, our artifact is packaged and installable, we have to give QA something they can test - but time just passes and this piece of shit software is still being killed or does nothing.
Now my knee is broken and can do no sports and are tied to my chair even more. To top it all my coffee machine broke and my internet connection was abysmal this week. Not the usual small disconnects, after which it would recover, but more annoying and enduring: often being throttled to 1.7 MB/s (ranking my connection in the slowest 7% even in Germany). My RDP sessions had compression artifacts all over the screen and a mouse click would only take effect 5 sec later.
But my Esspresso machine was just repaired. Not all hope is lost.7
I'm so sick of microservice architecture... in theory it was going to make scaling elastic and deployment easier. In reality it seems to slow productivity to a 🐌 pace.
Anyone have any brilliant suggestions on how to herd these cats in production?10
I like PHP but every new tech is about all the other languages ! Recently i was searching for microservice architecture and oh it's so easy with nodejs ! It's ready made for Go ! Java has a library build in ! What about Symfony ? Laravel ?18
For the love of GOD, if you're an architect or someone in the position where you can make drastic changes to the overarching design of a software system, if you're so keen on enforcing something "cool" just because you've read about it in a blog post/seen it on a youtube video, READ ABOUT IT THOROUGHLY, as in, pick up a fucking book or do actual research. An architect overseas just informed us that a whole legacy PHP application (a fucking monolith with a dysfunctional database, yes, I think someone demented designed it) should be rewritten to a microservice architecture (without a messaging broker, just plain API interaction through HTTP) AND WE'RE KEEPING THE DATABASE WHICH BEGS TO BE PUT DOWN FOR GOOD. So now we're gonna have a clusterfuck of tons of PHP microservices (Q_Q) which interact through plain HTTP APIs (swagger's gonna be put to a test) and all have a single broken database in the center. Talk about a microlithic design. Jesus Christ.9
When a microservice you created does not perform on prod like it performed on the load stage. There goes my week and my self-esteem 😥2
Feeling the pressure of writing back-end code for a golang microservice, after a year of fulltime front-end developement. Each commit feels like you're close to cause a bomb to detonate.
So I just saw a post on Facebook about a "Serverless Function as a Service on Kubernetes". Could someone give me eye bleach please?
I really can't wait after Scoped Storage and how goddamn slow it is, to have a "glorified microservice" take 5 minutes to spin up a container, just to do something silly like printing a hello world. In the post it was apparently showing a QR code.. that's it. A fucking QR code, because remember it's just a function.
A whole fucking operating system that goes up and back down, just to run a goddamn fucking function!!!9
Looking for a team mate for https://spot-next.io - a new fast and small OSS microservice-focussed ORM RAD framework
Here comes your millennial diagnosis of a hype word filled architecture and how its affecting me:
I was diagnosed with a mentally and socially crippling degree of OCD at a young age. As I got older and away from areas that contain hundreds of people on daily basis, my tendencies resided but still manifest themselves in lesser ways. Over the last 8 years of development, I've taught myself to steer these compulsive tendencies into the art of software architecture and code quality.
Over the last 3 years Ive become more obsessed with the concept of designing agnostic, pluggable pieces that are weighed down by very few dependencies. I had not read any books on pluggable architecture or dove deep into what SOLID means to me. It just "naturally" felt like an evolutionary step in where my software quality needed to go. I had never approached microservice architecture and at the time knew little of it so instead I went as far as breaking php or node components up into their own packages on npm/packagist. Making packages of them was as far as I could go to assure my components were entirely plug and play. It helped my mind understand them as separate entities and devs after me know that they in no way could depend on my core suite of services.
Then I ran into this "Clean architecture" book and my initial reaction through out it was "hmm, this is a much better way of achieving what I've organically been coming to". Inverted dependency was new to me. I had heard it a thousand times but never put it to practice. I approached agnostic behavior by much harder means of separating binaries into their own address spaces or combining them from different binaries to run in synchrony. The idea of pushing hard decisions off and separating concerns through interfaces was an eye opener but my it still does not solve the issue of monster repos.
I don't understand how teams allow services to grow exponentially with little check and Idk want to know. It doesnt take a principle dev with 20 years experience to say "this shits starting to get out of hand, lets split it". The minute you are forced to use your IDE's global search to work efficiently within the code base, it's too late. As silly as separating a project by npm packages sounds, it still was just a logical means of breaking up something far too complex so that it doesnt get in its own way.
Then came micro-services or my final realization of it. Ive found a perfect placement that satisfies my own compulsion for cleanliness between the principles of clean arch (or onion, or port arch) and service oriented arch. Teams work well within small codebases. They work well with low dependencies. They work well in a suite of services that can be plucked and rewritten without cascading dependencies to consider. Teams work well when given hard http specs to abide by when talking with other services or with a gateway.
Now someone tell me where in the flipping fuck I can work where these architectures are taken advantage of. Ive been through 3 companies in three years and each has been a shit show of monolithic web apps, mono api's. Shit our last suite thats now sunsetted was 600k lines of vanilla php, no framework, no orm, different approaches to architecture that did not unify, high dependencies, one repo.
The biggest thing coming out of that for me was knowing what I despise in architecture. Having these horror stories to pass on forever when discussing our bright futures. Im rambling now but I suppose that becomes the closure needed for this ted talk. Going through hell and coming out with a lesson learned. Feeding my mental disadvantages in life with best practices in my career.1
Let's see what's on the menu today:
* Web Application Catastrophe Special *
Includes, but not limited to:
- Orphaned server processes in the configuration management cluster
- Microservice back-end architecture with no API documentation
- Poorly implemented cache microservice with no documentation
- Stale data causing everything to be shown as down in production, despite everything running fine
Cost: 1 developer's sanity
Working on a project in college
we thought a messenger would be cool. (Language: C# )
We decided to use a microservice architecture, rabbitMQ, .Net Core and some frameworks like EF.
My project leader changed the whole project because he is convinced that we need bleeding edge technologies if we want to create a great product. He added over 15 frameworks like MassTransit which released 10 days ago (.Net Core support).
Now we have to put a lot of effort into learning how these frameworks work instead of implementing.
How would you handle this?4
Why do some people feel the need to prove their stupidity and utter lack of skill in the face of the world?!?!
Yesterday I learned that a sister company is hiring an intern civil engineer to code some application plugins connected to our IS ?!?!? How the fuck do you think he can only understand what the fuck we do?
To put it in context, I'm kind of the CDO of a French medium group (a little cluster of companies), as the group is in the construction industry I'm the CTO for all Computer things. Inside the group, I'm the CTO of the digital factory. So the group IS is a microservice decentralized API REST-based architecture.
Next Monday we'll have a meeting, so I can explain to them why it's a FUCKING STUPID IDEA!!!! The only good thing is that any application programming done outside of the Digital Factory will be handled as an External Company Application, so it's not my problem to secure it, debug it, or simply make it work. And they already know that I'll enforce this ruling!!!
But WHY the fuck do they still think any mother fucker can professionally program!!!!!! Every time I have to deal with them It's horrendous!!!! I had to prove them why using a not encrypted external drive for a high security mission It's stupid!!!, and why having the same password for every account is FUCKING STUPID!!!
The most ridiculous part is they have a guy who really believe he has some IT skills!! Saying things like "SVN" it's a today tool (WTF), firewall are useless, etc....
People who are too cool for old-and-busted OOP, but want to build microservices: networks of independent, encapsulated systems that look after their own data and communicate by passing messages.
Yup, that's totally not OOP you're doing there, kiddo.
I was in charge of updating the dependencies on a microservice we have. Everything was at least 3 years old. Nice. How bad was it gonna be?
Jesus Christ, I have never felt so wrong.
A certain dependency I will hide under the name w******js allowed custom functions for a feature that accepted three arguments: paramA, paramB, paramC. The old installed version, 1.x.x, did what was intended, but for some dumb reason, 2.x.x calls the function with paramA as paramB and paramB as paramA!
It took me a god damn while to find out why shit wasn't working as expected. Who thought this was a good idea?3
You've got the perfect architecture lined up, a great idea about how to transition and you're about to start refactoring. Then it hits you. You've got 10 clients running your code in production and you'll have to migrate their data to the new structures, from the blob to each new microservice datastore. For each release. FML.1
Another team didn't bother testing their code correctly, they passed us invalid values that another microservice needs, as a result they have a bunch of unstored data (that they would have seen wasn't being stored if they bothered checking the last stage of the process to see if it was stored). Now it's been so long that the data is expired. They want us to manually store millions of transactions for them.
Now tell me...if I came and shit in your bed, shouldn't I have to clean it up?
I can understand why in technical interviews they use those algorithmic questions. It's an incredibly short period of time to assess someone's coding capabilities. BUT can't we find a better way to do it? I mean, I've never implemented dijkstra in a work environment, and I had never heard of someone that faces those kind of problems in a normal day of work.
I may be judging by my limited information, but wouldn't be more useful to actually ask to solve a more plausible problem?
For example, create a microservice that implements this API, send us the GitHub link and the API url.
Just finished my first microservice project. I'm so happy that I was able to do that hardest thing I ever did. It's just a side project but I think it will do well on my cv as I will be finishing university next year.
Just wanted to share this with you guys as all the rants really helped me to calm down when I was wrecking my head over some weird bugs.4
So I'm in the process of learning go for building RESTful microservices and I have a really stupid but simple question.
Does a go microservice need more than one file? I ask this because my day job is c# and for those who know, asp.net rest projects have a lot of files6
For a new microservice we were designing, I recently had a design discussion with a team member on creating REST endpoints for a new entity. This discussion went on for almost 3 hours, most of the time was spent on why to have two endpoints for getting this resource, one is a POST using a graphQL-like query and another one is a GET using unique ID. I said, the client-side use case is different, one is a dashboard where search results need to be shown based on multiple fields and the unique ID won't be available there because it is a system generated value, second one will be used when the unique ID is present in the client as a result of previous search result. Their responses will be similar, first returns a list of entities, second returns a single entity of the same structure.
Then came the next argument: if both APIs are returning same response, why do we need two different requests ?
It was like saying, because 5+6=11, any sum of two numbers resulting in 11 should always use 5 & 6.
Are people so frustrated of working remotely all the time that they come with such weird arguments ?1
We've been working on a big application on-and-off for the last year (whenever we had time.) It was 99% working, and we left it to work on some other apps. We come back to it, only to find that some big features have magically stopped working. We dig into it and find thT some other dev team completely changed the functionality of one of the existing off-application microservices were utilizing without telling us, and then we had to spend days reverse-engineering what they did so we could retrofit our application to communicate with the microservice again.
We were able to get it fixed, but I just know that they're going to change something else in the future without telling us and it's gonna break again. A little interdepartmental communication would be greeeeaaaat!1
Colleague: I'll write a stored procedure that does fully qualified database table path names to access data from the other databases and then do business logic with all of it in the same proc.
Me: That will be 600 lashings.4
Okay... I am more than annoyed. 😵😵😵 I've been trying for days to get 2 small dockerized Spring REST-Services to communicate with each other. Without docker the services work fine with each other. I've tried many tutorials, examples, hints online and so on and none works or does something else. There is so much deprecated stuff or huge tutorials where I have to install 1000 other useless things. I kinda feel like this Microservice Stuff is a myth or I'm just stupid. Even my partner has no clue.18
Hmm. I've been wondering how I'll deploy an api based on a microservice style the smartest way... The general plan was to use salt to setup the base server and install dependencies and add the configuration.. Doing updates would be a git pull and pm2 restart api. I would love to know how you deploy your software ?1
My laptop was at 500% of course usage and ram... Wtf...
Had to stay till 1am because I was debugging a microservice system with 15 services, 3 databases and a full mail server. Freaking integration tests took all night to build and run. Don't have a remote server to run this on because the guy is sick. Arghhhhhh8
So who here likes working on a project, that utilizes microservice architecture AND has very minimal if any existing documentation anywhere? Someone has done a fine job hiding that stuff in Confluence, if it even exists there.
"Please implement this user story for us"
- "I have no idea where the API is or how it works. Best I can do is three months."1
A very long rant.. but I'm looking to share some experiences, maybe a different perspective.. huge changes at the company.
So my company is starting our microservices journey (we have a 359 retail websites at this moment)
First question was: What to build first?
The first thing we had to do was to decide what we wanted to build as our first microservice. We went looking for a microservice that can be used read only, consumers could easily implement without overhauling production software and is isolated from other processes.
We’ve ended up with building a catalog service as our first microservice. That catalog service provides consumers of the microservice information of our catalog and its most essential information about items in the catalog.
By starting with building the catalog service the team could focus on building the microservice without any time pressure. The initial functionalities of the catalog service were being created to replace existing functionality which were working fine.
Because we choose such an isolated functionality we were able to introduce the new catalog service into production step by step. Instead of replacing the search functionality of the webshops using a big-bang approach, we choose A/B split testing to measure our changes and gradually increase the load of the microservice.
Next step: Choosing a datastore
The search engine that was in production when we started this project was making user of Solr. Due to the use of Lucene it was performing very well as a search engine, but from engineering perspective it lacked some functionalities. It came short if you wanted to run it in a cluster environment, configuring it was hard and not user friendly and last but not least, development of Solr seemed to be grinded to a halt.
Elasticsearch started entering the scene as a competitor for Solr and brought interesting features. Still using Lucene, which we were happy with, it was build with clustering in mind and being provided out of the box. Managing Elasticsearch was easy since there are REST APIs for configuration and as a fallback there are YAML configurations available.
We decided to use Elasticsearch since it provides us the strengths and capabilities of Lucene with the added joy of easy configuration, clustering and a lively community driving the project.
Even bigger challenge? Which programming language will we use
What we’ve noticed during researching various languages is that almost all actions done by the catalog service will boil down to the following paradigm:
- Execute a HTTP call to fetch some JSON
- Transform JSON to a desired output
- Respond with the transformed JSON
Actions that easily can be done in a parallel and asynchronous manner and mainly consists out of transforming JSON from the source to a desired output. The programming language used for the catalog service should hold strong qualifications for those kind of actions.
Another thing to notice is that some functionalities that will be built using the catalog service will result into a high level of concurrent requests. For example the type-ahead functionality will trigger several requests to the catalog service per usage of a user.
To us, PHP and .NET at that time weren’t sufficient enough to us for building the catalog service based on the requirements we’ve set. Eventually we’ve decided to use Node.js which is better suited for the things we are looking for as described earlier. Node.js provides a non-blocking I/O model and being event driven helps us developing a high performance microservice.
The beauty of microservices and the isolation it provides, is that you can choose the best tool for that particular microservice. Not all microservices will be developed using Node.js and Elasticsearch. All kinds of combinations might arise and this is what makes the microservices architecture so flexible.
Even when Node.js or Elasticsearch turns out to be a bad choice for the catalog service it is relatively easy to switch that choice for magic ‘X’ or component ‘Z’. By focussing on creating a solid API the components that are driving that API don’t matter that much. It should do what you ask of it and when it is lacking you just replace it.
Many more headaches to come later this year ;)3
Today we finally launched Keycloak to secure our spring cloud microservice architecture!
Great feeling after 4 month of tailoring open source software, bug fixes and so much pain 😄
When you have to refactor the whole microservice to fix package structuring so it can be modular...why can't people do it correctly the first time?
We're doing single login with Azure AD for a Java-based site. We need to also sync the user changes with a microservice.
Now, here comes the fun part: Microsoft is working on a new API which looks promising, which they recommend to use as they've migrated their resources there. But this new API has SDK for a ton of languages but Java, so that's a no-no. On the other side, the js sdk for the old API is borderline unusable and has no deltas (which we need to sync users), although the new one is pretty good.
As a cherry on the cake, applications created with the old API are not transferrable to the new one, but it is otherwise. This is detailed in a very small section of their labrythinc docs and I'm really hoping that this is true or we're thoroughly screwed.
Alas, Microsoft, you've disappointed me again!2
as a seasoned systems eng myself, i had huge mental block of "i am not a programmer" whining when starting to incorperate agile/infrastructure as code for more seasoned syseng staff.
leadership made devops a role and not a practice so lots of growing pains. was finally able to win them over by asking them to look at how many 'scripts' and 'tools' they wrote to make life easier... and how much simpler and sustainable using puppet/ansible/chef/salt... and checking in all our sacred bin files and only approved 'scripts' would be pushed thru automation tool after post review.
we still are not programmers or developers, but using specific practices and source control took some time but saving us loads of time and gives us ability to actually do engineering
but just have 2 groups of younger guys that grew up wanting to be the bofh/crumudgen get off my systems types that are like not even 30... frustrating as they are the ones that should be more familiar with the shift from strictly ops to some overlap. and the devs that ask for root now that they can launch instances on aws or can launch docker containers and microservice..... ugggg. these 2 groups have never had to rack and stack servers, network gear, storage... just all magic to them because they can start 50 servers with a button click.
try to get past the iam roles, acls, facls, selinux and noshell i have been pushing. bitches.
What is a good start in go?
My wife wants to upgrade her coding skills from „I heard it at college“ to „I actually did something with it“.
I want to learn Go and start coding a bit more. My background is mostly C++ (Backend) and a bit Java (Fronted) some years ago before I went more into testing. For test automation I always use the language that makes the project happy, often Java.
We want want to join forces now, take a vacation and implement a small microservice in Go for my wife’s product (she is a PO) using pair programming.
I want to prepare that a bit. What is a good course or web tutorial to start, that some of you took and can recommend?
Thank you very much!!6
It's so frustrating going from being able to create a microservice stack prototype by yourself in a couple of months in your free time at home to having to wait 3 months at work for someone to add a build step in Team City so you can automate deploying a database project.
This seems to be a hard answer to find: with the api gateway approach to micro-servicing, Does the JWT or whatever token your working with thats sent to the gateway need to be passed to and from other services its acting on? Or are the services suppose to be behind a secure network that only the public gateway has access to ? Im trying to map out my first real take on this architectural approach and have had trouble finding support around this topic.5
"It's just as easy to create a small REST microservice as there is for a small one-off script, so let's follow the design pattern of creating a REST service first."
I don't think my manager understands how different in complexity these two things are.1
Hello fellow ranters!
So I just switched companies ( free from hell...rants if wanted) and now I'm facing a ton of new things I never worked with. Honestly I'm a little bit scared and thus I want to prepare a bit for the new stuff ahead of me. ( I'm a junior developer with only 4 years professional experience )
What are good resources to get some knowledge in the following topics, so I wont look stupid on my first day at the new company?
DevOps ( CI/CD, Kubernetes )
I know some basic stuff to describe what these words mean but no work experience at all for these. My old work was purely monolithic Java EE 😅
Resources should be in English or German.
Thanks for every help!! 🙇🏻♂️🙇🏻♂️6
Nothing like constantly having to spend 3-5 days of spin up on trying to help another team with their microservices because they have such a severe lack of documentation that I can't just follow a readme to get their projects running. Instead, I have to bug one of their developers to help me get it up and running (because they use non-standard project setups and dependency management), delaying things even more. No matter how much I scream that we need documentation no one makes it a priority.
Did I mention these are microservices written in Golang and I'm a front end developer? And I'm being made to work on back end tasks because we have a crazy high attrition rate and they won't back fill the back end positions.3
Best freelancer microservice website you use with quality programmers?
We really are desperate on that.
We request some tasks and on most of the websites we don't even get an answer.
(We tried guru, Fiverr, people per hour and freelancer.com)1
I made this: https://gitlab.com/snippets/1992288
Spare me if the title makes no sense grammatically, not a native speaker.
I'm building a microservice that keeps a check of the subscribers on my product, I wanted to keep an eye of those subscribers that may be missing the latest payment.
One way to do that is to make it work with my code and some ORM magic, but that may become unsustainable as the customer base grows, another idea that occurred to me was doing it fully inside (?) the database. The solution is to compare how many days have passed since the last payment and right now, if 'right now' is larger than 'last payment' then the subscriber is late with their next payment.
I did this as a PL/pgSQL function with an example usage accompanying that code.
Now, I just need to figure out how to use the result of those calculations in a WHERE statement...2
Why, oh god why, is our microservice component responsible for translating our output back into your obscure namespace xml format based on a Guidewire policy models? Why are we not just sending you back a Json - which is our output - and you have to figure it out into your details? Who messed this up?
I'd like to hear opinions from experienced devs/software architects... Referencing my two previous rants, the imposter's has been strong today. And I really don't know how to feel about the possible solution I've come up with... Adding the new feature as a microservice for an otherwise monolithic application 🙄 is that a sane idea?
The thing is I need to have a subscription type event-driven mechanism and since we're listening to service bus messages from another cloud provider, I apparently can't just have a serverless function to do the job, so unless there's a better option, I need a microservice with the subscription that can then invoke a serverless function to actually do what needs to be done. That's my idea, but I'm far from sure this is the best way...1
Question directed to devs who know a bit about setting up middle sized architecture.
Prestory: Joined into development of a middle sized online game. Figured they created a monolith over the last 6 years up to a point where nothing works properly and nothing can be changed without wrecking the whole system. Figured a monolithic approach isn't such a great idea.
Current Situation: In a different, same scale online game development team, game itself working but team is struggling with architecture.
My job is to come up with an approach on how to set up masterserver/matchmaking/database etc. Reading through various articles about common principles (SOLID etc.), i figured that a microservice+event-/servicebus architecture may work for that kind of project.
The idea would be to have a global interface in which microservices can be hooked. So a client registers to a client handler on startup, then starts to queue for a game, the client handler throws an event on the bus to register the user to matchmaking. The matchmaker happens to listen to those events (Observer Pattern) and adds him to matchmaking, when a match is found it throws an event on the bus to connect the user to the server, etc. One can easily imagine a banhandler throwing in a veto to cancel such an action, metrics and logging is fairly simple to add (just another service listening to all events), additionally Continuous Delivery, FRP and such are also beneficial advantages and it is said to scale well.
The question is, would you do the same, is there maybe something i might be overlooking? Do you have better ideas?
Keep in mind that we are not too experienced and are bound to different languages (python, C++ and java mostly) and are a small (4 Devs) Team with different strengths.
Thank you for your feedback and criticism!1
Learning how to build micro services using Spring Cloud. As I'm not familiar with this architecture, the company's lead engineer suggested me to do research & development on it, recommended me to follow official guide lines before involving me to the current under development project.
What's your advice regarding - what to keep in mind while learning micro service architecture (could be in one sentence)? It will be helpful to me. ^_^ Thank you.2
I love the idea behind arangodb and how fast and stable it is.
But why the fuck did they not make every possible effort to stick with SQL and build a friendlier microservice framework to make all that shit easier to use? (rhetorical question girls, no need to explain, I get it, they just rushed to butter some bread sooner rather than later)
they seem to have lost so much market share because of these two details...
Bit of a stupid oopsie I had today that someone might appreciate.
We’re working on a microservice project in Spring Boot, running in a docker swarm. Past few days I get a Spring Cloud config server going in separate stack, create an overlay network, and get CI deployments to use the right profiles etc. It’s looking great, and the first component is working spectacularly.
Now just to do the other 6. Move config files to the Git repo, tweak CI, all the other faffing and hoohas; and deploy. Health checks keep failing, the containers are murdering themselves and resurrecting ad infinitum. They’re doing this so quickly that by the time I get the container ID to exec in and curl health, it’s no longer running. Cue frustration, increased caffeine and nicotine consumption; my sanity is slipping.
No errors in the logs, because from experience the Cloud Config errors ar at debug level. Whhhyyyy?? Some time later (way longer than it should have been) I realize I had never actually included the Spring Cloud Config starter. Boot 101, get your starter!
Since config client is just additional setup in properties.yml, there’s no issue of the dep isn’t there, it just doesn’t try to get the config.
The containers are still unhealthy, I can hear them screaming. But now at least it’s about something else...
I got a task to work on during sprint planning on Monday. Felt the task was a 2hour task and procrastinated to do it on Tuesday. Tuesday came and I pushed it to Wednesday.
Today is Friday and I'm just realizing the task is a microservice on it's own. FML1
Today I submitted to code review the first iteration of a microservice done with Ramda and flow by request of my collegues. This is the first time they look at anything similar to functional code or typed js, and only one of them took the time to actually do a review.
I really like having my code reviewed and reviewing others', but please don't pester me to make a PR for a microservice you'll never look only to bail off as soon as you see something new that scares you. Buckle up and learn new stuff!
Just because the language/feamework/technology is trendy doesn't mean it is suitable for you. There is no silver bullet in software engineering.
I think it was a big mistake to use microservice architecture for our project.
So, in a microservice or web service pattern what is most important unitary test or functional test?