Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "kafka rant"
-
A lot of engineering fads go in circle.
Architecture in the 80s: Mainframe and clients.
Architecture in the 90s: Software systems connected by an ESB.
Architecture in the 2000s: Big central service and everyone connects to it for everything
Architecture in the 2010s: Decentralized microservices that communicate with queues.
Current: RabbitMQ and Kafka.
... Can't we just go back to the 90s?
I hate fads.
I hate when I have to get some data, and it's scattered on 20 different servers, and to load a fucking account page, a convoluted network of 40 apps have to be activated, some in PHP, others in JS, others on Java, that are developed by different teams, connected to different tiny ass DBs, all on huge clusters of tiny ass virtual machines that get 30% load at peak hours, 90% of which comes from serializing and parsing messages. 40 people maintaining this nightmare, that could've been just 7 people making a small monolithic system that easily handles this workload on a 4-core server with 32GB of RAM.
Tripple it, put it behind a load balancer, proper DB replication (use fucking CockroachDB if you really want survivability), and you've got zero downtime at a fraction of the cost.
Just because something's cool now, doesn't mean that everybody has to blindly follow it for fucks sake!
Same rant goes for functional vs OOP and all that crap. Going blindly with any of these is just a stupid fad, and the main reason why companies need refactoring of legacy code.12 -
For all the hate that Java gets, this *not rant* is to appreciate the Spring Boot/Cloud & Netty for without them I would not be half as productive as I am at my job.
Just to highlight a few of these life savers:
- Spring security: many features but I will just mention robust authorization out of the box
- Netflix Feign & Hystrix: easy circuit breaking & fallback pattern.
- Spring Data: consistent data access patterns & out of the box functionality regardless of the data source: eg relational & document dbs, redis etc with managed offerings integrations as well. The abstraction here is something to marvel at.
- Spring Boot Actuator: Out of the box health checks that check all integrations: Db, Redis, Mail,Disk, RabbitMQ etc which are crucial for Kubernetes readiness/liveness health checks.
- Spring Cloud Stream: Another abstraction for the messaging layer that decouples application logic from the binder ie could be kafka, rabbitmq etc
- SpringFox Swagger - Fantastic swagger documentation integration that allows always up to date API docs via annotations that can be converted to a swagger.yml if need be.
- Last but not least - Netty: Implementing secure non-blocking network applications is not trivial. This framework has made it easier for us to implement a protocol server on top of UDP using Java & all the support that comes with Spring.
For these & many more am grateful for Java & the big big community of devs that love & support it. -
Kafka lead after a perf test - we have amazing performance, processing 43 records a second😄
Me - WTF!!!! 🤬5 -
Why are some people incapable of reading documentation? THE "DUPLICATE RECORDS" IN OUR KAFKA TOPIC ARE BECAUSE ITS AN EVENT STREAM AND NOT A DATABASE. THIS IS LITERALLY ON THE FIRST PAGE OF THE GUIDE YOU ABSOLUTE MORONS.3
-
Today our customer tried to change the log dir of Kafka. Too bad he did not change the intended log4j.log, instead he changed the log.dir where kafka writes his data.
He basically tried to change the tires at 180km/h at the Autobahn. Only god knows how he managed to get the tires off. After that he slided a kilometer on his rims and wondered why the car wouldn't drive anymore.
Consumers fucked, Topics fucked and Producers fucked