After our Head Of Software has terminated.

I started to take control over our development crew. And in this year I did more then the old head in the last 6 years.
- Swapped from plain old SVN to Gitlab.
- Build a complete autonomous deployment with Gitlab.
- Introduced code reviews.
- Started to refactor the legacy product with 500.000 lines of code...
- learned how to use confluent apache kafka and kubernetes to split the legacy project in many small and maintainable one.(not done yet)
- Last 3 weeks I learned how to use elasticstack with kibana and co. That we aren't blind anymore. Big dashboards are now shown in the middle of the room :) and maybe convinced my coworker that we use unity3d for our business application cause of support for all devices and same design on them. And offline capabilities. (Don't know if this was my best idea)
When I look back, I'm proud to did that much in one year alone. And my coworkers are happy too that they have less work with deployments and everything.
But I can't decide what's the title for this. System or Software Architect cause I litterallity did both :/

I was looking for a part time job when I was 15, knew nothing about coding, got a developer job cause I said I'm keen to learn xD .. 5 years later.. I'm still learning but I have a better knowledge now!

First of all, I hate crammers so much. These people kill the industry without even understanding it. They turned interviews into exams, missed the point of hiring, and saw no distinction between knowledge and information all the time. They don't understand that if you can google an answer in five seconds, it's not knowledge. It's information.

They don't understand that questions like 'what will Python do if you delete an item from a dict while iterating over it' are complete nonsense. They don't understand that it's not 'dig deep'; it's just a bad practice that leads to errors, thus must be avoided. The fact of remembering 'RuntimeError: dictionary changed size during iteration' means that you haven't been avoiding it enough.

One more example. Which signature is correct?
- ApplicationListener<ContextRefreshedEvent>
- ApplicationListener<ContextRefreshEvent>
- ApplicationListener<RefreshedEvent>
- ApplicationListener<RefreshEvent>

Second. What's the point of forcing you to write compilable code in google docs? Do they really expect that one could possibly remember 'import org.springframework.beans.factory.annotation.Autowired;'? Seriously?

Third. Why do they expect me to know Spark, Java, J2EE, Spring Boot, Python, Kafka, Postgres, React/Redux, TypeScript, and work for miserable 70K EUR?

What's wrong with the European IT job market? Are they fucking nuts?

Every day I feel like Dr. Leonard McCoy.

"I'm an automation expert, not a ..."

* DBA
* ElasticSearch administrator
* Kafka Architect
* Email marketing specialist
* SEO Analyst
* Front end developer
* Doctor
* Project manager
* Accountant

The wife is asleep. The two year old wrecking ball of a boy is asleep. Wanted to finish a library to run kafka/grpc for nodejs microservices with plug and play functionality.

Make some tea. Figured the caffeine will keep me up. Maybe work late night. It'll be fun.

Then diarrhea hits. Now, staying up late because apparently I ate something I shouldn't have. Don't you hate it when that happens?

Excited for the following upcoming product releases:

Dark Mode GitHub
Dark Mode Jira
Dark Mode Kubernetes
Dark Mode Kafka Streams
Dark Mode WebEx

A lot of engineering fads go in circle.

Architecture in the 80s: Mainframe and clients.

Architecture in the 90s: Software systems connected by an ESB.

Architecture in the 2000s: Big central service and everyone connects to it for everything

Architecture in the 2010s: Decentralized microservices that communicate with queues.

Current: RabbitMQ and Kafka.

... Can't we just go back to the 90s?

I hate fads.
I hate when I have to get some data, and it's scattered on 20 different servers, and to load a fucking account page, a convoluted network of 40 apps have to be activated, some in PHP, others in JS, others on Java, that are developed by different teams, connected to different tiny ass DBs, all on huge clusters of tiny ass virtual machines that get 30% load at peak hours, 90% of which comes from serializing and parsing messages. 40 people maintaining this nightmare, that could've been just 7 people making a small monolithic system that easily handles this workload on a 4-core server with 32GB of RAM.
Tripple it, put it behind a load balancer, proper DB replication (use fucking CockroachDB if you really want survivability), and you've got zero downtime at a fraction of the cost.

Just because something's cool now, doesn't mean that everybody has to blindly follow it for fucks sake!

Same rant goes for functional vs OOP and all that crap. Going blindly with any of these is just a stupid fad, and the main reason why companies need refactoring of legacy code.

Serbia. $600/month for

- full stack
- angular dev
- java spring boot backend dev
- jenkins
- ci/cd pipelines
- jira
- unit integration E2E tests
- kubernetes
- docker
- graphql
- postgres
- sql queries
- aws
- microservices
- deployments
- scala
- kafka
- maven/gradle
- bsc or msc cs degree
- in depth knowledge of
-- observables
-- design patterns
-- jwt and how it works
-- ssl certificates
-- solid principles

There is more but i forgot the rest

I shit you not. This this a job qualifications qualifications entry level on LinkedIn.

7+ years working as part of a development team and with the following technologies:
Node.js Typescript and Java-based, microservice-driven applications using Spring Boot or similar framework
RESTful API design / microservice architectures
MongoDB or any other NoSQL DB
Message queues e.g. RabbitMQ, Kafka etc.
Modern MV*(MVC, MVVM, etc..) frameworks e.g. React, Angular, Vue etc.
JavaScript and design patterns, CSS and HTML
Modern CSS and view libraries e.g. RxJS, Angular Material, Typescript, JS ES6 etc.
Unit and UI testing using third party tools e.g. Jest, Cucumber, Groovy & Spock, etc.
Bachelor's degree in computer science or related field

Fun day, lots of relief and catharsis!

Client I was wanting to fire has apparently decided that the long term support contract I knew was bullshit from go will instead be handled by IBM India and it's my job to train them in the "application." Having worked with this team (the majority of whom have been out of university for less than a year), I can say categorically that the best of them can barely manage to copy and paste jQuery examples from SO, so best of fucking luck.

I said, "great!," since I'd been planning on quitting anyways. I even handed them an SOW stating I would train them for 2 days on the application's design and structure, and included a rider they dutifully signed that stated, "design and structure will cover what is needed to maintain the application long term in terms of its basic routing, layout and any 'pages' that we have written for this application. The client acknowledges that 3rd party (non-[us]) documentation is available for the technologies used, but not written by [us], effective support of those platforms will devolve to their respective vendors on expiry of the current support contract."

Contract in hand, and client being too dumb to realize that their severing of the maintenance agreement voids their support contract, I can safely share what's not contractually covered:

- ReactiveX
- Stream based programming
- Angular 9
- Any of the APIs
- Dotnet core
- Purescript
- Kafka
- Spark
- Scala
- Redis
- K8s
- Postgres
- Mongo
- RabbitMQ
- Cassandra
- Cake
- pretty much anything not in a commit

I'm a little giddy just thinking about the massive world of hurt they've created for themselves. Couldn't have happened to nicer assholes.

soo.. yeah.. I've just solved an annoying bug using only chatgpt.

My first commit in this new project. And it's based on chatgpt.

Literally just saved me from days of reading through kafka docs, auth mechanisms and other stuff. And no, the google did not provide me with a proper answer/hints. The only hint was "the configuration might be wrong". Well alright, but I was NOT using any configuration in the first place...

Fun times ahead :) I might even consider the pro version if it keeps delivering like that.

I work in a contract position and reviewed the code of a senior engineer recently. Regretfully I can't provide context to preserve anonymity.

He wrote awful JavaScript;

- handled a single DOM element with 2 different frontend libraries
- used the logical operator && to 'chain' two methods (it didn't work) instead of returning a boolean value,
- broke everything down into minute detail (a comment box had 7 components!),
- API calls were made for every component update instead of maintaining local component state where it made sense, which meant UI updates were slow,
- animated EVERYTHING, which made my Firefox on Xubuntu i7 64bit with 16GB RAM beg for mercy.

I had a rough couple of months with interviews, with 2nd stage technical interviewers throwing impossible tasks at me.

Example:

1. Create an online Python code editor with Javascript which can compile Python bytecode,
2. Use Mesos and Kafka to create real time architecture for Tensorflow with a Javascript frontend in 1 day. (I asked, and wasn't allowed to use Kubernetes or serverless architecture),
3. Hack a website from the browser's address bar using parameters ( what?!! ),

Obviously, the next time I meet a 'senior', I'm going to tell him talk is cheap;

'SHOW ME YOUR CODE.'

Pet peeve #91847 - when your non technical manager routinely forwards you articles about technical subjects, usually written by non technical idiots, and says "please see if this is something we should be using".

Yeah, I get that your business manager friend has heard Blockchain is amazing, Rsocket is revolutionary, and everyone should now be using Kafka, but none of that makes any sense for our use case.

The clincher had to be telling me to look at AWS groundstation though. And no, we don't have anything to do with satellites...

Inspired by @Billgates

everyone around is hyped about new tech they get to use, new toys to tinker with, I can see their eyes shining when they hear "let's try and introduce kafka" - they would wiggle their tails all day long if they had ones!

And me? Well, a new potential employer got me so excited I couldn't wash a smile off my face for a few days! You know what they said? "we don't use any frameworks, we focus on clean code, solid, kiss and we write with tdd". Bare java - that's the best position I've heard of in years!

I guess I'm oldschool. But I truly believe their approach is the right one. Not trashing the code with spring [which is turning into smth what systemd is for linux/unix], hibernate and what not.

Just good old java code. Db, multithreading, request-mapping -- all plain, manual and simple.

Amazing!

just learned that Kafka exists.
I was almost going to build an event bus from scratch, phew

just happend to me today

It's a job description for an entry level Software Developer role at Apple.

Basically they want
- A Front end dev
- A back end dev
- An iOS expert
- A Gimp/photoshop designer
- A streaming data expert
- An Oracle/Teradata expert
- A Kafka expert ( Not shown in the picture)

And on top of that the guy should be able to learn new tech faster.

Do they want a developer or a fucking terminator?

Daghhhhhhhh Kafka.

Set it up, seems to work fine.

Oh no...! Take a broker down, then messages go missing - hmm, that's not right. Fine, I'll just look into... Ah, bad replication factor, my fault. So then it's all fixed! Woop. Wait, no. Some messages still going missing occasionally. Oh, only set to "at most once" delivery. My bad, fix that, and... now everything is out of order. Oh, ok, partitions setup wrongly. Wtf, now the whole thing stalls when there's a network blip until a restart. Right, ok, looks like commits have to receive acks in the library I'm using before continuing. Switch to a library that uses CommitWithoutReply. Brilliant....

Apart from said library seems to have commits failing all over the place because it keeps trying to commit during a rebalance 🙄😒😤

The frustrating thing is I KNOW for a fact that Kafka is a fault tolerant, resilient, horizontally scalable thing capable of handling stupid amounts more than I'm throwing at it without missing a beat. But damn,configuring it, and checking you've configured it sanely is a royal, monumental PITA.

Regions....

!rant

We were talking about Kafka's "metamorphosis" and a friend said "what if the parents were programmers and did not want to have a bug in the family?"

If you plan on using Apache Kafka for whatever:
Don't do it. You are on the gates of hell.

If you are using Apache Kafka:
I feel sorry for you.

If you decided not to use Apache Kafka:
You chose wisely.

When you hear the data engineers talk about ambari, storm, sqoop, kafka & phoenix and you are not sure if they are talking about frameworks or pokemons..

Working with acutal BigData. Will be ''promoted'' to a new team where I will work on a system wrote in php+mysql with literal millions of requests and database rows. We are currently seeing server crashes around once a week on peak usage. Stack is a vps 64gb ram server + i dont know how many cpu/ cores. Apache, php, mysql.Best ways to optimise and adapt in this case? Kafka? Rabbitmq?ngnix? More hardware?

A 2d simulation of the mars rover curiosity and its commandControl station. Pure java. The two components talk to each other via apache-kafka. Rover has its own operating system (kernel) and resource management. Hooked up some sensors to NASA API.

Accounting software- and it’s even not the developers fault but because law is changing so often these days they barely keep up.

So I run sole business and visit my accountant from time to time to chat and give my documents.

Sometimes I also help with accounting software like finding why it’s doesn’t sending this crap and doing what it’s expected to do only in some cases.

It usually takes an hour to find out why something doesn’t work.

Also once I was sending some companies fiscal year summary documents cause no one was able to figure out how to sign those documents and how to fill the form so it’s accepted by the “system”.

Based on how I see bureaucracy is increasing cause of technology instead of decreasing and how stupid are those protocols that are required for sending some financial documents over internet. Seeing that those protocols are changing every year if not half a year and software to send those documents mostly doesn’t work.

I’d say any accounting software is temple of doom.
It’s classic “The trial” by Kafka.

For all the hate that Java gets, this *not rant* is to appreciate the Spring Boot/Cloud & Netty for without them I would not be half as productive as I am at my job.

Just to highlight a few of these life savers:
- Spring security: many features but I will just mention robust authorization out of the box
- Netflix Feign & Hystrix: easy circuit breaking & fallback pattern.
- Spring Data: consistent data access patterns & out of the box functionality regardless of the data source: eg relational & document dbs, redis etc with managed offerings integrations as well. The abstraction here is something to marvel at.
- Spring Boot Actuator: Out of the box health checks that check all integrations: Db, Redis, Mail,Disk, RabbitMQ etc which are crucial for Kubernetes readiness/liveness health checks.
- Spring Cloud Stream: Another abstraction for the messaging layer that decouples application logic from the binder ie could be kafka, rabbitmq etc
- SpringFox Swagger - Fantastic swagger documentation integration that allows always up to date API docs via annotations that can be converted to a swagger.yml if need be.
- Last but not least - Netty: Implementing secure non-blocking network applications is not trivial. This framework has made it easier for us to implement a protocol server on top of UDP using Java & all the support that comes with Spring.

For these & many more am grateful for Java & the big big community of devs that love & support it.

In a time where a web dev is expected to know, well.. everything... Backend -JAVA, python, nodejs and C++ would be great.
Front- angular, react, other 10 libs
DBs -sql, mongo, redis, elastic, kafka, rebbitmq
Also be devops on the side with AWS and docker kubernetis and more stuff

How the f is that possible?

In my real job for the last couple of years and different companies, I usually use 1 language/framework & 1 main DB.. and although it's possible in some companies, but in mine, ppl dont get access to AWS etc..

So let's say there's me.. a server side dev for years.
So I decide to be better and learn Golang.. cool lang, never needed in my job, after few days of not using it I forgot all I learned and that was it.

Then I realized I gotta know some frontend cause everyone want a fullstack ninja nowadays.. so I tried Vuejs.. it was amazing .. never got to use it at work, cause i was a backend, and we didnt use frameworks on our products back then..
Also forgotten.

Then I decided to learned nodejs, because this is the coolest thing ever.. hated it, but whatever... Never got to use it at work, cause everything was written in other lang which the whole team knew... Forgot the little i knew.

Then I decided, its time to see what Angular is, cause everyone started using it... similar idea to vuejs which i barely remembered, but wow it's a lot of code to remember, or I'll have to google everything.. so I went over it, but can't say i even learned it.

Now Im trying to move on to python, which, I really am learning in depth.. however, since I dont have real experience with it, no one gives me a shot at being a python dev, so again i feel like I'm trying to memorize syntax and wasting my time..

Tired of seeing React in all job ads, i decided to have a look what's that all about.. and whadoyaknow... It's fucking the same idea as vue/angular with again different syntax..

THIS IS CRAZY!

in how many syntaxes do i need to know how to make a fucking crud api, and a page with same fucking post form, TO BE A GOOD PROGRAMMER?!?

Why are some people incapable of reading documentation? THE "DUPLICATE RECORDS" IN OUR KAFKA TOPIC ARE BECAUSE ITS AN EVENT STREAM AND NOT A DATABASE. THIS IS LITERALLY ON THE FIRST PAGE OF THE GUIDE YOU ABSOLUTE MORONS.

Kafka lead after a perf test - we have amazing performance, processing 43 records a second😄
Me - WTF!!!! 🤬

How many sh*t days does it need to make me down?
3 ...

I hate my company, for making everything overcomplicated and annoying.... I have to discuss with 3 peoples for 3 days to getting some gitlab premium licenses (20$ per month for 10 licenses)... Why do you need it? Why we can't use the free version? Why Why Why... It's not enough to tell them it will save us much times and improves the quality of development.....

Also I wanted to ask if we can to Jaxb or another Dev Conference this year... Then I got the information that we have about 2000 Euro for 10 people for training.......... What should we do if everyone buys a book this budget is out .... f*ck company....

Second day, half of the day was taken for fixing the live db on the fly cause of a bad structure of tables... at least fixed some other inconsistence too... later the day fixed a freaking shitty bug with Spring Devtools and 2 Classloader to make the product that I'm presenting in 2 days running.

Today next shitty day with discussion that everything I did last half year (introducing Microservices, Kubernetes, Kafka and other DevOps things) could be maybe useless when the external company will say that they use another ecosystem -..- for their microservices...

Someone looking for a disappointed java developer? I just want to develop the best product ever... I'm happy with every area... Frontend, Backend, DevOps, Fullstack, Architect in some kinds depends on the wishes and technologies.

Being a Tech Co-Founder is both boon and a bane.

Boon: You know how to build it.

Bane: You just start building without thinking.

We often crave I will be using Redis😍😍, Kafka, Load Balancer, Muti Layer Neural Net

"Dude", Whatever you are building does anyone want it or not. Tell me that first.

A**hole of a "Technical" Delivery Manager, who has changed orgs after at-most every 14 months, and says she can't read code or even understand technical things like Kafka.
It's been 12 months she's joined here, I bet she's again gonna jump to somewhere. Why the hell do they give these roles if there's no relevance.
Only thing she knows is to blame/assign anything to anyone without giving a f**king read.
Any random thread, she'd just ++ and say, ensure this is done on priority. There are 7 priorities already assigned, atleast reply when I ask you on mail to arrange your priorities in order so that you won't say we messed up. We've no issues ticking one thing at a time from the top.
Atleast do something God damnit!

I realized something. No matter what tech i use to code a project there will always be a dev to take a shit on it

someone will recommend to use redis, after i use redis some other dev will trash me for having such a poor choice and recommend me socket.io

Then if i use socket.io some 3rd dev will trash me cause thats not the right way of building stuff and recommend me kafka

If i use kafka some 4th dev will trash me and say why i dont use angular

If I use angular 5th dev will trash me for not using react

If i use react 6th dev will trash me for not using nextjs

Tired of this bullshit

I'll use whatever tech i need. If i dont know what to use ill ask and take the first suggestion. I'll just build a saas and when it starts earning money ill pay other devs to refactor and scale the hell hole (which wont be cause i write good code following solid principles and not spaghetti). Much simpler solution than wrecking my head with decisions of tech stack

Well I was not actually in this meeting but I had to prepare everything so the CEO could show the new software to the employees and when I saw him clicking on stuff and it wasn't working... I was like: it works on my machine! After a little while I found out that I didnt push it to demo so it was still not updated..
Oh that embarrassing moment when I had to go to the room and say: refresh the page please!!

Development: we need Nginx installed on *insert server list*
Me: ok, let me get in tough with the platform team.
Platform team: This should be installed in the userspace, Unix teams don't support this.

And here I am, trying to get a reverse proxy running on servers on which I do not have sudo rights.
Since it doesn't work, it's my fault, both sides block the door.
I installed it locally on a virtual machine, but the compiled or installed code doesn't work once copied.
The joy of being an "application engineer". This job title means nothing!

Today our customer tried to change the log dir of Kafka. Too bad he did not change the intended log4j.log, instead he changed the log.dir where kafka writes his data.

He basically tried to change the tires at 180km/h at the Autobahn. Only god knows how he managed to get the tires off. After that he slided a kilometer on his rims and wondered why the car wouldn't drive anymore.

Consumers fucked, Topics fucked and Producers fucked

The senior engineer on my project is working with Kafka. Completely unaware of the possibility of rescheduling failed messages with a fixed delay he was trying to put a Thread.sleep somewhere in the consumer to emulate the feature.

Sometime i would like to burst out crying because I feel like I'm the only one who care about writing good code and using best practices.

The more in the industry the more I realise titles don't matters. Everything is shit, everything...

Scala is as horrible as Java. Been using Java at uni and once having discovered the simplicity and beauty of other languages (Python, golang), never went back.

Currently trying some apache projects (kafka streams, apache flink) where Scala is native. Same crap as Java. Needs 10x lines to write the same thing, abstraction over abstraction, and intuitive = 0. Why tf did it even got invented?

i just wanted to say:

FUCK HADOOP
FUCK KAFKA
FUCK JBOD
FUCK KERBEROS

that is all

fuck apache kafka this worthless piece of shit never fucking works >:(

Continuing to learn k8s ecosystem and to achieve acceptable level
With trying eventually Helm, Argo CD and even trying to use not managed setup for k8s.

Going though books to find out theory about being SRE.
And about data intensive apps.

Learning and trying Kafka

Learning and trying FastAPI and diving in generally to async python ecosystem

Learning Go.

Learning few more books to increase code quality and its compositioning.

Getting more practice in monitoring and logging systems with applicating them to k8s.

What is this format? Spaces are separated by + sign and columns are separated by & sign and it’s value is followed by = sign. Can I directly convert it to Spark dataframe?

- Finish "Introduction to algorithms"
- Learn some genetic algorithms
- Get my hands dirty on reinforcement learning
- Learn more about data streaming application (My currently app is still using plain stupid REST to transport image). I don't know, maybe Kafka and RabbitMQ.
- Learn to implement some distributed system prototypes to get fitter at this topic. There must be more than REST for communicating between components.
- Implementing a searching module for my app with elastic search.
- Employ redis at sometime for background tasks.
- Get my handy dirty on some operating system concepts (Interprocess Communication, I am looking at you)
- Take a look at Assembly (I dont want to do much with Assembly, maybe just want to implement one or two programs to know how things work)
- Learn a bit of parallel computing with CUDA to know what the hell Tensorflow is doing with my graphic card.
- Maybe finishing my first research paper
- Pass my electrical engineering exam (I suck at EE)

My interview...!

From civil background, not sure about internal system architectures.

Question: how does cpu switches between processes ?

Ans: how the f I know? Do I really need to know, internally!

Explained a bit.

He was like you are the one who are going to use kafka, not the one who are going to write kafka.

I was like devasted.

ONE WEEK i've been working on this migration from Mongo to Kafka for a service that will probably handle 5 requests A MONTH for this thing that launch an entire JVM and take 10 seconds just to create a so-called topic

I am working on distributed message stream/message bus system. Somewhat similar to Kafka but uses Memcached protocol, so you can use any memcached client already out there to speak with it.

Question:

For a real time chat (web) app. Whats the best technology to use for this? I dont want the chats to have delays or glitches (in case it gets sent but not delivered etc).

Backend stack is java spring boot

How about kafka? Or rabbitmq? Or socket.io? Should i use redis? Should i use AWS SQS? Talking about cloud what AWS tools should i be using to handle this the best way?

Note: it must be scalable. Meaning if i wanted to extend this software more by building a mobile app (aside from web), i should be able to use the same backend easily

Note 2: it is not ONLY a chat app. Chatting is just 1 out of many functionalities. So chatting is not the main component of the project just a side thing

Keep in mind the backend is a microservice architecture etc. Database postgresql.

I’ve been at this issue at work for four days now and no progress and I feel really bad because we have important stories to pick up and I feel I’m wasting my capacity like this because I haven’t fixed it. Basically, only in our QA environment (one before production) our services is not acknowledging duplicate events posted by Kafka, thus keeps reprocessing them. I’ve spent so long trying to diagnose the code, which is the same in all envs currently, seeing how this suddenly occurred, restarted things, went through complications of using different tools, asked for help from others a lot but IVE gotten NOWHERE. Idr wanna say to my team that I should prioritise other things because we have deadlines but I feel this issue is important to fix but I just can’t figure out how. Now I’m worried this whole sprint will go without me doing anything and then fingers pointed at me later

I am using python scripts for mathematical computations and deep learning wheres spring to build the server and now want to handle the scripts from spring guess what scripts were in python 3.6 and fucking jython does not support it.. so tried finding the different libs like jpy , jpe and analysisRPC.. Everything seems just bullshit .
Finally using Kafka to have my interprocess communication..

Originally I'm coming from Java , about 2 years ago, I switched Node with TypeScript and had a hard time getting accustomed to Promises. It was a big relief when I learned about async/await. Much cleaner code, no brainfuck anymore when thinking about how to handle stuff that requires multiple async values and so on.

Now I'm working on a clients project as a Java dev again. SOA, Spring Framework, Kafka and MongoDB, nothing too complicated... if they wouldn't use reactor to bring reactive functionalities to Java.

It feels like I'm back in Promise Hell...

Anyone have experience / opinions on Kafka? We currently don't have a messaging system like this, and I'm thinking it would be the best choice for things like providing webhooks because its persistent nature, instead of having a "synchronous" message queue that I push into and hope it gets handled. Might also want to store attempted webhook calls in a db but I don't know if that's overkill and could be done with just transforming messages (if they fail) and trying to handle them again.

I NEED HELP with Kafka

I'm working a thesis. I developed 4 different microservices (REST APIs). I would like to use Kafka to support large number of users. I may also place the microservices behind a HAPI Proxy. How can I use Kafka to stream requests and respond accordingly. I'm using Node.js. I think I haven't grasped Kafka. My Prof, suggested I try it to act as a broker but I'm blank right now. How do I tell Kafka I want it do a POST or GET etc?

My team is currently using a confluent cloud kafka architecture via protobuf and wordpress as website.
We realized this is quite impractical when you want login functionality and such and would like to make a page from scratch now.
problem is we are all java backend developers with minor php knowledge, and php doesnt work well with protobuf or kafka.
how would you get such a website started. what language and frameworks are easy to get going?

The day I'm fucking around with Kafka Streams API and had to work with Java after swearing it off 5 years ago for JS bae...

Is the day that Apache's everything is down. Can't even hit their fucking website. I just want a fat jar. Just one.

All the microservices are planning to communicate via kafka. And they are planning to put a REST API between kafka and the service. So all services hit the REST API with the events and the REST API send events to kafka.

I don't see any benefit of having a REST API between the service and kafka. What are the opinions in the wild?

Can someone send me a guide article or tutorial how to build a chat app with kafka/redis in spring boot?

Should kafka or redis be used?

Or something else?

Also in a chat app should i store messages in postgres or not?

I cant find 1 fucking course or guide how to PROPERLY build a chat app

My Spark Program to process changes from Kafka topic to DB table is working end-to-end. But when it is manually terminated when an insert/update/delete is in progress, the whole tables data gets dropped. How can I handle this in a safe way?

"Overengineering is terrible".
The project's owner wants to employ Kafka in the project, however, there are only a few hundred simultaneous connections. The programming team questioned, "Is it Overengineering?"
"It's that terrible," the project's owner said.

I'm looking into an app for work. It needs to connect to both a mongo and Kafka server using SSL or something, but different keys basically.

So need to load a jks file in the main() I think except I have 2 files.

I don't remember exactly how it works but I guess need to combine them into 1?

How do I load each though in app? Need to tell mongo, kafkaClient with one to use?