I’ve been at this issue at work for four days now and no progress and I feel really bad because we have important stories to pick up and I feel I’m wasting my capacity like this because I haven’t fixed it. Basically, only in our QA environment (one before production) our services is not acknowledging duplicate events posted by Kafka, thus keeps reprocessing them. I’ve spent so long trying to diagnose the code, which is the same in all envs currently, seeing how this suddenly occurred, restarted things, went through complications of using different tools, asked for help from others a lot but IVE gotten NOWHERE. Idr wanna say to my team that I should prioritise other things because we have deadlines but I feel this issue is important to fix but I just can’t figure out how. Now I’m worried this whole sprint will go without me doing anything and then fingers pointed at me later

  • 1
    Hey are using unique identifiers or any hashing of attributes for the messages in Kafka? Just trying to help haha
  • 0
    @duckycode aye were using unique identifiers that determine if an event has been seen before, now here’s the issue, the logs show that it’s recognised it as a duplicate and won’t do any more processing, BUT the dB shows it has....... and logs are perfect in the other environments
  • 2
    @pandasama oh wow yeah the issue sounds complicated, what I usually do in those types of situations is that I invert the question... sounds weird I know but bear with me, instead of asking how Do I fix X? ask yourself How do I screw up X? list all your answers and check everything.

    In your situation it might go like:

    How Do I create duplicates in the system:

    Maybe I can connect the database to the wrong environment (then you check if that’s the case)

    Perhaps I can create false positives in the logging system.

    I hope it helps even a bit dude!
  • 1
    @duckycode thank you ducky! That’s exactly how I’ve grown to do things too haha, I’ve checked those details too but sadly to no avail, the duplicates were intentional due to some chaos testing going on, and it’s fine to have duplicates, they just need to be ignored but aren’t even though the logs say they are ; _ ; and this works fine in other envs which is more annoying, I think anyone senior would be wasting their time on this so it’s just me for this but there’s really no clear solution to this, thanks got the tips though! Glad to know I was atleast thinking the right way :)
  • 2
    pat yourself in the back @pandasama It’s incredible how many times some like you goes further than anyone else and later fixes issues like no one else because they care, good job 👏
  • 0
    @duckycode thanks for the motivation man, I just hope I can be that guy you just described there XD
Add Comment