2

Machine learning algorithm with 20 threads. Machine is like: "bitch please" (and runs the algo clean until the end)

Increase parameter to 40 threads. NullPointerExceptions. NullPointerExceptions everywhere.

Comments
  • 1
    How many cores does the machine it's running on have? 40 threads seems a bit much, you may be better off using a hand full of threads in a thread pool layout and feed it with a work queue.
  • 0
    @hexc is right.

    Large number of threads does not improve performance. If you exceed the number of cores, you will degrade performance.
  • 0
    I have 8 cpu cores. It did manage to run with 40 threads for specific and lighter datasets. I think there are some methods I have synchronize. Or maybe I changed aomwthing in the code by accident in between these runs.
  • 0
    That could be but remember, if you have more threads than cores you usually get worse performance. The one exception being a Hyperthreaded processor. With one of those, you usually can get away with a bit more but I'd recommend benchmarking the code to find the sweet spot for the workload.
Add Comment