133
crafter
3y

I freaking love optimizing multithreaded applications.

Comments
  • 33
    @s0LA merging multiple csv files with about 70 million rows.
  • 67
    Plot twist: they're all spending 100% CPU to wait for other threads to finish
  • 11
    @s0LA well thanks to this new machine I'm now working on its only a matter of three minutes. The old application was written in one giant sql stored procedure doing the same in 8 to 9 hours
  • 3
    @jespersh well this would be funny but no the only thread wich is waiting is the main thread.
  • 4
    I loved making a multi-threaded program with C.

    @jespersh how would it show up like this? If the check for a mutex lock is constantly looped or something else stupid?
  • 3
    @electrineer Jup it's called a spin lock. Spin locks are used when the context switch from muticies takes to long.
  • 1
    @electrineer what he said. It is the wrong type of wait
  • 3
    @Chubak it's exciting and horrifically easy to screw up. Lots of benchmark testing is needed.
  • 3
    WHAT THE FUCK ARE YOU DOING? PROTEIN FOLDING?M
  • 0
    @crafter: Oh, boy. I know that struggle.
  • 1
    @hyperlisk importing and combining 14 gigs of csv files
  • 0
    @iKameo jep the ssd raid is to slow so 20 gigs are allocated as a ram disk where the source file is stored.
  • 2
    For those that don't know, C# doesn't take 1byte for 1 string character. And also an 8byte object overhead and some 4byte length overhead and 2byte null overhead

    So if you have a csv file with many small values, then the memory usage quickly rises.
    If you need a substring, then there's an additional allocation. Unless you use *Span<T>.

    It is more complicated than this, but hopefully it helps some here
  • 1
    Sweet rig.
  • 1
    What the fucking fuck is that?
  • 1
    @austudo Hardware whise, dual socket Intel Xeon e5 2620V4 Broadwell 8c16t CPUs so 2x 8c16t making it 16c32t.

    Sweet rig.
  • 0
    @RageBone asking about why is the cpu so loaded
  • 1
    @austudo because the code is well optimized
  • 0
    @electrineer oh wow I guess you could save the unoptimized code and have a fine firestarter
  • 0
    @austudo it wouldn't use the resources as optimally
  • 0
    @electrineer but it would be optimal to save your life
  • 1
    Looks like a cure to cancer
  • 0
    About to start multi-threading in a game engine that doesn't really want you to do that.... I would love someone to bounce questions off of
  • 1
    @Starkium wich engine are you using? And what are you trying to accomplish?
  • 1
    @crafter unreal engine 4, as many ai as possible in a multiplayer environment.
Add Comment