Spark is shit and is the definition of a write-only language. Change my mind.

You’re wrong

What would you use instead?

Being the only option does make it "the best" by definition... That still doesn't mean it's well designed :)

Fuck even is Spark

TIL the term write-only language

@rantydev its not the only option, there are similar software, some open source and some commercial, depending what you're aiming at.

But even if you think its the only option, then it makes it by definition:
1. An innovation
2. The best option
3. The worst option

Certainly an innovation, are you arguing against innovation or redundancy?

@bioDan I'm not arguing against the Spark system. The RDD system works (and well too).

I just hate the specific DSL used to interact with it :) the verbosity of the commands, the fact there that there are a million ways of doing the same thing (and most of them are inefficient), the plethora of side effects and implementation details that need to be kept in mind each time you want to run a command.