After working with a bunch of people it occurred to me that almost anyone can work and learn for most IT positions. This is something that most people will realize, especially if they work with newbies that learn on the go. There are exceptions of course, but most companies are just making crud apps with their business logic.

What I wonder then is, why is it that the way hiring is done seems to be completely against this idea? Rather than whining about recruiters and bad interviews, I am curious about why this is so common. What is it that a lot of companies think or see that make their hiring process so bad or convoluted?

Because of braindead stereotypes, trying to apply processes that don't fit IT, also not working their ass out towards the potential employees.

I switched from non-tech to tech.

What I found out was that there are much more arrogant assholes cut-throat people who likes to gate-keep working in tech compared to non-tech.

Law, stock trading, high level executives are also full of those kind of people.

I am just not sure why this field (who's supposed to be full of shy nerds) attracts the same kind of people

I don't actually think this is a job that can be done by anyone, or even by more than a small percentage of people. Can it be learned by a smart person? Of course, but most people aren't anywhere near capable of handling the complexity involved in a basic app. And that's before you add in the multiple layers of nonsense contributed by 'tools' at enterprise level.

Agreeing with @vintprox and @cho-uc, here is an example:

A container ship you might have many problems/"bugs". I.e. HVAC, hydraulics, drivetrain, engines, hull leaks, liquefication inside containers, vermin and literal bugs, fuel balance, radio and software bugs (those happen A LOT), freaking PIRATES and good ol' maintenance tasks like cleaning and valve calibration.
All for a deck crew of 2-7 to do for weeks before docking.

How do you hire those people?!? You want a single someone who can balance fluid dynamics equations, do underwater soldering, fix software bugs and lift 150Kg? That person does not exists in a commercial scale. You can't fit all the required skills in such a small crew.

So you plan ahead, choosing stops considering the skills of the dock crew and the needs of each individual ship. And you do not change routes easily.

Software companies HATE planning ahead, they want to be "dynamic" and "adaptable". So they make stupid hiring efforts to get freaking astronauts.

funny, my experience working with bunch of people is that often even people who claim to have learned programming don't know programming

"almost anybody can work in the IT and learn on the go" is the mindset why one of my colleagues could write O(n^2) SQL query and then trigger a discussion on buying a new DB server because "this one is too slow" and nobody in the chain could identify the root cause.

@qwwerty ^^ this
i mean... yeah, it took me about two days to realize why my query was taking 10 seconds to spit out about 10 rows...
... when I was 16, and my first week into learning sql.

You just live in a bubble. You made your realization basing on work with "newbies" who were not only motivated to work which already makes them special but presumably went through some verification process during the recrutation. So basically after working with the top of the candidates your conclusion is "hey everyone can do this". Empirical reality is sadly disappointing. Go google statistics on how many students struggle with things like reading with comprehension. An average person would most likely fail at a task as mundane as installing and configuring IDE

@PAKA I've worked in 3 different countries in various companies, and some of those people I hired myself by talking with them for a couple hours and seeing some of their work. Even crappy projects gave me good insight. Of course it takes time and they might struggle, but I just told them why x or y was a bad idea and to ask a lot of questions, and they got by just fine. I don't mean that absolutely everyone can do it. I dunno, as long as the person is interested in learning they seem to be fine.

@ars1 thats part of the "social bubble" again. Normal people usually dont have free time IT related projects they can showcase when the opportunity arises

I agree - most IT positions aren't actually technical in the sense of requiring code. A lot of it is just knowing a system or being able to Google.

Source: I've worked for two IT companies and three software companies