I had some fun with ChatGPT today. I wondered how good its problem solving skills are. Turns out, it's no better than an entry/junior dev armed with all the docs out there - it knows what's written there, how to use the thing (language/framework/tool/etc.), but it has no "understanding" neither of the problem nor the tool, in a holistic way. It's got the knowledge, but it neither has the skill nor understanding of how/why to use it to solve a problem (any problem beyond plain simple complexity).

So the problem I asked it to solve was related to this one I had: https://devrant.com/rants/6312527 .

It was painful to troubleshoot this problem with ChatGPT. It kept on focusing on this particular problem and reacting to errors while trying to fix its initial solution. It took us a good while. Eventually, it reached a working solution, but it was an ugly, convoluted approach that was not feasible to cover my use case with.

FWIW I think it is interesting to follow its line of thought. Eventually, a pattern emerges of how it tries to solve the problem. And it reminds me a lot of myself on the first week in the IT field :)