This week I got instructed to write a script that downloads an Excel sheet (which is used as a product database with >20000 products and lots of macros) from Dropbox, and parses it. In PHP. 🤦‍♂️

The company considers the project manager I work with to be the best. After working with him, I consider him to be everything that is wrong with project management.

This PM injects himself into everything and has a way of completely over-complicating the smallest of things. I will give an example:

We needed to receive around 1000 rows of data from our vendor, process each row, and host an endpoint with the data in json. This was a pretty simple task until the PM got involved and over complicated the shit out of it. He asks me what file format I need to receive the data. I say it doesnt really matter, if the vendor has the data in Excel, I can use that. After an hour long conversation about his concerns using Excel he decides CSV is better. I tell him not a problem for me, CSV works just as good. The PM then has multiple conversations with the Vendor about the specific format he wants it in. Everything seems good. The he calls me and asks how am I going to host the JSON endpoints. I tell him because its static data, I was probably going to simply convert each record into its own file and use `nginx`. He is concerned about how I would process each record into its own file. I then suggest I could use a database that stores the data and have an API endpoint that will retrieve and convert into JSON. He is concerned about the complexities of adding a database and unnecessary overhead of re-processing records every time someone hits the endpoint. No decision is made and two hours are wasted. Next day he tells me he figured out a solution, we should process each record into its own JSON file and host with `nginx`. Literally the first thing I said. I tell him great, I will do that.

Fast forward a few days and its time to receive the payload of 1000 records from the Vendor. I receive the file open it up. While they sent it in CSV format the headers and column order are different. I quietly without telling the PM, adjust my code to fit what I received, ran my unit test to make sure it processed correctly, and outputted each record into its own json file. Job is now done and the project manager gets credit for getting everything to work on the first try.

This is absolutely ridiculous, the PM has an absurd 120 hours to this task! Because of all the meetings, constant interruptions, and changing of his mind, I have 35 hours to this task. In reality the actual time I spent writing code was probably 2-3 hours and all the rest was dealing with this PM's meetings and questions and indecisiveness. From a higher level, he appears to be a great PM because of all the hours he logs but in reality he takes the easiest of tasks and turns them into a nightmare. This project could have easily been worked out between me and vendor in a 30 min conversation but this PM makes it his business to insert himself into everything. And then he has the nerve to complain that he is so overwhelmed with all the stuff going on. It drives me crazy because this inefficacy and unwanted help makes everything he touches turn into a logistical nightmare but yet he is viewed as one of the companies top Project Managers.

Storytime!

(I just posted this in a shorter form as a comment but wanted to write it as a post too)

TL;DR, smarts are important, but so is how you work.

My first 'real' job was a lucky break in the .com era working tech support. This was pretty high end / professional / well respected and really well paid work.

I've never been a super fast learner, I was HORRIBLE in school. I was not a good student until I was ~40 (and then I loved it, but no longer have the time :( )

At work I really felt like so many folks around me did a better job / knew more than me. And straight up I know that was true. I was competent, but I was not the best by far.

However .... when things got ugly, I got assigned to the big cases. Particularly when I transferred to a group that dealt with some fancy smancy networking equipment.

The reason I was assigned? Engineering (another department) asked I be assigned. Even when it would take me a while to pickup the case and catch up on what was going on, they wanted the super smart tech support guys off the case, and me on it.

At first this was a bit perplexing as this engineering team were some ultra smart guys, custom chip designers, great education, and guys you could almost see were running a mental simulation of the chip as you described what you observed on the network...

What was also amusing was how ego-less these guys seemed to be (I don't pretend to know if they really were). I knew for a fact that recruiting teams tried to recruit some of these guys for years from other companies before they'd jump ship from one company to the next ... and yet when I met them in person it was like some random meeting on the street (there's a whole other story there that I wish I understood more about Indian Americans (many of them) and American engineers treat status / behave).

I eventually figured out that the reason I was assigned / requested was simple:

1. Support management couldn't refuse, in fact several valley managers very much didn't like me / did not want to give me those cases .... but nobody could refuse the almighty ASIC engineers. No joke, ASIC engineers requests were all but handed down on stone tablets and smote any idols you might have.

2. The engineers trusted me. It was that simple.

They liked to read my notes before going into a meeting / high pressure conference call. I could tell from talking to them on the phone (I was remote) if their mental model was seizing up, or if they just wanted more data, and we could have quick and effective conversations before meetings ;)

I always qualified my answers. If I didn't know I said so (this was HUGE) and I would go find out. In fact my notes often included a list of unknowns (I knew they'd ask), and a list of questions I had sent to / pending for the customer.

The super smart tech support guys, they had egos, didn't want to say they didn't know, and they'd send eng down the rabbit hole. Truth be told most of what the smarter than me tech support guy's knew was memorization. I don't want to sound like I'm knocking that because for the most part memorization would quickly solve a good chunk of tech support calls for sure... no question those guys solved problems. I wish I was able to memorize like those guys.

But memorization did NOT help anyone solve off the wall bugs, sort of emergent behavior, recognize patterns (network traffic and bugs all have patterns / smells). Memorization also wouldn't lead you to the right path to finding ANYTHING new / new methods to find things that you don't anticipate.

In fact relying on memorization like some support folks did meant that they often assumed that if bit 1 was on... they couldn't imagine what would happen if that didn't work, even if they saw a problem where ... bro obviously bit 1 is on but that thing ain't happening, that means A, B, C.

Being careful, asking questions, making lists of what you know / don't know, iterating LOGICALLY (for the love of god change one thing at a time). That's how you solved big problems I found.

Sometimes your skills aren't super smarts, super flashy code, sometimes, knowing every method off the top of your head, sometimes you can excel just being more careful, thinking different.

As a tech lead i sometimes find it very hard to defend developers for no fault of theirs.

Management is completely incapable of noticing hard data like git logs or action items updated on an excel and seems to have an idea that the devs are incompetent , but the ba that sets impossible goals and crap business documentation is competent.

Should i just let the project and juniors burn.

This is why we can never have enough software developers

It's true. No matter how many people learn to program, there will never be enough people who know how to program. They don't have to be very good at it either. It is now a required skill.

Minimum wage in first world countries is way above 5$ per hour. A Raspberry PI 3B costs 40$, or at most 1 day of work for the worst paid jobs. And it will run for years, and do routine tasks up to thousands of times faster than any employee. With that, the only excuse that people still do routine tasks, is the inaccessibility of coder time.

Solution: everybody should know how to write code, even at the simplest level.

Blue-collar jobs: they will be obsolete. Many of them already are. The rest are waiting for their turn.

Marketing people - marketing is online. They need to know how to set up proper tracking in JS, how to get atomic data in some form of SQL, how to script some automated adjustments via APIs for ad budgets, etc. Right now they're asking for developers to do that. If they learn to do that, they'll be an independent, valued asset. Employers WILL ask for this as a bonus.
Project Managers - to manage developers, they need to know what they do. They need to know code, they have to know their way around repositories.
QA staff - scripted tests are the best, most efficient tests.
Finance - dropping Excel in favor of R with Markdown, Jupyter Notebooks or whatever, is much more efficient. Customizing / integrating their ERP with external systems is also something they could do if they knew how to code.

Operations / Category Management - most of it would go obsolete with more companies adopting APIs as a way to exchange important information, rather than phone calls and e-mails.

Who would not be replaced or who wouldn't benefit from programming? Innovative artists.

A lot of it might not be now now, but the current generation will see it already in their career.

If we educate people today, without advanced computer skills and some coding, then we are educating future deadbeats.

With all this, all education should include CS. And not just as a mandatory field or something. Make it more accessible, more interesting, more superficial if needed. Go straight to use cases, show its effectiveness in the easiest way possible. Inquisitive minds will fill in the blanks, and everyone else will at least know how to automate a part of their work.

Today I learned that in our team, where we usually process data for runtime usage through batch scripts, which is the dumbest shit anyone can think of, someone decided to do data processing through VBA inside an excel file.

So that proves, regardless of how bad a solution is, an even more stupid solution is still possible.

At least it's not documented, so my hope is no one will see and copy it.

What the hell am I!? I wonder if you guys can help me...

I've been programming most of my life but I've never actually been a developer by title or job role. I thought maybe if I list what I do and have done someone here could help? I'm sure there are more of you in a similar boat.

- C# and VB dev for some quick DBMS projects to help me understand and mine databases and create a nice simple view for project teams to show findings from the data to help make certain decisions.
- Automating a lot of my colleagues work with Python and if very restricted then just VBA macros in Excel and MSP. This did also include creating tools to gather data during workshops and converting the data for input into other systems.
- Brought Linux to the office with most team members now moving over to Linux with the peace of mind to know that though they do need to try solve their own problems, I can help if need be.
- Had to learn AWS and then implement an autoscaling and load balanced data center installation of a few Atlassian toolsets.
- Creating the architecture diagrams documentation needed for things like the above point.
- Having said that, also have ended up setting up all the Jira/Confluence etc. servers we use and have implemented so far whether cloud (Azure/AWS) or on prem and set up scripts to automate where possible.
- Implemented an automated workflow view in SharePoint based on SP list data and though in an ASPX page, primarily built in JS.
- Building test systems in PHP/JS with Laravel and Angular to help manage integration between systems. Having quite a time right looking into how to build middleware to connect between SOAP and REST API's, the trouble caused more by the systems and their reliance on frameworks we're trying to cut out of the picture.
- Working on BI and MI and training a team to help on the report creation so that I can do the fun creative stuff and then set them to work on the detail :)

Actually it seems safe to say that it seems that though I've finally moved into a dev office (beforehand being the only developer around) I seem to be the one they go to when a strategic solution is needed ASAP and the normal processes can't be followed (fun for someone with a CompSci degree and a number of project management courses under the belt... though I honestly do enjoy the challenges)

But I always end up Jack of all but master of, well hopefully some at least. let's not even get started on the tech related hobbies from circuit design and IoT to Andoid / iOS and game dev and enjoying a bit of pen testing to make sure we're all safe at work and at home.

As much as I don't like boxes, I'm interested to know if there is in fact a box for me? By the way, the above is just a snapshot of my last two years minus the project management work...