Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Search - "swagger"
Things have been a little too quiet on my side here, so its time for an exciting new series:
practiseSafeHex's new life as a manager.
Episode 1: Dealing with the new backend team
It's great to be back folks. Since our last series where we delved into the mind numbing idiocy of former colleagues, a lot has changed. I've moved to a new company and taken a step up as a Dev manager / Tech lead. Now I know what you are all thinking, sounds more dull and boring right? Well it wouldn't be a practiseSafeHex series if we weren't ...
DEALING! ... WITH! ... IDIOTS!
Bingo! so lets jump right in and kick us off with a good one.
So for the past few months i've been on an on-boarding / fact finding / figuring out this shit-storm, mission to understand more about what it is i'm suppose to do and how to do it. Last week, as part of this, I had the esteemed pleasure of meeting face to face with the remote backend team i've been working with. Lets rattle off a few facts to catch us all up:
- 8 hour time difference to me
- No documentation other than a non-maintained swagger doc
- Swagger is reporting errors and several of the input models are just `Type: String`
- The one model that seems accurate, has every property listed as optional, including what must be the primary key
- Properties go missing and get removed at the drop of a hat and we are never told.
- First email I sent them took 27 days to reply, my response to that hasn't been answered so far 31 days later (new record! way to go team, I knew we could do it!!!)
- I deal directly with 2 of them, the manager and the tech lead. Based on how things have gone so far, i've nick named them:
So lets look at some example of their work:
- I was trying to test the new backend, I saw no data in QA. They said it wouldn't show up until mid day their time, which is middle of the night for us. I said we need data in our timezone and I was told: a) "You don't understand how big this system is" (which is their new catch phrase) b) "Your timezone is not my concern"
- The whole org started testing 2 days later. The next day a member from each team was on a call and I was asked to give an update of how the testing was going on the mobile side. I said I was completely blocked because I can't get test data. Backend were asked to respond. They acknowledged they were aware, but that mobile don't understand how big the system is, and that the mobile team need to come up with ideas for the backend team, as to how mobile can test it. I said we can't do anything without test data, they said ... can you guess what? ... correct "you don't understand how big the system is"
- We eventually got something going and I noticed that only 1 of the 5 API changes due on their side was done. Opened tickets. 2 days later asked them for progress and was told that "new findings" always go to the bottom of the backlog, and they are busy with other things. I said these were suppose to be done days ago. They said you can't give us 2 days notice and expect everything done. I said the original ticket was opened a month a go *sends link* ......... *long silence* ...... "ok, but you don't understand how big the system is, this is a lot of work"
- We were on a call. Product was asking the backend manager (aka "Ass") a question about a slight upgrade to the new feature. While trying to talk, the tech lead (aka "Hole") kept cutting everyone off by saying loudly "but thats not in scope". The question was "is this possible in the future" and "how long would it take", coming from management and product development. Hole just kept saying "its not in scope", until he was told to be quiet by several people.
- An API was sending down JSON with a string containing a message for the user with 2 bits of data inside it. We asked for one of those pieces to also come down as a property as the string can change and we needed it client side. We got that. A few days later we found an edge case and asked for the second piece of data to be a property too. Now keep in mind, they clearly already have access to them in order to make the string. We were told "If you keep requesting changes like this, you are going to delay the release of the backend by up to 2 weeks"
Yes folks, there you have it, the most minuscule JSON modifications, can delay your release by up to 2 weeks ........ maybe I should just tell product, that they don't understand how big the app is, and claim we can't build it on our side? Seems to work for them
Thats all the time we have for today,
Tune in for more, where we'll be looking into such topics as:
- If god himself was an iOS developer ... not
- Why automate when you can spend all day doing it by hand
- Its more time-efficient to just give everything a story point of 5
- Why waste time replying to emails ... when you can do nothing instead
See you all next week,
I have replied to them with scripts, curl commands, and Swagger docs (PROVIDED TO SUPPORT THEIR API), everything that could possibly indicate there's a bug. Regardless, they refuse to escalate me to level 1 support because "We cant reproduce the issue in a dev environment"
Well of course you can't reproduce it in a dev environment otherwise you'd have caught this in your unit tests. We have a genuine issue on our hands and you couldnt give less of a shit about it, or even understand less than half of it. I literally gave them a script to use and they replied back with this:
"I cannot replicate the error, but for a resource ID that doesnt exist it throws an HTTP 500 error"
YOUR APP... throws a 500... for a resource NOT FOUND?????????!!!!!!!!!! That is the exact OPPOSITE of spec, in fact some might call it a MISUSE OF RESTFUL APIs... maybe even HTTP PROTOCOL ITSELF.
I'm done with IBM, I'm done with their support, I'm done with their product, and I'm DONE playing TELEPHONE with FIRST TIER SUPPORT while we pay $250,000/year for SHITTY, UNRELENTING RAPE OF MY INTELLECT.12
So, some time ago, I was working for a complete puckered anus of a cosmetics company on their ecommerce product. Won't name names, but they're shitty and known for MLM. If you're clever, go you ;)
Anyways, over the course of years they brought in a competent firm to implement their service layer. I'd even worked with them in the past and it was designed to handle a frankly ridiculous-scale load. After they got the 1.0 released, the manager was replaced with some absolutely talentless, chauvinist cuntrag from a phone company that is well known for having 99% indian devs and not being able to heard now. He of course brought in his number two, worked on making life miserable and running everyone on the team off; inside of a year the entire team was ex-said-phone-company.
Watching the decay of this product was a sheer joy. They cratered the database numerous times during peak-load periods, caused $20M in redis-cluster cost overrun, ended up submitting hundreds of erroneous and duplicate orders, and mailed almost $40K worth of product to a random guy in outer mongolia who is , we can only hope, now enjoying his new life as an instagram influencer. They even terminally broke the automatic metadata, and hired THIRTY PEOPLE to sit there and do nothing but edit swagger. And it was still both wrong and unusable.
Over the course of two years, I ended up rewriting large portions of their infra surrounding the centralized service cancer to do things like, "implement security," as well as cut memory usage and runtimes down by quite literally 100x in the worst cases.
It was during this time I discovered a rather critical flaw. This is the story of what, how and how can you fucking even be that stupid. The issue relates to users and their reports and their ability to order.
I first found this issue looking at some erroneous data for a low value order and went, "There's no fucking way, they're fucking stupid, but this is borderline criminal." It was easy to miss, but someone in a top down reporting chain had submitted an order for someone else in a different org. Shouldn't be possible, but here was that order staring me in the face.
So I set to work seeing if we'd pwned ourselves as an org. I spend a few hours poring over logs from the log service and dynatrace trying to recreate what happened. I first tested to see if I could get a user, not something that was usually done because auth identity was pervasive. I discover the users are INCREMENTAL int values they used for ids in the database when requesting from the API, so naturally I have a full list of users and their title and relative position, as well as reports and descendants in about 10 minutes.
I try the happy path of setting values for random, known payment methods and org structures similar to the impossible order, and submitting as a normal user, no dice. Several more tries and I'm confident this isn't the vector.
Exhausting that option, I look at the protocol for a type of order in the system that allowed higher level people to impersonate people below them and use their own payment info for descendant report orders. I see that all of the data for this transaction is stored in a cookie. Few tests later, I discover the UI has no forgery checks, hashing, etc, and just fucking trusts whatever is present in that cookie.
An hour of tweaking later, I'm impersonating a director as a bottom rung employee. Score. So I fill a cart with a bunch of test items and proceed to checkout. There, in all its glory are the director's payment options. I select one and am presented with:
"please reenter card number to validate."
Bupkiss. Dead end.
OR SO YOU WOULD THINK.
One unimportant detail I noticed during my log investigations that the shit slinging GUI monkeys who butchered the system didn't was, on a failed attempt to submit payment in the DB, the logs were filled with messages like:
"Failed to submit order for [userid] with credit card id [id], number [FULL CREDIT CARD NUMBER]"
One submit click later and the user's credit card number drops into lnav like a gatcha prize. I dutifully rerun the checkout and got an email send notification in the logs for successful transfer to fulfillment. Order placed. Some continued experimentation later and the truth is evident:
With an authenticated user or any privilege, you could place any order, as anyone, using anyon's payment methods and have it sent anywhere.
So naturally, I pack the crucifixion-worthy body of evidence up and walk it into the IT director's office. I show him the defect, and he turns sheet fucking white. He knows there's no recovering from it, and there's no way his shitstick service team can handle fixing it. Somewhere in his tiny little grinchly manager's heart he knew they'd caused it, and he was to blame for being a shit captain to the SS Failboat. He replies quietly, "You will never speak of this to anyone, fix this discretely." Straight up hitler's bunker meme rage.13
Somebody asked me my API doc.
I don't have any API at all.
I will lie, and I'll write a swagger specification in few hours and I'll send them.
They will try to read it and understand, and after maybe a week, when they will ask for testing and endpoint I'll pretend to be on holiday for 2 weeks.
3-4 weeks gone already, I checked they should be on holiday by then. Only then, I'll answer with a fake endpoint with fake data.
I'll get another 2 weeks if I'm lucky.
When they discover about fake data, I'll say there is a bug.
In total if I play well, I have 2/2.5 months to implement some kind of API server with some more or less true implementation.
Thanks to Swagger. Swag11
Swagger does not send request body for GET calls.! WHAT THE FUCK..! And the argument supporting is get calls should not have any request payloads and rather should have response payloads since its a "get" call. Are you serious?? What if there are parameters to be passed which cannot be accomodated in the params or the header. Even though people are kind of literally abusing on their issues page still they adamantly refuse to add support for this.
Swagger you had high standards in my book. You just fell so deep down there is no coming back.3
Beginning of a Hackaton and the CTO calls everyone up and tries to instigate everyone with this gem:
"Challenge yourself and what we have. For example, instead of AWS use Swagger."
Nobody says anything but has the same WTF expression ... How do these person become tech lead?2
Fuck this shitty day seriously. So I'm the only frontend on my team and our team lead knows some frontend from 6 years ago or something I guess and doesn't really understand that modern frontend is fucking complicated.
So now he decided that we are going to use some shitty protocol that is not json to communicate between frontend and backend just so they don't have to build a stupid controller and a simple API on fucking shit ass Java.
I told him it's a bad decision that brings no benefit since it has no easy integration with any popular frontend library, we are locking ourselves up to this solution so new people that come in are going to need to learn this shit just so they can communicate to the backend, we have absolutely no need to do this I'm just working on simple CRUDS anyway and the only thing this will bring Is pain in the frontend.
He just ignored me and say no no it's going to be easy so let's just do this. Bitch I was hired because I know my shit, if you're going to make a big decision like this just because you don't want to write a fucking simple API and some swagger documentation you can fucking burn in hell and don't deserve your position.
This kind of issues just perpetuate the eternal problem of frontend vs backend instead of just working as a team. Why do backenders think they're smarter than anyone that does frontend or that frontend is worthless? I don't go and tell them how to build their microservices or anything like that since I assume they know what they're doing and I respect their work, why can't I get the same? What is this fucking shit, why do I have to suffer because of this guy? Why can't we find a middle ground.
While we are here, also fuck monorepos and companies trying to copy google or Facebook just because without actually thinking of solving a problen16
Just received a test for a job I'm interviewing for. I was interviewing for a C++ position. Practice test: Create an REST API using SpringBoot, Spring Data, document with Swagger and implement continuous integration testing.
To be fair, I also mentioned I'm fluent in Java. But I've never touched SpringBoot or done any backend webdev, since my intention was to never get near it.
Deadline: Sunday. Game on...4
Oh I have quite a few.
#1 a BASH script automating ~70% of all our team's work back in my sysadmin days. It was like a Swiss army knife. You could even do `ScriptName INC_number fix` to fix a handful of types of issues automagically! Or `ScriptName server_name healthcheck` to run HW and SW healthchecks. Or things like `ScriptName server_name hw fix` to run HW diags, discover faulty parts, schedule a maintenance timeframe, raise a change request to the appropriate DC and inform service owners by automatically chasing them for CHNG approvals. Not to mention you could `ScriptName -l "serv1 serv2 serv3 ..." doSomething` and similar shit. I am VERY proud of this util. Employee liked it as well and got me awarded. Bought a nice set of Swarowski earrings for my wife with that award :)
#2 a JAVA sort-of-lib - a ModelMapper - able to map two data structures with a single util method call. Defining datamodels like https://github.com/netikras/... (note the @ModelTransform anno) and mapping them to my DTOs like https://github.com/netikras/... .
#3 a @RestTemplate annptation processor / code generator. Basically this dummy class https://github.com/netikras/... will be a template for a REST endpoint. My anno processor will read that class at compile-time and build: a producer (a Controller with all the mappings, correct data types, etc.) and a consumer (a class with the same methods as the template, except when called these methods will actually make the required data transformations and make a REST call to the producer and return the API response object to the caller) as a .jar library. Sort of a custom swagger, just a lil different :)
I had #2 and #3 opensourced but accidentally pushed my nexus password to gitlab. Ever since my utils are a private repo :/3
TLDR: What's your favorite REST API Documentation tool?
I'm about to start developing a really large REST API. I have never really had need to document my previous API's since they were small, self explanatory and had only me using them. Aside from this one being too large for me to keep track of there is also a remote team that will need to integrate with it.
Basically I need write exceptional documentation while using as little time as possible. I love postman and am planning to use it for documentation since I currently use it to test during development anyway but I have seen some really neat looking tools like swagger and apiary so I figured I would check for some other suggestions.
What is your current / preferred REST API documentation method?13
It's Friday and you're tired. You're working beyond 8 hrs already. Sure it's paid OT but you want to go home. Just finishing the last 10 API endpoints. But you execute the wrong script the overwrite the directory of the last 10 API endpoints instead of the swagger doc generator script. GRRRrrr.... NO!!!!! T_T2
For all the hate that Java gets, this *not rant* is to appreciate the Spring Boot/Cloud & Netty for without them I would not be half as productive as I am at my job.
Just to highlight a few of these life savers:
- Spring security: many features but I will just mention robust authorization out of the box
- Netflix Feign & Hystrix: easy circuit breaking & fallback pattern.
- Spring Data: consistent data access patterns & out of the box functionality regardless of the data source: eg relational & document dbs, redis etc with managed offerings integrations as well. The abstraction here is something to marvel at.
- Spring Boot Actuator: Out of the box health checks that check all integrations: Db, Redis, Mail,Disk, RabbitMQ etc which are crucial for Kubernetes readiness/liveness health checks.
- Spring Cloud Stream: Another abstraction for the messaging layer that decouples application logic from the binder ie could be kafka, rabbitmq etc
- SpringFox Swagger - Fantastic swagger documentation integration that allows always up to date API docs via annotations that can be converted to a swagger.yml if need be.
- Last but not least - Netty: Implementing secure non-blocking network applications is not trivial. This framework has made it easier for us to implement a protocol server on top of UDP using Java & all the support that comes with Spring.
For these & many more am grateful for Java & the big big community of devs that love & support it.
Swagger superposition: the generated server now works but the local hosted editor does not connect. I look at one screen and wonder: "Why does this work?" and then at my other screen (same code, same port) and think "Why DOESN'T this work?"
Once upon a time we had to integrate our backend with the billing service of another company. They sent us a Swagger API Description, describing the payload they will send to us. Everything was ok, and we implemented it.
We told them our background was ready to run some tests, but they got some error. After some quick debug i asked for the actual payload they just sent to me.
The payload had different field types, different field names and non-nullable keys was explicitly coming as null.
THEY DIDN'T FOLLOW THEIR OWN SPECIFICATIONS!
Everyone excited discussing a new data access API to provide to the clients when, le boss:
"Just so you guys think out of the box a bit. What if you deployed the API on Swagger instead of AWS? It seems a nice and fresh approach, huh?"
Everyone on the room remained in silence and internally questioning why do we work here...1
"Just start ahead"
I am supposed to transform calls from one api to another one. Yet there's no documentation, ambiguous code statements, no examples of what values are contained -- but sure, let me just start assuming how the whole thing is supposed to work. That won't lead us more into a murky waters at all.
Even more frustrating: We own the api. We should be able to tell by the access logs how we are queried. Yet for some reason, access logs cannot be accessed and I shall "just work from the swagger defintion".
Well, that swagger definition is broken, its example are shit (somebody liked to use undefined in optional fields, making me wonder even more what the heck is going on here), and I have no idea of what I am doing. Fun times.3
Fuck you azure and your goddamn swagger support. If you say that you support OAS than at least support the specification.
rent / question (there is a question at the end and I'd appreciate your opinion)
8 months ago, I agreed to help a not too distant relative of mine to do his master thesis at the company where I work. He was supposed to build something really MVP, but useful for us and I'd help him get some scientific questions out of it, and provide him with (computing) resources to test his theories / implementations under simulated and much heavier load.
Since then, he didn't get done anything even remotely useful, always just stuck on very rudimentary issues, claimed things are almost ready, I wrote a quick smoke test to prove that the whole application blows up when you touch it, in short - a disaster and went over to radio silence.
In the meanwhile, we didn't need it anymore, so 1.5 months ago, I got in touch with him again, with an even more technical proposal, something, at least I'd think, that's even cooler to do. He asked me some question about hypothetical load, the system should be able to handle eventually, to come up with alternative implementations to compare them against each other. He said that his exam period is going to be over soon and he'll get back to me with some initial version.
2 weeks ago, I got back in touch with him, trying to urge him, to get finally started and get something done. If he'd actually sit down and do it during the holidays as a "full time job", he'd be probably done in 2 weeks. Last week, he came back to me and said he has an initial PR ready to review.
I was excited about it, but basically froze when I realized what he did. He deleted all his previous work - some infrastructure stuff which took us basically 3 months of back and forth to get running - and as far as I could see, all the new code were only auto generated clients based on a swagger specification. In short - I could do it in less then an hour. If you really have no idea what you're doing, it might take you half a day, but definitely nowhere near to a week.
His brother, which a good friend of mine, thinks I'm being too hard on him. His argument was, that it's too hard, and he has to do it in C#, but he only knows Java (I gave him access to some of our repositories to copy paste code together, he didn't need to invent anything. I also prefer C# but wrote my master thesis in Java) Personally, I'm just pissed because he promises stuff that he never does. I totally understand him - I was like that as a student as well, I guess karma is a ... but still, he's wasting my time.
Right now I'm thinking how to get out of this, without having even more time wasted. I doubt he'd ever deliver anything useful. He got plenty of input from me about what he could consider for his scientific question, how to measure performance, ... He can keep his credentials to access our test environment with the test data, but I won't give him access to any additional computing resources, to compare how his solutions might scale on our company's cost. (mainly it's not the money, but I'd have to provide that stuff, and probably help him set it up)
does it sound like a fair deal (saying, I'm done with you. You can finish your topic on your own, but don't expect any help from me)? or am I being a dick about it and too demanding?1
Why the GMail API docs and example were sucks, thats totally not cool! Yeah I need your API right now, but wtf is this docs, its so fcuking ugly. Even my swagger generated docs much better than this... Arrrrghh!2
Does anyone know any good documentation tools for an asp.net core API. I've looked into swagger but it'd be good to have something that can produce an xsd4
Are there any good sites/outlets besides redbubble where you can purchase decent to high quality developer themed swag?3
Has anyone got exp with swagger, nodejs/express and sequealizejs
wondering how to where/how to use sequelizejs models if swagger defines it..