Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Get a devDuck
Rubber duck debugging has never been so cute! Get your favorite coding language devDuckBuy Now
Search - "directory structure"
Received "emergency update" code from internal enterprise security team. Wasn't given time to do code review; was assured code was reviewed and solid.
Pushed code to over 6k lower-level servers before finding this gem buried deep within:
cd /foo; rm -rf *; cd /
(This ran as root, and yes, the cwd was / from earlier in the code).
/foo, of course, did not exist on some servers.
Now, it is those servers which do not exist.
Worst legacy experience...
Called in by a client who had had a pen test on their website and it showed up many, many security holes. I was tasked with coming in and implementing the required fixes.
Site turned out to be Classic ASP built on an MS Access database. Due to the nature of the client, everything had to be done on their premises (kind of ironic but there you go). So I'm on-site trying to get access to code and server. My contact was *never* at her desk to approve anything. IT staff "worked" 11am to 3pm on a long day. The code itself was shite beyond belief.
The site was full of forms with no input validation, origin validation and no SQL injection checks. Sensitive data stored in plain text in cookies. Technical errors displayed on certain pages revealing site structure and even DB table names. Server configured to allow directory listing in file stores so that the public could see/access whatever they liked without any permission or authentication checks. I swear this was written by the child of some staff member. No company would have had the balls to charge for this.
Took me about 8 weeks to make and deploy the changes to client's satisfaction. Could have done it in 2 with some support from the actual people I was suppose to be helping!! But it was their money (well, my money as they were government funded!).1
Buckle up kids, this one gets saucy.
At work, we have a stress test machine that trests tensile, puncture and breaking strength for different materials used (wood construction). It had a controller software update that was supposed to be installed. I was called into the office because the folks there were unable to install it, they told me the executable just crashed, and wanted me to take a look as I am the most tech-savvy person there.
I go to the computer and open up the firmware download folder. I see a couple folders, some random VBScript file, and Installation.txt. I open the TXT, and find the first round of bullshit.
"Do not run the installer executable directly as it will not work. Run install.vbs instead."
Now, excuse me for a moment, but what kind of dick-cheese-sniffing cockmonger has end users run VBScript files to install something in 2018?! Shame I didn't think of opening it up and examining it for myself to find out what that piece of boiled dogshit did.
I suspend my cringe and run it, and lo and behold, it installs. I open the program and am faced with entering a license key. I'm given the key by the folks at the office, but quickly conclude no ways of entering it work. I reboot the program and there is an autofilled key I didn't notice previously. Whatever, I think, and hit OK.
The program starts fine, and I try with the login they had previously used. Now it doesn't work for some reason. I try it several times to no avail. Then I check the network inspector and notice that when I hit login, no network activity happens in the program, so I conclude the check must be local against some database.
I browse to the program installation directory for clues. Then I see a folder called "Databases".
"This can't be this easy", I think to myself, expecting to find some kind of JSON or something inside that I can crawl for clues. I open the folder and find something much worse. Oh, so much worse.
I find <SOFTWARE NAME>.accdb in the folder. At this point cold sweat is already running down my back at the sheer thought of using Microsoft Access for any program, but curiosity takes over and I open it anyway.
I find the database for the entire program inside. I also notice at this point that I have read/write access to the database, another thing that sent my alarm bells ringing like St. Pauls cathedral. Then I notice a table called "tUser" in the left panel.
Fearing the worst, I click over and find... And you knew it was coming...
Usernames and passwords in plain text.
Not only that, they're all in the format "admin - admin", "user - user", "tester - tester".
I suspend my will to die, login to the program and re-add the account they used previously. I leave the office and inform the peeps that the program works as intended again.
I wish I was making this shit up, but I really am not. What is the fucking point of having a login system at all when your users can just open the database with a program that nowadays comes bundled with every Windows install and easily read the logins? It's not even like the data structure is confusing like minified JSON or something, it's literally a spreadsheet in a program that a trained monkey could read.
God bless them and Satan condemn the developers of this fuckawful program.8
TLDR; I just screwed a production server and rendered it useless!!!
I went to install a product that we built at the customer's site, and was given a Linux running server, to deploy our app.
I work in windows, and barely know the basic Linux commands.
So I look at the files in the home directory, and see that the are a lot of files, so I ask the customer if it is ok that I move all the files to a separate directory.
He agrees, and me thinking that I am smart, proceed to enter the following commands in the terminal:
mv /* old
Of course I got an error that I don't have permission so my next command was:
sudo mv /* old
And that was the end of that computer.
The amazing part of the story is that as soon as it happened, I understood so much about Linux.
The file structure, sudo, the power of the terminal, aliases and so much more...15
I'm working on a programming language with a "bytecode" interpreter and a compiler that translates source code to said bytecode and... it sort of actually works!
I want to recreate an Erlang-style environment, currently you can write functions, call C++ functions via wrappers, have immutable-only values, and it has no explicit control structure apart from statement sequencing and the if-expression because I want to make it as functional as possible. Next thing on the list is to add a green threads implementation and ability to spawn and send messages to processes.
Still a WIP and heck even design-in-progress.
Now for the rant:
I'm using CMake for building C++ (interpreter) and Stack for Haskell (compiler) and I've been trying to get them to talk to each other for hours because I want CMake to manage the Stack build too and shove all the executables into one place. CMake documentation is weird and Stack isn't too helpful either, so I guess I'll just spend another few hours trying to get Stack to fuckin reveal its build directory to CMake and/or build to a given directory. Ugh.9
I love how the Keybase Linux client installs itself straight into /keybase. Unix directory structure guidelines? Oh no, those don't apply to us. And after uninstalling the application they don't even remove the directory. Leaving dirt and not even having the courtesy to clean it up. Their engineers sure are one of a kind.
Also, remember that EFAIL case? I received an email from them at the time, stating some stuff that was about as consistent as their respect for Unix directory structure guidelines. Overtyping straight from said email here:
[…] and our filesystem all do not use PGP.
> whatever that means.
The only time you'll ever use PGP encryption in Keybase is when you're sitting there thinking "Oh, I really want to use legacy PGP encryption."
> Legacy encryption.. yeah right. Just as legacy as Vim is, isn't it?
You have PGP as part of your cryptographic identity.
> OH REALLY?! NO SHIT!!! I ACTIVELY USED 3 OS'S AND FAILED ON 2 BECAUSE OF YOUR SHITTY CLIENT, JUST TO UPLOAD MY FUCKING PUBLIC KEY!!!
You'll want to remove your PGP key from your Keybase identity.
> Hmm, yeah you might want to do so. Not because EFAIL or anything, just because Keybase clearly is a total failure on all levels.
the Keybase team
> Well that's fucking clear. Could've taken some time to think before hitting "Send" though.
Don't get me wrong, I love the initiatives like this with all my heart, and greatly encourage secure messaging that leverages PGP. But when the implementation sucks this much, I start to ask myself questions about whether I should really trust this thing with my private conversations. Luckily I refrained from uploading my private key to their servers, otherwise I would've been really fucked.1
So, idiot me decided it would be a good idea to never get around to configuring my UPS to gracefully shutdown my server after a powercut lasting more than x duration...
Long story short, we had a powercut that lasted 4 minutes or so longer than the battery in the UPS could keep the server up for...
UPS died, server went pew, and after rebooting itself once the power came back on, my raid array wouldn’t mount anymore...
After Googling around, it seemed like running e2fsck would solve the problem.
Didn’t seem to do the trick... and tired me at 3am decided it would be a good idea to poke around.
Pretty sure I ran a command wrong, or two, because now I can’t even mount the fricken array in read only, and fsck complains with a shit ton errors...
Been researching for hours, and no dice...
Test Disk shows the ext4 partition, but fails to list any files...
I may have destroyed the tables or something... I’m a noob at this point.
I’m able to access files with the RStudio tool, however this doesn’t help with file names and directory structure 😭
Is it all over for my 5 years worth of photos and other bits and pieces that I don’t have any backups of ? 😂😭😭
If any of y’all are pros with data recovery and can help a fellow boi out, I’d be more than happy to pay for ya time !3
When your colleague just throws all their views into a single directory, despite you having a already created a very methodical and self-explanatory structure.
In today's episode of kidding on SystemD, we have a surprise guest star appearance - Apache Foundation HTTPD server, or as we in the Debian ecosystem call it, the Apache webserver!
So, imagine a situation like this - Its friday afternoon, you have just migrated a bunch of web domains under a new, up to date, system. Everything works just fine, until... You try to generate SSL certificates from Lets Encrypt.
Such a mundane task, done more than a thousand times already... Yet... No matter what you do, nothing works. Apache just returns a HTTP status code 403 - Forbidden.
Of course, what many folk would think of first when it came to a 403 error is - Ooooh, a permission issue somewhere in the directory structure!
So you check it... And re-check it to make sure... And even switch over to the user the webserver runs under, yet... You can access the challenge just fine, what the hell!
So you go deeper... And enable the most verbose level of logging apache is capable of - Trace8. That tells you... Not a whole lot more... Apparently, the webserver was unable to find file specified? But... Its right there, you can see it!
So you go another step deeper and start tracing the process' system calls to see exactly where it calls stat/lstat on the file, and you see that it... Calls lstat and... It... Returns -1? What the hell#2!
So, you compile a custom binary that calls lstat on the first argument given and prints out everything it returns... And... It works fine!
Until now, I chose to omit one important detail that might have given away the issue to the more knowledgeable right away. Our webservers have the URL /.well-known/acme-challenge/, used for ACME challenges, aliased somewhere else on the filesystem - To /tmp/challenges.
See the issue already?
Some *bleep* over at the Debian Package Maintainer group decided that Apache could save very sensitive data into /tmp, so, it would be for the best if they changed something that worked for decades, and enabled a SystemD service unit option "PrivateTmp" for the webserver, by default.
What it does is that, anytime a process started with this option enabled writes to /tmp/*, the call gets hijacked or something, and actually makes the write to a private /tmp/something/tmp/ directory, where something... Appeared as a completely random name, with the "apache2.service" glued at the end.
That was also the only reason why I managed fix this issue - On the umpteenth time of checking the directory structure, I noticed a "systemd-private-foobarbas-apache2.service-cookie42" directory there... That contained nothing but a "tmp" directory with 777 as its permission, owned by the process' user and group.
Overriding that unit file option finally fixed the issue completely.
I have just one question - Why? Why change something that worked for decades? I understand that, in case you save something into /tmp, it may be read by 3rd parties or programs, but I am of the opinion that, if you did that, its only and only your fault if you wrote sensitive data into the temporary directory.
And as far as I am aware, by default, Apache does not actually write anything even remotely sensitive into /tmp, so...
I wasted 4 hours of my life debugging this! Only to find out its just another SystemD-enabled "feature" now!
And as much as I love kidding on SystemD, this time, I see it more as a fault of the package maintainers, because... I found no default apache2/httpd service file in the apache repo mirror... So...9
I fucking HATE the Arduino environment right now.
First of all: you can't fucking put your project files in a sub folder to the main file. I can't write #include "src/motor.hpp" because it doesn't fucking know what that means.
Turns out you have to put all your header files in the fucking library folder common for all Arduino projects!
Secondly, you can't call your cpp headers hpp, they HAVE to be called h, or the Arduino environment throws a fit and begins whining about being unable to find the fucking files.
Not just that! You can't reference other Arduino libraries from within your library because the environment doesn't know what that means either.
To get around that you need to fucking include the library in your main file, AND THEN you can include it in the library file that uses it. After all, it should be the programmer's job to soon feed a so called IDE, right?
I'M SO FUCKING DONE WITH THIS SHIT! 😤
I'm ready to either program the Arduino directly with an AVR programmer or even port the entire project to the raspberry pi where I have a proper fucking Linux environment with a proper fucking directory structure so I can code proper fucking C++.
Hell I'm even fucking willing to spend all weekend porting all the code myself if necessary.
It's not reasonable that correct fucking C++ code is invalidated because I called the files something "wrong" and put them in the "wrong" directory.
"user friendly project board" my ass12
First week on the new job,
Looked at the existing (halfway done) react native code made by a third party vendor (again),
Fuck, they charge money for this shit?
Directory structure is shit
Redux code is shit
Api code is shit
They were given mock api and they still fucking hardcoded everything in the component shit
The only not-too-shit part is that it already used typescript, but just now I found it's because they used a fucking "under development" boilerplate,
that is still on version 0.0.6,
was last updated 6 months ago,
and it literally said "not ready for production" on the github,
Luckily I was given the authority to do a refactoring, which I'm gonna use to rewrite the app, because of that fucking boilerplate, and the only working part is only the UI, I can scrape what I can and scrap the rest
So I found some weird library included in this legacy code, didn't really get what it does and why it's there though.
Turns out there's nothing to be found on the internet about it. Absolutely nothing.
So after browsing through the directory structure a bit more I discover a README file. Hoping for answers I opened it, only to find this...3
Call with Customer for an upcoming software project (tone interpreted through rage)
them: "yeah we want to launch by end of march but our sales people would like to have a demo version asap, incl. structure of the forums and yaddayadda"
me: "earliest at end of feb"
them: "why do you need so long bro?"
me saying: "chill, we'll send you screenshots"
me (not saying): "because you ordered an azure based Active Directory as loginprovider at another company and our own white lable software needs to integrate that and we've never spoke about a demo version you mofos?"
me (also not saying): "and yet another partner that is working on the hardware component still hasn't logged into the API I crafted because he didn't knew how to send parameters to a REST API?"
What's the best nodejs framework for the MEAN stack? I need to do additional things to put TypeScript in node js and express. I have seen nestjs with a good directory structure and also uses TS by default. How about meteor or Koa?
Should I just add TS to my existing node and express? Or use nestjs or some other nodejs framework. What do you suggest?3
Just finished the prototype of my HTML5/Canvas implementation of a visual novel engine. The actual script exists behind the scenes on a REST like web service (to act as a sort of drm). The assets for the game and UI layouts are stored in what I call a shit file. Their is s a utility called the shitpacker that creates a shit file from a directory structure. The name of my engine is the Pyst engine. Pyst stands for Python Stub...as the game script is actually a subset of Python that I created. Eventually I will probably move Pyst to JS so I could hypothetically support offline games.
Early Django project from some random russian developers that they'd received it from a guy that was in-house at a recruitment company my old company worked for.
There were copies of directories everywhere. Everything was nested inside on large directory. The main site files were INSIDE THE ENVIRONMENT DIRECTORY. The assets were outside the main directory but were directly referenced. Everything had full access to everything.
I honestly don't know how they weren't hacked.
It was a disgusting piece of shit and it was so out-of-date I could have cried. There was no proper architecture. No structure. Models were put wherever someone saw fit. The few comments that existed were in Russian. Never again.
Even though I like a rolling Linux install that's been working for a long time, it's always fun to set up a fresh installation. Remember back when I had more time and setting up "Linux from scratch". Then there was Gentoo. Now Arch serves that purpose. Even though there is not that much time as when I was a student it's still brings pleasure starting from a clean slate. Only setting up the things you need and keeping config files clean and a nice directory structure. Keep it simple.4
markdown4documentation is a tool that can either convert a single markdown file, a complete directory or a complex structure to HTML or PDF. You can choose between several build-in themes (called templates) or define a custom theme.1
Side project update.
Made simple nlp library in python and published it’s first version to open source.
Now I can feed it with parsed pdf text.
See rant https://devrant.com/rants/2192388/...
Cause during reading book about nltk I couldn’t find simple extendible way to provide support for polish language and I wanted to abstract stemming, word normalization, tokenizer etc. so I can provide ex. different conditions for separate text files and don’t write much code what is an asset when you work solo.
It’s about 12GB of pdf public accessible law data I am trying to handle ( at first ) which is about 35000 files from last 90 years.
So far I automated downloading web pages and pdf documents from them. Extracting data from web pages and saving it to database. Extracting text from pdf files. I have about 5-6 projects to do all of it above maybe at the end I will put it to some workflow manager like Luigi or just run it by cronjob.
First thing for website version 1.0 part is find correlation between all documents inside law text using nlp library by building custom conditions. Then just generate directory structure and html files with links between documents.
Website version 2.0 is already in my mind but it will be creepy to make it and will take at least 1-2 months and I want to publish fast.
I have some pdfs with only images instead of text and tesseract worked quite good with them so maybe I will try to process them when everything go live.
Learned a lot about pdf as now I know that font in pdf is not always providing unicode characters ( stupid form of obfuscation) so when you extract text you need to build glyph vector to text map for every font.
Pdf is full vector representation - just like svg - what is logic if you think a bit and know that some printers are running using postscript.
Let’s hope next update will be about flutter mobile app which started all of shit above. It’s almost ready ( except getting data from api I am trying to do and logo for release version ). It’s last piece of puzzle.3
It’s funny how new developers are swaying between node and php(laravel) for their first run with api development. Laravels like your hand holding solution to warm you up before you dive in on express. Not shits configured for you with express or Koa. All the ORMs are fucked, everything and I mean everything is a separate npm package totally agnostic to your current environment. There’s barely a set of best practices or directory structure. It’s like being given some clay and water. Figure it out. I would never suggest anyone trying to find confidence in the web dev world to pick up node unless they’re working with an all in one framework like sales.js1
Oh how I wish there was more consensus on project directory structure in JS... sometimes keeps my mouse away from "fork" on GitHub.
What's your preferred structure?
It's 2016 and I still could not find a simple Markdown to HTML generator that mirrors the directory structure and does not require me to spend 20 million years with configs.
Had to roll my own.2
Working on a portfolio project - I ran some tests, everything is all good - 100% passing. Computer freezes - pop_os has been giving me shit lately, so I have to restart. Fine.
I get back into code, i re-run pytest - nothing works. Python apparently can't find my modules anymore. I try a few things, nothing. Why? Who the fuck knows - "oh you need this conftest file", "oh you need to remove __init__ from this directory or that one", "oh it's a pytest version iss---" no, no, no - listen here you little shit, it was working two seconds ago. Tell me why and how software I wrote with the most basic ass package structure - literally TWO directories suddenly has no fucking clue how to import the modules. hmm? Even from within the app directory, app.server now no longer recognizes imports from app.main or app.database.
ALL of this worked. It works in new directories without dedicated venvs - it works in new directories with my global python install - it works with any one of my conda envs - it works on other computers - WHY doesn't it work in THIS directory all of a sudden?? Ugh.
What's terrible is that relative imports will probably solve it within the app dir, but the tests dir won't accept them. Moreover, vs code autocompletion can find all the modules, but python itself cannot. Fucking infuriating shit like this is.1
Angular w/ Python or React w/ python. what why and how? I feel the web is full easy tutorials directing us to mainstream coding. I love angular 4 directory structure but react has more modules on git. help!1