Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "cronjob"
-
Like many others my favourote shameless hack is a cronjob to restart our app server at 2am, thus preventing out of memory exceptions5
-
The gift that keeps on giving... the Custom CMS Of Doom™
I've finally seen enough evidence why PHP has such a bad reputation to the point where even recruiters recommended me to remove my years of PHP experience from the CV.
The completely custom CMS written by company <redacted>'s CEO and his slaves features the following:
- Open for SQL injection attacks
- Remote shell command execution through URL query params
- Page-specific strings in most core PHP files
- Constructors containing hundreds of lines of code (mostly used to initialize the hundreds of properties
- Class methods containing more than 1000 lines of code
- Completely free of namespaces or package managers (uber elite programmers use only the root namespace)
- Random includes in any place imaginable
- Methods containing 1 line: the include of the file which contains the method body
- SQL queries in literally every source file
- The entrypoint script is in the webroot folder where all the code resides
- Access to sensitive folders is "restricted" by robots.txt 🤣🤣🤣🤣
- The CMS has its own crawler which runs by CRONjob and requests ALL HTML links (yes, full content, including videos!) to fill a database of keywords (I found out because the server traffic was >500 GB/month for this small website)
- Hundreds of config settings are literally defined by "define(...)"
- LESS is transpiled into CSS by PHP on requests
- .......
I could go on, but yes, I've seen it all now.12 -
I got a friend who likes to travel randomly to random places.. he now found some dudes who share this hobby.
So now there is a thing running on my server ticking everyday, 10am, with a 1:150 probability to send a sms to 7 numbers (including me) which tells them to travel to budapest.
FYI: he and his friends study engineering, i study too. So 1:150 may sound pretty low at first but we don't have that much time over the year actually.:D
Russian Roulette Travelling is a thing now.😁11 -
One Thursday noon,
operation manager: (looking at mobile)what the.....something is wrong i am getting bunch of emails about orders getting confirmed.
Colleague dev: (checks the main email where it gets all email sent/received) holy shit all of our clients getting confirmation email for orders which were already cancelled/incomplete.
Me: imediately contacting bluehost support, asking them to down the server so just that we can stopp it, 600+ emails were already sent and people keep getting it.
*calls head of IT* telling the situation because he's not in the office atm.
CEO: wtf is happening with my business, is it a hacker?
*so we have a intrusion somebody messed the site with a script or something*
All of us(dev) sits on the code finding the vulnerabilities , trying to track the issue that how somebody was able to do that.
*After an hour*
So we have gone through almost easch function written in the code which could possibly cause that but unable to find anything which could break it.
Head asking op when did you started getting it actually?
Op: right after 12 pm.
*an other hour passes*
Head: (checking the logs) so right after the last commit, site got updated too?. And....and.....wtf what da hell who wrote this shit in last commit?
* this fuckin query is missing damn where clause* 🤬
Me: me 😰
*long pause, everyone looking at me and i couldn't look at anyone*
The shame and me that how can i do that.
Head: so its you not any intrudor 😡
Further investigating, what the holy mother of #_/&;=568 why cronjob doesn't check how old the order is. Why why why.
(So basically this happened, because of that query all cancelled/incomplete orders got updated damage done already, helping it the cronjob running on all of them sending clients email and with that function some other values got updated too, inshort the whole db is fucked up.)
and now they know who did it as well.
*Head after some time cooling down, asked me the solution for the mess i create*
Me: i took backup just couple of days before i can restore that with a script and can do manual stuff for the recent 2 days. ( operation manager was already calling people and apologising from our side )
Head: okay do it now.
Me: *in panic* wrote a script to restore the records ( checking what i wrote 100000000 times now ), ran...tested...all working...restored the data.
after that wrote an apology email, because of me staff had to work alot and it becomes so hectic just because of me.
* at the end of the day CEO, head, staff accepted apology and asked me to be careful next time, so it actually teached me a lesson and i always always try to be more careful now especially with quries. People are really good here so that's how it goes* 🙂2 -
Alright, so my previous rant got a way better response than I expected! (https://devrant.io/rants/832897)
Hereby the first project that I cannot seem to get started on too badly :/.
DISCLAIMER: I AM NOT PROMOTING PIRACY, I JUST CAN'T FIND A SUITABLE SERVICE WHICH HAS ALL THE MUSIC I WANT. I REGULARLY BUY ALBUMS. before everyone starts to go batshit crazy regarding piracy, this is legal in The Netherlands for personal use. I think that supporting the artists you love is very good and I actually regularly pay for albums and so on but:
- I want all the music from about every artist in my scene. Either on Deezer or on Spotify this is not available and I'm not gonna get them both (they both have about half of the music I want). Their services are awesome but I'm not going to pay for something if I can't listen to all the music I like, hell even some artists (on deezer mostly) only have half their music on there and it's mostly not better on Spotify.
- I'd happily buy all albums because I love supporting the artists I love but buying everything is just way too fucking much."Get a premium music streaming subscription!" - see the first point.
You can either agree or disagree with me but that's not what this rant is about so here we go:
The idea is to create a commandline program (basically only needs to be called by a cron job every day or so) which will check your favourite youtube (sorry, haven't found a suitable non-google youtube replacement yet) channels every day through a cronjob and look for new uploads. If there are, it will download them, convert them to MP3 or whatever music format you'd like and place them in the right folder. Example with a favourite artist of mine:
1. Script checks if there are any new uploads from Gearbox Digital (underground raw hardstyle label).
2. Script detects two new uploads.
3. Script downloads the files (I managed to get that done through the (linux only or also mac?) youtube-dl software) and converts them to mp3 in my case (through FFMPEG maybe?).
4. Script copies them to the music library folder but then the specific sub-folder for Gearbox Digital in this case.
You should be able to put as many channels in there as you want, I've tried this with the official YouTube Data API which worked pretty fine tbh (the data gathering through that API). The ideal case would be to work without API as youtube-dl and youtube-dlg do. This is just too complicated for me :).
So, thoughts?43 -
I fucking HATE it when devs refer to "cronjob" as some kind of fucking webservice and not the actual crontab.3
-
Highlights from my week:
Prod access: Needed it for my last four tickets; just got it approved this week. No longer need it (urgently, anyway). During setup, sysops didn’t sync accounts, and didn’t know how. Left me to figure out the urls on my own. MFA not working.
Work phone: Discovered its MFA is tied to another coworker’s prod credentials. Security just made it work for both instead of fixing it.
My merchant communication ticket: I discovered sysops typo’d my cronjob so my feature hasn’t run since its release, and therefore never alerted merchants. They didn’t want to fix it outside of a standard release. Some yelling convinced them to do it anyway.
AWS ticket: wow I seriously don’t give a crap. Most boring ticket I have ever worked on. Also, the AWS guy said the project might not even be possible, so. Weee, great use of my time.
“Tiny, easy-peasy ticket”: Sounds easy (change a link based on record type). Impossible to test locally, or even view; requires environments I can’t access or deploy to. Specs don’t cover the record type, nor support creating them. Found and patched it anyway.
Completed work: Four of my tickets (two high-priority) have been sitting in code review for over a month now.
Prod release: Release team #2 didn’t release and didn’t bother telling anyone; Release team #1 tried releasing tickets that relied upon it. Good times were had.
QA: Begs for service status page; VP of engineering scoffs at it and says its practically impossible to build. I volunteered. QA cheered; VP ignored me.
Retro: Oops! Scrum master didn’t show up.
Coworker demo: dogshit code that works 1 out of 15 times; didn’t consider UX or user preferences. Today is code-freeze too, so it’s getting released like this. (Feature is using an AI service to rearrange menu options by usage and time of day…)
Micromanager response: “The UX doesn’t matter; our consumers want AI-driven models, and we can say we have delivered on that. It works, and that’s what matters. Good job on delivering!”
Yep.
So, how’s your week going?2 -
I once forgot to lock a cronjob, which dispatched queued text messages. Some people received the same message ~five times. I wasted around 4000 EUR in total ... :(1
-
Schrodingers cron: when a cronjob does not appear to be running and you turn email on it runs fine. Then you turn email off and doesn't appear to be running anymore...1
-
TL:DR
Why do so shitty "API"s exist that are even harder to write than proper ones? D:
Trying to hack my venilation at home.
This API is so horrible D:
The API is only based on POST requests no matter if you want to write values or get values and the response only contains XML with cryptic values like:
<?xml version="1.0" encoding="UTF-8"?>
<PARAMETER>
<LANG>de</LANG>
<ID>v01306</ID>
<VA>00011100000000000000000010000001</VA>
<ID>v00024</ID>
<VA>0</VA>
<ID>v00033</ID>
<VA>2</VA>
<ID>v00037</ID>
<VA>0</VA>
Also there are multiple API routes like
POST /data/werte1.xml
POST /data/werte2.xml
POST /data/werte3.xml
POST /data/werte4.xml
And actually the real API route is only given in the request body and not in the path.
Why is this so shitty? D:<
Btw in terms of security this is also top notch. It just globally saves if one computer sends the login password.
I mean why even ask for a password then? D:
That made me end up with a cronjob to send a login request so I don't have to login on any device.
PS:
You see, great piece of German engineering.3 -
I’m fairly new to maintaining my own webservers. For the past week the servers (two of them) kept crashing constantly.
After some investigation I figured it was due to someone running a script trying to get ssh access.
I learned about fail2ban, DOS and DDOS attacks and had quite a fight configuring it all since I had 20 seconds on average between the server shutdowns and had to use those 20 second windows to configure fail2ban bit by bit.
Finally after a few hours it was up and running on both servers and recognized 380 individual IPs spamming random e-mail / password combos.
I fet relieved seeing that it all stopped right after fail2ban installation and thought I was safe now and went to sleep.
I wake up this morning to another e-mail stating that pinging my server failed once again.
I go back to the logs, worried that the attack became more sophisticated or whatever only to see that the 06:25 cronjob is causing another fucking crash. I can’t figure out why.
Fuck this shit. I’m setting another cronjob to restart this son of a bitch at 06:30.
I’m done.3 -
So it turns out I had to set the memory_limit of a PHP cronjob to a whopping 8 Gigglebytes to make it run.
Call me haxX0r m4n from now on.2 -
All my respect goes to sysadmins, ive been trying to hack together a cronjob for tha past 2 hours and its still not working.4
-
The fact that there's only two characters between "run this job every 10 minutes" and "run this job every hour on the tenth minute" was the fix for the particular problem i just spent 5 hours on :facepalm:8
-
[CMS Of Doom™]
Ah, yes, their built-in bullshit newsletter module just sent the n-th user n emails. Wonderful considering n=368.
The culprit? Better don't ask...
OK, anyway: So the mailer is running as a CRONjob, but nah, not as a console script call but by a public HTTP GET URL call, fucking obviously (it's the CMS Of Doom for a reason).
So these fucking imbeciles "implemented" an ob_start() callback where HTML links are - for whatever fucking reason - modified by some regex (obviously everybody knows parsing HTML by Regex is trivial). In this case the link was somehow modified to recall the mailer Cronjob...
This must have upset the pngoing mailing process thus spamming mails. Whyyyy
And I've thought I've seen it all after 6 months in this legacy hell...
This is why you don't run a company consisting of only beginners in PHP (in cluding their "CEO")! -
Just had some problems with a cronjob I've been playing around which kept my pc running for like 10 mins after i used `shutdown -H`.. My mother asked me why i wa so mad at my pc
Me: A cronjob is stopping ubuntu from shutting down.
She: What's ubuntu?
Me: the os I've told you about the whole last weekend
She : what's OS?
I and my father literally explained her Ubuntu and Linux basics for at least 2 hours after she asked why my pc is looking "so purple" (i was using the console).
btw got the bug fixed :) -
Puts three months of work into this project; cronjob to ping 3 APIs at regular intervals, cleans, massive features extraction, dumps into PostgreSQL db. Got Django on top of that with a small neural net and interesting viz - absolutely gorgeous!
Can’t fuckin wait to showcase that.
Feedback: “is that the right blue? I think you have the old company logo! etc”
Mah.LahF3 -
I'm developing a web app, which is purely based on some commercial .NET driven API. The documentation is a 12 page MS Word file with incorrect parameters and non-existing endpoints. I think there's also a cronjob which purposely crashes their server every 15minutes. I just love getting client emails saying I need to fix my app and get my shit together.
-
I finally had a reason to setup an atjob on a server. Been looking for a reason to use it for so long. Today was finally the day!
Good feeling 😊2 -
Last night's cronjob was the first time I broke everything on the network via my automation. Fixed it before business hours of our clients.
At least I learned my lesson? 😀😁😂🤣😃😃😄😅 -
How do you restore partial data from a mysql backup? Don't worry, nothing is wrong, I'm just thinking about how would I restore something if shit hits the fan.
Our current strategy for database backups is to just run mysqldump during the night, using a cronjob (feel freue to suggest a better way ;))
1) Restoring the full db: just read that sql file into the mysql command.
2) Restoring just one (or some) tables: open the file in an IDE, just select the lines you're after, copy them to a new file, read that one (possible issues: let's say we have a table B to which entries of table A are related and we just want to restore table A. We can't nuke table B too, as also table C is refering to it, so we have to do some orphant removal in B afterwards)
3) Restoring selected entries in specific tables: setup a new db, read the full backup in there, dump these entries to a new file and read that into the real db
How do you so it? Any better aproaches/tools?8 -
TLDR;
Side project update.
Made simple nlp library in python and published it’s first version to open source.
Now I can feed it with parsed pdf text.
See rant https://devrant.com/rants/2192388/...
Why ?
Cause during reading book about nltk I couldn’t find simple extendible way to provide support for polish language and I wanted to abstract stemming, word normalization, tokenizer etc. so I can provide ex. different conditions for separate text files and don’t write much code what is an asset when you work solo.
It’s about 12GB of pdf public accessible law data I am trying to handle ( at first ) which is about 35000 files from last 90 years.
So far I automated downloading web pages and pdf documents from them. Extracting data from web pages and saving it to database. Extracting text from pdf files. I have about 5-6 projects to do all of it above maybe at the end I will put it to some workflow manager like Luigi or just run it by cronjob.
First thing for website version 1.0 part is find correlation between all documents inside law text using nlp library by building custom conditions. Then just generate directory structure and html files with links between documents.
Website version 2.0 is already in my mind but it will be creepy to make it and will take at least 1-2 months and I want to publish fast.
I have some pdfs with only images instead of text and tesseract worked quite good with them so maybe I will try to process them when everything go live.
Learned a lot about pdf as now I know that font in pdf is not always providing unicode characters ( stupid form of obfuscation) so when you extract text you need to build glyph vector to text map for every font.
Pdf is full vector representation - just like svg - what is logic if you think a bit and know that some printers are running using postscript.
Let’s hope next update will be about flutter mobile app which started all of shit above. It’s almost ready ( except getting data from api I am trying to do and logo for release version ). It’s last piece of puzzle.3 -
When waiting for that Cronjob to run to confirm its working okay, then modifying the script and go through the same thing over and over again.
-
My selenium script cronjob is not running on the VPS.
I am not sure what's wrong with it
Even the crond service status just shows
the session was opened for the user root
shows the command
the session closed for user root
I tried the tons of solution available but nothing worked
I am using Ubuntu 16.04 with LXDE13 -
Public reset API for every online food service?
That would be a great idea, if you consider running a cronjob every day witch randomly selects items form your FavoriteFoodArray.1 -
Always valued my every minute but seems I have given up the principle for a cron job which I have to wait for every minute to run so I can see what I am doing on the log file.
-
Spent a couple of weeks on writing a cronjob which updates a certain value in the application config, and spend the last few months on testing it in different environments to make sure it does not fail in production. Ran the deployment script, and the damn cronjob fails because of ssl certificate on production. fuck me
-
What is the best alternative to cronjobs, guaranteeing high availability and jobs not being duplicated?6
-
Not best practice whatsoever because the box was most likely owned, but...
SSH kept defaulting back to port 22 when it wasn't supposed to. So, wrote a cronjob that checked diff between SSH config and backed up SSH config. If different, reload backup. Didn't get locked out again.
Box has of course been replaced.2 -
[Prestashop question / rant]
Yes, it's not StackOverflow here, neither is it prestashop support forum - but I trust u people most :)
Proper solution for working with big(?) import of products from XML (2,5MB, ~8600 items) to MySQL(InnoDB) within prestashop backoffice module (OR standalone cronjob)
"solutions" I read about so far:
- Up php's max execution time/max memory limit to infinity and hope it's enough
- Run import as a cronjob
- Use MySQL XML parsing procedure and just supply raw xml file to it
- Convert to CSV and use prestashop import functionality (most unreliable so far)
- Instead of using ObjectModel, assemble raw sql queries for chunks of items
- Buy a pre-made module to just handle import (meh)
Maybe an expert on the topic could recommend something?3 -
So I think I have answered this, but here goes.
I have ddns service I need to update periodically. I chose once every 5 minutes. I am using this command:
/usr/bin/wget -O /dev/null -o /dev/null <webcall url>
I have it running every 5 minutes in a cronjob. I checked and wget is using port 443 to connect to my webcall url which is https. I am assuming this is hiding the details of the url. Is this true? Also, I don't like that the cronjob is sending the whole command to syslog. Is there a way to prevent it from syslogging this? I would rather keep the details of the url hidden as much as possible. I am the only user on the server, but am curious if there is a way.
So questions are:
1. Is wget hiding the details of the url from prying eyes? It is using port 443 for https.
2. Cannot I not log the cronjob command in syslog? I supose I could create a script that hides this.6 -
Does anybody know of a Java library that you can use to parse cron expressions? I mainly want to give it a cron expressions and have it calculate its next execution time.6