Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Get a devDuck
Rubber duck debugging has never been so cute! Get your favorite coding language devDuckBuy Now
Search - "server help needed"
Good Morning!, its time for practiseSafeHex's most incompetent co-worker!
Todays contestant is a very special one.
*sitcom audience: WHY?*
Glad you asked, you see if you were to look at his linkedin profile, you would see a job title unlike any you've seen before.
*sitcom audience oooooooohhhhhh*
were not talking software developer, engineer, tech lead, designer, CTO, CEO or anything like that, No No our new entrant "G" surpasses all of those with the title ..... "Software extraordinaire".
*sitcom audience laughs hysterically*
I KNOW!, wtf does that even mean! as a previous dev-ranter pointed out does this mean he IS quality code? I'd say he's more like a trash can ... where his code belongs
*ba dum tsssss*
Ok ok, lets get on with the show, heres some reasons why "G" is on the show:
One of G's tasks was to build an analytics gathering library for iOS, similar to google analytics where you track pages and events (we couldn't use google's). G was SO good at this job he implemented 2 features we didn't even ask for:
- If the library was unable to load its config file (for any reason) it would throw an uncatchable system integrity error, crashing the app.
- If anything was passed into any of the functions that wasn't expected (null, empty array etc.) it would crash the app as it was "more efficient" to not do any sanity checks inside the library.
This caused a lot of issues as some of the data needed to come from the clients server. The day we launched the app, within the first 3 hours we had over 40k crash logs and a VERY angry client.
Now, what makes this story important is not the bugs themselves, come on how many times have we all done something stupid? No the issue here was G defended all of this as the right thing to do!
.. and no he wasn't stoned or drunk!
G claimed if he couldn't get the right settings / params he wouldn't be able to track the event and then our CEO wouldn't have our usage data. To which I replied:
"So your solution was to not give the client an app instead? ... which also doesn't give the CEO his data".
He got very angry and asked me "what would you do then?". I offered a solution something like why not have a default tag for "error" or "unknown" where if theres an issue, we send up whatever we have, plus the file name and store it somewhere else. I was told I was being ridiculous as it wasn't built to track anything like that and that would never work ... his solution? ... pull the library out of the app and forget it.
... once again giving everyone no data.
G later moved onto another cross-platform style project. Backend team were particularly unhappy as they got no spec of what needed to be done. All they knew was it was a single endpoint dealing with very complex model. There was no Java classes, super classes, abstract classes or even interfaces, just this huge chunk of mocked data. So myself and the lead sat down with him, and asked where the interfaces for the backend where, or designs / architecture for them etc.
His response, to this day frightens me ... not makes me angry, not bewilders me ... scares the living shit out of me that people like this exist in the world and have successful careers.
G: "hhhmmm, I know how to build an interface, but i've never understood them ... Like lets say I have an interface, what now? how does that help me in any way? I can't physically use it, does it not just use up time building it for no reason?"
us: "... ... how are the backend team suppose to understand the model, its types, integrate it into the other systems?"
G: "Can I not just tell them and they can write it down?"
I'll just pause here for a moment, as you'll likely need to read that again out of sheer disbelief
I've never seen someone die inside the way the lead did. He started a syllable and his face just dropped, eyes glazed over and he instantly lost all the will to live. He replied:
" wel ............... it doesn't matter ... its not important ... I have to go, good luck with the project"
*killed the screen share and left the room*
now I know you are all dying in suspense to know what happened to that project, I can drop the shocking bombshell that it was in fact cancelled. Thankfully only ~350 man hours were spent on it
... yep, not a typo.
G's crowning achievement however will go down in history. VERY long story short, backend got deployed to the server and EVERYTHING broke. Lead investigated, found mistakes and config issues on every second line, load balancer wasn't even starting up. When asked had this been tested before it was deployed:
G: "Yeah I tested it on my machine, it worked fine"
lead: "... and on the server?"
G: "no, my machine will do the same thing"
lead: "do you have a load balancer and multiple VM's?"
G: "no, but Java is Java"
... and with that its time to end todays episode. Will G be our most incompetent? ... maybe.
Tune in later for more practiceSafeHex's most incompetent co-worker!!!33
So, I needed a package installed on one of our Unix servers. The package manager--which is obsolete garbage--was failing with a message which can only be described as a variant of "Go fuck yourself". A quick Google search didn't help.
3 espressos and an eternity later, I have descended into a manic state. My hair has turned grey and I have started lactating. As a last-ditch effort, I try a new search query on Google, and the first link takes me to a forum with a thread discussing a similar issue. The last post in the thread has a solution which works for me. After fixing the issue, everything in the world feels right and I decide to thank the generous poster, who is like an angel to me at this point.
Guess what? The poster is none other than me. 8 months back, I had created a user account on the forum just to post the solution to a similar issue I had on another server.13
I absolutely HATE "web developers" who call you in to fix their FooBar'd mess, yet can't stop themselves from dictating what you should and shouldn't do, especially when they have no idea what they're doing.
So I get called in to a job improving the performance of a Magento site (and let's just say I have no love for Magento for a number of reasons) because this "developer" enabled Redis and expected everything to be lightning fast. Maybe he thought "Redis" was the name of a magical sorcerer living in the server. A master conjurer capable of weaving mystical time-altering spells to inexplicably improve the performance. Who knows?
This guy claims he spent "months" trying to figure out why the website couldn't load faster than 7 seconds at best, and his employer is demanding a resolution so he stops losing conversions. I usually try to avoid Magento because of all the headaches that come with it, but I figured "sure, why not?" I mean, he built the website less than a year ago, so how bad can it really be? Well...let's see how fast you all can facepalm:
1.) The website was built brand new on Magento 188.8.131.52...what? I mean, if this were built a few years back, that would be a different story, but building a fresh Magento website in 2017 in 1.x? I asked him why he did that...his answer absolutely floored me: "because PHP 5.5 was the best choice at the time for speed and performance..." What?!
2.) The ONLY optimization done on the website was Redis cache being enabled. No merged CSS/JS, no use of a CDN, no image optimization, no gzip, no expires rules. Just Redis...
3.) Now to say the website was poorly coded was an understatement. This wasn't the worst coding I've seen, but it was far from acceptable. There was no organization whatsoever. Templates and skin assets are being called from across 12 different locations on the server, making tracking down and finding a snippet to fix downright annoying.
But not only that, the home page itself had 83 custom database queries to load the products on the page. He said this was so he could load products from several different categories and custom tables to show on the page. I asked him why he didn't just call a few join queries, and he had no idea what I was talking about.
4.) Almost every image on the website was a .PNG file, 2000x2000 px and lossless. The home page alone was 22MB just from images.
There were several other issues, but those 4 should be enough to paint a good picture. The client wanted this all done in a week for less than $500. We laughed. But we agreed on the price only because of a long relationship and because they have some referrals they got us in the door with. But we told them it would get done on our time, not theirs. So I copied the website to our server as a test bed and got to work.
So I show their developer the changes and he's stunned. He says he'll tell the hosting provider create a new server set up to migrate the optimized site over and cut over to, because taking the live website down for maintenance for even an hour or two in the middle of the night is "unacceptable".
So trying to be cool about it, I tell him I'd be happy to configure the server to the exact specifications needed. He says "we can't do that". I look at him confused. "What do you mean we 'can't'?" He tells me that even though this is a dedicated server, the provider doesn't allow any access other than a jailed shell account and cPanel access. What?! This is a company averaging 3 million+ per year in revenue. Why don't they have an IT manager overseeing everything? Apparently for them, they're too cheap for that, so they went with a "managed dedicated server", "managed" apparently meaning "you only get to use it like a shared host".
So after countless phone calls arguing with the hosting provider, they agree to make our changes. Then the client's developer starts getting nasty out of nowhere. He says my optimizations are not acceptable because I'm not using Redis cache, and now the client is threatening to walk away without paying us.
So I guess the overall message from this rant is not so much about the situation, but the developer and countless others like him that are clueless, but try to speak from a position of authority.
If we as developers don't stop challenging each other in a measuring contest and learn to let go when we need help, we can get a lot more done and prevent losing clients. </rant>14
I just can't understand what will lead an so called Software Company, that provides for my local government by the way, to use an cloud sever (AWS ec2 instance) like it were an bare metal machine.
They have it working, non-stop, for over 4 years or so. Just one instance. Running MySQL, PostgreSQL, Apache, PHP and an f* Tomcat server with no less than 10 HUGE apps deployed. I just can't believe this instance is still up.
By the way, they don't do backups, most of the data is on the ephemeral storage, they use just one private key for every dev, no CI, no testing. Deployment are nightmares using scp to upload the .war...
But still, they are running several several apps for things like registering citizen complaints that comes in by hot lines. The system is incredibly slow as they use just hibernate without query optimizations to lookup and search things (n+1 query problems).
They didn't even bother to get a proper domain. They use an IP address and expose the port for tomcat directly. No reverse proxy here! (No ssl too)
I've been out of this company for two years now, it was my first work as a developer, but they needed help for an app that I worked on during my time there. I was really surprised to see that everything still the same. Even the old private key that they emailed me (?!?!?!?!) back then still worked. All the passwords still the same too.
I have some good rants from the time I was there, and about the general level of the developers in my region. But I'll leave them for later!
Is it just me or this whole shit is crazy af?3
I hate people... I hate stupid people even more...
A person asked on slack about where download a Programming Language server called Railo. The official site is no longer up because the software was forked and acquired by a new company.
I suggested just to download that fork since it's more stable. They said no, they needed to mimic their production environment. Makes sense, so I left it alone since I couldn't help further.
Another person on slack asked which version of Railo they need. The OPs response was, "Oh whatever version you have."
My response was... "WTF... the latest version of Railo is 4.3 and the fork is 4.5... the only difference is the new name and a couple of security fixes. If you want to mimic production then you need the exact copy.. otherwise, the fork will be your best bet."
Nope.. I need Railo... any version. They say again.
I'm not sure if this entirely qualifies and I might have ranted about it a few years ago but fuck it.
My last internship. Company was awesome and my mentor/technical manager got along very well with me to the point that he often asked me to help out with Linux based stuff (he preferred Linux but was a C# guy and wasn't as familiar with it as me (Linux)).
We had to build an internal site thingy (don't remember what it was) and we delivered (me and some interns) and then the publishing moment came so I went to out project manager (a not-as-technical one) and asked if he could install a LetsEncrypt certificate on the site (he knew how and was one of the only ones who had direct access to the server).
He just stared at us and asked why the fuck we needed that since it was an internal thing anyways.
I kindly told that since it's free and can secure the connection, I preferred that and since its more secure, why the fuck not?
He wasn't convinced so it was off.
Next day I came in early and asked my mentor if he could do the SSL since he usually had access to that stuff. He stared at me with "what?" eyes and I explained what the PM said.
Then he immediately ssh'd in and got the damn cert with "we're going to go secure by default, of course!"
A minute later it was all set.2
A situation i just dealt with on my tech help Discord server:
"my gpu isn't working and i need it to update my bios"
"i have to have a cpu and ram to update my bios?"
this kid thought he only needed a mobo, GPU and PSU
So yeah, we released our app for 26k people to use two days ago. Due to circumstances not under our control, that was developed way faster than it should've. Today we're two days in and everything is going horrible. 26 thousand people are having trouble using it (not every problem is our fault, but ofc they don't give a shit) and our support team is not big enough to handle every request. We're not able to find any more people to help us handle it, so some developers are being used for support. But that slows the bugfixes a lot. We're at risk of losing our jobs because we will not be able to make it work in due time.
In Italy, at the beginning of the year a particular type of invoice became necessary and to handle that everyone needs a particular kind of email called "certified", you know, because Italy, and today our certified email server went offline for the whole fucking day because the provider was an overload of feces. We were overwhelmed by the amount of people that needed help.
I don't even know what to say, if we don't fix it we're fucking fucked, like literally.
I really hope everyone is having a nice new year. For us, this is going terrible.3
I had spent the last year working on a online store power by woocommerce with over 100k products from various suppliers. This online store utilized a custom API that would take the various formats that suppliers offer their inventory in and made them consistent. Now everything was going swimmingly initially, but then I began adding more and more products using a plug-in called WP all import. I reached around 100k products and the site would take up to an entire minute to load sometimes timing out. I got desperate so I installed several caching plugins, but to no avail this did not help me. The site was originally only supposed to take three to four months but ended up taking an entire year. Then, just yesterday I found out what went wrong and why this woocommerce website with all of these optimizations was still taking anywhere from 60 to 90 seconds to load, or just timing out entirely. I had initially thought that I needed a beefier server so I moved it to a high CPU digitalocean VM. While this did help a little bit, the site was still very slow and now I had very high CPU usage RAM usage and high disk IO. I was seriously stumped the Apache process was using a high amount of CPU and IO along with MYSQL as well. It wasn't until I started digging deeper into the database that I actually found out what the issue was. As I was loading the site I would run 'show process list' in the SQL terminal, I began to notice a very significant load time for one of the tables, so I went to go and check it out. What I did was I ran a select all query on that particular table just to see how full it was and SQL returned a error saying that I had exceeded the maximum packet size. So I was like okay what the fuck...
So I exited my SQL and re-entered it this time with a higher packet size. I ran a query that would count how many rows were in this particular table and the number came out to being in the millions. I was surprised, and what's worse is that this table belong to a plugin that I had attempted to use early in the development process to cache the site. The plugin was deactivated but apparently it had left PHP files within the wp content directory outside of the actual plugin directory, so it's still executing scripts even though the plugin itself was disabled. Basically every time I would change anything on the site, it would recache the whole thing, and it didn't delete any old records. So 100k+ products caching on saves with no garbage collection... You do the math, it's gonna be a heavy ass database. Not only that but it was serialized data, so when it did pull this metric shit ton of spaghetti from the database, PHP then had to deserialize it. Hence the high ass CPU load. I had caching enabled on the MySQL end of things so that ate the ram. I was really desperate to get this thing running.
Honest to God the main reason why this website took so long was because the load times made it miserable to work on. I just thought that the hardware that I had the site on was inadequate. I had initially started the development on a small Linux VM which apparently wasn't enough, which is why I moved it to digitalocean which also seemed to not be enough, so from there I moved to a dedicated server which still didn't seem to be enough. I was probably a few more 60-second wait times or timeouts from recommending a server cluster to my client who I know would not be willing to purchase it. The client who I promised this site to have completed in 3 months and has waited a year. Seriously, I would tell people the struggles that I would go through with this particular site and they would just tell me to just drop the site; just take the money, just take the loss. I refused to, this was really the only thing that was kicking my ass. I present myself as this high-and-mighty developer like I'm just really good at what I do but then I have this WordPress site that's just beating the shit out of me for a year. It was a very big learning experience and it was also very humbling as well, it made me realize that I really don't know as much as I think I might. It was evidence that there is still so much more to learn out there, I did learn a lot from that experience especially about optimizing websites the different types of methods to do that particular lonely on the server side and I'll be able to utilize this knowledge in the future.
I guess the moral of the story is, never really give up. Ultimately things might get so bad that you're running on hopes and dreams. Those experiences are generally the most humbling. Now I can finally present the site that I am basically a year late on to the client who will be so happy that I did not give up on the project entirely. I'll have experienced this feeling of pure euphoria, and help the small business significantly grow their revenue. Helping others is very fulfilling for me, even at my own expense.
Anyways, gonna stop ranting. Running out of characters. If you're still here... Ty for reading :')8
I work on a warehouse dev team. One day this past year, I was trying to deploy a new build to a QA server. Earlier that day I had been looking at the logs on the production server and had left the ssh session open. I had been working for less than a year out of college at this point and shouldn't have had access to deploy to the production server.
Long story short I deployed my QA build to the production server and saw there were problems connection to our production database. Then my heart dropped in my chest as I realized I had just brought down our production server.
I managed to get the server back up by rolling back in about 5 minutes and no one ever knew except some people on my team.
I felt horrible for the longest time. Later in the year another guy that joined my team that has about 20 years of experience under his belt did the exact same thing, but needed help rolling it back. Needless to say, that made me feel a lot better. 😂
Definitely the worst moment of my year.3
Welp, this made my night and sorta ruined my night at the same time.
He decided to work on a new gaming community but has limited programming knowledge, but has enough to patch and repair minor issues. He's waiting for an old friend of his to come back to start helping him again, so this leads to me. He needed a custom backend made for his server, which required pulling data from an SQL/API and syncing with the server, and he was falling behind pace and asked for my help. He's a good friend that I've known for a while, and I knew it wouldn't take to long to create this, so I decided to help him. Which lead to an interesting find, and sorta made my night.
It wasn't really difficult, got it done within an hour, took some time to test and fix any bugs with his SQL database. But this is where it get's interesting, at least for me. He had roughly a few hundred people that did beta testing of the server, anyways, once the new backend was hooked in and working, I realized that the other developer he works with had created a 'custom' script to make sure there are no leaks of the database. Well, that 'custom' script actually begins wiping rows/tables (Depends on the sub-table, some get wiped row by row, some just get completely dropped), I just couldn't comprehend what had happened, as rows/tables just slowly started disappearing. It took me a while of checking, before checking his SQL query logs (At least the custom script did that properly and logged every query), to realize it just basically wiped the database.
Welp, after that, it began to restrict the API I was using, and due to this it identified the server as foreign access (Since it wasn't using the same key as his plugin, even though I had an API key created just so it could only access ranks and such, to prevent abuse) and begin responding not with denied, but with a lovely "Fuck you hacker!" This really made my night, I don't know why, but I was genuinely laughing pretty hard at this response.
God, I love his developer. Luckily, I had created a backup earlier, so I patched it and just worked around the plugin/API to get it working. (Hopefully, it's not a clusterfuck to read, writing this at 2 am with less than an hour of sleep, bedtime! Goodnight everyone.)7
TL;DR: When picking vendors to outsource work to, vet them really well.
Got a large redesign project that involves rebuilding a website's main navigation (accessibility reasons).
Project is too big just for our dev team to handle with our workload so we got to bring a 3rd party vendor to help us. We do this often so no big deal.
But, this time the twist was Senior Management already had retained hours with a dev shop so they want us to use them for project. Okay...
Have our scope / discovery meeting about the changes and our expected DevOps workflow.
Devs work Local and push changes to our Github, that kicks off the build and we test on Dev, then it goes to Staging for more testing & PM review. Once ready we can push to prod, or whenever needed. All is agreed, everyone was happy.
Emailed the vendors' project manager to ask for their devs Github accounts so we can add them to the project. Got no reply for 3 days.
4th day, I get back "Who sets up the Github accounts?"
fuck me. they've never used Github before but in our scope meeting 4 days ago you said Github was fine...??
Whatever, fuck it. I'll make the accounts and add them.
Added 4 devs to the repo and setup new branch. 40min later get an email that they can't setup dev environment now, the dev doesn't know how to setup our CMS locally, "not working for some reason."
So, they ask for permission to develop on our STAGING server.. "because it's already setup"... they want to actively dev on our staging where we get PM/Senior Management approvals?
We have dev, staging, production instances and you want to dev in staging, not dev?... nay nay good sir.
This is whom senior management wants us to use, already paid for via retainer no less. They are a major dev shop and they're useless...
Cant wait for today's progress checkup meeting. 😐😐
Ex-colleague asked for help in regards to an old project we were working at my old job
Him:"We're experiencing an internal server error. What do we do?"
Me:"Restart tomcat, dude"
Then I explained how by finding tomcat in ps -ef in a Redhat server, because he's a Linux noob and needed a lesson in how services works. Proceeded to explain how to restart tomcat with an online guide available.
Him:"Couldn't find tomcat in any of the servers"
Me:"Are you sure? Send me screenshots"
Him: sent screenshots
Me:"it's there. Look carefully."
Him: finds it and proceeded to restart tomcat.
Him: "Can't restart. Some catalina.sh is stopping it."
Me:"Figure it out. You can do it".
Half a day passed...
Him:"I give up. If I restart the server, will tomcat also restart?"
Me:"Up to you man. It will work but it's bad practice."
He restarted the server vand now everything is honky dory. I feel sorry for him though.3
I've been putting a web application together that I'm looking to have published on AWS. Not having too much experience with AWS, I am looking to hire a contractor. I've had a number of quotes from different AWS admin's ranging from $40 an hour to $200 an hour, from 1-days worth of work to 2-months worth of work!
I'm not really sure what to make of it or to whom to trust. I believe they’re using my ignorance to overcharge me. I've listed my requirements below, could you guys use your professional experiences to let me know what you think is reasonable charge and where best I could find someone to help me.
My application is a US shopping website where people can set up an online shop and upload their products and maintain an inventory of the items.
This is what I’m looking for setup and configuration with the following two areas:
1) AWS SYSTEMS…
* AIM - Set up my server admin users.
* EC2 - Web Hosting.
* RDS - Fast DB.
* SES - To send emails.
* S3 Buckets - Uploaded image hosting.
Route 53 - I don’t know but someone said I should have this.
* Elastic Load Balancing - For, well, load balancing.
* A script that would back up the database once a day and save it to a private S3 Bucket.
* A script that will run once a day that calls an internal API, and POST a query to it.
* A script that runs once every 90 days, to refresh the SSL using ZeroSSL.com
Is there anything that I've missed such as security systems, firewalls, auto scaling and CDNs?
The quotes that I've received arranged from $320 to $64,000. I know I am being abused because of my ignorance. I would never overcharge someone because the customer doesn't know the efforts of the work. I hope someone here can help to understand the efforts needed and can tell me the true cost.
My work product: Or why I learned to get twitchy around Java...
I maintain a Java based test system, that tests a raster image processor. The client is a Java swing project that contains CORBA bindings to the internal API of the raster image processor. It also has custom written UI elements and duplicated functionality that became available in later versions of Java, but because some of the third party tools we use don't work with later versions of Java for some reason, it's not possible to upgrade Java to gain things as simple as recursive directory deletion, yes the version of Java we have to use does not support something as simple as that and custom code had to be written to support it.
Because of the requirement to build the API bindings along with the client the whole application must be built with the raster image processor build chain, which is a heavily customised jam build system. So an ant task calls out to execute a jam task and jam does about 90% of the heavy lifting.
In addition to the Java code there's code for interpreting PostScript files, as these can be used to alter the behaviour of the raster image processor during testing.
The server isn't much better though. It's a tomcat based application that was written by someone who had never built a tomcat application before, or any web application for that matter and uses raw SQL strings instead of an orm, it doesn't use MVC in any way, and insane amount of functionality is dumped into the jsp files.
It too interacts with a raster image processor to create difference masks of the output, running PostScript as needed. It spawns off multiple threads and can spend days processing hundreds of gigabytes of image output (depending on the size of the tests).
We're stuck on Tomcat seven because we can't upgrade beyond Java 6, which brings a whole manner of security issues, but that eager little Java updated will break the tool chain if it gets its way.
Between these two components we have the Java RMI server (sometimes) working to help generate image data on the client side before all images are pulled across a UNC network path onto the server that processes test jobs (in PDF format), by reading into the xref table of said PDF, finding the embedded image data (for our server consumed test files are just flate encoded TIFF files wrapped around just enough PDF to make them valid) and uses a tool to create a difference mask of two images.
This tool is very error prone, it can't difference images of different sizes, colour spaces, orientations or pixel depths, but it's the best we have.
The tool is installed in both the client and server if the client can generate images it'll query from the server which ones it needs to and if it can't the server will use the tool itself.
Our shells have custom profiles for linking to a whole manner of third party tools and libraries, including a link to visual studio 2005 (more indirectly related build dependencies), the whole profile has to ensure that absolutely no operating system pollution gets into the shell, most of our apps are installed in our home directories and we have to ensure our paths are correct for every single application we add.
And... Fucking and!
Most of the tools are stored as source bundles in a version control system... Not got or mercurial, not perforce or svn, not even CVS... They use a custom built version control system that is built on top of RCS, it keeps a central database of locked files (using soft and hard locks along with write protecting the files in the file system) to ensure users can't get merge conflicts by preventing other users from writing to the files at all.
Branching is heavy weight and can take the best part of a day to create a new branch and populate the history.
Gathering the tools alone to build the Dev environment to build my project takes the best part of a week.
What should be a joy come hardware refresh year becomes a curse ("Well fuck, now I loose a week spending it setting up the Dev environment on ANOTHER machine").
Needless to say, I enjoy NOT working with Java. A lot of this isn't Javas fault, but there's a lot of things that Java (specifically the Java 6 version we're stuck on) does not make easy.
This is why I prefer to build my web apps in python or node, hell, I'd even take Lua... Just... Compiling web pages into executable Java classes, why? I mean I understand the implementation of how this happens, but why did my predecessor have to choose this? Why?2
PHP dev help/advice needed!
We have problems with mysql. Still stuck with mariaDB, I'm using indexes (correct ones) and we have problems with scaling. we have a few tables with over 100mil rows, 1 of them is being read every morning with a subselect that counts unique rows, and fails every time because of timeout/lock, the temp table size was increased and helped for a little while but as time goes on the table grows and the problem reappears. I'm reading from a slave server that was purposely created for read only, yet we still have problems. We're using managed dedicated servers for out hosting and they aren't willing to optimise the database configs for our needs. What are the easiest options for scaling at this point? Going fully dedicated server and perconaDB? NOsql? Sharding the server? Anyone got any good blogposts or something to read about this? your own experience?11
I need some help speccing a server.
We have 5 video editors/ 3D animators, who’s currently working on their own machine and keeping a version on their machine.
I want a server that allows 5 people to work on it simultaneously. Preferably a few SSDs for the current project, then 50 TB++ of long storage. What raid should I use, how much ram, cpu, what system should I run, etc etc2
# This isn't THAT bad, but since I never had any collab before this one, this is the worst so far
I'm in a web development school where we need to do a yearly project. At the beginning, we started with the idea of doing an online wallet that would handle crypto-currencied (#blockchains), and other currencies too.
On the paper that sounds good, but the dude decided to create a NodeJS server api, and let's be honest, this was a gas factory. I couldn't help him because he was too fast in his ideas, and the third member was a bit more useful because he was the one creating the mobile app, so all he needed was an url that the dude couln't manage to create.
After a few weeks he started over the project, then over again a few weeks later, before coming to us and saying it was too difficult. We said "yeah, I mean you're own your own since the beginning, no wonders!" "Uh do you guys care if we change the whole project to do something else? Like a CV library"
Went a moment where he tried to over sell some incredible (read "overly common") features that already existed 10 years ago on some famous websites (ie. Monster), and he then eventually told me that this idea came from his new job, and that they needed this library. So we would have to work for his company for free. Nice.
The third guy and me came with a new idea (image recognition with IA and stuff), and we saw the dude maybe 5 times the whole week while we're supposed to work together
I was trying to set up my own "cloud" for iot experiments. I planned to use Intel Edison with mqtt broker (using mosca) and a node js app for providing API for mobiles and browser. And also to do other book keeping.
I spent the half day trying figure how to expose these servers to internet.
I configured ddns in noip.com and ddns settings configured to it in my router.
Port forwarded to the local server services I needed.
And then tested. Worked perfectly on any device in my router connected network. Tested on mobile network. Bam! It fucking doesn't work.
Then connected another router.
Double port forwarded. Again worked perfectly on router network. And failed on mobile.
Tested if ddns is right. Did nslookup it was fine as fuck.
Then disabled port forwarding. Did dmz. Nope. Nadda. No luck.
Then scratched my head so hard that I lost more already losing hairs.
Then remembered about router hardware firewall. Disabled it.
And there it didn't work.
My dreams shattered like a fucking deer hit by car on highway.
Then I see the IP pointing to my router in nslookup. Its 172.20.xx.xx. Its a fucking private IP.
My Asshole ISP is running another private network behind firewall. Which I fucking can't port forward
Now I think how much of a noob and idiot I am. Fuck this shit. Fuck all of these shit.
I am going for SaaS option for mqtt broker.
(Or help me?)
Fuck your firewall.
(PS: I had test the next day. FML)2
So I decided to collab with a website's maker (who i wont name here) to create something like r/place. (not an exact copy.)
I decided to start by learning their API, and customizing the server later.
I asked the guy for some help, and HOLY SHIT.
Let's start off by this: I had to request a chunk. The response data was in binary. 4 bits meant 1 pixel, so right away, I had to deal with that in my code.
No problem, just decided to use C# instead of JS. (see https://www.devrant.io/rants/547013)
I was finally done after a couple of mental breakdowns, and decided to implement updates.
I needed to use webhooks, and that was completely fine. But when I got "C1FFFF0000CA06" as response (in hex), I seeked some help.
C1 is the operation type: it means that a pixel was updated.
FFFF and 0000 were the chunk coordinates. But remeber: it's a signed integer. Guess what, I had to use Two's compliment. I decided to be a lazy asshole and only check for "00000000" because I was only displaying chunk 0,0.
CA06: This is a weird one. It's 2 bytes, and CA0 contains the X and Y coordinate of the pixel (in the chunk), and 6 contains the new color of the pixel.
I was sent the following code to work with 0xCA06:
color = 0xF & buffer
x = buffer >> 10
y = (buffer >> 4) & 0x3F
So I tried to do it, and it didn't work. I'm not blaming the developer of the server (original dev is reddit) because maybe I screwed up, but which guy will have a night of frustration and debugging?
P.S.: Dev, if you see this, I'm sorry. This API is way too complicated. I know we need to save bandwith and stuff, but damn.1
I needed to migrate one DB to another with one sql suite but instead I fucked up and suddenly disconnected both DBs, without being able to reconnect them again
I waisted a whole day for debugging, but found nothing
And guess which magic fixed all issues? On and Off a service of an app
On and Off!!!
The fun thing is that restarting the server didn't help, but the only service helped1
Anyone who has worked with OpenBazaar REST APIs. I cannot seem to get any response from the endpoints even though I think I've set the server correct. Any sort of help would be highly appreciated.
P.S. (Rant) the docs and the "helpful slack community" are total shit.
Follow up sorta...
So I got pulled into a support issue on a day off. Some system was facing timeouts on our servers so had to investigate.
Over the weekend as part of the release, I released the ELK stack I built and today I used that to help.
Pretty much immediately pinpointed which machine was hanging though still had to investigate and confirm so split between KQL and checking the server logs.
One thing I've always griped about is how no one created schema docs for it mongo collections so can't easily figure out what they do or your to get the document needed.
Well guess it's my turn.... Because only I know the schema :)
Okay, I hope a few people can help me with this; what are the benefits/reasons to use MS technologies? I'm talking about .NET, ASP, Windows Server, Powershell...
I've never understood it. I love Nodejs because you don't have any packages unless you ask for them. Alpine Linux is amazing! It runs on 8MB of RAM from fresh and doesn't need much more space to install.
You want .NET core? 140MB download. You're configuring database connection strings? Feel free to type in whatever you like, it'll parse and replace with some magic variables that have come from some other random file.
I was using Powershell recently, needed to set an env variable. Bash is happy with "export name=value". You want to do that in Powershell? I just googled it and found an entire 40-minute read discussing how to set env vars. Why?! It should be one command, and I don't know who thought that "Get-ChildItem" was _obviously_ referring to env variables.
It seems to me that everywhere MS has got their hands on development-wise, it inherits the typical sales bullshit. No no, you can't call them "websockets", they have to be branded "SignalR" and add tons of overhead. You can't say "disable notifications" it has to be "focus assist". I'm really surprised something as simple as a keyboard hasn't become a "varied user input device" or something of the like.
Am I alone in thinking this?4