Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Get a devDuck
Rubber duck debugging has never been so cute! Get your favorite coding language devDuckBuy Now
Search - "redis"
I absolutely HATE "web developers" who call you in to fix their FooBar'd mess, yet can't stop themselves from dictating what you should and shouldn't do, especially when they have no idea what they're doing.
So I get called in to a job improving the performance of a Magento site (and let's just say I have no love for Magento for a number of reasons) because this "developer" enabled Redis and expected everything to be lightning fast. Maybe he thought "Redis" was the name of a magical sorcerer living in the server. A master conjurer capable of weaving mystical time-altering spells to inexplicably improve the performance. Who knows?
This guy claims he spent "months" trying to figure out why the website couldn't load faster than 7 seconds at best, and his employer is demanding a resolution so he stops losing conversions. I usually try to avoid Magento because of all the headaches that come with it, but I figured "sure, why not?" I mean, he built the website less than a year ago, so how bad can it really be? Well...let's see how fast you all can facepalm:
1.) The website was built brand new on Magento 126.96.36.199...what? I mean, if this were built a few years back, that would be a different story, but building a fresh Magento website in 2017 in 1.x? I asked him why he did that...his answer absolutely floored me: "because PHP 5.5 was the best choice at the time for speed and performance..." What?!
2.) The ONLY optimization done on the website was Redis cache being enabled. No merged CSS/JS, no use of a CDN, no image optimization, no gzip, no expires rules. Just Redis...
3.) Now to say the website was poorly coded was an understatement. This wasn't the worst coding I've seen, but it was far from acceptable. There was no organization whatsoever. Templates and skin assets are being called from across 12 different locations on the server, making tracking down and finding a snippet to fix downright annoying.
But not only that, the home page itself had 83 custom database queries to load the products on the page. He said this was so he could load products from several different categories and custom tables to show on the page. I asked him why he didn't just call a few join queries, and he had no idea what I was talking about.
4.) Almost every image on the website was a .PNG file, 2000x2000 px and lossless. The home page alone was 22MB just from images.
There were several other issues, but those 4 should be enough to paint a good picture. The client wanted this all done in a week for less than $500. We laughed. But we agreed on the price only because of a long relationship and because they have some referrals they got us in the door with. But we told them it would get done on our time, not theirs. So I copied the website to our server as a test bed and got to work.
So I show their developer the changes and he's stunned. He says he'll tell the hosting provider create a new server set up to migrate the optimized site over and cut over to, because taking the live website down for maintenance for even an hour or two in the middle of the night is "unacceptable".
So trying to be cool about it, I tell him I'd be happy to configure the server to the exact specifications needed. He says "we can't do that". I look at him confused. "What do you mean we 'can't'?" He tells me that even though this is a dedicated server, the provider doesn't allow any access other than a jailed shell account and cPanel access. What?! This is a company averaging 3 million+ per year in revenue. Why don't they have an IT manager overseeing everything? Apparently for them, they're too cheap for that, so they went with a "managed dedicated server", "managed" apparently meaning "you only get to use it like a shared host".
So after countless phone calls arguing with the hosting provider, they agree to make our changes. Then the client's developer starts getting nasty out of nowhere. He says my optimizations are not acceptable because I'm not using Redis cache, and now the client is threatening to walk away without paying us.
So I guess the overall message from this rant is not so much about the situation, but the developer and countless others like him that are clueless, but try to speak from a position of authority.
If we as developers don't stop challenging each other in a measuring contest and learn to let go when we need help, we can get a lot more done and prevent losing clients. </rant>14
The guard at our school thinks I'm hacking
My parents thinks I'm hacking
My teacher thinks I'm hacking
But all I did was only build Redis from source...
FAC : Fucking annoying colleague
FAC: Hey how did you set up your microservices?
M: I used docke...
FAC: But docker is hard to setup, i want an easier option
FAC: Which services do you have?
M: I have one service for the api, one with redi..
FAC: Redis is not a service
FAC: Do you use AWS API gateway?
M: No, in set up my ow..
FAC: why would you set up your own? I just use the one from AWS.
FAC: How many instances are you have running
M: I have 5 replic...
FAC: 5 replicas? That's why i hate microservices,they are costly
FAC: How did you divide up your app?
M: Since I am starting, its better to run the monolithic and then break it up lat...
FAC: I knew it,you don't actually use microservices
M:(thinking)* Fucker, if you know it well why are you fucking disturbing me?? *2
Have been working on a frontend with actual stats for the DNS server I'm building. This is the result so far (real stats, red blocked domains are marked by me (in redis) as surveillance domains), thoughts?menu18
I just spent an hour debugging my company's web app. More specifically, I was trying to fix a bug that made a label on a comment I just made say "Posted 3 days ago".
After confirming timestamps on the server are correct with a calculator, fiddling around with the js debugger, and ruling out weird timezone-related shenanigans, I came to a conclusion.
The bug was in fact sitting, quite comfortably, between the chair and the keyboard.
Yesterday I had moved the date on my computer roughly 3 days into the future, because I was testing out some unrelated code that was dealing with Redis, and I wanted to expire all of the keys stored inside of it.
Don't blame me, my parents told me I had fallen onto my head as a child.5
Someone asked for an RSS feed for the security/privacy blog, I thought?
Well, hereby! There are three feeds:
https://much-security.nl/main.xml - a feed which is updated with both blog posts and external links relating to privacy/security I find interesting/useful.
https://much-security.nl/own.xml - a feed only containing the blogs posts themselves. For people who are only interested in that part.
https://much-security.nl/external.x... - a feed only containing external links. For people who'd like to stay updated on recent cyber security/privacy thingies.
Tracking: every time a feed is visited, a redis value for that feed get's incremented. No time, ip addresses, user agent or whatsoever is saved. Just one variable getting increased once.
New domain name will also be revealed soon (probs tomorrow, going to bed soon as I've just been sick) :D.
Oh and just a warning, the main/external feed are the only ones populated with exactly one item right now :P30
This tiny project is awesome. Thanks to @JoshBent (who partly got it from another repo as well) for providing a basic DNS server with hardcoded blacklisting functionality and thanks to @PerfectAsshole for correcting my mysql syntax I was stuck on for way too long.
I've now got this fucker to read blacklisted words from a redis list into an array which checks every requested domain to see if it matches. If yes, it proxies it through to another DNS server and if not, it'll log the requested domain to a mysql database and prints is as blocked onto the terminal.
If the domain matches any host from a service known to be integrated within a mass surveillance network, it also prints this out to thy terminal.
It's working yay! Gonna keep working on it today.13
if you are looking for
- Java,Python, PHP
- PostgreSQL, Redis, MongoDB
- AWS, S3, EC2, ECS, EKS
- *nix system administration
- Git and CI with TDD
- Docker, Kubernetes
That's not a Full Stack Developer
That’s an entire IT department
Yours truly #stolen10
Management eats shit for breakfast
I am the sole Dev on a project.
Stack: Postgresql, redis, nginx,Java with Spring Boot, Neo4j.
I am the only one nearly familiar with : Redis, Neo4j and anything Java.
I'm gonna be on vacation for the next 15 days since they have told me that we where gonna be on a "testing/feedback" period.
My vacation was approved.
Today's meeting: we have a URGENT deadline to meet some criteria that might be the difference between have further investment or not.
Urgent deadline: last day of my vacation.
My face: poker
My thoughts: attached image4
I think I will ship a free open-source messenger with end-to-end encryption soon.
With zero maintenance cost, it’ll be awesome to watch it grow and become popular or remain unknown and become an everlasting portfolio project.
So I created Heroku account with free NodeJS dyno ($0/mo), set up UptimeRobot for it to not fall asleep ($0/mo), plugged in MongoDB (around 700mb for free) and Redis for api rate limiting (30 mb of ram for free, enough if I’m going to purge the whole database each three seconds, and there’ll be only api hit counters), set up GitHub auto deployment.
So, backend will be in nodejs, cryptico will manage private/public keys stuff, express will be responsible for api, I also decided to plug in Helmet and Sqreen, just to be sure.
Actual data will be stored in mongo, rate limit counters – in redis.
Frontend will probably be implemented in React, hosted for free at GitHub pages. I also can attach a custom domain there, let’s see if I can attach it to Freenom garbage.
So, here we go, starting up modern nosql-nodejs-react application completely for free.
If it blasts off, I’m moving to Clojure + Cassandra for backend.
And the last thing. It’ll be end-to-end encrypted. That means if it blasts off, it will probably attract evil russian government. They’ll want me to give him keys. It’ll be impossible, you know. But they doesn’t accept that answer. So if I accidentally stop posting there, please tell my girl that I love her and I’m probably dead or captured32
Arrived today, hyped!
I have real-world experience with MongoDB, MySQL, Firebase, and caching with Redis.
A colleague in ops recommended this book a while back and I thought I'd give it a whirl to better understand what other options I have available.7
So I am running this crypto project that has dynamically generated private keys for a wallet stored in a Redis database. Nowhere else. The keys are generated on the fly.
At the moment of the happening the wallet had over 3.000 USD on it. I am testing new code locally, supposedly on a local Redis DB. Of a sudden, my code wipes the crypto keys and it turns out that I was connected to the live instance. 😱 Better don't ask me how.
Shock of my life. You know, when you turn pale and dark in your eyes, blood stops in your veins and you just want to die? Worst-case scenario that could have happened. All that money lost in crypto space.
Turns out, my good Redis hosting company kept backups for the past 7 days. Keys restored. Happiest moment of my life.3
Him: Relation databases are stupid; SQL injections, complex relationships, redundant syntax and so much more!
Me: so what should we use instead? Mongo, redis, some other fancy new db?
Him: no, I have this class in Java, it loads all the data into memory and handles transfers with http.
Me: ...... Bye!5
I showed a friend of mine a project I made in two days in Docker and Symfony php. It is a rather simple app, but it did involve my usual setup: Nginx with gzip/cache/security headers/ssl + redis caching db + php-fpm for symfony. I also used php7.4 for the lolz
He complained that he didn't like using Docker and would rather install dependencies with composer install and then run it with a Laravel command. He insisted that he wanted a non-docker installation manual.
I advised him to first install Nginx and generate some self-signed certificates, then copy all the config files and replace any environment-injected values (I use a self-made shell script for this) with the environment values in the docker-compose files.
Then I told him to download php-fpm with php 7.4 alpha, install and configure all the extensions needed, download and set up a local Redis database and at last re-implement a .env file since I removed those to replace them with a container environment.
He sent an angry emoji back (in a funny way)
God bless containerized applications, so easy to spin up entire applications (either custom or vendor like redis/mysql) and throw them away after having played with them. No need to clutter up your own pc with runtime environments.
I wonder if he relents :p9
There's so many cool technologies too look into now a days, but so little time.
Where do i download more hours to add to my days?16
I replaced a python/mysql daily process that takes 25 minutes to run with a perl/redis process that takes 1 minute to run, so it runs multiple times a day. Mgmt asks me to convert it to python/mysql, "...but keep the run time at one minute. That's great!"
I quit my job at a startup because the business guys did not respect my advices in business strategy.
just saw the job post for my position where they write:
"experience with MySQL databases (for example redis and git)"
Now I know that I was wrong. These guys seems to have informations that I do not have 😎2
Fun day, lots of relief and catharsis!
Client I was wanting to fire has apparently decided that the long term support contract I knew was bullshit from go will instead be handled by IBM India and it's my job to train them in the "application." Having worked with this team (the majority of whom have been out of university for less than a year), I can say categorically that the best of them can barely manage to copy and paste jQuery examples from SO, so best of fucking luck.
I said, "great!," since I'd been planning on quitting anyways. I even handed them an SOW stating I would train them for 2 days on the application's design and structure, and included a rider they dutifully signed that stated, "design and structure will cover what is needed to maintain the application long term in terms of its basic routing, layout and any 'pages' that we have written for this application. The client acknowledges that 3rd party (non-[us]) documentation is available for the technologies used, but not written by [us], effective support of those platforms will devolve to their respective vendors on expiry of the current support contract."
Contract in hand, and client being too dumb to realize that their severing of the maintenance agreement voids their support contract, I can safely share what's not contractually covered:
- Stream based programming
- Angular 9
- Any of the APIs
- Dotnet core
- pretty much anything not in a commit
I'm a little giddy just thinking about the massive world of hurt they've created for themselves. Couldn't have happened to nicer assholes.6
<just got out of this meeting>
Mgr: “Can we log the messages coming from the services?”
Me: “Absolutely, but it could be a lot of network traffic and create a lot of noise. I’m not sure if our current logging infrastructure is the right fit for this.”
Senior Dev: “We could use Log4Net. That will take care of the logging.”
Mgr: “Log4Net?…Yea…I’ve heard of it…Great, make it happen.”
Me: “Um…Log4Net is just the client library, I’m talking about the back-end, where the data is logged. For this issue, we want to make sure the data we’re logging is as concise as possible. We don’t want to cause a bottleneck inside the service logging informational messages.”
Mgr: “Oh, no, absolutely not, but I don’t know the right answer, which is why I’ll let you two figure it out.”
Senior Dev: “Log4Net will take care of any threading issues we have with logging. It’ll work.”
Me: “Um..I’m sure…but we need to figure out what we need to log before we decide how we’re logging it.”
Senior Dev: “Yea, but if we log to SQL database, it will scale just fine.”
Mgr: “A SQL database? For logging? That seems excessive.”
Senior Dev: “No, not really. Log4Net takes care of all the details.”
Me: “That’s not going to happen. We’re not going to set up an entire sql database infrastructure to log data.”
Senior Dev: “Yea…probably right. We could use ElasticSearch or even Redis. Those are lightweight.”
Mgr: “Oh..yea…I’ve heard good things about Redis.”
Senior Dev: “Yea, and it runs on Linux and Linux is free.”
Mgr: “I like free, but I’m late for another meeting…you guys figure it out and let me know.”
Me: “So..Linux…um…know anything about administrating Redis on Linux?”
Senior Dev: ”Oh no…not a clue.”
It was all I could do from doing physical harm to another human being.
I really hate people playing buzzword bingo with projects I’m responsible for.
Only good piece is he’s not changing any of the code.3
So I was browsing through some Redis docs and I find this. HOW THE HELL CAN YOU NOT KNOW HOW TO SPELL KOREA!!!!6
I don't know why, but each time I have the chance to create a caching system with redis to e.g. cache requests to APIs I get all excited about it.5
Me: "You could try using Redis, cache that baby and try and squeeze some speed"
Dev: "Hun?! Should I use it on the front end or the back end?"
Well... Webdev is not his thing to be fair!4
It's been a crazy month. Python, redis, Linux. But this one should be a bit closer to home.
Telltale games, a US game development studio is winding up and let go all their staff without severance and were told they had 30 minutes to leave the premises.
Most of the devs were living paycheck to paycheck because of the high cost of living. Some were hired a week earlier and uprooted their families to work there.
That situation is bad enough. But what's worse is the sheer lack of empathy by the customers for the staff. They just want their product cone hell or high water. One even insisted that the devs should work for free until they deliver...
Pleasing the customer should never come at the expense of your staff. We still have a long way to go as an industry
FYI. Copied from my FB stalked list.
Web developer roadmap 2018
Common: Git, HTTP, SSH, Data structures & Algorithms, Encoding
Back-end: PHP, Composer, Laravel > Nginx, REST, JWT, OAuth2, Docker > MariaDB, MemCached, Redis > Design Patterns, PSRs
DevOps: Linux, AWS, Travis-CI, Puppet/Chef, New Relic > Docker, Kubernetes > Apache, Nginx > CLI, Vim > Proxy, Firewall, LoadBalancer
Have been using redis for my new system and wanted to try some gui, so I stumbled on "redis desktop manager", it supports ssh tunnels, privatekeys and more, great isn't it?
BUT IT SAVES YOUR FUCKING PLAINTEXT PASSWORD AND PATH TO YOUR PRIVATE KEY IN %USER%\.rdm\connections.json
WHAT THE FUCK, fucking ask that password during connection, don't fucking save it in plaintext and give an attacker literally the path to my key, wanted to PR it, but fuck c++, probably thats why he doesnt have it, because hes just using some library, so he doesn't have to fuck with the actual implementation of it.2
I am so sick of getting random alerts for basic shit that don't work, like redis or memcached or some stupid shit that's there and seems ok.
i am so going bare metal.2
In love with Laravel events, listeners and mailables. What a beautiful way of doing this. Can't say how much I love this framework. <3
Can't wait to implement redis and queues. Am excited to try this for the first time. Share exp. if you have, pls.6
So this morning something wonderful happened. One of my teams deployed some cache for our API using Redis and they didn't config it!!!
So it's TROLL-time!!! And we have some really cool troll rules if you DOS the QA stack!! So if they resolve it in less than an hour they will only have to use a funny cap all day long (even for lunch) but if it takes more than 1 hour it will be a full cyclist outfit for the whole day (over their own cloth).
PS: if you ever use Redis don't mysql it (why config it? it'll work!!!), because Redis will murder your server!!!4
C: what technologies are you using in this product?
Me: we are using celery which uses kombu to communicate through redis. Additionally, we have beat tasks that run and check the state of workers
C: was someone hungry when they named these frameworks?
Me: Possibly, but I just got hungry naming them.
Got appraises on first day of my job by reducing the calls to redis by half.
All I did was replace SET and EXPIRE with SETEX.1
Just got in a discussion at a Facebook thread when the guy laughed at me because my application server stack uses 20 servers for redundancy and also because I like to make each service (mysql, redis, php) on its own box. He says that he'll teach me how to be awesome because a much larger app that he maintains uses only one server.3
I've implemented an in memory caching system for database queries with Redis in one of the blogs I manage.
Will it work well? Or do you think it will produce issues? I have no experience with Redis yet.14
We recently started using a Redis cache for our application, and our boss has now taken to blaming Redis for every single incident we have - all because he doesn't understand how it works, and doesn't want to >.>
It sometimes really sucks to see how many developers, mostly even much better than me, are too lazy to implement a function to its full UX finish.
Like how can you not implement pagination if you know there's going to be fuck ton of content, how can you not allow deleting entries, how can there be no proper search of content, but instead some google custom search, how can you not implement infinite loading everywhere, but only in parts of your application, how can you not caching a rendered version or improve the page, that loads EVERY SINGLE ENTRY in your database with 11k entries, by just adding a filter and loading only chunks of it.
I know sometimes you need to cut corners, but there's rarely any excuse with modern toolsets to just write 3 lines more and have it ready for such basic things.
I sometimes just wake up during the night or before falling asleep and think "oh, what if in the future he might want to manage that, it's just another view and another function handling a resource, laravel makes that very easy anyway", write in on my list and do it in a blink the next day, if there's nothing else like a major bug.
I have such high standard of delivery for myself, that it feels so weird, how somebody can just deliver such a shitty codebase (e.g. filled with "quick/temporary implementations"), not think of the future of the application or the complete user and or admin experience.
Especially it almost hurts seeing somebody so much more versatile in so many areas than me do it, like you perfectly fine know how to cache it in redis, you probably know a fuck ton of other ways I don't even know of yet to do it, yet you decide to make it such a fucking piece of shit and call it finished.3
Most exited I've been about some code? Probably for some random "build a twitter clone with Rails" tutorial I found online.
I've been working on my CS degree for a while (theoretical CS) but I really wanted to mess with something a bit more practical. I had almost none web dev experience, since I've been programming mostly OS-related stuff till then (C). I started looking around, trying to find a stack that's easy to learn since my time was limited- I still had to finish with my degree.
I played around with many languages and frameworks for a week or two. Decided to go with Ruby/Rails and built a small twitter clone blindly following a tutorial I found online and WAS I FUCKING EXITED for my small but handmade twitter clone had come to life. Coming from a C background, Ruby was weird and felt like a toy language but I fell in love.
The next few months were spent studying and working on my project. It was hard. I had no experience on any web dev technology so I had learn so many new things all at once. Picked up React, ditched it and rewrote the front end with Vue. Read about TDD, worked with PostgreSQL, Redis and a dozen third party APIs, bought a vps and deployed everything from scratch. Played it with node and some machine learning with python.
Long story short, one year and about 30 books later, my project is up and running, has about 4k active monthly users, is making a profit and is steadily growing. If everything goes well, next week I'll close a deal with a pretty big client and I CANT BE FKING HAPPIER AND MORE EXCITED :D Towards the end of the month I'll also be interviewed for a web dev position.
That stupid twitter clone tutorial made me excited enough to start messing with web technologies. Thank you stupid twitter clone tutorial, a part of my heart will be yours forever.2
There's very little good use cases for mongo, change my mind.
Prototyping maybe? Rails can prototype, create/update/destroy db schemas really quickly anyways.
If you're doing a web app, there's tons of libs that let you have a store in your app, even a fake mongo on the browser.
Are the reads fast? When I need that, use with redis.
Can it be an actual replacement for an app's db? No. Safety mechanisms that relational dbs have are pretty much must haves for a production level app.
Data type checks, null checks, foreign key checks, query checks.
All this robustness, this safety is something critical to maintain the data of an app sane.
Screw ups in the app layer affecting the data are a lot less visible and don't get noticed immediately (things like this can happen with relational dbs but are a lot less likely)
Let's not even get into mutating structures. Once you pick a structure with mongo, you're pretty much set.
Redoing a structure is manual, and you better have checks afterwards.
But at the same time, this is kind of a pro for mongo, since if there's variable data, as in some fields that are not always present, you don't need to create column for them, they just go into the data.
But you can have json columns in postgres too!
Is it easier to migrate than relational dbs? yes, but docker makes everything easy also.12
For all the hate that Java gets, this *not rant* is to appreciate the Spring Boot/Cloud & Netty for without them I would not be half as productive as I am at my job.
Just to highlight a few of these life savers:
- Spring security: many features but I will just mention robust authorization out of the box
- Netflix Feign & Hystrix: easy circuit breaking & fallback pattern.
- Spring Data: consistent data access patterns & out of the box functionality regardless of the data source: eg relational & document dbs, redis etc with managed offerings integrations as well. The abstraction here is something to marvel at.
- Spring Boot Actuator: Out of the box health checks that check all integrations: Db, Redis, Mail,Disk, RabbitMQ etc which are crucial for Kubernetes readiness/liveness health checks.
- Spring Cloud Stream: Another abstraction for the messaging layer that decouples application logic from the binder ie could be kafka, rabbitmq etc
- SpringFox Swagger - Fantastic swagger documentation integration that allows always up to date API docs via annotations that can be converted to a swagger.yml if need be.
- Last but not least - Netty: Implementing secure non-blocking network applications is not trivial. This framework has made it easier for us to implement a protocol server on top of UDP using Java & all the support that comes with Spring.
For these & many more am grateful for Java & the big big community of devs that love & support it.
In a time where a web dev is expected to know, well.. everything... Backend -JAVA, python, nodejs and C++ would be great.
Front- angular, react, other 10 libs
DBs -sql, mongo, redis, elastic, kafka, rebbitmq
Also be devops on the side with AWS and docker kubernetis and more stuff
How the f is that possible?
In my real job for the last couple of years and different companies, I usually use 1 language/framework & 1 main DB.. and although it's possible in some companies, but in mine, ppl dont get access to AWS etc..
So let's say there's me.. a server side dev for years.
So I decide to be better and learn Golang.. cool lang, never needed in my job, after few days of not using it I forgot all I learned and that was it.
Then I realized I gotta know some frontend cause everyone want a fullstack ninja nowadays.. so I tried Vuejs.. it was amazing .. never got to use it at work, cause i was a backend, and we didnt use frameworks on our products back then..
Then I decided to learned nodejs, because this is the coolest thing ever.. hated it, but whatever... Never got to use it at work, cause everything was written in other lang which the whole team knew... Forgot the little i knew.
Then I decided, its time to see what Angular is, cause everyone started using it... similar idea to vuejs which i barely remembered, but wow it's a lot of code to remember, or I'll have to google everything.. so I went over it, but can't say i even learned it.
Now Im trying to move on to python, which, I really am learning in depth.. however, since I dont have real experience with it, no one gives me a shot at being a python dev, so again i feel like I'm trying to memorize syntax and wasting my time..
Tired of seeing React in all job ads, i decided to have a look what's that all about.. and whadoyaknow... It's fucking the same idea as vue/angular with again different syntax..
THIS IS CRAZY!
in how many syntaxes do i need to know how to make a fucking crud api, and a page with same fucking post form, TO BE A GOOD PROGRAMMER?!?6
In Django code, looking at a class for caching REST calls. The cache is using Redis via Django's cache layer. In order to store different sets of parameters, each endpoint gets a "master" cache, that lists the other Redis keys, so they can be deleted when evicting the cache. Something isn't right, though. The cache has steadily increased in size and slowed down since 2014 even though many events clear the whole thing!
... And then it hit me. Nothing empties the list of cache keys. Nothing. So it has been growing endlessly since 2014. And everytime it grows, cache eviction gets a little more expensive, network traffic increases a little more, and cache evictions get a little slower.
Fixing this bug took things that were taking routinely an entire minute to complete and made them take a couple seconds.
I'm in the first semester at my third college, working on my B.S. degree. I get a job in the finance and operations IT department doing web development. I get to use lots of tech I've never used before, like Python Flask, Docker, Redis, Azure, Slack, Microsoft VSTS, Portainer, MongoDB, Oracle database (weeeeeeeeeeee), and who knows what else awaits me. It took a week to get me access permissions to various systems but that's fine. I expected that.
* It's the first time I've ever touched Docker. I manage to break it so badly (somehow) that I BSOD my Win10 machine. Everyone in the office congratulates me on such a feat.
* I get assigned to an internal app management site to not only patch because it's currently broken but also improve. Coming from a PHP background, I've seen some nasty code. What I didn't expect was the same exact same anti-patterns and coding "practices" (examples: everything in a single file, all forms on a page POSTing back to itself for submission, sloppy, mostly uncommented, densely packed code) in a Python Flask project. I was told it was thrown together quickly by a previous employee who made this as his first Flask app, but this was not what I envisioned.
It's as if everything I do in my career is to prepare me to untangle the next monster code base I come across. I feel like the universe is mocking me.
That's cool. Bring it on. I'm ready. I'm enjoying this job anyway. 😎3
I dug up my old ledger web app that I wrote when I was in my late twenties, as I realized with a tight budget toward the end of this year, I need to get a good view of future balances. The data was encrypted in gpg text files, but the site itself was unencrypted, with simple httpasswd auth. I dove into the code this week, and fixed a lot of crap that was all terrible practice, but all I knew when I wrote it in the mid-2000s. I grabbed a letsencrypt cert, and implemented cookies and session handling. I moved from the code opening and parsing a large gpg file to storing and retrieving all the data in a Redis backend, for a massive performance gain. Finally, I switched the UI from white to dark. It looks and works great, and most importantly, I have that future view that I needed.1
I know I'm pretty late to the party, but I've been playing with Redis a lot lately and it's pretty awesome. Sorted sets and the various Z functions seem very powerful. I'm hoping to get to use it in a prod environment soon.2
So a few days ago I sat down to write a redis adaptor to transfer data back and forth between redis and elasticsearch. I download the go-redis package and start writing a simple client.
I run the client and it gives me an error. So I'm stuck at it for about 30 mins and then I say to myself, "You dumb fuck you haven't started the redis-server". So I open up another terminal and type in `redis-server` and then I realise I don't even have redis installed on my machine.
I do such dumb things every weekend. If you have any dumb mistakes you made while writing code please share them in the comments. :-)
const topic = 'Laravel'
const subtopic = 'Queues'
Have you used Redis? Have you used it as your Queue driver? Is it cool? Are there better alternatives?5
Ugh, I feel like fucking crying every time I have to explain to the other developers and network sysadmins that we can't have the same Redis server for both session and caching data in the current project we're working on. I wrote all this down in the specifications which NO ONE reads!!
There are times like these I wish I just had access to the AWS account so I can do all of this myself.3
I usually hate Microsoft but they've gotten me positively surprised several times lately. I can run an SSH server and Redis just like that!
Trying to clear the redis cache for like half an hour, wondering why the redis server isn't even being filled with keys...
Then suddenly, FUCK wrong port
2:30 AM is too late for me apparently :(
Fuck you Redis
Goes in a docker container, calls bgrewriteaof, get success, checks info - no pending writes, last write success.
Tries to scp to remote, fails - Unexpected AOF
Decides to shut down the local redis to be able to port, in case it's blocking it
Calls redis-cli shudown (expect to just shut redis down rigth)
It fucking deleted all my data, now I see the docs
"Flush the Append Only File if AOF is enabled."
Why the fuck? Fuck you redis, fuck you1
In the past, apps I've written have used a flat file backend. It's very fast, but obviously clunky to have a big structure of flat files for an app. It ran circles around framework-based RDBMS backends, as performance is concerned, but again, it was clunky. Managing backups and permissions on tens or hundreds of thousands of small files was no fun. Optimizing code for scaling was fun- generating indexes, making shortcuts -but something was still missing. Early in 2017 I discovered redis. A nosql backend that just stores variables and lives almost entirely in memory. Excellent modules and frameworks for every language. It was EXACTLY what I'd needed, even though I didn't know I did. I spent a good deal of time in 2017 converting apps from flat files to redis, and cackled with glee as they became the apps I wanted them to be. Earlier this week, I started building my first app that started with redis, instead of flat files, and I can't stop gushing to anyone who will listen. Redis for president!
Im building a dead simple online store. So far the options are:
0) heroku, with redis
1) aws lambda functions and some aws storage
2) set up a vps
The domain will be a cheap .store domain, and 1 and 2 should not exceed the limit where its not free anymore. I dont want wordpress or anything premade, i dont expect more than 50 orders. Is there a more stupid solution that lets me try out different providers?13
whoever the fuck created the docker redis image:
why the fuck are you warning me hugepages are enabled and it eats up cpu ram when you don't let me disable it?
was it your point to waste 1h of my time? well you did. fuck you very much.4
This week I am finishing my brand new and probably buggy cluster manager for vert.x based on redis.
Can't wait to start battle testing.2
- Finish "Introduction to algorithms"
- Learn some genetic algorithms
- Get my hands dirty on reinforcement learning
- Learn more about data streaming application (My currently app is still using plain stupid REST to transport image). I don't know, maybe Kafka and RabbitMQ.
- Learn to implement some distributed system prototypes to get fitter at this topic. There must be more than REST for communicating between components.
- Implementing a searching module for my app with elastic search.
- Employ redis at sometime for background tasks.
- Get my handy dirty on some operating system concepts (Interprocess Communication, I am looking at you)
- Take a look at Assembly (I dont want to do much with Assembly, maybe just want to implement one or two programs to know how things work)
- Learn a bit of parallel computing with CUDA to know what the hell Tensorflow is doing with my graphic card.
- Maybe finishing my first research paper
- Pass my electrical engineering exam (I suck at EE)1
Is GraphQL worth it? It promisses to keep some load away from backend programmers but what about this scenario:
There's a list of items with scroll load/infinite scroll. There can be several filters as well as the Option to change the ordering of the list items.
With "traditional" REST, I'll hit the DB with one request, get the data in the right order to the backend, might itterate over it once to add additional information, cache the result in Redis/Memcache and send it off.
Using GraphQL, the frontend has to load all entries first, sort them in JS (which probably is slow on mobile devices), and then display it. No matter how "expensive" the query is, there's no caching.
Is that about it? Did I get something backwards?3
All those mine WTF moments are somehow related with caching which i keep on forgetting... the most fresh one was last week, i had some GIGANTIC mySQL query, and for the sake of response time I immediately made a cache function that kept Redis cache for a day or so... so last week i had to change something (good ol' client and his visions for app). So there i was with the query that returned same god damned results every time, i copy the query in some mySQL manager and it goes fine, but in the app it doesn't... what the actual FUCK!!! i was questioning my career until i figured it out, i was planning to buy some sheeps and a fife and to hell with this, a loud facepalm was echoed through the office that day...3
New startup - our backend stack is php, js, mysql, mongo & redis
Can you guys recommend nice-to-have internal tools that could make out lifes easier? We’ve been using confluence so far and thinking on grabbing a jira license.
Any advice is helpful 🤪11
Damn, I'm conducting some benchmarks on go and php...
A php page that simply returns "hi" with no framework is a lot slower than 1000 calls with 200 concurrency to a go function that queries redis and returns a json.3
Would you suggest MacBook Air Core i5 8th gen model for a hardcore Android Developer?
Current usage on regular basis -
- Android Studio
- 2 Android VM (MeMu, Genymotion etc)
- node.js MongoDB, Redis for server
- VS Code, Chrome, Mongodb compass
My old dell worked pretty well so far it has a core i3 with 8 gigs of RAM and 256GB SSD but processing always seemed slow but managed some how.
Suggest me if MacBook would make better choice than other windows laptop which are much more high end than MacBook on same price?13
To all Full Stack js employed devs here,
How much frontend vs how much backend you do?
It's just that I'm going to be in a job with a Full Stack node/vue/ember/AWS/Redis dev role and I don't know how much frontend I'll do compared to the backend stuff, I'm alot more of a backend guy...1
I was determined PHP advocate, always ready for debate with PHP criticizers. I am stacking with dozen other languages so I used to think I have all right to do just that. My code is fully OO, I used to scale FPM horizontally, eventually, with help of pthreds even vertically. With help of redis and chaching, I thought I was sorcerer, as I always find a way (or way around) to make things work, things that no one used to beleive it's possible. One day I started to work for language engineering company, when I suddenly realized how PHP often fails with it's come to localizations, translation, exotic charsets and over all multibyte operations. :( Whole this thing collapses. Wholes everywhere...3
Hey, Do you think it's common to get an interview take-home test like this...
Write a Python script that:- Downloads the Equity bhavcopy zip from the above page- Extracts and parses the CSV file in it- Writes the records into Redis into appropriate data structures(Fields: code, name, open, high, low, close)
Write a simple CherryPy python web application that:- Renders an HTML5 + CSS3 page that lists the top 10 stock entries from the Redis DB in a table- Has a searchbox that lets you search the entries by the 'name' field in Redis and renders it in a table- Make the page look nice!
Commit the code to Github. Host the application on AWS or Heroku or a similar provider and share both the links.14
Which is best package to be used as a PHP Session handler (https://cloudways.com/blog/... )?
Redis or Memcached?
!rant, so I'm trying to decide on what caching system I should use. It's for a PHP app, using Symfony as framework, tlgether with Doctrine for DB. The caches in Question are memcached, APCu or redis.
The goal: speed shit up.
The app currently uses Symfony 2.8 and is hosted on a single server (so no distributed system is needed). I'd currently opt for APCu, but more since it's not distributed, there won't be an overhead from that. A nice thing about memcached would be the abillity to store user seeions, even if we would decide to have multiple servers in the future.
What would you reccomend and why?3
What do you guys think about deploying elastic search on App Engine Custom Runtime?
(Basically, an empty folder with an elastic search Dockerfile.)
I think it's a good idea: you can now deploy your code and storage application (Elastic search, Redis, etc) as services on your cluster.
You can use GCP magic to auto scale those services, you have so many good stuff that come with it.
And it's inside the same network as your services running in the same AppEngine project.1
Need some advice on how to handle a unique problem. Im using Googles location services api with react natives geolocation service to fetch a clients location and gather more details on it. I essentially need users to be able to see other users on a map within the live time of their location. Im thinking about figuring what users are nearby to show by storing each users location in redis until the end of their session and fetching within a certain longitude/latitude range of the given user's location. Is this a solid way of looking up local users or is there a more solid approach to mapping users.4
I finally published my first medium article, It's on Sticky sessions for microservices.
It took so much of my time to piece everything together that I wrote a small how-to, so that my fellow devs should not suffer like I have.
Any one thoughts/opinions about Azure Service Bus? I'm using it a few months now in combination with a redis cache, cloud storage and the service bus.. works pretty nice so far..
I'm pretty impressed about the upgrade mechanism..
Anyone else pissed at the trend towards massive, clunky OO libraries in PHP (and other languages)?
I needed to clear Redis and the amount of code to use the library was more than the amount of code to open a socket and write the command.1
I need to know if I’m just going insane or if this is common in other work environments. I’ve had three feature improvements now that required me to connect to our staging Redis to obtain a hash map example that I would then manually put in my Local redis so that the process I am running can successfully run. That’s just one terrible example, I feel like I am constantly making changes to mongo and redis in order to make my environment act the way it is supposed to just so I can properly test my jobs. I don’t think I’ve ever written an application that has resulted in developers having to have a deep understanding of every database abstraction just so they can claw their way through manual database entries so they can continue doing their job5
I’m trying to add caching functionality in my scalable spark cluster on Docker. I am able to add redis to my Docker container, but that doesn’t add redis-CLI or any other useful tools that I need. And there are some redis-cache projects on PyPi, but they are very old and not compatible with python3.5
"[Elasticache isn't a managed service because] You are still choosing instance size, setting up replication and clustering more or less without their help"
From their website:
"Amazon offers a fully managed Redis service, Amazon ElastiCache for Redis"
Just because it's configurable doesn't mean it's not managed.
How (generally) do offer different persistence layers for an app?
So, I have used lots of apps (sorry, I'm talking a proper software system such as a Web based service (e.g. The Open Source XMPP server 'Openfire') in which you can choose what persistence back end you want (MySql or inbuilt H2/SQL light for example).
Within your code, how do you go about achieving this? Would you delegate the persistence to a separate class, and within that class figure out what the systems settings are and use the right connection string?
I'm currently using Java, Hibernate and would like to offer back ends of MySql, H2 and Redis, but the question is more conceptual than specific.
My load balancer with docker and redis database works fine bu5 I can’t access my web service. I get a “connection refused” error message.6