Does anybody have some good suggestion for horizontal scaling solution of relational database (MySQL) that's not commercial? Both read and write scalings are of relevance btw.

You want add more hardware?

It's horizontal - more machines.

if you need to scale you're probably comercial?

@heyheni good point, but I'm not a project owner and their decision is to go with something non-commercial, while I'm not sure if that even exist.

@myss what's the scope? How big do you need it to be?

The problem here is not so much in read scaling (ofc its also important) as in write scaling. We're building a central system that'll generate around 100GB of data every month.

What would be perfect solution is something like ClustrixDB.

Go commercial, go aurora.

Edit: oh well, you're talking about writing scaling, forget aurora. What about a beefy master (for writing) and other not so beefy replicas

@myss 100gb? per month for free in the cloud?

Tell your project owner he is dreaming! And that he should get that CORPORATE credit card. For a service like Amazon RDS for MySQL, Google Cloud SQL or Microsoft Azure SQL.

> 100GB per month
> non-commercial

L
O
L

@heyheni @devTea what's even better, we as contractors don't know where'll this system be deployed. Can happen to be on AWS, can be some other cloud-based service, can be their own datacenter (read probably server room somewhere in basement).. So yeah I know its fucked up, just wanted to check with bright minds of devrant, that maybe I wasn't missing something 🤷‍♂️🙈

@myss yeah I feel you, this place is great for not SO like questions

@myss sounds like a shitty project doomed to fail

@Kimmax it's actually pretty cool project (actual concept is too log as much of data as possible from automated stations based on several different protocols, am working on it for couple of months now and no rants until today). They'll just need to step up their money game a bit more together with transparency toward developer team in order not to fail.

Do you want to create another hardware for MySQL database as Slave where you can read data easily without any interference with master. Except for slight data inconsistency relative to time.

I had worked on legacy system.
So, i am using slave for reporting.
And master write operations.

And create best database design and fucking indices too.

I have created an indices on database table with 10 millions records. Fucking hell.

@myss if it's about logging large amouts of time sensitive data then maybe mysql isn't the ideal database?

Wikipedia - Time Series Database
https://en.m.wikipedia.org/wiki/...

example:
Dataloging with https://www.influxdata.com/
and dashboard analytics with https://grafana.com/

@heyheni hmm never worked with (or even heard of) TSDB before but thank you, will deffo lookin those

AWS just started with one of it's own Called AWS Time Stream

"1000 times faster and at a 10th of the cost of relational databases"

https://aws.amazon.com/timestream/

So far I heard of a few, never tried them myself though. All of them provide a commercial, managed version which can be scaled very easy but also give the option to run it manually.

For MySQL compatible scalable databases there is TiDB https://www.pingcap.com/en/
This is inspired by GCPs Cloud Spanner.

If PostgreSQL is an option, there are Citus Data https://www.citusdata.com

And also CockroachDB (although I thought they started out as a better Cassandra database but seems they're now postgres compatible)
https://www.cockroachlabs.com

You get cheap horizontal scaling in every rotten Thai brothel, STDs like HTML included.

@myss any news? 🙂

@-red mate you know quite well why :)

Regarding this topic, well currently they're passively skipping the issue on standup calls (like ahh its not a big issue) so it makes me think they'll probably opt out on spitting some $$$

Will definitely let you guys know of the outcome if/once we tackle the issue.

On a side-note, lately I've been looking in on database sharding, anyone of you good peoples has experience with it? https://doctrine-project.org/projec...

do you know graphQL?
https://www.graphql.com/

just slap it on top of your various databases and you get one interface you can query them all with a json like syntax.
www.prisma.io