Can someone try to sell me on NoSQL? I've never seen a use case for it so I want to hear a NoSQL Fan's perspective

I'm not a fan of NoSQL but I do use it.

Typically ill use it for a simple set of data used for lookups to prevent duplicate transactions occurring in ETL based systems.

Or logging where I don't need to worry about the 2.14 billion row issue coming my way at some point.

I use it for the stuff where running a SQL db for 1 or 2 tables doesn't make sense, nor does the cost difference.

Nosql are more specialized than sql and often faster than sql if the data and usage matches the goal of the nosql

For example
Redis is a fast cache database with support for queues and stacks and lists.

Mongodb is a clustered object db that is fast for lookup and allows atomic modifications without having to first load the object, making it easy to read, create and update single objects with no locking but not as good on search for objects

Neo4J is a graph db that excels in keeping track of connections like social network friends and similar.
Very fast to find the next 4 links from one person for example.

But you should almost never aim to use one one nosql since none of them is to be considered a universal db like sql, its more as complement for better performance or parallelism.

It can be your best option as long as the data lends itself to a flat schema. It’s more performant if you’re not doing lookups and/or joins.

It can simplify your stack when developing a restful backend; where you would be converting relational data to flat data, you’re now converting flat data to flat data.

You can actually make a restful backend with no code using couchdb. Which is nice because many people don’t know how to build restful backends.

SQL works only for relational data, and it requires all the data to be normalised. If a product can have multiple images, the images have to be in a separate table. If you want to retrieve a list of products with images, you need to make a join and then perform some grouping on the client-side as well. if you want to create a new product and add the images at the same time, you need to perform a transaction to ensure ACID. If you delete the product, you also need to delete the images (again with a transaction to ensure ACID). If you have a huge database and need to do some sharding, you have to be careful because you don't want your product to be assigned to a different shard than its images.

The whole point of NoSQL is to store a product and its images as a single document, so you don't need to worry about joins, transactions, or sharding which makes your database much faster.

it's much better for a bigger volume of data. i used to work with gps tracking in vehicles and we had an entire object for every position, it was too much for sql to handle well. we used both actually, sql for normal CRUD stuff and nosql for car data.

@darksideofyay when you say it was too much for SQL, what exactly does that mean? Were the queries too slow, or too complex?

@daxinator22 at a certain volume of data sql becomes too slow

I hate "NoSQL" as it says barely anything. It's the same as saying I travel by NoCar™ and than do bold statements about how much faster and environment friendly it is. It's faster if it's an airplane and a long distance. It's super cumbersome if destination is 20KM away and still using an airplane. Walking is very good for the environment and is also a NoCar™ solution.

So let's talk about at least two alternatives to relational databases. Key value store and document store.

We have a large number of items (above 100 million sometimes) that are picked up using a relational database and pushed on a queue for precessing. This is a single query to fill the queue. However each item needs some intermediate state during precessing. We want to process about 10000 items a second. Redis can handle this. It's ridiculously fast and has very little overhead per query. Our relational solution can't. You can use a relational database as key value store but a specialized one is needed is you need a certain performance.

A classical example where a document store fits is a contacts app. There is hardly any relation between the contacts however they do need to support flexible fields. Where you may need to create a separate solution so users can define special field for a contract you get this out of the box with a document store.
I use Google firestore if it's just a collection of data. Advantages:don't need schema/migrations and don't need to pay for a fat instance while getting good performance.

As with any tool use what fits the problem. Don't discount a filesystem, versioning tool, object store (some even have file/dir change triggers making it easy better than polling a queue in relationship table)

@darksideofyay not necessarily, I have worked with tables with billions of rows, thousands of inserts a second and hundreds of queries in sql, it works, BUT you need to know how to index and how to write those queries to not block, and it does have a limit, we probably could not have gone up another magnitude without adding separate tables.

But yrs, for high volume databases SQL can be much harder to use.

Still, if the site is small and you want only one single database, few can compete with a good sql, they are just more versatile.

Most nosql have some use case they really cannot do.

So unless the site is very specific in its requirements I would recommend at least one sql for general storage and then some flavor of nosql for the heavy lifting.

NoSql is split between document stores and key-value stores, maybe more.

I’m not sure why a key-value store would be a good choice for an app (aside from for obvious key-value data like caches) but it is used.
When I was forced to use one (DynamoDb) it involved using the Adjacent List Pattern to access data and simulate joins.
It always felt a bit clunky to me and I don’t think it would ever be my first choice.

For a document db I only have experience with MongoDb. This is completely different to my feelings about DynamoDb, I absolutely love working with it.
It’s very good provided the data you store is organised with some thought. What I mean by that is, if you’re working on a relational db and the system access pretty much every table directly, then that won’t translate well to Mongo.
But if you are working with an object hierarchy that is organised into aggregates, then that translates very well because a document closely meets the idea of an aggregate.

You can join across collections, but it’s not called a join as it works differently. It’s a ‘projection’.

And you can query, there’s a syntax to do it but it’s not traditional sql.

The system I worked on was very very fast and we never had to resort to using a projection or other techniques.

Best way I found to understand it was to do the free Mongo University online training sessions.

Non relational databases = dumb databases storing simple data, but without any bells and whistles holding it back.

@AlgoRythm some of them is not dumb :)

They are specialized.

Sure some are dumb because it suits their goal, like memcached that just stores a blob using a key with no understanding of what the blob is. Its just a cache, noting more, but it fast.

Redis have support for scripts and complex structures. Mongodb can to atomic updates like adding an item to a property of a stored object without loading the object.

All of these are special cases that if it matches you use case will give magnitudes better performance than sql for big data sets.

So dumb is the wrong word ;)