MongoDB database with really relational data. One main collection that had refs to four other collections, all of those references necessary to populate data for a page view. Complicated aggregate to populate all the necessary data and then filter based on criteria selected by the user. And then the client decides that he wants the information to be sortable by column. Some of those columns are fields on the main model, no problem. Others are fields on the refs, which is more of a problem. Especially given that these refs aren’t one single object. They’re arrays of objects.

The revelation was that I could just write an aggregate function to flat map the main collection, returning only the fields necessary for the search, and output it to a new collection and instead use that new collection for displaying and filtering/sorting search results.

But you can’t run the aggregate all the time, you surely say. If anything changes in the main collection, it won’t be reflected in the search results!

Mongoose post(‘findOneAndUpdate’) hooks, my friends. Mongoose post(‘findOneAndUpdate’) hooks.

Never been so happy to have a thing working properly in my life.

tried to stress-test an authenticated websocket endpoint (that makes 2-3 DB calls) by opening closing randomly and it crashed after 20-30 times within a few seconds

I was focused on the middleware glitching out, but error was in the DB-Postgres coz of multiple-connections

Even if I increase the upper limit of simultaneous open connections, the problem at-scale will still exist
If I tried to use a static forever-open connection, it errors out coz 1-command-at-a-time per connection

so im constrained on both sides -.-
Either I rate-limit the endpoint in general and force-close open connections or I cache Organisation-level info that rarely changes

this is one of the few times I miss MS-SQL, it can take a beating but still serve without much complains or losing data consistency -_-