I'm implementing a REST API that returns data as it's fetched from MongoDB.

If it's returned all at once it would be a JSON a string like [{}, {}, ...]

But what about when it's returned as a Stream?

Node by default seems to just return it as {}{}{}{} but that can't be parsed by the program that requested it?

What you need to understand is that they are returned as a stream no matter what.

HTTP sends data in chunks. The client has two options:

Hold that data in memory somewhere and process it after it has been fully downloaded (great for most things)

Process the data as it comes in (great for really big things, not possible in every case)

The actual server sends it as a stream no matter what, and the client is connected via a stream as well no matter what. Your processing is the only thing that may not be a stream.

@AlgoRythm so if the server returns 2 JSON documents that have inner objects, they would be getting it/reading it character by character basically?

So they would need to count the {} in order to figure out when one is fully received?

Or can we just randomly agree the a special char like ^ delimits each document?

And if they want to parse the whole thing at once, they would need to add the [] and replace the ^ with , themselves?

I'm wondering if there's a convention regarding returning JSON array as a Stream I guess.

@billgates stream parsing JSON is possible, and even encouraged. You'd need valid JSON, so you can just have an arbitrary number of objects, they need to be in an array. And yeah, every time the parser finds a complete and valid token (such as a string) it completed processing THAT part of the data.