MongoDb integrating with external db

328
July 01, 2017, at 4:25 PM

I have a database which contains data from two separate systems/servers. The first is generated locally [I develop and create this data] (users, activity logs, orders, ...). The second comes from a "product provider" [I only have READ access from API] These objects were created by MySQL and sent in JSON. They already have an "id" property.

With NodeJS, I use request to get a product by "id", and then store it with newProduct.save() appends an _id.

In products, "id" is necessary form relationships with the other collections in my database (such as products_price), and access dynamic endpoints, such as "products/:id/promos".

Note that products are constantly being updated externally and I need to be able to update my documents by "id" not by "_id" as the external server has no knowledge about "_id." [id is unique on a collection level, as each collection is a fresh iteration]

For my first question: should I treat "product.id" as a "regular" MongoDB field and use aggregate/lookup to merge documents from my collections? Or should I overwrite ObjectID() with id? (before saving rename "id" to "_id")

At some point, Orders (local) and Products (external) need to form a relationship where Order _id and Product id (or _id) are stored together for easy retrieval.

Which id do I use in this case?

Answer 1

if you are pretty sure that 'id' coming from your product provider API is unique you better use that as _id (overwrite _id), it will save you:

  • an unneeded index ('_id' is indexed any way)
  • some CPU cycles that mongoDB would take to produce the ObjectID
  • some disk and memory space

(*) even if you find yourself dealing with many different product providers, assuming its one is using his own unique product id you could use a combined _id to make it unique as:

  • _id = {provider: 'foo', id: xxx}
  • or _id = [provider_name, product_id]
  • or _id = provider_name + product_id etc. etc.

    in this use case of multiple providers format depends on how you plan to fetch those products later.

READ ALSO
Rangeslider.js does not install

Rangeslider.js does not install

I am trying to install and use the rangesliderjs API, I have downloaded the latest version and I have gone about installing it via the terminal using both npm and bower install commands

322
Deploying Angular 2 (quickstart app) to Google Cloud platform

Deploying Angular 2 (quickstart app) to Google Cloud platform

I was trying to deploy the simplest project on Google Cloud platformThe quickstart project can be found here

356
The Optimal Way to Generate Android Apps on a Node Server

The Optimal Way to Generate Android Apps on a Node Server

I am building a service that generates android apps using a template, the api receives a json file with data that is used to modify some of the template code, then I generate the app and send it back to the clientHere is how I currently do it:

209
How to remove boundary info from cURL post into node busboy -> writestream

How to remove boundary info from cURL post into node busboy -> writestream

The Issue: Resulting uploaded json file contains the curl boundary information

230