MongoDB Document Migrations

54
October 22, 2019, at 10:30 PM

We currently version our document "schemas" using a __schemaVersion stored against a given document e.g:

db.collection.insertOne({
  _id: "123",
  __schemaVersion: 1,
  name: "Lewis Hamilton"
}

This allows our application code to decide how to handle a document based on __schemaVersion so, for example, if we were to change the schema like this:

db.collection.insertOne({
  _id: "234",
  __schemaVersion: 2,
  firstName: "Lewis",
  lastName: "Hamilton"
});

We have a few options such as ignoring specific schema versions:

const user = db.collection.findOne({ _id: "123", __schemaVersion: { $gte: 2 } });

Or handle both document schemas in code:

const user = db.collection.findOne({ _id: "123" });
switch(user.__schemaVersion) {
  case 1:
     const nameParts = user.name.split(' ');
     return {
        ...user,
        name: undefined,
        firstName: nameParts[0],
        lastName: nameParts[1],
     }
  default:
    return user;
}

These approaches work fine, however there are times where we don't want to support multiple document schema versions as it can make our application code complicated.

I'd therefore like to perform ad hoc document migrations as and when they come into our application. e.g.

const user = db.collection.findOne({ _id: "123" });
switch(user.__schemaVersion) {
  case 1:
    const nameParts = user.name.split(' ');
    return db.collection.findOneAndUpdate({ _id: "123", __schemaVersion: 1 }, { $set: { __schemaVersion: 2, firstName: nameParts[0], lastName: nameParts[1] }, $unset: { name: false } }, { returnOriginal: false });
  default:
    return user;
}

However, I'm using node-mongodb-native and cannot find a hook to ensure that all documents which come into our application are migrated, i.e. I need to be able to hook into every mongodb driver method which possibly returns a document, e.g. find, findOne, findOneAndDelete, findOneAndReplace, findOneAndUpdate and probably many others? -- this would ensure my application has some form of guarantee that it's only working with migrated documents.

Does anybody know if node-mongodb-native offers a hook into the driver which handles this -- or of a wrapper which offers this functionality (ideally not Mongoose, but I would still be interested to hear whether Mongoose supports document migrations).

Additionally it'd be great if it supported a traditional ORM-style migration API where it'd migrate from 1 -> 2 -> 3 -> 4 so migration scripts just have to know how to migrate from one version to the next, e.g. 2 -> 3, not 2 -> 4.

READ ALSO
One package.json for a client + server React project?

One package.json for a client + server React project?

Right now, I have a project structure that looks like this:

29
Use of factory pattern in java

Use of factory pattern in java

is it a valid use of the factory pattern to "convert" objects from a given string? Example: If I have a text file where the data presents values for my object for example a numeric string "00101292877882" and I want it to create an object which will recieve...

38
How to capture log level Error and Warning both in Pyspark?

How to capture log level Error and Warning both in Pyspark?

I have tried to set log level as error or warn, but curious how to capture both ?

35
Inconsistent plotting behaviour with radial bar chart in matplotlib

Inconsistent plotting behaviour with radial bar chart in matplotlib

I'm trying to create a circular bar chart in Python/matplotlibI have the code working fine on my desktop (using Spyder), but the output of the program is completely different on different computers even in the same version of Python (3

49