Sequelize transaction - save transaction to commit in the next request

141
July 27, 2021, at 10:30 PM

I'm working with a feature to bulk upload more than 1000 users to DB Many insertions have to be done in different tables for each user

To avoid a timeout I am dividing the upload csv file in chunks of 200 lines and sending each chunk to the backend.

My problem is:

if I have something wrong with the data, and it was not catch by the middlewares, the application crashes. If it crashes in the third chunk, for example, all data from the 2 previous chunks were already inserted in DB.

I need an all or nothing approach. If the application crashes for some reason nothing should be inserted in DB.

Is there a way I can save or acumulate the sequelize transactions until the last chunk is completed and only then commit all transactions at once??

appreciate any suggestion

Answer 1

You can use DB transactions. Db transactions won't cause timeouts (But they may cause deadlocks if you misuse them)

You can use the following Promise for example;

 execInTransaction: function (pool) {
        return new Promise((resolve, reject) => {
            pool.getConnection(function (err, connection) {
                logger.debug("Connection opened " + connection.threadId);
                if (err) return reject(err);
                connection.beginTransaction(function (err1) {
                    if (err1) return reject(err1);
                    
                    // Loop and insert 200 items at each time.. 
                    
                    connection.commit(commitError => {
                        connection.release();
                        if (commitError != null) {
                            return reject(commitError);
                        }
                        return resolve("<some result>");
                    });
                }).catch(txnError => {
                    connection.rollback((rollbackError) => {
                        connection.release();
                        if (rollbackError != null) {
                            logger.error("Connection rollback failed. " + connection);
                            return reject(rollbackError);
                        } else {
                            return reject(txnError);
                        }
                    });
                });
            });
        });
    },
READ ALSO
CSS Custom property Url isn&#39;t working in Asp .Net

CSS Custom property Url isn't working in Asp .Net

I'm trying to style some cards in an mvc project and i have a custom property(variable) in my css "--bg-img" file as shown :

88
Load a Django Form into Template using jQuery?

Load a Django Form into Template using jQuery?

I am using Django function based views with a single template which contains multiple tabsDepending on which tab is selected I hide/show certain elements on the page by catching

93
Android webview app with keyboard up is not working, but with keyboard down it is working

Android webview app with keyboard up is not working, but with keyboard down it is working

In my android app, I am using web viewMy phone has an Android system webview version 91

120