How to do a bulk insert with node-postgres

22,346

Following the clarification provided by the author, to insert up to 1000 records at a time, the solution as suggested within Multi-row insert with pg-promise is exactly what the author needs, in terms of both performance and flexibility.

UPDATE

A must-read article: Data Imports.

Share:
22,346
Fabrizio Mazzoni
Author by

Fabrizio Mazzoni

Interested in programming and databases. Love to take pictures with smart phones and I'm also an aviation enthusiast. My business is industrial cleaning and love to apply technology to business to make people's lives easier!

Updated on July 09, 2022

Comments

  • Fabrizio Mazzoni
    Fabrizio Mazzoni almost 2 years

    I am importing an excel file into a postgres database with express and node-pg

    Currently I am looping through the excel rows and executing an insert for every row but I feel it's not the right way:

    workbook.xlsx.readFile(excel_file).then(function () {
            // get the first worksheet          
            var worksheet = workbook.getWorksheet(1);
            // Loop through all rows
            worksheet.eachRow(function (row, rowNumber) {
                // Commit to DB only from line 2 and up. We want to exclude headers from excel file
                if (rowNumber > 1) {
                    // Loop through all values and build array to pass to DB function
                    row.eachCell(function (cell, colNumber) {
                        arrSQLParams.push(cell.value)                   
                    })
    
                    // Add the user id from session to the array
                    arrSQLParams.push(user);
    
                    // Insert into DB
                    db.query(strSQL, arrSQLParams, function (err, result) {
                        if (err) {
                            console.log(err);
                                ret = false;
                            }
                    })
    
                    // Empty the array for new query
                    arrSQLParams = [];
                }
            })          
        });
    

    Is there a better way to do this to improve performance?