Inserting many records to database using Laravel's Chunk method becomes slower over time

157
June 07, 2018, at 03:50 AM

So basically I am trying to import data into around 15 different tables into my database from another. I am chunking the data from the source database, and within that function callback I am inserting it into my database using raw SQL. Code example:

foreach($tables as $table) {    
     DB::connection('source_db')->table($table)->select($fields)->orderBy('id'))->chunk(2500, function($records) use($table) {
          $this->runInsertQuery($records, $table);
     });
}

There are around 2 million records total, it seems that it starts out very fast but gets slower and slower as it moves from table to table, taking hours to complete. I was under the impression based on Laravel's documentation that chunking would make inserts run much faster, but it seems to be having the opposite effect. Any guidance would be greatly appreciated!

Rent Charter Buses Company
READ ALSO
Multi Jwt Auth with Laravel and Mongodb

Multi Jwt Auth with Laravel and Mongodb

I have three types of Authenticatable model and I need to have separate JWT authentication for eachLet me explain more about my issue

214
Uncaught Error in wordpress theme

Uncaught Error in wordpress theme

I am editing a theme and modifying it, i changed the name of all the files containing the original name of theme, and i also edited all files an replaced the name of theme with my name, still i got a error when i preview the theme

247
Opencart 3.x - How to get store_id in product page

Opencart 3.x - How to get store_id in product page

I have a multi-store OC3 and I want to do some changes on the product template of a store without changing anything of othersI want to get store_id in the product page

313
Route [voyager.database.bread.edit] not defined. Voyager Laravel 5.6

Route [voyager.database.bread.edit] not defined. Voyager Laravel 5.6

Can someone tell me what is this about? It works fine on my local server(i use valet) and on a live websiteBut on staging server when I try to open a database in the voyager admin panel it "Whoopses" and throw me this route not defined error

259