Laravel - seeding large SQL file - sql

Question

A memory exhaustion happens when I run my DB seed script in production.
Below is my seed script.
class MembershipTableSeeder extends Seeder
{
public function run()
{
DB::table('members')->delete();
foreach (range(1, 99) as $days){
Members::create(array('membership_code' => 'test'.$days));
}
DB::unprepared(file_get_contents(app_path()."/database/seeds/members.sql"));
}
}
So what I did was add a no-limit on my seed script.
ini_set('memory_limit', '-1');
The problem now is that when I run the script it logs the output into the terminal the content of the SQL script (which is very, very big).
Is there a good way of running a SQL dump inside my DB seeds that doesn't consume much memory? What I did now was run it manually:
mysql -uuser -p db < script.sql

Answer 1

For others who prefer a more Laravel-ish solution, this is how I handled it:
/**
* This class is responsible for running the data dump sql.
* It is recommended to update this class instead of creating new ones for new database content dumps.
*/
class DatabaseDumpSeeder extends Seeder
{
/**
* Run the database seeds.
* #throws \Exception
*/
public function run()
{
// Note: these dump files must be generated with DELETE (or TRUNCATE) + INSERT statements
$sql = file_get_contents(__DIR__ . '/dumps/dump-20150709.sql');
if (! str_contains($sql, ['DELETE', 'TRUNCATE'])) {
throw new Exception('Invalid sql file. This will not empty the tables first.');
}
// split the statements, so DB::statement can execute them.
$statements = array_filter(array_map('trim', explode(';', $sql)));
foreach ($statements as $stmt) {
DB::statement($stmt);
}
}
}

Answer 2

The problem happens because when using Db::unprepared it also logs the query to the laravel.log file, making in background much more actions then you think, from this side you have memory exhaust. If you are not running the safe mode I would stick to executing the console command like this:
exec("mysql -u ".\Config::get('database.mysql.user')." -p".\Config::get('database.mysql.password')." ".\Config::get('database.mysql.database')." < script.sql")

Answer 3

Create Seeder File "PostalCodeTableSeeder.php" in
Project_directory/database/seeds
use Illuminate\Database\Seeder;
class PostalCodeTableSeeder extends Seeder {
/**
* Run the database seeds.
*
* #return void
*/
public function run()
{
// =============================================================
// file Path -> Project/app/configs/database.php
// get the database name, database username, database password
// =============================================================
$db = \Config::get('database.connections.mysql.database');
$user = \Config::get('database.connections.mysql.username');
$pass = \Config::get('database.connections.mysql.password');
// $this->command->info($db);
// $this->command->info($user);
// $this->command->info($pass);
// running command line import in php code
exec("mysql -u " . $user . " -p" . $pass . " " . $db . " &lt postal_codes.sql");
// postal_codes.sql is inside root folder
}
}
Also add the class name into
Project_directory/database/seed/DatabaseSeeder.php like code below
use Illuminate\Database\Seeder;
class DatabaseSeeder extends Seeder
{
/**
* Run the database seeds.
*
* #return void
*/
public function run()
{
$this->call(PostalCodeTableSeeder::class);
// $this->call(UsersTableSeeder::class);
}
}

Answer 4

I had a strange issue where importing a large SQL file as a migration caused the line to not be added to the migrations table.
This is how I fixed it.
$path = 'database/data.sql';
$command = "mysql -h".env('DB_HOST')." -u".env('DB_USERNAME')." ".(env('DB_PASSWORD')?"-p'".env('DB_PASSWORD')."'":'')." ".env('DB_DATABASE')." < ".$path;
exec($command);
I notice that there are similar answers to this, but my method checks .env and not the app config, support for remote MySQL hosts, also works with no password on local and passwords with special characters that would break the command line.