I'm trying to execute SQL script that has lot of DML commands using Perl.
Basically there is some reference number which is required to be updated every time in SQL script before it executes, so I'm passing it as a parameter to Perl script which will further replace with parameter and execute. I'm doing this task in IntelliJ IDEA 2022.3.1 (Community Edition).
Problem: Unable to understand but Database connectivity is completely fine, and is fetching SQL script correctly, even replacement of reference number is also fine. But when trying to execute, it's not doing any changes in the Database. It looks like SQL commands are not really executing (as it is executing quickly which is not supposed to). I added warn with execute, so I got that warn message, but there is no error/exception message.
Please try to understand more details via code.
If anywhere I'm doing it wrong, please tell me. I have basic knowledge of Perl so might be doing it wrong. Can't provide more details about SQL script, it is confidential. But I can make sure that there is no error, as it is working fine if execute in it's dedicated tool. Thank you in advance!
#!/usr/bin/perl
use strict;
use warnings FATAL => 'all';
use DBI;
# Connect to the database
my $EnvDatabase = $ENV{"DATABASE"};
if (! defined $EnvDatabase) {
&ERROR ("The environment variable DATABASE is not defined");
exit (-1);
}
print "Updating in Database: $EnvDatabase\n" ;
my ($DBUser, $DBPass, $SID) = split (/\/|\#/,$EnvDatabase);
if (! (defined $DBUser && defined $DBPass && defined $SID )) {
&ERROR ('The environment variable DATABASE is not properly defined');
exit (-1);
}
my $dbh = DBI->connect( "dbi:Oracle:".$SID, $DBUser, $DBPass,{AutoCommit=>0,PrintError=>0})
|| &errorExit( "Unable to connect to database [$DBI::errstr] ");
# Get the input parameters
my $param1 = shift;
my $param2 = "###123###";
# Read the SQL file into a variable
my $sql = do {
local $/ = undef;
open my $fh, "<", "C:/someScript/someSampleSqlWithDMLCommands.sql" or die "could not open file: $!";
<$fh>;
};
# Replace with the parameter
$sql =~ s/$param2/$param1/g;
# Execute the SQL statements
$dbh->do($sql) || warn("Unable to execute ", $dbh->errstr);
$dbh->commit;
$dbh->disconnect;
OUTPUT: Updating in Database: database name declared in environment
Unable to execute ORA-00900: invalid SQL statement (DBD ERROR: OCIStmtExecute) at C:/**/autoScript.pl line 37.
Process finished with exit code 0
Related
I would like to find a way to run .sql file containing PL/SQL in PowerShell using .NET Data Proider for Oracle (System.Data.OracleClient).
I would deffinitely avoid using sqlplus for this task.
This is where I am now
add-type -AssemblyName System.Data.OracleClient
function Execute-OracleSQL
{
Param
(
# UserName required to login
[string]
$UserName,
# Password required to login
[string]
$Password,
# DataSource (This is the TNSNAME of the Oracle connection)
[string]
$DataSource,
# SQL File to execute.
[string]
$File
)
Begin
{
}
Process
{
$FileLines = Get-Content $File
$crlf = [System.Environment]::NewLine
$Statement = [string]::Join($crlf,$FileLines)
$connection_string = "User Id=$UserName;Password=$Password;Data Source=$DataSource"
try{
$con = New-Object System.Data.OracleClient.OracleConnection($connection_string)
$con.Open()
$cmd = $con.CreateCommand()
$cmd.CommandText = $Statement
$cmd.ExecuteNonQuery();
} catch {
Write-Error (“Database Exception: {0}`n{1}” -f $con.ConnectionString, $_.Exception.ToString())
stop-transcript
exit 1
} finally{
if ($con.State -eq ‘Open’) { $con.close() }
}
}
End
{
}
}
but I keep getting following error message
"ORA-00933: SQL command not properly ended
The content of the file is pretty basic:
DROP TABLE <schema name>.<table name>;
create table <schema name>.<table name>
(
seqtuninglog NUMBER,
sta number,
msg varchar2(1000),
usrupd varchar2(20),
datupd date
);
The file does not contain PL/SQL. It contains two SQL statements, with a semicolon statement separator between (and another one at the end, which you've said you've removed).
You call ExecuteNonQuery with the contents of that file, but that can only execute a single statement, not two at once.
You have a few options. Off the top of my head and in no particular order:
a) split the statements into separate files, and have your script read and process them in the right order;
b) keep them in one file and have your script split that into multiple statements, based on the separating semicolon - which is a bit messy and gets nasty if you will actually have PL/SQL at some point, since that has semicolons with one 'statement' block, unless you change everything to use /;
c) wrap the statements in an anonymous PL/SQL in the file, but as you're using DDL (drop/create) those would also then have to change to dynamic SQL;
d) have your script wrap the file contents in an anonymous PL/SQL block, but then that would have to work out if there is DDL and make that dynamic on the fly;
e) find a library to deal with the statement manipulation so you don't have to work out all the edge cases and combinations (no idea if such a thing exists);
f) use SQL*Plus or SQLcl, which you said you want to avoid.
There may be other options but they all have pros and cons.
I am unfamiliar with linux/linux environment so do pardon me if I make any mistakes, do comment to clarify.
I have created a simple perl script. This script creates a sql file and as shown, it would execute the lines in the file to be inserted into the database.
#!/usr/bin/perl
use strict;
use warnings;
use POSIX 'strftime';
my $SQL_COMMAND;
my $HOST = "i";
my $USERNAME = "need";
my $PASSWORD = "help";
my $NOW_TIMESTAMP = strftime '%Y-%m-%d_%H-%M-%S', localtime;
open my $out_fh, '>>', "$NOW_TIMESTAMP.sql" or die 'Unable to create sql file';
printf {$out_fh} "INSERT INTO BOL_LOCK.test(name) VALUES ('wow');";
sub insert()
{
my $SQL_COMMAND = "mysql -u $USERNAME -p'$PASSWORD' ";
while( my $sql_file = glob '*.sql' )
{
my $status = system ( "$SQL_COMMAND < $sql_file" );
if ( $status == 0 )
{
print "pass";
}
else
{
print "fail";
}
}
}
insert();
This works if I execute it while I am logged in as a user(I do not have access to Admin). However, when I set a cronjob to run this file let's say at 10.08am by using the line(in crontab -e):
08 10 * * * perl /opt/lampp/htdocs/otpms/Data_Tsunami/scripts/test.pl > /dev/null 2>&1
I know the script is being executed as the sql file is created. However no new rows are inserted into the database after 10.08am. I've searched for solutions and some have suggested using the DBI module but it's not available on the server.
EDIT: Didn't manage to solve it in the end. A root/admin account was used to to execute the script so that "solved" the problem.
First things first, get rid of the > /dev/null 2>&1 at the end of your crontab entry (at least temporarily) so you can actually see any errors that may be occurring.
In other words, change it temporarily to something like:
08 10 * * * perl /opt/lampp/htdocs/otpms/Data_Tsunami/scripts/test.pl >/tmp/myfile 2>&1
Then you can examine the /tmp/myfile file to see what's being output.
The most likely case is that mysql is not actually on the path in your cron job, because cron itself gives a rather minimal environment.
To fix that problem (assuming that's what it is), see this answer, which gives some guidelines on how best to expand the cron environment to give you what you need. That will probably just involve adding the MySQL executable directory to your PATH variable.
The other thing you may want to consider is closing the out_fh file before trying to pass it to mysql - if the buffers haven't been flushed, it may still be an empty file as far as other processes are concerned.
The expression glob(".* *") matches all files in the current working
directory.
- http://perldoc.perl.org/functions/glob.html
you should not rely on the wd in a cron job. If you want to use a glob (or any file operation) with a relative path, set the wd with chdir first.
source: http://www.perlmonks.org/bare/?node_id=395387
So if your working directory is, for example /home/user, you should insert
chdir('/home/user/');
before the WHILE, ie:
sub insert()
{
my $SQL_COMMAND = "mysql -u $USERNAME -p'$PASSWORD' ";
chdir('/home/user/');
while( my $sql_file = glob '*.sql' )
{
...
replace /home/user with wherever your sql files are being created.
It's better to do as much processing within Perl as possible. It avoids the overhead of generating a separate shell process and leaves everything under the control of the program so that you can handle any errors much more simply
Database access from Perl is done using the DBI module. This program demonstrates how to achieve what you have written using the mysql utility. As you can see it's also much more concise
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
my $host = "i";
my $username = "need";
my $password = "help";
my $dbh = DBI->connect("DBI:mysql:database=test;host=$host", $username, $password);
my $insert = $dbh->prepare('INSERT INTO BOL_LOCK.test(name) VALUES (?)');
my $rv = $insert->execute('wow');
print $rv ? "pass\n" : "fail\n";
I am trying to connect with database and perform some SQL queries by using this code, but every time it hangs.
my $connect_str = `/osp/local/etc/.oralgn $srv_name PSMF`;
my $sqlFile = "/osp/local/home/linus/amit/mytest.sql";
my ($abc, $cde)= split (/\#/ , $connect_str );
print "$abc";
$ORACLE_SID=SDDG00;
`export $ORACLE_SID`;
#chomp($abc);
#$abc=~ s/\s+$//;
`sqlplus $abc`;
open (SQL, "$sqlFile");
while (my $sqlStatement = <SQL>) {
$sth = dbi->prepare($sqlStatement)
or die (qq(Can't prepare $sqlStatement));
$sth->execute()
or die qq(Can't execute $sqlStatement);
}
How do I invoke a SQL command inside Perl?
Reading the documentation for the DBI module would be a good start.
Your problem seems to be this line.
$sth = dbi->prepare($sqlStatement)
You're trying to call the prepare method on the class "dbi". But you don't have a class called "dbi" in your program (or, at least, I can't see one in the code you've shown us).
To use a database from Perl you need to do these things:
1/ Load the DBI module (note, "DBI", not "dbi" - Perl is case sensitive).
use DBI;
2/ Connect to the database and get a database handle (Read the DBD::Oracle documentation for more details on the arguments to the connect() method).
my $dbh = DBI->connect('dbi:Oracle:dbname', $user, $password);
3/ You can then use this database handle to prepare SQL statements.
my $sth = $dbh->prepare($sqlStatement);
Getting something is wrong with the execute statement. It just seems to hang forever when I run in command prompt. It doesn't die either. does execute maybe need parameters?
#!/usr/bin/perl
use DBI;
use Data::Dumper;
$dbh = DBI->connect('DB', 'NAME', 'PASS',{ LongReadLen => 1000000} ) or die 'Error: cant connect to db';
$st= "update table set the_id = 7 where mid = 23 and tid = 22";
my $UpdateRecord = $dbh->prepare($st) or die "Couldn't prepare statement: $st".$dbh->errstr;
$UpdateRecord->execute() or die "can not execute $UpdateRecord".$dbh->errstr;
$UpdateRecord->finish;
$dbh->disconnect();
EDIT:
I tried binding in execute as well as using bind_param(), and it's still hanging up.
you need a do instead of prepare.
my $UpdatedRecord = $dbh->do($st) or die "Statement fails: $st".$dbh->errstr;
From DBI:
This method is typically most useful for non-SELECT statements that
either cannot be prepared in advance (due to a limitation of the
driver) or do not need to be executed repeatedly. It should not be
used for SELECT statements because it does not return a statement
handle (so you can't fetch any data).
Also it's always better to add/use db driver module,the one you are using, at top after use DBI; statement.
use DBD::Oracle;
Also add
use strict;
use warnings;
Problem was that I locked a bunch of objects from failing to put the disconnect in before I ran it once...yeah, don't do that.
We have a Perl script which runs a SQL and puts data in the table.
Now instead of supplying a single SQL statement, we want to pass bunch of them putting them together in a .sql file. We know that our program will fail because it expects a single SQL statement, not s bunch of them (that too from a .sql file). How do we make it work with a .sql file (having multiple INSERT statements?). We are using the DBI package.
A small snippet of code:
$sth = $dbh->prepare("/home/user1/tools/mytest.sql");
$sth->execute || warn "Couldn't execute statement";
$sth->finish();
There is a sort of workaround for DDL. You need to slurp SQL file first and then enclose it's contents into BEGIN ... END; keywords. Like:
sub exec_sql_file {
my ($dbh, $file) = #_;
my $sql = do {
open my $fh, '<', $file or die "Can't open $file: $!";
local $/;
<$fh>
};
$dbh->do("BEGIN $sql END;");
}
This subroutine allows to run DDL (SQL) scripts with multiple statements inside (e.g. database dumps).
Not exactly sure what you want...
Once you create a DBI object, you can use it over and over again. Here I'm reading SQL statement after SQL statement from a file and processing each and every one in order:
use DBI;
my $sqlFile = "/home/user1/tools/mytest.sql"
my $dbh = DBI::Connect->new($connect, $user, $password)
or die("Can't access db");
# Open the file that contains the various SQL statements
# Assuming one SQL statement per line
open (SQL, "$sqlFile")
or die("Can't open file $sqlFile for reading");
# Loop though the SQL file and execute each and every one.
while (my $sqlStatement = <SQL>) {
$sth = dbi->prepare($sqlStatement)
or die("Can't prepare $sqlStatement");
$sth->execute()
or die("Can't execute $sqlStatement");
}
Notice that I'm putting the SQL statement in the prepare and not the file name that contains the SQL statement. Could that be your problem?
You don't need perl for this at all. Just use the mysql command line client:
mysql -h [hostname] -u[username] -p[password] [database name] < /home/user1/tools/mytest.sql
replace the [variables] with your information.
Note no space after -u or -p. If your mysql server is running on the same machine you can omit -h[hostname] (it defaults to localhost)
Here is how I've done it. In my case I dont assume one SQL per line and I assume, my example is a bit better :)
sub get_sql_from_file {
open my $fh, '<', shift or die "Can't open SQL File for reading: $!";
local $/;
return <$fh>;
};
my $SQL = get_sql_from_file("SQL/file_which_holds_sql_statements.sql");
my $sth1 = $dbh1->prepare($SQL);
$sth1->execute();