Caveat: I am new to PowerShell...
I am trying to automate a load process for 48 tables. The table names are all of the format "Vocabulary_MMddyyyy", as in "Vocabulary_12052016". I have no trouble iterating through the table names and parsing out the MMddyyyy part (the release version for the vocabulary). Powershell does not recognize the '12052016' as a proper datetime format. So, I do some parsing to build a proper date string that I convert to DateTime with the Get-Date cmdlet:
$version = ($table.Table_name -replace 'Vocabulary_', '').Trim();
$mm = $version.Substring(0,2);
$dd = $version.Substring(2,2);
$yyyy = $version.Substring(4,4);
$date = $mm + "/" + $dd +"/" + $yyyy
$startDate = Get-Date -Date $date -Format "d"
$endDate = ($startDate).AddDays(-1)
write-host 'Version: ' $version
write-host 'StartDate: ' $startDate
write-host 'EndDate: ' $endDate
This properly sets $startDate and $endDates (though despite the "d" format, I still get HH:MM:SS AM part):
Version: 12052016
StartDate: 12/5/2016 12:00:00 AM
EndDate: 12/4/2016 12:00:00 AM
This is the SQL signature of the stored procedure I want to call:
CREATE PROCEDURE [vocab].[prInsertLoadProcessID] (
#CodeQualifierID INT,
#SourceVersion VARCHAR(100),
#SourceDate DATE,
#LoadTypeID INT,
#isTestLoad BIT,
#ValidationDate DATE,
#LoadDescription VARCHAR(500),
#GlobalEffectiveDateStart DATE,
#GlobalEffectiveDateEnd DATE
)
Now, I go to build up a SQL command to execute the stored procedure ($conn2 is properly defined and open; it executes simple select statements yielding correct results):
$cmd.Connection = $conn2;
$cmd.CommandText = "vocab.prInsertLoadProcessID ";
$cmd.CommandType = [System.Data.CommandType]::StoredProcedure;
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.Connection = $conn2;
$cmd.CommandText = "vocab.prInsertLoadProcessID ";
$cmd.CommandType = [System.Data.CommandType]::StoredProcedure;
$cmd.Parameters.AddWithValue("#CodeQualifierID", 6);
$cmd.Parameters.AddWithValue("#SourceVersion", $version);
$cmd.Parameters.AddWithValue("#SourceDate", $startDate);
$cmd.Parameters.AddWithValue("#LoadTypeID", 1);
$cmd.Parameters.AddWithValue("#isTestLoad", 0);
$cmd.Parameters.AddWithValue("#ValidationDate", 'NULL');
$cmd.Parameters.AddWithValue("#LoadDescription", $LoadDescription);
$cmd.Parameters.AddWithValue("#GlobalEffectiveDateStart", $startDate);
$cmd.Parameters.AddWithValue("#GlobalEffectiveDateEnd", $endDate);
Each of the dates (#SourceDate, #GlobalEffectiveDateStart, #GlobalEffectiveDateEnd) show as DateTime when I look at the output from adding the parameters (here is one example):
CompareInfo : None
XmlSchemaCollectionDatabase :
XmlSchemaCollectionOwningSchema :
XmlSchemaCollectionName :
ForceColumnEncryption : False
**DbType : DateTime**
LocaleId : 0
**ParameterName : #GlobalEffectiveDateEnd**
Precision : 0
Scale : 0
**SqlDbType : DateTime**
SqlValue : 12/6/2015 12:00:00 AM
UdtTypeName :
TypeName :
Value : 12/6/2015 12:00:00 AM
Direction : Input
IsNullable : False
Offset : 0
Size : 0
SourceColumn :
SourceColumnNullMapping : False
SourceVersion : Current
When I run $cmd.ExecuteNonQuery() I get:
Exception calling "ExecuteNonQuery" with "0" argument(s): "Error converting data
type nvarchar to date." this is a Category: MethodInvocationException and a FullyQualifiedErroID: SQLException.
So, after reading for hours and trying everything I can think of I am stuck. My variables appear to be typed as DATETIME, but ExecuteNonQuery() seems to read treat them as NVARCHAR.
What am I missing? I can certainly rewrite the stored procedure to take strings instead of dates and let SQL do the conversion, but it really seems I should be able to pass dates from Powershell to SQL.
There must be a solution to this. What is causing the $cmd.ExecuteNonQuery() to change the datatype of my inputs? Or, am I setting up the $cmd.Parameters incorrectly?
Kind thanks for any help.
Related
I wrote a script to execute SQL query and send email to user with SQL results.
Database column has Date field in format YYYY-MM-DD hh:mm:ss:mi but the SQL result in power shell giving me the format of YYYY/MM/DD hh:mm:ss. The result is not consisting of Milliseconds and date is in different format. If I have to get milliseconds also in the result date, how should I format??
param (
[Parameter(mandatory = "true")]
[string]$sqlserver,
[Parameter(mandatory = "true")]
[string]$sqlusername,
[Parameter(mandatory = "true")]
[string]$sqlpassword,
[Parameter(mandatory = "true")]
[string]$databasename
)
$query = "select * from dbo.UserDetails where createdate > '2020-04-13' order by createdate"
$Connection = New-Object System.Data.SQLClient.SQLConnection
$Connection.ConnectionString = "server='$Server';database='$databasename'; User ID = 'sqlusername' password= '$sqlpassword'"
$Connection.Open()
$Command = New-Object System.Data.SQLClient.SQLCommand
$Command.Connection = $Connection
$Command.CommandText = $query
$DataAdapter = new-object System.Data.SqlClient.SqlDataAdapter $Command
$Dataset = new-object System.Data.Dataset
$DataAdapter.Fill($Dataset)
$Connection.Close()
$noOfRows = $Dataset.Tables[0].Rows.Count
$UserCreateDate = $Dataset.Tables[0].Rows[$noOfRows - 1].ItemArray[6]
Write-Host ("The Latest User created " + $UserCreateDate)
Output:
4/14/2020 12:25:03
Value in DB: 2020-04-14 12:25:03.6028312
Powershell is converting your SQL output into a DateTime-Object. The default output format is determined by your set culture (try (Get-Culture).datetimeformat for more informations about your settings).
There are multiple ways to get the desired output:
Format the Output to a string using the .ToString-Method:
$UserCreateDate = ($Dataset.Tables[0].Rows[$noOfRows - 1].ItemArray[6]).ToString("yyyy-MM-dd HH:mm:ss.fff")
Using Get-Date and the -Format parameter:
$UserCreateDate = Get-Date ($Dataset.Tables[0].Rows[$noOfRows - 1].ItemArray[6]) -Format "yyyy-MM-dd HH:mm:ss.fff"
Please keep in mind that both options will convert the output into a string and won't keep the DateTime-Object.
Both solutions format the datetime with a given format-string. #Lee_Dailey was so nice to point out the possible formats from the documentation here: Custom date and time format strings
i have a perl script and i don't want to send a double request :
the request is '2018-03-15 12:30:00', 'Metric A', 62 and i want send only one time and not more :
in my mariadb bdd i have double line :
SELECT time, measurement, valueOne FROM `metric_values`;
results :
+---------------------+-----------------+----------+
| time | measurement | valueOne |
+---------------------+-----------------+----------+
| 2018-03-15 12:30:00 | Metric A | 62 |
| 2018-03-15 12:30:00 | Metric A | 62 |
my perl scipt :
use DBI;
open (FILE, 'logfile');
while (<FILE>) {
($word1, $word2, $word3, $word4, $word5, $word6, $word7, $word8, $word9, $word10, $word11, $word12, $word13, $word14) = split(" ");
$word13 =~ s/[^\d.]//g;
if ($word13 > 5) {
if ($word2 eq "Jan") {
$word2 = "01"
}
if ($word2 eq "Feb") {
$word2 = "02"
}
if ($word2 eq "Mar") {
$word2 = "03"
}
if ($word2 eq "Apr") {
$word2 = "04"
}
if ($word2 eq "May") {
$word2 = "05"
}
if ($word2 eq "Jun") {
$word2 = "06"
}
if ($word2 eq "Jul") {
$word2 = "07"
}
if ($word2 eq "Aug") {
$word2 = "08"
}
if ($word2 eq "Sep") {
$word2 = "09"
}
if ($word2 eq "Oct") {
$word2 = "10"
}
if ($word2 eq "Nov") {
$word2 = "11"
}
if ($word2 eq "Dec") {
$word2 = "12"
}
print "'$word5-$word2-$word3 $word4', $word11, $word13 \n";
}
# Connect to the database.
my $dbh = DBI->connect("DBI:mysql:database=db;host=ip",
"titi", 'mp!',
{'RaiseError' => 1}) ;
my $sth = $dbh->prepare(
"INSERT `metric_values` (time, measurement, valueOne) VALUES('$word5-$word2-$word3 $word4', $word11, $word13);")#result is ('2018-03-15 12:30:00', 'Metric A', 62)
or die "prepare statement failed: $dbh->errstr()";
$sth->execute() or die "execution failed: $dbh->errstr()";
print $sth->rows . " rows found.\n";
$sth->finish;
my log file:
Wed Oct 17 04:57:08 2018 : Resource = 'toto' cstep= 'titi' time =23.634s
Wed Oct 17 04:57:50 2018 : Resource = 'toto' cstep= 'titi' time =22.355s
thanks for your response
In a comment, you say this:
i execute this script every 5 minute and that create many same line in the table, i don't want same line in my table
I think this is what is happening.
Every five minutes you run your program. Each time you run the program you use exactly the same log file as input. So the same records get processed every time and new copies of the data are inserted on each run.
There's nothing wrong with your existing code. It's doing exactly what you've asked it to do. It's just not clever enough. You need to make it cleverer. You have a few options.
Remove from the log file the records that have been processed. That way you only insert each record once.
Add a flag to each record in your log file which indicates that it has been added to the database. You can then check that flag when processing the file and only insert records that don't have the flag.
Add an index to your table to ensure that it can only contain one copy of each record. You'll then need to change your code so it ignores any duplicate data errors that you get back from the database.
Use REPLACE instead of INSERT and ensure you have the correct primary key on your table to ensure that duplicate records aren't inserted.
Without knowing a log more about your particular application, it's hard to know which of these options is the best approach for you. I suspect you'll find the REPLACE option the easiest to implement.
Update: I hope you'll find some general comments on your code to be useful.
Your code to open the file works, of course, but it is some distance from current best practice. I recommend a) using a lexical filehandle, b) using the three-arg version of open() and c) checking the return value from the call.
open my $fh, '<', 'logfile'
or die "Could not open 'logfile': $!\n";
Using variables called $word1, $word2, etc is a terrible idea. A better idea would be to use an array:
my #words = split ' ',
If you really want individual variables, then please give them better names:
my ($day, $mon, $date, $time, $year, ... ) = split(' ');
Personally, I'd turn each record into a hash.
my #cols = qw[day mon date time year ... ];
# and then, in your loop
my %record;
#record{#cols} = split ' ';
Converting the month to a number the way you do it is clunky. Consider setting up a conversion hash.
my %months = (
Jan => 1,
Feb => 2,
...
);
Then your code becomes (assuming $mon instead of $word2):
$mon = sprintf '%02d', $months{$mon}
or die "$mon is not a valid month\n";
But, actually, you should use something like Time::Piece to deal with dates and times.
my $timestamp = "$day $mon $date $time $year";
my $tp = Time::Piece->strptime($timestamp, '%a %b %d %H:%M:%S $Y');
say $tp->ymd, ' ', $tp->hms;
I am trying to construct named parameters but receive an error.
Couldn't get data from Database
Oracle.DataAccess.Client.OracleException ORA-01858: a non-numeric
character was found where a numeric was expected at
Oracle.DataAccess.Client.OracleException.HandleErrorHelper...
private static void AddCriteria(IDbCommand command, string column, object value, string sqlOperator = "=")
{
var parameter = command.CreateParameter();
if (value is DateTime)
{
value = FormatSqlDate((DateTime)value);
}
parameter.ParameterName = DbHelper.GetParameterSql(parameter, "P" + (command.Parameters.Count + 1));
parameter.Value = value;
command.Parameters.Add(parameter);
command.CommandText += string.Format(" {0} {1} {2} {3}", (command.Parameters.Count > 1 ? "AND" : "WHERE"), column, sqlOperator, parameter.ParameterName);
}
Following query is constructed:
SELECT *
FROM trade LEFT JOIN
findetail
ON trade.trade = findetail.trade LEFT JOIN
fintransact
ON findetail.fintransact = fintransact.fintransact
WHERE trade.trade = :P1 AND acctdate = :P2
While parameters are
:P1 - 2298056
:P2 - TO_DATE('2014-12-31T00:00:00', 'YYYY-MM-DD"T"HH24:MI:SS')
Variables can only be values. You cannot have a function call like TO_DATE as text in a variable.
You need to put all SQL in the query text and only extract the real variables (like the actual time) into bound variables.
In this case, why don't you pass a properly parsed .NET DateTime as your value?
Using the following code I am getting the error:
Exception calling "Fill" with "1" argument(s): "The conversion of a
varchar data type to a datetime data type resulted in an out-of-range
value." At C:\Users\username\Desktop\TEST.ps1:47 char:1
+ $CommandCompl.fill($dt7) | out-null
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : SqlException
$Host.UI.RawUI.BufferSize = New-Object Management.Automation.Host.Size (200, 25)
$Date = (get-date).ToString("yyyyMMdd")
Clear
$SQLTableCOMPLIA = 'abc'
$SQLServerCOMPLIA = "123.123.123.123"
$SQLDBNameCOMPLIA = "test"
$UsernameCOMPLIA = "abc"
$PasswordCOMPLIA = "pasword"
$SQLServerLANDESK = "cba\test1"
$SqlConnectionLANDESK = New-Object System.Data.SqlClient.SqlConnection
$global:dt = new-object System.Data.DataTable
$dr = ""
$LONA = ""
$o = 0
$j = 0
$globalvuln , $globalNotvuln = 0
$global:dt7 = new-object System.Data.DataTable
$SqlConnectionCOMPLIA = New-Object System.Data.SqlClient.SqlConnection
$SqlConnectionCOMPLIA.ConnectionString = "Server=$SQLServerCOMPLIA; Database=$SQLDBNameCOMPLIA;uid=$UsernameCOMPLIA; pwd=$PasswordCOMPLIA"
$SqlConnectionCOMPLIA.Open() | out-null
$QueryCompl = "SELECT TOP 1 CONVERT(datetime,left(LD_publishdate,10),103) as R FROM vulns order by R DESC"
$CommandCompl = new-object System.Data.SqlClient.SqlDataAdapter ($QueryCompl, $SqlConnectionCOMPLIA)
$CommandCompl.fill($dt7) | out-null
What I am doing wrong? How can I solve this?
So problem here is simple and problem is with following line
CONVERT(datetime,left(LD_publishdate,10),103)
and with style parameter where you mentioned '103' which means you are trying to convert it to British/French standard. For British or french standard your varchar value should be in the format of 'dd/mm/yy' any other format will fail.
For example
following script will fail with the same error you are getting
select convert(datetime,left('2015/09/21',10),103)
While this will pass swiftly
select convert(datetime,left('21/09/2015',10),103)
Either solve this or try an appropriate value for your style. you can find appropriate values here
http://www.techonthenet.com/sql_server/functions/convert.php
Hope this helps.
Based on the error, it sounds like this part of your SQL query is invalid for the data in the table: CONVERT(datetime,left(LD_publishdate,10),103). Try running this query directly without the script and checking the output.
I ran into this, and thought my server was crazy, but after testing it in
Codepad I run into the same results. After using Datetime to try and
process my date stamp I end up one day and one month OFF my original date
after trying to format back to a string?? What madness is going on here?
Here's the code
echo $obj->attributes->timestamp; // output: Jun 25, 2013 11:43:52:875 AM
$date = New \DateTime();
$date->createFromFormat(
'M j, Y h:i:s:B A',
$obj->attributes->timestamp
);
echo $date->format('M j, Y'); // output: Jul 24, 2013
public static DateTime DateTime::createFromFormat ( string $format , string $time [, DateTimeZone $timezone ] )
It is static and returns new DateTime, but you are not using returned value but current date created by calling DateTime empty constructor.
Example from PHP docs, how to use it properly:
<?php
$date = DateTime::createFromFormat('j-M-Y', '15-Feb-2009');
echo $date->format('Y-m-d');