Powershell SQL insert randomly fails - sql

Question

$timelimit = (get-date).AddMinutes(-65)
$logpath = "C:\_SCRIPT_\_SCHED_\_Eventlog_to_SQL.txt"
$now = get-date
$nowstring = "{0:yyyy-MM-dd-HH:mm:ss}" -f $now
$dbServer = "myserver"
$dbDatabase = "logdb"
$dbuid = "logdbuser"
$dbpwd = "logdbpass"
$dbTableFWevents = "dbo.fwevents"
$FWEvents = Get-WinEvent -logname ForwardedEvents | where-object {$_.timecreated -ge $timelimit} | Select * | Sort-Object TimeCreated
$FWEventsstat = $FWEvents | Measure-Object
$dbConnection = New-Object System.Data.SqlClient.SqlConnection
$dbConnectionString = "Server=$dbServer;Database=$dbDatabase;Integrated Security=True;User ID=$dbuid;Password=$dbpwd;Connect Timeout=0"
$dbconnection.ConnectionString = $dbConnectionString
$dbconnection.Open()
$transaction = $dbConnection.BeginTransaction("LogParserUpload")
$nowstring + " ---START---" | out-file $logpath -Append
$nowstring + " EVENT COUNT: " +$FWEventsstat.Count | out-file $logpath -Append
foreach ($evnt in $FWEvents)
{
$Command = $dbconnection.CreateCommand()
$Command.CommandText = "INSERT INTO "+$dbDatabase+"."+$dbTableFWevents+" (Message, Id, Level, ProviderName, LogName, ProcessId, ThreadId, MachineName, UserId, TimeCreated, LevelDisplayName) VALUES (#Message, #Id, #Level, #ProviderName, #LogName, #ProcessId, #ThreadId, #MachineName, #UserId, #TimeCreated, #LevelDisplayName)";
if ([string]$evnt.message){$Command.Parameters.Add("#Message", [string]$evnt.Message);}else{$Command.Parameters.Add("#Message", [DBNull]::Value);}
if ([string]$evnt.id){$Command.Parameters.Add("#Id", [string]$evnt.id);}else{$Command.Parameters.Add("#Id", [DBNull]::Value);}
if ([string]$evnt.level){$Command.Parameters.Add("#Level", [string]$evnt.level);}else{$Command.Parameters.Add("#Level", [DBNull]::Value);}
if ([string]$evnt.providername){$Command.Parameters.Add("#ProviderName", [string]$evnt.ProviderName);}else{$Command.Parameters.Add("#ProviderName", [DBNull]::Value);}
if ([string]$evnt.logname){$Command.Parameters.Add("#LogName", [string]$evnt.LogName);}else{$Command.Parameters.Add("#LogName", [DBNull]::Value);}
if ([string]$evnt.processid){$Command.Parameters.Add("#ProcessId", [string]$evnt.ProcessId);}else{$Command.Parameters.Add("#ProcessId", [DBNull]::Value);}
if ([string]$evnt.threadid){$Command.Parameters.Add("#ThreadId", [string]$evnt.threadId);}else{$Command.Parameters.Add("#ThreadId", [DBNull]::Value);}
if ([string]$evnt.machinename){$Command.Parameters.Add("#MachineName", [string]$evnt.MachineName);}else{$Command.Parameters.Add("#MachineName", [DBNull]::Value);}
if ([string]$evnt.userid){$Command.Parameters.Add("#UserId", [string]$evnt.UserId);}else{$Command.Parameters.Add("#UserId", [DBNull]::Value);}
if ([string]$evnt.timecreated){$Command.Parameters.Add("#TimeCreated", [string]$evnt.TimeCreated);}else{$Command.Parameters.Add("#TimeCreated", [DBNull]::Value);}
if ([string]$evnt.leveldisplayname){$Command.Parameters.Add("#LevelDisplayName", [string]$evnt.LevelDisplayName);}else{$Command.Parameters.Add("#LevelDisplayName", [DBNull]::Value);}
$Command.Transaction = $transaction
$eredmenyin = $Command.ExecuteNonQuery()
$nowstring + " INSERT RESULT: " +$eredmenyin | out-file $logpath -Append
}
$eredmenytr = $transaction.Commit()
$nowstring + " TRANSACTION RESULT: " +$eredmenyin | out-file $logpath -Append
$nowstring + " ---END---" | out-file $logpath -Append
#$transaction.Rollback()
$dbconnection.Close()
Hello guys.
For historical reasons we collect a bunch of server info through Windows server Forwarded Event solution.
Forwarded Events is not easy to collect from a collector computer as logparser and other solutions hardly access it.
So I decided to collect and uplaod it with Powershell.
The script runs every 60 minutes as scheduled job, collects events backward for 65 minutes.
The script runs fine. But from time to time it does skips 1-2 lines. I do not understand why. It does not skips a whole session - but for example if in a 65 minute interval there are 5 events, it uploads 4. Next time it runns fluently. Next time it again skips from 10 event 2 randomly.
I would like to know why. But I do not understand the reason AND I have no clue how could I log the actual insert command to fix it, or create other error handling.
(As you can see I implemented transaction as well - hopefully not wrongly, and transaction gives an ok as well).
Have no Idea how to find the reason and/or source of error.

Answer 1

The first Column is uniqid - Allow Nulls is "no" - because its the row identifier.
Other rows are - Allow Nulls "yes".
(btw as I told the skript works - as 70-80% of the evenlogs are uploaded, but only 20-30% is missing - absolutley randomly ...)
So i doubt this is the source of the issue.
I suspect there is two way of finding the problem.
1) log the actual SQL command from $command variable - but I did not found out how can be that done.
write-host $command.parameters
only writes the content of $Command.CommandText - means no info only the #### variables in it. So I do not know, how to doublecheck if the actual INSERT is not "wrong"
2) use somehow the transaction log (or other resource) to find out if the actual INSERT command "arrives" and if did "arrived" if it was denied and for what reason. As I am not really an SQL expert this kind of magic is way beyond my knowledge (and google did not helped me out neither :( )