I know I could use Contains to find it but it doesn't work.
Full Story:
I have to get the PartNo, Ver, Rev from SQl db and check if they occur in the first line of the text file. I get the first line of the file and store it in $EiaContent.
The PartNo is associated with MAFN as in $partNo=Select PartNo Where MAFN=xxx. Most of the time MAFN returns one PartNo. But in some cases for one MAFN there could be multiple PartNo. So the query returns multiple PartNo(PartNo_1,PartNo_2,PartNo_3,and PartNo_4) but only one of these will be in the text file.
The issue is that each of these PartNo. is treated as a single character in PowerShell. $partNo.Length is 4. Therefore, my check If ($EiaContent.Contains("*$partNo*")) fails and it shouldn't in this case because I can see that one of the PartNo is mentioned in the file. Also, Contains wouldn't work if there was one PartNo. I use like as in If ($EiaContent -like "*$partNo*") to match the PartNo and it worked but it doesn't work when there are multiple PartNo.
Data type of $partNo is string and so is $EiaContent. The data type of PartNo. in SQL is varchar(50) collation is COLLATE SQL_Latin1_General_CP1_CI_AS
I am using PowerShell Core 7.2 and SQL 2005
Code:
$EiaContent = (Get-Content $aidLibPathFolder\$folderName\$fileName -TotalCount 1)
Write-host $EiaContent
#Sql query to get the Part Number
$partNoQuery = "SELECT PartNo FROM [NML_Sidney].[dbo].[vMADL_EngParts] Where MAFN = $firstPartTrimmed"
$partNoSql = Invoke-Sqlcmd -ServerInstance $server -Database $database -Query $partNoQuery
#Eliminate trailing spaces
$partNo = $partNoSql.PartNo.Trim()
If ($EiaContent.Contains("*$partNo*")) {
Write-Host "Part Matches"
}
Else {
#Send an email stating the PartNo discrepancy
}
Thank you in advance to those who try to help.
EDIT
Screenshot
[1]: https://i.stack.imgur.com/hIqJB.png
A1023 A1023MD C0400 C0400MD is the output of the variable $partNo and O40033( C0400 REV N VER 004, 37 DIA 4.5 BRAKE DRUM OP3 ) is the output of the variable $EiaContent
So the query returns multiple PartNo(PartNo_1,PartNo_2,PartNo_3,and PartNo_4) but only one of these will be in the text file.
A1023 A1023MD C0400 C0400MD is the output of the variable $partNo and O40033( C0400 REV N VER 004, 37 DIA 4.5 BRAKE DRUM OP3 ) is the output of the variable $EiaContent
So you first have to split $partNo and then for each sub string of $partNo, search for it in $EiaContent:
If ($partNo -split ' ' | Where-Object { $EiaContent.Contains( $_ ) }) {
Write-Host "Part Matches"
}
This is the generic form that most people are used to. We can simplify the query using the unary form of -split (as we split on the default separator) and use the intrinsic array method .Where() which is faster as it does not involve pipeline overhead.
If ((-split $partNo).Where{ $EiaContent.Contains( $_ ) }) {
Write-Host "Part Matches"
}
As correctly noted in comments, wildcards are not supported by the .Contains() string method.
Wildcards are supported only by the PowerShell -like operator. The following example is just for educational purposes, I wouldn't use it in your case as .Contains() string method is simpler and faster.
If ((-split $partNo).Where{ $EiaContent -like "*$EiaContent*" }) {
Write-Host "Part Matches"
}
Note that -contains would not be suitable here. A common misconception is that -contains does a substring search, when the LHS operand is a string. It doesn't! The operator tests whether a collection (such as an array) on the LHS contains the value given on the RHS.
Related
I have a database table with following columns (among others):
emailAttachment
emailAttachmentFileName
I am using the following Powershell to convert a .csv file to binary, to store in emailAttachment (defined as varbinary(max)):
$("0x$(([Byte[]](Get-Content -Encoding Byte -Path c:\temp\test.csv) | ForEach-Object ToString X2) -join '')")
As far as I can tell, that works fine.
To retrieve the data, I run this SQL query:
$query = #'
SELECT
*
FROM
[dbo].[mydatabase].[mytable]
WHERE
id = 1
'#
$file = Invoke-Sqlcmd -Query $query -ServerInstance server.domain.local -Credential (Get-Credential)
Now that I have the bytes from SQL, I want to write it to a file:
[IO.File]::WriteAllBytes($file.emailAttachmentFileName, $file.emailAttachment)
That creates the file with the correct name and some of the rows of the .csv, but not all of them (63 instead of 464).
I also tried both, but they have the same result:
[IO.File]::WriteAllLines($file.emailAttachmentFileName, [System.Text.Encoding]::ASCII.GetString($file.emailAttachment))
[IO.File]::WriteAllText($file.emailAttachmentFileName, [System.Text.Encoding]::ASCII.GetString($file.emailAttachment))
What am I doing wrong here?
please assist with the following. i m trying to run a script that accepts one argument $1. The argument can either be a string or character or an integer. I want to use the argument in there where clause to search for the element in the database.
This is the table i want to search from:enter image description here
When i use the multiple conditions with OR , it works only when either the argument is a number or text.
This what my code looks like enter image description here
`
ELEMENT=$($PSQL "SELECT * FROM elements e FULL JOIN properties p USING(atomic_number) WHERE symbol = '$1' OR name = '$1' OR atomic_number = $1;")
`
and this is the results i get when i run with different aurgumentsenter image description here
Please help.
Thank you in advance
This will always fail on any non-numeric argument.
You are passing in H for hydrogen, but taking whatever was passed in and using it in the atomic_number comparison as an unquoted number, which the DB engine is trying to figure out what to do with. H isn't a number, and isn't a quoted string, so it must be the name of a column...but it isn't, so you are using invalid syntax.
I don't have a postgres available right now, but try something like this -
ELEMENT=$( $PSQL "
SELECT *
FROM elements e
FULL JOIN properties p USING(atomic_number)
WHERE symbol = '$1'
OR name = '$1'
OR atomic_number = CAST(`$1` as INTEGER); " )
Also, as an aside... avoid all-capital variable names.
As a convention, those are supposed to be system vars.
And please - please don't embed images except as helpful clarification.
Never rely on them to provide info if it can be avoided. Copy/paste actual formatted text people can copy/paste in their own testing.
An alternate way to construct the query: requires bash
looks_like_a_number() {
# only contains digits
[[ "$1" == +([[:digit:]]) ]]
}
sanitize() {
# at a minimum, handle embedded single quotes
printf '%s' "${1//\'/\'\'}"
}
if looks_like_a_number "$1"; then
field="atomic_number"
value=$1
elif [[ ${#1} -eq 1 ]]; then
field="symbol"
printf -v value "'%s'" "$(sanitize "$1")"
else
field="name"
printf -v value "'%s'" "$(sanitize "$1")"
fi
q="SELECT *
FROM elements e
FULL JOIN properties p USING(atomic_number)
WHERE $field = $value;"
printf '%s\n' "$q"
result=$("$PSQL" "$q")
I am trying to scan SSIS .dtsx packages for table names. Yes, I know that I should use [xml] and a tool that parses SQL language. That does not seem to be possibe at this time. PowerShell can understand [xml], but SQL parsers generally cost++ and using ANTLR is more of an investment than is acceptable at this time. I am open to suggestions, but I am not asking for a tool recommendation.
There are two (2) problems.
1) `&.;` does not appear to be recognized as separate from the table name capture item
2) TABLE5 does not appear to be found
Yes, I also know that schema names should not be hardcoded into source. It makes it difficult/impossible for DBAs to manage the database. That is the way it is done here.
How can I make the regex omit the &.*; from the capture and recognize dbo.TABLE5
Here is the code I am using to scan the .dtsx files.
PS C:\src\sql> Get-Content .\Find-FromJoinSql.ps1
Get-ChildItem -File -Filter '*.dtsx' |
ForEach-Object {
$Filename = $_.Name
Select-String -Pattern '(FROM|JOIN)(\s|&.*;)+(\S+)(\s|&.*;)+' -Path $_ -AllMatches |
ForEach-Object {
if ($_.Matches.Groups.captures[3].value -match 'dbo') {
"$Filename === $($_.Matches.Groups.captures[3].value)"
}
}
}
Here is a tiny sample of the type of text from the .dtsx file.
PS C:\src\sql> Get-Content .\sls_test.dtsx
USE ADATABASE;
SELECT * FROM dbo.TABLE1 WHERE F1 = 3;
SELECT * FROM dbo.TABLE2 T2
FULL OUTER JOIN dbo.TABLEJ TJ
ON T2.KEY = TJ.KEY;
SELECT * FROM dbo.TABLE3 T3
INNER JOIN ADATABASE2.dbo.TABLEK
TK ON
T3.user_id = TK.user_id
SELECT * FROM dbo.TABLE4 T4 FULL OUTER JOIN dbo.TABLE5 T5
ON T4.F1 = T5.F1;
EXIT
Running the script on this data produces:
PS C:\src\sql> .\Find-FromJoinSql.ps1
sls_test.dtsx === dbo.TABLE1
sls_test.dtsx === dbo.TABLE2
sls_test.dtsx === dbo.TABLEJ
sls_test.dtsx === dbo.TABLE3
sls_test.dtsx === ADATABASE2.dbo.TABLEK
TK
sls_test.dtsx === dbo.TABLE4
PS C:\src\sql> $PSVersionTable.PSVersion.ToString()
7.1.5
Indeed strange that some entities (
) are not replaced in those files.
Change the regex pattern a bit to capture the dbo.table names like below.
Using Get-Content
$regex = [regex] '(?im)(?:FROM|JOIN)(?:\s|&[^;]+;)+([^\s&]+)(?:\s|&[^;]+;)*'
Get-ChildItem -Path D:\Test -File -Filter '*.dtsx' |
ForEach-Object {
$match = $regex.Match((Get-Content -Path $_.FullName -Raw))
while ($match.Success) {
"$($_.Name) === $($match.Groups[1].Value)"
$match = $match.NextMatch()
}
}
Using Select-String
As to why Select-String -AllMatches skipped your Table5.
From the docs: "When Select-String finds more than one match in a line of text, it still emits only one MatchInfo object for the line, but the Matches property of the object contains all the matches."
That means you need another loop to get all the $Matches from each $MatchInfo objects to get them in your output:
$pattern = '(?:FROM|JOIN)(?:\s|&[^;]+;)+([^\s&]+)(?:\s|&[^;]+;)*'
Get-ChildItem -Path 'D:\Test' -File -Filter '*.dtsx' |
ForEach-Object {
$Filename = $_.Name
Select-String -Pattern $pattern -Path $_.FullName -AllMatches |
ForEach-Object {
# loop again, because each $MatchInfo object may contain multiple
# $Matches objects if more matches were found in the same line
foreach ($match in $_.Matches) {
if ($match.Groups[1].value -match 'dbo') {
"$Filename === $($match.Groups[1].value)"
}
}
}
}
Output:
sls_test.dtsx === dbo.TABLE1
sls_test.dtsx === dbo.TABLE2
sls_test.dtsx === dbo.TABLEJ
sls_test.dtsx === dbo.TABLE3
sls_test.dtsx === ADATABASE2.dbo.TABLEK
sls_test.dtsx === dbo.TABLE4
sls_test.dtsx === dbo.TABLE5
Regex details:
(?im) Use case-insensitive matching and have '^' and '$' match at linebreaks
(?: Match the regular expression below
Match either the regular expression below (attempting the next alternative only if this one fails)
FROM Match the characters “FROM” literally
| Or match regular expression number 2 below (the entire group fails if this one fails to match)
JOIN Match the characters “JOIN” literally
)
(?: Match the regular expression below
| Match either the regular expression below (attempting the next alternative only if this one fails)
\s Match a single character that is a “whitespace character” (spaces, tabs, line breaks, etc.)
| Or match regular expression number 2 below (the entire group fails if this one fails to match)
& Match the character “&” literally
[^;] Match any character that is NOT a “;”
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
; Match the character “;” literally
)+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
( Match the regular expression below and capture its match into backreference number 1
[^\s&] Match a single character NOT present in the list below
A whitespace character (spaces, tabs, line breaks, etc.)
The character “&”
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
)
(?: Match the regular expression below
| Match either the regular expression below (attempting the next alternative only if this one fails)
\s Match a single character that is a “whitespace character” (spaces, tabs, line breaks, etc.)
| Or match regular expression number 2 below (the entire group fails if this one fails to match)
& Match the character “&” literally
[^;] Match any character that is NOT a “;”
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
; Match the character “;” literally
)* Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
Currently I have a perl script that accesses our database, performs certain queries and prints output to the terminal. Instead, I would like to output the results into a template latex file before generating a pdf. For most of my queries I pull out numbers and store these as scalar variables (eg how often a particular operator carries out a given task). eg.
foreach $op (#operator) {
$query = "SELECT count(task_name) FROM table WHERE date <= '$date_stop' and
date >= '$date_start' and task=\'$operator[$index]\';";
#execute query
$result=$conn->exec($query);
$conres = $conn->errorMessage;
if ($result->resultStatus eq PGRES_TUPLES_OK) {
if($result->ntuples > 0) {
($task[$index]) = $result->fetchrow;
}
printf("$operator[$index] carried out task: %d\n", $task[$index]);
} else {
die "Failed.\n$conres\n\n";
exit -1;
}
$index++;
}
printf("**********************************\n\n");
In the final report I will summarise how many times each operator completed each task in a table. In addition to this there will also be some incidents which must be reported. I can print these easily to the terminal using a command such as
$query = "SELECT operator, incident_type from table_name WHERE incident_type = 'Y'
and date <= '$date_stop' and date >= '$date_start';";
$result=$conn->exec($query);
$conres = $conn->errorMessage;
if ($result->resultStatus eq PGRES_TUPLES_OK) {
if($result->ntuples > 0) {
$result->print(STDOUT, 1, 1, 0, 0, 0, 1, "\t", "", "");
}
} else {
die "Failed.\n$conres\n\n";
exit -1;
}
An example of the output of this command is
operator | incident_type
-----------------------------
AB | Incomplete due to staff shortages
-------------------------------
CD | Closed due to weather
-----------------------------
How can I make my perl script pass the operator names and incidents into a string array rather than just sending the results to the terminal?
You should consider updating your script to use DBI. This is the standard for database connectivity in Perl.
DBI has a built in facility for inserting parameters into a query string. It is safer and faster than manually creating the string yourself. Before the loop, do this once:
#dbh is a database handle that you have already opened.
my $query = $dbh->prepare(
"SELECT count(task_name) FROM table WHERE date<=? and date>=? and task=?"
);
Then within the loop, you only have to do this each time:
$query->execute($date_stop,$date_start,$op);
Note that the parameters you pass to execute automatically get inserted in place of the ?'s in your statement. It handles the quoting for you.
Also in the loop, after you execute the statement, you can get the results like this:
my $array_ref = $query->fetchall_array_ref;
Now all of the rows are stored in a two-dimensional array structure. $array_ref->[0][0] would get the first column of the first row returned.
See the DBI documentation for more information.
As others have mentioned, there are quite a few other mistakes in your code. Make sure you start with use strict; use warnings;, and ask more questions if you need further help!
Lots of good feedback to your script, but nothing about your actual question.
How can I make my perl script pass the operator names and incidents into a string array rather than just sending the results to the terminal?
Have your tried creating an array and pushing items to it?
my #array;
push (#array, "foo");
Or using nested arrays:
push (#array, ["operator", "incident"]);
if ($node->taxonomy) {
$query = 'SELECT DISTINCT(t.nid), n.nid, n.title FROM {node} n INNER JOIN {term_node} t ON n.nid = t.nid WHERE n.nid != %d AND (';
$args = array($node->nid);
$tids = array();
foreach ($node->taxonomy as $term) {
$tids[] = 't.tid = %d';
$args[] = $term->tid;
}
$query .= implode(' OR ', $tids) . ')';
$result = db_query_range($query, $args, 0, 10);
while ($o = db_fetch_object($result)) {
echo l($o->title, 'node/' . $o->nid);
}
}
the code is from a drupal guru. . used to get the article's title under the same term in node.tpl.php, i have researched it two days, although know some part of it. the principle of the code i still don't know. expect someone can explain more details about it for me .many thanks.
Short version:
It gets the array of tags of the node, retrieves the first 10 nodes that use at least one of these tags and outputs a link for each of these 10 results.
Detailed version:
First of all, the variable "$node" is an object that contains the data about a specific node (e.g. a Page or Story node).
For example, "$node->title" would be the title of that node.
"$node->taxonomy" tests is that node is tagged (because if it has no tags, it cannot retrieve the other nodes using the same tag(s).
When there is one or several tags associated with that node/page/story, $node->taxonomy is an array .
Now about the SQL query:
"node" is the database table that stores the base fields (non-CCK) of every node.
"term_node" is the database table that contains the combination of tag (which is called a "taxonomy term") and node.
In both tables, "nid" is the "unique Node ID" (which is an internal autoincremented number). Because this column is in both tables, this is how the tables are joined together.
In "term_node", "tid" is the "unique Term ID" (which is also an internal autoincremented number).
The "node" table is aliased "n", therefore "n.nid" means "the Node ID stored in table node".
The "term_node" table is aliased "t", therefore "t.tid" means "the Term ID stored in table term_node".
The "foreach" loop goes thru the array of tags to extract the TermID of each tag used by the node in order to add it in the SQL query, and implode converts to a string.
The loop stores a piece of SQL query for each tag in variable $tids and stores the actual value in variable $args because Drupal database calls are safer when the arguments are passed separately from the SQL query: "%d" means "integer number".
"db_query_range" is a function that selects multiple rows in the database: here, "0 10" means "retrieve the first 10 results".
"db_fetch_object" in the "while" loop retrieves each result and stores it in the variable "$o", which is an object.
Therefore "$o->title" contains the value of the column "title" retrieved by the SQL query.
The function "l" is the drupal functin that creates the code for an HTML link: the first argument is the name of the link, the second argument is the drupal path: in Drupal, any node can be accessed by default using "www.yoursite.com/node/NodeID",
which is why it gives the path "node/123" (where 123 is the "Node ID").
This function is useful because it transparently handles custom paths, so if your node has a custom path to access it using "www.yoursite.com/my-great-page" instead, it will create a link to that page instead of "www.yoursite.com/node/123" automatically.
I wouldn't exactly call the guy who wrote this a guru, you could do this a lot prettier. Anyways what he does it create a query that looks like this:
SELECT DISTINCT(t.nid), n.nid, n.title FROM {node} n
INNER JOIN {term_node} t ON n.nid = t.nid
WHERE n.nid != %d
AND (t.tid = %d OR t.tid = %d OR ... t.tid = %d);
The end result is that he selects all the node ids and titles (only once) that share at least one term with the selected node, but isn't the node itself.