I have converted an HTML table from this site to an XML file.
I am trying to run a SQL query in PowerShell to copy the data from from the XML file to a database table. If I run the query within SSMS, it runs fine. However when I try to run the following code in Powershell, I get:
Error: input query is too long
[string] $dbCommand =
#"
Truncate table DB_NAME.dbo.SQL_LIFE_CYCLE_UPLOAD_IC
DECLARE #Data XML
SELECT #Data = BulkColumn
FROM OPENROWSET(BULK 'D:\Powershell\Temp\SQL_Life_Cycle.XML', SINGLE_BLOB) AS x
INSERT INTO DB_NAME.dbo.SQL_LIFE_CYCLE_UPLOAD_IC
(PRODUCT_RELEASED,LIFECYCLE_START_DATE,MAINSTREAM_SUPPORT_END_DATE,EXTENDED_SUPPORT_END_DATE,SERVICE_PACK__SUPPORT_END_DATE,NOTES)
Select max(case when col=1 then value else '' end) as PRODUCT_RELEASED,
max(case when col=2 then value else '' end) as LIFECYCLE_START_DATE,
max(case when col=3 then value else '' end) as MAINSTREAM_SUPPORT_END_DATE,
max(case when col=4 then value else '' end) as EXTENDED_SUPPORT_END_DATE,
max(case when col=5 then value else '' end) as SERVICE_PACK__SUPPORT_END_DATE,
max(case when col=6 then value else '' end) as NOTES
from
(SELECT
x.y.value('Col[1]', 'int') AS [Col],
x.y.value('Row[1]', 'int') AS [Row],
x.y.value('Value[1]', 'VARCHAR(200)') AS [Value]
FROM #data .nodes('//DocumentElement/TableData') AS x ( y )
) rawTableData
group by row
having row >0
order by row
"#
OSQL.EXE -E -Q $dbCommand
Any suggestions on how to rewrite this script where it will work?
I am assuming it is too long because you are using OSQL.exe and passing it as a command line parameter. Seeing you are using powershell I would just use built in .net capabilities and execute the query in that manner. If you need more info just search the internet for .net SQL ExecuteNonQuery and it will give you a lot of results.
The basics of it are as follows:
# Instantiate new SqlConnection object.
$Connection = New-Object System.Data.SQLClient.SQLConnection
# Set the SqlConnection object's connection string to the passed value.
$Connection.ConnectionString = "place a connection string here"
# Open the connection to the database.
$Connection.Open()
# Instantiate a SqlCommand object.
$Command = New-Object System.Data.SQLClient.SQLCommand
# Set the SqlCommand's connection to the SqlConnection object above.
$Command.Connection = $Connection
# Set the SqlCommand's command text to the query value passed in.
# this is where you pass the query string you wrote to
$Command.CommandText = $dbCommand
# Execute the command against the database without returning results (NonQuery).
$Command.ExecuteNonQuery()
# Close the currently open connection.
$Connection.Close()
I have written this code a few times but I did just grab it from this script which is available on Microsoft's Technet gallery https://gallery.technet.microsoft.com/scriptcenter/Perform-ExecuteNonQuery-a05eb40a
Related
In the code below I try to query an Excel file for a particular field "Username" Where there are blanks (= ''). But nothing gets returned to the data adapter.
The WHERE clause will return data for example if I specify a NOT LIKE 'thistext%', but again omits blanks.
Is there some way to better code the clause?
I have read there is an issue if fields are blank and need to be converted to DB NULL (or populated with a value) unsure how to do it.
$connection.ConnectionString = $connectstring
$connection.Open()
$objOleDbCommand.Connection = $connection
$objOleDbCommand.CommandText = "SELECT * FROM [$strSheetName] WHERE [Username] = ''"
$objOleDbAdapter.SelectCommand = $objOleDbCommand
$objOleDbAdapter.Fill($objDataTable)
$objDataTable | Export-Csv "C:\output\MyData_$dateandtime.csv" -NoTypeInformation
$connection.Close()
Try this instead to cover blanks and nulls.
Select * from [$strSheetName] where [Username] = '' OR [USERNAME] IS NULL
Note that this itself is not a powershell issue technically, you might encounter this same issue with other spreadsheets and databases.
I have a requirement where i need to see data for every week using a sql query.
I used query like SELECT * from table Between '27-08-2012' and '30-08-2012'.
Now my requirement is that i need a batch file that can given me this script based on user selection. Like user can give start date and end date and sql query should generate automatically.
A much less error prone way to get a date range is to use a graphical date picker
than manual user input which has to be checked for validity.
(month/day names will match your locale, not my German one)
This PowerShell script:
# Function Pick-Date
[void][System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms")
Function Pick-Date {
$Cal = new-object System.Windows.Forms.MonthCalendar
$Cal.ShowWeekNumbers = $true
$Cal.MaxSelectionCount = 10 # change this value for the max date distance
$Cal.Dock = 'Fill'
$Form = new-object Windows.Forms.Form
$Form.text = "Drag the mouse to select a date range then hit [enter]"
$Form.Size = new-object Drawing.Size #(656,620)
$btnSelect = new-object System.Windows.Forms.Button
$btnSelect.Size = "1,1"
$btnSelect.add_Click({ $Form.close() })
$Form.Controls.Add($btnSelect )
$Form.AcceptButton = $btnSelect
$Form.Controls.Add($Cal)
$Form.Add_Shown({$Form.Activate()})
[void]$Form.showdialog()
return ("SELECT * from table Between '"+
(Get-Date($Cal.SelectionStart) -format 'dd-MM-yyyy')+
"' and '"+
(Get-Date($Cal.SelectionEnd) -format 'dd-MM-yyyy')+
"'")
}
Pick-Date
Will have this output, you can save to a file.sql
PS> .\Pick-Date.ps1
SELECT * from table Between '27-08-2012' and '31-08-2012'
EDIT
This batch wrapper for the powerShell script will store the query in the variable SqlQuery
#Echo off&SetLocal EnableExtensions EnableDelayedExpansion
::Wrap Pick-Date.ps1 in same folder as batch
For /F "delims=" %%%A in (
'Powershell -NoP -NonI -NoLogo -File Pick-Date.ps1 '
) Do Set "SqlQuery=%%A"
Set SqlQuery
I have created a search query in VB.net using datasource which has two parameters.
SELECT [Product Code], Description, Input_Date, Price, Unit, Quantity, Markup, Total
FROM Inventory_Table
WHERE (? = ?)
I made two parameters because I want to search by specific columns, this is how i used the query:
Inventory_TableTableAdapter.SearchQuery(DBDataSet1.Inventory_Table, InvSearchCombo.Text, InvSearchTxt.Text)
First parameter would be a dropdown combobox containing all columns from the table, the second parameter would be an input textbox.
But whenever i try searching nothing would appear.
What seems to be the problem? I really want to implement this kind of search feature. Thanks in advance.
In this you can use a dynamic code
Dim columnQuery As String = "Description"
Using command As New SqlCommand( "select Description,Input_Date from dep where " & columnQuery & " = #par1", connection)
command.Parameters.AddWithValue("#par1", "descripcion")
End using
EDIT
A better form could be this:
First, create store procedure:
CREATE PROCEDURE SP_LIST_TABLA_BY_DYNAMIC_COLUMN
#PAR_COLUMN VARCHAR(20),
#PAR_VALUE VARCHAR(20)
AS
DECLARE #STRSQL NVARCHAR(MAX)
SET #STRSQL = 'SELECT PRODUCT_CODE,DESCRIP,INPUT_DATE FROM INVENTORY_TABLE WHERE ' + #PAR_COLUMN + ' = ' + #PAR_VALUE
EXEC sp_executesql #STRSQL
Then invoke it:
Using command As New SqlCommand( "SP_LIST_TABLA_BY_DYNAMIC_COLUMN", connection)
command.CommandType = CommandType.StoredProcedure
command.Parameters.AddWithValue("#PAR_COLUMN", "product_code")
command.Parameters.AddWithValue("#PAR_VALUE", "1")
Using reader As SqlDataReader = command.ExecuteReader()
While reader.Read()
End While
End using
End using
But like user #Basic says: if the column name is coming from user input (even if via a database) then you're going to be vulnerable to SQL injection attacks
One suggestion could be evaluate that par_column name exists and par_value don't have some special characters.
Imagine you have a combo filter and save the ID as #par1 and text field save as #par2
Combo have this values:
none (id: -999)
field1 (id: 0)
field2 (id: 1)
Not sure about how you set your parameter so I will use some pseudo code.
You can make a trick to dynamic set what filter to use.
SELECT *
FROM Inventory_Table
WHERE
(Field1 = #par2 and 0 = #par1)
OR (Field2 = #par2 and 1 = #par1)
OR (-999 = #par1)
So if you select Field1 then 0 = #par1 will be true and first filter will be active
if you select Field2 then 1 = #par1 will be true and second filter will be active
if none is select all rows are return.
I am trying to export the data from sql server tables to a csv file without ssms.
i am trying to achieve it by creating a stored procedure using bcp.
declare #sql nvarchar(4000);
select #sql = 'bcp "select * from table" queryout c:\file.csv -c -t, -T -S'+ ##servername
exec xp_cmdshell #sql
1 ) This query produces the expected results. But what i want is it should also include the column names in the csv files. So how can i achieve that ?
2) I want this result for all the tables in the given database. So how to do that ?
Please give some suggestions or solution as soon as possible
thanks
Only workarounds I know of...
Query the data dictionary and produce a csv field list, then
concatenate the header.csv with the data.csv. This would be querying the columns table, but you would need to take care to generate the SQL to match since you want to remove any and all chances that the column list doesn't match the data.
Create a view with the first row having all field names union all with a select on the data.
Something like:
SELECT 'a', 'b', 'c' UNION ALL SELECT a, b, c FROM table
This may require type conversions for dates, though.
I would suggest doing something along the lines of:
#echo off
bcp "SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<table_name>' AND TABLE_SCHEMA='<table_schema>'" queryout c:\file.csv -c -r, -T -S <server_name> -d <database_name>
bcp "select * from <table_schema>.<table_name>" queryout c:\data.csv -c -t, -T -S <server_name> -d <database_name>
echo. >> c:\file.csv
type c:\data.csv >> c:\file.csv
del c:\data.csv
in a .bat file.
I think that for what you want to do, it's better to just use the bcp command from a batch file/command line, instead of enabling the xp_cmdshell in SQL Server which could introduce a security issue.
Additionally, I'd like to point out that I'm not sure if the columns will always come out in the same order (in my case it did).
EDIT: Batch file explanation.
I basically ran 2 bcp commands and sent the output to 2 different files, as I couldn't find an option to append the output to another file. I then merely used the type command to add the data to the file already containing the column list, and deleted the file with the data as it is no longer needed.
Feel free to modify and mess around with it and let me know if you run into any problems.
Try this code if you want column name with SQL table data :-
public void ExportExcelFileToFolder()
{
string constr = ConfigurationManager.ConnectionStrings["constr"].ConnectionString;
using (SqlConnection con = new SqlConnection(constr))
{
using (SqlCommand cmd = new SqlCommand("SELECT * FROM MachineMaster"))
{
using (SqlDataAdapter sda = new SqlDataAdapter())
{
cmd.Connection = con;
sda.SelectCommand = cmd;
using (DataTable dt = new DataTable())
{
sda.Fill(dt);
using (XLWorkbook wb = new XLWorkbook())
{
wb.Worksheets.Add(dt, "SheetName");
HttpContext.Current.Response.Clear();
HttpContext.Current.Response.Buffer = true;
HttpContext.Current.Response.Charset = "";
HttpContext.Current.Response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
HttpContext.Current.Response.AddHeader("content-disposition", "attachment;filename=SqlExport.xlsx");
using (MemoryStream MyMemoryStream = new MemoryStream())
{
wb.SaveAs(MyMemoryStream);
string fileName = Guid.NewGuid() + ".xlsx";
string filePath = Path.Combine(System.Web.Hosting.HostingEnvironment.MapPath("~/ExeclFiles"), fileName);
MyMemoryStream.WriteTo(new FileStream(filePath, FileMode.Create));
HttpContext.Current.Response.Flush();
HttpContext.Current.Response.End();
}
}
}
}
}
}
}
Install-Package ClosedXML -Version 0.92.1
I have an SQL command I am attempting to execute.
The syntax is as follows
Public Shared Function CanRaiseWorkOrder(connection As IDbConnection, ProductID As Guid) As Boolean
Using sqlCmd As IDbCommand = connection.CreateCommand
With sqlCmd
.CommandTimeout = 30
.CommandType = CommandType.Text
.CommandText = "DECLARE #CanRaiseWorkOrder BIT, #WorkOrderQtyCount INT, #ProductAvailCount INT SET #WorkOrderQtyCount = (SELECT SUM(Qty) FROM WorkOrder WHERE ProductID = #ProductID AND WorkOrder.Status <> 4) --4 = Voided SET #ProductAvailCount = (SELECT Qty FROM Product WHERE ProductID = #ProductID) IF #WorkOrderQtyCount < #ProductAvailCount BEGIN SET #CanRaiseWorkOrder = 1 END ELSE BEGIN SET #CanRaiseWorkOrder = 0 END SELECT #CanRaiseWorkOrder AS CanRaiseWorkOrder"
Dim params As New List(Of IDbDataParameter)({
ProductDAL.CreateTSqlParameter("#ProductID", DbType.Guid, ProductID)
})
.Parameters.AddRange(params)
Return .ExecuteScaler(Of Boolean)()
End With
End Using
End Function
As you will probably notice there is some customization there in regards to how the parameters are created and the command executed but you can assume those aspects of the system work as required (We have a significant amount of code that functions correctly using those methods).
I will probably get some people asking why this is not a stored procedure and the answer is "because my boss said so".
I have run SQL profiler and here is the output that this query actually generates.
exec sp_executesql N'DECLARE #CanRaiseWorkOrder BIT, #WorkOrderQtyCount INT, #ProductAvailCount INT SET #WorkOrderQtyCount = (SELECT SUM(Qty) FROM WorkOrder WHERE ProductID = #ProductID AND WorkOrder.Status <> 4) --4 = Voided SET #ProductAvailCount = (SELECT Qty FROM Product WHERE ProductID = #ProductID) IF #WorkOrderQtyCount < #ProductAvailCount BEGIN SET #CanRaiseWorkOrder = 1 END ELSE BEGIN SET #CanRaiseWorkOrder = 0 END SELECT #CanRaiseWorkOrder',N'#ProductID uniqueidentifier',#ProductID='0908C780-763F-4CE6-B074-CEC01F4451B4'
Running the code in query analyser (when I originally created it) works fine but if I run the above query as outputted from the SQL command all I get is "Command(s) completed successfully."
Any ideas?
Figure I'll make it an answer, but also wanted to make sure I was accurate (considering how long I've been up now :grin:).
From what I can see (and this is probably due to condensing your statement for the sake of making it a one-liner in a code file) you have a comment declaration in the middle of your statement:
--4 = Voided [...]
What's happening is you're executing only your DECLARE and first SET command (which are done without error) but the rest of your statement is being ignored because it follows the comment declaration (--).
Make sure if you do condense your query that you remove commented lines. Once they're on one line, SQL doesn't care and WILL ignore anything past the --.