I am building an SSIS package to select some data as XML and export it to a flat file. I've running it via a for each loop, I have a sql execute task to select all records in a table and then loop through each one to create a single XML file.
Here is the SQL statement I am trying to execute as part of the data flow in the OLE DB Source Editor. The select is inside another SELECT (...) AS XML_TEXT to make the output to be text
SELECT
(SELECT SalesCode, ProductName, ManufacturingSite, Country, Language, Status, RevisionDate, ArchiveDate, PFFileName, XMLFileName, Version
FROM ir_cp_connect_data
WHERE (id = ?)
FOR XML RAW ('Data'), ROOT ('Sales'), ELEMENTS XSINIL)
AS XML_TEXT
I've tried to enter it direct as a SQL command and also in a variable. When I enter it directly into as a command, it errors saying it cannot parse the query, giving syntax error, permission violation or other non specific error.
When I switch it out to be a variable in data access mode, it errors with a long error message,
SSIS Error Code DTS_E_OLEDBERROR. An OLD DB Error has occurred, Error Code 0x80004005. An OLE DB record is available.
Again it then goes on to list 'syntax error, permission violation or other non specific error.'
If I run the select statement in SQL Management Studio it works, albeit I substitute the id = ? for id = #id and declare the #id in and assign it a value.
I'm working with VS2008 and SQL Server 2008R2.
I feel I am missing something very obvious but I cannot put my finger on it. If I drop the opening SELECT (..) AS XML_TEXT, the query parses fine, but the output is System.Byte[] which is no good for the flat file.
Any help would be much appreciated.
Related
Good Afternoon All,
I have spent about 6 hours trying to get formatting to work through SSIS using a Max Date Variable to identify into a where clause - Just have no luck!
I have created a variable called my_date which fetched the Max(Date) from a local SQL server table to understand the last load point for that table - using the below code:
SELECT CAST(FORMAT(MAX(Business_Date), 'dd-MMM-yyyy') AS varchar) AS my_date FROM Table
This fetches the date correctly as 17-Sep-2018.
I have then mapped my result set as my_date -> User::max_date
I have set my max_date variable to a string data type under the package scope.
I have tested my variable out by using breakpoints to ensure this runs all the way through in the correct format - and this works 100%.
I then have a data flow task running to fetch data from my ORACLE DB to insert into my SQL Server table which contains the following SQL command:
SELECT *
FROM Table2
WHERE (BUSINESS_DATE > to_date('#[User::max_date]', 'DD-MON-YYYY'))
However I get the ORA-01858 - [TABLE] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E07.
An OLE DB record is available. Source: "Microsoft OLE DB Provider for Oracle" Hresult: 0x80040E07 Description: "ORA-01858: a non-numeric character was found where a numeric was expected
".
If I go and replace my variable directly with the contents of the variable shown by the breakpoint in locales it works perfectly!
I have attempted multiple format types from the initial export through to the final where clause and this seems to be the closest I have come to pushing it through but it is still complaining about format.
Please help - Images below to help see setup.
Control Flow - displaying the execute SQL and the data flow task
Locale showing Variable is inserting after breakpoint is reached
Managed to get it working!
By adding an intermediary variable where the value and expression contains the following:
"SELECT *
FROM TABLE
WHERE (BUSINESS_DATE > to_date('"+#[User::max_date]+"' , 'DD-MON-YYYY'))"
I then amended my source OLEDB to SQL Command from variable and selected the above variable created and it worked perfectly!
Try mapping your user parameter in the "OLE DB Data Source Editor", under "Parameters".
1) Change SQL Command Text (change #[User::max_date] to ?), like this:
SELECT *
FROM Table2
WHERE (BUSINESS_DATE > to_date('?', 'DD-MON-YYYY'))
2) Then in the parameter editor, map parameter 1 to #[User::max_date].
https://learn.microsoft.com/en-us/sql/integration-services/data-flow/map-query-parameters-to-variables-in-a-data-flow-component?view=sql-server-2017
Also, the "Oracle Provider for OLE DB" behaves differently than the "Microsoft OLE DB Provider for Oracle", so it depends which you are using.
I have a pair of linked SQL servers: ServerA and ServerB. I want to write a simple INSERT INTO SELECT statement which will copy a row from ServerA's database to ServerB's database. ServerB's database was copied directly from ServerA's, and so they should have the exact same basic structure (same column names, etc.)
The problem is that when I try to execute the following statement:
INSERT INTO [ServerB].[data_collection].[dbo].[table1]
SELECT * FROM [ServerA].[data_collection].[dbo].[table1]
I get the following error:
Msg 16902, Level 16, State 48, Line 1
sp_cursor: The value of the parameter 'value' is invalid.
On the other hand, if I try to execute the following statement:
INSERT INTO [ServerB].[data_collection].[dbo].[table1] (Time)
SELECT Time FROM [ServerA].[data_collection].[dbo].[table1]
The statement works just fine, and the code is executed as expected. The above statement executes just fine, regardless of which or how many tables I specify to insert.
So my question here is why would my INSERT INTO SELECT statement function properly when I explicitly specify which columns to copy, but not when I tell it to copy everything using "*"? My second question would then be: how do I fix the problem?
Googling around to follow up on my initial hunch, I found a source I consider reliable enough to cite in an answer.
The 'value' parameter specified isn't one of your columns, it is the optional argument to sp_cursor that is called implicitly via your INSERT INTO...SELECT.
From SQL Server Central...
I have an ssis package that needs to populate a sql table with data
from a pipe-delimited text file containing 992 (!) columns per record.
...Initially I'd set up the package to contain a data flow task to use
an ole db destination control where the access mode was set to Table
or view mode. For some reason though, when running the package it
would crash, with an error stating the parameter 'value' was not valid
in the sp_cursor procedure. On setting up a trace in profiler to see
what this control actually does it appears it tries to insert the
records using the sp_cursor procedure. Running the same query in SQL
Server Management Studio gives the same result. After much testing and
pulling of hair out, I've found that by replacing the sp_cursor
statement with an insert statement the record populated fine which
suggests that sp_cursor cannot cope when more than a certain number
of parameters are attempted. Not sure of the figure.
Note the common theme here between your situation and the one cited - a bazillion columns.
That same source offers a workaround as well.
I've managed to get round this problem however by setting the access
mode to be "Table or view - fast load". Viewing the trace again
confirms that SSIS attempts this via a "insert bulk" statement which
loads fine.
I am trying following query in MS Access 2010 database:
Query:
SELECT ID, Title, Priority, Workflow_Instance_Step_ID:ID
FROM Task
Error:
Error Source: Microsoft Office Access Database Engine
Error Message: Syntax error (missing operator) in query expression 'Workflow_Instance_Step_ID:ID'
I know that field "Workflow_Instance_Step_ID:ID" is giving error as it has ':' operator, but I cant change it as it it coming from share point list.
I have tried '[Workflow_Instance_Step_ID:ID]' but still its giving an error.
Please give me your suggestion on the same.
I am unable to recreate your issue, at least with a native Access table in Access 2010. I created a table named [baz] with a single text field named [foo:bar]
foo:bar
-------
this
is
a
test
and the query
SELECT [foo:bar] FROM baz;
works fine, so a field name containing a colon character does not seem to cause problems for Access SQL as such.
An ODBC linked table to a SQL Server table with the same structure does not cause problems either.
In both cases the queries were run from within Access 2010 itself.
So we might be in diagnosis mode, so lets try
SELECT * FROM Task
and then look at the FieldNames that come back.
I am working with SQL Server Integration Services 2008. In my Data Flow task I got a OLEDB source. I choose SQL command as the Data Access mode and following is my sql text
Select * from table1 where ID=?
And from parameters tab my Parameter Name is zero "0" and value is coming from a package level variable.
I tried to replace the parameter name with "Parameter0" but I always get the same error that
Error: "No Value given for one or more required parameters"
But there is a value for my variable
You are doing the right thing, but it is the "Preview" that is not working for parameterized source tasks. Try executing the package and you should be ok.
I created an execute sql task to save the filenames encountered in a Foreach Loop task into a database table,
this works fine when I run the the individual task and it writes the filename to the database table (see below)
UNABLE TO POST IMAGES CURRENTLY
![task successfully runs][1]
However when i run the complete SSIS package, this task fails (see below)
UNABLE TO POST IMAGES CURRENTLY
![unsuccessful execution][2]
The error message I am getting is:
Error: String or binary data would be truncated.
[Execute SQL Task] Error: Executing the query "insert into filename_staging_table
(FileName) Val..." failed with the following error:
"The statement has been terminated.". Possible
failure reasons: Problems with the query,
"ResultSet" property not set correctly, parameters
not set correctly,
or connection not established correctly.
Any guidance with this greatly appreciated....I am completely stuck!!!!
Thanks
The error reads "String or binary data would be truncated." The reason the script works when run manualy is because you are only trying to insert one value. That value is within the length constraint of your column.
When you run the task, it is looping through a list of filenames and trying to insert each one. One of these names is too long for the field you are trying to insert it into.
You need to change the length of the field in SQL or you need to use a LEFT(filename, xx) on the filename itself so that a only the first xx characters are inserted into your table. (Change the xx to the number of characters to be inserted)