I have an SQL query written into a GET DATA command. It works perfectly as it is, but now I would like to modify it to take in a string value from my current dataset.
Sample query -
SELECT * FROM TABLE WHERE field IN ('AAA', 'BBB', 'CCC', 'DDD')
I would like the bracketed list to be a string value that already exists in my dataset (ref - previous post)
I have tried several things to date including using a concatenated string in the /SQL portion of the GET DATA command. For the concatenation I tried referencing the field itself and a scratch variable.
I've also attempted to call the command via a macro using the variable as an input. Unfortunately, the SQL string only goes through with the variable name rather than its contents.
I've seen a few posts that recommend using the WRITE OUT command for similar use cases but haven't found anything regarding concatenating into an SQL query. Would this be the only option for me?
Looks to me like write out is a good option for you.
Assuming you have one row of data in which myconcat has the value "'AAA', 'BBB', 'CCC', 'DDD'" (if you have more then one row in the data, first select only the one):
select if (the row with the full concatenated list) .
string cmd (a2000).
compute cmd=concat("define !mysqlcommand() 'SELECT * FROM TABLE WHERE field IN (", rtrim(myconcat), ")' !enddefine.").
write out = "path\create val list macro.sps" /cmd.
execute.
insert file = "path\create val list macro.sps".
The code writes a new syntax file where it creates a macro call. After the insert command runs this, you theoretically have a macro defined so when you use the macro call !mysqlcommand in syntax it will automatically write in the full SQL command you've created, so:
.
.
/sql = !mysqlcommand
.
.
Related
I have a table with a 'query' field containing a select sql and another 'parameters' field containing the sql parameters. I have merged these two fields into a new field containing a correct select sql statement. Now I need to execute this new field containing select sql, get the return from select (the output fields) and generate an excel file.
Use Table-Input if you are interested in a query result set. Table-Input supports SQL parameters, so no need to build the statement yourself using e.g. Replace-In-String, and tripping over escapes on your way. Also, there's variable substitution, just in case you can't live with a single template.
Update 21:14 GMT
I'm not very fond of the way you try to prepare the SELECT statement, but here we go, assuming it's a single statement we have:
Create a job with a Start entry and 2 Transformation entries (T1, T2). Let T1 produce the field containing your SELECT statement and use a Set-Variables step to make the statement available to T2 as variable SELECT. In T2 use a Table-Input step referencing ${SELECT} in the SQL statement text area. Don't forget to enable option "Replace variables in script".
From now on it's a matter of taste. I would prefer to create a CSV file using Text-File-Output. Using the right field separator Excel will open the file after double-clicking it. The advantage of Text-File-Output is that you don't have to specify the fields you don't know at design-time anyway. An empty field list will just handle all fields coming in. Comparable to the total projection in a Table-Input which will create the necessary fields from the retrieved columns downstream.
If you must produce an Excel workbook, you'll have to learn about metadata injection. That would be a separate project for a beginner, though. There are samples in your Kettle installation folder. And there is a very active community if you find yourself in trouble.
I have a table with a 'query' field containing a select sql and another 'parameters' field containing the sql parameters. I have already merged these two fields into a new field containing a correct select sql statement. Now I need to execute this new field containing select sql, get the return from select (the output fields) and generate an excel file.
In your transformation, you can use the Execute row SQL script (under scripting), which does exactly what you want, executing the SQL contained in the selected input field for each row passed to the step.
Please do consider if that's really what you want. It works fine for a small set or complicated logic, but is very inefficient compared to batch inserts and updates.
If your SQL statements do look like "INSERT [some data]" and you have many rows to process, consider splitting the streams with filter steps and applying calculations or constants to each case to set the values correctly, then directing all to a table output step.
I need to execute a simple INSERT INTO query on an old Access97 database.
I'm starting with a very short example - which doesn't work:
INSERT INTO [MY-TABLE]( [Field1] )
VALUES ( "blabla" )
MY-TABLE is the actual name of the table and Field1 is a String field.
I get the error:
Query input must contain at least one table or query
Because I need to insert literal value, I don't want to use a query here (i.e. a SELECT FROM)
Reading also the docs (https://msdn.microsoft.com/en-us/library/bb208861(v=office.12).aspx) I don't see where my SQL is wrong.
UPDATE
Here a couple of screen shot of the actual table and fields:
here the SQL code:
Anyway...
SOLVED!!
It works even with double quotes.
The problem was the push button I was using to check: using the "View" button leads to the error above. Instead I must use the "exclamation mark".
I need to execute the query using the "exlamation mark" button instead of the view one. This is because my query has not a result set to view - hence the error I saw.
By the way I confirm the syntax is accepted both with single or double quotes.
I am discovering Pentaho DI and i am stuck with this problem :
I want to insert data from a csv file to a custom DB, which does not support the "insert table" step. So i would like to use the sql script step, with one request :
INSERT INTO myTable
SELECT * FROM myInput
And my transformation would like this :
I don't know how to get all my data from the csv to be injected in the "myInput" field.
Could someone help me ?
Thanks a lot :)
When you first edit the SQL Script step, click 'Get fields' button. This is going to load the parameters(fields from your csv) into box on the bottom left corner. Delete the parameters(fields) you don't want to insert.
In your sql script write your query something like this where the question marks are your parameters in order.
insert into my_table (field1,field2,field3...) values ('?','?','?'...);
Mark the checkboxes execute for each row and execute as a single statement. That's really about it. Let me know if you have any more questions and if you provide sample data I'll make you a sample ktr file to look at.
I think you get the wrong way. You should get a cvs file input step and a table output step.
As rwilliams said, In cvs file input step Get fields; the more important, in table output there is a Database Fields tab. Enter field mapping is right choise.The guess function is amazing.
And more, system can generate target table create sql statement when target table not exists in target connection Db server.
Use the following code
with cte as
(
SELECT * FROM myInput
)
select *
into myTable
from cte;
We have a SQL Server table for user settings. Originally the settings were domain objects which had been serialized as XML into the table but we recently begun serializing them as binary.
However, as part of our deployment process we statically pre-populate the table with predefined settings for our users. Originally, this was as simple as copying the XML from a customized database and pasting it into an INSERT statement that was ran after the database was built. However, since we've moved to storing the settings as binary data we can't get this to work.
How can we extract binary data from a varbinary column in SQL Server and paste it into a static INSERT script? We only want to use SQL for this, we don't want to use any utilities.
Thanks in advance,
Jeremy
You may find it easier to store a template value in a config table somewhere, then read it into a variable and use that variable to fill your inserts:
DECLARE #v varbinary(1000)
SELECT #v = templatesettings from configtable
INSERT INTO usertable VALUES(name, #v, ....)
From SQL Server 2008 onwards you can use Tasks > Generate Scripts and choose to include data. That gives you INSERT statements for all rows in a table which you can modify as needed.
Here's the steps for SQL 2008. Note that the "Script Data" option in SQL 2008 R2 is called "Types of data to script" instead of "Script Data".
I presume you're OK with utilities like Query Analyzer/Mangement Studio?
You can just copy and paste the binary value returned by your select statement (make sure that you are returning sufficient data), and prefix it with "0x" in your script.
If I understand you correctly, you want to generate a static script from your data. If so, consider performing a query on the old data that concatenates strings to form the SQL statements you'll want in the script.
First, figure out what you want the scripted result to look like. Note that you'll need to think of the values you're inserting as constants. For example:
INSERT INTO NewTable VALUES 'value1', 'value2'
Now, create a query for the old data that just gets the values you'll want to move, like this:
SELECT value1, value2
FROM OldTable
Finally, update your query's SELECT statement to produce a single concatenated string in the form of the output you previous defined:
SELECT 'INSERT INTO NewTable VALUES ''' + value1 + ''', ''' + value2 + ''''
FROM OldTable
It's a convoluted way to do business, but it gets the job done. You'll need a close attention to detail. It will allow a small (but confusing) query to quickly output very large numbers of static DML statements.
David M's suggestion of using the 0x prefixing works but i had to add an extra 0 at the end of varbinary data that i was trying to insert.
See the stackoverflow entry below to see the issue with additional 0 that gets added when converting to varbinary or saving to varbinary column
Insert hex string value to sql server image field is appending extra 0