I am working on an SSIS 2010 package that makes use of multiple package variables that I want to populate with the results of a SQL query. We have an app settings table that is set up as such:
[ID] [settingName] [settingValue]
1 appName test app
2 scheduled yes
Within the package, I have variables set up for appName and scheduled but I'm having a hard time figuring out the best way to load them from the database. The query is running successfully, just a simple select statement, but I'm not sure what to do next. I've found suggestions saying I should parse it out in a script task but it seems like there should be a much easier way to do this.
Your query needs to return a single row that contains the values for each variable in separate columns:
Then map the result set columns to your variables:
In short, you need to pivot your query.
Edit: Like this. (minus the table variable; use your actual table)
declare #pivot table (settingname varchar(20), settingvalue varchar(20))
insert into #pivot
values
('appname','test app'),
('scheduled','yes')
SELECT *
FROM (
SELECT settingname, settingvalue from #pivot
) as source
pivot
(
max(settingvalue)
for settingname in ([appname],[scheduled])
)
as pvt
Related
With GDPR in the UK on the looming horizon and already have a team of 15 users creating spurious SELECT statements (in excess of 2,000) across 15 differing databases I need to be able to create a method to capture an already created SELECT statement and be able to assign surrogate keys/data WITHOUT rewriting every procedure we already have.
There will be a need to run the original team members script as normal and there will be requirements to pseudo the values.
My current thinking is to create a stored procedure along the lines of:
CREATE PROC Pseudo (#query NVARCHAR(MAX))
INSERT INTO #TEMP FROM #query
Do something with the data via a mapping table of real and surrogate/pseudo data.
UPDATE #TEMP
SET FNAME = (SELECT Pseudo_FNAME FROM PseudoTable PT WHERE #TEMP.FNAME = PT.FNAME)
SELECT * FROM #TEMP
So that team members can run their normal SELECT statements and get pseudo data simply by using:
EXEC Pseudo (SELECT FNAME FROM CUSTOMERS)
The problem I'm having is you can't use:
INSERT INTO #TEMP FROM #query
So I tried via CTE:
WITH TEMP AS (#query)
..but I can't use that either.
Surely there's a way of capturing the recordset from an existing select that I can pull into a table to amend it or capture the SELECT statement; without having to amend the original script. Please bear in mind that each SELECT statement will be unique so I can't write COLUMN or VALUES etc.
Does any anyone have any ideas or a working example(s) on how to best tackle this?
There are other lengthy methods I could externally do to carry this out but I'm trying to resolve this within SQL if possible.
So after a bit of deliberation I resolved it.
I passed the Original SELECT SQL to SP that used some SQL Injection, which when executed INSERTed data. I then Updated from that dataset.
The end result was "EXEC Pseudo(' Orginal SQL ;')
I will have to set some basic rules around certain columns for now as a short term fix..but at least users can create NonPseudo and Pseudo data as required without masses of reworking :)
Let me simplify with an example. Let's say I have the following query saved on:
C:\sample.sql
grp.id IN
(001 --Bob
,002 --Tom
,003 --Fay
)
Now, that group of IDs could change, but instead of updating those IDs in every query it's related to, I was hoping to just update in sample.sql and the rest of the queries will pull from that SQL file directly.
For example, I have several queries that would have a section like this:
SELECT *
FROM GROUP grp
WHERE grp.DATERANGE >= '2017-12-01 AND grp.DATERANGE <= '2017-12-31
AND -- **this is where I would need to insert that query (ie. C:\sample.sql)**
More explained update:
Issue: I have several reports/queries having the same ID filter (that's the only thing in common between those reports)
What's needed: Instead of updating those IDs every time they change on each report, I was wondering if I can update those IDs in it's own SQL file (like the example above) and have the rest of the queries pull from there.
Note. I can't create a table or database in the used database.
Maybe the bulk insert utility could help. Hold your data in csv files and load them into temp tables at run time. Use these temp tables to drive your query.
CREATE TABLE #CsvData(
Column1 VARCHAR(40),
Column2 VARCHAR(40)
)
GO
BULK
INSERT #CsvData
FROM 'c:\csvtest.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
--Use #CsvData to drive your query
SELECT *
FROM #CsvData
maybe what you could use is a CTE (Common Table Expression) to pull your IDs using an additional query, specially if you only have read access. It would look something like this:
WITH myIDs AS (select IDs from grp where (conditions to get the IDs))
SELECT *
FROM grp
WHERE grp.DATERANGE BETWEEN '2017-12-01 AND '2017-12-31'
AND IDs in (select * from myIDs)
I've changed the dates syntax to use BETWEEN since it's more practical but only works if you have a SQL Server 2008 or later
Hope this helps!
Cheers!
The only chance to build a query out of text fragments is dynamic SQL:
Try this:
DECLARE #SomeCommand VARCHAR(MAX)='SELECT * FROM sys.objects';
EXEC(#SomeCommand);
Returns a list of all sys.object entries
Now I append a WHERE clause to the string
SET #SomeCommand=#SomeCommand + ' WHERE object_id IN(1,2,3,4,5,6,7,8,9)';
EXEC(#SomeCommand);
And you get a reduced result.
Another option is dynamic IN-list with a CSV paramter.
This is forbidden: DECLARE #idList VARCHAR(100)='1,2,3,4' and use it like IN (#idList).
But this works:
DECLARE #idList VARCHAR(100)='1,2,3,4,5,6,7,8,9';
SELECT sys.objects.*
FROM sys.objects
--use REPLACE to transform the list to <x>1</x><x>2</x>...
OUTER APPLY(SELECT CAST('<x>' + REPLACE(#idList,',','</x><x>') + '</x>' AS XML)) AS A(ListSplitted)
--Now use the XML (the former CSV) within your IN() as set-based filter
WHERE #idList IS NULL OR LEN(#idList)=0 OR object_id IN(SELECT B.v.value('.','int') FROM ListSplitted.nodes('/x') AS B(v));
With a version of SQL Server 2016+ this can be done much easier using STRING_SPLIT().
This approach allows you to pass the id-list as simple text parameter.
I am trying to query bamboo's database to get the following information. I want to find out the trigger information, which is what environment is triggered by which branch on which build plan. The problem is that some environments can be triggered by other environments.
So far I have looked up the Bamboo database and I seem to have most if not all the information that I need. What I have done is there is a table in the Bamboo database called dbo.Deployment_Environment which has the fields, EnvironmentID and Triggers_XML_Data which is exactly what I need from it. Then there is another table called the dbo.Deployment_Result which has the EnvironmentID, Deployment_State and the Trigger_Reason.
From these two tables what I think I need to do is this. select all these columns but if the trigger_reason contains environment and also if the triggers_xml_data contains environment, then get that environment from the triggers_xml_Data (this is under the xml node /item/value). Once you get that ID select all the same columns until and repeat the process if environment is the reason for deployment again until it is not. The triggers_xml_data data type is ntext and the trigger_reason is nvarchar.
I am not exactly sure how I can do this in SQL this is what I am trying so far but I can't get the right information:
alter proc dbo.myStoredProc3
as
Declare #EnvironmentID int
select
#EnvironmentID as ENVIRONMENT_ID,
de.ENVIRONMENT_ID, dr.DEPLOYMENT_STATE,
de.TRIGGERS_XML_DATA, dr.TRIGGER_REASON, de.NAME
from
dbo.BUILDRESULTSUMMARY as br,
dbo.DEPLOYMENT_ENVIRONMENT as de,
dbo.DEPLOYMENT_RESULT as dr
where
dr.TRIGGER_REASON like '%environment%'
(
select de2.ENVIRONMENT_ID as 'test'
from dbo.DEPLOYMENT_ENVIRONMENT as de2
)
I know this is not going to get the right information but every time I tried to do case's or if statements in the SQL it would create an error too: How can I do this query correctly? I think that I all the information I need but if not I can add more.
I have also tried to do the following:
While (#Counter <= 5)
Begin
Select de.NAME As 'Deployment Name', dr.TRIGGER_REASON as 'Trigger Reason', dr.DEPLOYMENT_STATE as 'Status'
from dbo.DEPLOYMENT_ENVIRONMENT as de, dbo.DEPLOYMENT_RESULT as dr
Where de.TRIGGERS_XML_DATA like '%environment%' and dr.TRIGGER_REASON like '%environment%'
SET #Counter = #Counter + 1
END
But this will not link the columns that I am getting. If the Trigger_Reason is to do with the environment then in the ntext the it will have a node that has the environmentID, I want to get this and set it as a variable, which I can use to get further columns that link to together e.g. Select name from dbo.Environment where #NewEnvironmentID = de.EnvironmentID.
Edit
Using the following SQL query:
alter proc getEnvironmentTriggers
#EnvironmentID int
as
Select a.NAME, a.TRIGGERS_XML_DATA, a.TRIGGER_REASON, a.ENVIRONMENT_ID
From (Select de.NAME, de.TRIGGERS_XML_DATA, de.ENVIRONMENT_ID, dr.TRIGGER_REASON From dbo.DEPLOYMENT_ENVIRONMENT as de
Inner join dbo.DEPLOYMENT_RESULT as dr
on dr.ENVIRONMENT_ID = de.ENVIRONMENT_ID
Where #EnvironmentID = de.ENVIRONMENT_ID and de.TRIGGERS_XML_DATA like '%Environment%' and dr.TRIGGER_REASON like '%Environment%' ) a
I am getting close to what I need. Doing this and the executing the Stored procedure like this: exec dbo.getEnvironmentTriggers 15892483, this will return in the query inside the ntext l of the TRIGGERS_XML_DATA, will have a value of 18317322, and then using that ID I can get the value of 19234819, that is then the last environment that triggers all of my environments from that ID.
So what I am trying to do is this:
In an automated way get all of these ID's possibly by doing the following.
Exec the stored procedure using all of the ID's (maybe using some sort of for each way).
Search the ntext column in the SQL to get the environment ID (this is in the following node /item/value.
Store that as a variable, then use it to get all the columns from the other stored procedure.
If there is returned columns using the stored procedure, run the procedure again, if not export all of the TRIGGER_XML_DATA columns that were got from stored procedures.
Is there a way that I can do this?
See my answer here - https://answers.atlassian.com/questions/44908050/how-to-get-the-trigger-information-from-the-bamboo-database
Not sure it's possible to do with plain SQL
What about using Bamboo REST API
http://bamboo_host/rest/api/latest/plan.json?expand=plans.plan.actions
I am trying to move everything from one table to another table. I know how to do that if all of the columns had the same data type, however one column is different. On the new table its varcahar(24) and on the old table its a bigint.
This is what i have so far, Im using the first select statement to set the id, and the rest to add it to the new table. However this always returns the last id, and i need it to be synced up with the second select statement
Any suggestions on how to do this would be awesome. i tried to google it for about an hour but couldnt come up with anything. I am using SQL Server 2012
use DatabaseA
GO
DECLARE #id bigint;
SELECT #id= columnName FROM differentDB.tableB
INSERT INTO tableA
(buyer_id, restOfTheColumns)
Select CAST(#id AS varchar(24)), restOfTheColumns
FROM differentDB.tableB
GO
Variables don't work like that. T-SQL variables hold data values, not object names. Furthermore, that's a scalar variable. It only holds one value.
That said, you don't need the variable at all. You can just do this:
INSERT INTO tableA (buyer_id, restOfTheColumns)
SELECT CAST(columnName AS varchar(24)), restOfTheColumns
FROM differentDB.tableB
I want to know if this is possible to do in SSIS Execute SQL task. I want to check for existence of a table, and if it doesn't exist, create the table. I want to keep the table name in a variable so I can execute the task against different databases and be able to change the table name.
As an example I have this code:
if not exists (select * from sysobjects where name='Table1' and xtype='U')
Create Table Table1(
Col11 varchar(10) primary key,
Col12 varchar(10)
);
For flexibility I want to store the value Table1 in a variable and keep changing it to create different tables each time I run the package.
Yes, This is possible.
Here is how I would do it:
Build another variable for your SqlStatement from your TableName
variable. Depending on your version of SSIS this can bedone in two
ways:
In SSIS 2008 you need to use Script Task to do the string concatenations.
In SSIS 2012 you can leverage the Expression Task.
You can use the Exressions tab in Execute SQL Task and add an
expression for SqlStatementSource and point it to your Variable.
the variable should be the complete statement.