I return a JSON array from a TSQL procedure to Azure Data Factory. I want to know if at least 1 value in the array is equal to true. The JSON array has multiple fields included and multiple rows.
Setup overview:
Data Factory lookup activity.
TSQL procedure that returns 2 or more
rows.
Data Factory IF activity with conditional that checks JSON
returned for at least 1 instance of x.
Dummy procedure:
CREATE PROC dbo.usp_dummyProc
AS
SET NOCOUNT ON;
SELECT 1, 'a', 1
UNION
SELECT 2, 'b', 0
;
Data Factory pipe:
I tried:
#contains(activity('ActivityName').output.value.SqlFieldName, true)
Which, unsurprisingly led to:
The expression
'contains(activity('ActivityName').output.value.SqlFieldName, true)'
cannot be evaluated because property 'SqlFieldName' cannot be
selected. Array elements can only be selected using an integer index.
I cannot see an expression component that can iterate over the list returned to check for a value. I could write another procedure to deal with this, but ideally, I would prefer not to need to do so every time I want to solve this problem. This is where I looked.
You can cast your the result of activity('ActivityName').output.value to String, then use contains() to check whether it contains '"SqlFieldName":true'. Something like the following expression: #contains(join(activity('ActivityName').output.value,','),'"SqlFieldName":true')
Related
I've been tasked with creating a report for my company. The report is generated from the results returned by the Stored Procedure spGenerateReport, which has multiple filters.
Inside the SP, this is how the filter is expected to work:
SELECT * FROM MyTable WHERE column1 IN (
'filters', 'for', 'this', 'report'
)
Entering the code above yields ~30000 rows in 9s. However, I want to be able to change my SP's filter by passing it a single argument (since I may use 1 or 2 or n filters), like so:
spGenerateReport 'Filters,for,this,report'
For this I have the User-Created Function fnSplitString (yes, I do know that there is a STRING_SPLIT function but I can't use it due to a lower compatibility level of my database) which splits a single string into a table, like so:
SELECT splitData FROM fnSplitString('Filters,for,this,report')
Returns:
splitData
------
Filters
for
this
report
Thus the final code in my SP is:
SELECT * FROM MyTable WHERE column1 IN (
SELECT * FROM fnSplitString('Filters,for,this,report')
)
However, this instead yields ~10000 rows in 60s. The time taken to complete this SP is weird but isn't too much of a problem, however nearly a quarter of my rows disappearing into the void certainly is. The results only have rows from the first couple filters (for example, 'Filters' and 'for'; if I change the order of the arguments (e.g.: fnSplitString('report,for,Filters,this')), I get a different number of rows, and only from filters 'report', 'for', 'Filters'! I don't understand why using the function returns different results than those obtained when using the literal strings. Is there some inside gimmick that I'm not aware of?
PS - I'm sorry in advance for being bad at explaining myself, and for any grammar mistakes
You should definitely be getting the same results with both techniques. Something is wrong.
You havent posted the fnSplitString code but I suspect fnSplitString is not outputting the last string in the list, or maybe the last string in the list is being truncated before it reaches fnSplitString so that no matches are found.
e.g. if the parameter going into your spGenerateReport stored procedure is varchar(20) then what will reach the function is 'Filters,for,this,rep' with the last bit truncated.
SSRS, for example, will truncate strings that are being passed into an SP instead of warning you with an error message
I am trying to use a cell as a parameter in Excel powerquery. The query works without this, but I have to manually input the values, which I need to constantly change them in the query in other to get the results that I want.
Query (Advanced Editor):
let
Criteria01 = Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{0},
Criteria02 = Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{1},
Criteria03 = Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{2},
Source = Sql.Database("SERVERNAMEHERE", "DATABASENAMEHERE", [Query="SELECT DISTINCT [...........] AND (TABLEPREF.COLUMNHERE like '%MANUALVALUE01%' OR#(lf)TABLEPREF.COLUMNHERE like '%MANUALVALUE02%' OR#(lf)TABLEPREF.COLUMNHERE like '%MANUALVALUE03%' OR#(lf)TABLEPREF.COLUMNHERE like Criteria01)#(lf)#(lf)#(lf)order by 1 asc"])
in
Source
"Servers" is the table name and "ServerSearch" is the column header. If I check the step for Criteria01/etc it will show me the correct value of that table that I need to use.
Original query done in Sql-Server. I have no problems when running the query with only LIKE '%MANUALVALUES%' lines.
My main goal is to automatically get N values of "MANUALVALUES" from a table in a sheet, which will be used as an input for comparing WHERE TABLEPREF.COLUMNHERE like '%VALUEHERE%'. I must use this and I can't get the whole table/database because there are way too many results besides the ones that I want.
However for test purposes at this moment, I am trying to use only 1-3 values, the first 3 of this table (Criteria{0}{1}{2} in the query above). However, if I try to do something like TABLEPREF.COLUMNHERE like Criteria01 I get the following error:
DataSource.Error: Microsoft SQL: Invalid column name 'Criteria01'.
Details:
DataSourceKind=SQL
DataSourcePath=dalsql390;itdw
Message=Invalid column name 'Criteria01'.
Number=207
Class=16
So my questions are:
I am getting the table cell value by the right way? Meaning:
Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{0}.
How do I refer this value in my query? Since the way that I wrote
that query bought me that error.
Also please note that if change TABLEPREF.COLUMNHERE like
Criteria01 to CHG1.CI_Name like "Criteria01" I get the
following error:
Expression.SyntaxError: Token Comma expected.
After fixed 1 and 2, how can I use this dynamically? For
example, instead of getting values of index 1 2 3, what if I want to
use a whole table? I know that using
Excel.CurrentWorkbook(){[Name="Servers"]}[Content] will bring me the whole table of values (1 column, unknown number of rows), but
how do I use this table content 1 by 1 in my query?
That will get the value, but you can't refer to steps inside of text values by putting the step name inside of it.
You have a couple options for doing this dynamically.
Use Value.NativeQuery to create a parameterized query where you can pass in other values as parameters. For example, Value.NativeQuery(Sql.Database("SERVERNAMEHERE", "DATABASENAMEHERE"), "select #a, #b", [a = 1, b = "x"]) will return the table [1, x]. You can put in the step name in the record value to pass that it (e.g. replace "x" with Criteria01).
Add the text values directly in the query field, e.g. [Query = "select " & Criteria01 ";"]. This is highly discouraged since this can lead to SQL injection issues.
For the third question, it depends what you want to do with the list of values. At some point you will likely need List.Accumulate to turn them all into a single text value which can be placed in the query value, and maybe to turn them into a record to place into the parameters value.
I have a string variable dest which holds a certain value. I need to check if this variable exists on a registered temptable . I use the below query to find it.
terminatecheck = sqlContext.sql("""
SELECT 1 as op from known where node = """+dest +""" and 1=1
""")
Now i need to compare the value of terminatecheck to "1" and terminate a loop.
I checked and found that terminatecheck is a row object. How exactly do i compare this ?
if terminatecheck.op =="1":
does not work
Calling sqlContext.sql("Select...") will return a dataframe which is lazy evaluated. You need to call an action like first, take or collect to get the expression evaluated and the value returned. You probably want to call first which returns a single row (of type Row). take and collect returns an array (of type Array[Row]).
terminatecheck = sqlContext
.sql("""SELECT 1 as op from known where node = """+dest +""" and 1=1""")
.first
On an Oracle DB I have a table with SDO_GEOMETRY objects. I would like to query the database for those polygons with less than x edges. In theory this would be easy with a query like
SELECT * FROM myTable t WHERE LENGTH(t.geometry.sdo_ordinates) < x
Obviously the LENGTH funtion is defined for char and the type of
t.geometry.sdo_ordinates is oracle.sql.ARRAY so that doesn't work. Shouldn't there be a trivial way to SELECT the length or an array in Oracle? Somehow I'm unable to get the syntax right.
PS: I kind of solved my search with the following query, still the original questerion remains, isn't there an array size/length function?
SELECT * FROM myTable t WHERE LENGTH(t.geomety.Get_WKT()) < (x * c)
No, there is no simple sql function that counts the elements of an array.
However as mentioned here, another idea is a PL/SQL script.
create or replace function get_count(ar in SDO_ORDINATE_ARRAY) return number is
begin
return ar.count;
end get_count;
t.geometry.sdo_ordinates.COUNT is a PL/SQL attribute that can be used within functions/procedures. Thus that is not a function useable in plain SQL.
Attribute:
value.someAttribute
Function:
doSomething(value)
Clarification: Functions have return values, procedures don't. Source
I have a report in 2005 SSRS which I want to add a parameter to. The parameter would be comprised of a group of zip codes, but be selected as a single item in the list.
For example, I would like to have 5 zip codes as one selection in the list and 3 for another, etc:
Select 11111,22222,33333,44444,55555,66666 AS Boondock
Select 77777,88888,99999 AS Timbuck
Select Zip Codes NOT IN (11111-99999) AS Everything Else
So my selections in the dropdown would be:
Boondock
Timbuck
Everything Else
Can anyone help me with how I should go about creating this parameter?
Create a simple string parameter to present to the user. Let's call it ZipCodeSet.
Create a dataset that examines the #ZipCodeSet parameter and returns the appropriate list of zip codes. Call it ZipCodeSelection.
Create an internal multivaue parameter that uses ZipCodeSelection as both its Available Values and Default Values. Call it SelectedZipCodes.
Use SelectedZipCodes in your report's datasets.
The easiest solution here would probably to use a Calculated Field on your dataset, called LocationDescription, for example:
=SWITCH(Fields!ZipCode >= 11111 and Fields!ZipCode <= 66666, "Boondock", Fields!ZipCode >= 77777 and Fields!ZipCode <= 99999, "Timbuck",True, "Everywhere Else")
The lone true statement at the end is due to the SWITCH expression reading left-to-right and exiting once it evaluates one of the switches as TRUE. This way for each of the items in your table of ZipCodes you will always end up with a TRUE result.
I assume you're evaluating a range of ZipCodes, and not exact values of 11111,22222, and so on? If so, the switch will have more values. A sample of your data would help if you want an exact answer.
Once you have built your Calculated Field, you can then set up a Parameter (called #LocationParameter) with available values based on a query of your LocationDescription field, then just filter your dataset using:
Expression:
= Fields!LocationDescription
Operator: =
Value:
#LocationParameter
(if you want multiple selections on your parameter, change the operator to IN)
Hope that helps.