I have a string variable dest which holds a certain value. I need to check if this variable exists on a registered temptable . I use the below query to find it.
terminatecheck = sqlContext.sql("""
SELECT 1 as op from known where node = """+dest +""" and 1=1
""")
Now i need to compare the value of terminatecheck to "1" and terminate a loop.
I checked and found that terminatecheck is a row object. How exactly do i compare this ?
if terminatecheck.op =="1":
does not work
Calling sqlContext.sql("Select...") will return a dataframe which is lazy evaluated. You need to call an action like first, take or collect to get the expression evaluated and the value returned. You probably want to call first which returns a single row (of type Row). take and collect returns an array (of type Array[Row]).
terminatecheck = sqlContext
.sql("""SELECT 1 as op from known where node = """+dest +""" and 1=1""")
.first
Related
I have a dataframe with column of array (or list) with each element being a map of String, complex data type (meaning --String, nested map, list etc; in a way you may assume column data type is similar to List[Map[String,AnyRef]])
now i want to query on this table like..
select * from the tableX where column.<any of the array element>['someArbitaryKey'] in ('a','b','c')
I am not sure how to represent <any of the array element> in the spark SQL. Need help.
The idea is to transform the list of maps into a list of booleans, where each boolean indicates if the respective map contains the wanted key (k2 in the code below). After that all we have to check if the boolean array contains at least one true element.
select * from tableX where array_contains(transform(col1, map->map_contains_key(map,'k2')), true)
I have assumed that the name of the column holding the list of maps is col1.
The second parameter of the transform function could be replaced by any expression that returns a boolean value. In this example map_contains_key is used, but any check resulting in a boolean value would work.
A bit unrelated: I believe that the data type of the map cannot be Map[String,AnyRef] as there is no encoder for AnyRef available.
I return a JSON array from a TSQL procedure to Azure Data Factory. I want to know if at least 1 value in the array is equal to true. The JSON array has multiple fields included and multiple rows.
Setup overview:
Data Factory lookup activity.
TSQL procedure that returns 2 or more
rows.
Data Factory IF activity with conditional that checks JSON
returned for at least 1 instance of x.
Dummy procedure:
CREATE PROC dbo.usp_dummyProc
AS
SET NOCOUNT ON;
SELECT 1, 'a', 1
UNION
SELECT 2, 'b', 0
;
Data Factory pipe:
I tried:
#contains(activity('ActivityName').output.value.SqlFieldName, true)
Which, unsurprisingly led to:
The expression
'contains(activity('ActivityName').output.value.SqlFieldName, true)'
cannot be evaluated because property 'SqlFieldName' cannot be
selected. Array elements can only be selected using an integer index.
I cannot see an expression component that can iterate over the list returned to check for a value. I could write another procedure to deal with this, but ideally, I would prefer not to need to do so every time I want to solve this problem. This is where I looked.
You can cast your the result of activity('ActivityName').output.value to String, then use contains() to check whether it contains '"SqlFieldName":true'. Something like the following expression: #contains(join(activity('ActivityName').output.value,','),'"SqlFieldName":true')
How to find string values in text array using SQL query.
Suppose I have:
id location
1 {Moscow,New york}
2 {Mumbai}
3 {California,Texas}
I want to find id whose location is Moscow.I used:
select id from table where location in ('Moscow'); but get error:
ERROR: malformed array literal: "Moscow"
LINE 1: select id from table where location in ('Moscow');
DETAIL: Array value must start with "{" or dimension information.
I am using Postgres.
For DataType=Array, you can use the method=Any.
select id from table where 'Moscow' = Any(location)
As the document which describes DataType=Array:
8.14.5. Searching in Arrays
To search for a value in an array, each value must be checked. This
can be done manually, if you know the size of the array.
or use the method = Any:
9.21.3. ANY/SOME (array)
expression operator ANY (array expression) expression operator SOME
(array expression) The right-hand side is a parenthesized expression,
which must yield an array value. The left-hand expression is evaluated
and compared to each element of the array using the given operator,
which must yield a Boolean result. The result of ANY is "true" if any
true result is obtained. The result is "false" if no true result is
found (including the case where the array has zero elements).
For Searching in Array DataType you can use the ANY()
SELECT id FROM table WHERE 'Moscow' = ANY(location);
Live Demo
http://sqlfiddle.com/#!17/a6c3a/2
select id from demo where location like '%Moscow'%'
I did a rather easy view to return only rows where there is number is CONTRACT_ID column. CONTRACT_ID has data type number(8).
CREATE OR REPLACE VIEW cid AS
SELECT *
FROM transactions
WHERE contract_id IS NOT NULL
AND LENGTH(contract_id) > 0;
View works just fine until I scroll down to row ~2950 where I get ORA-01722. Same thing happens if I want to export data to Excel, my file gets only ~2950 rows instead of expected ~20k.
Any idea what might be causing this and how to resolve this issue?
Many thanks!
You wrote too much SQL.. The following will provide all the results you require:
CREATE OR REPLACE VIEW cid AS
SELECT *
FROM transactions
WHERE contract_id IS NOT NULL
You can't LENGTH() a number - a number is either null or it's a value, so you don't need this kind of check.
Passing a number to LENGTH() will turn it into a string first, i.e. LENGTH(TO_CHAR(numbercolumn)). You don't even need a LENGTH() check for null strings, as to oracle NULL string and a zero length string are equivalent, and calling LENGTH() on an empty string or a null, will return null, not 0 (so LENGTH(myNullStr) = 0 doesnt work out; it's not comparing 0 = 0, it's comparing null = 0 and null compared with anything is always false).
The only time this seems to cause confusion is when the string columns in the table are CHAR types rather than VARCHAR types, and people forget that assigning an empty string to a CHAR causes it to become space padded out to the CHAR length hence, not a zero length string any more
First of all, you should remove redundant condition about length(), it's senseless. I'm not sure how it can produce such error, but check whether error disappered after it.
If no, replace star (*) to some field names, say, contract_id. If it will fix error - it would appoint that error source somewhere into removed fields (say, if generated column used).
I cannot imagine how error can be still alive after that, by if so, I'd tried to move it into other tablespace and add into fields list a call of logging function which stores rowid's of rows read - thus check which row produces error.
I have a report in 2005 SSRS which I want to add a parameter to. The parameter would be comprised of a group of zip codes, but be selected as a single item in the list.
For example, I would like to have 5 zip codes as one selection in the list and 3 for another, etc:
Select 11111,22222,33333,44444,55555,66666 AS Boondock
Select 77777,88888,99999 AS Timbuck
Select Zip Codes NOT IN (11111-99999) AS Everything Else
So my selections in the dropdown would be:
Boondock
Timbuck
Everything Else
Can anyone help me with how I should go about creating this parameter?
Create a simple string parameter to present to the user. Let's call it ZipCodeSet.
Create a dataset that examines the #ZipCodeSet parameter and returns the appropriate list of zip codes. Call it ZipCodeSelection.
Create an internal multivaue parameter that uses ZipCodeSelection as both its Available Values and Default Values. Call it SelectedZipCodes.
Use SelectedZipCodes in your report's datasets.
The easiest solution here would probably to use a Calculated Field on your dataset, called LocationDescription, for example:
=SWITCH(Fields!ZipCode >= 11111 and Fields!ZipCode <= 66666, "Boondock", Fields!ZipCode >= 77777 and Fields!ZipCode <= 99999, "Timbuck",True, "Everywhere Else")
The lone true statement at the end is due to the SWITCH expression reading left-to-right and exiting once it evaluates one of the switches as TRUE. This way for each of the items in your table of ZipCodes you will always end up with a TRUE result.
I assume you're evaluating a range of ZipCodes, and not exact values of 11111,22222, and so on? If so, the switch will have more values. A sample of your data would help if you want an exact answer.
Once you have built your Calculated Field, you can then set up a Parameter (called #LocationParameter) with available values based on a query of your LocationDescription field, then just filter your dataset using:
Expression:
= Fields!LocationDescription
Operator: =
Value:
#LocationParameter
(if you want multiple selections on your parameter, change the operator to IN)
Hope that helps.