Related
In this project I'm currently working on (I'm building a bridge between a desktop app and a new e-shop) there is a products table that has some spare columns defined that can be used for whatever reason the end user might need some custom data to be stored into.
So, the user needed to set a true/false flag to determine whether the products would appear in three different sliders... Unfortunately, the person who implemented this, didn't even use the same type of spare columns... So,
Slider1's flag is stored in a varchar(50) column
Slider2's flag is stored in a float column
Slider3's flag is stored in a float column
Additionally I ran a SELECT DISTINCT <column> for each one of them to get an idea of the actual data stored in each column and got the following results:
The varchar column has the following data stored in it:
FLDSTRING1
NULL
''
0
1
194276400456
The float column has the following data stored:
FLDFLOAT5
NULL
0
1
And the other float column has this:
FLDFLOAT6
NULL
1
Also, I ran the following query to find the different combinations of the data stored for each column:
SELECT FLDSTRING1, FLDFLOAT5, FLDFLOAT6
FROM MATERIAL
GROUP BY FLDSTRING1, FLDFLOAT5, FLDFLOAT6
and got the following combinations...
FLDSTRING1
FLDFLOAT5
FLDFLOAT6
NULL
NULL
NULL
NULL
NULL
1
NULL
0
NULL
NULL
1
NULL
NULL
1
1
''
NULL
NULL
''
NULL
1
0
NULL
NULL
0
0
NULL
1
NULL
NULL
1
NULL
1
1
0
NULL
1
1
NULL
1
1
1
194276400456
0
NULL
What I need after all this introduction...
I want a concatenated string of three comma-separated values like this
NEWPROD for when FLDSTRING1 would evaluate to true - anything not NULL, 0, or ''
CUSTOM1 for when FLDFLOAT5 would evaluate to true - basically the value 1
CUSTOM2 for when FLDFLOAT6 would evaluate to true - again value 1
After some trial and error I managed to bring this to a point that it kind of works, in the sense that it brings the correct values, not comma-separated though...
SELECT
FLDSTRING1, FLDFLOAT5, FLDFLOAT6,
CONCAT(CASE WHEN ISNULL(FLDSTRING1, '') = '' THEN '' ELSE 'NEWPROD' END,
CASE WHEN ISNULL(FLDFLOAT5, '') = '' THEN '' ELSE 'CUSTOM1' END,
CASE WHEN ISNULL(FLDFLOAT6, '') = '' THEN '' ELSE 'CUSTOM2' END) AS TAGS
FROM
MATERIAL
GROUP BY
FLDSTRING1, FLDFLOAT5, FLDFLOAT6;
FLDSTRING1
FLDFLOAT5
FLDFLOAT6
TAGS
NULL
NULL
NULL
NULL
NULL
1
CUSTOM2
NULL
0
NULL
NULL
1
NULL
CUSTOM1
NULL
1
1
CUSTOM1CUSTOM2
''
NULL
NULL
''
NULL
1
CUSTOM2
0
NULL
NULL
NEWPROD
0
0
NULL
NEWPROD
1
NULL
NULL
NEWPROD
1
NULL
1
NEWPRODCUSTOM2
1
0
NULL
NEWPROD
1
1
NULL
NEWPRODCUSTOM1
1
1
1
NEWPRODCUSTOM1CUSTOM2
194276400456
0
NULL
NEWPROD
Problem #1 is I don't quite understand how this works... I mean, value 0 isn't '', but still for the combination of NULL 0 NULL I get an empty value, which is what I wanted... But how does it do that?
And also, can someone update my final query to comma-separate the calculated TAGS column? Problem #2 is that I don't want it to contain just two commas, like ,,, when the combination wouldn't justify any of the three values to appear... It should work like PHP's implode() works...
To help you help me with this, I'm including a fiddle with the setup of the scenario I describe here... Thanks in advance!
Since you are using SQL Server 2014, instead of CONCAT_WS you may try STUFF as shown below. By prepending the delimiter , before all strings ,the STUFF will remove the first comma found.
SELECT
FLDSTRING1,
FLDFLOAT5,
FLDFLOAT6,
STUFF(
CONCAT(
CASE WHEN FLDSTRING1 IS NULL OR FLDSTRING1 IN ('0','') THEN '' THEN '' ELSE ',NEWPROD' END,
CASE WHEN FLDFLOAT5 IS NULL THEN '' ELSE ',CUSTOM1' END,
CASE WHEN FLDFLOAT6 IS NULL THEN '' ELSE ',CUSTOM2' END
),
1,1,''
) AS TAGS
FROM #MATERIAL
GROUP BY FLDSTRING1, FLDFLOAT5, FLDFLOAT6;
View working demo db fiddle
Let me know if this works for you.
Use the CONCAT_WS() function to concat values into a comma (or other separator) separated list, which ignores nulls.
To use CONCAT_WS(), you want to pass it a true NULL if the value is "blank" (by your definition), otherwise your custom label:
SELECT DISTINCT
FLDSTRING1,
FLDFLOAT5,
FLDFLOAT6,
CONCAT_WS(',',
CASE WHEN FLDSTRING1 IS NULL OR FLDSTRING1 = '' OR FLDSTRING1 = '0' THEN NULL ELSE 'NEWPROD' END,
CASE WHEN FLDFLOAT5 IS NULL OR FLDFLOAT5 = 0 THEN NULL ELSE 'CUSTOM1' END,
CASE WHEN FLDFLOAT6 IS NULL OR FLDFLOAT6 = 0 THEN NULL ELSE 'CUSTOM2' END) AS TAGS
FROM MATERIAL
Replaced GROUP BY with DISTINCT because it's simpler and (here) achieves the same thing.
If CONCAT_WS is not available:
SELECT DISTINCT
FLDSTRING1,
FLDFLOAT5,
FLDFLOAT6,
REPLACE(REPLACE(REPLACE(CONCAT(
CASE WHEN FLDSTRING1 IS NULL OR FLDSTRING1 = '' OR FLDSTRING1 = '0' THEN 'X' ELSE 'NEWPROD' END,
',',
CASE WHEN FLDFLOAT5 IS NULL OR FLDFLOAT5 = 0 THEN 'X' ELSE 'CUSTOM1' END,
',',
CASE WHEN FLDFLOAT6 IS NULL OR FLDFLOAT6 = 0 THEN 'X' ELSE 'CUSTOM2' END
), ',X', ''), 'X,', ''), 'X', '') AS TAGS
FROM MATERIAL
See dbfiddle.
I have a query where i used case when to KNOWN_AS column stating when its null then 'null' else 'not null' end as known_as2. Now in where clause i want to bring rows which only contain 'Not null' .
SELECT i.individual_ref, 0, 'KNOWNAS', i.FORENAMES, i.KNOWN_AS,
case when KNOWN_AS is null then 'Null' else ' Not null' end as known_as
FROM TestDatabase.dbo.INDIVIDUAL I
JOIN TestDatabase.dbo.MEMBER M ON M.INDIVIDUAL_REF=I.INDIVIDUAL_REF
WHERE m.member_status IN(33,1316)
AND i.KNOWN_AS IS null or i.KNOWN_AS=''
and m.MEMBER_STATUS in (33,1316)
and LEN(i.FORENAMES) > '1' and i.FORENAMES !=''
AND i.FORENAMES IS NOT NULL
Reason i'm want help is :-
I have a table which contain Forename, surname and known_as field.
I want to get members who's known_as field is blank/null and forename is not null or blank and forename length is >1 . How can i achieve it. the member status is from another table call member where i want member who are in active and pending status hence i said WHERE m.member_status IN (33,1316). Any solution please.
Finally I have solved it using
SELECT i.individual_ref,0,'KNOWNAS',
case when KNOWN_AS is null then 'Null' else ' Not null' end as knownas2
FROM TestDatabase.dbo.INDIVIDUAL I
JOIN TestDatabase.dbo.MEMBER M ON M.INDIVIDUAL_REF=I.INDIVIDUAL_REF
WHERE m.member_status IN(33,1316)
and len(i.forenames)>2 and
(IsNull(i.forenames, '') <> '') and (i.known_as is null or i.known_as='')
I had to take len(i.forenames, '')>2 instead of 1 because some members also has forename by mistakenly updated as Mr.
Now in where clause i want to bring rows which only contain 'Not null' .
Just modify your WHERE clause with IS NOT NULL as :
SELECT i.individual_ref, 0, 'KNOWNAS', i.FORENAMES, i.KNOWN_AS
FROM DiTestDatabase.dbo.INDIVIDUAL I JOIN
DiTestDatabase.dbo.MEMBER M
ON M.INDIVIDUAL_REF = I.INDIVIDUAL_REF
WHERE m.member_status IN (33,1316) AND
LEN(i.FORENAMES) > 1 AND i.FORENAMES != '' AND
i.FORENAMES IS NOT NULL AND i.KNOWN_AS IS NOT NULL;
Note :
LEN() will return INT type. So, you don't need to use ''.
Might be below query will help you.
SELECT i.individual_ref, 0, 'KNOWNAS', i.FORENAMES, i.KNOWN_AS
,CASE WHEN i.KNOWN_AS IS NULL THEN 'null'
ELSE 'Not null' END AS KNOWN_AS2
FROM DiTestDatabase.dbo.INDIVIDUAL I JOIN
DiTestDatabase.dbo.MEMBER M
ON M.INDIVIDUAL_REF = I.INDIVIDUAL_REF
WHERE m.member_status IN (33,1316) AND
LEN(i.FORENAMES) > 1 AND i.FORENAMES != '' AND
i.FORENAMES IS NOT NULL AND i.KNOWN_AS IS NOT NULL;
this way we can get solution
SELECT * FROM (
SELECT i.individual_ref, 0, 'KNOWNAS', i.FORENAMES, i.KNOWN_AS,
case when KNOWN_AS is null then 'Null' else 'Not null' end as known_as
FROM DiTestDatabase.dbo.INDIVIDUAL I
JOIN DiTestDatabase.dbo.MEMBER M ON M.INDIVIDUAL_REF=I.INDIVIDUAL_REF
WHERE m.member_status IN(33,1316)
AND i.KNOWN_AS IS null or i.KNOWN_AS=''
and m.MEMBER_STATUS in (33,1316)
and LEN(i.FORENAMES) > 1 and i.FORENAMES !=''
AND i.FORENAMES IS NOT NULL) t
WHERE t.known_as='Not null'
I receive raw data files from external sources and need to provide analysis on them. I load the files into a table & set the fields as varchars, then run a complex SQL script that does some automated analysis. One issue I've been trying to resolve is: How to tell if a column of data is duplicated with 1 or more other columns in that same table?
My goal is to have, for every column, a hash, checksum, or something similar that looks at a column's values in every row in the order they come in. I have dynamic SQL that loops through every field (different tables will have a variable number of columns) based on the fields listed in INFORMATION_SCHEMA.COLUMNS, so no concerns on how to accomplish that part.
I've been researching this all day but can't seem to find any sensible way to hash every row of a field. Google & StackOverflow searches return how to do various things to rows of data, but I couldn't find much on how to do the same thing vertically on a field.
So, I considered 2 possibilities & hit 2 roadblocks:
HASHBYTES - Use 'FOR XML PATH' (or similar) to grab every row & use a delimiter between each row, then use HASHBYTES to hash the long string. Unfortunately, this won't work for me since I'm running SQL Server 2014, and HASHBYTES is limited to an input of 8000 characters. (I can also imagine performance would be abysmal on tables with millions of rows, looped for 200+ columns).
CHECKSUM + CHECKSUM_AGG - Get the CHECKSUM of each value, turning it into an integer, then use CHECKSUM_AGG on the results (since CHECKSUM_AGG needs integers). This looks promising, but the order of the data is not considered, returning the same value on different rows. Plus the risk of collisions is higher.
The second looked promising but doesn't work as I had hoped...
declare #t1 table
(col_1 varchar(5)
, col_2 varchar(5)
, col_3 varchar(5));
insert into #t1
values ('ABC', 'ABC', 'ABC')
, ('ABC', 'ABC', 'BCD')
, ('BCD', 'BCD', NULL)
, (NULL, NULL, 'ABC');
select * from #t1;
select cs_1 = CHECKSUM(col_1)
, cs_2 = CHECKSUM(col_2)
, cs_3 = CHECKSUM(col_3)
from #t1;
select csa_1 = CHECKSUM_AGG(CHECKSUM([col_1]))
, csa_2 = CHECKSUM_AGG(CHECKSUM([col_2]))
, csa_3 = CHECKSUM_AGG(CHECKSUM([col_3]))
from #t1;
In the last result set, all 3 columns bring back the same value: 2147449198.
Desired results: My goal is to have some code where csa_1 and csa_2 bring back the same value, while csa_3 brings back a different value, indicating that it's its own unique set.
You could compare every column combo in this way, rather than using hashes:
select case when count(case when column1 = column2 then 1 else null end) = count(1) then 1 else 0 end Column1EqualsColumn2
, case when count(case when column1 = column3 then 1 else null end) = count(1) then 1 else 0 end Column1EqualsColumn3
, case when count(case when column1 = column4 then 1 else null end) = count(1) then 1 else 0 end Column1EqualsColumn4
, case when count(case when column1 = column5 then 1 else null end) = count(1) then 1 else 0 end Column1EqualsColumn5
, case when count(case when column2 = column3 then 1 else null end) = count(1) then 1 else 0 end Column2EqualsColumn3
, case when count(case when column2 = column4 then 1 else null end) = count(1) then 1 else 0 end Column2EqualsColumn4
, case when count(case when column2 = column5 then 1 else null end) = count(1) then 1 else 0 end Column2EqualsColumn5
, case when count(case when column3 = column4 then 1 else null end) = count(1) then 1 else 0 end Column3EqualsColumn4
, case when count(case when column3 = column5 then 1 else null end) = count(1) then 1 else 0 end Column3EqualsColumn5
, case when count(case when column4 = column5 then 1 else null end) = count(1) then 1 else 0 end Column4EqualsColumn5
from myData a
Here's the setup code:
create table myData
(
id integer not null identity(1,1)
, column1 nvarchar (32)
, column2 nvarchar (32)
, column3 nvarchar (32)
, column4 nvarchar (32)
, column5 nvarchar (32)
)
insert myData (column1, column2, column3, column4, column5)
values ('hello', 'hello', 'no', 'match', 'match')
,('world', 'world', 'world', 'world', 'world')
,('repeat', 'repeat', 'repeat', 'repeat', 'repeat')
,('me', 'me', 'me', 'me', 'me')
And here's the obligatory SQL Fiddle.
Also, to save you having to write this here's some code to generate the above. This version will also include logic to handle scenarios where both columns' values are null:
declare #tableName sysname = 'myData'
, #sql nvarchar(max)
;with cte as (
select name, row_number() over (order by column_id) r
from sys.columns
where object_id = object_id(#tableName, 'U') --filter on our table
and name not in ('id') --only process for the columns we're interested in
)
select #sql = coalesce(#sql + char(10) + ', ', 'select') + ' case when count(case when ' + quotename(a.name) + ' = ' + quotename(b.name) + ' or (' + quotename(a.name) + ' is null and ' + quotename(b.name) + ' is null) then 1 else null end) = count(1) then 1 else 0 end ' + quotename(a.name + '_' + b.name)
from cte a
inner join cte b
on b.r > a.r
order by a.r, b.r
set #sql = #sql + char(10) + 'from ' + quotename(#tableName)
print #sql
NB: That's not to say you should run it as dynamic SQL; rather you can use this to generate your code (unless you need to support the scenario where the number or name of columns may vary at runtime, in which case you'd obviously want the dynamic option).
NEW SOLUTION
EDIT: Based on some new information, namely that there may be more than 200 columns, my suggestion is to compute hashes for each column, but perform it in the ETL tool.
Essentially, feed your data buffer through a transformation that computes a cryptographic hash of the previously-computed hash concatenated with the current column value. When you reach the end of the stream, you will have serially-generated hash values for each column, that are a proxy for the content and order of each set.
Then, you can compare each to all of the others almost instantly, as opposed to running 20,000 table scans.
OLD SOLUTION
Try this. Basically, you'll need a query like this to analyze each column against the others. There is not really a feasible hash-based solution. Just compare each set by its insertion order (some sort of row sequence number). Either generate this number during ingestion, or project it during retrieval, if you have a computationally-feasible means of doing so.
NOTE: I took liberties with the NULL here, comparing it as an empty string.
declare #t1 table
(
rownum int identity(1,1)
, col_1 varchar(5)
, col_2 varchar(5)
, col_3 varchar(5));
insert into #t1
values ('ABC', 'ABC', 'ABC')
, ('ABC', 'ABC', 'BCD')
, ('BCD', 'BCD', NULL)
, (NULL, NULL, 'ABC');
with col_1_sets as
(
select
t1.rownum as col_1_rownum
, CASE WHEN t2.rownum IS NULL THEN 1 ELSE 0 END AS col_2_miss
, CASE WHEN t3.rownum IS NULL THEN 1 ELSE 0 END AS col_3_miss
from
#t1 as t1
left join #t1 as t2 on
t1.rownum = t2.rownum
AND isnull(t1.col_1, '') = isnull(t2.col_2, '')
left join #t1 as t3 on
t1.rownum = t3.rownum
AND isnull(t1.col_1, '') = isnull(t2.col_3, '')
),
col_1_misses as
(
select
SUM(col_2_miss) as col_2_misses
, SUM(col_3_miss) as col_3_misses
from
col_1_sets
)
select
'col_1' as column_name
, CASE WHEN col_2_misses = 0 THEN 1 ELSE 0 END AS is_col_2_match
, CASE WHEN col_3_misses = 0 THEN 1 ELSE 0 END AS is_col_3_match
from
col_1_misses
Results:
+-------------+----------------+----------------+
| column_name | is_col_2_match | is_col_3_match |
+-------------+----------------+----------------+
| col_1 | 1 | 0 |
+-------------+----------------+----------------+
I am trying to write a SQL Select statement to return records based on a user input through a front end.
I want to write the Select statement like this:
SELECT somefields
FROM sometable
WHERE CASE variable
WHEN 'blank' THEN field IS NULL
ELSE field = field
END
Basically I either want to filter a column to find NULL values or ignore the filter and return all values depending on the value of the variable. I know that the results of the CASE statement is not executable but how can I do this?
When variable is 'blank', the following query will give you rows where field is NULL. When variable is anything else, it will give you all rows:
SELECT somefields
FROM sometable
WHERE
(variable = 'blank' AND field IS NULL)
OR (variable <> 'blank')
You can use NULLIF() (link is for SQL Server, but NULLIF() should be standard):
SELECT somefields
FROM sometable
WHERE field = NULLIF(variable, 'blank')
The following snippet should behave as follows:
when #variable is null, return all rows
when #variable = 'blank', return all rows where field is null or field = 'blank'
otherwise, return rows where #variable equals field
Code snippet:
WHERE 1 = CASE
WHEN #variable is null then 1
WHEN #variable = 'blank' and field is null then 1
WHEN #variable = field then 1
END
SELECT somefields
FROM sometable
WHERE ((variable IS NULL OR variable = 0) OR (variable = field))
WHERE Criteria is apply when variable have value
For Example:
DECLARE #CityName VARCHAR(50)
SET #CityName = NULL
SELECT CityName
FROM City
WHERE ((#CityName IS NULL ) OR (#CityName = CityName ))
When City is null then tables return all rows
I think I get what you're after. Something like this maybe?
SELECT field1,
field2,
CASE variable
WHEN 'blank' THEN NULL
ELSE field3
END as field3
FROM sometable
Think I understand what you mean....for example....
SELECT
House, Postcode
from
SomeTable
where
(House=isnull(#House,House) or (House is null and #House is null))
and
(Postcode=isnull(#Postcode,Postcode) or (Postcode is null and #Postcode is null))
First bit of the conditional where is to use the variable, when present (the isnull bit is to ignore the variable if it's null)
Second bit of the conditional where is in case your evaluative field is null also as effectively fields don't = null they are 'is null'.
Confused? Good. Works on what I'm doing though!
Here is my solution based on #Andomar answer above aimed at anyone testing an input varchar value, you need to test the parameter in the right order as per the example below:
FIELD1 = CASE
WHEN #inputParameter = '' THEN FIELD1
WHEN #inputParameter <> FIELD1 THEN NULL -- if input is some text but does not match
WHEN #inputParameter IS NULL THEN FIELD1
WHEN #inputParameter != '' AND FIELD1 = #inputParameter THEN FIELD1
END
Hope this helps someone.
I have a procedure which receive a bit variable called #FL_FINALIZADA.
If it is null or false I want to restrict my select to show only the rows that contain null DT_FINALIZACAO values. Otherwise I want to show the rows containing not null DT_FINALIZACAO values.
Something like this:
SELECT
*
FROM
MyTable
WHERE
...
AND
(
OPE.DT_FINALIZACAO = (
CASE
WHEN (#FL_FINALIZADA <> 1)
THEN NULL
END
) OR
OPE.DT_FINALIZACAO IS NOT NULL
)
In this case I receive the message:
None of the result expressions in a
CASE specification can be NULL.
How can I achieve this?
Thanks in advance.
SELECT
*
FROM
MyTable
WHERE
(ISNULL(#FL_FINALIZADA, 0) = 0
AND
OPE.DT_FINALIZACAO IS NULL
)
OR
(#FL_FINALIZADA = 1
AND
OPE.DT_FINALIZACAO IS NOT NULL
)
Change the AND to be:
AND (((#FL_FINALIZADA <> 1) AND (OPE.DT_FINALIZACAO IS NULL)) OR ( (#FL_FINALIZADA = 1) AND (OPE.DT_FINALIZACAO IS NOT NULL)))
If the bit flag is 1 then DT_FINALIZACAO can't be null.
IF #FL_FINALIZADA IS NULL
SET #FL_FINALIZADA = 0
SELECT * FROM NewsletterSubscribers
WHERE
(#FL_FINALIZADA = 0 AND OPE.DT_FINALIZACAO IS NULL)
OR
(#FL_FINALIZADA = 1 AND OPE.DT_FINALIZACAO IS NOT NULL)
My detailed SQL is a little rusty, but have you tried using 0 insted of NULL? I would expect 0 to evaluate the same as NULL in that select