Transactions column's names in below code are dynamicaly generated (so it means that sometimes particular name/column doesn't exist). Using this select it finishes successfully only in case when every of those names exists, if not, I got error like this (example):
Error(s), warning(s): 42703: column "TransactionA" does not exist
SELECT
*,
((CASE WHEN "TransactionA" IS NULL THEN 0 ELSE "TransactionA" END) -
(CASE WHEN "TransactionB" IS NULL THEN 0 ELSE "TransactionB" END) +
(CASE WHEN "TransactionC" IS NULL THEN 0 ELSE "TransactionC" END)) AS "Account_balance"
FROM Summary ORDER BY id;
Could you tell me please how can I check first if the column exists and then how can I nest another CASE statement or other condition statement to make it working in a correct way?
You can build any query dynamically with information from the Postgres catalog tables. pg_attribute in your case. Alternatively, use the information schema. See:
Query to return output column names and data types of a query, table or view
How to check if a table exists in a given schema
Basic query to see which of the given columns exist in a given table:
SELECT attname
FROM pg_attribute a
WHERE attrelid = 'public.summary'::regclass -- tbl here
AND NOT attisdropped
AND attnum > 0
AND attname IN ('TransactionA', 'TransactionB', 'TransactionC'); -- columns here
Building on this, you can have Postgres generate your whole query. While being at it, look up whether columns are defined NOT NULL, in which case they don't need COALESCE:
CREATE OR REPLACE FUNCTION f_build_query(_tbl regclass, _columns json)
RETURNS text AS
$func$
DECLARE
_expr text;
BEGIN
SELECT INTO _expr
string_agg (op || CASE WHEN attnotnull
THEN quote_ident(attname)
ELSE format('COALESCE(%I, 0)', attname) END
, '')
FROM (
SELECT j->>'name' AS attname
, CASE WHEN j->>'op' = '-' THEN ' - ' ELSE ' + ' END AS op
FROM json_array_elements(_columns) j
) j
JOIN pg_attribute a USING (attname)
WHERE attrelid = _tbl
AND NOT attisdropped
AND attnum > 0;
IF NOT FOUND THEN
RAISE EXCEPTION 'No column found!'; -- or more info
END IF;
RETURN
'SELECT *,' || _expr || ' AS "Account_balance"
FROM ' || _tbl || '
ORDER BY id;';
END
$func$ LANGUAGE plpgsql;
The table itself is parameterized, too. May or may not be useful for you. The only assumption is that every table has an id column for the ORDER BY. Related:
Table name as a PostgreSQL function parameter
I pass columns names and the associated operator as JSON document for flexibility. Only + or - are expected as operator. Input is safely concatenated to make SQL injection impossible.About json_array_elements():
Query for element of array in JSON column
Example call:
SELECT f_build_query('summary', '[{"name":"TransactionA"}
, {"name":"TransactionB", "op": "-"}
, {"name":"TransactionC"}]');
Returns the according valid query string, like:
SELECT *, + COALESCE("TransactionA", 0) - COALESCE("TransactionB", 0) AS "Account_balance"
FROM summary
ORDER BY id;
"TransactionC" isn't there in this case. If both existing columns happen to be NOT NULL, you get instead:
SELECT *, + "TransactionA" - "TransactionB" AS "Account_balance"
FROM summary
ORDER BY id;
db<>fiddle here
You could execute the generated query in the function immediately and return result rows directly. But that's hard as your return type is a combination of a table rows (unknown until execution time?) plus additional column, and SQL demands to know the return type in advance. For just id and sum (stable return type), it would be easy ...
It's odd that your CaMeL-case column names are double-quoted, but the CaMeL-case table name is not. By mistake? See:
Are PostgreSQL column names case-sensitive?
How to pass column names containing single quotes?
Addressing additional question from comment.
If someone used column names containing single quotes by mistake:
CREATE TABLE madness (
id int PRIMARY KEY
, "'TransactionA'" numeric NOT NULL -- you wouldn't do that ...
, "'TransactionC'" numeric NOT NULL
);
For the above function, the JSON value is passed as quoted string literal. If that string is enclosed in single-quotes, escape contained single-quotes by doubling them up. This is required on top of valid JSON format:
SELECT f_build_query('madness', '[{"name":"''TransactionA''"}
, {"name":"TransactionB", "op": "-"}
, {"name":"TransactionC"}]'); --
("''TransactionA''" finds a match, "TransactionC" does not.)
Or use dollar quoting instead:
SELECT f_build_query('madness', $$[{"name":"'TransactionA'"}
, {"name":"TransactionB", "op": "-"}
, {"name":"TransactionC"}]$$);
db<>fiddle here with added examples
See:
Insert text with single quotes in PostgreSQL
Assuming that id is a unique id in summary, then you can use the following trick:
SELECT s.*,
(COALESCE("TransactionA", 0) -
COALESCE("TransactionB", 0) +
COALESCE("TransactionC", 0)
) AS Account_balance
FROM (SELECT id, . . . -- All columns except the TransactionX columns
FROM (SELECT s.*,
(SELECT TransactionA FROM summary s2 WHERE s2.id = s.id) as TransactionA,
(SELECT TransactionB FROM summary s2 WHERE s2.id = s.id) as TransactionB,
(SELECT TransactionC FROM summary s2 WHERE s2.id = s.id) as TransactionC
FROM Summary s
) s CROSS JOIN
(VALUES (NULL, NULL, NULL)) v(TransactionA, TransactionB, TransactionC)
) s
ORDER BY s.id;
The trick here is that the correlated subqueries do not qualify the TransactionA. If the value is defined for summary, then that will be used. If not, it will come from the values() clause in the outer query.
This is a bit of a hack, but it can be handy under certain circumstances.
Check this example:
UPDATE yourtable1
SET yourcolumn = (
CASE
WHEN setting.value IS NOT NULL
THEN CASE WHEN replace(setting.value,'"','') <> '' THEN replace(setting.value,'"','') ELSE NULL END
ELSE NULL
END
)::TIME FROM (SELECT value FROM yourtable2 WHERE key = 'ABC') AS setting;
Related
I am working on a sql database which will provide with data some grid. The grid will enable filtering, sorting and paging but also there is a strict requirement that users can enter free text to a text input above the grid for example
'Engine 1001 Requi' and that the result will contain only rows which in some columns contain all the pieces of the text. So one column may contain Engine, other column may contain 1001 and some other will contain Requi.
I created a technical column (let's call it myTechnicalColumn) in the table (let's call it myTable) which will be updated each time someone inserts or updates a row and it will contain all the values of all the columns combined and separated with space.
Now to use it with entity framework I decided to use a table valued function which accepts one parameter #searchQuery and it will handle it like this:
CREATE FUNCTION myFunctionName(#searchText NVARCHAR(MAX))
RETURNS #Result TABLE
( ... here come columns )
AS
BEGIN
DECLARE #searchToken TokenType
INSERT INTO #searchToken(token) SELECT value FROM STRING_SPLIT(#searchText,' ')
DECLARE #searchTextLength INT
SET #searchTextLength = (SELECT COUNT(*) FROM #searchToken)
INSERT INTO #Result
SELECT
... here come columns
FROM myTable
WHERE (SELECT COUNT(*) FROM #searchToken WHERE CHARINDEX(token, myTechnicalColumn) > 0) = #searchTextLength
RETURN;
END
Of course the solution works fine but it's kinda slow. Any hints how to improve its efficiency?
You can use an inline Table Valued Function, which should be quite a lot faster.
This would be a direct translation of your current code
CREATE FUNCTION myFunctionName(#searchText NVARCHAR(MAX))
RETURNS TABLE
AS RETURN
(
WITH searchText AS (
SELECT value token
FROM STRING_SPLIT(#searchText,' ') s(token)
)
SELECT
... here come columns
FROM myTable t
WHERE (
SELECT COUNT(*)
FROM searchText
WHERE CHARINDEX(s.token, t.myTechnicalColumn) > 0
) = (SELECT COUNT(*) FROM searchText)
);
GO
You are using a form of query called Relational Division Without Remainder and there are other ways to cut this cake:
CREATE FUNCTION myFunctionName(#searchText NVARCHAR(MAX))
RETURNS TABLE
AS RETURN
(
WITH searchText AS (
SELECT value token
FROM STRING_SPLIT(#searchText,' ') s(token)
)
SELECT
... here come columns
FROM myTable t
WHERE NOT EXISTS (
SELECT 1
FROM searchText
WHERE CHARINDEX(s.token, t.myTechnicalColumn) = 0
)
);
GO
This may be faster or slower depending on a number of factors, you need to test.
Since there is no data to test, i am not sure if the following will solve your issue:
-- Replace the last INSERT portion
INSERT INTO #Result
SELECT
... here come columns
FROM myTable T
JOIN #searchToken S ON CHARINDEX(S.token, T.myTechnicalColumn) > 0
I'm trying to convert an INT but it is having an issue with the conversion.
Conversion failed when converting the nvarchar value '245428,246425' to data type int.
The query I am using:
SELECT STUFF
(
(
SELECT DISTINCT ',' + CONVERT(VARCHAR(20), NumField)
FROM Table A
WHERE ID = 218554
FOR XML PATH('')
) ,1,1,''
)
I use this as a subquery in a larger table like so:
SELECT
Field1,
Field2,
CASE WHEN criteria = '1'
THEN (SELECT STUFF(
(
SELECT DISTINCT ',' + CONVERT(VARCHAR(20), NumField)
FROM Table A
WHERE ID = 218554
FOR XML PATH('')
) ,1,1,''
))
END
FROM
Table B
The STUFF query runs fine when it's executed on it's own but when I run it in the full query it comes up with the conversion error.
I don't think you are not showing the full query -- or at least the full case expression. A case expression returns a single value with a single type.
When there are type conflicts, then SQL Server has to determine the single overall type, according to its rules. If one then returns an integer and another returns a string, then the case expression is an integer (not a string). So, the string is converted to an integer.
You can see this problem with much simpler logic:
select (case when 1=1 then 'a' else 0 end)
Even though the else is never execution, the type of the expression is determined at compile time -- and 'a' cannot be converted to an integer.
I have a pretty simple Stored Procedure that I am in trouble to do because i'm new to SQL and PL/SQL. I Have a table with a name column that is a varchar(55).
I discovered that if the user executes my procedure with an empty string as a paramter the LIKE statment brings all rows from TABLE1
SELECT *
FROM TABLE1
WHERE COLUMN LIKE VARIABLE || '%'
AND...
So I tried to change the query so if the VARIABLE is passed with a empty string it can still perform other conditions in the where statment.
SELECT *
FROM TABLE1
WHERE (VARIABLE <> '' AND COLUMN LIKE VARIABLE || '%')
AND...
But now wherever I pass as variable ('', NULL, 'anystring') I get no rows returned.
How can I build a query that validates if the variable is different of empty string and if it is it performs the LIKE statment with the variable correctly?
If I understand you correctly, it is not difficult thing to do. You can use conditional WHERE clause using CASE WHEN. So your query will support different scenarios, something like this:
SELECT *
FROM TABLE1
WHERE (CASE WHEN variable IS NULL AND column IS NULL THEN 1
WHEN variable LIKE '%' AND column LIKE variable||'%' THEN 1
ELSE 0
END) = 1
AND...
Basically, it checks if the variable = '' then it will compare the column against ''. Otherwise, it will compare it against variable||'%'.
Notice, Oracle treats empty string of the type VARCHAR as NULL (this does not apply to CHAR). So, in the first scenario we compare against NULL.
Hello Just a thought for this we can use Dynamic sql too. If you may try this approach. Hope it helps.
CREATE OR REPLACE PROCEDURE SPS_TEST_OUT(
p_input_in IN VARCHAR2
)
AS
lv_sql LONG;
lv_where VARCHAR2(100);
BEGIN
lv_where:= CASE WHEN p_input_in IS NULL OR p_input_in = '' THEN
''
ELSE
' AND COLUMN1 LIKE '''||p_input_in||'''%'
END;
lv_sql:='SELECT * FROM TABLE
WHERE 1 = 1
' ||lv_where;
dbms_output.put_line(lv_sql);
END;
The way these rows usually come into the target table the first time are with a sparse number of columns populated with mostly text data with the remainder of the columns set to NULL. On subsequent passes, the fresh data populates existing known (non null) and unknown (NULL) data. I've ascertained that the fresh data ( #pld) do indeed contain different data. The data does not appear to change. Here's what I have:
BEGIN TRANSACTION
BEGIN TRY
MERGE INTO [metro].listings AS metroList
USING #pld as listnew
ON metroList.id = listnew.id
AND metroList.sid = listnew.sid
WHEN MATCHED AND (
metroList.User != listnew.User
or metroList.Email != listnew.Email
or metroList.LocName != listnew.LocName
) THEN
UPDATE SET
metroList.User = listnew.User,
metroList.Email = listnew.Email,
metroList.LocName = listnew.LocName,
WHEN NOT MATCHED THEN
INSERT
( User,
Email,
LocName
)
VALUES
(
listnew.User,
listnew.Email,
listnew.LocName
);
COMMIT TRANSACTION
END TRY
IF ##TRANCOUNT > 0
ROLLBACK TRANSACTION;
END CATCH
I've tried replacing the != to under the update portion of the statement with <> . Same results. This has to be related to a comparison of a possible (likely) null value against a string--maybe even another null? Anyway, I'm calling on all sql-geeks to untangle this.
Also you can use option with NULLIF() function.
NULLIF returns the first expression if the two expressions are not equal. If the expressions are equal, NULLIF returns a null value of the type of the first expression.
WHEN MATCHED AND (
NULLIF(ISNULL(metroList.[User],''), listnew.[User]) IS NOT NULL
OR NULLIF(ISNULL(metroList.Email, ''), listnew.Email) IS NOT NULL
OR NULLIF(ISNULL(metroList.LocName, ''), listnew.LocName) IS NOT NULL
)
THEN
Comparing NULL with an empty string will not work.
If either side could be NULL, you could do something like:
WHEN MATCHED AND (
COALESCE(metroList.User, '') <> COALESCE(listnew.User, '')
or COALESCE(metroList.Email, '') <> COALESCE(listnew.Email, '')
or COALESCE(metroList.LocName, '') <> COALESCE(listnew.LocName, '')
) THEN
Of course, this assumes that you're fine with NULL meaning the same as an empty string (which may not be appropriate).
Take a look at this BOL article on NULL comparisons.
As I understand the question you are looking for an expression that emulates IS DISTINCT FROM.
The answer you have accepted is not correct then
WITH metroList([User])
AS (SELECT CAST(NULL AS VARCHAR(10))),
listnew([User])
AS (SELECT 'Foo')
SELECT *
FROM metroList
JOIN listnew
ON NULLIF(metroList.[User], listnew.[User]) IS NOT NULL
Returns zero rows. Despite the values under comparison being NULL and Foo.
I would use the technique from this article: Undocumented Query Plans: Equality Comparisons
WHEN MATCHED AND EXISTS (
SELECT metroList.[User], metroList.Email,metroList.LocName
EXCEPT
SELECT listnew.[User], listnew.Email,listnew.LocName
)
I currently have a prepared statement in Java which uses the following SQL statement in the WHERE clause of my query, but I would like to re-write this into a function to limit the user parameters passed to it and possibly make it more efficient.
(
(USER_PARAM2 IS NULL AND
( COLUMN_NAME = nvl(USER_PARAM1, COLUMN_NAME) OR
(nvl(USER_PARAM1, COLUMN_NAME) IS NULL)
)
)
OR
(USER_PARAM2 IS NOT NULL AND COLUMN_NAME IS NULL)
)
USER_PARAM1 and USER_PARAM2 are passed into the prepared statement by the user.
USER_PARAM1 represents what the application user wants to search this particular COLUMN_NAME for. If the user does not include this parameter, it will default to NULL.
USER_PARAM2 was my way to allow a user to request a NULL value only search on this COLUMN_NAME. Additionally I have some server logic that sets USER_PARAM2 to 'true' if passed in by the user or NULL if it wasn't specified by the user.
The intended behavior is that if USER_PARAM2 was declared then only COLUMN_NAME values of NULL are returned. If USER_PARAM2 wasn't declared and USER_PARAM1 was declared then only COLUMN_NAME = USER_PARAM1 are returned. If neither user params are declared then all rows are returned.
Could anyone help me out on this?
Thanks in advance...
EDIT:
Just to clarify this is how my current query looks (without the other WHERE clause statements..)
SELECT *
FROM TABLE_NAME
WHERE (
(USER_PARAM2 IS NULL AND
( COLUMN_NAME = nvl(USER_PARAM1, COLUMN_NAME) OR
(nvl(USER_PARAM1, COLUMN_NAME) IS NULL)
)
)
OR
(USER_PARAM2 IS NOT NULL AND COLUMN_NAME IS NULL)
)
... and this is where I would like to get to...
SELECT *
FROM TABLE_NAME
WHERE customSearchFunction(USER_PARAM1, USER_PARAM2, COLUMN_NAME)
EDIT #2:
OK, so another co-worker helped me out with this...
CREATE OR REPLACE function searchNumber (pVal IN NUMBER, onlySearchForNull IN CHAR, column_value IN NUMBER)
RETURN NUMBER
IS
BEGIN
IF onlySearchForNull IS NULL THEN
IF pVal IS NULL THEN
RETURN 1;
ELSE
IF pVal = column_value THEN
RETURN 1;
ELSE
RETURN 0;
END IF;
END IF;
ELSE
IF column_value IS NULL THEN
RETURN 1;
ELSE
RETURN 0;
END IF;
END IF;
END;
... this seems to work in my initial trials..
SELECT *
FROM TABLE_NAME
WHERE 1=searchNumber(USER_PARAM1, USER_PARAM2, COLUMN_NAME);
... the only issues I have with it would be
1)possible performance concerns vs the complex SQL statement I started with.
2)that I would have to create similar functions for each data type.
However, the latter would be less of an issue for me.
EDIT #3 2012.02.01
So we ended up going with the solution I chose below, while using the function based approach where code/query cleanliness trumps performance. We found that the function based approach performed roughly 6x worse than using pure SQL.
Thanks everyone for the great input everyone!
EDIT #4 2012.02.14
So looking back I noticed that applying the virtual table concept in #Alan's solution with the clarity of #danihp's solution gives a very nice overall solution in terms of clarity and performance. Here's what I now have
WITH params AS (SELECT user_param1 AS param, user_param2 AS param_nullsOnly FROM DUAL)
SELECT *
FROM table_name, params p
WHERE ( nvl(p.param_nullsOnly, p.param) IS NULL --1)
OR p.param_nullsOnly IS NOT NULL AND column_name IS NULL --2)
OR p.param IS NOT NULL AND column_name = p.param --3)
)
-- 1) Test if all rows should be returned
-- 2) Test if only NULL values should be returned
-- 3) Test if param equals the column value
Thanks again for the suggestions and comments!
There's a simple way of to pass your parameters only once and refer to them as many times as needed, using common-table expressions:
WITH params AS (SELECT user_param1 AS up1, user_param2 AS up2 FROM DUAL)
SELECT *
FROM table_name, params p
WHERE ((p.up2 IS NULL
AND (column_name = NVL(p.up1, column_name)
OR (NVL(p.up1, column_name) IS NULL)))
OR (p.up2 IS NOT NULL AND column_name IS NULL))
In effect, you're creating a virtual table, where the columns are your parameters, that is populated with a single row.
Conveniently, this also ensures that all of your parameters are collected in the same place and can be specified in an arbitrary order (as opposed to the order that the naturally appear in the query).
There are a couple big advantages to this over a function-based approach. First, this will not prevent the use of indexes (as pointed out by #Bob Jarvis). Second, this keeps the query's logic in the query, rather than hidden in functions.
I don't know if my approach has more performance, but it has best readability:
Sending 2 additionals parameters to query you can rewrite query like:
where
( P_ALL_RESULTS is not null
OR
P_ONLY_NULLS is not null AND COLUMN_NAME IS NULL
OR
P_USE_P1 is not null AND COLUMN_NAME = USER_PARAM1
)
Disclaimer: answered before OP question clarification