Remove NULL values and corresponding column names from SQL Insert statements - sql

I very frequently have to deal with sql written by others.
A typical sql insert statements can hold 5-6 uneeded values (NULL).
An example:
INSERT INTO Texts (Text_EN, Text_NO, Text_DK, Text_SV) VALUES ('English', NULL, NULL, 'Svenska')
I've been looking for a way to transform a line like this into:
INSERT INTO Texts (Text_EN, Text_SV) VALUES ('English', 'Svenska')
So what I want to accomplish is:
Find the NULL values
Find the columns that correspond to those values
Remove those values and the column names
I've been trying various regexes with positive lookbehinds, since I think I'd need to find the NULLs first, but no dice. Perhaps there's another way already?
I've been checking out sql formatters as well, but they doesn't seem to have this very specific functionality.

You can do it by using dynamic SQL. For example, write function for 2 input array type parameters. One of the arrays is for fields and another array for values. Inside the function, you can declare 2 string-type variables, for dynamic SQLs, and use one for-loop operation. In this loop check values by the if operation. If the value is null then don't add to string values and fields, if-else add. After the loop is set to your main string variable 'insert into table' then execute dynamic SQL.
Shortly example:
declare
v_sql varchar(1000);
v_fields varchar(1000);
v_valuesm varchar(1000);
begin
-- for-loop statement
-- if check generate striong variables for fiedls and values
-- end of for-loop statement
v_sql = 'INSERT INTO Texts (' + v_fields + ') VALUES (' + v_values + ')';
-- executing your v_sql

Related

How do you pass values for a parameter by position when you need to check multiple values?

I created a stored procedure (spBalanceRange) with 2 optional parameters. They've been set to a default value and the sp works fine when I pass only 1 value per parameter by position. However, I have a situation where I'm trying to pass, by position, two strings immediately followed by a wildcard. I want the user to be able to search for Vendor names that start with either 'C%' or 'F%'. Here's the gist of the CREATE PROC statement:
CREATE PROC spBalanceRange
#VendorVar varchar(40) = '%',
#BalanceMin money = 1.0
...
Here's what I've tried so far, but doesn't work:
EXEC spBalanceRange '(C%|F%)', 200.00;
EXEC spBalanceRange 'C%|F%', 200.00;
Is there a way to check for 2 or more string values with a wildcard when passed by position? Thanks.
EDIT: According to your comments you are looking for the first letter of a vendor's name only.
In this special case I could suggest an easy, not well performing but really simple approach. CHARINDEX returns a number greater than zero, if a character appears within a string. So you just have to pass in all your lookup-first-characters as a simple "chain":
DECLARE #DummyVendors TABLE(VendorName VARCHAR(100));
INSERT INTO #DummyVendors VALUES
('Camel Industries')
,('Fritz and Fox')
,('some other');
DECLARE #ListOfFirstLetters VARCHAR(100)='CF';
SELECT VendorName
FROM #DummyVendors AS dv
WHERE CHARINDEX(LEFT(dv.VendorName,1),#ListOfFirstLetters)>0
This was the former answer
Checking against more than one value needs either a dedicated list of compares
WHERE val=#prm1 OR val=#prm2 OR ... (you know the count before)
...or you use the IN-clause
WHERE LEFT(VenoderName,1) IN ('C','F', ...)
...but you cannot pass the IN-list with a parameter like ... IN(#allValues)
You might think about a created TYPE to pass in all your values like a table and use an INNER JOIN as filter: https://stackoverflow.com/a/337864/5089204 (and a lot of other examples there...)
Or you might think of dynamic SQL: https://stackoverflow.com/a/5192765/5089204
And last but not least you might think of one of the many split string approaches. This is one of my own answers, section "dynamic IN-statement": https://stackoverflow.com/a/33658220/5089204
I'm answering my own question, and maybe other solutions exist but here is what had to happen with my stored procedure in order to pass variables by position:
CREATE PROC spBalanceRange
#VendorVar varchar(40) = '%',
#BalanceMin money = 1.0
AS
IF (#VendorVar = '%' AND #BalanceMin IS NULL OR #BalanceMin = '')
BEGIN
PRINT 'BalanceMin cannot be null.';
END
IF (#VendorVar = % AND #BalanceMin IS NOT NULL)
BEGIN
(sql statement using parameters)
END
EXEC spBalanceRange '[C,F]%', 200.00;
That's what I know.

SELECT #local_variable=values from table where values can have multiple values

I am using below query
SELECT #local_variable=Adtid from table where Adtid can have multiple values stored into it. As I don't know what to use instead of '=' in #local_variable=Adtid. Can anyone suggest please what I should use instead of '=' so that my local varaible can have all values of Adtid
The variable can't actually hold multiple values. You can declare a table variable instead, where you can then do something like
declare #tableVariable table
(
Adtid int
);
insert into #tableVariable
select Adtid from table where Adtid ...
This puts the relevant rows inside your table variable. Now you can use the table variable to eg. create a cursor (to go row by row in the data - you can also do that on the original select if you only need to go through once) or use it in a join clause.
You can use below syntax for get the Adtid as comma separator in result
DECLARE #local_variable VARCHAR(100) =''
SELECT #local_variable += CAST(Adtid AS VARCHAR) + ',' FROM TABLE
PRINT #local_variable
and another way as per below answer.

How to concatenate variables into SQL strings

I need to concatenate a variable table name into my SQL query such as the following...
ALTER FUNCTION fn_myfunction(#KeyValue text)
BEGIN
INSERT INTO #tmpTbl1
SELECT #KeyValue AS fld1
FROM tbl + #KeyValue + KeyValue.fld1
I also attempted the following but it told me I had to declare the table variable?
ALTER FUNCTION fn_myfunction(#KeyValue text, #KeyTable text)
FROM #KeyTable.fld1
You can accomplish this (if I understand what you are trying to do) using dynamic SQL.
The trick is that you need to create a string containing the SQL statement. That's because the tablename has to specified in the actual SQL text, when you execute the statement. The table references and column references can't be supplied as parameters, those have to appear in the SQL text.
So you can use something like this approach:
SET #stmt = 'INSERT INTO #tmpTbl1 SELECT ' + #KeyValue
+ ' AS fld1 FROM tbl' + #KeyValue
EXEC (#stmt)
First, we create a SQL statement as a string. Given a #KeyValue of 'Foo', that would create a string containing:
'INSERT INTO #tmpTbl1 SELECT Foo AS fld1 FROM tblFoo'
At this point, it's just a string. But we can execute the contents of the string, as a dynamic SQL statement, using EXECUTE (or EXEC for short).
The old-school sp_executesql procedure is an alternative to EXEC, another way to execute dymamic SQL, which also allows you to pass parameters, rather than specifying all values as literals in the text of the statement.
FOLLOWUP
EBarr points out (correctly and importantly) that this approach is susceptible to SQL Injection.
Consider what would happen if #KeyValue contained the string:
'1 AS foo; DROP TABLE students; -- '
The string we would produce as a SQL statement would be:
'INSERT INTO #tmpTbl1 SELECT 1 AS foo; DROP TABLE students; -- AS fld1 ...'
When we EXECUTE that string as a SQL statement:
INSERT INTO #tmpTbl1 SELECT 1 AS foo;
DROP TABLE students;
-- AS fld1 FROM tbl1 AS foo; DROP ...
And it's not just a DROP TABLE that could be injected. Any SQL could be injected, and it might be much more subtle and even more nefarious. (The first attacks can be attempts to retreive information about tables and columns, followed by attempts to retrieve data (email addresses, account numbers, etc.)
One way to address this vulnerability is to validate the contents of #KeyValue, say it should contain only alphabetic and numeric characters (e.g. check for any characters not in those ranges using LIKE '%[^A-Za-z0-9]%'. If an illegal character is found, then reject the value, and exit without executing any SQL.
You could make use of Prepared Stements like this.
set #query = concat( "select name from " );
set #query = concat( "table_name"," [where condition] " );
prepare stmt from #like_q;
execute stmt;

Create INSERT statement using parameter

I need to create a INSERT statement using parameters. Say I have two variable name #DestinationFields, #InsertValues.
Here #DestinationFields contain the column name like: product,price and #InsertValues contains the values for those two columns, like: Book,100.
Now, How i create a insert command to insert those values where each value need to add a quotation mark .I already tried as
I already tried as
EXEC('INSERT into tbl_test('+#DestinationFields+')values('+#InsertValues+')')
But it's returning an error.
The name "book" is not permitted in this context. Valid expressions are constants, constant expressions, and (in some
contexts) variables. Column names are not permitted.
How do I do it? Thanks in advance.
Pretending there is no problem of SQL injection here*, you can quickly fix your code by adding quotation marks around Book. The value of # InsertValues should be
'Book', 100
instead of simply
Book, 100
You need to add quotation marks around each string value; otherwise, strings are interpreted as names, which is not valid.
EDIT : (in response to a comment) If all columns are of varchar type, you can put quotes around the entire string, and replace all commas with the quote-comma-quote pattern, like this:
values('''+REPLACE(#InsertValues,',',''',''')+''')'
* You should not put code like this into production, because it can be manipulated to harm your system rather severely. Here is a good illustration of the problem (link).
Try:
DECLARE #DestinationFields VARCHAR(200);
SET #DestinationFields = 'Col1, Col2, Col3'
DECLARE #InsertValues VARCHAR(200);
SET #InsertValues = '1, 2, 3'
DECLARE #SQLString VARCHAR(1000);
SET #SQLString = 'INSERT INTO tbl_test (' + #DestinationFields + ') VALUES (' + #InsertValues + ')';
EXEC (#SQLString)
However, this is very open to SQL Injection attacks. But, it will do what you require.
The Curse and Blessing of Dynamic SQL

PL/SQL varchar(10) to varchar(9)

How would I write a loop that has a select in the “in” clause of the loop that selects a col of type varchar(10), but then inserts those values in a col that wants them to be varchar(9)? Basically I’m trying to “typecast” from one precision to another, if that makes any sense. Example:
FOR V_TEN IN (SELECT THIS_IS_VARCHAR_TEN FROM TABLE WHERE SOMETHING=’VALUE’)
LOOP
INSERT INTO OTHER_TABLE
(THIS_IS_VARCHAR_NINE)
VALUES
(V_TEN);
END LOOP;
The error is that the column types aren’t the same. I’ve tried looking at to_char() and cast() but neither seem to be what I want. I realize there is a loss of precision here and am okay with that, since I actually know that the values in the varchar(10) column are always going to be 9 chars.
You are looking for the SUBSTR function.
Also, do not use PL/SQL for this, plain SQL will do and be faster.
INSERT INTO OTHER_TABLE
SELECT OTHER_COLUMN, SUBSTR(THIS_IS_VARCHAR_TEN,1,9)
FROM TABLE WHERE SOMETHING=’VALUE’;
And if there are really no values longer than nine character, you do not even need to call the substr function (it will be converted automatically, and raise an error if too long).
since I actually know that the values
in the varchar(10) column are always
going to be 9 chars.
If that's true, then you don't even need to use SUBSTR as others have been suggesting.
I believe the reason that you're getting an error is that you are trying to insert the value of V_TEN. When you use a construct like FOR x IN (SELECT ...) LOOP, x is implicitly declared as a record type. In your case, it's a record with only one field, but you still can't use it directly as a scalar type.
You just need to reference the field of the record by name in your insert.
FOR V_TEN IN (SELECT THIS_IS_VARCHAR_TEN FROM TABLE WHERE SOMETHING=’VALUE’)
LOOP
INSERT INTO OTHER_TABLE
(THIS_IS_VARCHAR_NINE)
VALUES
(V_TEN.THIS_IS_VARCHAR_TEN);
END LOOP;
In any case, as Thilo pointed out, there's no reason to do this in an explicit loop at all. Just write it as a single INSERT ... SELECT.
Use:
FOR V_TEN IN (SELECT SUBSTR(t.this_is_varchar_ten, 1, 9)
FROM TABLE t
WHERE t.something = 'VALUE')
LOOP
INSERT INTO OTHER_TABLE
(THIS_IS_VARCHAR_NINE)
VALUES
(V_TEN);
END LOOP;
Use the SUBSTR function to substring the VARCHAR(10) data so it is returned as VARCHAR(9)