Store selected column names in user-defined table - sql

I'm currently facing a problem with searching values from columns specified in user-defined table. I've got my .NET program in which I define which parameters are going to be searched for. I've decided that the best way to achieve this is to store this values (selected columns) in a list. Now I have to do something like that:
SELECT
#MyColumnNamesInAList <- I do not know how to iterate it properly
FROM
xyz
What is more, I have to insert NULL if there is no corresponding column in xyz.
In C# code I would do something like this:
foreach(var column in MyColumnNamesInAList)
{
if(!xyz.Contains(column)
"NULL AS " + column.Name;
}

#MyColumnNamesInAList <- I do not know how to iterate it properly
DECLARE #MyColumnNamesInAList VARCHAR(50) = 'ID,Name'
DECLARE #xml AS XML
SET #xml = CAST(('<X>'+REPLACE(#MyColumnNamesInAList,',' ,'</X><X>')+'</X>') AS XML)
SELECT N.value('.', 'varchar(10)') AS Columns FROM #xml.nodes('X') AS T(N)
/*
Columns
--------
ID
Name
*/
In C# code I would do something like this:
foreach(var column in MyColumnNamesInAList)
{
if(!xyz.Contains(column)
"NULL AS " + column.Name;
}
Consider we have a table as
CREATE TABLE [dbo].[tblTesttable]([ID] [int] IDENTITY(1,1) NOT NULL,[Name] [varchar](50) NULL)
Now If we execute the below query, we can figure out the extra columns -
DECLARE #MyColumnNamesInAList VARCHAR(50) = 'ID,Name,NewColumn1,NewColumn2'
DECLARE #xml AS XML
SET #xml = CAST(('<X>'+REPLACE(#MyColumnNamesInAList,',' ,'</X><X>')+'</X>') AS XML)
SELECT X.Column_Name
FROM (SELECT N.value('.', 'varchar(10)') AS Column_Name FROM #xml.nodes('X') AS T(N))X
EXCEPT
(SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = N'tblTesttable')
/*
Column_Name
-----------
NewColumn1
NewColumn2
*/
What is more, I have to insert NULL if there is no corresponding
column in xyz.
Now here I will give you a hint (I want you to solve this part) (:
You need to issue
ALTER TABLE N'tblTesttable' ADD NewColumn1 VARCHAR(20) NULL
ALTER TABLE N'tblTesttable' ADD NewColumn2 VARCHAR(20) NULL
You need to build the query dynamically and using the previous stuffs that we shared.
Hope you understand.
All the best!!!

You should dynamically generate the query directly in your .NET program.
You could do it with system view and the EXEC command but that will run poorly on SQL Server.
Watch out for SQL injection if the values you're using come from the end-user.

Related

split string from a string by sql

I have this type of data in a column in my table,,,
QCIM1J77477, 4650125572, QCCR1J77891, 4650312729, QCCR1J74974 --- remove without comma
or
QCIM1E107498,QCIM1E109835,4650092399/ QCCR1E91190, -- remove 4650092399
I want only that string which starts from QC,remove apart from QC,
so please tell me how can I achive it?
Beneath a piece of t-sql script that creates a temporary table #t with temporary variables. Here the temporary table exists untill you break your session, temporary variables exist during the execution of the script. I have a drop table statement at the bottom. Figure out yourself what you want with the table data and whether you want the data put in somewhere else, for example in a not-temporary table :).
I assume you want all the pieces of the string that contain 'QC' as seperate values. If you want your data back as it was originally, that is multiple strings per one column, then you could also do a group by trick. Then you do need a unique identifier of some sort, like name, id, guid of each row or identity.
create table #t ([QCs] nvarchar(100))
declare #str nvarchar(500)
set #str = 'QCIM1E107498,QCIM1E109835,4650092399/ QCCR1E91190'
--replace the above temporary variable with the column you are selecting
declare #sql nvarchar(4000)
select #sql = 'insert into #t select '''+ replace(#str,',',''' union all select ''') + ''''
print #sql
exec ( #sql )
select
QCs
,PATINDEX('%QC%',QCs) as StartPosition
,SUBSTRING(QCs,PATINDEX('%QC%',QCs),12) as QCsNew
from #t where QCs like '%QC%'
drop table #t
With PATINDEX you find the position where in the string 'QC' starts, and with SUBSTRING you tell the dbms to give back (here) 12 characters starting from the found StartPosition.
Beneath what the result looks like. QCsNew is your desired result.
QCs StartPosition QCsNew
QCIM1E107498 1 QCIM1E107498
QCIM1E109835 1 QCIM1E109835
4650092399/ QCCR1E91190 13 QCCR1E91190

update table with dynamic sql query

For a project, we are using a table (named txtTable) that contains all the texts. And each column contains a different language (for example column L9 is English, column L7 is German, etc..).
TextID L9 L7 L16 L10 L12
------------------------------------------------------
26 Archiving Archivierung NULL NULL NULL
27 Logging Protokollierung NULL NULL NULL
28 Comments Kommentar NULL NULL NULL
This table is located in a database on a Microsoft SQL Server 2005. The big problem is that this database name changes each time the program is restarted. This is a behavior typically for this third-party program and cannot be changed.
Next to this database and on the same server is our own database. In this database are several tables that point to the textID for generating data for reporting (SQL Server Reporting Services) in the correct language. This database contains also a table "ProjectSettings" with some properties like the name of the texttable database, and the stored procedures to generate the reporting data.
The way we now are requesting the right texts of the right language from this table with the changing database name is by creating a dynamic SQL query and execute it in a stored procedure.
Now we were wondering if there is a cleaner way to get the texts in the right language. We were thinking about creating a function with the textID and the language as a parameter, but we cannot find a good way to do this. We thought about a function so we just can use it in the select statement, but this doesn’t work:
CREATE FUNCTION [dbo].[GetTextFromLib]
(
#TextID int,
#LanguageColumn Varchar(5)
)
RETURNS varchar(255)
AS
BEGIN
-- return variables
DECLARE #ResultVar varchar(255)
-- Local variables
DECLARE #TextLibraryDatabaseName varchar(1000)
DECLARE #nvcSqlQuery varchar(1000)
-- get the report language database name
SELECT #TextLibraryDatabaseName = TextLibraryDatabaseName FROM ProjectSettings
SET #nvcSqlQuery = 'SELECT #ResultVar =' + #LanguageColumn + ' FROM [' + #TextLibraryDatabaseName + '].dbo.TXTTable WHERE TEXTID = ' + cast(#TextID as varchar(30))
EXEC(#nvcSqlQuery)
-- Return the result of the function
RETURN #ResultVar
END
Is there any way to work around this so we don’t have to use the dynamic sql in our stored procedures so it is only ‘contained’ in 1 function?
Thanks in advance & kind regards,
Kurt
Yes, it is possible with the help of synonym mechanism introduced with SQL Server 2005. So, you can create synonym during your setting up procedure based on data from ProjectSettings table and you can use it in your function. Your code will look something like this:
UPDATE: The code of function is commented here because it still contains dynamic SQL which does not work in function as Kurt said in his comment. New version of function is below this code.
-- Creating synonym for TXTTable table
-- somewhere in code when processing current settings
-- Suppose your synonym name is 'TextLibrary'
--
-- Drop previously created synonym
IF EXISTS (SELECT * FROM sys.synonyms WHERE name = N'TextLibrary')
DROP SYNONYM TextLibrary
-- Creating synonym using dynamic SQL
-- Local variables
DECLARE #TextLibraryDatabaseName varchar(1000)
DECLARE #nvcSqlQuery varchar(1000)
-- get the report language database name
SELECT #TextLibraryDatabaseName = TextLibraryDatabaseName FROM ProjectSettings
SET #nvcSqlQuery = 'CREATE SYNONYM TextLibrary FOR [' + #TextLibraryDatabaseName + '].dbo.TXTTable'
EXEC(#nvcSqlQuery)
-- Synonym created
/* UPDATE: This code is commented but left for discussion consistency
-- Function code
CREATE FUNCTION [dbo].[GetTextFromLib]
(
#TextID int,
#LanguageColumn Varchar(5)
)
RETURNS varchar(255)
AS
BEGIN
-- return variables
DECLARE #ResultVar varchar(255)
-- Local variables
DECLARE #nvcSqlQuery varchar(1000)
SET #nvcSqlQuery = 'SELECT #ResultVar =' + #LanguageColumn + ' FROM TextLibrary WHERE TEXTID = ' + cast(#TextID as varchar(30))
EXEC(#nvcSqlQuery)
-- Return the result of the function
RETURN #ResultVar
END
*/
UPDATE This is one more attempt to solve the problem. Now it uses some XML trick:
-- Function code
CREATE FUNCTION [dbo].[GetTextFromLib]
(
#TextID int,
#LanguageColumn Varchar(5)
)
RETURNS varchar(255)
AS
BEGIN
-- return variables
DECLARE #ResultVar varchar(255)
-- Local variables
DECLARE #XmlVar XML
-- Select required record into XML variable
-- XML has each table column value in element with corresponding name
SELECT #XmlVar = ( SELECT * FROM TextLibrary
WHERE TEXTID = #TextID
FOR XML RAW, ELEMENTS )
-- Select value of required element from XML
SELECT #ResultVar = Element.value('(.)[1]', 'varchar(255)')
FROM #XmlVar.nodes('/row/*') AS T(Element)
WHERE Element.value('local-name(.)', 'varchar(50)') = #LanguageColumn
-- Return the result of the function
RETURN #ResultVar
END
Hope this helps.
Credits to answerer of this question at Stackoverflow - How to get node name and values from an xml variable in t-sql
To me, it sounds like a total PITA... However, how large is this database of "words" you are dealing with. Especially if it is not changing much and remains pretty constant. Why not have on some normal cycle (such as morning), just have one dynamic query generated that queries the one that changes and synchronize it to a "standard" table name in YOUR database that won't change. Then, all your queries run against YOUR version and completely remove the constant dynamic queries every time. Yes there would need to be this synchronizing stored procedure to run, but if it can be run on a schedule, you should be fine, and again, how large is the table of "words" for proper language context.

Regarding Dynamic sql construction

when we load xml into cursor then we specify column name and their datatype and size. instead of specifying manually how could i make that area dynamic. suppose my tsql as follows
Exec sp_xml_preparedocument #XMLFormat OUTPUT, #DetailXml
-- Create Cursor from XML Table
Declare CurDetailRecord
Cursor For
Select productid,unit,rate,qty,amount
From Openxml (#XMLFormat, '/NewDataSet/PurchaseOrderDetail', 2)
With
(
productid Varchar(10),
unit Varchar(50),
rate decimal(18,2),
qty decimal(18,3),
amount decimal(18,2)
)
here as a example
productid Varchar(10),
unit Varchar(50)
etc i am specifying and also specify their data tyoe & size.
so how could i construct this area dynamically and fetch the column name and data type & size dynamically.
please guide me thanks.
You can get column names(which are nodes inside PurchasePrderDetail node) like this:
declare #xml xml='<NewDataSet><PurchaseOrderDetail>
<productid>19125</productid>
</PurchaseOrderDetail></NewDataSet>'
SELECT b.value('local-name(.)','nvarchar(128)')ColumnName,
LEN(b.value('.','nvarchar(128)'))MaxLength
FROM #xml.nodes('/NewDataSet/PurchaseOrderDetail/*') a(b)
So you can generate dynamic SQL statement to create cursor with appropriate column names and length like varchar(MaxLength).
But you can not get datatypes from XML without knowing real column names because data in xml is just text and f.e. "5" can be int type and also just a text.
EDIT
If you know table name, you can built dynamic SQL statement using metadata from that table using this:
; With cols as(
SELECT COLUMN_NAME,
UPPER(DATA_type)
+
case when data_type like '%char' then
case when CHARACTER_MAXIMUM_LENGTH=-1 THEN ' (MAX)'
else ' ('+CAST(CHARACTER_MAXIMUM_LENGTH as nvarchar)+')'
END
ELSE ''
END ColConv
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME='PurchaseOrderDetail'),
XMLS as(
SELECT b.value('local-name(.)','nvarchar(128)')ColumnName,
b.value('.','nvarchar(128)')Value
FROM #xml.nodes('/NewDataSet/PurchaseOrderDetail/*') a(b)
)
SELECT XMLS.ColumnName,'CAST ('''+XMLS.Value+''' AS '+ ColConv+''')' FROM XMLS
JOIN cols ON XMLS.ColumnName=cols.COLUMN_NAME
As output you will have column name and value with appropriate CAST clause. Then you can build dynamic statement what you need.
Usually the information of data types and field names are describes in the XSD file (XML Schema Definition).
So you need to have a valid XSD file for each of your XML file then you can retrieve the fields name and data type.
Here a link to understand better the XSD
And here how to deal with XSD and XML step by step
Hope it helps you

SQL query like filter

I need to execute a search query in SQL Server where I need to filter out data based upon an user input textfield.
The problem is, this query needs to be executed on several tables (so I only know the tablecolumns at runtime).
This is the query I have:
SELECT * FROM [BTcegeka.C2M].[dbo].[Lookup_Country] WHERE Name LIke '%test%'
Now the problem is I need to do the Like function on every column (I only know the columnname at runtime) in the table. I am calling this query from an ASP.NET website. The user selects a table from a dropdownlist and can then enter the search field.
This is what I really want to accomplish:
SELECT * FROM [BTcegeka.C2M].[dbo].[Lookup_Country] WHERE * LIke '%test%'
Obviously 'Where * Like' Fails. How can I accomplish this?
You can query all columns in a table like:
select name from sys.columns where object_id = object_id('YourTable')
Then you can construct a query that does a like for each column.
Another approach is to create a calculated column called SearchField that contains a concatenation of all strings you'd like to search for. Then you can search like:
create table #tmp (id int identity, col1 varchar(10), col2 varchar(10),
SearchField as col1 + '|' + col2 persisted)
insert #tmp (col1, col2) values
('alfa', 'beta'),
('gamma', 'DELTA'),
('GAMMA', 'delta')
select * from #tmp where SearchField like '%alfa%'
Try using your SQL query like this.
SELECT * FROM [BTcegeka.C2M].[dbo].[Lookup_Country]
WHERE
COL1 LIke '%test%'
OR COL2 LIke '%test%'
OR COL3 LIke '%test%'
You may use AND instead of OR if your requirement needs that.
If you know the column names at run time, then you should build you query in .NET before passing it to sql. You can build it with the correct column name. This way you can account also for the type of the column you search in.
Careful though this path you chose is prone to SQL injection so before sending a query to the SQL you should check it.
If you really need to do this you can search in sqlserver meta tables and find the description of selected user table. Make a good use of this data is easy and you can make any sql you want with this information, but performance may not the that good
you have to use dynamic sql for implementing this. Your column name needs to be passed as parameter to this stored procedure or if you dont want to create stored procedure just declare one paramter and assign the value selected from the drop down list to it and use that in the query.
create procedure sp_dynamicColumn
(
#columnName varchar(10)
)
as
begin
declare #DYNAMICSQL nvarchar(4000);
SET #DYNAMICSQL = 'Select * from [BTcegeka.C2M].[dbo].[Lookup_Country] where '+ #columnName + ' like ''%test%'''
EXECUTE SP_EXECUTESQL #DYNAMICSQL
end
go

Column name or number of supplied values does not match table definition

In the SQL Server, I am trying to insert values from one table to another by using the below query:
delete from tblTable1
insert into tblTable1 select * from tblTable1_Link
I am getting the following error:
Column name or number of supplied values does not match table definition.
I am sure that both the tables have the same structure, same column names and same data types.
They don't have the same structure... I can guarantee they are different
I know you've already created it... There is already an object named ‘tbltable1’ in the database
What you may want is this (which also fixes your other issue):
Drop table tblTable1
select * into tblTable1 from tblTable1_Link
I want to also mention that if you have something like
insert into blah
select * from blah2
and blah and blah2 are identical keep in mind that a computed column will throw this same error...
I just realized that when the above failed and I tried
insert into blah (cola, colb, colc)
select cola, colb, colc from blah2
In my example it was fullname field (computed from first and last, etc)
for inserts it is always better to specify the column names see the following
DECLARE #Table TABLE(
Val1 VARCHAR(MAX)
)
INSERT INTO #Table SELECT '1'
works fine, changing the table def to causes the error
DECLARE #Table TABLE(
Val1 VARCHAR(MAX),
Val2 VARCHAR(MAX)
)
INSERT INTO #Table SELECT '1'
Msg 213, Level 16, State 1, Line 6
Insert Error: Column name or number of
supplied values does not match table
definition.
But changing the above to
DECLARE #Table TABLE(
Val1 VARCHAR(MAX),
Val2 VARCHAR(MAX)
)
INSERT INTO #Table (Val1) SELECT '1'
works. You need to be more specific with the columns specified
supply the structures and we can have a look
The problem is that you are trying to insert data into the database without using columns. SQL server gives you that error message.
Error: insert into users values('1', '2','3') - this works fine as long you only have 3 columns
If you have 4 columns but only want to insert into 3 of them
Correct: insert into users (firstName,lastName,city) values ('Tom', 'Jones', 'Miami')
Beware of triggers. Maybe the issue is with some operation in the trigger for inserted rows.
Dropping the table was not an option for me, since I'm keeping a running log. If every time I needed to insert I had to drop, the table would be meaningless.
My error was because I had a couple columns in the create table statement that were products of other columns, changing these fixed my problem. eg
create table foo (
field1 as int
,field2 as int
,field12 as field1 + field2 )
create table copyOfFoo (
field1 as int
,field2 as int
,field12 as field1 + field2) --this is the problem, should just be 'as int'
insert into copyOfFoo
SELECT * FROM foo
The computed columns make the problem.
Do not use SELECT *. You must specify each fields after SELECT except computed fields
some sources for this issues are as below
1- Identity column ,
2- Calculated Column
3- different structure
so check those 3 , i found my issue was the second one ,
For me the culprit is int value assigned to salary
Insert into Employees(ID,FirstName,LastName,Gender,Salary) values(3,'Canada', 'pa', 'm',15,000)
in salary column When we assign 15,000 the compiler understand 15 and 000.
This correction works fine for me.
Insert into Employees(ID,FirstName,LastName,Gender,Salary) values(4,'US', 'sam', 'm',15000)
Update to SQL server 2016/2017/…
We have some stored procedures in place to import and export databases.
In the sp we use (amongst other things) RESTORE FILELISTONLY FROM DISK where we create a
table "#restoretemp" for the restore from file.
With SQL server 2016, MS has added a field SnapshotURL nvarchar(360) (restore url Azure) what has caused the error message.
After I have enhanced the additional field, the restore has worked again.
Code snipped (see last field):
SET #query = 'RESTORE FILELISTONLY FROM DISK = ' + QUOTENAME(#BackupFile , '''')
CREATE TABLE #restoretemp
(
LogicalName nvarchar(128)
,PhysicalName nvarchar(128)
,[Type] char(1)
,FileGroupName nvarchar(128)
,[Size] numeric(20,0)
,[MaxSize] numeric(20,0)
,FileID bigint
,CreateLSN numeric(25,0)
,DropLSN numeric(25,0) NULL
,UniqueID uniqueidentifier
,ReadOnlyLSN numeric(25,0)
,ReadWriteLSN numeric(25,0)
,BackupSizeInByte bigint
,SourceBlockSize int
,FilegroupID int
,LogGroupGUID uniqueidentifier NULL
,DifferentialBaseLSN numeric(25,0)
,DifferentialbaseGUID uniqueidentifier
,IsReadOnly bit
,IsPresent bit
,TDEThumbprint varbinary(32)
-- Added field 01.10.2018 needed from SQL Server 2016 (Azure URL)
,SnapshotURL nvarchar(360)
)
INSERT #restoretemp EXEC (#query)
SET #errorstat = ##ERROR
if #errorstat <> 0
Begin
if #Rueckgabe = 0 SET #Rueckgabe = 6
End
Print #Rueckgabe
Check your id. Is it Identity? If it is then make sure it is declared as ID not null Identity(1,1)
And before creating your table , Drop table and then create table.
The problem I had that caused this error was that I was trying to insert null values into a NOT NULL column.
I had the same problem, and the way I worked around it is probably not the best but it is working now.
It involves creating a linked server and using dynamic sql - not the best, but if anyone can suggest something better, please comment/answer.
declare #sql nvarchar(max)
DECLARE #DB_SPACE TABLE (
[DatabaseName] NVARCHAR(128) NOT NULL,
[FILEID] [smallint] NOT NULL,
[FILE_SIZE_MB] INT NOT NULL DEFAULT (0),
[SPACE_USED_MB] INT NULL DEFAULT (0),
[FREE_SPACE_MB] INT NULL DEFAULT (0),
[LOGICALNAME] SYSNAME NOT NULL,
[DRIVE] NCHAR(1) NOT NULL,
[FILENAME] NVARCHAR(260) NOT NULL,
[FILE_TYPE] NVARCHAR(260) NOT NULL,
[THE_AUTOGROWTH_IN_KB] INT NOT NULL DEFAULT(0)
,filegroup VARCHAR(128)
,maxsize VARCHAR(25)
PRIMARY KEY CLUSTERED ([DatabaseName] ,[FILEID] )
)
SELECT #SQL ='SELECT [DatabaseName],
[FILEID],
[FILE_SIZE_MB],
[SPACE_USED_MB],
[FREE_SPACE_MB],
[LOGICALNAME],
[DRIVE],
[FILENAME],
[FILE_TYPE],
[THE_AUTOGROWTH_IN_KB]
,filegroup
,maxsize FROM OPENQUERY('+ QUOTENAME('THE_MONITOR') + ','''+ ' EXEC MASTER.DBO.monitoring_database_details ' +''')'
exec sp_executesql #sql
INSERT INTO #DB_SPACE(
[DatabaseName],
[FILEID],
[FILE_SIZE_MB],
[SPACE_USED_MB],
[FREE_SPACE_MB],
[LOGICALNAME],
[DRIVE],
[FILENAME],
[FILE_TYPE],
THE_AUTOGROWTH_IN_KB,
[filegroup],
maxsize
)
EXEC SP_EXECUTESQL #SQL
This is working for me now.
I can guarantee the number of columns and type of columns returned by the stored procedure are the same as in this table, simply because I return the same table from the stored procedure.
In my case, I had:
insert into table1 one
select * from same_schema_as_table1 same_schema
left join...
and I had to change select * to select same_schema.*.
You're missing column name after TableName in insert query:
INSERT INTO TableName**(Col_1,Col_2,Col_3)** VALUES(val_1,val_2,val_3)
In my case the problem was that the SP I was executing returned two result sets, and only the second result set was matching the table definition.