Find a value from all table columns - sql

I am building a functionality that will filter a data on all column table.
Let's say I have this table:
-------------------------------------
| ID | NAME | Address | Remarks |
| 1 | Manny | Phil | Boxer-US |
| 2 | Timothy | US | Boxer |
| 3 | Floyd | US | Boxer |
| 4 | Maidana | US | Boxer |
| 5 | Marquez | MEX | Boxer |
-------------------------------------
I search for "US", it should give me IDs 1-4 since "US" exists in their columns.
I could have this to filter it:
SELECT ID FROM tbl_Boxers
WHERE ID LIKE '%US%' OR NAME LIKE '%US%' OR Address LIKE '%US%' OR Remarks LIKE '%US%'
But I'm trying to avoid a long WHERE clause here since in actual, I have around 15 columns to look at.
Is there any other way to minimize the where clause?
Please help.
Thanks

The way this is normally done is to make a 'Searchable Field' column where you concatinate all search columns into one field for searching.
So while that provides an easier overview and querying it adds some management of the data you need to be aware off on inserts and updates.
Also on its own - it's not an optimal way of searching, so if performance is important - then you should look to implement full text search.
So the question is where you want to have the 'overhead' and whether that functionality your building is going to be run often or just once in a while.
If it is the former, and performance is important - look to full text. If it is just a once-in-a-while query, then I'd properly just do the long WHERE clause myself to avoid adding more overhead on the maintenance of the data

Check the following solution. Here the query is generated dynamically based on the column names in your table.
This is applicable if the given table is a physical table. This solution wont work for temporary tables or table variables.
BEGIN TRAN
--Simulate your table structure
--Should be a physical table. Cannot be a temp table or a table variable
CREATE TABLE TableA
(
ID INT,
NAME VARCHAR(50),
ADDRESS VARCHAR(50),
REMARKS VARCHAR(50)
)
--Added values for testing
INSERT INTO TableA(ID, name , address ,remarks) VALUES(1,'Manny','Phil','Boxer-US')
INSERT INTO TableA(ID, name , address ,remarks) VALUES(2,'Timothy','US','Boxer')
INSERT INTO TableA(ID, name , address ,remarks) VALUES(3,'Floyd','US','Boxer')
INSERT INTO TableA(ID, name , address ,remarks) VALUES(4,'Maidana','US','Boxer')
INSERT INTO TableA(ID, name , address ,remarks) VALUES(5,'Marquez',' MEX','Boxer')
--Solution Starts from here
DECLARE #YourSearchValue VARCHAR(50)--Will be passed
SET #YourSearchValue = 'US' --Simulated passed value
CREATE TABLE #TableCols
(
ID INT IDENTITY(1,1),
COLUMN_NAME VARCHAR(1000)
)
INSERT INTO #TableCols
(COLUMN_NAME)
SELECT COLUMN_NAME
FROM information_schema.columns
WHERE table_name = 'TableA';
DECLARE #STARTCOUNT INT, #MAXCOUNT INT, #COL_NAME VARCHAR(1000), #QUERY VARCHAR(8000), #SUBQUERY VARCHAR(8000)
SELECT #STARTCOUNT = 1, #MAXCOUNT = MAX(ID) FROM #TableCols;
SELECT #QUERY = '', #SUBQUERY = ''
WHILE(#STARTCOUNT <= #MAXCOUNT)
BEGIN
SELECT #COL_NAME = COLUMN_NAME FROM #TableCols WHERE ID = #STARTCOUNT;
SET #SUBQUERY = #SUBQUERY + ' CONVERT(VARCHAR(50), ' + #COL_NAME + ') LIKE ''%' + #YourSearchValue + '%''' + ' OR ';
SET #STARTCOUNT = #STARTCOUNT + 1
END
SET #SUBQUERY = LEFT(#SUBQUERY, LEN(#SUBQUERY) - 3);
SET #QUERY = 'SELECT * FROM TableA WHERE 1 = 1 AND (' + #SUBQUERY + ')'
--PRINT (#QUERY);
EXEC (#QUERY);
ROLLBACK
Hope this helps.

Related

SQL pivot table1 into table2 without using dynamic SQL pivot or hardcode query

I have seen many questions and answers given about pivoting table with SQL, with dynamic SQL pivot or hard code query with CASE WHEN.
However is there any way I can pivot table without using those 2?
Table 1:
| col1 | col2 | col3 |
|--------|-------|--------|
| ABCD | 1 | XY123 |
| ABCD | 2 | RT789 |
| PQST | 3 | XY123 |
| PQST | 4 | RT789 |
Pivoting to
| col1 | ABCD | PQST |
|--------|-------|-------|
| XY123 | 1 | 3 |
| RT789 | 2 | 4 |
My idea was to retrieve the structure of the col with:
WITH
structure AS (
SELECT DISTINCT
col3 AS col1, col1 AS colName, col2 AS values
FROM table1 ori
)
and then extracting matched values of each cell with joins and storing them temporarily. At last JOIN again populating them in the output. However I am stuck after the above step. I can't use PIVOT and have to do this dynamically (i.e. can't use the method to hardcode each value with CASE WHEN)
How can I achieve this?
This is not as efficient (and not as easy to code) as a dynamic pivot. However, it is doable.
It does all need to be dynamic e.g., creating each SQL statement as a string and executing that.
The process involves
Determine the column names (store in a temporary table)
Creating the table with the first column only
Populating that first column
For each additional column name
Adding a column to the table (dynamically)
Populating that column with data
You haven't specified the database - I'll illustrate the following below using SQL Server/T-SQL.
The following are in this db<>fiddle so you can see what's going on.
CREATE TABLE #ColNames (ColNum int IDENTITY(1,1), ColName nvarchar(100), ColNametxt nvarchar(100));
INSERT INTO #ColNames (ColName, ColNametxt)
SELECT DISTINCT QUOTENAME(Col1), Col1
FROM table1;
This will populate the #ColNames table with the values 1, [ABCD], ABCD, 2, [PQST], PQST.
The next step is to create your output table - I'll call it #pvttable
CREATE TABLE #pvttable (col1 nvarchar(100) PRIMARY KEY);
INSERT INTO #pvttable (col1)
SELECT DISTINCT Col3
FROM table1;
This creates your table with 1 column (col1) with values XY123 and RT789).
The write your favorite loop (e.g., cursor, while loop). In each step
Get the next column name
Add the column to the table
Update that column with appropriate data
e.g., the following is an illustrative example with your data.
DECLARE #CustomSQL nvarchar(4000);
DECLARE #n int = 1;
DECLARE #ColName nvarchar(100);
DECLARE #ColNametxt nvarchar(100);
SELECT #ColName = ColName,
#ColNameTxt = ColNameTxt
FROM #ColNames
WHERE ColNum = #n;
WHILE #ColName IS NOT NULL
BEGIN
SET #CustomSQL = N'ALTER TABLE #pvttable ADD ' + #ColName + N' nvarchar(100);';
EXEC (#CustomSQL);
SET #CustomSQL =
N'UPDATE #pvttable SET ' + #Colname + N' = table1.col2'
+ N' FROM #pvttable INNER JOIN table1 ON #pvttable.col1 = table1.col3'
+ N' WHERE table1.col1 = N''' + #ColNametxt + N''';';
EXEC (#CustomSQL);
SET #n += 1;
SET #ColName = NULL;
SET #ColNametxt = NULL;
SELECT #ColName = ColName,
#ColNameTxt = ColNameTxt
FROM #ColNames
WHERE ColNum = #n;
END;
SELECT * FROM #pvttable;

Single column from Multiple tables SQL-SERVER 2014 Exprs

I have a DB with 50 tables having the same structure (same column names, types) clustered Indexed on the Created Date column . Each of these tables have around ~ 100,000 rows and I need to pull all of them for some columns.
select * from customerNY
created date | Name | Age | Gender
__________________________________
25-Jan-2016 | Chris| 25 | M
27-Jan-2016 | John | 24 | M
30-Jan-2016 | June | 34 | F
select * from customerFL
created date | Name | Age | Gender
__________________________________
25-Jan-2016 | Matt | 44 | M
27-Jan-2016 | Rose | 24 | F
30-Jan-2016 | Bane | 34 | M
The above is an example of the tables in the DB. I need an SQL that runs quickly pulling all the data. Currently, I am using UNION ALL for this but it takes a lot of time for completing the report. Is there another way for this where I can pull in data without using UNION ALL such as
select Name, Age, Gender from [:customerNY:customerFL:]
Out of context: Can I pull in the table name in the result?
Thanks for any help. I've been putting my mind to this but I can't find a way to do it quicker.
This dynamic SQL approach should meet your criteria, it selects table names from the schema and creates a SELECT statement at runtime for it to execute, and to meet the criteria of the UNION ALL each SELECT statement is given a UNION ALL then I use STUFF to remove the first one.
DECLARE #SQL AS VarChar(MAX)
SET #SQL = ''
SELECT #SQL = #SQL + 'UNION ALL SELECT Name, Age, Gender FROM ' + TABLE_SCHEMA + '.[' + TABLE_NAME + ']' + CHAR(13)
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE 'Customer%'
SELECT #SQL = STUFF(#SQL,1,10,'')
EXEC (#SQL)
However I do not recommend using this and you should do what people have suggested in the comments to restructure your data.
Memory Optimising the test tables below gave a 7x speed increase compared to the same data in regular tables. Samples are 50 tables of 100000 rows. Please only run this on a test server as it creates filegroups/tables etc.:
USE [master]
GO
ALTER DATABASE [myDB] ADD FILEGROUP [MemOptData] CONTAINS MEMORY_OPTIMIZED_DATA
GO
ALTER DATABASE [myDB] ADD FILE ( NAME = N'Mem', FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\DATA' ) TO FILEGROUP [MemOptData] --Change Path for your version
Go
use [myDB]
go
set nocount on
declare #loop1 int = 1
declare #loop2 int = 1
declare #NoTables int = 50
declare #noRows int = 100000
declare #sql nvarchar(max)
while #loop1 <= #NoTables
begin
set #sql = 'create table [MemCustomer' + cast(#loop1 as nvarchar(6)) + '] ([ID] [int] IDENTITY(1,1) NOT NULL,[Created Date] date, [Name] varchar(20), [Age] int, Gender char(1), CONSTRAINT [PK_Customer' + cast(#loop1 as nvarchar(6)) + '] PRIMARY KEY NONCLUSTERED
(
[ID] ASC
)) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA)'
exec(#sql)
while #loop2 <= #noRows
begin
set #sql = 'insert into [MemCustomer' + cast(#loop1 as nvarchar(6)) + '] ([Created Date], [Name], [Age], [Gender]) values (DATEADD(DAY, ROUND(((20) * RAND()), 0), DATEADD(day, 10, ''2018-06-01'')), (select top 1 [name] from (values(''bill''),(''steve''),(''jack''),(''roger''),(''paul''),(''ozzy''),(''tom''),(''brian''),(''norm'')) n([name]) order by newid()), FLOOR(RAND()*(85-18+1))+18, iif(FLOOR(RAND()*(2))+1 = 1, ''M'', ''F''))'
--print #sql
exec(#sql)
set #loop2 = #loop2 + 1
end
set #loop2 = 1
set #loop1 = #loop1 + 1
end
;with cte as (
Select * from MemCustomer1
UNION
Select * from MemCustomer2
UNION
...
UNION
Select * from MemCustomer50
)
select * from cte where [name] = 'tom' and age = 27 and gender = 'F'

SQL Server - validate table column values against with values provided

I have the following task that want to solve using SQL Server's query and/or stored procedure, and I would really appreciated if someone can give me some directions.
Basically we have a data warehouse based on SQL Server. I want to validate columns in certain tables to ensure the values in these columns are valid.
Example as below:
Table1 ColumnsToValidate specifies the table/columns in which values need to be validated. In the example I want to validate the Gender column of the Customer table, and the State column of the Address table. And the validationID is a foreign-key to a table holding all the valid values (Table2).
Table2 ValidationValues: this table holds all valid values for specific validation rules. In the example, validation rule #1 (ValidationID = 1) has two valid values, and validation rule #2 specified 3 valid values.
I'd like to (using SQL) dynamically create a query based on values in Table 1, which accordingly selects the Customer.Gender column and the Address.State column, so the values in these columns can be validated against the values in Table 2.
Table1: ColumnsToValidate
TableName | ColumnName | ValidationID
-----------+----------------+-----------------
Customer | Gender | 1
Address | State | 2
Table2: ValidationValues
ValidationID | Values
-------------+----------------
1 | Male
1 | Female
2 | NY
2 | WA
2 | CA
Table3: Customer
CustomerID | Gender
-----------+----------------
111 | Male
112 | Female
113 | Unknown
114 | NULL
Table4: Address
AddressID | State
-----------+----------------
211 | AL
212 | NY
213 | WA
214 | NULL
EDIT: I could write this in a C# program, but the program will be slow. I would think there could be a way in pure SQL (SQL Server)
Here is my solution, in two steps -
1) To validate the values "as if" the target table and column are known, e.g. the following outer join query finds invalid values on the Customer.Gender column.
select * from Customer a
left join ValidationValues b
on a.Gender = b.values
and a.ValidationID = b.ValidationID
where b.values is null
2) Use dynamic-SQL to generate above SQL script, using values from table ColumnsToValidate, substituting table-name 'Customer' and column-name 'Gender' with variables #tab and #col1 :
declare #tableCursor cursor,
#tab varchar(100),
#col1 varchar(100),
#val_id varchar(20)
set #tableCursor = cursor for select * from ColumnsToValidate
open #tableCursor
fetch next from #tableCursor into #tab, #col1, #val_id
while(##fetch_status = 0)
begin
--dynamic sql
declare #sql varchar(max)
set #sql =
N'select * from '+ #tab +' a ' +
N'left join ValidationValues b ' +
N'on a.' + #col1 + ' = b.values ' +
N'and a.' + #val_id + ' = b.ValidationID ' +
N'where b.values is null'
--print #sql
exec #sql
fetch next from #tableCursor into #tab, #col1, #val_id
end
close #tableCursor
deallocate #tableCursor
As being said, these are mock-up codes and they are not tested. However I'd just to share the ideas to people having similar problems - the key to the solution is "Dynamic SQL".

Retrieve column value using column name in SQL Server

I am using SSMS 2014, I want to retrieve column value using column name in SQL Server. The user can select any column from the table and retrieve the value of that column.
Example:
exec employee Name 'karl'
Output as follows:
| Id | Name | ManagerId | ManagerName | Gender | Dept |
| 5 | Karl | 1 | Luke | M | 1 |
I am creating procedure to resolve this but i am not getting any value in output.
Create proc sp_getEmpDetail
#colname varchar(50),
#colvalue varchar(50)
as
Select *
from employees1
where #colname = #colvalue
I am not getting any value in output.
When I am debugging it then variables are getting value which I supplied.
Please help to resolve this.
Thanks in advance.
--for this you need to create a dynamic Query
Create proc sp_getEmpDetail
#colname varchar(50),
#colvalue varchar(50)
AS
DECLARE #Sql_String NVARCHAR(MAX)
SET #Sql_String='Select * from employees1 where '+#colname+' = '''+#colvalue+''''
PRINT #Sql_String
EXEC(#Sql_String)
END

Count number of rows across multiple tables in one query

I have a SQL Server 2005 database that stores data for multiple users. Each table that contains user-owned data has a column called OwnerID that identifies the owner; most but not all tables have this column.
I want to be able to count number of rows 'owned' by a user in each table. In other words, I want a query that returns the names of each table that contains an OwnerID column, and counts the number of rows in each table that match a given OwnerID value.
I can return just the names of the matching tables using this query:
SELECT OBJECT_NAME(object_id) [Table] FROM sys.columns
WHERE name = 'OwnerID' ORDER BY OBJECT_NAME(object_id);
That query returns a list of table names like this:
+---------+
| Table |
+---------+
| Alpha |
| Beta |
| Gamma |
| ... |
+---------+
But is it possible to write a query that can also count the number of rows in each table that match a given OwnerID? ie:
+---------+------------+
| Table | RowCount |
+---------+------------+
| Alpha | 2042 |
| Beta | 49 |
| Gamma | 740 |
| ... | ... |
+---------+------------+
Note: The list of table names needs to be returned dynamically, it is not suitable to hard-code table names into this query.
Edit: the answer...
(I can't edit your answers yet but I can edit my own question so I'm putting it here...)
Damien_The_Unbeliever had essentially the correct answer, but SQL Server doesn't allow string concatenation in an exec statement so I had to set the query prior to the exec statement. The final query is as follows:
DECLARE #OwnerID int;
SET #OwnerID = 1;
DECLARE #ForEachSQL varchar(100);
SET #ForEachSQL = 'INSERT INTO #t(TableName,RowsOwned) SELECT ''?'', COUNT(*) FROM ? WHERE OwnerID = ' + CONVERT(varchar(11), #OwnerID);
CREATE TABLE #t(TableName sysname, RowsOwned int);
EXEC sp_MSforeachtable #ForEachSQL,
#whereAnd = 'AND o.id IN (SELECT id FROM syscolumns where name=''OwnerID'')';
SELECT * FROM #t ORDER BY TableName;
DROP TABLE #t;
You can use sp_MSForeachtable, and the #whereand parameter, to specify a filter so you're only working against tables with an OwnerID column. Create a temp table, and populate that for each matching table. Something like:
create table #t(tablename sysname,Cnt int)
sp_MSforeachtable 'insert into #t(tablename,Cnt) select ''?'',COUNT(*) from ?',#whereAnd='and o.id in (select id from syscolumns where name=''OwnerID'')'
select * from #t
Two major caveats to mention - first is that sp_MSforeachtable is "undocumented", so you use it at your own risk - it could be suddenly removed from SQL Server by any kind of servicing, or in the next release.
The second is that, having a dynamic schema is usually a sign that something else has gone wrong in modelling - possibly attribute splitting (where sales for January and February are given different tables, even though they're logically the same thing and should appear in the same table, with possibly an additional column to distinguish them)
And, of course, you wanted to filter based on a particular clientID, so the query would be more like:
'insert into #t(tablename,Cnt) select ''?'',COUNT(*) from ? where OwnerID=' + #OwnerID
(Assuming #OwnerID is the owner sought, and is an int)
This would get the info from sysindexes. It can be slightly out of date but will give you a rough count
SELECT
[TableName] = so.name,
[RowCount] = MAX(si.rows)
FROM
sysobjects so,
sysindexes si
WHERE
so.xtype = 'U'
AND
si.id = OBJECT_ID(so.name)
GROUP BY
so.name
ORDER BY
2 DESC
If you needed it to be 100% right then you could use the undocumented feature sp_MSForEachTable
DECLARE #SQL VARCHAR(255)
SET #SQL = 'DBCC UPDATEUSAGE (' + DB_NAME() + ')'
EXEC(#SQL)
CREATE TABLE #foo
(
tablename VARCHAR(255),
rc INT
)
INSERT #foo
EXEC sp_msForEachTable
'SELECT PARSENAME(''?'', 1),
COUNT(*) FROM ?'
SELECT tablename, rc
FROM #foo
ORDER BY rc DESC
DROP TABLE #foo
You can use this:
DECLARE #nSQL NVARCHAR(MAX)
SELECT #nSQL = COALESCE(#nSQL + 'UNION ALL ' + CHAR(10), '')
+ 'SELECT ''' + TABLE_NAME + ''' AS TableName, COUNT(*) FROM ' + QUOTENAME(TABLE_NAME) + CHAR(10)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE COLUMN_NAME = 'strKey'
-- This will PRINT out the dynamically generated SQL statement. Just replace this with EXECUTE(#nSQL) when you are happy to run it.
PRINT #nSQL
Update: To search for a specific OwnerId:
DECLARE #nSQL NVARCHAR(MAX)
DECLARE #OwnerId INTEGER
SET #OwnerId = 1
SELECT #nSQL = COALESCE(#nSQL + 'UNION ALL ' + CHAR(10), '')
+ 'SELECT ''' + TABLE_NAME + ''' AS TableName, COUNT(*) FROM ' + QUOTENAME(TABLE_NAME) + ' WHERE OwnerId = #OwnerId' + CHAR(10)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE COLUMN_NAME = 'strKey'
EXECUTE sp_executesql #nSQL, '#OwnerId INTEGER', #OwnerId
SELECT
O.ID,
O.NAME,
I.ROWCNT
FROM SYSOBJECTS O
INNER JOIN SYSINDEXES I
ON O.ID = I.ID
WHERE O.UID = 5
AND O.XTYPE = 'U'
AND I.STATUS = 0
Try using this query it will give you id of the table, table name and no of rows for that table.
UID = 5 means I want to check in particular schema which has id = 5.You can check schema id using SELECT SCHEMA_ID('<schema name>');
XTYPE = 'U' means User defined tables only.