Related
How to Select a Column's Value Dynamically from a Table, where columns are not fixed?
I have a table name EMP which contains a numbers of dynamic columns.
id Items Col1 Col2 Col3 Col4 ... .... ... Col20 -----> Columns
1 ABC 20 40 100 32 .. .. . 200 -----> Values
Now I have a Dynamic Formula like
Round(Col1+Col4+Col20,2)+100
For Solving this formula I have to fetch the particular columns's value, which are associated to that columns and finally execute the query according to the formula.
I can Apply 2 methods
First Method
select col1,col4,col20 from EMP
but it will not work because I do not know the columns's name, even I get extract the required columns form the formula, but after getting the columns name. how can i select them dynamically from the EMP table?
****Second Method****
declare #holder int,#holder2 int.....
SELECT #holder=col1,#holder=col2..... FROM emp
It will also not work, because how can I Declare the a numbers of Dynamic variable for every column?
How can I do it?
Please Help
This is a very dubious requirement. Tables shouldn't have dynamic columns.
You can do something horrible with XML though (DB Fiddle)
The below will dynamically find all columns in the table whose name starts with the prefix "Col" and then cast the contents to decimal and sum them.
SELECT t.*, (SELECT t.* FOR XML PATH('x'), TYPE).value('
let $sum := sum(
for $col in x/*
where substring(local-name($col), 1, 3) eq "Col"
return ($col cast as xs:decimal ?)
)
return $sum',
'decimal(20,10)') AS Sum
FROM YourTable t
Declare
#sql1 as nvarchar(MAX),
#para as nvarchar(Max),-----'Holds all the columns name, that will extract from formula like col1,col2'
#formula nvarchar(MAX),--- Holds the formula like Round(Col1+Col4+Col20,2)+100
#ABC float,
set #sql1=N'
select * into datafetch from (
SELECT '+ #para+', '+#formula+' as newcol'+' FROM Emp)abc'
exec(#sql1)
SELECT #ABC=newcol FROM datafetch
---#ABC variable holds the final calculated value from the formula
You should only use Dynamic SQL for this purpose.
like this :
declare #columns varchar(max) = 'Col1 , Col2 , Col3' ;
declare #sql varchar(mx) = 'SELECT '+ #columns +'FROM emp' ;
I want to build a dynamic SQL query where I can use data from another table as where condition. Let's assume I have two tables: one table with financial data and the other one with conditions. They look something like this:
Table sales
c006 mesocomp c048 c020 c021
----- ---------- ------- ----- ----
120 01TA MICROSOFT 2 239
and a condition table with the following data:
dimension operator wert_db
--------- -------- -------
sales.c006 < 700
sales.c048 not like 'MIC%'
sales.c021 in (203,206)
I want to select all data from sales with the conditions stated in the condition table. So I have an SQL Query as follows:
SELECT *
FROM sales
WHERE sales.c006 < 700
AND sales.c048 NOT LIKE 'MIC%'
AND sales.c021 IN (203, 206)
Since you've posted no attempt to solve or research this yourself, I'll point you in a direction to get you started.
Your question already mentions using Dynamic SQL, so I assume you know at least what that is. You're going to populate a string variable, starting with 'SELECT * FROM Sales '.
You can use the STUFF...FOR XML PATH technique to assemble the conditions rows into a WHERE clause.
One change to the linked example is that you'll need to concatenate dimension, operator and wert_db into one artificial column in the innermost SELECT. Also instead of separating with a comma, you'll separate with ' AND '. And change the parameters of the STUFF function to take off the length of ' AND ' instead of the length of a comma.
DECLARE #tblSales TABLE
(
c006 VARCHAR(10),
mesocomp VARCHAR(100),
c048 VARCHAR(100),
c020 VARCHAR(100),
c021 VARCHAR(100)
)
INSERT INTO #tblSales(c006, mesocomp, c048, c020, c021)
VALUES(120,'01Ta','Microsoft','2','239')
SELECT * FROM #tblSales
DECLARE #tblCondition TABLE
(
Id INT,
dimension VARCHAR(100),
operator VARCHAR(10),
wert_db VARCHAR(100)
)
INSERT INTO #tblCondition(Id, dimension, operator, wert_db) VALUES(1,'sales.c006','<','700')
INSERT INTO #tblCondition(Id, dimension, operator, wert_db) VALUES(1,'sales.c048','not like','''MIC%''')
INSERT INTO #tblCondition(Id, dimension, operator, wert_db) VALUES(1,'sales.c021','in','(203,206)')
DECLARE #whereCondition VARCHAR(400)
SELECT #whereCondition = COALESCE(#whereCondition + ' ', '') + dimension + ' ' + operator + ' ' + wert_db + ' AND '
FROM #tblCondition
SET #whereCondition = SUBSTRING(#whereCondition,0, LEN(#whereCondition) - 3)
PRINT #whereCondition
DECLARE #sql VARCHAR(4000)
SET #sql = 'SELECT * FROM #tblSales Where ' + #whereCondition
PRINT #sql
EXEC(#sql)
--please use real tables so you will get everything working.
I have a table including:
ID Name Period0Id Period1Id Period2Id
What I would like to receive data based on a user-defined parameter #check.
Lets assume:
declare #check int = 1;
In this case I need to get Period1Id value from the table. So I need to have something like that:
Select ID, Name, StatusId = Period + #check + Id -- #check is the parameter
From mytable
However, my query is not working. How can I fix this?
Your table looks like it is not in first normal form.
Instead of three columns for Period0Id to Period2Id you could have a column for PeriodIndex with values of (0,1,2) and a single column for PeriodId and then it would be just a WHERE PeriodIndex = #Check
You can't select a column using string interpolation with a variable as you are attempting. You can use dynamic SQL to create the SQL String dynamically. Or simply hardcode the options if they all have the same dataype.
Select ID,
Name,
StatusId = CASE #Check WHEN 0 THEN Period0Id
WHEN 1 THEN Period1Id
WHEN 2 THEN Period2Id
END
From mytable
Here is an alternative way that will create dynamic columns, which is essentially using your original query:
DECLARE #check VARCHAR = 1
DECLARE #sqlquery NVARCHAR(MAX)
SET #sqlquery = N'SELECT ID, Name, StatusId = Period'+#check+'Id
FROM mytable'
EXEC sp_executesql #sqlquery
How to select all the columns of a table except one column?
I have nearly 259 columns I cant mention 258 columns in SELECT statement.
Is there any other way to do it?
You can use this approach to get the data from all the columns except one:-
Insert all the data into a temporary table
Then drop the column which you dont want from the temporary table
Fetch the data from the temporary table(This will not contain the data of the removed column)
Drop the temporary table
Something like this:
SELECT * INTO #TemporaryTable FROM YourTableName
ALTER TABLE #TemporaryTable DROP COLUMN Columnwhichyouwanttoremove
SELECT * FROM #TemporaryTable
DROP TABLE #TemporaryTable
Create a view. Yes, in the view creation statement, you will have to list each...and...every...field...by...name.
Once.
Then just select * from viewname after that.
This is not a generic solution, but some databases allow you to use regular expressions to specify the columns.
For instance, in the case of Hive, the following query selects all columns except ds and hr:
SELECT `(ds|hr)?+.+` FROM sales
You can get the column name details from sys.columns table
Try the following query:
SELECT * FROM SYS.COLUMNS
WHERE object_id = OBJECT_ID('dbo.TableName')
AND [Name] <> 'ColumnName'
DECLARE #sql as VARCHAR(8000)
SET #sql = 'SELECT '
SELECT #sql += [Name] + ', ' FROM SYS.COLUMNS
WHERE object_id = OBJECT_ID('dbo.TableName')
AND [Name] <> 'ColumnName'
SELECT #sql += ' FROM Dbo.TableName'
EXEC(#sql)
I just wanted to echo #Luann's comment as I use this approach always.
Just right click on the table > Script table as > Select to > New Query window.
You will see the select query. Just take out the column you want to exclude and you have your preferred select query.
There are lot of options available , one of them is :
CREATE TEMPORARY TABLE temp_tb SELECT * FROM orig_tb;
ALTER TABLE temp_tb DROP col_x;
SELECT * FROM temp_tb;
Here the col_x is the column which u dont want to include in select statement.
Take a look at this question : Select all columns except one in MySQL?
You can retrieve the list of column name by simple query and then remove those column by apply where query like this.
SELECT * FROM (
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = N'TableName'
) AS allColumns
WHERE allColumns.COLUMN_NAME NOT IN ('unwantedCol1', 'unwantedCol2')
If you are using DataGrip you can do the following:
Enter your SELECT statement SELECT * FROM <your_table>;
Put your cursor over * and press Alt+Enter
You will get pop up menu with Expand column list option
Click on it and it will convert * with full list of columns
Now you can remove columns that you don't need
Here is a link for an example on how to do it.
Without creating new table you can do simply (e.g with mysqli):
get all columns
loop through all columns and remove wich you want
make your query
$r = mysqli_query('SELECT column_name FROM information_schema.columns WHERE table_name = table_to_query');
$c = count($r); while($c--) if($r[$c]['column_name'] != 'column_to_remove_from_query') $a[] = $r[$c]['column_name']; else unset($r[$c]);
$r = mysqli_query('SELECT ' . implode(',', $a) . ' FROM table_to_query');
Try the following query:
DECLARE #Temp NVARCHAR(MAX);
DECLARE #SQL NVARCHAR(MAX);
SET #Temp = '';
SELECT #Temp = #Temp + COLUMN_NAME + ', ' FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME ='Person' AND COLUMN_NAME NOT IN ('Id')
SET #SQL = 'SELECT ' + SUBSTRING(#Temp, 0, LEN(#Temp)) +' FROM [Person]';
EXECUTE SP_EXECUTESQL #SQL;
In your case, expand columns of that database in the object explorer. Drag the columns in to the query area.
And then just delete one or two columns which you don't want and then run it. I'm open to any suggestions easier than this.
Only one way to achieve this giving column name. There is no other method found. You must have to list all column name
Is there any RDBMS that implements something like SELECT * EXCEPT? What I'm after is getting all of the fields except a specific TEXT/BLOB field, and I'd like to just select everything else.
Almost daily I complain to my coworkers that someone should implement this... It's terribly annoying that it doesn't exist.
Edit: I understand everyone's concern for SELECT *. I know the risks associated with SELECT *. However, this, at least in my situation, would not be used for any Production level code, or even Development level code; strictly for debugging, when I need to see all of the values easily.
As I've stated in some of the comments, where I work is strictly a commandline shop, doing everything over ssh. This makes it difficult to use any gui tools (external connections to the database aren't allowed), etc etc.
Thanks for the suggestions though.
As others have said, it is not a good idea to do this in a query because it is prone to issues when someone changes the table structure in the future. However, there is a way to do this... and I can't believe I'm actually suggesting this, but in the spirit of answering the ACTUAL question...
Do it with dynamic SQL... this does all the columns except the "description" column. You could easily turn this into a function or stored proc.
declare #sql varchar(8000),
#table_id int,
#col_id int
set #sql = 'select '
select #table_id = id from sysobjects where name = 'MY_Table'
select #col_id = min(colid) from syscolumns where id = #table_id and name <> 'description'
while (#col_id is not null) begin
select #sql = #sql + name from syscolumns where id = #table_id and colid = #col_id
select #col_id = min(colid) from syscolumns where id = #table_id and colid > #col_id and name <> 'description'
if (#col_id is not null) set #sql = #sql + ','
print #sql
end
set #sql = #sql + ' from MY_table'
exec #sql
Create a view on the table which doesn't include the blob columns
Is there any RDBMS that implements something like SELECT * EXCEPT?
Yes, Google Big Query implements SELECT * EXCEPT:
A SELECT * EXCEPT statement specifies the names of one or more columns to exclude from the result. All matching column names are omitted from the output.
WITH orders AS(
SELECT 5 as order_id,
"sprocket" as item_name,
200 as quantity
)
SELECT * EXCEPT (order_id)
FROM orders;
Output:
+-----------+----------+
| item_name | quantity |
+-----------+----------+
| sprocket | 200 |
+-----------+----------+
EDIT:
H2 database also supports SELECT * EXCEPT (col1, col2, ...) syntax.
Wildcard expression
A wildcard expression in a SELECT statement. A wildcard expression represents all visible columns. Some columns can be excluded with optional EXCEPT clause.
EDIT 2:
Hive supports: REGEX Column Specification
A SELECT statement can take regex-based column specification in Hive releases prior to 0.13.0, or in 0.13.0 and later releases if the configuration property hive.support.quoted.identifiers is set to none.
The following query selects all columns except ds and hr.
SELECT `(ds|hr)?+.+` FROM sales
EDIT 3:
Snowflake also now supports: SELECT * EXCEPT (and a RENAME option equivalent to REPLACE in BigQuery)
EXCLUDE col_name EXCLUDE (col_name, col_name, ...)
When you select all columns (SELECT *), specifies the columns that should be excluded from the results.
RENAME col_name AS col_alias RENAME (col_name AS col_alias, col_name AS col_alias, ...)
When you select all columns (SELECT *), specifies the column aliases that should be used in the results.
and so does Databricks SQL (since Runtime 11.0)
star_clause
[ { table_name | view_name } . ] * [ except_clause ]
except_clause
EXCEPT ( { column_name | field_name } [, ...] )
and also DuckDB
-- select all columns except the city column from the addresses table
SELECT * EXCLUDE (city) FROM addresses;
-- select all columns from the addresses table, but replace city with LOWER(city)
SELECT * REPLACE (LOWER(city) AS city) FROM addresses;
-- select all columns matching the given regex from the table
SELECT COLUMNS('number\d+') FROM addresses;
DB2 allows for this. Columns have an attribute/specifier of Hidden.
From the syscolumns documentation
HIDDEN
CHAR(1) NOT NULL WITH DEFAULT 'N'
Indicates whether the column is implicitly hidden:
P Partially hidden. The column is implicitly hidden from SELECT *.
N Not hidden. The column is visible to all SQL statements.
Create table documentation As part of creating your column, you would specify the IMPLICITLY HIDDEN modifier
An example DDL from Implicitly Hidden Columns follows
CREATE TABLE T1
(C1 SMALLINT NOT NULL,
C2 CHAR(10) IMPLICITLY HIDDEN,
C3 TIMESTAMP)
IN DB.TS;
Whether this capability is such a deal maker to drive the adoption of DB2 is left as an exercise to future readers.
Is there any RDBMS that implements something like SELECT * EXCEPT
Yes! The truly relational language Tutorial D allows projection to be expressed in terms of the attributes to be removed instead of the ones to be kept e.g.
my_relvar { ALL BUT description }
In fact, its equivalent to SQL's SELECT * is { ALL BUT }.
Your proposal for SQL is a worthy one but I heard it has already been put to the SQL standard's committee by the users' group and rejected by the vendor's group :(
It has also been explicitly requested for SQL Server but the request was closed as 'won't fix'.
Yes, finally there is :) SQL Standard 2016 defines Polymorphic Table Functions
SQL:2016 introduces polymorphic table functions (PTF) that don't need to specify the result type upfront. Instead, they can provide a describe component procedure that determines the return type at run time. Neither the author of the PTF nor the user of the PTF need to declare the returned columns in advance.
PTFs as described by SQL:2016 are not yet available in any tested database.10 Interested readers may refer to the free technical report “Polymorphic table functions in SQL” released by ISO. The following are some of the examples discussed in the report:
CSVreader, which reads the header line of a CVS file to determine the number and names of the return columns
Pivot (actually unpivot), which turns column groups into rows (example: phonetype, phonenumber) -- me: no more harcoded strings :)
TopNplus, which passes through N rows per partition and one extra row with the totals of the remaining rows
Oracle 18c implements this mechanism. 18c Skip_col Polymorphic Table Function Example Oracle Live SQL and Skip_col Polymorphic Table Function Example
This example shows how to skip data based on name/specific datatype:
CREATE PACKAGE skip_col_pkg AS
-- OVERLOAD 1: Skip by name
FUNCTION skip_col(tab TABLE, col columns)
RETURN TABLE PIPELINED ROW POLYMORPHIC USING skip_col_pkg;
FUNCTION describe(tab IN OUT dbms_tf.table_t,
col dbms_tf.columns_t)
RETURN dbms_tf.describe_t;
-- OVERLOAD 2: Skip by type --
FUNCTION skip_col(tab TABLE,
type_name VARCHAR2,
flip VARCHAR2 DEFAULT 'False')
RETURN TABLE PIPELINED ROW POLYMORPHIC USING skip_col_pkg;
FUNCTION describe(tab IN OUT dbms_tf.table_t,
type_name VARCHAR2,
flip VARCHAR2 DEFAULT 'False')
RETURN dbms_tf.describe_t;
END skip_col_pkg;
and body:
CREATE PACKAGE BODY skip_col_pkg AS
/* OVERLOAD 1: Skip by name
* NAME: skip_col_pkg.skip_col
* ALIAS: skip_col_by_name
*
* PARAMETERS:
* tab - The input table
* col - The name of the columns to drop from the output
*
* DESCRIPTION:
* This PTF removes all the input columns listed in col from the output
* of the PTF.
*/
FUNCTION describe(tab IN OUT dbms_tf.table_t,
col dbms_tf.columns_t)
RETURN dbms_tf.describe_t
AS
new_cols dbms_tf.columns_new_t;
col_id PLS_INTEGER := 1;
BEGIN
FOR i IN 1 .. tab.column.count() LOOP
FOR j IN 1 .. col.count() LOOP
tab.column(i).pass_through := tab.column(i).description.name != col(j);
EXIT WHEN NOT tab.column(i).pass_through;
END LOOP;
END LOOP;
RETURN NULL;
END;
/* OVERLOAD 2: Skip by type
* NAME: skip_col_pkg.skip_col
* ALIAS: skip_col_by_type
*
* PARAMETERS:
* tab - Input table
* type_name - A string representing the type of columns to skip
* flip - 'False' [default] => Match columns with given type_name
* otherwise => Ignore columns with given type_name
*
* DESCRIPTION:
* This PTF removes the given type of columns from the given table.
*/
FUNCTION describe(tab IN OUT dbms_tf.table_t,
type_name VARCHAR2,
flip VARCHAR2 DEFAULT 'False')
RETURN dbms_tf.describe_t
AS
typ CONSTANT VARCHAR2(1024) := upper(trim(type_name));
BEGIN
FOR i IN 1 .. tab.column.count() LOOP
tab.column(i).pass_through :=
CASE upper(substr(flip,1,1))
WHEN 'F' THEN dbms_tf.column_type_name(tab.column(i).description)
!=typ
ELSE dbms_tf.column_type_name(tab.column(i).description)
=typ
END /* case */;
END LOOP;
RETURN NULL;
END;
END skip_col_pkg;
And sample usage:
-- skip number cols
SELECT * FROM skip_col_pkg.skip_col(scott.dept, 'number');
-- only number cols
SELECT * FROM skip_col_pkg.skip_col(scott.dept, 'number', flip => 'True')
-- skip defined columns
SELECT *
FROM skip_col_pkg.skip_col(scott.emp, columns(comm, hiredate, mgr))
WHERE deptno = 20;
I highly recommend to read entire example(creating standalone functions instead of package calls).
You could easily overload skip method for example: skip columns that does not start/end with specific prefix/suffix.
db<>fidde demo
Related: How to Dynamically Change the Columns in a SQL Query By Chris Saxon
Stay away from SELECT *, you are setting yourself for trouble. Always specify exactly which columns you want. It is in fact quite refreshing that the "feature" you are asking for doesn't exist.
I believe the rationale for it not existing is that the author of a query should (for performance sake) only request what they're going to look at/need (and therefore know what columns to specify) -- if someone adds a couple more blobs in the future, you'd be pulling back potentially large fields you're not going to need.
Temp table option here, just drop the columns not required and select * from the altered temp table.
/* Get the data into a temp table */
SELECT * INTO #TempTable
FROM
table
/* Drop the columns that are not needed */
ALTER TABLE #TempTable
DROP COLUMN [columnname]
SELECT * from #TempTable
declare #sql nvarchar(max)
#table char(10)
set #sql = 'select '
set #table = 'table_name'
SELECT #sql = #sql + '[' + COLUMN_NAME + '],'
FROM INFORMATION_SCHEMA.Columns
WHERE TABLE_NAME = #table
and COLUMN_NAME <> 'omitted_column_name'
SET #sql = substring(#sql,1,len(#sql)-1) + ' from ' + #table
EXEC (#sql);
I needed something like what #Glen asks for easing my life with HASHBYTES().
My inspiration was #Jasmine and #Zerubbabel answers. In my case I've different schemas, so the same table name appears more than once at sys.objects. As this may help someone with the same scenario, here it goes:
ALTER PROCEDURE [dbo].[_getLineExceptCol]
#table SYSNAME,
#schema SYSNAME,
#LineId int,
#exception VARCHAR(500)
AS
DECLARE #SQL NVARCHAR(MAX)
BEGIN
SET NOCOUNT ON;
SELECT #SQL = COALESCE(#SQL + ', ', ' ' ) + name
FROM sys.columns
WHERE name <> #exception
AND object_id = (SELECT object_id FROM sys.objects
WHERE name LIKE #table
AND schema_id = (SELECT schema_id FROM sys.schemas WHERE name LIKE #schema))
SELECT #SQL = 'SELECT ' + #SQL + ' FROM ' + #schema + '.' + #table + ' WHERE Id = ' + CAST(#LineId AS nvarchar(50))
EXEC(#SQL)
END
GO
It's an old question, but I hope this answer can still be helpful to others. It can also be modified to add more than one except fields. This can be very handy if you want to unpivot a table with many columns.
DECLARE #SQL NVARCHAR(MAX)
SELECT #SQL = COALESCE(#SQL + ', ', ' ' ) + name FROM sys.columns WHERE name <> 'colName' AND object_id = (SELECT id FROM sysobjects WHERE name = 'tblName')
SELECT #SQL = 'SELECT ' + #SQL + ' FROM ' + 'tblName'
EXEC sp_executesql #SQL
Stored Procedure:
usp_SelectAllExcept 'tblname', 'colname'
ALTER PROCEDURE [dbo].[usp_SelectAllExcept]
(
#tblName SYSNAME
,#exception VARCHAR(500)
)
AS
DECLARE #SQL NVARCHAR(MAX)
SELECT #SQL = COALESCE(#SQL + ', ', ' ' ) + name from sys.columns where name <> #exception and object_id = (Select id from sysobjects where name = #tblName)
SELECT #SQL = 'SELECT ' + #SQL + ' FROM ' + #tblName
EXEC sp_executesql #SQL
For the sake of completeness, this is possible in DremelSQL dialect, doing something like:
WITH orders AS
(SELECT 5 as order_id,
"foobar12" as item_name,
800 as quantity)
SELECT * EXCEPT (order_id)
FROM orders;
Result:
+-----------+----------+
| item_name | quantity |
+-----------+----------+
| foobar12 | 800 |
+-----------+----------+
There also seems to be another way to do it here without Dremel.
Your question was about what RDBMS supports the * EXCEPT (...) syntax, so perhaps, looking at the jOOQ manual page for * EXCEPT can be useful in the future, as that page will keep track of new dialects supporting the syntax.
Currently (mid 2022), among the jOOQ supported RDBMS, at least BigQuery, H2, and Snowflake support the syntax natively. The others need to emulate it by listing the columns explicitly:
-- ACCESS, ASE, AURORA_MYSQL, AURORA_POSTGRES, COCKROACHDB, DB2, DERBY, EXASOL,
-- FIREBIRD, HANA, HSQLDB, INFORMIX, MARIADB, MEMSQL, MYSQL, ORACLE, POSTGRES,
-- REDSHIFT, SQLDATAWAREHOUSE, SQLITE, SQLSERVER, SYBASE, TERADATA, VERTICA,
-- YUGABYTEDB
SELECT LANGUAGE.CD, LANGUAGE.DESCRIPTION
FROM LANGUAGE
-- BIGQUERY, H2
SELECT * EXCEPT (ID)
FROM LANGUAGE
-- SNOWFLAKE
SELECT * EXCLUDE (ID)
FROM LANGUAGE
Disclaimer: I work for the company behind jOOQ
As others are saying: SELECT * is a bad idea.
Some reasons:
Get only what you need (anything more is a waste)
Indexing (index what you need and you can get it more quickly. If you ask for a bunch of non-indexed columns, too, your query plans will suffer.