I have a table in which one column represent a name of a table in my db.
TableA
| tableName |
-------------
| table 1 |
| table 2 |
| ....... |
| table n |
What I need to do is to count all the records that are inside each table listed in tableA.
What I managed to do so far is this:
select count(*)
from (
select tableName
from tableA
) tables
but this is wrong because it counts the number of rows in the tableA table, which is not what I need.
The list of table don't have any relationship pretty much so there are no join operations, I just need to add to a counter the number of all records in each table.
Can that be done with a plain sql query? I'm using postgresql but would like to come up with a solution that doesn't depend on any db vendor.
select sum(row_count) as totalRecords
from(
select table_schema,
table_name,
(xpath('/row/cnt/text()', xml_count))[1]::text::int as row_count
from (
select table_name, table_schema,
query_to_xml(format('select count(*) as cnt from %I.%I', table_schema, table_name), false, true, '') as xml_count
from information_schema.tables t
join tableA ta on ta.tablename = t.table_name
)t
) t
A pure SQL solution that works in about any DBMS? Well, as you know you cannot simply query the table names from your control table and use these in a SQL FROM clause, because the table names in the control table are data while the table names in the FROM cause are structure, so to say.
As you should know your database, however, you can use the table names known to you and compare them to the entries in your control table:
select 'TABLE_1', count(*) from table_1 having 'TABLE_1' in (select tablename from tablea)
union all
select 'TABLE_2', count(*) from table_2 having 'TABLE_2' in (select tablename from tablea)
union all
select 'TABLE_3', count(*) from table_3 having 'TABLE_3' in (select tablename from tablea)
union all
select 'TABLE_4', count(*) from table_4 having 'TABLE_4' in (select tablename from tablea)
union all
select 'TABLE_5', count(*) from table_5 having 'TABLE_5' in (select tablename from tablea)
This query must be altered when new tables get added to the database. As adding tables to a database is something that happens seldom if ever, this shouldn't be an issue.
Demo: https://dbfiddle.uk/Dfk9nIFo
Related
I used the search function which brought me to the following solution.
Starting Point is the following: I have one table A which stores all data.
From that table I select a certain amount of records and store it in table B.
In a new statement I want to select new records from table A that do not appear in table B and store them in table c. I tried to solve this with a AND ... NOT IN statement.
But I still receive records in table C that are in table B.
Important: I can only work with select statements, each statement needs to start with select as well.
Does anybody have an idea where the problem in the following statement could be:
Select *
From
(Select TOP 10000 *
FROM [table_A]
WHERE Email like '%#domain_A%'
AND Id NOT IN (SELECT Id
FROM [table_B]))
Union
(Select TOP 7500 *
FROM table_A]
WHERE Email like '%#domain_B%'
AND Id NOT IN (SELECT Id
FROM [table_B]))
Union
(SELECT TOP 5000 *
FROM [table_A]
WHERE Email like '%#domain_C%'
AND Id NOT IN (SELECT Id
FROM [table_B]))
Try NOT EXISTS instead of NOT IN
SELECT
*
FROM TableA A
WHERE NOT EXISTS
(
SELECT 1 FROM TableB WHERE Id = A.Id
)
So Basically the idea here is to select everything from table A that doesnt exists in table B and Insert all that into Table C?
INSERT INTO Table_C
SELECT a.colum1, a.column2,......
FROM [table_A]
LEFT JOIN [table_B] ON a.id = b.ID
WHERE a.Email like '%#domain_A%' AND b.id IS NULL
Thank you guys all for your feedback, from which I learned a lot.
I was able to fix the statement with your help. Above is the statement which is working now with the desired results:
Select Id
From
(Select TOP 10000 * FROM Table_A
WHERE Email like '%#domain_a%'
AND Id NOT IN (SELECT Id
FROM Table_B)
order by No desc) t1
Union
Select Id
From
(Select TOP 7500 * FROM Table_A
WHERE Email like '%#domain_b%'
AND Id NOT IN (SELECT Id
FROM Table_B)
order by No desc) t2
Union
Select Id
From
(SELECT TOP 5000 * FROM Table_A
WHERE Email like '%#domain_c%'
AND Id NOT IN (SELECT Id
FROM Table_B)
order by No desc) t3
I have 9 tables each with value like
Level_1_tab
Code Name
ae1 hdgdgd
ae2 dhdh
level_2_tab
code Name
2 jfjfjf
3 fkfjfjf
similarly level_3_tab , level_4_tab, level_5 table so on and so forth till level_9_tab.
I am inserting the code column into a new table and checking for duplicates.
SELECT
code, name, COUNT(*)
FROM
new_table
GROUP BY
code, name
HAVING
COUNT(*) > 1;
Can i write a query and compare the code column of these 9 tables and check for duplicates ? that all the rows with duplicate code values should be retrieved
You can do a union all of the 9 tables and run your same query on that.
select code, name, count(*) from
(select code, name from table 1 union all
select code, name from table 2 union all
select code, name from table 3 union all
select code, name from table 4 union all
.....)
group by code, name
having count(*) > 1;
Everyone's suggestion of union all is great to build your initial table to look for duplicates in. But you say you already have a temp table with all of the values from the 9 tables which is perfect and another great way of doing it if your dataset isn't huge.
The only step your missing from your description to get the actually duplicate rows is to use your duplicate query above to re-query your temp table and return the rows you want. A great way of doing this is through common table expressions which basically allows you to build a query on top of your other query without another temp table. So use a cte and join back to your temp table.
;WITH CommonTableExpression AS (
SELECT
code, name, COUNT(*)
FROM
new_table
GROUP BY
code, name
HAVING
COUNT(*) > 1;
)
SELECT t.*
FROM
new_table t
INNER JOIN CommonTableExpression c
ON t.code = c.code
AND t.name = c.name
If you want to do it to each of the 9 tables independently rather than to your temp table. Place the duplicates into another temp table and join on it.
SELECT
code, name, COUNT(*)
INTO #Duplicates
FROM
new_table
GROUP BY
code, name
HAVING
COUNT(*) > 1
SELECT
l.*
FROM
leve_1_tab l
INNER JOIN #Duplicates d
ON l.Code = d.Code
AND l.name = d.name
Seeing everyone loves union all here is a way to do it with out temp tables and lots of union all s I wonder which would be a more optimized query though.
;WITH cteAllCodeValues AS (
select code, name from table 1 union all
select code, name from table 2 union all
select code, name from table 3 union all
select code, name from table 4 union all
--.....)
)
, cteDuplicates AS (
SELECT code, name, RecordCount = COUNT(*)
FROM
cteAllCodeValues
GROUP BY
code, name
)
SELECT c.*
FROM
cteDuplicates d
INNER JOIN cteAllCodeValues c
ON d.code = c.code
AND d.name = c.name
The table contains about 300 million rows. I need to select those rows based on two columns.
SELECT *
FROM table_1
WHERE column_1
IN (SELECT column FROM table_2)
AND column_2
IN (SELECT column FROM table_2)
table_1 has 300 million rows. table_2 has 1 million distinct rows.
I also used the exists method:
SELECT *
FROM table_1
WHERE EXISTS (
SELECT 1
FROM table_2
WHERE column=table_1.column_1)
AND EXISTS (
SELECT 1
FROM table_2
WHERE column=table_1.column_2)
But it is too slow. I created index on both columns in table_1 and column in table_2. It would take more than two hours on a 12G RAM Dell server.
Is there any better way to deal with such big table? Can Hadoop solve this problem?
USE THIS :
SELECT *
FROM table_1
INNER JOIN
(SELECT DISTINCT column FROM table_2) tab2_1
ON colum_1 = tab2_1.column
INNER JOIN
(SELECT DISTINCT column FROM table_2) tab2_2
ON colum_2 = tab2_2.column
Hope this will helps you
With a such big database, I would create a materialized view on this query, and then can do a simple SELECT * FROM table_view :
CREATE MATERIALIZED VIEW table_view AS
SELECT * FROM table_1
WHERE column_1 IN (SELECT column FROM table_2)
AND column_2 IN (SELECT column FROM table_2);
And you just have to create a TRIGGER to update this view whenever you add or remove a row from table1 or table2.
If I have two tables A and B. I want to know if this table are same or not ?
i.e check does columns match ?
and does data match ?
SELECT *
FROM
(
(SELECT * FROM Table1
MINUS
SELECT * FROM Table2)
UNION ALL
(SELECT * FROM Table2
MINUS
SELECT * FROM Table1)
)
Query USER_TAB_COLUMNS for each table
SELECT table_name, column_name,
FROM USER_TAB_COLUMNS
WHERE table_name = 'MYTABLE'
And you can compare it for matching columns .
(hawk's answer will do it in one step.)
I am new to sql and I need a little help. I have got a union query with 3 tables. I have got a virtual column so I can tell which table each row is from. I would like to be able to filter this virtual column to show only rows from a specific table.
I am using Microsoft Access and this is what I have so far:
SELECT Table1 as Table_Name, Table1.1
FROM Table1
UNION ALL
SELECT Table2 as Table_Name, Table2.1
FROM Table2
UNION ALL
SELECT Table3 as Table_Name, Table3.1
FROM Table3
UNION ALL
WHERE Table_Name = Form1.TextBox
ORDER BY Table1.1;
I am trying to link this to a listbox on a form and then have some textboxs to filter results.
How about:
SELECT * FROM
( SELECT Table1 as Table_Name, Table1.1
FROM Table1
UNION ALL
SELECT Table2 as Table_Name, Table2.1
FROM Table2
UNION ALL
SELECT Table3 as Table_Name, Table3.1
FROM Table3 ) q
WHERE Table_Name = Forms!Form1!TextBox
ORDER BY Table1.1;
See: http://access.mvps.org/access/forms/frm0031.htm