Dual data entry checking. Same data is enterd by two persons and now i want to compare this to ensure data quality.
This will depend a lot the measure of quality that you want to use.
As an example, you can just check for the fraction of entries that match exactly,
CASE WHEN COLUMN1 = COLUMN2 THEN '1' ELSE '0' END AS MatchedData
Then you can sum MatchedData and divide by the total number of entries
You can use a correlated subquery for this. First you need to decide which are columns when two records have the identical value in are considered duplicate records. Like you said the records entered by different users so they may have created_by_user column (if exists) different value and all other same. then put them in below sub query below to get the list of duplicate records.
SELECT
*
FROM
MY_TABLE t1
WHERE
ROWID <> (
SELECT
MAX(ROWID)
FROM
MY_TABLE t2
WHERE
t1.col1 = t2.col1
AND
t1.col2 = t2.col2
)
Related
I need to render a query such that every column contains the count of a respective table.
The code I have now is:
SELECT COUNT(table1.Id),
COUNT(table2.Id),
COUNT(table3.Id)
FROM table1,
table2,
table3
WHERE table1.done = 'No' OR
table2.done = 'No' OR
table3.done = 'No' OR
But I need the query to return the same result values as if every table would be counted independently, like:
SELECT COUNT(tableX.Id) FROM tableX WHERE talbeX.done = 'No'
where the 'X' stands for 1,2 or 3.
How can this be achived with SQL?
Thanks beforhand for the help.
Just use a nested sub query, exactly as you have explained it:
SELECT
(SELECT COUNT(table1.Id) FROM table1 WHERE table1.done = 'No') as T1Count,
(SELECT COUNT(table2.Id) FROM table2 WHERE table2.done = 'No') as T2Count,
(SELECT COUNT(table3.Id) FROM table3 WHERE table3.done = 'No') as T3Count,
(SELECT COUNT(tableN.Id) FROM tableN) as TNCount;
This will query the tables independently so you are free to use what ever additional criteria you may need without trying to correlate the results from each query
FROM in this case is not strictly necessary in the outer query as we are not returning rows from any specific table, there is no table that we could specify in the from clause. Each RDBMS has their own convention for these types of queries, MS SQL Server and Oracle are to predominant database engines used in Outsystems
If we did specify a table in FROM then this would return 1 row for every record in that table, which is inefficient and not required. So it is important that we do not include a FROM clause.
Transact-SQL - FROM
The FROM clause is usually required on the SELECT statement. The exception is when no table columns are listed, and the only items listed are literals or variables or arithmetic expressions.
ORACLE - DUAL Table
DUAL is a table automatically created by Oracle Database along with the data dictionary. DUAL is in the schema of the user SYS but is accessible by the name DUAL to all users. It has one column, DUMMY, defined to be VARCHAR2(1), and contains one row with a value X. Selecting from the DUAL table is useful for computing a constant expression with the SELECT statement. Because DUAL has only one row, the constant is returned only once. Alternatively, you can select a constant, pseudocolumn, or expression from any table, but the value will be returned as many times as there are rows in the table.
Update - OP is using Oracle!
After attempting the solution, OP responded that it raised the following error:
Error in advanced query SQL2: ORA-00923: FROM keyword not found where expected
The ORA prefix of this error number indicates that the data store is actually an Oracle implementation, so we need to append the FROM DUAL to the query.
SELECT
(SELECT COUNT(table1.Id) FROM table1 WHERE table1.done = 'No') as T1Count,
(SELECT COUNT(table2.Id) FROM table2 WHERE table2.done = 'No') as T2Count,
(SELECT COUNT(table3.Id) FROM table3 WHERE table3.done = 'No') as T3Count,
(SELECT COUNT(tableN.Id) FROM tableN) as TNCount
FROM DUAL;
I have a a large customer database where customers have been added multiple times in some circumstances which is causing problems. I am able to use a query to identify the records which are an exact match, although some records have slight variations such as different addresses or given names.
I want to query across 10 fields, some records will match all 10 which is clearly a duplicate although other fields may only match 5 fields with another record and require further investigation. Therefore i want to create a results set which has field with a count how many fields have been matched. Basically to create a rating of the likely hood the result is an actual match. All 10 would be a clear dup but 5 would only be a possible duplicate.
Some will only match on POSTCODE and FIRSTNAME which is generally can be discounted.
Something like this helps but as it only returns records which explicitly match on all 3 records its not really useful due the sheer amount of data.
SELECT field1,field2,field3, count(*)
FROM table_name
GROUP BY field1,field2,field3
HAVING count(*) > 1
You are just missing the magic of CUBE(), which generates all the combinations of columns automatically
DECLARE #duplicate_column_threshold int = 5;
WITH cte AS (
SELECT
field1,field2,...,field10
,duplicate_column_count = (SELECT COUNT(col) FROM (VALUES (field1),(field2),...,(field10)) c(col))
FROM table_name
GROUP BY CUBE(field1,field2,...,field10)
HAVING COUNT(*) > 1
)
SELECT *
INTO #duplicated_rows
FROM cte
WHERE duplicate_column_count >= #duplicate_column_threshold
Update: to fetch the rows from the original table, join it against the #duplicated_rows using a technique that treats NULLs as wildcards when comparing the columns.
SELECT
a.*
,b.duplicate_column_count
FROM table_name a
INNER JOIN #duplicated_rows b
ON NULLIF(b.field1,a.field1) IS NULL
AND NULLIF(b.field2,a.field2) IS NULL
...
AND NULLIF(b.field10,a.field10) IS NULL
You might try something like
Select field1, field2, field3, ... , field10, count(1)
from customerdatabase
group by field1, field2, field3, ... , field10
order by field1, field2, field3, ... , field10
Where field1 through field10 are ordered by the "most identifiable/important" to least.
This is as close I've got to what i'm trying to achieve, which will return all records which have any duplicate fields. I want to add a column to the results which indicate how many fields have matched any other record in the table. There are around 40,000 records in total.
select * from [CUST].[dbo].[REPORTA] as a
where exists
(select [GIVEN.NAMES],[FAMILY.NAME],[DATE.OF.BIRTH],[POST.CODE],[STREET],[TOWN.COUNTRY]
from [CUST].[dbo].[REPORTA] as b
where a.[GIVEN.NAMES] = b.[GIVEN.NAMES]
or a.[FAMILY.NAME] = b.[FAMILY.NAME]
or a.[DATE.OF.BIRTH] = b.[DATE.OF.BIRTH]
or a.[POST.CODE] = b.[POST.CODE]
or a.[STREET] = b.[STREET]
or a.[TOWN.COUNTRY] = b.[TOWN.COUNTRY]
group by [GIVEN.NAMES],[FAMILY.NAME],[DATE.OF.BIRTH],[POST.CODE],[STREET],[TOWN.COUNTRY]
having count(*) >= 1)
This query will return thousands of records but I'm generally interested in the record with a high count of exactly matching fields
I have two table one is having all field VARCHAR2 but other having different type for different data.
For Example :
Table One
==========================
Col 1 VARCHAR2 UNIQUE KEY
Col 2 VARCHAR2
Col 3 VARCHAR2
===========================
Table Two
==========================
Col One VARCHAR2 UNIQUE KEY
Col Two TIMESTAMP
Col Three NUMBER
==========================
we are having one mapping table. it denotes which column of Table One has to compare with which column of Table Two.
For Example
Mapping Table
==============================
Table One Table Two
==============================
Col 1 Col One
Col 2 Col Three
Col 3 Col Two
==============================
Now with the help of UNIQUE KEY of TABLE ONE we have to find same row in TABLE TWO and compare rows column by column and get changes in data.
Currently we are using java program for comparing data row by row and column by column and getting changes between data in rows with same UNIQUE KEY. it is working fine but taking too much time as we are having 100000 records in DB.
Now my question is : is there any way i can compare data at SQL level and get changes in data?
You can do it 'manually' with a query like this: It's a lot of work, but there are only three different types of checks you need to do, so it's not very complex:
select
*
from
Table1 t1
full outer join Table2 t2 on t2.ID = t1.ID
where
-- Check ID, either record does not exist in either table.
t1.ID is null or
t2.ID = null or
-- Not nullable field can be easily compared.
t1.NotNullableField1 <> t2.NotNUllableField1 or
-- Nullable field is slightly more work.
t1.NullableField1 <> t2.NullableField1 or
(t1.NullableField1 is null and t2.NullableField1 is not null) or
(t1.NullableField1 is not null and t2.NullableField1 is null)
Another solution is to use MINUS, which is a bit like UNION, only it returns a dataset minus the records in a second dataset:
select * from Table1 t1
MINUS
select * from Table2 t2
This works only one way (which might be fine for your purpose), but you can also combine it with UNION to make it bidirectional.
select
*
from
( select * from Table1
MINUS
select * from Table2)
UNION ALL
( select * from Table2
MINUS
select * from Table1)
The output of both solutions is a bit different.
In the FULL OUTER JOIN query, the IDs will be joined and the values of the matching rows will be displayed next to each other as a single row.
In the MINUS query, the result will be presented as a single dataset. If a record does not exist in either one table, it will be displayed. If a record (ID) exists in both tables, but other fields are different, you will get both rows. So it's a bit harder to compare them.
See: http://www.techonthenet.com/oracle/minus.php
Hey i have a requirement to compare two tables of same structure.
Table1
EmpNO - Pkey
EmpName
DeptName
FatherName
IssueDate
ValidDate
I need to pass the EMPNO as parameter and I need to compare whether any of the column get changes? and return YES OR NO value.
can I able to do that using a PL/SQL Funcation? I was thinking of using the CONCAT in-build function to do that.
I'm trying the below one
Table1Concat = Select CONCAT(Column1.....6) from tbale1 where emp_no= in_empno;
Table2Concat = Select CONCAT(Column1.....6) from tbale2 where emp_no= in_empno;
IF(Table1Concat<>Table2Concat ) THEN return data_changed :='YES';
else data_changed :='NO';
END;
If you only want to detect whether any value is different then ...
select count(*)
from (select * from table1 where emp_no = my_emp_no
union
select * from table2 where emp_no = my_emp_no
)
If it returns 1 then the rows are the same, if it returns 2 then there is a difference.
The columns must be in the same order for this to work, or you'll have to list out all the column names in the order in which they match.
If you wanted to do this in bulk for a great many rows then you'd most likely use a different solution, s do not loop through every emp_no running this code for each one.
For bulk data where all emp_id's are present in both tables, use a query of the form:
select table1.emp_no,
case when table1.column1 = table2.column1 and
table2.column2 = table2.column2 and
table2.column3 = table2.column3 and
...
then 'Yes'
else 'No
end columns_match
from table1
join table2 on table1.emp_no = table2.emp_no
You can insert this result directly into a logging table.
Take care of null values though. "any_value = null" is never true, and "any_value != Null" is also never true, so you might need to add logic to take care of cases where one or both values are null.
I am attempting to use a case statement but keep getting errors. Here's the statement:
select TABLE1.acct,
CASE
WHEN TABLE1.acct_id in (select acct_id
from TABLE2
group by acct_id
having count(*) = 1 ) THEN
(select name
from TABLE3
where TABLE1.acct_id = TABLE3.acct_id)
ELSE 'All Others'
END as Name
from TABLE1
When I replace the TABLE1.acct_id in the THEN expression with a literal value, the query works. When I try to use TABLE1.acct_id from the WHEN part of the query, I get a error saying the result is more than one row. It seems like the THEN expression is ignoring the single value that the WHEN statement was using. No idea, maybe this isn't even a valid use of the CASE statement.
I am trying to see names for accounts that have one entry in TABLE2.
Any ideas would be appreciated, I'm kind of new at SQL.
First, you are missing a comma after TABLE1.acct. Second, you have aliased TABLE1 as acct, so you should use that.
Select acct.acct
, Case
When acct.acct_id in ( Select acct_id
From TABLE2
Group By acct_id
Having Count(*) = 1 )
Then ( Select name
From TABLE3
Where acct.acct_id = TABLE3.acct_id
Fetch First 1 Rows Only)
Else 'All Others'
End as Name
From TABLE1 As acct
As others have said, you should adjust your THEN clause to ensure that only one value is returned. You can do that by add Fetch First 1 Rows Only to your subquery.
Then ( Select name
From TABLE3
Where acct.acct_id = TABLE3.acct_id
Fetch First 1 Rows Only)
Fetch is not accepting in CASE statement - "Keyword FETCH not expected. Valid tokens: ) UNION EXCEPT. "
select name from TABLE3 where TABLE1.acct_id = TABLE3.acct_id
will give you all the names in Table3, which have a accompanying row in Table 1. The row selected from Table2 in the previous line doesn't enter into it.
Must be getting more than one value.
You can replace the body with...
(select count(name) from TABLE3 where TABLE1.acct_id = TABLE3.acct_id)
... to narrow down which rows are returning multiples.
It may be the case that you just need a DISTINCT or a TOP 1 to reduce your result set.
Good luck!
I think that what is happening here is that your case must return a single value because it will be the value for the "name" column. The subquery (select acct_id from TABLE2 group by acct_id having count(*) = 1 ) is OK because it will only ever return one value. (select name from TABLE3 where TABLE1.acct_id= TABLE3.acct_id) could return multiple values depending on your data. The problem is you trying to shove multiple values into a single field for a single row.
The next thing to do would be to find out what data causes multiple rows to be returned by (select name from TABLE3 where TABLE1.acct_id= TABLE3.acct_id), and see if you can further limit this query to only return one row. If need be, you could even try something like ...AND ROWNUM = 1 (for Oracle - other DBs have similar ways of limiting rows returned).