Topic : Data Cleaning - Checking for outlier - Out of pattern
I'm trying update custID from value 'A123' to '123', '22A4' to '224' I only want to keep integer inside custID, I don't want the custID contain any non-integer character (A to Z and a to z)
CREATE TABLE customer (
custID VARCHAR2(10) PRIMARY KEY,
custName VARCHAR2(30)
);
INSERT INTO customer(custID,custName) VALUES ('A123','Angel');
INSERT INTO customer(custID,custName) VALUES ('22A4','Chris');
INSERT INTO customer(custID,custName) VALUES ('2333','Chris');
UPDATE customer
SET custID = -- I want to change 'A123' to '123', '22A4' to '224'
WHERE ;
Use this...
UPDATE customer
SET custID = REGEXP_REPLACE(custID, '[^0-9]+', '')
WHERE ;
or try this...
UPDATE customer
SET custID = REGEXP_REPLACE(custID, '[^[:digit:]]+', '')
WHERE ;
UPDATE customer
SET custid = REGEXP_SUBSTR ( custid , '[0-9]*' )
WHERE
REGEXP_SUBSTR Oracle documentation
Related
I have table of transaction which contains a column transactionId that has values like |H000021|B1|.
I need to make a join with table Category which has a column CategoryID with values like H000021.
I cannot apply join unless data is same.
So I want to split or remove the unnecessary data contained in TransctionId so that I can join both tables.
Kindly help me with the solutions.
Create a computed column with the code only.
Initial scenario:
create table Transactions
(
transactionId varchar(12) primary key,
whatever varchar(100)
)
create table Category
(
transactionId varchar(7) primary key,
name varchar(100)
)
insert into Transactions
select'|H000021|B1|', 'Anything'
insert into Category
select 'H000021', 'A category'
Add computed column:
alter table Transactions add transactionId_code as substring(transactionid, 2, 7) persisted
Join using the new computed column:
select *
from Transactions t
inner join Category c on t.transactionId_code = c.transactionId
Get a straighforward query plan:
You should fix your data so the columns are the same. But sometimes we are stuck with other people's bad design decisions. In particular, the transaction data should contain a column for the category -- even if the category is part of the id.
In any case:
select . . .
from transaction t join
category c
on transactionid like '|' + categoryid + |%';
Or if the category id is always 7 characters:
select . . .
from transaction t join
category c
on categoryid = substring(transactionid, 2, 7)
You can do this using query :
CREATE TABLE #MyTable
(PrimaryKey int PRIMARY KEY,
KeyTransacFull varchar(50)
);
GO
CREATE TABLE #MyTransaction
(PrimaryKey int PRIMARY KEY,
KeyTransac varchar(50)
);
GO
INSERT INTO #MyTable
SELECT 1, '|H000021|B1|'
INSERT INTO #MyTable
SELECT 2, '|H000021|B1|'
INSERT INTO #MyTransaction
SELECT 1, 'H000021'
SELECT * FROM #MyTable
SELECT * FROM #MyTransaction
SELECT *
FROM #MyTable
JOIN #MyTransaction ON KeyTransacFull LIKE '|'+KeyTransac+'|%'
DROP TABLE #MyTable
DROP TABLE #MyTransaction
Create table query
CREATE TABLE ID_TAB (
ID VARCHAR2(20),
ID_VALUE VARCHAR2(20), FLAG VARCHAR2(20)
);
CREATE TABLE FACT_TABLE (
ID VARCHAR2(20),
VALUE VARCHAR2(20),
NAME VARCHAR2(100)
);
Insert Query
INSERT INTO ID_TAB VALUES('100','ABC','N');
INSERT INTO ID_TAB VALUES('120','ABC','Y');
INSERT INTO FACT_TABLE VALUES('100','MAX','ORANGE');
My objective is to update the fact table 'ID' column to 120 because it has the FLAG value as 'Y'
My original table has 50 million records.
How can we write the query in merge or using Update?
You could try the following (I didn't try it though):
UPDATE fact_table
SET fact_table.id = (SELECT yes_tab.id
FROM id_tab no_tab
JOIN id_tab yes_tab
ON no_tab.id_value = yes_tab.id_value
AND yes_tab.flag = 'Y'
WHERE no_tab.id = fact_table.id)
WHERE EXISTS (SELECT *
FROM id_tab
WHERE id_tab.id = fact_table.id
AND id_tab.flag = 'N');
The WHERE EXISTS makes sure you update only the elements with the flag in the corresponding ID_TAB set to 'N'.
The query for the SET searches the ID of the corresponding element having the flag set to 'Y'.
To get a performant solution (i.e. avoid the join completely) you may switch to a dynamic SQL.
I extended you example a bit to have more updated keys as follows.
Note that there is maximal one new value per ID_VALUE, but more old value with N flag are allowed, which all must be updated.
INSERT INTO ID_TAB VALUES('100','ABC','N'); -- old key
INSERT INTO ID_TAB VALUES('110','ABC','N'); -- old key
INSERT INTO ID_TAB VALUES('120','ABC','Y'); -- new key
INSERT INTO ID_TAB VALUES('200','EFG','N'); -- old key
INSERT INTO ID_TAB VALUES('210','EFG','N'); -- old key
INSERT INTO ID_TAB VALUES('220','EFG','Y'); -- new key
INSERT INTO FACT_TABLE VALUES('100','MAX','ORANGE');
INSERT INTO FACT_TABLE VALUES('110','MIN','ORANGE');
INSERT INTO FACT_TABLE VALUES('200','MAX','APPLE');
INSERT INTO FACT_TABLE VALUES('210','MIN','APPLE');
INSERT INTO FACT_TABLE VALUES('220','NEW','APPLE');
commit;
The UPDATE statements reflects all th eupdate options, e.g. the keys 100 and 110 should be chnaged to 200.
This leads to the following update
update FACT_TABLE
set ID = case
when ID in ('100','110') then '200'
when ID in ('200','210') then '220'
end
where ID in ('100','110','200','210');
Note that the where condition consist of all the keys with flag N and the case statement is produced per ID_VALUE mapping all keys with the flag N to the (only one) key with flag Y.
This makes the generation of the UPDATE statement an easy little task of a query on the table ID_TAB. See the query below. For the list creation the function LISTAGG is used. The main parts are commented in the query.
with old_keys as (
select /* get list of all old values */
listagg(''''||ID||'''',',') within group (order by ID) old_keys
from ID_TAB where flag = 'N'),
case_stmt as (
select ID_VALUE,
listagg(case when flag = 'N' then ''''||ID||'''' end,',') within group (order by ID) old_keys,
max(''''||case when flag = 'Y' then ID end||'''') new_key
from ID_TAB
group by ID_VALUE),
case_stmt2 as (
select
'when ID in ('||OLD_KEYS ||') then ' ||NEW_KEY ||' /* update for '|| ID_VALUE || ' */' case_when, ID_VALUE
from case_stmt),
case_stmt3 as ( /* concatenate CASE WHEN */
select listagg(CASE_WHEN,chr(13)) within group (order by ID_VALUE) case_when from case_stmt2)
select
'update FACT_TABLE
set ID = case
'||
case_when ||
'
end
where ID in ('||
(select old_keys from old_keys)||')' as update_stmt
from case_stmt3
On the sample data this SQL string is returned
update FACT_TABLE
set ID = case
when ID in ('100','110') then '120' /* update for ABC */
when ID in ('200','210') then '220' /* update for EFG */
end
where ID in ('100','110','200','210')
I have a table that record events as it happens and as such could record same ID multiple times. I want to return single row for each unique RefID as 'Y' is substituted for 'N' in relevant columns. Below is a mock-up
DECLARE #Ref TABLE
(
RefID INT
, InvoiceNo INT
, InvoicedDate Date
, CustID INT
, PaidOnTime CHAR(1)
, Paidlate CHAR(1)
, PaidByCash CHAR(1)
, PaidByCard CHAR(1)
)
INSERT INTO #Ref VALUES
(23,50,'22-jun-2015', 11,'Y','N','Y','N')
, (23,50,'22-jun-2015', 11,'Y','N','N','Y')
, (27,11,'12-Aug-2015', 11,'Y','N','N','Y')
, (27,11,'22-Aug-2015', 11,'N','Y','N','Y')
, (45,67,'28-jun-2015', 11,'N','Y','Y','N')
, (45,67,'28-jun-2015', 11,'N','N','N','Y')
, (48,51,'18-jun-2015', 11,'Y','N','Y','N')
SELECT * FROM #Ref --would return values like so:
For example, RefID of 23 should be "23,50,22/06/2015,11,Y,N,Y,Y"
Group by the columns you want to be unique and use max() to get the highest value for every group (since Y is alphabetically higher than N)
SELECT RefID, InvoiceNo, InvoicedDate, CustID,
max(PaidOnTime), max(Paidlate), max(PaidByCash), max(PaidByCard)
FROM #Ref
GROUP BY RefID, InvoiceNo, InvoicedDate, CustID
How can I use an auto-generated column to populate another column during an INSERT statement?
Long story short: we are reusing a database table and an related ASP page to display completely different data than was originally intended.
I have a table similar in structure to the following. It's structure is out of my control.
ID int NON-NULL, IDENTITY(1,1)
OrderNo varchar(50) NON-NULL, UNIQUE
More ...
The table has been repurposed and we are not using the OrderNo column. However, it's NON-NULL and UNIQUE. As dummy data, I want to populate it with the row's ID column.
I have the following SQL so far, but can't work out how to use the row's generated ID.
INSERT INTO MyTable (OrderNo, More)
OUTPUT INSERTED.ID
VALUES (CAST(ID AS varchar(50)))
This just gives:
Msg 207, Level 16, State 1, Line 3
Invalid column name 'ID'.
Here's a solution using the OUTPUT clause. Unfortunately, you won't be able to do it in a single statement.
CREATE TABLE Orders (
ID int not null identity(1,1),
OrderNo varchar(50) not null unique
)
CREATE TABLE #NewIDs ( ID int )
INSERT Orders (OrderNo)
OUTPUT INSERTED.ID INTO #NewIDs
SELECT 12345
UPDATE o
SET o.OrderNo = i.ID
FROM Orders o
JOIN #NewIDs i
ON i.ID = o.ID
SELECT * FROM Orders
One option would be:
create trigger YourTable_Trigger
on YourTable
INSTEAD OF INSERT
as begin
INSERT INTO YourTable (OrderNo, AnotherField)
SELECT 0, AnotherField FROM Inserted
UPDATE YourTable SET OrderNo = SCOPE_IDENTITY() WHERE ID = SCOPE_IDENTITY()
end;
And here is the Fiddle.
Good luck.
CREATE TABLE customer
(id varchar,
sal int);
insert into customer values (1,500);
insert into customer values (2,500);
insert into customer values (3,500);
insert into customer values (4,500);
select * from customer;
update customer set id = 'sss'+id;
select * from customer;
My ideone link
I missed the 'sss' word before the id while creating, I thought update would do it, but how?
update customer set id = 's' || cast(id as varchar(10));
Works in your IdeOne environment.