Deleting billion of rows in an oracle db - sql

Question

I have to delete a lot of rows from our log db.
Currently, the table holds about 6.3 billion of rows that I want to delete.
First of all, I'm doing it right now (until I have a better solution) in million increments via Python, which takes around ~600 seconds on average per million.
I've tried to copy the data we need (which is around 3% of all data in this table) in another table, but an error occurred because the data amounts too much, it seems, to insert that via select into another table.
(my plan was to get the data we need, insert it into another table, and just delete the old one and rename the new table)
I found this one too:
Delete specific rows from a table with billions of rows
Is that the best approach for me? (my procedure writing skills are nearly non existent, so I'm a little scared to try this, that's why I'm doing it with Python right now)
Thank you for any advice
Edit:
My Python Code:
import time
from helperfunc import dbExecAny
def delete_lg_log():
dbExecAny("delete from inspire.lg_log WHERE lg_wp_type != 'Matching.SAP.Zahlstatus' and rownum <= 1000000")
i = 0
while i <= 6200:
i += 1
print(f"Process loop {i}")
start = time.time()
delete_lg_log()
end = time.time()
print(start - end)
delete function:
def dbExecAny(stmt, db_name=""):
try:
if db_name.lower() == "cobra":
a_cur = cobra_conn.cursor()
else:
a_cur = conn.cursor()
if "drop" in stmt.lower() in stmt.lower() or "truncate" in stmt.lower() or "where" not in stmt.lower():
print("Invalid statement: " + stmt)
return []
a_cur.execute(stmt)
return a_cur.execute(stmt)
except Exception as e:
print("Coult not execute statement " + stmt + ", error: " + str(e))
return []
finally:
conn.commit()

Answer 1

The quickest way to save your 3% of data that should remain is to do a ctas (Create Table As Select) from your source table with no logging option. After that truncate your source table, drop it and rename the newly created table.
Don’t forget about things like dependencies like indexes, triggers, constraints and privileges.
This no logging only works if the database is not in force logging mode because it needs to keep a standby database in sync.