We have seven tables in postgres database.t1,t2,t3,t4,t5,t,t7 Each table contains various columns with duplicate product_id number
The product_id number is exists is each table.That means
t1 --> 123(product_id)
t1 --> 123(with various other column data)
t2 --> 123 upto t7
this "123" product id will be existing in each table upto t7.And also,the table will have more than one same product_ids.
Current requirement is to process all product_id's in my server, I need to create intermediate table with unique product ids.
whenever i am updating the tables(t1..t7) the intermediate table has to be triggered to update.
Edit1:
The Intermediate view has to be generated by making all seven tables together.
When I am again importing few more rows from csv/(copy tablename from csvpath...) to these seven tables.The intermediate view also need to be computed and updated by the trigger method
Because this is the frequent operation.Updating the tables from csv and again computing and updating the intermediate view.
So ,How it supposed to write the trigger when updating the seven tables by importing from csv?
Don't create a table, create a view that selects from those tables.
create or replace view all_product_ids
as
select product_id
from t1
union
select product_id
from t2
union
... you get the picture ...
Once you have done that, re-think your database model. By the little information you have provided it sure sounds like your model is not ideal.
Take a look at the PostgreSQL documentation on PL/pgSQL triggers, and especially on this example, your activities looks similar.
Related
I have tables A, B, C with millions of rows each. Tables B and C reference table A. The tables are mainly used for one query with multiple filters but only one of those filters vary between queries. since the constant parameters are adding significant time to the query execution time, I was wondering if there is a way to precompute these params into a new table. I was looking at materialized views but the issue is that the computed type I want will be different from the original column type. To explain I will give an example.
lets say these tables represent a book store database. Table A contains general information and table B contain multiple codes for each book to indicate what categories they fall under such as 406, 678, 252.. . I'm building a query to search for books that only fall under 3 of those categories. The variable here is the keyword search in the discreption of the book. I will always need books under those 3 categories (codes) so these are constants.
What I want to do is create a table where it will have a column that tells me whether a given serial falls under those 3 codes or not. this can be done with a boolean type. I don't want to have to join these table and filter for these 3 codes (and more in the real scenario) for every query.. As I understand materialized views can't have generated fields?
What do you think is a good solution here?
You have multiple options.
Partial Index
PostgreSQL allows you to create an index with a where clause like so:
create index tableb_category on tableb (category)
where category in (406, 678, 252);
Create a view for those categories:
create view v_books_of_interest
as
select tablea.*, tableb.*
from tablea
inner join table b
on tableb.bookid = tablea.bookid
and tableb.category in (406, 678, 252);
Now, your queries can use this book_of_interest rather than books. Frankly, I would start with this first. Query optimization with the right indexes goes a long way. Millions of rows in multiple table are manageable.
Materialized view
create materialized view mv_books_of_interest
as
select tablea.*, tableb.*
from tablea
inner join table b
on tableb.bookid = tablea.bookid
and tableb.category in (406, 678, 252);
with no data;
Periodically, run a cron job (or the like) to refresh it:
refresh materialized view mv_books_of_interest;
Partitioning data
https://www.postgresql.org/docs/9.3/ddl-partitioning.html will get you started. If your team is on-board with table inheritance, great. Give it a shot and see how that works for your use case.
Trigger
Create a field is_interesting in tableA (or tableB, depending on how you want to access data). Create a trigger that checks for a certain criteria when data is inserted in dependencies and then turns the book's flag true/false. That will allow your queries to run faster but could slow down your inserts and updates.
I have two database tables: TABLE 1 and TABLE 2, both have the same number of columns and have the same columns, the problem is that TABLE 2 has a different order of it columns, So I want to arrange the columns of TABLE 2 like they are arranged in TABL1, but i Dont know how to do that.
I'll be very grateful if you can help me.
Re-create Table2 as create table Table2_Temp as select <sequence of fields like Table1> from Table2. After that re-name tables: Table2 -> Table2_Old (or just drop this one), Table2_Temp -> Table2. This is the simplest way and can be realized in any version of DB.
Change order of columns using non-standard SQL language facilities but if it's possible in using a type of DB.
I have a table people with less than 100,000 records and I have taken a backup of this table using the following:
create table people_backup as select * from people
I add some new records to my people table over time, but eventually I want to merge the records from my backup table into people. Unfortunately I cannot simply DROP my table as my new records will be lost!
So I want to update the records in my people table using the records from people_backup, based on their primary key id and I have found 2 ways to do this:
MERGE the tables together
use some sort of fancy correlated update
Great! However, both of these methods use SET and make me specify what columns I want to update. Unfortunately I am lazy and the structure of people may change over time and while my CTAS statement doesn't need to be updated, my update/merge script will need changes, which feels like unnecessary work for me.
Is there a way merge entire rows without having to specify columns? I see here that not specifying columns during an INSERT will direct SQL to insert values by order, can the same methodology be applied here, is this safe?
NB: The structure of the table will not change between backups
Given that your table is small, you could simply
DELETE FROM table t
WHERE EXISTS( SELECT 1
FROM backup b
WHERE t.key = b.key );
INSERT INTO table
SELECT *
FROM backup;
That is slow and not particularly elegant (particularly if most of the data from the backup hasn't changed) but assuming the columns in the two tables match, it does allow you to not list out the columns. Personally, I'd much prefer writing out the column names (presumably those don't change all that often) so that I could do an update.
I have two tables in my database, one is Transactions and the other is TransactionHistories. The latter is essentially an auditing table, whereby a trigger executes on insert, update and delete on Transactions to capture a screenshot of the data.
I am successfully retrieving all of the data stored in the Transactions table where the columns match, but the difficulty comes where I am trying to retrieve data from another table using a foreign key. For instance:
The transaction table has a field "TransactionType_TransactionTypeId", but in the audit table we wish to store its 'name' equivalent as "TransactionTypeName". This needs to be populated from the "TransactionTypes" table, which has the fields "TransactionTypeId" and "Name".
I am struggling to write a query to retrieve this as we wish. I am trying something similar to the following but having little success:
SELECT #TransactionTypeName=Name
FROM TransactionTypes
WHERE inserted.TransactionType_TransactionTypeId=TransactionTypes.TransactionTypeId;
I'm assuming that is a syntactic nightmare. If someone could point me in the right direction I would be extremely grateful!
well to get a name you should do the following
select #TransactionTypeName = TT.Name
from inserted as i
left outer join TransactionTypes as TT on TT.TransactionTypeId = i.TransactionType_TransactionTypeId
but you have to know that inserted table can have more than one row, and you are getting value for only one row.
I have a table ( A ) in a database that doesn't have PK's it has about 300 k records.
I have a subset copy ( B ) of that table in other database, this has only 50k and contains a backup for a given time range ( july data ).
I want to copy from the table B the missing records into table A without duplicating existing records of course. ( I can create a database link to make things easier )
What strategy can I follow to succesfully insert into A the missing rows from B.
These are the table columns:
IDLETIME NUMBER
ACTIVITY NUMBER
ROLE NUMBER
DURATION NUMBER
FINISHDATE DATE
USERID NUMBER
.. 40 extra varchar columns here ...
My biggest concern is the lack of PK. Can I create something like a hash or a PK using all the columns?
What could be a possible way to proceed in this case?
I'm using Oracle 9i in table A and Oracle XE ( 10 ) in B
The approximate number of elements to copy is 20,000
Thanks in advance.
If the data volumes are small enough, I'd go with the following
CREATE DATABASE LINK A CONNECT TO ... IDENTIFIED BY ... USING ....;
INSERT INTO COPY
SELECT * FROM table#A
MINUS
SELECT * FROM COPY;
You say there are about 20,000 to copy, but not how many in the entire dataset.
The other option is to delete the current contents of the copy and insert the entire contents of the original table.
If the full datasets are large, you could go with a hash, but I suspect that it would still try to drag the entire dataset across the DB link to apply the hash in the local database.
As long as no duplicate rows should exist in the table, you could apply a Unique or Primary key to all columns. If the overhead of a key/index would be to much to maintain, you could also query the database in your application to see whether it exists, and only perform the insert if it is absent