I have materialized view manually partitioned by month
create materialized view MV_MVNAME_201001 as
select MONTH_ID, AMOUNT, NAME
from TABLE_NAME_201001
201002, 201003, ..., 201112, 2012, 2009
Now i need query from these views, take only the required views.
Is it possible, without involving the client side?
example query
select AMOUNT, NAME
from (
--union all mview
)
where month_id >= 201003
and month_id < 201101
should look only to the MV_MVNAME_201003 .. MV_MVNAME_201012
The materialized view is "materialized". It is a phisical table with data within it.
The query that produce the materialized view is used only when you refresh data, not on querys.
Oracle doesn't know where data came from(in your case - a union from several distinct tables), unless you specify it somehow, for example - a column.
But in your specific case you have the column month_id, on witch you can partition the table.
When you specify the month or range of months, it will scan only the correspondent partitions.
UDPATE:
Now I understand better your question, but I cannot give you a better answer. Your question has nothing to do with mviews. Mviews can be tables. Your problem is the same. You want to select only from some tables, dynamic. For this was created partitioning. Probably old dogs know a trick...
Related
I have a database of around 90 schemas. In each of these schemas, I go to "Materialized Views" and go to a one of the views called, "product_visitor_view" and I create a SELECT script and I write this script and run it and see the results:
SELECT priority, count(*)
FROM ag_modem_01.product_visitor_view
group by priority;
However, I cannot do this for all 90 around schemas. Is there a way I can do this for all schemas and the results would be shown for each schema in a page and how can I do this?
Thank you in advance.
You can prepare the SQL for each schema using below query.
The idea is to instead of manually writing all the query, you can use the system table pg_matviews which contains information about materialized views.
Once you have the list, just need to do union all between all the rows.
select
string_agg('select '''||schemaname||''' as schema,priority,count(*) from '||schemaname||'.'||matviewname
||' group by priority '||chr(10),' Union All '||chr(10)
)
from pg_matviews
where matviewname='product_visitor_view'
and ispopulated=true; -- to filter out Views which has not been populated
Take the output from this query and run it in the Querytool.
I am trying to create a materialized view on table which will have latest data.
The query looks liks this
Create Materialized view t1_latest as
select c1,c2,dt from t1
join
(select max(dt) maxdt from t1) t2
ON t1.dt = t2.maxdt
dt being the date field.
Now as we know Materialized view does not allow subquery or window function. Is there a way to rewrite the query to create the Materialized view with latest date. latest date cannot be considered as current_date or hardcoded.
Another approach is to create a view with join and then create the Materialized view on top of that. But the problem there is we will loose the advantage of Materialized view being calculated before hand.
Any suggestion.
You'd actually gain exactly the same amount of performance gain if your table was clustered by dt and you just put a standard view over it with the same logic you have above. This will prune the table in the same manner as creating a materialized view would do.
If the table is already clustered on something else, then creating a simple materialized view with the dt as your cluster key will provide the same benefits, but also have the benefits of the ability to simply query the base table and have the query optimizer assist with choosing the best pruning option:
https://docs.snowflake.com/en/user-guide/views-materialized.html#how-the-query-optimizer-uses-materialized-views
Edited per comment:
Not sure I understand why you can't use a task/stream, but what if you did something a bit different with task & stream.
Create a stream...when the stream has data, you could execute a task that executes a stored procedure.
The Stored Procedure would evaluate the table to see if there were
more than 1 distinct value for dt. If there is, it would delete the older data from the table.
In Snowflake, the delete would essentially just be a metadata operation, since all of the micropartitions would contain the same date based on how you describe the data being loaded.
I have a view which looks like this:
CREATE VIEW My_View AS
SELECT * FROM My_Table UNION
SELECT * FROM My_External_Table
What I have found is that performance is very slow when ordering the data which I need to do for pagination. For example the following query takes almost 2 minutes despite only returning 20 rows:
SELECT * FROM My_View
ORDER BY My_Column
OFFSET 20 ROWS FETCH NEXT 20 ROWS ONLY
In contrast the following (useless) query takes less than 2 seconds:
SELECT * FROM My_View
ORDER BY GETDATE()
OFFSET 20 ROWS FETCH NEXT 20 ROWS ONLY
I cannot add indexes to the view as it is not SCHEMABOUND and I cannot make it SCHEMABOUND as it references an external table.
Is there any way I can improve the performance of the query or otherwise get the desired result. All the databases involved are AzureSQL.
If all items are unique in My_table and My_external_table using OUTER UNION would help you to improve the performance.
And adding an index to table would help to run your query faster.
You can't really get around the order by so I don't think there is anything you can do.
I'm a bit surprised the order by getdate() works, because ordering by a constant does not usually work. I imagine it is equivalent to order by (select null) and no ordering takes place.
My recommendation? You probably need to replicate the external table on the local system and have a process to create a new local table. That sounds complicated, but you may be able to do it using a materialized view. However this works with the "external" table depends on what you mean by "external".
Note that you will also want an index on my_column to avoid the sort.
What is the best practice to use aggregation function with a view in sql query with a huge data
select name , sum(value) from v_transactions Group by name
this view have 150,000 Record
If the view is presenting 150k rows, but take 5 minutes to aggregate them, the problem is the performance of the view not the aggregation that you're doing.
SELECT * FROM v_transactions will probably take almost exactly the same amount of time
It is possible that the view was only designed to be queried for a small number of rows
FROM v_transaction WHERE some_key = some_value
In which case, trying to query the view for all of its rows may be "misusing" it.
It is possible that the view is doing work unrelated to your query
Complicated calculations and joins to generate other fields
Again, if the view exists specifically to generate fields that you're not using, your use of the view may be considered "misuse".
In any case, as it's a view there will be an alternative source to aggregate from. The could be other views or tables that v_transactions derives it's data from.
Ask around, find out what views, tables and functions you have access to. You may or may not have access to what you want, but until you investigate you're not going to find out.
Another possibility is that the name field is the expensive calculation. Is there another field you can group by instead? Some id or key that corresponds to the name? This would give you several options to test for performance...
SELECT some_id, SUM(value) FROM v_transactions GROUP BY some_id
SELECT some_id, name, SUM(value) FROM v_transactions GROUP BY some_id, name
SELECT some_id, MAX(name), SUM(value) FROM v_transactions GROUP BY some_id
etc, etc
I'm running on SQL Server 2014, and have a Customer table related to a table with orders. The Order table has e.g 500.000 rows - 100.000 for each of 5 customers. So when looking for all orders for a specific date for a specific customer, the SQL would be
Select *
from Orders
Where idCustomer = 1
and OrderDate = '20160225'
This query should look through 500.000 records to find relevant orders.
Would it be better for performance to create 5 views, a view for each customer and then querying the view:
Select *
from View_Orders_Customer1
Where OrderDate = '20160225'
Or do I have to create 5 tables Order_Customer1, Order_Customer2 etc. - instead ??
I don't recommend create view for each customer .. but you can add Index in your table http://www.w3schools.com/sql/sql_create_index.asp
Also you can add Index on your view to enhance your select performance
https://technet.microsoft.com/en-us/library/ms187864(v=sql.105).aspx
I have table contains more than 700,000 records and applying indexing in table and view helped me very much
For same data model, use same table(s). More than one table or view is not good design,
You should look at execution plan, and create necessary indexes,
After performance tuning with table indexes, you can use Indexed Views,
https://msdn.microsoft.com/en-us/library/ms191432(v=sql.120).aspx