Application connected to MS SQL Server will create views where a single row result is an analysis including aggregations of 1-10k records. The applicable criteria across the resulting view will have dozens to tens of thousands of results. The view+criteria will then be ordered by some column (user specified) in the view which are most likely to be the aggregated columns. Response times are expected to degrade quickly when aggregated column is used for ordering.
A while back, this problem was solved pretty easily (in Oracle 9i) with materialized views.
Any ideas on how to get a similar solution in MS SQL Server 2005.
You can use Indexed views for this.
Read here for SQL 2005: http://msdn.microsoft.com/en-us/library/dd171921.aspx
Read here for SQL 2008: http://msdn.microsoft.com/en-us/library/dd171921.aspx
Materialized views are not same as indexed views. MS SQL server indexed views have multiple limitations such as use of outer joins, aggregates and common table expressions.
Related
I am working on PowerBi and use SQL server as database. I used views or direct tables as source to PowerBi . My views are simple select queries with simple joins. I am not finding any scope for query optimizations. Query execution takes time in SQL and table has millions of data increasing day by day.
Now I am thinking to use impala as well as SQL server. I am getting clean data from Rapidminer. I didn't use impala before. So I have some doubts. Please answer if you can. I have zero knowledge of impala.
Can we create connection between rapid miner and impala? then what will be the steps? google give me some steps which is difficult to understand.
Can we create connection between impala and sql?
Can we create view on impala and create joins in views? I know we can create view as well as joins in impala. But my question is can we create it together?
suppose SQl and impala connection is made then suppose I have one table from impala and one table from sql server management studio. can I join both tables in impala? for this can we create connection between impala and sql server management studio?
5.Can I use all tables or views created in sql to impala (after making connection between sql and impala). That means my tables or views are in sql. but I am fetching data in impala.
All tables stored in sql server. can I do join operation on these tables in impala.
7.Can I make views in impala using tables which are stored in sql
8.Can I create all tables in impala and do etl operation like sum, add, dateadd in impala
9.Can I create all tables in impala and do etl operation like sum, add, dateadd in power query
10.Can I create views from sql and put it in impala table. and use in power query
Can I create all tables and views with joins in impala?
12.How can I optimise my query in sql and if I run same query for same data in impala then my execution time will reduce or not?
My SQL query is like this
create view as test
select * from table a
inner join table b on a.id=b.id
inner join table c on b.name=c.name
go
output is 3000000 row. increasing day by day
also instead of using view I use table directly. but execution time is not decreasing.
I am using a SQL database and MS Access 2019 as the front end. The SQL database tables are linked to the Access db using an ODBC connection.
All my queries (they have multiple joined linked tables) run just fine, but as soon as I add a join to a table stored in the Access app (for example, a small table just for mapping values) the query will slow to a crawl. Doesn't matter if the joined fields are indexed or what type of join I'm using.
If anyone has seen this behaviour and found a solution I would much appreciate hearing it.
Joining tables from two separate databases requires the client app to retrieve both tables in their entirety in order to determine the rows needed. That's why it's slow.
If your Access table is small, try using a stored procedure on the SQL side with the data from Access moved to a temporary table. (Or better yet, move the Access table to SQL).
I created an indexed view that joined a number of tables to get better performance, but when I use the indexed view, the performance is not better than before. When I survey the execution plan, I don't see any change between the view and the indexed view on these joined tables.
If you are not using sql server enterprise edition (see feature Automatic use of indexed views by query optimizer), sql server query optimizer is not taking in account indexed views..
In other versions you can make sql server use it- With (NoExpand):
Select col1, col2, col3
From dbo.vw_MyView With (NoExpand)
I need to pull a large amount of data from various tables across a line that has very low bandwidth. I need to minimize the amount of data that gets sent too and fro.
On that side is a Sybase database, on this side SQL Server 2008.
What I need is to pull all the tables from the Sybase database that have to do with this office. Lets say I have the following tables as an example:
Farm
Tree
Branch
etc.
(one farm has many trees, one tree has many branches etc.)
Lets say the "Farm" table has a field called "CountryID", and I only want the data for where CountryID=12. The actual table structures I am looking at are very complex (and I am also not very familiar with them) so I want to try to keep the queries simple.
So I am thinking of setting up a series of views:
CREATE VIEW vw_Farm AS
SELECT * from Farm where CountryID=12
CREATE VIEW vw_Tree AS
SELECT * from Tree where FarmID in (SELECT FarmID FROM vw_Farm)
CREATE VIEW vw_Branch AS
SELECT * from Tree where BranchID in (SELECT BranchID FROM vw_Branch)
etc.
To then pull the actual data across I would then do:
SELECT * from vw_Farm into localDb.Farm
SELECT * from vw_Tree into localDb.Tree
SELECT * from vw_Branch into localDb.Branch
etc.
Simple enough to set up. I am wondering how this will perform though? Will it perform all the SELECT statements on the Sybase side and then just send back the result? Also, since this will be an iterative process, is it possible to index the views for subsequent calls?
Any other optimisation suggestions would also be welcome!
Thanks
Karl
EDIT: Just to clarify, the views will be set up in SQL Server. I am using a linked server using Sybase ASE to set up those views. What is worrying me in particular is whether the fact that the view is in SQL Server on this side and not on Sybase on that side will mean that for each iteration the data from the preceeing view will get pulled across to SQL Server first before the calculations get executed. I want Sybase to do all the calcs and just pass the results across.
It's difficult to be certain without testing, but my somewhat-relevant experience (using linked servers to platforms other than Sybase, and on SQL Server 2005) has been that using subqueries (such as your code for vw_Tree and vw_Branch) more or less guarantees that SQL Server will pull all the data for the outer table into a local temp table, then match it to the results of the inner query.
The problem is that SQL Server has no access to the linked server's table statistics, so can make no meaningful decisions about how to optimise the query.
If you want to be sure to have the work done on the Sybase server, your best bet will be to write code (could be views or stored procedures) on the Sybase side and reference them from SQL Server.
Linked server connections are, in my experience, not particularly resilient over flaky networks. If it's available, you could consider using Integration Services rather than linked-server queries - but even that may not be much better. You may need to consider falling back on moving text files with robocopy and bcp.
Does MySQL have an equalavent to SQL Servers "indexed view" functionality?
Is a view faster than a simple query?
What I'm specifically looking for is a way for MySQL to create a "view" that will return results faster than simply performing the underline view's query/sql.
An Indexed View is SQL Server terminology for a materialized view, which MySQL does not support.
You can either:
re-create a temporary table, populated with the desired columns, at a given interval
use an actual table, populated and indexed though again - there'd have to be a process to keep the data and indexes current