I have a large query that can't go in memory. This query involves joining a few tables.. We are using a "DIRECT QUERY" to pull the data out of a database.
How would I write a query that has joins in the SQL with Qlikview direct discovery? Is it possible to do?
This guide outlines how to join in direct discovery. You can do a where = join or use the JOIN keyword.
https://help.qlik.com/sense/1.1/en-US/online/Subsystems/Hub/Content/DirectDiscovery/MultiTableSupportDirectDiscovery.htm
Related
I have a Xamarin Forms App that uses Azure Mobile Services offline sync. I want to use a Join to query two tables, but it looks like the use of JOIN isn't supported on IMobileServiceSyncTables. Can I use a regular SQL query? For a single-table query, I'm doing the following:
IEnumerable<Job> jobs = await jobTable
.Where(job => job.JobPackId == jobPackId)
.ToEnumerableAsync();
I can join the Job table to the JobPack table on job.JobPackID == JobPack.Id.
My workaround at the moment is to get all the jobs from the above query, then for each job, query the JobPack table for the item I want. But I don't think it's very efficient, so looking for a better way of doing it.
Can I use a regular SQL query?
AFAIK, you could not use the join query when using IMobileServiceSyncTables and you could not write your own SQL query. Per my understanding, you are using Azure Mobile Services offline sync, you could not achieve your purpose via a single table query. For querying all records from your Job table and the relevant JobPack record, you need to retrieve your expected records with two queries. Or you could install the SQLite client library and write your own SQL query for join operation. Here is a similar issue. Moreover, you could add your feature request here.
I have a question about how to perform database queries against different database systems?
For example, I have a following SQL query string:
SELECT A.F1, A.F2, B.F3, B.F4
FROM TableA A, TableB B
WHERE A.ID=B.ID AND B.ID=xyz
Is there any solution that I can perform the above query when:
TableA is from an Oracle database, TableB is also from an Oracle database from another instance
TableA is from a SQL Server database, TableB is from an Oracle database
TableA is from an Oracle database, TableB is from a SQL Server database
TableA is from a SQL Server database, TableB is also from a SQL Server database from another SQL Server instance
I know that for situation #1 I can use the ORACLE DATABASE LINK feature (also maybe #3). But is there any common solution which can address all of the scenarios above? For instance, maybe there is another scenario that I want to join two tables from MySQL and SQL Server databases.
For coding I am using C#/.NET, any recommendation is welcome, including joining the data in the code.
Thanks!
This is known as a federated query
You can use SQL Servers federation ability (linked servers) and run the query in SQL Server. You can use Oracles federation ability (Oracle heterogeneous services) and run the query in Oracle.
Either way you need to pick a database server, make the other (foreign) database known to that server then execute the query on the server.
Keep in mind you cannot expect good performance from this as in most cases the executing server is loading all the source records locally and joining locally.
Otherwise you need to write your own 'federation' ability in your app. This is non trivial, i.e. data types don't match exactly between vendors so you need to build some smarts. Also if you are joining particularly large tables you'd better make sure your algorithm is optimised or you'll run our of memory. Further to this, federated query ability in existing products (SQL, Oracle, Cognos etc.) is the result of very large companies doing a lot of development.
Can you tell us how many records you expect to join, and are the various source database servers mostly fixed or is this for an ad hoc query tool?
I have a simple Access database I use to create a number of reports. I've linked to a Teradata database server in our organization to pull some additional employee-level details. There is a simple left join on employee number, and all I pull is the name and the role.
The query without the connect takes maybe a minute or so to run and is very quick once loaded. Left joining on the Teradata connection slows everything down to a crawl. It can take 10 minutes or so to run the query through Excel. When the query is loaded in Access, scrolling through it is very slow.
I should note there's no performance issues with the Teradata server. I pull unrelated reports from the same and different tables, with complex joins and the speed is very quick.
I tried creating an even simpler query that does almost noting, and the performance issues are still there. Here is the code:
SELECT EMPL_DETAILS_CURR.NM_PREFX, EMPL_DETAILS_CURR.NM_GIVEN,
MC.DT_APP_ENTRY, MC.CHANNEL_IND
FROM MC LEFT JOIN EMPL_DETAILS_CURR ON MC.EMP_ID = EMPL_DETAILS_CURR.EMP_ID;
There are only 7000 records in MC.
If you are joining data between MS Access tables and Teradata tables the join has to be completed using the Microsoft JET Engine on your local machine. That means the data that exists in your Teradata tables is being brought down to your local machine to so that it can be joined.
If the tables are all on Teradata and accessed via linked tables in MS Access the join may still be occurring locally. I would suggest running the query as an ODBC Direct (I forget the exact term) query so that the SQL is passed on to Teradata to be executed and the results returned to MS Access when the query completes.
I have two different databases (on same server) and i want to join tables across databases. I am using Hibernate, is it possible to create a query in hibernate which can join two tables in those databases?
Hibernate will create an SQL query for your HQL or Criteria query and this SQL will be sent through jdbc to the database. This means that hibernate does not support for what you are trying to do.
However, you can achieve the same result in some cases. Some databases give you the option to create an alias for a table that resides in a different database. So you will be able to write an SQL query that joins the two tables and execute it on the database.
We are doing that with DB2.
If you can do that, it depends on your database.
I guess, that it would impossible if you have two different databases (example DB2 and MySQL) but if both databases are of the same vendor, then maybe it's achievable.
You should try to find more info in you database server's documentation.
Here's the situation: we have an Oracle database we need to connect to to pull some data. Since getting access to said Oracle database is a real pain (mainly a bureaucratic obstacle more than anything else), we're just planning on linking it to our SQL Server and using the link to access data as we need it.
For one of our applications, we're planning on making a view to get the data we need. Now the data we need is joined from two tables. If we do this, which would be preferable?
This (in pseudo-SQL if such a thing exists):
OPENQUERY(Oracle, "SELECT [cols] FROM table1 INNER JOIN table2")
or this:
SELECT [cols] FROM OPENQUERY(Oracle, "SELECT [cols1] FROM table1")
INNER JOIN OPENQUERY(Oracle, "SELECT [cols2] from table2")
Is there any reason to prefer one over the other? One thing to keep in mind: we are on a limitation on how long the query can run to access the Oracle server.
I'd go with your first option especially if your query contains a where clause to select a sub-set of the data in the tables.
It will require less work on both servers, assuming there are indices on the tables in the Oracle server that support the join operation.
If the inner join significantly reduces the total number of rows, then option 1 will result in much less network traffic (since you won't have all the rows from table1 having to go across the db link
What hamishmcn said applies.
Also, SQL Server doesn't really know anything about the indexes or statistics or cache kept by the oracle server. Therefore, the oracle server can probably do a much more efficient job with the join than the sql server can.