I know in Impala (and other databases) I can run both of the following:
SHOW DATABASES
SHOW TABLES
I also know I can add optional LIKE or IN arguments e.g. to show me all the tables in database Bananas I could write:
SHOW TABLES IN Bananas
What I really want to know is a way of returning all the tables in the databases without having to recurse through (also showing database name and table name in separate fields.
I'll be running this via impala shell so I'd have to first return back all the database names and then produce a script line per database to give me the tables.
It's not a problem to do this as such, I just can't help wondering there must be a better way to end up with:
Unfortunately not yet. Impala will eventually support this by exposing tables for schema metadata (e.g. ANSI INFORMATION_SCHEMA), and IMPALA-1761 tracks that feature request.
Related
I have to copy data from one table to another, the tables are held in two different databases within Azure. I did a quick search for answers to this and whilst a query seems fairly straight forward i.e.
INSERT INTO table1 (make, model, type, serial)
SELECT the_make, the_model, the_type, ref_no
FROM database2.dbo.table2
I encountered issues because I'm using Azure.
Msg 40515, Level 15, State 1, Line 16 Reference to database and/or
server name in 'database2.dbo.table2' is not supported in this version of
SQL Server.
The above issue led me to the Cross-Database Queries articles. My requirements are a little more complicated than some of the scenarios provided and I need some help in making it work.
I also need to convert some columns such as reg_no which is a 'string' to an 'int' and then copy the value to the 'serial' column.
My question is, what the best way to create a script for this that allows me to reference both databases without any errors, copy the data and convert the columns at the same time? I tried the simple way of exporting data and importing it, editing the mappings for the columns, it wasn't that good I found and was causing problems all over the place.
Any guidance is appreciated on this.
You're getting this error because there's no linked server by default. You'll need to add it, in order to access the secondary db server. Here's a link about how to do it:
https://www.sqlshack.com/create-linked-server-azure-sql-database/
In terms of the transformation. It depends on many factors e.g. amount of rows, frequency, etc..
Usually the best alternative is by using an external tool (ETL) such as SSIS / Azure Data Factory because you can schedule it's execution and get the status of each execution.
#GregGalloway was able to answer the question I should have asked. I am adding a more concise question here, while maintaining the original lengthy text
How do I use a table valued function as the query for a partition, when the function is in separate database from my fact and referenced dimensions?
Overview: I am building a SSAS multidimensional cube that is built off of a single fact table in our application's data warehouse, and want to use the result set from a table valued function as my fact table's partition query. We are using SQL Server (and SSAS) 2014
Condition: For each environment (Dev,Tst,Prd) there are 2 separate databases on the same server, one for the application data warehouse [DW_App], the other for custom objects [DW_Custom]. I cannot create any objects in [DW_App], but have a lot of freedom in [DW_Custom]
Background info: I have not been able to find much information on using a TVF and partitions in this way. My thinking is that it will help streamline future development by giving me a single place to update the SQL if/when I modify the fact table.
So in testing out my crazy idea of using a TVF as the query for my partitions I have run into a bit of a conundrum. I am able to use my TVF when I explicitly state the Database in my FROM clause.
SELECT * FROM [DW_Custom].[dbo].[CubePartition](#StartDate, #EndDate)
However, that will not work, because the cube will be deployed in multiple environments before production, and it needs to point to different DBs for each. So I tried adding a new data source, setting my partition query to point to the new data source, and then remove the database name. IE:
SELECT * FROM [dbo].[CubePartition](#StartDate, #EndDate)
I get an error that
The SQL syntax is not valid. The relational database returned the following error message: Deferred prepare could not be completed. Invalid object name 'dbo.CubePartition'
If I click through this error and the subsequent warnings about the cube not being able to process if I continue I am able to build and deploy the cube. However I cannot process it, because I get an error that one of my dimensions does not exist.
Looking into the query that was generated and it is clear that it is querying my dimensions as well as fact, which do not exist inside of '[DW_Custom]' which explains that error perfectly fine.
So I guess 2 questions:
Is it possible to query another DB (on the same server) from inside of an SSAS partition query?
If not, is there any way I can use a variable as the database name in the query, and update that variable based on the project configuration (Dev,Tst,Prd)
Bonus question: Is the reason that I can not find much about doing it this way because it is an obviously bad idea that I am overlooking, and if so why?
How about creating a second SSAS Data Source pointing to the DW_Custom database (or whatever it's called in the particular environment you're deploying to)? Then when you deploy from Dev to Prod, you need only change that connection string. When you create your partitions, then specify the DW_Custom data source and then specify the query without database name:
SELECT * FROM [dbo].[CubePartition](#StartDate, #EndDate)
As long as the query plan for that table-valued function is efficient compared to a plain SELECT statement, then I don't see a problem with that.
Excuse me but I am very new at learning to use oracle and sql.
I was given a bat file by my teacher that runs an sql to setup a database. The sql basically just drops 5 tables and recreates them with data.
What I don't understand is why I am seeing tons of other tables (that came with oracle?)
Why am I seeing all of these tables http://i.imgur.com/AvliJ.jpg
the only tables that are supposed to be created are
REP
CUSTOMER
ORDERS
PART
ORDER_LINE
Is there a way to only display those?
The other tables are tables Oracle uses.
Whether you see them or not depends on your permissions in the database. Ask your teacher to show you how to reduce your permissions so that you can only see the five tables your supposed to or look at the Oracle documentation here User Admin.
ETA : Be careful if you do start messing with permissions.
I am wondering if it is possible to get the real name of the tables/fields from which each field in a select statement comes from.
Lets say you have two tables, Creditors and Debtors both have the fields Code, Name and Phone.
If a user enters the following sql statement:
SELECT Code AS CustomerCode, Name AS CustomerName, Phone AS ContactNumber FROM Debtors.
This will result in SQL server returning field names CustomerCode, CustomerName and ContactNumber.
Is it possible to get from the SQL server some sort of meta data that maps each field to its real name and the table it comes from?
Programmically, given an SQL select statement, I want to be able to determine the real name of each field and the real name of the tables they come from.
I don't want to parse the SQL myself so I thought that there might be away to send the statement to the sql server and get back this information without the row data.
What we are trying to do is implement table/field level security. A user can enter an sql statement to select fields from a table or across multiple tables (using joins) and have the results displayed in a table. The fields get added dynamically to a grid control but only the ones that the user is allowed to see.
If the user joins multiple tables how do we know programmically which fields come from which table? the problem gets worse if they use aliases in the SQL.
Currently this is working on a legacy in house built sql engine (that uses a proprietary database) which can return all of the required table/field information without the row data so part of the applications security model is built around this. However moving this application to something like SQL server might prove difficult if we can't get this to work.
Besides Sql server, do any other sql databases support this type of functionality?
To the best of my knowledge, you cannot get that information.
You can probably handle your problem in many databases, however, by using GRANT / REVOKE security in the database itself. Assuming that users are logging into the database itself (not just your application), many DBMSes allow you to GRANT SELECT privileges on restricted columns from a table. Using that technique, it will not fool the server if the user specifies ALIASes for columns.
A quick google indicates that at least PostgreSQL, SQL Server, and Oracle offer column level GRANT SELECT protection based on userid.
Interesting question, by the way.
No. The whole point is you should only know the name presented to you and not where it came from.
The way to solve your problem is to remove all access from tables and only grant access through views with appropriate access permissions.
When I right click on the database I want to export data from, I only get to select a single table or view, rather than being able to export all of the data. Is there a way to export all of the data?
If this is not possible, could you advise on how I could do the following:
I have two databases, with the same table names, but one has more data than the other
They both have different database names (Table names are identical)
They are both on different servers
I need to get all of the additional data from the larger database, into the smaller database.
Both are MS SQL databases
Being that both are MS SQL Servers, on different hosts... why bother with CSV when you can setup a Linked Server instance so you can access one instance from the other via a SQL statement?
Make sure you have a valid user on the instance you want to retrieve data from - it must have access to the table(s)
Create the Linked Server instance
Reference the name in queries using four name syntax:
INSERT INTO db1.dbo.SmallerTable
SELECT *
FROM linked_server.db.dbo.LargerTable lt
WHERE NOT EXISTS(SELECT NULL
FROM db1.dbo.SmallerTable st
WHERE st.col = lt.col)
Replace WHERE st.col = lt.col with whatever criteria you consider to be duplicate values between the two tables.
There is also a very good tool by Redgate software that syncs data between two databases.
I've also used SQL scripter before to generate a SQL file with insert statements that you can run on the other database to insert the data.
If you right-click on the database, under the Tasks menu, you can use the Generate Scripts option to produce SQL scripts for all the tables and data. See this blog post for details. If you want to sync the second database with the first, then you're better off using something like Redgate as suggested in mpenrow's answer.