I am connecting to Hive via an ODBC driver from a .NET application. Is there a query to determine if a table already exists?
For example, in MSSQL you can query the INFORMATION_SCHEMA table and in Netezza you can query the _v_table table.
Any assistance would be appreciated.
Execute the following command : show tables in DB like 'TABLENAME'
If the table exists, its name will be returned, otherwise nothing will be returned.
This is done directly from hive. for more options see this.
DB is the database in which you want to see if the table exists.
TABLENAME is the table name you seek,
What actually happens is that Hive queries its metastore (depends on your configuration but it can be in a standard RDBMS like MySQL) so you can optionally connect directly to the same metastore and write your own query to see if the table exists.
There are two approaches by which you can check that:
1.) As #dimamah suggested, just to add one point here, for this approach you need to
1.1) start the **hiveserver** before running the query
1.2) you have to run two queries
1.2.1) USE <database_name>
1.2.2) SHOW TABLES LIKE 'table_name'
1.2.3) Then you check your result using Result set.
2.) Second approach is to use HiveMetastoreClient APIs, where you can directly use the APIs to check whether the table_name exist in a particular database or not.
For further help please go through this Hive 11
When programming on Hive by Spark SQL, you can use following method to check whether Hive table exists.
if (hiveContext.hql("SHOW TABLES LIKE '" + tableName + "'").count() == 1) {
println(tableName + " exists")
}
If someone is using shell script like me then my answer could be useful. Assume that your table is in the default namespace.
table=your_hive_table
validateTable=$(hive --database default -e "SHOW TABLES LIKE '$table'")
if [[ -z $validateTable ]]; then
echo "Error:: $table cannot be found"
exit 1
fi
If you're using SparkSQL you can do the following.
if "table_name" in sqlContext.tableNames("db_name"):
...do something
http://spark.apache.org/docs/2.1.0/api/python/pyspark.sql.html#pyspark.sql.SQLContext.tableNames
Code similar to below one can find in many of my Spark notebooks:
stg_table_exists = sqlCtx.sql("SHOW TABLES IN "+ stg_db)
.filter("tableName='%s'" % stg_tab_name) .collect()
(made two-liner for readability)
I wish Spark would have an API call to check the same.
If you're using a scala spark app and SparkSQL you can do the following
if spark.catalog.tableExists("tablename") {do something}
Related
I need make an Sql query into Hql query.
select count(column_name) from user_tab_columns where table_name='EMP_TABLE';
i do not know how to make it into an hql query if any one know please assist me.
I'm pretty sure Hive doesn't have this kind of metadata information available in a nice tabular format. You could look into querying the metastore directly, but that's ugly, and not really what you want anyway.
If you just want to know the number of columns in a Hive table, you can do that through a shell script that calls hive, for example:
hive -S -e 'describe my_table' | wc -l
I am writing a database abstraction layer that also abstracts some of the different query types. One of them is called "field_exists" - its purpose should be pretty self-explanatory.
And I want to implement that for SQLite.
The problem I am having is that I need to use one query that either returns a row confirming that the field exists or none if it doesn't. Thus, I cannot use the PRAGMA approach.
So, what query can I use to check whether a field exists in SQLite, that fulfills the above criteria?
EDIT: I should add that the query needs to be able to run in PHP code (using PDO).
Also, the query should look something like this (which only works with MySQL):
SHOW COLUMNS FROM table LIKE 'field'
Trying to select a field that doesn't exist will return an exception, then you can catch it and return nothing.
Use the .schema TABLENAME command. It will tell you the command that was issued to create the table. For more info chekcout the SQLite command shell documentation.
If you don't have access to the sqlite command line, you can always query the sqlite_master table. Let's say you want to know the command used to create the table MyTable. You'd issue this:
select sql from sqlite_master where name='MyTable';
This then gives you the sql command that was used to create the table. Then just grep through that output and see if the column you're looking for is in the command used to create the table.
UPDATE 2:
Actually better than the sql I posted above, you can use this:
PRAGMA table_info(*table_name*)
This will show you all the columns in a given table along with their types and other info.
Is there a way to send an SQL query to HSQLDB (edit : prior to version 2) to know the columns of a certain table ? (Similar to the inspection of the schemas through INFORMATION_SCHEMA in MySQL.)
Right now it seems to me I'm stuck with trying to SELECT column_name FROM table_name and see if I get an SQL error...
HSQLDB supports the INFORMATION_SCHEMA as well:
http://hsqldb.org/doc/2.0/guide/databaseobjects-chapt.html#dbc_information_schema
You use queries to the special SYSTEM tables. See the answers to [this question].1
I have the following problem, I need to put in a script that is going to run before the new version is rolled the SQL code that enables the pgAgent in PostgreSQL. However, this code should be run on the maintenance database (postgres) and the database where we run the script file is another one.
I remember that in SQL Server there is a command "use " so you could do something like:
use foo
-- some code
use bar
-- more code
is there something similar in PostgreSQL?
You can put in your file something like:
\c first_db_name
select * from t; --- your sql
\c second_db_name
select * from t; --- your sql
...
Are you piping these commands through the psql command? If so, \c databasename is what you want.
psql documentation
You can't switch databases in Postgres in this way. You actually have to reconnect to the other database.
PostgreSQL doesn't have the USE command. You would most likely use psql with the --dbname option to accomplish this, --dbname takes the database name as a parameter. See this link for details on the other options you can pass in you will also want to check out the --file option as well. http://www.postgresql.org/docs/9.0/interactive/app-psql.html
well after looking on the web for some time I found this which was what I need it
http://www.postgresonline.com/journal/archives/44-Using-DbLink-to-access-other-PostgreSQL-Databases-and-Servers.html
I usually use SQLDeveloper to browse the database, but I couldn't make it work with HSQLDB and I don't know which tables are already created…
I guess it's a vendor-specific question and not plain SQL, but the point is: how can I see the tables so I can drop/alter them?
The ANSI SQL92 standard for querying database metadata is contained within the INFORMATION_SCHEMA data structures.
I have no idea whether your database supports this or not, but try the following:
SELECT *
FROM INFORMATION_SCHEMA.TABLES
On further research, it appears that HSQLDB does support INFORMATION_SCHEMA, but with slightly non-standard naming.
All of the tables have SYSTEM_* prepended to them, so the above example would read
SELECT *
FROM INFORMATION_SCHEMA.SYSTEM_TABLES
I have no means of testing this, and the answer was found on sourceforge.
Awesome, thanks! Been scouring the Web for that info.
This will fetch only your tables' field info:
SELECT TABLE_NAME, COLUMN_NAME, TYPE_NAME, COLUMN_SIZE, DECIMAL_DIGITS, IS_NULLABLE FROM INFORMATION_SCHEMA.SYSTEM_COLUMNS WHERE TABLE_NAME NOT LIKE 'SYSTEM_%'
You can retrieve indexes, primary key info, all kinds of stuff from INFORMATION_SCHEMA.SYSTEM_TABLES.
Gotta love oo documentation :p
If you're on the command line, you may want to try the Hsqldb SqlTool, documented in the SqlTool Manual (hsqldb.org).
Put your database connection information in "~/sqltool.rc" and choose any DBNAME you want, substitute correct username and password, if known.
urlid DBNAME
url jdbc:hsqldb:/path/to/hsql/database
username SA
password
Install tool with: apt-get install hsqldb-utils (on Ubuntu)
Connect with hsqldb-sqltool DBNAME # on Ubuntu
Hint for other systems: java -jar YourHsqlJar.jar DBNAME
Show tables with: \dt
Show columns with: \d TABLENAME
Standard queries like: SELECT * FROM …;
Edit (append) last command with: :a
Quit with: \q
View special commands with: \? OR :?
Good luck!
Use the \dt command when you hit the >sql prompt in the command line for HSQLDB.
Check out DBVisualiser and SQuirreL SQL Client. Both of these have support for HSQLDB, and a GUI for editing/modifying/viewing the tables.
You run querying using hsql database manager, are you?
If you use this, below may give some hints:
Select your connection:
type: HSQL DATABASE ENGINE SERVER
Driver: jdbc.hsqldb.jdbcDriver
URL: jdbc:hsqldb:hsql://localhost/
Then, you will browse the database.