My script is
beeline <<EOF
!connect -u jdbc:hive://someurl; principal=hive/someprincipal;set hive.msck.repair.batch.size=10000; msck repair table default.test_table sync partitions;
EOF
Script returns Nonzero exit value 127.
The problem is located at last part: msck repair table default.test_table sync partitions.
May be msck requires some more syntax inside beeline scripts?
Related
Sometimes my commands in psql seem to be having no effect. Any idea why?
The below is the list of all tables in the database library_development:
library_development=> \d
List of relations
Schema | Name | Type | Owner
--------+-------------------+-------+----------
public | Pavan | table | postgres
public | schema_migrations | table | sai
(2 rows)
After this I dropped the table Pavan using:
library_development-> drop table Pavan
But the Table isn't dropped and its shows as shown:
library_development=> \d
List of relations
Schema | Name | Type | Owner
--------+-------------------+-------+----------
public | Pavan | table | postgres
public | schema_migrations | table | sai
(2 rows)
Also:
I am using PostgreSQL in Windows. Is there any command to clear the console (Like cl scr present in Oracle)?
Is there any concept of a "commit" I need to perform in Postgresql when working with DML scripts?
Statements end with semicolons.
In psql, pressing enter without a semicolon continues the statement onto the next line, adding what you wrote to the query buffer rather than executing it. You will notice that the prompt changes from dbname=> to dbname-> to indicate that you're on a continuation line.
regress=> DROP TABLE sometable
regress-> \r
Query buffer reset (cleared).
regress=> DROP TABLE sometable;
ERROR: table "sometable" does not exist
regress=>
Notice how after I press enter without a semicolon, the prompt changes to regress-# and no action is taken. There is no table sometable, so if the statement had run an error would be reported.
Next, see the use of \r on the next line? That clears the query buffer. Notice that the prompt changes back to regress=# when the buffer is cleared, as there's no partial statement buffered anymore.
This shows how statements can be split across lines:
regress=> DROP TABLE
regress-> sometable
regress-> ;
ERROR: table "sometable" does not exist
The confusing thing is that psql backslash commands like \d are newline-terminated, not semicolon terminated, so they do run when you press enter. That's handy when you want to (say) view a table definition while writing a statement, but it's a bit confusing for newcomers.
As for your additional questions:
If there's a "clear screen" command in psql for Windows I haven't found it yet. On Linux I just use control-L, same as any other readline-using program. In Windows \! cls will work.
DDL in PostgreSQL is transactional. You can BEGIN a transaction, issue some DDL, and COMMIT the transaction to have it take effect. If you don't do your DDL in an explicit transaction then it takes effect immediately.
I just encountered something similar, but the cause was different.
I was trying to use the dropdb command and what I was seeing was like my command having no effect (change of prompt means my command is in the buffer, but that's not the issue here):
postgres=# dropdb databasename
postgres-#
Commands like dropdb should be entered OUTSIDE psql. What i had been doing was:
$ psql postgres
postgres=# dropdb databasename
Notice the first command is in Bash and fires psql, and the second is sent within psql.
The correct way is to send the command directly in bash (the console):
$ dropdb databasename
Problem
Error running schema migration, err: cmd failed, cmd:
ALTER TABLE "schema"."table" ALTER COLUMN kill_id TYPE VARCHAR(MAX)
err: pq: ALTER TABLE ALTER COLUMN cannot run inside a transaction block
AWS Redshift does not support running the above ALTER COMMAND in an transaction block.
Does database/sql supports running sql commands without a transaction block?
How to run raw sql commands without a transaction block in Golang?
I'm trying to execute two sql commands (create a new schema and table), in a way that would enable a rollback of both commands if the execution fails. The database I'm connecting to is AWS Redshift.
create schema if not exists test_schema;
create table test_schema.test_table as select 1;
Initially I tried to execute these commands programatically with python, using both psycopg2 and pyodbc, and got the following error:
ERROR: schema "test_schema" does not exist
I realised that it fails because the first command isn't being comitted, so to fix that , I tried setting the autocommit mode on, and wrapping the statements with "begin/end" block, which didn't help.
When I used psql CLI and ran the following, everything worked as intended (there was no "schema does not exist" error, and after the rollback, both schema and table were gone):
dev=# begin;
BEGIN
dev=# create schema test_schema;
CREATE SCHEMA
dev=# create table test_schema.test_table as select 1;
SELECT
dev=# rollback;
ROLLBACK
I tried to get the same results by running the following in the command line:
psql -c "begin; create schema test_schema; create table test_schema.test_table as select 1;"
This results in the same error:
ERROR: schema "test_schema" does not exist
However, when I put the above code in a file and ran the same command, this time using -f, it worked:
psql -f create_schema_and_table.sql
My questions are:
What is the difference between executing queries with "psql -c" and "psql -f"?
How can the same result be achieved programatically, with python?
Thanks a lot!
I don't know what you are doing wrong, your "psql -c" command works perfectly fine:
ads#diamond:~$ psql -c "begin; create schema test_schema; create table test_schema.test_table as select 1;" postgres
SELECT 1
psql will send the entire string to the server, and execute it in one single transaction. Your problem is that you start a transaction using "begin", but never commit it. Therefore at the end of the psql run, all your changes are rolled back. The next psql command will not find the schema, nor the table. But as long as everything stays in a single psql call, subsequent queries in the same command can see newly created objects.
Your query string should instead look like:
begin; create schema test_schema; create table test_schema.test_table as select 1; commit;
Or, more easy:
create schema test_schema; create table test_schema.test_table as select 1;
Both will work.
I am a complete beginner with postgresql. I created a test database called iswdp and now I want to delete it. When I do:
dropdb iswdp
the command returns no output and when I \list the table iswdp is still there.
dropdb iswdp;
returns:
ERROR: syntax error at or near "dropdb"
LINE 1: dropdb iswdp
I used:
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE datname = current_database()
AND pid <> pg_backend_pid();
which I found from another stackoverflow post to disconnect from all databases then attempted dropdb iswdp again with the same result.
I'm stuck, can someone help me? I'm doing this on linux mint from the bash terminal.
The command dropdb is a command issued from a shell prompt. From a sql prompt (like psql), you would want to issue a DROP DATABASE command.
I recommend opening psql and issuing DROP DATABASE iswdp;. That should work.
You may get an error that looks like ERROR: cannot drop the currently open database, which will happen if you connect to iswdp and try to drop it. If that happens, try instead to connect to the postgres database, and issue the same DROP DATABASE command. It should work.
dropdb is a commando from shell not from psql program, from psql the commnado is drop database iswp;
I am learning SQL at the moment, and am using Oracle SQL Developer.
Right now I am having an issue with the spool command.
I insert the file path and name the file but I keep getting an "unknown command" error.
spool C:\Users\user\Desktop\AAA_Task1\output.txt
set echo on
--drop table TESTTABLE;
create table TESTTABLE (TestCol int);
set echo off
spool off
I alternate between the create and drop statements for each test.
I run the and the table is created or dropped correctly, but I get a line one error "Unknown Command" and no text file is created.
I have attached a picture of a test command and the error.
If someone could point me in the right direction I would be grateful.
There is difference between run script and run statement, see here.
Use following commands and run script.
spool 'C:\Users\user\Desktop\AAA_Task1\output.txt'
set echo on
--drop table TESTTABLE;
create table TESTTABLE (TestCol int);
set echo off
spool off
Making the file path in quotes did not work for me in the Oracle database.
spool data.txt
set echo on
Query -- the query
set echo off
spool off
this will save the query and output to the data.txt file in the
"C:\Users\NELSON JOSEPH\AppData\Roaming\SQL Developer\data.txt" in this location likewise username varies for others.