I am trying to get the current date into a Hive database (version 0.13 running on an HDInsight cluster) with the following script
SET curdt = from_unixtime(unix_timestamp());
DROP TABLE IF EXISTS curtime_test;
CREATE TABLE curtime_test (
dateEntered STRING
);
INSERT INTO TABLE curtime_test
SELECT '${hivevar:curdt}' FROM hivesampletable limit 3;
SELECT * FROM curtime_test;
Note that I want to have the same insert date for all the inserted records, this is a toy example, but the real one I want to use it on has millions of records to insert. This version I tried above just inserts the string '${hivevar:curdt}' into the database, which is not what I want:
${hivevar:curdt}
${hivevar:curdt}
${hivevar:curdt}
Omitting the quotes causes the insert to error out because of the spaces in the string. How can I do this right?
Update:
Using the line
SELECT ${hiveconf:curdt} FROM hivesampletable limit 3;
as per the comment from Charlie Haley (I mixed up ${hivevar} and ${hiveconf}), gives me the results that I want. If he writes it up as an answer I will mark it as right.
The following code sample works for me. Does this solve your problem?
DROP TABLE IF EXISTS curtime_test;
CREATE TABLE curtime_test (
dateEntered STRING
);
INSERT INTO TABLE curtime_test
SELECT unix_timestamp() FROM hivesampletable limit 1;
SELECT * FROM curtime_test;
Related
I have a table where I inserted my fixed file data to single varchar2 column and called this table tmp_table
CREATE TABLE "tmp_table"
( "COLUMN1" VARCHAR2(256 BYTE)
)
Now I am trying to transform and insert data from this tmp_table to final table where I am breaking the data using SUBSTR function
Now I created my next table where I will insert these columns
CREATE TABLE "TABLE"
(
"COLUMN A" VARCHAR2(4),
"CODE" NUMBER(6,0),
"DATE_LOADED_TIMESTAMP" DATE
)
Now I run a select query to check if I can get right data from my tmp_table that I want to insert to final TABLE
So I run the query
SELECT Cast(SUBSTR(COLUMN1,8,6)as number(6)) as Code
from TMP_TABLE;
The results I see were good and what I want to insert into CODE column.
Now I try to run final query:
insert into TABLE(CODE)
SELECT Cast(SUBSTR(COLUMN1,8,6)as number(6)) as Code
from TMP_TABLE;
Now it gives me error
Error report -
ORA-01722: invalid number
Why am I not able to insert number values to a 6 digit number column?
Ok I found what happened I was using SQL Developer and the query I ran below was returning on some results in my tool and the few results it was showing there was no characters in those fields.
SELECT Cast(SUBSTR(COLUMN1,8,6)as number(6)) as Code
from TMP_TABLE;
I ran following query to search if there are characters in the records I am looking at:
SELECT SUBSTR(COLUMN1,8,6)
from TMP_TABLE
where REGEXP_LIKE(SUBSTR(COLUMN1,8,6),'[A-Za-z]');
and here I saw characters now I know not to trust SQL developer blindly :)
I know how to find the CREATE statement for a table in SQL Server but is there any place that stores the actual SQL code if I use SELECT INTO ... to create a table and if so how do I access it?
I see two ways of creating tables with SELECT INTO.
First: You know the Schema, then you can declare a #Table Variable and perform the Select INSERT
Second: You can create a temp table:
SELECT * INTO #TempTable FROM Customer
There are some limitations on the second choice:
- You need to drop the temp table afterwards.
- If there is a VARCHAR Column and the maximum number of characters of that given SELECT is 123 characters (example), and then you try to insert into the TEMP table afterwards with a greater number of characters, it will throw an error.
My recommendation is always declare a table in order to use, it makes it clear what is the intentions and increases readability.
Using SQL Server 2008-R2
I have a csv of purchase IDs and in my database there is a table with these purchase IDs and there corresponding User IDs in our system. I need these to run a more complicated query after that using. I tried to bulk insert or run import wizard but I don't have permission. My new idea is to create a #temp using SELECT INTO and then have the query inside that like below.
SELECT *
INTO ##PurchaseIDs
FROM
(
SELECT PurchaseID, UserID, Added
FROM Users
WHERE PurchaseID IN (
/* These are the csv IDs just copied and pasted in */
'49397828',
'49397883',
etc.
What happens is that there are ~55,000 IDs so I get this error.
The query processor ran out of internal resources and could not
produce a query plan. This is a rare event and only expected for
extremely complex queries or queries that reference a very large
number of tables or partitions. Please simplify the query. If you
believe you have received this message in error, contact Customer
Support Services for more information.
It works if I upload about 30,000 so my new plan is to see if I can make a temp table, then append a new table to the end of that. I am also open to other ideas on how to accomplish what I am looking to do. I attached an idea of what I am thinking below.
INSERT *
INTO ##PurchaseIDs
FROM (
SELECT PurchaseID, UserID, Added
FROM Users
WHERE PurchaseID IN (
/* These are the OTHER csv IDS just copied and pasted in */
'57397828',
'57397883',
etc.
You need to create a temp table and insert the values in IN clause to the temp table and Join the temp table to get the result
Create table #PurchaseIDs (PurchaseID int)
insert into #PurchaseIDs (PurchaseID)
Select '57397828'
Union All
Select '57397828'
Union All
......
values from csv
Now use Exists to check the existence of PurchaseID in temp table instead of IN clause
SELECT PurchaseID,
UserID,
Added
FROM Users u
WHERE EXISTS (SELECT 1
FROM #PurchaseIDs p
WHERE u.PurchaseID = p.PurchaseID)
i am trying to run this SQL query to copy data from one table to another but it shows zero rows returned:
INSERT INTO ticket_updates
VALUES
(
SELECT *
FROM ticket_updates2
WHERE sequence = '4715'
)
You cannot put a select statement into a values method.
Your syntax should look like this:
INSERT into ticket_updates(all,my,columns)
select all,my,columns from ticket_updates2 where sequence = '4715'
I have a sql statement to insert data into a table for archiving, but I need a merge statement to run on a monthyl basis to update the new table(2) with any data that changed in the old table(1) that should now be moved into archive.
Part of the issue is to remove the moved data from the old table. My insert is not doing that, but I need to have it to where the saved data is purged from the original table.
Is there a single sql statement that will move data out of one table into another in this way? Or does it need to be a two step operation?
the initial statement moved data depending on age and a few other relative factors.
insert is:
INSERT /*+ append */
INTO tab1
SELECT *
FROM tab2
WHERE (Postingdate < TO_DATE ('2001/07/01', 'yyyy/mm/dd')
OR jobname IS NULL)
AND STATUS <> '45';
All help appreciated...
The merge statement will let you do this in one statement by adding a delete statement in the update clause. See Oracle Documentation on Merge.
I think you should try this with a partition table. My idea is to create table which have range partition on date:
create table(id number primary key,name varchar,J_date date )
partition by range(J_date)(PARTITION one_mnth VALUES LESS THAN(sysdate-30)),
partition by range(J_date)(PARTITION one_mnth VALUES LESS THAN(maxvalue)));
then move that partition in to another table and and truncate that partition