I have 2 rows like below:
941 78 252 3008 86412 1718502 257796 2223252 292221 45514 114894
980 78 258 3064 88318 1785623 269374 2322408 305467 46305 116970
I want to insert current time stamp while inserting each row.
finally in my hive table row should be like below:
941 78 252 3008 86412 1718502 257796 2223252 292221 45514 114894
2014-10-21
980 78 258 3064 88318 1785623 269374 2322408 305467 46305 116970
2014-10-22
Is there any way I can insert timestamp directly into hive without using pig script?
You can use from_unixtime(unix_timestamp()) while inserting.
For example, suppose you have following tables:
create table t1(c1 String);
create table t2(c1 String, c2 timestamp);
Now you can populate table t2 from t1 with current timestamp:
insert into table t2 select *, from_unixtime(unix_timestamp()) from t1;
Related
We have an app to manage our Member's information that is tied to a SQL Database. We have attributes that the users can set that apply to the whole family. I am trying to write a SQL Script that will update the values for the whole family.
Example:
Here is a sample of a few columns of our dbo.AttributeValue column:
AttributeID
EntityID
Value
CreatedDateTime
ModifiedDateTime
5856
733
True
2021-11-06 17:30:38.207
2021-11-10 13:52:09.843
5856
613
Fale
2021-11-05 12:12:08.207
2021-11-16 3:32:01.843
Here is a sample of a few columns in our dbo.Person Table:
ID
PrimaryFamilyID
733
187
709
187
137
187
I would like for anyone with the same value in PrimaryFamilyID to have the same values in the dbo.AttributeValue table. Bonus points if we can make it update to the value with the most recent ModifiedDateTime in the dbo.AttributeValue table so that if someone in the family modifies the value after every has an assigned attribute, it will go ahead an update those as well.
Desired outcome:
AttributeID
EntityID
Value
CreatedDateTime
ModifiedDateTime
5856
733
True
2021-11-06 17:30:38.207
2021-11-10 13:52:09.843
5856
709
True
2021-11-06 17:30:38.207
2021-11-10 13:52:09.843
5856
137
True
2021-11-06 17:30:38.207
2021-11-10 13:52:09.843
It took me a while to get to a solution what you want is but here it is
DBFiddleRunningSolution
You can start somewhere from here.
With PersonCTE as (
Select Count(*) as cnt,PrimaryFamilyId
from Person
group by PrimaryFamilyId
having count(*)>1
)
Select AV.AttributeId,P.Id,AV.Value,AV.CreatedDateTime,AV.ModifiedDateTime
into NewAttributeValue
from Person P inner join PersonCTE C
ON P.PrimaryFamilyId = C.PrimaryFamilyId
cross join AttributeValue AV
where AV.EntityId in (Select distinct Id from Person)
I have this table on Oracle 11g:
CREATE TABLE USER.HISTORY_LOG
(
ID NUMBER,
DATE DATE,
COUNT_INSERTED_ROWS NUMBER,
EXEC_TIME VARCHAR2(50 BYTE)
)
Some data:
ID DATE COUNT_INSERTED_ROWS EXEC_TIME
6356 04/04/2016 09:36:03 1074 00h:00m:58s:769ms
6355 04/04/2016 09:33:00 1074 00h:00m:56s:221ms
6354 04/04/2016 09:30:11 1074 00h:01m:06s:719ms
6353 04/04/2016 09:27:13 1074 00h:01m:08s:977ms
6352 04/04/2016 09:24:13 1074 00h:01m:09s:361ms
6351 04/04/2016 09:21:12 1074 00h:01m:07s:685ms
6350 04/04/2016 09:18:11 1074 00h:01m:06s:657ms
6349 04/04/2016 09:15:01 1074 00h:00m:57s:074ms
This table is fed by a console app writen in c#, which runs every 3 minutes "forever"...
But it can crash by some server's issue like unavailability.. and I must check if there was any time - and when - this happened. how can I do that using SQL?
What I need is something like that:
Between date X and Y, it took more than 3 minutes to execute.
I want to solve using SQL.... is there any way I can do that? I confess that I'm out of ideas to build a query to do that.
You can try with something like this:
select *
from (
select lag(date_)over ( order by date_) as date_before_ko,
date_ as date_after_ko,
floor((date_ - lag(date_) over ( order by date_))*24*60) as minutes_passed
from HISTORY_LOG
)
where minutes_passed > 4
This computes, for each row, the preceeding record and checks if the difference in minutes between the two dates is more than 4.
I am trying to migrate an excel database into sqlite, although I have no expertise in the latter.
My first step is to import a series of CSV files into an sql database (which I found fairly easy to do).
These tables have a common structure,i.e.
column1: timestamp
column2: temperature
column3: humidity
The second step is to merge rows from imported table, according to the value in the first column (timestamp). The data in the rows might overlap or have gaps e.g.:
timestamp,temperature,humidity
04/01/2016 09:00:00, 23.1, 45.5
04/01/2016 09:15:00, 23.3, 46
...
20/01/2016 15:15:00, 25, 40
with
timestamp,temperature,humidity
10/01/2016 09:00:00, 23.1, 45.5
10/01/2016 09:15:00, 23.3, 46
...
30/01/2016 15:15:00, 25, 40
How can I merge the two (or more) imported CSV, overwriting the common data and leaving blank (or NULL) values for the gaps into one master sql table?
Thanks,
Andrea
My_Table has following columns:
time | temp | humidity
example.csv has the following data:
10/01/2016 09:15:00,23.1,45.5
10/01/2016 09:11:00,22.3,41.5
10/01/2016 09:15:00,23.1,42.5
Go to Sqlite3 terminal and run the following commands.
.separator ","
.import example.csv My_Table
Likewise you can import all of your csv files at last you can delete duplicate lines based on time.
delete from My_Table where rowid not in (select max(rowid) from My_Table group by time);
I have a table looking like this:
ReadingDate,=avg(Cost)
11/04/2011,£10.00
28/05/2011,£326.00
02/06/2011,£12.00
28/06/2011,£53.00
10/09/2011,£956.00
11/10/2011,£63.00
01/01/2012,£36.00
11/04/2012,£150.00
12/05/2012,£100.00
I know how to make an avg of a day or month, but how do I make limitations like 'between 01.05.2012 and 11.11.2013' and getting one average from it?
If you would like to do this in the load script, you can create a temporary table where you can perform the average over your desired range and then store this in a variable.
I used your source data for the below example:
SET DateFormat='DD/MM/YYYY';
MyData:
LOAD * INLINE [
ReadingDate, Cost
11/04/2011, 10.00
28/05/2011, 26.00
02/06/2011, 12.00
28/06/2011, 53.00
10/09/2011, 956.00
11/10/2011, 63.00
01/01/2012, 36.00
11/04/2012, 150.00
12/05/2012, 100.00
];
AverageData:
LOAD
avg(Cost) as AvgCost
RESIDENT MyData
WHERE (ReadingDate > '28/05/2011') AND (ReadingDate < '01/01/2012');
DROP TABLE AverageData;
LET AverageCost = peek('AvgCost',0,'AverageData');
Here, AverageCost is your variable and contains a single number (in this case 271). which you can then use later on in the script, for example:
MyData2:
NOCONCATENATE
LOAD
ReadingDate,
Cost
$(AverageCost)
RESIDENT MyData;
This then results in the following:
11/04/2011, 10.00, 271
28/05/2011, 26.00, 271
02/06/2011, 12.00, 271
28/06/2011, 53.00, 271
10/09/2011, 956.00, 271
11/10/2011, 63.00, 271
01/01/2012, 36.00, 271
11/04/2012, 150.00, 271
12/05/2012, 100.00, 271
i have this values in a coulmn say accountno in table record:
1,002
1,044
1,086
1,108
1,126
1,190
1,226
1,258
1,260
now i want to update them as
1002
1044
1086
1108
1126
1190
1226
1258
1260
the column is of type string. how can i do it??
assuming you are using SQL server -
update table
set accountno = REPLACE(accountno,',', '')
UPDATE record
SET accountno = replace(accountno,',','')