I'm trying to extract the name of space organisations from a table but the closest i can get is the amount of times it appears next to the name of the organisation but i just want the name of the organisation not the amount of times it is named in the table.
if you can help me please leave a comment on my google colab.
https://colab.research.google.com/drive/1m4zI4YGguQ5aWdDVyc7Bdpr-78KHdxhR?usp=sharing
What I get:
variable number
organisation
time of launch
0
SpaceX
Fri Aug 07, 2020 05:12 UTC
1
CASC
Thu Aug 06, 2020 04:01 UTC
2
SpaceX
Tue Aug 04, 2020 23:57 UTC
3
Roscosmos
Thu Jul 30, 2020 21:25 UTC
4
ULA
Thu Jul 30, 2020 11:50 UTC
...
...
...
4319
US Navy
Wed Feb 05, 1958 07:33 UTC
4320
AMBA
Sat Feb 01, 1958 03:48 UTC
4321
US Navy
Fri Dec 06, 1957 16:44 UTC
4322
RVSN USSR
Sun Nov 03, 1957 02:30 UTC
4323
RVSN USSR
Fri Oct 04, 1957 19:28 UTC
etc
etc
etc
What I want:
organisation
RVSN USSR
Arianespace
CASC
General Dynamics
NASA
VKS RF
US Air Force
ULA
Boeing
Martin Marietta
etc
I am trying to merge two dataframes with different time delta. One represents the returns of an asset (df2) on a daily basis and the other one is the inflation rate (df1) which is published once a month but not in a regular inverval. I am trying to merge those two.
df1 =
First Release
Original Release Date
30 Jun 2010 10:01 1.4%
30 Jul 2010 10:00 1.7%
31 Aug 2010 10:00 1.6%
30 Sep 2010 10:00 1.8%
29 Oct 2010 10:02 1.9%
... ...
17 Mar 2022 11:00 5.9%
21 Apr 2022 10:00 7.4%
18 May 2022 10:00 7.4%
17 Jun 2022 10:00 8.1%
19 Jul 2022 10:00 8.6%
[145 rows x 1 columns]
df2 =
Date
2010-08-11 -0.001654
2010-08-12 -0.028538
2010-08-13 0.001072
2010-08-16 -0.007665
2010-08-17 0.002667
...
2022-01-25 0.029663
2022-01-26 0.026082
2022-01-27 -0.000115
2022-01-28 0.002425
2022-01-31 0.007184
Obviously inflation rate should be placed in the new column from the day after it is released until there is a new release. For example 30. June is the first anouncement and 30 Jul the second. So from 1. July to the 30. July should be 1.4 %. The result is published on the 30. but to avoid look-ahead-bias it is more appropriate to have it . Does someone have an idea or maybe encountered some similar problem ?
I want to take col1 of file1 and if there is a match in col1 of file2, update the "date updated" in the last column. If there is no match, I want to append the entire line of file1 to file2 and append a "date updated" value to that line as well.
I am currently using awk 'NR==FNR{c[$1]++;next};c[$1] > 0' file2 file1 for a baseline comparison, but that wrongly prints the whole line IF there is a match and I also cannot figure out how to add another condition for updating the date column. I am also trying to do this in a shell script.
file 1
userName | cpu% | command | date created
user1, 101.6, plasma-de+, Thu Aug 8 09:30:17 MDT 2019
user2, 100.0, plasma-de+, Thu Aug 8 09:30:17 MDT 2019
user3, 102.0, plasma-de+, Thu Aug 8 09:30:17 MDT 2019
file 2
userName | cpu% | command | date created | date updated
user1, 101.6, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Mon Aug 5 06:35:39 MDT 2019
user2, 100.0, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Mon Aug 5 06:35:39 MDT 2019
file 2 after command is run
userName | cpu% | command | date created | date updated
user1, 101.6, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
user2, 100.0, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
user3, 102.0, plasma-de+, Thu Aug 8 09:30:17 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
One non-awk way that assumes your files are sorted:
$ (join -t, -j1 -o 0,2.2,2.3,2.4,1.4 file1 file2; \
join -t, -j1 -v1 -o 0,1.2,1.3,1.4,1.4 file1 file2)
user1, 101.6, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
user2, 100.0, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
user3, 102.0, plasma-de+, Thu Aug 8 09:30:17 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
or using awk without that restriction:
$awk 'BEGIN { FS = OFS = "," }
NR == FNR { a[$1] = $0; b[$1] = $4; next }
$1 in a { $5 = b[$1]; delete a[$1]; print }
END { for (u in a) print a[u], b[u] }' file1 file2
user1, 101.6, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
user2, 100.0, plasma-de+, Mon Aug 5 06:35:39 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
user3, 102.0, plasma-de+, Thu Aug 8 09:30:17 MDT 2019, Thu Aug 8 09:30:17 MDT 2019
I have a column varchar type with dates like:
Fri Mar 3 12:55:17 EST 2017
Thu Jul 27 10:12:07 EDT 2017
Fri Jul 21 12:11:35 EDT 2017
Wed Jan 31 13:15:34 EST 2018
And I would like to return just the date and time something like:
03/03/2017 12:55:17
07/27/2017 10:12:07
07/21/2017 12:11:35
01/31/2018 13:15:34
I tried several ways with substring and convert statement but nothing work.
Any assistance in this regard will be greatly appreciated.
Perhaps something like this
Example
Declare #YourTable table (SomeCol varchar(50))
Insert Into #YourTable values
('Fri Mar 3 12:55:17 EST 2017'),
('Thu Jul 27 10:12:07 EDT 2017'),
('Fri Jul 21 12:11:35 EDT 2017'),
('Wed Jan 31 13:15:34 EST 2018')
Select *
,AsDateTime = try_convert(datetime,substring(SomeCol,4,len(SomeCol)-11)+right(SomeCol,4))
From #YourTable
Returns
SomeCol AsDateTime
Fri Mar 3 12:55:17 EST 2017 2017-03-03 12:55:17.000
Thu Jul 27 10:12:07 EDT 2017 2017-07-27 10:12:07.000
Fri Jul 21 12:11:35 EDT 2017 2017-07-21 12:11:35.000
Wed Jan 31 13:15:34 EST 2018 2018-01-31 13:15:34.000
There is table called t1, and there are columns id, created_at, text, for example, as following table:
id created text
1 Thu Jun 30 01:00:57 +0000 2016 I like this movie1
2 Thu Jun 30 02:59:57 +0000 2016 I like this movie2
3 Thu Jun 30 03:49:57 +0000 2016 I like this movie3
4 Thu Jun 30 04:59:50 +0000 2016 I like this movie4
5 Thu Jun 30 05:39:57 +0000 2016 I like this movie5
6 Thu Jun 30 06:39:57 +0000 2016 I like this movie6
7 Thu Jun 30 06:29:57 +0000 2016 I like this movie6
8 Thu Jun 30 07:09:57 +0000 2016 I like this movie7
9 Thu Jun 30 07:39:57 +0000 2016 I like this movie8
10 Thu Jun 30 08:39:57 +0000 2016 I like this movie9
11 Thu Jun 30 09:39:57 +0000 2016 I like this movie10
12 Thu Jun 30 10:29:57 +0000 2016 I like this movie11
13 Thu Jun 30 11:29:57 +0000 2016 I like this movie12
12 Thu Jun 30 12:29:57 +0000 2016 I like this movie13
I want to select data separated by hour time.
For example, I want to select all the data that hour is less or equal 06, then I want to select the data that hour is more than 07. Since the data of column is datetime form: Thu Jun 30 12:29:57 +0000 2016, I don't know how to deal with this. Thanks for your help!
The sql is presto(presto sql):
select id, created, text from t1 where created_at <= 6
You could use datepart for this if you are using mssql:
select
id,
created,
text
from
t1
where
datepart(hour, created) <= 6
References:
DATEPART (Transact-SQL)
I done it, use the hour(datestamp)can solve it.
select id, created, text from t1 where hour(created_at) <= 6