What is the cause of MCOConnectionLogTypeErrorParse ?Command to send the QUIT to return to this mistake。
2015-07-15 11:38:23.501 xxxxxxxxxxx[1588:845458] type==>1
2015-07-15 11:38:23.502 xxxxxxxxxxx[1588:845458] data==>QUIT
2015-07-15 11:38:23.502 xxxxxxxxxxx[1588:845458] type==>-1
2015-07-15 11:38:23.502 xxxxxxxxxxx[1588:845458] data==>
2015-07-15 11:38:23.502 xxxxxxxxxxx[1588:845458] type==>-1
2015-07-15 11:38:23.503 xxxxxxxxxxx[1588:845458] data==>
2015-07-15 11:38:53.505 xxxxxxxxxxx[1588:845458] type==>4
2015-07-15 11:38:53.506 xxxxxxxxxxx[1588:845458] data==>
2015-07-15 11:38:53.514 xxxxxxxxxxx[1588:839802] 邮件发送失败错误:Error
Domain=MCOErrorDomain Code=1 "A stable connection to the server could
not be established." UserInfo=0x17426bb80 {NSLocalizedDescription=A
stable connection to the server could not be established.}
But in fact is sent successfully
Related
I have something like the following dataframe (notice dt is the index)
fx fy
dt
2019-05-29 0.000000 0.000000
2019-05-30 65.410004 156.449997
2019-05-31 70.279999 125.040001
2019-06-01 49.220001 147.979996
2019-06-02 100.580002 232.539993
2019-06-03 262.230011 468.809998
2019-06-04 383.779999 525.390015
2019-06-05 761.609985 1147.380005
2019-06-06 1060.750000 1727.380005
2019-06-07 1640.300049 2827.120117
What I want to achieve is the have a new column named fz where each day's value, is the previous month's max value of fy - so the result would be
fx fy fz
dt
2019-05-29 0.000000 0.000000 NaN
2019-05-30 65.410004 156.449997 NaN
2019-05-31 70.279999 125.040001 NaN
2019-06-01 49.220001 147.979996 156.449997
2019-06-02 100.580002 232.539993 156.449997
2019-06-03 262.230011 468.809998 156.449997
2019-06-04 383.779999 525.390015 156.449997
2019-06-05 761.609985 1147.380005 156.449997
2019-06-06 1060.750000 1727.380005 156.449997
2019-06-07 1640.300049 2827.120117 156.449997
The first month's fz is empty because there is no previous month. I tried a combination of pd.Grouper(freq='M') with .transform() and .shift(-1, freq='M') but failed miserably as it changed the index entirely, and I would like to keep the index as is.
How can I solve this for arbitrary N months back?
Use DatetimeIndex.to_period for month period with shifting and mapping by Index.map:
#changed datetimeindex
print (df)
fx fy
dt
2019-05-29 0.000000 0.000000
2019-05-30 65.410004 156.449997
2019-05-31 70.279999 125.040001
2019-06-01 49.220001 147.979996
2019-06-02 100.580002 232.539993
2019-07-03 262.230011 468.809998
2019-07-04 383.779999 525.390015
2019-08-05 761.609985 1147.380005
2019-08-06 1060.750000 1727.380005
2019-09-07 1640.300049 2827.120117
N = 2
s = df.index.to_period('m')
df['fz'] = s.map(df.groupby(s)['fy'].max().shift(N))
print (df)
fx fy fz
dt
2019-05-29 0.000000 0.000000 NaN
2019-05-30 65.410004 156.449997 NaN
2019-05-31 70.279999 125.040001 NaN
2019-06-01 49.220001 147.979996 NaN
2019-06-02 100.580002 232.539993 NaN
2019-07-03 262.230011 468.809998 156.449997
2019-07-04 383.779999 525.390015 156.449997
2019-08-05 761.609985 1147.380005 232.539993
2019-08-06 1060.750000 1727.380005 232.539993
2019-09-07 1640.300049 2827.120117 525.390015
Solution if datetimes are not conecutive, missing some months with add N to PeriodIndex by rename:
print (df)
fx fy
dt
2019-05-29 0.000000 0.000000
2019-05-30 65.410004 156.449997
2019-05-31 70.279999 125.040001
2019-06-01 49.220001 147.979996
2019-06-02 100.580002 232.539993
2019-08-03 262.230011 468.809998
2019-08-04 383.779999 525.390015
2019-09-05 761.609985 1147.380005
2019-09-06 1060.750000 1727.380005
2019-09-07 1640.300049 2827.120117
N = 1
s = df.index.to_period('m')
df['fz'] = s.map(df.groupby(s)['fy'].max().rename(lambda x: x + N))
print (df)
fx fy fz
dt
2019-05-29 0.000000 0.000000 NaN
2019-05-30 65.410004 156.449997 NaN
2019-05-31 70.279999 125.040001 NaN
2019-06-01 49.220001 147.979996 156.449997
2019-06-02 100.580002 232.539993 156.449997
2019-08-03 262.230011 468.809998 NaN
2019-08-04 383.779999 525.390015 NaN
2019-09-05 761.609985 1147.380005 525.390015
2019-09-06 1060.750000 1727.380005 525.390015
2019-09-07 1640.300049 2827.120117 525.390015
you can do it in two steps:
create a table with maximum values + shift per monthly period:
maximum_shift = df.resample('M')['fy'].max().shift().to_period('M')
concatenate/merge it to the original data frame:
pd.DataFrame(pd.concat([df.to_period('M'), maximum_shift], axis=1).values, index=df.index, columns=df.columns.tolist()+['fz'])
Premise: You have a table with one column, original_date, of datatype string:
ORIGINAL_DATE
20190825
20190826
20190827
20190828
20190829
20190830
20190831
20190901
Question: Write a SQL query to calculate two more columns – end_of_week - the date of the next Sunday from original_date. If original_date is already a Sunday, this field should be the same value end_of_month - the value of the end of month date An acceptable solution is one which works for any valid date in the string format of original_date. With end_of_month and end_of_week computed
ORIGINAL_DATE END_OF_WEEK END_OF_MONTH
20190825 20190825 20190831
20190826 20190901 20190831
20190827 20190901 20190831
20190828 20190901 20190831
20190829 20190901 20190831
20190830 20190901 20190831
20190831 20190901 20190831
20190901 20190901 20190930
Additional Info:
20190825 is a Sunday, so the end_of_week for that value is still that same date.
20190827 is a Tuesday, and the next Sunday is on 20190901
CREATE TABLE random_dates ( original_date VARCHAR(8) NOT NULL );
INSERT INTO random_dates(original_date) values('20190825');
INSERT INTO random_dates(original_date) values('20190826');
INSERT INTO random_dates(original_date) values('20190827');
INSERT INTO random_dates(original_date) values('20190828');
INSERT INTO random_dates(original_date) values('20190829');
INSERT INTO random_dates(original_date) values('20190830');
INSERT INTO random_dates(original_date) values('20190831');
INSERT INTO random_dates(original_date) values('20190901');
EXPECTED OUTPUT:
20190825 2019-08-25 2019-08-31
20190826 2019-09-01 2019-08-31
20190827 2019-09-01 2019-08-31
20190828 2019-09-01 2019-08-31
20190829 2019-09-01 2019-08-31
20190830 2019-09-01 2019-08-31
20190831 2019-09-01 2019-08-31
20190901 2019-09-01 2019-09-30
Solution for Hive:
with random_dates as (--this is your example dataset
select stack(8,
'20190825', '20190826', '20190827', '20190828', '20190829', '20190830', '20190831', '20190901'
) as original_date
)
select original_date,
date_add(date_formatted, 6-days) end_of_week,
last_day(date_formatted) end_of_month
from
(
select original_date,
regexp_replace(original_date,'^(\\d{4})(\\d{2})(\\d{2})$','$1-$2-$3') date_formatted,
pmod(datediff(regexp_replace(original_date,'^(\\d{4})(\\d{2})(\\d{2})$','$1-$2-$3'),'1900-01-08'),7) days
from random_dates
)s
;
Result:
original_date end_of_week end_of_month
20190825 2019-08-25 2019-08-31
20190826 2019-09-01 2019-08-31
20190827 2019-09-01 2019-08-31
20190828 2019-09-01 2019-08-31
20190829 2019-09-01 2019-08-31
20190830 2019-09-01 2019-08-31
20190831 2019-09-01 2019-08-31
20190901 2019-09-01 2019-09-30
SELECT original_date,
CASE DAYOFWEEK(STR_TO_DATE(original_date,'%Y%m%d'))
WHEN 1 THEN DATE_ADD(STR_TO_DATE(original_date,'%Y%m%d'),INTERVAL 0 DAY)
WHEN 2 THEN DATE_ADD(STR_TO_DATE(original_date,'%Y%m%d'),INTERVAL 6 DAY)
WHEN 3 THEN DATE_ADD(STR_TO_DATE(original_date,'%Y%m%d'),INTERVAL 5 DAY)
WHEN 4 THEN DATE_ADD(STR_TO_DATE(original_date,'%Y%m%d'),INTERVAL 4 DAY)
WHEN 5 THEN DATE_ADD(STR_TO_DATE(original_date,'%Y%m%d'),INTERVAL 3 DAY)
WHEN 6 THEN DATE_ADD(STR_TO_DATE(original_date,'%Y%m%d'),INTERVAL 2 DAY)
WHEN 7 THEN DATE_ADD(STR_TO_DATE(original_date,'%Y%m%d'),INTERVAL 1 DAY)
END AS END_OF_WEEK,
LAST_DAY(STR_TO_DATE(original_date,'%Y%m%d')) AS END_OF_MONTH
FROM random_dates;
I have once csv with missing dates, I have created a new df of that same date range, without the missing dates. I want to compare the two csvs and place an NaN wherever there are blank dates in the original csv:
Example:
DateTime Measurement Dates
0 2016-10-09 00:00:00 1021.9 2016-10-09
1 2016-10-11 00:00:00 1019.9 2016-10-10
2 2016-10-12 00:00:00 1015.8 2016-10-11
3 2016-10-13 00:00:00 1013.2 2016-10-12
4 2016-10-14 00:00:00 1005.9 2016-10-13
so I want the new table to be:
DateTime Measurement Dates
0 2016-10-09 00:00:00 1021.9 2016-10-09
1 Nan 00:00:00 Nan 2016-10-10
2 2016-10-11 00:00:00 1015.8 2016-10-11
3 2016-10-12 00:00:00 1013.2 2016-10-12
4 2016-10-13 00:00:00 1005.9 2016-10-13
And then I will remove the DateTime column so the final df is a complete list of dates with the missing measurements.
The code I have used thus far:
new_dates = pandas.date_range(start = '2016-10-09 00:00:00', end = '2017-10-09 00:00:00')
merged = pandas.merge(measurements, updated_dates,left_index=True, right_index=True)
If I understand your correctly you want to resample your DateTime column to a daily frequency and fill the gaps with NaN:
# Use this line if your DateTime column is not datetime type yet
# df['DateTime'] = pd.to_datetime(df['DateTime'])
dates = pd.date_range(df['DateTime'].min(), df['DateTime'].max(), freq='D')
df = df.set_index('DateTime').reindex(dates).reset_index()
Output
index Measurement
0 2016-10-09 1021.9
1 2016-10-10 NaN
2 2016-10-11 1019.9
3 2016-10-12 1015.8
4 2016-10-13 1013.2
5 2016-10-14 1005.9
If you have unique dates, you can use resample as well. If your dates are not unique it would aggregate them and take the mean of two dates:
df.set_index('DateTime').resample('D').mean()
Output
DateTime Measurement
0 2016-10-09 1021.9
1 2016-10-10 NaN
2 2016-10-11 1019.9
3 2016-10-12 1015.8
4 2016-10-13 1013.2
5 2016-10-14 1005.9
How can i a Series like this:
2016-11-09 00:07:00 0 days 00:00:15.000000000
2016-11-09 00:07:15 0 days 00:20:14.000000000
2016-11-09 00:07:30 0 days 10:00:15.000000000
into in integer values like this:
2016-11-09 00:07:00 15
2016-11-09 00:07:15 1214 // 20*60+14
2016-11-09 00:07:30 36015 // 10*60*60+15
Those are TimeDeltas. You should be able to use the total_seconds method. However, you'd need to access that method via the datetime accessor dt. Assuming your series is named s
s.dt.total_seconds()
2016-11-09 00:07:00 15.0
2016-11-09 00:07:15 1214.0
2016-11-09 00:07:30 36015.0
dtype: float64
Hower, if by chance those are strings. It might be better to do use pd.to_timedelta
pd.to_timedelta(s).dt.total_seconds()
2016-11-09 00:07:00 15.0
2016-11-09 00:07:15 1214.0
2016-11-09 00:07:30 36015.0
dtype: float64
I need to SUM column something by month:
date something
2010-01-02
2010-01-03
2010-01-04
2010-01-07
2010-01-10
2010-01-12
2010-01-13
2010-01-14
2010-01-15
2010-01-16
2010-01-17
2010-01-18 3
2010-01-19 1
2010-01-21
2010-01-22 11
2010-01-23 1
2010-01-24
2010-01-25
2010-01-26
2010-01-27
2010-01-28
2010-01-29
2010-01-30
2010-01-05 5
2010-01-06 8
2010-01-09
2010-01-08 3
2010-01-11
2010-01-01
2010-01-20 0
2010-01-31 13
Output should be e.g. for JAN 2010 SUM OF SOMETHING 45:
date something
2010-01 45
How to write SQL query for that?
This is a simple aggregation based on the month of the date column:
select to_char("date", 'yyyy-mm'), sum(something)
from the_table
group by to_char("date", 'yyyy-mm')
This assumes the column date has the data type date (or timestamp)