Drop multiple columns in python - pandas
I have 1000 columns in my dataframe, and I want to drop 3rd, 5th, 7th, 9th....999th columns, but need to keep the 1st column. I am not sure if df.drop function can deal with this issue. any ideas? The following code drops all odd columns include 1st column. but I need to keep the 1st column
col_to_drop = df.columns[np.array([i for i in range(df.shape[1]) if i%2!= 1 and i != 1])]
df.drop(col_to_drop, axis=1)
Try this:
df = pd.DataFrame(np.arange(100).reshape(10,-1), columns=[*'ABCDEFGHIJ'])
df.drop(df.columns[np.r_[2,4,5,6,9]], axis=1)
# or df.drop(df.columns[np.array([2,4,5,6,9])], axis=1)
Output:
A B D H I
0 0 1 3 7 8
1 10 11 13 17 18
2 20 21 23 27 28
3 30 31 33 37 38
4 40 41 43 47 48
5 50 51 53 57 58
6 60 61 63 67 68
7 70 71 73 77 78
8 80 81 83 87 88
9 90 91 93 97 98
Update for 1,000 columns.
Drops every odd colum except 1. -> 3,5,7,9...999
col_to_drop = df.columns[np.array([i for i in range(df.shape[1]) if i%2 != 0 and i != 0])]
df.drop(col_to_drop, axis=1)
I think you can try like below:
>>> df
A B C D E
0 A1 B1 C1 D1 E1
1 A2 B2 C2 D2 E2
2 A3 B3 C3 D3 E3
3 A4 B4 C4 D4 E4
4 A5 B5 C5 D5 E5
Drop unwanted columns:
>>> df.drop(ex.columns[[0, 4, 2]], axis = 1, inplace = True)
Result:
>>> df
B D
0 B1 D1
1 B2 D2
2 B3 D3
3 B4 D4
4 B5 D5
Another way around:
>>> df
A B C D E F G H I J K L M N O P
0 24 27 61 40 12 20 52 95 47 23 92 97 86 73 89 84
1 88 33 39 38 87 7 38 31 77 0 66 41 30 34 80 4
2 8 45 68 99 21 66 6 54 28 48 29 18 12 56 30 3
3 22 79 37 1 32 5 53 13 0 57 54 97 71 86 84 12
4 94 19 8 99 91 91 77 90 56 50 41 4 70 36 56 58
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
95 70 22 73 15 41 50 81 38 31 78 94 72 62 56 78 17
96 13 45 40 83 25 24 73 30 67 85 98 63 32 1 70 97
97 24 58 74 64 0 43 36 27 17 76 51 36 85 39 15 77
98 29 3 73 57 39 70 45 53 83 98 92 51 49 45 85 71
99 74 22 77 87 4 16 69 77 39 88 71 73 39 8 63 34
[100 rows x 16 columns]
As you want to drop all the odd columns hence other way around is to opt all the even column then while using .loc indexer as follows.
>>> df.loc[:, ::2]
A C E G I K M O
0 24 61 12 52 47 92 86 89
1 88 39 87 38 77 66 30 80
2 8 68 21 6 28 29 12 30
3 22 37 32 53 0 54 71 84
4 94 8 91 77 56 41 70 56
.. .. .. .. .. .. .. .. ..
95 70 73 41 81 31 94 62 78
96 13 40 25 73 67 98 32 70
97 24 74 0 36 17 51 85 15
98 29 73 39 45 83 92 49 85
99 74 77 4 69 39 71 39 63
[100 rows x 8 columns]
OR
>>> df[df.columns[::2]]
A C E G I K M O
0 24 61 12 52 47 92 86 89
1 88 39 87 38 77 66 30 80
2 8 68 21 6 28 29 12 30
3 22 37 32 53 0 54 71 84
4 94 8 91 77 56 41 70 56
.. .. .. .. .. .. .. .. ..
95 70 73 41 81 31 94 62 78
96 13 40 25 73 67 98 32 70
97 24 74 0 36 17 51 85 15
98 29 73 39 45 83 92 49 85
99 74 77 4 69 39 71 39 63
[100 rows x 8 columns]
A little change from scott Boston's answer works perfectly for this case
col_to_drop = df.columns[np.array([i for i in range(df.shape[1]) if i%2 != 1 and i != 0])]
df.drop(col_to_drop, axis=1)
Related
Dataframe Operation Splicing
I have a single column dataframe without headers and I want to split it into multiple columns as follows The current dataframe - 1 2 3 4 5 . . 100 I want to represent it as - 1 6 .. .. 96 2 7 .. .. 97 3 8 .. .. 98 4 9 .. .. 99 5 10 .. .. 100
Assuming such a DataFrame: df = pd.DataFrame({'col': range(1, 101)}) you can use the underlying numpy array to reshape: df2 = pd.DataFrame(df['col'].to_numpy().reshape(5, -1, order='F')) output: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 \ 0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 1 2 7 12 17 22 27 32 37 42 47 52 57 62 67 72 77 82 87 92 2 3 8 13 18 23 28 33 38 43 48 53 58 63 68 73 78 83 88 93 3 4 9 14 19 24 29 34 39 44 49 54 59 64 69 74 79 84 89 94 4 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 19 0 96 1 97 2 98 3 99 4 100
Optimize Oracle Query - for View
I have single question, to optimize a View Query using Oracle. Is there anyone able to help me out. I'm about to select circuitname and oldname (the old name of circuitname) from only one table. But each circuitname, possibly has more than 1 sub name (oldname, sub-oldname, etc), and i want to join all of them with this View Query. This query is good, but when I select 1 by 1 circuitname, oldname, sub-oldname, etc until 20 circuitname, it takes too much time. I want to optimize the query, to perform less time, before the app timeout. Here's the query CREATE OR REPLACE FORCE EDITIONABLE VIEW "IMS"."VW_SI_CASCADE" ("R1", "L1", "L2", "L3", "L4", "L5", "L6", "L7", "L8", "L9", "L10", "L11", "L12", "L13", "L14", "L15", "L16", "L17", "L18", "L19", "L20", "L21", "M1", "M2", "M3", "M4", "M5", "M6", "M7", "M8", "M9", "M10", "M11", "M12", "M13", "M14", "M15", "M16", "M17", "M18", "M19", "M20", "M21") AS SELECT ROWNUM AS R1, V1.OLDNAME AS L1, V1.CIRCUITNAME AS L2, V2.CIRCUITNAME AS L3, V3.CIRCUITNAME AS L4, V4.CIRCUITNAME AS L5, V5.CIRCUITNAME AS L6, V6.CIRCUITNAME AS L7, V7.CIRCUITNAME AS L8, V8.CIRCUITNAME AS L9, V9.CIRCUITNAME AS L10, V10.CIRCUITNAME AS L11, V11.CIRCUITNAME AS L12, V12.CIRCUITNAME AS L13, V13.CIRCUITNAME AS L14, V14.CIRCUITNAME AS L15, V15.CIRCUITNAME AS L16, V16.CIRCUITNAME AS L17, V17.CIRCUITNAME AS L18, V18.CIRCUITNAME AS L19, V19.CIRCUITNAME AS L20, V20.CIRCUITNAME AS L21, V1.OLDCIRCUITID AS M1, V1.CIRCUITID AS M2, V2.CIRCUITID AS M3, V3.CIRCUITID AS M4, V4.CIRCUITID AS M5, V5.CIRCUITID AS M6, V6.CIRCUITID AS M7, V7.CIRCUITID AS M8, V8.CIRCUITID AS M9, V9.CIRCUITID AS M10, V10.CIRCUITID AS M11, V11.CIRCUITID AS M12, V12.CIRCUITID AS M13, V13.CIRCUITID AS M14, V14.CIRCUITID AS M15, V15.CIRCUITID AS M16, V16.CIRCUITID AS M17, V17.CIRCUITID AS M18, V18.CIRCUITID AS M19, V19.CIRCUITID AS M20, V20.CIRCUITID AS M21 FROM VW_SI_OLDNAME V1 LEFT JOIN VW_SI_OLDNAME V2 ON V2.OLDNAME = V1.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V3 ON V3.OLDNAME = V2.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V4 ON V4.OLDNAME = V3.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V5 ON V5.OLDNAME = V4.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V6 ON V6.OLDNAME = V5.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V7 ON V7.OLDNAME = V6.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V8 ON V8.OLDNAME = V7.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V9 ON V9.OLDNAME = V8.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V10 ON V10.OLDNAME = V9.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V11 ON V11.OLDNAME = V10.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V12 ON V12.OLDNAME = V11.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V13 ON V13.OLDNAME = V12.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V14 ON V14.OLDNAME = V13.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V15 ON V15.OLDNAME = V14.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V16 ON V16.OLDNAME = V15.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V17 ON V17.OLDNAME = V16.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V18 ON V18.OLDNAME = V17.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V19 ON V19.OLDNAME = V18.CIRCUITNAME LEFT JOIN VW_SI_OLDNAME V20 ON V20.OLDNAME = V19.CIRCUITNAME; Pardon if I have any mistakes, and kindly need your help guys, my fellow heroes!
You're querying the view twenty times. That's going to be slow. You could try using subquery factoring, so you query the view once and re-use the result set multiple times. CREATE OR REPLACE FORCE EDITIONABLE VIEW "IMS"."VW_SI_CASCADE" ("R1", "L1", "L2", "L3", "L4", "L5", "L6", "L7", "L8", "L9", "L10", "L11", "L12", "L13", "L14", "L15", "L16", "L17", "L18", "L19", "L20", "L21", "M1", "M2", "M3", "M4", "M5", "M6", "M7", "M8", "M9", "M10", "M11", "M12", "M13", "M14", "M15", "M16", "M17", "M18", "M19", "M20", "M21") AS with cte as ( select * from VW_SI_OLDNAME ) SELECT ROWNUM AS R1, V1.OLDNAME AS L1, V1.CIRCUITNAME AS L2, V2.CIRCUITNAME AS L3, V3.CIRCUITNAME AS L4, V4.CIRCUITNAME AS L5, V5.CIRCUITNAME AS L6, V6.CIRCUITNAME AS L7, V7.CIRCUITNAME AS L8, V8.CIRCUITNAME AS L9, V9.CIRCUITNAME AS L10, V10.CIRCUITNAME AS L11, V11.CIRCUITNAME AS L12, V12.CIRCUITNAME AS L13, V13.CIRCUITNAME AS L14, V14.CIRCUITNAME AS L15, V15.CIRCUITNAME AS L16, V16.CIRCUITNAME AS L17, V17.CIRCUITNAME AS L18, V18.CIRCUITNAME AS L19, V19.CIRCUITNAME AS L20, V20.CIRCUITNAME AS L21, V1.OLDCIRCUITID AS M1, V1.CIRCUITID AS M2, V2.CIRCUITID AS M3, V3.CIRCUITID AS M4, V4.CIRCUITID AS M5, V5.CIRCUITID AS M6, V6.CIRCUITID AS M7, V7.CIRCUITID AS M8, V8.CIRCUITID AS M9, V9.CIRCUITID AS M10, V10.CIRCUITID AS M11, V11.CIRCUITID AS M12, V12.CIRCUITID AS M13, V13.CIRCUITID AS M14, V14.CIRCUITID AS M15, V15.CIRCUITID AS M16, V16.CIRCUITID AS M17, V17.CIRCUITID AS M18, V18.CIRCUITID AS M19, V19.CIRCUITID AS M20, V20.CIRCUITID AS M21 FROM cte V1 LEFT JOIN cte V2 ON V2.OLDNAME = V1.CIRCUITNAME LEFT JOIN cte V3 ON V3.OLDNAME = V2.CIRCUITNAME LEFT JOIN cte V4 ON V4.OLDNAME = V3.CIRCUITNAME LEFT JOIN cte V5 ON V5.OLDNAME = V4.CIRCUITNAME LEFT JOIN cte V6 ON V6.OLDNAME = V5.CIRCUITNAME LEFT JOIN cte V7 ON V7.OLDNAME = V6.CIRCUITNAME LEFT JOIN cte V8 ON V8.OLDNAME = V7.CIRCUITNAME LEFT JOIN cte V9 ON V9.OLDNAME = V8.CIRCUITNAME LEFT JOIN cte V10 ON V10.OLDNAME = V9.CIRCUITNAME LEFT JOIN cte V11 ON V11.OLDNAME = V10.CIRCUITNAME LEFT JOIN cte V12 ON V12.OLDNAME = V11.CIRCUITNAME LEFT JOIN cte V13 ON V13.OLDNAME = V12.CIRCUITNAME LEFT JOIN cte V14 ON V14.OLDNAME = V13.CIRCUITNAME LEFT JOIN cte V15 ON V15.OLDNAME = V14.CIRCUITNAME LEFT JOIN cte V16 ON V16.OLDNAME = V15.CIRCUITNAME LEFT JOIN cte V17 ON V17.OLDNAME = V16.CIRCUITNAME LEFT JOIN cte V18 ON V18.OLDNAME = V17.CIRCUITNAME LEFT JOIN cte V19 ON V19.OLDNAME = V18.CIRCUITNAME LEFT JOIN cte V20 ON V20.OLDNAME = V19.CIRCUITNAME; Beyond that you may need to consider the internal logic of the view. There may be a better way to re-construct the chain of names. You also might be exposed to cycles in the re-naming.
For example, your VW_SI_OLDNAME contains a data like this: with VW_SI_OLDNAME as ( select 'name_'||to_char(level-1,'fm0000') OLDNAME ,'name_'||to_char(level ,'fm0000') CIRCUITNAME ,level-1 OLDCIRCUITID ,level CIRCUITID from dual connect by level<=100 ) select * from VW_SI_OLDNAME; Example: OLDNAME CIRCUITNAM OLDCIRCUITID CIRCUITID ---------- ---------- ------------ ---------- name_0000 name_0001 0 1 name_0001 name_0002 1 2 name_0002 name_0003 2 3 name_0003 name_0004 3 4 name_0004 name_0005 4 5 ... name_0028 name_0029 28 29 name_0029 name_0030 29 30 name_0030 name_0031 30 31 ... name_0097 name_0098 97 98 name_0098 name_0099 98 99 name_0099 name_0100 99 100 In this case, you can easily create your view using MATCH_RECOGNIZE that will scan your VW_SI_OLDNAME just once: VW_SI_CASCADE as ( select * from VW_SI_OLDNAME match_recognize( MEASURES classifier() event ,match_number() match ,prev(CIRCUITID,1) CIRCUITID_1 ,prev(CIRCUITID,2) CIRCUITID_2 ,prev(CIRCUITID,3) CIRCUITID_3 ,prev(CIRCUITID,4) CIRCUITID_4 ,prev(CIRCUITID,5) CIRCUITID_5 ,prev(CIRCUITID,6) CIRCUITID_6 ,prev(CIRCUITID,7) CIRCUITID_7 ,prev(CIRCUITID,8) CIRCUITID_8 ,prev(CIRCUITNAME,1) CIRCUITNAME_1 ,prev(CIRCUITNAME,2) CIRCUITNAME_2 ,prev(CIRCUITNAME,3) CIRCUITNAME_3 ,prev(CIRCUITNAME,4) CIRCUITNAME_4 ,prev(CIRCUITNAME,5) CIRCUITNAME_5 ,prev(CIRCUITNAME,6) CIRCUITNAME_6 ,prev(CIRCUITNAME,7) CIRCUITNAME_7 ,prev(CIRCUITNAME,8) CIRCUITNAME_8 ALL ROWS PER MATCH PATTERN (p) DEFINE p AS OLDNAME = prev(CIRCUITNAME) ) ) I've created only 1-8 just to make this example shorter. Full example with test data: with VW_SI_OLDNAME as ( select 'name_'||to_char(level-1,'fm0000') OLDNAME ,'name_'||to_char(level ,'fm0000') CIRCUITNAME ,level-1 OLDCIRCUITID ,level CIRCUITID from dual connect by level<=100 ) ,VW_SI_CASCADE as ( select * from VW_SI_OLDNAME match_recognize( MEASURES classifier() event ,match_number() match ,prev(CIRCUITID,1) CIRCUITID_1 ,prev(CIRCUITID,2) CIRCUITID_2 ,prev(CIRCUITID,3) CIRCUITID_3 ,prev(CIRCUITID,4) CIRCUITID_4 ,prev(CIRCUITID,5) CIRCUITID_5 ,prev(CIRCUITID,6) CIRCUITID_6 ,prev(CIRCUITID,7) CIRCUITID_7 ,prev(CIRCUITID,8) CIRCUITID_8 ,prev(CIRCUITNAME,1) CIRCUITNAME_1 ,prev(CIRCUITNAME,2) CIRCUITNAME_2 ,prev(CIRCUITNAME,3) CIRCUITNAME_3 ,prev(CIRCUITNAME,4) CIRCUITNAME_4 ,prev(CIRCUITNAME,5) CIRCUITNAME_5 ,prev(CIRCUITNAME,6) CIRCUITNAME_6 ,prev(CIRCUITNAME,7) CIRCUITNAME_7 ,prev(CIRCUITNAME,8) CIRCUITNAME_8 ALL ROWS PER MATCH PATTERN (p) DEFINE p AS OLDNAME = prev(CIRCUITNAME) ) ) select * from VW_SI_CASCADE; Results: EVENT MATCH CIRCUITID_1 CIRCUITID_2 CIRCUITID_3 CIRCUITID_4 CIRCUITID_5 CIRCUITID_6 CIRCUITID_7 CIRCUITID_8 CIRCUITNAM CIRCUITNAM CIRCUITNAM CIRCUITNAM CIRCUITNAM CIRCUITNAM CIRCUITNAM CIRCUITNAM OLDNAME CIRCUITNAM OLDCIRCUITID CIRCUITID ----- ------ ----------- ----------- ----------- ----------- ----------- ----------- ----------- ----------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ------------ ---------- P 1 1 name_0001 name_0001 name_0002 1 2 P 2 2 1 name_0002 name_0001 name_0002 name_0003 2 3 P 3 3 2 1 name_0003 name_0002 name_0001 name_0003 name_0004 3 4 P 4 4 3 2 1 name_0004 name_0003 name_0002 name_0001 name_0004 name_0005 4 5 P 5 5 4 3 2 1 name_0005 name_0004 name_0003 name_0002 name_0001 name_0005 name_0006 5 6 P 6 6 5 4 3 2 1 name_0006 name_0005 name_0004 name_0003 name_0002 name_0001 name_0006 name_0007 6 7 P 7 7 6 5 4 3 2 1 name_0007 name_0006 name_0005 name_0004 name_0003 name_0002 name_0001 name_0007 name_0008 7 8 P 8 8 7 6 5 4 3 2 1 name_0008 name_0007 name_0006 name_0005 name_0004 name_0003 name_0002 name_0001 name_0008 name_0009 8 9 P 9 9 8 7 6 5 4 3 2 name_0009 name_0008 name_0007 name_0006 name_0005 name_0004 name_0003 name_0002 name_0009 name_0010 9 10 P 10 10 9 8 7 6 5 4 3 name_0010 name_0009 name_0008 name_0007 name_0006 name_0005 name_0004 name_0003 name_0010 name_0011 10 11 P 11 11 10 9 8 7 6 5 4 name_0011 name_0010 name_0009 name_0008 name_0007 name_0006 name_0005 name_0004 name_0011 name_0012 11 12 P 12 12 11 10 9 8 7 6 5 name_0012 name_0011 name_0010 name_0009 name_0008 name_0007 name_0006 name_0005 name_0012 name_0013 12 13 P 13 13 12 11 10 9 8 7 6 name_0013 name_0012 name_0011 name_0010 name_0009 name_0008 name_0007 name_0006 name_0013 name_0014 13 14 P 14 14 13 12 11 10 9 8 7 name_0014 name_0013 name_0012 name_0011 name_0010 name_0009 name_0008 name_0007 name_0014 name_0015 14 15 P 15 15 14 13 12 11 10 9 8 name_0015 name_0014 name_0013 name_0012 name_0011 name_0010 name_0009 name_0008 name_0015 name_0016 15 16 P 16 16 15 14 13 12 11 10 9 name_0016 name_0015 name_0014 name_0013 name_0012 name_0011 name_0010 name_0009 name_0016 name_0017 16 17 P 17 17 16 15 14 13 12 11 10 name_0017 name_0016 name_0015 name_0014 name_0013 name_0012 name_0011 name_0010 name_0017 name_0018 17 18 P 18 18 17 16 15 14 13 12 11 name_0018 name_0017 name_0016 name_0015 name_0014 name_0013 name_0012 name_0011 name_0018 name_0019 18 19 P 19 19 18 17 16 15 14 13 12 name_0019 name_0018 name_0017 name_0016 name_0015 name_0014 name_0013 name_0012 name_0019 name_0020 19 20 P 20 20 19 18 17 16 15 14 13 name_0020 name_0019 name_0018 name_0017 name_0016 name_0015 name_0014 name_0013 name_0020 name_0021 20 21 P 21 21 20 19 18 17 16 15 14 name_0021 name_0020 name_0019 name_0018 name_0017 name_0016 name_0015 name_0014 name_0021 name_0022 21 22 P 22 22 21 20 19 18 17 16 15 name_0022 name_0021 name_0020 name_0019 name_0018 name_0017 name_0016 name_0015 name_0022 name_0023 22 23 P 23 23 22 21 20 19 18 17 16 name_0023 name_0022 name_0021 name_0020 name_0019 name_0018 name_0017 name_0016 name_0023 name_0024 23 24 P 24 24 23 22 21 20 19 18 17 name_0024 name_0023 name_0022 name_0021 name_0020 name_0019 name_0018 name_0017 name_0024 name_0025 24 25 P 25 25 24 23 22 21 20 19 18 name_0025 name_0024 name_0023 name_0022 name_0021 name_0020 name_0019 name_0018 name_0025 name_0026 25 26 P 26 26 25 24 23 22 21 20 19 name_0026 name_0025 name_0024 name_0023 name_0022 name_0021 name_0020 name_0019 name_0026 name_0027 26 27 P 27 27 26 25 24 23 22 21 20 name_0027 name_0026 name_0025 name_0024 name_0023 name_0022 name_0021 name_0020 name_0027 name_0028 27 28 P 28 28 27 26 25 24 23 22 21 name_0028 name_0027 name_0026 name_0025 name_0024 name_0023 name_0022 name_0021 name_0028 name_0029 28 29 P 29 29 28 27 26 25 24 23 22 name_0029 name_0028 name_0027 name_0026 name_0025 name_0024 name_0023 name_0022 name_0029 name_0030 29 30 P 30 30 29 28 27 26 25 24 23 name_0030 name_0029 name_0028 name_0027 name_0026 name_0025 name_0024 name_0023 name_0030 name_0031 30 31 P 31 31 30 29 28 27 26 25 24 name_0031 name_0030 name_0029 name_0028 name_0027 name_0026 name_0025 name_0024 name_0031 name_0032 31 32 P 32 32 31 30 29 28 27 26 25 name_0032 name_0031 name_0030 name_0029 name_0028 name_0027 name_0026 name_0025 name_0032 name_0033 32 33 P 33 33 32 31 30 29 28 27 26 name_0033 name_0032 name_0031 name_0030 name_0029 name_0028 name_0027 name_0026 name_0033 name_0034 33 34 P 34 34 33 32 31 30 29 28 27 name_0034 name_0033 name_0032 name_0031 name_0030 name_0029 name_0028 name_0027 name_0034 name_0035 34 35 P 35 35 34 33 32 31 30 29 28 name_0035 name_0034 name_0033 name_0032 name_0031 name_0030 name_0029 name_0028 name_0035 name_0036 35 36 P 36 36 35 34 33 32 31 30 29 name_0036 name_0035 name_0034 name_0033 name_0032 name_0031 name_0030 name_0029 name_0036 name_0037 36 37 P 37 37 36 35 34 33 32 31 30 name_0037 name_0036 name_0035 name_0034 name_0033 name_0032 name_0031 name_0030 name_0037 name_0038 37 38 P 38 38 37 36 35 34 33 32 31 name_0038 name_0037 name_0036 name_0035 name_0034 name_0033 name_0032 name_0031 name_0038 name_0039 38 39 P 39 39 38 37 36 35 34 33 32 name_0039 name_0038 name_0037 name_0036 name_0035 name_0034 name_0033 name_0032 name_0039 name_0040 39 40 P 40 40 39 38 37 36 35 34 33 name_0040 name_0039 name_0038 name_0037 name_0036 name_0035 name_0034 name_0033 name_0040 name_0041 40 41 P 41 41 40 39 38 37 36 35 34 name_0041 name_0040 name_0039 name_0038 name_0037 name_0036 name_0035 name_0034 name_0041 name_0042 41 42 P 42 42 41 40 39 38 37 36 35 name_0042 name_0041 name_0040 name_0039 name_0038 name_0037 name_0036 name_0035 name_0042 name_0043 42 43 P 43 43 42 41 40 39 38 37 36 name_0043 name_0042 name_0041 name_0040 name_0039 name_0038 name_0037 name_0036 name_0043 name_0044 43 44 P 44 44 43 42 41 40 39 38 37 name_0044 name_0043 name_0042 name_0041 name_0040 name_0039 name_0038 name_0037 name_0044 name_0045 44 45 P 45 45 44 43 42 41 40 39 38 name_0045 name_0044 name_0043 name_0042 name_0041 name_0040 name_0039 name_0038 name_0045 name_0046 45 46 P 46 46 45 44 43 42 41 40 39 name_0046 name_0045 name_0044 name_0043 name_0042 name_0041 name_0040 name_0039 name_0046 name_0047 46 47 P 47 47 46 45 44 43 42 41 40 name_0047 name_0046 name_0045 name_0044 name_0043 name_0042 name_0041 name_0040 name_0047 name_0048 47 48 P 48 48 47 46 45 44 43 42 41 name_0048 name_0047 name_0046 name_0045 name_0044 name_0043 name_0042 name_0041 name_0048 name_0049 48 49 P 49 49 48 47 46 45 44 43 42 name_0049 name_0048 name_0047 name_0046 name_0045 name_0044 name_0043 name_0042 name_0049 name_0050 49 50 P 50 50 49 48 47 46 45 44 43 name_0050 name_0049 name_0048 name_0047 name_0046 name_0045 name_0044 name_0043 name_0050 name_0051 50 51 P 51 51 50 49 48 47 46 45 44 name_0051 name_0050 name_0049 name_0048 name_0047 name_0046 name_0045 name_0044 name_0051 name_0052 51 52 P 52 52 51 50 49 48 47 46 45 name_0052 name_0051 name_0050 name_0049 name_0048 name_0047 name_0046 name_0045 name_0052 name_0053 52 53 P 53 53 52 51 50 49 48 47 46 name_0053 name_0052 name_0051 name_0050 name_0049 name_0048 name_0047 name_0046 name_0053 name_0054 53 54 P 54 54 53 52 51 50 49 48 47 name_0054 name_0053 name_0052 name_0051 name_0050 name_0049 name_0048 name_0047 name_0054 name_0055 54 55 P 55 55 54 53 52 51 50 49 48 name_0055 name_0054 name_0053 name_0052 name_0051 name_0050 name_0049 name_0048 name_0055 name_0056 55 56 P 56 56 55 54 53 52 51 50 49 name_0056 name_0055 name_0054 name_0053 name_0052 name_0051 name_0050 name_0049 name_0056 name_0057 56 57 P 57 57 56 55 54 53 52 51 50 name_0057 name_0056 name_0055 name_0054 name_0053 name_0052 name_0051 name_0050 name_0057 name_0058 57 58 P 58 58 57 56 55 54 53 52 51 name_0058 name_0057 name_0056 name_0055 name_0054 name_0053 name_0052 name_0051 name_0058 name_0059 58 59 P 59 59 58 57 56 55 54 53 52 name_0059 name_0058 name_0057 name_0056 name_0055 name_0054 name_0053 name_0052 name_0059 name_0060 59 60 P 60 60 59 58 57 56 55 54 53 name_0060 name_0059 name_0058 name_0057 name_0056 name_0055 name_0054 name_0053 name_0060 name_0061 60 61 P 61 61 60 59 58 57 56 55 54 name_0061 name_0060 name_0059 name_0058 name_0057 name_0056 name_0055 name_0054 name_0061 name_0062 61 62 P 62 62 61 60 59 58 57 56 55 name_0062 name_0061 name_0060 name_0059 name_0058 name_0057 name_0056 name_0055 name_0062 name_0063 62 63 P 63 63 62 61 60 59 58 57 56 name_0063 name_0062 name_0061 name_0060 name_0059 name_0058 name_0057 name_0056 name_0063 name_0064 63 64 P 64 64 63 62 61 60 59 58 57 name_0064 name_0063 name_0062 name_0061 name_0060 name_0059 name_0058 name_0057 name_0064 name_0065 64 65 P 65 65 64 63 62 61 60 59 58 name_0065 name_0064 name_0063 name_0062 name_0061 name_0060 name_0059 name_0058 name_0065 name_0066 65 66 P 66 66 65 64 63 62 61 60 59 name_0066 name_0065 name_0064 name_0063 name_0062 name_0061 name_0060 name_0059 name_0066 name_0067 66 67 P 67 67 66 65 64 63 62 61 60 name_0067 name_0066 name_0065 name_0064 name_0063 name_0062 name_0061 name_0060 name_0067 name_0068 67 68 P 68 68 67 66 65 64 63 62 61 name_0068 name_0067 name_0066 name_0065 name_0064 name_0063 name_0062 name_0061 name_0068 name_0069 68 69 P 69 69 68 67 66 65 64 63 62 name_0069 name_0068 name_0067 name_0066 name_0065 name_0064 name_0063 name_0062 name_0069 name_0070 69 70 P 70 70 69 68 67 66 65 64 63 name_0070 name_0069 name_0068 name_0067 name_0066 name_0065 name_0064 name_0063 name_0070 name_0071 70 71 P 71 71 70 69 68 67 66 65 64 name_0071 name_0070 name_0069 name_0068 name_0067 name_0066 name_0065 name_0064 name_0071 name_0072 71 72 P 72 72 71 70 69 68 67 66 65 name_0072 name_0071 name_0070 name_0069 name_0068 name_0067 name_0066 name_0065 name_0072 name_0073 72 73 P 73 73 72 71 70 69 68 67 66 name_0073 name_0072 name_0071 name_0070 name_0069 name_0068 name_0067 name_0066 name_0073 name_0074 73 74 P 74 74 73 72 71 70 69 68 67 name_0074 name_0073 name_0072 name_0071 name_0070 name_0069 name_0068 name_0067 name_0074 name_0075 74 75 P 75 75 74 73 72 71 70 69 68 name_0075 name_0074 name_0073 name_0072 name_0071 name_0070 name_0069 name_0068 name_0075 name_0076 75 76 P 76 76 75 74 73 72 71 70 69 name_0076 name_0075 name_0074 name_0073 name_0072 name_0071 name_0070 name_0069 name_0076 name_0077 76 77 P 77 77 76 75 74 73 72 71 70 name_0077 name_0076 name_0075 name_0074 name_0073 name_0072 name_0071 name_0070 name_0077 name_0078 77 78 P 78 78 77 76 75 74 73 72 71 name_0078 name_0077 name_0076 name_0075 name_0074 name_0073 name_0072 name_0071 name_0078 name_0079 78 79 P 79 79 78 77 76 75 74 73 72 name_0079 name_0078 name_0077 name_0076 name_0075 name_0074 name_0073 name_0072 name_0079 name_0080 79 80 P 80 80 79 78 77 76 75 74 73 name_0080 name_0079 name_0078 name_0077 name_0076 name_0075 name_0074 name_0073 name_0080 name_0081 80 81 P 81 81 80 79 78 77 76 75 74 name_0081 name_0080 name_0079 name_0078 name_0077 name_0076 name_0075 name_0074 name_0081 name_0082 81 82 P 82 82 81 80 79 78 77 76 75 name_0082 name_0081 name_0080 name_0079 name_0078 name_0077 name_0076 name_0075 name_0082 name_0083 82 83 P 83 83 82 81 80 79 78 77 76 name_0083 name_0082 name_0081 name_0080 name_0079 name_0078 name_0077 name_0076 name_0083 name_0084 83 84 P 84 84 83 82 81 80 79 78 77 name_0084 name_0083 name_0082 name_0081 name_0080 name_0079 name_0078 name_0077 name_0084 name_0085 84 85 P 85 85 84 83 82 81 80 79 78 name_0085 name_0084 name_0083 name_0082 name_0081 name_0080 name_0079 name_0078 name_0085 name_0086 85 86 P 86 86 85 84 83 82 81 80 79 name_0086 name_0085 name_0084 name_0083 name_0082 name_0081 name_0080 name_0079 name_0086 name_0087 86 87 P 87 87 86 85 84 83 82 81 80 name_0087 name_0086 name_0085 name_0084 name_0083 name_0082 name_0081 name_0080 name_0087 name_0088 87 88 P 88 88 87 86 85 84 83 82 81 name_0088 name_0087 name_0086 name_0085 name_0084 name_0083 name_0082 name_0081 name_0088 name_0089 88 89 P 89 89 88 87 86 85 84 83 82 name_0089 name_0088 name_0087 name_0086 name_0085 name_0084 name_0083 name_0082 name_0089 name_0090 89 90 P 90 90 89 88 87 86 85 84 83 name_0090 name_0089 name_0088 name_0087 name_0086 name_0085 name_0084 name_0083 name_0090 name_0091 90 91 P 91 91 90 89 88 87 86 85 84 name_0091 name_0090 name_0089 name_0088 name_0087 name_0086 name_0085 name_0084 name_0091 name_0092 91 92 P 92 92 91 90 89 88 87 86 85 name_0092 name_0091 name_0090 name_0089 name_0088 name_0087 name_0086 name_0085 name_0092 name_0093 92 93 P 93 93 92 91 90 89 88 87 86 name_0093 name_0092 name_0091 name_0090 name_0089 name_0088 name_0087 name_0086 name_0093 name_0094 93 94 P 94 94 93 92 91 90 89 88 87 name_0094 name_0093 name_0092 name_0091 name_0090 name_0089 name_0088 name_0087 name_0094 name_0095 94 95 P 95 95 94 93 92 91 90 89 88 name_0095 name_0094 name_0093 name_0092 name_0091 name_0090 name_0089 name_0088 name_0095 name_0096 95 96 P 96 96 95 94 93 92 91 90 89 name_0096 name_0095 name_0094 name_0093 name_0092 name_0091 name_0090 name_0089 name_0096 name_0097 96 97 P 97 97 96 95 94 93 92 91 90 name_0097 name_0096 name_0095 name_0094 name_0093 name_0092 name_0091 name_0090 name_0097 name_0098 97 98 P 98 98 97 96 95 94 93 92 91 name_0098 name_0097 name_0096 name_0095 name_0094 name_0093 name_0092 name_0091 name_0098 name_0099 98 99 P 99 99 98 97 96 95 94 93 92 name_0099 name_0098 name_0097 name_0096 name_0095 name_0094 name_0093 name_0092 name_0099 name_0100 99 100 So you can take this example and modify it for your needs: add required number of previous data(9-20), change column names, remove unneeded columns.
TLS handshake failure
I have an embedded IOT device connected to real time DB FireBase. However, the server cancel the TLS handshake with fatal alert 0x46 = protocol_version. I have managed to get an hex dump of the whole handshake process. Can someone explain me why the handshake was canceled? My device is using TLS V1.2 as firebase expected as it can be seen in the header (16 3 3). Update: I've noticed that the first 4 bytes in the client random are zero. They need to be a calculated in seconds from 1970AD. There is no internal Time function (like we have in WIN32 or UNIX) and I used br_x509_minimal_set_time() to set the time for the BearSSL Thanks, Hexa dump: Client Hello (Output from my IOT device to the peer) 16 3 1 0 E2 1 0 0 DE 3 3 0 0 0 0 E6 1F 4 48 79 54 47 D3 D7 CB FF C8 69 43 C8 66 7 ED DE 9C AF 96 8C F2 E 97 AD A6 0 0 5A CC A9 CC A8 C0 2B C0 2F C0 2C C0 30 C0 AC C0 AD C0 AE C0 AF C0 23 C0 27 C0 24 C0 28 C0 9 C0 13 C0 A C0 14 C0 2D C0 31 C0 2E C0 32 C0 25 C0 29 C0 26 C0 2A C0 4 C0 E C0 5 C0 F 0 9C 0 9D C0 9C C0 9D C0 A0 C0 A1 0 3C 0 3D 0 2F 0 35 C0 8 C0 12 C0 3 C0 D 0 A 1 0 0 5B FF 1 0 1 0 0 0 0 24 0 22 0 0 1F 69 72 6F 6E 6D 61 78 73 6F 73 62 75 74 74 6F 6E 2E 66 69 72 65 62 61 73 65 69 6F 2E 63 6F 6D 0 D 0 16 0 14 4 3 3 3 5 3 6 3 2 3 4 1 3 1 5 1 6 1 2 1 0 A 0 A 0 8 0 17 0 18 0 19 0 1D 0 B 0 2 1 0 Server hello (Input from the peer) 16 3 3 0 5B 2 0 0 57 3 3 5F DD D0 79 2B C1 D 5C 7 F3 C1 95 A2 D9 36 8C C7 4B DA 9F A2 22 D4 31 CF C9 C1 AB 86 5D CC A2 20 E3 4A 89 12 38 A0 31 D 9A 0 76 D0 E9 AE F2 F 9D 56 4C 36 4E A0 39 A5 AD D1 39 6B 3 33 8F 88 CC A8 0 0 F 0 0 0 0 FF 1 0 1 0 0 B 0 2 1 0 Certificate (Input) 16 3 3 9 FB B 0 9 F7 0 9 F4 0 5 A0 30 82 5 9C 30 82 4 84 A0 3 2 1 2 2 11 0 F0 CB F2 8F DF 22 56 9C 8 0 0 0 0 2F EE 22 30 D 6 9 2A 86 48 86 F7 D 1 1 B 5 0 30 42 31 B 30 9 6 3 55 4 6 13 2 55 53 31 1E 30 1C 6 3 55 4 A 13 15 47 6F 6F 67 6C 65 20 54 72 75 73 74 20 53 65 72 76 69 63 65 73 31 13 30 11 6 3 55 4 3 13 A 47 54 53 20 43 41 20 31 4F 31 30 1E 17 D 32 30 30 32 32 30 31 39 33 36 30 36 5A 17 D 32 31 30 32 31 38 31 39 33 36 30 36 5A 30 68 31 B 30 9 6 3 55 4 6 13 2 55 53 31 13 30 11 6 3 55 4 8 13 A 43 61 6C 69 66 6F 72 6E 69 61 31 16 30 14 6 3 55 4 7 13 D 4D 6F 75 6E 74 61 69 6E 20 56 69 65 77 31 13 30 11 6 3 55 4 A 13 A 47 6F 6F 67 6C 65 20 4C 4C 43 31 17 30 15 6 3 55 4 3 13 E 66 69 72 65 62 61 73 65 69 6F 2E 63 6F 6D 30 82 1 22 30 D 6 9 2A 86 48 86 F7 D 1 1 1 5 0 3 82 1 F 0 30 82 1 A 2 82 1 1 0 DF 51 84 12 1 76 47 89 16 CC BC E2 2E F5 27 E4 D2 44 2D 3A F7 D8 F2 9F F2 44 95 F4 A9 57 F4 D8 B2 66 EB 83 36 1D B5 94 99 A0 21 3E 17 CE 64 5A 61 EB 7F 2F F0 5C BD 44 5A 3 B9 E1 74 A7 BD 28 F0 51 9E 59 CE 98 5B 66 E3 E 1 22 50 A3 AA 9E 57 35 D0 AC 11 F6 2D 58 A7 13 C4 49 FA 3C 89 65 60 6E F9 E8 10 95 71 37 4A FA 11 F 4F 1B FA 7D 8C DB EC 2F 4A D4 EE 2A 86 A5 2F 11 44 85 E3 6B 6E 94 A9 BE 5A C6 FD 30 82 EE BB 2B 3C 69 BD 2B 3A E7 83 15 C1 3C B6 AB AB 62 DC 13 F8 9B 51 41 DF 21 C6 70 BA 4D BD F0 77 60 16 34 56 F CF B5 CF 29 24 46 D 21 6 E6 37 B 4D AE 8E C8 8D 2E 43 7E 81 20 50 2C 70 F6 55 12 2C 7B 1 62 1B B3 2F 66 C1 1D 61 8C 56 13 C B0 90 3A A 2 0 4C 22 1B 33 14 35 DF DD 41 C3 1A 6A 7D EA 3E EC 73 3E 4F FB 5A 46 40 98 10 35 4D 8A 73 AB 3B 63 E9 2 3 1 0 1 A3 82 2 65 30 82 2 61 30 E 6 3 55 1D F 1 1 FF 4 4 3 2 5 A0 30 13 6 3 55 1D 25 4 C 30 A 6 8 2B 6 1 5 5 7 3 1 30 C 6 3 55 1D 13 1 1 FF 4 2 30 0 30 1D 6 3 55 1D E 4 16 4 14 DA 22 8D DC A2 42 AB 7B 5 4A 6E F6 D3 60 3E D2 58 2C 13 42 30 1F 6 3 55 1D 23 4 18 30 16 80 14 98 D1 F8 6E 10 EB CF 9B EC 60 9F 18 90 1B A0 EB 7D 9 FD 2B 30 64 6 8 2B 6 1 5 5 7 1 1 4 58 30 56 30 27 6 8 2B 6 1 5 5 7 30 1 86 1B 68 74 74 70 3A 2F 2F 6F 63 73 70 2E 70 6B 69 2E 67 6F 6F 67 2F 67 74 73 31 6F 31 30 2B 6 8 2B 6 1 5 5 7 30 2 86 1F 68 74 74 70 3A 2F 2F 70 6B 69 2E 67 6F 6F 67 2F 67 73 72 32 2F 47 54 53 31 4F 31 2E 63 72 74 30 2B 6 3 55 1D 11 4 24 30 22 82 E 66 69 72 65 62 61 73 65 69 6F 2E 63 6F 6D 82 10 2A 2E 66 69 72 65 62 61 73 65 69 6F 2E 63 6F 6D 30 21 6 3 55 1D 20 4 1A 30 18 30 8 6 6 67 81 C 1 2 2 30 C 6 A 2B 6 1 4 1 D6 79 2 5 3 30 2F 6 3 55 1D 1F 4 28 30 26 30 24 A0 22 A0 20 86 1E 68 74 74 70 3A 2F 2F 63 72 6C 2E 70 6B 69 2E 67 6F 6F 67 2F 47 54 53 31 4F 31 2E 63 72 6C 30 82 1 3 6 A 2B 6 1 4 1 D6 79 2 4 2 4 81 F4 4 81 F1 0 EF 0 75 0 F6 5C 94 2F D1 77 30 22 14 54 18 8 30 94 56 8E E3 4D 13 19 33 BF DF C 2F 20 B CC 4E F1 64 E3 0 0 1 70 64 50 91 CE 0 0 4 3 0 46 30 44 2 20 2 A0 A AA 37 B4 C9 31 3F E9 41 83 35 63 EF D3 EE 45 E2 D9 CE E4 E E8 A4 1 A2 FC 9C 91 51 F1 2 20 60 2B 2 EB 2E 4D 49 C6 E0 D2 87 33 B0 BA EF 4D 79 A3 BE BA A9 17 92 16 6A 26 4F FC 0 50 5C 51 0 76 0 44 94 65 2E B0 EE CE AF C4 40 7 D8 A8 FE 28 C0 DA E6 82 BE D8 CB 31 B5 3F D3 33 96 B5 B6 81 A8 0 0 1 70 64 50 92 27 0 0 4 3 0 47 30 45 2 20 4D 5 84 E0 5 CA A6 14 51 9D C6 3E 5D 84 D6 A6 84 80 80 C B7 1C DD 46 E2 73 61 40 C1 E3 90 D1 2 21 0 CD 4C B6 65 EE 99 5F B1 BE 1D D6 45 83 C F4 33 AB 5C 84 7B EA AC D0 61 C7 48 F2 CE 44 9C D7 64 30 D 6 9 2A 86 48 86 F7 D 1 1 B 5 0 3 82 1 1 0 93 6B AB 76 2D D2 BB 81 1B 91 28 49 78 BD 8F EB 36 13 E6 6C F2 E4 92 26 68 9 E6 5 8E CB D7 62 6A 3F 1E 97 8C 47 57 AD 46 2E 16 A5 78 27 A3 49 46 F1 AE 76 60 83 C1 BB 75 65 18 9B C7 B5 EA D7 F9 24 4D 90 FC B9 57 ED C1 73 6A 88 EA 41 AB FC C D3 96 39 8E 56 CA A3 A3 45 E9 D6 70 AD 1C E5 15 15 6 74 1E B9 C6 D 51 A AE 31 6F 9 3F 50 69 59 7 43 FF 6E 6B ED 1 A3 1C FD 4E 21 9D 5 EB B8 74 8D A7 43 DB B2 C5 2 79 EB A 8D 1A B9 8 39 E FA 6D 4B 5 92 A6 15 93 FE 87 81 C9 80 AC 56 1 1A D4 7 85 F1 70 FC 76 57 7D EF 7E 78 3D C5 E9 FE F D5 2E D3 B8 27 6 B4 85 58 B2 F5 3F 8B 31 DF F6 AA 5B B AB 3 9C A B2 FB C2 90 AF FB 22 A1 7B 8F 6C 64 F8 C7 B4 85 6A 17 2B 91 CC CB 7 75 7D FE F5 76 C8 D2 9E 8E C1 37 D9 E7 73 CA 3C 7A 4 59 EA 59 C8 93 8B CB 7A 78 51 C0 0 4 4E 30 82 4 4A 30 82 3 32 A0 3 2 1 2 2 D 1 E3 B4 9A A1 8D 8A A9 81 25 69 50 B8 30 D 6 9 2A 86 48 86 F7 D 1 1 B 5 0 30 4C 31 20 30 1E 6 3 55 4 B 13 17 47 6C 6F 62 61 6C 53 69 67 6E 20 52 6F 6F 74 20 43 41 20 2D 20 52 32 31 13 30 11 6 3 55 4 A 13 A 47 6C 6F 62 61 6C 53 69 67 6E 31 13 30 11 6 3 55 4 3 13 A 47 6C 6F 62 61 6C 53 69 67 6E 30 1E 17 D 31 37 30 36 31 35 30 30 30 30 34 32 5A 17 D 32 31 31 32 31 35 30 30 30 30 34 32 5A 30 42 31 B 30 9 6 3 55 4 6 13 2 55 53 31 1E 30 1C 6 3 55 4 A 13 15 47 6F 6F 67 6C 65 20 54 72 75 73 74 20 53 65 72 76 69 63 65 73 31 13 30 11 6 3 55 4 3 13 A 47 54 53 20 43 41 20 31 4F 31 30 82 1 22 30 D 6 9 2A 86 48 86 F7 D 1 1 1 5 0 3 82 1 F 0 30 82 1 A 2 82 1 1 0 D0 18 CF 45 D4 8B CD D3 9C E4 40 EF 7E B4 DD 69 21 1B C9 CF 3C 8E 4C 75 B9 F 31 19 84 3D 9E 3C 29 EF 50 D 10 93 6F 5 80 80 9F 2A A0 BD 12 4B 2 E1 3D 9F 58 16 24 FE 30 9F B 74 77 55 93 1D 4B F7 4D E1 92 82 10 F6 51 AC C C3 B2 22 94 F 34 6B 98 10 49 E7 B 9D 83 39 DD 20 C6 1C 2D EF D1 18 61 65 E7 23 83 20 A8 23 12 FF D2 24 7F D4 2F E7 44 6A 5B 4D D7 50 66 B0 AF 9E 42 63 5 FB E0 1C C4 63 61 AF 9F 6A 33 FF 62 97 BD 48 D9 D3 7C 14 67 DC 75 DC 2E 69 E8 F8 6D 78 69 D0 B7 10 5 B8 F1 31 C2 3B 24 FD 1A 33 74 F8 23 E0 EC 6B 19 8A 16 C6 E3 CD A4 CD B DB B3 A4 59 60 38 88 3B AD 1D B9 C6 8C A7 53 1B FC BC D9 A4 AB BC DD 3C 61 D7 93 15 98 EE 81 BD 8F E2 64 47 20 40 6 4E D7 AC 97 E8 B9 C0 59 12 A1 49 25 23 E4 ED 70 34 2C A5 B4 63 7C F9 A3 3D 83 D1 CD 6D 24 AC 7 2 3 1 0 1 A3 82 1 33 30 82 1 2F 30 E 6 3 55 1D F 1 1 FF 4 4 3 2 1 86 30 1D 6 3 55 1D 25 4 16 30 14 6 8 2B 6 1 5 5 7 3 1 6 8 2B 6 1 5 5 7 3 2 30 12 6 3 55 1D 13 1 1 FF 4 8 30 6 1 1 FF 2 1 0 30 1D 6 3 55 1D E 4 16 4 14 98 D1 F8 6E 10 EB CF 9B EC 60 9F 18 90 1B A0 EB 7D 9 FD 2B 30 1F 6 3 55 1D 23 4 18 30 16 80 14 9B E2 7 57 67 1C 1E C0 6A 6 DE 59 B4 9A 2D DF DC 19 86 2E 30 35 6 8 2B 6 1 5 5 7 1 1 4 29 30 27 30 25 6 8 2B 6 1 5 5 7 30 1 86 19 68 74 74 70 3A 2F 2F 6F 63 73 70 2E 70 6B 69 2E 67 6F 6F 67 2F 67 73 72 32 30 32 6 3 55 1D 1F 4 2B 30 29 30 27 A0 25 A0 23 86 21 68 74 74 70 3A 2F 2F 63 72 6C 2E 70 6B 69 2E 67 6F 6F 67 2F 67 73 72 32 2F 67 73 72 32 2E 63 72 6C 30 3F 6 3 55 1D 20 4 38 30 36 30 34 6 6 67 81 C 1 2 2 30 2A 30 28 6 8 2B 6 1 5 5 7 2 1 16 1C 68 74 74 70 73 3A 2F 2F 70 6B 69 2E 67 6F 6F 67 2F 72 65 70 6F 73 69 74 6F 72 79 2F 30 D 6 9 2A 86 48 86 F7 D 1 1 B 5 0 3 82 1 1 0 1A 80 3E 36 79 FB F3 2E A9 46 37 7D 5E 54 16 35 AE C7 4E 8 99 FE BD D1 34 69 26 52 66 7 3D A BA 49 CB 62 F4 F1 1A 8E FC 11 4F 68 96 4C 74 2B D3 67 DE B2 A3 AA 5 8D 84 4D 4C 20 65 F A5 96 DA D 16 F8 6C 3B DB 6F 4 23 88 6B 3A 6C C1 60 BD 68 9F 71 8E EE 2D 58 34 7 F0 D5 54 E9 86 59 FD 7B 5E D 21 94 F5 8C C9 A8 F8 D8 F2 AD CC F 1A F3 9A A7 A9 4 27 F9 A3 C9 B0 FF 2 78 6B 61 BA C7 35 2B E8 56 FA 4F C3 1C C ED B6 3C B4 4B EA ED CC E1 3C EC DC D 8C D6 3E 9B CA 42 58 8B CC 16 21 17 40 BC A2 D6 66 EF DA C4 15 5B CD 89 AA 9B 9 26 E7 32 D2 D 6E 67 20 2 5B 10 B0 90 9 9C C 1F 9E AD D8 3B EA A1 FC 6C E8 10 5C 8 52 19 51 2A 71 BB AC 7A B5 DD 15 ED 2B C9 8 2A 2C 8A B4 A6 21 AB 63 FF D7 52 49 50 D0 89 B7 AD F2 AF FB 50 AE 2F E1 95 D F3 46 AD 9D 9C F5 CA Server key exchange (Input) 16 3 3 1 2C C 0 1 28 3 0 1D 20 69 4A A 2A 99 52 9D E1 E3 32 BB 19 71 1F CC 9A 31 19 99 9B B 3D 7B 2A 87 E 53 F2 B6 E8 57 56 4 1 1 0 5 D4 B0 F5 6C 91 1C 62 94 8 B3 E F8 CE 21 32 69 A5 1B C5 4 62 64 EA C 7E 0 99 98 FD B1 60 78 78 9D A6 33 72 2E 36 65 71 F6 5 73 1F 34 4D 37 13 71 D A3 6A B7 39 F9 DD A6 F8 37 B 84 E7 62 FE 66 42 98 93 45 5 35 DD 8A B9 2C 29 A 34 D6 22 CA B8 8 FA 5A C7 49 27 F9 47 8F B2 CF FC 51 75 5D 5C 33 EC 28 E F7 EC 70 95 E8 C7 B3 10 2B E9 9 B E B2 B3 7E 3D 53 60 BD 38 99 8E 66 83 A6 9C EF 4F 39 1F 8D 2A 10 6B 6E DE 41 B2 0 37 E9 3 C1 1 FB 43 C6 42 95 AC 40 1 12 9 4 2C BC 29 50 DE 8B BC B5 D1 12 7A 71 B4 7F DB EC CB B0 35 7D 84 8B 60 19 72 5D 70 D FA 7A 4B 1 C1 BA BC BC D 5F 18 43 56 E4 B7 1 75 1E B0 92 C5 5E 9 36 F8 65 EC DE 23 22 79 5F E4 75 FD AB 85 DB DF 25 D9 32 EA D0 4D E5 64 9B 17 EE 8D EE D2 3A 36 3D 97 FB DA 33 FC FD B7 A3 A0 E1 A1 5A Server hello done (Input) 16 3 3 0 4 E 0 0 0 Client key exchange (Output) 16 3 3 0 25 10 0 0 21 20 4B EA B1 D1 7F 14 56 5D 76 1A 6B 56 B6 93 8B 76 3D EA E5 D2 95 2C C3 1 45 B5 5D F2 4F FF 91 75 Client change cipher spec (Output) 14 3 3 0 1 1 Client handshake finish (Output) 16 3 3 0 20 C4 46 D6 2C 6F A9 E9 E8 E4 BE 7B DA 5F 3E 7F 11 D6 83 31 EA 71 D7 4F CF 5E 62 36 5A F 6 AC 7F Server fatal alert (Input) 15 3 3 0 2 2 46
Boto's multipart upload API is not working in AWS eu-central-1 region
I am trying to use boto's multipart upload API in my application. But it is failing with The request signature we calculated does not match the signature you provided. Check your key and signing method Here are the error details import boto import boto3 s3_conn = boto.s3.connect_to_region('eu-central-1', aws_access_key_id=******, aws_secret_access_key=*****) buckt = s3_conn.get_bucket('phoenix-4-147757042135') mpuid = buckt.initiate_multipart_upload('4/5137/data_6000C298-a994-efe2-7e3b-efb3fd5a6c76/FULL/amar') mpuid.copy_part_from_key('phoenix-4-147757042135', '4/5137/data_6000C298-a994-efe2-7e3b-efb3fd5a6c76/FULL/10000000', 1, 0, 1) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.6/site-packages/boto-2.39.0-py2.6.egg/boto/s3/multipart.py", line 306, in copy_part_from_key query_args=query_args) File "/usr/lib/python2.6/site-packages/boto-2.39.0-py2.6.egg/boto/s3/bucket.py", line 889, in copy_key response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden <Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>ASIAIZZPTQJ3AWY3WG2Q</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256 20161110T051801Z 20161110/eu-central-1/s3/aws4_request bb44a9b3366ce1702d2850188d998eab092746bea906b50481ffe814137efde9</StringToSign><SignatureProvided>2d8b63e0c6f29d602e34ab426bf5704fcaaf6ae15c19550f047931a1da351211</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 36 31 31 31 30 54 30 35 31 38 30 31 5a 0a 32 30 31 36 31 31 31 30 2f 65 75 2d 63 65 6e 74 72 61 6c 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 62 62 34 34 61 39 62 33 33 36 36 63 65 31 37 30 32 64 32 38 35 30 31 38 38 64 39 39 38 65 61 62 30 39 32 37 34 36 62 65 61 39 30 36 62 35 30 34 38 31 66 66 65 38 31 34 31 33 37 65 66 64 65 39</StringToSignBytes><CanonicalRequest>PUT /4/5137/data_6000C298-a994-efe2-7e3b-efb3fd5a6c76/FULL/amar partNumber=1&uploadId=PMd350jeac8vLRuC7qf9CEBckNBqXGWdaEL32a.bCw3Zet.K1IInwYmDEEnkU98.xHSnlucJLeis49yMg2M.NoyOdwAVU8VMYcve6Sv5ExJ8Yh9pc7UJrxli5O2p6PqxH83.I_pcIGttaybTT0ul2nwc79DOUSIpKO8TlMosfNI- host:phoenix-4-147757042135.s3.eu-central-1.amazonaws.com user-agent:Boto/2.39.0 Python/2.6.6 Linux/2.6.32-431.20.3.el6.x86_64 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-copy-source:phoenix-4-147757042135/4/5137/data_6000C298-a994-efe2-7e3b-efb3fd5a6c76/FULL/10000000 x-amz-copy-source-range:bytes=0-1 x-amz-date:20161110T051801Z x-amz-security-token:FQoDYXdzEFYaDOR7x87//+eMX2hqwyK9A6+FKe1SX8gN2bpcbRJPuJxLZC+zP62TnzqoXA1lPvdKhjWpJI0v0UZ46Y+IcvbflIQXq438R8kC5QHJZHLx/fsDC7lPYLuDPA3d43V+cc28cn44pjD34DQpy0RKGMc7GJljZR+ijOb3m8e6bxv4c4kN83Kv+Gs7UBBa5EoGeg4NHN3neNN8g2u68Ad0uouPDV1Wvuiijbfp6ZIaT4LZFbq3mE11ZXiBjQDSDTDcLpO+ek3VCck6bpzquT/jDc8Jzrm6/rS8Giqs/xWLiATa4VqxVXzvbQ2Pu4Tv0kfCU7uc6DWMrBMhejj97s/HKLU0b12mcQ65T+FcieYALPfM4qPf27ZZVsjXWkCzkrMu8qOAblqpVCG15A2snZn0Ke6T7JF6ONGKkr1jSQ3Tv93C5/j6HuQOIcsuRKDLkBLgffOAyIzDzY2xBQvBIABTk0ENcLkgYzNLKsWLFuI3Pry24qcXVCSzXruyEE2neqZp8ohAvVh0WPZOtxmPOTbGgqRn2xEgb3yODAyz7+KOemRvfyGgxBGmvvSnuLrr24vY/6Inm49Q736eT7d5xmzflbLG6cdtBejyKmgttcOpuiEo9f2PwQU= host;user-agent;x-amz-content-sha256;x-amz-copy-source;x-amz-copy-source-range;x-amz-date;x-amz-security-token e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 55 54 0a 2f 34 2f 35 31 33 37 2f 64 61 74 61 5f 36 30 30 30 43 32 39 38 2d 61 39 39 34 2d 65 66 65 32 2d 37 65 33 62 2d 65 66 62 33 66 64 35 61 36 63 37 36 2f 46 55 4c 4c 2f 61 6d 61 72 0a 70 61 72 74 4e 75 6d 62 65 72 3d 31 26 75 70 6c 6f 61 64 49 64 3d 50 4d 64 33 35 30 6a 65 61 63 38 76 4c 52 75 43 37 71 66 39 43 45 42 63 6b 4e 42 71 58 47 57 64 61 45 4c 33 32 61 2e 62 43 77 33 5a 65 74 2e 4b 31 49 49 6e 77 59 6d 44 45 45 6e 6b 55 39 38 2e 78 48 53 6e 6c 75 63 4a 4c 65 69 73 34 39 79 4d 67 32 4d 2e 4e 6f 79 4f 64 77 41 56 55 38 56 4d 59 63 76 65 36 53 76 35 45 78 4a 38 59 68 39 70 63 37 55 4a 72 78 6c 69 35 4f 32 70 36 50 71 78 48 38 33 2e 49 5f 70 63 49 47 74 74 61 79 62 54 54 30 75 6c 32 6e 77 63 37 39 44 4f 55 53 49 70 4b 4f 38 54 6c 4d 6f 73 66 4e 49 2d 0a 68 6f 73 74 3a 70 68 6f 65 6e 69 78 2d 34 2d 31 34 37 37 35 37 30 34 32 31 33 35 2e 73 33 2e 65 75 2d 63 65 6e 74 72 61 6c 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 75 73 65 72 2d 61 67 65 6e 74 3a 42 6f 74 6f 2f 32 2e 33 39 2e 30 20 50 79 74 68 6f 6e 2f 32 2e 36 2e 36 20 4c 69 6e 75 78 2f 32 2e 36 2e 33 32 2d 34 33 31 2e 32 30 2e 33 2e 65 6c 36 2e 78 38 36 5f 36 34 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3a 70 68 6f 65 6e 69 78 2d 34 2d 31 34 37 37 35 37 30 34 32 31 33 35 2f 34 2f 35 31 33 37 2f 64 61 74 61 5f 36 30 30 30 43 32 39 38 2d 61 39 39 34 2d 65 66 65 32 2d 37 65 33 62 2d 65 66 62 33 66 64 35 61 36 63 37 36 2f 46 55 4c 4c 2f 31 30 30 30 30 30 30 30 0a 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 2d 72 61 6e 67 65 3a 62 79 74 65 73 3d 30 2d 31 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 36 31 31 31 30 54 30 35 31 38 30 31 5a 0a 78 2d 61 6d 7a 2d 73 65 63 75 72 69 74 79 2d 74 6f 6b 65 6e 3a 46 51 6f 44 59 58 64 7a 45 46 59 61 44 4f 52 37 78 38 37 2f 2f 2b 65 4d 58 32 68 71 77 79 4b 39 41 36 2b 46 4b 65 31 53 58 38 67 4e 32 62 70 63 62 52 4a 50 75 4a 78 4c 5a 43 2b 7a 50 36 32 54 6e 7a 71 6f 58 41 31 6c 50 76 64 4b 68 6a 57 70 4a 49 30 76 30 55 5a 34 36 59 2b 49 63 76 62 66 6c 49 51 58 71 34 33 38 52 38 6b 43 35 51 48 4a 5a 48 4c 78 2f 66 73 44 43 37 6c 50 59 4c 75 44 50 41 33 64 34 33 56 2b 63 63 32 38 63 6e 34 34 70 6a 44 33 34 44 51 70 79 30 52 4b 47 4d 63 37 47 4a 6c 6a 5a 52 2b 69 6a 4f 62 33 6d 38 65 36 62 78 76 34 63 34 6b 4e 38 33 4b 76 2b 47 73 37 55 42 42 61 35 45 6f 47 65 67 34 4e 48 4e 33 6e 65 4e 4e 38 67 32 75 36 38 41 64 30 75 6f 75 50 44 56 31 57 76 75 69 69 6a 62 66 70 36 5a 49 61 54 34 4c 5a 46 62 71 33 6d 45 31 31 5a 58 69 42 6a 51 44 53 44 54 44 63 4c 70 4f 2b 65 6b 33 56 43 63 6b 36 62 70 7a 71 75 54 2f 6a 44 63 38 4a 7a 72 6d 36 2f 72 53 38 47 69 71 73 2f 78 57 4c 69 41 54 61 34 56 71 78 56 58 7a 76 62 51 32 50 75 34 54 76 30 6b 66 43 55 37 75 63 36 44 57 4d 72 42 4d 68 65 6a 6a 39 37 73 2f 48 4b 4c 55 30 62 31 32 6d 63 51 36 35 54 2b 46 63 69 65 59 41 4c 50 66 4d 34 71 50 66 32 37 5a 5a 56 73 6a 58 57 6b 43 7a 6b 72 4d 75 38 71 4f 41 62 6c 71 70 56 43 47 31 35 41 32 73 6e 5a 6e 30 4b 65 36 54 37 4a 46 36 4f 4e 47 4b 6b 72 31 6a 53 51 33 54 76 39 33 43 35 2f 6a 36 48 75 51 4f 49 63 73 75 52 4b 44 4c 6b 42 4c 67 66 66 4f 41 79 49 7a 44 7a 59 32 78 42 51 76 42 49 41 42 54 6b 30 45 4e 63 4c 6b 67 59 7a 4e 4c 4b 73 57 4c 46 75 49 33 50 72 79 32 34 71 63 58 56 43 53 7a 58 72 75 79 45 45 32 6e 65 71 5a 70 38 6f 68 41 76 56 68 30 57 50 5a 4f 74 78 6d 50 4f 54 62 47 67 71 52 6e 32 78 45 67 62 33 79 4f 44 41 79 7a 37 2b 4b 4f 65 6d 52 76 66 79 47 67 78 42 47 6d 76 76 53 6e 75 4c 72 72 32 34 76 59 2f 36 49 6e 6d 34 39 51 37 33 36 65 54 37 64 35 78 6d 7a 66 6c 62 4c 47 36 63 64 74 42 65 6a 79 4b 6d 67 74 74 63 4f 70 75 69 45 6f 39 66 32 50 77 51 55 3d 0a 0a 68 6f 73 74 3b 75 73 65 72 2d 61 67 65 6e 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3b 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 2d 72 61 6e 67 65 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 73 65 63 75 72 69 74 79 2d 74 6f 6b 65 6e 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>3EDF2BFB155307CF</RequestId><HostId>q5lLYag9EJC/liFWejbNHWgFPZUDMLG3bBP5xM2+u87HlsO4alkoEu5r7xIcOih+YPcGFeCVJ20=</HostId></Error>
Thanks John Rotenstein & Jordon Phillips for your help. My issue got resolved by using boto3 for multipart. I was facing above issue with boto for multipart in eu-central-region
SQL SERVER order by random in series
I want to display records in random order, the simplest answer is order by NEWID(), that's fine. I have EmployeesID from 1 to 100. The output needs to be displayed in random series, i.e. 30.31.32.33...40..20.21.22..30..80.81.82..90.1.2.3..10..and so on.. If there are 100 records, it should be divided into 10 parts and every part displays randomly. CODE I tried is: select empId,empName from employees where empID between 1 and 100 order by NEWID() INPUT 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 OUTPUT 50 51 52 53 54 55 56 57 58 59 60 90 91 92 93 94 95 96 97 98 99 100 30 31 32 33 34 35 36 37 38 39 40 10 11 12 13 14 15 16 17 18 19 20 20 21 22 23 24 25 26 27 28 29 30 70 71 72 73 74 75 76 77 78 79 80 60 61 62 63 64 65 66 67 68 69 70 1 2 3 4 5 6 7 8 9 10 40 41 42 43 44 45 46 47 48 49 50 80 81 82 83 84 85 86 87 88 89 90
I will use NORTHWND.[dbo].[Employees], it got 9 employees. USE tempdb; GO SET NOCOUNT ON; CREATE TABLE #SAMPLE_DATA ( empID int, ni uniqueidentifier ); INSERT INTO #SAMPLE_DATA(empID, ni ) SELECT [EmployeeID], NEWID() FROM NORTHWND.[dbo].[Employees] ORDER BY NEWID() DECLARE #Values VARCHAR(1000) SELECT #Values = COALESCE(#Values + ' ', '') + CAST(empID AS varchar(50)) FROM #SAMPLE_DATA ORDER BY ni SELECT #Values; DROP TABLE #SAMPLE_DATA; Output: 2 6 4 1 3 9 7 8 5 Next run: 8 5 3 1 6 4 7 9 2 etc