This is the hexdump of the file I pipe into redis via a nodejs converter:
me#myself ~/scripts $ cat example.txt | node redisProtocol.js | hexdump -C
00000000 2a 39 0d 0a 24 34 0d 0a 53 41 44 44 0d 0a 24 37 |*9..$4..SADD..$7|
00000010 0d 0a 64 6f 6d 61 69 6e 73 0d 0a 24 31 34 0d 0a |..domains..$14..|
00000020 77 77 77 2e 72 65 64 64 69 74 2e 63 6f 6d 0d 0a |www.reddit.com..|
00000030 24 34 0d 0a 53 41 44 44 0d 0a 24 37 0d 0a 64 6f |$4..SADD..$7..do|
00000040 6d 61 69 6e 73 0d 0a 24 31 33 0d 0a 77 77 77 2e |mains..$13..www.|
00000050 34 63 68 61 6e 2e 6f 72 67 0d 0a 24 34 0d 0a 53 |4chan.org..$4..S|
00000060 41 44 44 0d 0a 24 37 0d 0a 64 6f 6d 61 69 6e 73 |ADD..$7..domains|
00000070 0d 0a 24 31 36 0d 0a 77 77 77 2e 66 61 63 65 62 |..$16..www.faceb|
00000080 6f 6f 6b 2e 63 6f 6d 0d 0a |ook.com..|
00000089
when piping to redis-cli --pipe I get:
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 1
Which is good.
Now looking into the redis DB executing smembers domains I get:
redis 127.0.0.1:6379> smembers domains
1) "domains"
2) "SADD"
3) "www.reddit.com"
4) "www.4chan.org"
5) "www.facebook.com"
Where do the additional entries "domains" and "SADD" come from? The hexdump looks good, doesn't it?
Using redis version redis-cli 2.6.7. Thanks a lot for any help provided.
Best,
Alex
No, it does not look good. I suppose you expect the hexdump to represent:
SADD domains www.reddit.com
SADD domains www.4chan.org
SADD domains www.facebook.com
However, it starts by '*9' which means Redis expects a command with nine parameters. So Redis processes:
SADD domains www.reddit.com SADD domains www.4chan.org SADD domains www.facebook.com
which gives the result you had.
You need either to execute 3 commands in your stream, each of them starting with *3, or just one command containing:
SADD domains www.reddit.com www.4chan.org www.facebook.com
starting with *5
Related
I want to create an Openshift route to a RabbitMQ.
As far as I understand the documentation at https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html, the "TLS with SNI" support should be able to inspect the SNI header and then route the traffic to the appropriate service.
Unfortunately, I'm having problems both with edge termination and with reencrypt termination. In both cases, the HAProxy seems to inspect the traffic and throws an error because the content is no HTTP.
With a simple Java application which uses the AMQP Java library, I can see the following traffic:
javax.net.ssl|DEBUG|01|main|2021-02-26 13:35:47.001 CET|SSLSocketOutputRecord.java:331|WRITE: TLS12 application_data, length = 8
javax.net.ssl|DEBUG|01|main|2021-02-26 13:35:47.001 CET|SSLCipher.java:1770|Plaintext before ENCRYPTION (
0000: 41 4D 51 50 00 00 09 01 AMQP....
)
javax.net.ssl|DEBUG|0D|AMQP Connection 192.168.0.1:443|2021-02-26 13:35:47.044 CET|SSLSocketInputRecord.java:249|READ: TLSv1.2 application_data, length = 211
javax.net.ssl|DEBUG|0D|AMQP Connection 192.168.0.1:443|2021-02-26 13:35:47.045 CET|SSLCipher.java:1672|Plaintext after DECRYPTION (
0000: 48 54 54 50 2F 31 2E 30 20 34 30 30 20 42 61 64 HTTP/1.0 400 Bad
0010: 20 72 65 71 75 65 73 74 0D 0A 43 61 63 68 65 2D request..Cache-
0020: 43 6F 6E 74 72 6F 6C 3A 20 6E 6F 2D 63 61 63 68 Control: no-cach
0030: 65 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 63 e..Connection: c
0040: 6C 6F 73 65 0D 0A 43 6F 6E 74 65 6E 74 2D 54 79 lose..Content-Ty
0050: 70 65 3A 20 74 65 78 74 2F 68 74 6D 6C 0D 0A 0D pe: text/html...
0060: 0A 3C 68 74 6D 6C 3E 3C 62 6F 64 79 3E 3C 68 31 .<html><body><h1
0070: 3E 34 30 30 20 42 61 64 20 72 65 71 75 65 73 74 >400 Bad request
0080: 3C 2F 68 31 3E 0A 59 6F 75 72 20 62 72 6F 77 73 </h1>.Your brows
0090: 65 72 20 73 65 6E 74 20 61 6E 20 69 6E 76 61 6C er sent an inval
00A0: 69 64 20 72 65 71 75 65 73 74 2E 0A 3C 2F 62 6F id request..</bo
00B0: 64 79 3E 3C 2F 68 74 6D 6C 3E 0A dy></html>.
)
(The output is generated with
java -Djavax.net.debug=all -jar rabbitmqtest-1.0-SNAPSHOT-all.jar amqps://myroute:443 2> output.txt)
The traffic does not get routed to the RabbitMQ. If I open the router in the web browser, RabbitMQ receives a connection request (but of course cannot understand it, because it is HTTP traffic).
The Helm route template is:
{{- if .Values.route.enabled }}
apiVersion: route.openshift.io/v1
kind: Route
metadata:
name: rabbitmq
spec:
host: rabbitmq-{{ .Values.route.identifier}}.{{ .Values.route.host }}
port:
targetPort: amqp-ssl
tls:
termination: reencrypt
destinationCACertificate: {{ .Values.tls.ca_crt | quote }}
to:
kind: Service
weight: 100
name: rabbitmq
status:
ingress: []
{{- end }}
Is there any way to use the router as a raw TCP proxy? I would prefer not to use a node port, as I would have to manage SSL certs on the RabbitMQ (Currently I have installed a long-lived self signed cert).
In theory, using an Ingress Controller should work in this scenario, as you are using TLS with SNI. It may however require additional work on the infrastructure side (adding additional ports to the loadbalancer, firewalling). Using the "standard" port 443 will not work (as you noted), as the Router expects HTTP traffic on that port.
Alternatives are to use externalIP (non-cloud environments) or exposing services using loadbalancers (cloud environment).
We are having troubles with sending a 'large' SOAP request to one of our sources over SSL. When we send the same request, but with less data in it, it works without any problems. The small file is 10kb, the larger file is 30kb. The SOAP requests are send from OSB (11.1.1.4) in Weblogic (10.3.4).
Our source has checked what happens in their proxy and they see that the proxy waits for a part of the message, but never receives it:
[14/Sep/2016:11:04:56 +0200] [someurl.something.com/sid#7fcdebef0ae8][rid#7fcdebe080a0][/cxf/someServiceService-01-01][4] Input filter: Reading request body.
[14/Sep/2016:11:04:56 +0200] [someurl.something.com/sid#7fcdebef0ae8][rid#7fcdebe080a0][/cxf/someServiceService-01-01][9] Input filter: Bucket type HEAP contains 1 bytes.
[14/Sep/2016:11:04:56 +0200] [someurl.something.com/sid#7fcdebef0ae8][rid#7fcdebe080a0][/cxf/someServiceService-01-01][9] Input filter: Bucket type HEAP contains 8000 bytes.
[14/Sep/2016:11:04:56 +0200] [someurl.something.com/sid#7fcdebef0ae8][rid#7fcdebe080a0][/cxf/someServiceService-01-01][9] Input filter: Bucket type HEAP contains 192 bytes.
[14/Sep/2016:11:04:56 +0200] [someurl.something.com/sid#7fcdebef0ae8][rid#7fcdebe080a0][/cxf/someServiceService-01-01][9] Input filter: Bucket type HEAP contains 534 bytes.
[14/Sep/2016:11:04:56 +0200] [someurl.something.com/sid#7fcdebef0ae8][rid#7fcdebe080a0][/cxf/someServiceService-01-01][9] Input filter: Bucket type HEAP contains 7376 bytes.
[14/Sep/2016:11:05:38 +0200] [someurl.something.com/sid#7fcdebef0ae8][rid#7fcdebe080a0][/cxf/someServiceService-01-01][4] Error reading request body: The timeout specified has expired
We have HTTP dump on our server and from the logging I can see that the SSL handshaking is ok, we start sending our message, but it stops before it's completly done.
Padded plaintext before ENCRYPTION: len = 328
0000: 50 4F 53 54 20 2F 63 78 00 00 00 00 00 00 00 00 POST /cxf/
0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 some
0020: 53 65 72 76 69 63 65 2D 30 31 2D 30 31 20 48 54 Service-01-01 HT
0030: 54 50 2F 31 2E 31 0D 0A 43 6F 6E 74 65 6E 74 2D TP/1.1..Content-
0040: 54 79 70 65 3A 20 74 65 78 74 2F 78 6D 6C 3B 20 Type: text/xml;
0050: 63 68 61 72 73 65 74 3D 75 74 66 2D 38 0D 0A 53 charset=utf-8..S
0060: 4F 41 50 41 63 74 69 6F 6E 3A 20 22 42 65 77 61 OAPAction: "
0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 someAction"..
0080: 55 73 65 72 2D 41 67 65 6E 74 3A 20 4A 61 76 61 User-Agent: Java
0090: 31 2E 36 2E 30 5F 33 31 0D 0A 48 6F 73 74 3A 20 1.6.0_31..Host:
00A0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 some.host
00B0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00C0: 6E 65 74 2E 62 65 0D 0A 41 63 63 65 70 74 3A 20 ..Accept:
00D0: 74 65 78 74 2F 68 74 6D 6C 2C 20 69 6D 61 67 65 text/html, image
00E0: 2F 67 69 66 2C 20 69 6D 61 67 65 2F 6A 70 65 67 /gif, image/jpeg
00F0: 2C 20 2A 2F 2A 3B 20 71 3D 2E 32 0D 0A 43 6F 6E , */*; q=.2..Con
0100: 6E 65 63 74 69 6F 6E 3A 20 4B 65 65 70 2D 41 6C nection: Keep-Al
0110: 69 76 65 0D 0A 43 6F 6E 74 65 6E 74 2D 4C 65 6E ive..Content-Len
0120: 67 74 68 3A 20 34 32 35 39 36 0D 0A 0D 0A CB F2 gth: 42596......
0130: DB 39 D1 16 D4 4C D3 05 BB 08 3C 2B A0 1E 39 BF .9...L....<+..9.
0140: A9 15 05 05 05 05 05 05 ........
Padded plaintext before ENCRYPTION: len = 16128
0000: 3F 78 6D 6C 20 76 65 72 73 69 6F 6E 3D 22 31 2E ?xml version="1.
0010: 30 22 20 65 6E 63 6F 64 69 6E 67 3D 22 55 54 46 0" encoding="UTF
0020: 2D 38 22 3F 3E 0A 3C 73 6F 61 70 65 6E 76 3A 45 -8"?>.<soapenv:E
0030: 6E 76 65 6C 6F 70 65 20 78 6D 6C 6E 73 3A 73 6F nvelope xmlns:so
0040: 61 70 65 6E 76 3D 22 68 74 74 70 3A 2F 2F 73 63 apenv="http://sc
... and so forth
This ends before I've seen the complete SOAP request pass through. And we receive:
Padded plaintext after DECRYPTION: len = 328
0000: 48 54 54 50 2F 31 2E 31 20 34 30 30 20 42 61 64 HTTP/1.1 400 Bad
0010: 20 52 65 71 75 65 73 74 0D 0A 44 61 74 65 3A 20 Request..Date:
0020: 4D 6F 6E 2C 20 31 30 20 4F 63 74 20 32 30 31 36 Mon, 10 Oct 2016
0030: 20 30 37 3A 35 32 3A 30 37 20 47 4D 54 0D 0A 53 07:52:07 GMT..S
0040: 65 72 76 65 72 3A 20 41 70 61 63 68 65 0D 0A 53 erver: Apache..S
0050: 74 72 69 63 74 2D 54 72 61 6E 73 70 6F 72 74 2D trict-Transport-
0060: 53 65 63 75 72 69 74 79 3A 20 6D 61 78 2D 61 67 Security: max-ag
0070: 65 3D 33 31 35 33 36 30 30 30 3B 20 69 6E 63 6C e=31536000; incl
0080: 75 64 65 53 75 62 44 6F 6D 61 69 6E 73 0D 0A 4C udeSubDomains..L
0090: 61 73 74 2D 4D 6F 64 69 66 69 65 64 3A 20 54 75 ast-Modified: Tu
00A0: 65 2C 20 30 33 20 4D 61 72 20 32 30 31 35 20 31 e, 03 Mar 2015 1
00B0: 32 3A 32 35 3A 32 31 20 47 4D 54 0D 0A 45 54 61 2:25:21 GMT..ETa
00C0: 67 3A 20 22 62 65 38 2D 35 31 30 36 31 36 64 61 g: "be8-510616da
00D0: 38 63 62 31 64 22 0D 0A 41 63 63 65 70 74 2D 52 8cb1d"..Accept-R
00E0: 61 6E 67 65 73 3A 20 62 79 74 65 73 0D 0A 43 6F anges: bytes..Co
00F0: 6E 74 65 6E 74 2D 4C 65 6E 67 74 68 3A 20 33 30 ntent-Length: 30
0100: 34 38 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 48..Connection:
0110: 63 6C 6F 73 65 0D 0A 43 6F 6E 74 65 6E 74 2D 54 close..Content-T
0120: 79 70 65 3A 20 74 65 78 74 2F 68 74 6D 6C 0D 0A ype: text/html..
0130: 0D 0A 7E 01 14 86 D8 1F DA 05 97 49 26 2B 2F 65 ...........I&+/e
0140: DB 5E ED 05 F2 AA 01 01 .^......
Our server team has already tried increasing the timeouts in weblogic under Servers - Protocols - HTTP by a factor of 10, but with no success.
Increasing any possible timeout settings in OSB didn't help.
We are pretty sure it is a Weblogic (maybe OSB) issue as sending the larger request from the same server with curl doesn't give any problems.
Our Development environment does NOT have this issue. The problem is, we currently have an upgraded Weblogic (10.3.6)/OSB (11.1.1.7) installed there for an upcoming upgrade of the software on other environments. Same configuration though.
Any insight on what could be going wrong or what we could try would be helpfull. Let me know if you need any additional information.
An Avro file contains the schema in plain text followed by the data in binary format. I'd like to know whether the schema (or some part of it) also exists in the binary part? I got a hunch that the schema (or just the field names) also get coded in the binary part because when I make some changes in the plain schema part of an AVRO file I get an error message when exporting the schema using the Avro-tool.jar .
When the binary encoding is used, the whole file is using a binary format.
The file starts with a 4 bytes header, then a map containing some metadata immediately follows. This map contains a "avro.schema" entry. The value of this entry is the schema stored as a string. After the map you will find your data.
If you edit the schema manually, read change its size, then length prefix stored just before this string will be incoherent and the file is corrupted.
See Binary encoding specification to learn how various types are binary encoded.
I'm not sure what you are trying to achieve, and I quite sure that is should not be done. But for fun, lets try to edit the schema in place.
For this example I will use the weather.avro file from the avro's source tree:
$ java -jar avro-tools-1.8.0.jar getmeta weather-orig.avro
avro.codec null
avro.schema {"type":"record","name":"Weather","namespace":"test","fields":[{"name":"station","type":"string"},{"name":"time","type":"long"},{"name":"temp","type":"int"}],"doc":"A weather reading."}
$ java -jar avro-tools-1.8.0.jar getschema weather-orig.avro
{
"type" : "record", "name" : "Weather", "namespace" : "test", "doc" : "A weather reading.",
"fields" : [
{"name" : "station", "type" : "string"},
{"name" : "time", "type" : "long"},
{"name" : "temp", "type" : "int"}
]
}
$ java -jar /avro-tools-1.8.0.jar tojson weather-orig.avro
{"station":"011990-99999","time":-619524000000,"temp":0}
{"station":"011990-99999","time":-619506000000,"temp":22}
{"station":"011990-99999","time":-619484400000,"temp":-11}
{"station":"012650-99999","time":-655531200000,"temp":111}
{"station":"012650-99999","time":-655509600000,"temp":78}
OK. This is our source file. Plain simple, two metadata entries and the schema defines three fields. Now, we will try to understand how things are stored in binary and how we can edit the file to change the rename station int station-id.
$ hexdump weather-orig.avro -n 256 -C
00000000 4f 62 6a 01 04 14 61 76 72 6f 2e 63 6f 64 65 63 |Obj...avro.codec|
00000010 08 6e 75 6c 6c 16 61 76 72 6f 2e 73 63 68 65 6d |.null.avro.schem|
00000020 61 f2 02 7b 22 74 79 70 65 22 3a 22 72 65 63 6f |a..{"type":"reco|
00000030 72 64 22 2c 22 6e 61 6d 65 22 3a 22 57 65 61 74 |rd","name":"Weat|
00000040 68 65 72 22 2c 22 6e 61 6d 65 73 70 61 63 65 22 |her","namespace"|
00000050 3a 22 74 65 73 74 22 2c 22 66 69 65 6c 64 73 22 |:"test","fields"|
00000060 3a 5b 7b 22 6e 61 6d 65 22 3a 22 73 74 61 74 69 |:[{"name":"stati|
00000070 6f 6e 22 2c 22 74 79 70 65 22 3a 22 73 74 72 69 |on","type":"stri|
00000080 6e 67 22 7d 2c 7b 22 6e 61 6d 65 22 3a 22 74 69 |ng"},{"name":"ti|
00000090 6d 65 22 2c 22 74 79 70 65 22 3a 22 6c 6f 6e 67 |me","type":"long|
000000a0 22 7d 2c 7b 22 6e 61 6d 65 22 3a 22 74 65 6d 70 |"},{"name":"temp|
000000b0 22 2c 22 74 79 70 65 22 3a 22 69 6e 74 22 7d 5d |","type":"int"}]|
000000c0 2c 22 64 6f 63 22 3a 22 41 20 77 65 61 74 68 65 |,"doc":"A weathe|
000000d0 72 20 72 65 61 64 69 6e 67 2e 22 7d 00 b0 81 b3 |r reading."}....|
000000e0 c4 0a 0c f6 62 fa c9 38 fd 7e 52 00 a7 0a cc 01 |....b..8.~R.....|
000000f0 18 30 31 31 39 39 30 2d 39 39 39 39 39 ff a3 90 |.011990-99999...|
First four bytes, 4f 62 6a 01, are the header
The next thing is a long describing the size of the first block of the "metadata" map. long are encoded using variable-length zig-zag coding, so here 04 means 2 which coherent with the output of getmeta. (remember to read the Avro specification to know how various data type are encoded)
Just after you will find the first key of the map. A key is a string and a string is prefixed by its length in bytes. Here 0x14 means 10 bytes which is the length of "avro.codec" when encoded in UTF-8.
You can then skip the next 10 bytes and go to the next element. Etc. You can advance until you spot the avro.schema part.
Just after this string is the length of the map value (which is a string since it is our schema). That is what you want to modify. We are renaming station into station-id so you want to add 3 to the current length, so f2 02 should now be f8 02 (remember variable length zig zag coding ?).
You can now update the schema string to add "-id"
Enjoy
java -jar /home/cmathieu/Sources/avro-trunk/lang/java/tools/target/avro-tools-1.8.0-SNAPSHOT.jar tojson weather.avro
{"station-id":"011990-99999","time":-619524000000,"temp":0}
{"station-id":"011990-99999","time":-619506000000,"temp":22}
{"station-id":"011990-99999","time":-619484400000,"temp":-11}
{"station-id":"012650-99999","time":-655531200000,"temp":111}
{"station-id":"012650-99999","time":-655509600000,"temp":78}
But as I said, you most likely don't want to do that.
I'm implementing RFC 5246(TLS 1.2), and I'm stuck at the certificate message.I'm debugging the server with a combination of Openssl s_client and Browsers.The server hello message is received and interpreted fine, and with -msg option in openssl I can see the message has been interpreted properly as Client_hello, without any errors.
When the certificate message is sent, the browser doesn't respond, and openssl s_client with -msg doesn't respond either. Openssl s_client with -debug reads the message but doesn't respond after the server_hello_done message. No errors are logged.
I suspect the problem is with the structure of my certificate message, because anything I send after the Record layer length gets received without any errors, even if it's just random text or binary data.
Modifying the structure of the record layer to incorrect/inappropriate values throws an error with the appropriate error message, for example setting the version to 9.3 throws the error
5256:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
as expected.
This is the structure I am currently using:
/Record layer/
One byte for the message type, two bytes for the protocol version(major and minor), and two bytes for the message length.
/Handshake message data/
Two bytes for the length, two bytes for the certificate(s) length.Finally the certificate(s).
I'm currently working with a self-signed certificate, and neither the browser nor openssl s_client include a signature_algorithms extension in the Client_hello, so I send the certificate as it is, in .PEM format, and in plain text.Below is openssl's hex dump of the handshake thus far:
CONNECTED(00000164)
write to 0x14ad698 [0x13f1ffb] (210 bytes => 210 (0xD2))
0000 - 16 03 01 00 cd 01 00 00-c9 03 01 53 5e 5c d6 a0 ...........S^\..
0010 - 34 27 ea 22 ed 01 dc 36-bb 0b 84 1e 5a 58 3e d5 4'."...6....ZX>.
0020 - 95 4d 5f 81 9f 2a f0 27-75 fb 1f 00 00 5c c0 14 .M_..*.'u....\..
0030 - c0 0a 00 39 00 38 00 88-00 87 c0 0f c0 05 00 35 ...9.8.........5
0040 - 00 84 c0 12 c0 08 00 16-00 13 c0 0d c0 03 00 0a ................
0050 - c0 13 c0 09 00 33 00 32-00 9a 00 99 00 45 00 44 .....3.2.....E.D
0060 - c0 0e c0 04 00 2f 00 96-00 41 00 07 c0 11 c0 07 ...../...A......
0070 - c0 0c c0 02 00 05 00 04-00 15 00 12 00 09 00 14 ................
0080 - 00 11 00 08 00 06 00 03-00 ff 01 00 00 44 00 0b .............D..
0090 - 00 04 03 00 01 02 00 0a-00 34 00 32 00 01 00 02 .........4.2....
00a0 - 00 03 00 04 00 05 00 06-00 07 00 08 00 09 00 0a ................
00b0 - 00 0b 00 0c 00 0d 00 0e-00 0f 00 10 00 11 00 12 ................
00c0 - 00 13 00 14 00 15 00 16-00 17 00 18 00 19 00 23 ...............#
00d2 - <SPACES/NULS>
read from 0x14ad698 [0x13edaab] (5 bytes => 5 (0x5))
0000 - 16 03 01 00 2c ....,
read from 0x14ad698 [0x13edab0] (44 bytes => 44 (0x2C))
0000 - 02 00 00 26 03 01 53 5e-5c d6 4a 5a 5b 4d 63 38 ...&..S^\.JZ[Mc8
0010 - 57 47 53 45 4d 5b 3f 3c-5f 38 23 67 26 32 38 4c WGSEM[?<_8#g&28L
0020 - 2e 67 47 67 28 56 00 00-2f .gGg(V../
002c - <SPACES/NULS>
read from 0x14ad698 [0x13edaab] (5 bytes => 5 (0x5))
0000 - 16 03 01 02 99 .....
read from 0x14ad698 [0x13edab0] (665 bytes => 665 (0x299))
0000 - 0b 02 96 02 94 2d 2d 2d-2d 2d 42 45 47 49 4e 20 .....-----BEGIN
0010 - 43 45 52 54 49 46 49 43-41 54 45 2d 2d 2d 2d 2d CERTIFICATE-----
0020 - 0a 4d 49 49 42 75 54 43-43 41 53 49 43 43 51 43 .MIIBuTCCASICCQC
0030 - 43 65 67 31 46 6f 4f 76-43 4b 6a 41 4e 42 67 6b Ceg1FoOvCKjANBgk
0040 - 71 68 6b 69 47 39 77 30-42 41 51 55 46 41 44 41 qhkiG9w0BAQUFADA
0050 - 68 4d 51 73 77 43 51 59-44 56 51 51 47 45 77 4a hMQswCQYDVQQGEwJ
0060 - 4c 0a 52 54 45 53 4d 42-41 47 41 31 55 45 41 78 L.RTESMBAGA1UEAx
0070 - 4d 4a 62 47 39 6a 59 57-78 6f 62 33 4e 30 4d 42 MJbG9jYWxob3N0MB
0080 - 34 58 44 54 45 30 4d 44-51 79 4d 54 45 32 4e 44 4XDTE0MDQyMTE2ND
0090 - 4d 30 4e 56 6f 58 44 54-45 31 4d 44 51 79 4d 54 M0NVoXDTE1MDQyMT
00a0 - 45 32 0a 4e 44 4d 30 4e-56 6f 77 49 54 45 4c 4d E2.NDM0NVowITELM
00b0 - 41 6b 47 41 31 55 45 42-68 4d 43 53 30 55 78 45 AkGA1UEBhMCS0UxE
00c0 - 6a 41 51 42 67 4e 56 42-41 4d 54 43 57 78 76 59 jAQBgNVBAMTCWxvY
00d0 - 32 46 73 61 47 39 7a 64-44 43 42 6e 7a 41 4e 42 2FsaG9zdDCBnzANB
00e0 - 67 6b 71 0a 68 6b 69 47-39 77 30 42 41 51 45 46 gkq.hkiG9w0BAQEF
00f0 - 41 41 4f 42 6a 51 41 77-67 59 6b 43 67 59 45 41 AAOBjQAwgYkCgYEA
0100 - 72 51 71 76 50 36 4c 35-41 71 31 31 67 76 38 2b rQqvP6L5Aq11gv8+
0110 - 2f 59 55 53 62 50 46 4b-34 66 51 71 30 74 42 79 /YUSbPFK4fQq0tBy
0120 - 36 53 39 6c 0a 78 6f 45-6d 50 47 79 52 49 7a 44 6S9l.xoEmPGyRIzD
0130 - 31 46 78 78 52 65 50 79-55 6a 69 78 63 39 66 41 1FxxRePyUjixc9fA
0140 - 59 6f 74 5a 31 53 71 71-2f 2b 37 77 69 52 2b 7a YotZ1Sqq/+7wiR+z
0150 - 33 46 6f 65 51 58 73 53-64 32 78 32 44 4b 63 62 3FoeQXsSd2x2DKcb
0160 - 73 62 64 62 76 0a 2f 73-49 2b 68 63 57 39 4c 5a sbdbv./sI+hcW9LZ
0170 - 48 4d 45 75 49 6a 4d 42-73 6f 6f 4d 52 6a 76 35 HMEuIjMBsooMRjv5
0180 - 79 78 79 50 67 38 33 35-34 66 46 6d 51 50 38 4a yxyPg8354fFmQP8J
0190 - 49 73 54 54 48 31 54 56-5a 4f 47 48 79 49 62 76 IsTTH1TVZOGHyIbv
01a0 - 6e 34 7a 36 35 6b 0a 48-63 62 75 4f 52 38 43 41 n4z65k.HcbuOR8CA
01b0 - 77 45 41 41 54 41 4e 42-67 6b 71 68 6b 69 47 39 wEAATANBgkqhkiG9
01c0 - 77 30 42 41 51 55 46 41-41 4f 42 67 51 41 72 73 w0BAQUFAAOBgQArs
01d0 - 36 62 74 6f 38 6b 76 67-6b 48 70 4d 56 50 42 42 6bto8kvgkHpMVPBB
01e0 - 50 62 4e 71 6d 2f 62 0a-4a 78 4a 34 4a 78 5a 72 PbNqm/b.JxJ4JxZr
01f0 - 51 4c 6b 6a 48 7a 39 34-44 75 57 7a 64 67 41 6a QLkjHz94DuWzdgAj
0200 - 56 2b 70 39 72 7a 65 76-37 56 65 57 44 49 5a 41 V+p9rzev7VeWDIZA
0210 - 78 39 6a 43 6b 65 39 59-51 4c 68 42 67 62 2b 7a x9jCke9YQLhBgb+z
0220 - 48 73 6d 30 39 70 50 43-0a 4c 4f 50 37 4b 67 4b Hsm09pPC.LOP7KgK
0230 - 42 78 6e 68 52 49 33 4f-43 48 41 39 6f 43 78 30 BxnhRI3OCHA9oCx0
0240 - 78 46 35 46 65 34 53 38-34 65 6b 30 5a 37 65 5a xF5Fe4S84ek0Z7eZ
0250 - 4c 55 63 7a 52 41 6f 55-57 50 42 65 70 32 6e 62 LUczRAoUWPBep2nb
0260 - 44 77 39 78 6c 6e 30 57-49 0a 53 33 72 6d 2f 47 Dw9xln0WI.S3rm/G
0270 - 4d 6d 6e 4e 73 78 75 74-68 69 63 41 3d 3d 0a 2d MmnNsxuthicA==.-
0280 - 2d 2d 2d 2d 45 4e 44 20-43 45 52 54 49 46 49 43 ----END CERTIFIC
0290 - 41 54 45 2d 2d 2d 2d 2d-0a ATE-----.
read from 0x14ad698 [0x13edaab] (5 bytes => 5 (0x5))
0000 - 0e 03 01 ...
0005 - <SPACES/NULS>
read from 0x14ad698 [0x13edaab] (5 bytes => -1 (0xFFFFFFFF))
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 724 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
PSK identity: None
PSK identity hint: None
Start Time: 1398693078
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
PEM is certainly not the right format to send on the wire. You should send it in binary form (use a base-64 decoded version of this PEM content).
You'll also need to wrap this in other layers, since a certificate_list in the Certificate struct is expected.
If you want to learn a bit more about all this, it's probably worth looking at existing traffic produced by working implementations using Wireshark. Its SSL wiki page even has an existing capture file that you can use.
(You're also mentioning you want to implement TLS 1.2, but you're sending 03 01, which is for TLS 1.0, although this shouldn't matter very much at this stage of your implementation.)
I'm making an HTTP POST request using curl like this:
curl -v -i -d 'var=something' 'http://mysite.com/whatever.json'
How can I see the body of the outgoing request?
Motivation: I'm debugging an Objective-C HTTP library. POST requests that are failing in this library are working from the Terminal using curl. I think comparing the bodies of the requests will help debug.
You can use --trace or --trace-ascii to get an traffic dump:
curl --trace - -d 'var=something' 'http://mysite.com/whatever.json'
[...]
=> Send header, 246 bytes (0xf6)
0000: 50 4f 53 54 20 2f 77 68 61 74 65 76 65 72 2e 6a POST /whatever.j
0010: 73 6f 6e 20 48 54 54 50 2f 31 2e 31 0d 0a 55 73 son HTTP/1.1..Us
0020: 65 72 2d 41 67 65 6e 74 3a 20 63 75 72 6c 2f 37 er-Agent: curl/7
0030: 2e 32 31 2e 36 20 28 78 38 36 5f 36 34 2d 70 63 .21.6 (x86_64-pc
0040: 2d 6c 69 6e 75 78 2d 67 6e 75 29 20 6c 69 62 63 -linux-gnu) libc
0050: 75 72 6c 2f 37 2e 32 31 2e 36 20 4f 70 65 6e 53 url/7.21.6 OpenS
0060: 53 4c 2f 31 2e 30 2e 30 65 20 7a 6c 69 62 2f 31 SL/1.0.0e zlib/1
0070: 2e 32 2e 33 2e 34 20 6c 69 62 69 64 6e 2f 31 2e .2.3.4 libidn/1.
0080: 32 32 20 6c 69 62 72 74 6d 70 2f 32 2e 33 0d 0a 22 librtmp/2.3..
0090: 48 6f 73 74 3a 20 6d 79 73 69 74 65 2e 63 6f 6d Host: mysite.com
00a0: 0d 0a 41 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a 43 ..Accept: */*..C
00b0: 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a 20 31 ontent-Length: 1
00c0: 33 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 3..Content-Type:
00d0: 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 2d 77 application/x-w
00e0: 77 77 2d 66 6f 72 6d 2d 75 72 6c 65 6e 63 6f 64 ww-form-urlencod
00f0: 65 64 0d 0a 0d 0a ed....
=> Send data, 13 bytes (0xd)
0000: 76 61 72 3d 73 6f 6d 65 74 68 69 6e 67 var=something
[...]
--trace-ascii has less detail:
curl --trace-ascii - -d 'var=something' 'http://mysite.com/whatever.json'
[...]
=> Send header, 246 bytes (0xf6)
0000: POST /whatever.json HTTP/1.1
001e: User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 Ope
005e: nSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
0090: Host: mysite.com
00a2: Accept: */*
00af: Content-Length: 13
00c3: Content-Type: application/x-www-form-urlencoded
00f4:
=> Send data, 13 bytes (0xd)
0000: var=something
[...]
You won't get friendly parsing, but you could always use tcpdump or Wireshark to see exactly what's being sent back and forth.
If you have it available, setting up netcat to listen on a port and then pointing both curl and your failing requests at that host/port would also work.