Why doesnt client.connect accept a full adress - ssl

i am trying to write a piece of code in which the ESP32 connect securely to my api-server and post some sensor data. For this i use the WifiClientSecure library.
Through trial and error i have found out that client.connect(www.myserver.com, 443); works.
But client.connect(www.myserver.com/api, 443); will throw an error.
I have searched the internet for a solution but can't manage to find one.
Full code:
#include <arduino.h>
#include <WiFiClientSecure.h>
const char* ssid = "filled out ssid"; // your network SSID
const char* password = "filled out password"; // your network password
const char* server = "www.server.com/api"; // Server URL
/* use
openssl s_client -showcerts -connect www.howsmyssl.com:443 </dev/null
to get this certificate */
const char* ca_cert = \
"-----BEGIN CERTIFICATE-----\n" \
"MIIF3jCCA8agAwIBAgIQAf1tMPyjylGoG7xkDjUDLTANBgkqhkiG9w0BAQwFADCB\n" \
"iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl\n" \
"cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV\n" \
"BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTAw\n" \
"MjAxMDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV\n" \
"BAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0plcnNleSBDaXR5MR4wHAYDVQQKExVU\n" \
"aGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNVBAMTJVVTRVJUcnVzdCBSU0EgQ2Vy\n" \
"dGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK\n" \
"AoICAQCAEmUXNg7D2wiz0KxXDXbtzSfTTK1Qg2HiqiBNCS1kCdzOiZ/MPans9s/B\n" \
"3PHTsdZ7NygRK0faOca8Ohm0X6a9fZ2jY0K2dvKpOyuR+OJv0OwWIJAJPuLodMkY\n" \
"tJHUYmTbf6MG8YgYapAiPLz+E/CHFHv25B+O1ORRxhFnRghRy4YUVD+8M/5+bJz/\n" \
"Fp0YvVGONaanZshyZ9shZrHUm3gDwFA66Mzw3LyeTP6vBZY1H1dat//O+T23LLb2\n" \
"VN3I5xI6Ta5MirdcmrS3ID3KfyI0rn47aGYBROcBTkZTmzNg95S+UzeQc0PzMsNT\n" \
"79uq/nROacdrjGCT3sTHDN/hMq7MkztReJVni+49Vv4M0GkPGw/zJSZrM233bkf6\n" \
"c0Plfg6lZrEpfDKEY1WJxA3Bk1QwGROs0303p+tdOmw1XNtB1xLaqUkL39iAigmT\n" \
"Yo61Zs8liM2EuLE/pDkP2QKe6xJMlXzzawWpXhaDzLhn4ugTncxbgtNMs+1b/97l\n" \
"c6wjOy0AvzVVdAlJ2ElYGn+SNuZRkg7zJn0cTRe8yexDJtC/QV9AqURE9JnnV4ee\n" \
"UB9XVKg+/XRjL7FQZQnmWEIuQxpMtPAlR1n6BB6T1CZGSlCBst6+eLf8ZxXhyVeE\n" \
"Hg9j1uliutZfVS7qXMYoCAQlObgOK6nyTJccBz8NUvXt7y+CDwIDAQABo0IwQDAd\n" \
"BgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rIDZsswDgYDVR0PAQH/BAQDAgEGMA8G\n" \
"A1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEMBQADggIBAFzUfA3P9wF9QZllDHPF\n" \
"Up/L+M+ZBn8b2kMVn54CVVeWFPFSPCeHlCjtHzoBN6J2/FNQwISbxmtOuowhT6KO\n" \
"VWKR82kV2LyI48SqC/3vqOlLVSoGIG1VeCkZ7l8wXEskEVX/JJpuXior7gtNn3/3\n" \
"ATiUFJVDBwn7YKnuHKsSjKCaXqeYalltiz8I+8jRRa8YFWSQEg9zKC7F4iRO/Fjs\n" \
"8PRF/iKz6y+O0tlFYQXBl2+odnKPi4w2r78NBc5xjeambx9spnFixdjQg3IM8WcR\n" \
"iQycE0xyNN+81XHfqnHd4blsjDwSXWXavVcStkNr/+XeTWYRUc+ZruwXtuhxkYze\n" \
"Sf7dNXGiFSeUHM9h4ya7b6NnJSFd5t0dCy5oGzuCr+yDZ4XUmFF0sbmZgIn/f3gZ\n" \
"XHlKYC6SQK5MNyosycdiyA5d9zZbyuAlJQG03RoHnHcAP9Dc1ew91Pq7P8yF1m9/\n" \
"qS3fuQL39ZeatTXaw2ewh0qpKJ4jjv9cJ2vhsE/zB+4ALtRZh8tSQZXq9EfX7mRB\n" \
"VXyNWQKV3WKdwrnuWih0hKWbt5DHDAff9Yk2dDLWKMGwsAvgnEzDHNb842m1R0aB\n" \
"L6KCq9NjRHDEjf8tM7qtj3u1cIiuPhnPQCjY/MiQu12ZIvVS5ljFH4gxQ+6IHdfG\n" \
"jjxDah2nGN59PRbxYvnKkKj9\n" \
"-----END CERTIFICATE-----\n" ;
/* create an instance of WiFiClientSecure */
WiFiClientSecure client;
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
/* waiting for WiFi connect */
while (WiFi.status() != WL_CONNECTED) {
Serial.print(".");
delay(100);
}
Serial.print("Connected to ");
Serial.println(ssid);
/* set SSL/TLS certificate */
client.setCACert(ca_cert);
Serial.println("Connect to server via port 443");
if (!client.connect(server, 443)){
Serial.println("Connection failed!");
} else {
Serial.println("Connected to server!");
/* create HTTP request */
Serial.print("Waiting for response ");
while (!client.available()){
delay(50); //
Serial.print(".");
}
/* if data is available then receive and print to Terminal */
while (client.available()) {
char c = client.read();
Serial.write(c);
}
/* if the server disconnected, stop the client */
if (!client.connected()) {
Serial.println();
Serial.println("Server disconnected");
client.stop();
}
}
}
void loop() {
}

www.server.com/api is a partial URL. The complete URL will be https://www.server.com/api,
where:
https - This part is called scheme or protocol. HTTPS implies a default TCP port 443 (http mean port 80 by default)
www.server.com - The domain part, can be called also server address.
api- This is a path on the server
WiFiClient, or WiFiClientSecure, is the TCP client, it doesn't know anything about URLs. TCP client recognizes just address, given in form of IP address (192.168.1.20) or FQDN (which is then translated into IP address).
It is easier to use HTTPClient for requesting HTTP resources.
edit: FNDQ->FQDN

Related

ESP32 MQTT TLS errors (self signed cert)

I cannot seem to make MQTT+TLS+self signed certs work on my ESP32.
I am using this "setup" in my home network without major hickups with paho-python and mosquitto_pub/sub. The latter works with the following bash command (note the insecure there):
mosquitto_pub -h 192.168.1.X -p 8883 --cafile /etc/mosquitto/ca_certificates/mosq_ca.crt --cert /etc/mosquitto/certs/mosq_client.crt --key /etc/mosquitto/certs/mosq_client.key --debug --insecure --topic "test/message" -m "off"
My broker's mosquitto config file contains these lines apart from cert paths and general settings:
tls_version tlsv1.3
require_certificate true
When I send the sketch to my ESP32, the result is that it is stuck in the reconnect() loop (see sketch below).
When I do not specify a client cert and key ( // for client verification rows are commented in setup()), I get the following error (I guess it is attributable to the config which requires certificate):
1594936874: New connection from 192.168.1.162 on port 8883.
1594936874: OpenSSL Error: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
1594936874: Socket error on client <unknown>, disconnecting.
When I do specify a client certificate, I get this:
1594936887: New connection from 192.168.1.162 on port 8883.
1594936888: Socket error on client <unknown>, disconnecting.
Not sure if it is a host name mismatch on the certificate or what. This would be my main tip, but I am uncertain. I also do not know if a similar option to --insecure can be specified for WiFiClientSecure class, so far I have not found anything like that. But I am a beginner, too.
My sketch if pretty unpolished yet, but it should be still readable:
#include <WiFiClientSecure.h>
#include <time.h>
#include <PubSubClient.h>
const char* ssid = "mySSID";
const char* password = "myPasswd";
const char* server = "192.168.1.X"; // Server URL
const char* MQTT_subscribe_topic = "test/esp32";
int timediff_hr = 2;
const char* root_ca = \
"-----BEGIN CERTIFICATE-----\n" \
"pem format broken into lines\n" \
"pem format broken into lines\n" \
"pem format broken into lines\n" \
"-----END CERTIFICATE-----\n" ;
// You can use x.509 client certificates if you want
const char* test_client_key = \
"-----BEGIN CERTIFICATE-----\n" \
"pem format broken into lines\n" \
"pem format broken into lines\n" \
"pem format broken into lines\n" \
"-----END CERTIFICATE-----\n" ;
const char* test_client_cert = \
"-----BEGIN CERTIFICATE-----\n" \
"pem format broken into lines\n" \
"pem format broken into lines\n" \
"pem format broken into lines\n" \
"-----END CERTIFICATE-----\n" ;
WiFiClientSecure wifiClient;
time_t now;
void msgReceived(char* topic, byte* payload, unsigned int len);
PubSubClient pubSubClient(server, 8883, msgReceived, wifiClient);
void setup() {
//Initialize serial and wait for port to open:
Serial.begin(115200);
delay(100);
Serial.print("Attempting to connect to SSID: ");
Serial.println(ssid);
WiFi.begin(ssid, password);
// attempt to connect to Wifi network:
while (WiFi.status() != WL_CONNECTED) {
Serial.print(".");
// wait 1 second for re-trying
delay(1000);
}
Serial.print("Connected to ");
Serial.println(ssid);
getTime();
wifiClient.setCACert(root_ca);
wifiClient.setCertificate(test_client_key); // for client verification
wifiClient.setPrivateKey(test_client_cert); // for client verification
}
unsigned long lastPublish;
int msgCount;
void loop() {
if (!pubSubClient.connected()) {
reconnect();
}
pubSubClient.loop();
if (millis() - lastPublish > 10000) {
String msg = String("Hello from ESP32: ") + ++msgCount;
boolean rc = pubSubClient.publish("outTopic", msg.c_str());
Serial.print("Published, rc="); Serial.print( (rc ? "OK: " : "FAILED: ") );
Serial.println(msg);
lastPublish = millis();
}
}
void msgReceived(char* topic, byte* payload, unsigned int length) {
Serial.print("Message received on "); Serial.print(topic); Serial.print(": ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
}
void reconnect() {
Serial.print(F("PubSubClient attempting connection to: ")); Serial.println(server);
uint8_t i = 0;
while ( ! pubSubClient.connected()) {
// Attempt to connect
if (pubSubClient.connect("ESP32Client")) { //or "ESP8266Client"
Serial.println("connected");
// Subscribe
pubSubClient.subscribe(MQTT_subscribe_topic);
} else { //unsuccessful connect
Serial.print("failed, rc=");
Serial.print(pubSubClient.state());
Serial.print("\nNext try in 5 seconds: connection to "); Serial.println(server);
// Wait 5 seconds before retrying
delay(5000);
}
}
}
void getTime(){
// Synchronize time useing SNTP. This is necessary to verify that
// the TLS certificates offered by the server are currently valid.
Serial.print("Setting time using SNTP");
configTime(timediff_hr * 3600, 0, "de.pool.ntp.org");
time_t now = time(nullptr);
while (now < 1000) {
delay(500);
Serial.print(".");
now = time(nullptr);
}
Serial.println("");
struct tm timeinfo;
gmtime_r(&now, &timeinfo);
Serial.print("Current time: ");
Serial.print(asctime(&timeinfo));
}
P.S.: The base of this sketch is a test sketch of mine for MQTT:1883 unsecured, which also works fine. So the issue should definitely be in the TLS part.
Your test_client_key contains a certificate.
It would be helpful if you post the sample code with keys and certificates. Obviously not the ones you will be using later, but as you are self signing, generate some test key material that you can post here.
The configuration of your broker would also be useful.

GRPC Objective-C not calling handler on response

I'm running into an issue where I'm sending the message to the server (golang) but the response is not getting to the client (objective-c). To be clear, I see the log statement from the server with the correct message from the client. However, I don't see the response on the client. Furthermore, I've noticed that if my server is shutdown, the client does not have any issues getting to the same wait(), but that may be just my lack of understanding of the API. One important thing to note is that I'm not using cocopods or any other build tool for the client. As a learning exercise I'm attempting to build directly against the grpc objective-c source. I've attached my Makefile. Take note of the GRPC_CLIENT_SOURCE variable in the Makefile. I've had to do that as there are header files included there that do not exist in the grpc repo. That "feels" like where the problem is but I've learning that my feelings were useless through the years.
Many thanks in advance!
Here is my code:
Server:
package main
import (
"context"
"log"
"net"
"github.com/asharif/chatterbox/pb"
"google.golang.org/grpc"
)
// UserService is used for creating accounts and logins
type UserService struct{}
func main() {
initLogger()
startServer()
}
func initLogger() {
log.SetFlags(log.Ldate | log.Ltime | log.Lmicroseconds | log.Llongfile)
}
func startServer() {
log.Println("Starting server on port 9000")
lis, err := net.Listen("tcp", ":9000")
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterUserServiceServer(s, &UserService{})
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
// Create creates a new user
func (i *UserService) Create(ctx context.Context, req *pb.Request) (*pb.Response, error) {
log.Printf("Received: %v", req.Username)
return &pb.Response{Success: true, Msg: "Hello from server"}, nil
}
// Login attempts a login
func (i *UserService) Login(ctx context.Context, req *pb.Request) (*pb.Response, error) {
return &pb.Response{Success: true, Msg: "Hello "}, nil
}
Client:
#import <Foundation/Foundation.h>
#import "pb/User.pbobjc.h"
#import "pb/User.pbrpc.h"
int main (int argc, const char * argv[]) {
#autoreleasepool {
NSString* host = #"localhost:9000";
[GRPCCall useInsecureConnectionsForHost:host];
UserService* client = [[UserService alloc] initWithHost:host];
Request* req = [Request message];
req.username = #"yo from obj-c";
NSLog(#"callling server yo...");
NSCondition* cond = [[NSCondition alloc] init];
__block BOOL done = NO;
[client createWithRequest:req handler:^(Response *response, NSError *error) {
if (response) {
// Successful response received
NSLog(#"YES");
} else {
// RPC error
NSLog(#"GAH");
}
[cond lock];
done=YES;
[cond signal];
[cond unlock];
}];
[cond lock];
while(!done)
[cond wait];
[cond unlock];
}
return 0;
}
Makefile:
SHELL:=/bin/bash
VERSION:=local
BIN_DIR:=bin
# Protobuf stuff
PB_SOURCE_DIR:=/opt/protobuf/objectivec/
PB_SOURCE:=$(PB_SOURCE_DIR)/GPBProtocolBuffers.m
PB_DIR:=`pwd`/src/protobuf
PB_DEF_DIR:=`pwd`/src/protobuf/def
DEF:=$(shell find $(PB_DEF_DIR) -name '*.proto')
PB:=$(patsubst %.proto, %.pb, $(DEF))
# GRPC stuff
GRPC_SOURCE_DIR:=/opt/grpc/src/objective-c
GRPC_INC:=-I $(GRPC_SOURCE_DIR)
GRPC_CLIENT_SOURCE:=$(GRPC_SOURCE_DIR)/GRPCClient/GRPCCall.m $(GRPC_SOURCE_DIR)/GRPCClient/GRPCCallOptions.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCCallInternal.m $(GRPC_SOURCE_DIR)/GRPCClient/GRPCInterceptor.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCChannelPool.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCChannel.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCCompletionQueue.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCCronetChannelFactory.m $(GRPC_SOURCE_DIR)/GRPCClient/GRPCCall+ChannelCredentials.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCHost.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCInsecureChannelFactory.m $(GRPC_SOURCE_DIR)/GRPCClient/private/ChannelArgsUtil.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCWrappedCall.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCRequestHeaders.m $(GRPC_SOURCE_DIR)/GRPCClient/private/GRPCSecureChannelFactory.m $(GRPC_SOURCE_DIR)/GRPCClient/private/NSError+GRPC.m $(GRPC_SOURCE_DIR)/GRPCClient/GRPCCall+Interceptor.m $(GRPC_SOURCE_DIR)/GRPCClient/private/NSDictionary+GRPC.m $(GRPC_SOURCE_DIR)/GRPCClient/private/NSData+GRPC.m $(GRPC_SOURCE_DIR)/GRPCClient/GRPCCall+Tests.m
GRPC_PROTORPC_SOURCE:=$(shell find $(GRPC_SOURCE_DIR)/ProtoRPC -name '*.m')
GRPC_RXLIBRARY_SOURCE:=$(shell find $(GRPC_SOURCE_DIR)/RxLibrary -name '*.m')
GRPC_CLIENT_OBJ:=$(patsubst %.m, %.o, $(GRPC_CLIENT_SOURCE))
GRPC_PROTORPC_OBJ:=$(patsubst %.m, %.o, $(GRPC_PROTORPC_SOURCE))
GRPC_RXLIBRARY_OBJ:=$(patsubst %.m, %.o, $(GRPC_RXLIBRARY_SOURCE))
# Server stuff
SERVER_DIR:=`pwd`/src/server
SERVER_PB_DIR:=$(SERVER_DIR)/pb
SERVER_BIN:=$(shell basename $(CURDIR))-server
# Client stuff
CC:=clang
grpc: CFLAGS:=-fobjc-arc
PB_OBJ:=$(patsubst %.m, %.o, $(PB_SOURCE))
CLIENT_DIR:=`pwd`/src/client
CLIENT_PB_DIR:=$(CLIENT_DIR)/pb
CLIENT_INC:=-I $(PB_SOURCE_DIR) $(GRPC_INC)
CLIENT_LIBS:=-lgrpc
CLIENT_SOURCE:=$(shell find $(CLIENT_DIR) -name '*.m')
CLIENT_OBJ:=$(patsubst %.m, %.o, $(CLIENT_SOURCE))
CLIENT_FRAMEWORKS:=-framework Foundation
CLIENT_OBJ_DIR:=$(CLIENT_DIR)/obj
CLIENT_BIN:=$(shell basename $(CURDIR))-client
# Build the protobuf dist
protobuf: $(PB)
#mv $(PB_DIR)/*.go $(SERVER_PB_DIR)
#mv $(PB_DIR)/*.h $(CLIENT_PB_DIR)
#mv $(PB_DIR)/*.m $(CLIENT_PB_DIR)
%.pb : %.proto
#protoc -I $(PB_DEF_DIR) $< --go_out=plugins=grpc:$(PB_DIR)
#protoc -I $(PB_DEF_DIR) $< --objc_out=$(PB_DIR) --objcgrpc_out=$(PB_DIR)
# Build the server
server:
#mkdir -p $(BIN_DIR)
#cd src/server/; go build -o ../../$(BIN_DIR)/$(SERVER_BIN)-$(VERSION)
# Build the client
client: |dir pb grpc $(CLIENT_OBJ)
#mv *.o $(CLIENT_OBJ_DIR)
#$(CC) $(CFLAGS) $(CLIENT_FRAMEWORKS) $(CLIENT_LIBS) -o $(BIN_DIR)/$(CLIENT_BIN)-$(VERSION) $(CLIENT_OBJ_DIR)/**
pb: $(PB_OBJ)
grpc: $(GRPC_PROTORPC_OBJ) $(GRPC_CLIENT_OBJ) $(GRPC_RXLIBRARY_OBJ)
%.o: %.m
$(CC) $(CFLAGS) -c $(CLIENT_INC) $<
dir:
#mkdir -p $(BIN_DIR)
#mkdir -p $(CLIENT_OBJ_DIR)
clean:
#rm -f bin/*
#rm -f src/server/pb/**
#rm -f src/client/pb/**
#for i in `find . -iname "*.o"`; do rm $$i; done;
#for i in `find $(PB_SOURCE_DIR)/ -iname "*.o"`; do rm $$i; done;
GRPC puts your handler methods in dispatch_queue. You need to process your dispatch queue since your are using a command line application.
Just replace return 0 with dispatch_main() and you should be able to get into your handler.
Once you used dispatch_main() your program has no idea if all the handlers are processed or not so it blocks forever. You can do this to exit your program normally once all your handlers has been processed.
// create a dispatch group
dispatch_group_t serviceGroup = dispatch_group_create();
dispatch_group_enter(serviceGroup);
[client createWithRequest:req handler:^(Response *response, NSError *error) {
...
dispatch_group_leave(serviceGroup);
}];
...
// exit once the main queue is dispatched
dispatch_group_notify(serviceGroup, dispatch_get_main_queue(), ^{
exit(EXIT_SUCCESS);
});
This is the link to another example.

How to pass data in Poco HTTPRequest

This is regarding to my project, Where I am write Poco SSL client to communicate with a server.
I am able to do (i) Basic Auth (ii) Cert exchange. But after sending post request I am facing "Error 500".
Let me explain it=>
I have a working curl:
curl -d '{"name":"com.my.session.value"}' -H 'Content-Type: application/json' -H 'Accept: application/json' -E MyCert.pem --key MyKey.pem -u testuser -k -vvv https://<server-ip>:<port>/Internal/directory/path
using this I am able to print data on console. So tried to write same in Poco/C++:
// I have handler for keyPassparse, cert, invalid-cert-handler and used them in below line
SSLManager::instance().initializeClient(parse_str, Cert, ptrContext);
URI uri(argv[1]);
Poco::Net::HTTPSClientSession session(uri.getHost(), uri.getPort());
session.setKeepAlive(true);
Poco::Net::HTTPRequest req(Poco::Net::HTTPRequest::HTTP_POST, uri.getPath(), Poco::Net::HTTPMessage::HTTP_1_1);
HTTPBasicCredentials cred("testuser", "secret");
//Here I tried to add headers and data
cred.authenticate(req);
req.add("Content-Type","application/json");
req.add("Accept","application/json");
req.add("data","com.my.session.value"); // try-1 to add data
req.setKeepAlive(true);
std::ostream& myOStream = session.sendRequest(req);
std::string body("name=com.my.session.value"); // try-2 to add data
myOStream << body;
Poco::Net::HTTPResponse response;
std::istream& rs = session.receiveResponse(response);
std::cout << response.getStatus() << " " << response.getReason() << std::endl;
}
catch (Exception& exc)
{
std::cerr << exc.displayText() << std::endl;
return 1;
}
return 0;
}
This is always returning Error:500 (Internal server error)
Which means my data section is not reaching properly.
Please suggest me a way to pass proper "data" section to server.
Thanks in advance.
I found the solution for this:
1) I sent data as json:
Poco::JSON::Object obj;
obj.set("name", "com.my.session.value");
std::stringstream ss;
obj.stringify(ss);
2) Content herders should not be added by "add", used below for them:
req.setContentType("application/json");
req.setContentLength(ss.str().size());
3) Now sending body like this:
std::ostream& myOStream = session.sendRequest(req);
obj.stringify(myOStream);
Approach used:
I wrote code for http.
Sent same data by curl and exe, captured packets for both.
Compared and fixed gaps one by one.
I hope this will help someone in future.

Arduino MKR1000 not able to connect to AWS API Gateway

I tried modifying the basic Arduino code from here in order to send HTTP Requests to AWS API Gateway. While the example code from the link worked, I was not able to get a successful connection with AWS API Gateway.
I have tried a combination of things such as removing the https:// from server[], changing the port to 443 instead of 80, removing the /beta from server[], using client.connectSSL instead of client.connect, but none of these have worked so far.
The line:
int err = client.connect(server, 80);
returns me a value of 0.
There are no certificates set up with the AWS API Gateway, so I don't think it's a problem with that. Wifi works perfectly.
Any help would be greatly appreciated!
#include <SPI.h>
#include <WiFi101.h>
#include "arduino_secrets.h"
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
char ssid[] = SECRET_SSID; // your network SSID (name)
char pass[] = SECRET_PASS; // your network password (use for WPA, or
use as key for WEP)
int keyIndex = 0; // your network key Index number (needed only for WEP)
int status = WL_IDLE_STATUS;
// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
char server[] = "https://**********.execute-api.us-west-2.amazonaws.com/beta"; // name address for Google (using DNS)
WiFiClient client;
void setup() {
//Initialize serial and wait for port to open:
Serial.begin(9600);
while (!Serial) {
; // wait for serial port to connect. Needed for native USB port only
}
// check for the presence of the shield:
if (WiFi.status() == WL_NO_SHIELD) {
Serial.println("WiFi shield not present");
// don't continue:
while (true);
}
// attempt to connect to WiFi network:
while (status != WL_CONNECTED) {
Serial.print("Attempting to connect to SSID: ");
Serial.println(ssid);
// Connect to WPA/WPA2 network. Change this line if using open or WEP network:
status = WiFi.begin(ssid, pass);
// wait 10 seconds for connection:
delay(10000);
}
Serial.println("Connected to wifi");
printWiFiStatus();
Serial.println("\nStarting connection to server...");
// if you get a connection, report back via serial:
int err = client.connect(server, 80);
Serial.println(err);
if (err) {
Serial.println("connected to server");
// Make a HTTP request:
client.println("GET /beta HTTP/1.1");
client.println("Host: https://**********.execute-api.us-west-2.amazonaws.com");
client.println("Connection: close");
client.println();
}
}
void loop() {
// if there are incoming bytes available
// from the server, read them and print them:
while (client.available()) {
char c = client.read();
Serial.write(c);
}
// if the server's disconnected, stop the client:
if (!client.connected()) {
Serial.println();
Serial.println("disconnecting from server.");
client.stop();
// do nothing forevermore:
while (true);
}
}
void printWiFiStatus() {
// print the SSID of the network you're attached to:
Serial.print("SSID: ");
Serial.println(WiFi.SSID());
// print your WiFi shield's IP address:
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);
// print the received signal strength:
long rssi = WiFi.RSSI();
Serial.print("signal strength (RSSI):");
Serial.print(rssi);
Serial.println(" dBm");
}
Tips:
(1) client.connect(fqdn, port) is expecting an FQDN for the first parameter
The example in the docs is client.connect("Arduino.cc", 80) this seems to work well in my tests. The docs say "URL" but they mean FQDN.
(2) If you need SSL then you MUST load up your certs using the firmware updater
first. If you are using non-standard pins for the WiFi101 board then you MUST use wifi.setPins() to set the pins or the firmware updater will fail. Adafruit Feather M0 1500 owners will know what I am talking about here.
Reference: https://www.arduino.cc/en/Reference/WiFi101ClientConnect
I hope this helps.

SSL certificate verification on ESP8266 Wemos D1 Mini with MQTT broker

I have a raspberry pi 3 with raspbian stretch as its operating system. I have installed and fully configured a MQTT broker on the raspberry pi following this tutorial: https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-the-mosquitto-mqtt-messaging-broker-on-ubuntu-16-04
Everything works fine and well on the broker's side. The certificates get renewed after 60 days and you can only connect to port 1883 via the localhost and the other ports (8883 and 8083) are open but can only be accessed using TLS version 1.2 and for the latter also using websockets. Below you can find the code of my configuration of mosquitto (/etc/mosquitto/conf.d/default.conf).
allow_anonymous false
password_file /etc/mosquitto/passwd
listener 1883 localhost
listener 8883
certfile /etc/letsencrypt/live/home.kamidesigns.be/cert.pem
cafile /etc/letsencrypt/live/home.kamidesigns.be/chain.pem
keyfile /etc/letsencrypt/live/home.kamidesigns.be/privkey.pem
tls_version tlsv1.2
listener 8083
protocol websockets
certfile /etc/letsencrypt/live/home.kamidesigns.be/cert.pem
cafile /etc/letsencrypt/live/home.kamidesigns.be/chain.pem
keyfile /etc/letsencrypt/live/home.kamidesigns.be/privkey.pem
tls_version tlsv1.2
I also bought a ESP8266 Wemos D1 Mini to connect to this broker in a secure way. I used the pubsubclient library from this link: https: //github.com/knolleary/pubsubclient for my MQTT client.
I use the master branch of this link: https://github.com/esp8266/Arduino for my secure SSL connection. Below you see the code I used for programming my Wemos D1 Mini
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <time.h>
void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
}
const char* ssid = "ssid";
const char* password = "wifipassword";
const char* host = "home.kamidesigns.be";
const int port = 8883;
WiFiClientSecure espClient;
PubSubClient client(host, port, callback, espClient);
long lastMsg = 0;
char msg[50];
int value = 0;
void setup() {
Serial.begin(115200);
Serial.println();
Serial.print("connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
// Synchronize time useing SNTP. This is necessary to verify that
// the TLS certificates offered by the server are currently valid.
Serial.print("Setting time using SNTP");
configTime(8 * 3600, 0, "pool.ntp.org", "time.nist.gov");
time_t now = time(nullptr);
while (now < 1000) {
delay(500);
Serial.print(".");
now = time(nullptr);
}
Serial.println("");
struct tm timeinfo;
gmtime_r(&now, &timeinfo);
Serial.print("Current time: ");
Serial.print(asctime(&timeinfo));
}
void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
Serial.print("Attempting MQTT connection...");
// Attempt to connect
if (client.connect("ESP8266LightController","username","password")) {
Serial.println("connected");
// Once connected, publish an announcement...
client.publish("outTopic", "hello world");
// ... and resubscribe
client.subscribe("inTopic");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(5000);
}
}
}
When I start my Wemos D1, the serial monitor says:
connecting to ssid
..
WiFi connected
IP address:
192.168.0.213
Setting time using SNTP.
Current time: Sat Oct 14 02:26:25 2017
Attempting MQTT connection...connected
This is good and it is exactly what I wanted but I'm confused by how my Wemos D1 is able to connect to port 8883 without it verifying the certificate chain of the server? Remember that I never uploaded a certificate to the Wemos D1 or implemented a certificate into the code, and still it can connect.
One of 2 options
The WiFiClientSecure includes a list of public CA certs and is verifying your certificate against this list
The WiFiClientSecure defaults to not verifying remote certs by default.
Looking at this issue it looks like option 2 is most likely as it implies you have to verify the cert yourself after the connection.