I am making a simple arduino app that send a GET request to a HTTPS site. The code I'm using is exactly the same as the code in the MKRGSM library examples (GsmSslWebClient). But for whatever reason I always get the same response when connecting to a HTTPS site: "301 moved permanently". I kind of know what that means, I am aware that you are supposed to just make another request to the location specified in the header. But I don't know what I have to change to be able to address a https site. I'm sorry for my ignorance and I do know that in the example it clearly states that it connects to http://www.arduino.cc/asciilogo.txt but why is it then any different than the normal http example?
I would also point out here, that I have tried changing port to 80 and client settings, to work for unprotected http which works just fine. So its just the https that doesn't work.
this is the code:
/*
Web client
This sketch connects to a website using SSL through a MKR GSM 1400 board. Specifically,
this example downloads the URL "http://www.arduino.cc/asciilogo.txt" and
prints it to the Serial monitor.
Circuit:
* MKR GSM 1400 board
* Antenna
* SIM card with a data plan
created 8 Mar 2012
by Tom Igoe
*/
// libraries
#include <MKRGSM.h>
#include "arduino_secrets.h"
// Please enter your sensitive data in the Secret tab or arduino_secrets.h
// PIN Number
const char PINNUMBER[] = SECRET_PINNUMBER;
// APN data
const char GPRS_APN[] = SECRET_GPRS_APN;
const char GPRS_LOGIN[] = SECRET_GPRS_LOGIN;
const char GPRS_PASSWORD[] = SECRET_GPRS_PASSWORD;
// initialize the library instance
GSMSSLClient client;
GPRS gprs;
GSM gsmAccess;
// URL, path and port (for example: arduino.cc)
char server[] = "arduino.cc";
char path[] = "/asciilogo.txt";
int port = 443; // port 443 is the default for HTTPS
void setup() {
// initialize serial communications and wait for port to open:
Serial.begin(9600);
while (!Serial) {
; // wait for serial port to connect. Needed for native USB port only
}
Serial.println("Starting Arduino web client.");
// connection state
bool connected = false;
// After starting the modem with GSM.begin()
// attach the shield to the GPRS network with the APN, login and password
while (!connected) {
if ((gsmAccess.begin(PINNUMBER) == GSM_READY) &&
(gprs.attachGPRS(GPRS_APN, GPRS_LOGIN, GPRS_PASSWORD) == GPRS_READY)) {
connected = true;
} else {
Serial.println("Not connected");
delay(1000);
}
}
Serial.println("connecting...");
// if you get a connection, report back via serial:
if (client.connect(server, port)) {
Serial.println("connected");
// Make a HTTP request:
client.print("GET ");
client.print(path);
client.println(" HTTP/1.1");
client.print("Host: ");
client.println(server);
client.println("Connection: close");
client.println();
} else {
// if you didn't get a connection to the server:
Serial.println("connection failed");
}
}
void loop() {
// if there are incoming bytes available
// from the server, read them and print them:
if (client.available()) {
char c = client.read();
Serial.print(c);
}
// if the server's disconnected, stop the client:
if (!client.available() && !client.connected()) {
Serial.println();
Serial.println("disconnecting.");
client.stop();
// do nothing forevermore:
for (;;)
;
}
}
and this is the output:
Starting Arduino web client.
connecting...
connected
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 19 Nov 2020 20:24:07 GMT
Content-Type: text/html
Content-Length: 178
Connection: close
Location: https://www.arduino.cc/asciilogo.txt
Strict-Transport-Security: max-age=500; includeSubDomains
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
disconnecting.
It is also possible that I'm just stupid and the answer is clear as day, but if somebody could please tell me exactly what to do, maybe even put an example, that would be greatly appreciated.
I haven't really tried much except changing the setting to see if they work for normal http and some other libraries that worked even worse. Sadly that's all I could find on the internet since I'm not that skilled to be messing with libraries on my own. I am using arduino mkr gsm 1400.
Be sure to call me out if I missed to mention any detail that could help solving this issue
Thanks to anybody that can help me in advance.
Related
Having this as my sketch, the serial monitor saying that the JSON is successfully sent. But it does not reflect to my Cloud DB.
I changed my HTTPS to HTTP but no luck. Where could this go wrong?
My objective is all input in my Arduino will be sent to my server and store to my Cloud DB.
EDIT: After replacing all suggested Edits, I am getting 400 Bad request.
In Postman, the request is working so i know that my request is valid. But I can't make it work in arduino using ethernet shield
void setup() {
Serial.begin(9600);
// initialize the Ethernet shield using DHCP:
if (Ethernet.begin(mac) == 0) {
Serial.println("Failed to obtaining an IP address using DHCP");
while(true);
}
delay(1000);
Serial.println("connecting...");
if (client.connect(HOST_NAME, HTTP_PORT)) {
Serial.println("connected");
} else {
Serial.println("connection failed");
}
//Create JSON doc and write a "name" attribute
const size_t capacity = JSON_OBJECT_SIZE(3);
DynamicJsonDocument doc(capacity);
doc["tank_id"] = "2a";
doc["branch_name"] = "aurora";
doc["water_level"] = "high level";
//POST request
Serial.println("Begin POST Request");
client.println("POST /myURL HTTP/1.1");
Serial.println("POST /myURL HTTP/1.1");
client.println("Host: host.net");
Serial.println("Host: host.net");
client.println("User-Agent: Arduino/1.0");
Serial.println("User-Agent: Arduino/1.0");
client.println("Content-Type: application/json");
Serial.println("Content-Type: application/json");
client.println("Connection: keep-alive");
Serial.println("Connection: keep-alive");
client.print("Content-Length: ");
Serial.print("Content-Length: ");
client.println(measureJson(doc));
Serial.println(measureJson(doc));
client.println();
Serial.print(F("Sending: "));
serializeJson(doc, Serial);
Serial.println();
//This works like client.println, but prints doc to client
serializeJsonPretty(doc, client);
//To let me know that request has been completed
Serial.println("Sent POST Request");
while (client.available()) {
char c = client.read();
Serial.print(c);
}
}
void loop() {
while(client.connected()) {
if(client.available()){
// read an incoming byte from the server and print it to serial monitor:
char c = client.read();
Serial.print(c);
}
}
}
I had exactly the same issue... after hours and hours of trying different codes, I found the problem:
serializeJsonPretty(doc, client);
This code, prints this:
POST /api/ HTTP/1.1
Host: api.xxxxxxx.com
Content-Type: application/json
Connection: close
Content-Length: 63
->
-> {
-> "data": "2a",
-> "branch_name": "aurora",
-> "water_level": "high level"
-> }
But, if you change to serializeJson(doc, client); you'll send an HTTP like this:
POST /api/ HTTP/1.1
Host: api.xxxxxxx.com
Content-Type: application/json
Connection: close
Content-Length: 63
->
-> {"data":"2a","branch_name":"aurora","water_level":"high level"}
I understood that the problem is to send the data in each line.. for me, changing data for only one line, the problem is gone.
I hope to help!
To describe my problem I was trying to connect my Arduino UNO to website created by me in Heroku.
Main purpose was to called rest api function in arduino conected to Internet and get a json data.
My Arduino code:
#include <ArduinoJson.h>
#include <Ethernet.h>
#include <SPI.h>
void setup() {
// Initialize Serial port
Serial.begin(9600);
while (!Serial) continue;
// Initialize Ethernet library
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED};
Ethernet.init(8); // use pin 53 for Ethernet CS
if (!Ethernet.begin(mac)) {
Serial.println(F("Failed to configure Ethernet"));
return;
}
delay(1000);
Serial.println(F("Connecting..."));
// Connect to HTTP server
EthernetClient client;
client.setTimeout(10000);
if (!client.connect("https://salty-cliffs-06856.herokuapp.com", 80)) {
Serial.println(F("Connection failed"));
return;
}
Serial.println(F("Connected!"));
// Send HTTP request
client.println(F("GET /api/command/ HTTP/1.1"));
client.println(F("Host: https://salty-cliffs-06856.herokuapp.com"));
client.println(F("Connection: close"));
Serial.println(F("Done"));
if (client.println() == 0) {
Serial.println(F("Failed to send request"));
return;
}
// Check HTTP status
char status[32] = {0};
client.readBytesUntil('\r', status, sizeof(status));
Serial.println(status);
if (strcmp(status, "HTTP/1.1 200 OK") != 0) {
Serial.print(F("Unexpected response: "));
Serial.println(status);
return;
}
// Skip HTTP headers
char endOfHeaders[] = "\r\n\r\n";
if (!client.find(endOfHeaders)) {
Serial.println(F("Invalid response"));
return;
}
// Allocate JsonBuffer
// Use arduinojson.org/assistant to compute the capacity.
const size_t capacity = JSON_OBJECT_SIZE(3) + JSON_ARRAY_SIZE(2) + 60;
DynamicJsonBuffer jsonBuffer(capacity);
// Parse JSON object
JsonObject& root = jsonBuffer.parseObject(client);
if (!root.success()) {
Serial.println(F("Parsing failed!"));
return;
}
// Extract values
Serial.println(F("Response:"));
Serial.println(root["command"].as<char*>());
// Disconnect
client.stop();
Everything was working fine with that code when I was trying putting there non-secured HTTP address. After putting there my web powered by Heroku secrued by HTTPS I always got error.
The program annouced error when I was checking HTTP status and in my Arduino port terminal I got response:
Unexpected response: HTTP/1.1 400 Bad Request
I checked my heroku logs but there are not listed any request from Arduino. (Just to be sure I tried to called API from web browser and it works)
Could you help me where could be a problem ? I was thinking that it could be because of secured HTTPS. What do you think ?
Thanks for every help :)
First, change client.connect("https://salty-cliffs-06856.herokuapp.com", 80)
from
`https`
to
`http`
as port 80 is not https port and Ethernet shield does not support SSL.
Secondly, you have a wrong http header for Host. the HTTP 1.1 requires that only the domain name to use used without the protocol (i.e. http://) prefix. So change the line:
client.println(F("Host: https://salty-cliffs-06856.herokuapp.com"));
to:
client.println(F("Host: salty-cliffs-06856.herokuapp.com"));
I tried modifying the basic Arduino code from here in order to send HTTP Requests to AWS API Gateway. While the example code from the link worked, I was not able to get a successful connection with AWS API Gateway.
I have tried a combination of things such as removing the https:// from server[], changing the port to 443 instead of 80, removing the /beta from server[], using client.connectSSL instead of client.connect, but none of these have worked so far.
The line:
int err = client.connect(server, 80);
returns me a value of 0.
There are no certificates set up with the AWS API Gateway, so I don't think it's a problem with that. Wifi works perfectly.
Any help would be greatly appreciated!
#include <SPI.h>
#include <WiFi101.h>
#include "arduino_secrets.h"
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
char ssid[] = SECRET_SSID; // your network SSID (name)
char pass[] = SECRET_PASS; // your network password (use for WPA, or
use as key for WEP)
int keyIndex = 0; // your network key Index number (needed only for WEP)
int status = WL_IDLE_STATUS;
// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
char server[] = "https://**********.execute-api.us-west-2.amazonaws.com/beta"; // name address for Google (using DNS)
WiFiClient client;
void setup() {
//Initialize serial and wait for port to open:
Serial.begin(9600);
while (!Serial) {
; // wait for serial port to connect. Needed for native USB port only
}
// check for the presence of the shield:
if (WiFi.status() == WL_NO_SHIELD) {
Serial.println("WiFi shield not present");
// don't continue:
while (true);
}
// attempt to connect to WiFi network:
while (status != WL_CONNECTED) {
Serial.print("Attempting to connect to SSID: ");
Serial.println(ssid);
// Connect to WPA/WPA2 network. Change this line if using open or WEP network:
status = WiFi.begin(ssid, pass);
// wait 10 seconds for connection:
delay(10000);
}
Serial.println("Connected to wifi");
printWiFiStatus();
Serial.println("\nStarting connection to server...");
// if you get a connection, report back via serial:
int err = client.connect(server, 80);
Serial.println(err);
if (err) {
Serial.println("connected to server");
// Make a HTTP request:
client.println("GET /beta HTTP/1.1");
client.println("Host: https://**********.execute-api.us-west-2.amazonaws.com");
client.println("Connection: close");
client.println();
}
}
void loop() {
// if there are incoming bytes available
// from the server, read them and print them:
while (client.available()) {
char c = client.read();
Serial.write(c);
}
// if the server's disconnected, stop the client:
if (!client.connected()) {
Serial.println();
Serial.println("disconnecting from server.");
client.stop();
// do nothing forevermore:
while (true);
}
}
void printWiFiStatus() {
// print the SSID of the network you're attached to:
Serial.print("SSID: ");
Serial.println(WiFi.SSID());
// print your WiFi shield's IP address:
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);
// print the received signal strength:
long rssi = WiFi.RSSI();
Serial.print("signal strength (RSSI):");
Serial.print(rssi);
Serial.println(" dBm");
}
Tips:
(1) client.connect(fqdn, port) is expecting an FQDN for the first parameter
The example in the docs is client.connect("Arduino.cc", 80) this seems to work well in my tests. The docs say "URL" but they mean FQDN.
(2) If you need SSL then you MUST load up your certs using the firmware updater
first. If you are using non-standard pins for the WiFi101 board then you MUST use wifi.setPins() to set the pins or the firmware updater will fail. Adafruit Feather M0 1500 owners will know what I am talking about here.
Reference: https://www.arduino.cc/en/Reference/WiFi101ClientConnect
I hope this helps.
I'm building a hardware device which connects to the AWS IOT platform. According to the documentation the authentication with the aws iot platform is done with TLS. I have the Root CA, client key and client certificate files on the device that authorize the access. Is there a way to use these files in the HTTP header while making the POST request? If so, how? So far here is the code for the Energia IDE (based on the Arduino IDE) and using the WiFiClient methods.
if (client.sslConnect(aws_endpoint, 443))
{
Serial.println("\nConnected to AWS endpoint");
String PostData = "{\"value1\" : \"testValue\", \"value2\" : \"Hello\", \"value3\" : \"World!\" }";
request = "POST /things/";
request += thingname;
request += "/shadow";
request += " HTTP/1.1";
Serial.print("Request:\t"); Serial.println(request);
Serial.print("Post data:\t"); Serial.println(PostData);
client.println(request);
client.println("Host: ");
client.println(aws_endpoint);
client.println(":443");
client.println("User-Agent: Energia/1.1");
client.println("Connection: close");
client.println("Content-Type: application/json");
client.print("Content-Length: "); client.println(PostData.length());
client.println();
client.println(PostData);
client.println();
}
else
{
Serial.println("Connection failed");
}
Serial.println();
Serial.println("Server response:");
Serial.println();
// Capture response from the server. (10 second timeout)
long timeOut = 5000;
long lastTime = millis();
while((millis()-lastTime) < timeOut)
{ // Wait for incoming response from server
while (client.available())
{ // Characters incoming from the server
char c = client.read(); // Read characters
Serial.write(c);
}
}
This however, gives an authentication error:
HTTP/1.1 403 Forbidden
content-type: application/json
content-length: 91
date: Tue, 26 Jul 2016 11:46:59 GMT
x-amzn-RequestId: 4d5388a9-e3c4-460a-b674-c3f971f3330d
connection: Keep-Alive
x-amzn-ErrorType: ForbiddenException:
{"message":"Missing Authentication Token","traceId":"4d5388a9-e3c4-460a-b674-c3f971f3330d"}
The TLS client certificates would be sent/used as part of your client.sslConnect() call, not as part of the HTTP request. The TLS handshake (and exchange/validation of client and server certificates) happens before any HTTP message is sent.
This AWS forums post suggests that you may need to be using port 8443 (not port 443), for the shadow API. It looks like the use/requirement of TLS mutual authentication (via certificates), versus the use of AWS SIGv4 headers, is determined by AWS IOT based on the port used.
Hope this helps!
I have a code for the Arduino Ethernet Shield that will send a GET request to a server and return a PHP echo statement.
However, most of the time it fails to connect to the server.
When it does connect, I keep getting 403 Forbidden error or it says bad header format for "Host:".
I have checked every forum and all StackOverflow links related to the topic, but none of their solutions worked. My code is attached below.
#include <SPI.h>
#include <Ethernet.h>
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
char server[] = "kanishkaganguly.byethost5.com";
IPAddress ip(192,168,0,103);
EthernetClient client;
void setup() {
Serial.begin(9600);
while (!Serial) {
;
}
Ethernet.begin(mac, ip);
delay(1000);
Serial.println("connecting...");
if (client.connect(server, 80)) {
Serial.println("connected");
client.println("GET /test.php HTTP/1.1");
client.println("Host: www.arduino.cc");
client.println("User-Agent: arduino-ethernet");
//client.println("User-Agent: Mozilla/5.0");
//This supposedly fixed 403 error for another user
client.println("Connection: close");
client.println();
}else {
Serial.println("connection failed");
}
}
void loop(){
// if there are incoming bytes available
// from the server, read them and print them:
if (client.available()) {
char c = client.read();
Serial.print(c);
}
// if the server's disconnected, stop the client:
if (!client.connected()) {
Serial.println();
Serial.println("disconnecting.");
client.stop();
// do nothing forevermore:
for(;;) ;
}
}
I figured out the problem. The client.println() as a new line isn't working for some reason. So, here is the updated code
client.print("GET /checkcontrol.php HTTP/1.1\r\n");
client.print("Host: shahz.webatu.com\r\n");
client.print("User-Agent: arduino-ethernet\r\n");
client.print("Connection: close\r\n\r\n");
The \r\n is the right way to go about adding a new line for the server to recognize.
The Host header specifies the hostname of the site you're connecting to. In this case, you are trying to connect to the domain kanishkaganguly.byethost5.com, but your Host header is set to www.arduino.cc. Usually this is incorrect. The Host header should match the domain, so both should be kanishkaganguly.byethost5.com.
When a client connects to a domain, the client first resolves the domain name to an IP, then makes the connection to that IP. Because the connection is made to the IP, the server does not know what domain name was looked up by the client. Because IPs are a limited resource, many hosting providers host multiple domains on a single IP. Without the Host header, the server would not know which page to return to the client.
The println command sends the text followed by "\r\n" so instead of changing every println for print, you could have added CRLF to the close line.
Client.println("Connection: close\r\n")