I am new to this, but i am using AWS IOT CORE with esp8266. So, here i am connecting esp8266 to mqtt broker which is on AWS cloud and sending data to it. Which mean my esp8266 is a client which is connected to a broker AWS IOT CORE.
SO, my main question is that when i connect single esp8266 to AWS IOT CORE then i need some certificates like CA, CERTI & Private key etc.. These file's provide secure TLS connection of esp8266 with AWS MQTT IOT core. But now i want to connect 2x esp8266 with the same Mqtt Broker with same Topic but most essentially with same TLS certificates that i generated earlier for esp8266 (1). So is it possible to use those same certificates to provide secure TLS connection to both esp8266 at the same time.
Need a answer to that in a described manner. Do help me out here.
For reference i used this project to connect my esp8266 with AWS IOT CORE through MQTT
https://electronicsinnovation.com/how-to-connect-nodemcu-esp8266-with-aws-iot-core-using-arduino-ide-mqtt/
No, client certificates are used to uniquely identify each connecting client.
You need to use a separate certificate per device.
Related
I am trying to setup Azure IoT Edge as transparent gateway to which I want to connect directly via MQTT (without any IoT SDKs).
###---Short Version---###
I want to connect directly to IoT Edge via MQTT like this to IoT Hub. What is the host, username and password for connection via symmetric keys to connect to the IoT Edge Gateway?
###---Long Version---###
The communication should work like this: Device (MQTT) -> Azure IoT Edge -> Azure IoT Central.
I started with provisioning the Device to Azure IoT Central and was able to connect there directly. I used a .NET sample to provision the devices on Azure IoT Central and get the Iot-Hub name. Then I created the SAS Token to connect directly with a MQTT client to Azure IoT Central (assigned IoT-Hub)
I used the following credentials:
"ClientID": "plc2"
"Host": "mqtts://iotc-x-x-x-x-x.azure-devices.net"
"Port": "8883"
"Username": "iotc-x-x-x-x-x.azure-devices.net/plc2/?api-version=2021-04-12"
"Password": "SharedAccessSignature sr=iotc-x-x-x-x-x.azure-devices.net%2Fdevices%2Fplc2&sig=P......D&se=1652037456"
I created the SAS Token following this documentation for MQTT direct device connection: SAS Token structure
As ressourceUri for creating SAS Token I used the following: "iotc-x-x-x-x-x.azure-devices.net.azure-devices.net/devices/plc2"
For testing I used MQTTX windows client with the Azure Baltimore Root CA Certificate and the credentials explained and everything was fine.
The problem
Now I want the communication to run via the Azure IoT Edge as transparent gateway. I followed following documentation from Microsoft:
Created the demo certificates from Microsoft
Configured "config.toml" -> Set the hostname to: iotedge-prod1
Provisioned the device in Azure IoT Central -> Set its gateway to the Edge device
Used a sample Manifest file for edge where the ports are open and upstream is allowed
On my windows pc created a host file entry with "192.168.0.100 iotedge-prod1"
Ping is ok for the new hostname of the edge device
Generated a .PKCS12 certificate from the .PEM certificate for the windows machine
But now I cannot connect via my MQTT Client -> It says Username or Password wrong
My big question
What does the MQTT connection data to the Edge device look like? The structure with Edge Hub I have not yet understood I think. I tried many combinations of credentials but I didn't get it
"ClientID": "plc2" --> Should be ok
"Host": "mqtts://iotedge-prod1" --> In my opinion the IP-Adress is important right?
"Port": "8883" --> Same like before for TLS
"Username": "iotedge-prod1.iotc-x-x-x-x-x.azure-devices.net/plc2/?api-version=2021-04-12" or just "iotedge-prod1/plc2/?api-version=2021-04-12" --> What must the username be like?
"Password": "SharedAccessSignature sr=iotc...." --> Same thing for the password. Which information is necessary in the SAS Token? Is the ressourceUri for creation: "iotedge-prod1.myHub.azure-devices.net/devices/plc2" or just "iotedge-prod1/devices/plc2" or anything different?
I hope I could explain halfway understandable what my problem is.
Additional information:
Edge-config_toml.jpg
Edge-IoTCentral.jpg
Edge-Manifest (MS Docs)
The username and password in case of direct connection to IoT Central or via IoT Edge as transparent gateway will be the same. This is because IoT Edge will pass through the token to IoTHub for attestation.
Only change required will be the Host, which should be mqtts://hostname or ip
Ensure in MQTTX client the Certificate is selected as Self signed and provide path of the Test Root CA, no need for client cert and key. The parent of the device also needs to be correctly set in IoT Central.
e.g. the deviceId here is mqttclient and iotedge gateway is raspi###
Check edgeHub log to see if you are getting connection attempt with "iotedge logs -f edgeHub' from MQTTX client
One possible problem also was that the certificate hierachy was not ok. If Azure IoT Edge is installed there are some certificates installed. After using the demo certificates from Azure the hierarchy should be mainly the same like in the documentation from Microsoft - starting with "Azure ... Test Only"
Demo Azure Certificate Hierarchy
I am trying to connect my NodeMCU ESP-12N to Azure IoT hub.
I have created client with code like this:
esp8266 = mqtt.Client(DEVICE, 240, USER, PASSWD)
and receiving error on calling esp8266:connect
PANIC: unprotected error in call to Lua API (init.lua:127: ssl not
available)
When I previously have connect my Arduino, then it was important to flash Wi-Fi with SSL certificate. However I can't do it with NodeMCU (or more right to say - don't know how).
I can get certificates with:
openssl s_client -showcerts -connect ArduinoDemoHub.azure-devices.net:8883
When I have add (thank you #MarcelStör for hint, I have tried previously net.cert.verify)
tls.cert.verify([[
-----BEGIN CERTIFICATE-----
MIIGsTCCBJmgADJKJFdWgAIQF5Koy/50vI3cQAAAAhAXjANBgkqhkiG9w0BAQsF
......
lz3ZDUcyzRgG6TRtA3SjASEUlJMt3f7xuI2nNpt8p5gy9pXuRw==
-----END CERTIFICATE-----
]])
tls.cert.verify(true)
I'v received another error:
Error Connecting: -5
UPDATE:
There is another similar topic, and I have set time, but still without result.
UPDATE 2:
Here are some links that I trying to use:
Use Azure IoT Hub without client libraries (MQTT)
Sending Device-to-Cloud (D2C) Messages
sankarcheppali gitHub with samples
Here is official Link
Communicate with your IoT hub using the MQTT protocol
that tells:
To use the MQTT protocol directly, your client must connect over
TLS/SSL. Attempts to skip this will fail with connection errors.
This link also provides username that is different from old code samples:
{iothubhostname}/{device_id}/api-version=2016-11-14
TLS/SSL by default is disabled in the firmware build configuration. CLIENT_SSL_ENABLE in user_config.h must be uncommented before compiling/building the firmware.
The easiest is to use the cloud builder at https://nodemcu-build.com (by yours truly) and check the TLS checkbox.
Also, all TLS/SSL related functions in the net module (you use net.cert.verify) were extracted into the tls module a while ago. Unless there's convincing arguments for not using a current firmware build I suggest you try the dev branch. When you enable "TLS/SSL support" in the builder you get the tls module automatically.
UPDATE: There is nice article about Azure IoT hub and certificates: Certificate between IoT hub and devices connection
I have created SAS with help of Device Explorer, that is looks like:
SharedAccessSignature sr={your hub name}.azure-devices.net%2Fdevices%2FMyDevice01%2Fapi-version%3D2016-11-14&sig=vSgHBMUG.....Ntg%3d&se=1456481802
This value is set to PASSWD variable. Have changed USER variable to:
USER = "MyAzureIoTHub.azure-devices.net/MyDeviceID/api-version=2016-11-14"
And has set to false tls.cert.verify
tls.cert.verify(false)
Works now :)
You can refer to the following two libraries to connect NodeMCU ESP-12N to Azure IoT hub, these two libraries could be searched in Library Manager:
AzureIoTHubMQTTClient
AzureIoTProtocol_MQTT
After installation,you can refer to the samples.
Is kafka suitable for Internet-use?
More precisely, what I want is to expose kafka topics as "public interface", then external consumers (or producers) can connect to it. Is it possible?
I hear there are problems if I want to use the cluster in both internal and external networks, because it is then hard to configure advertised.host.name. Is that true?
And do I have to expose zookeeper as well? I think the new consumer/producer api no longer need that.
Kafka's wire protocol is TCP-based and works fine over the public internet. In the latest versions of Kafka you can configure multiple interfaces for both internal and external traffic. Examples of Kafka over the internet in production include several Kafka-as-a-Service offerings from Heroku, IBM MessageHub, and Confluent Cloud.
You do not need to expose zookeeper if the Kafka clients use the new consumer API.
You may also choose to expose a REST Proxy such as the open source Confluent REST Proxy as a more client firewall friendly interface since it runs over HTTP(S) and will not be blocked by most corporate or personal firewalls.
I would personally not expose the Kafka server directly to clients via TCP for these reasons, only to name a few:
If a bad client opens too many connections this may affect the stability of the Kafka platform and may affects other clients too
Too many open files on the Kafka server, HW/SW settings and OS tuning is needed to limit uncontrolled clients
If you need to add a Kafka server to increase scalability, you may need to go through a lot of low level configuration (firewall, IPs visibility, certificates, etc.) on both client and server side. Other product address these problems using gateways or proxies: Coherence uses extend proxy clients, tibco EMS uses routed destinations, other SW (many JMS servers) use Store&Forward mechanisms, etc.
Maintenance of the Kafka nodes, in case of clients attached to the Kafka servers, will have to consider also the needs of clients and the SLA (service level aggreement) that have been defined with the client (ex. 24*7*365)
If you use Kafka also as a back end service, a multi layered architecture should be taken into consideration: FE gateways and BE services, etc.
Other considerations require to understand what exacly you consider to be an external (over the internet) consumer/producer in your system. Is it a component of your system that needs to access the Kafka servers? Are they internal or external to your organization, etc.
...
Naturally all these considerations can be correctly addressed also using a TCP direct connection to the Kafka servers, but I would personally use a different solution.
HTTP proxies
Or at least I would use a dedicated FE Kafka server (or couple of servers for HA) dedicated for each client that forward the messages to the main Kafka group of servers
It is possible to expose Kafka over the internet (in fact, that's how managed Kafka providers such as Aiven and Instaclustr make their money) but you have to ensure that it is adequately secured. At minimum:
ZooKeeper nodes should reside in a private subnet and not be routable from outside. ZK's security is inadequate and, at any rate, it is no longer required to bootstrap Kafka clients with ZK address(es).
Limit access to the brokers at the network level. If all your clients connect from a trusted network, then set appropriate firewall rules. If in AWS, use VPC peering or Direct Connect if you are connecting cloud-to-cloud or cloud-to-ground. If most of your clients are on a trusted network but a relative minority are not, force the latter to go via a VPN tunnel. Finally, if you want to allow connectivity from arbitrary locations, you'll just have to allow * on port 9092 (or whichever port you configure the brokers to listen on); just make sure that the other ports are closed.
Enable TLS (SSL) for client-broker connections. This is easily configured with a self-signed CA. Depending on how you expose your listeners, you may need to disable SSL hostname verification on the client. (The certificate chain of trust breaks if the advertised host names don't match the certificate's common name.) The clients will need the CA certificate installed. (Same CA that signed the brokers' certs.)
Optionally, you may enable mutual TLS authentication; however, this is logistically more taxing, as it requires each client to have its own private key that is signed by a CA trusted by the broker.
Use SASL to authenticate the client to the broker and create individual users for each application and each person that is expected to access the cluster.
Issue minimally-sufficient cluster- and topic-level access privileges in the ACLs for each user, following the Principle of Least Privilege (PoLP).
One other thing to bear in mind: Not all tooling supports SASL/SSL connectivity and some tools actually require a connection to ZooKeeper nodes (which will not be reachable in the above setup). Make sure any tooling you rely on uses the 'new' style of connectivity directly to the Kafka brokers and does not require a Zookeeper connection.
Beyond configuring client TLS, brokers have to have public IPs which we try to avoid. Normally for other services we hide everything behind load balancers. Would this be possible with kafka?
I'm not sure the Confluent REST proxy hosted on a public server is a real option when you need the high performance batching of the java producer client.
We have the MobileFirst platform running on our Linux machine. Business Logic Web services are running on a separate Server. The Certificate is issued by our Business Server.
We have overwritten the MobileFirst certificate mfp-default-cert.crt with the certificate issued by our Server. This certificate is used to establish the HTTPS SSL connection between the MobileFirst adapter and the web service.
We have successfully integrated this encrypted web service (using SSL) between MobileFirst (using HTTP Adapter) and Server in a Native app.
Now we want to implement/configure SSL connection between Mobile Device/emulator and MobileFirst server for secure communication using the same issued certificate.
Can anyone please advise how to do that?
As we know for SSL Connection, there should be a proper handshake to happen between the server and client.
Client(Device/Emulator) which you are trying to connect with server should have a corresponding CA Certificate installed on it so that it will have a successful secured communication with sever machine .
so you should have the client side certificate(example.crt) installed on your device/emulator to connect the server on HTTPS port.
And it depends even on the Device u are using to connect .
For iOS devices from 9.0. Apple mandates more secure way of connection
https://developer.ibm.com/mobilefirstplatform/2015/09/07/preparing-ibm-mobilefirst-platform-server-app-transport-security-ios-9/
Hope this serves your purpose. :-)
I have a C++ application that has a SSL/TLS communication with its own server and i don't have any access to that server. I'm trying to find out what is it sending from my PC to the server.
I tried burp and fiddler as man-in-middle but it didn't work. The application does not support Proxy so i tried routing the traffic using proxifier to burp and fiddler but it didn't work.
So I came up with these articles https://isc.sans.edu/forums/diary/Psst+Your+Browser+Knows+All+Your+Secrets+/16415 and http://ask.wireshark.org/questions/4229/follow-ssl-stream-using-master-key-and-session-id
I just need to know , How I can find Master-key and Session-ID to decrypt SSL/TLS trafic.
It depends on the TLS cipher suite being used. If the ciphersuite uses forward secrecy (DHE) you cannot decrypt the stream. If it uses RSA encryption then you need at least the private key of the server. If it also uses client authentication then you would also need the private key of the client. if it uses symmetric encryption you need the symmetric (master) key from either one of them.
But if you do have a C++ application, I would simply add logging to that application (at the lowest level).
You can use following alrternates on a x86 windows
STRACE - http://blogs.msdn.com/b/emmanubo/archive/2007/06/04/introduction-to-strace-httpreplay-support-tools.aspx
SOCKTRC if this app is on windows checkout
http://blogs.msdn.com/b/emmanubo/archive/2007/08/03/socktrc-tool.aspx
HTTPREPLAY -
generally used for browsers but here can be used to view the responses in the UI