Connect a microcontroller to the internet and download data from API - embedded

Let me start by saying, I am a complete newbie on microcontrollers. So please help!
I want to use a microcontroller with a stored memory of timestamps for one year. The reason being that I want to write a simple conditional which will trigger an output depending on these times of the day (e.g. today if time == X, set output = 1)
My question is, how can I get the timestamp data into the microcontroller? It is actually downloadable via an API - can I do an API call and download the information through the microcontroller, or is there another way to store the data into its memory?

A "microcontroller" is not a complete system and they are not all the same. It could be a lowly 8-bit 8051 running bare-metal code, or it could be a 32 bit chip capable of running Linux. There is a lot of additional hardware and software between a "microcontroller" and The Internet.
From a software point of view (and that is the scope in which the question is valid on StackOverflow), you need at least a TCP/IP stack and drivers for the network interface (Ethernet most commonly). How you store the data is entirely within your design; your system may have a filesystem, or it may just have a small amount of EEPROM, or you might store it in on-chip flash memory for example. You have to tailor your software solution to the hardware resources available on your system (and your system is not just the microcontroller).
Given a TCP/IP stack the "API" will be whatever that stack provides - it may be a complete BSD socket API or something more lightweight. It may or may not provide application layer protocols such as FTP, Telnet or SSH. For this simple application a proprietary application protocol would probably suffice allowing you to work at the TCP/IP socket level.
Another thing to consider is where time comes from. Will the system have an RTC (requiring an RTC crystal and battery), or will it get time via the Internet connection, GPS or other source?

Answer to your question depends on your design requirements and constrains:
what microcontroller do you want to use, and how much memory will it have available?
can it connect to the internet? Is internet connection available all the time?
how does it know what time it is?
do timestamps change over time? E.g. once downloaded can timestamps list become obsolete?
There are many possible approaches: you can download data manually and write the to SD card, or internal memory of microcontroller (if dataset is small). Or you can program microcontroller to download data using API. Just keep in mind its memory limitations. Many units have only 1-2kB of RAM, so downloading all data at once and storing it in RAM can become a problem.

Related

Is it possible to send jpg files over CANbus

I'm trying to interface a board level USB camera with a STM32 family microcontroller and send the image file to a central computer using CANbus. Just want to know if this is possible/ has been done before and how involved a task it would be.
I worked at a company where we sent live (low-resolution infra-red) video streams over CAN, but towards the end of my time there they shifted towards ethernet.
So it is possible, but certainly not what it is best suited for. The main advantages of CAN are that it is a multi-point, multi-master bus with built in arbitration. It is meant for short packets, typically 8 bytes (CAN FD allows you to increase that).
If your camera is USB, why not just get a USB repeater cable or USB-over-ethernet gateway?
If there is already a CAN network in place that you are piggy-backing onto then you need to consider what impact you will have on the existing traffic.
If you are starting from scratch then of course CAN will work but it would be an odd choice.
Depending on if its CAN or CANFD (Affects the maximum bulk transfer packet size) you have higher level protocol options to packetise your images and send them over canbus like any other block of data.
For just reguarlar CAN your after part of the standard called J1939.21 Data Link Layer, there are public versions of this floating around online, however due to the agreement when purchasing the standard, I am not able to share the specifics from what I have.
Its on pages 27-28 of the 2001 revision.

How to make the embedded system configurable without update the whole firmware

I'm totally a newbie in embedded software. Currently, I'm working on a project that implements an image processing pipeline on an ARM Cortex-M4 based MCU(board model: STM32F446RE).
I would like to be able to configure the parameters of the pipeline on the fly without actually update the entire firmware since we're using LoRa which has low bandwidth.
I have googled for several hours and could not find any valid solution. So could you please point me in a direction? Thank you very much.
BTW, I don't know if this is relevant, but I'm using FreeRTOS kernel with CMSIS RTOS API v2.
If you are asking this question, I would hope that either:
The board is still under design or
You have a board that was designed by someone who has thought about these issues.
If #2, speak to whoever designed the board, and find out what resources were put in, to handle these issues.
If #1, presumably you have input into the design.
Necessary resources:
Non-volatile storage: flash, eeprom, etc.
One or more ways to write parameters to that non-volatile storage
Desirable resource: communication line for input/output while running (serial is often used).
Once you have these resources, you do the following:
Design the variables, data structures, etc. to hold the parameters
Design your non-volatile storage, taking into account:
a. The features/limitations of your media (for example, flash memory generally requires an erase before writing. Erase takes time and must be done by sector, not individual bytes.
b. Verification: your program should have a way to verify that the non-volatile storage has valid values, not garbage, not all 0xFFs, and either fail or use defaults or some such, if it is not valid
Then you can write a program using this.
You need to consider how you will write the values to the non-volatile memory
during development
in production
They are not likely to be the same.
During development, you want to be able to easily change values. You may have a way to burn your flash chip via a JTAG. You may have a communications port which either runs some kind of simple CLI, accepts commands via some protocol, asks questions and reads the answers via a terminal emulator, etc. The program can then write the values to the non-volatile memory.
In production, you will likely want to burn the 'correct' values once, when setting up the system, without too much operator involvement.
This is just a starting guideline...as mentioned in the comments, your question is very general.

Hacking computer hardware to do experiment control

I am a physicist, and I had a revelation a few weeks ago about how I might be able to use my personal computer to get much finer control over laboratory experiments than is typically the case. Before I ran off to try this out though, I wanted to check the feasibility with people who have more expertise than myself in such matters.
The idea is to use the i/o ports---VGA, ethernet, speaker jacks, etc.---on the computer to talk directly to the sensors and actuators in the experimental setup. E.g. cut open one side of an ethernet cable (with the other end attached to the computer) and send each line to a different device. I knew a postdoc who did something very similar using a BeagleBone. He wrote some assembly code that let him sync everything with the internal clock and used the GPIO pins to effectively give him a hybrid signal generator/scope that was completely programmable. It seems like the same thing should be possible with a laptop, and this would have the additional benefit that you can do data analysis from the same device.
The main potential difficulty that I foresee is that the hardware on a BeagleBone is designed with this sort of i/o in mind, whereas I expect the hardware on a laptop will probably be harder to control directly. I know for example (from some preliminary investigation, http://ask.metafilter.com/125812/Simple-USB-control-how-to-blink-an-LED-via-code) that USB ports will be difficult to access this way, and VGA is (according to VGA 15 pin port data read and write using Matlab) impossible. I haven't found anything about using other ports like ethernet or speaker jacks, though.
So the main question is: will this idea be feasible (without investing many months for each new variation of the hardware), and if so what type of i/o (ethernet, speaker jacks, etc.) is likely to be the best bet?
Auxiliary questions are:
Where can I find material to learn how I might go about executing this plan? I'm not even sure what keywords to plug in on Google.
Will the ease with which I can do this depend strongly on operating system or hardware brand?
The only cable I can think of for a pc that can get close to this would be a parallel printer cable which is pretty much gone away. It's a 25 wire cable that data is spread across so that it can send more data at the same time. I'm just not sure if you can target a specific line or if it's more of a left to right fill as data is sent.
To use one on a laptop today would definitely be difficult. You won't find any laptops with parallel ports. There are usb to parallel cables and serial to parallel cables but I would guess that the only control you would have it to the usb or serial interface and not the parallel.
As for Ethernet, you have 4 twisted pair with only 2 pair in use and 2 pair that are extra.
There's some hardware that available called Zwave that you might want to look into. Zwave will allow you to build a network of devices that communicate in a mesh. I'm not sure what kind of response time you need.
I actually just thought of something that might be a good solution. Check out security equipment. There's a lot of equipment available for pc's that monitor doors, windows, sensors, etc. That industry might what your looking for.
I think the easiest way would be to use the USB port as a Human Interface Device (HID) and using a custom built PIC program and a PIC that includes the USB functionality to encode the data to be sent to the computer and in that way be able to program it independently from the OS due to the fact that all mayor OS have the HID USB functionality.
Anyways if you used your MIC/VGA/HDMI whatever other port you still need a device to encode the data or transmit it, and another program inside the computer to decode that data being sent.
And remember that different hardware has different software (drivers) that might decode the raw data in other odd ways rendering your IO hardware dependent.
Hope this helps, but thats why the USB was invented in the first place to make it hardware and os independent.

Details on USB- no luck so far

I've been looking for a detailed description for how USB protocol and cabling works for a long time with no luck. I am looking for a detailed yet not overcomplicated explanation of how things work on the software and hardware side of USB. Links and explanations would be appreciated. I've really run out of ideas, so it would be great if you can help me out.
This is what I do know:
USB hardware carries 4 lines- 5V power, ground, and 2 full duplex lines.
When connecting, the device can ask for a specified amount of current.
The transfer speeds for USB are quite fast compared to traditional serial connections.
When connecting, a device will output descriptors to the host describing itself. These descriptors will also be used for data.
What I don't know:
How does a program in C/C++ write directly to a USB port? Does it write to an address in the port?
How do some devices describe themselves as HID?
How do drivers work?
Everything else...
Thank you!
Identification
Every device has a (unique) Vendor and Product ID. These are provided (sold) by usb.org to identify a device. You can use a library like libusbx to enumerate all connected devices and select the one with the Vendor and Product ID you are looking for.
HID Descriptors
The point of HID descriptors is actually to do away with drivers. HID descriptors are a universal way of describing your device so you don't need to waste time on a driver for every system/architecture/etc/. (Same concept as the JVM.)
Reports
You will use either the input, output, or feature reports to read or write to your device. You send a stream to your device on the input or feature report. This is typically 8 bytes I believe. Only one of which is a single character you wish to write. The HID descriptor contains all the information you need to put together a report. Although I'm struggling to find a related link to clarify this.
Potential Libraries
In an effort to be open-minded here are all the libraries I am familiar with and some info about them.
libusb-0.1
First off is libusb-0.1. This used to be the go to and was built in to many Linux kernels and Windows I believe. It is very easy to use and there is a lot of documentation. However, the owner never updated and it wasn't edited for many years. It supports only synchronous transfers. (If an error occurs, the program can wait infinitely while it expects a transfer.)
libusbx
Next is libusbx. This is what most people would suggest today and I agree. It was published by those frustrated by the owner of libusb-0.1. The code is much more lightweight, up-to-date, and importantly does not require root privileges like libusb-0.1 and libusb-1.0 (Discussed in a second). It supports synchronous or asynchronous transfers.
libusb-1.0
Then there is libusb-1.0. This was the first update to libusb-0.1 in some number of years. It is not compatible with libusb-0.1. This was published the same day as libusbx as a retaliation (I assume) and an attempt to rectify the lack of updated content and conserve a user-base. It supports synchronous or asynchronous transfers.
hid.h
Finally, there is the hid library. This was built on top of libusb as another layer of abstraction. But honestly, I think it's just really confusing and it just adds more overhead than necessary.
Some Good Resources
Understanding HID Descriptors
Control Message Transfer Documentation (Very Good Link IMO)
Rolling Your Own HID Descriptor
Good Visual of HID Reports for Transfers
Great List of bmRequestType constants (You will need this or similar)
A simple terminal app for speaking with DigiSpark using libusbx and libusb-0.1
I know this isn't exactly what you are looking for, but maybe it will get you started!
This website has a general overview of how USB devices work:
https://www.beyondlogic.org/usbnutshell/usb1.shtml
Particular sections give answers to things from the list of things you don't know yet about USB.
E.g. to find out how USB devices identify themselves, read about USB descriptors:
https://www.beyondlogic.org/usbnutshell/usb5.shtml#DeviceDescriptors
To learn how a C/C++ program can talk to a USB device, see examples on using the libusb library:
https://github.com/libusb/libusb/tree/master/examples
To learn how USB drivers work, see a tutorial from Bootlin:
https://bootlin.com/blog/usb-slides/

Reading live RAM variables from a Micro controller in VB.net

I want to read the global variables via the JTAG port, live, when a program is running on the microcontroller. Is it possible?
JTAG defines only a physical interface, it does not describe the on-chip debug capabilities of a particular processor which may or may not support access during execution.
Moreover whether it can be done in VB is not really the issue, the important issue is what hardware device and/or I/O port you are using for the JTAG interface, and whether a driver and API to access via .Net is available. That said VB.Net is not the first language I'd choose for that in any case.
A good place to start perhaps is OpenOCD, though it is not .Net specific.
"Almost-Live" is possibly doable, depending on the JTAG implementation. Often JTAG activity which reads memory does so by stealing cycles from the micro (or sometimes even inserting instructions into the pipeline). I'm not sure there's a micro which allows completely transparent access to memory over JTAG.
"All you need to do" is understand the JTAG implementation, know where the variable is located and issue a "memory read" command by wiggling the JTAG pins in the appropriate fashion. This is not a small task, which is why professional engineers are willing to pay (sometimes large amounts of) money for tools which perform this task.
Often the free (limited) toolchains the vendors provide can perform this also.
Yes, I suppose it is possible. But you'll need to drive the JTAG port (that sounds painful!) and know exactly where the data is stored on the chip, and what the formatting is.