How to use DLLs to exchange values between MQL4 programs? - dll

First of all I need to say I don't know much about DLLs.
I am trying to send data from one program to another, using functions from kernel32.dll. My programs are coded in MQL4.
This is the code I use for the server part, which is supposed to save the data:
#define INVALID_HANDLE_VALUE -1
#define BUF_SIZE 256
#define PAGE_READWRITE 0x0004
#define FILE_MAP_ALL_ACCESS 0xf001F
#import "kernel32.dll"
int CreateFileMappingA(int hFile, int lpAttributes, int flProtect, int dwMaximumSizeHigh, int dwMaximumSizeLow, string lpName);
int MapViewOfFile(int hFileMappingObject, int dwDesiredAccess, int dwFileOffsetHigh, int dwFileOffsetLow, int dwNumberOfBytesToMap);
int UnmapViewOfFile(int lpBaseAddress);
int RtlMoveMemory(int DestPtr, string s, int Length);
int CloseHandle(int hwnd);
int CreateMutexA(int attr, int owner, string mutexName);
int ReleaseMutex(int hnd);
int WaitForSingleObject(int hnd, int dwMilliseconds);
bool started = False;
int hMapFile = 0;
int pBuf=0;
int hMutex;
int OnInit()
{
if(!started) {
started = true;
string szName="Global\\Value1";
int hMapFile = CreateFileMappingA(INVALID_HANDLE_VALUE,0,PAGE_READWRITE,0,BUF_SIZE,szName);
if(hMapFile==0) {
Alert("CreateFileMapping failed!");
return;
}
pBuf = MapViewOfFile(hMapFile, FILE_MAP_ALL_ACCESS, 0, 0, BUF_SIZE);
if(pBuf==0) {
Alert("Map View failed!");
return;
}
hMutex = CreateMutexA(0,0,"PriceMapMutex");
}
}
void OnTick()
{
WaitForSingleObject(hMutex,1000);
if(pBuf==0) return;
string szMsg = DoubleToStr(Bid,Digits);
Comment("Data: ",szMsg);
RtlMoveMemory(pBuf, szMsg, StringLen(szMsg)+1);
ReleaseMutex(hMutex);
return(0);
}
int deinit()
{
switch(UninitializeReason()) {
case REASON_CHARTCLOSE:
case REASON_REMOVE:
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
break;
}
return(0);
}
This is what I use for my client part, which is supposed to pick up the data:
#define INVALID_HANDLE_VALUE -1
#define BUF_SIZE 1024
#define FILE_MAP_READ 4
extern int BufferSize = 1024;
#import "kernel32.dll"
int OpenFileMappingA(int dwDesiredAccess, bool bInheritHandle, string lpName);
string MapViewOfFile(int hFileMappingObject, int dwDesiredAccess, int dwFileOffsetHigh, int dwFileOffsetLow, int dwNumberOfBytesToMap);
int UnmapViewOfFile(string lpBaseAddress);
int CloseHandle(int hwnd);
int CreateMutexA(int attr, int owner, string mutexName);
int ReleaseMutex(int hnd);
int WaitForSingleObject(int hnd, int dwMilliseconds);
string szName;
int hMapFile;
string obj;
string data;
int hMutex;
double dd;
int OnInit()
{
szName="Global\\Value1";
hMapFile = OpenFileMappingA(FILE_MAP_READ,False,szName);
if(hMapFile==0) {
Alert("CreateFile Failed!");
return;
}
obj="data";
ObjectCreate(obj,OBJ_HLINE,0,0,0);
ObjectSet(obj,OBJPROP_COLOR,Gold);
hMutex = CreateMutexA(0,0,"PriceMapMutex");
}
void OnDeinit(const int reason)
{
CloseHandle(hMapFile);
Comment("");
ObjectDelete(obj);
return(0);
}
void start()
{
getsignal();
Comment("Data: ",DoubleToStr(dd,Digits));
Sleep(50);
}
void getsignal() {
WaitForSingleObject(hMutex,333);
data = MapViewOfFile(hMapFile,FILE_MAP_READ,0,0,BUF_SIZE);
dd = StrToDouble(data);
ReleaseMutex(hMutex);
UnmapViewOfFile(data);
ObjectMove(obj,0,Time[0],dd);
}
The code basically works. However I am facing 2 major problems with it.
Problem number 1:
I want to exchange multiple values ( value1, value2, value3, ... ). For some reason it seems to be irrelevant which name I use for szName="Global\\Value1". The server saves the value and the client picks it up no matter what names I use szName="Global\\Value1", szName="Global\\Value2" or szName="Global\\Value3". So for example in the server code I use szName="Global\\Value1" and in my client code I use szName="Global\\Value3" the client still picks up the value which the server writes to szName="Global\\Value1".
Problem number 2:
my client is only stable for about 5 hours. After that I get a message in the client programme saying
"There is a problem and the program needs to be closed...".
Then I close and restart my client and its again working for the next 5 hours.
Has anyone any idea?

FILE MEDIUM
I agree that Kernel32 is not a good option if you need to do MT4-to-MT4 interfacing. The reason is that Kernel32 is Windows specific. The EA won't work on Mac. Also, messing around with Kernel32 DLL might cause memory leaks (eg, your 5hr live). Plus, it requires user to know how to enable DLL (you'd be surprise how many users have no idea how to enable it).
Recommendation:
If you only need SAME MT4 exchange (EAs between charts), then use the GlobalVariableGet(), GlobalVariableSet(), etc.
If you need exchange between 2 different MT4s (on the same PC) --even if it is across different broker MT4, then use the FILE system: Files\FilePipe.mqh which allows you to write to the common MT4 folder:
#include <Files\FilePipe.mqh>
CFilePipe voPipeOut;
voPipeOut.Open("yourFileName.txt", FILE_WRITE|FILE_COMMON|FILE_BIN);
voPipeOut.WriteString("WhatEverMessage, probably some CSV value here");
voPipeOut.Close();
and subsequently
CFilePipe voPipeFile;
string vsInString = "";
voPipeFile.Open("yourFileName.txt", FILE_SHARE_READ|FILE_COMMON|FILE_BIN);
voPipeFile.Seek(0,SEEK_SET);
voPipeFile.ReadString( vsInString );
voPipeFile.Close();
This way, your EA won't depend on DLLs and also works in a wide range of environments. It is very fast (under 2ms for a 1Mb pipe). It works even for cross-broker interfacing (exchanging info [feed?] between 2 different brokers).

The best idea?
The best thing I can advise you is to stop trying to tweak KERNEL32.DLL published API to make it use with MetaTrader Terminal 4 code-execution ecosystem and to rather start designing a professional distributed system, independently of injecting objects into the O/S pagefile and hassling with semaphores and MUTEX-es.
Besides the best next step:
MQL4 code ought NEVER block. A MUTEX-signalling turned into a non-blocking state is a must
MQL4 code / API mapper ought respect the data-types and their actual memory sizes in MQL4
MQL4 code ought conform to the recent New-MQL4 rules ( sections are in "old"-MQL4 )
MQL4 declared string is not a C-lang string, but rather a struct! Handle with care!
MQL4 code violated in several places syntax rules, just test with #property strict
MQL4 code is "shooting itself in leg" when ignoring namespace boundaries / scopes of declaration
MQL4 code ignores potential error states, not inspecting any GetLastError() to handle such collision(s)
MQL4 code does not gracefully return resources ( forgets to clear 'em )
MQL4 code proposed exposes itself into an immense risk of KERNEL32.DLL API usage unlocked stealth security flaw / enabling a run-time hijacking hack
better use separation of concerns, using ZeroMQ or nanomsg messaging to "exchange values between ( not only ) MQL4 programs"

Related

How to put BG96 on power save mode between sending messages to Azure IoT Hub over HTTP

I'm using a Nucleo L496ZG, X-NUCLEO-IKS01A2 and the Quectel BG96 module to send sensor data (temperature, humidity etc..) to Azure IoT Central over HTTP.
I've been using the example implementation provided by Avnet here, which works fine but it's not power optimized and with a 6700mAh battery pack it only lasts around 30 hours sending telemetry ever ~10 seconds. Goal is for it to last around a week. I'm open to increasing the time between messages but I also want to save power in between sending.
I've gone over the Quectel BG96 manuals and I've tried two things:
1) powering off the device by driving the PWRKEY and turning it back on when I need to send a message
I've gotten this to work, kinda… until I get a hardfault exception which happens seemingly randomly anywhere from within ~5 minutes of running to 2 hours (messages successfully sending prior to the exception). Output of crash log parser is the same every time:
Crash location = strncmp [0x08038DF8] (based on PC value)
Caller location = _findenv_r [0x0804119D] (based on LR value)
Stack Pointer at the time of crash = [20008128]
Target and Fault Info:
Processor Arch: ARM-V7M or above
Processor Variant: C24
Forced exception, a fault with configurable priority has been escalated to HardFault
A precise data access error has occurred. Faulting address: 03060B30
The caller location traces back to my .map file and I don't know what to make of it.
My code:
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
//#define USE_MQTT
#include <stdlib.h>
#include "mbed.h"
#include "iothubtransporthttp.h"
#include "iothub_client_core_common.h"
#include "iothub_client_ll.h"
#include "azure_c_shared_utility/platform.h"
#include "azure_c_shared_utility/agenttime.h"
#include "jsondecoder.h"
#include "bg96gps.hpp"
#include "azure_message_helper.h"
#define IOT_AGENT_OK CODEFIRST_OK
#include "azure_certs.h"
/* initialize the expansion board && sensors */
#include "XNucleoIKS01A2.h"
static HTS221Sensor *hum_temp;
static LSM6DSLSensor *acc_gyro;
static LPS22HBSensor *pressure;
static const char* connectionString = "xxx";
// to report F uncomment this #define CTOF(x) (((double)(x)*9/5)+32)
#define CTOF(x) (x)
Thread azure_client_thread(osPriorityNormal, 10*1024, NULL, "azure_client_thread");
static void azure_task(void);
EventFlags deleteOK;
size_t g_message_count_send_confirmations;
/* create the GPS elements for example program */
BG96Interface* bg96Interface;
//static int tilt_event;
// void mems_int1(void)
// {
// tilt_event++;
// }
void mems_init(void)
{
//acc_gyro->attach_int1_irq(&mems_int1); // Attach callback to LSM6DSL INT1
hum_temp->enable(); // Enable HTS221 enviromental sensor
pressure->enable(); // Enable barametric pressure sensor
acc_gyro->enable_x(); // Enable LSM6DSL accelerometer
//acc_gyro->enable_tilt_detection(); // Enable Tilt Detection
}
void powerUp(void) {
if (platform_init() != 0) {
printf("Error initializing the platform\r\n");
return;
}
bg96Interface = (BG96Interface*) easy_get_netif(true);
}
void BG96_Modem_PowerOFF(void)
{
DigitalOut BG96_RESET(D7);
DigitalOut BG96_PWRKEY(D10);
DigitalOut BG97_WAKE(D11);
BG96_RESET = 0;
BG96_PWRKEY = 0;
BG97_WAKE = 0;
wait_ms(300);
}
void powerDown(){
platform_deinit();
BG96_Modem_PowerOFF();
}
//
// The main routine simply prints a banner, initializes the system
// starts the worker threads and waits for a termination (join)
int main(void)
{
//printStartMessage();
XNucleoIKS01A2 *mems_expansion_board = XNucleoIKS01A2::instance(I2C_SDA, I2C_SCL, D4, D5);
hum_temp = mems_expansion_board->ht_sensor;
acc_gyro = mems_expansion_board->acc_gyro;
pressure = mems_expansion_board->pt_sensor;
azure_client_thread.start(azure_task);
azure_client_thread.join();
platform_deinit();
printf(" - - - - - - - ALL DONE - - - - - - - \n");
return 0;
}
static void send_confirm_callback(IOTHUB_CLIENT_CONFIRMATION_RESULT result, void* userContextCallback)
{
//userContextCallback;
// When a message is sent this callback will get envoked
g_message_count_send_confirmations++;
deleteOK.set(0x1);
}
void sendMessage(IOTHUB_CLIENT_LL_HANDLE iotHubClientHandle, char* buffer, size_t size)
{
IOTHUB_MESSAGE_HANDLE messageHandle = IoTHubMessage_CreateFromByteArray((const unsigned char*)buffer, size);
if (messageHandle == NULL) {
printf("unable to create a new IoTHubMessage\r\n");
return;
}
if (IoTHubClient_LL_SendEventAsync(iotHubClientHandle, messageHandle, send_confirm_callback, NULL) != IOTHUB_CLIENT_OK)
printf("FAILED to send! [RSSI=%d]\n", platform_RSSI());
else
printf("OK. [RSSI=%d]\n",platform_RSSI());
IoTHubMessage_Destroy(messageHandle);
}
void azure_task(void)
{
//bool tilt_detection_enabled=true;
float gtemp, ghumid, gpress;
int k;
int msg_sent=1;
while (true) {
powerUp();
mems_init();
/* Setup IoTHub client configuration */
IOTHUB_CLIENT_LL_HANDLE iotHubClientHandle = IoTHubClient_LL_CreateFromConnectionString(connectionString, HTTP_Protocol);
if (iotHubClientHandle == NULL) {
printf("Failed on IoTHubClient_Create\r\n");
return;
}
// add the certificate information
if (IoTHubClient_LL_SetOption(iotHubClientHandle, "TrustedCerts", certificates) != IOTHUB_CLIENT_OK)
printf("failure to set option \"TrustedCerts\"\r\n");
#if MBED_CONF_APP_TELUSKIT == 1
if (IoTHubClient_LL_SetOption(iotHubClientHandle, "product_info", "TELUSIOTKIT") != IOTHUB_CLIENT_OK)
printf("failure to set option \"product_info\"\r\n");
#endif
// polls will happen effectively at ~10 seconds. The default value of minimumPollingTime is 25 minutes.
// For more information, see:
// https://azure.microsoft.com/documentation/articles/iot-hub-devguide/#messaging
unsigned int minimumPollingTime = 9;
if (IoTHubClient_LL_SetOption(iotHubClientHandle, "MinimumPollingTime", &minimumPollingTime) != IOTHUB_CLIENT_OK)
printf("failure to set option \"MinimumPollingTime\"\r\n");
IoTDevice* iotDev = (IoTDevice*)malloc(sizeof(IoTDevice));
if (iotDev == NULL) {
return;
}
setUpIotStruct(iotDev);
char* msg;
size_t msgSize;
hum_temp->get_temperature(&gtemp); // get Temp
hum_temp->get_humidity(&ghumid); // get Humidity
pressure->get_pressure(&gpress); // get pressure
iotDev->Temperature = CTOF(gtemp);
iotDev->Humidity = (int)ghumid;
iotDev->Pressure = (int)gpress;
printf("(%04d)",msg_sent++);
msg = makeMessage(iotDev);
msgSize = strlen(msg);
sendMessage(iotHubClientHandle, msg, msgSize);
free(msg);
iotDev->Tilt &= 0x2;
/* schedule IoTHubClient to send events/receive commands */
IOTHUB_CLIENT_STATUS status;
while ((IoTHubClient_LL_GetSendStatus(iotHubClientHandle, &status) == IOTHUB_CLIENT_OK) && (status == IOTHUB_CLIENT_SEND_STATUS_BUSY))
{
IoTHubClient_LL_DoWork(iotHubClientHandle);
ThisThread::sleep_for(100);
}
deleteOK.wait_all(0x1);
free(iotDev);
IoTHubClient_LL_Destroy(iotHubClientHandle);
powerDown();
ThisThread::sleep_for(300000);
}
return;
}
I know PSM is probably the way to go since powering on/off the device draws a lot of power but it would be useful if someone had an idea of what is happening here.
2) putting the device to PSM between sending messages
The BG96 library in the example code I'm using doesn't have a method to turn on PSM so I tried to implement my own. When I tried to run it, it basically runs into an exception right away so I know it's wrong (I'm very new to embedded development and have no prior experience with AT commands).
/** ----------------------------------------------------------
* this is a method provided by current library
* #brief Tx a string to the BG96 and wait for an OK response
* #param none
* #retval true if OK received, false otherwise
*/
bool BG96::tx2bg96(char* cmd) {
bool ok=false;
_bg96_mutex.lock();
ok=_parser.send(cmd) && _parser.recv("OK");
_bg96_mutex.unlock();
return ok;
}
/**
* method I created in an attempt to use PSM
*/
bool BG96::psm(void) {
return tx2bg96((char*)"AT+CPSMS=1,,,”00000100”,”00000001”");
}
Can someone tell me what I'm doing wrong and provide any guidance on how I can achieve my goal of having my device run on battery for longer?
Thank you!!
I got Power Saving Mode working by using Mbed's ATCmdParser and the AT+QPSMS commands as per Quectel's docs. The modem doesn't always go into power saving mode right away so that should be noted. I also found that I have to restart the modem afterwards or else I get weird behaviour. My code looks something like this:
bool BG96::psm(char* T3412, char* T3324) {
_bg96_mutex.lock();
if(_parser.send("AT+QPSMS=1,,,\"%s\",\"%s\"", T3412, T3324) && _parser.recv("OK")) {
_bg96_mutex.unlock();
}else {
_bg96_mutex.unlock();
return false;
}
return BG96Ready(); }//restarts modem
To send a message to Azure, the modem will need to be manually woken up by driving the PWRKEY to start bi-directional communication, and a new client handle needs to be created and torn down every time since Azure connection uses keepAlive and the modem will be unreachable when it's in PSM.

For the following race condition example, why an inconsistent result comes up only in some environments, but not in some others?

The following code is about the classical producer and consumer problem and there is a race condition here. When I run this example in MinGW in Windows 10, we have a very large probability (e.g., 80%) of having an inconsistent result. However, when I run it in Ubuntu, the probability is extremely low.
Besides, when I run a similar code in Java, the probability of having an inconsistent result is also very high.
So, could anyone explain what is different behind. I think it is because of different scheduling mechanisms and different time slice settings.
#include <stdio.h>
#include <pthread.h>
int count = 0;
void *producer(){
for(int i = 0;i<10000;i++)
count++;
}
void *consumer() {
for(int i = 0;i<10000;i++)
count--;
}
int main(int argc, const char *argv[]){
printf("initial count: %d\n",count);
pthread_t t1, t2;
pthread_create(&t1, NULL, producer,NULL);
pthread_create(&t2, NULL, consumer,NULL);
pthread_join(t1, NULL);
pthread_join(t2, NULL);
printf("final count: %d\n",count);
return 0;
}

pthread_mutex_t struct: What does lock stand for?

I am looking at the pthread_mutex_t structure in the pthreadtypes.h file. What does the "__lock" stand for? Is it like a lock number assigned to the mutex?
typedef union
{
struct __pthread_mutex_s
{
int __lock;
unsigned int __count;
int __owner;
#if __WORDSIZE == 64
unsigned int __nusers;
#endif
/* KIND must stay at this position in the structure to maintain
binary compatibility. */
int __kind;
#if __WORDSIZE == 64
int __spins;
__pthread_list_t __list;
# define __PTHREAD_MUTEX_HAVE_PREV 1
#else
unsigned int __nusers;
__extension__ union
{
int __spins;
__pthread_slist_t __list;
};
#endif
} __data;
char __size[__SIZEOF_PTHREAD_MUTEX_T];
long int __align;
} pthread_mutex_t;
The __lock member of struct __pthread_mutex_s __data is used as a futex object on Linux. Many of the following details may differ depending on the architecture you're looking at:
See the pthread_mutex_lock.c code for the high level locking function for pthread mutexes - __pthread_mutex_lock(), which generally will end up calling LLL_MUTEX_LOCK() and the definitions of LLL_MUTEX_LOCK() and friends, which end up calling lll_lock(), etc., in lowlevellock.h.
The lll_lock() macro in turn calls __lll_lock_wait_private(), which calls lll_futex_wait(), which makes the sys_futex system call.

How to write a Linux Driver, that only forwards file operations?

I need to implement a Linux Kernel Driver, that (in the first step) only forwards all file operations to another file (in later steps, this should be managed and manipulated, but I don't want to discuss this here).
My idea is the following, but when reading, the kernel crashes:
static struct {
struct file *file;
char *file_name;
int open;
} file_out_data = {
.file_name = "/any_file",
.open = 0,
};
int memory_open(struct inode *inode, struct file *filp) {
PRINTK("<1>open memory module\n");
/*
* We don't want to talk to two processes at the same time
*/
if (file_out_data.open)
return -EBUSY;
/*
* Initialize the message
*/
Message_Ptr = Message;
try_module_get(THIS_MODULE);
file_out_data.file = filp_open(file_out_data.file_name, filp->f_flags, filp->f_mode); //here should be another return handling in case of fail
file_out_data.open++;
/* Success */
return 0;
}
int memory_release(struct inode *inode, struct file *filp) {
PRINTK("<1>release memory module\n");
/*
* We're now ready for our next caller
*/
file_out_data.open--;
filp_close(file_out_data.file,NULL);
module_put(THIS_MODULE);
/* Success */
return 0;
}
ssize_t memory_read(struct file *filp, char *buf,
size_t count, loff_t *f_pos) {
PRINTK("<1>read memory module \n");
ret=file_out_data.file->f_op->read(file_out_data.file,buf,count,f_pos); //corrected one, false one is to find in the history
return ret;
}
So, can anyone please tell me why?
Don't use set_fs() as there is no reason to do it.
Use file->f_fop->read() instead of the vfs_read. Take a look at the file and file_operations structures.
Why are you incrementing file_out_data.open twice and decrementing it once? This could cause you to use file_out_data.file after it has been closed.
You want to write memory in your file ou read?
Because you are reading and not writing...
possible i'm wrong

Kcachegrind/callgrind is inaccurate for dispatcher functions?

I have a model code on which kcachegrind/callgrind reports strange results. It is kind of dispatcher function. The dispatcher is called from 4 places; each call says, which actual do_J function to run (so the first2 will call only do_1 and do_2 and so on)
Source (this is a model of actual code)
#define N 1000000
int a[N];
int do_1(int *a) { int i; for(i=0;i<N/4;i++) a[i]+=1; }
int do_2(int *a) { int i; for(i=0;i<N/2;i++) a[i]+=2; }
int do_3(int *a) { int i; for(i=0;i<N*3/4;i++) a[i]+=3; }
int do_4(int *a) { int i; for(i=0;i<N;i++) a[i]+=4; }
int dispatcher(int *a, int j) {
if(j==1) do_1(a);
else if(j==2) do_2(a);
else if(j==3) do_3(a);
else do_4(a);
}
int first2(int *a) { dispatcher(a,1); dispatcher(a,2); }
int last2(int *a) { dispatcher(a,4); dispatcher(a,3); }
int inner2(int *a) { dispatcher(a,2); dispatcher(a,3); }
int outer2(int *a) { dispatcher(a,1); dispatcher(a,4); }
int main(){
first2(a);
last2(a);
inner2(a);
outer2(a);
}
Compiled with gcc -O0; Callgrinded with valgrind --tool=callgrind; kcachegrinded with kcachegrind and qcachegrind-0.7.
Here is a full callgraph of the application. All paths to do_J go through dispatcher and this is good (the do_1 is just hided as too fast, but it is here really, just left to do_2)
Lets focus on do_1 and check, who called it (this picture is incorrect):
And this is very strange, I think, only first2 and outer2 called do_1 but not all.
Is it a limitation of callgrind/kcachegrind? How can I get accurate callgraph with weights (proportional to running time of every function, with and without its childs)?
Yes, this is limitation of callgrind format. It doesn't store full trace; it only stores parent-child calls information.
There is a google-perftools project with pprof/libprofiler.so CPU profiler, http://google-perftools.googlecode.com/svn/trunk/doc/cpuprofile.html . libprofiler.so can get profile with calltraces and it will store every trace event with full backtrace. pprof is converter of libprofile's output to graphic formats or to callgrind format. In full view the result will be the same as in kcachegrind; but if you will focus on some function, e.g. do_1 using pprof's option focus; it will show accurate calltree when focused on function.