Qt4 and MySQL5 - Binding QByteArray "1:1" as binary data - sql

I have some issues with character encoding of a binary value using Qt4 and MySQL5.
Let's say we want to bind a value containing the four bytes \xDE \xAD \xBE \xEF. I check the bound value using the MySQL function HEX() using this code:
#include <QtGui/QApplication>
#include <QDebug>
#include <QSqlQuery>
#include <QVariant>
#include <QSqlRecord>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL");
if(!db.open("test", "test"))
exit(1);
QSqlQuery q("SELECT HEX(?)");
q.addBindValue(QVariant(QByteArray::fromHex("DEADBEEF")));
if(!q.exec())
exit(1);
if(!q.next())
exit(1);
qDebug() << q.record().value(0).toString();
return a.exec();
}
The output of this code is "DEADEFBFBDEFBFBD" which is obviously the HEX code of \xDE \xAD \xBE \xEF interpreted as a latin1-encoded string and then encoded as a UTF8 string.
If I do not bind the value using addBindValue() but placing it directly into the query using UNHEX('DEADBEEF') results in the expected behaviour (which isn't surprising...).
Where does the UTF8 encoding step take place?
(Finally, I want to store a binary value "1:1" into a BLOB field.)
OS: Ubuntu 10.10 (32 bit)
Qt Version: 4.7.0 (Ubuntu package)
MySQL Version: 5.1.49-1ubuntu8.1
Thanks in advance!

After weeks of trial and error, the only solution I found is to transfer the binary data in hexadecimal code and UNHEX() it in the query.
This is a solution, so I allow myself to accept my own answer, but it isn't a nice solution and I have no explanation to the behavior of the code above.
So if you have any advice, I am looking forward for any further answer. (I will then accept your answer.) Thanks!

Related

DLL silently ignored even though the library links

I'm using a third part DLL that I've used successfully for ages. Now the linker links the dll lib without complaint but the exe doesn't load the dll.
I recently upgraded from the 32 bit to 64 bit cygwin.
I'm doing a mingw cross compile to 32 bits.
I'm trying to use the FTDI USB interface FTD2XX dll.
I have the version 2.04.06 FTD2XX lib, .h, and dll.
I had been using that dll successfully for ages but with older versions of cygwin and mingw.
Recently upgraded to cygwin64.
The app appears to link with the FTD2XX.lib without complaint.
But when I run the app it doesn't seem to look for or load the FTD2XX.dll.
The app runs but crashes as soon as it tries to call something in FTD2XX dll.
I created a simple hello_dll.dll for side by side test. That works.
The app.c does calls on both hello_dll.dll and ftd2xx.dll.
Is starts without complain, successfully calls function in hello_dll, and then it crashes on a call to ft2xx.dll.
(I renamed the lib to ftd2xx_2.04.06 to distinguish them from other versions I have. Newer versions don't work any better.)
Link with -verbose gives:
i686-w64-mingw32-gcc -Wall -m32 -g -O2 -c -I . -o app.o app.c
i686-w64-mingw32-gcc -Wall -m32 -o app.exe app.o -Wl,-verbose -L. -lhello_dll -lftd2xx_2.04.06
GNU ld (GNU Binutils) 2.34.50.20200227
Supported emulations:
i386pe
using internal linker script:
<snip>
/usr/lib/gcc/i686-w64-mingw32/9.2.0/../../../../i686-w64-mingw32/bin/ld: mode i386pe
attempt to open /usr/i686-w64-mingw32/sys-root/mingw/lib/../lib/crt2.o succeeded
/usr/i686-w64-mingw32/sys-root/mingw/lib/../lib/crt2.o
attempt to open /usr/lib/gcc/i686-w64-mingw32/9.2.0/crtbegin.o succeeded
/usr/lib/gcc/i686-w64-mingw32/9.2.0/crtbegin.o
attempt to open app.o succeeded
app.o
<snip>
attempt to open ./hello_dll.lib succeeded
./hello_dll.lib
(./hello_dll.lib)d000001.o
(./hello_dll.lib)d000000.o
(./hello_dll.lib)d000002.o
<snip>
attempt to open ./ftd2xx_2.04.06.lib succeeded
./ftd2xx_2.04.06.lib
(./ftd2xx_2.04.06.lib)FTD2XX.dll
(./ftd2xx_2.04.06.lib)FTD2XX.dll
(./ftd2xx_2.04.06.lib)FTD2XX.dll
(./ftd2xx_2.04.06.lib)FTD2XX.dll
::::::::::::::::::::::::::::
I obtained a 32 bit compatible version of gdb. When I run gdb:
GNU gdb (GDB) 7.7.50.20140303-cvs
<snip>
This GDB was configured as "i686-pc-mingw32".
<snip>
(gdb) break main
(gdb) Breakpoint 1 at 0x40267b: file app.c, line 28.
(gdb) run
(gdb) Starting program: C:\_d\aaa\pd\src\dll\pathological\app.exe
[New Thread 1428.0x2528]
Breakpoint 1, main (argc=1, argv=0x9b2f70) at app.c:28
28 dostuff();
(gdb) info share
(gdb) From To Syms Read Shared Object Library
0x774e0000 0x77644ccc Yes (*) C:\Windows\SysWOW64\ntdll.dll
0x753d0000 0x754cadec Yes (*) C:\Windows\syswow64\kernel32.dll
0x75ea1000 0x75ee6a3a Yes (*) C:\Windows\syswow64\KernelBase.dll
0x64081000 0x6408a1d8 Yes C:\_d\aaa\pd\src\dll\pathological\hello_dll.dll
0x75041000 0x750eb2c4 Yes (*) C:\Windows\syswow64\msvcrt.dll
(*): Shared library is missing debugging information.
(gdb) A debugging session is active.
(gdb) c
Continuing.
Hello dll. <--- The function in hello_dll.dll prints this.
Program received signal SIGSEGV, Segmentation fault.
0x8000004c in ?? () <----- call to FT_GetLibraryVersion()
(gdb) bt
#0 0x8000004c in ?? ()
#1 0x0040158e in dostuff () at app.c:49
#2 0x00402680 in main (argc=1, argv=0x8e2f70) at app.c:28
(gdb)
It links with the lib without complaint but when I run the exe it (silently) doesn't load the dll.
Anybody have any ideas? Is there some linker control that I am missing? Are there other diagnostic or debug tools to dig into this further?
:::::::::::::::::::::::
edit 7/11/20
I'll post some code. (If I know how. I'm new here.)
It should be shown in the "info share", but it isn't, as you can see above.
I'm suspecting name decoration. Objdump -x of the .exe shows an entry for FTD2XX.dll in the Import Tables. But it doesn't show any vma or bound name under it. I suspect that at program load the loader sees no vma/name and decides it doesn't really need to load the dll.
There is an import table in .idata at 0x406000
<snip>
The Import Tables (interpreted .idata section contents)
vma: Hint Time Forward DLL First
Table Stamp Chain Name Thunk
00006000 0000607c 00000000 00000000 00006218 0000614c
DLL Name: FTD2XX.dll
vma: Hint/Ord Member-Name Bound-To
<----- empty?
00006014 00006080 00000000 00000000 000064f8 00006150
DLL Name: hello_dll.dll
vma: Hint/Ord Member-Name Bound-To
6224 1 hello_dll
00006028 00006088 00000000 00000000 00006554 00006158
DLL Name: KERNEL32.dll
vma: Hint/Ord Member-Name Bound-To
6230 277 DeleteCriticalSection
6248 310 EnterCriticalSection
<snip>
:::::::::::::::::::::::::::::::::::::::::::::
edit 2, 7/11/20
This is the program that calls functions in the DLLs.
/* app.c
Demonstrates using the function imported from the DLL.
*/
// 200708 pathological case. Based on the simple hello_dll.
//#include <stdlib.h>
// for sleep
#include <unistd.h>
#include <stdio.h>
// for dword
#include <windef.h>
// for lpoverlapped
#include <minwinbase.h>
#include "hello_dll.h"
// My legacy app, and really all others too, use 2.04.06.h
#include "ftd2xx_2.04.06.h"
//#include "ftd2xx_2.02.04.h"
///////////////////////////
void dostuff( void );
void call_ft_listdevices( void );
///////////////////////////
int main(int argc, char** argv)
{
FT_STATUS status;
DWORD libver;
//dostuff();
printf( "Calling hello_dll():\n" );
fflush( stdout );
hello_dll();
fflush( stdout );
printf( "Back from hello_dll()\n" );
fflush( stdout );
sleep( 1 );
printf( "Calling FT_GetLibraryVersion().\n" );
fflush( stdout );
status = FT_GetLibraryVersion( &libver );
if( status == FT_OK ){
printf( "FTD2XX library version 0x%lx\n", libver );
fflush( stdout );
}
else{
printf( "Error reading FTD2XX library version.\n" );
fflush( stdout );
}
// 200710 Adding call to different ft function did
// not result in entries in the import table.
//call_ft_listdevices( );
return 0;
}
I don't think there is a need to include the code for my hello_dll. It works.
I have three versions of the FTD2XX. I'm pretty careful about tracking versions. Plus, when one is beating one's head against the wall, double checking the versions appeals early on as a way to end the pain.
I found a surprise copy of FTD2XX.dll. It's in c:/Windows/SysWOW64. It is the oldest of the three versions I have. Versions of my app that were compiled before this problem started run correctly using that dll in that place.
Solved.
There's a bug in the 2.34.50.20200227 i686-w64-mingw32-ld.exe. It won't work with ftd2xx.lib, regardless of ftd2xx version as far as I can tell.
2.25.51.20150320 and 2.29.1.20171006 work with ftd2xx.lib. I've reverted back to 2.29 mingw64-i686-binutils. I'm running again.

Generating .gcda coverage files via QEMU/GDB

Executive summary: I want to use GDB to extract the coverage execution counts stored in memory in my embedded target, and use them to create .gcda files (for feeding to gcov/lcov).
The setup:
I can successfully cross-compile my binary, targeting my specific embedded target - and then execute it under QEMU.
I can also use QEMU's GDB support to debug the binary (i.e. use tar extended-remote localhost:... to attach to the running QEMU GDB server, and fully control the execution of my binary).
Coverage:
Now, to perform "on-target" coverage analysis, I cross-compile with
-fprofile-arcs -ftest-coverage. GCC then emits 64-bit counters to keep track of execution counts of specific code blocks.
Under normal (i.e. host-based, not cross-compiled) execution, when the app finishes __gcov_exit is called - and gathers all these execution counts into .gcdafiles (that gcov then uses to report coverage details).
In my embedded target however, there's no filesystem to speak of - and libgcov basically contains empty stubs for all __gcov_... functions.
Workaround via QEMU/GDB: To address this, and do it in a GCC-version-agnostic way, I could list the coverage-related symbols in my binary via MYPLATFORM-readelf, and grep-out the relevant ones (e.g. __gcov0.Task1_EntryPoint, __gcov0.worker, etc):
$ MYPLATFORM-readelf -s binary | grep __gcov
...
46: 40021498 48 OBJECT LOCAL DEFAULT 4 __gcov0.Task1_EntryPoint
...
I could then use the offsets/sizes reported to automatically create a GDB script - a script that extracts the counters' data via simple memory dumps (from offset, dump length bytes to a local file).
What I don't know (and failed to find any relevant info/tool), is how to convert the resulting pairs of (memory offset,memory data) into .gcda files. If such a tool/script exists, I'd have a portable (platform-agnostic) way to do coverage on any QEMU-supported platform.
Is there such a tool/script?
Any suggestions/pointers would be most appreciated.
UPDATE: I solved this myself, as you can read below - and wrote a blog post about it.
Turned out there was a (much) better way to do what I wanted.
The Linux kernel includes portable GCOV related functionality, that abstracts away the GCC version-specific details by providing this endpoint:
size_t convert_to_gcda(char *buffer, struct gcov_info *info)
So basically, I was able to do on-target coverage via the following steps:
Step 1
I added three slightly modified versions of the linux gcov files to my project: base.c, gcc_4_7.c and gcov.h. I had to replace some linux-isms inside them - like vmalloc,kfree, etc - to make the code portable (and thus, compileable on my embedded platform, which has nothing to do with Linux).
Step 2
I then provided my own __gcov_init...
typedef struct tagGcovInfo {
struct gcov_info *info;
struct tagGcovInfo *next;
} GcovInfo;
GcovInfo *headGcov = NULL;
void __gcov_init(struct gcov_info *info)
{
printf(
"__gcov_init called for %s!\n",
gcov_info_filename(info));
fflush(stdout);
GcovInfo *newHead = malloc(sizeof(GcovInfo));
if (!newHead) {
puts("Out of memory!");
exit(1);
}
newHead->info = info;
newHead->next = headGcov;
headGcov = newHead;
}
...and __gcov_exit:
void __gcov_exit()
{
GcovInfo *tmp = headGcov;
while(tmp) {
char *buffer;
int bytesNeeded = convert_to_gcda(NULL, tmp->info);
buffer = malloc(bytesNeeded);
if (!buffer) {
puts("Out of memory!");
exit(1);
}
convert_to_gcda(buffer, tmp->info);
printf("Emitting %6d bytes for %s\n", bytesNeeded, gcov_info_filename(tmp->info));
free(buffer);
tmp = tmp->next;
}
}
Step 3
Finally, I scripted my GDB (driving QEMU remotely) via this:
$ cat coverage.gdb
tar extended-remote :9976
file bin.debug/fputest
b base.c:88 <================= This breaks on the "Emitting" printf in __gcov_exit
commands 1
silent
set $filename = tmp->info->filename
set $dataBegin = buffer
set $dataEnd = buffer + bytesNeeded
eval "dump binary memory %s 0x%lx 0x%lx", $filename, $dataBegin, $dataEnd
c
end
c
quit
And finally, executed both QEMU and GDB - like this:
$ # In terminal 1:
qemu-system-MYPLATFORM ... -kernel bin.debug/fputest -gdb tcp::9976 -S
$ # In terminal 2:
MYPLATFORM-gdb -x coverage.gdb
...and that's it - I was able to generate the .gcda files in my local filesystem, and then see coverage results over gcov and lcov.
UPDATE: I wrote a blog post showing the process in detail.

How to access debug information in a running application

I was wondering if it is possible to access debug information in a running application that has been compiled with /DEBUG (Pascal and/or C), in order to retrieve information about structures used in the application.
The application can always ask the debugger to do something using SS$_DEBUG. If you send a list of commands that end with GO then the application will continue running after the debugger does its thing. I've used it to dump a bunch of structures formatted neatly without bothering to write the code.
ANALYZE/IMAGE can be used to examine the debugger data in the image file without running the application.
Although you may not see the nice debugger information, you can always look into a running program's data with ANALYZE/SYSTEM .. SET PROCESS ... EXAMINE ....
The SDA SEARCH command may come in handy to 'find' recognizable morcels of date, like a record that you know the program must have read.
Also check out FORMAT/TYPE=block-type, but to make use of data you'll have to compile your structures into .STB files.
When using SDA, you may want to try run the program yourself interactively in an other session to get sample sample addresses to work from.... easier than a link map!
If you programs use RMS a bunch (mine always do :-), then SDA> SHOW PROC/RMS=(FAB,RAB) may give handy addresses for record and key buffers, allthough those may also we managed by the RTL's and thus not be meaningful to you.
Too long for a comment ...
As far as I know, structure information about elements is not in the global symbol table.
What I did, on Linux, but that should work on VMS/ELF files as well:
$ cat tests.c
struct {
int ii;
short ss;
float ff;
char cc;
double dd;
char bb:1;
void *pp;
} theStruct;
...
$ cc -g -c tests.c
$ ../extruct/extruct
-e-insarg, supply an ELF object file.
Usage: ../extruct/extruct [OPTION]... elf-file variable
Display offset and size of members of the named struct/union variable
extracted from the dwarf info in the elf file.
Options are:
-b bit offsets and bit sizes for all members
-lLEVEL display level for nested structures
-n only the member names
-t print base types
$ ../extruct/extruct -t ./tests.o theStruct
size of theStruct: 0x20
offset size type name
0x0000 0x0004 int ii
0x0004 0x0002 short int ss
0x0008 0x0004 float ff
0x000c 0x0001 char cc
0x0010 0x0008 double dd
0x0018 0x0001 char bb:1
0x001c 0x0004 pp
$

How to get full path (including mount directory) for file path?

I'm working on a plugin for Coda, and I'm running into a strange problem when trying to compare file paths to a sqlite row.
Basically, NSOpenPanel returns an NSURL which returns a path like this:
/Users/michael/Documents/xcode stuff/hM github/Plugin/filename
And Coda returns the current file path as this:
/Volumes/Macintosh HD/Users/michael/Documents/xcode stuff/hM github/Plugin/filename
I need to be able to retrieve a sqlite row tied to this file (which was initially created when the user selects a file through NSOpenPanel) given the path from Coda.
Is there a way to get a file's actual full path, including Volume information?
Alternatively, is there a better way to store a reference to a file that could be easily retrievable, regardless of the given path?
update
I've realized that Coda modifies saved files in such a way that you cannot rely on a file's inode number to remain consistent, so trojanfoe's answer will not work for me.
Looking at the following:
miranda-6:~ jeremyp$ ls -l /Volumes
total 8
drwxr-xr-x 1 jeremyp staff 8192 23 Oct 06:54 BOOTCAMP
drwxrwxr-x# 20 jeremyp staff 748 23 Oct 11:22 G-DRIVE
lrwxr-xr-x 1 root admin 1 28 Oct 07:25 Macintosh HD -> /
drwxrwxrwx 0 root wheel 0 1 Nov 16:28 MobileBackups
miranda-6:~ jeremyp$
/Volumes/Macintosh HD is a symbolic link to /.
So what you should do (IMO), is iterate through the items in /Volumes until you find the one that is a symbolic link to / Then whenever you get a file path from Coda that starts with /Volumes/<the item linked to / >, you strip that off before doing a search for it in your database.
You can use NSFileManager attributesOfItemAtPath:error: to find out if you have a symlink and NSFileManager -destinationOfSymbolicLinkAtPath:error: to find out where the symlink goes.
A better way to uniquely identify a file, regardless of how it's specified (absolute, relative, or including the mount) is to use the file's device and inode number which are unique across the system:
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/stat.h>
static void get_file_details(const char *filename) {
struct stat statbuf;
if (stat(filename, &statbuf) == 0) {
printf("%s = %016llx%016llx\n", filename, (uint64_t)statbuf.st_dev, (uint64_t)statbuf.st_ino);
} else {
fprintf(stderr, "Failed to stat '%s': %s\n", filename, strerror(errno));
}
}
int main(int argc, const char **argv) {
for (int i = 1; i < argc; i++)
get_file_details(argv[i]);
return 0;
}
$ clang -o stattest stattest.c
$ ./stattest stat*
stattest = 00000000010000010000000001b1d48a
stattest.c = 00000000010000010000000001b1d43b
You'll need to decide how you format the device/inode combo, but the above format should work fine for most cases.

Sending signals from DCL command line on OpenVMS

I'm trying to send a signal via the command line on an OpenVMS server. Using Perl I have set up signal handlers between processes and Perl on VMS is able to send Posix signals. In addition, C++ programs are able to send and handle signals too. However, the problem I run into is that the processes could be running on another node in the cluster and I need to write a utility script to remotely send a signal to them.
I'm trying to avoid writing a new script and would rather simply execute a command remotely to send the signal from the command line. I need to send SIGUSR1, which translates to C$_SIGUSR1 for OpenVMS.
Thanks.
As far as I know, there is no supported command line interface to do this. But you can accomplish the task by calling an undocumented system service called SYS$SIGPRC(). This system service can deliver any condition value to the target process, not just POSIX signals. Here's the interface described in standard format:
FORMAT
SYS$SIGPRC process-id ,[process-name] ,condition-code
RETURNS
OpenVMS usage: cond_value
type: longword (unsigned)
access: write only
mechanism: by value
ARGUMENTS
process-id
OpenVMS usage: process_id
type: longword (unsigned)
access: modify
mechanism: by reference
Process identifier of the process for which is to receive the signal. The
process-id argument is the address of an unsigned longword containing the
process identifier. If you do not specify process-id, process-name is
used.
The process-id is updated to contain the process identifier actually
used, which may be different from what you originally requested if you
specified process-name.
process-name
OpenVMS usage: process_name
type: character string
access: read only
mechanism: by descriptor
A 1 to 15 character string specifying the name of the process for
which will receive the signal. The process-name argument is the
address of a descriptor pointing to the process name string. The name
must correspond exactly to the name of the process that is to receive
the signal; SYS$SIGPRC does not allow trailing blanks or abbreviations.
If you do not specify process-name, process-id is used. If you specify
neither process-name nor process-id, the caller's process is used.
Also, if you do not specify process-name and you specify zero for
process-id, the caller's process is used.
condition-value
OpenVMS usage: cond_value
type: longword (unsigned)
access: read only
mechanism: by value
OpenVMS 32-bit condition value. The condition-value argument is
an unsigned longword that contains the condition value delivered
to the process as a signal.
CONDITION VALUES RETURNED
SS$_NORMAL The service completed successfully
SS$_NONEXPR Specified process does not exist
SS$_NOPRIV The process does not have the privilege to signal
the specified process
SS$_IVLOGNAM The process name string has a length of 0 or has
more than 15 characters
(plus I suspect there are other possible returns having to do
with various cluster communications issues)
EXAMPLE CODE
#include <stdio.h>
#include <stdlib.h>
#include <ssdef.h>
#include <stsdef.h>
#include <descrip.h>
#include <errnodef.h>
#include <lib$routines.h>
int main (int argc, char *argv[]) {
/*
**
** To build:
**
** $ cc sigusr1
** $ link sigusr1
**
** Run example:
**
** $ sigusr1 := $dev:[dir]sigusr1.exe
** $ sigusr1 20206E53
**
*/
static unsigned int pid;
static unsigned int r0_status;
extern unsigned int sys$sigprc (unsigned int *,
struct dsc$descriptor_s *,
int);
if (argc < 2) {
(void)fprintf (stderr, "Usage: %s PID\n",
argv[0]);
exit (EXIT_SUCCESS);
}
sscanf (argv[1], "%x", &pid);
r0_status = sys$sigprc (&pid, 0, C$_SIGUSR1);
if (!$VMS_STATUS_SUCCESS (r0_status)) {
(void)lib$signal (r0_status);
}
}