we have couple of C applications which talk using the shared memory. first application adds the message and the second one always reads from the shared memory.
struct messagestruct {
unsigned int sessionid;
uint8_t message[16]; //unsigned 8bit int
}__attribute__ ((__packed__));
we need to have 3 sessions (for 3 users). so defined shared memory size as
#define SHARED_SIZE ( 3 * sizeof(messagestruct)) + sizeof(int)
we access the shared memory as
int sesskey = ftok("/path/to/a/file", "B");
int shmemoryid = shmget(sesskey, SHARED_SIZE, 0666 | IPC_CREAT);
during copying structs to shared memory, valgrind reports error (invalid write size 1)
void *shmaddr = shmat(shmemoryid, NULL, 0);
int *sessnum;
struct messagestruct *msgstruct;
sessnum = (int *)shmaddr;
msgstruct = (struct messagestruct*)((void*) shmaddr + sizeof(int));
memcpy(shmaddr, currentsessionsstruct, SHARED_SIZE); //-->valgrind error invalid write size 1
thanks for any helpful info.
memcpy(shmaddr, currentsessionsstruct, SHARED_SIZE);
You are copying 3 * sizeof(messagestruct) + sizeof(int). I believe you want to copy only sizeof(currentsessionstruct).
Related
I am trying communicating vector's in between RTP's all is well but i am getting memPartFree error!!
#include <iostream>
#include <taskLib.h>
#include <cstdlib>
#include <vector>
#include <msgQLib.h>
using namespace std;
#define TEN_BYTES 10
#define HUNDERED_BYTES 100
#define THOUSAND_BYTES 1000
#define SIZE_OF_EACH_MESSAGE 10
#define ONE_MESSAGE 1
#define TEN_MESSAGE 10
#define HUNDERED_MESSAGE 100
#define WAIT_FOR_EVER -1
void sender();
void receiver();
struct test
{
short int num1;
short int num2;
short int num3;
short int num4;
short int num5;
};
MSG_Q_ID MsgQ_ID;
int main()
{
cout<<__FUNCTION__<<endl;
MsgQ_ID = msgQCreate(ONE_MESSAGE,SIZE_OF_EACH_MESSAGE,MSG_Q_FIFO);
taskSpawn("receiver",150,VX_FP_TASK,10000,FUNCPTR (receiver),0,0,0,0,0,0,0,0,0,0);
taskSpawn("sender",150,VX_FP_TASK,10000,FUNCPTR (sender),0,0,0,0,0,0,0,0,0,0);
cout<<"wait here"<<endl;
}
void sender()
{
cout<<__FUNCTION__<<endl;
vector<test> vec;
test Obj;
while((vec.size() * SIZE_OF_EACH_MESSAGE) != TEN_BYTES)
{
Obj.num1 = rand();
Obj.num2 = rand();
Obj.num3 = rand();
Obj.num4 = rand();
Obj.num5 = rand();
vec.push_back(Obj);
}
cout<<"Size of vector to be sent "<<vec.size()<<endl;
vector<test>::iterator it;
for(it = vec.begin();it!=vec.end();it++)
{
cout<<"Send Data:"<<endl;
cout<<it->num1<<"\t"<<it->num2<<"\t"<<it->num3<<"\t"<<it->num4<<"\t"<<it->num5<<endl;
}
int MsgQStatus = msgQSend(MsgQ_ID,(char*)&vec,(SIZE_OF_EACH_MESSAGE * ONE_MESSAGE),WAIT_FOR_EVER,MSG_PRI_NORMAL);
cout<<"Status of MsgQ Send:"<<MsgQStatus<<endl;
}
void receiver()
{
cout<<__FUNCTION__<<endl;
vector<test> vec;
vec.reserve(ONE_MESSAGE);// to initialize the vector otherwise it's size will become random
int MsgQStatus = msgQReceive(MsgQ_ID,(char*)&vec,(SIZE_OF_EACH_MESSAGE * ONE_MESSAGE),WAIT_FOR_EVER);
cout<<"Status of MsgQ Receive:"<<MsgQStatus<<endl;
vector<test>::iterator it;
cout<<"size of the received vector"<<vec.size()<<endl;
for(it = vec.begin();it!=vec.end();it++)
{
cout<<"Received data:"<<endl;
cout<<it->num1<<"\t"<<it->num2<<"\t"<<it->num3<<"\t"<<it->num4<<"\t"<<it->num5<<endl;
}
}
and here is the output :
main
receiver
sender
Size of vector to be sent 1
Send Data:
7403 -19371 19159 -10975 -24349
Status of MsgQ Send:0
Status of MsgQ Receive:10
size of the received vector1
Received data:
7403 -19371 19159 -10975 -24349
memPartFree: invalid block 0xff873850 in partition 0xff8593a0
i have no idea why i am getting memPartFree error and due to this my RTP is getting stopped!! HELP!!
My guess is that since you are sending 10 bytes of vector<test> vec that you are sending internals of the vector class.
Although it looks like you're sending your struct test I assume that you're sending an internal pointer to the first element of the vector instead. Your sender frees the memory after it successfully sent the messages causing the receiver to try to free already freed memory after the message was received (both sender and receiver work on the same memory).
This can be tested by putting a while (1) taskDelay(1); at the end of either the sender or receiver task before leaving the function. This inhibits one of them to free the memory. Then the memPartFree error shouldn't be appearing if I'm correct (since the memroy is not freed twice).
To solve this do the following:
Change #define SIZE_OF_EACH_MESSAGE 10 to #define SIZE_OF_EACH_MESSAGE sizeof(struct test) to avoid trouble if the size of your struct changes in the future...
Put the data of each element struct test into the message queue instead of 10 bytes starting at &vec (e.g. &(vec[0])) so your msgQSend call should look something like this: int MsgQStatus = msgQSend(MsgQ_ID, (char*)&(vec[0]),(SIZE_OF_EACH_MESSAGE * ONE_MESSAGE), WAIT_FOR_EVER, MSG_PRI_NORMAL);.
Receive the data using a temporary struct test Obj; variable int MsgQStatus = msgQReceive(MsgQ_ID, (char*)&Obj, (SIZE_OF_EACH_MESSAGE * ONE_MESSAGE), WAIT_FOR_EVER);.
After you have successfully received the data push it to the end of your receiver vector (vec.push_back(Obj);).
I have a trivial program:
int main(void)
{
const char sname[]="xxx";
sem_t *pSemaphor;
if ((pSemaphor = sem_open(sname, O_CREAT, 0644, 0)) == SEM_FAILED) {
perror("semaphore initilization");
exit(1);
}
sem_unlink(sname);
sem_close(pSemaphor);
}
When I run it under valgrind, I get the following error:
==12702== Syscall param write(buf) points to uninitialised byte(s)
==12702== at 0x4E457A0: __write_nocancel (syscall-template.S:81)
==12702== by 0x4E446FC: sem_open (sem_open.c:245)
==12702== by 0x4007D0: main (test.cpp:15)
==12702== Address 0xfff00023c is on thread 1's stack
==12702== in frame #1, created by sem_open (sem_open.c:139)
The code was extracted from a bigger project where it ran successfully for years, but now it is causing segmentation fault.
The valgrind error from my example is the same as seen in the bigger project, but there it causes a crash, which my small example doesn't.
I see this with glibc 2.27-5 on Debian. In my case I only open the semaphores right at the start of a long-running program and it seems harmless so far - just annoying.
Looking at the code for sem_open.c which is available at:
https://code.woboq.org/userspace/glibc/nptl/sem_open.c.html
It seems that valgrind is complaining about the line (270 as I look now):
if (TEMP_FAILURE_RETRY (__libc_write (fd, &sem.initsem, sizeof (sem_t)))
== sizeof (sem_t)
However sem.initsem is properly initialised earlier in a fairly baroque manner, firstly by explicitly setting fields in the sem.newsem (part of the union), and then once that is done by a call to memset (L226-228):
/* Initialize the remaining bytes as well. */
memset ((char *) &sem.initsem + sizeof (struct new_sem), '\0',
sizeof (sem_t) - sizeof (struct new_sem));
I think that this particular shenanigans is all quite optimal, but we need to make sure that all of the fields of new_sem have actually been initialised... we find the definition in https://code.woboq.org/userspace/glibc/sysdeps/nptl/internaltypes.h.html and it is this wonderful creation:
struct new_sem
{
#if __HAVE_64B_ATOMICS
/* The data field holds both value (in the least-significant 32 bytes) and
nwaiters. */
# if __BYTE_ORDER == __LITTLE_ENDIAN
# define SEM_VALUE_OFFSET 0
# elif __BYTE_ORDER == __BIG_ENDIAN
# define SEM_VALUE_OFFSET 1
# else
# error Unsupported byte order.
# endif
# define SEM_NWAITERS_SHIFT 32
# define SEM_VALUE_MASK (~(unsigned int)0)
uint64_t data;
int private;
int pad;
#else
# define SEM_VALUE_SHIFT 1
# define SEM_NWAITERS_MASK ((unsigned int)1)
unsigned int value;
int private;
int pad;
unsigned int nwaiters;
#endif
};
So if we __HAVE_64B_ATOMICS then the structure has a data field which contains both the value and the nwaiters, otherwise these are separate fields.
In the initialisation of sem.newsem we can see that these are initialised correctly, as follows:
#if __HAVE_64B_ATOMICS
sem.newsem.data = value;
#else
sem.newsem.value = value << SEM_VALUE_SHIFT;
sem.newsem.nwaiters = 0;
#endif
/* pad is used as a mutex on pre-v9 sparc and ignored otherwise. */
sem.newsem.pad = 0;
/* This always is a shared semaphore. */
sem.newsem.private = FUTEX_SHARED;
I'm doing all of this on a 64-bit system, so I think that valgrind is complaining about the initialisation of the 64-bit sem.newsem.data with a 32-bit value since from:
value = va_arg (ap, unsigned int);
We can see that value is defined simply as an unsigned int which will usually still be 32 bits even on a 64-bit system (see What should be the sizeof(int) on a 64-bit machine?), but that should just be an implicit cast to 64-bits when it is assigned.
So I think this is not a bug - just valgrind getting confused.
The code below attempts to save a data stream to a file using fwrite. The first example using malloc works but with the second example the data stream is %70 corrupted. Can someone explain to me why the second example is corrupted and how I can remedy it?
short int fwBuffer[1000000];
// short int *fwBuffer[1000000];
unsigned long fwSize[1000000];
// Not Working *********
if (dataFlow) {
size = sizeof(short int)*length*inchannels;
short int tmpbuffer[length*inchannels];
int count = 0;
for (count = 0; count < length*inchannels; count++)
{
tmpbuffer[count] = (short int) (inbuffer[count]);
}
memcpy(&fwBuffer[saveBufferCount], tmpbuffer, sizeof(tmpbuffer));
fwSize[saveBufferCount] = size;
saveBufferCount++;
totalSize += size;
}
// Working ***********
if (dataFlow) {
size = sizeof(short int)*length*inchannels;
short int *tmpbuffer = (short int*)malloc(size);
int count = 0;
for (count = 0; count < length*inchannels; count++)
{
tmpbuffer[count] = (short int) (inbuffer[count]);
}
fwBuffer[saveBufferCount] = tmpbuffer;
fwSize[saveBufferCount] = size;
saveBufferCount++;
totalSize += size;
}
// Write to file ***********
for (int i = 0; i < saveBufferCount; i++) {
if (isRecording && outFile != NULL) {
// fwrite(fwBuffer[i], 1, fwSize[i],outFile);
fwrite(&fwBuffer[i], 1, fwSize[i],outFile);
if (fwBuffer[i] != NULL) {
// free(fwBuffer[i]);
}
fwBuffer[i] = NULL;
}
}
You initialize your size as
size = sizeof(short int) * length * inchannels;
then you declare an array of size
short int tmpbuffer[size];
This is already highly suspect. Why did you include sizeof(short int) into the size and then declare an array of short int elements with that size? The byte size of your array in this case is
sizeof(short int) * sizeof(short int) * length * inchannels
i.e. the sizeof(short int) is factored in twice.
Later you initialize only length * inchannels elements of the array, which is not entire array, for the reasons described above. But the memcpy that follows still copies the entire array
memcpy(&fwBuffer[saveBufferCount], &tmpbuffer, sizeof (tmpbuffer));
(Tail portion of the copied data is garbage). I'd suspect that you are copying sizeof(short int) times more data than was intended. The recipient memory overflows and gets corrupted.
The version based on malloc does not suffer from this problem since malloc-ed memory size is specified in bytes, not in short int-s.
If you want to simulate the malloc behavior in the upper version of the code, you need to declare your tmpbuffer as an array of char elements, not of short int elements.
This has very good chances to crash
short int tmpbuffer[(short int)(size)];
first size could be too big, but then truncating it and having whatever size results of that is probably not what you want.
Edit: Try to write the whole code without a single cast. Only then the compiler has a chance to tell you if there is something wrong.
I have a 2D matrix in the main. I want to transfer if from host to device. Can you tell me how I can allocate memory for it and transfer it to the device memory?
#define N 5
__global__ void kernel(int a[N][N]){
}
int main(void){
int a[N][N];
cudaMalloc(?);
cudaMemcpy(?);
kernel<<<N,N>>>(?);
}
Perhaps something like this is what you really had in mind:
#define N 5
__global__ void kernel(int *a)
{
// Thread indexing within Grid - note these are
// in column major order.
int tidx = threadIdx.x + blockIdx.x * blockDim.x;
int tidy = threadIdx.y + blockIdx.y * blockDim.y;
// a_ij = a[i][j], where a is in row major order
int a_ij = a[tidy + tidx*N];
}
int main(void)
{
int a[N][N], *a_device;
const size_t a_size = sizeof(int) * size_t(N*N);
cudaMalloc((void **)&a_device, a_size);
cudaMemcpy(a_device, a, a_size, cudaMemcpyHostToDevice);
kernel<<<N,N>>>(a_device);
}
The point you might have missed is that when you statically declare an array like this A[N][N], it is really just a row major ordered piece of linear memory. The compiler is automatically converting between a[i][j] and a[j + i*N] when it emits code. On the GPU, you must use the second form of access to read the memory you copy from the host.
I have the following questions regarding BLOBs in sqlite:
Does sqlite keep track of sizes of BLOBs?
I'm guessing that it does, but then, does the length function use it, or does it read the BLOB's content?
If sqlite keeps track of the size of the BLOB and length doesn't use it, is the size accessible via some other functionality?
I'm asking this because I'm wondering if I should implement triggers that set BLOBs' sizes in additional columns, of if I can obtain the sizes dynamically without the performance hit of sqlite reading the BLOBs.
From the source:
** In an SQLite index record, the serial type is stored directly before
** the blob of data that it corresponds to. In a table record, all serial
** types are stored at the start of the record, and the blobs of data at
** the end. Hence these functions allow the caller to handle the
** serial-type and data blob seperately.
**
** The following table describes the various storage classes for data:
**
** serial type bytes of data type
** -------------- --------------- ---------------
** 0 0 NULL
** 1 1 signed integer
** 2 2 signed integer
** 3 3 signed integer
** 4 4 signed integer
** 5 6 signed integer
** 6 8 signed integer
** 7 8 IEEE float
** 8 0 Integer constant 0
** 9 0 Integer constant 1
** 10,11 reserved for expansion
** N>=12 and even (N-12)/2 BLOB
** N>=13 and odd (N-13)/2 text
In other words, the blob size is in the serial, and it's length is simply "(serial_type-12)/2".
This serial is stored before the actual blob, so you don't need to read the blob to get its size.
Call sqlite3_blob_open and then sqlite3_blob_bytes to get this value.
Write a 1byte and a 10GB blob in a test database. If length() takes the same time for both blobs, the blob's length is probably accessed. Otherwise the blob is probably read.
OR: download the source code and debug through it: http://www.sqlite.org/download.html. These are some relevant bits:
/*
** Implementation of the length() function
*/
static void lengthFunc(
sqlite3_context *context,
int argc,
sqlite3_value **argv
){
int len;
assert( argc==1 );
UNUSED_PARAMETER(argc);
switch( sqlite3_value_type(argv[0]) ){
case SQLITE_BLOB:
case SQLITE_INTEGER:
case SQLITE_FLOAT: {
sqlite3_result_int(context, sqlite3_value_bytes(argv[0]));
break;
}
case SQLITE_TEXT: {
const unsigned char *z = sqlite3_value_text(argv[0]);
if( z==0 ) return;
len = 0;
while( *z ){
len++;
SQLITE_SKIP_UTF8(z);
}
sqlite3_result_int(context, len);
break;
}
default: {
sqlite3_result_null(context);
break;
}
}
}
and then
/*
** Return the number of bytes in the sqlite3_value object assuming
** that it uses the encoding "enc"
*/
SQLITE_PRIVATE int sqlite3ValueBytes(sqlite3_value *pVal, u8 enc){
Mem *p = (Mem*)pVal;
if( (p->flags & MEM_Blob)!=0 || sqlite3ValueText(pVal, enc) ){
if( p->flags & MEM_Zero ){
return p->n + p->u.nZero;
}else{
return p->n;
}
}
return 0;
}
You can see that the length of text data is calculated on the fly. That of blobs... well, I'm not fluent enough in C... :-)
If you have access to the raw c api sqlite3_blob_bytes will do the job for you. If not please provide additional information.