TensorFlow - how to serialize and send Status to remote - tensorflow

I am writing a distributed rendezvous implementation.
Requester - requests a tensor.
Responder - looks the tensor up and sends it to the requester.
If an error occurs on the responder side, I would like to send the error code to the requester which will propagate it to upper levels.
I saw that on GRPC the Status is converted to a ::grpc::status. Is that enough for Status serialization (i.e. can I simply mem-copy the ::grpc::status bytes inside my message body and send it, or is there another call required to flatten the data)? Thanks.

So, it seems clear that I can't mem-copy the ::grpc::status directly because it has the error_message and error_details fields of type ::grpc::string which is basically a typedef of std::string. So the data is not flat as far as I know, nor can I count on it.
I couldn't find a code example of how the serialization is done, so I did the following:
Create a Proto object with 3 fields: error_code, error_message, error_details.
When serializing, I would set those fields manually from the ::grpc::status and then call proto.SerializeToArray().
When deserializing, ParseProtoUnlimited(&proto) and then create a ::grpc::status from the proto fields.
Seems clean enough. Maybe just a little code duplication.
Proto:
message ErrorStatusProto {
int32 error_code = 1;
string error_message = 2;
string error_details = 3;
}
Serialize:
::grpc::Status gs = ToGrpcStatus(rm.status_);
ErrorStatusProto gsProto;
gsProto.set_error_code(gs.error_code());
gsProto.set_error_message(gs.error_message());
gsProto.set_error_details(gs.error_details());
gsProto.SerializeToArray(&message[kErrorStatusStartIndex],
gsProto.ByteSize());
if (gsProto.ByteSize() > kErrorStatusMaxSize) {
LOG(ERROR) << "Error status (" << gsProto.ByteSize() << " bytes) "
<< "is too big to fit in RDMA message ("
<< kErrorStatusMaxSize << " bytes). Truncated.";
}
Deserialize:
ErrorStatusProto gsProto;
CHECK(ParseProtoUnlimited(
&gsProto, &message[kErrorStatusStartIndex], kErrorStatusMaxSize))
<< "Failed to parse error status proto from message. Aborting.";
::grpc::Status gs((::grpc::StatusCode)gsProto.error_code(),
gsProto.error_message(), gsProto.error_details());
rm.status_ = FromGrpcStatus(gs);

Related

How to detect termination character in SChannel-based HTTPS client

I've searched StackOverflow trying to find a similar problem, but haven't come across it, so I am posting this question.
I am trying to write an C++ HTTPS client using Microsoft's SChannel libraries, and I'm getting stochastic errors with chunked message transfer. This issue only seems to occur on very long downloads -- short ones generally work OK. Most of the time the code works properly -- even for long downloads -- but occasionally the recv() command gracefully timesout, disconnecting my TLS session, and other times, I get an incomplete last packet. The stochastic errors appear to be the result of the different size chunks and encryption blocks the server is using to pass the data. I know I need to handle this variation, but while this would be easy to solve on an unencrypted HTTP connection, the encryption aspect is causing me problems.
First, the timeout problem, which occurs about 5% of the time I request large HTTP requests (about 10 MB of data from a single HTTP GET request).
The timeout is resulting because on the last chunk I have specified a bigger receive buffer than the data remaining on a blocking socket. The obvious fix to this is to only request exactly the number of bytes I need for the next chunk, and that is what I did. But for some reason, the amount received from each request is less than what I request, yet appears to be missing no data after decryption. I'm guessing this must be due to some compression in the data stream, but I don't know. IN any event, if it is using compression, I have no idea how to translate the size of the decrypted uncompressed byte stream into the size of compressed encrypted byte stream including the encryption headers and trailers to request the exact right number of bytes. Can anyone help me do that?
The alternative approach is for me to just look for two CR+LFs in a row, which would also signal the end of the HTTPS response. But because the data is encrypted, I can't figure out how to look byte by byte. SChannel's DecryptMessage() seems to do its decryptions in blocks, not byte by byte. Can anyone in this forum provide any advice on how to do byte-by-byte decryption to enable me to look for the end of the chunked output?
The second problem is DecryptMessage sometimes erroneously thinks it is done decrypting before I reach the actual end of the message. The resultant behavior is I go on to the next HTTP request, and I get the rest of the previous response where I am expecting to see the header of the new request.
The obvious solution to this is to check the contents of the decrypted message to see if we actually reached the end, and if not, try to receive more data before sending the next HTTP request. But when I do this, and try to decrypt, I get a decryption error message.
Any advice/help anyone can provide on a strategies would be appreciated. I've attached the relevant code sections for the read/decrypt process of the HTTP body -- I'm not including the header read and parsing because that is working without any problems.
do
{
// Note this receives large files OK, but I can't tell when I hit the end of the buffer, and this
// hangs. Need to consider a non-blocking socket?
// numBytesReceived = recv(windowsSocket, (char*)inputBuffer, inputBufSize, 0);
m_ErrorLog << "Next read size expected " << nextReadSize << endl;
numBytesReceived = recv(windowsSocket, (char*)inputBuffer, nextReadSize, 0);
m_ErrorLog << "NumBytesReceived = " << numBytesReceived << endl;
if (m_BinaryBufLen + numBytesReceived > m_BinaryBufAllocatedSize)
::EnlargeBinaryBuffer(m_BinaryBuffer,m_BinaryBufAllocatedSize,m_BinaryBufLen,numBytesReceived+1);
memcpy(m_BinaryBuffer+m_BinaryBufLen,inputBuffer,numBytesReceived);
m_BinaryBufLen += numBytesReceived;
lenStartDecryptedChunk = decryptedBodyLen;
do
{
// Decrypt the received data.
Buffers[0].pvBuffer = m_BinaryBuffer;
Buffers[0].cbBuffer = m_BinaryBufLen;
Buffers[0].BufferType = SECBUFFER_DATA; // Initial Type of the buffer 1
Buffers[1].BufferType = SECBUFFER_EMPTY; // Initial Type of the buffer 2
Buffers[2].BufferType = SECBUFFER_EMPTY; // Initial Type of the buffer 3
Buffers[3].BufferType = SECBUFFER_EMPTY; // Initial Type of the buffer 4
Message.ulVersion = SECBUFFER_VERSION; // Version number
Message.cBuffers = 4; // Number of buffers - must contain four SecBuffer structures.
Message.pBuffers = Buffers; // Pointer to array of buffers
scRet = m_pSSPI->DecryptMessage(phContext, &Message, 0, NULL);
if (scRet == SEC_E_INCOMPLETE_MESSAGE)
break;
if( scRet == SEC_I_CONTEXT_EXPIRED )
{
m_ErrorLog << "Server shut down connection before I finished reading" << endl;
m_ErrorLog << "# of Bytes Requested = " << nextReadSize << endl;
m_ErrorLog << "# of Bytes received = " << numBytesReceived << endl;
m_ErrorLog << "Decrypted data to this point = " << endl;
m_ErrorLog << decryptedBody << endl;
m_ErrorLog << "BinaryData just decrypted: " << endl;
m_ErrorLog << Buffers[0].pvBuffer << endl;
break; // Server signalled end of session
}
if( scRet != SEC_E_OK &&
scRet != SEC_I_RENEGOTIATE &&
scRet != SEC_I_CONTEXT_EXPIRED )
{
DisplaySECError((DWORD)scRet,errmsg);
m_ErrorLog << "CSISPDoc::ReadDecrypt(): " << "Failed to decrypt message--Error=" << errmsg;
if (decryptedBody)
m_ErrorLog << decryptedBody << endl;
return scRet;
}
// Locate data and (optional) extra buffers.
pDataBuffer = NULL;
pExtraBuffer = NULL;
for(i = 1; i < 4; i++)
{
if( pDataBuffer == NULL && Buffers[i].BufferType == SECBUFFER_DATA )
pDataBuffer = &Buffers[i];
if( pExtraBuffer == NULL && Buffers[i].BufferType == SECBUFFER_EXTRA )
pExtraBuffer = &Buffers[i];
}
// Display the decrypted data.
if(pDataBuffer)
{
length = pDataBuffer->cbBuffer;
if( length ) // check if last two chars are CR LF
{
buff = (PBYTE)pDataBuffer->pvBuffer; // printf( "n-2= %d, n-1= %d \n", buff[length-2], buff[length-1] );
if (decryptedBodyLen+length+1 > decryptedBodyAllocatedSize)
::EnlargeBuffer(decryptedBody,decryptedBodyAllocatedSize,decryptedBodyLen,length+1);
memcpy_s(decryptedBody+decryptedBodyLen,decryptedBodyAllocatedSize-decryptedBodyLen,buff,length);
decryptedBodyLen += length;
m_ErrorLog << buff << endl;
}
}
// Move any "extra" data to the input buffer -- this has not yet been decrypted.
if(pExtraBuffer)
{
MoveMemory(m_BinaryBuffer, pExtraBuffer->pvBuffer, pExtraBuffer->cbBuffer);
m_BinaryBufLen = pExtraBuffer->cbBuffer; // printf("inputStrLen= %d \n", inputStrLen);
}
}
while (pExtraBuffer);
if (decryptedBody)
{
if (incompletePacket)
p1 = decryptedBody + lenStartFragmentedPacket;
else
p1 = decryptedBody + lenStartDecryptedChunk;
p2 = p1;
pEndDecryptedBody = decryptedBody+decryptedBodyLen;
if (lastDecryptRes != SEC_E_INCOMPLETE_MESSAGE)
chunkSizeBlock = true;
do
{
while (p2 < pEndDecryptedBody && (*p2 != '\r' || *(p2+1) != '\n'))
p2++;
// if we're here, we probably found the end of the current line. The pattern we are
// reading is chunk length, chunk, chunk length, chunk,...,chunk lenth (==0)
if (*p2 == '\r' && *(p2+1) == '\n') // new line character -- found chunk size
{
if (chunkSizeBlock) // reading the size of the chunk
{
pStartHexNum = SkipWhiteSpace(p1,p2);
pEndHexNum = SkipWhiteSpaceBackwards(p1,p2);
chunkSize = HexCharToInt(pStartHexNum,pEndHexNum);
p2 += 2; // skip past the newline character
chunkSizeBlock = false;
if (!chunkSize) // chunk size of 0 means we're done
{
bulkReadDone = true;
p2 += 2; // skip past the final CR+LF
}
nextReadSize = chunkSize+8; // chunk + CR/LF + next chunk size (4 hex digits) + CR/LF + encryption header/trailer
}
else // copy the actual chunk
{
if (p2-p1 != chunkSize)
{
m_ErrorLog << "Warning: Actual chunk size of " << p2 - p1 << " != stated chunk size = " << chunkSize << endl;
}
else
{
// copy over the actual chunk data //
if (m_HTTPBodyLen + chunkSize > m_HTTPBodyAllocatedSize)
::EnlargeBuffer(m_HTTPBody,m_HTTPBodyAllocatedSize,m_HTTPBodyLen,chunkSize+1);
memcpy_s(m_HTTPBody+m_HTTPBodyLen,m_HTTPBodyAllocatedSize,p1,chunkSize);
m_HTTPBodyLen += chunkSize;
m_HTTPBody[m_HTTPBodyLen] = 0; // null-terminate
p2 += 2; // skip over chunk and end of line characters
chunkSizeBlock = true;
chunkSize = 0;
incompletePacket = false;
lenStartFragmentedPacket = 0;
}
}
p1 = p2; // move to start of next chunk field
}
else // got to end of encrypted body with no CR+LF found --> fragmeneted chunk. So we need to read and decrypt at least one more chunk
{
incompletePacket = true;
lenStartFragmentedPacket = p1-decryptedBody;
}
}
while (p2 < pEndDecryptedBody);
lastDecryptRes = scRet;
}
}
while (scRet == SEC_E_INCOMPLETE_MESSAGE && !bulkReadDone);
TLS does not support byte-by-byte decryption.
TLS 1.2 breaks its input into blocks of up to 16 kiB, then encrypts them into ciphertext blocks that are slightly larger due to the need for encryption IVs/nonces and integrity protection tags/MACs. It is not possible to decrypt a block until the entire block is available. You can find the full details at https://www.rfc-editor.org/rfc/rfc5246#section-6.2.
Since you're already able to decrypt the first few blocks (containing the headers), you should be able to read the HTTP length so that you at least know the plaintext length that you're expecting, which you can then compare to the number of bytes that you've decrypted from the stream. That won't tell you how many bytes of ciphertext you need, though -- you can get an upper bound on the size of a fragment by calling m_pSPPI->QueryContextAttributes() and then should read either at least that number of bytes or until end of stream before trying to decrypt.
Have you tried looking at other examples? http://www.coastrd.com/c-schannel-smtp appears to contain a detailed example of an SChannel-based TLS client.
I was finally able to figure this out. I fixed this by decrypting each TCP/IP packet as it came in to check for the CR+LF+CR+LF in the decrypted packet instead of what I had been doing -- trying to consolidate all of the encrypted packets into one buffer prior to decrypting it.
On the "hang" problem, what I thought was happening was that recv() wasn't returning because the amount of data actually received was smaller than my expected receive size. But what actually happened was I had actually received the entire transmission, but I didn't realize it. Thus, I was making additional recv() calls when there was actually no more data to receive. The fact that there was no more data to receive was what caused the connection to time out (causing a "hang").
The truncation problem was occurring because I couldn't detect the CR+LF+CR+LF sequence in the encrypted stream, and I erroneously thought SChannel returned SEC_E_OK on DecryptMessage() only when the entire response was processed.
Both problems were eliminated once I was able to detect the true end of the message by decrypting in piecemeal fashion vs. in bulk.
In order to figure this out, I had to completely restructure the sample SChannel code from www.coastRD.com. While the www.coastRD.com code was very helpful in general, it was written for SMTP transfers, not chunked HTTP encoding. In addition, the way it was written, it was hard to follow the logic for processing variations in how messages were received and processed. Lastly, I spent a lot of time "hacking" Schannel to understand how it behaves and which codes are returned under which conditions, because unfortunately none of that is discussed in any of the Microsoft documentation (that I've seen).
The first thing I needed to understand was how SChannel tries to decrypt a message. In Schannel, the 1st 13 bytes of an encrypted message are the encryption header, and the last 16 bytes are the encryption trailer. I still don't know what the trailer does, but I did realize that the encryption header is never actually encrypted/decrypted. The 1st 5 bytes are just the TLS record header for "application data" (hex code 0x17), followed by two bytes defining the TLS version used, followed by 2 bytes of the TLS record fragment size, followed by leading 0s and one byte which I still haven't figured out.
The reason this matters is that DecryptMessage() only works if the record type is "application data". For any other record type (such as a TLS handshake "finished message), DecryptMessage() won't even try to decrypt it-- it will just return a SEC_E_DECRYPT_FAILURE code.
In addition, I needed to understand that DecryptMessage() often can't decrypt the entire contents of the receive buffer in one pass when using chunked transfer encoding. In order to successfully process the entire contents of the receive buffer and the remainder of the server HTTPS response, I needed to understand two key return codes from DecryptMessage() -- SEC_E_OK and SEC_E_INCOMPLETE_MESSAGE.
When I received SEC_E_OK, it meant DecryptMessage() was able to successfully decrypt at least part of the receive buffer. When this occurred, the 1st 13 bytes (the encryption header) remained unchanged. However, the bytes immediately following the header were decrypted in-place, followed by the encryption trailer (which is also unchanged). Often, there will be additional encrypted data still in the receive buffer after the end of the encryption trailer, which is also unchanged.
Since I was using the SecBufferDesc output buffer structures and 4 SecBuffer structures described in www.coastRD.com's code, I needed to understand that these are not actually 4 separate buffers -- they are just pointers to different locations within the receive buffer. The first buffer is a pointer to the encryption header. The second buffer is a pointer to the beginning of the decrypted data. The 3rd buffer is a pointer to the beginning of the encryption trailer. Lastly, the 4th buffer is a pointer to the "extra" encrypted data that DecryptMessage() was not able to process on the last call.
Once I figured that out, I realized that I needed to copy the decrypted data (the pointer in the second buffer) into a separate buffer, because the receive buffer would probably be overwritten later.
If there was no "extra" data in the 4th buffer, I was done done for the moment -- but this was the exception rather than the rule.
If there was extra data (the usual case), I needed to move that data forward to the very beginning of the receive buffer, and I needed to call DecryptMessage() again. This decrypted the next chunk, and I appended that data to the data I already copied to the separate buffer, and repeated this process until there was either no more data left in the receive buffer to decrypt, or I received a SEC_E_INCOMPLETE_MESSAGE.
If I received a SEC_E_INCOMPLETE_MESSAGE, the data remaining in the receive buffer was unchanged. It wasn't decrypted because it was an incomplete encryption block. Thus, I needed to call recv() again to get more encrypted data from the server to complete the encryption block.
Once that occurred, I appended newly received data to the receive buffer. I appended it to the contents of the receive buffer vs. overwriting it because the latter approach would have overwritten the beginning of the encryption block, producing a SEC_E_DECRYPT_FAILURE message the next time I called DecryptMessage().
Once I appended this new block of data to the receive buffer, I repeated the steps above to decrypt the contents of the receive buffer, and continued to repeat this whole process until I got a SEC_E_OK message on the last chunk of data left in the receive buffer.
But I wasn't necessarily done yet -- there may still be data being sent by the server. Stopping at this point is what caused the truncation issue I had occasionally encountered.
So I now checked the last 4 bytes of the decrypted data to look for CR+LF+CR+LF. If I found that sequence, I knew I had received and decrypted a complete HTTPS response.
But if I hadn't, I needed to call recv() again and repeat the process above until I saw the CR+LF+FR+LF sequence at the end of the data.
Once I implemented this process, I was able to definitively identify the end of the encrypted HTTPS response, which prevented me from making an unnecessary recv() call when no data was remaining, preventing a "hang", as well as prematurely truncating the response.
I apologize for the long answer, but given the lack of documentation on SChannel and its functions like DecryptMessage(), I thought this description of what I learned might be helpful to others who may have also been struggling to use SChannel to process TLS HTTP responses.
Thank you again to user3553031 for trying to help me with this over 7 months ago -- those attempts helped me narrow down the problem.

How to connect a Flow to a Stream in SpinalHDL

I have a write sensitive APB register which shall push into a FIFO with clock domain crossing. I though I would write:
val myFlow = Flow(...)
busCtrl.driveFlow(myFlow,address=4)
val myFifo = StreamFifoCC(...)
myFifo.io.push << myFlow
But the << operator requires a Stream, so a type error happens.
myFifo.io.push << myFlow.toStream
Should be fine, if the fifo overflow, the myFlow transaction will be lost

What is Tag-matching interface?

I heard that PSM is a library supporting tag-matching.
What is Tag-matching interface? Why is tag-matching important for performance in the context of MPI?
Short intro in tag matching for MPI: https://www.hpcwire.com/2006/08/18/a_critique_of_rdma-1/ section "Matching"
MPI is a two-sided interface with a large matching space: a MPI Recv is associated with a MPI Send according to several criteria such as Sender, Tag, and Context, with the first two possibly ignored (wildcard). Matching is not necessarily in order and, worse, a MPI Send can be posted before the matching MPI Recv ... MPI requires 64 bits of matching information, and MX, Portals, and QsNet provide such a matching capability.
InfiniBand Verbs and other RDMA-based APIs do not support matching at all
So, it sounds like PSM is way to include fast matching to Infiniband-style network adapters (first versions with software matching, but with possibility of moving part of matching to the hardware).
I can't find public documentation of PSM (there are no details in User Guide http://storusint.com/pdf/qlogic/InfiniPath%20User%20Guide%202_0.pdf).
But there are sources of the library: https://github.com/01org/psm
Some details are listed in PSM2 presentation https://www.openfabrics.org/images/eventpresos/2016presentations/304PSM2Features.pdf
What is PSM?
Matched Queue (MQ) component
• Semantically matched to the needs of MPI using tag matching
• Provides calls for communication progress guarantees
• MQ completion semantics (standard vs. synchronized)
PSM API
• Global tag matching API with 64-bit tags
• Scale up to 64K processes per job
• MQ APIs provide point-to-point message passing between endpoints
• e.g. psm_mq_send, psm_mq_irecv
• No “recvfrom” functionality – needed by some applications
So, there are 64-bit tags. Every message has a tag, and Matched Queue has tag (in some tag matching implementations there is also tag mask). According to the source psm_mq_internal.h: mq_req_match() https://github.com/01org/psm/blob/67c0807c74e9d445900d5541358f0f575f22a630/psm_mq_internal.h#L381, there is mask in PSM:
typedef struct psm_mq_req {
...
/* Tag matching vars */
uint64_t tag;
uint64_t tagsel; /* used for receives */
...
} psm_mq_req_t;
mq_req_match(struct mqsq *q, uint64_t tag, int remove)
)
{
psm_mq_req_t *curp;
psm_mq_req_t cur;
for (curp = &q->first; (cur = *curp) != NULL; curp = &cur->next) {
if (!((tag ^ cur->tag) & cur->tagsel)) { /* match! */
if (remove) {
if ((*curp = cur->next) == NULL) /* fix tail */
q->lastp = curp;
cur->next = NULL;
}
return cur;
}
}
So, match is when the incoming tag is xored with tag of receives, posted to the MQ, result anded with tagsel of receive. If after these operations there are only zero bits, the match is found, else next receive is processed.
Comment from psm_mq.h, psm_mq_irecv() function, https://github.com/01org/psm/blob/4abbc60ab02c51efee91575605b3430059f71ab8/psm_mq.h#L206
/* Post a receive to a Matched Queue with tag selection criteria
*
* Function to receive a non-blocking MQ message by providing a preposted
* buffer. For every MQ message received on a particular MQ, the tag and #c
* tagsel parameters are used against the incoming message's send tag as
* described in tagmatch.
*
* [in] mq Matched Queue Handle
* [in] rtag Receive tag
* [in] rtagsel Receive tag selector
* [in] flags Receive flags (None currently supported)
* [in] buf Receive buffer
* [in] len Receive buffer length
* [in] context User context pointer, available in psm_mq_status_t
* upon completion
* [out] req PSM MQ Request handle created by the preposted receive, to
* be used for explicitly controlling message receive
* completion.
*
* [post] The supplied receive buffer is given to MQ to match against incoming
* messages unless it is cancelled via psm_mq_cancel #e before any
* match occurs.
*
* The following error code is returned. Other errors are handled by the PSM
* error handler (psm_error_register_handler).
*
* [retval] PSM_OK The receive buffer has successfully been posted to the MQ.
*/
psm_error_t
psm_mq_irecv(psm_mq_t mq, uint64_t rtag, uint64_t rtagsel, uint32_t flags,
void *buf, uint32_t len, void *context, psm_mq_req_t *req);
Example of encoding data into tag:
* uint64_t tag = ( ((context_id & 0xffff) << 48) |
* ((my_rank & 0xffff) << 32) |
* ((send_tag & 0xffffffff)) );
With tagsel mask we can encode both "match everything", "match tags with some bytes or bits equal to value, and anything in other", "match exactly".
There is newer PSM2 API, open source too - https://github.com/01org/opa-psm2, programmer's guide published at http://www.intel.com/content/dam/support/us/en/documents/network/omni-adptr/sb/Intel_PSM2_PG_H76473_v1_0.pdf.
In PSM2 tags are longer, and the matching rule is defined (stag is "Message Send Tag" - the tag value sent in message, and rtag is tag of receive request):
https://www.openfabrics.org/images/eventpresos/2016presentations/304PSM2Features.pdf#page=7
Tag matching improvement
• Increased tag size to 96 bits
• Fundamentally ((stag ^ rtag) & rtagsel) == 0
• Supports wildcards such as MPI_ANY_SOURCE or MPI_ANY_TAG using zero bits in rtagsel
• Allows for practically unlimited scalability
• Up to 64M processes per job
PSM2 TAG MATCHING
#define PSM_MQ_TAG_ELEMENTS 3
typedef
struct
psm2_mq_tag {
union {
uint32_t tag[PSM_MQ_TAG_ELEMENTS] __attribute__((aligned(16)));
struct {
uint32_t tag0;
uint32_t tag1;
uint32_t tag2;
};
};
} psm2_mq_tag_t;
• Application fills ‘tag’ array or ‘tag0/tag1/tag2’ and passes to PSM
• Both tag and tag mask use the same 96 bit tag type
And actually there is source peer address near matching variables in psm2_mq_req struct: https://github.com/01org/opa-psm2/blob/master/psm_mq_internal.h#L180
/* Tag matching vars */
psm2_epaddr_t peer;
psm2_mq_tag_t tag;
psm2_mq_tag_t tagsel; /* used for receives */
And software list scanning for match, mq_list_scan() called from mq_req_match() https://github.com/01org/opa-psm2/blob/85c07c656198204c4056e1984779fde98b00ba39/psm_mq_recv.c#L188:
psm2_mq_req_t
mq_list_scan(struct mqq *q, psm2_epaddr_t src, psm2_mq_tag_t *tag, int which, uint64_t *time_threshold)
{
psm2_mq_req_t *curp, cur;
for (curp = &q->first;
((cur = *curp) != NULL) && (cur->timestamp < *time_threshold);
curp = &cur->next[which]) {
if ((cur->peer == PSM2_MQ_ANY_ADDR || src == cur->peer) &&
!((tag->tag[0] ^ cur->tag.tag[0]) & cur->tagsel.tag[0]) &&
!((tag->tag[1] ^ cur->tag.tag[1]) & cur->tagsel.tag[1]) &&
!((tag->tag[2] ^ cur->tag.tag[2]) & cur->tagsel.tag[2])) {
*time_threshold = cur->timestamp;
return cur;
}
}
return NULL;
}

Error: cannot dynamically allocate this value type object on native heap

I am trying to create an instance of System::DateTimeon-the-fly and assign it to System::DateTime gDate in case user uses 'P' argument. But, I get the error shown following the code snippet.
case 'P':
gDate=new DateTime(std::stoi(year), std::stoi(month), std::stoi(day));
cout << "Persian Date is: " << pDate.GetDayOfMonth(gDate) << "/" <<
pDate.GetMonth (gDate) << "/" << pDate.GetYear(gDate) << endl;
break;
Error C3255 'System::DateTime': cannot dynamically allocate this value
type object on native heap
What causes the error and how should I prevent it?
Update:
I probably should have said in the first place, I tried also the following definition:
DateTime gDate(std::stoi(year), std::stoi(month), std::stoi(day));
But, I received the error, Error C2360 initialization of 'gDate' is skipped by 'case' label
If you don't need gDate to be a pointer, and you almost certainly do not, try:
case 'P':
{
DateTime gDate(std::stoi(year), std::stoi(month), std::stoi(day));
cout << "Persian Date is: " << pDate.GetDayOfMonth(gDate) << "/" <<
pDate.GetMonth (gDate) << "/" << pDate.GetYear(gDate) << endl;
}
break;
The braces establish a scope for gDate, ensuring deletion when the program exits the braces.
CLI/CLR C++ is a different beast from C++ and has some different semantics.
CLI/C++ has added a concepts of value and ref structs and classes. These are objects with automatic lifetime control. The .Net runtime, not the programmer, decides when they live and die, and this requires different syntax.
Those tagged value are intended to be used as one would use a Plain Old Datatype like an int or a double. Create 'em as a temporary, use them, and let the stack or whatever other method of managing temporary variables is in use take care of the clean-up. You can point to them, but it is not recommended.
ref structs and classes are designed with referenced use in mind and are open game for pointers, so long as they are garbage collected pointers.
System::DateTime is a value struct, so what follows strays from its recommended use. As a pointer, System::DateTime must either be used as a garbage collected pointer with ^ in place of * and allocated with gcnew in place of new or as a variable with a defined scope.
If gDate must be a pointer, it must be defined
DateTime ^ gDate;
And allocating it requires
gDate = gcnew DateTime(std::stoi(year), std::stoi(month), std::stoi(day));
When there are no further references to this allocated object, gDate and any copies of gDate have gone out of scope, the .Net runtime's garbage collector will destroy it.
As explained here, you can create DateTime on the stack, but not on the heap.
Try like this:
DateTime gDate(std::stoi(year), std::stoi(month), std::stoi(day));
cout << "Persian Date is: " << pDate.GetDayOfMonth(gDate) << "/" <<
pDate.GetMonth (gDate) << "/" << pDate.GetYear(gDate) << endl;
break;
alternatively, you can use gcnew to allocate managed memory:
DateTime^ gDate = gcnew DateTime(std::stoi(year), std::stoi(month), std::stoi(day));

How to output the concrete contents of a QsqlQuery before execution

For debuging prurpose I wouls like to print a sql query I am executing.
Here is my code:
QSqlQuery query;
query.prepare("INSERT INTO GeoAndEnergies VALUES(:smi,:chismi,:index,:rank,:comp,:met,:ba,:nha, :na, :gr, :gconv, :scfconv, :ener,:chemf,:prog,:ver,:cha,:mult,:sol,:geo, :freq, :enth, :free_e, :wei)");;
query.bindValue(":smi",QVariant(SMILES));
query.bindValue(":chismi",QVariant(ChiralSMILES));
query.bindValue(":index",QVariant(IndexCS));
query.bindValue(":rank",QVariant(Confrank));
query.bindValue(":comp",QVariant(Comptype));
query.bindValue(":met",QVariant(Method));
query.bindValue(":ba",QVariant(BASE));
query.bindValue(":nha",QVariant(NheavyAtom));
query.bindValue(":na",QVariant(NAtoms));
query.bindValue(":gr",QVariant(Grid));
query.bindValue(":gconv",QVariant(GeoConvergence));
query.bindValue(":scfconv",QVariant(SCFConvergence));
query.bindValue(":ener",QVariant(Energy));
query.bindValue(":chemf",QVariant(ChemicalFormula));
query.bindValue(":prog",QVariant(SOFTWARE));
query.bindValue(":ver",QVariant(VERSION));
query.bindValue(":cha",QVariant(Charge));
query.bindValue(":mult",QVariant(Multiplicity));
query.bindValue(":sol",QVariant(SOLVANT));
query.bindValue(":geo",QVariant(Geometry));
query.bindValue(":freq",QVariant(freq));
query.bindValue(":enth",QVariant(enthalpy));
query.bindValue(":free_e",QVariant(free_enthalpy));
query.bindValue(":wei",QVariant(weight));
if (!query.exec()){
std::cout << "Une erreur s'est produite. :(" << std::endl << q2c(query.lastError().text()) << std::endl;
}
return;
Thanks for tips.
query.executedQuery() will return the text of the last query that was successfully executed, with placeholder values replaced with concrete values. Hopefully, it'll also work if there was an error due to bad values, etc.
Note also that the explicit QVariant constructions are never necessary. For types that are handled by QVariant, the conversion will be done automatically. For custom types, there's no QVariant constructor available and the code won't compile anyway. You'd need to use QVariant::fromValue(xyz), where xyz has a custom type that has been Q_DECL_METATYPE'd in the header where the type is declared.
Your code could be rewritten as follows:
QSqlQuery query;
query.prepare("INSERT INTO GeoAndEnergies VALUES(:smi,:chismi,:index,:rank,:comp,:met,:ba,:nha, :na, :gr, :gconv, :scfconv,"
":ener,:chemf,:prog,:ver,:cha,:mult,:sol,:geo, :freq, :enth, :free_e, :wei)");
query.bindValue(":smi", SMILES);
query.bindValue(":chismi", ChiralSMILES);
query.bindValue(":index", IndexCS);
query.bindValue(":rank", Confrank);
query.bindValue(":comp", Comptype);
query.bindValue(":met", Method);
query.bindValue(":ba", BASE);
query.bindValue(":nha", NheavyAtom);
query.bindValue(":na", NAtoms);
query.bindValue(":gr", Grid);
query.bindValue(":gconv", GeoConvergence);
query.bindValue(":scfconv", SCFConvergence);
query.bindValue(":ener", Energy);
query.bindValue(":chemf", ChemicalFormula);
query.bindValue(":prog", SOFTWARE);
query.bindValue(":ver", VERSION);
query.bindValue(":cha", Charge);
query.bindValue(":mult", Multiplicity);
query.bindValue(":sol", SOLVANT);
query.bindValue(":geo", Geometry);
query.bindValue(":freq", freq);
query.bindValue(":enth", enthalpy);
query.bindValue(":free_e", free_enthalpy);
query.bindValue(":wei", weight);
if (!query.exec()) {
qWarning() << "The query has failed:" << query.executedQuery();
}