Content in received ActiveMQBytesMessage is empty - activemq

When I receive a ActiveMQBytesMessage, the Content property has exactly the expected length, but all bytes are zero, it looks like an uninitialized buffer with just the correct length.
I also tried ReadBytes(buffer, length) on the message object, and got the same empty buffer.
I'm pretty sure that the messages arrives correctly at the broker, because I can look into the XML representation of the message via the ActiveMQ web interface, and there I can see the correct Base64 representation in <content><data>
I'm using the latest version 1.5.6.
Any ideas what I might be doing wrong?
Update
How I create the message before sending it using a publisher:
var binMessage = new ActiveMQBytesMessage();
binMessage.Content = /* ... */
This is what a message looks like on the web interface (queue browser):
<org.apache.activemq.command.ActiveMQBytesMessage>
<commandId>5</commandId>
<!-- ... -->
<content>
<data>H4sIAN[...]AA=</data>
<offset>0</offset>
<length>305</length>
</content>
<!-- ... -->
</org.apache.activemq.command.ActiveMQBytesMessage>
This is what I basically do in my MessageListener:
Console.WriteLine("Message ID: " + message.NMSMessageId);
var bytesMessage = message as IBytesMessage;
if (bytesMessage != null)
{
Console.WriteLine("Content length: " + bytesMessage.Content.Length);
Console.WriteLine("Content: " + BitConverter.ToString(bytesMessage.Content));
}
This gives me the correct content length, but the actual content is just empty (hex output is just 00-00-00 etc.).

I believe this is working as its currently designed. The Content field is read and returned to you by the first call to Content where you display the length. At this point the message is now read to the end of its data stream. Before you can call Content again and re-read the data you need to reset the message.
Something like this should work.
Console.WriteLine("Message ID: " + message.NMSMessageId);
var bytesMessage = message as IBytesMessage;
if (bytesMessage != null)
{
byte[] content = bytesMessage.Content;
Console.WriteLine("Content length: " + content.Length);
Console.WriteLine("Content: " + BitConverter.ToString(content));
}

Related

System.Text.Json adds an extra curly bracket ONLY TO END so it causes exception

I am sending JSON object through TcpSocket. I deserialize it after destination receives. Usually first a few objects are sent and deserialized without issue! And then suddenly one comes with an extra curly braket only at the end then run time exception.
Seriosuly, what the hell is this ?
System.Text.Json.JsonException: ''}' is invalid after a single JSON value. Expected end of data. Path: $ | LineNumber: 0 | BytePositionInLine: 32.'
{"Value":3,"Name":"Blood Sugar"}}
while(true)
{
seperateSocketForEachRequest.Receive(byteMessage);
seperateSocketForEachRequest.Send(Encoding.UTF8.GetBytes("FF"));
string stringMessage = Encoding.UTF8.GetString(byteMessage);
stringMessage = stringMessage.Substring(0, stringMessage.IndexOf('\0'));
Object message = JsonSerializer.Deserialize<Object>(stringMessage);
}
////////////////////////////////////////
while (Form.isGenerate)
{
Data newData = dataType.generate(person.generatingParameters);
Thread.Sleep(500);
clientSocket.Send(Encoding.UTF8.GetBytes(JsonSerializer.Serialize<Data>(newData)));
byte[] messageReceivedByte = new Byte[1024];
clientSocket.Receive(messageReceivedByte);
}
I found the issue. It is caused of data transmission. Apparently same buffer is used for writing data received from socket and new data is written over old data. Therefore, when value of data is 2 digit number, no issue, when a data comes with 1 digit, boom.
{"Value":76,"Name":"Blood Sugar"}
{"Value":99,"Name":"Blood Sugar"}
{"Value":76,"Name":"Blood Sugar"}
{"Value":1,"Name":"Blood Sugar"}}

what is correct use of consumer groups in spring cloud stream dataflow and rabbitmq?

A follow up to this:
one SCDF source, 2 processors but only 1 processes each item
The 2 processors (del-1 and del-2) in the picture are receiving the same data within milliseconds of each other. I'm trying to rig this so del-2 never receives the same thing as del-1 and vice versa. So obviously I've got something configured incorrectly but I'm not sure where.
My processor has the following application.properties
spring.application.name=${vcap.application.name:sample-processor}
info.app.name=#project.artifactId#
info.app.description=#project.description#
info.app.version=#project.version#
management.endpoints.web.exposure.include=health,info,bindings
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration
spring.cloud.stream.bindings.input.group=input
Is "spring.cloud.stream.bindings.input.group" specified correctly?
Here's the processor code:
#Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
public Object transform(String inputStr) throws InterruptedException{
ApplicationLog log = new ApplicationLog(this, "timerMessageSource");
String message = " I AM [" + inputStr + "] AND I HAVE BEEN PROCESSED!!!!!!!";
log.info("SampleProcessor.transform() incoming inputStr="+inputStr);
return message;
}
Is the #Transformer annotation the proper way to link this bit of code with "spring.cloud.stream.bindings.input.group" from application.properties? Are there any other annotations necessary?
Here's my source:
private String format = "EEEEE dd MMMMM yyyy HH:mm:ss.SSSZ";
#Bean
#InboundChannelAdapter(value = Source.OUTPUT, poller = #Poller(fixedDelay = "1000", maxMessagesPerPoll = "1"))
public MessageSource<String> timerMessageSource() {
ApplicationLog log = new ApplicationLog(this, "timerMessageSource");
String message = new SimpleDateFormat(format).format(new Date());
log.info("SampleSource.timeMessageSource() message=["+message+"]");
return () -> new GenericMessage<>(new SimpleDateFormat(format).format(new Date()));
}
I'm confused about the "value = Source.OUTPUT". Does this mean my processor needs to be named differently?
Is the inclusion of #Poller causing me a problem somehow?
This is how I define the 2 processor streams (del-1 and del-2) in SCDF shell:
stream create del-1 --definition ":split > processor-that-does-everything-sleeps5 --spring.cloud.stream.bindings.applicationMetrics.destination=metrics > :merge"
stream create del-2 --definition ":split > processor-that-does-everything-sleeps5 --spring.cloud.stream.bindings.applicationMetrics.destination=metrics > :merge"
Do I need to do anything differently there?
All of this is running in Docker/K8s.
RabbitMQ is given by bitnami/rabbitmq:3.7.2-r1 and is configured with the following props:
RABBITMQ_USERNAME: user
RABBITMQ_PASSWORD <redacted>:
RABBITMQ_ERL_COOKIE <redacted>:
RABBITMQ_NODE_PORT_NUMBER: 5672
RABBITMQ_NODE_TYPE: stats
RABBITMQ_NODE_NAME: rabbit#localhost
RABBITMQ_CLUSTER_NODE_NAME:
RABBITMQ_DEFAULT_VHOST: /
RABBITMQ_MANAGER_PORT_NUMBER: 15672
RABBITMQ_DISK_FREE_LIMIT: "6GiB"
Are any other environment variables necessary?

PUT blob with Blob Service API in Postman - Authorization header

I need help constructing the Authorization header to PUT a block blob.
PUT\n\n\n11\n\n\n\n\n\n\n\n\nx-ms-blob-type:BlockBlob\nx-ms-date:Sat, 25 Feb 2017 22:20:13 GMT\nx-ms-version:2015-02-21\n/myaccountname/mycontainername/blob.txt\n
I take this, UTF 8 encode it. Then I take my access key in my Azure account and HMAC sha256 this UTF 8 encoded string with the key. Then I output that in base64. Let's call this output string.
My authorization header looks like this: SharedKey myaccountname:output string
It is not working.
The header in Postman also has x-ms-blob-type, x-ms-date, x-ms-version, Content-Length, and Authorization. The body for now says hello world.
Can anyone help me make this successful request in Postman?
<?xml version="1.0" encoding="utf-8"?>
<Error>
<Code>AuthenticationFailed</Code>
<Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:cdeb9a5e-0001-0029-5fb5-8f7995000000
Time:2017-02-25T22:22:32.0300016Z</Message>
<AuthenticationErrorDetail>The MAC signature found in the HTTP request 'jiJtirohvi1syXulqkPKESnmQEJI4GpDU5JBn7BM/xY=' is not the same as any computed signature. Server used following string to sign: 'PUT
11
text/plain;charset=UTF-8
x-ms-date:Sat, 25 Feb 2017 22:20:13 GMT
x-ms-version:2015-02-21
/myaccountname/mycontainername/blob.txt'.</AuthenticationErrorDetail>
</Error>
EDIT:
First, I want to thank you and everyone who responded. I truly truly appreciate it. I have one last question and then I think I'll be set!! I'm not using that code - I'm doing this all by hand. If I have my key: X2iiy6v47j1jZZH5555555555zzQRrIAdxxVs55555555555av8uBUNGcBMotmS7tDqas14gU5O/w== changed slightly for anonymity - do I decode it: using an online base64decoder. Then, when I have my string which now looks like this: PUT\n\n\n11\n\ntext/plain;charset=UTF-8\n\n\n\n\n\n\nx-ms-blob-type:BlockBlob\nx-ms-date:Mon, 27 Feb 2017 21:53:13 GMT\nx-ms-version:2015-02-21\n/myaccount/mycontainer/blob.txt\n so I run this in https://mothereff.in/utf-8 and then use this in HMAC with my decoded key: https://www.liavaag.org/English/SHA-Generator/HMAC/ - using sha256 and base64 at the end.
Is that how I get the correct string to put here?: SharedKey myaccount:<string here>
I believe there's an issue with how you're specifying StringToSign here:
PUT\n\n\n11\n\n\n\n\n\n\n\n\nx-ms-blob-type:BlockBlob\nx-ms-date:Sat,
25 Feb 2017 22:20:13
GMT\nx-ms-version:2015-02-21\n/myaccountname/mycontainername/blob.txt\n
If you notice the error message returned from the server, string to sign by server is different than yours and the difference is that the server is using Content-Type (text/plain;charset=UTF-8) in signature calculation while you're not. Please include this content type in your code and things should work just fine.
Here's the sample code (partial only) I used:
var requestMethod = "PUT";
var urlPath = "test" + "/" + "myblob.txt";
var storageServiceVersion = "2015-12-11";
var date = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
var blobType = "BlockBlob";
var contentBytes = Encoding.UTF8.GetBytes("Hello World");
var canonicalizedResource = "/" + accountName + "/" + urlPath;
var canonicalizedHeaders = "x-ms-blob-type:" + blobType + "\nx-ms-date:" + date + "\nx-ms-version:" + storageServiceVersion + "\n";
var stringToSign = requestMethod + "\n" +
"\n" + //Content Encoding
"\n" + //Content Language
"11\n" + //Content Length
"\n" + //Content MD5
"text/plain;charset=UTF-8" + "\n" + //Content Type
"\n" + //Date
"\n" + //If - Modified - Since
"\n" + //If - Match
"\n" + //If - None - Match
"\n" + //If - Unmodified - Since
"\n" + //Range +
canonicalizedHeaders +
canonicalizedResource;
string authorizationHeader = GenerateSharedKey(stringToSign, accountKey, accountName);
private static string GenerateSharedKey(string stringToSign, string key, string account)
{
string signature;
var unicodeKey = Convert.FromBase64String(key);
using (var hmacSha256 = new HMACSHA256(unicodeKey))
{
var dataToHmac = Encoding.UTF8.GetBytes(stringToSign);
signature = Convert.ToBase64String(hmacSha256.ComputeHash(dataToHmac));
}
return string.Format(CultureInfo.InvariantCulture, "{0} {1}:{2}", "SharedKey", account, signature);
}
According to your error message, it indicates that authorization signature is incorrect.
If the Content-Type "text/plain; charset=UTF-8" is not included in the header, please add it in the stringTosign and postman.
When we try to get the signature, we need to make sure the length of the blob.txt matches the Content length in the stringTosign. That means request body length should match the content length in the stringTosign.
I test it with Postman, it works correctly. We can get the signature with the code in another SO Thread. The following is my detail steps
Add the following header
Add the request body (example: Hello World)
Send the put blob request.
Update :
Please have a try to use the online tool to generate signature for test.

Global variable assigned inside while loop not available outside the loop

I'm calling an api using ESP8266 WiFi Chip. Im declaring a global variable to assign the token sent by the server to it. The variable assignment is made inside a while loop. In this loop i'm reading the contents of the reply "the token" and saving it in the global variable.
But when i print out the result outside the while loop to variable is empty. But inside the loop it is not.
String deviceToken;
WiFiClient client;
if (!client.connect(hostIp, 80)){
Serial.println("Connection to server failed.");
Serial.print(client.connect(hostIp, 80));
return;
}
String authURL = "/api/";
authURL += "startauth/";
authURL += serialValue;
Serial.print("Requesting URL: ");
Serial.println(authURL);
// This will send auth request to the server
client.print(String("GET ") + authURL + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Connection: close\r\n\r\n");
unsigned long timeout = millis();
while (client.available() == 0) {
if (millis() - timeout > 5000) {
Serial.println(">>> Client Timeout !");
client.stop();
return;
}
}
// Read Auth reply from server and print them to Serial
while(client.available()){
String token = client.readStringUntil('\r');
deviceToken = token;
writeDeviceToken(token.c_str()); //Function to write data in eeprom
Serial.print(deviceToken); //NOT EMPTY
}
Serial.print("Device Token:" + deviceToken); //EMPTY VARIABLE
In your while loop you are looping over the HTTP headers and the data line by line and overwrite deviceToken each time. This is ok, except that each line ends with \r\n and not only \r - (also, you probably only want to write the token to the eeprom, not every header).
So the last token that gets read is a single \n which is then put into deviceToken. You will need to read that extra byte after each line.
An alternative way would be to read until you reach \r\n\r\n (end of header block), then read the size (28 hex - this is a chunked transfer), and then you can read and store the token you want. Otherwise your token would be that last 0, or the newline following it.
Also keep in mind that there is no guarantee that the reply will allways be chunked (which is actually rather strange in this case), and you should be prepared for a 'normal' reply too.
An easy way to fix this, is to send a HTTP/1.0 request instead of HTTP/1.1 (as far as that server supports it). Since HTTP/1.0 doesn't support/allow chunked transfers you will always have a 'clean' deviceToken in the reply.
Thanks to Danny_ds instructions i was able to solve my issue and thought of sharing the working code in here as a reference.
// Read Auth reply from server and print them to Serial
while(client.available()){
char c = client.read();
response += c;
}
String testStatus = response;
if((strstr(testStatus.c_str(),"HTTP/1.1 500")) != NULL) {
Serial.println("Found reponse 500");
return;
} else if ((strstr(testStatus.c_str(),"HTTP/1.1 200")) != NULL) {
Serial.println("Found reponse 200");
//Grabing the token after the header
String token = response.substring(response.indexOf("\r\n\r\n") + 8 );
//Removing the \r in end of token
token.remove(40);
deviceToken = token;
writeDeviceToken(deviceToken.c_str());
delay(500);
Serial.print("Device Token:" + deviceToken);
return;
}

The ':' character, hexadecimal value 0x3A, cannot be included in a name

I saw this question already, but I didnt see an answer..
So I get this error:
The ':' character, hexadecimal value 0x3A, cannot be included in a name.
On this code:
XDocument XMLFeed = XDocument.Load("http://feeds.foxnews.com/foxnews/most-popular?format=xml");
XNamespace content = "http://purl.org/rss/1.0/modules/content/";
var feeds = from feed in XMLFeed.Descendants("item")
select new
{
Title = feed.Element("title").Value,
Link = feed.Element("link").Value,
pubDate = feed.Element("pubDate").Value,
Description = feed.Element("description").Value,
MediaContent = feed.Element(content + "encoded")
};
foreach (var f in feeds.Reverse())
{
....
}
An item looks like that:
<rss>
<channel>
....items....
<item>
<title>Pentagon confirms plan to create new spy agency</title>
<link>http://feeds.foxnews.com/~r/foxnews/most-popular/~3/lVUZwCdjVsc/</link>
<category>politics</category>
<dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/" />
<pubDate>Tue, 24 Apr 2012 12:44:51 PDT</pubDate>
<guid isPermaLink="false">http://www.foxnews.com/politics/2012/04/24/pentagon-confirms-plan-to-create-new-spy-agency/</guid>
<content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/"><![CDATA[|http://global.fncstatic.com/static/managed/img/Politics/panetta_hearing_030712.jpg<img src="http://feeds.feedburner.com/~r/foxnews/most-popular/~4/lVUZwCdjVsc" height="1" width="1"/>]]></content:encoded>
<description>The Pentagon confirmed Tuesday that it is carving out a brand new spy agency expected to include several hundred officers focused on intelligence gathering around the world.&amp;#160;</description>
<dc:date xmlns:dc="http://purl.org/dc/elements/1.1/">2012-04-4T19:44:51Z</dc:date>
<feedburner:origLink>http://www.foxnews.com/politics/2012/04/24/pentagon-confirms-plan-to-create-new-spy-agency/</feedburner:origLink>
</item>
....items....
</channel>
</rss>
All I want is to get the "http://global.fncstatic.com/static/managed/img/Politics/panetta_hearing_030712.jpg", and before that check if content:encoded exists..
Thanks.
EDIT:
I've found a sample that I can show and edit the code that tries to handle it..
EDIT2:
I've done it in the ugly way:
text.Replace("content:encoded", "contentt").Replace("xmlns:content=\"http://purl.org/rss/1.0/modules/content/\"","");
and then get the element in the normal way:
MediaContent = feed.Element("contentt").Value
The following code
static void Main(string[] args)
{
var XMLFeed = XDocument.Parse(
#"<rss>
<channel>
....items....
<item>
<title>Pentagon confirms plan to create new spy agency</title>
<link>http://feeds.foxnews.com/~r/foxnews/most-popular/~3/lVUZwCdjVsc/</link>
<category>politics</category>
<dc:creator xmlns:dc='http://purl.org/dc/elements/1.1/' />
<pubDate>Tue, 24 Apr 2012 12:44:51 PDT</pubDate>
<guid isPermaLink='false'>http://www.foxnews.com/politics/2012/04/24/pentagon-confirms-plan-to-create-new-spy-agency/</guid>
<content:encoded xmlns:content='http://purl.org/rss/1.0/modules/content/'><![CDATA[|http://global.fncstatic.com/static/managed/img/Politics/panetta_hearing_030712.jpg<img src='http://feeds.feedburner.com/~r/foxnews/most-popular/~4/lVUZwCdjVsc' height='1' width='1'/>]]></content:encoded>
<description>The Pentagon confirmed Tuesday that it is carving out a brand new spy agency expected to include several hundred officers focused on intelligence gathering around the world.&amp;#160;</description>
<dc:date xmlns:dc='http://purl.org/dc/elements/1.1/'>2012-04-4T19:44:51Z</dc:date>
<!-- <feedburner:origLink>http://www.foxnews.com/politics/2012/04/24/pentagon-confirms-plan-to-create-new-spy-agency/</feedburner:origLink> -->
</item>
....items....
</channel>
</rss>");
XNamespace contentNs = "http://purl.org/rss/1.0/modules/content/";
var feeds = from feed in XMLFeed.Descendants("item")
select new
{
Title = (string)feed.Element("title"),
Link = (string)feed.Element("link"),
pubDate = (string)feed.Element("pubDate"),
Description = (string)feed.Element("description"),
MediaContent = GetMediaContent((string)feed.Element(contentNs + "encoded"))
};
foreach(var item in feeds)
{
Console.WriteLine(item);
}
}
private static string GetMediaContent(string content)
{
int imgStartPos = content.IndexOf("<img");
if(imgStartPos > 0)
{
int startPos = content[0] == '|' ? 1 : 0;
return content.Substring(startPos, imgStartPos - startPos);
}
return string.Empty;
}
results in:
{ Title = Pentagon confirms plan to create new spy agency, Link = http://feeds.f
oxnews.com/~r/foxnews/most-popular/~3/lVUZwCdjVsc/, pubDate = Tue, 24 Apr 2012 1
2:44:51 PDT, Description = The Pentagon confirmed Tuesday that it is carving out
a brand new spy agency expected to include several hundred officers focused on
intelligence gathering around the world. , MediaContent = http://global
.fncstatic.com/static/managed/img/Politics/panetta_hearing_030712.jpg }
Press any key to continue . . .
A few points:
You never want to treat Xml as text - in your case you removed the namespace declaration but actually if the namespace was declared inline (i.e. without binding to the prefix) or a different prefix would be defined your code would not work even though semantically both documents would be equivalent
Unless you know what's inside CDATA and how to treat it you always want to treat is as text. If you know it's something else you can treat it differently after parsing - see my elaborate on CDATA below for more details
To avoid NullReferenceExceptions if the element is missing I used explicit conversion operator (string) instead of invoking .Value
the Xml you posted was not a valid xml - there was missing namespace Uri for feedburner prefix
This is no longer related to the problem but may be helpful for some folks so I am leaving it
As far as the contents of the encode element is considered it is inside CDATA section. What's inside CDATA section is not an Xml but plain text. CDATA is usually used to not have to encode '<', '>', '&' characters (without CDATA they would have to be encoded as < > and & to not break the Xml document itself) but the Xml processor treat characters in the CDATA as if they were encoded (or to be more correct in encodes them). The CDATA is convenient if you want to embed html because textually the embedded content looks like the original yet it won't break your xml if the html is not a well-formed Xml. Since the CDATA content is not an Xml but text it is not possible to treat it as Xml. You will probably need to treat is as text and use for instance regular expressions. If you know it is a valid Xml you can load the contents to an XElement again and process it. In your case you have got mixed content so it is not easy to do unless you use a little dirty hack. Everything would be easy if you have just one top level element instead of mixed content. The hack is to add the element to avoid all the hassle. Inside the foreach look you can do something like this:
var mediaContentXml = XElement.Parse("<content>" + (string)item.MediaContent + "</content>");
Console.WriteLine((string)mediaContentXml.Element("img").Attribute("src"));
Again it's not pretty and it is a hack but it will work if the content of the encoded element is valid Xml. The more correct way of doing this is to us XmlReader with ConformanceLevel set to Fragment and recognize all kinds of nodes appropriately to create a corresponding Linq to Xml node.
You should use XNamespace:
XNamespace content = "...";
// later in your code ...
MediaContent = feed.Element(content + "encoded")
See more details here.
(Of course, you the string to be assigned to content is the same as in xmlns:content="...").