I'm using Jedis as the java client to connect to Redis servers. I am also using protocol buffers to write the data in jedis or redis cache. But I'm not finding the way how to write or set the protobuf object to redis. Am I missing something or Jedis doesn't provide the way to store protobuf object as value?
This is a late answer...
You can serialize the Protobuf message into a binary string, and save it to Redis using the SET command. However, whenever you need to get the Protobuf message, you have to get the binary string back, and deserialize it. Even if you only want to get a single field.
Another solution is that you can use the redis-protobuf module to save the Protobuf message into Redis. With this module, you can read or write the Protobuf message. If you only need a single field, you don't need to get the whole message back, redis-protobuf will only return that field.
Disclaimer: I'm the author of redis-protobuf.
Related
We are programming a MULE REST service which is divided in several layers.
The API layer (RAML-based) receives the inbound requests and prepares some flowVars so that the lower layers know how to proceed.
The second layer is also service defined, so there's one flow for each service oferred.
Finally, the third layer contains a unique flow and is the one which, depending on the flowVars configured in the upper layer, carries out a call using a HTTP Request component to the third-party service needed.
In this third layer, some audit registers are made in order to know what we are sending and what we are receiving. So, our audit component (a custom MULE connector) needs to write the content of the payload to our database, so a message.getPayloadAsString() (or similar) is needed. If we use a clean getter (like message.getPayload()), only the data type is obtained and thus written into the database.
The problem lays right in here. Every single payload received seems to be a BufferInputStream and, when doing the message.getPayloadAsString(), an inner casting seems to be affecting the payload. This, normally, wouldn't be a problem except for one of the cases that we have found: one of the services we invoke returns a PNG file, so message.getPayloadAsString() turns it into a String and breaks the image.
We've tried to clone the payload in order to keep one of the copies safe from the casting but, as an Object, it's not implementing Cloneable interface; we've tried to make a copy of the payload in any other single way, but only a new reference is generated; we've tried to serialize the payload to create a new copy from the serialized data but the Object doesn't either implement Serializable interface... Everything useless.
Any help, idea or piece of advice would be appreciated.
We finally managed to solve the problem by using message.getPayloadAsBytes();, which return value is a brand new byte[] object. This method doesn't either alter the payload within the message. By using the byte array we can create a String object to be written in our audit like this:
byte[] auditByteArray[] = message.getPayloadAsBytes();
String auditString = new String(auditByteArray);
Moreover, we tried a test consisting in stablishing that byte array as the new payload in the message and both JSON and PNG responses are managed correctly by the browser.
I inherited a Mirth Connect (v2.2.1) instance and am learning how it works. I'm now learning how globalChannelMap variables work, and I'm stumped by a misbehaving filter on a source connector.
In theory I can edit a csv text file in the Mirth Connect folders directory to update the globalChannelMap that is called by the filter.
But in practice the csv file is updated yet the source connector filter continues to call a prior globalChannelMap for the txt file. What step am I missing to update the globalChannelMap? Is there a simple way to output the current contents of a globalChannelMap?
You may need to redeploy. If you're seeing that you're using an old global channel map (using calKno's method), it means you need to redeploy the channel.
Channel's need to be redeployed anytime their code content is changed, be it an internal library (such as a code template), a transformer, or a global channel map.
You can get the map at the beginning of your filter and update it at the end or wherever it makes sense.
//get map
var map = globalChannelMap.get('mapName');
//log map value
logger.info('This is your map content: '+map);
//update map value
globalChannelMap.put('mapName', value);
I want to update ONE SINGLE attribute on the Core Data Object and send the change to the server using RestKit.
I see that ResKit is always sending Object with ALL attributes and not only with the changed ones. That makes my app slow(er).
I also see that update response from server should return whole object back to the RestKit which is again slower that it could be (All I need is success/failure response)
Is there elegant solution for this? (I am pretty new to the RestKit)
You can create a request descriptor on-the-fly which includes only the attributes you want to send, you would either simply need to use a mapping operation yourself or be sure to only run one such operation at a time and call removeRequestDescriptor: on the object manager (this could be tricky for you to manage). The alternate is to put the data to upload into a dictionary and upload that, but that isn't ideal.
For the response, the mapping says what to map but it doesn't all have to be there, RestKit will take whatever it can and map that.
I am using node.js to write a web service, it calls an API for some data but I am limited by the API to a number of calls per month, so I wish to cache the data I retrieve from the API so I can serve it up with the cached data, and re-fetch the data from the API at a timed interval.
Is this a good approach for this problem? And what caching framework should I use? I looked at node-redis but I don't think a key value store is appropriate for the data.
Thanks!
I would disagree with you regarding Redis. Redis is a very powerful key-value store that can easily be used for what you want. It is designed to have stuff dumped in it and taken out again. In your situation, you can easy cache the API response by saving it into Redis with the query as the key (if this is a REST API you're calling, you could just use the URL or serialized data as the key) and simply cache the response as a stringified JSON object (or XML string if you happen to be getting that).
You can also set an expiry on the cached data, and it will be cleared when the time is expired.
You could then wrap your API call in a helper function which checks the cache, and returns the value if it's present. If it's not it makes the API request, adds it to the cache, then returns it.
This is probably the most straightforward solution and seems to cover your use case pretty well.
Okay, so my goal is to build a easy to use protocol for sending data over tcp. basically, it would send a message, and an object(of unknown type) over tcp. To send, it would only require one method call and to receive it would only require one also.
So this is how I was thinking to format the "message".
length_of_message - "A string that is a message" - length_of_Object - object
length_of_message would be a set number of bytes. along with length_of_Object.
the actual message string and the actual object would be of variable length.
If the actual class of the object wouldn't be know, could I just declare it as a "generic object" somehow? and then get its class name from the "generic object" and the message would tell the receiver what to do with the object?
It would be simple if it was a constant object type but i want to be able use one send function and one receive function for ever object that needs to be send/recieved.
Any suggestions?
Thanks,
Andrew
Make sure you aren't reinventing the wheel (unless doing so is your primary goal).
With that in mind, consider:
• Implement and use the NSCoding protocol. It allows for the efficient archival of complexly connected object graphs, including cycles.
• Instead of raw TCP, use HTTP. While it adds a bit over overhead in the headers, the body can be straight encoded data. More importantly, HTTP is ubiquitous. It routes through just about anything whereas other protocols might be blocked (think proxy servers).
• Via HTTP, you can leverage compression. If one side of your communication pipe is an existing web server of some kind, it probably already supports gzip'd communication. Compressing an NSData (that would be the result of NSCoding) is trivial.
• Alternatively, stick with straight plists.
Unless you truly have some requirement that makes the above inviable, you are likely better off leveraging the above technologies instead of rolling a new one.
With that said, what you propose is fine. I would add, possibly, a structure like:
[HEADER][MSGID][LEN][TYPE][DATA of len][POST]
Where the POST is a known sequence of bytes that the receiver can verify to make sure that, maybe, all the data was received correctly. Or you could go whole hog and integrate a checksum. Or sub-pieces could be repeated, as needed (i.e. [LEN][TYPE][DATA] over and over.