Which one of the two is recommended approach given my server API is expecting a C# string? Which one will result in lowest string length?
1) Protobuf-net
Using protobuf-net to convert object <-> byte array
Use Convert.ToBase64String methods for converting byte array <-> string
2) Use Json .Net directly to convert object <-> string
We have Protobuf-net working in our project with byte[] server APIs. Now our server is migrating to string APIs instead of byte[]. We are not sure whether we should move to Json .Net or stay with protobuf-net and use Convert Base 64 for extra string to byte[] conversion.
What do you suggest?
Okay, so this is my thought process which I'm hoping can help you decide between the two:
Before deciding which one is better we need to have a better grasp of the context of the problem. Optimization is always something that has to be done under well defined "fitness" parameters.
What I mean by this is:
If you're most constrained by CPU usage, I would test to see which code uses more CPU to execute.
If bandwidth is an issue, you'd want to look at the method that sends the smallest packets. (In which case base64 of binary serialization should be the answer.)
If code readability is a factor, you should probably look at which code is easier to read / understand while taking less text to write. (In which case I suspect that the JSON route will have better readability)
In general, I would caution against over-optimization. Mainly because you might spend more time thinking and comparing than would be lost by your "unoptimized" code :)
That is to say, only optimize when you can clearly define your bottle-neck.
Hope this helped :)
Related
A little while ago, I started learning Kotlin, and I have done its basics, variables, classes, lists, and arrays, etc. but the book I was learning from seemed to miss one important aspect, reading and writing to a file, maybe a function like "fwrite" in C++
So I searched google, and yes, reading and writing bytes were easy enough. However, I being used to C++'s open personality, wanted to make a "kind of" database.
In C++ I would simply make a struct and keep appending it to a file, and then read all the stored objects one by one, by placing "fread" in a for loop or just reading into an array of the struct in one go, as the struct was simply just the bytes allocated to the variables inside it.
However in Kotlin, there is no struct, instead, we use Data Class to group data. I was hoping there was an equally easy way to store data in a file in form of Data Class and read it into maybe a List of that class, or if that is not possible, maybe some other way to store grouped data that would be easy to read and write.
Easiest way is to use a serialization library. Kotlin already provides something for that
TL;DR;
Add KotlinX Serialization to your project, choose the serialization format you prefer (protobuf or cbor will fit, go for json if you prefer something more human readable although bigger in size), use the proper serializer for generating your ByteArray and write it to a file using Kotlin methods for that
Generating the ByteArray might be tricky, not sure as I'm telling this from memory. What I can tell for sure is that if you choose JSON you can get the string representation and write to a file. So I'm assuming the same will be valid for binary formats (but writing to a file in binary instead of strings)
What you need can be fulfilled by ROOM DATABASE
It is officially recommended by GOOGLE, It uses your Android application's internal Database which is made using SQLITE
You can read more info about ROOM at
https://developer.android.com/jetpack/androidx/releases/room?gclid=Cj0KCQjw5ZSWBhCVARIsALERCvwjmJqiRPAnSYjhOzhPXg8dJEYnqKVgcRxSmHRKoyCpnWAPQIWD4gAaAlBnEALw_wcB&gclsrc=aw.ds
It provided Data Object Class (DAO) and Entity Classes through which one can access the database TABLE using SQL Queries.
Also, it will check your queries at compile time for any errors in it.
Note: You need to have basic SQL Knowledge for building the queries for CRUD Operations
Isn't storing only strings as data type a big overhead in terms of memory consumption?
e.g.: To store "304.2" in any application is more expensive than to store 304.2 as float/double.
Even if internally the value is indeed stored as a numeric value, delegating to every client the responsibility of "parsing" the string isn't another source of inefficiency?
I was getting super excited to start using redis but my case of usage is to cache a key x value structure like "string" x "doubles[]". Even if it would probably pay off in comparison with disk those two points really turns me off in adopting the technology.
I would love to be proven wrong, this is why I'm asking the question.
Thank you,
For point 1: You can't store 304.2 as a float/double; you can only store a close approximation to it. To store it, you need e.g. a dedicated decimal type, or more general rational type. Or a string.
For point 2:
RESP is a compromise between the following things:
Simple to implement.
Fast to parse.
Human readable.
Human readable means that no matter how numbers are stored internally, they still would be sent as strings and clients would have to parse them.
After all I've chosen Infinispan which gave me the APIs was looking for. Pros of the choosen solution is the actual ability to refer to the cache as a generic key x value concurrent map. Cons: probably less flexible in terms of out of the box client supported programming languages, even though you can always use google protobuff.
I am using VB.net and I want to make some cryptographic computations with keys length 1024bit (128bytes). I do not want to use a know algorithm thus I cannot use the Security library.
The biggest data type in vb.net is decimal (16bytes).
how can I do those computations? is there a different data type that I am not aware of?
You may like to check System.Numerics.BigInteger in .NET 4.0.
Also you may find it interesting to check Large Number Calculations in VB.NET
It would be instructive, even if you're not going to use them, to inspect the existing classes in the System.Security.Cryptography namespace.
You'll note that most of the methods that need to deal with keys, blocks, etc, are specced in terms of byte[]. A byte[] can be as big as you want/need it to be.
(Insert usual warnings about rolling your own crypto code)
I need to find the best implementation to send a byte array to the key space of a Redis Server with Booksleeve.
I tried different implementation like UTF8 Encoding but i don't know what is the most optimized one in memory of redis server (i will worked with millions of key like this so i really need the shortest in memory key).
Is anyone has already had this requirement?
In the current build for simplicity I've stuck to string keys, however the code would handle binary fine - it uses the binary API. IIRC I received a patch in my inbox just this week that adds binary key support.
Since it seems to be in demand I'll look at that this week.
Edit: a week came and went; the reason being that I'm also doing some work on redis-cluster support, which is going to need some new interfaces anyway, because:
not all operations are supported
parallel (numbered) databases aren't supported
So basically my plan is to roll both pieces of work into the same branch, giving:
a new set of interfaces
which use a struct for the key parameter with an implicit conversion operator from string and byte, allowing either to be used interchangeably
with the redis-cluster and redis-server commands on separate APIs
and a new method on the old connection to get one of the new APIs on a per-DB basis, i.e. Database(3).Keys.Remove(key); or something like that
ETA is still imaginary, but I wanted to explain why I hadn't simply thrown in the existing patch - I think the advent of redis-cluster makes it a good time to revisit the entire API, (but obviously in a way that doesn't break existing code).
Task: transfer a server side binary tree to client.
I got this task in an interview. Is there any efficient way to do this?
I don't understand the task very well myself.
This is what I came up with, but not sure about server to client trasfer. Any ideas?
void copyInOrder(TNode *orgTree, Tnode *& copyTree)
{
if(orgTree !=NULL){
//left side
TNode newLeftNode = cloneNode(orgTree->left_link);
copyTree->left_link = newLeftNode;
copyInOrder(orgTree->left_link, copyTree->left_link);
//right side
TNode newRightNode = cloneNode(orgTree->right_link);
copyTree->right_link = newRightNode;
copyInOrder(orgTree->right_link, copyTree->right_link);
}
}
Honestly, defining "efficient" would be a good first step. What is considered important? Network bandwidth? Server side computation involved? Client side computation involved?
Along the same lines, what is the data?
For example, if the data is integers and network bandwidth is important, you could do construct an array of ints from the tree, transfer that with minimal overhead, and then convert it back to a binary tree on the user's end.
If server side load is the most important thing, you'd go a different route. If client side load, possibly yet another.
Its worth noting that, since this is an interview question, discussing with the interviewer what they find important for efficiency and how it affects the solution may be just as important as the actual solution you come up with.
To me, the problem seems to be not so much about the transfer itself, but how one would package the data to be transferred. The important thing to note is that since you are transferring bytes, as opposed to C++ objects, you will not be able to preserve the structure of the tree so easily during the transfer.
Therefore, I would suggest that you serialize the tree, and then send it over the network. You would obviously need to a pre-determined serialization scheme that the client knows about, so that the client can recreate the tree from the bytes it receives.
It suffices to show how to convert a given binary tree T to a sequence S of values in such a way that we can reconstruct T from S at the client side. One possible way is to use the well know fact:
We can construct a binary tree uniquely from its inorder and preorder
traversals.
Thus we can let S be the inorder traversal followed by the preorder traversal. Check the answer of this question.