I have multiple servers (for redundancy) sending data to clients. The clients need to process these messages in sequence and ignore duplicates.
We use external information to determine a special sequencing string that is deterministic across all our servers, as it would be too slow to keep the servers in sync.
The sequencing strings generated have remnants of top-secret information in them, and we can't reveal them to the clients.
Suppose the sequencing string just contains an integer. Is there a way of hashing this data such that the clients can order the messages without learning any additional information about its content?
Suppose a more complicated sequence string is used. The string is split into sub-sequences, and each sub-sequence is given a category, something like "a:12477/t:637" and "a:12477/e:456", where the comparison function between sequences is given below. Is it possible to hash the sequencing string in such a way that even a complicated function like this can operate on the data and nothing else?
JavaScript pseudo-code:
function compare(seq_a: string, seq_b: string) {
function decode(seq) {
seq_a.split("/").map(segment => {
let [category, sub_seq] = segment.split(":");
return { category, sub_seq: Number(sub_seq) }
});
}
let a = decode(seq_a);
let b = decode(seq_b);
for (let i = 0; i < Math.max(a.length, b.length); i++) {
let segment_a = a[i] || { category: "empty", sub_seq: 0 };
let segment_b = b[i] || { category: "empty", sub_seq: 0 };
if (segment_a.category != segment_b.category) {
return "UNKNOWN";
}
if (segment_a.sub_seq > segment_b.sub_seq) {
return "A";
} else if (segment_a.sub_seq < segment_b.sub_seq) {
return "B";
} else if (segment_a.sub_seq == segment_b.sub_seq) {
continue;
}
}
return "UNKNOWN";
}
I have very little knowledge in the cryptographic and zero-knowledge area so there's nothing I have yet tried, so the furthest I have gotten is just realizing the idea of what is needed.
Related
I have a list of states, which are defined to be ordered by min to max. the sequence is the following:
Cancelled - complete - draft - reservation - reserved - ordered - confirmed
So the cancelled is the minimum state, and confirmed is the maximum state. I may have different instances with different states, so I use a for-each loop to run through all states, and select the minimum state present in the loop.
That is: if in a list I have states [complete, reserved, draft, ordered] I need to check all the values and select complete -as it appears to be the minimum state. OR
if I have [reserved, confirmed, ordered, draft, cancelled, confirmed, confirmed] I need to select the cancelled value, as it appears to be the minimum.
I am doing the following check, but it does not seem to be working:
string globstatus = " ";
foreach (var currentstatus in list)
{
if (currentstatus == "cancelled")
{
globstatus = "cancelled";
}
else
{
if (globstatus == "cancelled")
{
return globstatus;
}
else
{
if (currentstatus == "complete")
{
globstatus = "complete";
}
else
{
if (globstatus == "complete")
{
return globstatus;
}
else
{
if (currentstatus == "draft")
{
globstatus = "draft";
}
else
{
if (globstatus == "reservation")
{
return globstatus;
}
else
{
if (currentstatus == "reserved")
{
globstatus = "reserved";
}
else
{
if (globstatus == "ordered")
{
return globstatus;
}
else
{
if (currentstatus == "confirmed")
{
globstatus = "confirmed";
}
else
{
return currentstatus;
}
}
}
}
}
}
}
}
}
}
return globstatus;
What can be the best solution to achieve the desired behavior?
I find a rule of thumb helpful that if I need more than three levels of braces, I need to rethink my code. It's hard to follow, easy to make mistakes, and a nightmare to debug. I suggest that applies here - trying to follow the flow of what all those nested if..else statements is extremely difficult.
Using Enum
My preferred solution is to achieve this using an Enum, e.g.:
var list = new List<Status>
{
Status.Complete,
Status.Draft,
Status.Draft,
Status.Confirmed
};
var minStatus = (Status)list.Select(l => (int)l).Min();
// minStatus = Status.Complete
public enum Status
{
Cancelled,
Complete,
Draft,
Reservation,
Reserved,
Ordered,
Confirmed
}
How it works: by default Enums give each value a zero-based integer, i.e. Cancelled = 0, Complete = 1 and so on. You can override this with your own values if you wish (e.g. 1/2/4/8/16 if you want to combine multiple values).
I recommend using Enum types for things like this, rather than strings. It helps avoid typos, gives someone else looking at your code a clear understanding of how your program works and its flow, and represents hierarchy in a way in which simple strings don't. (For example - does 'complete' come before or after 'draft'? Without context, I imagine most people would say after, but in this case it comes before - that is much more obvious when using an Enum.)
Parse strings to Enum
However if the statuses have to be strings, you could parse them into an enum like so:
var stringList = new List<string>
{
"complete",
"draft",
"draft",
"confirmed",
"this will be ignored"
};
var statusList = new List<int>();
foreach (var str in stringList)
{
if(Enum.TryParse(typeof(Status), str, ignoreCase: true, out object? parsed) && parsed is Status status)
{
statusList.Add((int)status);
}
}
var minStatus = (Status)statusList.Min();
// minStatus = Status.Complete
However, if it's possible to refactor your code to use the Enum in the first place, that would be a better solution, and much quicker as parsing strings has an overhead that would be good to avoid.
I use Redis Streams in my Spring Boot application. Within a scheduler I regularly want to get all the pending messages and check how long they are already processing and re-trigger them if necessary.
My problem is now that I can get the pending messages, but I'm not sure how to get the payload.
My first approach used the pending and range operations. The downside here is that the totalDeliveryCount is not increased with range - so I cannot use the range method
val pendingMessages = stringRedisTemplate.opsForStream<String, Any>().pending(redisStreamName, Consumer.from(redisConsumerGroup, instanceName))
return pendingMessages.filter { pendingMessage ->
if (pendingMessage.totalDeliveryCount < maxDeliveryAttempts && pendingMessage.elapsedTimeSinceLastDelivery > Duration.ofMillis(pendingTimeout.toLong())) {
return#filter true
} else {
...
return#filter false
}
}.map { //map from PendingMessage::class to a MapRecord with the content
val map = stringRedisTemplate.opsForStream().range(redisStreamName, Range.just(it.idAsString)) // does not increase totalDeliveryCount !!!
if (map != null && map.size > 0) {
return#map map[0]
} else {
return#map null
}
}.filterNotNull().toList()
My second approach used the pending and read operations. For the read operation I can specify an offset with the current ID. The problem is that I only get IDs back which are higher then the one specified.
val pendingMessages = stringRedisTemplate.opsForStream().pending(redisStreamName, Consumer.from(redisConsumerGroup, instanceName))
return pendingMessages.filter { pendingMessage ->
if (pendingMessage.totalDeliveryCount < maxDeliveryAttempts && pendingMessage.elapsedTimeSinceLastDelivery > Duration.ofMillis(pendingTimeout.toLong())) {
return#filter true
} else {
...
return#filter false
}
}.map { //map from PendingMessage::class to a MapRecord with the content
val map = stringRedisTemplate.opsForStream<String, Any>()
.read(it.consumer, StreamReadOptions.empty().count(1),
StreamOffset.create(redisStreamName, ReadOffset.from(it.id)))
if (map != null && map.size > 0 && map[0].id.value == it.idAsString) { // map[0].id.value == it.idAsString does not match
return#map map[0]
} else {
return#map null
}
}.filterNotNull().toList()
So when I use ReadOffset.from('1234-0') I don't get the message with 1234-0 but everything after that message. Is there a way to get the exact message and also honoring the totalDeliveryCount and elapsedTimeSinceLastDelivery statistic?
I'm using spring-data-redis 2.3.1.RELEASE
I'm using following workaround now, which should be good for most cases:
return if (id.sequence > 0) {
"${id.timestamp}-${id.sequence - 1}"
} else {
"${id.timestamp - 1}-99999"
}
It relies on the fact that there are not more then 99999 messages inserted per ms.
I have three collections.
SalaryCollection
BonusCollection
Deduction Collection
All of them have date which is common in some of them.
I want to merge these three into one collection in a way that object with same date in three becomes one as a result.
Something like this:
#items:array:2[
0=>{
+"date":"1-2020"
+"salaries":36500.0
+"deductions":1500.0
+"bonuses":7000.0
}
1=>{
+"date":"2-2020"
+"salaries":20000.0
+"deductions":1000.0
+"bonuses":5000.0
}
]
How can i do it?
I am not sure if this is the best way to do it but this is how i made it worked.
$salaryCollection = $salaryCollection->map(function ($item, $key) use ($bonusCollection) {
$single_bonus = $bonusCollection->where('date', $item->date);
if (!$single_bonus->isEmpty()) {
return collect($item)->put('bonuses', $single_bonus->first()->bonuses);
} else {
return collect($item)->put('bonuses', 0);
}
});
$salaryCollection = $salaryCollection->map(function ($item, $key) use ($deductionCollection) {
$single_deduction = $deductionCollection->where('date', $item['date']);
if (!$single_deduction->isEmpty()) {
return collect($item)->put('deductions', $single_deduction->first()->deductions);
} else {
return collect($item)->put('deductions', 0);
}
});
The below code will look for "=" and then split them. If there's no "=", filter them away first
myPairStr.asSequence()
.filter { it.contains("=") }
.map { it.split("=") }
However seeing that we have both
.filter { it.contains("=") }
.map { it.split("=") }
Wonder if there's a single operation that could combine the operation instead of doing it separately?
You can use mapNotNull instead of map.
myPairStr.asSequence().mapNotNull { it.split("=").takeIf { it.size >= 2 } }
The takeIf function will return null if the size of the list returned by split method is 1 i.e. if = is not present in the string. And mapNotNull will take only non null values and put them in the list(which is finally returned).
In your case, this solution will work. In other scenarios, the implementation(to merge filter & map) may be different.
I see your point and under the hood split is also doing an indexOf-check to get the appropriate parts.
I do not know of any such function supporting both operations in a single one, even though such a function would basically just be similar to what we have already for the private fun split-implementation.
So if you really want both in one step (and require that functionality more often), you may want to implement your own splitOrNull-function, basically copying the current (private) split-implementation and adapting mainly 3 parts of it (the return type List<String>?, a condition if indexOf delivers a -1, we just return null; and some default values to make it easily usable (ignoreCase=false, limit=0); marked the changes with // added or // changed):
fun CharSequence.splitOrNull(delimiter: String, ignoreCase: Boolean = false, limit: Int = 0): List<String>? { // changed
require(limit >= 0, { "Limit must be non-negative, but was $limit." })
var currentOffset = 0
var nextIndex = indexOf(delimiter, currentOffset, ignoreCase)
if (nextIndex == -1 || limit == 1) {
if (currentOffset == 0 && nextIndex == -1) // added
return null // added
return listOf(this.toString())
}
val isLimited = limit > 0
val result = ArrayList<String>(if (isLimited) limit.coerceAtMost(10) else 10)
do {
result.add(substring(currentOffset, nextIndex))
currentOffset = nextIndex + delimiter.length
// Do not search for next occurrence if we're reaching limit
if (isLimited && result.size == limit - 1) break
nextIndex = indexOf(delimiter, currentOffset, ignoreCase)
} while (nextIndex != -1)
result.add(substring(currentOffset, length))
return result
}
Having such a function in place you can then summarize both, the contains/indexOf and the split, into one call:
myPairStr.asSequence()
.mapNotNull {
it.splitOrNull("=") // or: it.splitOrNull("=", limit = 2)
}
Otherwise your current approach is already good enough. A variation of it would just be to check the size of the split after splitting it (basically removing the need to write contains('=') and just checking the expected size, e.g.:
myPairStr.asSequence()
.map { it.split('=') }
.filter { it.size > 1 }
If you want to split a $key=$value-formats, where value actually could contain additional =, you may want to use the following instead:
myPairStr.asSequence()
.map { it.split('=', limit = 2) }
.filter { it.size > 1 }
// .associate { (key, value) -> key to value }
I'm having a page that contains several dropdownlists, all filled with the same values. I would like to compare them on the client as well as on the server side.
The problem is though, that the dropdownlists are generated dynamically because their quantity can vary.
Client side comparing:
<script type="text/javascript">
function CompareSelectedValues(dropDown1ID, dropDown2ID) {
var DropDownList1 = document.getElementById(dropDown1ID);
var DropDownList2 = document.getElementById(dropDown2ID);
if (DropDownList1.selectedIndex != -1 && DropDownList2.selectedIndex != -1) {
if (DropDownList1.options[DropDownList1.selectedIndex].value != DropDownList2.options[DropDownList2.selectedIndex].value)
alert('not same');
}
}
</script>
Classic server side comparing with C#:
private bool AreDropDownListValuesEqual(DropDownList ddlist1, DropDownList ddlist2)
{
// Check for invalid input or different number of items for early return
if (ddlist1 == null || ddlist2 == null || ddlist1.Items.Count != ddlist2.Items.Count)
{
return false;
}
// Check items one by one. We need a nested loop because the list could be sorted differently while having the same values!
foreach (ListItem outerItem in ddlist1.Items)
{
bool hasMatch = false;
foreach (ListItem innerItem in ddlist2.Items)
{
if (innerItem.Value == outerItem.Value && innerItem.Text == outerItem.Text)
{
hasMatch = true;
break;
}
}
if (!hasMatch)
{
return false;
}
}
// All items from ddlist1 had a match in ddlist2 and we know that the number of items is equal, so the 2 dropdownlist are matching!
return true;
}
What kind of comparison do you need? If you don't keep them in a List and that list in Session, you can never do anything with them since you add them dynamically. Add your dropdownlists where you create them (this should me when Page.IsPostBack == false) and keep that list in session. On postbacks, load your dropdowns from the list. You can compare them using the list you keep.