Redis Secondary Indicing maintenance add,remove are fine what about updates - redis

Question

All articles and write ups Ive read all talking about insertion only, and i have a problem.
I have implemented a secondary index mechanism using transactions (MULTI) and sets in order to save time when looking up entities, the index is saved by a set names by property name and value.
say we have a Person
Person Jack = new Person() { Id = 1, Name = Jack, Age = 30 }
Person Jena = new Person() { Id = 2, Name = Jena, Age = 30 }
when I choose to index the Age property and insert both, I look up the Age of 1 and 2 prep them and insert them to the corresponding set and update the index in the same transaction.
age_30_index holds ids 1,2
when deleting Jack, I prep Jacks age, 30, and remove id 1 from age_30_index and remove Jack from its set again within one transaction, all great, well .. almost.
the problem start when I want to change the Age and update the cache, look at the following scenario:
var p = GetEntity<Person>(id: 1)
p.Age = 31
UpdateEntity(p)
now with the concept above, i will have age_30_index -> 1,2 and age_31_index -> 1
that is because when updating the entity in cache I don't know what is the value of the property stored in cache.. therefore cant remove it from the index.
another problem is deleting by Id, or deleting like this:
var p = GetEntity<Person>(id: 1)
p.Age = 31
DeleteEntity(p)
An easy solution would be using a distributed lock and lock by entity name, get the entities from cache, delete the indexes and continue, but with tests I ran, the performance is lacking.
any other option I thought about is not thread safe because its not atomic.
Is there any other way to achieve what I'm tying to do?
the project is c# .net framework with redis on windows, redisearch.io seems nice but its out of scope unfortunately.

Answer 1

The Hard Way
If RediSearch is really something you don't want to get into because you are running Redis on Windows - my big comment is that you are using the wrong data structure for storing age, numerics generally should be stored in sorted sets so you can query them more easily
so for your age you would have the sorted set Person:Age and when you add Jack (let's say Jack's Id is 1 and age is 30 like your example) you would set your index like so:
ZADD Person:Age 30 1
then when you need to update Jack's age to 31, all you would need to do is update the member in the sorted set:
ZADD Person:Age 31 1
you'd then be able to query all the 31 year olds with a ZRANGE which in StackExchange.Redis looks like:
db.SortedSetRangeByScore("Person:Age", 31, 31, Exclude.None);
that's it, no need for you to look it up in an older collection and a newer one.
Your bigger concern about atomicity should be surrounding if you need to scale out redis. If you need to do that, there's no good atomic way to update a distributed index without RediSearch. That's because all the keys in a multi-key operation or transaction have to be in the same slot, or redis will not accept the operation. Because you are on windows I'm assuming you are not using a clustered multi-shard environment and that you are either stand-alone or sentinel.
Assuming you are running stand-alone or sentinel you can run the updates inside of a lua script which would allow you to run all of your commands sequentially basically your script would update your entity and then update all the accompanying indexes so if you have a sorted set for age in you example Person:Age your script to update that would look something like:
redis.call('HSET', KEYS[1], ARGV[1])
redis.call('ZADD', 'Person:Age', ARGV[1], KEYS[1])
naturally, you will have to change this script out with whatever the reality of your index is, so you'll probably want to do some kind of dynamic script generation. I'd use scripting over MULTI because MULTI on StackExchange.Redis has slightly different behavior than what's expected by redis due to it's architecture.
The best way with RediSearch and Redis.OM
The best way to handle this would be to use RediSearch (I'm putting this part second even though it really is the right answer because it sounds like something you want to avoid, given your environment). With RediSearch + Redis.OM .NET you would just update the item in question and call collection.Save() so for your example here - the following does everything you're looking for (insert, retrieve, update, delete)
using Redis.OM;
using TestRedisOmUpdate;
var provider = new RedisConnectionProvider("redis://localhost:6379");
provider.Connection.CreateIndex(typeof(Person));
var collection = provider.RedisCollection<Person>();
//insert
var id = await collection.InsertAsync(new() {Name = "Jack", Age = 30});
//update
foreach (var person in collection.Where(p => p.Name == "Jack"))
{
person.Age = 31;
}
await collection.SaveAsync();
//check
foreach (var person in collection.Where(p => p.Name == "Jack"))
{
Console.WriteLine($"{person.Name} {person.Age}");
}
//delete
provider.Connection.Unlink(id);
that all would work off of the model:
[Document]
public class Person
{
[Indexed]
public string Name { get; set; }
[Indexed]
public int Age { get; set; }
}
And you don't have the headache or hassle of maintaining the indexes yourself.