How can a single instance of Redis be used in a multi-tenant environment. Meaning multiple and different applications using the same Redis instance.
Suppose I have two apps, one is Baking App and the other one is Delivery App. Both apps will be using the same Redis instance and both apps will be saving similar keys with similar key patterns (e.g. userid:uuid -> johnsmith) etc. Obviously, using the same Redis will have collisions, is there a way to "Namespace" the database as such even the same key will be isolated from each other allowing multiple apps to use the same Redis instance concurrently?
And also work with Redis search, and in the same way, search and indexing would be isolated from each app. So if the search is on the Delivery App namespace it would not fetch anything from the Baking App namespace.
How can this be achieved?
So there are multiple things that you can do:
You can prefix the keys with app name like app1:userid:uuid etc
You can use different in memory db provided by redis. Redis supports upto 16 DBs. You can store keys for different apps in different db. To fetch them connect with respective DB.
You can use both of the above methods.
To improve security so that the Apps cannot access other App's data:
Implement Redis ACL - If you are using Redis version 6+, you can leverage the feature of using ACL(Access Control Lists). You can create users with passwords for each app and pass these credentials while making Redis connection. You can even add permissions/commands etc. to the users.
Data in different DB cannot be accessed i.e. if you make connection to DB 0, you cannot fetch data from DB 1.
Related
I have large number of key-value pairs of different types to be stored in Redis cache. Currently I use a single Redis node. When my app server starts, it reads a lot of this data in bulk (using mget) to cache it in memory.
To scale up Redis further, I want to set up a cluster. I understand that in cluster mode, I cannot use mget or mset if keys are stored on different slots.
How can I distribute data into different nodes/slots and still be able to read/write in bulk?
It's handled in redis client library. You need to find if a library exists with this feature in the language of your choice. For example, if you are using golang - per docs redis-go-cluster provides this feature.
https://redis.io/topics/cluster-tutorial
redis-go-cluster is an implementation of Redis Cluster for the Go language using the Redigo library client as the base client. Implements MGET/MSET via result aggregation.
So I would like to ask if there will be any contention issues due to shared access to the same Redis cluster by multiple separate applications which use Redisson library (each application in turn has several instances of themselves).
Does Redisson library support such use case? Or do I need to configure Redisson in each application, for example add some kind of prefix or app name (as it is possible with Quartz where you can define prefixes for tables used by separate applications having access to the same db and using Quartz independently).
Won't the tasks submitted to ExecutorService in one app be forwarded to completely another application which also uses Redisson and not to another instance of the same application?
I would recommend you to use prefix/suffix in Redisson's object names when you share same Redis setup in cluster mode across multiple independent applications.
Scenario: Two instances of an application share the same redis instance, but use different databases. The application makes use of the redis pub/sub functions to exchange data between services.
Problem: When application instance A publishes something (on redis database 1), application instance B (running on redis database 2) receives the message.
Expectation: As both instances of the application use a different database, I would expect not only that the keys in redis are hold separately, but pub/sub subscribers aswell.
Question: Can I tell redis to keep pub/sub separate for each database?
No - PubSub is shared across all clients connected to the server, regardless of their currently SELECTed database (shared database/numbered database/keyspace). While you can use different channels and such, real separation is possible only by using two Redis instances.
Note: using shared/numbered databases isn't recommended - always use dedicated Redis instances per app/service/use case
As https://redis.io/docs/manual/pubsub/#database--scoping suggests
If you need scoping of some kind, prefix the channels with the name of
the environment (test, staging, production...).
I currently have some different project that works on different redis instance ( consider the sample where I've 3 different asp.net application that are on different server each one with its redis server).
We've been asked to virtualize and to remove useless instances so I was wondering what happens if I have only one redis server and all the 3 asp.net points to the same redis instance.
For the application key I think there's no problem, I can prefix my own key with the application name , for example "fi-agents", "ga-agents", and so on... but I was wondering for the auth session what happens?
as far as I've read the Prefix is used as internal and it can't be used by final user to separate... it's just enought to use different Db?
Thanks
Generally and unless there are truely compelling reasons, you don't want to mix different applications and their data in the same database. Yes, it does lower ops costs initially but it can quickly deteriorate to scaling and performance nightmare. This, I believe, is true for any database.
Specifically with Redis, technically yes - you could use a key prefix or the shared/numbered database approach. I'm not sure what you meant by "auth" sessions but you can probably apply the same approach to them. But you really shouldn't... since Redis is a single-threaded process you can end up where one of the apps is blocking the other two. Since Redis by itself is so lightweight, just spin up dedicated servers - one per app - even in the same VM if you must. You can read more background information on why you don't want to opt for the shared approach here: https://redislabs.com/blog/benchmark-shared-vs-dedicated-redis-instances
I'm creating a mobile app and it requires a API service backend to get/put information for each user. I'll be developing the web service on ServiceStack, but was wondering about the storage. I love the idea of a fast in-memory caching system like Redis, but I have a few questions:
I created a sample schema of what my data store should look like. Does this seems like it's a good case for using Redis as opposed to a MySQL DB or something like that?
schema http://www.miles3.com/uploads/redis.png
How difficult is the setup for persisting the Redis store to disk or is it kind of built-in when you do writes to the store? (I'm a newbie on this NoSQL stuff)
I currently have my setup on AWS using a Linux micro instance (because it's free for a year). I know many factors go into this answer, but in general will this be enough for my web service and Redis? Since Redis is in-memory will that be enough? I guess if my mobile app skyrockets (hey, we can dream right?) then I'll start hitting the ceiling of the instance.
What to think about when desigining a NoSQL Redis application
1) To develop correctly in Redis you should be thinking more about how you would structure the relationships in your C# program i.e. with the C# collection classes rather than a Relational Model meant for an RDBMS. The better mindset would be to think more about data storage like a Document database rather than RDBMS tables. Essentially everything gets blobbed in Redis via a key (index) so you just need to work out what your primary entities are (i.e. aggregate roots)
which would get kept in its own 'key namespace' or whether it's non-primary entity, i.e. simply metadata which should just get persisted with its parent entity.
Examples of Redis as a primary Data Store
Here is a good article that walks through creating a simple blogging application using Redis:
http://www.servicestack.net/docs/redis-client/designing-nosql-database
You can also look at the source code of RedisStackOverflow for another real world example using Redis.
Basically you would need to store and fetch the items of each type separately.
var redisUsers = redis.As<User>();
var user = redisUsers.GetById(1);
var userIsWatching = redisUsers.GetRelatedEntities<Watching>(user.Id);
The way you store relationship between entities is making use of Redis's Sets, e.g: you can store the Users/Watchers relationship conceptually with:
SET["ids:User>Watcher:{UserId}"] = [{watcherId1},{watcherId2},...]
Redis is schema-less and idempotent
Storing ids into redis sets is idempotent i.e. you can add watcherId1 to the same set multiple times and it will only ever have one occurrence of it. This is nice because it means you don't ever need to check the existence of the relationship and can freely keep adding related ids like they've never existed.
Related: writing or reading to a Redis collection (e.g. List) that does not exist is the same as writing to an empty collection, i.e. A list gets created on-the-fly when you add an item to a list whilst accessing a non-existent list will simply return 0 results. This is a friction-free and productivity win since you don't have to define your schemas up front in order to use them. Although should you need to Redis provides the EXISTS operation to determine whether a key exists or a TYPE operation so you can determine its type.
Create your relationships/indexes on your writes
One thing to remember is because there are no implicit indexes in Redis, you will generally need to setup your indexes/relationships needed for reading yourself during your writes. Basically you need to think about all your query requirements up front and ensure you set up the necessary relationships at write time. The above RedisStackOverflow source code is a good example that shows this.
Note: the ServiceStack.Redis C# provider assumes you have a unique field called Id that is its primary key. You can configure it to use a different field with the ModelConfig.Id() config mapping.
Redis Persistance
2) Redis supports 2 types persistence modes out-of-the-box RDB and Append Only File (AOF). RDB writes routine snapshots whilst the Append Only File acts like a transaction journal recording all the changes in-between snapshots - I recommend adding both until your comfortable with what each does and what your application needs. You can read all Redis persistence at http://redis.io/topics/persistence.
Note Redis also supports trivial replication you can read more about at: http://redis.io/topics/replication
Redis loves RAM
3) Since Redis operates predominantly in memory the most important resource is that you have enough RAM to hold your entire dataset in memory + a buffer for when it snapshots to disk. Redis is very efficient so even a small AWS instance will be able to handle a lot of load - what you want to look for is having enough RAM.
Visualizing your data with the Redis Admin UI
Finally if you're using the ServiceStack C# Redis Client I recommend installing the Redis Admin UI which provides a nice visual view of your entities. You can see a live demo of it at:
http://servicestack.net/RedisAdminUI/AjaxClient/