I managed database sharding using sequelize and express app like below,
Create connections on bootup.
Used hash key based(hash ring) algorithm to get the partition on insert/query data.
e.g model.getPartition(userId).get({...})
Looking for the suggestion to handle it using NestJS/Typescript/TypeORM.
Related
I have an existing Mongo DB that was created and managed using PyMongo. I would like to implement MongoEngine and use it from now on on the existing data.
What would be the best way to approach it? Is it possible to migrate the current DB structure to and ODM based approach using MongoEngine?
I would like to know, is there a way to do CRUD operations using non primary key for a cache defined in grid gain using thin client API (Not by using sql query).
Without using SQL, Ignite is basically a key-value store. You could use a ScanQuery, which iterates over every record in the named cache.
But really, the answer is to use SQL.
I need advice for design pattern how to use RabbitMQ to select data from Database.
RabbitMQ looks very good solution for inserting and updating data into Database but what about selecting data from DB?
In my case I have REST API module and Database module connected to Maria DB which communicate via queues.
REST API module -> Database module -> Maria DB
But I need to select configuration from database via database module. I can use RPC as a solution but probably there is better way?
Can you advice?
In general, some sort of RPC is the way to go.
However: The point of a queue (asynchronous tasks) is the opposite of a database select (return my data now). If the direct database select requests are performing adequately, use them, avoid the extra complexity. Or some caching system for your config. This might not work for your system architecture and load needs, but is simpler.
I have a project in which i need to replace the SQL DB with REDIS. Its a job scheduling system. There are tables like JobInfo, TaskInfo, Result, BatchInfo etc.
What is the best way to map DB tables in REDIS server key value pair?
There are join and group by kind of queries used in the project.
What is the best way to replace the sql server with the redis server? Also does redis provides a way with which i can query the data like i can in join and group by queries?
Redis is basically a key-value store (a bit more sophisticated than just a simple one, but yet - a key-value db). the value may be a document that follows some schema, but Redis isn't optimized to search for those documents and query them like other Document Databases or like relational database such as SQL Server.
I dont know why you're trying to migrate from SQL Server to Redis, but you need to re-check yourself if that's the right design choice. If you need fixed schema and join operations - it may suggest that Redis isn't the right solution.
If all you're looking for is caching, you can cache in the application layer, or use other solution to integrate your Redis and SQL Server (I wrote simple open-source project that does that: http://redisql.ishahar.net ).
Hope this helps.
I guess its not possible though you can see below post to implement JOIN like feature in Redis.
Can we take join in Redis?
Please refer below post as well:
Redis database table desing like sql?
I'm currently developing a service for an App with WCF. I want to host this data on windows-azure and it should host data from differed users. I'm searching for the right design of my database. In my opinion there are only two differed possibilities:
Create a new database for every customer
Store a customer-id to every table (or the main table when every table is connected via entities)
The first approach has very good speed and isolating, but it's very expansive on windows azure (or am I understanding something of the azure pricing wrong?). Also I don't know how to configure a WCF- Service that way, that it always use another database.
The second approach is low on speed and the isolating is poor. But it's easy to implement and cheaper.
Now to my question:
Is there any other way to get high isolation of data and also easy integration in a WCF- service using azure?
What design should I use and why?
You have two additional options: build multiple schema containers within a database (see my blog post about this technique), or even better use SQL Database Federations (you can use my open-source project called Enzo SQL Shard to access federations). The links I am providing give you access to other options as well.
In the end it's a rather complex decision that involves a tradeoff of performance, security and manageability. I usually recommend Federations, even if it has its own set of limitations, because it is a flexible multitenant option for the cloud with the option to filter data automatically. Check out the open source project - you will see how to implement good separation of customer of data independently of the physical storage.