I'm designing database for an mobile-based ecom platform.
Currently, the system have one and only one shop. The design for the database of the system has the tables below:
User
Product
Category
Review
Order
Now, I want to scale up the system so that it can support multiple shops. User can sign-up as a seller and create their own shop, manage their own products and orders.
How can I design such a database from the original database which was designed only for one shop?
I have two options in mind but I have no idea whether they will works or not:
For each record in the table, I add a field shopId that ref to the id of the shop it belongs to. Then I will index this field to increase query's performance.
For each shop, I create a new collection/table to store the data that belongs to it. For example: shop1_product, shop1_order,... are the tables i will create for the shop1.
Are the approach above valid? Is there any other better approach.
P/s: I'm using MongoDB, and the system doesn't require operations across many shops.
Thank you guys!
Related
Suggest me the best way to do relationship in Microsoft Access,
For example: I have 10 user, each user has 10 products total 100 products, and each product has 10 orders total 1000 order,
How can I show each user only their products and each product only its orders?
I don't have any experience with MS Access
How you structure this data will not only depend on the specific data you will be storing, but also on how you will be accessing it. You will need to keep in mind things like, how often you will be writing data into Firestore vs how often you will be reading it; will fields of one object (e.g. Order) be shown when showing other objects (User/Product), how will you query the data?
I suggest you check this documentation, which compares pros and cons of different ways of structuring data and use cases in which they are recommended.
We have a need to know which database architecture makes more sense to use and why.
We have a list of customers who are all going to use the same table structure (with very few exceptions).
We would have about 10 thousand customers who might all have all about 50 thousand products each.
The processing on products may not be the same for each customer and we would also want to provide a plan where customers could have API access to their data.
Our customers do sell products and their SQL table structure would all have columns such as :
Feed_ID
Product_ID
Product_Description
Price
Weight
etc...
The Feed_ID is used to differentiate the origin of these products and will be unique for each customer - of course.
The 3 choices of relational table structure that we have thought about:
Each customer has its own database and in that database, he has 1 table per product-feed
All customers are hosted under 1 unique database under which all customers all have 1 table per feed - in that case, 1 customer can have 2 tables if he as 2 different product feed.
All customers are hosted under 1 unique database, HOWEVER, in this 3rd solution, we only have 1 unique table that host all products feed of all customers.
Which solution would you use and why you think that the solution you selected is better?
Thank you.
You haven't quite provided enough information. Under almost all circumstances (see below for exceptions), you want one set of tables for all customers. Here are some reasons:
Performance. A proliferation of tables means the data is spread through more data pages, so you have lots of partially filled data pages. The database is bigger and processing is slower.
Coding efficiency. If the tables for a customer all have different names, then all the code is dynamic SQL. That is harder to maintain.
Maintenance. Adding a column or index is very arduous when there are zillions of similar tables.
Analytics. When similar data is spread through tables, it is really hard to answer questions such as "Which client has the most products?".
Security. Granting access permissions on a single set of tables is less error prone than on zillions of tables.
And no doubt, I've missed a few reasons. You can see that it is almost a no-brainer to have a single database with a small number of tables.
There are situations where separate databases might be called for. I cannot think of a good reason to have separate tables for each client in a single database.
The number one reason would be security and isolation. There might be a business or even legal reason for storing data into "physically" separate databases, to further minimize the possibility of one client seeing another client's data (accidentally or through hacking).
Another reason would be if clients had bespoke solutions. That is, there are per-client customizations. I would still be inclined to try to put this into a single database solution, but that might not be possible.
Related to this would be an application that you intend to support both in the cloud and on premises. In that case, separate databases per client would probably simplify the application design.
But, in general, you would store the data in a pretty normalized single database, with one table per entity.
I think having separate tables (or ideally schemas) for each customer is not that bad idea. In addition to benefits you mentioned, this way you can scale your database easily, and you can give customers full control of their data if they want to.
Regarding the drawbacks:
Managing it is more complicated but not as bad either - you can write
a script to create columns/tables/indexes/etc. You
don't have to do it manually.
It will be a challenge to perform analytics on 10K tables,
although it's not the best idea to mix it with production anyway.
I'd create a separate database (or server) for analytics, running
some overnight job to update reporting tables.
Also, if your table is going to have hundreds of millions rows (10Kx50k?), it's a good idea to split it into smaller pieces, regardless which option you'll choose. If not by customer, then by region or some other bigger group (assuming you are building on premises RDBMS)
I am working for a big online store. At the moment our architecture is something weird where we have microservices which actually all share the same DB (doesn't work well at all...).
I am considering improving that but have some challenges on how to make them independant.
Here is a use case. I have customers, customers purchase products. Let say I have 3 microservices : customer authentication, order management, product management.
An order is linked to a customer and a product.
Could you describe a solution for the following problems :
How do you make the link between an order and a customer?
Let say both services share a customer ID, how do you handle data consistency? If you remove a customer on the customer service side, you end up with inconsistency. If your service has to notify the other services then you end up with tighlty coupled services which to me sounds like what you wanted to avoid in the first place. You could kind of avoid that by having an event mechanism which notify everyone but what about network errors when you don't even know who is supposed to receive the event?
I want to do a simple query : retrieve the customers from US that bought product A. Given that 3million people bought product A and we have 1 million customers in the US; How could you make that reasonably performant? (Our current DB would execute that in few milliseconds)
I can't think of any part of our code where we don't have this kind of relation. One of the solution I can think of is duplicating data. E.g. When a customer purchase something, the order management service will store the customer details and the product details. You end up with massive data replication, not sure if that's a good thing and I would still be worried about consistency.
I couldn't find a paper addressing those issues. What are the different options?
At the moment our architecture is something weird where we have microservices which actually all share the same DB (doesn't work well at all...). I am considering improving that but have some challenges on how to make them independant.
IMHO the architecture is more simple by having one OLTP database for orders, customers, and products since it allows you to make use of JOINS and stored procedures. It could be the case that the DB could use some configuration and tuning TLC vs. software re-architecture. Just keep that door open when you consider how to fix performance problems.
How do you make the link between an order and a customer?
In the orders table have a column for customer_id. The customer_id field in the orders table would be a foreign key to the id field on the customers table. This will give you the best performance.
You can do either periodic cleanup or event based cleanup of deleted users (and their orders). But please make sure that somewhere these old orders and customers are stored. Maybe archive tables or back-end data-warehouse where reports and analysis (OLAP) can be done on this data.
Let say both services share a customer ID, how do you handle data consistency? If you remove a customer on the customer service side, you end up with inconsistency. If your service has to notify the other services then you end up with tighlty coupled services which to me sounds like what you wanted to avoid in the first place. You could kind of avoid that by having an event mechanism which notify everyone but what about network errors when you don't even know who is supposed to receive the event?
There are various ways this can be done. As mentioned you can either create an event to deal with customer deletions or do periodic db cleanups. But one thing is for certain, the orders service does not NEED to be notified when this cleanup is done, unless you want it to. Not a need but could be a want if you want order culling to be done via the order services. The naive way to do this is to create a stored procedure that takes a customer_id (or list of customer_id's) as input and deletes all orders that match that customer_id from the orders table. Please make sure to backup the data for future data analysis and auditing.
I want to do a simple query : retrieve the customers from US that bought product A. Given that 3million people bought product A and we have 1 million customers in the US; How could you make that reasonably performant? (Our current DB would execute that in few milliseconds)
Again this is why it makes sense to keep the customers, products, and orders tables in the same DB as this query can more easily be made to execute quickly when they are on the same DB. You can take advantage of your DB's designing and optimization tools, and EXPLAIN/DESCRIBE output to tweak your tables indexes and such. If you are using Mysql you can change DB engines (I recommend TokuDB DB engine).
In the end my main suggestion is to leave in one DB for OLTP as you will get more efficiency and performance for the same amount of hardware. Splitting the DB into multiple DB's will have an overhead cost for your code, architecture, network, and CPU's. The important thing is that your DB can scale horizontally and is finely tuned for the queries being done on it. Move OLAP to its own DB. This can be done using ETL to move data from OLTP DB to OLAP DB. The query in your example sounds like something that would be done in an OLAP DB. For your OLAP database you can use a columnar DB, like Vertica or something equivalent that can easily scale horizontally. The important thing to note is that by splitting up your OLAP and OLTP you can tune and configure each for their respective purpose.
Whether you run your customer, orders, and products services as a monolith (my recommendation) or as microservices the DB design should not change. What will cause your queries in your code to change is if you split the OLTP DB into multiple DB's because now you can not do simple JOINs or stored procedures.
This is what Martin Fowler calls the Monolith First. http://martinfowler.com/bliki/MonolithFirst.html
I'm trying to create a proper database layout for a project I'm working on, however I can't seem to work out which is best.
Basically the "app" is something where a User can be assigned to many Products, and a Product can be have many Customers.
From here, each Customer has a Service, which is specific to that Customer and Product.
A Service can have many Incidents, but an Incident can only be assigned to one Service.
A User can also have Incidents, but an Incident can only have one User.
Here is the two designs I have made for this:
http://i.imgur.com/ZcCFcdg.png
As you can see, the left design has a table specific tables for the Many-Many relationships, where as the right one has a overall pivot table for them all.
I see both of these methods working (in my head) - however since I'm not the best with this, are there any downsides to either of these methods? And do you see any problems I'll run into, in the future?
Also, is the right one even a proper way of doing it?
I'm also going to be using the Eloquent ORM.
Look up 'Third Normal Form'. This gives rules on how to design tables. You could take one or other of your current designs and apply the three rules to see where you get to.
I would say the one on the right is wrong: too much duplicate information, and unclear relationships. The one on the left is OK, but the two Many-Many tables are redundant. You know a Product and Customer are linked because it's in the Service table. You know a User and Product are linked because its in the Incident + Service tables.
Cheers -
After clarification, a User belongs to a Customer. In that case, add CustomerID to the User table - this is important data about the user and should be included. It will allow you to stop the User raising Incidents on Products not associated with the Customer.
This also will enable you to list the Products associated with the User via the Customer the User is associated with.
Further, the Customer should have it's ID on Product, and Service should have the ProductId, not the CustomerID, as the Service is associated with a Product and only with a Customer via the Product it is associated with.
I am building a POS system that support multi branches.
The system is going to support these features.
Each store should have a local database for it's own inventory list and invoice. (Local database to avoid internet failure).
There is a reporting DB that contains information of all shop (inventory, invoice, etc), the reporting DB can be async to shopDB.
Each shop contains a unique shop code to indentify the record ownership, also a part of key (to avoid issue with primary key).
Shop system can query Reporting DB for inventory list on other shop (customer can place an order, shop may query for full inventory list and get other branches ship them the inventory).
Currently I am building the system with Java, PostgreSQL and Cayenne, but I am open to change the DB or ORM tool in case there is any technology limitation.
I tried to read a lot with Replication and Clustering, but it doesn't appears to suit my need.
Any clues on what I should look for ? Or should I build the replication on app layer instead of DB layer ?
The thing that strikes me here is what happens when shop A sells inventory for shop B while shop B sells the same inventory?
why cant the application access other shops dbs?
have you read about federated databases - http://dev.mysql.com/doc/refman/5.1/en/federated-description.html
http://en.wikipedia.org/wiki/Federated_database_system