I’ve hit sort of a roadblock in a current project I’m working on, I don’t have a lot of web developers in my office and as a matter in fact the only other web dev just went on vacation. Anyway I was wondering if anyone could help me with structuring two of my postgres tables.
The user needs to be able to create custom data tables, one for each specific program (a parent record). The form I’ve setup for these tables allows you to add or remove inputs based on how many fields you need and then specify the name, data_type, etc.
My initial idea was to create a new table in the dB each time a user created one of these custom tables. The other web dev, who has created something similar, said it would be better to create a fields table that stores each custom field information and then have a data table that stores every cell of data tying to a field id.
I understand having the fields table so that I can retrieve just the field information and build my front-end tables and edit forms dynamically, but I’m a little confused on how to get the data into the table. I’m used to having an array of objects and each object relating to an entire row. But with this method it’s storing each cell of data instead of row of data and I don’t know the best way to select and organize it on the backend.
Data for these tables are going to be imported in from CSV files formatted to the custom table structure, below is the current structure I have for my two tables. I got a suggestion on reddit to use JSON to store each rows data, but I'm wondering how I'll be able to do sorting and filtering with this data. My current table structure is listed below, and this is before I got the suggestion to use the json data. I'm guessing if I went that route I would remove the fieldId column and instead use it for
the JSON key name, and store that fields data with it.
fields
id -- name -- program_id -- type -- required -- position -- createdAt -- updatedAt
data
id -- fieldId -- data -- createdAt -- updatedAt
So I guess my question is does this sound like the right way to structure these tables for my needs and if so can I still perform sorting and filtering on it?
I have used this type of data storage in a VBA bsed application, storing a record, where one of the fields was an array. Is this possible on a SQL table?
eg. I need to store data relating to customers and their assets. Each client has their own list of assets. I would use a second joined table, but then each customer would require their own new table.
Is it possible to store this in an array within the original table?
If RDMS, you can either use
1) XML data type or JSON to maintain the asset information against each client
2) OR create a separate table for assets and link to the client table.
It depends on individual use case and context.
1) if you are having only 1 or 2 assets to be maintained against each client. then XML/ JSON would be ideal.
2) if you maintaining high volumes of assets against each client, then the creation of separate table is ideal.
3)If you are unsure of the volume, then a separate table for an asset is ideal.
If I have multiple key value pairs in Azure Blob Storage such as:
-/files/key1
-/files/key2
-/files/key3
And each key is uploaded by a user, but a user can upload multiple keys, what is the best table design in my SQL database to reference what keys are associated with what user?
A) Table with single column - Everytime I add a file to BLOB storage I add a row to a single column table with the username and the associated key value i.e:
AssociationColumn
-User1+key1
-User2+key2
-User1+key3
Will this be slow in looking up all the keys for User1 for example if I query using some sort of regex starts with? Will making this two column with User as one column and key as another column affect performance at all? How can I achieve this one to many relationship?
Also is it bad to store keys using an identifier such as 1-2-n? Any suggestions on how to create unique identifiers that can fit in the space of varchar(MAX)?
The correct approach in a relational database is to have a junction table. This would have at least two columns:
User
Key
You wouldn't put these in a single column -- I don't think even in Azure.
I'm new to databases and I'm thinking of creating one for a website. I started with SQL, but I really am not sure if I'm using the right kind of database.
Here's the problem:
What I have right now is the first option. So that means that, my query looks something like this:
user_id photo_id photo_url
0 0 abc.jpg
0 1 123.jpg
0 2 lol.png
etc.. But to me that seems a little bit inefficient when the database becomes BIG. So the thing I want is the second option shown in the picture. Something like this, then:
user_id photos
0 {abc.jpg, 123.jpg, lol.png}
Or something like that:
user_id photo_ids
0 {0, 1, 2}
I couldn't find anything like that, I only find the ordinary SQL. Is there anyway to do something like that^ (even if it isn't considered a "database")? If not, why is SQL more efficient for those kinds of situations? How can I make it more efficient?
Thanks in advance.
Your initial approach to having a user_id, photo_id, photo_url is correct. This is the normalized relationship that most database management systems use.
The following relationship is called "one to many," as a user can have many photos.
You may want to go as far as separating the photo details and just providing a reference table between the users and photos.
The reason your second approach is inefficient is because databases are not designed to search or store multiple values in a single column. While it's possible to store data in this fashion, you shouldn't.
If you wanted to locate a particular photo for a user using your second approach, you would have to search using LIKE, which will most likely not make use of any indexes. The process of extracting or listing those photos would also be inefficient.
You can read more about basic database principles here.
Your first example looks like a traditional relational database, where a table stores a single record per row in a standard 1:1 key-value attribute set. This is how data is stored in RDBMS' like Oracle, MySQL and SQL Server. Your second example looks more like a document database or NoSQL database, where data is stored in nested data objects (like hashes and arrays). This is how data is stored in database systems like MongoDB.
There are benefits and costs to storing data in either model. With relational databases, where data is spread accross multiple tables and linked by keys, it is easy to get at data from multiple angles and aggregate it for multiple purposes. With document databases, data is typically more difficult to join in single queries, but much faster to retrieve, and also typically formatted for quicker application use.
For your application, the latter (document database model) might be best if you only care about referencing a user's images when you have a user ID. This would not be ideal for say, querying for all images of category 'profile pic' or for all images uploaded after a certain date. You could probably accomplish your task with either database type, and choosing the right database will always depend on the application(s) that it will be used for, but as a general rule-of-thumb, relational databases are more flexible and hard to go wrong with.
What you want (having user -> (photo1, photo2, ...)) is kind of an INDEX :
When you execute your request, it will go to the INDEX and fetch the INDEX "user" in the photos table, and get the photo list to fetch. Not all the database will be looked up, it's optimised.
I would do something like
Users_Table(One User - One Photo)
With all the column that every user will have. if one user will have only one photo then just add a column in this table with photo_url
One User Many Photos
If one User Can have multiple Photos. then create a table separately for photos which contains only UserID from Users_Table and the Photo_ID and Photo_File.
Many Users Many Photos
If One Photo can be assigned to multiple users then Create a Separate table for Photos Where there are PhotoID and Photo_File. Third Table User_Photos which can have UserID from Users_Table and Photo_ID from Photos Table.
Hey all i'm getting hung up on how to search a database for a specific key pair. What i'm doing is sending a combine array of objects that represent data not sent for that user for the day. Here is the basic logic
Unless the database contains the object_id and user_id
insert it into the array
The association is many objects to many users.
Is their a way to search the table cases_users so that it is a query like this
unless cases_users.contains? {object_id, user_id}
objectarray.push(object_id)
I think you want
objectarray.push(object_id) unless User.find_by_object_id_and_user_id(object_id, user_id)