database design in django for 2-rows multiple colums - sql
I am trying to create a model for the following data in django.
I have around 20 columns and they can increase with time but I will only have one row for the values of each column which will update with time. What would be the best way of creating a model for such a data? I cannot transpose the rows into columns because the data is coming from another source which needs the database in the above format.
you can define a model say TheModel (a relevent name, may be). and each column would be one attribute. This way you can add more attributes as the column increases.
According to your description, This model will have only one object. but that is fine right.
class TheModel(models.Model):
mda_table_ndx = models.CharField(max_length=200)
mf_p030_.._.._.. = models.IntegerField()
..............
.............
Let us know if there is a downside to this.
Related
How to populate all possible combination of values in columns, using Spark/normal SQL
I have a scenario, where my original dataset looks like below Data: Country,Commodity,Year,Type,Amount US,Vegetable,2010,Harvested,2.44 US,Vegetable,2010,Yield,15.8 US,Vegetable,2010,Production,6.48 US,Vegetable,2011,Harvested,6 US,Vegetable,2011,Yield,18 US,Vegetable,2011,Production,3 Argentina,Vegetable,2010,Harvested,15.2 Argentina,Vegetable,2010,Yield,40.5 Argentina,Vegetable,2010,Production,2.66 Argentina,Vegetable,2011,Harvested,15.2 Argentina,Vegetable,2011,Yield,40.5 Argentina,Vegetable,2011,Production,2.66 Bhutan,Vegetable,2010,Harvested,7 Bhutan,Vegetable,2010,Yield,35 Bhutan,Vegetable,2010,Production,5 Bhutan,Vegetable,2011,Harvested,2 Bhutan,Vegetable,2011,Yield,6 Bhutan,Vegetable,2011,Production,3 Image of the above csv: Now there is a very small country lookup table which has all possible countries the source data can come with, listed. PFB: I want to have the output data's number of columns always fixed (this is to ensure the reporting/visualization tool doesn't get dynamic number columns with every day's new source data ingestions depending on the varying distinct number of countries present). So, I've to somehow join the source data with the country_lookup csv and populate all those columns with default value as F. Every country column would be binary with T or F being the possible values. The original dataset from the above has to be converted into below: Data (I've kept the Amount field unsolved for column Type having Derived Yield as is, rather than calculating them below for a better understanding and for you to match with the formulae): Country,Commodity,Year,Type,Amount,US,Argentina,Bhutan,India,Nepal,Bangladesh US,Vegetable,2010,Harvested,2.44,T,F,F,F,F,F US,Vegetable,2010,Yield,15.8,T,F,F,F,F,F US,Vegetable,2010,Production,6.48,T,F,F,F,F,F US,Vegetable,2010,Derived Yield,(2.44+15.2)/(6.48+2.66),T,T,F,F,F,F US,Vegetable,2010,Derived Yield,(2.44+7)/(6.48+5),T,F,T,F,F,F US,Vegetable,2010,Derived Yield,(2.44+15.2+7)/(6.48+2.66+5),T,T,T,F,F,F US,Vegetable,2011,Harvested,6,T,F,F,F,F,F US,Vegetable,2011,Yield,18,T,F,F,F,F,F US,Vegetable,2011,Production,3,T,F,F,F,F,F US,Vegetable,2011,Derived Yield,(6+10)/(3+9),T,T,F,F,F,F US,Vegetable,2011,Derived Yield,(6+2)/(3+3),T,F,T,F,F,F US,Vegetable,2011,Derived Yield,(6+10+2)/(3+9+3),T,T,T,F,F,F Argentina,Vegetable,2010,Harvested,15.2,F,T,F,F,F,F Argentina,Vegetable,2010,Yield,40.5,F,T,F,F,F,F Argentina,Vegetable,2010,Production,2.66,F,T,F,F,F,F Argentina,Vegetable,2010,Derived Yield,(2.44+15.2)/(6.48+2.66),T,T,F,F,F,F Argentina,Vegetable,2010,Derived Yield,(15.2+7)/(2.66+5),F,T,T,F,F,F Argentina,Vegetable,2010,Derived Yield,(2.44+15.2+7)/(6.48+2.66+5),T,T,T,F,F,F Argentina,Vegetable,2011,Harvested,10,F,T,F,F,F,F Argentina,Vegetable,2011,Yield,90,F,T,F,F,F,F Argentina,Vegetable,2011,Production,9,F,T,F,F,F,F Argentina,Vegetable,2011,Derived Yield,(6+10)/(3+9),T,T,F,F,F,F Argentina,Vegetable,2011,Derived Yield,(10+2)/(9+3),F,T,T,F,F,F Argentina,Vegetable,2011,Derived Yield,(6+10+2)/(3+9+3),T,T,T,F,F,F Bhutan,Vegetable,2010,Harvested,7,F,F,T,F,F,F Bhutan,Vegetable,2010,Yield,35,F,F,T,F,F,F Bhutan,Vegetable,2010,Production,5,F,F,T,F,F,F Bhutan,Vegetable,2010,Derived Yield,(2.44+7)/(6.48+5),T,F,T,F,F,F Bhutan,Vegetable,2010,Derived Yield,(15.2+7)/(2.66+5),F,T,T,F,F,F Bhutan,Vegetable,2010,Derived Yield,(2.44+15.2+7)/(6.48+2.66+5),T,T,T,F,F,F Bhutan,Vegetable,2011,Harvested,2,F,F,T,F,F,F Bhutan,Vegetable,2011,Yield,6,F,F,T,F,F,F Bhutan,Vegetable,2011,Production,3,F,F,T,F,F,F Bhutan,Vegetable,2011,Derived Yield,(2.44+7)/(6.48+5),T,F,T,F,F,F Bhutan,Vegetable,2011,Derived Yield,(10+2)/(9+3),F,T,T,F,F,F Bhutan,Vegetable,2011,Derived Yield,(6+10+2)/(3+9+3),T,T,T,F,F,F The image of the above expected output data for a structured look at it: Part 1 - Part 2 - Formulae for populating Amount Field for Derived Type: Derived Amount = Sum of Harvested of all countries with T (True) grouped by Year and Commodity columns divided by Sum of Production of all countries with T (True)grouped by Year and Commodity columns. So, the target is to have a combination of all the countries from source and calculate the sum of respective Harvested and Production values which then has to be divided. The commodity can be more than one in the actual scenario for any given country, but that should not bother as the summation of amount happens on grouped commodity and year. Note: The users in the frontend can select any combination of countries. The sole purpose of doing it in the backend rather than dynamically doing it in the frontend is because AWS QuickSight (our visualisation tool), even though can populate sum on selected column filters but doesn't yet support calculation on those derived summed fields. Hence, the entire calculation of all combination of countries has to be pre-populated (very naive approach) in order to make it available in report on dynamic users selection of countries. Also if you've any better approach (than the above naive approach mentioned in note) to solve this problem, you are most welcome to guide me. I've also posted a question on the same problem without writing my expected approach for experts to show me the path on how we can solve this kind of a problem better than this naive approach. If you want to help solve it with some other technique, you're most welcome, here is the link to that question. Any help shall be greatly acknowledged.
Co dependent fields in Odoo 9.0 view
Hello everybody and thanks for taking the time to read my question: I need to make two fields update on each other changes in my view, let's say one of them list's ingredients and the other lists food requirements requirements = fields.Selection([(0,"Vegan"),(1,"Lactose intolerant")]) ingredients_id = fields.Many2Many(comodel_name="sample.ingredients") What i need is to update ingredients_id's domain to show only those who match a requirement and vice versa, if you happend to select and ingredient the other field selecting the proper requirement. I'm thinking about making field A a computed field depending on fieldB and then add an on_change to field B that updates field A. But I'm pretty certain that will turn into some kind of infinite loop situation. Is that my only option or am I missing something obviuos?
Use #api.depends decorator and pass both fields as parameters in the decorator. This will trigger function every time on change on both values and will be dependent on each other.
#api.depends('requirements','ingredients_id')
def function (self): ....
do something....
return
retrive data using multiple table in core data in iPhone
Actually I have three tables as empName(empId,empName),empSalary(empId,salary),empBonus(salary,bonus). And I have a name and need to find the bonus. So My logic is that first find using name I will find the empId then using empId find the salary and finally using salary find the bonus from empBonus table. How can I achieve following things in core data in iPhone.
Use relationships between data, so that whenever you find the name, it also finds the salary and bonus from the other tables. See here I would not do raw sqlite queries if you have this level of knowledge of Core Data and Sql.
First of all you should stop thinking in tables. Core Data is not a wrapper for SQlite. If you want tables don't use Core Data and use SQLite. Core Data is an object graph and persistence framework. You have entities instead of tables. Because you are talking about tables I have no idea what you actually want but your Data Model could look like this: or this: (the Employee->Salary relationship in the last screenshot should be to-many, I forgot to put that into the model.) and because you have an object graph you can get all the objects in that graph. For the last example model your code could look like this: Employee *myEmployee = /* do a fetch to get an Employee */ NSSet *salaries = myEmployee.salaries; Salary *salary = /* choose one from the salaries set */ Bonys = salary.bonus;
What is the best way to fake a SQL array or list?
I'm building a chatroom application, and I want to keep track of which users are currently in the chatroom. However, I can't just store this array of users (or maybe a list would be better) in a field in one of my records in the Chatroom table. Obviously one of the SQL data types is not an array, which leads me to this issue: what is the best way to fake/mock array functionality in a SQL database? It seems there are 3 options: 1: Store the list/array of users as a string separated by commas, and just do some parsing when I want to get it back to an array 2: Since the max amount of users is allowed to be 10, just have 10 extra fields on each Chatroom record representing the users who are currently there 3: Have a new table Userchats, which has two fields, a reference to the chatroom, and a user name I dunno, which is the best? I'm also open to other options. I'm also using Rails, which seems irrelevant here, but may be of interest.
Option 3 is the best. This is how you do it, in a relational schema. It is also the most flexible and future-proof option. It can grow easier in width (extra columns say, a date joined, a channel status, a timestamp last talked) and length (extra rows when you decide there now can be 15 users in a room instead of 10).
The proper way to do this is to add an extra table representing an instance of a user being in a chatroom. In most cases, this is probably what you will want to do, since it gives you more flexibility in the types of queries you can do (for instance: list all chatrooms a particular user is in, find the average number of people in each chatroom, etc.) You would just need to add a new table - something like chat_room_users, with a chat_room_id, and a user_id. If you're deadset on not adding an extra table, then Rails (or more specifically ActiveRecord), does have some functionality to store data structures like arrays in a SQL column. Just set up your column as a string or text type in a Rails migration, and add: serialize :users You can then use this column as a normal Ruby array / object, and ActiveRecord will automatically serialize / deserialize this object as you work with it. Keep in mind that's there are a lot of tradeoffs with this approach - you will never be able to query what users are in a particular room using SQL and will instead need to pull all data down to Ruby before working with it.
Accommodating Dynamic Hierarchies in a Data Warehouse Model
I am building a data warehouse for the company's (which I am working for) core ERP application, for a particular client. Most of the data in the source database, which is related to hierarchies in the data warehouse are in columns as shown below: But traditionally the model to store dimension data according to my knowledge is as: I could pivot the data and fit them in the model shown above. But the issue comes when a user introduces a new hierarchy value. Say for instance the user in the future decides to define a new level called Product Sub Category. Then my entire data warehouse model will collapse without a way to accommodate the new hierarchy level defined. Do let me know a way to overcome this situation. I hope my answer is clear enough. Just let me know if further details are needed.
Well, nothing should collapse -- the ETL should extract and load the data as always. Here are a few options to consider: Simply add one more column for the new hierarchy to the dimProduct. Try using hierarchy helper table. Consider adding path string attribute to the dimProduct.