I want to store a large number of filled-out government forms, like the Application for Federal Assistance. The forms are varied and change yearly. Field types vary, and can be: boolean, string, date, int, among others.
Is the best way to store these forms to completely normalize data?
À la:
form
+-----------------+-----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+-----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| govt_identifier | char(40) | YES | | NULL | |
| description | char(100) | YES | | NULL | |
+-----------------+-----------+------+-----+---------+----------------+
filled_form (a form a person has actually filled out)
+-----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| form_id | int(11) | NO | | NULL | |
| person_id | int(11) | NO | | NULL | |
+-----------+---------+------+-----+---------+----------------+
text_field (a class of input; belongs to a form)
+---------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | char(40) | YES | | NULL | |
| form_id | int(11) | NO | | NULL | |
+---------+----------+------+-----+---------+----------------+
text_value (a particular input record; belongs to a class and filled_form)
+----------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| value | text | YES | | NULL | |
| text_field_id | int(11) | NO | | NULL | |
| filled_form_id | int(11) | NO | | NULL | |
+----------------+---------+------+-----+---------+----------------+
... continue for all input types
While this would work, your SQL will be slightly awkward and quite non-intuitive. Have you considered actually creating data models for each form individually and then using those to populate your forms. It may seem more work up front, but the development of your data capture will potentially be simpler.
I would have a look at single table inheritance.
Model each field as a base class Field with subclasses IntField, BoolField, etc.
The Field class will have a member Name (string), IntField will have IntValue (int), BoolField will have BoolValue (bit), etc.
This requires you to have one column for each possible type in your Field-table, that is a bit space overhead, but on the other hand it gives you type safety. If you model as single table inheritance you can probably hook up your favorite ER-mapper without problem.
Related
Two table are populated from a CSV file :
routers:
+------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| type_id | int(11) | YES | MUL | NULL | |
| identificationID | varchar(255) | NO | | NULL | |
| wep | varchar(255) | NO | | NULL | |
| ssid | varchar(255) | NO | UNI | NULL | |
+------------------+--------------+------+-----+---------+----------------+
and
pickups:
+--------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| type_id | int(11) | YES | MUL | NULL | |
| enable | tinyint(1) | YES | | NULL | |
| name | varchar(255) | NO | | NULL | |
| adress | varchar(255) | YES | | NULL | |
| zip | varchar(255) | YES | | NULL | |
| city | varchar(255) | YES | | NULL | |
| googleGeoCode | varchar(255) | YES | | NULL | |
| agent | tinyint(1) | YES | | NULL | |
| userRestricted | tinyint(1) | YES | | NULL | |
| superUserPickup_id | int(11) | YES | UNI | NULL | |
| telephone | varchar(255) | YES | | NULL | |
+--------------------+--------------+------+-----+---------+----------------+
Because everything comes from Excel, I need a SQL code to populate a third table which is
pickup_has_router :
+-----------+----------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+----------+------+-----+-------------------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| router_id | int(11) | YES | MUL | NULL | |
| pickup_id | int(11) | YES | MUL | NULL | |
| timeStamp | datetime | NO | | CURRENT_TIMESTAMP | |
+-----------+----------+------+-----+-------------------+----------------+
For example, I know that below routers have the pickup ID n°1, and I know how to fetch them
SELECT * FROM `routers` WHERE `identificationID` IN(
'UCDDU17805001464',
'UCDDU17805001029',
'UCDDU17805000137',
'UCDDU17805000129',
'UCDDU17805001394'
)
But from the result of above statment, how to loop the INSERT statment which would be something like:
INSERT INTO `pickup_has_router` (`id`, `router_id`, `pickup_id`, `timeStamp`)
WHERE `identificationID` IN(
'UCDDU17805001464',
'UCDDU17805001029',
'UCDDU17805000137',
'UCDDU17805000129',
'UCDDU17805001394'
)
VALUES (NULL, 'router.id', '1', CURRENT_TIMESTAMP);
Thanks for the help.. I have been searching quite a lot and yes SQL is not my best skill :-))))))
insert into `pickup_has_router` (`router_id`, `pickup_id`)
select `id`, 1 from `routers`
where `identificationID` in(
'UCDDU17805001464',
'UCDDU17805001029',
'UCDDU17805000137',
'UCDDU17805000129',
'UCDDU17805001394'
)
When doing an insert, instead of the values clause you can use a select statement, where the columns returned by the select statement match the columns required by the insert.
I'm trying to salvage a Gitorious installation that has gone bad. I've dumped the SQL table using mysqldump, but now I'm running into the problem that the new version of Gitorious changed its SQL schema in a few places.
In particular, the old version has a table taggings, which looks like
mysql> describe taggings;
+---------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| tag_id | int(11) | YES | MUL | NULL | |
| taggable_id | int(11) | YES | MUL | NULL | |
| taggable_type | varchar(255) | YES | | NULL | |
| created_at | datetime | YES | | NULL | |
+---------------+--------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
In the new version, this table has gotten three extra columns:
mysql> describe taggings;
+---------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| tag_id | int(11) | YES | MUL | NULL | |
| taggable_id | int(11) | YES | MUL | NULL | |
| taggable_type | varchar(255) | YES | | NULL | |
| created_at | datetime | YES | | NULL | |
| tagger_id | int(11) | YES | | NULL | |
| tagger_type | varchar(255) | YES | | NULL | |
| context | varchar(255) | YES | | NULL | |
+---------------+--------------+------+-----+---------+----------------+
8 rows in set (0.00 sec)
so that
grep 'INSERT INTO `taggings`' inuse.sql | mysql -uroot gitorious_production
fails with
ERROR 1136 (21S01) at line 1: Column count doesn't match value count at row 1
Is there an easy way to tell MySQL that the final two fields should be left at their default value, NULL?
(The new Gitorious' taggings table starts out empty.)
As a general best practice, you should mention the field names in which you're inserting :
Insert into taggings (id,tag_id,taggable_id,taggable_type,created_at) values (...your values...)
Rename your new table taggings as taggings_old
Create a table named taggings with your old schema
Insert your data
Add the new column to your table taggings
I have a rails 3.2 application where I have a table called menu_items. My rails application barks when I read data that has an apostrophe in it (ex "Devil's food cake"). I am able to input the field however reading it is a different story. My application works perfectly as long as I delete the record with the apostrophe. My view is an ajax form where I remotely retrieve the record. My schema is below.
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| description | text | YES | | NULL | |
| price | decimal(8,2) | YES | | NULL | |
| serves | varchar(255) | YES | | NULL | |
| measurement | varchar(255) | YES | | NULL | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| section_id | int(11) | YES | | NULL | |
| position | int(11) | YES | | NULL | |
+-------------+--------------+------+-----+---------+----------------+
Following is the error I receive when trying to edit the record within my application.
ActionView::Template::Error (SyntaxError: reserved word "class" can't be assigned on line 343):
1: $('#edit_form').empty()
2: $('#available_menu_items_container').empty()
3: $('#available_menu_items_container').html('<%= render :partial => "menu_item" %>')
4: $('.new_menu').hide()
app/views/menus/edit.js.coffee:1:in `_app_views_menus_edit_js_coffee__4311426478414483561_70345298165400'
app/controllers/menus_controller.rb:33:in `edit'
According to the error message, you're setting an attribute called class somewhere
ActionView::Template::Error (SyntaxError: reserved word "class" can't be assigned on line 343):
Look at line 343 (I think in your view) for something like
Perhaps you're setting a CSS class?
Really weird that an apostrophe would effect that - I think removing the record with the apostrophe is just hiding this bug rather than solving the issue.
Below is schema description. I would like to construct a query that for a given user will return all the cases that are shared directly via case_users OR indirectly via case_groups table. Here is my attempt, where I pull the groups the user belongs to upfront:
SELECT * FROM `cases`
INNER JOIN `case_users` ON `cases`.`id` = `case_users`.`case_id`
INNER JOIN `case_groups` ON `cases`.`id` = `case_groups`.`case_id`
WHERE `case_users`.`user_id` = '<USER_ID>'
OR `case_groups`.`group_id` IN (<USER_GROUP_LIST>)
EXPLAIN returns the following: Impossible WHERE noticed after reading const table...
How can I get it done? Ideally I would like to retrieve all the cases in a single shot - without pulling the USER_GROUP_LIST - groups that the user belongs to.
mysql> describe users;
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
+-------------+--------------+------+-----+---------+----------------+
mysql> describe cases;
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
+-------------+--------------+------+-----+---------+----------------+
mysql> describe case_users;
+-------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+-------+
| user_id | int(11) | NO | PRI | NULL | |
| case_id | int(11) | NO | PRI | NULL | |
+-------------+---------+------+-----+---------+-------+
mysql> describe case_groups;
+-------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+-------+
| case_id | int(11) | NO | PRI | NULL | |
| group_id | int(11) | NO | PRI | NULL | |
+-------------+---------+------+-----+---------+-------+
mysql> describe group_users;
+-------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+-------+
| group_id | int(11) | NO | PRI | NULL | |
| user_id | int(11) | NO | PRI | NULL | |
+-------------+---------+------+-----+---------+-------+
Your joins will only return cases whose Id is in both the case_users and case_groups..
If its one or the other, then you need 2 queries, which you can UNION to get all the results in a single resultset:
SELECT `cases`.* FROM `cases`
INNER JOIN `case_users` ON `cases`.`id` = `case_users`.`case_id`
WHERE `case_users`.`user_id` = '<USER_ID>'
UNION
SELECT `cases`.* FROM `cases`
INNER JOIN `case_groups` ON `cases`.`id` = `case_groups`.`case_id`
WHERE `case_groups`.`group_id` IN (SELECT `group_users`.`group_id`
FROM `group_users`
WHERE `group_users`.`user_id` = '<USER_ID>')
i have user list , and i have select box to filter userlist one of the select box options is show by most viewed so i have also need user information too .
i want to sort my users based on most viewed profile in my user list .
i have these two tables but i don't know how to right correct query to make this happen .
i used grouping like this :
$sql ="select userid , count(*) form profile_visit group by userid " ;
but it's not make sense to me , i don't think this query will help me at all .
+-----------+---------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------------+------+-----+-------------------+----------------+
| userid | int(11) | NO | PRI | NULL | auto_increment |
| username | varchar(128) | NO | | NULL | |
| password | char(40) | NO | | NULL | |
| email | varchar(128) | NO | | NULL | |
| name | varchar(256) | NO | | NULL | |
| lastname | varchar(256) | NO | | NULL | |
| job | varchar(256) | NO | | NULL | |
| birthdate | varchar(100) | NO | | NULL | |
| address | varchar(1024) | NO | | NULL | |
| website | varchar(100) | NO | | NULL | |
| tel | varchar(100) | NO | | NULL | |
| role | tinyint(1) | NO | | 0 | |
| reg_date | timestamp | NO | | CURRENT_TIMESTAMP | |
+-----------+---------------+------+-----+-------------------+----------------+
and profile_visit table like this
+------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| ip_address | varchar(70) | NO | | NULL | |
| userid | int(11) | NO | | NULL | |
+------------+-------------+------+-----+---------+----------------+
Try something like this:
$sql ="SELECT userid , COUNT(*) AS visits FROM profile_visit GROUP BY userid ORDER BY visits DESC" ;
That should group as you were expecting, but order the results in descending order based on the number of visits they have had.
I would ask whether it is necessary to have a separate table? Do you need details of all the visits to be stored, or could you just increment a "visits" integer for each user?
Disregarding the typo in form from your query looks reasonable. It should give you the profile identifiers and their view counts.
As I understand your question, you simply need to get the data out sorted in descending order, which is achievable by simply appending an order by to your query (using an alias for the aggregated column makes it an easier read):
select userid , count(*) as visitcount
from profile_visit
group by userid
order by visitcount