PS: Added a new image to better describe what i would like to achieve
As I couldn't find a way to phase the question, therefore it limits the possibility that I could get a ''ready'' solution, pardon me if this is available.
I am self-learning SQL, and would hope to gain some valuable lessons and information on how to write something as the following, greatly appreciated!
Seeking to write lines that allows me to add a master job (eg. 05-16-00000)
in each master job, there will be other "jobs" so it should generate (eg. 05-16-0000 - 01 .. XX). How can I have it written in a way?
[2
[]2
Just hold an id for each record and if a row has a parent, you set a parent_job_id to the corresponding id. Rows with no parent have the parent_job_id set to NULL.
CREATE TABLE `dbname`.`job`
( `id` BIGINT NOT NULL,
`description` VARCHAR(45) NULL,
`parent_job_id` BIGINT NULL,
PRIMARY KEY (`id`)
);
Get master jobs:
SELECT
`job`.`id`,
`job`.`description`,
`job`.`parent_job_id`
FROM
`testdb`.`job`
WHERE
`job`.`parent_job_id` IS NULL
;
If you are looking for children of job 3 replace the WHERE clause with
WHERE `job`.`parent_job_id` = 3
As you show in your later added example you want to m:n link the table with itself. Create a table with parent and child IDs.
CREATE TABLE `dbname`.`job_parent_child`
( `parent_id` BIGINT NOT NULL,
`child_id` BIGINT NOT NULL,
PRIMARY KEY (`parent_id`, `child_id`)
);
Same example - get all childs with parent job 3
SELECT * FROM `dbname`.`job` AS `child`
INNER JOIN `dbname`.`job_parent_child` AS `mn`
ON `child`.`id` = `mn`.`child_id`
WHERE `mn`.`parent_id` = 3
;
According to your last edit just select the job ids (and possibly other data if needed) from the table and iterate over the rows.
SELECT DISTINCT `JOB ID` FROM `jobs`;
Output the master job row of the html table. Then query with prepared statement
SELECT * FROM `jobs` WHERE `JOB ID` = ?;
Output all the rows. That's really all.
Related
Goal: Insert all data into a table from staging table. Each piece of data in the staging table has 2 names which can be found in a separate table. By using the 2 two names, I want to find their respective IDs and insert them into the foreign keys of the main table.
Question: How do I insert the data from a staging table into a table while using data from the staging to query IDs from a separate table?
Example tables:
TABLE location:
id int PRIMARY KEY,
location varchar(255) NOT NULL,
person_oneID int FOREIGN KEY REFERENCES people(person_id),
person_twoID int FOREIGN KEY REFERENCES people(person_id)
TABLE staging_location:
id int PRIMARY KEY,
location varchar(255) NOT NULL,
p1_full_name varchar(255) NOT NULL,
p2_full_name varchar(255) NOT NULL
TABLE people:
person_id int PRIMARY KEY,
first_name varchar(255) NOT NULL,
last_name varchar(255) NOT NULL,
full_name varchar(255) NOT NULL,
This question was the closest example to what I have been looking for. Though I haven't been able to get the query to work. Here is what I've tried:
INSERT INTO location(id,location,person_oneID,person_twoID)
SELECT (l.id,l.location,p1.person_oneID,p2.person_twoID)
FROM staging_location AS l
INNER JOIN people p1 ON p1.full_name = l.p1_full_name
INNER JOIN people p2 ON p2.full_name = l.p2_full_name
Additional info: I would like to do this in the same insert statement without using an update because of the number of locations being inserted. I'm using staging tables as a result of importing data from csv files. The csv file with people didn't have an ID field, so I created one for each person by following steps similar to the first answer from this question. Please let me know if any additional information is required or if I can find the answer to my question somewhere I haven't seen.
Use this code even though I do not know what your data structure is and a duplicate field may be inserted
INSERT INTO location(id,location,person_oneID,person_twoID)
SELECT (l.id,l.location,p1.person_id as person_oneID,p2.person_id as person_twoID)
FROM staging_location AS l
INNER JOIN people p1 ON p1.full_name = l.p1_full_name
INNER JOIN people p2 ON p2.full_name = l.p2_full_name
I have 2 SQL tables and I'm trying to generate a new table with data from the 2 tables.
Jobs table:
jobs (
id SERIAL PRIMARY KEY,
position TEXT NOT NULL,
location TEXT NOT NULL,
pay NUMERIC NOT NULL,
duration TEXT NOT NULL,
description TEXT NOT NULL,
term TEXT NOT NULL,
user_id INTEGER REFERENCES users(id) ON DELETE SET NULL
)
Applied table:
applied (
id SERIAL PRIMARY KEY,
completed BOOLEAN DEFAULT FALSE,
user_id INTEGER REFERENCES users(id) ON DELETE SET NULL,
job_id INTEGER REFERENCES jobs(id) ON DELETE SET NULL,
UNIQUE (user_id, job_id)
);
The tabled populated with data look like this:
Jobs table
Applied table
I want my final query to be a table that matches the jobs table but that has a new column called js_id with true or false based on whether the user has applied to that job. I want the table to look like this:
Here is the query I came up with to generate the above table:
SELECT DISTINCT on (jobs.id)
jobs.*, applied.user_id as applicant,
CASE WHEN applied.user_id = 1 THEN TRUE
ELSE FALSE END as js_id
FROM jobs
JOIN applied on jobs.id = applied.job_id;
However this doesn't work as I add more applicants to the table. I get different true and false values and I haven't been able to get it working. When I remove DISTINCT on (jobs.id) my true values are consistent but I wind up with a lot more than the 3 jobs I want. Here are the results without the DISTINCT on (jobs.id):
I think you want exists:
SELECT j.*,
(exists (select 1
from applied a
where a.job_id = j.id and a.user_id = 1
) as js_id
FROM jobs j;
I'm working on a database for my school project in which I have to produce a functional database by normalizing sample tables given to us.
One table I'm having trouble with is itineraries. I produce 3 tables from the normalization which are "Destinations", "Itineraries" and "Itinerary_Destinations".
The code for Destinations is:
create table Destinations
(
DestinationID varchar(5) primary key,
Name varchar(45)
);
The code for Itineraries is:
create table Itineraries
(
ItineraryID varchar(5),
Name varchar(45)
);
The code for the last table is:
create table Itinerary_Destinations
(
DI varchar(5) primary key,
ItineraryID varchar(5) foreign key references Itineraries(ItineraryID),
Itinerary_Name varchar(45),
DestinationID varchar(5) foreign key references Destinations(DestinationID),
Destination_Name varchar(45)
);
Data has already been inserted into all 3 tables with the exception of 'Destination_Name' and 'Itinerary_Name' columns. The code I'm attempting to use is returning as error. The code is shown below.
insert into Itinerary_Destinations (Itinerary_name)
select Name from Itineraries where
Itineraries.ItineraryID = ItineraryID;
The error it returns is
Msg 515, Level 16, State 2, Line 1 Cannot insert the value NULL into
column 'DI', table 'DDDAssignment.dbo.Itinerary_Destinations'; column
does not allow nulls. INSERT fails. The statement has been terminated.
Is there a method to accomplish the task of inserting the Destination_Name and Itinerary_Name without creating new records that require primary keys?
Or should I do it manually?
If you want to modify records which already exist, then you should be using an UPDATE rather than an INSERT:
UPDATE a
SET Itinerary_name = b.Name
FROM Itinerary_Destinations a
INNER JOIN Itinerary_name b
ON a.ItineraryID = b.ItineraryID;
But, if you do have some data which is not already logically associated with the Itinerary_Destinations table, then using an insert is appropriate.
use coalesce funtion in case null it will insert blank string, as your column does not allow null value thats why you got that error in your query
insert into Itinerary_Destinations (Itinerary_name)
select coalesce(Name,' ') from Itineraries where
Itineraries.ItineraryID = ItineraryID;
I have this table:
CREATE TABLE `search_engine_rankings` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`keyword_id` int(11) DEFAULT NULL,
`search_engine_id` int(11) DEFAULT NULL,
`total_results` int(11) DEFAULT NULL,
`rank` int(11) DEFAULT NULL,
`url` varchar(255) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`indexed_at` date DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `unique_ranking` (`keyword_id`,`search_engine_id`,`rank`,`indexed_at`),
KEY `search_engine_rankings_search_engine_id_fk` (`search_engine_id`),
CONSTRAINT `search_engine_rankings_keyword_id_fk` FOREIGN KEY (`keyword_id`) REFERENCES `keywords` (`id`) ON DELETE CASCADE,
CONSTRAINT `search_engine_rankings_search_engine_id_fk` FOREIGN KEY (`search_engine_id`) REFERENCES `search_engines` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=244454637 DEFAULT CHARSET=utf8
It has about 250M rows in production.
When I do:
select id,
rank
from search_engine_rankings
where keyword_id = 19
and search_engine_id = 11
and indexed_at = "2010-12-03";
...it runs very quickly.
When I add the url column (VARCHAR):
select id,
rank,
url
from search_engine_rankings
where keyword_id = 19
and search_engine_id = 11
and indexed_at = "2010-12-03";
...it runs very slowly.
Any ideas?
The first query can be satisfied by the index alone -- no need to read the base table to obtain the values in the Select clause. The second statement requires reads of the base table because the URL column is not part of the index.
UNIQUE KEY `unique_ranking` (`keyword_id`,`search_engine_id`,`rank`,`indexed_at`),
The rows in tbe base table are not in the same physical order as the rows in the index, and so the read of the base table can involve considerable disk-thrashing.
You can think of it as a kind of proof of optimization -- on the first query the disk-thrashing is avoided because the engine is smart enough to consult the index for the values requested in the select clause; it will already have read that index into RAM for the where clause, so it takes advantage of that fact.
Additionally to Tim's answer. An index in Mysql can only be used left-to-right. Which means it can use columns of your index in your WHERE clause only up to the point you use them.
Currently, your UNIQUE index is keyword_id,search_engine_id,rank,indexed_at. This will be able to filter the columns keyword_id and search_engine_id, still needing to scan over the remaining rows to filter for indexed_at
But if you change it to: keyword_id,search_engine_id,indexed_at,rank (just the order). This will be able to filter the columns keyword_id,search_engine_id and indexed_at
I believe it will be able to fully use that index to read the appropriate part of your table.
I know it's an old post but I was experiencing the same situation and I didn't found an answer.
This really happens in MySQL, when you have varchar columns it takes a lot of time processing. My query took about 20 sec to process 1.7M rows and now is about 1.9 sec.
Ok first of all, create a view from this query:
CREATE VIEW view_one AS
select id,rank
from search_engine_rankings
where keyword_id = 19000
and search_engine_id = 11
and indexed_at = "2010-12-03";
Second, same query but with an inner join:
select v.*, s.url
from view_one AS v
inner join search_engine_rankings s ON s.id=v.id;
TLDR: I solved this by running optimize on the table.
I experienced the same just now. Even lookups on primary key and selecting just some few rows was slow. Testing a bit, I found it not to be limited to the varchar column, selecting an int also took considerable amounts of time.
A query roughly looking like this took around 3s:
select someint from mytable where id in (1234, 12345, 123456).
While a query roughly looking like this took <10ms:
select count(*) from mytable where id in (1234, 12345, 123456).
The approved answer here is to just make an index spanning someint also, and it will be fast, as mysql can fetch all information it needs from the index and won't have to touch the table. That probably works in some settings, but I think it's a silly workaround - something is clearly wrong, it should not take three seconds to fetch three rows from a table! Besides, most applications just does a "select * from mytable", and doing changes at the application side is not always trivial.
After optimize table, both queries takes <10ms.
In a certain TABLE, I have a VARTEXT field which includes comma-separated values of country codes. The field is named cc_list. Typical entries look like the following:
'DE,US,IE,GB'
'IT,CA,US,FR,BE'
Now given a country code, I want to be able to efficiently find which records include that country. Obviously there's no point in indexing this field.
I can do the following
SELECT * from TABLE where cc_list LIKE '%US%';
But this is inefficient.
Since the "IN" function is supposed to be efficient (it bin-sorts the values), I was thinking along the lines of
SELECT * from TABLE where 'US' IN cc_list
But this doesn't work - I think the 2nd operand of IN needs to be a list of values, not a string. Is there a way to convert a CSV string to a list of values?
Any other suggestions? Thanks!
SELECT *
FROM MYTABLE
WHERE FIND_IN_SET('US', cc_list)
In a certain TABLE, I have a VARTEXT field which includes comma-separated values of country codes.
If you want your queries to be efficient, you should create a many-to-many link table:
CREATE TABLE table_country (cc CHAR(2) NOT NULL, tableid INT NOT NULL, PRIMARY KEY (cc, tableid))
SELECT *
FROM tablecountry tc
JOIN mytable t
ON t.id = tc.tableid
WHERE t.cc = 'US'
Alternatively, you can set ft_min_word_len to 2, create a FULLTEXT index on your column and query like this:
CREATE FULLTEXT INDEX fx_mytable_cclist ON mytable (cc_list);
SELECT *
FROM MYTABLE
WHERE MATCH(cc_list) AGAINST('+US' IN BOOLEAN MODE)
This only works for MyISAM tables and the argument should be a literal string (you won't be able to join on this condition).
The first rule of normalization says you should change multi-value columns such as cc_list into a single value field for this very reason.
Preferably into it's own table with IDs for each country code and a pivot table to support a many-to-many relationship.
CREATE TABLE my_table (
my_id INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
mystuff VARCHAR NOT NULL,
PRIMARY KEY(my_id)
);
# this is the pivot table
CREATE TABLE my_table_countries (
my_id INT(11) UNSIGNED NOT NULL,
country_id SMALLINT(5) UNSIGNED NOT NULL,
PRIMARY KEY(my_id, country_id)
);
CREATE TABLE countries {
country_id SMALLINT(5) UNSIGNED NOT NULL AUTO_INCREMENT,
country_code CHAR(2) NOT NULL,
country_name VARCHAR(100) NOT NULL,
PRIMARY KEY (country_id)
);
Then you can query it making use of indexes:
SELECT * FROM my_table JOIN my_table_countries USING (my_id) JOIN countries USING (country_id) WHERE country_code = 'DE'
SELECT * FROM my_table JOIN my_table_countries USING (my_id) JOIN countries USING (country_id) WHERE country_code IN('DE','US')
You may have to group the results my my_id.
find_in_set seems to be the MySql function you want. If you could actually store those comma-separated strings as MySql sets (no more than 64 possible countries, or splitting countries into two groups of no more than 64 each), you could keep using find_in_set and go a bit faster.
There's no efficient way to find what you want. A table scan will be necessary. Putting multiple values into a single text field is a terrible misuse of relational database technology. If you refactor (if you have access to the database structure) so that the country codes are properly stored in a separate table you will be able to easily and quickly retrieve the data you want.
One approach that I've used successfully before (not on mysql, though) is to place a trigger on the table that splits the values (based on a specific delimiter) into discrete values, inserting them into a sub-table. Your select can then look like this:
SELECT * from TABLE where cc_list IN
(
select cc_list_name from cc_list_subtable
where c_list_subtable.table_id = TABLE.id
)
where the trigger parses cc_list in TABLE into separate entries in column cc_list_name in table cc_list_subtable. It involves a bit of work in the trigger, too, as every change to TABLE means that associated rows in cc_list_table have to be deleted/updated/inserted as appropriate, but is an approach that works in situations where the original table TABLE has to retain its original structure, but where you are free to adapt the query as you see fit.