I'm retrieving images stored as blobs in the database using a python script running on the same server.
SELECT *
FROM imagedb_production.imagedb IMAGE
LEFT JOIN dccms_production.tblmedia MEDIA ON IMAGE.name = MEDIA.name
LEFT JOIN dccms_production.tblmultimedia CAP ON MEDIA.contentItemID = CAP.contentItemID
LIMIT 5000,100;
An EXPLAIN returns
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE IMAGE index NULL name_idx 767 NULL 10145962 Using index
1 SIMPLE MEDIA ref name name 63 imagedb_production.IMAGE.name 1
1 SIMPLE CAP eq_ref PRIMARY,idx_contentItemID PRIMARY 4 dccms_production.MEDIA.contentItemID 1 Using index
(Sorry the output looks like crap)
This query takes close to 12 minutes is there any way I can speed this up before going through and tuning the mysql db instance?
Additional information
'imagedb', 'CREATE TABLE `imagedb` (
`multimediaID` int(11) NOT NULL auto_increment,
`name` varchar(255) NOT NULL,
`content` mediumblob,
`description` longtext,
`mime_type` varchar(255) default NULL,
PRIMARY KEY (`multimediaID`),
KEY `name_idx` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=2320759 DEFAULT CHARSET=utf8'
'tblmedia', 'CREATE TABLE `tblmedia` (
`mediaID` int(11) NOT NULL auto_increment,
`contentItemID` int(11) NOT NULL default ''0'',
`name` varchar(255) default NULL,
`width` int(11) default NULL,
`height` int(11) default NULL,
`file1Size` bigint(20) default NULL,
`file2Size` bigint(20) default NULL,
`mediaSlug` int(11) default NULL,
PRIMARY KEY (`mediaID`),
KEY `idx_contentItemID` (`contentItemID`),
KEY `name` (`name`(20))
) ENGINE=InnoDB AUTO_INCREMENT=899975 DEFAULT CHARSET=utf8'
'tblmultimedia', 'CREATE TABLE `tblmultimedia` (
`contentItemID` int(11) NOT NULL default ''0'',
`caption` text,
`mimeType` varchar(255) default NULL,
PRIMARY KEY (`contentItemID`),
KEY `idx_contentItemID` (`contentItemID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8'
You have 10,000,000 rows with no sorting, I would fix that. Add a default order by clause.
Older versions of MySQL did not take limit clauses into account until much later. I think newer versions do a better job of that. You might want to check into different ways to limit the result set.
Related
So I wanted to put a sql file in a database (es_extended.sql) and it keeps giving me errors like: duplicate column name name etc. and
SQL error (1050): Table 'items' already exists
I have tried to change the names of columns and all but it keeps giving the same errors.
here is the code:
ALTER TABLE `users`
ADD COLUMN `money` VARCHAR(50) DEFAULT NULL,
ADD COLUMN `name` VARCHAR(50) NULL DEFAULT '' AFTER `money`,
ADD COLUMN `skin` LONGTEXT NULL AFTER `name`,
ADD COLUMN `job` VARCHAR(50) NULL DEFAULT 'unemployed' AFTER `skin`,
ADD COLUMN `job_grade` INT NULL DEFAULT 0 AFTER `job`,
ADD COLUMN `loadout` LONGTEXT NULL AFTER `job_grade`,
ADD COLUMN `position` VARCHAR(36) NULL AFTER `loadout`
;
CREATE TABLE `items` (
`name` varchar(50) NOT NULL,
`label` varchar(50) NOT NULL,
`limit` int(11) NOT NULL DEFAULT '-1',
`rare` int(11) NOT NULL DEFAULT '0',
`can_remove` int(11) NOT NULL DEFAULT '1',
PRIMARY KEY (`name`)
);
CREATE TABLE `job_grades` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`job_name` varchar(50) DEFAULT NULL,
`grade` int(11) NOT NULL,
`name` varchar(50) NOT NULL,
`label` varchar(50) NOT NULL,
`salary` int(11) NOT NULL,
`skin_male` longtext NOT NULL,
`skin_female` longtext NOT NULL,
PRIMARY KEY (`id`)
);
INSERT INTO `job_grades` VALUES (1,'unemployed',0,'unemployed','Unemployed',200,'{}','{}');
CREATE TABLE `jobs` (
`name` varchar(50) NOT NULL,
`label` varchar(50) DEFAULT NULL,
PRIMARY KEY (`name`)
;
INSERT INTO `jobs` VALUES ('unemployed','Unemployed');
CREATE TABLE `user_accounts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`identifier` varchar(22) NOT NULL,
`name` varchar(50) NOT NULL,
`money` double NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
);
CREATE TABLE `user_inventory` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`identifier` varchar(22) NOT NULL,
`item` varchar(50) NOT NULL,
`count` int(11) NOT NULL,
PRIMARY KEY (`id`)
);
Sql databases are a way to implement the persistence layer: data that should be kept permanently or at least independently from sessions.
The error you're getting: SQL error (1050): Table 'items' already exists means that you've already created a table with the same name you're using to try and create a new table.
You do not need to recreate the table each time you try to add data to it. The table is persistent as long as you do not DROP (delete) it or the database it belongs to.
If you want to recreate the table (either because you need to change its schema or because you want to start over), you can run the command DROP TABLE items which will delete the table and all data in the table.
After dropping, you can run the CREATE TABLE items... command again.
If you don't want to delete the table, you can just run the INSERT INTO items... command to add data to the existing table.
The SQL Error (xxxx): duplicate column name 'name' error means you are trying to add a column that already exists in the table. If the users table already has a column called name, then you cannot run the command ALTER TABLE users ADD COLUMN 'name'... without an error.
If the column doesn't meet your needs anymore, you can use ALTER TABLE users ALTER COLUMN 'name'... to change the schema for the column.
Hello,
first of all I want you to show my table I have:
CREATE TABLE `channels` (
`channel_id` int(11) NOT NULL AUTO_INCREMENT,
`channel_name` varchar(45) DEFAULT NULL,
PRIMARY KEY (`channel_id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8
CREATE TABLE `gamepoints` (
`gp_id` int(11) NOT NULL AUTO_INCREMENT,
`gamble` int(11) DEFAULT NULL,
`roulette` int(11) DEFAULT NULL,
`blackjack` int(11) DEFAULT NULL,
`slots` int(11) DEFAULT NULL,
PRIMARY KEY (`gp_id`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8
CREATE TABLE `player_channel` (
`pc_id` int(11) NOT NULL AUTO_INCREMENT,
`players_id` int(11) DEFAULT NULL,
`channel_id` int(11) DEFAULT NULL,
PRIMARY KEY (`pc_id`),
KEY `players_id_idx` (`players_id`),
KEY `channel_id_idx` (`channel_id`),
CONSTRAINT `channel_id` FOREIGN KEY (`channel_id`) REFERENCES `channels` (`channel_id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `players_id` FOREIGN KEY (`players_id`) REFERENCES `players` (`players_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8
CREATE TABLE `players` (
`players_id` int(11) NOT NULL AUTO_INCREMENT,
`p_name` varchar(45) DEFAULT NULL,
`p_right` varchar(45) DEFAULT NULL,
`gp_id` int(11) DEFAULT NULL,
PRIMARY KEY (`players_id`),
KEY `gp_id_idx` (`gp_id`),
CONSTRAINT `gp_id` FOREIGN KEY (`gp_id`) REFERENCES `gamepoints` (`gp_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8
I wanted to this query:
UPDATE gamepoints SET gamble = 1
WHERE gamepoints.gp_id = players.gp_id
AND players.p_name = "test"
AND player_channel.players_id = players.players_id
AND player_channel.channel_id = channels.channel_id
AND channels.channel_name = "test";
But when I do this i get this Error:
Error Code: 1054. Unknown column 'players.p_name' in 'where clause'
I don't know what I am doing wrong, i tryed also after set to add from with all tables, but this works also not.
I would be really happy if someone can help me :)
Thanks in advance
You omitted the table names of all but gamepoints. I would recommend that you use explicit join update syntax, which makes it harder to have this sort of error:
UPDATE gamepoints t1
INNER JOIN players t2
ON t1.gp_id = t2.gp_id
INNER JOIN player_channel t3
ON t3.players_id = t2.players_id
INNER JOIN channels t4
ON t4.channel_id = t3.channel_id
SET t1.gamble = 1
WHERE t2.p_name = 'test' AND
t4.channel_name = 'test';
If you wanted to salvage your current query, then it would start off looking something like this:
UPDATE gamepoints, players, player_channel, channels
SET gamble = 1
WHERE -- a very large number of conditions
One reason implicit joins are frowned upon is that they mix normal restrictions on the result set with join conditions, all in the same WHERE clause. This doesn't happen with the query as I wrote it using explicit joins.
I am using Yii framework and I have got a problem with CRUD generator.
I have got two tables called Users and news with the following structures:
CREATE TABLE IF NOT EXISTS `news` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`keyword` varchar(1000) COLLATE utf8_persian_ci DEFAULT NULL,
`user_id` tinyint(3) unsigned NOT NULL,
`title` varchar(100) COLLATE utf8_persian_ci DEFAULT NULL,
`body` varchar(1000) COLLATE utf8_persian_ci DEFAULT NULL,
`publishedat` date DEFAULT NULL,
`state` tinyint(1) unsigned DEFAULT NULL,
`archive` tinyint(1) unsigned DEFAULT NULL,
`last_modified` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `news_FKIndex1` (`keyword`(255)),
KEY `news_FKIndex2` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_persian_ci AUTO_INCREMENT=3 ;
CREATE TABLE IF NOT EXISTS `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`username` varchar(20) NOT NULL,
`password` varchar(128) NOT NULL,
`create_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`lastvisit_at` timestamp NULL DEFAULT NULL,
`is_disabled` tinyint(1) NOT NULL DEFAULT '1',
PRIMARY KEY (`id`),
UNIQUE KEY `username` (`username`),
KEY `status` (`is_disabled`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=7 ;
when I generate a CRUD using Gii for my news table I cannot see the fields for users table. Instead of user_id I want to see the username in the table created by CRUD generator. How can I make a change in the code to get the result as above?
First, user_id needs to be a foreign key field not just a key field.
Second, gii will not generate the field as you require by default. For such functionality an extension such as Giix might help. However, since a relation exists you could always use relationName.username to display the username in a grid view or a list view.
I have these classes, abbreviated for practical reasons:
class CV {
Date dateCreated
static hasMany=[proposals: Proposal]
}
class Proposal {
String name
Date date_started
static hasMany = [CVs: CV]
static belongsTo = CV
}
Grails creates tables for both these classes, and a third class named "cv_proposals" joining them. So far, so good. I have data in both the CV and the Proposal tables, they both have autoincremented "id" values. All good.
in Oracle MySQL Workbench, I try to manually add values to the joining table to get some dummy data to work with. I get an error message with this trace:
ERROR 1452: Cannot add or update a child row: a foreign key constraint fails
(cvreg_utv.cv_proposals, CONSTRAINT FK17D946F55677A672 FOREIGN KEY (cv_id) REFERENCES cv (id))
I made sure both the tables had several lines of data in them, and that I could edit both of them separately.
After trying dropping and recreating the table, altering the classes back and forth, I'm kind of convinced that this operation somehow has to be done through a running Grails application. So I write this script in a controller and run it:
def g = CV.get(1)
Proposal proposal = g.addToProposals(new Proposal(
name: "SavingTest",
date_started: new Date())).save()
I still get the same error, though. Is this not the right way to define a proposal that is connected to a certain CV? Am I wrong in using a many-to-many connection here somehow?
Edit: adding the schema-create script for the joining table
delimiter $$
CREATE TABLE `cv_proposals` (
`proposal_id` bigint(20) NOT NULL,
`cv_id` bigint(20) NOT NULL,
PRIMARY KEY (`cv_id`,`proposal_id`),
KEY `FK17D946F55677A672` (`cv_id`),
KEY `FK17D946F5F7217832` (`proposal_id`),
CONSTRAINT `FK17D946F5F7217832` FOREIGN KEY (`proposal_id`) REFERENCES `proposal` (`id`),
CONSTRAINT `FK17D946F55677A672` FOREIGN KEY (`cv_id`) REFERENCES `cv` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1$$
And the CV table:
CREATE TABLE `cv` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`version` bigint(20) NOT NULL,
`user_id` bigint(20) NOT NULL,
`version_name` varchar(255) DEFAULT NULL,
`date_created` datetime NOT NULL,
`last_updated` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `FKC734A9AB992` (`user_id`)
) ENGINE=MyISAM AUTO_INCREMENT=101 DEFAULT CHARSET=latin1$$
And the Proposal table:
CREATE TABLE `proposal` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`version` bigint(20) NOT NULL,
`date_ended` datetime NOT NULL,
`date_started` datetime NOT NULL,
`description` varchar(500) DEFAULT NULL,
`name` varchar(500) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=latin1$$
This is the insert script I tried to run:
INSERT INTO `cvreg_utv`.`cv_proposals` (`proposal_id`, `cv_id`)
VALUES ('1', '1');
You crated the tables manually? It's interesting that cv table is using MyISAM engine and the others uses InnoDB.
I think you want to use InnoDB to all your tables, since this engine is transactional. In my test, I also was unable to create the cv_proposals table until I changed the cv creation:
CREATE TABLE cv (
id bigint(20) NOT NULL AUTO_INCREMENT,
version bigint(20) NOT NULL,
user_id bigint(20) NOT NULL,
version_name varchar(255) DEFAULT NULL,
date_created datetime NOT NULL,
last_updated datetime NOT NULL,
PRIMARY KEY (id),
KEY FKC734A9AB992 (user_id)
) ENGINE=InnoDB AUTO_INCREMENT=101
After that, the insert's worked smoothly.
I added the jquery autocomplete plugin to my places textfield to help users better select a location. What I didn't realize before building is that the query would be very slow.
select * from `geoplanet_places` where name LIKE "%San Diego%" AND (place_type = "County" OR place_type = "Town")
The query above took 1.18 seconds. Then I tried adding indexes for name and place_type but that only slowed it down (1.93s).
Is there a way to optimize this query or is there another technique to speed up the query.
This geoplanet_places table has 437,715 rows (mysql)
CREATE TABLE `geoplanet_places` (
`id` int(11) NOT NULL auto_increment,
`woeid` bigint(20) default NULL,
`parent_woeid` bigint(20) default NULL,
`country_code` varchar(255) collate utf8_unicode_ci default NULL,
`name` varchar(255) collate utf8_unicode_ci default NULL,
`language` varchar(255) collate utf8_unicode_ci default NULL,
`place_type` varchar(255) collate utf8_unicode_ci default NULL,
`ancestry` varchar(255) collate utf8_unicode_ci default NULL,
`activity_count` int(11) default '0',
`activity_count_updated_at` datetime default NULL,
`bounding_box` blob,
`slug` varchar(255) collate utf8_unicode_ci default NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `index_geoplanet_places_on_woeid` (`woeid`),
KEY `index_geoplanet_places_on_ancestry` (`ancestry`),
KEY `index_geoplanet_places_on_parent_woeid` (`parent_woeid`),
KEY `index_geoplanet_places_on_slug` (`slug`),
KEY `index_geoplanet_places_on_name` (`name`),
KEY `index_geoplanet_places_on_place_type` (`place_type`)
) ENGINE=InnoDB AUTO_INCREMENT=5652569 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
EXPLAIN
id 1
select_type SIMPLE
table geoplanet_places
type ALL
possible_keys index_geoplanet_places_on_place_type
key NULL
key_len NULL
ref NULL
rows 441273
Extra Using where
You can switch the storage engine of the table to MyISAM to take advantage of full text indexing.
The name index wont help you unless you change the like to LIKE 'San Diego%' which can do a prefix search on the index
Get rid of the leading '%' in your where-like clause, so it becomes: where name like "San Diego%". For auto complete, this seems a reasonable limitation (assumes that the user starts typing correct characters) that should speed up the query significantly, as MySql will be able to use an existing index (index_geoplanet_places_on_name).