Including a set of rows in a view column - sql

Design:
A main table where each entry in it can have zero of more of a set of options “checked”. It seems to me that it would be easier to maintain (adding/removing options) if the options were part of a separate table and a mapping was made between the main table and an options table.
Goal:
A view that contains the information from the main table, as well as all options to which that row has been mapped. However the latter information exists in the view, it should be possible to extract the option’s ID and its description easily.
The implementation below is specific to PostgreSQL, but any paradigm that works across databases is of interest.
The select statement that does what I want is:
WITH tmp AS (
SELECT
tmap.MainID AS MainID,
array_agg(temp_options) AS options
FROM tstng.tmap
INNER JOIN (SELECT id, description FROM tstng.toptions ORDER BY description ASC) AS temp_options
ON tmap.OptionID = temp_options.id
GROUP BY tmap.MainID
)
SELECT tmain.id, tmain.contentcolumns, tmp.options
FROM tstng.tmain
INNER JOIN tmp
ON tmain.id = tmp.MainID;
However, attempting to create a view from this select statement generates an error:
column "options" has pseudo-type record[]
The solution that I’ve found is to cast the array of options (record[]) to a text array (text[][]); however, I’m interested in knowing if there is a better solution out there.
For reference, the create instruction:
CREATE OR REPLACE VIEW tstng.vsolution AS
WITH tmp AS (
SELECT
tmap.MainID AS MainID,
array_agg(temp_options) AS options
FROM tstng.tmap
INNER JOIN (SELECT id, description FROM tstng.toptions ORDER BY description ASC) AS temp_options
ON tmap.OptionID = temp_options.id
GROUP BY tmap.MainID
)
SELECT tmain.id, tmain.contentcolumns, CAST(tmp.options AS text[][])
FROM tstng.tmain
INNER JOIN tmp
ON tmain.id = tmp.MainID;
Finally, the DDL in case my description has been unclear:
CREATE TABLE tstng.tmap (
mainid INTEGER NOT NULL,
optionid INTEGER NOT NULL
);
CREATE TABLE tstng.toptions (
id INTEGER NOT NULL,
description text NOT NULL,
unwanted_column text
);
CREATE TABLE tstng.tmain (
id INTEGER NOT NULL,
contentcolumns text
);
ALTER TABLE tstng.tmain ADD CONSTRAINT main_pkey PRIMARY KEY (id);
ALTER TABLE tstng.toptions ADD CONSTRAINT toptions_pkey PRIMARY KEY (id);
ALTER TABLE tstng.tmap ADD CONSTRAINT tmap_pkey PRIMARY KEY (mainid, optionid);
ALTER TABLE tstng.tmap ADD CONSTRAINT tmap_optionid_fkey FOREIGN KEY (optionid)
REFERENCES tstng.toptions (id);
ALTER TABLE tstng.tmap ADD CONSTRAINT tmap_mainid_fkey FOREIGN KEY (mainid)
REFERENCES tstng.tmain (id);

You could create composite type e.g. temp_options_type with:
DROP TYPE IF EXISTS temp_options_type;
CREATE TYPE temp_options_type AS (id integer, description text);
After that just cast temp_options to that type within array_agg, so it returns temp_options_type[] instead of record[]:
DROP VIEW IF EXISTS tstng.vsolution;
CREATE OR REPLACE VIEW tstng.vsolution AS
WITH tmp AS
(
SELECT
tmap.MainID AS MainID,
array_agg(CAST(temp_options AS temp_options_type)) AS options
FROM
tstng.tmap INNER JOIN
(
SELECT id, description
FROM tstng.toptions
ORDER BY description
) temp_options
ON tmap.OptionID = temp_options.id
GROUP BY tmap.MainID
)
SELECT tmain.id, tmain.contentcolumns, tmp.options
FROM tstng.tmain
INNER JOIN tmp ON tmain.id = tmp.MainID;
Example result:
TABLE tstng.vsolution;
id | contentcolumns | options
----+----------------+-----------------------
1 | aaa | {"(1,xxx)","(2,yyy)"}
2 | bbb | {"(3,zzz)"}
3 | ccc | {"(1,xxx)"}
(3 rows)

Related

How do I move data from one table to another and update references?

I have a table:
select * from users
id | name | company_name
1 | Sam | Sam's Plumbing
2 | Pat | Pat's Bakery
3 | Vic |
I want to move users.company_name to a new companies table, with users.company_id referencing companies.id. Preferably, I'd like to do this in one transaction.
This expresses what I want conceptually, but isn't valid SQL:
BEGIN;
-- 1: add companies
CREATE TABLE companies (
id integer PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY,
name varchar(255) not null
);
-- 2: add users.company_id -> companies.id
ALTER TABLE users
ADD COLUMN company_id INT
CONSTRAINT users_company_id_fk REFERENCES companies (id);
-- 3: move users.company_name to companies.name; update FK
UPDATE users
SET users.company_id = inserted_companies.id
FROM (
INSERT INTO companies (name)
SELECT company_name FROM users
WHERE company_name IS NOT NULL
-- this isn't valid; RETURNING can't reference users
RETURNING companies.id, users.id AS user_id
) AS inserted_companies;
-- 4: drop users.company_name
ALTER TABLE users
DROP COLUMN company_name;
COMMIT;
Similar questions that don't quite help:
Error : ERROR: table name specified more than once
How do I move a column (with contents) to another table in a Rails migration?
Adding a LEFT JOIN on a INSERT INTO....RETURNING
create table companies
(
id serial primary key ,
company_name text
);
insert into companies (company_name)
select distinct company_name
from users
where company_name is not null;
alter table users add company_id int null;
update users set company_id = companies.id
from companies where companies.company_name = users.company_name;
ALTER TABLE users
DROP COLUMN company_name;
ALTER TABLE users
ADD CONSTRAINT users_company_id_fk foreign key (company_id) REFERENCES companies (id);
DBFiddle demo
Building off #Cetin.Basoz's excellent answer, here's what I ended up with:
-- 1: add companies:
CREATE TABLE companies
(
id serial primary key,
company_name text
);
-- 2: move users.company_name to companies.name
-- using the users.id as the companies.id for the initial import:
INSERT INTO companies (id, company_name)
SELECT id, company_name
FROM users
WHERE company_name IS NOT NULL;
-- 3: update the companies PK index
-- so we don't try to insert duplicate IDs:
SELECT setval('companies_id_seq', (SELECT MAX(id) FROM companies));
-- 4: add references to the newly-inserted companies:
ALTER TABLE users ADD company_id int null;
UPDATE users
SET company_id = users.id
WHERE company_name IS NOT NULL;
-- 5: drop the vestigial users.company_name
ALTER TABLE users DROP COLUMN company_name;
-- 6: add a FK index:
ALTER TABLE users
ADD CONSTRAINT users_company_id_fk
FOREIGN KEY (company_id)
REFERENCES companies (id);
The companies table will have holes in it, but as long as the id space is sufficiently large, I don't see that as a problem. It would have holes if records were deleted.

postgres: (sub)select and combine optional content into an array

i have the following table structure:
Location----- * Media ----1 Attribute --------* AttributeTranslation
Each Location has n mediaitems attached, containing one optional attribute (text) and n associated translationa for that attribute.
I need to select this data into an array, so that i get for each location the associated medialist for each language.
what i currently do and what i get:
SELECT m.location_id, t.language_id,
array_agg_mult(
ARRAY[ARRAY[m.sortorder::text, m.filename, t.name]] ORDER BY m.sortorder
) as medialist
FROM Media m
LEFT JOIN ATTRIBUTE a ON a.id = m.attribute_id
LEFT JOIN AttributeTranslation t ON a.id = t.attribute_id
WHERE m.location_id = ?
GROUP BY m.location_id, t.language_id
This gives me following result for the given scenario: the current location has 4 images attached, only the first image has an associated attribute containing two translations:
Location_ID Language_ID MEDIALIST
AT_014 1 {{1,'location_image1.jpg','attribute german'}}
AT_014 2 {{1,'location_image1.jpg','attribute english'}}
AT_014 {{2,'location_image2.jpg',null},{3,'location_image3.jpg',null},{4,'location_image4.jpg',null}}
but what i need instead is this:
Location_ID Language_ID MEDIALIST
AT_014 1 {{1,'location_image1.jpg','attribute german'},{2,'location_image2.jpg',null},{3,'location_image3.jpg',null},{4,'location_image4.jpg',null}}
AT_014 2 {{1,'location_image1.jpg','attribute english'},{2,'location_image2.jpg',null},{3,'location_image3.jpg',null},{4,'location_image4.jpg',null}}
those 3 columns are part of a view, so that i can do later:
select * from locationview where location_id = ? and language_id = ?
how can i achieve the desired result here? thanks in advance!
Simplified Table Definitions:
CREATE TABLE LOCATION (
location_id numeric(20) primary key,
description text
);
CREATE TABLE MEDIA (
media_id numeric(20) primary key,
fileName text,
sortorder smallint,
location_id numeric(20) references LOCATION(location_id),
attribute_id numeric(20) references ATTRIBUTE(attribute_id)
);
CREATE TABLE ATTRIBUTE (
attribute_id numeric(20) primary key,
attributetype varchar(100),
);
CREATE TABLE ATTRIBUTETRANSLATION (
translation_id numeric(20),
language_id smallint,
name text,
description text,
attribute_id numeric(20) references ATTRIBUTE(attribute_id)
);
ALTER TABLE ATTRIBUTETRANSLATION add constraint AT_ID primary key(translation_id, language_id)
I am not sure I fully understand your question, but here's an attempt. You could take the output of your query, and match each row that has a language_id with the corresponding rows where language_id is NULL, so that you can then concatenate the medialist arrays. Here's a way to do that by creating an alias of your query with a CTE:
WITH t AS (
SELECT m.location_id, t.language_id,
array_agg(
ARRAY[ARRAY[m.sortorder::text, m.filename, t.name]] ORDER BY m.sortorder
) as medialist
FROM Media m
LEFT JOIN ATTRIBUTE a ON a.attribute_id = m.attribute_id
LEFT JOIN AttributeTranslation t ON a.attribute_id = t.attribute_id
WHERE m.location_id = ?
GROUP BY m.location_id, t.language_id
)
SELECT location_id, t1.language_id, t1.medialist || t2.medialist AS medialist
FROM (SELECT * FROM t WHERE language_id IS NOT NULL) t1
RIGHT OUTER JOIN (SELECT * FROM t WHERE language_id IS NULL) t2 USING (location_id);
I am not sure if this does exactly what you want, but hopefully it will give you some ideas.

Role basesd access control recursive SQL query

I made the following role based access database schema which is able to hold roles, operations and types. A role can perform a specific operation on a type. Connections to users or types is not important here because this will be application specific. Every of these three tables can have as many parents as they want.
At the moment I'm struggling with a query which outputs every possible combination from the role_operation_type table.
Every role should inherit every permission on a types from the ancestors which can be more than one. In my opinion I need three nested recursive with queries for that or is there any faster way to achieve that?
My intention is to put that query in a view and select the needed values when a user requests an operation on a type.
Here is the database schema:
CREATE TABLE IF NOT EXISTS `role` (
`id` INTEGER PRIMARY KEY,
`name` VARCHAR NOT NULL UNIQUE
);
CREATE TABLE IF NOT EXISTS `role_role` (
`role_id` INTEGER NOT NULL REFERENCES `role`(`id`) ON UPDATE CASCADE ON DELETE CASCADE,
`parent_role_id` INTEGER NOT NULL REFERENCES `role`(`id`) ON UPDATE CASCADE ON DELETE CASCADE,
CONSTRAINT role_uq UNIQUE (`role_id`, `parent_role_id`),
CONSTRAINT role_chk CHECK(`role_id` != `parent_role_id`)
);
CREATE TABLE IF NOT EXISTS `operation` (
`id` INTEGER PRIMARY KEY,
`name` VARCHAR NOT NULL UNIQUE
);
CREATE TABLE IF NOT EXISTS `operation_operation` (
`operation_id` INTEGER NOT NULL REFERENCES `operation`(`id`) ON UPDATE CASCADE ON DELETE CASCADE,
`parent_operation_id` INTEGER NOT NULL REFERENCES `operation`(`id`) ON UPDATE CASCADE ON DELETE CASCADE,
CONSTRAINT operation_uq UNIQUE (`operation_id`, `parent_operation_id`),
CONSTRAINT operation_chk CHECK(`operation_id` != `parent_operation_id`)
);
CREATE TABLE IF NOT EXISTS `type` (
`id` INTEGER PRIMARY KEY,
`name` VARCHAR NOT NULL UNIQUE
);
CREATE TABLE IF NOT EXISTS `type_type` (
`type_id` INTEGER NOT NULL REFERENCES `type`(`id`) ON UPDATE CASCADE ON DELETE CASCADE,
`parent_type_id` INTEGER NOT NULL REFERENCES `type`(`id`) ON UPDATE CASCADE ON DELETE CASCADE,
CONSTRAINT type_uq UNIQUE (`type_id`, `parent_type_id`),
CONSTRAINT type_chk CHECK(`type_id` != `parent_type_id`)
);
CREATE TABLE IF NOT EXISTS `role_operation_type` (
`role_id` INTEGER NOT NULL REFERENCES `role`(`id`) ON DELETE CASCADE ON UPDATE CASCADE,
`operation_id` INTEGER NOT NULL REFERENCES `operation`(`id`) ON DELETE CASCADE ON UPDATE CASCADE,
`type_id` INTEGER NOT NULL REFERENCES `type`(`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT role_id_operation_id_type_id_uq UNIQUE (`role_id`, `operation_id`, `type_id`)
);
CREATE VIEW IF NOT EXISTS role_role_recursive_view AS
WITH RECURSIVE p(role_id, r, parent_role_id) AS (
SELECT ROLE.id, ROLE.id, role_role.parent_role_id
FROM ROLE
INNER JOIN role_role ON ROLE.id = role_role.role_id
UNION
SELECT p.r, p.role_id, role_role.parent_role_id
FROM p
INNER JOIN role_role ON p.parent_role_id = role_role.role_id
WHERE p.r != role_role.parent_role_id
)
SELECT p.role_id, p.parent_role_id FROM p ORDER BY role_id;
CREATE VIEW IF NOT EXISTS operation_operation_recursive_view AS
WITH RECURSIVE o(operation_id, o, parent_operation_id) AS (
SELECT operation.id, operation.id, operation_operation.parent_operation_id
FROM operation
INNER JOIN operation_operation ON operation.id = operation_operation.operation_id
UNION
SELECT o.o, o.operation_id, operation_operation.parent_operation_id
FROM o
INNER JOIN operation_operation ON o.parent_operation_id = operation_operation.operation_id
WHERE o.o != operation_operation.parent_operation_id
)
SELECT o.operation_id, o.parent_operation_id FROM o ORDER BY operation_id;
CREATE VIEW IF NOT EXISTS type_type_recursive_view AS
WITH RECURSIVE t(type_id, t, parent_type_id) AS (
SELECT TYPE.id, TYPE.id, type_type.parent_type_id
FROM TYPE
INNER JOIN type_type ON TYPE.id = type_type.type_id
UNION
SELECT t.t, t.type_id, type_type.parent_type_id
FROM t
INNER JOIN type_type ON t.parent_type_id = type_type.type_id
WHERE t.t != type_type.parent_type_id
)
SELECT t.type_id, t.parent_type_id FROM t ORDER BY type_id;
Now I will answer to my own question with a solution which works quite well. This is a recursive query and generates every possible combination. This is a recursive query which might be slow when the inheritance level becomes deeper.
CREATE VIEW IF NOT EXISTS role_operation_type_recursive_view AS
WITH RECURSIVE T(role_id, operation_id, type_id) AS (
WITH RECURSIVE O(role_id, operation_id, type_id) AS (
WITH RECURSIVE R(role_id, operation_id, type_id) AS (
SELECT role_id, operation_id,type_id
FROM role_operation_type
UNION
SELECT role_role_recursive_view.role_id, R.operation_id, R.type_id
FROM R
INNER JOIN role_role_recursive_view ON R.role_id = role_role_recursive_view.parent_role_id
)
SELECT * FROM R
UNION
SELECT O.role_id,operation_operation_recursive_view.parent_operation_id ,O.type_id
FROM O
INNER JOIN operation_operation_recursive_view ON O.operation_id = operation_operation_recursive_view.operation_id
)
SELECT * FROM O
UNION
SELECT T.role_id, T.operation_id, type_type_recursive_view.type_id
FROM T
INNER JOIN type_type_recursive_view ON T.type_id = type_type_recursive_view.parent_type_id
)
SELECT * FROM T;

Get rows that no foreign keys point to

I have two tables
CREATE TABLE public.city_url
(
id bigint NOT NULL DEFAULT nextval('city_url_id_seq'::regclass),
url text,
city text,
state text,
country text,
common_name text,
CONSTRAINT city_url_pkey PRIMARY KEY (id)
)
and
CREATE TABLE public.email_account
(
id bigint NOT NULL DEFAULT nextval('email_accounts_id_seq'::regclass),
email text,
password text,
total_replied integer DEFAULT 0,
last_accessed timestamp with time zone,
enabled boolean NOT NULL DEFAULT true,
deleted boolean NOT NULL DEFAULT false,
city_url_id bigint,
CONSTRAINT email_accounts_pkey PRIMARY KEY (id),
CONSTRAINT email_account_city_url_id_fkey FOREIGN KEY (city_url_id)
REFERENCES public.city_url (id) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
)
I want to come up with a query that fetches rows in the city_url only if there is no row in the email_account pointing to it with the city_url_id column.
NOT EXISTS comes to mind:
select c.*
from city_url c
where not exists (select 1
from email_account ea
where ea.city_url_id = c.id
);
There's also this option:
SELECT city_url.*
FROM city_url
LEFT JOIN email_account ON email_account.city_url_id = city_url.id
WHERE email_account.id IS NULL
A NOT EXISTS is absolutely the answer to the "... if there is no row ...".
Nonetheless it would be preferable to accomplish this by selecting then difference quantity.
Which is in principle:
SELECT a.*
FROM table1 a
LEFT JOIN table2 b
ON a.[columnX] = b.[columnY]
WHERE b.[columnY] IS NULL
Using the tablenames here, this would be:
SELECT c.*
FROM city_url c
LEFT JOIN email_account e
ON c.id = e.city_url
WHERE e.city_url IS NULL
I believe NOT IN could be used here as well, although this might be less performant on large datasets:
SELECT *
FROM city_url
WHERE id NOT IN (
SELECT city_url_id FROM email_account
)

Search entities by weighted keywords and spelling correction

For starters, a little diagram relations-entities
Diagram relations-entities http://img11.hostingpics.net/pics/32979039DB.png
And now, a dataset
Archive
create :
CREATE TABLE archive (
id integer NOT NULL,
parent_id integer,
code character varying(15) NOT NULL,
label text NOT NULL
);
ALTER TABLE ONLY archive ADD CONSTRAINT archive_pkey PRIMARY KEY (id);
CREATE INDEX idx_142 ON archive USING btree (parent_id);
CREATE UNIQUE INDEX uniq_14242 ON archive USING btree (code);
ALTER TABLE ONLY archive ADD CONSTRAINT fk_14242 FOREIGN KEY (parent_id) REFERENCES archive(id);
insert :
INSERT INTO archive VALUES (1, NULL, 'B28', 'Confidential');
INSERT INTO archive VALUES (2, 1, 'B28.0', 'Nuclear zone');
Keyword
create :
CREATE TABLE keyword (
id integer NOT NULL,
label text NOT NULL,
label_double_metaphone text NOT NULL
);
ALTER TABLE ONLY keyword ADD CONSTRAINT eyword_pkey PRIMARY KEY (id);
CREATE UNIQUE INDEX uniq_242 ON keyword USING btree (label);
insert :
INSERT INTO keyword VALUES (1, 'SECURITY', 'SKRT');
INSERT INTO keyword VALUES (2, 'AREA', 'AR');
INSERT INTO keyword VALUES (3, 'NUCLEAR', 'NKLR');
Assoc_kw_archive
create :
CREATE TABLE assoc_kw_archive (
id integer NOT NULL,
keyword_id integer,
archive_id integer,
weight integer NOT NULL
);
ALTER TABLE ONLY assoc_kw_archive ADD CONSTRAINT assoc_kw_archive_pkey PRIMARY KEY (id);
CREATE INDEX idx_3421 ON assoc_kw_archive USING btree (archive_id);
CREATE INDEX idx_3422 ON assoc_kw_archive USING btree (keyword_id);
ALTER TABLE ONLY assoc_kw_archive ADD CONSTRAINT fk_3421 FOREIGN KEY (archive_id) REFERENCES archive(id);
ALTER TABLE ONLY assoc_kw_archive ADD CONSTRAINT fk_3422 FOREIGN KEY (keyword_id) REFERENCES keyword(id);
insert :
INSERT INTO assoc_kw_archive VALUES (1, 1, 1, 10);
INSERT INTO assoc_kw_archive VALUES (2, 1, 2, 20);
INSERT INTO assoc_kw_archive VALUES (3, 2, 2, 30);
INSERT INTO assoc_kw_archive VALUES (4, 3, 2, 30);
The target
The goal here is to search in the database. The research is based on a string typed by a user. Output a list of archives sorted by relevance. Relevant archive depends on three factors:
The people can make a mistake in the spelling of a word, etc...
The weight of a word to give it importance
Give a gain to the archives include the x keywords typed by the user
I worked on different versions of sql query, but, now I can't to step back and look at the overall problem.
The archive table is composed of 100,000 tuples, 80 000 for the table of keywords and 1,000,000 associations between these two entities.
This is my last version, she is functional, but is very slowly :
select f.id, f.code, f.label, min(f.dist) as distF, max(f.poid) as poidF
from
(
select
a.id,
a.code,
a.label,
( ( levenshtein(lower('Security'), lower(k1.label)) + 1 ) + ( levenshtein(lower('Nuclear'), lower(k2.label)) + 1 ) ) as dist,
( ka1.weight + ka2.weight ) as poid
from archive a
inner join assoc_kw_archive ka1
on ka1.archive_id = a.id
inner join keyword k1
on k1.id = ka1.keyword_id
inner join assoc_kw_archive ka2
on ka2.archive_id = a.id
inner join keyword k2
on k2.id = ka2.keyword_id
where levenshtein(dmetaphone('Security'), k1.label_double_metaphone) < 2
and levenshtein(dmetaphone('Nuclear'), k2.label_double_metaphone) < 2
) as f
group by f.id, f.code, f.label
order by distF asc, poidF desc
limit 10;
I made one join by keyword, it's this that makes it slow! But I can't find another solution.
I think the problem is doing the full join with the distance calculation. here is an alternative approach. Filter the keywords first. Keep the information in the where clause by using a subquery. Then use conditional aggregation to get the information you want.
The query ends up looking something like:
select a.id, a.code, a.label,
min( (levenshtein(lower('Security'), lower(case when securityl < 2 then k.label end)) + 1 ) +
(levenshtein(lower('Nuclear'), lower(case when nuclearl < 2 then k.label end)) + 1 )
) as mindist,
sum(weight) as poid
from archive a inner join
assoc_kw_archive ka
on ka.archive_id = a.id inner join
(select k.*, levenshtein(dmetaphone('Security'), k.label_double_metaphone) as securityl,
levenshtein(dmetaphone('Nuclear'), k.label_double_metaphone) as nuclearl
from keyword k
having securityl < 2 or
nuclearl < 2
) k
on k.id = ka.keyword_id
group by a.id, a.code, a.label