In order to manage column updates more effectively cockroach db has a family feature: https://www.cockroachlabs.com/docs/stable/column-families.html
CREATE TABLE test (
id INT PRIMARY KEY,
last_accessed TIMESTAMP,
data BYTES,
FAMILY f1 (id, last_accessed),
FAMILY f2 (data)
);
Is there any to describe liquibase script to create table with two or more families?
Something like
changes:
- createTable:
tableName: test
columns:
- column:
constraints:
primaryKey: true
primaryKeyName: pk_id
name: id
type: INT
- column:
name: last_accessed
type: TIMESTAMP
- column:
name: data
type: bytes
# not real liquibase yaml just question
families:
- family:
name: f1
columns:
- id
- last_accessed
- family:
name: f2
columns:
- data
You can simply use SQL changeset or SQL change types.
Related
I am creating a table using the following code:
from sqlmodel import Field, SQLModel, JSON, Column, Time
class MyTable(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
name: str
type: str
slug: str = Field(index=True, unique=True)
resource_data: dict | None = Field(default=None, sa_column=Column(JSON)) # type: ignore
# ... create engine
SQLModel.metadata.create_all(engine)
The CREATE table script generated for the model above ends up putting the resource_data column above everything else, instead of preserving the natural order of 'id' first
CREATE TABLE mytable (
resource_data JSON, <----- why is this the FIRST column created?
id SERIAL NOT NULL,
name VARCHAR NOT NULL,
type VARCHAR NOT NULL,
slug VARCHAR NOT NULL,
PRIMARY KEY (id)
)
This feels unusual when I inspect my postgresql tables in a db tool like pgAdmin.
How do I ensure the table is created with the 'natural' order of the declarative model, that is 'id' first?
I'm currently trying to get some dummydata for my project. So I setup a docker file and I am trying to load in some data from a csv file. I'm not getting any error's just an empty database.
Docker:
version: "3.3"
services:
maria-db:
image: 'mariadb:latest'
ports:
- '3307:3306'
environment:
MYSQL_DATABASE: 'researchprojectdb'
# So you don't have to use root, but you can if you like
MYSQL_USER: 'user'
# You can use whatever password you like
MYSQL_PASSWORD: 'password'
# Password for root access
MYSQL_ROOT_PASSWORD: 'admin'
volumes:
- ./scripts/researchprojectdb.sql:/docker-entrypoint-initdb.d/1.sql
- ./data:/var/lib/mysql-files
SQL Script:
CREATE TABLE ROLES (
role_id int PRIMARY KEY,
role varchar(20)
);
CREATE TABLE USERS (
user_id int PRIMARY KEY,
firstname varchar(83),
lastname varchar(83),
email varchar(83),
password varchar(83)
role_id int,
foreign key (role_id) references roles(role_id)
);
INSERT INTO ROLES VALUES(1, 'student');
INSERT INTO USERS VALUES(1,'Jan', 'Met de pet', 'jan#depet.com', 'depet', 1);
-- LOAD DATA LOCAL INFILE '/var/lib/mysql-files/rolesdata.csv' into TABLE ROLES FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 0 ROWS;
-- LOAD DATA LOCAL INFILE '/var/lib/mysql-files/usersdata.csv' into TABLE USERS FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 0 ROWS;
and then I use 2 .csv files with the following data:
Roles:
1 student
2 administrator
3 company
Userdata
1 Jan heas jan#metdepet.be depet 1
Let's take a PostgreSQL DB with the following structure:
(all the relations from top to down are OneToMany)
Where a Brandlift always has exactly 2 brandlift_question each having 1 brandlift_answer (itself having many brandlift_answer_content) by brandlift_respondent
Goal:
Given a 'brand' (let's call it the "flagship brand")
and a brandlift.campaign_id,
for each brand of this brandlift,
retrieve by respondent segment,
the count of brand.id = 'answer_content.brand_id' of the second (right) question answers,
where the previous (left) question answer of the same respondent has an answer_content.brand_id equal to the "flagship brand" id
Sample data (dump) :
https://dbfiddle.uk/?rdbms=postgres_12&fiddle=a127374b9a327204db40dc9f4e769fc1
-- -- PostgreSQL database dump -- -- Dumped from database version 12.1 -- Dumped by pg_dump version 12.3 SET statement_timeout = 0; SET lock_timeout = 0; SET idle_in_transaction_session_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SELECT pg_catalog.set_config('search_path', '', false); SET check_function_bodies = false; SET xmloption = content; SET client_min_messages = warning; SET row_security = off; SET default_tablespace = ''; SET default_table_access_method = heap; -- -- Name: brandlift; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.brandlift ( campaign_id uuid NOT NULL, respondent_goal integer NOT NULL ); ALTER TABLE public.brandlift OWNER TO postgres; -- -- Name: COLUMN brandlift.campaign_id; Type: COMMENT; Schema: public; Owner: postgres -- COMMENT ON COLUMN public.brandlift.campaign_id IS '(DC2Type:uuid)'; -- -- Name: brandlift_answer; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.brandlift_answer ( id integer NOT NULL, question_id uuid NOT NULL, respondent_id integer NOT NULL ); ALTER TABLE public.brandlift_answer OWNER TO postgres; -- -- Name: COLUMN brandlift_answer.question_id; Type: COMMENT; Schema: public; Owner: postgres -- COMMENT ON COLUMN public.brandlift_answer.question_id IS '(DC2Type:uuid)'; -- -- Name: brandlift_answer_content; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.brandlift_answer_content ( id integer NOT NULL, answer_id integer NOT NULL, brand_id integer ); ALTER TABLE public.brandlift_answer_content OWNER TO postgres; -- -- Name: brandlift_answer_content_id_seq; Type: SEQUENCE; Schema: public; Owner: postgres -- CREATE SEQUENCE public.brandlift_answer_content_id_seq START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1; ALTER TABLE public.brandlift_answer_content_id_seq OWNER TO postgres; -- -- Name: brandlift_answer_id_seq; Type: SEQUENCE; Schema: public; Owner: postgres -- CREATE SEQUENCE public.brandlift_answer_id_seq START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1; ALTER TABLE public.brandlift_answer_id_seq OWNER TO postgres; -- -- Name: brandlift_brand; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.brandlift_brand ( id integer NOT NULL, campaign_id uuid NOT NULL, name character varying(255) NOT NULL ); ALTER TABLE public.brandlift_brand OWNER TO postgres; -- -- Name: COLUMN brandlift_brand.campaign_id; Type: COMMENT; Schema: public; Owner: postgres -- COMMENT ON COLUMN public.brandlift_brand.campaign_id IS '(DC2Type:uuid)'; -- -- Name: brandlift_brand_id_seq; Type: SEQUENCE; Schema: public; Owner: postgres -- CREATE SEQUENCE public.brandlift_brand_id_seq START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1; ALTER TABLE public.brandlift_brand_id_seq OWNER TO postgres; -- -- Name: brandlift_question; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.brandlift_question ( id uuid NOT NULL, campaign_id uuid NOT NULL, title character varying(255) NOT NULL ); ALTER TABLE public.brandlift_question OWNER TO postgres; -- -- Name: COLUMN brandlift_question.id; Type: COMMENT; Schema: public; Owner: postgres -- COMMENT ON COLUMN public.brandlift_question.id IS '(DC2Type:uuid)'; -- -- Name: COLUMN brandlift_question.campaign_id; Type: COMMENT; Schema: public; Owner: postgres -- COMMENT ON COLUMN public.brandlift_question.campaign_id IS '(DC2Type:uuid)'; -- -- Name: brandlift_respondent; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.brandlift_respondent ( id integer NOT NULL, campaign_id uuid NOT NULL, segment_id integer NOT NULL, aam_uuid character varying(255) DEFAULT NULL::character varying, "timestamp" timestamp(0) without time zone NOT NULL ); ALTER TABLE public.brandlift_respondent OWNER TO postgres; -- -- Name: COLUMN brandlift_respondent.campaign_id; Type: COMMENT; Schema: public; Owner: postgres -- COMMENT ON COLUMN public.brandlift_respondent.campaign_id IS '(DC2Type:uuid)'; -- -- Name: brandlift_respondent_id_seq; Type: SEQUENCE; Schema: public; Owner: postgres -- CREATE SEQUENCE public.brandlift_respondent_id_seq START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1; ALTER TABLE public.brandlift_respondent_id_seq OWNER TO postgres; -- -- Name: brandlift_segment; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.brandlift_segment ( id integer NOT NULL, name character varying(255) NOT NULL ); ALTER TABLE public.brandlift_segment OWNER TO postgres; -- -- Name: brandlift_segment_id_seq; Type: SEQUENCE; Schema: public; Owner: postgres -- CREATE SEQUENCE public.brandlift_segment_id_seq START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1; ALTER TABLE public.brandlift_segment_id_seq OWNER TO postgres; -- -- Name: campaign; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.campaign ( id uuid NOT NULL, format_id integer NOT NULL, advertiser character varying(255) NOT NULL, name character varying(255) NOT NULL, date_start timestamp(0) without time zone NOT NULL, date_end timestamp(0) without time zone NOT NULL ); ALTER TABLE public.campaign OWNER TO postgres; -- -- Name: COLUMN campaign.id; Type: COMMENT; Schema: public; Owner: postgres -- COMMENT ON COLUMN public.campaign.id IS '(DC2Type:uuid)'; -- -- Name: doctrine_migration_versions; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.doctrine_migration_versions ( version character varying(191) NOT NULL, executed_at timestamp(0) without time zone DEFAULT NULL::timestamp without time zone, execution_time integer ); ALTER TABLE public.doctrine_migration_versions OWNER TO postgres; -- -- Name: format; Type: TABLE; Schema: public; Owner: postgres -- CREATE TABLE public.format ( id integer NOT NULL, name character varying(255) NOT NULL ); ALTER TABLE public.format OWNER TO postgres; -- -- Name: format_id_seq; Type: SEQUENCE; Schema: public; Owner: postgres -- CREATE SEQUENCE public.format_id_seq START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1; ALTER TABLE public.format_id_seq OWNER TO postgres; -- -- Data for Name: brandlift; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.brandlift (campaign_id, respondent_goal) FROM stdin; 8d8c79ce-188c-4f5e-bd72-edb854faf34c 500 \. -- -- Data for Name: brandlift_answer; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.brandlift_answer (id, question_id, respondent_id) FROM stdin; 28 6d6596f4-9418-4f76-9234-c3c943ca56cf 18 29 c84760de-be6f-4f8e-be32-9195846307cf 18 30 6d6596f4-9418-4f76-9234-c3c943ca56cf 19 31 c84760de-be6f-4f8e-be32-9195846307cf 19 32 6d6596f4-9418-4f76-9234-c3c943ca56cf 20 33 c84760de-be6f-4f8e-be32-9195846307cf 20 34 6d6596f4-9418-4f76-9234-c3c943ca56cf 21 35 c84760de-be6f-4f8e-be32-9195846307cf 21 36 6d6596f4-9418-4f76-9234-c3c943ca56cf 22 37 c84760de-be6f-4f8e-be32-9195846307cf 22 \. -- -- Data for Name: brandlift_answer_content; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.brandlift_answer_content (id, answer_id, brand_id) FROM stdin; 54 28 5 55 28 7 56 29 5 57 29 8 58 30 5 59 30 7 60 31 7 61 31 5 62 32 5 63 32 7 64 33 6 65 33 7 66 33 8 67 34 7 68 34 5 69 35 6 70 35 7 71 35 8 72 36 7 73 36 5 74 37 6 75 37 7 \. -- -- Data for Name: brandlift_brand; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.brandlift_brand (id, campaign_id, name) FROM stdin; 5 8d8c79ce-188c-4f5e-bd72-edb854faf34c Nike 6 8d8c79ce-188c-4f5e-bd72-edb854faf34c Adidas 7 8d8c79ce-188c-4f5e-bd72-edb854faf34c Lacoste 8 8d8c79ce-188c-4f5e-bd72-edb854faf34c Puma \. -- -- Data for Name: brandlift_question; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.brandlift_question (id, campaign_id, title) FROM stdin; 6d6596f4-9418-4f76-9234-c3c943ca56cf 8d8c79ce-188c-4f5e-bd72-edb854faf34c Parmi les propositions suivantes, pour lesquelles avez-vous vu de la publicité vidéo en ligne au cours du dernier mois ? c84760de-be6f-4f8e-be32-9195846307cf 8d8c79ce-188c-4f5e-bd72-edb854faf34c Si vous deviez prochainement choisir une marque de streetwear la ou lesquelles choisiriez-vous ? \. -- -- Data for Name: brandlift_respondent; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.brandlift_respondent (id, campaign_id, segment_id, aam_uuid, "timestamp") FROM stdin; 10 8d8c79ce-188c-4f5e-bd72-edb854faf34c 3 912407327014 2021-02-01 18:04:43 11 8d8c79ce-188c-4f5e-bd72-edb854faf34c 3 2692416912404 2021-02-01 18:06:01 12 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 9741094120421 2021-02-01 18:06:56 13 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 8973093247093 2021-02-01 18:07:59 14 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 2147124472178421 2021-02-01 18:08:34 15 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 2147124472178421 2021-02-01 18:09:20 16 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 23253532532352 2021-02-01 18:09:53 17 8d8c79ce-188c-4f5e-bd72-edb854faf34c 3 357325732577352 2021-02-01 18:20:01 18 8d8c79ce-188c-4f5e-bd72-edb854faf34c 3 357325732577352 2021-02-01 18:21:12 19 8d8c79ce-188c-4f5e-bd72-edb854faf34c 3 4354646464223 2021-02-01 18:22:14 20 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 6855985895808 2021-02-01 18:25:41 21 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 6855985895808 2021-02-01 18:26:03 22 8d8c79ce-188c-4f5e-bd72-edb854faf34c 4 6855985895808 2021-02-01 18:26:33 \. -- -- Data for Name: brandlift_segment; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.brandlift_segment (id, name) FROM stdin; 3 expo 4 no expo \. -- -- Data for Name: campaign; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.campaign (id, format_id, advertiser, name, date_start, date_end) FROM stdin; 8d8c79ce-188c-4f5e-bd72-edb854faf34c 2 Nike Air Max 270 2021-01-25 00:00:00 2021-02-28 00:00:00 \. -- -- Data for Name: doctrine_migration_versions; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.doctrine_migration_versions (version, executed_at, execution_time) FROM stdin; DoctrineMigrations\\Version20210131122327 2021-01-31 13:23:34 199 \. -- -- Data for Name: format; Type: TABLE DATA; Schema: public; Owner: postgres -- COPY public.format (id, name) FROM stdin; 2 Brandlift \. -- -- Name: brandlift_answer_content_id_seq; Type: SEQUENCE SET; Schema: public; Owner: postgres -- SELECT pg_catalog.setval('public.brandlift_answer_content_id_seq', 75, true); -- -- Name: brandlift_answer_id_seq; Type: SEQUENCE SET; Schema: public; Owner: postgres -- SELECT pg_catalog.setval('public.brandlift_answer_id_seq', 37, true); -- -- Name: brandlift_brand_id_seq; Type: SEQUENCE SET; Schema: public; Owner: postgres -- SELECT pg_catalog.setval('public.brandlift_brand_id_seq', 8, true); -- -- Name: brandlift_respondent_id_seq; Type: SEQUENCE SET; Schema: public; Owner: postgres -- SELECT pg_catalog.setval('public.brandlift_respondent_id_seq', 22, true); -- -- Name: brandlift_segment_id_seq; Type: SEQUENCE SET; Schema: public; Owner: postgres -- SELECT pg_catalog.setval('public.brandlift_segment_id_seq', 4, true); -- -- Name: format_id_seq; Type: SEQUENCE SET; Schema: public; Owner: postgres -- SELECT pg_catalog.setval('public.format_id_seq', 2, true); -- -- Name: brandlift_answer_content brandlift_answer_content_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_answer_content ADD CONSTRAINT brandlift_answer_content_pkey PRIMARY KEY (id); -- -- Name: brandlift_answer brandlift_answer_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_answer ADD CONSTRAINT brandlift_answer_pkey PRIMARY KEY (id); -- -- Name: brandlift_brand brandlift_brand_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_brand ADD CONSTRAINT brandlift_brand_pkey PRIMARY KEY (id); -- -- Name: brandlift brandlift_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift ADD CONSTRAINT brandlift_pkey PRIMARY KEY (campaign_id); -- -- Name: brandlift_question brandlift_question_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_question ADD CONSTRAINT brandlift_question_pkey PRIMARY KEY (id); -- -- Name: brandlift_respondent brandlift_respondent_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_respondent ADD CONSTRAINT brandlift_respondent_pkey PRIMARY KEY (id); -- -- Name: brandlift_segment brandlift_segment_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_segment ADD CONSTRAINT brandlift_segment_pkey PRIMARY KEY (id); -- -- Name: campaign campaign_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.campaign ADD CONSTRAINT campaign_pkey PRIMARY KEY (id); -- -- Name: doctrine_migration_versions doctrine_migration_versions_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.doctrine_migration_versions ADD CONSTRAINT doctrine_migration_versions_pkey PRIMARY KEY (version); -- -- Name: format format_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.format ADD CONSTRAINT format_pkey PRIMARY KEY (id); -- -- Name: answer_content_unique; Type: INDEX; Schema: public; Owner: postgres -- CREATE UNIQUE INDEX answer_content_unique ON public.brandlift_answer_content USING btree (answer_id, brand_id); -- -- Name: answer_unique; Type: INDEX; Schema: public; Owner: postgres -- CREATE UNIQUE INDEX answer_unique ON public.brandlift_answer USING btree (question_id, respondent_id); -- -- Name: idx_1f1512ddd629f605; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_1f1512ddd629f605 ON public.campaign USING btree (format_id); -- -- Name: idx_3be90c3a44f5d008; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_3be90c3a44f5d008 ON public.brandlift_answer_content USING btree (brand_id); -- -- Name: idx_3be90c3aaa334807; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_3be90c3aaa334807 ON public.brandlift_answer_content USING btree (answer_id); -- -- Name: idx_80d627bdf639f774; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_80d627bdf639f774 ON public.brandlift_question USING btree (campaign_id); -- -- Name: idx_d5009950db296aad; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_d5009950db296aad ON public.brandlift_respondent USING btree (segment_id); -- -- Name: idx_d5009950f639f774; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_d5009950f639f774 ON public.brandlift_respondent USING btree (campaign_id); -- -- Name: idx_d671e368f639f774; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_d671e368f639f774 ON public.brandlift_brand USING btree (campaign_id); -- -- Name: idx_fcce59931e27f6bf; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_fcce59931e27f6bf ON public.brandlift_answer USING btree (question_id); -- -- Name: idx_fcce5993ce80cd19; Type: INDEX; Schema: public; Owner: postgres -- CREATE INDEX idx_fcce5993ce80cd19 ON public.brandlift_answer USING btree (respondent_id); -- -- Name: campaign fk_1f1512ddd629f605; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.campaign ADD CONSTRAINT fk_1f1512ddd629f605 FOREIGN KEY (format_id) REFERENCES public.format(id); -- -- Name: brandlift_answer_content fk_3be90c3a44f5d008; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_answer_content ADD CONSTRAINT fk_3be90c3a44f5d008 FOREIGN KEY (brand_id) REFERENCES public.brandlift_brand(id); -- -- Name: brandlift_answer_content fk_3be90c3aaa334807; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_answer_content ADD CONSTRAINT fk_3be90c3aaa334807 FOREIGN KEY (answer_id) REFERENCES public.brandlift_answer(id); -- -- Name: brandlift fk_5b5d1287f639f774; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift ADD CONSTRAINT fk_5b5d1287f639f774 FOREIGN KEY (campaign_id) REFERENCES public.campaign(id); -- -- Name: brandlift_question fk_80d627bdf639f774; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_question ADD CONSTRAINT fk_80d627bdf639f774 FOREIGN KEY (campaign_id) REFERENCES public.brandlift(campaign_id); -- -- Name: brandlift_respondent fk_d5009950db296aad; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_respondent ADD CONSTRAINT fk_d5009950db296aad FOREIGN KEY (segment_id) REFERENCES public.brandlift_segment(id); -- -- Name: brandlift_respondent fk_d5009950f639f774; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_respondent ADD CONSTRAINT fk_d5009950f639f774 FOREIGN KEY (campaign_id) REFERENCES public.brandlift(campaign_id); -- -- Name: brandlift_brand fk_d671e368f639f774; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_brand ADD CONSTRAINT fk_d671e368f639f774 FOREIGN KEY (campaign_id) REFERENCES public.brandlift(campaign_id); -- -- Name: brandlift_answer fk_fcce59931e27f6bf; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_answer ADD CONSTRAINT fk_fcce59931e27f6bf FOREIGN KEY (question_id) REFERENCES public.brandlift_question(id); -- -- Name: brandlift_answer fk_fcce5993ce80cd19; Type: FK CONSTRAINT; Schema: public; Owner: postgres -- ALTER TABLE ONLY public.brandlift_answer ADD CONSTRAINT fk_fcce5993ce80cd19 FOREIGN KEY (respondent_id) REFERENCES public.brandlift_respondent(id); -- -- PostgreSQL database dump complete --
Desired output with :
campaign_id => 8d8c79ce-188c-4f5e-bd72-edb854faf34c
"flagship brand" => Nike (id=5)
brand_name
segment_name
count
Adidas
expo
0
Adidas
no expo
3
Lacoste
expo
1
Lacoste
no expo
3
Nike
expo
2
Nike
no expo
0
Puma
expo
1
Puma
no expo
2
UPDATE :
Ended up adding an 'ordinal' column to brandlift_question table, so that later if I ever have to add more questions in a Brandlift, I can compare answers based on other questions indexes and not just between first & second
SELECT
bb.name AS brand_name,
s.name AS segment_name,
COUNT(q2ac.brand_id)
FROM
brandlift b
LEFT JOIN brandlift_brand bb ON b.campaign_id = bb.campaign_id
LEFT JOIN brandlift_respondent r ON b.campaign_id = r.campaign_id
LEFT JOIN brandlift_segment s ON s.id = r.segment_id
LEFT JOIN brandlift_question q1 ON b.campaign_id = q1.campaign_id AND q1.ordinal = 1
LEFT JOIN brandlift_answer q1a ON r.id = q1a.respondent_id AND q1.id = q1a.question_id
INNER JOIN brandlift_answer_content q1ac ON q1a.id = q1ac.answer_id AND q1ac.brand_id = 5 -- the "flagship brand" id
LEFT JOIN brandlift_question q2 ON q1.campaign_id = q2.campaign_id AND q2.ordinal = 2
INNER JOIN brandlift_answer q2a ON q1a.respondent_id = q2a.respondent_id AND q2.id = q2a.question_id
LEFT JOIN brandlift_answer_content q2ac ON q2a.id = q2ac.answer_id AND bb.id = q2ac.brand_id
WHERE
b.campaign_id = '8d8c79ce-188c-4f5e-bd72-edb854faf34c'
GROUP BY
bb.name,
s.name
But there is still a point, if there is no respondent on a segment, the result output of this query will not contain rows with that segment values filled with 0. Need them too if possible
I'm keen for any advice
If you don't want to do mod, then you need to select questionID instead.
and I used cross join to get some other segment where not in respond. and in select statement, you case statement to count segmentName to 0.
SELECT
bb.name,
s.name,
CASE WHEN s.id IN(
SELECT
r.segment_id FROM brandlift_respondent r) THEN
COUNT(q2ac.brand_id)
ELSE
0
END AS CountNumber
FROM
brandlift b
LEFT JOIN brandlift_brand bb ON b.campaign_id = bb.campaign_id
LEFT JOIN brandlift_respondent r ON b.campaign_id = r.campaign_id
CROSS JOIN brandlift_segment s
LEFT JOIN brandlift_question q1 ON b.campaign_id = q1.campaign_id AND q1.ordinal = 1
LEFT JOIN brandlift_answer q1a ON r.id = q1a.respondent_id AND q1.id = q1a.question_id
INNER JOIN brandlift_answer_content q1ac ON q1a.id = q1ac.answer_id AND q1ac.brand_id = 1 -- the "flagship brand" id
LEFT JOIN brandlift_question q2 ON q1.campaign_id = q2.campaign_id AND q2.ordinal = 2
LEFT JOIN brandlift_answer q2a ON q1a.respondent_id = q2a.respondent_id AND q2.id = q2a.question_id
LEFT JOIN brandlift_answer_content q2ac ON q2a.id = q2ac.answer_id AND bb.id = q2ac.brand_id
WHERE
b.campaign_id = :campaign_id -- parameter
AND R.segment_id = s.id OR s.id NOT IN(SELECT r.segment_id FROM brandlift_respondent r)
GROUP BY
bb.name,
s.name,
s.id
the result shows like that
I need to restrict the number of digits in the 'integer' type fields to 6 or 7 in postgres.
Basically it should accept 123456 or 1234567 but it should not accept 12345 or 12345678.
How can I achieve this??
You can use floor(log(i)+1 to get the number of digits in a number (that's base10 log).
DB> CREATE TEMP TABLE t (
i integer CONSTRAINT i_has_6_or_7_digits CHECK (floor(log(abs(i))+1) BETWEEN 6 AND 7)
);
CREATE TABLE
Time: 5,676 ms
DB> INSERT INTO t VALUES (123456), (1234567);
INSERT 0 2
Time: 0,471 ms
DB> INSERT INTO t VALUES (12345);
ERROR: 23514: new row for relation "t" violates check constraint "i_has_6_or_7_digits"
DETAIL: Failing row contains (12345).
SCHEMA NAME: pg_temp_2
TABLE NAME: t
CONSTRAINT NAME: i_has_6_or_7_digits
LOCATION: ExecConstraints, execMain.c:1661
Time: 0,468 ms
DB> INSERT INTO t VALUES (12345678);
ERROR: 23514: new row for relation "t" violates check constraint "i_has_6_or_7_digits"
DETAIL: Failing row contains (12345678).
SCHEMA NAME: pg_temp_2
TABLE NAME: t
CONSTRAINT NAME: i_has_6_or_7_digits
LOCATION: ExecConstraints, execMain.c:1661
Time: 0,215 ms
Have a check constraint with
check (value>99999 and value<=9999999)
I want to remove duplicate rows from a table measurement in a PostgreSQL 9.1 data base.
Some table information:
select column_name, data_type from information_schema.columns where table_name = 'measurement';
column_name | data_type
-------------+-----------
s_sum | real
s_l3 | real
s_l2 | real
s_l1 | real
q_sum | real
q_l3 | real
q_l2 | real
q_l1 | real
p_sum | real
p_l3 | real
p_l2 | real
p_l1 | real
irms_n | real
irms_l3 | real
irms_l2 | real
irms_l1 | real
urms_l3 | real
urms_l2 | real
urms_l1 | real
timestamp | integer
site | integer
id | integer
(22 rows)
and
select count(*) from measurement;
count
----------
56265678
(1 row)
So what I want to do is to remove duplicate rows where all columns except id are equal. I went ahead and tried this with the approach in this answer.
SET temp_buffers = '1GB';
BEGIN;
CREATE TEMPORARY TABLE t_tmp AS
SELECT DISTINCT site,
timestamp,
urms_l1,
urms_l2,
urms_l3,
irms_l1,
irms_l2,
irms_l3,
irms_n,
p_l1,
p_l2,
p_l3,
p_sum,
q_l1,
q_l2,
q_l3,
q_sum,
s_l1,
s_l2,
s_l3,
s_sum
FROM measurement;
TRUNCATE measurement;
INSERT INTO measurement
SELECT * FROM t_tmp;
COMMIT;
where the echo / error is:
SET
BEGIN
SELECT 56103537
TRUNCATE TABLE
ERROR: duplicate key value violates unique constraint "measurement_pkey"
DETAIL: Key (id)=(1) already exists.
ROLLBACK
so it looks as if it would remove the duplicates alright (compare with number of rows of original table measurement above) but then a primary key constraint is violated. I do not really know what is going on here, I assume that the INSERT is not operating on the truncated table...
Update:
The requested sql schema is as follows:
--
-- PostgreSQL database dump
--
SET statement_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SET check_function_bodies = false;
SET client_min_messages = warning;
--
-- Name: plpgsql; Type: EXTENSION; Schema: -; Owner: -
--
CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;
--
-- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner: -
--
COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';
SET search_path = public, pg_catalog;
SET default_tablespace = '';
SET default_with_oids = false;
--
-- Name: measurement; Type: TABLE; Schema: public; Owner: -; Tablespace:
--
CREATE TABLE measurement (
id integer NOT NULL,
site integer,
"timestamp" integer,
urms_l1 real,
urms_l2 real,
urms_l3 real,
irms_l1 real,
irms_l2 real,
irms_l3 real,
irms_n real,
p_l1 real,
p_l2 real,
p_l3 real,
p_sum real,
q_l1 real,
q_l2 real,
q_l3 real,
q_sum real,
s_l1 real,
s_l2 real,
s_l3 real,
s_sum real
);
--
-- Name: measurement_pkey; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace:
--
ALTER TABLE ONLY measurement
ADD CONSTRAINT measurement_pkey PRIMARY KEY (id);
--
-- Name: public; Type: ACL; Schema: -; Owner: -
--
REVOKE ALL ON SCHEMA public FROM PUBLIC;
REVOKE ALL ON SCHEMA public FROM postgres;
GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO PUBLIC;
--
-- PostgreSQL database dump complete
--
And then
SELECT id
FROM measurement
GROUP BY id
HAVING COUNT(*) > 1;
yields
id
----
(0 rows)
The primary key is a unique constraint on a subset of the fields in your measurement table, while your SELECT DISTINCT returns only unique records from the fields you list, but looks at every field in each record, not just the primary key
That is, you have records which have the same primary key (id, apparently), but have different values in the non-key fields.
You can find keys that have duplicate ids by running this:
SELECT id
FROM t_tmp
GROUP BY id
HAVING COUNT(*) > 1;
And you can display the records related to that by doing this:
SELECT *
FROM t_tmp
WHERE id IN (
SELECT id
FROM t_tmp
GROUP BY id
HAVING COUNT(*) > 1
);
[Note that I specify t_tmp above, but if you haven't actually run TRUNCATE TABLE measurement; yet that you can use measurement instead.]
Those are the records that have duplicated ids that are causing your key violations, assuming that the key is just on id, which it looks like it is from the error message. You'll need to decide on which one to keep and which one to delete, or otherwise consider updating the id field to a new unique value.
It's not clear if id is tied to a sequence or was created as a SERIAL or BIGSERIAL in your new table. You should really just generate a CREATE TABLE script from pgAdmin to give us the full schema. It's also not clear if there are other unique constraints on the table, either.