I have been playing with postgresql for a while now and this one caught my eye. What is a "mapped relation" in postgresql. According to the documentation,
When the name of on-disk file is zero, it is called a "mapped"
relation whose disk file name is determined by low - level state.
Is it a simple relation that doesnt have a fixed OID to reference it with. Why is it created? What is its significance?Or is it similar to a temp table?
can some one thow light on this?
https://www.postgresql.org/docs/current/static/storage-file-layout.html
Also, for certain system catalogs including pg_class itself,
pg_class.relfilenode contains zero. The actual filenode number of
these catalogs is stored in a lower-level data structure, and can be
obtained using the pg_relation_filenode() function.
t=# select relfilenode from pg_class where relname = 'pg_class';
relfilenode
-------------
0
(1 row)
t=# select pg_relation_filenode('pg_class');
pg_relation_filenode
----------------------
12712
(1 row)
now a little barbarian (yet user friendly) way to make sure it is the file:
t=# create table very_special_name(i int);
CREATE TABLE
t=# CHECKPOINT; --to actually write to disk
CHECKPOINT
t=# select oid from pg_database where datname='t';
oid
----------
13805223
(1 row)
so we check the readable strings:
-bash-4.2$ strings /pg/data/base/13805223/12712 | grep very_special
very_special_name
New table name is in...
Related
In the following, I will present two examples. The first one has a non-numeric name and the second one has a numeric name. The TimescaleDB extension is already added to the database.
1.1 Creating the table:
create table "one"(time timestamp);
CREATE TABLE
1.2 Checking the OID of the table:
SELECT 'one'::regclass::oid;
oid
-------
61962
(1 row)
1.3 Creating the Hypertable:
SELECT create_hypertable('one','time');
NOTICE: adding not-null constraint to column "time"
DETAIL: Time dimensions cannot have NULL values.
create_hypertable
-------------------
(1,public,one,t)
(1 row)
2.1 Creating the table:
create table "1"(time timestamp);
CREATE TABLE
2.2 Checking the OID of the table:
SELECT '1'::regclass::oid;
oid
-----
1
(1 row)
2.3 Creating the Hypertable:
SELECT create_hypertable('1','time');
ERROR: could not open relation with OID 1
One possible solution would be to create the Hypertable with the OID of "1". That seems pretty complicated to me in the long run.
Getting the OID of "one" and "1":
SELECT oid, relname from pg_class where relname='one' or relname='1';
oid | relname
-------+---------
61959 | 1
61962 | one
(2 rows)
Creating the hypertable with the OID of "1" as name:
SELECT create_hypertable('61959','time');
NOTICE: adding not-null constraint to column "time"
DETAIL: Time dimensions cannot have NULL values.
create_hypertable
-------------------
(2,public,1,t)
(1 row)
PostgreSQL seems to be 'confused' as to whether it should look for the name "1" or the OID "1". Can I work with numeric names? How can I create the Hypertable?
If you really wish to commit this atrocity, you can do it by passing the double quotes to timescaledb.
SELECT create_hypertable('"1"','time');
Maintaining code from a former employee found a piece of SQL i cannot understand:
CREATE OR REPLACE VIEW my_view AS Not a view
There's no info on official documentation and I've beenseeking for information on this query but to no avail.
any hint?
The view was deleted.
Your script was generated by an application which uses pg_get_viewdef(view_oid). When there is no a view with a given oid then the function returns the string not a view. Simple test:
create view my_view as select 1;
select oid
from pg_class
where relname = 'my_view';
oid
--------
151388
(1 row)
select pg_get_viewdef(151388);
pg_get_viewdef
----------------
SELECT 1;
(1 row)
drop view my_view;
select pg_get_viewdef(151388);
pg_get_viewdef
----------------
Not a view
(1 row)
Note that it does not mean that my_view does not exist. If you recreate the view it'll have another oid. The only certain conclusion is that your script is not up-to-date (It's inconsistent with the current content of the database). As a remedy you should dump the schema in SQL format, e.g.
pg_dump --schema-only --format=plain my_database > my_database.dump
We need to count the number of rows in a PostgreSQL table. In our case, no conditions need to be met, and it would be perfectly acceptable to get a row estimate if that significantly improved query speed.
Basically, we want select count(id) from <table> to run as fast as possible, even if that implies not getting exact results.
For a very quick estimate:
SELECT reltuples FROM pg_class WHERE relname = 'my_table';
There are several caveats, though. For one, relname is not necessarily unique in pg_class. There can be multiple tables with the same relname in multiple schemas of the database. To be unambiguous:
SELECT reltuples::bigint FROM pg_class WHERE oid = 'my_schema.my_table'::regclass;
If you do not schema-qualify the table name, a cast to regclass observes the current search_path to pick the best match. And if the table does not exist (or cannot be seen) in any of the schemas in the search_path you get an error message. See Object Identifier Types in the manual.
The cast to bigint formats the real number nicely, especially for big counts.
Also, reltuples can be more or less out of date. There are ways to make up for this to some extent. See this later answer with new and improved options:
Fast way to discover the row count of a table in PostgreSQL
And a query on pg_stat_user_tables is many times slower (though still much faster than full count), as that's a view on a couple of tables.
Count is slow for big tables, so you can get a close estimate this way:
SELECT reltuples::bigint AS estimate
FROM pg_class
WHERE relname='tableName';
and its extremely fast, results are not float, but still a close estimate.
reltuples is a column from pg_class table, it holds data about "number of rows in the table. This is only an estimate used by the planner. It is updated by VACUUM, ANALYZE, and a few DDL commands such as CREATE INDEX" (manual)
The catalog pg_class catalogs tables and most everything else that has columns or is otherwise similar to a table. This includes indexes (but see also pg_index), sequences, views, composite types, and some kinds of special relation (manual)
"Why is "SELECT count(*) FROM bigtable;" slow?" : http://wiki.postgresql.org/wiki/FAQ#Why_is_.22SELECT_count.28.2A.29_FROM_bigtable.3B.22_slow.3F
Aside from running COUNT() on an indexed field (which hopefully 'id' is) - the next best thing would be to actually cache the row count in some table using a trigger on INSERT. Naturally, you'll be checking the cache instead.
For an approximation you can try this (from https://wiki.postgresql.org/wiki/Count_estimate):
select reltuples from pg_class where relname='tablename';
You can get an estimate from the system table "pg_stat_user_tables".
select schemaname, relname, n_live_tup
from pg_stat_user_tables
where schemaname = 'your_schema_name'
and relname = 'your_table_name';
You can ask for the exact value of the count in the table by simply using trigger AFTER INSERT OR DELETE
Something like this
CREATE TABLE tcounter(id serial primary key,table_schema text, table_name text, count serial);
insert into tcounter(table_schema, table_name,count) select 'my_schema', 'my_table', count(*) from my_schema.my_table;
and use trigger
CREATE OR REPLACE FUNCTION ex_count()
RETURNS trigger AS
$BODY$
BEGIN
IF (TG_OP='INSERT') THEN
UPDATE tcounter set count = count + 1 where table_schema = TG_TABLE_SCHEMA::TEXT and table_name = TG_TABLE_NAME::TEXT;
ELSIF (TG_OP='DELETE') THEN
UPDATE tcounter set count = count - 1 where table_schema = TG_TABLE_SCHEMA::TEXT and table_name = TG_TABLE_NAME::TEXT;
END IF;
RETURN NEW;
END$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
CREATE TRIGGER tg_counter AFTER INSERT OR DELETE
ON my_schema.my_table FOR EACH ROW EXECUTE PROCEDURE ex_count();
And ask for count
select * from tcounter where table_schema = 'my_schema' and table_name = 'my_table'
it means you select count(*) once for initialize first record
If your database is small, you can get an estimate of all your tables like #mike-sherrill-cat-recall suggested. This command will list all the tables though.
SELECT schemaname,relname,n_live_tup
FROM pg_stat_user_tables
ORDER BY n_live_tup DESC;
Output would be something like this:
schemaname | relname | n_live_tup
------------+--------------------+------------
public | items | 21806
public | tags | 11213
public | sessions | 3269
public | users | 266
public | shops | 259
public | quantities | 34
public | schema_migrations | 30
public | locations | 8
(8 rows)
In our database we have users: A, B, C.
Each user has its own corresponding schema: A, B, C.
Normally if I wanted to select from a table in one of the schemas I would have to do:
select * from A.table;
My question is:
Is there a way to make:
select * from table
go to the correct schema based on the user that is logged in?
This is the default behavior for PostgreSQL. Make sure your search_path is set correctly.
SHOW search_path;
By default it should be:
search_path
--------------
"$user",public
See PostgreSQL's documentation on schemas for more information. Specifically this part:
You can create a schema for each user with the same name as that user. Recall that the default search path starts with $user, which resolves to the user name. Therefore, if each user has a separate schema, they access their own schemas by default.
If you use this setup then you might also want to revoke access to the public schema (or drop it altogether), so users are truly constrained to their own schemas.
Update RE you comment:
Here is what happens on my machine. Which is what I believe you are wanting.
skrall=# \d
No relations found.
skrall=# show search_path;
search_path
----------------
"$user",public
(1 row)
skrall=# create schema skrall;
CREATE SCHEMA
skrall=# create table test(id serial);
NOTICE: CREATE TABLE will create implicit sequence "test_id_seq" for serial column "test.id"
CREATE TABLE
skrall=# \d
List of relations
Schema | Name | Type | Owner
--------+-------------+----------+--------
skrall | test | table | skrall
skrall | test_id_seq | sequence | skrall
(2 rows)
skrall=# select * from test;
id
----
(0 rows)
skrall=#
I have a postgres table. I need to delete some data from it. I was going to create a temporary table, copy the data in, recreate the indexes and the delete the rows I need. I can't delete data from the original table, because this original table is the source of data. In one case I need to get some results that depends on deleting X, in another case, I'll need to delete Y. So I need all the original data to always be around and available.
However it seems a bit silly to recreate the table and copy it again and recreate the indexes. Is there anyway in postgres to tell it "I want a complete separate copy of this table, including structure, data and indexes"?
Unfortunately PostgreSQL does not have a "CREATE TABLE .. LIKE X INCLUDING INDEXES'
New PostgreSQL ( since 8.3 according to docs ) can use "INCLUDING INDEXES":
# select version();
version
-------------------------------------------------------------------------------------------------
PostgreSQL 8.3.7 on x86_64-pc-linux-gnu, compiled by GCC cc (GCC) 4.2.4 (Ubuntu 4.2.4-1ubuntu3)
(1 row)
As you can see I'm testing on 8.3.
Now, let's create table:
# create table x1 (id serial primary key, x text unique);
NOTICE: CREATE TABLE will create implicit sequence "x1_id_seq" for serial column "x1.id"
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "x1_pkey" for table "x1"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "x1_x_key" for table "x1"
CREATE TABLE
And see how it looks:
# \d x1
Table "public.x1"
Column | Type | Modifiers
--------+---------+-------------------------------------------------
id | integer | not null default nextval('x1_id_seq'::regclass)
x | text |
Indexes:
"x1_pkey" PRIMARY KEY, btree (id)
"x1_x_key" UNIQUE, btree (x)
Now we can copy the structure:
# create table x2 ( like x1 INCLUDING DEFAULTS INCLUDING CONSTRAINTS INCLUDING INDEXES );
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "x2_pkey" for table "x2"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "x2_x_key" for table "x2"
CREATE TABLE
And check the structure:
# \d x2
Table "public.x2"
Column | Type | Modifiers
--------+---------+-------------------------------------------------
id | integer | not null default nextval('x1_id_seq'::regclass)
x | text |
Indexes:
"x2_pkey" PRIMARY KEY, btree (id)
"x2_x_key" UNIQUE, btree (x)
If you are using PostgreSQL pre-8.3, you can simply use pg_dump with option "-t" to specify 1 table, change table name in dump, and load it again:
=> pg_dump -t x2 | sed 's/x2/x3/g' | psql
SET
SET
SET
SET
SET
SET
SET
SET
CREATE TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
And now the table is:
# \d x3
Table "public.x3"
Column | Type | Modifiers
--------+---------+-------------------------------------------------
id | integer | not null default nextval('x1_id_seq'::regclass)
x | text |
Indexes:
"x3_pkey" PRIMARY KEY, btree (id)
"x3_x_key" UNIQUE, btree (x)
[CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } ] TABLE table_name
[ (column_name [, ...] ) ]
[ WITH ( storage_parameter [= value] [, ... ] ) | WITH OIDS | WITHOUT OIDS ]
[ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
[ TABLESPACE tablespace ]
AS query][1]
Here is an example
CREATE TABLE films_recent AS
SELECT * FROM films WHERE date_prod >= '2002-01-01';
The other way to create a new table from the first is to use
CREATE TABLE films_recent (LIKE films INCLUDING INDEXES);
INSERT INTO films_recent
SELECT *
FROM books
WHERE date_prod >= '2002-01-01';
Note that Postgresql has a patch out to fix tablespace issues if the second method is used
There are many answers on the web, one of them can be found here.
I ended up doing something like this:
create table NEW ( like ORIGINAL including all);
insert into NEW select * from ORIGINAL
This will copy the schema and the data including indexes, but not including triggers and constraints.
Note that indexes are shared with original table so when adding new row to either table the counter will increment.
I have a postgres table. I need to
delete some data from it.
I presume that ...
delete from yourtable
where <condition(s)>
... won't work for some reason. (Care to share that reason?)
I was going to create a temporary
table, copy the data in, recreate the
indexes and the delete the rows I
need.
Look into pg_dump and pg_restore. Using pg_dump with some clever options and perhaps editing the output before pg_restoring might do the trick.
Since you are doing "what if"-type analysis on the data, I wonder if might you be better off using views.
You could define a view for each scenario you want to test based on the negation of what you want to exclude. I.e., define a view based on what you want to INclude. E.g., if you want a "window" on the data where you "deleted" the rows where X=Y, then you would create a view as rows where (X != Y).
Views are stored in the database (in the System Catalog) as their defining query. Every time you query the view the database server looks up the underlying query that defines it and executes that (ANDed with any other conditions you used). There are several benefits to this approach:
You never duplicate any portion of your data.
The indexes already in use for the base table (your original, "real" table) will be used (as seen fit by the query optimizer) when you query each view/scenario. There is no need to redefine or copy them.
Since a view is a "window" (NOT a shapshot) on the "real" data in the base table, you can add/update/delete on your base table and simply re-query the view scenarios with no need to recreate anything as the data changes over time.
There is a trade-off, of course. Since a view is a virtual table and not a "real" (base) table, you're actually executing a (perhaps complex) query every time you access it. This may slow things down a bit. But it may not. It depends on many issues (size and nature of the data, quality of the statistics in the System Catalog, speed of the hardware, usage load, and much more). You won't know until you try it. If (and only if) you actually find that the performance is unacceptably slow, then you might look at other options. (Materialized views, copies of tables, ... anything that trades space for time.)
A simple way is include all:
CREATE TABLE new_table (LIKE original_table INCLUDING ALL);
Create a new table using a select to grab the data you want. Then swap the old table with the new one.
create table mynewone as select * from myoldone where ...
mess (re-create) with indexes after the table swap.