Read-only access for Heroku Postgres Databases - sql

I have a Heroku Postgres database for my application. I can easily access the psql shell using the provided DATABASE_URL from Heroku
psql $(heroku config:get DATABASE_URL -a my_app)
psql (9.6.1)
SSL connection (protocol: TLSv1.2, cipher: ECDSE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.
d5i032ahpfiv07=>
And by default, this user seems to have full access to update and drop tables
d5i032ahpfiv07=> SELECT
table_name,
string_agg(privilege_type, ', ') as privileges
FROM information_schema.role_table_grants
WHERE table_schema = 'public'
AND grantee = current_user
GROUP BY 1
;
table_name | privileges
-----------------------------------+---------------------------------------------------------------
articles | INSERT, TRIGGER, REFERENCES, TRUNCATE, DELETE, UPDATE, SELECT
comment_flags | TRIGGER, INSERT, SELECT, UPDATE, DELETE, TRUNCATE, REFERENCES
comment_likes | TRUNCATE, REFERENCES, TRIGGER, INSERT, SELECT, UPDATE, DELETE
comments | INSERT, SELECT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER
communities | TRIGGER, REFERENCES, TRUNCATE, DELETE, UPDATE, SELECT, INSERT
.....
I'd love to set up "Read Only" users in the database so that people can view this data without fear of running "DROP TABLE" or some other destructive command.
How can I go about setting up a read-only user on Heroku Postgres?
Thank you!

Create a Heroku Postgres Follower Database
A database follower is a read-only copy of the leader database that
stays up-to-date with the leader database data. As writes and other
data modifications are committed in the leader database, the changes
are streamed, in real-time, to the follower databases.
Then simply run your analytics, dataclips, and other read-only applications against the follower. This is a very standard configuration that safeguards your main DB, and has added performance benefits: you can hammer it with queries (which are often intensive and have different cache profiles) without affecting your user-facing application.

Adam's suggestion to create a follower database is usually the right solution for this, but it's possible to create a read-only role (or any other permissions you want), for any Heroku Postgres database, even one that isn't a follower.
Heroku has detailed documentation for this at Heroku Postgres Credentials, but briefly:
Create Heroku credential
heroku pg:credentials:create <addon_name> --name new_credential -a <app_name>
<addon_name> is the name of your Postgres addon, which you can find by running heroku pg:info -a <app_name> and checking the "Add-on" line in the output.
The new credential is named new_credential in the above command, but you can use whatever name you want.
Set permissions
Creating the new_credential credential also creates a new_credential role (see Postgres roles documentation) in your Postgres database. You can set the permissions for new_credential like any other role, using psql. By default, the role has no permissions. The Heroku docs give this example, to run in psql:
GRANT USAGE ON SCHEMA PUBLIC TO new_credential;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO new_credential;
ALTER DEFAULT PRIVILEGES IN SCHEMA public
GRANT SELECT ON TABLES TO new_credential;
The above sets the new_credential role to have read-only privileges. See the Postgres documentation on privileges for more details.
Attach the credential
Before you can use the credential, you need to attach it to your app:
heroku addons:attach <addon_name> --credential new_credential -a <app_name>
When you run the addons:attach command, the output will include a new config var named HEROKU_POSTGRESQL_<COLOUR>_URL, where <COLOUR> is some colour. This config var holds a Postgres connection URL you can use to connect to the database using the new_credential role.
Using the credential
In an app, connect to the database using the new config var created in the last step instead of the default DATABASE_URL (which still contains the default, read-write credentials).
You can also use the new role in psql:
heroku pg:psql <addon_name> --credential new_credential -a <app_name>
Or, if you want to use psql directly as in your example, you can use the new colour URL:
psql $(heroku config:get HEROKU_POSTGRESQL_<COLOUR>_URL -a <app_name>)
When you connect to Postgres using the new role, any attempts to create, update, or drop data will be rejected by Postgres due to the role's permissions.

It has kind of already been said, but this is a simple solution for those who don't want to get real technical.
https://data.heroku.com/dataclips
You can just create dataclips here for each or any individual Heroku app that you have. You run whatever queries you like, and then you can share it via URL, CSV, JSON, Google Sheet, or you can even add individual people or teams to have access to them.
It's nice cause you have the Heroku web UI to use (rather than through CLI), and it doesn't necessarily give someone complete access to the DB, just what you want them to see; which is suitable for some people's specific needs/scenarios.

Related

Can't limit Postgres user access

I am trying to automate creating databases and users in Postgres.
Currently, after I create the databases and users for each database, all users seem to have admin access and can do and see anything, including other databases.
Here is the SQL I'm running:
CREATE DATABASE MY_DB WITH ENCODING 'UTF8';
CREATE USER MY_DB_ADMIN WITH ENCRYPTED PASSWORD 'SUPER_SECRET_PASSWORD';
GRANT ALL PRIVILEGES ON DATABASE MY_DB TO MY_DB_ADMIN;
I'm relatively new to Postgres, so not sure if this is a Postgres nuance thing or SQL in general.
Thanks in advance
Update 1
By "do anything", I mean, I am able to perform selects, create tables etc in other databases.
I have now tried this:
REVOKE ALL ON DATABASE MY_DB FROM MY_DB_ADMIN;
GRANT CREATE, CONNECT, TEMPORARY ON DATABASE MY_DB TO MY_DB_ADMIN;
But this still doesn't work.
User DB_ONE_ADMIN is able to create tables in DB_TWO
GRANT ALL PRIVILEGES ON DATABASE MY_DB TO MY_DB_ADMIN;
The problem is with the last line here, you are granting all privileges on database MY_DB to MY_DB_ADMIN. You need to decide on the access levels for different users if this is something you don't want.
If restriction needs to be applied and current privilege needs to be revoked then use REVOKE command

Postgres - new role with SELECT shows empty table, whereas superuser role shows data in same table

thank for taking the time to try answer/understand this question.
I am using AWS Aurora Postgres (Engine version: 13.4) database.
I referred to this document for creating readwrite and readonly roles for 2 new rdsiam users -> "dev_ro" and "dev_rw". I have granted readwrite role to "dev_rw" and readonly to "dev_ro". The additional changes are:
myschema is "public" - which is my default schema
I add the same permissions as "myschema" to another schema called "graphile_worker" (from graphile/worker - which is a job queue).
With this in mind, here is what I have done:
I run my application which adds some repeating jobs (jobs schedule itself), implying that the jobs table can never be empty
Connect to RDS using the IAM user (doesn't matter dev_ro or dev_rw)
I run SELECT * FROM graphile_worker.jobs in my IDE (dbeaver - shouldn't matter, I think)
The table shows up empty
Disconnect and Re-connect to RDS using superuser credentials (which are created when server is created)
Run same query as above
See data in the table
I don't know why this is happening.
I double-checked, both "dev_ro/w" (through the roles) and superuser, have:
CONNECT to database (without doubt)
SELECT on all tables of graphile_worker schema
USAGE on the graphile_worker schema
Moreover, I can query graphile_worker.migrations and the migration records show up as expected (on both devro/w and superuser)!
Please let me know if there is any more information that I can provide to help debug this issue.
Removing Row-Level Security (RLS) solved this issue.
Thanks #Hambone for asking the right question.
RLS is removed by executing
ALTER ROLE <username> WITH BYPASSRLS

Grant privileges for a particular database in PostgreSQL

I'm moving from MySQL to PostgreSQL and have hit a wall with user privileges. I am used to assigning a user all privileges to all tables of a database with the following command:
# MySQL
grant all privileges on mydatabase.* to 'myuser'#'localhost' identified by 'mypassword';
It appears to me that the PostgreSQL 9.x solution involves assigning privileges to a "schema", but the effort required of me to figure out exactly what SQL to issue is proving excessive. I know that a few more hours of research will yield an answer, but I think everyone moving from MySQL to PostgreSQL could benefit from having at least one page on the web that provides a simple and complete recipe. This is the only command I have ever needed to issue for users. I'd rather not have to issue a command for every new table.
I don't know what scenarios have to be handled differently in PostgreSQL, so I'll list some of the scenarios that I have typically had to handle in the past. Assume that we only mean to modify privileges to a single database that has already been created.
(1a) Not all of the tables have been created yet, or (1b) the tables have already been created.
(2a) The user has not yet been created, or (2b) the user has already been created.
(3a) Privileges have not yet been assigned to the user, or (3b) privileges were previously assigned to the user.
(4a) The user only needs to insert, update, select, and delete rows, or (4b) the user also needs to be able to create and delete tables.
I have seen answers that grant all privileges to all databases, but that's not what I want here. Please, I am looking for a simple recipe, although I wouldn't mind an explanation as well.
I don't want to grant rights to all users and all databases, as seems to be the conventional shortcut, because that approach compromises all databases when any one user is compromised. I host multiple database clients and assign each client a different login.
It looks like I also need the USAGE privilege to get the increasing values of a serial column, but I have to grant it on some sort of sequence. My problem got more complex.
Basic concept in Postgres
Roles are global objects that can access all databases in a db cluster - given the required privileges.
A cluster holds many databases, which hold many schemas. Schemas (even with the same name) in different DBs are unrelated. Granting privileges for a schema only applies to this particular schema in the current DB (the current DB at the time of granting).
Every database starts with a schema public by default. That's a convention, and many settings start with it. Other than that, the schema public is just a schema like any other.
Coming from MySQL, you may want to start with a single schema public, effectively ignoring the schema layer completely. I am using dozens of schema per database regularly.
Schemas are a bit (but not completely) like directories in the file system.
Once you make use of multiple schemas, be sure to understand search_path setting:
How does the search_path influence identifier resolution and the "current schema"
Default privileges
Per documentation on GRANT:
PostgreSQL grants default privileges on some types of objects to
PUBLIC. No privileges are granted to PUBLIC by default on tables,
columns, schemas or tablespaces. For other types, the default
privileges granted to PUBLIC are as follows: CONNECT and CREATE TEMP TABLE
for databases; EXECUTE privilege for functions; and USAGE privilege for languages.
All of these defaults can be changed with ALTER DEFAULT PRIVILEGES:
Grant all on a specific schema in the db to a group role in PostgreSQL
Group role
Like #Craig commented, it's best to GRANT privileges to a group role and then make a specific user member of that role (GRANT the group role to the user role). This way it is simpler to deal out and revoke bundles of privileges needed for certain tasks.
A group role is just another role without login. Add a login to transform it into a user role. More:
Why did PostgreSQL merge users and groups into roles?
Predefined roles
Update: Postgres 14 or later adds the new predefined roles (formally "default roles") pg_read_all_data and pg_write_all_data to simplify some of the below. See:
Grant access to all tables of a database
Recipe
Say, we have a new database mydb, a group mygrp, and a user myusr ...
While connected to the database in question as superuser (postgres for instance):
REVOKE ALL ON DATABASE mydb FROM public; -- shut out the general public
GRANT CONNECT ON DATABASE mydb TO mygrp; -- since we revoked from public
GRANT USAGE ON SCHEMA public TO mygrp;
To assign "a user all privileges to all tables" like you wrote (I might be more restrictive):
GRANT ALL ON ALL TABLES IN SCHEMA public TO mygrp;
GRANT ALL ON ALL SEQUENCES IN SCHEMA public TO mygrp; -- don't forget those
To set default privileges for future objects, run for every role that creates objects in this schema:
ALTER DEFAULT PRIVILEGES FOR ROLE myusr IN SCHEMA public
GRANT ALL ON TABLES TO mygrp;
ALTER DEFAULT PRIVILEGES FOR ROLE myusr IN SCHEMA public
GRANT ALL ON SEQUENCES TO mygrp;
-- more roles?
Now, grant the group to the user:
GRANT mygrp TO myusr;
Related answer:
PostgreSQL - DB user should only be allowed to call functions
Alternative (non-standard) setting
Coming from MySQL, and since you want to keep privileges on databases separated, you might like this non-standard setting db_user_namespace. Per documentation:
This parameter enables per-database user names. It is off by default.
Read the manual carefully. I don't use this setting. It does not void the above.
Maybe you could give me an example that grants a specific user
select/insert/update/delete on all tables -- those existing and not
yet created -- of a specific database?
What you call a database in MySQL more closely resembles a PostgreSQL schema than a PostgreSQL database.
Connect to database "test" as a superuser. Here that's
$ psql -U postgres test
Change the default privileges for the existing user "tester".
ALTER DEFAULT PRIVILEGES IN SCHEMA public
GRANT INSERT, SELECT, UPDATE, DELETE ON TABLES
TO tester;
Changing default privileges has no effect on existing tables. That's by design. For existing tables, use standard GRANT and REVOKE syntax.
You can't assign privileges for a user that doesn't exist.
You can forget about the schema if you only use PUBLIC.
Then you do something like this: (see doc here)
GRANT { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER }
[, ...] | ALL [ PRIVILEGES ] }
ON { [ TABLE ] table_name [, ...]
| ALL TABLES IN SCHEMA schema_name [, ...] }
TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ]
I don't want to grant rights to all users and all databases, as seems to be the conventional shortcut, because that approach compromises all databases when any one user is compromised. I host multiple database clients and assign each client a different login.
OK. When you assign tables to the correct role, the privileges granted will be role-specific and not to all users! Then you can decide who to give roles to.
Create a role for each database. A role can hold many users.
Then assign a client-username to the correct role.
Also assign your-username to each role if needed.
(1a) Not all of the tables have been created yet, or (1b) the tables have already been created.
OK. You can create tables later.
When you are ready, assign tables to the correct client role.
CREATE TABLE tablename();
CREATE ROLE rolename;
ALTER TABLE tablename OWNER TO rolename;
(2a) The user has not yet been created, or (2b) the user has already been created.
OK. Create usernames when you are ready. If your client needs more than one username simply create a second client-username.
CREATE USER username1;
CREATE USER username2;
(3a) Privileges have not yet been assigned to the user, or (3b) privileges were previously assigned to the user.
OK. When you are ready to give privileges, create the user and assign the correct role to her.
Use GRANT-TO command to assign roles to users.
GRANT rolename TO username1;
GRANT rolename TO username2;
(4a) The user only needs to insert, update, select, and delete rows, or (4b) the user also needs to be able to create and delete tables.
OK. You run these commands to add permissions to your users.
GRANT SELECT, UPDATE, INSERT, DELETE ON dbname TO role-or-user-name;
ALTER USER username1 CREATEDB;

Hide databases in Amazon Redshift cluster from certain users

Is it possible to hide the existence of and access to databases (incl. their schemas, tables etc) from certain users within Amazon Redshift.
By default, it seems like every user is able to see other DBs even though he doesnt have permission to select data nor any other (non-default) privileges.
I tried
REVOKE ALL PRIVILEGES ON DATABASE testdb FROM testdbuser;
and similar but still testdbuser can connect to the testdb DB and even see all other objects in his object browser in a SQL tool (here: Aginity Redshift Workbench).
Ideally, testdbuser would not be able to see anything else except what he got explicitly granted access to.
Note, testdbuser is not a superuser.
Thanks!
Try to revoke from the PUBLIC group vs the specific user
REVOKE USAGE ON SCHEMA information_schema FROM PUBLIC;
REVOKE USAGE ON SCHEMA pg_catalog FROM PUBLIC; -- This should suffice, but...
REVOKE SELECT ON TABLE pg_catalog.pg_database FROM PUBLIC; -- just to be sure.
Note that this could have an undesirable effect on all users within the selected database.
You will need to do this on all databases, since the user can guess another database name and see pg_catalog information there.
The user could still find all the databases via a brute force attack simply by trying to switch or connect to all possible strings.
Unfortunately it is not possible today. Redshift does not support the REVOKE CONNECT FROM DATABASE command, so users can connect to any database.
Because Redshift is built on PostgreSQL, once connected, users can read a list of all databases in the cluster from the system tables, and by connecting to each database can read the list of schemas, tables, and even table columns from the system tables, even if they are prevented from reading the data within those tables through the use of REVOKE ... FROM SCHEMA or REVOKE ... FROM TABLE.

expdp on a different SID and use of EXCLUDE, INCLUDE and CONTENT switch

I am trying to import a db into my local xe instance. I have a problem in that it is greater than 11GB and things fall over. My db has audit tables (courtesy of envers). I do not need this data.
Two questions here:
can I expdp table structures and not data for the aud_ tables that are my audit tables
can I expdp on a different SID. My SID is not orcl but orcllo (for historical reason)
For the second question I have done the following but
sqlplus / as sysdba#orcllo
alter user MY_DB identified by MY_PASS ACCOUNT UNLOCK;
CREATE OR REPLACE DIRECTORY db_dumps AS '/tmp/db_dumps';
GRANT READ, WRITE ON DIRECTORY db_dumps TO MY_DB;
but when I run expdp I get an error about db_dumps not being found.
Thanks
Q.1:
You can use 'EXCLUDE' or 'INCLUDE' switch during oracle impdp (data pump) to escape/include few tables.
The following code shows how they can be used as command line parameters.
impdp scott/tiger#db10g schemas=SCOTT exclude=TABLE:"IN ('EMP', 'DEPT')" directory=TEST_DIR dumpfile=SCOTT.dmp logfile=expdpSCOTT.log
Same can be done during export as well:
expdp scott/tiger#db10g schemas=SCOTT exclude=TABLE:"IN ('EMP', 'DEPT')" directory=TEST_DIR
CONTENT=METADATA_ONLY switch should be used if you indeed want to create structure on target schema/db.
Q.2:
You can not use expdp / impdp on a different SID as is but via DB Link route (using NETWORK_LINK switch).
However, i think, Ans-1 should address your real problem due to Oracle XE limitations. Moreover, there may be performance issues due to cross network/DB data flow.
Both the local and remote users require the IMP_FULL_DATABASE role granted to them.
impdp test/test#db10g tables=SCOTT.EMP network_link=REMOTE_SCOTT directory=TEST_DIR logfile=impdpSCOTT.log remap_schema=SCOTT:TEST
HTH
Check the following for more options with data pump:
http://www.oracle-base.com/articles/10g/oracle-data-pump-10g.php