Is there any way to set up a custom quota for each user?
Example:
User1 = 10 Tb,
User2 = 20 Tb
User3 (Project owner) = unlimited
There is not a default feature to preform this, but it is possible to export your billing to another Big Query table and keep a track of each user consumption. A very good example of this excercise is shown by Mike Zinni. https://medium.com/google-cloud/visualize-gcp-billing-using-bigquery-and-data-studio-d3e695f90c08
With this feed Cloud Functions with a result of consumption and revoke a user.
This solution might have several hours of delay before revoking the permission.
I like Nilo's answer - meanwhile you could also leverage BigQuery's cost controls:
When you run a query there are at least 2 projects involved: The project used to run the query (costs go here), and the project/dataset containing the data (which could be the same project too).
You can set up cost controls that apply to every user in one project. You can give users in said project permission to query data which lives in a different project. For users in this project, set up a custom quota (https://cloud.google.com/bigquery/cost-controls).
For different group of users, make them members of different projects. Each of these projects can have a different custom quota per user. Then give them all permission to query the project containing the data.
Related
I have a project with more than 10 datasets and one dataset has more than 70 tables(tables created for beginner level and intermediate level). If I share my project publicly, my project name and project id will be exposed to many students. Is it safe to do like this? Will I be hacked or attacked by using the project name and project id by others?
As long as you only give the role dataViewer, the people whom you shared the project with will only be able to get the data from your tables, not modify it or insert jobs (that's what actually has cost).
This will force users to use their own projects to run jobs querying your datasets.
However, I would suggest you not to share this completely publicly. If you're using G Suite in your school, or if you know the Google accounts (most likely Gmail) from your students, you should create a Google group with the people who requires the access, and then giving permission to that group.
I already have access to Google analytics provided by my client and the bigquery has been configured to the project. But i want to know if i can create jobs. How do i find the role assigned to my id ?
i want to know if i can create jobs
Below is simple way to get this:
Just open Web UI and try to switch to project of your interest
a. If you do have it in the list of available projects – just select it and then run (just in case) some simple query (SELECT 1)
If it is run successfully - you can create jobs in this project (because any query is in reality a job)
b. If it is not in the initial list – select “Display Project” and enter project of your interest and also check “Make this my current project” box. If result is successful – most likely you again lucky and can create jobs in this project (but still – run some simple query to be 110% sure
How do i find the role assigned to my id
This would be more involved – you will need to use respective IAM (Google Identity and Access Management) APIs
For example you can use testIamPermissions() API that allows you to test Cloud IAM permissions on a user for a resource. It takes the resource URL and a set of permissions as input parameters, and returns the set of permissions that the caller is allowed.
The permission you should look for is bigquery.jobs.create, but yo can pass to this API list of any permissions you want to check if you have
How can I change the permissions in CKAN, so every editor/admin of an organization can add a group to a dataset (right now, the editor or admin has to be a member of the group to be able to add a certain group to a dataset)?
The dataportal I am developing only has 8 groups and every admin/editor should be able to add these groups to a dataset.
It seems to me that Group permissions are a bit of a hangover from the past (there was a time in CKAN without Organizations and only groups, and groups are based heavily on Organizations internally) and I wanted to make a proposal to the CKAN developers for providing an option to remove them (i.e. every Org editor or Admin can add anything to any group). Apart from that possibility there are a couple of workarounds:
Make everyone of these users a superuser (probably bad)
Make a "master" group which contains all existing groups (this is only
possible via the API) and then use the cascading authorization
configuration:
http://docs.ckan.org/en/latest/maintaining/configuration.html#ckan-auth-roles-that-cascade-to-sub-groups
Make an extension that hooks into the user creation process and
automatically adds all Org editors and Admins to all groups
Make an
extension that doesn't check or relaxes auth when trying to add
datasets to groups (this seems like overkill compared to the general
change I mentioned at the beginning: both would need to be coded and
I am more interested in doing the first one!)
I know if any user query data from a dataset, that person will get billed for the query, while the project will get billed for the storage. Is it possible to set it up so the project or billing account which creates the project get billed for the query instead of the person who did the query?
I guess one solution to this is to create a service account and have that service account does the querying through a web app.
Who is billed for BigQuery queries?
it is not the user per se who is being billed for querying data - it is rather project (from which query is being executed) get's billed. Of course if that given user happen to be a billing owner - it means that the user gets billed.
The only way I know to change this is:
be added (as Project Viewer at least ) to the project that queryable data is in
log into that project and execute query from within that project
Note if you are in Web UI - the active project becomes a billing project. and if you are using bq command line - you need to set default or billing project using respectve flag
Does anyone know of some good resources related to setting up heirarchical user account systems? I'm currently setting one up and am struggling with some of the more complex logic (especially with determining permissions). I was hoping I might be able to find some resources to help me along.
Some Background:
I'm building a user account system for a web CMS that allows for a nested group hierarchy. Each group can be allowed/denied access to read, write, add, and delete (either explicitly for that group, or implicitly by one of its parents). As if that weren't complicated enough, the system also allows for users to be members of multiple groups. -- This is where I'm stuck. I've got everything set up, but I'm struggling with the actual logic for determining pemissions for a given user.
The manual for CakePHP has an excellent description of how Access Control Lists work.
http://book.cakephp.org/2.0/en/core-libraries/components/access-control-lists.html
Represent the permissions set for a given group as a bit mask. OR-ing the bit masks together will give you the resultant permission set.
Update for #Alex:
I wrote this answer 3 years ago, but I believe I was alluding to the following...
From the question
a nested group hierarchy. Each group can be allowed/denied access to
read, write, add, and delete (either explicitly for that group, or
implicitly by one of its parents). As if that weren't complicated
enough, the system also allows for users to be members of multiple
groups. -- This is where I'm stuck. I've got everything set up, but
I'm struggling with the actual logic for determining pemissions for a
given user.
Assign a bitmask matching the total permission set of a group (or role) in the system:
e.g. 00 (using two bits keeps it simple here!)
The first bit confers Permission A and the second Permission B.
Now say Group A confers the following permission set: 01.
... and say Group B confers the following permission set: 10.
To get the resultant permission set for a user in an arbitrary set of groups you could perform a logical OR on the permission set bit masks:
Permission set for Group A 01
Permission set for Group B 10 OR
----
Resultant permission set 11 (i.e. both permission A and B are conferred)
I do not know the details of the questioner's system, but the system outlined here could be augmented to achieve different group-composition behaviors using different logical operators.
Look at the permissions in the Andrew File System. It allows users to create and administer groups of their own, while selectively assigning admin rights and ACLs. You might find that many of the pesky details are already worked out for you in their model.
Edit: here's a better link to AFS documentation:
http://www.cs.cmu.edu/~help/afs/index.html
Here's the section on groups:
http://www.cs.cmu.edu/~help/afs/afs_groups.html
I've done exactly this before and its no trivial implementation. You're going to want to look at the SecurityPermission class.
[http://msdn.microsoft.com/en-us/library/system.security.permissions.securitypermission.aspx][1]
I have done this before by utilizing XML (which I'm not sure I'd do again) and storing that XML as permission list inside of SQL server in an XML column through a CLR stored proc. The XML would have an element called a "permission" and then the permission would actually be a ENUM inside of the code. Each permission was a new implementation of the SecurityPermission class (linked above) Users were tied to groups which were defined in SQL server and then as the user was added/removed to groups, the XML doc would get updated to reflect which groups they were apart of.
As soon as the user logged in, the users credentials would be loaded into the application store (session) and then would be accessed accordingly. When authorization needed to take place the XMl in the application store would be pulled down loaded into the SecurityPermission via the "FromXML" method. At that point I would use the following methods to determine if the user had permission:
Demand
Intersect
Union
IsUnrestricted
IsSubSetOf
etc., etc, etc.
At that point after performing the Demand I was able to determine if the caller had access according to how I implemented my security routines in the SecurityPermissions.
Again, this is leaving out a TON of detail, but this should get you going down the right path.
Take a look at this name space as well: [2]: http://msdn.microsoft.com/en-us/library/system.security.permissions.aspx "System.Security.Permissions"