In my current dbt project, I run everything on the same Google cloud project (let's say project : dataA). Since datasets becomes a lot, I decide to split the project into 2: The current project for import of raw data and a new project (for example : dataB) for production environment where I stock all data marts.
I use a service account to manage the lecture or editing data sources for both two projects. And I am sure that there is no issues on rights. The profile setting is quite similar to my current settings which work fine.
But I am experiencing some Database Error issues from dbt say that I don't have Insufficient Permission.
Does anyone have an idea about the reason of the issue? And how to fix it?
Many thanks!
Related
We've made use of environment specific migrations for things like seeding data, data correction, applying table grants. There are times when we'd like to take a copy of production, for example, and import it to another lower environment, either as a periodic refresh, or to start a new test environment. However, as expected, we end up with various failures like Detected applied migration not resolved locally and Detected resolved migration not applied to database. I see there are various flags (ignoreIgnoredMigrations, ignoreMissingMigrations and outOfOrder) to allow us to bypass these issues.
Are there best practices for handling scenarios like I described? Is there a way to run an environment specific migration that doesn't file an entry in the flyway_schema_history table? Other approaches to this issue that I haven't mentioned?
Thanks in advance for any insights.
We have used ignoreMissingMigrations as one approach around this issue.
I am new to using BigQuery. I tried to select my trial project vertical-idea-303617 and run a query against it. But the browser keeps complaining about permissions related to erudite-buckeye-3032181. I don't know what that erudite-buckeye-3032181 project is and I do not have it selected. Why am I still getting errors? See this image here:
How do I run queries against my vertical-idea-303617.perftest.reportgraphs ?
I had a similar problem, not exactly the same, and to solve it I just had to click that Hide Preview Features button.
apparently Google released some new features that are all broken in the new Front-end, but not in the old
I created a project on Google BigQuery and enabled billing.
Went on to create few datasets that were shared with my team members (Can EDIT premissions).
However, my team mates are unable to load data into the respective datasets shared with them. Whenever they try it says billing not enabled for this project.
I am able to load data into the datasets but not my team.
It's been more than 24 hours
Thanks in advance
Note that in order to load data, they need to run a load job, and that load job needs to be run in a project. Perhaps billing is not enabled on the project they are using?
You can give your team members read access to the project (or greater) to allow them to run jobs in your own billing-enabled project.
You can share a BigQuery project at the project level and at the dataset level.
See https://developers.google.com/bigquery/access-control.
I assume you are sharing at the dataset level. Can you try sharing the project instead with your team members? (here: https://cloud.google.com/console/project)
Please report back!
So, we have this one project which uses Cloud Storage and BigQuery as services. All has been well.
Then, I wanted to add Cloud SQL to this project to try it out. It asked for a unique Project ID so I gave it one. (The Project ID is different than the Project Number.)
Ever since then, I've been having a difficult time accessing my BigQuery tables. When I go to the BigQuery web interface, the URL contains the Project ID instead of the original Project Number. It shows the list of datasets, but now shows the Project Number before each dataset name and the datasets are greyed out and inaccessible. If I manually change the URL to contain the Project Number instead of the Project ID, it appears to work although it shows the list of datasets in the left nav twice, one set greyed out and inaccessible and the other set seemingly accessible.
At the same time, some code that I've been successfully using in Apps Script that accesses BigQuery is now regularly failing with a generic "We're sorry, a server error occurred. Please wait a bit and try again." I'm not sure if this is related to the Project ID/Project Number confusion, or if it's just a Red Herring.
Since we actively use the Cloud Storage service of this project, I am trying to be cautious with further experimentation with this project. I'm not sure if I should delete the Cloud SQL service in this project to get it back to the way it was, or if this is a known issue with some back-end solution. Please advise.
After setting the project id, there can be a delay where BigQuery picks up the change. It should happen within 15 minutes or so, but sometimes it takes longer.
If you send the project ID I can make sure it has been updated.
I'm using MigratorDotNet to manage Rails-style migrations for my web app. I have a workflow where, if I delete all the tables in the database, I can access an installation view that will run MigratorDotNet and create all the necessary tables.
This works locally. For some reason, when I upload my code to my Arvixe hosting, the migrations just never run. I get this odd error:
There is already an object named 'SchemaInfo' in the database.
This is odd because, prior to running migrations, I manually deleted all the tables in the database (to make sure it wasn't left over from a previous install).
My code essentially boils down to:
new Migrator.Migrator("SqlServer", connectionString.ToString(), migrationsAssembly).MigrateToLastVersion();
I've already verified by logging that the connection string is correct (production/hosting settings), and the assembly is correctly loaded (name and version).
Works locally, but not on Arvixe. How do I troubleshoot this?
This is a dark day.
It turns out (oddly) that the root cause was my hosting company used a schema other than dbo for my database. Because of this, the error message I saw (SchemaInfo already exists) was talking about their table.
My solution, unfortunately, was to rip out MigratorDotNet and go with FluentMigator instead. not only did this solve the problem, but it also gave me a more intelligible error message (one referring to the schema names).
While it doesn't seem possible to auto-set the schema, and while I need to switch the schema on my dev vs. production machine, it's still a solvable problem (and a better API, IMO). I googled, but did not find any way to change the default schema in migratordotnet.
I'm sorry for the issues that you were having. On shared hosting, unfortunately the only way that we may be able to change the schema is manually. If you are still looking for a solution that requires our assistance, please forward your ticket ID to qa .at. arvixe.com as well as arvand .at. arvixe.com and we can look into the best way to resolve this.