I'm currently creating sharded tables daily using the Python API.
Is there a way to add the comments for the table's columns when creating the table ? I couldn't see it in the doc but it might still be implemented.
Thanks in advance.
From the REST API for BigQuery documentation, tables.Insert takes a table resource, and in the schema.fields[].description property, you can put in a description.
Related
I am trying to populate populate and collect metadata for the business in GBQ. Basically, the business doesn't have access to tables, we create authorised views for them that they use in their reports.
The problem is, if I populate the column description field in the table, the views based on that table won't inherit the metadata, the same with sharded tables.
There is going to be a degree of data entry to populate the metadata, but I would really like to be able to share it across related views.
Is it possible to automate BQ metadata in any way?
There are some different options to both get information about a table or a view (https://cloud.google.com/bigquery/docs/tables#getting_information_about_tables), and to update that information (https://cloud.google.com/bigquery/docs/updating-views#update-description).
Depending on your specific case you can use the bq command line or a programming language SDK to automatize the process for retrieving and updating BigQuery's metadata.
I am trying to see if I can set up a trigger system, so whenever a new row of data is populated in these tables A, B, and C --> it populates new rows into a new table I created (table D, for example)?
I'm using Bigquery. Does this platform allow this capability?
Not sure what kind of coding should be used for this...(Insert into, etc.)
Maybe late to the party, but this is possible now. https://cloud.google.com/blog/topics/developers-practitioners/how-trigger-cloud-run-actions-bigquery-events
Triggers are not supported on BigQuery, basically because they are not aligned to the intended use patterns of the product. You may also refer to this existing question.
There is a Feature Request in place for an interesting approach to trigger an action when rows are loaded to BigQuery, but currently there is no ETA for it.
You my want to consider Cloud Composer as different alternative, instead of using triggers, you may orchestrate your data ingestion tasks.
BigQuery is a data warehouse, and there is no trigger support.
Have a look at https://cloud.google.com/sql/ , maybe it will help you.
The kusto tutorial references a Logs table. Is this data available publicly somewhere so as to try kusto out on a very large data store?
The samples in the tutorial which reference the StormEvents table can be run against the Samples database in the Help cluster: https://dataexplorer.azure.com/clusters/help/databases/Samples
Data sets used in other samples (such as those referencing the Logs table or the Events table) are not publicly available at this moment.
Good morning all,
How to create employees table in the marklogic by using sql or javascript?
Thanks in advance,
Naveen
You do not create tables in MarkLogic. You insert individual structured documents (XML/JSON) where each document is as close as you would get to the idea of a table row in a relational database.
For use of SQL to query these documents, MarkLogic allows you to create views. These are similiar to views in SQL and expose specific elements/attributes/properties of your documents.
I think you need to read the section on SQL in MarkLogic as a starting point and then come back with more specific questions if you get stuck. https://docs.marklogic.com/guide/sql/intro
MarkLogic is schema agnostic, you can achieve a structure thru a Schema database to enforce document structure. Just create your XSD file and your document can be validated against that.
Check this link for more information:
https://developer.marklogic.com/learn/2007-04-schema
Since MarkLogic is schema-agnostic database, you can restrict the element/attribute or define the XML structure through XSD.
However if you want to create tables using SQL, you actually have to create views.
Please check more on https://docs.marklogic.com/guide/sql/intro.
N.B : If you are creating views, make sure range-index is created for all the elements or attributes that you wish to use.
How to create BigQuery Partition table in Bigquery WebUI. I know we can create from gcloud and api.
I searched webUI but couldn't find any option for creating partition table from WebUI.
Thanks,
We added support for this recently and expect the UI to show it once the changes are deployed. I will provide an update once that is done. Thanks!