I need to locate a guild that shows how to get the dataset from bigquery-public-data to a table under my projects.
You could use
select * from dataset.tablename
using this query your could create a table under your project name and dataset.
Related
bigquery-public-data.samples.shakespeare
just from below example;
-- Which names appear as words in Shakespeare's plays?
SELECT
name AS shakespeare_name
FROM top_names
WHERE name IN (
SELECT word
FROM bigquery-public-data.samples.shakespeare
);
BigQuery tables are stored in datasets and datasets are set up within projects.
The dots separate project id, dataset name and table name. In your example, "bigquery-public-data" is the project id, "samples" is the dataset and "shakespeare" is the name of the table.
You can validate this visually when you look at the explorer and where the "shakespeare" table is located:
I have a table in AWS Athena which contains 2 records. Is there a SQL query using which a new column can be inserted in to the table?
You can find more information about adding columns to table in Athena documentation
Or you can use CTAS
For example, you have a table with
CREATE EXTERNAL TABLE sample_test(
id string)
LOCATION
's3://bucket/path'
and you can create another table from sample_test with the query
CREATE TABLE new_test
AS
SELECT *, 'new' AS new_col FROM sample_test
You can use any available query after AS
This is mainly for future readers like me, who was struggling to get this working for Hive table with AVRO data and if you don't want to create new table i.e updating schema of the existing table. It works for csv using 'add columns', but not for Hive + AVRO. For Hive + AVRO, to append columns at the end, before partition columns, the solution is available at this link. However, there are couple of things to note that, we need to pass full schema to the literal attribute and not just the changes; and (not sure why but) we had to alter hive table for all 3 things in the same order - 1. add columns using add columns 2. set tblproperties and 3. set serdeproperties. Hopefully it helps someone.
How to create a table on the basis of result of select query in BigQuery?
For example:
create table abc as select x,y,z from mnp;
Is there any way or workaround to achieve the same in BigQuery?
Any leads?
Just try giving the GCP dataset name before table name in create table statement.
Example:
create table `GCP_Dataset.abc` as select x,y,z from mnp
CREATE OR REPLACE TABLE [GCP_Folder].abc AS SELECT x,y,z FROM mnp
This wont get the error Cannot set write disposition in jobs with DDL statements. no source to refer, its trial and error from my end.
already it can be done like
CREATE OR REPLACE TABLE project.dataset.table AS SELECT x,y,z FROM 'project.dataset.mnp'
You can create a table using another table as the starting point.
This method basically allows you to duplicate another table (or a part
of it, if you add a WHERE clause in the SELECT statement).
CREATE TABLE project_name.dataset_name.table (your destination) AS SELECT column_a,column_b,... FROM (UNION/JOIN for example)
Actually, I want to move one table to another database.
But spark don't permit this.
Then, how to copy table by spark-sql?
I already tried this.
SELECT *
INTO table1 IN new_database
FROM old_database.table1
But it was not working.
maybe try:
CREATE TABLE new_db.new_table AS
SELECT *
FROM old_db.old_table;
To preserve partitioning and storage format do the following-
Get the complete schema of the existing table by running-
show create table db.old_table
The above query will output the table schema which you can just execute after changing the path name and table name.
Then insert all the rows into the new blank table using-
insert into db.new_table select * from db.old_table
The following snippet will create a new table while preserving the definition of the "old" table.
CREATE TABLE db.new_table LIKE db.old_table;
For more info, check the doc's CREATE TABLE.
I want to create a table in sql using the columns details (name, data type etc.) stored in anther table in the database.
Depending on the database you can use the information schema tables. They hold the information you are looking for. Look for the table that describes the columns.
Postgres: http://www.postgresql.org/docs/8.4/interactive/information-schema.html
MySQL: http://dev.mysql.com/doc/refman/5.0/en/information-schema.html
You can query these tables and use 'select into' to insert the results into your other table.
One opinion is to create CREATE TABLE query and execute it in ADO.NET like shown here this
Try out this code
CREATE TABLE new_table
AS
SELECT *
FROM old_table
WHERE 1 = 2;