I want to run a Bigquery SQL query using insert method.
I ran the following code just like so:
JobConfigurationQuery = Google::Apis::BigqueryV2::JobConfigurationQuery
bq = Google::Apis::BigqueryV2::BigqueryService.new
scopes = [Google::Apis::BigqueryV2::AUTH_BIGQUERY]
bq.authorization = Google::Auth.get_application_default(scopes)
bq.authorization.fetch_access_token!
query_config = {query: "select colA from [dataset.table]"}
qr = JobConfigurationQuery.new(configuration:{query: query_config})
bq.insert_job(projectId, qr)
and I got an error as below:
Caught error invalid: Job configuration must contain exactly one job-specific configuration object (e.g., query, load, extract, spreadsheetExtract), but there were 0:
Please let me know how to use the insert_job method.
I'm not sure what client library you're using, but insert_job probably takes a JobConfiguration. You should create one of those and set the query parameter to equal your JobConfigurationQuery you've created.
This is necessary because you can insert various jobs (load, copy, extract) with different types of configurations to this one API method, and they all take a single configuration type with a subfield that specifies which type and details about the job to insert.
More info from BigQuery's documentation:
jobs.insert documentation
job resource: note the "configuration" field and its "query" subfield
Related
I am trying to export large amounts of data from snowflake into a CSV. I saw a similar question and the solution given was to “Run the query as part of a COPY INTO {location} command to an internal stage, and then use a GET command to pull it down locally.”
I tried following the guide and ran the following but receives the error, “SQL compilation error: syntax error line 4 at position 3 unexpected 'file_format'.”
I am not sure how to fix this or even if the first part of my syntax is correct. Can someone please help.
copy into #my_stage/result/data_ from (select *
from"IRIS"."PRODUCTION"."VW_ALL_IIS_LHJ"
where (RECIP_ADDRESS_COUNTY = 06065 or ADMIN_ADDRESS_COUNTY = 06065)
file_format=(TYPE='CSV');
[ HEADER = TRUE]
get #%my_stage/result/data.csv/;
I'm pretty sure the issue is that you're missing a closing parenthesis. Try:
copy into #my_stage/result/data_ from (select *
from"IRIS"."PRODUCTION"."VW_ALL_IIS_LHJ"
where (RECIP_ADDRESS_COUNTY = 06065 or ADMIN_ADDRESS_COUNTY = 06065))
file_format=(TYPE='CSV');
[ HEADER = TRUE]
get #%my_stage/result/data.csv/;
Sorry - I don't have a way to test this.
You are missing a parentheses after the where clause. You opened a parentheses after the first FROM and then another one at the WHERE clause, but you only closed the WHERE parentheses.
Also, AFAIK, you don't need to call a get if the stage was properly set. The copy into command will place it in your stage, you then retrieve it from that stage but you can do this by the normal way of accessing the stage you specified. So if you sent it to a s3 bucket, you'd just access the resource from S3 as if it were any other file.
Lastly, remember there are many useful parameters you can indicate in the FILE_FORMAT, such as Record_delimiter, compression and how to handle nulls.
And remove the last semicolon after csv, that's going to cause another error because HEADER is not a valid instruction on its own.
Also you don't have to put HEADER = TRUE between brackets. Brackets in documentation mean it's an optional parameter.
In Amazon Neptune I would like to run multiple Gremlin commands in Java as a single transactions. The document says that tx.commit() and tx.rollback() is not supported. It suggests this - Multiple statements separated by a semicolon (;) or a newline character (\n) are included in a single transaction.
Example from the document show that Gremlin is supported in Java but I don't understand how to "Multiple statements separated by a semicolon"
GraphTraversalSource g = traversal().withRemote(DriverRemoteConnection.using(cluster));
// Add a vertex.
// Note that a Gremlin terminal step, e.g. next(), is required to make a request to the remote server.
// The full list of Gremlin terminal steps is at https://tinkerpop.apache.org/docs/current/reference/#terminal-steps
g.addV("Person").property("Name", "Justin").next();
// Add a vertex with a user-supplied ID.
g.addV("Custom Label").property(T.id, "CustomId1").property("name", "Custom id vertex 1").next();
g.addV("Custom Label").property(T.id, "CustomId2").property("name", "Custom id vertex 2").next();
g.addE("Edge Label").from(g.V("CustomId1")).to(g.V("CustomId2")).next();
The doc you are referring is for using the "string" mode for query submission. In your approach you are using the "bytecode" mode by using the remote instance of the graph traversal source (the "g" object). Instead you should submit a string script via the client object
Client client = gremlinCluster.connect();
client.submit("g.V()...iterate(); g.V()...iterate(); g.V()...");
Gremlin sessions
Java Example
After getting the cluster object,
String sessionId = UUID.randomUUID().toString();
Client client = cluster.connect(sessionId);
client.submit(query1);
client.submit(query2);
.
.
.
client.submit(query3);
client.close();
When you run .close() all the mutations get committed.
You can also capture the response from the Query reference.
List<Result> results = client.submit(query);
results.stream()...
You can also use the SessionedClient, which will run all queries in the same transaction upon close().
More information is here: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-sessions.html#access-graph-gremlin-sessions-glv
What is the use / purpose of logging menu in the
Settings -> Technical -> Database structure -> Logging
It seems that you can store all the log messages in that view (model ir.logging) as long as you use the parameter --log-db your_database_name when executing odoo-bin in the command line (or add log_db = your_database_name in your Odoo config file).
Check out Odoo 11 info about command line parameters: https://www.odoo.com/documentation/11.0/reference/cmdline.html
--log-db
logs to the ir.logging model (ir_logging table) of the specified database. The database can be the name of a database in the “current” PostgreSQL, or a PostgreSQL URI for e.g. log aggregation
This is the theory, but honestly, I was not able to make it work, and I did not waste much time in trying to know why.
EDIT
As #CZoellner says, it seems that log messages stored in ir_logging table (log messages you see clicking on the menuitem Settings -> Technical -> Database structure -> Logging) come only from scheduled actions. If you create an scheduled action which executes some Python code, you have the following available variables to use in your method code:
env: Odoo Environment on which the action is triggered.
model: Odoo Model of the record on which the action is triggered; is a void recordset.
record: record on which the action is triggered; may be void.
records: recordset of all records on which the action is triggered in multi-mode; may be void.
time, datetime, dateutil, timezone: useful Python libraries.
log: log(message, level='info'): logging function to record debug information in ir.logging table.
Warning: Warning Exception to use with raise To return an action, assign: action = {...}.
If you use the log one, for example:
log('This message will be stored in ir_logging table', level='critical')
That log message and its details will be stored in ir_logging table each time the scheduled action is executed (automatically or manually). This answers your question, but now I am wondering what is the parameter --log-db, as I have tested it and these log messages are stored in ir_logging no matter this parameter is set or not.
I am new to JSON Schema, and am trying to validate JSON based on the HL7-FHIR schemas. Data I think should be invalid (and that the official Java-based validator says are invalid) shows up as valid.
For example, {"dog": "food"} should be invalid, because when I run the validator, I get:
> java -jar org.hl7.fhir.validator.jar bad.json -defn definitions.json.zip
.. load FHIR from definitions.json.zip
.. connect to tx server # http://tx.fhir.org/r3
(vnull-null)
.. validate
*FAILURE* validating bad.json: error:1 warn:0 info:0
Fatal # $ (line 1, col2) : Unable to find resourceType property
But if I paste the fhir.schema.json file from here into a JSON Schema validator like the one here, and evaluate {"dog": "food"}, it's valid.
It's valid even if I supply a resourceType, which I thought might cause the restrictions to kick in. It's also valid if I copy an example I expect to be valid—say, this Practitioner example—and change some of the types (set name to be a string rather than an array, for example).
I'm not sure if I'm running into a problem with the HL7-FHIR JSON Schema in particular or with JSON Schemas in general. I believe my question is different than this one because it appears that we're up to release 3.0, and so the schema I'm using is updated.
I am trying to list the jdbcprovider list at cell scope but it also list the jdbcproviders at node and server scope, how to get rid off the providers at node and server scope from the list?
AdminConfig.list('JDBCProvider', AdminConfig.getid( '/Cell:CellV70A/'))
output:
'"DB2 Universal JDBC Driver Provider(cells/CellV70A/nodes/nodename|resources.xml#JDBCProvider_1302300228086)"\n"DB2 Universal JDBC Driver Provider(cells/CellV70A|resources.xml#JDBCProvider_1263590015775)"\n"WebSphere embedded ConnectJDBC driver for MS SQL Server(cells/CellV70A|resources.xml#JDBCProvider_1272027151294)"'
If you look at the help for the AdminConfig.list command:
wsadmin>print AdminConfig.help('list')
WASX7056I: Method: list
...
Method: list
Arguments: type, scope
Description: Lists all the configuration objects of the type named
by "type" within the scope of the configuration object named by "scope."
...
It says "within the scope". Since node and server-scoped JDBCProviders are within the scope of the cell, they are returned by your command. If you list all JDBCProviders at cell scope using the Admin Console and then look at the Command Assistance, you'll see something like:
Note that scripting list commands may generate more information than is displayed by the administrative console because the console generally filters with respect to scope, templates, and built-in entries. AdminConfig.list('JDBCProvider', AdminConfig.getid('/Cell:MyCell/'))
So you'll need to filter your return list similarly. You could throw together a very simple script to do so:
jdbcProviders = AdminConfig.list('JDBCProvider', AdminConfig.getid('/Cell:MyCell')).split('\r\n')
for jdbcProvider in jdbcProviders:
if "/nodes/" or "/servers/" in jdbcProvider:
continue
print jdbcProvider