I wrote a User Defined function in BigQuery SQL and the documentation doesn't show how to view an existing function code. In other engines, the sintax is something like SHOW FUNCTION ... but that doesn't work here.
Thanks.
You can try to use INFORMATION_SCHEMA views to view the source code and other parameters of the UDF:
#standardSQL
select *
from <project_id>.<dataset_name>.INFORMATION_SCHEMA.ROUTINES;
Pay attention to the location of your UDF, because INFROMATION_SCHEMA view results depend on the location of objects.
You can create function as below :-
CREATE FUNCTION `project.dataset.Function`(x INT64) AS (x * 3);
Use of function will be as :-
select `project.dataset.Function` (Coll) from `project.dataset.Table`
To view function you need to click on function on dataset, you will get entire code.
Related
I am trying to convert our legacy functions into standard SQL and so by using UDFs as a gig to reuse the old function names with the native functions.
Here I have my UDF to_char and inside use the parameters to run FORMAT_DATE but when I execute it even though a routine is created I get errors with no additional info.
Once the rountine has been created and I comment out the code the errors come as such function to_char not found
If I run a CREATE TEMP FUNCTION I get really long loading times(ca 30 - 90secs) and that obviously cant be right.
CREATE OR REPLACE FUNCTION mytable.TO_CHAR(dstamp DATETIME, fmt STRING)
RETURNS STRING AS
(
FORMAT_DATE(fmt, dstamp)
);
SELECT
TO_CHAR(End_Time, 'yyyy-mm') AS Month_and_Year,
I don't want to create a temp function. Ideally I'd like to have a routine that I can just keep calling when need be but I am messing something up with my UDF and the FORMATE_DATE function. Any help would be much appreciated, thank you!
btw running this in GBQ.
You need reference UDF with dataset prefix, otherwise BigQuery think it is a builtin function or temporary UDF. In your case, just change TO_CHAR to mytable.TO_CHAR. Assuming your mytable is actually a dataset.
SELECT mytable.TO_CHAR(End_Time, 'yyyy-mm') AS Month_and_Year,
I am trying to define a sql view that picks a subset of elements from a source struct data type and creates a new struct. In hive I can do this:
create view myview as
select
id,
named_struct("cnt", bkg.cnt, "val", bkg.val) as bkg
from mybkgtable
This works. The trouble is, when this view is invoked from presto, it fails with: Function named_struct not registered
Found that presto has no struct data type, but has ROW instead. It works with this syntax:
select
id,
CAST(ROW(bkg.cnt, bkg.val) as row(cnt integer, val double)) as bkg
from mybkgtable
However, this syntax isn't understood by Hive.
Question is, is it possible to have one view definition that works on both hive and presto?
Question is, is it possible to have one view definition that works on both hive and presto?
Sadly, no.
You can use coral to write the view definition in hiveql and translate it to presto.
https://github.com/linkedin/coral
https://engineering.linkedin.com/blog/2020/coral
It may seem a little strange, but there are already tables with names for each date.
In my project, I have tables for each date to make statistics easier to handle.
Of course, I don't think this is always the best way, but this is the table structure for my project.
(It's a common technique in Google BigQuery and Amazon Athena. This question is about Google BigQuery)
So to get the data, I want to generate today's date. If I use TODAY, I can get the data of the latest day without rewriting the code even if it is the next day.
I tried, but the code didn't work.
Not work 1:
CONCAT in FROM
SELECT
*
FROM
CONCAT('foo_', FORMAT_TIMESTAMP('%Y%m%d', CURRENT_TIMESTAMP(), 'Asia/Tokyo'))
Error:
Table-valued function not found: CONCAT at [4:3]
Not work 2:
create temporary function:
create temporary function getTableName() as (CONCAT('foo_', FORMAT_TIMESTAMP('%Y%m%d', CURRENT_TIMESTAMP(), 'Asia/Tokyo')));
Error:
CREATE TEMPORARY FUNCTION statements must be followed by an actual query.
Question
How do I generate a table name that contains TODAY's date?
In this case, I would recommend you to use Wild tables in BigQuery, which allows you to use some features in Standard SQL.
With Wild Tables you can use _TABLE_SUFFIX, it grants you the ability to filter/scan tables containing this parameter. The syntax would be as follows:
SELECT *
FROM `test-proj-261014.sample.test_*`
where _TABLE_SUFFIX = FORMAT_DATE('%Y%m%d', CURRENT_DATE)
I hope it helps.
Your first query should go like this:
select CONCAT('foo_', FORMAT_TIMESTAMP('%Y%m%d', CURRENT_TIMESTAMP(), 'Asia/Tokyo'))
For creating temporary function, use the below code:
create temp function getTableName() as
((select CONCAT('foo_', FORMAT_TIMESTAMP('%Y%m%d', CURRENT_TIMESTAMP(), 'Asia/Tokyo'))
));
select getTableName()
The error "CREATE TEMPORARY FUNCTION statements must be followed by an actual query." is because once the temporary functions are defined then you have to use the actual query to use that function and then the validity of function dies out. To define persistent UDFs and use them in multiple queries please go through the link to define permanent functions.You can reuse persistent UDFs across multiple queries, whereas you can only use temporary UDFs in a single query.
I have a simple query that runs successfully, but when I introduce a variable into the query, I am unable to save a view using that query. For example:
SELECT * FROM mytable WHERE color = 'red';
This query runs fine. Then:
DECLARE color STRING DEFAULT 'red';
SELECT * FROM mytable WHERE color = color;
This query also runs fine. Then in the BigQuery UI I click to "Save view", but I get an error saying Unexpected keyword DECLARE. Why is that?
As explained in the documentation:
BigQuery views are subject to the following limitations:
You cannot reference query parameters in views.
What you want to do is not allowed. A view is limited to a single SELECT statement.
Normally, I call my function like so:
SELECT *
FROM TABLE(
package_name.function(parameters)
)
I'm trying to call this function across a database link. My intuition is that the following is the correct syntax, but I haven't gotten it to work:
SELECT *
FROM TABLE(
package_name.function#DBLINK(parameters)
)
> ORA-00904: "PACKAGE_NAME"."FUNCTION": invalid identifier
I've tried moving around the database link to no effect. I've tried putting it after the parameter list, after the last parenthesis, after the package name...I've also tried all of the above permutations including the schema name before the package name. I'm running out of ideas.
This is oracle 10g. I'm suspicious that the issue may be that the return type of the function is not defined in the schema in which I'm calling it, but I feel like I should be getting a different error if that were the case.
Thanks for your help!
What you're trying is the correct syntax as far as I know, but in any case it would not work due to the return type being user-defined, as you suspect.
Here's an example with a built-in pipelined function. Calling it locally works, of course:
SELECT * FROM TABLE(dbms_xplan.display_cursor('a',1,'ALL'));
Returns:
SQL_ID: a, child number: 1 cannot be found
Calling it over a database link:
SELECT * FROM TABLE(dbms_xplan.display_cursor#core('a',1,'ALL'));
fails with this error:
ORA-30626: function/procedure parameters of remote object types are not supported
Possibly you are getting the ORA-904 because the link goes to a specific schema that does not have access to the package. But in any case, this won't work, even if you define an identical type with the same name in your local schema, because they're still not the same type from Oracle's point of view.
You can of course query a view remotely, so if there is a well-defined set of possible parameters, you could create one view for each parameter combination and then query that, e.g.:
CREATE VIEW display_cursor_a_1_all AS
SELECT * FROM TABLE(dbms_xplan.display_cursor('a',1,'ALL'))
;
If the range of possible parameter values is too large, you could create a procedure that creates the needed view dynamically given any set of parameters. Then you have a two-step process every time you want to execute the query:
EXECUTE package.create_view#remote(parameters)
SELECT * FROM created_view#remote;
You have to then think about whether multiple sessions might call this in parallel and if so how to prevent them from stepping on each other.