I am trying to create a function in Bigquery which pulls a Cloud Functions:
CREATE OR REPLACE FUNCTION `DATASET.XXXXX`(user_id int64, corp_id STRING) RETURNS STRING
REMOTE WITH CONNECTION `myPROJECTID.REGION.MY_CONNECTION`
OPTIONS (
endpoint = 'https://XXXX.cloudfunctions.net/XXXXX'
)
previously create a connection in the Bigquery shell, but I get the following error, does anyone know?
Keyword REMOTE is not supported at [2:1]
or
Not found: Connection my-connection
Your project must be allowlisted. It's a private preview (I asked 2 month ago, still nothing....)
Related
I'm using postgres 14 for creating this function in Datagrip:
CREATE EXTENSION plpython3u;
CREATE OR REPLACE FUNCTION pymax (x integer, y integer) RETURNS integer
AS $$
if x>y:
return x
$$ LANGUAGE plpython3u;
The user I use is a superuser.
I get the following error when I execute this. How do I resolve this? I've tried increasing the connection timeout in the settings which did not help. Any suggestions here?
In datagrip, the error looks like this.
I tried to replicate this in pgadmin4, the error looks like this:
How to create U-SQL Data Source with connection string ?
This is my attempt, it has issue in setting the CREDENTIAL parameter.
CREATE DATA SOURCE MyAzureSQLDBDataSource
FROM AZURESQLDB
WITH
(
PROVIDER_STRING = "Database=mySampleDB;",
CREDENTIAL = "Server=mySampleDB.database.windows.net;User ID=myUser;Password=myPasswd",
REMOTABLE_TYPES = (bool, byte, sbyte, short, ushort, int, uint, long, ulong, decimal, float, double, string, DateTime)
);
Error message:
error External0: E_CSC_USER_INVALIDDATASOURCEOPTIONVALUE: Invalid value '"Server=mySampleDB.database.windows.net;User ID=myUser;Password=myPasswd"' for data source option 'CREDENTIAL'.
Description:
The only valid values for 'CREDENTIAL' are identifiers or two-part identifiers.
Resolution:
Use a valid value.
Have you reviewed CREATE DATA SOURCE (U-SQL)?
To expand on David's answer.
Since U-SQL scripts are stored at least temporarily in the cluster, we cannot allow the inclusion of secrets in the script. Instead, you need to create the credential in the metadata via an Azure PowerShell command (or SDK) and then refer to that credential in the CREATE DATA SOURCE statement. The documentation link provided by David contains some examples.
I would like to reference a UDF inside a View. According to BigQuery documentation ('bq help mk') and to this post How do I create a BigQuery view that uses a user-defined function?, it is possible to do it with the "--view_udf_resource" syntax.
However, when I try it I get the following error:
# gsutil cat gs://mybucket/bar.js
CREATE TEMP FUNCTION GetWord() AS ('fire');
# bq mk --nouse_legacy_sql --view_udf_resource="gs://mybucket/bar2.js" --view="SELECT 1 as one, GetWord() as myvalue" mydataset.myfoo
Error in query string: Function not found: GetWord at [1:18]
I have also tried it with the Java API and I get the same error:
public void foo(){
final String viewQuery = "#standardSQL\n SELECT 1 as one, GetWord() as myvalue";
UserDefinedFunction userDefinedFunction = UserDefinedFunction.inline("CREATE TEMP FUNCTION GetWord() AS ('fire');");
ViewDefinition tableDefinition = ViewDefinition.newBuilder(viewQuery)
.setUserDefinedFunctions(userDefinedFunction)
.build();
TableId viewTableId = TableId.of(projectName, dataSetName, "foobar");
final TableInfo tableInfo = TableInfo.newBuilder(viewTableId, tableDefinition).build();
bigQuery.create(tableInfo);
}
com.google.cloud.bigquery.BigQueryException: Function not found: GetWord at [2:19]
Am I doing something wrong? Or is the Google's documentation misleading and it is not possible to reference any custom UDF from a View?
You cannot (currently) create a view using standard SQL that uses UDFs. You need to have all of the logic be inline as part of the query itself, and the post that you are looking at is about JavaScript UDFs using legacy SQL. There is an open feature request to support permanent registration of UDFs, however, which would enable you to reference UDFs from views.
I created a table with the BLOB datatype, in Oracle Database,
I tried to retrieve/store data in BLOB Datatype using developer Machine, it is working Fine,
same application was installed on Oracle Client Machine that it is giving error like below
Object Reference is not set to an instance of an object
Procedure Call or arugument is not valid
Code like
da = New Odbc.OdbcDataAdapter("Select * From DOCUMENTSTAB", OdbcCon)
da.fill(ds,"DOCUMENTSTAB")
I am using VB .NET 2012 with SYSTEM.DATA.ODBC Connection Type.
Thanks
My coworker wrote a package in VB.NET to manipulate strings. I have been asked to implement them through CLR. The package accesses a reference database on the same server as the CLR dll file deployed to standardize the strings. I imported the package to my CLR code to create the functions for SQL Server.
After I deploy the dll file to sql server, the functions work fine except occasionally I get the error
System.NullReferenceException: Object reference not set to an instance of an object.
I have to execute my query several times to make it work. It seems the table is locked or sleeping. My query is like
SELECT EMAIL_ADDRESS, dbo.ufn_getEmailDomain(Email_Address) AS EDOMAIN FROM CONTACT_TEMP
WHERE Email_Address IS NOT NULL AND Email_Address<>''
dbo.ufn_getEmailDomain is the CLR function.
There is no online access to this server at all. I searched for a while and couldn't find why this error comes up occasionally or how to fix it.
Your feedback is greatly appreciated.
my CLR functions here.
<Microsoft.SqlServer.Server.SqlFunction(DataAccess:=DataAccessKind.Read)> _
Public Shared Function ufn_getEmailDomainSLD(ByVal email As String) As String
If email Is Nothing Then
Return Nothing
End If
Dim de As New DataEmail(email)
Dim dm As New DataDomain
Dim emailDomain As String
dm = de.Domain
emailDomain = dm.SLD
Return emailDomain
End Function
If I understood right - function returns domain name from email string.
Try to get it in such a way (if there are not milti-mail string values)
SELECT
EMAIL_ADDRESS,
SUBSTRING(EMAIL_ADDRESS, CHARINDEX('#', Item)+1, 2147483647) AS EDOMAIN
FROM CONTACT_TEMP
WHERE Email_Address IS NOT NULL AND Email_Address<>''
and - find your bug in your clr code. Try to recreate function with option RETURNS NULL ON NULL INPUT. If it is not help - then you need to review and to clean your CLR code.
I found the reason eventually. In the connection string in the CLR code, I should set ENLIST=FALSE because it connects to a different database. Once I set ENLIST=FALSE, the problem seems to be fixed.