I'm using Apache Calcite to validate SQL. I add tables and UDFs dynamically.
The problem is when I add a UDF with variable number parameter, the validator can not find this function.
Version of Calcite is 1.18.0
And this is my code.
TestfuncFunction.java
public class TestfuncFunction {
public String testfunc(String... arg0) {
return null;
}
}
Add UDF
Function schemafunction = ScalarFunctionImpl.create(TestfuncFunction.class),"testfunc");
SchemaPlus schemaPlus = Frameworks.createRootSchema(true);
schemaPlus.add("testfunc", schemafunction);
SQL
select testfunc(field1, field2) from test_table
testfunc is a ScalarFunction with variable number parameter,field1 and field2 are columns of test_table. So this is a legal SQL. But I got this CalciteContextException when validating:
No match found for function signature testfunc(<CHARACTER>, <CHARACTER>)
I tryed to change my sql into one parameter like this:
select testfunc(field1) from test_table
and got this exception
java.lang.AssertionError: No assign rules for OTHER defined
at org.apache.calcite.sql.type.SqlTypeAssignmentRules.canCastFrom(SqlTypeAssignmentRules.java:386)
at org.apache.calcite.sql.type.SqlTypeUtil.canCastFrom(SqlTypeUtil.java:864)
at org.apache.calcite.sql.SqlUtil.lambda$filterRoutinesByParameterType$4(SqlUtil.java:554)
...
It seems that calcite transform java array type into SqlTypeName.OTHER.
I have tryed to override method "createJavaType" in JavaTypeFactoryImpl like this:
private static class CustomJavaTypeFactoryImpl extends JavaTypeFactoryImpl {
#Override
public RelDataType createJavaType(Class clazz) {
if (clazz.isArray()) {
return new ArraySqlType(super.createJavaType(clazz.getComponentType()), true);
}
return super.createJavaType(clazz);
}
}
but it did not work.
Do Calcite support UDF with variable number parameter, and what should I do.
Related
I have the following function in my OperationWalker:
public override void VisitDynamicInvocation(IDynamicInvocationOperation operation)
{
var memberReferenceOp = (IDynamicMemberReferenceOperation)operation.Operation;
switch (memberReferenceOp.Instance.Type)
{
case INamedTypeSymbol type:
{
var memberName = memberReferenceOp.MemberName;
var members = type.GetMembers(memberName);
if (members.Length > 1)
{
// WHAT DO I DO HERE ???
}
else
{
Result.Add((IMethodSymbol)members[0]);
}
break;
}
case IDynamicTypeSymbol dynamicType:
Unresolved.Add((operation.Syntax, memberReferenceOp.MemberName));
break;
}
}
I am clueless when a method on a normal type (non dynamic) is called with a dynamic parameter and there is a choice of target methods with the same name. E.g.:
class A
{
public void Get(int i){}
public void Get(string s){}
public void Get(object o){}
public void Get(double s, int precision){}
}
...
dynamic x = ...;
A a;
a.Get(x)
In this case any of the first 3 A.Get methods may be called, depending on the actual type of x. But not the fourth method.
Is there a way in Roslyn to get this information? Specifically in this example, I would like to get the symbols for first 3 Get methods.
The logic is non trivial, because one needs to take into account:
Default parameters, so just counting the arguments may not be enough
Type conversions
Visibility Scope
Number of arguments
Parameters may be passed using the named syntax in arbitrary order
Combining it all together we get non trivial logic. Is there anything in the SemanticModel or anywhere else to help get the answer?
I figured it out and it is straightforward - SemanticModel.GetSymbolInfo. When there is exact match its Symbol property returns it. When there are multiple candidates, as may be the case when one of the passed arguments is dynamic, then the property CandidateSymbols holds all the options.
I have not tested it with extension methods, so it is possible there is a gap there.
I created a custom PostgreSQL function:
create function delete_classes() returns void as
$$
DELETE FROM ...;
$$ language sql;
I would like to call this function with Spring`s JDBCTemplate. I tried the following:
#Repository
class Repository {
private final JdbcTemplate template; // initialized in constructor
public void deleteClasses() {
template.update("select delete_classes()");
}
}
I get the following error:
Caused by: org.springframework.dao.DataIntegrityViolationException:
PreparedStatementCallback; SQL [select delete_classes()];
A result was returned when none was expected.;
nested exception is org.postgresql.util.PSQLException:
A result was returned when none was expected.
The PostgreSQL function delete_classes returns void, so I am not really sure what is meant here. The DELETE statement has no RETURNING ... clause.
What is the correct way to execute the function?
update
I tried the following (execute instead of update):
#Repository
class Repository {
private final JdbcTemplate template; // initialized in constructor
public void deleteClasses() {
template.execute("select delete_classes()", statement -> null);
}
}
Here, this implements a PreparedStatementCallback in lambda-fashion which just returns null.
The function delete_classes is not called now.
There is a UDF java class shown as below:
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
public class Strip extends UDF {
private Text result = new Text();
public Text evaluate(Text str) {
if (str == null) {
return null;
}
result.set(StringUtils.strip(str.toString()));
return result;
}
public Text evaluate(Text str, String stripChars) {
if (str == null) {
return null;
}
result.set(StringUtils.strip(str.toString(), stripChars));
return result;
}
}
Hive actually supports Java
primitives in UDFs (and a few other types, such as java.util.List and
java.util.Map), so a signature like:
public String evaluate(String str)
would work equally well. However, by using Text we can take advantage of object reuse,
which can bring efficiency savings, so this is preferred in general.
Can someone tell me the reason why Text is preferred? Why we could take advantage of object reuse by using Text. When we execute the following command in Hive:
hive> SELECT strip(' bee ') FROM dummy;
After that we execute another command using that Strip function, then the Strip object is created again, right? So we cannot reuse it, right?
You can reuse a Text instance by calling one of the set() methods on it. For example:
Text t = new Text("hadoop");
t.set("pig");
I am confused by some code results using dynamic.
I am parsing some JSON in C# using the Newtonsoft json.net libraries. The JSON string ends like this:
}],"Result":{"820":1,"821":1,"822":2,"823":0}}
and so I set up a class to contain the parsed JSON:
public class Poll
{
public Answer[] Answer { get; set; }
public dynamic Result { get; set; }
}
and then parse my JSON string with this line
var poll = JsonConvert.DeserializeObject<BallotsharePoll>(json);
and then this code to delve into the dynamic object:
if (poll.Result["820"] != null)
{
var votes = (int)(poll.Result["820"]).Value;
result[i] = votes;
}
This works. But my original code used int? and the line
var votes = (poll.Result["820"]).Value as int?;
which fails as shown in the watch window here:
This shows that
the dynamic value of poll.Result["820"] is {1},
the value of (int)(poll.Result["820"]).Value is 1,
but the value of (poll.Result["820"]).Value as int? is null
QUESTION: Why does as int? return null while casting to int results in 1?
I would rather use int? to cover the case where the dynamic value cannot be coerced into an int.
Consider this program:
using System;
static class Program {
static void Main() {
dynamic s = (short)1;
Console.WriteLine(s as int?);
Console.WriteLine((int)s);
}
}
The first WriteLine prints nothing, the second prints 1. Likely, the problem in your case is similar.
This is not intuitive behaviour, I'll happily admit. When using as with dynamic, what you get is pretty much s is int? ? (int?)s : null, and s is int? would return false with or without dynamic.
Sorry for exhuming a very old question but I know the answer behind this behavior.
To be short, the keyword "int?" is just a shortcut for the generic class Nullable, with the generic type T as Int.
So a dynamic, to be considered an int?, should be a Nullable instance.
if your Json was
}],"Result":{"820":{"Value":1,"HasValue":true},}}
the operation "as int?" would have worked.
I'm experimenting with PetaPoco to convert a table into POCOs.
In my table, I've got a column named TheEnum. The values in this column are strings that represent the following enum:
public enum MyEnum
{
Fred,
Wilma
}
PetaPoco chokes when it tries to convert the string "Fred" into a MyEnum value.
It does this in the GetConverter method, in the line:
Convert.ChangeType( src, dstType, null );
Here, src is "Fred" (a string), and dstType is typeof(MyEnum).
The exception is an InvalidCastException, saying Invalid cast from 'System.String' to 'MyEnum'
Am I missing something? Is there something I need to register first?
I've got around the problem by adding the following into the GetConverter method:
if (dstType.IsEnum && srcType == typeof(string))
{
converter = delegate( object src )
{
return Enum.Parse( dstType, (string)src ) ;
} ;
}
Obviously, I don't want to run this delegate on every row as it'll slow things down tremendously. I could register this enum and its values into a dictionary to speed things up, but it seems to me that something like this would likely already be in the product.
So, my question is, do I need to do anything special to register my enums with PetaPoco?
Update 23rd February 2012
I submitted a patch a while ago but it hasn't been pulled in yet. If you want to use it, look at the patch and merge into your own code, or get just the code from here.
I'm using 4.0.3 and PetaPoco automatically converts enums to integers and back. However, I wanted to convert my enums to strings and back. Taking advantage of Steve Dunn's EnumMapper and PetaPoco's IMapper, I came up with this. Thanks guys.
Note that it does not handle Nullable<TEnum> or null values in the DB. To use it, set PetaPoco.Database.Mapper = new MyMapper();
class MyMapper : PetaPoco.IMapper
{
static EnumMapper enumMapper = new EnumMapper();
public void GetTableInfo(Type t, PetaPoco.TableInfo ti)
{
// pass-through implementation
}
public bool MapPropertyToColumn(System.Reflection.PropertyInfo pi, ref string columnName, ref bool resultColumn)
{
// pass-through implementation
return true;
}
public Func<object, object> GetFromDbConverter(System.Reflection.PropertyInfo pi, Type SourceType)
{
if (pi.PropertyType.IsEnum)
{
return dbObj =>
{
string dbString = dbObj.ToString();
return enumMapper.EnumFromString(pi.PropertyType, dbString);
};
}
return null;
}
public Func<object, object> GetToDbConverter(Type SourceType)
{
if (SourceType.IsEnum)
{
return enumVal =>
{
string enumString = enumMapper.StringFromEnum(enumVal);
return enumString;
};
}
return null;
}
}
You're right, handling enums is not built into PetaPoco and usually I just suggest doing exactly what you've done.
Note that this won't slow things down for requests that don't use the enum type. PetaPoco generates code to map responses to pocos so the delegate will only be called when really needed. In other words, the GetConverter will only be called the first time a particular poco type is used, and the delegate will only be called when an enum needs conversion. Not sure on the speed of Enum.Parse, but yes you could cache in a dictionary if it's too slow.
If you are using PetaPoco's T4 generation and you want enums in your generated type, you can use the PropertyType override in Database.tt:
tables["App"]["Type"].PropertyType = "Full.Namespace.To.AppType";
I you want to store the value of the enum instead of the index number (1,2,4 for example) you can locate the update function in PetaPoco class because the code is "managed" etc, when you add it as nuget package it will store the .cs file to your project. If we would have the enum variable Color = {red, yellow, blue}
Instead of:
// Store the parameter in the command
AddParam(cmd, pc.GetValue(poco), pc.PropertyInfo);
change to:
//enum?
if (i.Value.PropertyInfo.PropertyType.IsEnum)
{
AddParam(cmd, i.Value.GetValue(poco).ToString(), i.Value.PropertyInfo);
}
else
{
// Store the parameter in the command
AddParam(cmd, i.Value.GetValue(poco), i.Value.PropertyInfo);
}
It would store "yellow" instead of 2