Apache Ignite : Ignite Repository query with "IN" clause, returns no records - sql

I am using Apache Ignite as the back-end data store in a SpringBoot Application.
I have a requirement where I need to get all the entities whose name matches one of the names from a set of names.
Hence i am trying to get it implemented using a #Query configuration and a method named findAllByName(Iterable<String> names)as below:
Here on the Query, I am trying to use the 'IN' clause and want to pass an array of names as an input to the 'IN' clause.
#RepositoryConfig(cacheName = "Category")
public interface CategoryRepository extends IgniteRepository<Category, Long>
{
List<Category> findByName(String name);
#Query("SELECT * FROM Category WHERE name IN ( ? )")
Iterable<Category> findAllByName(Iterable<String> names); // this method always returns empty list .
}
In this the method findAllByName always returns empty list, even when ignite has Categories for which the name field matches the data passed in the query.
I am unable to figure out if there is a problem with the Syntax or the query of the method signature or the parameters.

Please try using String[] names instead for supplying parameters.
UPDATE: I have just checked the source, and we don't have tests for such scenario. It means that you're on uncharted territory even if it is somehow possible to get to work.
Otherwise looks unsupported currently.

I know your question is more specific to Spring Data Ignite feature. However, as an alternate, you can achieve it using the SqlQuery abstraction of Ignite.
You will form your query like this. I have pasted the sample below with custom sql function inSet that you will write. Also, the below tells how this is used in your sql.
IgniteCache<String, MyRecord> cache = this.ignite
.cache(this.environment.getProperty(Cache.CACHE_NAME));
String sql = "from “my-ignite-cache”.MyRecord WHERE
MyRecord.city=? AND inSet(?, MyRecord.flight)"
SqlQuery<String, MyRecord> sqlQuery = new SqlQuery<>(MyRecord.class,
sql);
sqlQuery.setArgs(MyCity, [Flight1, Flight2 ] );
QueryCursor<Entry<String, MyRecord>> resultCursor = cache.query(sqlQuery);
You can iterate the result cursor to do something meaningful from the extracted data.
resultCursor.forEach(e -> {
MyRecord record = e.getValue();
// do something with result
});
Below is the Ignite Custom Sql function which is used in the above Query - this will help in replicating the IN clause feature.
#QuerySqlFunction
public static boolean inSet(List<String> filterParamArgIds, String id) {
return filterParamArgIds.contains(id);
}
And finally, as a reference MyRecord referred above can be defined something like this.
public class MyRecord implements Serializable {
#QuerySqlField(name = "city", index = true)
private String city;
#QuerySqlField(name = "flight", index = true)
private String flight;
}

Related

Apache Ignite Continuous Queries : How to get the field names and field values in the listener updates when there are dynamic fields?

I am working on a POC on whether or not we should go ahead with Apache Ignite both for commerical and enterprise use. There is a use case though that we are trying to find an answer for.
Preconditions
Dynamically creation of tables i.e. there may be new fields that come to be put into the cache. Meaning there is no precompiled POJO(Model) defining the attributes of the table/cache.
Use case
I would like to write a SELECT continuous query where it gives me the results that are modified. So I wrote that query but the problem is that when the listener gets a notification, I am not able to find all the field names that are modified from any method call. I would like to be able to get all the field names and field values in some sort of Map, which I can use and then submit to other systems.
You could track all modified field values using binary object and continuous query:
IgniteCache<Integer, BinaryObject> cache = ignite.cache("person").withKeepBinary();
ContinuousQuery<Integer, BinaryObject> query = new ContinuousQuery<>();
query.setLocalListener(events -> {
for (CacheEntryEvent<? extends Integer, ? extends BinaryObject> event : events) {
BinaryType type = ignite.binary().type("Person");
if (event.getOldValue() != null && event.getValue() != null) {
HashMap<String,Object> oldProps = new HashMap<>();
HashMap<String,Object> newProps = new HashMap<>();
for (String field : type.fieldNames()) {
oldProps.put(field,event.getOldValue().field(field));
newProps.put(field,event.getValue().field(field));
}
com.google.common.collect.MapDifference<Object, Object> diff = com.google.common.collect.Maps.difference(oldProps, newProps);
System.out.println(diff.entriesDiffering());
}
}
});
cache.query(query);
cache.put(1, ignite.binary().builder("Person").setField("name","Alice").build());
cache.put(1, ignite.binary().builder("Person").setField("name","Bob").build());

Ignite CacheJdbcPojoStoreFactory using Enum fields

I am to using the CacheJdbcPojoStoreFactory
I want to have a VARCHAR field in the database which maps to an Enum in Java.
The way I am trying to achieve this is something like the following. I want the application code to work with the enum, but the persistence to use the string so that it is human readable in the database. I do not want to use int values in the database.
This seems to work fine for creating new objects, but not for reading them out. It seems that it tries to set the field directly, and the setter (setSideAsString) is not called. Of course there is no field called sideAsString. Should this work? Any suggestions?
Here is the code excerpt
In some application code I would do something like
trade.setSide(OrderSide.Buy)
And this will persist fine. I can read "Buy" in the side column as a VARCHAR.
In Trade
private OrderSide side; // OrderSide is an enum with Buy,Sell
public OrderSide getSide() {
return side;
}
public void setSide(OrderSide side) {
this.side = side;
}
public String getSideAsString() {
return this.side.name();
}
public void setSideAsString(String s) {
this.side = OrderSide.valueOf(s);
}
Now when configuring the store, I do this
Collection<JdbcTypeField> vals = new ArrayList<>();
vals.add(new JdbcTypeField(Types.VARCHAR, "side", String.class, "sideAsString"));
After a clean start, If I query Trade using Ignite SQL query, and call trade.getSide() it will be null. Other (directly mapped) columns are fine.
Thanks,
Gordon
BinaryMarshaller deserialize only fields which used in query.
Please try to use OptimizedMarshaller:
IgniteConfiguration cfg = new IgniteConfiguration();
...
cfg.setMarshaller(new OptimizedMarshaller());
Here's the ticket for support enum mapping in CacheJdbcPojoStore.

Returning composite class with Neo4jClient

The Neop4jClient cypher wiki (https://github.com/Readify/Neo4jClient/wiki/cypher) contains an example of using lambda expressions to return multiple projections...
var query = client
.Cypher
.Start(new { root = client.RootNode })
.Match("root-[:HAS_BOOK]->book-[:PUBLISHED_BY]->publisher")
.Return((book, publisher) => new {
Book = book.As<Book>(),
Publisher = publisher.As<Publisher>(),
});
So the query will return details of both book nodes and publisher nodes. But I want to do something slightly different. I want to combine the contents of a single node type with a property of the matched path. Lets say I have Person nodes with a property 'name', and a class defined so,,,
public class descendant
{
public string name { get; set; }
public int depth { get; set; }
}
A cypher query like this will return what I want, which is all descendants of a given node with the depth of the relationship...
match p=(n:Person)<-[*]-(child:Person)
where n.name='George'
return distinct child.name as name, length(p) as depth
If I try a Neo4jClient query like this...
var query =
_graphClient.Cypher
.Match("p=(n:Person)<-[*]-(child:Person)")
.Where("n.name='George'")
.Return<descendant>("child.name, length(p)") ;
I get an error that the syntax is obsolete, but I can't figure out how should I project the cypher results onto my C# POCO. Any ideas anyone?
The query should look like this:
var query =
_graphClient.Cypher
.Match("p=(n:Person)<-[*]-(child:Person)")
.Where((Person n) => n.name == "George")
.Return((n,p) => new descendant
{
name = n.As<Person>().Name,
depth = p.Length()
});
The Return statement should have the 2 parameters you care about (in this case n and p) and project them via the lambda syntax (=>) to create a new descendant instance.
The main point this differs from the example, is that the example creates a new anonymous type, whereas you want to create a concrete type.
We then use the property initializer (code inside the { } braces) to set the name and depth, using the As<> and Length extension methods to get the values you want.
As a side note, I've also changed the Where clause to use parameters, you should always do this if you can, it will make your queries both faster and safer.

JSF selectItems value from sql query

I want to populate selectOneMenu component with values extracted from database by sql query.
Query returns only store names which I want to enter as values to selectOneMenu
I get java.lang.IllegalArgumentException with stack starting with :
java.lang.IllegalArgumentException at com.sun.faces.renderkit.SelectItemsIterator.initializeItems(SelectItemsIterator.java:216)
at com.sun.faces.renderkit.SelectItemsIterator.hasNext(SelectItemsIterator.java:135)
at com.sun.faces.renderkit.html_basic.MenuRenderer.renderOptions(MenuRenderer.java:762)
This is my xhtml code (This is the only use of selectItems):
<h:selectOneMenu id="storeName" value="#{shoplist.store}">
<f:selectItems value="#{buyHistory.stores}" />
</h:selectOneMenu>
This is query from buyHistory bean:
public ResultSet getStores() throws SQLException {
...
PreparedStatement getStores = connection.prepareStatement(
"SELECT distinct STORE_NAME "
+ "FROM BuyingHistory ORDER BY STORE_NAME");
CachedRowSet rowSet = new com.sun.rowset.CachedRowSetImpl();
rowSet.populate(getStores.executeQuery());
return rowSet;
}
What am I doing wrong? Should I convert somehow from resultSet to SelectItem array/list?
Should I convert somehow from resultSet to SelectItem array/list?
Yes, that's one of the solutions. See also our h:selectOneMenu wiki page. The IllegalArgumentException will be thrown when the value is not an instance of SelectItem, or an array, or Iterable or Map.
Ultimately, your JSF backing beans should be completely free of java(x).sql dependencies. I.e. you should have no single line of import java(x).sql.Something; in your JSF code. Otherwise, it's bad design anyway (tight-coupling). Learn how to create proper DAO classes.
Why do you think that JSF would know how to transform a ResultSet from the persistence layer? JSF is a presentation layer framework :)
Yes you need to convert it to a SelectItem-List like this:
private List<SelectItem> transformToSelectItems(ResultSet resultSet) {
List<SelectItem> selectItems = new ArrayList<SelectItem>();
while(resultSet.next()) {
String storeName = resultSet.getString("STORE_NAME");
SelectItem item = new SelectItem(storeName, storeName);
selectItems.add(item);
}
return selectItems;
}
Be sure to notice BalusC's answer. This is just an example of how to construct a dynamic SelectItem-List. But you should definetely not have a ResultSet in your JSF-ManagedBeans.

How to intercept and modify SQL query in Linq to SQL

I was wondering if there is any way to intercept and modify the sql generated from linq to Sql before the query is sent off?
Basically, we have a record security layer, that given a query like 'select * from records' it will modify the query to be something like 'select * from records WHERE [somesecurityfilter]'
I am trying to find the best way to intercept and modify the sql before its executed by the linq to sql provider.
Ok, first to directly answer your question (but read on for words of caution ;)), there is a way, albeit a finicky one, to do what you want.
// IQueryable<Customer> L2S query definition, db is DataContext (AdventureWorks)
var cs = from c in db.Customers
select c;
// extract command and append your stuff
DbCommand dbc = db.GetCommand(cs);
dbc.CommandText += " WHERE MiddleName = 'M.'";
// modify command and execute letting data context map it to IEnumerable<T>
var result = db.ExecuteQuery<Customer>(dbc.CommandText, new object[] { });
Now, the caveats.
You have to know which query is generated so you would know how to modify it, this prolongs development.
It falls out of L2S framework and thus creates a possible gaping hole for sustainable development, if anyone modifies a Linq it will hurt.
If your Linq causes parameters (has a where or other extension causing a WHERE section to appear with constants) it complicates things, you'll have to extract and pass those parameters to ExecuteQuery
All in all, possible but very troublesome. That being said you should consider using .Where() extension as Yaakov suggested. If you want to centrally controll security on object level using this approach you can create an extension to handle it for you
static class MySecurityExtensions
{
public static IQueryable<Customer> ApplySecurity(this IQueryable<Customer> source)
{
return source.Where(x => x.MiddleName == "M.");
}
}
//...
// now apply it to any Customer query
var cs = (from c in db.Customers select c).ApplySecurity();
so if you modify ApplySecurity it will automatically be applied to all linq queries on Customer object.
If you want to intercept the SQL generated by L2S and fiddle with that, your best option is to create a wrapper classes for SqlConnection, SqlCommand, DbProviderFactory etc. Give a wrapped instance of SqlConnection to the L2S datacontext constructor overload that takes a db connection. In the wrapped connection you can replace the DbProviderFactory with your own custom DbProviderFactory-derived class that returns wrapped versions of SqlCommand etc.
E.g.:
//sample wrapped SqlConnection:
public class MySqlConnectionWrapper : SqlConnection
{
private SqlConnecction _sqlConn = null;
public MySqlConnectionWrapper(string connectString)
{
_sqlConn = new SqlConnection(connectString);
}
public override void Open()
{
_sqlConn.Open();
}
//TODO: override everything else and pass on to _sqlConn...
protected override DbProviderFactory DbProviderFactory
{
//todo: return wrapped provider factory...
}
}
When using:
using (SomeDataContext dc = new SomeDataContext(new MySqlConnectionWrapper("connect strng"))
{
var q = from x in dc.SomeTable select x;
//...etc...
}
That said, do you really want to go down that road? You'll need to be able to parse the SQL statements and queries generated by L2S in order to modify them properly. If you can instead modify the linq queries to append whatever you want to add to them, that is probably a better alternative.
Remember that Linq queries are composable, so you can add 'extras' in a separate method if you have something that you want to add to many queries.
first thing come to my mind is to modify the query and return the result in Non-LINQ format
//Get linq-query as datatable-schema
public DataTable ToDataTable(System.Data.Linq.DataContext ctx, object query)
{
if (query == null)
{
throw new ArgumentNullException("query");
}
IDbCommand cmd = ctx.GetCommand((IQueryable)query);
System.Data.SqlClient.SqlDataAdapter adapter = new System.Data.SqlClient.SqlDataAdapter();
adapter.SelectCommand = (System.Data.SqlClient.SqlCommand)cmd;
DataTable dt = new DataTable("sd");
try
{
cmd.Connection.Open();
adapter.FillSchema(dt, SchemaType.Source);
adapter.Fill(dt);
}
finally
{
cmd.Connection.Close();
}
return dt;
}
try to add your condition to the selectCommand and see if it helps.
Try setting up a view in the DB that applies the security filter to the records as needed, and then when retrieving records through L2S. This will ensure that the records that you need will not be returned.
Alternatively, add a .Where() to the query before it is submitted that will apply the security filter. This will allow you to apply the filter programmatically (in case it needs to change based on the scenario).