Is it possible to execute query independently on each partition using colocated computing in IgniteCallable without explicitly joining on keys? - ignite

Question

I have one client code which does get all the keys and then call the affinityCall which execute on server-node
Code on client side for computing on each partition
Map<Integer, List<Integer>> partitionMap = new HashMap<>();
for(Integer affKey : affKeys) {
Integer partitionId = affinityFunction.partition(affKey);
partitionMap.computeIfAbsent(partitionId, k -> new ArrayList<>()).add(affKey);
}
Collection<IgniteFuture<Map<Integer, Double>>> futures = new ArrayList<>();
Collection<String> cacheList = new ArrayList<>();
cacheList.add(CACHE_NAME);
for (int partitionId : partitionMap.keySet()) {
IgniteFuture<Map<Integer, Double>> future = compute.affinityCallAsync(cacheList, partitionId,
new Worker(CACHE_NAME, partitionMap.get(partitionId)));
futures.add(future);
}
Code for the worker is
public class Worker implements IgniteCallable<Map<Integer, Double>> {
#IgniteInstanceResource
private Ignite ignite;
private String cacheName;
private List<Integer> keys;
public Worker(String cacheName, List<Integer> keys){
this.cacheName = cacheName;
this.keys = keys;
}
#Override
public Map<Integer, Double> call() throws Exception {
IgniteCache<Object, Object> cache = ignite.cache(cacheName);
SqlFieldsQuery query =
new SqlFieldsQuery("select affKey, sum(count) from Summary s
join table(affKey integer = ?) a
on a.affKey = s.affKey
join table(studyId integer = ?) st
on st.studyId = s.studyId
group by affKey")
.setArgs((Object) keys.toArray());
Map<Integer, Double> result = new HashMap<>();
try (QueryCursor<List<?>> cursor = cache.query(query)) {
for (List<?> record : cursor) {
Integer affKey = (Integer) record.get(0);
Double sum = (Double) record.get(1);
result.put(affKey, sum);
}
}
return result;
}
}
In above case I'm joining on all the keys present in the partition. can anybody help how can we get rid of the extra join here.
More specifically this is query on which I'm working on.
public SqlFieldsQuery getQueryStringForGroup1() {
String queryString = "select g.geneId as geneId, sum(count) as count, " +
"sum(g.rawMean) as sumRawMean, " +
"sum(g.squareRawMean) as sumSquareRawMean, " +
"sum(g.rawSess) as rawSess, " +
"sum(g.mean) as sumMean, " +
"sum(g.squareMean) as sumSquareMean, " +
"sum(g.squareSd) as sEss, " +
"sum(g.greaterThan0) as sumGreaterThan0 " +
"from GeneSummary g " +
"join table(geneId integer = ?1) gn on g.geneId = gn.geneId " +
(filter.getStudyId().isEmpty() ? ""
: "join table(studyId integer = ?4) s on g.studyId = s.studyId ")
+
(filter.getCellType().isEmpty() ? ""
: "join table(cellType integer = ?7) ct on g.cellType = ct.cellType ")
+
(filter.getTissueType().isEmpty() ? ""
: "join table(tissueType integer = ?6) t on g.tissueType = t.tissueType ")
+
(filter.getCondition().isEmpty() ? ""
: "join table(condition integer = ?5) cd on g.condition = cd.condition ")
+
(filter.getAge().isEmpty() ? ""
: "join table(age integer = ?3) a on g.age = a.age ")
+
(filter.getGender().isEmpty() ? ""
: "join table(gender integer = ?2) gd on g.gender = gd.gender ")
+
"group by g.geneId";
SqlFieldsQuery query = new SqlFieldsQuery(queryString).setArgs(
(Object) geneMap.keySet().toArray(),
(Object) filter.getGender().toArray(),
(Object) filter.getAge().toArray(),
(Object) filter.getStudyId().toArray(),
(Object) filter.getCondition().toArray(),
(Object) filter.getTissueType().toArray(),
(Object) filter.getCellType().toArray());
//.setPartitions(this.partitionId);
// query.setPartitions(this.partitionId);
return query;
}
As you mentioned in the comment I modified argument to set for the query.
Modified query looks like this.
public SqlFieldsQuery getQueryStringForGroup1() {
String queryString = "select g.geneId as geneId, sum(count) as count, " +
"sum(g.rawMean) as sumRawMean, " +
"sum(g.squareRawMean) as sumSquareRawMean, " +
"sum(g.rawSess) as rawSess, " +
"sum(g.mean) as sumMean, " +
"sum(g.squareMean) as sumSquareMean, " +
"sum(g.squareSd) as sEss, " +
"sum(g.greaterThan0) as sumGreaterThan0 " +
"from GeneSummary g " +
// "join \"GeneName\".\"GENENAME\" gn on g.geneId = gn.geneId and gn.partitionId
// = ?1 " +
// "join table(geneId integer = ?1) gn on g.geneId = gn.geneId " +
(filter.getStudyId().isEmpty() ? ""
: "join table(studyId integer = ?4) s on g.studyId = s.studyId ")
+
(filter.getCellType().isEmpty() ? ""
: "join table(cellType integer = ?7) ct on g.cellType = ct.cellType ")
+
(filter.getTissueType().isEmpty() ? ""
: "join table(tissueType integer = ?6) t on g.tissueType = t.tissueType ")
+
(filter.getCondition().isEmpty() ? ""
: "join table(condition integer = ?5) cd on g.condition = cd.condition ")
+
(filter.getAge().isEmpty() ? ""
: "join table(age integer = ?3) a on g.age = a.age ")
+
(filter.getGender().isEmpty() ? ""
: "join table(gender integer = ?2) gd on g.gender = gd.gender ")
+
"group by g.geneId";
SqlFieldsQuery query = new SqlFieldsQuery(queryString);
if (!filter.getCellType().isEmpty())
query.setArgs(
(Object) geneMap.keySet().toArray(),
(Object) filter.getGender().toArray(),
(Object) filter.getAge().toArray(),
(Object) filter.getStudyId().toArray(),
(Object) filter.getCondition().toArray(),
(Object) filter.getTissueType().toArray(),
(Object) filter.getCellType().toArray());
if (!filter.getTissueType().isEmpty())
query.setArgs(
(Object) geneMap.keySet().toArray(),
(Object) filter.getGender().toArray(),
(Object) filter.getAge().toArray(),
(Object) filter.getStudyId().toArray(),
(Object) filter.getCondition().toArray(),
(Object) filter.getTissueType().toArray());
if (!filter.getCondition().isEmpty())
query.setArgs(
(Object) geneMap.keySet().toArray(),
(Object) filter.getGender().toArray(),
(Object) filter.getAge().toArray(),
(Object) filter.getStudyId().toArray(),
(Object) filter.getCondition().toArray());
if (!filter.getStudyId().isEmpty())
query.setArgs(
(Object) geneMap.keySet().toArray(),
(Object) filter.getGender().toArray(),
(Object) filter.getAge().toArray(),
(Object) filter.getStudyId().toArray());
if (!filter.getAge().isEmpty())
query.setArgs(
(Object) geneMap.keySet().toArray(),
(Object) filter.getGender().toArray(),
(Object) filter.getAge().toArray());
if (!filter.getGender().isEmpty())
query.setArgs(
(Object) geneMap.keySet().toArray(),
(Object) filter.getGender().toArray());
query.setPartitions(this.partitionId);
return query;
}
Workaround to it was set atleast last argument in the query.
Now my question here is, which approach is better here ?
With join on affinity key or querying on each partition.

Answer 1

With the caveat that I've not studied your query, you can limit a query to a specific partition like this:
var q = new SqlFieldsQuery("select * from table")
.setPartitions(1, 2, 3);