RavenDB: How to index dictionary keys on a multi-map index? - indexing

Question

I've got the below RavenDB MultiMap index that works and returns results. Now when I want to use the query and try to filter data I get the following message:
The field 'Stock_Key' is not indexed, cannot query/sort on fields that are not indexed.
I am trying to get all the products which have some stock at certain warehouses. _request.Warehouses is a list of warehouse IDS that can be provided to the query. The product stock is saved in a separate collection in the database which holds the same SKU.
var query = await _session
.Query<Products_SearchUniqueBySku.Result, Products_SearchUniqueBySku>()
.Where(x => x.Stock.Any(y => y.Key.In(_request.Warehouses) && y.Value > 0))
.ToListAsync();
I've been trying to get the keys indexed all day but failed to do so. Would appreciate some help with this. I've also tried to do the query via some RQL variants in the RavenDB studio but getting more of the same message in there. Not sure if the RQL queries are written correctly though.
from index 'Products/SearchUniqueBySku'
where Stock.589e90c9-09bb-4a04-94fb-cf92bde88f97 > 0
The field 'Stock.589e90c9-09bb-4a04-94fb-cf92bde88f97' is not indexed, cannot query/sort on fields that are not indexed
from index 'Products/SearchUniqueBySku'
where Stock_589e90c9-09bb-4a04-94fb-cf92bde88f97 > 0
The field 'Stock_589e90c9-09bb-4a04-94fb-cf92bde88f97' is not indexed, cannot query/sort on fields that are not indexed
The index that I've used:
using System;
using System.Collections.Generic;
using System.Linq;
using Raven.Client.Documents.Indexes;
using MyProject.Models.Ordering.Entities;
using MyProject.Models.Ordering.Enums;
namespace MyProject.Ordering.Indexes;
public class Products_SearchUniqueBySku : AbstractMultiMapIndexCreationTask<Products_SearchUniqueBySku.Result>
{
public class Result
{
public string Sku { get; set; }
public ProductTypes Type { get; set; }
public string Name { get; set; }
public IDictionary<string, decimal> Stock { get; set; }
}
public Products_SearchUniqueBySku()
{
AddMap<Product>(
products => from product in products
where product.Type == ProductTypes.Simple
where product.Variants.Count == 0
select new
{
product.Sku,
product.Type,
product.Name,
Stock = new Dictionary<string, decimal>()
}
);
AddMap<Product>(
products => from product in products
where product.Type == ProductTypes.Simple
where product.Variants.Count > 0
from variant in product.Variants
select new
{
variant.Sku,
Type = ProductTypes.Variant,
Name = $"{product.Name} ({string.Join("/", variant.Mappings.Select(y => y.Value.Name))})",
Stock = new Dictionary<string, decimal>()
});
AddMap<StockItem>(
items => from item in items
group item by item.Sku
into grouped
select new
{
Sku = grouped.Key,
Type = ProductTypes.Variant,
Name = (string) null,
Stock = grouped.ToDictionary(x => x.Warehouse.Id, x => x.Stock)
}
);
Reduce = results => from result in results
group result by result.Sku
into grouped
let product = grouped.Single(x => x.Stock.Count == 0)
select new
{
Sku = grouped.Key,
product.Type,
product.Name,
Stock = grouped.SelectMany(x => x.Stock).ToDictionary(x => x.Key, x => x.Value),
};
}
}
The results when using it the RavenDB studio (only showing some, you get the idea):
from index 'Products/SearchUniqueBySku'
{
"Sku": "VANS-SH-38",
"Type": "Variant",
"Name": "Vans Men's Suede (Maat 38)",
"Stock": {
"589e90c9-09bb-4a04-94fb-cf92bde88f97": 10,
"98304a84-0f44-49ce-8438-8a959ca29b9d": 11
},
"#metadata": {
"#change-vector": null,
"#index-score": 1
}
},
{
"Sku": "889376",
"Type": "Simple",
"Name": "Apple Magic Trackpad (2021)",
"Stock": {
"589e90c9-09bb-4a04-94fb-cf92bde88f97": 15
},
"#metadata": {
"#change-vector": null,
"#index-score": 1
}
}
The models (most properties omitted for brevity):
public class StockItem
{
public EntityReference Warehouse { get; set; }
public string Sku { get; set; }
public decimal Stock { get; set; }
}
public class EntityReference
{
public string Id { get; set; }
public string Name { get; set; }
}
public class Product
{
public string Id { get; set; }
public string Name { get; set; }
public ProductTypes Type { get; set; }
public List<ProductVariant> Variants { get; set; }
}
public class ProductVariant
{
public string Sku { get; set; }
}
EDIT:
Building on the answer from #Ayende Rahien. I had to change the Reduce to the following:
Reduce = results => from result in results
group result by result.Sku
into grouped
let product = grouped.Single(x => x.Stock.Count == 0)
let stock = grouped.SelectMany(x => x.Stock).ToDictionary(x => x.Key, x => x.Value)
select new
{
product.Id,
....
Stock = stock,
_ = stock.Select(x => CreateField("Stock_" + x.Key, x.Value)) <--
};
See Dynamic Fields for indexes (docs). I then got the following message:
Map and Reduce functions of a index must return identical types.
This I solved by adding _ = (string) null as a property (not sure if it is the perfect solution but hey, it worked) to each of the AddMap functions and then the following query worked:
from index 'Products/SearchUniqueBySku'
where Stock_589e90c9-09bb-4a04-94fb-cf92bde88f97 > 0

Answer 1

You need to do this in the Reduce of the index:
_ = grouped.SelectMany(x => CreateField("Stock_" +x.Stock.Key, x.Stock.Value))
That uses _ to mark the field as containing dynamic fields.
The fields that it will emit will be in the format of Stock_$key