Querying nested dictionaries in RavenDB - indexing

This question regards querying nested dictionaries.
I have a case which can be simplified into the following setup with a style containing a list of SKUs containing a list of Collis.
CLASS DEFINITIONS:
public class Style
{
public string Name { get; set; }
public Dictionary<string, Sku> Skus = new Dictionary<string, Sku>();
}
public class Sku
{
public string Name { get; set; }
public Dictionary<string, Colli> Collis = new Dictionary<string, Colli>();
}
public class Colli
{
public string Name { get; set; }
}
JSON DATA IN RAVEN DB:
{
"Skus": {
"Sku1": {
"Collis": {
"Right": {
"Name": "Right"
},
"Right again": {
"Name": "Right again"
},
"Wrong": {
"Name": "Wrong"
}
},
"Name": "Sku1"
},
"Sku2": {
"Collis": {
"Wrong 1": {
"Name": "Wrong 1"
},
"Wrong 2": {
"Name": "Wrong 2"
},
"Wrong 3": {
"Name": "Wrong 3"
}
},
"Name": "Sku2"
}
},
"Name": "Style1"
}
VALID QUERIES:
(Ask for style with skus of specific names)
var existingStyleWithSku1 = session.Query<Style>().Where(s => s.Skus["Sku1"] != null).ToList();
var nonexistingStyleWithSku4 = session.Query<Style>().Where(s => s.Skus["Sku4"] != null).ToList();
INVALID NESTED QUERY
(Ask for style containing a sku named "Sku1" that contains a colli named "Right")
var styleWithSpecificColli = session.Query<Style>().Where(s => s.Skus["Sku1"].Collis["Right"] != null).ToList();
When i attempt to execute the last query, I get the message:
{ "Url":
"/indexes/dynamic/Styles?query=-Skus.get_Item(%2522Sku1%2522).Collis.Right%253A%255B%255BNULL_VALUE%255D%255D%2520AND%2520Skus.get_Item(%2522Sku1%2522).Collis.Right%253A*&start=0&pageSize=128&aggregation=None",
"Error": "System.ArgumentException: The field ')CollisRight' is not
indexed, cannot query on fields that are not indexed\r\n at
Raven.Database.Indexing.Index.IndexQueryOperation.AssertQueryDoesNotContainFieldsThatAreNotIndexes()
in c:\Builds\raven\Raven.Database\Indexing\Index.cs:line 628\r\n
at
Raven.Database.Indexing.Index.IndexQueryOperation.d__1c.MoveNext()
in c:\Builds\raven\Raven.Database\Indexing\Index.cs:line 542\r\n
at
System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()\r\n
at
System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()\r\n
at System.Collections.Generic.List1.InsertRange(Int32 index,
IEnumerable1 collection)\r\n at
........
Is there a way that I can be able to execute the last query? Maybe defining what to index in RavenDB?
Thankyou in advance.

I posted the example above as failing test, but synhershko corrected my code in order to make it work.
It is actually possible to do this. The query just looks like this instead:
WRONG:
var styleWithSpecificColli = session.Query<Style>()
.Where(s => s.Skus["Sku1"].Collis["Right"] != null)
.ToList();
RIGHT:
var styleWithSpecificColli = session.Query<Style>()
.Select(s => s.Skus["Sku1"])
.Where(c => c.Collis["Right"] != null)
.ToList();

Related

IConfiguration.GetSection with Where selector

I'm using IConfiguration.GetSection to retrieve configuration information from a config file:
var loggingProviders = Config.GetSection( "Logging" ).Get<LoggingProviders>();
which works just fine, but I want to only retrieve entries that are enabled, so I want to do either of these:
var loggingProviders = Config.GetSection( "Logging" ).Get<LoggingProviders>().Where( x => x.Enabled == true );
var loggingProviders = Config.GetSection( "Logging" ).Where( x => x.Enabled == true ).Get<LoggingProviders>();
But I keep getting hitting a dead end, any advice would be appreciated!
If you want to use .Where,it needs to be a list,here is a demo:
public class LoggingProviders
{
public int Id { get; set; }
public bool Enabled { get; set; }
}
appsettings.json:
"Logging1": [
{
"Id": "1",
"Enabled": "true"
},
{
"Id": "2",
"Enabled": "true"
},
{
"Id": "3",
"Enabled": "false"
}
]
startup:
public IConfiguration Configuration { get; }
...
List<LoggingProviders> loggingProviders = Configuration.GetSection("Logging1").Get<List<LoggingProviders>>().Where(x => x.Enabled == true).ToList();
result:
If you don't get a list,and want to use .where,you can try to change it to list first.Here is a demo.
appsettings.json:
"Logging1":
{
"Id": "1",
"Enabled": "true"
},
startup:
public IConfiguration Configuration { get; }
...
List<LoggingProviders> l= new List<LoggingProviders>();
l.Add(Configuration.GetSection("Logging1").Get<LoggingProviders>());
List<LoggingProviders> loggingProviders = l.Where(x => x.Enabled == true).ToList();
result:

Find the same key in other localization json-file

I want to find the same key in an other jsons localization file.
My json’s looks like this.
{
"general": {
"main": {
"title": "This is a title",
"header": "This is a header",
"sub": {
"sub_thing_1": "Something",
"sub_thing_2": "Something2"
}
}
},
"other_things": {
"title": "Other title"
"sub": {
"sub_thing_1": "Something",
"sub_thing_2": "Something2"
}
}
}
As you can see the same key can be in more places
My thinking is, to generate a collection that contains, the depth values like this
eg: this "sub_thing_2" can be found at -> ["general", "main", "sub"]
and with this information i can find the exact key in other files.
And with this code I geathering the depth information for each key, it’s just a scratch, thinkering code fragement, I'm new to Kotlin by the way
psi?.accept(object : JsonRecursiveElementVisitor() {
override fun visitObject(o: JsonObject) {
val childrens = o.children
for (child in childrens) {
val childProp = child as JsonProperty
if (child.lastChild is JsonObject) {
depth.add(childProp.name)
visitObject(child.lastChild as JsonObject)
} else {
println(depth.toString() + childProp.name + ": " + childProp.lastChild.text)
}
}
if (depth.isNotEmpty()) {
val lastItem = depth.last();
depth.remove(lastItem)
}
}
})
My question is there is other easier way to do this maybe without the PSI or I'm in the right direction?

BigQueryIO.write DynamicDestination withCreateDisposition - Clustering fields

BigQueryIO.write.withCreateDisposition(CreateDisposition.CREATE_IF_NEEDED) along with DynamicDestinations we can write to the dynamic table and if the table does not exist it will create the table from TableSchema provided from DynamicDestinations.
I am not able to add clustering fields part of TableSchema model, because it does not have such a feature.
how we can add DynamicDestinations having TableSchema with clustering fields?
bigQuery API is one way to add cluster field to a table
Using this link you can test the API before writing your code
function execute() {
return gapi.client.bigquery.jobs.insert({
"resource": {
"configuration": {
"query": {
"clustering": {
"fields": [
"Field1",
"Field2"
]
},
"query": "select 5",
"destinationTable": {
"datasetId": "Id1",
"projectId": "Project1",
"tableId": "T1"
}
}
}
}
})
.then(function(response) {
// Handle the results here (response.result has the parsed body).
console.log("Response", response);
},
function(err) { console.error("Execute error", err); });
}
And This is a JS example on how to manipulate the parameters:
static setConfiguration(params, configuration) {
//To have a destination table we MUST have a tableId
if (params.destinationTable && params.destinationTable.tableId) {
configuration.query.destinationTable = params.destinationTable
}
if (params.clusteringFields) {
configuration.query.clustering = {fields: params.clusteringFields}
}
if (params.timePartitioning) {
configuration.query.timePartitioning = {
type: 'DAY',
field: params.timePartitioning
}
}
if (params.writeDisposition) {
configuration.query.writeDisposition = params.writeDisposition
}
if (params.queryPriority && params.queryPriority.toUpperCase() === "BATCH") {
configuration.query.priority = "BATCH"
}
if (params.useCache === false) {
configuration.query.useQueryCache = params.useCache
}
if (params.maxBillBytes) {
configuration.query.maximumBytesBilled = params.maxBillBytes
}
if (params.maxBillTier) {
configuration.query.maximumBillingTier = params.maxBillTier
}
}
Now after version 2.16.0, BigQueryIO does provide an option to add clusteringFields in dynamic destinations.
#Override
public TableDestination getTable(String eventName) {
return new TableDestination(tableSpec,
tableDescription, timePartitioning, clustering);
}
Notice that the 4th parameter is clustering, which you can use.

Serialize data from PostGIS to JSON

I'm working on a .NET Core REST service which will have a controller that will send geometry data back a GeoJson so I can show it on my Leaflet map.
I'm having trouble converting my class to JSON.
I've tried this:
StringBuilder sb = new StringBuilder();
JsonTextWriter writer = new JsonTextWriter(new StringWriter(sb));
var jsonSerializer = GeoJsonSerializer.Create(new JsonSerializerSettings()
{NullValueHandling = NullValueHandling.Ignore});
jsonSerializer.Serialize(writer, _context.Locations);
writer.Flush();
return Ok(sb.ToString());
But then I get this error
Self referencing loop detected for property 'CoordinateValue' with type 'GeoAPI.Geometries.Coordinate'
I've been looking for examples and reading docs all day but can't get it to work.
The class I need to serialize:
public class SobekLocation
{
public int Id { get; set; }
public string LocationId { get; set; }
public bool Use { get; set; }
public Point Location { get; set; }
}
Any suggestions are much appreciated.
Edit:
I'm making my own featurecollection and am trying to serialize that. It is doing something but it is not valid GeoJSON.
public FeatureCollection GetLocationsAsFeatureCollection()
{
var featureCollection = new FeatureCollection();
foreach (var location in GetLocations().Take(5))
{
var attr = new AttributesTable
{
{"Id", location.Id},
{"Use", location.Use},
{"LocationId", location.LocationId}
};
var feature = new Feature(location.Location, attr);
featureCollection.Add(feature);
}
return featureCollection;
}
And de serialization part:
var fc = _service.GetLocationsAsFeatureCollection();
var sb = new StringBuilder();
var jsonSerializer = JsonSerializer.Create(new JsonSerializerSettings
{
// To prevent the Self referencing loop error:
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
});
using (var sw = new StringWriter(sb))
{
jsonSerializer.Serialize(sw, fc);
}
return Ok(sb.ToString());
The serialized data:
{
"Features":[
{
"Geometry":{
"CoordinateSequence":{
"Dimension":3,
"Ordinates":7,
"Count":1
},
"Coordinates":[
{
"X":191390.015,
"Y":523064.716597462,
"Z":"NaN"
}
],
"NumPoints":1,
"IsEmpty":false,
"Dimension":0,
"BoundaryDimension":-1,
"X":191390.015,
"Y":523064.716597462,
"Coordinate":{
"X":191390.015,
"Y":523064.716597462,
"Z":"NaN"
},
"GeometryType":"Point",
"OgcGeometryType":1,
"Boundary":[
],
"Z":"NaN",
"M":"NaN",
"Factory":{
"PrecisionModel":{
"IsFloating":true,
"MaximumSignificantDigits":16,
"Scale":0,
"PrecisionModelType":0,
"OffsetX":0,
"OffsetY":0
},
"CoordinateSequenceFactory":{
"Ordinates":7
},
"SRID":28992
},
"UserData":null,
"SRID":28992,
"PrecisionModel":{
"IsFloating":true,
"MaximumSignificantDigits":16,
"Scale":0,
"PrecisionModelType":0,
"OffsetX":0,
"OffsetY":0
},
"NumGeometries":1,
"IsSimple":true,
"IsValid":true,
"Area":0,
"Length":0,
"EnvelopeInternal":{
"IsNull":false,
"Width":0,
"Height":0,
"MinX":191390.015,
"MaxX":191390.015,
"MinY":523064.716597462,
"MaxY":523064.716597462,
"Area":0,
"MinExtent":0,
"MaxExtent":0,
"Centre":{
"X":191390.015,
"Y":523064.716597462,
"Z":"NaN"
}
},
"IsRectangle":false
},
"Attributes":[
{
"Key":"Id",
"Value":1
},
{
"Key":"Use",
"Value":false
},
{
"Key":"LocationId",
"Value":"KNP_1"
}
],
"BoundingBox":null
}
],
"Type":"FeatureCollection",
"CRS":null,
"Count":5,
"BoundingBox":null
}
As you can see all properties start with an uppercase but they should be in lowercase. And after changing the properties manually geojsonlint.com reports: GeoJSON features must have a type=feature member
Edit2:
I've made some progress. I'm using the GeoJSON.Net package now and I can now create valid GeoJSON accordin to http://geojsonlint.com/. But still my Leaflet map is complaining about invalid GeoJSON so the map is still not showing anything ;)
Here's my new converter:
public GeoJSON.Net.Feature.FeatureCollection GetLocationsAsGeoJsonFeatureCollection()
{
var featureCollection = new GeoJSON.Net.Feature.FeatureCollection();
foreach (var location in GetLocations())
{
var attr = new Dictionary<string, object>()
{
{"Id", location.Id},
{"Use", location.Use},
{"LocationId", location.LocationId}
};
var position = new Position(location.Location.Coordinate.X, location.Location.Coordinate.Y);
var geom = new GeoJSON.Net.Geometry.Point(position);
var feature = new GeoJSON.Net.Feature.Feature(geom, attr, location.Id.ToString());
featureCollection.Features.Add(feature);
}
return featureCollection;
}
And the GeoJSON it produces:
{
"type":"FeatureCollection",
"features":[
{
"type":"Feature",
"id":"1",
"geometry":{
"type":"Point",
"coordinates":[
523064.716597462,
191390.015
]
},
"properties":{
"id":1,
"use":false,
"locationId":"KNP_1"
}
},
{
"type":"Feature",
"id":"2",
"geometry":{
"type":"Point",
"coordinates":[
519349.860113963,
170162.249352578
]
},
"properties":{
"id":2,
"use":false,
"locationId":"CP_6"
}
},
{
"type":"Feature",
"id":"3",
"geometry":{
"type":"Point",
"coordinates":[
519952.507022603,
170402.383673312
]
},
"properties":{
"id":3,
"use":false,
"locationId":"CP_UR_5_1"
}
},
{
"type":"Feature",
"id":"4",
"geometry":{
"type":"Point",
"coordinates":[
519948.062527202,
170582.612655391
]
},
"properties":{
"id":4,
"use":false,
"locationId":"CP_UR_6_1"
}
},
{
"type":"Feature",
"id":"5",
"geometry":{
"type":"Point",
"coordinates":[
519902.432252114,
170602.894875503
]
},
"properties":{
"id":5,
"use":false,
"locationId":"CP_UR_6_2"
}
}
]
}
This is my code for Leaflet:
L.geoJson('http://localhost:5000/api/data/',
{
attribution: 'Mine'
}
).addTo(map);
I'm still looking for advice how to easily plot my PostGIS data on a Leaflet map.

How to serialize c# object array into json object without an array

I want to use JsonConvert.Serialize in order to serialize a c# array class object into a json non-array object.
public list<employee> employees;
Output:
"{\"employees\":
{\"name\":\"Alex\",\"number\":\"25860340\"},
{\"name\":\"Tom\",\"number\":\"94085345\"}
}"
The format you have asked for in your question would not be valid JSON, because objects are not allowed to follow one another directly unless they are part of an array (see JSON.org). However, you could transform your employee list into a dictionary and serialize that instead, so long as you had a suitable key to use. One idea would be to use the employee number as a key, for example:
var employees = new List<Employee>
{
new Employee { name = "Alex", number = "25860340" },
new Employee { name = "Tom", number = "94085345" }
};
var obj = new
{
employees = employees.ToDictionary(e => e.number)
};
string json = JsonConvert.SerializeObject(obj, Formatting.Indented);
Console.WriteLine(json);
That would give you this output, which is close to what you wanted:
{
"employees": {
"25860340": {
"name": "Alex",
"number": "25860340"
},
"94085345": {
"name": "Tom",
"number": "94085345"
}
}
}
If the employee number isn't actually unique, you could instead use each employee's position in the list as a key like this:
int i = 0;
var obj = new
{
employees = employees.ToDictionary(e => i++)
};
This would give you the following output instead:
{
"employees": {
"0": {
"name": "Alex",
"number": "25860340"
},
"1": {
"name": "Tom",
"number": "94085345"
}
}
}