Add exception to natural sorting in datatables - datatables

Is there a way to add exceptions to the natural ordering plugin, so that it ignores things like c. , [, ], ? ?
This is an example of my data:
161?
1604
[1563]
c. 1476
I'd like the sorted asc. output to be:
c. 1476
[1563]
1604
161?
Right now what I get is all the numbers first, and the strings beginning with [ afterwards.
My initialisation code:
<script type="text/javascript" src="//cdn.datatables.net/plug-ins/1.10.24/sorting/natural.js"></script>
$('#sourcesList').DataTable({
"paging": false,
"columnDefs": [
{ type: 'natural-nohmtl', targets: '_all' }
],
[...]
PS: this is my data in the wild.

The natural sorting option takes care of certain types of numeric data which you can reasonably expect to encounter in the "real world" - such as numbers with different thousands separators, or with currency symbols of different types in different positions - or "1st", "2nd" "3rd" and so on. Don't quote me on these exact examples, as I have not looked at that add-on in detail. But that is the overall idea.
Items such as question-marks - I would not expect that add-on to handle those. Items in square brackets - same thing.
Given your clarification in a comment, I think you do not need or want this add-on.
Instead you can use a column renderer with DataTables' ability to store different versions of a value - for display, sorting, and filtering.
var dataSet = [ ["161?"], ["1604"], ["[1563]"], ["c. 1476"] ];
$(document).ready(function() {
$('#example').DataTable( {
data: dataSet,
columnDefs: [ {
type: "natural-nohmtl",
targets: [ 0 ],
title: "My Data",
"render": function ( data, type, row, meta ) {
if ( type === 'sort' ) {
//return parseInt(data.replace(/\D/g, '')); // numbers as numbers
return data.replace(/\D/g, ''); // numbers as strings
} else {
return data;
}
}
} ]
} );
} );
This saves a version of each value with all non-digits stripped from the value, using replace(/\D/g, ''). This altered version of the data is stored as the value which will be used when sorting. The unaltered version is what the user will see.
My result:
This is the case where numbers are treated as text.
You can uncomment the commented-out return statement to get numbers as numbers.
However...
This crude approach of stripping non-digits from the data meets the narrow example from your question, but it may not be sufficient for all the data you are expecting to handle. So, you may need to refine the "replace" logic.
But the render function should meet your needs, more generally.

Related

How to identify if one json contains specific value in json schema

There is a json result like this below, and I need to check if a specific value is contained in shop_ids, for example, 232323 is contained, while 232324 is not. Is it possible to achieve that with JSON Schema? How?
{
"request":{
"0":{
"cids":{
"0":"a",
"1":"b",
"2":"c",
"3":"d",
"4":"e",
"5":"f",
"6":"g",
"7":"h"
},
"data_type":"1",
"key_type":"1",
"one_item_id":"0",
"queries":{
"0":{
"0":"query1"
},
"1":{
"0":"query2"
}
},
"shop_ids":{
"0":"11111",
"1":"232323",
"2":"33333",
"3":"124151",
"4":"55555",
"5":"12121"
},
"num":"30"
}
}
}
note: the shop_ids may contain more numeric fields, not just 0-5.
I tried "patternProperties", it does work for checking specific key-value but returns fail on other key-value. What I need is just any of "shop_ids" match the specific value is ok.

Is there a way to use the graphLookup aggregation pipeline stage for arrays?

I am currently working on an application that uses MongoDB as the data repository. I am mainly concerned about the graphLookup query to establish links between different people, based on what flights they took. My document contains an array field, that in turn contains key value pairs. I need to establish the links based on one of the key:value pairs of that array.
I have already tried some queries of aggregation pipeline with $graphLookup as one of the stages and they have all worked fine. But now that I am trying to use it with an array, I am hitting a blank.
Below is the array field from the first document :
"movementSegments":[
{
"carrierCode":"MO269",
"departureDateTimeMillis":1550932676000,
"arrivalDateTimeMillis":1551019076000,
"departurePort":"DOH",
"arrivalPort":"LHR",
"departurePortText":"HAMAD INTERNATIONAL AIRPORT",
"arrivalPortText":"LONDON HEATHROW",
"serviceNameText":"",
"serviceKey":"BA007_1550932676000",
"departurePortLatLong":"25.273056,51.608056",
"arrivalPortLatLong":"51.4706,-0.461941",
"departureWeeklyTemporalSpatialWindow":"DOH_8",
"departureMonthlyTemporalSpatialWindow":"DOH_2",
"arrivalWeeklyTemporalSpatialWindow":"LHR_8",
"arrivalMonthlyTemporalSpatialWindow":"LHR_2"
}
]
The other document has the below field :
"movementSegments":[
{
"carrierCode":"MO269",
"departureDateTimeMillis":1548254276000,
"arrivalDateTimeMillis":1548340676000,
"departurePort":"DOH",
"arrivalPort":"LHR",
"departurePortText":"HAMAD INTERNATIONAL AIRPORT",
"arrivalPortText":"LONDON HEATHROW",
"serviceNameText":"",
"serviceKey":"BA003_1548254276000",
"departurePortLatLong":"25.273056,51.608056",
"arrivalPortLatLong":"51.4706,-0.461941",
"departureWeeklyTemporalSpatialWindow":"DOH_4",
"departureMonthlyTemporalSpatialWindow":"DOH_1",
"arrivalWeeklyTemporalSpatialWindow":"LHR_4",
"arrivalMonthlyTemporalSpatialWindow":"LHR_1"
},
{
"carrierCode":"MO270",
"departureDateTimeMillis":1548254276000,
"arrivalDateTimeMillis":1548340676000,
"departurePort":"DOH",
"arrivalPort":"LHR",
"departurePortText":"HAMAD INTERNATIONAL AIRPORT",
"arrivalPortText":"LONDON HEATHROW",
"serviceNameText":"",
"serviceKey":"BA003_1548254276000",
"departurePortLatLong":"25.273056,51.608056",
"arrivalPortLatLong":"51.4706,-0.461941",
"departureWeeklyTemporalSpatialWindow":"DOH_4",
"departureMonthlyTemporalSpatialWindow":"DOH_1",
"arrivalWeeklyTemporalSpatialWindow":"LHR_4",
"arrivalMonthlyTemporalSpatialWindow":"LHR_1"
}
]
And I am running the below query :
db.person_events.aggregate([
{ $match: { eventId: "22446688" } },
{
$graphLookup: {
from: 'person_events',
startWith: '$movementSegments.carrierCode',
connectFromField: 'carrierCode',
connectToField: 'carrierCode',
as: 'carrier_connections'
}
}
])
The above query creates an array field in the document, but there are no values in it. As per the expectation, both my documents should get linked based on the carrier number.
Just to be clear about the query, the documents contain an eventId field, and the match pipeline returns one document to me after the match stage.
Well, I don't know how I missed it, but here is the solution to my problem which gives me the required results :
db.person_events.aggregate([
{ $match: { eventId: "22446688" } },
{
$graphLookup: {
from: 'person_events',
startWith: '$movementSegments.carrierCode',
connectFromField: 'movementSegments.carrierCode',
connectToField: 'movementSegments.carrierCode',
as: 'carrier_connections'
}
}
])

query for Time Stamp in mongo [duplicate]

I have a problem when querying mongoDB with nested objects notation:
db.messages.find( { headers : { From: "reservations#marriott.com" } } ).count()
0
db.messages.find( { 'headers.From': "reservations#marriott.com" } ).count()
5
I can't see what I am doing wrong. I am expecting nested object notation to return the same result as the dot notation query. Where am I wrong?
db.messages.find( { headers : { From: "reservations#marriott.com" } } )
This queries for documents where headers equals { From: ... }, i.e. contains no other fields.
db.messages.find( { 'headers.From': "reservations#marriott.com" } )
This only looks at the headers.From field, not affected by other fields contained in, or missing from, headers.
Dot-notation docs
Since there is a lot of confusion about queries MongoDB collection with sub-documents, I thought its worth to explain the above answers with examples:
First I have inserted only two objects in the collection namely: message as:
> db.messages.find().pretty()
{
"_id" : ObjectId("5cce8e417d2e7b3fe9c93c32"),
"headers" : {
"From" : "reservations#marriott.com"
}
}
{
"_id" : ObjectId("5cce8eb97d2e7b3fe9c93c33"),
"headers" : {
"From" : "reservations#marriott.com",
"To" : "kprasad.iitd#gmail.com"
}
}
>
So what is the result of query: db.messages.find({headers: {From: "reservations#marriott.com"} }).count()
It should be one because these queries for documents where headers equal to the object {From: "reservations#marriott.com"}, only i.e. contains no other fields or we should specify the entire sub-document as the value of a field.
So as per the answer from #Edmondo1984
Equality matches within sub-documents select documents if the subdocument matches exactly the specified sub-document, including the field order.
From the above statements, what is the below query result should be?
> db.messages.find({headers: {To: "kprasad.iitd#gmail.com", From: "reservations#marriott.com"} }).count()
0
And what if we will change the order of From and To i.e same as sub-documents of second documents?
> db.messages.find({headers: {From: "reservations#marriott.com", To: "kprasad.iitd#gmail.com"} }).count()
1
so, it matches exactly the specified sub-document, including the field order.
For using dot operator, I think it is very clear for every one. Let's see the result of below query:
> db.messages.find( { 'headers.From': "reservations#marriott.com" } ).count()
2
I hope these explanations with the above example will make someone more clarity on find query with sub-documents.
The two query mechanism work in different ways, as suggested in the docs at the section Subdocuments:
When the field holds an embedded document (i.e, subdocument), you can either specify the entire subdocument as the value of a field, or “reach into” the subdocument using dot notation, to specify values for individual fields in the subdocument:
Equality matches within subdocuments select documents if the subdocument matches exactly the specified subdocument, including the field order.
In the following example, the query matches all documents where the value of the field producer is a subdocument that contains only the field company with the value 'ABC123' and the field address with the value '123 Street', in the exact order:
db.inventory.find( {
producer: {
company: 'ABC123',
address: '123 Street'
}
});

understand azure search charFilters mapping

I create my index with following custom analyzer
"analyzers":[
{
"name":"shinglewhite_analyzer",
"#odata.type":"#Microsoft.Azure.Search.CustomAnalyzer",
"charFilters":[
"map_dash"
],
"tokenizer":"whitespace",
"tokenFilters":[
"shingle"
]
}
],
"charFilters":[
{
"name":"map_dash",
"#odata.type":"#Microsoft.Azure.Search.MappingCharFilter",
"mappings":[ "_=> " ]
}
]
The problem is that word like ice_cream from input will not match query ice cream, it matches icecream though. Can someone help me understand how this works and if I have done something wrong?
Also we'd like query "ice cream" to match "ice cream", "icecream" and "ice and cream" but favor those in order.
in order to map to a space please use the following notation (we'll update the docs to include this information):
{
"name":"map_dash",
"#odata.type":"#Microsoft.Azure.Search.MappingCharFilter",
"mappings":[ "_=>\\u0020" ]
}
Also, by default the shingle token filter separates tokens with a space. If you want to join subsequent tokens into one without a separator you need to customize your filter like in the following example:
{
"name": "my_shingle",
"#odata.type":"#Microsoft.Azure.Search.ShingleTokenFilter",
"tokenSeparator": ""
}
With those two changes for token ice_cream your analyzer will generate: ice, icecream, cream.
I hope that helps

MongoDB like statement with multiple fields

With SQL we can do the following :
select * from x where concat(x.y ," ",x.z) like "%find m%"
when x.y = "find" and x.z = "me".
How do I do the same thing with MongoDB, When I use a JSON structure similar to this:
{
data:
[
{
id:1,
value : "find"
},
{
id:2,
value : "me"
}
]
}
The comparison to SQL here is not valid since no relational database has the same concept of embedded arrays that MongoDB has, and is provided in your example. You can only "concat" between "fields in a row" of a table. Basically not the same thing.
You can do this with the JavaScript evaluation of $where, which is not optimal, but it's a start. And you can add some extra "smarts" to the match as well with caution:
db.collection.find({
"$or": [
{ "data.value": /^f/ },
{ "data.value": /^m/ }
],
"$where": function() {
var items = [];
this.data.forEach(function(item) {
items.push(item.value);
});
var myString = items.join(" ");
if ( myString.match(/find m/) != null )
return 1;
}
})
So there you go. We optimized this a bit by taking the first characters from your "test string" in each word and compared the tokens to each element of the array in the document.
The next part "concatenates" the array elements into a string and then does a "regex" comparison ( same as "like" ) on the concatenated result to see if it matches. Where it does then the document is considered a match and returned.
Not optimal, but these are the options available to MongoDB on a structure like this. Perhaps the structure should be different. But you don't specify why you want this so we can't advise a better solution to what you want to achieve.