How to optimize the Azure Cosmos index for this query - indexing

Question

I have a Cosmos DB with approx. 10GB of data that is used to store analytic data.
The model is as below:
{
"publisherID": "",
"managerID": "",
"mediaID": "",
"type": "",
"ip": "",
"userAgent": "",
"playerRes": "",
"title": "",
"playerName": "",
"videoTimeCode": 0,
"geo": {
"country": "",
"region": "",
"city": "",
"ll": []
},
"date": "",
"uuid": "",
"id": ""
}
I sometimes have very heavy query that are throttling because my RU limit is reached. Before considering increasing my RU limit, I want to make sure my queries are optimized.
All my queries follow the below pattern:
SELECT c.id,c.date,c.uuid,c.type FROM c WHERE c.mediaID = "{ID}" AND (c.type = "Load OR c.type = "Progress" OR c.type = "Play") AND (c.date BETWEEN "2021-06-30T22:00:00.000Z" AND "2021-07-31T21:59:59.999Z")
So after doing some research, I reached the conclusion that the best index I could have is:
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/type/?"
},
{
"path": "/mediaID/?"
},
{
"path": "/date/?"
}
],
"excludedPaths": [
{
"path": "/*"
}
]
}
I got the below stats for this query:
Request Charge: 324.51000000000005 RUs
Showing Results: 1 - 100
Retrieved document count: 10024
Retrieved document size: 8597509 bytes
Output document count: 200
Output document size: 28324 bytes
Index hit document count: 199.24
Index lookup time: 2.41 ms
Document load time: 62.93 ms
Query engine execution time: 15.709800000000001 ms
System function execution time: 0 ms
User defined function execution time: 0 ms
Document write time: 0.47000000000000003 ms
Round Trips: 1
What is worried me is the difference between the Retrieved document count and the Output document count. I'm guessing this is why I need 324 RU just to get the first 100 results...
I'm not sure how to set the index to optimize the performance of the query (always the same pattern: WHERE mediaID = {ID} AND type = {TYPE} AND date Between 2 dates)
Any help would be welcome here.

Answer 1

Thanks for your feedback, that helped a lot!
I added composite index as below:
"compositeIndexes": [
[
{
"path": "/mediaID",
"order": "ascending"
},
{
"path": "/type",
"order": "ascending"
},
{
"path": "/date",
"order": "ascending"
}
]
]
The RU for the initial request in my first post is now 7 RUs (it was 320 before!). Thanks #404
If you don't mind I would like to push a bit further... :)
For another request (same structure), where I know there is a lot of data that need to be retrieved I need 42.02 RUs just to get the first 100 results. Does it make sense?
Request Charg 42.02 RUs
Showing Results 1 - 100
Retrieved document count 200
Retrieved document size 164847 bytes
Output document count 200
Output document size 28544 bytes
Index hit document count 200
Index lookup time 6.98 ms
Document load time 1.3399 ms
Query engine execution time 0.6601 ms
System function execution time 0 ms
User defined function execution time 0 ms
Document write time 0.47000000000000003 ms
Round Trips 1
Is there more optimization that can be done here, other than increasing the RU limit?