I am trying to get information about files in a folder using https://apis.live.net/v5.0/folderid/files?
This particular folder of mine has around 5200 files. So I am getting a readtimeout when I make the above mentioned request. Is there any restriction on the number of files that I can make the request.
Note : I am able to successfully retrieve the file information from folder if I restrict the file count to 500 say https://apis.live.net/v5.0/folderid/files?limit=500
In general it's good to page queries that could potentially return a large number of requests. You could try using the limit query parameter in combination with the offset query parameter to read sets of the children at a time and see if that works better for you.
I'll quote in the relevant information from the documentation for ease of reference:
Specify the first item to get by setting the offset parameter in the preceding code to the index of the first item that you want to get. For example, to get two items starting with the third item, use FOLDER_ID/files?limit=2&offset=3.
Note In the JavaScript Object Notation (JSON)-formatted object that's returned, you can look in the paging object for the previous and next structures, if they apply, to get the offset and limit parameter values of the previous and next entries, if they exist.
You may also want to consider swapping to the new API, which has it's own paging model (using the next links).
Related
Good day everyone,
we are trying to have through the use of the integrations of the Apigee service of google all the rows in a bigquery table that have a certain value in a field.
this operation is quite easy to do, but when we have more than 200 lines as a result, problems arise.
The problem is that using the integration to connect to BigQuery I am not returning any listEntitiesPageToken value and not even any listEntitiesNextPageToken value
so i can't figure out how i can go about navigating the result pages
Has anyone had the same problem? What do you suggest?
In the tutorial: "https://cloud.google.com/apigee/docs/api-platform/integration/connectors-task#configure-the-connectors-task" is write : "For example, if you are expecting 1000 records in your result set, you can set the listEntitiesPageSize to 100. So when the Connectors task runs for the first time, it returns the first 100 records, the next 100 records in the second run and so on."
And there is a tip: "Use the listEntitiesPageSize parameter in conjunction with the listEntitiesPageToken parameter to navigate through the pages."
I used the tutorial to understand how to use the task for loop and I understood that I should create a "subintegration" which must be called by a "main integration" for each element present in a list / array.
But what what can i do since these tokens are empty?
https://rally1.rallydev.com/slm/webservice/v2.0/typedefinition/<defect_id>/Attributes
After hitting the specified url we get the fields for the specified defect id but in order to fetch the allowed values for dropdown fields we have to hit another api.
Is there any other way through which we can get all the fields with the allowed values instead of the reference to the allowed values in a single api call?
Unfortunately with the introduction of WSAPI version 2.0, the ability to load sub collections of data within the initial request was removed. This was done in order to improve performance as it was previously possible to request too large a set of data and have a significant impact on the performance of the system.
So the only way to fetch the lists of allowed values for fields is to loop through the list of defect attributes, grab the necessary endpoint url from the _ref value and load it from there.
It could be worth saving the references to these attribute IDs as they shouldn't change as long as the fields aren't removed from the object model.
MarkLogic 9.0.8.2
We have around 20M records in MarkLogic.
For one of the business requirement, we need to generate additional data for each xml and then need user will search this data.
As we can't change original document, so need input on what is best way to manage additional data. Following are the few which we have thought of
Create separate collection and store additional data in separate xml with same unique number i.e. same as original xml. So when user search for it, search in this collection and then retrieved original documents and send response back.
Store additional data in original document properties
We also need to create element range index to make sure it works when end user provide data in range operators.
<abc>
<xyz>
<quan>qty1</quan>
<value1>1.01325E+05</value1>
<unit>Pa</unit>
</xyz>
<xyz>
<quan>qty2</quan>
<value1>9.73E+02</value1>
<value2>1.373E+03</value2>
<unit>K</unit>
</xyz>
<xyz>
<quan>qty3</quan>
<value1>1.8E+03</value1>
<unit>s</unit>
</xyz>
<xyz>
<quan>qty4</quan>
<value1>3.6E+03</value1>
<unit>s</unit>
</xyz>
</abc>
We need to process data from value1 element. User will then search for something like
qty1 >= minvalue AND qty1<=maxvalue
qty2 >= minvalue AND qty2<=maxvalue
qty3 >= minvalue AND qty3<=maxvalue
So when user will search for qty1 then it should only get data from element where value is qty1 and so on.
So would like to know
What is best approach to store data like this
What kind of index i should create to implement this
I would recommend wrapping the original data in an envelope, which allows adding extra data in the header. It could also allow creating a canonical view on the relevant pieces of the data, and either store that as instance, and original as 'attachment' (sub-property, not an attached binary), or keep the instance as-is, and put canonical values for indexing in the header.
There is a lengthy blog article about the topic, that discusses pros and cons in high detail: https://www.marklogic.com/blog/envelope-design-pattern/
HTH!
Grtjn's answer would be the recommended solution, as it is more performant to keep all the information inside the document itself, versus having to query across both the document with the properties, but it would require changes to the document.
Option 1 & 2 could both work.
Properties documents already exist, so it doesn't add fragments, but the properties must conform to the schema.
Creating a sidecar document provides more flexibility, because you are creating new documents, it will increase number of fragments.
I want to use DataTables to show data to a user.
I read the document about "Server-side processing", but
I don't know PHP, so I can't figure out what has happened.
How client-side code sends the data to server-side script?
And how to let server-side script know how many records should be returned?
Please refer the Server-side processing chapter in the DataTables manual. The length parameter determines how many records are requested and start parameter determines first record index (zero-based).
Below is an excerpt from the manual:
start
Paging first record indicator. This is the start point in the current
data set (0 index based - i.e. 0 is the first record).
length
Number of records that the table can display in the current draw. It
is expected that the number of records returned will be equal to this
number, unless the server has fewer records to return. Note that this
can be -1 to indicate that all records should be returned (although
that negates any benefits of server-side processing!)
This sample will give me ALL the GUIDs for HOURS (Yes over 2000 you need to page to get the rest)
htps://YOURCOMAPNY.attask.com/attask/api/v4.0/hour/search?$$LIMIT=2000&fields=ID:*
Object=Hour
field=ID:* (wildcard for all)
If I wanted to do the same thing to SCANS (ScoreCard Answer)
htps://YOURCOMAPNY.attask.com/attask/api/v4.0/scans/search?$$LIMIT=2000&fields=ID:*
Object=scans (Scorecard Answer)
Field=ID:* (Wildcard All)
I get an error:
{"error":{"class":"java.lang.IllegalArgumentException","message":"SCANS is not a top level object and can't be requested directly in internal"}}
I Can do this to pull a ton of project information and I have to parse it all such as produced by this code.
htps://YOURCOMAPNY.attask.com/attask/api/v4.0/proj/search?fields=alignmentScoreCard:,alignmentValues:,alignmentValues:scoreCardQuestion:*
And if I click thru it I can find a ID for a scans and a scoreq and if I take that scans or scoreq and attempt to pull it using the field=ID or
htps://YOURCOMPANY.attask.com/attask/api/v4.0/scans/YOURGUIDHEREXXXXXXXXX/
This again tells me:
{"error":{"class":"java.lang.IllegalArgumentException","message":"SCANS is not a top level object and can't be requested directly in internal"}}
I can pull just about anything and get a list of ID's and then manually pull those items by ID but the scores/scorecard answers/scorecard questions (Even as the API seems to indicate that I can pull this like anything else does NOT allow me to do so?
Suggestions? I know its all associated with projectIDs and such I just want to get a list of them all and work with the ones that I want on a one - one basis without having to pull the PROJ and then rummage thru to locate all the ID's associated with it (If there are one or many) Is this possible?
Thank you!
I don't think there is any workaround for this..
you have to go via projects only..
I am also searching a method for this...
I had to update scorecard answers and I had to go via projects..