I am trying to figure out that if there is something that can help the use case mentioned below:
My use case is that I have an API that gives the response for the DB for a given time period. Now for the smaller time period, there is no issue but if the time period for which the query is being made increases then it will add a significant amount of time until the API responds. I do not want the UI to keep loading until the full response is received.
Thus I was thinking that there should be some mechanism using which I can get the response from the API in an incremental fashion(in batch) so that I can show the user and the user do not have to wait until the API is executed completely.
Any help on coding or design would be greatly appreciated.
Well I did some R/D and was able to fulfill my requirement with the Pagination package of flask_mongoengine
http://docs.mongoengine.org/projects/flask-mongoengine/en/latest/
this has the sample implementations.
I'm trying to add ~20 tracking category options programmatically and I've seen that calls to add the first two values returns in < 1 second, but after that calls timeout.
If I restart with a new session, I see the same behavior (first two calls are quick and then calls timeout).
My question is, is there rate limiting on the Xero API? Has anyone seen behavior similar to this? Workarounds or fixes?
UPDATE: I forgot to add that i did look at rate limits posted by Xero, I'm not anywhere close to any of those limits
I think xero API give you the option to save a bunch of tracking categories within one request
I would like to do the following using OKTA api:
One time, I would like to pull the entire system log.
Going forward I would like to pull only the days log information.
The challenge that I am facing is whenever I get the logs, I only get 1000 records. How do I get the whole days log, it maybe more that 1000 records. Is there some body who can help me with a piece of code which shows how to do this.
Thanks
You can use the Events API to retrieve this information. This API supports Pagination so you can retrieve all the events for a particular filter (like all events after a certain point in time).
1000 is the default limit for the Events API because this object can potentially contain a lot of data.
However, you can specify how many records for a specific time range are returned via the Events API using filters. For example, the following GET statement would retrieve the first 100 successful login requests since 1-Mar.
https://{{YOUR_COMPANY}}.okta.com/api/v1/events?limit=100&filter=published gt "2015-01-01T00:00:00.000Z" and action.objectType eq “core.user_auth.login_success"
If there are more than 100 records, you can get the next set by passing rel=“next” in the next request header. If you wanted to get only messages for today, you could change the date.
Right, this is really working on my nerves, but Instagram has to do something about their bloody documentation.
I am already trying for a week to live update my website with new instagram posts without refreshing the page. Twitter was fairly easy, but instagram is just one big mess. Basically I use the realtime Instagram API, the callback and all that stuff is working fine, but thanks to Instagram it does not return me an ID from the post that is new, the callback only returns some basic stuff:
[{"changed_aspect": "media", "object": "tag", "object_id": "nofilter", "time": 1391091743, "subscription_id": xxxxx, "data": {}}]
with this data you are nothing, except for the Tag, but I knew the tag before this callback too so doesn't matter. It actually only tells me that there is a new post. I have tried doing the same request as when the page loads, when this callback occurs, and get the posts that are newer than those already on the page. Unfortunately I have not succeeded in this yet. I have picked the ID from the last posted instagram post, and checked if it is in the callback request, and it's not.
What am I doing wrong?
I'd appreciate some help, thanks!
Edit:
I'd like to note that this is not only a problem with the realtime api, but also with the normal API. I just don't know how to compare data so I don't get duplicates in my database(normal api), or on my website (realtime). I can't find any tutorial or documentation (Yes, I might be blind), that explains to me how to compare data. I can only find the min_id and max_id, but no explanation what these id's contains. I checked these id's with id's from results, and they do not match. It's not an ID from a media item.
I also checked the next_url, and in my logic thinking, this should be a URL to the next page (like Twitter).
Am I looking at this all wrong?
Ok strike my old answer, I changed the way I do this. Here's how I'll do it now.
I still wait for 10 hits on my Real-time subscription, when I reach 10 I send off a new thread (if one is not already running).
The sync thread queries my DB for a value, I need the last min_tag_id I used. Then I query:
https://api.instagram.com/v1/tags/*/media/recent?access_token=*&min_tag_id=*
Try it out here: https://api.instagram.com/v1/tags/montreal/media/recent?access_token=*
You'll get 20 results, and a min_tag_id value. Append that to your url, you'll see you get no results. Wait a couple of seconds and refresh. Eventually you'll get some media, and a new min_tag_id.
(You can ignore the "next_url" value they give you, you won't be using that).
Basically you only need to store that min_tag_id and query until you have no more results, that means you're done then.
When you get a subscription push, you need to query that endpoint (tag / recent).
I normally start an synchronous thread to perform this so I can answer in under 2 seconds to Instagram.
Then you parse that endpoint and look for a "next url" value.
Keep querying that end point, parsing the media and going to the next url until you find your stop condition.
For me I try to match 10 consecutive records in my DB. Basically from the tag, I store media when then meet my business rules.
The Instagram documentation is accurate and actually well written.
The realtime API is working correctly. As stated in the documentation:
The changed data is not included in the payload, so it is up to you
how you'd like to fetch the new data. For example, you may decide only
to fetch new data for specific users, or after a certain number of
photos have been posted.
http://instagram.com/developer/realtime/
You only receive a notification that an update has happened to your subscribed object. It is up to you to call the API to find out what that data is.
You can call the /tags/[tag-name]/media/recent with an access token that you have previously stored on your own server or DB. Then, you should be able to compare the data returned from that endpoint with any data you have retrieved prior, and just pull the objects that you do not yet have.
I've noticed when retrieving a story/defect after first updating it, sometimes the retrieve response returns the field values as if the update never happened. Retrying the retrieve after a short delay (~500ms) returns the updated field values as expected. Is this a known behaviour? Is there any way of avoiding this?
I'm using the Rally API 2.0 - https://rally1.rallydev.com/slm/webservice/v2.0/
The update is being performed using this URI:
POST /slm/webservice/v2.0/Defect/14173461229?key=<key> HTTP/1.1
I'm retrieving the story after update as follows:
GET /slm/webservice/v2.0/artifact?query=(ObjectId%20=%2014173461229)&start=1&pagesize=20&fetch=true HTTP/1.1
What is your integration doing that it needs to re-poll the artifact within < 1 second of POST'ing an update? Is there a second process that does polling that is revealing the latency for the updates? Does your integration run multiple threads? Does the response time vary at all depending on time of day, etc.? There are any number of factors that could be at play here, but 500 ms doesn't seem like an un-reasonable refresh rate given factors such as latency over HTTP/S as well as server-side database and cache updates. That said, for an in-depth look you may wish to inquire with Rally Support (rallysupport#rallydev.com) as they have tools that can help evaluate server-side response time corresponding to requests by specific UserID.