I'm working on an app that gets a list of documents/source URL from an api. I'd like to periodically check for new or updated contents within that API so users can update saved items in the database. I'm at a loss on the correct wording to search, thus Google and Stack Overflow have both failed me. My fetching function is below:
The URL for the API is https://api.afiexplorer.com
private fun fetchPubs() {
_binding.contentMain.loading.visibility = View.VISIBLE
request = JsonArrayRequest(
Request.Method.GET,
Config.BASE_URL,
JSONArray(),{ response ->
val items: List<Pubs> =
Gson().fromJson(response.toString(), object : TypeToken<List<Pubs>>() {}.type)
val sortedItems = items.sortedWith(compareBy { it.Number })
pubsList?.clear()
pubsList?.addAll(sortedItems)
// Hardcoded pubs moved to Publications Gitlab Repo
// https://gitlab.com/afi-explorer/pubs
_binding.contentMain.recyclerView.recycledViewPool.clear()
adapter?.notifyDataSetChanged()
_binding.contentMain.loading.visibility = View.GONE
setupData()
Log.i("LENGTH OF DATA", "${items.size}")
},
{error ->
println(error.printStackTrace())
Toasty.error(applicationContext, getString(string.no_internet), Toast.LENGTH_SHORT, true).show()
}
)
MyApplication.instance.addToRequestQueue(request!!)
}
private fun setupData(){
adapter = MainAdapter(applicationContext, pubsList!!, this)
_binding.contentMain.recyclerView.adapter = adapter
}
I tried using ChatGPT to see if that would get me started and that failed miserably. Also searched Google, Reddit and Stack Overflow for similar projects, but mine is a unique scenario I guess. I'm just a hobbyist and intermediate dev I guess. First time working with Volley, everything works, but I would like to find a way to send a notification (preferably not Firebase) if there is updated info within the API listed above. I'm not sure if this is actually doable.
Are you asking if you can somehow find if the remote API has changed its content? If so, how would that service advise you? If the service provider provides a web hook or similar callback you could write a server-based program to send a push notification to your Android app.
Perhaps you intent to poll the API periodically, and then you want to know if there is a change?
If you use a tool such as Postman or curl to easily see the headers of the API https://api.afiexplorer.com you will see, unfortunately, there is no Last-Modified header or ETag header which would allow you to easily determine if there was a change.
Next looking at the content of the API, the author does not provide an obvious version/change date, so no luck there.
What you could do is receive the content as a String, and perform a checksum operation on it, and if it differs you know there has been a change
or if you are deserialising the received JSON in Kotlin data classes, then out of the box, Kotlin will enable you to perform an equality operation on a previous copy of the data to know if there was a change.
This looks like an android app; if so, why don't you create a background service that makes requests to the API and updates the data as needed? You can use an AlarmManager class to set the interval threshold for polling by using the setInexactRepeating() method.
Most apps are updated in this fashion; sometimes, a separate table is created to catalog changesets.
Let me know if this helps.
Related
I am using spring-websocket together with Kotlin Multiplatform, SockJS and StompJS.
I've got my application up and running, mainly using this tutorial from Spring. The frontend is able to send messages to the backend (via the #MessageMapping annotation), which responds back to frontend (via the #SendTo annotation).
The frontend code currently looks like this:
val socket = SockJS("http://localhost:8080/my-app")
val stompClient = Stomp.over(socket)
stompClient.connect(json(), { frame ->
stompClient.subscribe("/topic/a-topic", { message ->
// message handling code
})
})
stompClient.send("/app/hello", null, "")
The subscription approach is useful when an action from some user needs to affect all users (like sending a message in a group chat). However, there are times when some actions should affect only the user who started the action, and that is my problem.
Ideally, I would like to do something to the likes of this:
val socket = SockJS("http://localhost:8080/my-app")
val stompClient = Stomp.over(socket)
stompClient.connect(json(), {})
stompClient.send("/app/hello", null, "").then { response ->
// response handling code
}
But I don't know if it is possible.
I also thought about having a REST API handle this kind of request, but it doesn't seem optimal since there's the existing websocket connection already.
Any enlightment on the matter is appreciated.
I'm trying to launch a crawl via the rest api. A crawl starts with injecting urls. Using a chrome developer tool "Advanced Rest Client" I'm trying to build this POST payload up but the response I get is a 400 Bad Request.
POST - http://localhost:8081/job/create
Payload
{
"crawl-id":"crawl-01",
"type":"INJECT",
"config-id":"default",
"args":{ "path/to/seedlist/directory"}
}
My problem is in the args, I think more is needed but I'm not sure. In the NutchRESTAPI page this is the sample it gives for creating a job.
POST /job/create
{
"crawlId":"crawl-01",
"type":"FETCH",
"confId":"default",
"args":{"someParam":"someValue"}
}
POST /job/create
{
"crawlId":"crawl-01",
"jobClassName":"org.apache.nutch.fetcher.FetcherJob"
"confId":"default",
"args":{"someParam":"someValue"}
}
I'm not sure what param or value to give each of the commands to complete a job. (eg. Inject, Generate, Fetch, Parse, and UpdateDb) Can someone clear this up? How do I tell the api where to look for the seedlist at?
UPDATE
When trying to complete the Generate command I came into a classException error where the value for the topN key is to be of type long but the api reads it as either a string or an int. I found a fix that is supposed to included in the 2.3.1 release (release date: TBA) and applied it and recompiled my code. It can now work.
At the time of this posting, the REST API is not yet complete. A much more detailed document exists, though it's still not comprehensive. It is linked to in the following email from the user mailing list (which you might want to consider joining):
http://www.mail-archive.com/user%40nutch.apache.org/msg13652.html
But to answer your question about the seedlist, you can create the seedlist through REST, or you can use the argument "seedDir"
{
"args":{
"seedDir":"/path/to/seed/directory"
},
"confId":"default",
"crawlId":"sample-crawl-01",
"type":"INJECT"
}
I've run into the following problem when porting an app from REST API to GDAA.
The app needs to download some of (thousands of) JPEG images based on user selection. The way this is solved in the app is by downloading a thumbnail version first, using this construct of the REST API:
private static InputStream getCont(String rsid, boolean bBig){
InputStream is = null;
if (rsid != null) try {
File gFl = bBig ?
mGOOSvc.files().get(rsid).setFields("downloadUrl" ).execute():
mGOOSvc.files().get(rsid).setFields("thumbnailLink").execute();
if (gFl != null){
GenericUrl url = new GenericUrl(bBig ? gFl.getDownloadUrl() : gFl.getThumbnailLink());
is = mGOOSvc.getRequestFactory().buildGetRequest(url).execute().getContent();
}
} catch (UserRecoverableAuthIOException uraEx) {
authorize(uraEx.getIntent());
} catch (GoogleAuthIOException gauEx) {}
catch (Exception e) { }
return is;
}
It allows to get either a 'thumbnail' or 'full-blown' version of an image based on the bBig flag. User can select a thumbnail from a list and the full-blown image download follows (all of this supported by disk-base LRU cache, of course).
The problem is, that GDAA does not have an option to ask for reduced size / thumbnail version of an object (AFAIK), so I have to resort to combining both APIs, which makes the code more convoluted then I like (bottom of the page). Needles to state that the 'Resource ID' needed by the REST may not be immediately available.
So, the question is: Is there a way to ask GDAA for a 'thumbnail' version of a document?
Downloading thumbnails isn't currently available in the Drive Android API, and unfortunately I can't give a timeframe to when it will be available. Until that time, the Drive Java Client Library is the best way to get thumbnails on Android.
We'd appreciate if you go ahead and file a feature request against our issue tracker: https://code.google.com/a/google.com/p/apps-api-issues/
That gives requests more visibility to our teams internally, and issues will be marked resolved when we release updates.
Update: I had an error in the discussion of the request fields.
As Ofir says, you can't get thumbnails with the Drive Android API and you can get thumbnails with the Drive Java Client Library. This page has is a really good primer for getting started:
https://developers.google.com/drive/v3/web/quickstart/android
Oddly, I can't get the fields portion of the request to work as it is on that quick start. As I've experienced, you have to request the fields a little differently.
Since you're doing a custom field request you have to be sure to add the other fields you want as well. Here is how I've gotten it to work:
Drive.Files.List request = mService.files()
.list()
.setFields("files/thumbnailLink, files/name, files/mimeType, files/id")
.setQ("Your file param and/or mime query");
FileList files = request.execute();
files.getFiles(); //Each File in the collection will have a valid thumbnailLink
A sample query might be:
"mimeType = 'image/jpeg' or mimeType = 'video/mp4'"
Hope this helps!
I want to use breezejs api for storing data in local storage (indexdb or websql) and also want to sync local data with sql server.
But I am failed to achieve this and also not able to find sample app of this type of application using breezejs, knockout and mvc api.
My requirement is:
1) If internet is available, the data will come from sql server by using mvc web api.
2) If internet is shutdown, the application will retrieve data from cached local storage (indexdb or websql).
3) As soon as internet is on, the local data will sync to sql server.
Please let me know Can I achieve this requirement by using breezejs api or not?
If yes, please provide me some and links and sample.
If no, what other can we use for achieving this type of requirement?
Thanks.
Please help me to meet this requirement.
You can do this, but I would suggest simply using localstorage. Basically, every time you read from the server or save to the server, you export the entities and save that to local storage. THen, when you need to read in the data, if the server is unreachable, you read the data from localstorage and use importentities to get it into the manager and then query locally.
function getData() {
var query = breeze.EntityQuery
.from("{YourAPI}");
manager.executeQuery.then(saveLocallyAndReturnPromise)
.fail(tryLocalRestoreAndReturnPromise)
// If query was successful remotely, then save the data in case connection
// is lost
function saveLocallyAndReturnPromise(data) {
// Should add error handling here. This code
// assumes tis local processing will be successful.
var cacheData = manager.exportEntities()
window.localStorage.setItem('savedCache',cacheData);
// return queried data as a promise so that this detour is
// transparent to viewmodel
return Q(data);
}
function tryLocalRestoreAndReturnPromise(error) {
// Assume any error just means the server is inaccessible.
// Simplified for example, but more robust error handling is
// warranted
var cacheData = window.localStorage.getItem('savedCache');
// NOTE: should handle empty saved cache here by throwing error;
manager.importEntities(cacheData); // restore saved cache
var query = query.using(breeze.FetchStrategy.FromLocalCache);
return manager.executeQuery(query); // this is a promise
}
}
This is a code skeleton for simplicity. You should check catch and handle errors, add an isConnected function to determine connectivity, etc.
If you are doing editing locally, there are a few more hoops to jump through. Every time you make a change to the cache, you will need to export either the whole cache or the changes (probably depending on the size of the cache). When there is a connection, you will need to test for local changes first and, if found, save them to the server before requerying the server. In addition, any schema changes made while offline complicate matters tremendously, so be aware of that.
Hope this helps. A robust implementation is a bit more complex, but this should give you a starting point.
I have a Backbone application, which has a collection called Links. Links maps to a REST API URI of /api/links.
The API will give the user the latest links. However, I have a system in place that will add a job to the message queue when the user hits this API, requesting that the links in the database are updated.
When this job is finished, I would to push the new links to the Backbone collection.
How should I do this? In my mind I have two options:
From the Backbone collection, long poll the API for new links
Setup WebSockets to send a "message" to the collection when the job is done, sending the new data with it
Scrap the REST API for my application and just use WebSockets for everything, as I am likely to have more realtime needs later down the line
WebSockets with the REST API
If I use WebSockets, I'm not sure of the best way to integrate this into my Backbone collection so that it works alongside the REST API.
At the moment my Backbone collection looks like this:
var Links = Backbone.Collection.extend({
url: '/api/links'
});
I'm not sure how to enable the Backbone collection to handle AJAX and WebSockets. Do I continue to use the default Backbone.sync for the CRUD Ajax operations, and then deal with the single WebSocket connection manually? In my mind:
var Links = Backbone.Collection.extend({
url: '/api/links',
initialize: function () {
var socket = io.connect('http://localhost');
socket.on('newLinks', addLinks)
},
addLinks: function (data) {
// Prepend `data` to the collection
};
})
Questions
How should I implement my realtime needs, from the options above or any other ideas you have? Please provide examples of code to give some context.
No worries! Backbone.WS got you covered.
You can init a WebSocket connection like:
var ws = new Bakcbone.WS('ws://exmaple.com/');
And bind a Model to it like:
var model = new Backbone.Model();
ws.bind(model);
Then this model will listen to messages events with the type ws:message and you can call model.send(data) to send data via that connection.
Of course the same goes for Collections.
Backbone.WS also gives some tools for mapping a custom REST-like API to your Models/Collections.
My company has a fully Socket.io based solution using backbone, primarily because we want our app to "update" the gui when changes are made on another users screen in real time.
In a nutshell, it's a can of worms. Socket.IO works well, but it also opens a lot of doors you may not be interested in seeing behind. Backbone events get quite out of whack because they are so tightly tied to the ajax transactions...you're effectively overriding that default behavior. One of our better hiccups has been deletes, because our socket response isn't the model that changed, but the entire collection, for example. Our solution does go a bit further than most, because transactions are via a DDL that is specifically setup to be universal across the many devices we need to be able to communicate with, now and in the future.
If you do go the ioBind path, beware that you'll be using different methods for change events compared to your non-socket traffic (if you mix and match) That's the big drawback of that method, standard things like "change" becomes "update" for example to avoid collisions. It can get really confusing in late-night debug or when you have a new developer join the team. For that reason, I prefer either going sockets, or not, not a combination. Sockets have been good so far, and scary fast.
We use a base function that does the heavy lifting, and have several others that extend this base to give us the transaction functionality we need.
This article gives a great starter for the method we used.