I've built two APIs with Kimonlabs.
API "A" gets a list of URLs.
API "B" gets several values from a web page.
I've set the crawl strategy for "B" to "URLs from Source API", and set the API to "A".
My question is about whether the list of URLs that "B" will process is dynamic, i.e. will the list change automatically the next time that "A" updates? Or is the list of URLs static at the time I create B?
Every API acts according it's settings:
"Specify how often this API will automatically fetch new data from the target page(s). APIs are limited to 1 URL for a hourly auto-run, <1000 URLs for a daily auto-run, and <10,000 URLs for a weekly auto-run."
Dynamic. B will crawl the up-to-date set of URLs from A's last run.
Related
I have seen that there are various APIs and various tools that allow you to see the most visited pages of the Wikimedia projects such as Wikipedia, but all these services have a limit, they do not allow to show more than 1000 pages, while I would like to have the list of 5000-10000(or more) most visited pages in order of traffic.
these are all the services that I checked and with which I found this limit:
https://en.wikipedia.org/w/api.php?action=help&modules=query%2Bmostviewed
https://stats.wikimedia.org/#/en.wikipedia.org/reading/top-viewed-articles/normal|table|last-month|~total|monthly
https://pageviews.toolforge.org/topviews/?project=en.wikipedia.org&platform=all-access&date=last-month&excludes=
https://wikimedia.org/api/rest_v1/#/Pageviews%20data
I have also found services like https://quarry.wmflabs.org/ or
https://query.wikidata.org/ where you can run a query, technically perhaps through this service you could but I don't know the query to be performed to show the pages with most visits.
I also found an interesting article here: https://www.reddit.com/r/bigquery/comments/3dg9le/analyzing_50_billion_wikipedia_pageviews_in_5/ where it is explained that it is possible to use Google's BigQuery but it is an external service and before using it I wanted to know if it existed a simpler method.
If the REST API doesn't suit your purpose, you'd need to parse the raw data yourself. That's because all the tools you've linked just consume the REST API.
The raw data are available at https://dumps.wikimedia.org/other/pageviews/. There are two groups of files there. One starts with pageviews-, which lists the number of views of individual pages, the second starts with projectviews-, which lists the number of views of individual projects.
For your target, you need the pageviews ones. Download the files for your timespan, and then analyze them using a script.
The file is space-separated. Each row represents one page that was visited in that hour. First column represents the project (en is English Wikipedia, for instance), second is the page title (spaces are represented by underscores) and then there are total pageviews.
The technical documentation is available at https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Pageviews.
Here is what I have:
Project in Google API (Translate API)
Billing Enabled
Upgraded Quota limit up to 10 000 000 chars per day
One server key
Many sites, which use this key to translate text
And here is the problem: it seems our sites request more texts for translations than we expect and we can't find which of the sites sends so many texts. We have looked at our sites and found nothing. We want to determine translations requests volume by each site or module.
I can see overall usage and quota usage in API Manager. But there is no information about IP / domain name input to the overall usage. Is it possible to see translations usage by site / API usage / other parameter.
I have some ideas and questions. First, I can create a unique API key for each site/module so each site will use its own key. But it seems it is not possible to see translations usage by keys within one project.
The second idea was about adding parameter in URL like user_id. That method was mentioned in API docs but, again, I can't see how to generate translations usage by a certain parameter.
Could you please advise?
I have sort of a problem how to use Google Analytics properly with Boostrap.
My page has 3 level deep subpages and the last subpage has it's own subdomain. In GA I see I can use max. 50 tracking codes within one service. What if I need more than that?
You are limited to 50 properties not 50 pages. Each property can track many pages and (up to 10 million hits a month for the free version) and events.
Typically you would use the same property code on all pages on the same site so you can see all that data together (though with option to drill down).
You would only use a new property code for a new site (though your subdomain might qualify for that if you want to track it separately).
So the two questions you want to ask yourself are:
Do you want to be able to report on two pages together? E.g. To see that your site gets 10,000 hits and 20% are for this page and 5% are for that page. Or people start at this page and then go to that page and then on to this page. If so it should be the same analytics property.
Do different people need to see these page stats? And is it a problem if they do? If so put as a separate property so you can permission separately.
It sounds like these are part of the same site so I'd be veering towards tracking them together on same property.
On a different note you should set one page as the main version (with a rel canonical tag) and redirect other version to that page to avoid confusing search engines thinking you have duplicated content. Do you have a reason for having the same content on two different addresses? It can cause SEO and other problems.
I have an android application connected to magento server by rest api. The application must preform some actions, but I have no examples of requests needed for the application.
For example I have this list of requests, but it is superficial enough for my application. For example, I do request:
"http://myUrl.com/api/rest/products?limit=15" and receive the list of 15 products. But after I need to receive next 15 product, and next, and next... What request can do it?
Also I need to do another requests which have no in site examples.
Where can I get more information and examples about rest api requests? Thank You in advance.
You can use various get filters available for REST request in magento GET FILTERS. To access second page you can use page filter available i.e. "http://myUrl.com/api/rest/products?limit=15&page=2". You can also combine many filters and use them as you want. Eg. For getting a result of products which have their name like 'product123', you can use: magentohost/api/rest/products?filter[1][attribute]=name&filter[1][like]=%product123%.
Scenario
I am working on a web application assume it www.abc.com which having a profile for all users
www.abc.com/username
and all users have a dashboard for controlling their profiles
Requirement
i have one analytics profile for www.abc.com but my requirement is
a to show stats to all users on their dashboard
can i get this by google analytics API
Visits
demographics
all traffic source
and keywords
i have integrated reporting by API on one of my project but that is for the domain . i am not sure for my requirement.
Resource-guru.com, what you can do is to pull all the data with page path dimension included, and then simply filter the results if the username string is found in the page path.
As for the second part of your question - you can get:
visits (metric)
traffic sources as well as keywords (dimensions, but remember (not provided) might make this useless report)
you can NOT get demographics data via API.
Hope this helps.
You can use the API to do this but remember your going to have an issue with the fact that you can only make 10k requests per day per view (profile).
The Demographics report displays age and gender. Those dimensions can be found under the Audience - Dimensions & Metrics Reference
ga:visitorAgeBracket
ga:visitorGender
ga:interestOtherCategory
ga:interestAffinityCategory
Traffic Source is just really just a mix up of ga:sourceMedium , ga:campaign maybe a few others depending on what information you want to display.
You may have issues with Keywords because due to ssl and trying to keep user info private Google has stopped recording this sometimes you get (not provided). But you can get that information from webmaster tools. Its just hard to merge it with your GA data then.
Keyword: The keywords that visitors searched are usually captured in
the case of search engine referrals. This is true for both organic and
paid search. Note, however, that when SSL search is employed, Keyword
will have the value (not provided).