Is it possible to get a list of photos from Picasa (or Google Plus that still uses the Picasa API) that are older than a certain date? I would like to get all photos from today a year ago from a large album, the "InstantUpload" album.
These are the things I looked into:
The example in the Java Developers Guide only retrieves a limited number of "recently uploaded" photo's.
This reported issue makes clear that the PicasaWeb query does not allow the published-min (or published-max) as query parameters.
The start-index in this query does not seem to have any effect.
The work around using a "Partial response", as suggested in the issue mentioned above by grab...#gmail.com, does not work to get photo's earlier than a certain (seemingly rather random) date.
EDIT:
googlecl has been discontinued. Its deprecated OAuth version is no longer accepted by Picasa. The approach described below no longer works. You may find an alternative in Picasa Web Albums Data API.
I automated my own Picasa-related tasks with googlecl. I am using it on Linux, but there is a Windows version too. Yes, it's a command line tool; don't let that scare you off, you should be able to call the tool and capture its output from your favorite programming language (Java, C#, whatever).
Running googlecl for the very first time on your machine requires you to log on with your Google account, but after you have done that once, googlecl will remember your authentication and will not bother you ever again.
So your album is named InstantUpload. The following command will download a list of photos (date/time published + URL).
google picasa list --title="InstantUpload" --fields=published,url-direct --delimiter=" " > FullList.txt
AFAIK, there are no command line options to filter on a certain date range. You can filter on tags, but that would require you to assign a date-like tag to every photo in your album. Unless the download time of the list bothers you (and you are having some really good reason not to split up that single big album into multiple smaller ones), I would suggest to do the filtering yourself, immediately following the download.
I find CLI convenient, so I will demonstrate some simple shell scripting. But of course you can do the filtering in any programming language you like.
First, let's retrieve the UTC date of one year ago (formatted yyyy-mm-dd):
maxdate=$(date -u --date="1 year ago" +%F)
Use your favorite filter tool (grep, sed, awk, perl) to filter the list; keep only the lines with publish date < maxdate.
awk "\$1 < \"$maxdate\" { print \$2; }" FullList.txt > FilteredList.txt
(AWK is available for virtually every OS.)
You can even automate downloading the photos themselves. Make sure you cd to an appropriate data folder first.
while read url; do
wget "$url"
done < FilteredList.txt
Put it all together in a shell script file, and a single call is enough to do all the hard work for you.
Related
This is my first time working with Jira and their API. My company wants me to fetch all "ASAPSD" issues, but I don't understand how to. The core problem in itself is that I do not understand exactly how Jira works, and how issues are "built" up.
The issue starts with "ASAPSD" followed by some random characters and numbers. For example "ASAPSD-334". How can I, with a GET request, get all issues that start with ASAPSD?
Basic information about Jira and projects
The first part (prefix) is the Project Key representing a project/collection where all similar issues are stored (in this case, ASAPSD may stand for ASAP Service Desk:-). There are certainly more projects in every Jira instance. Some other projects are intended to track different activities.
Searching for the project issues
You can search for any issues using the search function (available also via REST API).
First, log in to the Jira and try to search for the issues manually by yourself - in Issue Navigator (via Issues top menu bar). Here you'll find that you can search all issues via Basic (Project is ASAPSD) or Advanced search (project = ASAPSD). This advanced search is called JQL (Jira Query Language).
You can then use this JQL in your REST API search method:
https://docs.atlassian.com/software/jira/docs/api/REST/latest/#api/2/search-search
Example
GET https://jira.yourdomain.com/rest/api/2/search?jql=project%3DASAPSD
Notes
The output lists only limited number of issues (usually 50) - to get more issues, you need to increase the limit (maxResults param) or paginate (startAt param) over next results.
Using expand and fields params you can alter the output to get more/less information.
There two types of Jira instances - on-premise Server/Data Center and Cloud one. REST API and usage might slightly differ.
Alternatively, you can get CSV export. When you search for issues in the Issue Navigator, there's an option to export results to CSV. Save this URL and you can request it in your script via GET.
The documentation I can find for the Canonical Landscape API lets you do lots of things with scripts, but I can't find anything suggesting that you can get output. However, if you use the Canonical web interface, script output is available, so it's presumably exposed somehow...?
I just had this issue as well and since you're the first hit right now on google, I wanted to share the answer for everyone - if you run ExecuteScript on a landscape client and get back an ID of 123, and let's assume the job finished already - you want to then use that ID to ask the GetActivities API, with an input argument of "query" with value "parent-id:123". If there is a result there, you will find the script output you are looking for under the result_text field of the response. Good luck!!! It worked over here very well.
I'll get this out of the way first: I'm an amateur programmer (at best)...i have some knowledge of how APIs work, but little to no experience manipulating the podio API directly (ie I use zapier/globiflow a lot and don't write any php/ruby). I'm sure other people can figure this out via the API documentation, but I can't. So i'm really hoping someone can help clarify and give some more detailed instruction.
My Overall objective:
I frequently export podio files as xlsx from the podio front-end. This is used by my team and me to do regular data analysis tasks in excel. I would like to make this process easier by automating the function of getting an updated podio export into my excel. My plan is to do this via excel VBA. I understand from other searching that it is possible to send an HTTP request using VBA, so i want to make sure i understand what I need to send to the Podio API to get what I need. The method of how to write the HTTP request in excel VBA is outside the intentional scope of this question (though i'd accept any help on this!)
What I've tried so far:
I know that 'get items as xlsx' is part of the podio API: https://developers.podio.com/doc/items/get-items-as-xlsx-63233
However I cannot seem to get this to work in the sandbox environment on that page so that i can figure out a valid request url. I get this message: 'Invalid filtering key' ... because i have no idea how to fill in that field. The information on that page is not clear on this. Nor is it evident on the referenced 'views page'. There are no examples to follow!
I don't even want to do any filtering. I just want to get ALL items in the app. Or i can give it a pre-existing view_id, but that doens't seem to work either without a {key}
I realise this is probably dead simple. Please help a noob? :)
Unfortunately, the interactive API sandbox does not behave appropriately for this particular endpoint. For filtering, this API endpoint expects query string parameters where the field-value pairs consist of integer field IDs and the allowed values for each field. Filtering by fields is totally optional. It looks like this sandbox page isn't built for this kind of operation with dynamic query string field names; wherever you see the {key} field on that page is meant as a placeholder for whatever field IDs that you would use for filtering.
If you want to experiment with this endpoint, I would encourage you to try another dedicated HTTP client first. I was able to get this simple example working with the command-line program wget:
wget --header="Authorization:OAuth2 $MY_SECRET_TOKEN" \
--content-disposition \
"https://api.podio.com/item/app/16476850/xlsx/"
In this case, wget downloaded an Excel file containing all the items in my app without any filtering applied. The additional --content-disposition argument tells wget to save the output as a file with a name using the information in the server's Content-Disposition response header.
With a filter applied:
wget --header="Authorization:OAuth2 $MY_SECRET_TOKEN" \
--content-disposition \
"https://api.podio.com/item/app/16476850/xlsx/?130654431=galaxy"
In this case, the downloaded file filtered the results to items where field id 130654431 (which is a category field) contain the value galaxy.
I have tagged over 700 blog posts with tags containing hyphens, and these tags suddenly stopped working in 2011, because Tumblr decided (without any notice) to forbid hyphens in tags (I guess hyphens are blocked now, because spaces in tags (which are allowed) get changed to hyphens.). Unfortunately, Tumblr is not willing to globally rename all tags containg hyphens (although these tags are of no use anymore → 404).
Now I want to rename my tags myself.
I tried to do it with the "Mass Post Editor" (tumblr.com/mega-editor), but it's not possible to select posts by tag. I'd have to manually select post after post and look if a certain tag was used, and if so, delete it and add a new one instead. This would be a huge job (700 tagged posts, but more than 1000 in total).
So I thought that the Tumblr API might help me. I'm no programmer, but I'd be willing to dig into it, if I could get some help here as a starting point.
I think I need the following process:
select all posts that are tagged with x (= a tag containing hyphens)
tag all these posts with y (= a tag without hyphens)
delete the tag x on all these posts
I'd start this process for every affected tag manually.
I see that the method (or whatever you call it) /post knows the request parameter tag:
Limits the response to posts with the specified tag
(I guess I can only hope that this works for tags containing hyphens, too.)
After that I'd need a way to add and remove tags from that result set. /post/edit doesn't say anything about tags. Did I miss something? Isn't it possible to add/remove tags with the API?
Have you an idea how I could "easily" rename my tags?
Is it possible with the API? Could you give me a starting point, tip etc. how I could manage to do it?
I don't know if this might be helpful, but I noticed that the search function is still able to find posts "tagged" with tags that contain hyphens.
Example: let's say I have the tag foo-bar. It is linked with /tagged/foo-bar (→ 404). I can find the posts with /search/foo-bar (but this is of course not ideal because it might also find posts that contain (in the body text) words similar/equal to the tag name).
I tried to encode the hyphen (/tagged/foo%2Dbar), but no luck.
just for the record, because this is a popular google search: i've done it! you can use it at http://dev.goose.im/tags/.
i used a combo of PHP and jquery, basing my jquery off of a previous tumblr api script i wrote a year or two ago, and used this tumblr php oauth script for the authentication. if anyone wants me to put up the source code, i'd be happy to.
If you aren't a programmer, how much is your time is worth to you? As they say, time is money. Not only do you have to figure out how to use the API, but choose a language and learn to write in it. That's no small task. You could higher a freelancer for $50 for an hour worth of work.
To answer your question, yes it is possible to do this with the API. It mentions "These parameters are used for /post, /post/edit and /post/reblog methods." and tags is mentioned as a string of comma separated words.
What you want to do is get a listing of every single blog post using the /posts method. You'll want to look at the "Request" section to figure out the criteria to pass to this URL. You want it to be as general as possible to get a complete listing of all your posts.
After you get a listing of posts you'll want to iterate over it and modify the tags parameter provided in the response for each post. You'll want to use the id paramater along with /post/edit, which again takes tags as a string.
The simplest language you can use for this task is PHP. You'll want to look at the curl extension to make your requests. You'll want to read up on arrays as you'll be using them a lot. You'll also need to look at explode, implode, str_replace (for the dashes), and foreach for iterating over the result.
When you do this I would highly recommend you use break at the end of your foreach loop so it only affects one post at first. Testing it first will be important, as you don't want to accidentally erase your tags/posts. print and var_dump are good ways to help you debug the code. xdebug is a nice extension that allows you to step through the code line by line as it runs. Netbeans is an IDE that has good xdebug support.
There's also a nice page here to get you started with PHP. You'll need to install PHP on your machine. You don't need to install a web server - for this PHP-CLI (command line) sapi is good enough.
How does one write a script to download one's Google web history?
I know about
https://www.google.com/history/
https://www.google.com/history/lookup?hl=en&authuser=0&max=1326122791634447
feed:https://www.google.com/history/lookup?month=1&day=9&yr=2011&output=rss
but they fail when called programmatically rather than through a browser.
I wrote up a blog post on how to download your entire Google Web History using a script I put together.
It all works directly within your web browser on the client side (i.e. no data is transmitted to a third-party), and you can download it to a CSV file. You can view the source code here:
http://geeklad.com/tools/google-history/google-history.js
My blog post has a bookmarklet you can use to easily launch the script. It works by accessing the same feed, but performs the iteration of reading the entire history 1000 records at a time, converting it into a CSV string, and making the data downloadable at the touch of a button.
I ran it against my own history, and successfully downloaded over 130K records, which came out to around 30MB when exported to CSV.
EDIT: It seems that number of foks that have used my script have run into problems, likely due to some oddities in their history data. Unfortunately, since the script does everything within the browser, I cannot debug it when it encounters histories that break it. If you're a JavaScript developer, use my script, and it appears your history has caused it to break; please feel free to help me fix it and send me any updates to the code.
I tried GeekLad's system, unfortunately two breaking changes have occurred #1 URL has changed ( I modified and hosted my own copy which led to #2 type=rss arguments no longer works.
I only needed the timestamps... so began the best/worst hack I've written in a while.
Step 1 - https://stackoverflow.com/a/3177718/9908 - Using chrome disable ALL security protocols.
Step 2 - https://gist.github.com/devdave/22b578d562a0dc1a8303
Using contentscript.js and manifest.json, make a chrome extension, host ransack.js locally to whatever service you want ( PHP, Ruby, Python, etc ). Goto https://history.google.com/history/ after installing your contentscript extension in developer mode ( unpacked ). It will automatically inject ransack.js + jQuery into the dom, harvest the data, and then move on to the next "Later" link.
Every 60 seconds, Google will force you to re-login randomly so this is not a start and walk away process BUT it does work and if they up the obfustication ante, you can always resort to chaining Ajax calls and send the page back to the backend for post processing. At full tilt, my abomination script collected 1 page a second of data.
On moral grounds I will not help anyone modify this script to get search terms and results as this process is not sanctioned by Google ( though not blocked apparently ) and recommend it only to sufficiently motivated individuals to make it work for them. By my estimates it took me 3-4 hours to get all 9 years of data ( 90K records ) # 1 page every 900ms or faster.
While this thing is going, DO NOT browse the rest of the web because Chrome is running with no safeguards in place, most of them exist for a reason.
One can download her search logs directly from Google (In case downloading it using a script is not the primary purpose),
Steps:
1) Login and Go to https://history.google.com/history/
2) Just below your profile picture logo, towards the right side, you can find an icon for settings. See the second option called "Download". Click on that.
3) Then click on "Create Archive", then Google will mail you the log within minutes.
maybe before issuing a request to get the feed the script shuld add a User-Agent HTTP header of well known browser, for Google to decide that the request came from that browser.