Receive a WOEID by Lat, Long with Yahoos API - api

i searched a while but found nothing, thats simular to my problem.
i'm trying to use the YAHOO Weather API, for example: http://weather.yahooapis.com/forecastrss?w=4097
i don't know the WOEID in my case, but i got latitude and longitude points.
so my question is:
is there a way to get the WOEID of a place by using lat and long points?

This is now available through the recently released PlaceFinder API. Kudos to Yahoo! for providing yet another important piece of the Geo puzzle.

Yahoo! PlaceFinder API allows you to find a corresponding WOEID for a latitude/longitude pair. Consider this example web service method call:
http://where.yahooapis.com/geocode?location=37.42,-122.12&flags=J&gflags=R&appid=zHgnBS4m
You can play with request parameters according to your needs, see Yahoo! PlaceFinder API documentation for more.
And you should replace appid with your Yahoo! appid, you can create one here.
This request returns a response like that, which includes a lot of useful data along with the WOEID:
{
"ResultSet": {
"version": "1.0",
"Error": 0,
"ErrorMessage": "No error",
"Locale": "us_US",
"Quality": 99,
"Found": 1,
"Results": [
{
"quality": 99,
"latitude": "37.420000",
"longitude": "-122.120000",
"offsetlat": "37.420000",
"offsetlon": "-122.120000",
"radius": 500,
"name": "37.42,-122.12",
"line1": "3589 Bryant St",
"line2": "Palo Alto, CA 94306-4207",
"line3": "",
"line4": "United States",
"house": "3589",
"street": "Bryant St",
"xstreet": "",
"unittype": "",
"unit": "",
"postal": "94306-4207",
"neighborhood": "",
"city": "Palo Alto",
"county": "Santa Clara County",
"state": "California",
"country": "United States",
"countrycode": "US",
"statecode": "CA",
"countycode": "",
"hash": "",
"woeid": 12797284,
"woetype": 11,
"uzip": "94306"
}
]
}
}

This is not using Yahoo's API but I found this blog post:
http://geomojo.org/?p=38
Mentioning this service:
http://www.geomojo.org/cgi-bin/reversegeocoder.cgi?long=-117.699444&lat=35.4775
Perhaps you can use that? It solved my problem, I hope it helps in solving yours.

It is somewhat ridiculous that Yahoo doesn't provide a lookup method for WOEIDs via lat/lon--it's been on their todo list since 2008--but that's the state of things.
I would caution you against using the suggested workaround implemented at Geomojo. If it works for your data, great, but the Yahoo service that Geomojo relies on is unpredictable. Geomojo uses Yahoo's PlaceMaker, which extracts location information from unstructured text to get a WOEID. It does this by creating a microformat containing your lat/lon pair and submitting it to PlaceMaker. However, since PlaceMaker returns WOEIDs for zip codes there's a loss of resolution and you will sometimes not be able to identify even the town for submitted coordinates. I have a number of example points on the east coast of the U.S. where the PlaceMaker WOEIDs do not correspond to the submitted lat/lon pairs.
Strangely, as HD writes, only Flickr's API provides a simple way to lookup a WOEID from lat/lon. Flickr's findByLatLon method has great resolution. It will usually return a neighborhood (one level below town) for a pair of coordinates.

First get city name from lat/long using this code.
CLGeocoder *geocoder = [[CLGeocoder alloc] init] ;
[geocoder reverseGeocodeLocation:location
completionHandler:^(NSArray *placemarks, NSError *error) {
if (error){
NSLog(#"Geocode failed with error: %#", error);
return;
}
CLPlacemark *placemark = [placemarks objectAtIndex:0];
NSLog(#"cityname - %#",placemark.locality);
}];
Then use that cityname in below url
https://search.yahoo.com/sugg/gossip/gossip-gl-location/?appid=weather&output=sd1&p2=pt&command=YOURCITYNAME
Example - https://search.yahoo.com/sugg/gossip/gossip-gl-location/?appid=weather&output=sd1&p2=pt&command=sydney
This will return json and you find get WOEID in this...
{ "l" : { "gprid" : "eIL89mltSzSfgDWdP7uyBA" },
"q" : "sydney",
"r" : [ { "d" : "pt:iso=AU&woeid=1105779&lon=151.021&lat=-33.8563&s=New South Wales&c=Australia&sc=NSW&n=Sydney, Australia",
"k" : "Sydney"
} ]
}

Seems like you got it the wrong way around. This is the URL on weather.yahoo.com:
weather.yahoo.com/united-states/illinois/chicago-2379574/
The last bit is the WOEID for Chicago, i.e. 2379574
WOEIDs are described in the GeoPlanet docs:
http://developer.yahoo.com/geo/geoplanet/guide/concepts.html#woeids

You can use Flickr's reverse geocoding API through YQL.
Here is a link to the YQL with an example query to find the WOEID for a given lat/lon:
http://developer.yahoo.com/yql/console/#h=select%20place.woeid%20from%20flickr.places%20where%20lat%3D43%20and%20lon%3D-94
The above query can be called directly from your app with this URL (XML/JSON formats available):
http://query.yahooapis.com/v1/public/yql?q=select%20place.woeid%20from%20flickr.places%20where%20lat%3D43%20and%20lon%3D-94&format=xml

There is a topic about this issue at the YDN forums http://developer.yahoo.net/forum/index.php?showtopic=69
Looks like it's buried in the to-do list, from 2008 "The ability to map a set of longitude and latitude coordinates to a WOEID, from which information such as ZIP and State may be derived, has already been identified as a valuable feature and it is on our enhancement request list."
Other quotes;
"Flickr has a method: flickr.places.findByLatLon which returns a WOEID, but they truncate coordinates to three decimal places."
In this topic a Yahoo dev also suggests using the advice at http://geomojo.org/?p=38 as an interim solution.

Instead using lng & lat, you can use the current online IP Address to get the City Name then use Yahoo GeoPlanet web service to get the WOEID.
Follow this tutorial to get the detail: http://4rapiddev.com/php/get-woeid-of-a-city-name-from-ip-address-with-php/

The former Yahoo Weather API has became deprecated. The new Yahoo Weather API requires a query string to get weather.
Use the following query string to get weather data by latitude and longitude -
"https://query.yahooapis.com/v1/public/yql?q=select * from weather.forecast where woeid in (select woeid from geo.places(1) where text=\"(" + latitude + "," + longitude + ")\")&format=json"
Eg - https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text%3D%22(31.63%2C74.87)%22)&format=json

Yahoo api uses weather.com actually, so go to weather.com and search for your local weather. I'm in Chicago so I entered 'Chicago, IL' and here's the link in my browser bar showing my weather:
http://www.weather.com/weather/today/Chicago+IL+USIL0225?lswe=chicago,%20il&from=searchbox_localwx
In the link is the woeid - which is USIL0225
You can get yours the same way.

Related

Proper way to convert Data type of a field in MongoDB

Possible Replication of How to change the type of a field?
I am currently newly learning MongoDB and I am facing problem while converting Data type of field value to another data type.
Below is an example of my document
[
{
"Name of Restaurant": "Briyani Center",
"Address": " 336 & 338, Main Road",
"Location": "XYZQWE",
"PriceFor2": "500.0",
"Dining Rating": "4.3",
"Dining Rating Count": "1500",
},
{
"Name of Restaurant": "Veggie Conner",
"Address": " New 14, Old 11/3Q, Railway Station Road",
"Location": "ABCDEF",
"PriceFor2": "1000.0",
"Dining Rating": "4.4",
}]
Like above I have 12k documents. Notice the datatype of PriceFor2 is a string. I would like to convert the data type to Integer data type.
I have referred many amazing answers given in the above link. But when I try to run the query, I get .save() is not a function error. Please advice what is the problem.
Below is the code I used
db.chennaiData.find().forEach( function(x){ x.priceFor2= new NumberInt(x.priceFor2);
db.chennaiData.save(x);
db.chennaiData.save(x);});
This is the error I am getting..
TypeError: db.chennaiData.save is not a function
From MongoDB's save documentation:
Starting in MongoDB 4.2, the
db.collection.save()
method is deprecated. Use db.collection.insertOne() or db.collection.replaceOne() instead.
Likely you are having a MongoDB with version 4.2+, so the save function is no longer available. Consider migrate to the usage of insertOne and replaceOne as suggested.
For your specific scenario, it is actually preferred to do with a single update as mentioned in another SO answer. It only does one db call(while your approach fetches all documents in the collection to the application level) and performs n db call to save them back.
db.collection.update({},
[
{
$set: {
PriceFor2: {
$toDouble: "$PriceFor2"
}
}
}
],
{
multi: true
})
Mongo Playground

How to get new Search Engine results in the past 24h using a SERP API?

Assume I am in possession of a SERP API, which given a keyword, returns me the Google results of that keyword in JSON format (for example: https://serpapi.com/):
{
"organic_results": [
{
"position": 1,
"title": "Coffee - Wikipedia",
"link": "https://en.wikipedia.org/wiki/Coffee",
"displayed_link": "https://en.wikipedia.org › wiki › Coffee",
"snippet": "Coffee is a brewed drink prepared from roasted coffee beans, the seeds of berries from certain Coffea species. From the coffee fruit, the seeds are ...",
"sitelinks":{/*snip*/}
,
"rich_snippet":
{
"bottom":
{
"extensions":
[
"Region of origin: Horn of Africa and ‎South Ara...‎",
"Color: Black, dark brown, light brown, beige",
"Introduced: 15th century"
]
,
"detected_extensions":
{
"introduced_th_century": 15
}
}
}
,
"about_this_result":
{
"source":
{
"description": "Wikipedia is a free content, multilingual online encyclopedia written and maintained by a community of volunteers through a model of open collaboration, using a wiki-based editing system. Individual contributors, also called editors, are known as Wikipedians.",
"source_info_link": "https://en.wikipedia.org/wiki/Wikipedia",
"security": "secure",
"icon": "https://serpapi.com/searches/6165916694c6c7025deef5ab/images/ed8bda76b255c4dc4634911fb134de53068293b1c92f91967eef45285098b61516f2cf8b6f353fb18774013a1039b1fb.png"
}
,
"keywords":
[
"coffee"
]
,
"languages":
[
"English"
]
,
"regions":
[
"the United States"
]
}
,
"cached_page_link": "https://webcache.googleusercontent.com/search?q=cache:U6oJMnF-eeUJ:https://en.wikipedia.org/wiki/Coffee+&cd=4&hl=en&ct=clnk&gl=us",
"related_pages_link": "https://www.google.com/search?q=related:https://en.wikipedia.org/wiki/Coffee+Coffee"
},
/* Results 2,3,4... */
]}
What is a good way to get new results from the past 24h? I added the &tbs=qdr:d query parameter, which only shows the results from the past day. That's a good first step.
The 2nd step is to filter out only relevant results. When there are no relevant results, Google shows this message box:
What is their algorithm to show this box?
Idea 1: "grep -i {exact_keywords}"
For example, if I search a keyword like "Alexander Pope", the 24h Google query might return results about the pope, written by a guy called Alexander. That's not super relevant. My naive idea is to grep (case insensitive) the exact keyword "Alexander Pope".
But that might leave out some good results.
Any other ideas?

Mapping Apple podcast episode id to rss feed element

I'm trying to map Apple podcast's episode id to that specific podcast entry in RSS feed. Say I have the episode with the following link https://podcasts.apple.com/us/podcast/the-numberphile-podcast/id1441474794?i=1000475383420 so the podcast_id=1441474794 and episode_id=1000475383420. Now I'm able to get the RSS feed with podcast id through this code:
from urllib.request import urlopen
import json
import xmltodict
podcast_id = "1441474794"
ITUNES_URL = 'https://itunes.apple.com/lookup?id='
with urlopen(ITUNES_URL + podcast_id) as response:
res = json.load(response)
feedUrl = res['results'][0]['feedUrl']
print(feedUrl)
with urlopen(feedUrl) as response:
res = xmltodict.parse(response)
with open('res.json', "w") as f:
f.write(json.dumps(res))
This gives me a JSON with some general info about the podcast and an array with all the episodes. For a specific episode the result looks like this:
"item": [
{
"title": "The Parker Quiz - with Matt Parker",
"dc:creator": "Brady Haran",
"pubDate": "Thu, 21 May 2020 16:59:08 +0000",
"link": "https://www.numberphile.com/podcast/matt-parker-quiz",
"guid": {
"#isPermaLink": "false",
"#text": "5b2cf993266c07b1ca7a812f:5bd2f1a04785d353e1b39d76:5ec683354f70a700f9f04555"
},
"description": "some description here...",
"itunes:author": "Numberphile Podcast",
"itunes:subtitle": "Matt Parker takes a quiz prepared by Brady. The YouTube version of this quiz contains a few visuals at https://youtu.be/hMwQwppzrys",
"itunes:explicit": "no",
"itunes:duration": "00:55:34",
"itunes:image": {
"#href": "https://images.squarespace-cdn.com/content/5b2cf993266c07b1ca7a812f/1541821254439-PW3116VHYDC1Y3V7GI0A/podcast_square2_2000x2000.jpg?format=1500w&content-type=image%2Fjpeg"
},
"itunes:title": "The Parker Quiz - with Matt Parker",
"enclosure": {
"#url": "https://traffic.libsyn.com/secure/numberphile/numberphile_parker_quiz.mp3",
"#type": "audio/mpeg"
},
"media:content": {
"#url": "https://traffic.libsyn.com/secure/numberphile/numberphile_parker_quiz.mp3",
"#type": "audio/mpeg",
"#isDefault": "true",
"#medium": "audio",
"media:title": {
"#type": "plain",
"#text": "The Parker Quiz - with Matt Parker"
}
}
},
...]
The episode_id=1000475383420 doesn't appear anywhere in the RSS feed response so there is no way to find which episode corresponds to this id. Is there a clean way to find the episode by id? For example an Apple api call with episode id which will give me info about the episode and then I can match the info with RSS feed entry.
The element/tag that is supposed to uniquely identify an episode in a podcast RSS feed is:
<guid>
Here is some related info from the Apple Podcasts Connect Guide to RSS that might be helpful.
If you can get a hold of the <guid> then you can access the episode from the feed.
A less reliable option would be to try the <link> tag for the episode. On that URL that you provided, there is a link down toward the end of the page that is named 'Episode Website'
That may also get you a unique key to the episode in the RSS feed. But it may not work as you would expect in all cases. i.e. say the creator/publisher of the podcast RSS simply just put the same URL in each episode instead of a unique URL per episode.
Yeah the second response is a general-purpose podcast rss feed, independent of Apple or other sources. I'd not expect it ever to have Apple / podcast player-specific results.
Best I've been able to do is do a title match based on json-ld metadata on the podcsat episode html page. json-ld data is semantic data (vs presentation) so much less likely to change. I use the extruct library for some semblance of hope of extracting meaningful metadata and jsonpath_rw for parsing json text (amazing library)
import extruct
from jsonpath_rw import parse
metadata = extruct.extract(itunes_podcast_episode_html, uniform=True)
title_pattern = "[json-ld][*]['name']"
expr = parse(title_pattern)
title = [match.value for match in expr.find(metadata)][0]
print(f"itunes podcast episode name = '{title}'")

amadeus api where to get a list of hotel chain codes

In Amadeus API, in hotel search, in the result there is "chainCode", is there a list in csv of all chain codes ?
https://developers.amadeus.com/self-service/category/hotel/api-doc/hotel-search/api-reference
I'm referring to "chainCode" in this result example
{
"data": [
{
"type": "hotel-offers",
"hotel": {
"type": "hotel",
"hotelId": "XKPARC12",
"chainCode": "XK",
"dupeId": "501132260",
"name": "Holiday Inn Paris-notre Dame",
How to get the hotel chain name from "XK" in this example ?
sorry we totally missed this question. Since you cannot retrieve it via API, please take a look to the data-collection repository, which contains a list of Hotel chain codes.

Get data Amadeus API Flight low fare search

I need parameters to get data that contains stops array. I tried about 100 different combinations, and i didn't get any response that returns stops array in results.
If anyone knows how to accomplish this, please provide your answer.
Thanks.
Having stops is not that common and it usually depends on the distance between origin and destination. For example, having London as origin and Sydney as destination:
https://test.api.amadeus.com/v1/shopping/flight-offers?origin=LON&destination=SYD&departureDate=2019-08-01&nonStop=false&returnDate=2019-08-28
You can check in the response that most of the segments contain stops:
"stops": [
{
"iataCode": "HKG",
"duration": "0DT1H0M",
"arrivalAt": "2019-08-28T12:00:00+08:00",
"departureAt": "2019-08-28T13:00:00+08:00"
},
{
"iataCode": "DOH",
"duration": "0DT1H0M",
"arrivalAt": "2019-08-28T14:00:00+03:00",
"departureAt": "2019-08-28T15:00:00+03:00"
},
{
"iataCode": "BAH",
"duration": "0DT1H0M",
"arrivalAt": "2019-08-28T16:00:00+03:00",
"departureAt": "2019-08-28T17:00:00+03:00"
}
]
Where stop means that an aircraft lands for refueling, for instance, but passengers don't necessary get out of the plane.