I am trying to create a Bigquery UDF function that uses Googles Geocoding service.
It seems we can import external libraries with the option parameter but I feel that I cant use the Geocoding service here.
Following my function approach:
CREATE OR REPLACE FUNCTION
functions.returnGeoCode(address STRING)
RETURNS Array<String>
LANGUAGE js AS """
var geocoder = new google.maps.Geocoder();
geocoder.geocode( { 'address': address}, function(results, status) {
if (status == google.maps.GeocoderStatus.OK) {
var latitude = results[0].geometry.location.lat();
var longitude = results[0].geometry.location.lng();
alert(latitude);
}
});
"""
which complains because it does not know google of course saying ReferenceError: google is not defined at UDF$1(STRING) line 2, columns 23-24 when I try to use the function.
My ultimate goal is to convert addresses that I have in a Bigquery dataset to lat/longs so I can then create a heatmap in a visualization tool.
Any tips for my approach or something totally different? I saw some suggestions to use some public Bigquery datasets (openstreetmaps suggestion) but I have addresses from Germany and it does not cover that well.
Also Bigquery does not support the conversion this way it seems.
Thank you in advance!
Since the feature you need is from Geocoding, what I could suggest is you can script out everything where you use BigQuery API (to execute queries) and Geocoding API (to perform Geocoding calculations) in a javascript I suppose. You can perform the calculation of Geocoder separately from the query, afterwards use the returned value from Geocoder on your query using BigQuery API.
You can include a self-contained external javascript library, but it would not work with Geocoder service - here the javascript library makes external HTTP calls, which is not allowed for javascript UDF.
I think the appropriate solution is Cloud DataFlow - you can include arbitrary code there, without security and performance restrictions of UDF, read data out of BigQuery table, and write the results back.
If you have a lot of data, and Geocoder service becomes expensive, I think OpenStreetMaps can help - try to resolve data using OSM tables, then resolve the remaining addresses using Geocoder service.
Related
I am developing a search engine with angular 2.
Therefore I use APIs from multiple platforms.
It works if I call the search function from every api service manually.
But is it possible to do the same foreach api service?
Every api service has the same function:
search (query: string): Observable<Array<SearchResult>> { ... }
In the UI I want to separate the results by tabs.
Therefore every api service has a title:
public title: string = "the title";
For storing the search results locally I have a class that is extended by every api service. This class has helper functions etc.
Depending on the behaviour you need you can use merge, concat or forkJoin to merge multiple streams into one.
The code would look pretty much the same.
For example using merge in order to merge 2 streams into one.
If you have a list of apis you need to call for the search. Your code would look like this.
let apis: string[] = [];
let observables = apis.map(api => search(api)); // get an array of observables
let merged = observables.reduce((previous, current) => previous.merge(current), new EmptyObservable()); // merge all obserbable in the list into one.
merged.subscribe(res => doSomething(res));
This article might be helpful.
With the impending deprecation of the Fusion Tables SQL API, I want to ensure my application continues to operate as expected.
I use the Google Visualization api to visualize and query the Fusion tables like this:
To visualize the map:
map = new google.maps.Map(document.getElementById("map_canvas"), myOptions);
layer = new google.maps.FusionTablesLayer({
query: {
select : 'Latitude',
from : table
}
});
layer.setMap(map);
To query the fusion tables:
var query = "SELECT * FROM " + tableid;
query = encodeURIComponent(query);
var gvizQuery = new google.visualization.Query(
'http://www.google.com/fusiontables/gvizdata?tq=' + query);
My question is, what URL endpoints do I have to change to ensure my application continues working?
I would really appreciate some guidance on this subject.
Only the Fusion Tables API changed, see the migration guide for details.
To visualize the map you use FusionTablesLayer, there you don't have a API endpoint, just make sure to use the encrypted table id.
Here you use the Google Visualization API, which did not change. But if you use the Google Visualization API to get data (rather than to visualize data), you should consider to use the Fusion Tables API instead.
I am a new to using Mongo DB and exploring the frameworks around for migrating from mysql to mongodb. So far from my findings I have been able to figure out SpringMongo as the best solution to my requirements.
The only problem is that instead of using a DSL based or abstract querying mechanism, I wished the framework allowed me to pass plain json string as arguments to the different methods exposed by the API(find, findOne) so that the query parameters can be written out to an external file (using a key to refer) and passed to the methods by reading and parsing at run time. But the framework should be capable of mapping the results to the domain objects.
Is there a way in spring-mongo to achieve this? Or is there any other frameworks on the same lines
You could use Spring Data to do that, just use the BasicQuery class instead of Query class. Your code will look like the following:
/* Any arbitrary string that could to parsed to DBObject */
Query q = new BasicQuery("{ filter : true }");
List<Entity> entities = this.template.find(q, Entity.class);
If you want more details:
http://static.springsource.org/spring-data/data-mongo/docs/current/reference/html/#mongo.query
http://static.springsource.org/spring-data/data-mongodb/docs/current/api/org/springframework/data/mongodb/core/query/BasicQuery.html
Well I got to find this one in the Spring data MongoOperations...
String jsonCommand = "{username: 'mickey'}";
MongoOperations mongoOps = //get mongooperations implemantation
mongoOps.executeCommand(jsonCommand)
It returns an instance of CommandResult that encapsulates the result.
Question
How do I return different results for the same resource?
Details
I have been searching for some time now about the proper way to build a RESTful API. Tons of great information out there. Now I am actually trying to apply this to my website and have run into a few snags. I found a few suggestions that said to base the resources on your database as a starting point, considering your database should be structured decently. Here is my scenario:
My Site:
Here is a little information about my website and the purpose of the API
We are creating a site that allows people to play games. The API is supposed to allow other developers to build their own games and use our backend to collect user information and store it.
Scenario 1:
We have a players database that stores all player data. A developer needs to select this data based on either a user_id (person who owns the player data) or a game_id (the game that collected the data).
Resource
http://site.com/api/players
Issue:
If the developer calls my resource using GET they will receive a list of players. Since there are multiple developers using this system they must specify some ID by which to select all the players. This is where I find a problem. I want the developer to be able to specify two kinds of ID's. They can select all players by user_id or by game_id.
How do you handle this?
Do I need two separate resources?
Lets say you have a controller name 'Players', then you'll have 2 methods:
function user_get(){
//get id from request and do something
}
function game_get(){
//get id from request and do something
}
now the url will look like: http://site.com/api/players/user/333, http://site.com/api/players/game/333
player is the controller.
user/game are the action
If you use phil sturgeon's framework, you'll do that but the url will look like:
http://site.com/api/players/user/id/333, http://site.com/api/players/game/id/333
and then you get the id using : $this->get('id');
You can limit the results by specifying querystring parameters, i.e:
http://site.com/api/players?id=123
http://site.com/api/players?name=Paolo
use phil's REST Server library: https://github.com/philsturgeon/codeigniter-restserver
I use this library in a product environment using oauth, and api key generation. You would create a api controller, and define methods for each of the requests you want. In my case i created an entirely seperate codeigniter instance and just wrote my models as i needed them.
You can also use this REST library to insert data, its all in his documentation..
Here is a video Phil threw together on the basics back in 2011..
http://philsturgeon.co.uk/blog/2011/03/video-set-up-a-rest-api-with-codeigniter
It should go noted, that RESTful URLs mean using plural/singular wording e.g; player = singular, players = all or more than one, games|game etc..
this will allow you to do things like this in your controller
//users method_get is the http req type.. you could use post, or put as well.
public function players_get(){
//query db for players, pass back data
}
Your API Request URL would be something like:
http://api.example.com/players/format/[csv|json|xml|html|php]
this would return a json object of all the users based on your query in your model.
OR
public function player_get($id = false, $game = false){
//if $game_id isset, search by game_id
//query db for a specific player, pass back data
}
Your API Request URL would be something like:
http://api.example.com/player/game/1/format/[csv|json|xml|html|php]
OR
public function playerGames_get($id){
//query db for a specific players games based on $userid
}
Your API Request URL would be something like:
http://api.example.com/playerGames/1/format/[csv|json|xml|html|php]
I have a list of addresses from a Database for which I'd like to put markers on a Yahoo Map. The addMarker() method on YMap takes a YGeoPoint, which requires a latitude and longitude. However, Yahoo Maps must know how to convert from addresses because drawZoomAndCenter(LocationType,ZoomLevel) can take an address. I could convert by using drawZoomAndCenter() then getCenterLatLon() but is there a better way, which doesn't require a draw?
You can ask the map object to do the geoCoding, and catch the callback:
<script type="text/javascript">
var map = new YMap(document.getElementById('map'));
map.drawZoomAndCenter("Algeria", 17);
map.geoCodeAddress("Cambridge, UK");
YEvent.Capture(map, EventsList.onEndGeoCode, function(geoCode) {
if (geoCode.success)
map.addOverlay(new YMarker(geoCode.GeoPoint));
});
</script>
One thing to beware of -- in this example the drawAndZoom call will itself make a geoCoding request, so you'll get the callback from that too. You might want to filter that out, or set the map's centre based on a GeoPoint.
If you're working with U.S. addresses, you can use geocoder.us, which has APIs.
Also, Google Maps Hacks has a hack, "Hack 62. Find the Latitude and Longitude of a Street Address", for that.