I have a ruby on rails 3.0 application which will need a database containing the name and address of all the listed places in a certain area indexed by google maps. I don't need to display the map provided by google maps itself, I just need to be able to get the names and addresses of all places in a certain area from google, store this on my server, and then match the address/name a user enters with the place in my database if it exists, or add it if it doesn't. I have some questions about this:
Are there any gems out there that would help with this? A quick google search brought up gems which show places graphically on the google map, but this isn't what I need.
Approximately how much space will I need to store the names and addresses of, say every place in a city indexed by google maps?
I'd appreciate any feedback on how to go about building the places database using the google maps database as a source and making sure it's quickly accessible.
I just implemented an application for Google Places . There were about 4 million records in the Google places Db so the approach i used it you just make a curl call for a google place and store it in Database ,so next time i serve the place from DB. Don't store all the records in Db as this will make your Db too heavy. also google places allow only 1000 queries per day from a single IP. you also need ranking of the places with Db eg Paris in Canada and Paris in NYC will come together when you will search for paris. Google doesnot provide data according to the ranking . there is a different mechanism for finding ranking of the places
Related
Probably I could not ask the question in a proper way. But Here is what I want to do.
I have folders in hierarchical manner. Folders within folders. (For example - I am working on a DL project. I have images of blades of a image. Every fan is in one separate room of a house. House has many rooms hence we have assign number to these fans. A Fan has room number, A Blade of fan has three sides. We have assign some characters to these sides of blade also. like this we have images of blade and folders is something like this.
House - Room - Blade side
Now I want to build a front end which has these tabs(House name, room number, blade side etc) on web page . And I want to use local system's or azure storage drive to store the huge image dataset. But I want to use SQL database to store the information (like sides of the blade, room number, fan number etc) in csv format. And use this SQL as backend database for the web page.
When a user enters just any one of these information in the tab on webpage. I want this to give image as output
How to do this?
Thank you very much in advance.
I need to find a way how to (SQL)-join my GA360 tables in BigQuery(BQ) with data within AdsDataHub(ADH).
I already know how to query tables from BQ within ADH:
SELECT *
FROM 'projectname.table_name'
But I cant find any resources on what matching key to use in the Join statement
SELECT
*
FROM
adh.*** AS adh_data
adh_data LEFT JOIN ???
ON ga360.??? = ???
I read through this https://developers.google.com/ads-data-hub/guides/join-your-data
But it's not really clear to me what to get/use from it and I couldn't find any information on this topic anywhere.
Thank you in advance!
AFAIK, ADH doesn't currently allow for querying across google analytics data sets (which would already be in ADHs "clean room" if they wanted you to be able to make such queries...)
Your best option might be to A: make sure that you're capturing 1st party IDs in your google analytics implementation and B: ensuring those IDs are also captured in your CRM platforms as they interact with your properties (assumption being your CRM can capture, along with that ID, any Google Analytics related data you may find useful, though it won't be log level I don't think...)
From there, with "onboarding" of sorts, you may be able to eventually drop your CRM data into ADH queryable tables which can be joined (per the link you shared, "join your data") and then well... you're at google's behest for the most part, but I think that's the path you're looking for...
PS: Google may have some solutions with guides that include some useful example queries regarding join keys across CM/DV/GoogleAds tables, and they may be high quality, but they may not be EXACTLY what you're looking for... It's entirely possible they are not publicly available though...
I am trying to pull all the store locations from several different websites, for example Sephora http://www.sephora.com/store-locations-events and Freshpet http://freshpet.com/storelocator/.
Every site constrains the radius for the search, so to find all locations, the only thing I have come up with is cycling through every zip code and then filtering out duplicates. Is there a way to use VBA to manipulate the search radius using VBA to just search the entire United States, instead of only up to 100 miles? That way, I could just do one search for each of these sites.
Thanks!
I don't think you can do it on those javascript sites. VBA works great with HTML, but not so great with javascript and flash. Also, the code isn't even exposed, so the search results must be generated dynamically from the server. I just did a search on that freshpet site and got the results then did a right-click and viewed the source code and nothing that I searched for was in the code.
I need to create a list of all DNS Queries required to display a large number of sites (ideally up to 1 000 000). The list needs to assign the queries to the page that required them.
Example: Visiting google.com required a DNS query for google.com, ssl.gstatic.com, apis.google.com and other sites. My List would read something along the lines of
google.com:google.com,ssl.gstatic.com,apis.google.com,...
(exact format not relevant here)
I currently have two ideas on how to do this:
Set up a DNS Server with logging, build a script that visits a given list of domains using my DNS Server as a resolver
Building a script that loads the source code of the site (think python's urllib2, for example), parsing all embedded content and constructing a list of queries that would be needed
Both ideas have problems though. Visiting 1 000 000 Domains with a space of 2 seconds between visits (to make it possible to assign queries to the visited site afterwards), taking about 1 second to load (which is pretty optimistic) would take over 34 days, probably longer. But to build a parser I would need a complete list of all possible forms of embedded content that would result in a DNS Query, and I would need to query some of the target URLs as well (think iframes), and some content would be impossible to check for further queries (think flash content which connects to other servers).
I'm kind of stuck here, and would appreciate some input on how to deal with this. It would be possible to shorten the List of URLs to maybe 100 000, but any less would dramatically reduce the use of the result.
For context: I need this list for my bachelor thesis dealing with a attack strategy on a proposed DNS privacy extension.
You can use PhantomJS to do this, as it provides an interface that will let you capture network requests and log them, something along the lines of this example.
You'd need to write some simple Javascript, but as it's Node, it should be fairly easy to run this asynchronously to gather the data you need within a reasonable time.
There is a tool that can do this and produce a graphic representation. It is part of dnssec-tools called DNSpktflow (DNS Packet Flow)
It may not do what you want exactly but it is open source so you can see how they do it.
So I am looking at creating a very small app for the town I live in, I would like to offer people the ability to log into a website and create a business listing that would have things like (name, address, phone, opening hrs, logo -image).
I have limited knowledge in sql and creating Databases but am able to with abit of reading and time.. However I would like to know the best approach for storing images.
I am planning on just uploading the image to the webspace and then having a URL link stored in the DB... But not really sure if this is the best option?
So my question first of all, is this the best way to go about it? secondly how could I achieve this?
You've got everything right.
Yes, go for the plain table in the database, contains all the required fields.
Upon inserting another company data, get the autoincremented id, extract logo's filename extension and move uploaded file into a file called "$id.$ext" and you're all set.
How to achieve it - just try some online manuals.
There are plenty on working with database, while handling files is better to learn from the man page http://php.net/manual/en/features.file-upload.post-method.php