Testing IP based geolocation [closed] - testing

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
We are implementing an IP based geolocation service, and we need to find some IP's from various markets (LA, NY etc) to fully test the service.
Does anybody know of a directory where we could find what IP ranges are used where?
EDIT: We have already implemented the system, it uses a 3rd party DB and a webservice. We just want some IP's from known markets to verify its working properly.
I'm going to see if I can get what I need from the free maxmind database.


Not sure if cost is a factor but there are a few open source databases knocking about. This one claims 99.3% accuracy on its free version with 99.8% for its paid version. They've also got a Free & Open Source City Database (76% accuracy at city level).
They're both available as CSV-based databases so you can easily take a known location and get an IP range for ISPs in the area.
The tougher part is getting access to a computer in that IP range.


Try looking for sites providing lists of anonymizers. They usually list the countries for the anonymizer sites. Then either use the IP provided or do a lookup on the anonymizer name.
Also try searching for lists of anonymous proxies.
We trawled the logs for our huge web site and built up a test collection.
Sorry I can't pass it on. )-:
cheers,
Rob


maybe this database will be useful for you:
http://www.hostip.info/dl/index.html
it's a collection of ip adresses with countries and cities.


Many open source projects have worldwide mirrors; you can find a country-indexed list of Debian mirrors and kernel.org mirrors. (Note that kernel.org specifically has many mirrors per country; there are eleven United States mirrors, which are located in different regions of the country and would give different information.)


You could try using an automation tool, such as AutoIT, to fire off a series of IP addresses at a whois database service such as arin or RIPE, and harvest the addressed responses, probably just varying the first two parts of the IP.


Use Tor with a strict exit node.
You'll need to use these options in your config:
ExitNodes server1, server2, server3
StrictExitNodes 1
You'll also need to identify exit nodes that work for you in the region that you want. I suggest using the Search Whois feature at ARIN to see it's location if the Tor country icon isn't good enough. It can be a bit of a pain to identify working Tor nodes in each region that you wish to test, but it's possible and free.

Related

Apache: IP addresses vs users

Suppose you wanna analyze your access log files in order to check users activities. One common way is to assume that a same IP address corresponds to a same user.
However, several internet providers use CGNAT. Which, briefly, allows multiple end users to use a common public IP address.
In that case, users behind a CGNAT and sharing the same public address might be confused with each other. Therefore, causing problems to calculate view counts and to ban disruptive traffic.
Question
Any alternative to mitigate that?
(Preferably using only Apache)

You could consider unique users are unique combinations of IP+user-agent. It would be a bit better but still wouldn't be able to differentiate users on the same IP and using the same browser, on the same platform.
Other than that, you'd need to use a server side scripting technology and track sessions. That would require cookies tho, which is not too much of a biggie. You can't track static assets using that method tho.

Set up authoritative DNS server [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 years ago.
Improve this question
I am trying to set up a hosting company. The hosting company is going to have a client with the domain widgets.de
The name of my company is hostingcompany.de. The name servers I am setting up are called ns1.hostingcompany.de and ns2.hostingcompany.de
In the zone file for widgets.de, I have
NS ns1.hostingcompany.de.
NS ns2.hostingcompany.de.
In the zone file for hostingcompany.de, I have
hostingcompany.de 300 IN NS ns-110.awsdns-13.com
hostingcompany.de 300 IN NS ns-1130.awsdns-15.com
ns1.hostingcompany.de. 300 IN A 34.65.125.52
ns2.hostingcompany.de. 300 IN A 52.43.124.76
Also, I created two more hosted zones per Amazon's guidance
124.43.52.in-addr.arpa.
NS ns-2035.awsdns-62.co.uk.
SOA ns-2035.awsdns-62.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400
34 PTR ns1.hostingcompany.de
and
76.124.43.in-addr.arpa.
NS ns-799.awsdns-35.net.
SOA ns-2435.awsdns-62.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400
52 PTR ns2.hostingcompany.de
However, this is not working. When I try to submit these zone files, RIPE rejects them saying that ns1.hostingcompany.de and ns2.hostingcompany.de do not exist as objects. I think I have to do something with PTR records, but I don't know what.

PTR records are usually necessary if you are running a DNS or SMTP server to provide some proof that you are legitimate. I found this article to be quite illuminating.
I think the answer to this question is found towards the bottom of the link in the question. You have to fill out a form and AWS will create the PTR record for you. Creating a hosted zone in Route 53 for the pointer record does not appear to have any effect. Nothing in the RFC prohibits the owner of the public IP address from allowing a customer to create a PTR record for that public IP address. Although AWS could allow customers to create PTR records for their Elastic IP addresses, they do not.
There are a lot of articles discussing how you need to create your own hosted zones for the PTR records, such as but not limited to Amazon's own article the question linked to. You can definitely do this for private IP addresses if you are running a DNS server for a private network. However, if you are running a publicly available DNS or SMTP server on a public IP address, more vetting is required.
In order to verify that the records are set up correctly, you have to get an answer to:
dig -x 34.65.125.52 (must answer ns1.hostingcompany.de)
Unless you do this, the TLD registrar will not accept your nameserver, and your SMTP mail will probably be rejected as spam.
In addition to the above, another problem was that these lines should also be included in the zone file for hostingcompany.de
hostingcompany.de. 300 IN NS ns1.hostingcompany.de.
hostingcompany.de. 300 IN NS ns2.hostingcompany.de.
It is still unclear to me why the top level domain requires that the domains own nameservers are listed as being nameservers for its own domain, but this does appear to be a requirement for some top-level domains. After correcting the above problems, everything works.
I spent a long time trying to track down the above problems, and it did not seem to be documented anywhere, so I hope this helps someone. I also found this RFC to be quite interesting and informative. It is always good to read stuff written by the authorities.

Adding a new URL (domain) to a page

A friend of mine has opened a local business and not knowing what to do choose a URL with the name of her company (which is a playword). The chosen name is quite bad for google ranking because not meaningful: not indicating the nature of business nor the location (city).
I would like her to buy two new domains:
businessname-business-type-city.com
businessnamebusinesstypecity.com
is that still ok with google? I was doing that some years ago and ranked first on the search.

You can get as many domain names as you want, and set up the DNS to point to the ip of playword. But don't think it would be worth it, and no guarantee it would generate more hits.. Google search take location in consideration, so you're probably best of branding playword, and generate buzz in other ways; social media, flyers, many Google ads, and sponsored posts on Facebook
And be sure to have good semantics on your page

If the domain name is the name of the business then it is meaningful. Value of keywords in domain names has been diminished in recent times. Market the business, not the domain name. (And google local business results will help)

Country based on IP Address - A new approach to Improving Accuracy?

For my web app, I was wondering how to improve the accuracy of the country lookup based on IP address. Has anyone tried using a number of the API services in parallel and taking a vote, i.e. if 4 out of 5 return 'GB' and one returns 'US' then go with 'GB'. Statistically this should improve accuracy, but that depends on the independence of the IP databases behind each API.
Would this be a feasible approach?

You can maintain database for mapping of IP address to country and check on each request to determine the country associated with IP address.
As IP address's are always assigned country-wise so you wont face any problems following this approach,make sure you update this database from time to time.

Travel/Hotel API's? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I am working on a large project involving creating a worldwide hotel database with rich data such as Addresses, Images, Descriptions, Policies, Coordinates, Facilities, Reviews, Local area descriptions and their amenities etc.
I am looking for a public hotel API where I can both retrieve the data and run (preferably RESTful) real-time queries for specific locations and available rooms, and where I can subsequently let my customers make bookings.
Can anyone share experience, knowledge and possible guidelines for such a project, which APIs are suitable and what are the best practices?

In my search for hotel APIs I have found only one API giving unrestricted open access to their hotel database and allowing you to book their hotels:
Expedia's EAN http://developer.ean.com/
You need to sign for their affiliate program, which is very easy.
You get immediate access to their hotel databases plus you can make availability/booking requests with several response options, including JSON, which is more convenient and lightweight than the (unfortunately) more widespread XML.
As you immediately access their API, you can start developing and testing, but still need their approval to launch the site, basically to make sure it provides the needed quality and security, which is reasonable.
They also offer "deep linking", i.e. you may customize your requests by adding parameters. Then if it sufficient for your purpose (for mine it is not), you don't even need to store their content on your server.
I have also signed for HotelsCombined program:
(link removed as this site doesn't seem to let me put more links)
However, they do not immediately allow you to use their API even for testing. From their answer:
"Apologies for the inconvenience caused, but it’s simply a business decision to limit access to our rich hotel content. Please kindly check back within the next 2-3 months, where we will be able to judge your traffic, and in turn judge your status on standard data feeds."
I have also signed for Booking.com affiliate program:
(link removed as this site doesn't seem to let me put more links)
Unfortunately, again, they limit access, from their answer:
"Please do note that, since there's a high amount of time and cost involved in the XML integration, we are only able to offer the XML integration to a small amount of partners with a high potential."
I did not explore Tripadvisor as they seem only to offer top 10 hotels and only as widgets, but most importantly for me, they wouldn't allow booking through them.
I've checked the hotelbase.org mentioned above, they have very extensive list but not as rich as by Expedia, also they don't seem to have images and don't allow booking either.

I've used the TripAdvisor API before and its suited me well. It returns, per destination, a list of top-rated hotels, along with options to retrieve reviews, photos, nearby restaurants and a couple other useful things.
http://www.tripadvisor.com/help/what_type_of_tripadvisor_content_is_available
From the API page (available API content) :
* Hotel, attraction and restaurant ratings and reviews
* Top 10 lists of hotels, attractions and restaurants in a destination
* Traveler photos of a destination
* Travelers' Choice award badges for hotels and destinations
To expand upon #nstehr's answer, you could also use Yahoo Pipes to facilitate a more granular local search. Go to pipes.yahoo.com and do a search for existing hotel pipes and you'll get the idea..

HotelsCombined has an easy-to-access and useful service to download the data feed files with hotels. Not exactly API, but something you can get, parse and use. Here is how you do it:
Go to http://www.hotelscombined.com/Affiliates.aspx
Register there (no company or bank data is needed)
Open “Data feeds” page
Choose “Standard data feed” -> “Single file” -> “CSV format” (you may get XML as well)
If you are interested in details, you may find the sample Python code to filter CSV file to get hotels for a specific city here:
http://mikhail.io/2012/05/17/api-to-get-the-list-of-hotels/
Update:
Unfortunately, HotelsCombined.com has introduced the new regulations: they've restricted the access to data feeds by default. To get the access, a partner must submit some information on why one needs the data. The HC team will review it and then (maybe) will grant access.

You could probably trying using Yahoo or Google's APIs. They are generic, but by specifying the right set of parameters, you could probably narrow down the results to just hotels. Check out Yahoo's Local Search API and Google's Local Search API

Check out api.hotelsbase.org - its a free xml hotel api
No images as of yet though

Try Tixik.com and their API there. They have a very different data that big players, really good coverage mostly in Europe and good API conditions.

After several days of searching found the EAN API - http://developer.ean.com/ - it is a very big one, but it provides really good information. Free demos, XML\JSON format. Looks good.