I would like to report the number of players in each of my game server nodes to a central authority that will be used to load balance connections to those nodes. This is a real time multiplayer web game. This is a game design question for a game that could go in any load balance direction and at this point any solution is possible. At a high level if you were designing this would it be better to open an outbound tcp connection to the authority and report game stats in the game loop OR would it be better to wait for the authority to ask the game server for stats over an incoming tcp connection? Which solution would be better if you had 1000 nodes?
If this were outbound from game server:
If the game server reports that this is my ip address, my port and my number of players the authority could catalog that data and update the number of players if the same ip/port combo exists. In order to prevent security issues with rogue agents reporting themselves as game servers you would then need to try and secure the connection with both encryption and some type of authorization mechanism.
If this were inbound to the game server
If the authority already knows the address of each node it needs to connect too then you may only need to worry about man in the middle servers pretending to be the authority. Given the data is only game stats though and nothing sensitive, encrypting it shouldn't matter in the grand scheme for a connection inbound to a game server. I could have for example a config file or database table with all the known game servers to connect too. A mitm would get game stats or maybe just block the connection. To me this seems like the better option to just connect out to your game server unencrypted and get the stats. The downside is anyone else could connect to your game server as well to ask for game stats. Then in your game server you'd have to figure out either which connection is the real authority or just broadcast the game stats to anyone asking for game stats. The game server from a security standpoint though could get overloaded with to many connections wanting to get game stats and you don't have a good way to know which authority is the one that matters for your load balancing vs adhoc agents. I FEEL LIKE THIS IS THE WAY TO GO.
I agree that inbound to the game servers are the way to go, for the reasons you stated.
Here are a few other considerations.
I think practically that it is a lot harder for the servers to connect to users' hosts because of ip changes and firewalls at the user-local level.
You should always encrypt your customer's over the wire. And speaking of MITM, you should have some way of preventing one user from spoofing another user. No-one wants their game play taken over by a hacker.
You could have, as part of your client-server protocol, a way for the server to dictate a minimum wait time before the client sends their next update.
You could have the client updates put into a queue (rabbit, kafka, etc.) and have the servers update at their own capabilities. Queues are a lot more forgiving of elastic traffic, if you can handle them.
I am currently just using a STUN server and am wondering whether TURN is necessary for an MVP. The number of users accessing the website from a workplace with super secure firewalls should be near-zero.
Let's say 4 people are testing WebRTC connection reliability. Sometimes they all successfully connect and can see/hear one another, but other times they cannot see/hear someone and refresh the page to try again.
Does the fact that they can sometimes all see/hear each other rule out whether a TURN server would make a difference?
In other words, is it possible for a STUN server to identify my IP so I can connect one second, but fail if I try again a few seconds later? Or is it just network-based, so if I a STUN doesn't work for me on my network now, it 100% will not work in 5 minutes either?
If the latter (and a TURN is either always or never needed for a given network), I guess that tells me the problem in my code is elsewhere...
I'm looking to write a toy application for my own personal use (and possibly to share with friends) for peer-to-peer shared status on a local network. For instance, let's say I wanted to implement it for the name of the current building you're in (let's pretend the network topology is weird, and multiple buildings occupy the same LAN). The idea is if you run the application, you can set what building you're in, and you can see the buildings of every other user running the application on the local network.
The question is, what's the best transport/network layer technology to use to implement this?
My initial inclination was to use UDP Multicast, but the more research I do about it, the more I'm scared off by it: while the technology is great and seems easy to use, if the application is not tailored for a particular site deployment, it also seems most likely to get you a visit from an angry network admin.
I'm wondering, therefore, since this is a relatively low bandwidth application — probably max one update every 4–5 minutes or so from each client, with likely no more than 25–50 clients — whether it might be "cheaper" in many ways to use another strategy:
Multicast: find a way to pick a well-known multicast address from 239.255/16 and have interested applications join the group when they start up.
Broadcast: send out a single UDP Broadcast message every time someone's status changes (and one "refresh" broadcast when the app launches, after which every client replies directly to the requesting user with their current status).
Unicast: send a UDP Broadcast at application start to announce interest, and when a client's status changes, it sends a UDP packet directly to every client who has announced. This results in the highest traffic, but might be less likely to annoy other systems with needless broadcast packets. It also introduces potential complications when apps crash (in terms of generating unnecessary traffic).
Multicast is most certainly the best technology for the job, but I'm wondering if the associated hassles are worth avoiding since this is just a "toy application," not a business-critical service intended for professional network admin deployment and configuration.
I am working on an app that monitors network usage. However I noticed many ways to do this does not allow exclusion of local traffic (say, Time Machine).
I am looking for a way to exclude local traffic, and only monitors usage that goes directly to/from the internet.
Update: Thank you for your replies, now I know how to find if the traffic is local, but I still don't know how I can calculate total in/out bytes (sorry if I didn't elaborate earlier). I have no way of knowing how many bytes are sent/received locally (or to the internet) in a certain period of time, or since the OS starts. This problem is further complicated by the fact processes are launched or killed when the OS is running.
The answer to the question How to get network adapter stats in linux/Mac OSX? gives an interesting way of summing up total usage but it doesn't help because the usage it sums up are interface statistics.
Update 2: I've posted my final solution to this. Please scroll down a bit to see.
you need to read the source for ifconfig(8), which describes how to get the status of every attached network interface.
pay particular attention to in_status(), which gets the inet address and netmask of an interface.
when the source or destination address in the traffic has the same host as a local interface
int is_local =
(src && netmask) == (ifaddr && netmask)
|| (dst && netmask) == (ifaddr && netmask)
then you can be sure that it is local
http://www.opensource.apple.com/source/network_cmds/network_cmds-307/ifconfig.tproj/ifconfig.c
Answering you comment about which interfaces carry local traffic is actually complicated, because it depends on what you mean by local traffic.
What “Local” Means
The easiest meaning of "local traffic" is traffic that does not leave the machine its generated on (two programs on the same machine talking to each other, for example). This traffic all goes over lo. This is one thing that people mean when they say local (and what I was thinking of when I answered).
The next easiest meaning would be "IP traffic destined to machines on the same subnet". That'd be traffic that has a destination address inside the local subnet. The easiest way to count this is going to be either the routing table (if Mac OS X counts traffic stats per route, the routes on the various gateways will give you non-local traffic) or with a firewall rule. This probably isn't want anyone means when they say "local traffic".
Another meaning would be "IP traffic destined to machines in this (physical) location". E.g., at my office we have several subnets in use, with routers between them, but traffic from one subnet to the other is still clearly local. You need network knowledge to distinguish local from non-local traffic with this definition.
Another meaning would be "IP traffic destined to machines in my organization". This is a reasonable meaning depending on how your network is set up (e.g., maybe you have fast fiber between your locations, but your Internet connections are much slower, or charged per-GB). Requires in-depth knowledge of the network to figure if a destination is going to be local or not—and, with things like VPNs, that may vary over time.
Finally, "Internet traffic" isn't the opposite of any of those. Sometimes, for example, what appears to be a local machine on your Ethernet segment is actually over a VPN, over the Internet (this isn't crazy, it's very useful for when remote users need to use various Windows services). Traffic inside your organization can easily travel over an Internet VPN.
Cheating in Simple Networks
If the network is very simple, with there being only one internal subnet, only one router, and all traffic not to that internal subnet being Internet traffic, you can cheat and solve this. This probably applies to the vast majority of home networks, and many small business ones as well.
Using firewall rules
In a simple network setup, you can probably make some assumptions, and get a close enough answer by counting traffic as non-local if:
the destination MAC address is the default gateway's MAC address; and
the destination IP address is not the default gateway's IP address
alternatively:
the destination IP address is not within the subnet of the network interface the default route goes out
You can probably create a firewall rule to count either of those. At least with Linux iptables you can, and I'm pretty sure BSD pf, and probably Mac OS X.
Alternate Approach: SNMP
Finally, if you can't use a firewall rule (as that'd require root), you could hope that the default gateway responds to SNMP community public, explore all its interfaces, and find the one with a off-subnet IP address, and then assume that is the Internet link. Then you can ask the router for traffic counts on that interface.
Of course, you'll find that many SOHO routers don't support SNMP, and those that do probably don't have it turned on.
The best way is to find the 'external' ip address through the eth0, eth1, or whatever adapter with a system call to ifconfig. Then pull logs for whatever system (messages, syslog, whatever) and write a filter for that external ip address. To make it nicer and more portable, write a regex that will filter for publicly routable IPs only and just filter messages log for that 'external' ip address.
I think, an approximate solution: getifaddrs can be used to get statistics on network usage.
It can get separate statistics for Wi-Fi and WWAN interfaces.
You might find more information from :
http://www.gsp.com/cgi-bin/man.cgi?section=3&topic=getifaddrs
It depends on how you define "local", but a common definition would be to look at the network mask.
For example, if your IP (ie the IP of the interface you monitor is
10.33.52.123
netmask 255.255.255.0
that would mean every IP-packet with both source-IP and destination-IP 10.33.52.xx is local.
I don't know cocoa or objective-c, but you can probably use some of these functions helping you extract the network from an IP-address: http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/inet_network.3.html
Don't know how to implement it in objective-c but the idea is that you get the address of the network you are in (you can figure this out from network class(A,B,C) based from your local ip or from bits in netmask if it's not standard), then just check the outgoing connection's address. If the destination is not in your local network, calculate traffic; if it's in, just do nothing.
There are three ranges of non-routable IP addresses, and they are commonly used as the address ranges for NAT services. Any address that is not in one of the non-routable address ranges is an external address.
Of course if you are not behind a NAT router, the task is harder (and technically all the addresses short of 127.0.0.1 are external at this point).
The non-routable IP ranges are:
10.0.0.0 - 10.255.255.255
172.16.0.0 - 172.31.255.255
192.168.0.0 - 192.168.255.255
The final working solution I have is to use libpcap to achieve this. Of course there are some downsides, which includes it requires elevated privileges and must capture all filtered packets to calculate statistics, but at least it works perfectly well.
Many documentations and tutorials on libpcap is fairly thorough and clear, I suggest every one interested in this solution to look at those with relatively little google-fu effort.
Also it may interest a few that my filter for internet traffic is simply the following -
- (NSString *)_internetFilterStringForInterface:(AKNetworkInterface *)interface
inOrOut:(BOOL)inYesOutNo
{
if (![interface net] || ![interface mask] || IsEmpty([interface addresses]))
{
return nil;
}
NSString *hostType = inYesOutNo ? #"dst" : #"src";
NSString *host = nil;
for (NSString *hostComponent in [interface addresses])
{
if (IsEmpty(hostComponent)) continue;
if (!host)
host = [NSString stringWithFormat:#"(%# host %#", hostType, hostComponent];
else
host = [host stringByAppendingFormat:#" or %# host %#", hostType, hostComponent];
}
host = [host stringByAppendingString:#")"];
NSString *net = [interface netString];
net = [net stringByReplacingOccurrencesOfString:#".0" withString:#""];
NSString *filter = [NSString stringWithFormat:
#"ip and (not %# net %#) and %#",
inYesOutNo ? #"src" : #"dst",
net, host];
return filter;
}
The filter is designed with some of the answers about what counts as 'local traffic', I know it does not encompass some edge cases such as double NAT configurations, etc., but I would like to see suggestions about this.
I know net = [net stringByReplacingOccurrencesOfString:#".0" withString:#""]; is just a quick hack which could easily fail under some peculiar circumstances but hey no one is complaining, at least not yet.
I'm trying to find my external IP address, but I get local ones only, behind the NAT.
NSArray *addresses = [[NSHost currentHost] addresses];
Is there a way to print out the public address? Using NSHost is it a good idea?
There may not always be a reliable way to get at your public IP, but the DNSService API in OS X will use UPnP and/or the NAT port mapping protocol to get the public IP (amongst other things). The code illustrating how to use the C API would be a bit large (50-60 lines) to show here, but there's some Apple sample code which implements a nice ObjC wrapper around the functionality, and even offers a pair of functions to directly return UInt32 and NSString representations of the public IP address.
The relevant code itself is located here, but you're probably best off just downloading the zip file and including PortMapper.h and PortMapper.m in your project and using them directly. Then all you'd need to do is:
NSString * publicAddressString = [PortMapper findPublicAddress];
If you ever get to look at a network topology chart for a major organization, do so. It's enlightening. The whole point of NAT, firewalls and all that other "black magic" is to allow the network to manage addresses (including protecting you) without your knowledge.
There are only three ways in which I've been able to reliably (and reliable is a relative concept here) get the external-facing IP address of a server.
The first is to as the network gods themselves (and make sure you refer to them as gods when asking, this will assist you in getting the information). Sometimes (not always), it's a simple mapping of the top bytes of your IP address whilst retaining the low-order bytes. Sometimes it's more complex, but still follows rules that you can use. Just keep in mind these rules can change at any time.
The second is to have a box outside of your network which you can query and it, in turn, can let you know your IP address.
The third is to specifically attach to an outside DNS server (not your corporate one) to retrieve the information.
Of course, you should question the need to know your external IP address. The whole point of DNS is to avoid having to worry too much about IP addresses and just refer to machines by the domain names.
There is no reliable way to get your public IP. Depending on your network topology, you may not be able to depend on the outgoing address for a number of reasons: IP address pools, dynamic routes, multiple layers of NAT, proxy servers, etc.