This probably has a very obvious answer, but what is the common way to get the router/gateway IP address of the packet I just received in pcap.net?
I know how to get the IP address source:
packet.Ethernet.IpV4.Source.ToString()
I tried looking through the object browser, but I didn't find a property that seemed to match. Any way I could find it?
It's more of networking question, than programming one. A short answer would be - You can't.
The source IP address will always (unless strangely translated by the gateway) belong to the endpoint You wanted to connect with. This way Your application will get the response to any request You send. Unless You're using NAT the router does not alter the packet in any way so it's transparent from a connectivity point of view. The source address of the packet You just got would almost always contain the IP address of the server You connected to. That's the way Ethernet works.
A poor man's solution would be to use traceroute to find out which way the packets go and therefore get the address of the router, which generally would be the first hop along the way. From a programmer's perspective this would mean sending out several packets to the destination You got the packet from, each time incrementing the packet's TTL (starting from 1) and looking at the ICMP responses. This however could mislead You if some sort of load balancing is being done.
Maybe if You clarified what You would like to achieve I could point You in a better direction.
My NIC has multiple virtual interfaces corresponding to 10 unique IPs from my network. I have an application which I would like to run 10 instances of; each instance utilizing a unique IP address. Assuming each of the 10 instances of the app knows which IP number it should be associated with:
How can I instruct my NSMutableURLRequest's to send the requests off of the specified IPs?
Is this something that can should be set at the NSURLConnection level?
I don't believe you can tell the NSURLRequest what IP to use, or the NSURLConnection. There certainly doesn't seem to be anything in the headers. You might have to go down to the C API's and try it there, though I'm not sure if you could do it on that level, either.
Edit:
https://developer.apple.com/library/mac/documentation/Networking/Reference/SCNetworkConfiguration/Reference/reference.html#//apple_ref/c/func/SCNetworkSetSetCurrent and the related functions appear to be what you need.
How to select the network interface to use in my Cocoa app
I am working with pcap in an OS X application to understand packet analysis.
I am working with a app https://github.com/jpiccari/MacAlyzer
but I am getting only raw data but I want to differentiate every domain request into separate and clear way to read request and response value. Please guide me the way to how to develop an application with pcap.
I have tried some code but they translate data into hex format. How do I convert that data into meaningful request and response objects like Charles and Fiddler show?
MacAlyzer wasn't developed for your needs. I know because I'm the author. As already stated, Charles and Fiddler are web proxies and work entirely different (and serve different purposes).
Diving a bit deeper into your question, communication between client and server happens IP-to-IP and not domain-to-domain. Domain information is not contained in the packets at the either the IP or TCP level. Instead computers request domain-to-IP lookup information which is then stored and communication is carried out using the client and server IP addresses.
MacAlyzer, and really libpcap, don't have sophisticated packet dissection (like say Wireshark) and cannot display packet information as verbosely as other programs. Before I lost interest in the project I was planning a library that would allow much richer packet dissection and analysis, but free time became very limited.
As for adding domain information to MacAlyzer, I'll explain at a high-level since it seems you know what you're doing. To include domain information instead of IP address in the Source and Destination columns you could edit function ip_host_string() in ip.m. This function controls how the client and server addresses are displayed. Modifying it to lookup the hostname from IP address and returning the resulting string would cause the domains to be displayed instead of IP addresses.
If you come up with some nice updates, consider submitting a pull request.
Here is the food for thoughts:
http://www.binarytides.com/packet-sniffer-code-c-linux/
Anyway, you will need to use C. Therefore, check the codes of the includes, for example:
http://www.eg.bucknell.edu/~cs363/2014-spring/code/tcp.h
Here is the documentation of "pcap":
http://www-01.ibm.com/support/knowledgecenter/#!/ssw_aix_71/com.ibm.aix.basetrf1/pcap_close.htm
I am working on an app that monitors network usage. However I noticed many ways to do this does not allow exclusion of local traffic (say, Time Machine).
I am looking for a way to exclude local traffic, and only monitors usage that goes directly to/from the internet.
Update: Thank you for your replies, now I know how to find if the traffic is local, but I still don't know how I can calculate total in/out bytes (sorry if I didn't elaborate earlier). I have no way of knowing how many bytes are sent/received locally (or to the internet) in a certain period of time, or since the OS starts. This problem is further complicated by the fact processes are launched or killed when the OS is running.
The answer to the question How to get network adapter stats in linux/Mac OSX? gives an interesting way of summing up total usage but it doesn't help because the usage it sums up are interface statistics.
Update 2: I've posted my final solution to this. Please scroll down a bit to see.
you need to read the source for ifconfig(8), which describes how to get the status of every attached network interface.
pay particular attention to in_status(), which gets the inet address and netmask of an interface.
when the source or destination address in the traffic has the same host as a local interface
int is_local =
(src && netmask) == (ifaddr && netmask)
|| (dst && netmask) == (ifaddr && netmask)
then you can be sure that it is local
http://www.opensource.apple.com/source/network_cmds/network_cmds-307/ifconfig.tproj/ifconfig.c
Answering you comment about which interfaces carry local traffic is actually complicated, because it depends on what you mean by local traffic.
What “Local” Means
The easiest meaning of "local traffic" is traffic that does not leave the machine its generated on (two programs on the same machine talking to each other, for example). This traffic all goes over lo. This is one thing that people mean when they say local (and what I was thinking of when I answered).
The next easiest meaning would be "IP traffic destined to machines on the same subnet". That'd be traffic that has a destination address inside the local subnet. The easiest way to count this is going to be either the routing table (if Mac OS X counts traffic stats per route, the routes on the various gateways will give you non-local traffic) or with a firewall rule. This probably isn't want anyone means when they say "local traffic".
Another meaning would be "IP traffic destined to machines in this (physical) location". E.g., at my office we have several subnets in use, with routers between them, but traffic from one subnet to the other is still clearly local. You need network knowledge to distinguish local from non-local traffic with this definition.
Another meaning would be "IP traffic destined to machines in my organization". This is a reasonable meaning depending on how your network is set up (e.g., maybe you have fast fiber between your locations, but your Internet connections are much slower, or charged per-GB). Requires in-depth knowledge of the network to figure if a destination is going to be local or not—and, with things like VPNs, that may vary over time.
Finally, "Internet traffic" isn't the opposite of any of those. Sometimes, for example, what appears to be a local machine on your Ethernet segment is actually over a VPN, over the Internet (this isn't crazy, it's very useful for when remote users need to use various Windows services). Traffic inside your organization can easily travel over an Internet VPN.
Cheating in Simple Networks
If the network is very simple, with there being only one internal subnet, only one router, and all traffic not to that internal subnet being Internet traffic, you can cheat and solve this. This probably applies to the vast majority of home networks, and many small business ones as well.
Using firewall rules
In a simple network setup, you can probably make some assumptions, and get a close enough answer by counting traffic as non-local if:
the destination MAC address is the default gateway's MAC address; and
the destination IP address is not the default gateway's IP address
alternatively:
the destination IP address is not within the subnet of the network interface the default route goes out
You can probably create a firewall rule to count either of those. At least with Linux iptables you can, and I'm pretty sure BSD pf, and probably Mac OS X.
Alternate Approach: SNMP
Finally, if you can't use a firewall rule (as that'd require root), you could hope that the default gateway responds to SNMP community public, explore all its interfaces, and find the one with a off-subnet IP address, and then assume that is the Internet link. Then you can ask the router for traffic counts on that interface.
Of course, you'll find that many SOHO routers don't support SNMP, and those that do probably don't have it turned on.
The best way is to find the 'external' ip address through the eth0, eth1, or whatever adapter with a system call to ifconfig. Then pull logs for whatever system (messages, syslog, whatever) and write a filter for that external ip address. To make it nicer and more portable, write a regex that will filter for publicly routable IPs only and just filter messages log for that 'external' ip address.
I think, an approximate solution: getifaddrs can be used to get statistics on network usage.
It can get separate statistics for Wi-Fi and WWAN interfaces.
You might find more information from :
http://www.gsp.com/cgi-bin/man.cgi?section=3&topic=getifaddrs
It depends on how you define "local", but a common definition would be to look at the network mask.
For example, if your IP (ie the IP of the interface you monitor is
10.33.52.123
netmask 255.255.255.0
that would mean every IP-packet with both source-IP and destination-IP 10.33.52.xx is local.
I don't know cocoa or objective-c, but you can probably use some of these functions helping you extract the network from an IP-address: http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/inet_network.3.html
Don't know how to implement it in objective-c but the idea is that you get the address of the network you are in (you can figure this out from network class(A,B,C) based from your local ip or from bits in netmask if it's not standard), then just check the outgoing connection's address. If the destination is not in your local network, calculate traffic; if it's in, just do nothing.
There are three ranges of non-routable IP addresses, and they are commonly used as the address ranges for NAT services. Any address that is not in one of the non-routable address ranges is an external address.
Of course if you are not behind a NAT router, the task is harder (and technically all the addresses short of 127.0.0.1 are external at this point).
The non-routable IP ranges are:
10.0.0.0 - 10.255.255.255
172.16.0.0 - 172.31.255.255
192.168.0.0 - 192.168.255.255
The final working solution I have is to use libpcap to achieve this. Of course there are some downsides, which includes it requires elevated privileges and must capture all filtered packets to calculate statistics, but at least it works perfectly well.
Many documentations and tutorials on libpcap is fairly thorough and clear, I suggest every one interested in this solution to look at those with relatively little google-fu effort.
Also it may interest a few that my filter for internet traffic is simply the following -
- (NSString *)_internetFilterStringForInterface:(AKNetworkInterface *)interface
inOrOut:(BOOL)inYesOutNo
{
if (![interface net] || ![interface mask] || IsEmpty([interface addresses]))
{
return nil;
}
NSString *hostType = inYesOutNo ? #"dst" : #"src";
NSString *host = nil;
for (NSString *hostComponent in [interface addresses])
{
if (IsEmpty(hostComponent)) continue;
if (!host)
host = [NSString stringWithFormat:#"(%# host %#", hostType, hostComponent];
else
host = [host stringByAppendingFormat:#" or %# host %#", hostType, hostComponent];
}
host = [host stringByAppendingString:#")"];
NSString *net = [interface netString];
net = [net stringByReplacingOccurrencesOfString:#".0" withString:#""];
NSString *filter = [NSString stringWithFormat:
#"ip and (not %# net %#) and %#",
inYesOutNo ? #"src" : #"dst",
net, host];
return filter;
}
The filter is designed with some of the answers about what counts as 'local traffic', I know it does not encompass some edge cases such as double NAT configurations, etc., but I would like to see suggestions about this.
I know net = [net stringByReplacingOccurrencesOfString:#".0" withString:#""]; is just a quick hack which could easily fail under some peculiar circumstances but hey no one is complaining, at least not yet.
We're adding a second web server for redundancy and load sharing purposes. All connections are mandated to be SSL, and adding a dedicated appliance is not possible at this moment.
I'd like to use round robin DNS, where both servers answer to the same domain using different IPs (we have a wildcard SSL certificate, so that's OK). I can get the DNS to return in random/round robin order no problem.
Is this a bad setup when using SSL?
Our user pattern is consistent -- users will consistently be utilizing the web app for 8-10 hours. We want each page view to be as fast as possible, and my concern is users could constantly flip between the servers, potentially negating any SSL handshake caching/keep alive.
Thanks!
Firstly, SSL has the ability to resume an earlier session, so flipping between servers will cost you a few hundred ms per request (longer if several clients are accessing the site simultaneously, since this is CPU time we're talking about).
Whether the clients will actually flip depends, though - DNS "load balancing" is fiddly business:
if many of your users are using the same recursive nameservers, they'll get the same "first IP" hence no load balancing
if the DNS record has a high TTL (several hours), caching nameservers will store a particular permutation of IP addresses until they expire (good so long as your users aren't all using the same recursive nameservers)
if your users have multiple recursive nameservers configured, they may flip if each nameserver has a different "first IP" (bad)
if you have no mechanism for removing "bad" records, and a low TTL, then if one server goes down 50% of your clients will get the "bad" server and have to wait for a timeout before they can see your site
As you can see there are various tradeoffs depending on whether you're more concerned about redundancy/failover or load balancing; DNS isn't really the best tool here - you really need the servers to share an IP using either a reverse proxy, or something like Heartbeat (assuming you're Linux-based).
An aside: if both servers are answering to the same domain then you don't need a wildcard cert, although CAs often charge more if you intend to use a cert on more than one server.
TLDR: You will be fine. The SSL renegotiations shouldn't happen frequently enough to be noticeable by your end user.
Rant starts here:
Load distribution using DNS is a commonly misunderstood topic that leads into a lot of anecdotal evidence and straw-man arguments. I've been in too many of these meetings.
Here's how I usually settle these arguments:
"Wow yeah that sounds really exoteric [long dramatic pause] but it really can't be that bad since google uses it"
$host encrypted.google.com
encrypted.google.com is an alias for www3.l.google.com.
www3.l.google.com has address 74.125.224.195
www3.l.google.com has address 74.125.224.202
www3.l.google.com has address 74.125.224.193
www3.l.google.com has address 74.125.224.197
www3.l.google.com has address 74.125.224.207
www3.l.google.com has address 74.125.224.206
www3.l.google.com has address 74.125.224.203
www3.l.google.com has address 74.125.224.204
www3.l.google.com has address 74.125.224.196
www3.l.google.com has address 74.125.224.199
www3.l.google.com has address 74.125.224.201
www3.l.google.com has address 74.125.224.194
www3.l.google.com has address 74.125.224.192
www3.l.google.com has address 74.125.224.200
www3.l.google.com has address 74.125.224.205
www3.l.google.com has address 74.125.224.198
Updates:
Is this setup redundant?
It is not inherently redundant in the engineering sense since if one of those ips were to fail it would continue to be served to the customer until a DNS zone change is performed and all downstream caches expire. With that said, most browsers are smart enough to try another ip under these circumstance - reference.
Moreover, a system could easily be devise that instead of requiring a DNS zone change to remove the failed node would, instead, route the ip of the failed instance to a servicing device by simple ip takeover.
Is this setup resilient?
Yes, resilience is achieved by minimizing your failure domain. Back to our example failure of a single ip (and remember these ips may represent load balancers backed by hundred of servers or even an entire data center) the likelihood of a customer hitting that ip is 1/16, or ~6%, (using the google example above). This is inherently more resilient than a system with a single A address, wich would impact 100% of the users, or a system with 2 A records in which the user has an even 50/50 change of hitting a failed resource.
Don't worry about it. There are multiple levels of DNS caches so user is not going to flip between 2 IPs on every request. The IP will stay the same for hours for each client.
We have an opposite problem. When server goes down, the user still has the bad IP. We set the TTL to 1 minute but very few browsers honor it. Due to this issue, VIP is a much better option than DNS for load-balancing on the same network.
DNS round robin does not provide redundancy.
Without substantial additional help it only provides dumb load sharing (nb: not load "balancing", which implies dynamic load distribution based on server load).
Having the same cert on two IPs should be no problem, though.