UDP against routers : hole punching
Hello there! Today, I ran a few tests with UDP networking (yes, my hobbies are really exciting), and my results made me think I could write another article… This is the first one related to networking, hopefully it’ll be understandable enough. For this first time, we’ll talk about UDP hole punching. Let’s go!
What is UDP hole punching ?
Networking notions
For those of you who aren’t interested in networking (yet), here’s a little schema of mine :
Beautiful is it not (well no, it isn’t) ? Basically, your machine is connected to a router, which connects you to the rest of the world. With this very common setup, it is impossible for A and C to communicate directly. In order to do so, the routers must be configured accordingly, usually through port forwarding rules, and relay messages sent on the communication ports to the specified machines. That’s great, but not everyone enjoys going into the router’s configuration to allow home-made networking applications to work.
Punching a hole
Punching a hole makes the problem go away. It relies on a very simple thing : they are ports opened when receiving and when sending. This means that when you send a packet outside (through your router), a port will be automatically assigned to the outgoing transmission. Let’s explain the process with a list this time :
- Your machine sends to packet to 1.2.3.4:5000 (this is an example IP which happens to belong to Google).
- The packet goes through your router, which will handle the transmission from here.
- The router assigns a random port for the outgoing message, let’s say 12345.
- The message goes from your router (port 12345) to its destination (1.2.3.4, port 5000).
There it is, this random port. 12345 is used for the transmission, but more importantly, it is connected to your local machine : you punched a hole. This means that any message sent to your router, on port 12345 will be relayed to your machine. Point is : once you close your socket descriptor, the one you used to send the message, this assignation goes away (the hole is closed). The trick here will be to maintain the socket descriptor so that 1.2.3.4 can answer back!
Some code
The trick needs to be performed by both sides, client and server. For this reason, there will be two samples here. See you in the comments.
The server
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
#include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <string.h> int main(int argc, char** argv){ /* * Define the local host. * - Listen all IPs (INADDR_ANY). * - Listen on port 5000. */ struct sockaddr_in local; local.sin_family = AF_INET; local.sin_port = htons(5000); local.sin_addr.s_addr = INADDR_ANY; // Prepare the socket and bind it. int sock = socket(AF_INET, SOCK_DGRAM, 0); if(bind(sock, (struct sockaddr*)&local, sizeof(local)) == -1){ perror("bind"); exit(1); } /* * Ready to listen. * Let's prepare a structure for the connecting host. */ struct sockaddr_in remote; int addr_len = 0; char buffer[256]; /* * The UDP server will listen to 10 messages * before terminating. Let's see who's talking... */ int i = 0; for(i = 0; i < 10; i++){ memset(buffer, 0, 256); memset(&remote, 0, sizeof(struct sockaddr_in)); if(recvfrom(sock, buffer, 256, 0, (struct sockaddr*)&remote, &addr_len) < 0){ perror("recvfrom"); exit(2); } fprintf(stdout, "Packet received from %s:%d (%s)\n", inet_ntoa(remote.sin_addr), remote.sin_port, buffer); } return EXIT_SUCCESS; } |
Now, write some code, compile it, and run it on a distant server, which has a public IP. Keep it aside, and let’s program the client.
The client
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
#include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <string.h> int main(int argc, char** argv){ if(argc < 2){ fprintf(stderr, "Usage: %s [remote host]\n", argv[0]); exit(1); } /* * Define the remote host. */ struct sockaddr_in remote; remote.sin_addr.s_addr = inet_addr(argv[1]); remote.sin_port = htons(5000); remote.sin_family = AF_INET; int addr_len = 0; int sock = socket(AF_INET, SOCK_DGRAM, 0); /* * Same principle. * The client will send 10 messages saying "Hi" to * the UDP server, without EVER closing/reopening * the above socket. */ int i = 0; for(i = 0; i < 10; i++){ if(sendto(sock, "Hi\0", 3, 0, (struct sockaddr*)&remote, sizeof(struct sockaddr_in)) < 0){ perror("sendto"); exit(2); } sleep(1); // Let's take our time... } return EXIT_SUCCESS; } |
Run your favourite C compiler on this, and run it on your local machine. Watch those funny little lines printing on the server terminal. You should recognise your router’s IP, and a port, always the same (if it differs, then you probably abandoned the socket of the client side).
Now, you’ll probably tell me : “this doesn’t help! the server isn’t my destination, I want to contact another machine behind another router!”. You’re right, that’s what you want to do. But the above code is a way to set it up! Let’s see…
Communicating over the Internet, behind routers
Here’s how we can actually punch a hole to make A and B communicate.
- Machine A sends a packet to a world-visible server, let’s call it S.
- Machine B does just the same.
- S receives the packet from A, which transited through A’s router (with the random port!)
- S receives the packet from B, which transited through B’s router (with another random port!)
- S sends B’s IP and random port to A.
- S sends A’s IP and random port to B.
From there, A and B know exactly how to contact each other : through the ports which were allocated when contacting S. And because they’re super kind and all, the client applications on these machines didn’t close any socket descriptor, meaning there’s still a random port associated for both of them. Now, another cute little schema :
There you go ! As long as A and B keep their descriptors opened, and don’t renew them, then A and B will be able to communicate through their routers, without any manual port forwarding configured. As it so happens, UDP hole punching is a very common mechanism, especially in VoIP or P2P software. Even though other solutions exist (STUN, UPnP), this one requires less setup, and works on most routers. Be careful though : this might fail on some NAT types… You cannot win every time, right?
That’s all for today, see you next time!
Very Interesting article.. this is what i need.
I am using a similar type of tool called “pwnat”, but am not sure if the connection is a success or not…
i am a student and i want to code remote connection app(c/c++), but as a student its difficult to understand all this networking and coding concepts.
Can you explain some messaging or desktop sharing concepts on this topic?
I am not familiar with pwnat but I remain quite sceptical about the way it works. However, it seems to rely on both ICMP and IP spoofing. Basically, the server sends dummy packets into the void (3.3.3.3), and when a client wishes to connect, it simulates responses from this dummy host.
Server > NAT > 3.3.3.3 (ICMP Echo Request)
… no response.
Server > NAT > 3.3.3.3 (ICMP Echo Request)
… no response.
Then a client comes in…
Client > Client’s NAT > Server’s NAT (ICMP Time Exceeded with fake source)
At this moment, the client’s ICMP TE is sent with “3.3.3.3” spoofed as the source IP. With this, the server’s NAT will believe that these packets are originating from 3.3.3.3 and let them through. Of course, this requires the client to know the dummy IP address…
That’s where I’m doubting… It takes root privileges to build up ICMP packets, and I don’t quite see how a simple user could fill the “IP header of original datagram’s data“.
Now, supposing the client successfully sends this fake “ICMP Time Exceeded” packet, the server will now know the client’s IP address, since it’ll be in the IP headers of the fake ICMP TE. At this moment, both the server and the client start flooding each other’s NAT with UDP packets. While these are supposed to get dropped by both NATs, since the source and destination addresses match, both NATs should eventually let them through and consider them responses to each other, therefore creating a client-to-client link.
All in all, I don’t believe this piece of software can be used for a production application, since it might fail to connect the clients behind specific types of NAT. Besides, when trying to connect a client to a server behind a NAT, it is much easier to set a forwarding rule on the server side… Since there’s only one server (and many clients), there is no need for dynamic STUN here. A simple rule will do.
Additionally, it is very unlikely that the client app’ gets root privileges, required to fake a first transmission to the server. Finally, this requires the client to know two things: the server IP and the dummy IP. When building a messaging software, these two are completely unknown to the clients. In this case, I would recommend using a TURN server, UPnP or STUN techniques where they work, yet these are slightly trickier to build up…
Well when you say “set a forwarding rule on the server side” do you mean port forwarding?
Well i have seen that too. port forwarding manually is possible, but what if there are multiple servers or the user don’t know how to port forward .
Stun:- Discover type of NAT and firewall between application and public internet.
(firewall permission is needed)
Turn:-More trickier than Stun.
UPNP:- Possible. but (Universal pnp must be enable or enable it manually ).
For application development this things should be automatically adjusted and which is very difficult to obtain.
I have found some useful resource but haven’t tested it yet these are as listed:-
1.UPNP port forwarding :- (http://www.fluxbytes.com/csharp/upnp-port-forwarding-the-easy-way/).
It say’s “manual port forwarding is not needed, at least for UPNP enabled routers.”
2.UDP Hole Punching c# :- (http://www.fluxbytes.com/csharp/udp-hole-punching-implementation-in-c/) (another guide).
what should i do? should i use this UPNP port forwarding?
or should i do hole punching which sounds very difficult to implement?
“what if there are multiple servers or the user don’t know how to port forward”
> Here’s the trick. If there is a server, then it belongs to you: you’re the administrator, you setup port forwarding, and your clients just connect to your NAT-ed server. However, when you purchase a server from an online provider, most of the time, the machines are non-NATed, which solves the problem.
You need to see the difference between client-server STUN and client-client STUN here. If there’s only one server to which clients connect, then there is no need for hole punching, and probably no need for port forwarding since the server is visible worldwide.
Now, if you want to connect two NAT-ed clients together, it gets tricky, because it would require both clients to setup a port forwarding rule on their NAT. Of course, as you said:
– Most people don’t know how to set it up.
– Some people just can’t access their NAT settings.
UPnP is very recent, and very unpopular among developers. Most of the time, applications offer UPnP communications as an option, which is unused by default. If you take that option away, you can either:
– STUN, basically hole punching, which is what this article is about. Both clients send a packet to the server, therefore opening outgoing ports, which can be reused for incoming streams. Unfortunately, this technique will not work with every router, and those with which it works are getting rarer and rarer.
– TURN, which is what Skype (and so many others) use. Both clients connect to a remote (non NAT-ed) server, and all their communications are relayed through it. This solution is much much much easier to develop, yet, as an administrator, you can read absolutely everything that goes through your server (which is what Skype does… you know).
Also: with a STUN technique, you’ll probably be stuck with UDP. Additionally, if your client EVER closes their socket descriptor or break the link, you have to start punching again. With a intermediary TURN server, you can use both TCP and UDP, and get into far much less trouble in the end.
Concerning your C# links, I’m afraid I can’t help you: I despise this ridiculous language. UPnP is not particularly difficult to use, yet most NAT don’t enable it. Port forwarding is even easier, but cannot be setup by every user. All that’s left is STUN (which is very tricky to set up, if at all possible sometimes) and TURN (which may raise questions about your users’ privacy).
Very descriptive help but I think i have to read more about TURN to understand about it…
i will stay updated with this article…
Anyway …
THANKS FOR THE HELP…
Hello from Ireland JWH Smith!
You seem like a guy who has exactly the answer to a problem that has been roasting my brain for more than a week and any help would be very much appreciated.
I’m trying to build a chatroom here in VC++ to get people to practise the Irish Language as well as other celtic and minority languages. It’s going well but I am completely jammed on that very point. Hole-punching. I can get the windows client to access the Linux router, ascertain the external port number, access the client back from the server, successfully bypassing the router.
The hassle starts when I want another windows client to access the first client by using that external port number. The recvfrom (or recv) will only accept communication from the machine IT contacted. If it sends its initial overture to a windows machine, it will only accept contact from that machine and likewise if it first contacts the Linux machine.
Sadly, since I wrote this article, my router has changed, and I am now unable to work with UDP hole punching (or most kinds of ICEs for that matter). Until I find more time and resources to study the matter, and perhaps publish another article, let me provide you with some links. First of all, a few Wikipedia pages might be a good start for once. This page about ICE is a quick introduction to the “main” technique, and you’ll find details about TURN in this one. You’ll also find more details about STUN here. Details about UDP hole punching can be found here, and this is the technique I used here. You may also be interested in TCP hole punching, but that’s a challenge.
Then, I also came across this post by Oscar Rodriguez, but it dates back to 2011, and you’re probably looking for more recent material (especially if you’re using a modern router). Yet, his code is very well commented and it may give you a hint as to how you may fix your program.
Now, for more motivated readers, I would recommend reading the RFCs, which are pretty much the most complete documents you may come across:
– RFC 5389 ; Session Traversal Utilities for NAT (STUN).
– RFC 5766 ; Traversal Using Relays around NAT (TURN).
– RFC 5245 ; Interactive Connectivity Establishment (ICE)
Now, another thing to keep in mind is that my article gives instructions to Linux users. I cannot guarantee that Windows won’t mess up your whole process through its Winsock interface. The point where you’re stuck is basically punching the hole, and it is very likely that one of your NATs is among those who won’t allow you to do hole punching (symmetric NATs). In this case, you’ll probably have to rely on TURN and relay servers.