Dirty Hack of the Day: Python DNS Edition

In Python, you can set most request timeouts w/ socket.setdefaulttimeout(). In recent versions, urllib2 has also added a timeout field to urllib2.urlopen(). So far so good, right?

Unfortunately, while these work fine when looking up IPs or domains in /etc/hosts, this fails miserable when querying a FQDN as you’re at the mercy of socket.gethostbyname() and your DNS resolver which does not let you adjust the timeout. On my Mac this defaults to 30 seconds. It’ll ruin your day, really. (A good recent thread, old summary)

This is a somewhat common problem and you can see a lot of various workarounds (using signals didn’t work for me). The proper modern way is to probably using multiprocessing with a join(timeout) (sample) but that seemed awfully wordy, so here’s my simple one-line hack that I ended up with instead:


Just set 1 to the timeout you want. It’s hacky, but it works and it’s much easier and shorter – a one liner in a try block without any other libraries. Another advantage this has is that it works as it should both with DNS and mDNS (zeroconf) without any additional lookups. I’m using this for finding local machines so this is quite useful.

Some extra references:


So, at the end of 2016 I encountered this problem, and decided that I could do better, especially because I wanted to do a sub-second query. I decided that of course the way to go would be to use concurrent.futures, but that was actually wrong, it turns out. When you call whether on the ThreadPoolExecutor or ProcessPoolExecutor version, it still waits for socket.gethostbyname() to finish. Here’s the simplest code that I implemented that worked:

from   multiprocessing import Process, Queue
def dns_lookup(host, q):

q = Queue()
p = Process(target=dns_lookup, args=('example.com', q,))

if q.empty():
  print('dns timeout')

Using the multiprocessing library turns out to be the way to go because the terminate() function actually works like you expect it to, killing with extreme prejudice and w/o too much extra code. Hope that helps anyone dealing with the same problem.