In Python, you can set most request timeouts w/ socket.setdefaulttimeout(). In recent versions, urllib2 has also added a timeout field to urllib2.urlopen(). So far so good, right?
Unfortunately, while these work fine when looking up IPs or domains in /etc/hosts, this fails miserable when querying a FQDN as you’re at the mercy of socket.gethostbyname() and your DNS resolver which does not let you adjust the timeout. On my Mac this defaults to 30 seconds. It’ll ruin your day, really. (A good recent thread, old summary)
This is a somewhat common problem and you can see a lot of various workarounds (using signals didn’t work for me). The proper modern way is to probably using multiprocessing with a join(timeout) (sample) but that seemed awfully wordy, so here’s my simple one-line hack that I ended up with instead:
subprocess.check_call(['/sbin/ping','-t','1','FQDN'])
Just set 1 to the timeout you want. It’s hacky, but it works and it’s much easier and shorter – a one liner in a try block without any other libraries. Another advantage this has is that it works as it should both with DNS and mDNS (zeroconf) without any additional lookups. I’m using this for finding local machines so this is quite useful.
Some extra references:
- pybonjour, zeroconf – a couple zeroconf libs that are way overkill for what I wanted to do
- monkeypatch/custom dns – creative way to solve this
- another monkeypatch example
- dnspython – for custom resolver
- timeout w/ subprocess – you’ll need the subprocess32 backport if you need this
- tornado-dns – async dns w/ Tornado (fyi Tornado blocks on DNS for async_http calls by default – still as of late 2012)
UPDATE:
So, at the end of 2016 I encountered this problem, and decided that I could do better, especially because I wanted to do a sub-second query. I decided that of course the way to go would be to use concurrent.futures, but that was actually wrong, it turns out. When you call socket.gethostbyname() to finish. Here’s the simplest code that I implemented that worked:
from multiprocessing import Process, Queue
def dns_lookup(host, q):
try:
socket.gethostbyname(host)
q.put('OK')
except:
pass
q = Queue()
p = Process(target=dns_lookup, args=('example.com', q,))
p.start()
time.sleep(0.4)
if q.empty():
p.terminate()
print('dns timeout')
Using the multiprocessing library turns out to be the way to go because the terminate() function actually works like you expect it to, killing with extreme prejudice and w/o too much extra code. Hope that helps anyone dealing with the same problem.