python - Scapy fails to sniff packets when using multiple threads

Question

I'll try to demonstrate my problem with a simplified example.

Following is a very simple (single threaded) packet sniffer (ICMP):

from scapy.all import *

m_iface = "wlan0"
m_dst = "192.168.0.1"

def print_summary(pkt):
  print pkt.summary()

def plain_sniff():
  sniff(iface = m_iface, count = 10, filter = "icmp and src {0}".format(m_dst), prn = print_summary)

This sniffer works just fine and I get the output:

WARNING: No route found for IPv6 destination :: (no default route?)
Ether / IP / ICMP 192.168.0.1 > 192.168.0.9 echo-reply 0 / Raw
Ether / IP / ICMP 192.168.0.1 > 192.168.0.9 echo-reply 0 / Raw
Ether / IP / ICMP 192.168.0.1 > 192.168.0.9 echo-reply 0 / Raw
...

Next, I create a separate thread for sniffing packets and use a Queue to communicate the captured packets between the sniffer thread and the main thread:

from threading import Thread
from Queue import Queue, Empty
from scapy.all import *

m_iface = "wlan0"
m_finished = False
m_dst = "192.168.0.1"

def print_summary(pkt):
  print pkt.summary()

def threaded_sniff_target(q):
  global m_finished
  sniff(iface = m_iface, count = 10, filter = "icmp and src {0}".format(m_dst), prn = lambda x : q.put(x))
  m_finished = True

def threaded_sniff():
  q = Queue()
  sniffer = Thread(target = threaded_sniff_target, args = (q,))
  sniffer.daemon = True
  sniffer.start()
  while (not m_finished):
    try:
      pkt = q.get(timeout = 1)
      print_summary(pkt)
    except Empty:
      pass

This sniffer also works fine and I get the same output as above. However, when I modify the main thread just a little bit so that it uses the send() function between reads from the packet queue as below:

def threaded_sniff_with_send():
  q = Queue()
  sniffer = Thread(target = threaded_sniff_target, args = (q,))
  sniffer.daemon = True
  sniffer.start()
  while (not m_finished):
    send(IP(dst = m_dst) / ICMP()) # Here
    try:
      pkt = q.get(timeout = 1)
      print_summary(pkt)
    except Empty:
      pass

Then I get the following bizarre output (the filter doesn't seem to work):

WARNING: No route found for IPv6 destination :: (no default route?)
Sent 1 packets.
Ether / ARP who has 192.168.0.1 says 192.168.0.9
Sent 1 packets.
Ether / ARP is at a0:21:b7:1a:7a:db says 192.168.0.1
Sent 1 packets.
Ether / IP / ICMP 192.168.0.9 > 192.168.0.1 echo-request 0
Sent 1 packets.
Ether / IP / ICMP 192.168.0.1 > 192.168.0.9 echo-reply 0
...

The script for the three sniffers can be downloaded from here.

My current system configuration is as below:

Python: 2.7.3
Scapy: 2.2.0
OS: Fedora 18

Interestingly, all the three sniffers work fine on my older computer:

Python: 2.6.4
Scapy: 2.0.0.10 (beta)
OS: Fedora 13

First I thought it might be the Scapy / Python versions. But even when I installed the exact same versions on my new computer, the behaviour persisted.

I'm not entirely sure if this is a question suitable for SO (may be a bug report to Scapy?). Please excuse me in that case.

Answer 1

The main reason for this is most likely because of resource locks. Scapy's sniff() function probably have to lock on to low-level network resources in order to be able to sniff packets.

Delaying the two threads (after starting the sniffer thread) you'll ensure that Scapy will get the time it needs to do so.

To see how we came to this conclusion see the discussion above in the comment section. Gl Asiri Rathnayake :)