introspection - what's the biggest difference between dir and __dict__ in python

Question

class C(object):
    def f(self):
        print self.__dict__
        print dir(self)
c = C()
c.f()

output:

{}

['__class__', '__delattr__','f',....]

why there is not a 'f' in self.__dict__

Answer 1

dir() does much more than look up __dict__

First of all, dir() is a API method that knows how to use attributes like __dict__ to look up attributes of an object.

Not all objects have a __dict__ attribute though. For example, if you were to add a __slots__ attribute to your custom class, instances of that class won't have a __dict__ attribute, yet dir() can still list the available attributes on those instances:

>>> class Foo(object):
...     __slots__ = ('bar',)
...     bar = 'spam'
... 
>>> Foo().__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Foo' object has no attribute '__dict__'
>>> dir(Foo())
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', 'bar']

The same applies to many built-in types; lists do not have a __dict__ attribute, but you can still list all the attributes using dir():

>>> [].__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__dict__'
>>> dir([])
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']

What dir() does with instances

Python instances have their own __dict__, but so does their class:

>>> class Foo(object):
...     bar = 'spam'
... 
>>> Foo().__dict__
{}
>>> Foo.__dict__.items()
[('__dict__', <attribute '__dict__' of 'Foo' objects>), ('__weakref__', <attribute '__weakref__' of 'Foo' objects>), ('__module__', '__main__'), ('bar', 'spam'), ('__doc__', None)]

The dir() method uses both these __dict__ attributes, and the one on object to create a complete list of available attributes on the instance, the class, and on all ancestors of the class.

When you set attributes on a class, instances see these too:

>>> f = Foo()
>>> f.ham
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Foo' object has no attribute 'ham'
>>> Foo.ham = 'eggs'
>>> f.ham
'eggs'

because the attribute is added to the class __dict__:

>>> Foo.__dict__['ham']
'eggs'
>>> f.__dict__
{}

Note how the instance __dict__ is left empty. Attribute lookup on Python objects follows the hierarchy of objects from instance to type to parent classes to search for attributes.

Only when you set attributes directly on the instance, will you see the attribute reflected in the __dict__ of the instance, while the class __dict__ is left unchanged:

>>> f.stack = 'overflow'
>>> f.__dict__
{'stack': 'overflow'}
>>> 'stack' in Foo.__dict__
False

TLDR; or the summary

dir() doesn't just look up an object's __dict__ (which sometimes doesn't even exist), it will use the object's heritage (its class or type, and any superclasses, or parents, of that class or type) to give you a complete picture of all available attributes.

An instance __dict__ is just the 'local' set of attributes on that instance, and does not contain every attribute available on the instance. Instead, you need to look at the class and the class's inheritance tree too.