python data and non-data descriptors

Question

According to Python's documentation,

Data descriptors with __set__() and __get__() defined always override a redefinition in an instance dictionary.

I have no problem understanding this sentence, but can someone clarify for me why such a rule is in place? After all, if I want to override an attribute in an instance dictionary, I already need to do that explicitely (inst.__dict__["attr"] = val), as a naive inst.attr = val would call the descriptor's __set__ method, which would (usually) not override the attribute in the instance dictionary.

edit: just to make it clear, I understand what is happening, my question is about why such a rule was put in place.

Answer 1

The override applies to descriptors that are part of the class __dict__.

Python will always look up type(instance).__dict__[attributename].__get__(instance, type(instance)), and will not use instance.__dict__ to search for a instance-override.

Here is an example using a contrived Descriptor class and a property (which is a descriptor with a __get__ and a __set__:

>>> class Descriptor(object):
...     def __init__(self, name):
...         self.name = name
...     def __get__(self, instance, cls):
...         print 'Getting %s, with instance %r, class %r' % (self.name, instance, cls)
... 
>>> class Foo(object):
...     _spam = 'eggs'
...     @property
...     def spam(self):
...         return self._spam
...     @spam.setter
...     def spam(self, val):
...         self._spam = val
... 
>>> Foo().spam
'eggs'
>>> foo = Foo()
>>> foo.__dict__['spam'] = Descriptor('Override')
>>> foo.spam
'eggs'

As you can see, even though I add a spam entry in the instance __dict__, it is completely ignored and the Foo.spam property is used still. Python is ignoring the instance __dict__ because the spam property defines both __get__ and a __set__.

If you use a descriptor that doesn't define a __set__ the override works (but it's __get__ is not called:

>>> class Foo(object):
...     desc = Descriptor('Class-stored descriptor')
... 
>>> Foo.desc
Getting Class-stored descriptor, with instance None, class <class '__main__.Foo'>
>>> Foo().desc
Getting Class-stored descriptor, with instance <__main__.Foo object at 0x1018df510>, class <class '__main__.Foo'>
>>> foo = Foo()
>>> foo.__dict__['desc'] = Descriptor('Instance-stored descriptor')
>>> foo.desc
<__main__.Descriptor object at 0x1018df1d0>