ICS 33 Fall 2024 | News | Course Reference | Schedule | Project Guide | Notes and Examples | Reinforcement Exercises | Grade Calculator | About Alex

ICS 33 Fall 2024
Notes and Examples: Inheritance

Background

Introduced in your prior coursework and solidified in this course, the idea that Python relates classes to one another somewhat loosely is a familiar one. As long as there's a common protocol defining how the objects of many classes can be asked to do a particular job — such as giving us their length, being indexed or sliced like a list or string, or compared for equality or ordering — objects of many classes can be used interchangeably for that purpose, meaning that we can ask them to do that job without depending on what types of objects they really are. This allows us to write simple functions that are extremely flexible, in terms of what kinds of inputs they can take.

What can we pass as an argument to this function, for example?


def total_length(items):
     return sum(len(item) for item in items)

The short answer is "way too many possible types for us to count," but here are a few of the many possibilities.

There are two common threads that tie together all of the ways that total_length can be used, each of which can be succinctly described in terms of a Python protocol we've seen.

A combination of the Python rules we've learned this quarter takes care of the rest.

That last rule is the one that enables many different kinds of objects to be processed by the same function, yet behave in the appropriate way automatically depending on their types. When we ask a string for its length, it's the str.__len__ method that decides what the answer is. When we ask a MyFancyDataStructure object for its length, MyFancyDataStructure.__len__ answers the question instead. The combination of protocols and type-based lookups are where Python derives a lot of its flexibility, allowing a function like total_length to be written with minimal ceremony yet be wildly flexible. When the types fit, it just works. When they don't, an exception is raised. (Requiring that the types will fit before a program runs is a topic for another day.)

So, as we see here, despite not having specified the relationships between these classes anywhere, their objects are naturally interchangeable. As long as we follow agreed-upon protocols and well-known language rules, we gain this flexibility automatically.

But the story of how classes relate to one another in Python doesn't end with protocols, because this kind of relationship, while useful and powerful, isn't the only one we'll find that we need. What we do when we follow a protocol in a class is to make objects of that class interchangeable with others that follow the same protocol. In other words, we're giving its objects the ability to be asked to do the same jobs as other kinds of objects, but in their own particular way.

Sometimes, we want a different kind of relationship, where we instead want to define a new class whose objects can do the same jobs the same way as objects of an existing class, except we may also give objects of our new class some additional abilities, or make them behave differently in some cases, even if they behave the same way in others.

That relationship between classes is categorically different from one in which two classes implement the same protocol. We don't just need to be sure we've written methods with the same signatures to share behavior between two classes. We need methods with the same bodies, since they need to do the same things in the same way. That could require us to copy and paste potentially large swaths of code from one class to another, with the usual downsides of our program becoming larger and, more importantly, having to maintain the same code in multiple places — meaning that fixing a bug in one necessitates finding and fixing the same bug everywhere else we've copied the same code.

What we'd rather do, then, is tell Python about this relationship explicitly. If there's a way to express this in our program directly — if we could say "Our new class Y should be just like the class X, except I'll explain here only the ways that it's different" — then we'd enjoy the dual benefits of not having to copy the code (meaning that Y contains less code to read and understand on its own) and not having to maintain the two copies in the long run (which is, by far, the bigger win here, since a change to X would automatically become a change to Y going forward).

What we've just described is what Python calls inheritance. When we define a new class that inherits from another, we can think of that as a way of saying that objects of the new class are just like objects of an existing one, except to the extent that we describe their differences in the body of the new class. That doesn't just mean "Objects of our new class have the same method signatures as objects of the existing class"; it means also that "objects of our new class have the same methods (including their bodies) as objects of the existing class, except where we've said otherwise."

There are plenty of situations where inheritance is an approach that's too heavy-handed in a language like Python, but there are also plenty of situations where it's exactly what the doctor ordered. Before we contrast those situations, though, let's understand the mechanisms by which inheritance works, because it's exactly those mechanisms that inform its usefulness. Once we have a handle on that, we'll have a way to differentiate between when we might want to use inheritance and when it might not be the best choice.

Single inheritance

We've learned many details governing how Python resolves attribute lookups. If we have a variable x and we write x.y, a dizzying array of rules is brought to bear, including various ways that our classes might customize the result. But, leaving customizations like __getattr__ and descriptors out of the conversation, the rules we've learned for resolving x.y are pretty simple.

By our understanding, if y cannot be found within x or type(x), an AttributeError will be raised. But there's once again more to the story than we've seen. (We're learning the entire story piecemeal, so that we can understand it piecemeal. Each layer of the story adds an ability we didn't previously have, but otherwise doesn't contradict our previous understanding. This is a great way to learn a complex topic, but it helps to have a guide when you're taking such a tour. So, think of me as your guide.)

To begin the next layer of the story, let's consider one thing we've seen before: Even if we don't specify how equality is determined for objects of a class by writing an __eq__ method, we can nonetheless check objects of that class for equality.


>>> class Thing:
...     pass
...
>>> t1 = Thing()
>>> t2 = Thing()
>>> t1 == t2
    False      # If we don't specify how equality is determined for objects of our class,
               # identity is used to determine it.

As we've seen repeatedly in this course, these kinds of things rarely depend on special magic in Python; there's usually a specific mechanism that makes them happen, and it's usually both visible and customizable. When we specify how equality should be implemented for objects of a class, we do so by writing an __eq__ method, because that's the method that Python calls when we use == to compare two objects. That observation raises an interesting question: Does a Thing object have an __eq__ method?


>>> t1.__eq__
    <method-wrapper '__eq__' of Thing object at 0x00000163EA71F370>
                       # Yes, it does!  If so, then we can call it.  What happens if we do?
>>> t1.__eq__(t2)
    NotImplemented     # This is the mechanism that tells Python to fall back on using
                       # identity to determine equality instead.
>>> sorted(t1.__dict__.keys())
    []                 # The __eq__ method is not in the object's dictionary.
>>> sorted(Thing.__dict__.keys())
    ['__annotations__', '__dict__', '__doc__', '__module__', '__weakref__']
                       # It's not in the class' dictionary, either.

If neither the object nor the class has an __eq__ method listed in its dictionary, then where did Python get its __eq__ method from? The answer, as it turns out, revolves around inheritance. Whenever we write a class in Python, whether we say so or not, it inherits from at least one class. When a class Y inherits from a class X, we say that X is a base class of Y and that Y is a derived class of X. All classes have at least one base class, except the object class that's built into Python; as a necessary special case, it has no base class. Many classes, on the other hand, won't have any derived classes.

What are the base classes of our Thing class, then? We didn't specify any, so what are they? The __bases__ attribute of the class can tell us.


>>> Thing.__bases__
    (<class 'object'>,)
             # Classes can have more than one base class, so a tuple is used to store
             # them, even if there's only one.
>>> object.__bases__
    ()       # The object class has no base classes.  An empty tuple is a reasonable
             # and compatible way to specify that.

When we don't specify any base classes for a class, it will have exactly one base class: Python's built-in object class. When a class has one base class, we call it single inheritance; Thing singly inherits from object, in other words.

The use of object as a base class when we don't specify one is a sensible default; that way, no matter what we say, all classes inherit from object, one way or another, which means that there are some features we're sure are provided by all objects, no matter their type. (Those features will even be present in objects of the object class, since they won't need to be inherited from a base class; they'll just be there.)

All of this suggests that it's the object class that's supplying the __eq__ method that was called when we evaluated t1 == t2 or t1.__eq__(t2).


>>> object.__eq__(t1, t2)
    NotImplemented     # It behaves the same way.
>>> object.__eq__ is Thing.__eq__
    True               # And indeed it is the same.

And there's a simple mechanism that would allow object.__eq__ to be used when we don't write an __eq__ method of our own, while allowing ours to take precedence when we do. When we write x.y to access an attribute, there's a slightly more complex set of rules at work than we've seen previously.

So, when we wrote t1.__eq__, that triggered the following lookup steps.

As is often the case in Python, a lot of power is derived from a relatively simple mechanism. To understand its power, though, we'll need to look a little more carefully at it.

Inheriting a class from a base class other than object

When we write a class, we can specify its base class explicitly by following its name with parentheses, within which we write the name of the desired base class.


>>> class Base:
...     def first(self):
...         return 13
...
>>> class Derived(Base):       # Derived inherits from Base.
...     def second(self):
...         return 17
...
>>> d = Derived()
>>> d.second()
    17         # We can call the method we defined in Derived itself.
>>> d.first()
    13         # We can call the method inherited from Base, too.
>>> Derived.__bases__
    (<class '__main__.Base'>,)
               # Derived singly inherits from Base.
>>> Base.__bases__
    (<class 'object'>,)
               # Base singly inherits from object.

It's worth noting that the presence of base classes complicates what it means to ask questions about an object's type, because there are now two questions we might be asking.

While there are times you might ask either of these questions, it's much more often the case in practice that you'd want to be asking the latter one, because it's much more often that you're really asking "Is it safe to call all of Base's methods on this object?" If so, then a Derived object is as good as a Base object for your purposes.


>>> type(d) is Derived
    True            # Is the type of d specifically Derived?  Yes.
>>> type(d) is Base
    False           # Is the type of d specifically Base?  No.
>>> isinstance(d, Derived)
    True            # Is it safe to treat d as a Derived?
                    # Yes, because d has all of Derived's methods,
                    # including the ones inherited from Base (and object).
>>> isinstance(d, Base)
    True            # Is it safe to treat d as a Base?
                    # Yes, because d has all of Base's methods,
                    # including the ones inherited from object.
>>> isinstance(d, object)
    True            # Is it safe to treat d as an object?
                    # Yes.  All objects have the methods inherited from object.

How a derived class reuses and enriches base class functionality

The mechanics of attribute lookup in the presence of base classes suggests two ways that derived classses can be different from their base classes, along with one way they can't be. Suppose that a class Y singly inherits from X.

The third of these is not an oversight; there's a good reason that it's not permitted, which, in turn, led Python's designers to choose mechanics that disallowed it. Suppose you've written a class X with a method determine_cost that calculates a "cost" of some kind, and you've then written the following function that calls determine_cost on every X object in an iterable and returns the standard deviation of the results.


def process_xs(xs):
    return statistics.stdev(x.determine_cost() for x in xs)

Now, suppose there's a class Y that singly inherits from X. What do we expect to happen if we call process_xs and pass it a collection of Y objects? Or if we pass it a collection containing a mixture of X and Y objects? We might expect something to be different, but we'd expect the basic assumption to hold: X objects (including objects of classes that derive from X) have a determine_cost method that returns a numeric result. In other words, we might expect the way that Y objects determine a cost to be different from how X objects determine it, but we don't expect the ability to have changed.

The idea that objects of derived classes should be able to safely take the place of objects of base classes is referred to as the Liskov substitution principle, which is part of the bedrock of object-oriented software design. If derived classes could remove attributes from base classes, that would necessarily make objects of those derived classes unable to safely take the place of base class objects; since no good can come from it, it's better to simply disallow it altogether, so Python's mechanism is designed in a way that disallows it. We also want to be careful about the way that we replace base class functionality in derived classes, so that our alterations still allow safe substitution, but the devil there is in the details, so that's a battle we win with finesse rather than having it enforced on us by Python.

Replacing an attribute in a derived class is simply a matter of providing a new definition for it, so that the attribute will be present in the derived class' dictionary — and, thus, will hide the attribute in the base class' dictionary.


>>> class Base:
...     def value(self):
...         return 11
...
>>> class Derived(Base):
...     def value(self):
...         return 17
...
>>> Derived().value()
    17       # The value attribute in Derived is the one that "wins".
>>> Base().value()
    11       # If the object is a Base, we get Base's version of value.

Sometimes, particularly when we're replacing a base class method with a new implementation in a derived class, we want the derived class version to make use of the base class version. For example, suppose we want Derived's value method to return the square of the value returned by Base's version. Here's how we might write that.


>>> class Derived(Base):
...     def value(self):
...         base_value = super().value()
...         return base_value * base_value
...
>>> Derived().value()
    121      # This is the square of 11.

When we use the built-in super() function in a method within a class that singly inherits from a base class, it returns an object that allows access to the attributes of the base class. So, for example, when we write super().value(), we mean "The value method in the base class of this class."

(The name super may seem out of place here, but it comes from another common object-oriented software design term. When a class Y derives from a class X, we sometimes say that X is a superclass or supertype of Y and that Y is a subclass or subtype of X. So, asking to treat something as "an object of a superclass" is the same idea as asking to treat it as "an object of a base class.")

Multiple inheritance

We say that multiple inheritance in Python is the act of a class deriving from more than one base class. If single inheritance is a way of saying "Objects of this class can be substituted in place of objects of that other class," then multiple inheritance is a way of saying "Objects of this class can be substituted in place of objects of either of those other classes." Or, thought differently, it allows us to say "Objects of this class can do all of the things that objects of both of those classes can do."

Not all programming languages with features analogous to classes and inheritance allow multiple inheritance, but Python is not the only one that does; C++ is another well-known example of a programming language with both classes and multiple inheritance. The languages that were designed to disallow it were most often designed that way because of the complexities it adds to the language, both in terms of its effect on the underlying implementation, as well as on people being able to read and understand the subtleties of a program that uses it. When a language disallows multiple inheritance, it's generally because its designers decided that those complexities were too costly relative to the added benefit. (Java is a notable example of a language that made the choice opposite of Python's; Java has both classes and inheritance, but does not allow multiple inheritance.) Reasonable people can disagree about whether the benefits of multiple inheritance outweigh its costs, but our best bet is to understand a little bit about both sides of that argument, which we can consider by examining how multiple inheritance is implemented in Python.

Since the effect of inheritance in Python arises primarily from its impact on the attribute lookup algorithm, then we should consider how attribute lookup might be done differently on an object whose class has multiple base classes. A good way to start that process is to experiment, so we'll start with some classes that demonstrate various combinations of what's possible. (Download the file linked below and read through it before proceeding.)

Let's start by experimenting with Derived12, which inherits from both Base1 and Base2. Our rough expectation is that a Derived12 object can do anything a Base1 object can do and anything a Base2 object can do, but it's wise to verify that, and to see if there are details that stand out as interesting.


>>> Derived12().derived_only()
    'Derived12.derived_only'   # We can call a Derived12 method on a Derived12 object.
                               # That's no surprise.
>>> Derived12().one_only()
    'Base1.one_only'           # Derived12 objects can be treated as Base1 objects, so we can
                               # call a Base1 method on a Derived12.
>>> Derived12().two_only()
    'Base2.two_only'           # Derived12 objects can be treated as Base2 objects, too, so we
                               # can also call a Base2 method on a Derived12.
>>> Derived12().both()
    'Base1.both'               # If a Derived12 can be treated as either a Base1 or a Base2,
                               # then why did we get the Base1 version and not the Base2 version?
                               # Or, why didn't we get both of them?  Or, why didn't this result
                               # in an exception?

That last expression demonstrates where multiple inheritance leads to complexity. The assumptions we can make in the presence of single inheritance don't always compose cleanly when we use multiple inheritance.

So, given this choice, which is the right one? Resolving that tension — what to do when two base classes provide the same attribute with different values — is the central problem that needs to be solved when designing a multiple inheritance feature like Python's. Not all languages with multiple inheritance make the same choice (and some avoid it altogether by disallowing multiple inheritance), but we'll concentrate for now on Python's design and build a fuller understanding by continuing our experiment.


>>> Derived21().both()
    'Base2.both'               # Derived21 handles this differently from Derived12.  How and why?
>>> Derived12Override().both()
    'Derived12Override.both'   # This is less surprising.  Since the class has its own both method,
                               # it overrides whatever is in any base class.

Because we see that Derived12 and Derived21 behave differently, we ought to take a look at how they're written differently from each other. The key difference is in the first line of their definitions, where they inherit from the same classes, but do so in a different order.


class Derived12(Base1, Base2):
    ...

class Derived21(Base2, Base1):
    ...

So, it seems clear that the order in which we list the base classes has an effect on the rules that govern attribute lookup, the details of which we should familiarize ourselves with.

Method resolution order (MRO)

Suppose there's a variable x referring to some object and the expression x.y is evaluated. How does Python determine what the value of y is? Once again, there's a little bit more to the story than we've seen, with the story having an additional layer that doesn't invalidate anything we've seen, but adds the ability to handle multiple inheritance in a definitive way.

When a class is defined in Python, one of the things that's determined automatically is the order in which class dictionaries will be checked when looking for an attribute. This ordering is called the method resolution order (or, MRO), and can be found in an attribute named __mro__ within the class. (Note that the method resolution order isn't just for methods; despite its slightly misleading name, all attribute lookups use it.)


>>> object.__mro__
    (<class 'object'>,)
               # Given an object whose type is object, the only class that will be searched
               # is object.  This is in line with what we've seen before: object has no base
               # classes, so there's nowhere else to look.
>>> class Thing:
...     pass
...
>>> Thing.__mro__
    (<class '__main__.Thing'>, <class 'object'>)
               # Thing inherits from object, so we'll first check in Thing
               # and, failing that, we'll check in object.
>>> class SpecificThing(Thing):
...     pass
...
>>> SpecificThing.__mro__
    (<class '__main__.SpecificThing'>, <class '__main__.Thing'>, <class 'object'>)
               # Similarly, SpecificThing singly inherits from Thing, so it abides
               # by the same rule.

So, in reality, the attribute lookup x.y follows a slightly different algorithm than we've seen — albeit one with the same overall effect when all we have is single inheritance.

If that's true, then our best route to understanding the effect of multiple inheritance is to understand the MROs of classes with more than one base class. Let's take a look at the MRO for our Derived12 and Derived21 classes.


>>> Derived12.__mro__
    (<class '__main__Derived12'>, <class '__main__.Base1'>, <class '__main__.Base2'>, <class 'object'>)
>>> Derived21.__mro__
    (<class '__main__Derived21'>, <class '__main__.Base2'>, <class '__main__.Base1'>, <class 'object'>)

These orderings are exactly why we saw that calling both on a Derived12 object calls Base1.both, while calling both on a Derived21 object calls Base2.both instead. Whichever class is listed first in the MRO determines where an attribute will be found, given a choice between them.

When Python determines the MRO for a class, it's looking for an ordering of all of the related classes — the class itself, all of its base classes, all of their base classes, and so on — that adheres to two restrictions.

An ordering of classes that meets those two requirements is known as a linearization. You can verify that the MROs for Derived12 and Derived21 are valid linearizations, according to these rules.

If you find yourself curious about precisely how the MRO is calculated for classes like Derived12, the algorithm that determines it — which is known as the C3 algorithm and has been a part of Python for many years — is explained at the link below. (The details are beyond the scope of this course, so don't feel like you need to study this algorithm in depth, but do follow your curiosity if it leads you there.)

There's one more thing to be aware of: It's possible to define classes for which there is no linearization possible, which can lead to a rather perplexing error message.


>>> class MoreDerived(Derived12, Derived21):
...     pass
...
    Traceback (most recent call last):
      ...
    TypeError: Cannot create a consistent method resolution
    order (MRO) for bases Base1, Base2

In this case, the complaint revolves around Derived12 needing Base1 to be listed before Base2, while Derived21 needs Base2 to be listed before Base1. Both of these things can't be true simultaneously, so the C3 algorithm fails to find a linearization and we're unable to define a class that doesn't have one. Using inheritance only when it's warranted and avoiding its overuse is a good way to avoid a problem like this — complex inheritance hierarchies riddled with multiple inheritance are usually going to be difficult for people to understand, anyway — but it's worth knowing that this can happen, in case you stumble upon it.

How exceptions and inheritance are related

When I teach ICS 32 or ICS H32, I often show students how to create their own new kinds of exceptions, without much explanation of the underlying details. We usually do something like this.


class CustomError(Exception):
    pass

We now know a lot more about what this statement means: CustomError singly inherits from the built-in class Exception, which means that CustomError automatically inherits the common machinery that exceptions have, such as the ability to carry tracebacks and error messages. That's one reason we use inheritance when defining custom exception classes, a classic example of reusing functionality that we'd otherwise have to copy and paste.

The other reason we use inheritance when writing exception classes is categorization, which is to say that we group related kinds of errors within the hierarchy of exception classes. For example, if we have a custom error related to an invalid value being passed into a method, we might derive it from ValueError rather than Exception, because that's a more specific way to categorize its meaning. It's not just an error, after all; it's an error related to a value, and that's what ValueError means.

Why that categorization is useful is because of the mechanics of how an except clause in a try statement matches against an exception.


>>> class CustomValueError(ValueError):
...     pass
...
>>> try:
...     raise CustomValueError
... except ValueError:
...     print('Caught a ValueError')
... except CustomValueError:
...     print('Caught a CustomValueError')
...
    Caught a ValueError       # Why wasn't it caught as a CustomValueError?

You've likely seen before that in a try statement with multiple except clauses, the except clauses are matched to an exception in the order specified, with the first match being the only one that catches and handles the exception.

What does it mean to match? It's effectively just like a call to Python's built-in isinstance function, which means that catching ValueError will catch an exception of any class that derives from ValueError (i.e., any class with ValueError in its MRO). This way, we can handle entire categories of errors in one way, rather than handling each variant separately, which is great news if we want them all to be handled in the same way. But this also means that we need to exercise some caution when deciding on the order in which we handle exceptions in a try statement, so that handlers for more specific errors precede handlers for less specific ones, when we want more specific errors to be handled differently from the others within the same category.


>>> class CustomValueError(ValueError):
...     pass
...
>>> try:
...     raise CustomValueError
... except CustomValueError:
...     print('Caught a CustomValueError')
... except ValueError:
...     print('Caught a ValueError')
...
    Caught a CustomValueError