Object Orientation vs Functional Programming in Python

Editor's Notes

• #3 - Software Engineer at DemonWare - Online Services for some of the worlds most popular games, as well game online experiences like COD Elite - Python is one of our core technologies. - We're hiring!
• #4 The sub-title of this talk is writing modular python programs. What do I mean by modularity and why do we want it? I don't want to make too much of a song and dance about it, because I think it is an obvious goal and programming languages have always been trying to help us achieve it. Never the less I think it is useful to at least loosely define it. We want the code we write to be individual components that do one job. That allows us to recombine those bits in different ways to solve new problems. We can build new stuff, making use of what we built before repeating ourselves as little as possible. To achieve this there are some properties our code needs to have. If there is an idea/concern we dealing with we would like that be expressed in as few places as possible ideally one. Conversely if we have a single module/class/method it would be better if it dealt with one concern rather than being cluttered with many different concerns. The diagrams on the screen try to illustrate this point. If each colour represents a particular concern we see that is scattered all over the place, if you were assigned the task of modifying the functionality represented by the yellow bits that would be a pretty annoying, grep helps but we would rather not be in that position in the first place. It is similarly problematic to have modules like we have on the left, in which lots of concerns are tangled together. One has to be aware of all the concerns in the module to change just one of them, or run the risk of inadvertently breaking other stuff that just happens to be near by. If you have ever had to say but I only changed the comments you know what I am talking about. Instead what we want is what we have on the right. Modules that related but separate, with each module dealing with a particular concern. If we can achieve that we can better reuse, re-purpose and extend our code to solve new problems. This is the open-closed principle ( should be open for extension, but closed for modification ). We should only modify a component when it's intrinsic behaviour changes, all other extension ideally should be made from the outside.   I think most modern programming languages provide features that help us achieve that goal. They succeed to varying degrees, we are going to look at some of the ways Python helps us do this.
• #5 - I am not a classical rhetorician, but I aspire to play one on TV. The structure of classical rhetorical argument requires us to present the thesis, then explore the antithesis, and finally present the synthesis   - We are going take an example based approach, taking a look at a few Python OO and FP features that exemplify the point I am trying to make.   - Some of the examples themselves are taken mostly taken from the Python standard library, not so much the internals buy how we are supposed to use it. This is useful because the standard library (or any library for that matter) necessarily needs to extensible and reusable from the outside without modifying the library itself.
• #6 Python OO, it is class oriented like most OO languages there are so exceptions that are prototype based a notable one being javascript, but are some others like self. - So OO is packaging state and behaviour generally to achive 3 things: 1. delegation which explicitly or implicitly passing on the responsibility of performing an action to somewhere else. implicit when you lookup an attribute: instance -> class class -> base classes explicit: delegating to another object that we have a reference to or even explicit delegation delegation to the object itself 2. Polymorphism allowing objects to act like other objects depending on the context. 3. Instantiation that is essentially having one blueprint that can be used to create many instances. - That is really all I am going to say about OO in general, I am kind of assuming we are all familiar with the basics of OO in Python.
• #7 - base class offers "organizing method" which calls "hook methods" - in base class, most hook methods stay abstract - concrete subclasses implement the hooks - client code calls organizing method - A couple of things to note the Chess class does not specialise the winner printing; Data overriding something we cad do in python but not in many other languages   Mention how this structure would be hard to recreate with functions only, the obviousness of what is required is great.
• #8 The methods on a dict do not necessarily call the methods you would expect. The reason for this is that python needs to protect the internal invariants of collections to prevent crashes. So the internal calling patterns are not obvious. ABC declare these constraints in an obvious way. This is really a variant of the template method design pattern with the sort of checks you would normally get in a staticly typed language. This still of course happens at runtime, but the failure is early, obvious and descriptive. Better still using an abc allows your code to move with the language. Because abc define the minimum set of the methods for each type. All other methods can be implemented in terms of these. You can override more methods to do things more efficiently, but if you don't things will still work.   The abc's really provide structured modularity and extendibility. We can extend them in ways that are sure to be consistent with intention of the base class.
• #9 This can be thought of a form of the template method design pattern. Except the organising methods come from one or more base classes.   Inheriting from a mixin is not a form of specialization but is rather a means of collecting functionality. A class may inherit most or all of its functionality from one or more mixins through multiple inheritance .   Diamond problem: Python creates a list of a classes using the C3 linearization algorithm. That algorithm enforces two constraints: children precede their parents and if a class inherits from multiple classes, they are kept in the order specified in the tuple of base classes. Thus, the method resolution order is: D, B, C, A
• #10 No discussion of multiple inheritance is complete without mentioning the "Diamon Problem"   The diamond problem such as it is, is In object-oriented programming languages with multiple inheritance and knowledge organization , the diamond problem is an ambiguity that arises when two classes B and C inherit from A, and class D inherits from both B and C. If a method in D calls a method defined in A (and does not override the method), and B and C have overridden that method differently, then from which class does it inherit: B, or C? A well designed language needs to have a deterministic resolution to this ambiguity. Python creates a list of a classes using the C3 linearization algorithm. That algorithm enforces two constraints: children precede their parents and if a class inherits from multiple classes, they are kept in the order specified in the tuple of base classes. Thus, the method resolution order is: D, B, C, A   Personally I think it is a mistake to rely on this order even it is deterministic. This situation can avoided in most cases just by taking care what you inherit from. Since the main reason for mixins is to get functionality defined in another class if inheritance is causing problems it may be time to consider composition instead.
• #11 The design patterns literature. talks about preferring composition over inheritance. There is a good reason for this, for one thing inheritance always  expands the interface of the subclass with all of the parents attributes this may not always be what you want, especially if the base class is just a service that you may want to swap out in the in end. Inheritance cannot restrict it can only expand. This is a general mechanism but specific things you may be trying to achieve is to adapt or proxy some object. In all these cases though the general mechanism is the same. It is useful to distinguish between them (as is done in design patterns literature) to help in deciding when to apply the technique. However at the level of abstraction we are working today
• #12 The design patterns literature. talks about preferring composition over inheritance. There is a good reason for this, for one thing inheritance always  expands the interface of the subclass with all of the parents attributes this may not always be what you want, especially if the base class is just a service that you may want to swap out in the in end. Inheritance cannot restrict it can only expand. This is a general mechanism but specific things you may be trying to achieve is to adapt or proxy some object. In this example say had a lot of code that knows how to deal with file like objects (perhaps it uses context lib closing()This is a way to reuse code in a context that it would not otherwise be usable, by creating a very thin wrapper.
• #13 python def __getattr__(self, n) magic really make life easy for us, how does this help modularity well we only need to specify what we care about and the whole world making this a lot easier to change
• #14 python def __getattr__(self, n) magic really make life easy for us, how does this help modularity well we only need to specify what we care about and the whole world making this a lot easier to change
• #15 - I am not a classical rhetorician, but I aspire to play one on TV. The structure of classical rhetorical argument requires us to present the thesis, then explore the antithesis, and finally present the synthesis   - We are going take an example based approach, taking a look at a few Python OO and FP features that exemplify the point I am trying to make.   - The examples themselves are taken mostly taken from the Python standard library, not so much the internals buy how we are supposed to use it. This is useful because the standard library (or any library for that matter) necessarily needs to extensible and reusable from the outside without modifying the library itself.
• #16 - In functional programming Functions are the primary means of abstraction. Programs are decomposed into a set of functions, that take some input and produce some output.   - Ideally a function should compute outputs from its inputs and do nothing else. That is takes values and produces new values. No mutable shareable state should be modified as the result of calling a function. Pure functional languages are strict about side effect freeness. (The type systems in such languages are used to control the limited amount of necessary side effects).   - This purity is what gives functional languages some of advantages you sometime here mentioned like provable program correctness. Also since side effects are limited or none existent, functional languages are very useful for concurrent programming.   - Python is not a pure functional language, because the language does permit side effects. Assignment of values to names which is not possible in pure functional languages is commonplace in Python program.    - Rather It is a language the supports functional style programming. The main thing that makes this possible is the fact that the language supports higher order functions. That is functions are first class objects. So we can functions that accept other functions as inputs and even return functions as outputs.   - Never the less we can get pretty for with the less draconian functional support we have in python.
• #17 One of the first examples most people encounter is programming python in a function style is callbacks. Before we take a look at some examples to make it more concrete let briefly discuss the concept of a callback. for: customization (flexibility)    "event-driven" architectures ("actual" events such as in GUI or network programming OR "pseudo" events for structuring of control-flow, a common example being a SAX XML parser.   This style of programming really begs for first class callables. The same effect can be achieved in other languages using interfaces, or functor objects or anonymous classes and so on, just to get around the lack of higher order functions.
• #18 Mention how simple this is to achieve because we are not encumbered by unnecessary OO infrastructure.   Imagine create a subclass of list with different sorting, or subclasses of the objects you are comparing to implement __eq__ and __lt__, this would result in a combinatorial explosion of different subclasses to do sort by different combinations of fields.
• #19 The sorted function is a higher order function, it accepts a function as its key argument and uses that key to customise how it does the sorting, that is rather than using the natural order of the class as defined by lt and eq.   Notice how much more flexible this is you can easily switch between sorting mechanisms. More importantly the users of the person class can easily sort this object in ways not envisaged by the creator of the class without having to modify or subclass the class itself. Remember our modularity goals, see how that fits in here? Some creates a class we can customise it behaviour from the outside.
• #20 why write your own functions when all of pythons operators are available to you as functions itemgetter and attrgetter are examples of higher order functions, they return functions (or callables) as their results.   sorted requires a function that it can call we wrote a simple one for ourselves but this code was basically boilerplate that would have to be repeated. The operators in python can really be viewed as functions, they take operands (arguments) and produce results. There are many operators in Python which would be valuable viewed in this way.    The operator module does just that. The . operator is no execption and we can use it in our previous example. This is a higher order function its result is a function the extracts the field of the specified name from any object passed in. These and many more are available, they really come into their own in combination with what we are going to discuss in the next section.
• #21 A powerful feature of functional programs is the ability to declare that we would like to do something to a set of objects, what is that something? Well we can pass that as an argument. This simple idea can lead to profound insights. Google MapReduce is inspired by this functional view of the world allow operations to be parallelzed then the result collated later. Since the operations on each object a separate this can be done transparently to the user. The process of iteration itself can be further customised through the use of filter. This may seem like a lot of trouble to go to just to iterate over a bunch of objects, but this pattern comes up so often that it is more than worth it.
• #22 The idea of looping over a collection of objects and doing a customisable something to them is so fundamental to functional programming in python that there is whole module dedicated to that cause. It contains many useful variations on that theme. cycle() - infinitely cycle through an iterable repeat() - repeats an element a specified number of times chain()    - chains iterators together so they be iterated over as one sequnce groupby() - returns sub-iterators grouped by value of keyfunc(v) tee() - splits one iterator into n product() - cartesian product, equivalent to a nested for-loop
• #23 Syntactic sugar for higher order functions. No real difference but the way the clearly indicate the intentions make it much much nicer to use higher order functions. I think it is fair to say that most peoples use of higher order functions in python is via decorators. Decorators really cause an explosion of this sort of higher order functions in the wild and of course the standard library has many useful decorators built in. Do not write code like this it is no only a bad caching decorator (that contains a memory leak), it is unnecessary since python 2.7 it is in the standard library.
• #24 Allows you to define functions in terms of existing ones by freezing some of the arguments and keyword arguments. This avoids repetition of those parameters all over the code. This is another of the problems we discussed at the beginning, scattering the knowledge about a particular concern all over the code. This infact a rather more subtle version, it is not obvious in the heat of to see how having many function call for which a subset of the arguments must always be the same is modularity and maintenance problem. But thinking about it for a moment you can see that it really is.  functools partial allows you to avoid this without having to write trivial wrapper functions which people tend not to do anyway.   You may also here it referred to as currying , but if I call it currying people who really know about functional programming will come and beat me up because apparently currying is something different.
• #25 - I am not a classical rhetorician, but I aspire to play one on TV. The structure of classical rhetorical argument requires us to present the thesis, then explore the antithesis, and finally present the synthesis   - We are going take an example based approach, taking a look at a few Python OO and FP features that exemplify the point I am trying to make.   - The examples themselves are taken mostly taken from the Python standard library, not so much the internals buy how we are supposed to use it. This is useful because the standard library (or any library for that matter) necessarily needs to extensible and reusable from the outside without modifying the library itself.
• #26 We have seen how python's oo features allow us to express well the structural relationships between object and better communicate the constraints we want to impose as in the abc example. We have also seen how the functional approach makes it easy to express performing different operations over a given set of objects and how higher order functions allow us to easily customise those operations.   I could end the talk right here and legitimately claim that I have shown what I set to. However the story only gets better from here the OO and functional aspects of python are not separate worlds that are not intended to meet. Quite the contrary in Python the two approaches complement each other nicely. We are now approaching our foreshadowed conclusion that the design of Python takes both the OO and functional view into account. This section contains a selection of examples that demonstrate this multi-paradigm  approach to programming that Python supports and how it allows for really clean solutions.
• #27 The functional style of programming relies heavily on passing functions around, than applying them to objects later (using itertools or some of the other callback we talked about previously. Functions are descriptors Override binding behaviour Override differently for A.x and a.x Unbound methods know their class but not their instance Ideal for use in functional scenarios where the target object is specified directly.
• #28 Using the property higher order function to bring new functionality in an OO context. Avoids the unnecessary getter setter desease in Java other languages. C# does infact have properties because it is designed by Anders so has influence from Delphi The decorators are just sugar for the property higher order function.
• #29 property and inheritance there is an unintentional pun there.   Property results in early binding. The property has a handle to the specific function object passed into it when it was created. It cannot know about the methods in the derived class. We talked about python helping smooth the impedance mismatch between OO and FP well this is a case where the abstraction leaks a bit. Luckily the fix is not that complicated. A layer of indirection solved every problem in computer science. I do not think that is quite true it solves many problems though and this is one of them.
• #30 Do not scatter dependencies on specific classed throughout your  code this makes it harder to change implementations down the line    I know what you thinking I can use grep,    but if you now need to do some checking and perhaps initialise the object a little differently based on some previously on unforeseen factors this is now going to be much more of a pain. Invert the dependency, use a factory instead. Of course we all do this every time we are instantiating a class right. Modularity again we want to be able to change things in future without too much pain.
• #31 What I really mean is that python classes are callables. Python classes are callables, like functions and methods Indistinguishable from other callables to the caller, for the most part of course via introspection you can tell the difference but in typical cases they are the same Allow us to postpone the creation of a factory until it actually needed. this is similar to how property
• #32 One of the reasons python is so suitable for programming in a functional style is that there are several different types of callables, which are mostly inter changeable. Notice how the documentation for what I have been calling higher order functions, in the operator module such as itemgetter attrgetter and method caller say they return callables not functions. If they are not functions what else could they be? Class based callables can be handy if you want a callable that has internal state that is uses when called.
• #33 - I am not a classical rhetorician, but I aspire to play one on TV. The structure of classical rhetorical argument requires us to present the thesis, then explore the antithesis, and finally present the synthesis   - We are going take an example based approach, taking a look at a few Python OO and FP features that exemplify the point I am trying to make.   - The examples themselves are taken mostly taken from the Python standard library, not so much the internals buy how we are supposed to use it. This is useful because the standard library (or any library for that matter) necessarily needs to extensible and reusable from the outside without modifying the library itself.