class Duck:
    """Ducks

    :param color: Duck color
    """

    def __init__(self, color):
        self.color = color

    def quack(self):
        print(f"Hi, I'm a {self.color} duck!")


def create_random_ducks(number):
    """Create a batch of random color ducks

    :param number: Number of ducks to create
    """
    ducks = []
    for _ in number:
        color = random.choice(['yellow', 'white', 'gray'])
        ducks.append(Duck(color=color))
    return ducks

from typing import List


class Duck:
    def __init__(self, color: str): 
        self.color = color

    def quack(self) -> None: 
        print(f"Hi, I'm a {self.color} duck!")


def create_random_ducks(number: int) -> List[Duck]: 
    ducks: List[Duck] = [] 
    for _ in number:
        color = random.choice(['yellow', 'white', 'gray']) 
        ducks.append(Duck(color=color))
    return ducks

Hacker News (hereinafter referred to as HN) is a well-known foreign science and technology information site, which is very popular in the programmer circle. On the homepage of HN, you can read the current popular articles and participate in the discussion. At the same time, you can also submit new article links to the home page. The system will sort the articles according to the scoring algorithm, and the most popular articles will be ranked first. The screenshot of HN home page is shown in Figure 10-1.

I usually like to visit HN. I often go to it to find some popular articles. But every time I visit HN, I need to open my browser and find website bookmarks in my favorites. The steps are still very cumbersome - programmers are "lazy"!

In order to make browsing HN more convenient, I want to write a program to automatically obtain the title and links of the most popular items in the HN home page and save them in ordinary files. Isn't it beautiful that I can browse popular articles directly on the command line?

As a Python programmer, it's easy to write a small script. utilize   requests,lxml   Such as the powerful functions provided by the module, I wrote the program in less than half an hour.

By executing this script, I can see the Top5 entry on the HN site on the command line:

You can clearly see that the above code conforms to the object-oriented style. Because in the code, I define two classes as follows:

1. Post: represents an HN content entry, including fields such as title and link. It is a typical "data class", which is mainly used to connect the "data capture" and "file write" behaviors of the program.

2. HNTopPostsSpider: a crawler class that grabs HN content, including page grabbing, parsing, result writing and other behaviors. It is a class that completes the main work.

Although the script is written in an object-oriented style (in other words, it defines only a few class es), it can meet my needs. However, from a design perspective, it violates the first of the SOLID principles: "Single responsibility principle" (hereinafter referred to as SRP). Let's see why.

The single responsibility principle is the first article in the SOLID principle, which holds that a class should have only one reason to be modified. In other words, each class should assume only one responsibility.

To understand the SRP principle, the most important thing is to understand what the "reason for modification" in the principle represents. Obviously, the software itself is lifeless, and the reason for modification will not come from the software itself. Your program won't suddenly jump up and say, "I think I'm a little slow and need to be optimized.".

All the reasons for modifying the software come from the people related to the software. People are the "culprit" who causes the program to be modified.

For example, in the above crawler script, you can easily find two that need to be modified   HNTopPostsSpider   Class reasons.

• Reason 1: the programmer of HN website suddenly updated the page style, and the old xpath parsing algorithm could not parse the new page normally, so   fetch()   The parsing logic in the method needs to be modified.

• Reason 2: the user of the program (that is, me) thinks the plain text format is not good-looking and wants to change it to Markdown style, so   write_to_file()   The output logic in the method needs to be modified.

From these two reasons, HNTopPostsSpider   It obviously violates the SRP principle. It undertakes two completely different responsibilities of "grabbing the post list" and "writing the post list to file".

If a class violates the SRP principle, we will often modify it for different reasons, which may lead to the interaction between different functions. For example, one day I adjusted the page parsing logic in order to adapt to the new style of Hacker News Site, but found that the contents of the output file were also destroyed.

In addition, the more responsibilities a single class undertakes, the more complex and difficult it is to maintain. In the object-oriented field, there is a notorious class: God Class. God Class refers to those classes that contain too many responsibilities, have too much code and can do anything. God Class is a nightmare for all programmers. After encountering God Class, every rational programmer always runs away with his first thought. The farther he runs, the better.

Finally, classes that violate SRP principles are also difficult to reuse. If I want to write another HN related script now, I need to reuse it   HNTopPostsSpider   Class. I'll find that I can't do it at all, because I have to provide an inexplicable file object to you   HNTopPostsSpider   Class.

There are a lot of disadvantages of violating the SRP principle, so how to modify the script to make it comply with the SRP principle? There are many methods, among which the most traditional is to divide the large category into small categories.

To make   HNTopPostsSpider   The responsibility of class becomes more pure. I split the content related to "write file" into a new class: PostsWriter

Then, for   HNTopPostsSpider   Class, I just put   write_to_file()   Method is deleted so that it only remains   fetch()   method.

After this modification, hntoppostsspider   and   PostsWriter   Each class conforms to the principle of single responsibility. Only when the parsing logic changes will I modify it   HNTopPostsSpider   Class, similarly, modify   PostsWriter   Class has only one reason to adjust the output format.

These two classes can be modified separately without affecting each other.

Finally, since both classes are now responsible for only one thing, I need a new role to concatenate their work, so I implemented a new function   get_hn_top_posts():

New functions by combining   HNTopPostsSpider   And   PostsWriter   Class, completed the main work.

>>> l = [5, 3, 2, 4, 1]
>>> sorted(l)
[1, 2, 3, 4, 5]

>>> l = [8, 1, 9]
>>> sorted(l, key=lambda i: i % 3) 
[9, 1, 8]

Three days have passed since the last time I modified the Hacker News crawler script with "single responsibility". In these three days, I found that although the script can quickly retrieve the content and is very convenient to use, in most cases, the content retrieved by the script is not what I want to see.

The current version of the script will capture all the popular items regardless of their source, but in fact, I am only interested in the content from specific sites, such as GitHub.

Therefore, I need to make a little modification to the script. I need to modify it   HNTopPostsSpider   Class to filter the results.

Soon, the code was modified:

Next, let me briefly test the modified effect:

It seems that the newly written filter code works, and now only when the content item comes from   github.com   Is written to the result.

However, as the Greek philosopher Heraclitus said: the only constant in the world is the change itself. Within a few days, my interest changed. I suddenly felt that except GitHub, I came from Bloomberg[ 2 ]   So I have to add a new domain name to the script's filtering logic: bloomberg.com.

Then I found that in order to increase   bloomberg.com, I have to modify the existing   HNTopPostsSpider   Class code, adjust that line   if parsed_link.netloc == 'github.com'   Judgment sentences can achieve my goal.

Remember what the OCP principle says? "Classes should change their behavior by extension rather than modification". According to this definition, the current code obviously violates the OCP principle, because I have to modify the class code to adjust the domain name filtering conditions.

So, how can we make the class conform to the OCP principle so that the behavior can be adjusted without changing the code? The first way is to use inheritance.

Inheritance is an important concept in object-oriented programming. It provides powerful code reuse ability.

There is an important connection between inheritance and OCP principles. Inheritance allows us to extend the behavior of programs by adding new subclasses instead of modifying the original classes, which is just in line with OCP principles. To achieve effective expansion, the key point is to find the unstable and changeable contents in the parent class first. Only by encapsulating these changes into methods (or properties) Subclasses can override this behavior through inheritance.

The topic goes back to my crawler script. In the current demand scenario, HNTopPostsSpider   The unstable logic that will change in the class is actually the part of "whether users are interested in items" (who makes me think one idea a day?).

Therefore, I can extract this part of logic and refine it into a new method:

With such a structure, if I only care about   github.com   So I just need to define an inheritance   HNTopPostsSpider   And then override the parent class   interested_in_post()   Method.

If one day, my interest changes? It doesn't matter. You don't have to modify the old code, just add a new subclass:

In this framework, as long as the change of requirements is related to "whether I am interested in the item", I do not need to modify the original   HNTopPostsSpider   Parent class, I just need to create new subclasses based on it. Through inheritance, I finally realized what the OCP principle says: open to extension and close to change, as shown in Figure 10-2.

Although inheritance is powerful, it is not the only way to the OCP principle. In addition to inheritance, we can also use another idea: composition. More specifically, we can use dependency injection technology based on composition.

Different from inheritance, dependency injection allows us to inject the changeable part of business logic (often referred to as "algorithm") into the object through initialization parameters when creating the object, and finally use polymorphism to achieve the effect of "extending the class without changing the code".

As previously analyzed, in this script, "item filtering algorithm" is a changeable part of business logic. To implement dependency injection, we need to model the filtering algorithm first.

First, a file named   PostFilter   Abstract classes for:

Then, in order to implement the original logic of the script: no entries are filtered. We have created a default algorithm class that inherits this abstract class: DefaultPostFilter. Its filtering logic is to retain all results.

To implement dependency injection, HNTopPostsSpider   Class also needs to make some adjustments. It must receive a file named   post_filter   Result filter object for:

As the code indicates, when I don't provide   post_filter   Parameter, HNTopPostsSpider.fetch()   All results will be retained without any filtering. If the requirements change, the current filtering logic needs to be modified. Then I just need to create a new one   PostFilter   Class.

Here are two ways to filter GitHub and Bloomberg respectively   PostFilter   Class:

Creating   HNTopPostsSpider   Object, I can choose to pass in different filter objects to meet different filtering requirements:

By abstracting and refining the filter algorithm, and combining polymorphism and dependency injection technology, I also make the code comply with the OCP principle.

In addition to inheritance and dependency injection, there is another commonly used way to implement OCP principles: data-driven. The core idea of data-driven is to extract the frequently changing parts in the form of data. When the demand changes, only the data can be changed, and the code logic can remain unchanged.

It sounds a bit like data-driven and dependency injection. They both pull the changing things out of the class. The difference between the two is that dependency injection is usually extracted from classes, while data-driven extraction is pure data.

Next, let's try the data-driven scheme in the script.

The first step in transforming to data-driven is to define the data format. In this requirement, the changed part is "the site address I am interested in", so I can simply use a string list   filter_by_hosts: [List[str]]   To refer to this address.

Here are the modified   HNTopPostsSpider   Class code:

Modified   HNTopPostsSpider   Class, its caller also needs to adjust. Creating   HNTopPostsSpider   When creating an instance, I have to pass in the list of sites to be filtered:

After that, whenever I need to adjust the filtering site, just modify it   hosts   List without adjustment   HNTopPostsSpider   Class. This data-driven approach also meets the requirements of OCP principles.

Compared with the previous inheritance and dependency injection, data-driven code is obviously more concise because it does not need to define any additional classes.

But data-driven also has a disadvantage: its customizability is not as good as the other two methods. For example, if I want to filter by "whether the link ends in a string", the current data-driven code can't.

The fundamental reason affecting the customizability of each scheme is that the abstraction level of each scheme is different. For example, in the dependency injection scheme, the abstract content I choose is "item filtering behavior", while in the data-driven scheme, the abstract content is "effective site address of item filtering behavior". Obviously, the latter has a lower level and pays more attention to specific content, so the flexibility is naturally not as flexible as the former.

In daily work, if you want to write code that conforms to OCP principles, there are many different processing methods in addition to inheritance, dependency injection and data-driven demonstrated here. Each method has its own advantages and disadvantages. You need to deeply analyze the specific demand scenarios to determine which method is the most suitable. This is a process that cannot be achieved overnight and requires a lot of practice.

1. Single responsibility principle (SRP)

   • The SRP principle holds that a class should only have one reason to be modified

   • Writing smaller classes is often less likely to violate SRP principles

   • The SRP principle also applies to functions. You can make functions and classes work together

2. Open close principle (OCP)

   • The OCP principle holds that classes should be closed to changes and open to extensions

   • By analyzing the requirements and finding the changeable parts of the code, it is the key to make the class conform to the OCP principle

   • Subclass inheritance can make classes conform to OCP principles

   • Through algorithm class and dependency injection, you can also make the class conform to OCP principles

   • Separating data from logic and using data-driven method is also a good way to comply with OCP principles