Differences between Python's generators and iterators

Posted by Zetusko on Sun, 19 Jan 2020 05:48:38 +0100

What's the difference between an iterator and a generator? Some examples of when to use each case will help.

#1 building

Iterator:

An iterator is an object that uses the next() method to get the next value of a sequence.

Alternator:

A generator is a function that uses the yield method to generate or generate a sequence of values.

The generator object returned by the generator function, such as the ex: foo() function in the following example, each next() method call on the ex: f next() method in the following example, generates the next value in sequence.

When a generator function is called, it returns the generator object without even starting to execute it. When the next() method is called for the first time, the function starts execution until it reaches the yield statement, which returns the resulting value. Revenue tracking (that is, remember the last execution). The second next () call continues from the previous value.

The following example demonstrates the interaction between yield and the call to the next method on the generator object.

>>> def foo():
...     print "begin"
...     for i in range(3):
...         print "before yield", i
...         yield i
...         print "after yield", i
...     print "end"
...
>>> f = foo()
>>> f.next()
begin
before yield 0            # Control is in for loop
0
>>> f.next()
after yield 0             
before yield 1            # Continue for loop
1
>>> f.next()
after yield 1
before yield 2
2
>>> f.next()
after yield 2
end
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

#2 building

iterator is a more general concept: any object's class has the next method (\\\\\\\\\\\.

Each generator is an iterator, but vice versa. The generator is built by calling a function that has one or more yield expressions (yield statements in Python 2.5 and earlier), and that function is an object that satisfies the previous definition of iterator.

When you need a class with some complex state maintenance behavior, or want to expose methods other than the next method (as well as other methods other than "iter" and "init"), you may want to use a custom iterator instead of a generator. Generally, a generator (sometimes, for a simple enough requirement, a generator expression) is enough, and it's easier to write code, because state maintenance (to a reasonable extent) is basically done for you by suspending and restoring frames.

For example, a generator, for example:

def squares(start, stop):
    for i in range(start, stop):
        yield i * i

generator = squares(a, b)

Or equivalent generator expression (genexp)

generator = (i*i for i in range(a, b))

More code will be needed to build as a custom iterator:

class Squares(object):
    def __init__(self, start, stop):
       self.start = start
       self.stop = stop
    def __iter__(self): return self
    def next(self): # __next__ in Python 3
       if self.start >= self.stop:
           raise StopIteration
       current = self.start * self.start
       self.start += 1
       return current

iterator = Squares(a, b)

But, of course, with the Squares class, you can easily provide other methods, namely

    def current(self):
       return self.start

If you actually need this extra functionality in your application.

#3 building

What's the difference between an iterator and a generator? Some examples of when to use each case will help.

To summarize: an Iterator is an object with the methods' Iterator 'and' next '(next in Python 2). The generator provides a simple built-in way to create an instance of Iterator.

The function containing yield is still a function, and when it is called, it returns an instance of the generator object:

def a_function():
    "when called, returns generator object"
    yield

The builder expression also returns the Builder:

a_generator = (i for i in range(0))

Read on for more in-depth instructions and examples.

Generators are iterators

Specifically, generators are subtypes of iterators.

>>> import collections, types
>>> issubclass(types.GeneratorType, collections.Iterator)
True

We can create generators in several ways. A very common and simple way is to use functions.

Specifically, the function containing yield is a function that returns the generator when it is called:

>>> def a_function():
        "just a function definition with yield in it"
        yield
>>> type(a_function)
<class 'function'>
>>> a_generator = a_function()  # when called
>>> type(a_generator)           # returns a generator
<class 'generator'>

Similarly, generators are iterators:

>>> isinstance(a_generator, collections.Iterator)
True

Iterators are iterative

Iterators are iterative

>>> issubclass(collections.Iterator, collections.Iterable)
True

This requires a return iter ator method:

>>> collections.Iterable()
Traceback (most recent call last):
  File "<pyshell#79>", line 1, in <module>
    collections.Iterable()
TypeError: Can't instantiate abstract class Iterable with abstract methods __iter__

Built in tuples, lists, dictionaries, sets, frozen sets, strings, byte strings, byte arrays, ranges, and memory views are some examples of iteratable objects:

>>> all(isinstance(element, collections.Iterable) for element in (
        (), [], {}, set(), frozenset(), '', b'', bytearray(), range(0), memoryview(b'')))
True

Iterator requires next or next method

In Python 2:

>>> collections.Iterator()
Traceback (most recent call last):
  File "<pyshell#80>", line 1, in <module>
    collections.Iterator()
TypeError: Can't instantiate abstract class Iterator with abstract methods next

In Python 3:

>>> collections.Iterator()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Iterator with abstract methods __next__

We can use the iter function to get iterators from built-in objects (or custom objects):

>>> all(isinstance(iter(element), collections.Iterator) for element in (
        (), [], {}, set(), frozenset(), '', b'', bytearray(), range(0), memoryview(b'')))
True

When you try to use an object with a for loop, the \\\\\\\\. The next method is then called on the iterator object so that each item enters a loop. When exhausted, the iterator raises a StopIteration and cannot be reused at this time.

From the document

In "built in type" File In the generator types section of the iterator types section of:

Python's generator provides a convenient way to implement the iterator protocol. If you implement the container object's uiter uuu; () method as a generator, it will automatically return the iterator object (technically, the generator object) that provides the uiter u; () and the u next u; () in the [Python 3] methods. More information about the generator can be found in the document for the yield expression.

(emphasis added.)

So we learned that generators are (handy) iterator types.

Example iterator object

You can create objects that implement the Iterator protocol by creating or extending your own objects.

class Yes(collections.Iterator):

    def __init__(self, stop):
        self.x = 0
        self.stop = stop

    def __iter__(self):
        return self

    def next(self):
        if self.x < self.stop:
            self.x += 1
            return 'yes'
        else:
            # Iterators must raise when done, else considered broken
            raise StopIteration

    __next__ = next # Python 3 compatibility

However, using the Generator to do this is easier:

def yes(stop):
    for _ in range(stop):
        yield 'yes'

Perhaps simpler, a generator expression (similar to list derivation):

yes_expr = ('yes' for _ in range(stop))

They can all be used in the same way:

>>> stop = 4             
>>> for i, y1, y2, y3 in zip(range(stop), Yes(stop), yes(stop), 
                             ('yes' for _ in range(stop))):
...     print('{0}: {1} == {2} == {3}'.format(i, y1, y2, y3))
...     
0: yes == yes == yes
1: yes == yes == yes
2: yes == yes == yes
3: yes == yes == yes

conclusion

When you need to extend Python objects to iteratable objects, you can use the Iterator protocol directly.

However, in most cases, yield is best used to define functions that return Generator iterators or consider Generator expressions.

Finally, note that the generator provides more collaborator functionality. What role will I play in my "yield" keyword The Generators and yield statements are explained in depth in the answer to.

#4 building

The answer was added because none of the existing answers specifically addressed the confusion in the official literature.

Generator functions are ordinary functions defined using yield rather than return. When called, the generator function returns a generator object, which is an iterator - it has the next() method. When you call next(), the next value generated by the generator function is returned.

Functions or objects can be called "generators," depending on which Python source document you are reading. Python glossary display Generator functions, and Python Wiki Implied generator object. Python tutorial Conspicuously managed to suggest these two uses between two sentences:

Generators are simple and powerful tools for creating iterators. They are written in a similar way to regular functions, but use the yield statement whenever you want to return data. Every time next() is called, the generator continues where it was last interrupted (it remembers all the data values and the last statement executed).

The first two sentences identify generators with generator functions and the third with generator objects.

Despite all this confusion, you can still find Python Language Reference To get clear words:

The yield expression is used only when defining generator functions, and only in the body of the function definition. Using a yield expression in a function definition is enough to make the definition create a generator function instead of a normal function.

When a generator function is called, it returns an iterator called the generator. The generator then controls the execution of generator functions.

Therefore, in formal and precise usage, a "generator" disqualification represents a generator object rather than a generator function.

The above reference is for Python 2, but Python 3 language reference But said the same thing. However, Python 3 Glossary point out

Generator... Usually refers to generator functions, but in some cases may refer to generator iterators. When the expected meaning is not clear, ambiguity can be avoided by using complete terms.

#5 building

Generator function, generator object, generator:

The generator function is like a regular function in Python, but contains one or more yield statements. Generator functions are a great tool for creating Iterator objects as easily as possible. The Iterator object returned through the generator function is also known as the generator object or generator.

In this example, I create a generator function that returns the generator object < generator object FIB at 0x01342480 >. Like other iterators, the generator object can be used in a for loop, or it can be used with the built-in function next(), which returns the next value from the generator.

def fib(max):
    a, b = 0, 1
    for i in range(max):
        yield a
        a, b = b, a + b
print(fib(10))             #<generator object fib at 0x01342480>

for i in fib(10):
    print(i)               # 0 1 1 2 3 5 8 13 21 34


print(next(myfib))         #0
print(next(myfib))         #1
print(next(myfib))         #1
print(next(myfib))         #2

Therefore, the generator function is the easiest way to create an Iterator object.

Iterator:

Each generator object is an iterator, but vice versa. You can create a custom iterator object if its class implements the iterator and next methods (also known as the iterator protocol).

However, using generator functions to create iterators is much easier because they simplify the creation of iterators, but custom iterators give you more freedom, and you can implement other methods as needed, as shown in the following example.

class Fib:
    def __init__(self,max):
        self.current=0
        self.next=1
        self.max=max
        self.count=0

    def __iter__(self):
        return self

    def __next__(self):
        if self.count>self.max:
            raise StopIteration
        else:
            self.current,self.next=self.next,(self.current+self.next)
            self.count+=1
            return self.next-self.current

    def __str__(self):
        return "Generator object"

itobj=Fib(4)
print(itobj)               #Generator object

for i in Fib(4):  
    print(i)               #0 1 1 2

print(next(itobj))         #0
print(next(itobj))         #1
print(next(itobj))         #1

Topics: Python