iterator and generator

Iterable

A iterable is an object which implements the _iter_ method. __iter\ has to implement yield statement.

iterable concepts

python

from collections import abc
from collections import Iterable

class IterableObjc:
    def __iter__(self):
        cont = 0
        while cont < 3:
            cont += 1
            yield cont


a_iterable = IterableObjc()
assert isinstance(a_iterable, abc.Iterable)  # no exception
assert isinstance(a_iterable, Iterable)  # no exception
abc.Iterable is Iterable # collection.Iterable is the same as a_iterable

True

‘for’ syntax is able to be used for iterable.

python

Iterable_instance =  IterableObjc()

python

for i in Iterable_instance:
    print(i)

1
2
3

Iterable could be also understood as any accepted object by iter() function.

_getitem_() is another way to define a Iterable object.

note: StopIteration() exception has to be raised in _getitem_() function, otherwise infinite loop will be triggered when ‘for’ syntax is used!

python

class GetitemObjc:
    def __getitem__(self, item):
        cont = 0
        while cont <= item:
            cont += 1
            if cont >= 5:
                raise StopIteration()
        return cont


a_getitem = GetitemObjc()
assert isinstance(a_getitem, abc.Iterable)  # exception

---------------------------------------------------------------------------

AssertionError                            Traceback (most recent call last)

<ipython-input-4-52f2bca5d0ef> in <module>()
     10 
     11 a_getitem = GetitemObjc()
---> 12 assert isinstance(a_getitem, abc.Iterable)  # exception


AssertionError: 

Code

Getitem_instance =  GetitemObjc()

python

for i in Getitem_instance:
    print(i)

1
2
3
4

‘for’ syntax

Here is how for syntax works.

for syntax is able to call iter() method for an iterable object, which transfers the iterable to a generator, and then call next(generator_object) until the StopIteration() exception is raised.

‘any’ syntax

any syntax checks if the iterable is not empty.

python

any(Iterable_instance)

True

python

any(Getitem_instance)

True

python

class EmptyIterableObjc:
    def __iter__(self):
        cont = 0
        while cont < 0:
            cont += 1
            yield cont

empty_iterable = EmptyIterableObjc()
assert isinstance(empty_iterable, abc.Iterable)  # no exception

This iterable is not able to provide any iteration element, therefore any returns False.

python

any(empty_iterable)

False

‘list’ syntax

list syntax is able to iterate through the iterable object, and add all the elements into a list.

python

list(Iterable_instance)

[1, 2, 3]

Iterator

• Iterable is an object with _iter_ method
• Iterator is an object with both _iter_ and __next\ methods

A class with both iter and next methods is a iterator object. for or next() are able to be called for an iterator.

python

class IteratorObjc:
    def __init__(self):
        self.cont = 0
    def __iter__(self):
        while self.cont < 3:
            self.cont += 1
            yield self.cont

    def __next__(self):
        while self.cont < 3:
            self.cont += 1
            return self.cont
        raise StopIteration()

python

self_defined_generator = IteratorObjc()

python

for i in self_defined_generator:
    print(i)

1
2
3

python

self_defined_generator = IteratorObjc()

python

print(next(self_defined_generator))
print(next(self_defined_generator))
print(next(self_defined_generator))
print(next(self_defined_generator)) #raise stop iteration

1
2
3



---------------------------------------------------------------------------

StopIteration                             Traceback (most recent call last)

<ipython-input-59-042c29e8c989> in <module>()
      2 print(next(self_defined_generator))
      3 print(next(self_defined_generator))
----> 4 print(next(self_defined_generator)) #raise stop iteration


<ipython-input-55-75f8519e3c93> in __next__(self)
     11             self.cont += 1
     12             return self.cont
---> 13         raise StopIteration()
     14 


StopIteration: 

manual implementation of next

An inheritant class from a iterable class, which implements next method will become an iterator.

e.g: str is an iterable object but not an iterator. Here is a inheritant class from str class which implements next method.

python

normal_string = 'abcde'
next(normal_string)

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-60-7a029a4ee3bb> in <module>()
      1 normal_string = 'abcde'
----> 2 next(normal_string)


TypeError: 'str' object is not an iterator

python

class IteratorStr(str):
    def __init__(self, string):
        super().__init__()
        self._count = 0
        
    def __iter__(self):
        self._count = 0
        return self

    def __next__(self):
        while self._count < len(self):
            self._count += 1
            return self.__getitem__(self._count - 1)
        raise StopIteration()

python

iterator_string = IteratorStr('abcde')

python

print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string)) # raise StopIteration() exception

a
b
c
d
e



---------------------------------------------------------------------------

StopIteration                             Traceback (most recent call last)

<ipython-input-83-2fbad4156275> in <module>()
      4 print(next(iterator_string))
      5 print(next(iterator_string))
----> 6 print(next(iterator_string)) # raise StopIteration() exception


<ipython-input-75-a498191acf75> in __next__(self)
     12             self._count += 1
     13             return self.__getitem__(self._count - 1)
---> 14         raise StopIteration()


StopIteration: 

Of course the for syntax is able to be used for an iterator

python

iterator_string = IteratorStr('abcde')
for i in iterator_string:
    print(i)

a
b
c
d
e

auto implementation of next by iter()

python

normal_string = 'abcde'
iterator_string = iter(normal_string)

python

print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string))
print(next(iterator_string)) # raise StopIteration() exception

a
b
c
d
e



---------------------------------------------------------------------------

StopIteration                             Traceback (most recent call last)

<ipython-input-87-2fbad4156275> in <module>()
      4 print(next(iterator_string))
      5 print(next(iterator_string))
----> 6 print(next(iterator_string)) # raise StopIteration() exception


StopIteration: 

python

iterator_string = IteratorStr('abcde')
for i in iterator_string:
    print(i)

a
b
c
d
e

Generator

function with ‘yield’ syntax is called a generator function.

python

def generator_function():
    for i in range(3):
        yield i

A generator function has to be ‘instantiated’. The syntax looks like the instance of the generator function. But actually it is creating a generator from the generator function yield statement.

python

gen = generator_function()

python

type(gen)

generator

python

for i in gen:
    print(i)

0
1
2

A generator will automatically have the _next_ method implemented

python

gen = generator_function()

python

print(next(gen))
print(next(gen))
print(next(gen))
print(next(gen)) # StopIteration() exception raised

0
1
2



---------------------------------------------------------------------------

StopIteration                             Traceback (most recent call last)

<ipython-input-16-648ef4ded63a> in <module>()
      2 print(next(gen))
      3 print(next(gen))
----> 4 print(next(gen)) # StopIteration() exception raised


StopIteration: 

Advance material

Python中iteration(迭代)、iterator(迭代器)、generator(生成器)等相关概念的理解

python