Python : Yield Keyword & Generators explained with examples

In this article we will discuss what’s the use of yield keyword, What are generators and how to Iterate over Generator objects.

Yield Keyword

In Python inside a function instead of using return keyword, we can use yield keyword to return the value. But unlike return keyword, the yield keyword do not terminates the function, it just pauses the function by saving its current state like last executed line number, variables on stack and then it returns the yielded value. So, now next time when this function will be called, then it will start from the point it left off. If there is no value to yield and function reaches its end then StopIteration is raised. Otherwise if any another yield is encountered then it will again pause the function execution and returns the yielded value.

Let’s understand by an example,

A function can yield multiple values at different locations inside a function i.e.

def primeNumbers():
    ''' A function that will yield 2 different values at different places'''
    print('step 1')
    # Yield a value
    yield 5
    
    print('step 2')
    # Yield a value
    yield 7

Generator Function & yield keyword

The function that uses yield keyword is called Generator function. Because if we call this function it returns a Generator object ( which internally has control over the generating function)

# Get a generator object
generator = primeNumbers()

# Print Generator object
print(generator)

Output:

<generator object primeNumbers at 0x03A19AB0>

A generator object is a kind of Iterator, on which we can call the next() function to fetch the next yielded value from the associated function i.e.

# Get a generator object
generator = primeNumbers()

# Get First element
num = next(generator)
print('First Element : ', num)

Now when first time we call the next() function on this Generator object, then it calls the associated generator function ( primeNumbers() in our case). primeNumbers() function is executed from start. As soon as it encounters a yield statement, execution is paused and yielded value is returned i.e.

step 1
First Element :  5

Now when second time we call the next() function on Generator object, then it calls the primeNumbers() again. But this time primeNumbers() function starts from the point where it was last paused. As soon as it encounters another yield statement, it pauses the execution and returns the value i.e

# Get Second element
num = next(generator)
print('Second Element : ', num)

Output:

step 2
Second Element :  7

Similarly when we call the next() function third time then function primeNumbers() continues from the last paused location. Then it continues till it encounters any yield statement. As there is no more yield statement and function end is reached, therefore StopIteration is raised i.e.

# Get third element
num = next(generator)
print('Third Element : ', num)

Output:

Traceback (most recent call last):
....../Python/Iterator_generator/gen_1.py", line 90, in <module>
....../Python/Iterator_generator/gen_1.py", line 77, in main
num = next(generator)
StopIteration

As no more yield statement was found and function terminated, therefore it raised the StopIteration.

Iterate over Generator object Using while loop & next() function

Now instead of calling next() function on generator object one by one, we can use while loop too i.e.

# Get a generator object
generator = primeNumbers()

while True:
    try:
        # Get next value from Generator object
        num = next(generator)
        print(num)
    except StopIteration:
        break

Output:

step 1
5
step 2
7

Using generator with For loop

A generator function yields multiple values and returns an Generator object. We can Iterate over this yielded values of generator function by
using the returned Generator object. This Generator object is an Iterator and can be used at any place where Iterator can be used like for loop.

As primeNumbers() returns a Generator, so we can for loop over this Generator object to fetch all the yielded values
from generator function primeNumbers() one by one i.e.

# Iterate over all the yielded values one by one
for num in primeNumbers():
    print(num)

Output:

step 1
5
step 2
7

for loop internally calls the next() function on returned Generator object and also handles the StopIteration.

Another example:

Let’s create a function that accepts a string and yields every alternate character in the given string i.e.

def alternateChar(message):
    ''' A Generator Function that yields characters at ind '''
    for i in range(len(message)):
        if i % 2 == 0:
            yield message[i]

This function returns a Generator object and we can use this generate object to iterate over every alternate character of the given string i.e.

for elem in alternateChar('Hello Sample Code'):
    print(elem)

Output:

H
l
o
S
m
l
 
o
e

Now we hope that you understand what is the use of yield keyword, Generator Function & Generator objects.

But why do we need Generators ?

Memory Efficient

Generators help us implement memory efficient solutions. Many times we want to do some processing on a collection of elements but on one element at a time. For such scenarios don’t need to create huge list or set, we can use generators to iterate over the elements one at a time.

Infinite Generators

Many times we want to generate stuff continuously, like continuous monitoring of memory usage etc. For these kind of purposes we can create Infinite Generators. Infinite generator is a kind of generator that will keep on returning values and never raises a Stop Iteration.
For example, we want a generator that should keep on returning values from 0 to 4 in a cycle i.e.

def numbersInCycle():
    ''' An Infinite Generator Function,
    it yields numbers 0 to 4 in a cycle and never raises Stop Iteration '''
    i = -1
    while i < 5:
        if i == 4 :
            i = -1
        i = i + 1
        yield i

Now we loop through this generator object then it will continously return values and never raises StopIteration i.e.

for elem in numbersInCycle():
    print(elem)

Output

0
1
2
3
4
0
1
.....

It will keep on printing numbers 0 to 4 in a loop.

Complete example is as follows:

def numbersInCycle():
    ''' An Infinite Generator Function,
    it yields numbers 0 to 4 in a cycle and never raises Stop Iteration '''
    i = -1
    while i < 5:
        if i == 4 :
            i = -1
        i = i + 1
        yield i


def alternateChar(message):
    ''' A Generator Function that yields characters at ind '''
    for i in range(len(message)):
        if i % 2 == 0:
            yield message[i]


def primeNumbers():
    ''' A function that will yield 2 different values at different places'''
    print('step 1')
    # Yield a value
    yield 5

    print('step 2')
    # Yield a value
    yield 7


def main():

    # Get a generator object
    generator = primeNumbers()

    # Print Generator object
    print(generator)

    # Get First element
    num = next(generator)
    print('First Element : ', num)

    # Get Second element
    num = next(generator)
    print('Second Element : ', num)

    # Get third element
    #num = next(generator)
    #print('Third Element : ', num)


    print('*** Iterate over Generator object Using while loop & next() function ***')
    # Get a generator object
    generator = primeNumbers()

    while True:
        try:
            # Get next value from Generator object
            num = next(generator)
            print(num)
        except StopIteration:
            break


    print('*** Using Generator with for loop ***')

    # Iterate over all the yielded values one by one
    for num in primeNumbers():
        print(num)

    for elem in alternateChar('Hello Sample Code'):
        print(elem)

    print('*** Using Infinite Generator ***')

#    for elem in numbersInCycle():
#        print(elem)

if __name__ == '__main__':
  main()

Output:

<generator object primeNumbers at 0x02F59AB0>
step 1
First Element :  5
step 2
Second Element :  7
*** Iterate over Generator object Using while loop & next() function ***
step 1
5
step 2
7
*** Using Generator with for loop ***
step 1
5
step 2
7
H
l
o
S
m
l
 
o
e
*** Using Infinite Generator ***

 

1 thought on “Python : Yield Keyword & Generators explained with examples”

  1. Pingback: Python : List Comprehension vs Generator expression explained with examples – thispointer.com

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top