Python : List Comprehension vs Generator expression explained with examples
In this article we will discuss the differences between list comprehensions and Generator expressions.
Need of Generator Expression ?
Sometimes we want to operate on selected elements only and that too one by one. For such scenarios we should avoid keeping unnecessary elements in memory by creating lists or set or tuple etc.
Let’s understand by an example,
Suppose we want to calculate the sum of all the numbers divisible by 3 & 5 in the range 1 to 1000.
Now let’s see how to do that using list comprehension,
1 2 3 4 5 6 7 8 9 
# Create a list of numbers which are divisible by 3 & 5 and are in range from 1 to 1000 listOfNums = [n for n in range(1000) if n % 3 == 0 and n %5 == 0] # get the sum of all numbers in list total = 0 for num in listOfNums: total += num print('Total = ', total) 
Output:
1 
Total = 33165 
We created a list of numbers i.e. listOfNums and the iterated over each element in the list to calculate the sum. Here creating a list of so many elements and keeping them in memory was unnecessary and could be avoided using Generators.
Generators in Rescue
Getting the Sum of selected numbers using Generators
As we wanted the sum only, therefore there is no need to create a heavy list of these numbers and then calculate the sum. The efficient way is creating a Generator for these selected numbers and then iterate over the selected numbers one by one using generator object and get the sum. This way we will avoid keeping all the elements unnecessary in memory.
Let’s create a Generator that yields numbers divisible by 3 & 5 one by one i.e.
1 2 3 4 5 
def selectedNumbers(): ''' A Generator that yields multiples of 3 & 5 in range 0 to 1000''' for num in range(1000): if num % 3 == 0 and num % 5 == 0: yield num 
selectedNumbers() function returns a Generator object. A Generator is like an Iterator using which we can iterate over all the yielded values one by one i.e.
1 2 3 4 5 6 7 8 9 
# Get a Generator Object generatorObj = selectedNumbers() # Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total) 
Output:
1 
Total = 33165 
It does not keep any thing in memory, during iteration Generator object calls the associated function again and again for next yielded value.
To know more about Generators check this article –> Python : Yield Keyword and Generators explained with examples
So, we avoided keeping unnecessary numbers in memory using Generator. But, do we always need to create a functions for creating Generator?
The answer is no. Here comes the Generator Expression in picture.
What is a generator expression ?
In python, a generator expression is used to generate Generators. It looks like List comprehension in syntax but (} are used instead of [].
Let’s get the sum of numbers divisible by 3 & 5 in range 1 to 1000 using Generator Expression.
Create a Generator expression that returns a Generator object i.e.
1 2 
# Get a Generator object using Generator Expression generatorObj = (n for n in range(1000) if n % 3 == 0 and n %5 == 0) 
This generator expression did exactly same thing as function selectedNumbers() i.e. it returned the Generator object. Now let’s use this Generator object to Iterate over all the numbers and get their sum i.e.
1 2 3 4 5 6 
# Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total) 
Output:
1 
Total = 33165 
Instead of Iterating using for loop we could have directly passed the returned Generator object to sum() function i.e.
1 2 3 4 
# Pass the Generator object returned by Generator Expression to sum() total = sum((n for n in range(1000) if n % 3 == 0 and n %5 == 0)) print('Total = ', total) 
Output:
1 
Total = 33165 
List Comprehension vs Generator Expression
Generator expression’s syntax is just like List comprehension except the brackets, but the main difference between List Comprehension & Generator Expression is that later returns a Generator object instead of list. We should use Generators when we are only interested in looping over the items one at a time and avoid keeping unnecessary elements in memory, as we explained in above examples.
Complete example is as follows,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 
from datetime import datetime def selectedNumbers(): ''' A Generator that yields multiples of 3 & 5 in range 0 to 1000''' for num in range(1000): if num % 3 == 0 and num % 5 == 0: yield num def main(): print('*** Getting the Sum of selected numbers using List Comprehension ***') # Create a list of numbers which are divisible by 3 & 5 and are in range from 1 to 1000 listOfNums = [n for n in range(1000) if n % 3 == 0 and n %5 == 0] # get the sum of all numbers in list total = 0 for num in listOfNums: total += num print('Total = ', total) print('*** Getting the Sum of selected numbers using Generators ***') # Get a Generator Object generatorObj = selectedNumbers() # Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total) print('*** Getting the Sum of selected numbers using Generator Expression ***') # Get a Generator object using Generator Expression generatorObj = (n for n in range(1000) if n % 3 == 0 and n %5 == 0) # Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total) print('*** Getting the Sum of selected numbers using Generator Expression & sum() ***') # Pass the Generator object returned by Generator Expression to sum() total = sum((n for n in range(1000) if n % 3 == 0 and n %5 == 0)) print('Total = ', total) if __name__ == '__main__': main() 
Output:
1 2 3 4 5 6 7 8 
*** Getting the Sum of selected numbers using List Comprehension *** Total = 33165 *** Getting the Sum of selected numbers using Generators *** Total = 33165 *** Getting the Sum of selected numbers using Generator Expression *** Total = 33165 *** Getting the Sum of selected numbers using Generator Expression & sum() *** Total = 33165 
Python Recommendations:
C++ & C++11 Recommendations:

Best C++ Courses for beginners to learn from scratch

5 Top Courses to learn C++11 / C+14

Best books to learn C++11
Leave a Reply