In this article we will discuss the differences between list comprehensions and Generator expressions.
Need of Generator Expression ?
Sometimes we want to operate on selected elements only and that too one by one. For such scenarios we should avoid keeping unnecessary elements in memory by creating lists or set or tuple etc.
Let’s understand by an example,
Suppose we want to calculate the sum of all the numbers divisible by 3 & 5 in the range 1 to 1000.
Now let’s see how to do that using list comprehension,
# Create a list of numbers which are divisible by 3 & 5 and are in range from 1 to 1000 listOfNums = [n for n in range(1000) if n % 3 == 0 and n %5 == 0] # get the sum of all numbers in list total = 0 for num in listOfNums: total += num print('Total = ', total)
Output:
Total = 33165
We created a list of numbers i.e. listOfNums and the iterated over each element in the list to calculate the sum. Here creating a list of so many elements and keeping them in memory was unnecessary and could be avoided using Generators.
Generators in Rescue
Getting the Sum of selected numbers using Generators
As we wanted the sum only, therefore there is no need to create a heavy list of these numbers and then calculate the sum. The efficient way is creating a Generator for these selected numbers and then iterate over the selected numbers one by one using generator object and get the sum. This way we will avoid keeping all the elements unnecessary in memory.
Let’s create a Generator that yields numbers divisible by 3 & 5 one by one i.e.
def selectedNumbers(): ''' A Generator that yields multiples of 3 & 5 in range 0 to 1000''' for num in range(1000): if num % 3 == 0 and num % 5 == 0: yield num
selectedNumbers() function returns a Generator object. A Generator is like an Iterator using which we can iterate over all the yielded values one by one i.e.
# Get a Generator Object generatorObj = selectedNumbers() # Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total)
Output:
Total = 33165
It does not keep any thing in memory, during iteration Generator object calls the associated function again and again for next yielded value.
To know more about Generators check this article –>Â Python : Yield Keyword and Generators explained with examples
So, we avoided keeping unnecessary numbers in memory using Generator. But, do we always need to create a functions for creating Generator?
The answer is no. Here comes the Generator Expression in picture.
What is a generator expression ?
In python, a generator expression is used to generate Generators. It looks like List comprehension in syntax but (} are used instead of [].
Let’s get the sum of numbers divisible by 3 & 5 in range 1 to 1000 using Generator Expression.
Create a Generator expression that returns a Generator object i.e.
# Get a Generator object using Generator Expression generatorObj = (n for n in range(1000) if n % 3 == 0 and n %5 == 0)
This generator expression did exactly same thing as function selectedNumbers() i.e. it returned the Generator object. Now let’s use this Generator object to Iterate over all the numbers and get their sum i.e.
# Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total)
Output:
Total = 33165
Instead of Iterating using for loop we could have directly passed the returned Generator object to sum() function i.e.
# Pass the Generator object returned by Generator Expression to sum() total = sum((n for n in range(1000) if n % 3 == 0 and n %5 == 0)) print('Total = ', total)
Output:
Total = 33165
List Comprehension vs Generator Expression
Generator expression’s syntax is just like List comprehension except the brackets, but the main difference between List Comprehension & Generator Expression is that later returns a Generator object instead of list. We should use Generators when we are only interested in looping over the items one at a time and avoid keeping unnecessary elements in memory, as we explained in above examples.
Complete example is as follows,
from datetime import datetime def selectedNumbers(): ''' A Generator that yields multiples of 3 & 5 in range 0 to 1000''' for num in range(1000): if num % 3 == 0 and num % 5 == 0: yield num def main(): print('*** Getting the Sum of selected numbers using List Comprehension ***') # Create a list of numbers which are divisible by 3 & 5 and are in range from 1 to 1000 listOfNums = [n for n in range(1000) if n % 3 == 0 and n %5 == 0] # get the sum of all numbers in list total = 0 for num in listOfNums: total += num print('Total = ', total) print('*** Getting the Sum of selected numbers using Generators ***') # Get a Generator Object generatorObj = selectedNumbers() # Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total) print('*** Getting the Sum of selected numbers using Generator Expression ***') # Get a Generator object using Generator Expression generatorObj = (n for n in range(1000) if n % 3 == 0 and n %5 == 0) # Iterate over yielded values one by one and calculate the sum total = 0 for num in generatorObj: total += num print('Total = ', total) print('*** Getting the Sum of selected numbers using Generator Expression & sum() ***') # Pass the Generator object returned by Generator Expression to sum() total = sum((n for n in range(1000) if n % 3 == 0 and n %5 == 0)) print('Total = ', total) if __name__ == '__main__': main()
Output:
*** Getting the Sum of selected numbers using List Comprehension *** Total = 33165 *** Getting the Sum of selected numbers using Generators *** Total = 33165 *** Getting the Sum of selected numbers using Generator Expression *** Total = 33165 *** Getting the Sum of selected numbers using Generator Expression & sum() *** Total = 33165
Pandas Tutorials Learn Data Analysis with Python

Pandas Tutorial Part #1  Introduction to Data Analysis with Python

Pandas Tutorial Part #2  Basics of Pandas Series

Pandas Tutorial Part #3  Get & Set Series values

Pandas Tutorial Part #4  Attributes & methods of Pandas Series

Pandas Tutorial Part #5  Add or Remove Pandas Series elements

Pandas Tutorial Part #6  Introduction to DataFrame

Pandas Tutorial Part #7  DataFrame.loc[]  Select Rows / Columns by Indexing

Pandas Tutorial Part #8  DataFrame.iloc[]  Select Rows / Columns by Label Names

Pandas Tutorial Part #9  Filter DataFrame Rows

Pandas Tutorial Part #10  Add/Remove DataFrame Rows & Columns

Pandas Tutorial Part #11  DataFrame attributes & methods

Pandas Tutorial Part #12  Handling Missing Data or NaN values

Pandas Tutorial Part #13  Iterate over Rows & Columns of DataFrame

Pandas Tutorial Part #14  Sorting DataFrame by Rows or Columns

Pandas Tutorial Part #15  Merging or Concatenating DataFrames

Pandas Tutorial Part #16  DataFrame GroupBy explained with examples
Are you looking to make a career in Data Science with Python?
Data Science is the future, and the future is here now. Data Scientists are now the most soughtafter professionals today. To become a good Data Scientist or to make a career switch in Data Science one must possess the right skill set. We have curated a list of Best Professional Certificate in Data Science with Python. These courses will teach you the programming tools for Data Science like Pandas, NumPy, Matplotlib, Seaborn and how to use these libraries to implement Machine learning models.
Checkout the Detailed Review of Best Professional Certificate in Data Science with Python.
Remember, Data Science requires a lot of patience, persistence, and practice. So, start learning today.