Drop Infinite Values from a Pandas DataFrame

In this article, we will discuss different ways to Drop infinite values from a Pandas DataFrame.

Table of Contents

A DataFrame is a data structure that stores the data in rows and columns. We can create a DataFrame using pandas.DataFrame() method. Let’s create a dataframe with 4 rows and 5 columns with infinite values

In python , we can create infinite values using numpy module. Call the “numpy.inf” to get an infinite value. Let’s use this to create a dataframe with some infinite values,

import pandas as pd
import numpy as np

# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
                  'two':[np.inf, -np.inf,1,1],
                  'three':[0,0,0,0],
                  'four':[0,1,89, -np.inf],
                  'five':[34,np.inf,45,34]})

# Display the Dataframe
print(df)

Output:

Advertisements
   one  two  three  four  five
0    0  inf      0   0.0  34.0
1    0 -inf      0   1.0   inf
2    0  1.0      0  89.0  45.0
3    0  1.0      0  -inf  34.0

Drop Infinite Values from dataframe using set_option()

We can drop infinite values by using set_option() method. This is used to set the values by using infinity values as NaN values . NaN values means not a number. So we have to convert the mode of operation. Lets see the syntax

pandas.set_option('mode.use_inf_as_na', True)

So the option used is – mode.use_inf_as_na. This will use infinity values as NaN values. After this line, contents of the dataframe will be ,

   one  two  three  four  five
0    0  NaN      0   0.0  34.0
1    0  NaN      0   1.0   NaN
2    0  1.0      0  89.0  45.0
3    0  1.0      0   NaN  34.0

And finally, we have to use the dropna() method to remove the rows that contains NaN values. Syntax is as follows:

df.dropna()

where, df is the input dataframe. . After this line, contents of the dataframe will be,

   one  two  three  four  five
2    0  1.0      0  89.0  45.0

Let’s see the complete example. In this example, We are going to convert the infinity values to NA and then drop NA values from the dataframe. By this we can drop infinity values.

import pandas as pd
import numpy as np

# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
                  'two':[np.inf, -np.inf,1,1],
                  'three':[0,0,0,0],
                  'four':[0,1,89, -np.inf],
                  'five':[34,np.inf,45,34]})

# Display the Dataframe
print(df)


# Changing option to use infinite as NaN
pd.set_option('mode.use_inf_as_na', True)

print(df)

# Drop the rows with NaN values
df = df.dropna()

print('Modified Dataframe')

# Display the Dataframe
print(df)

Output:

   one  two  three  four  five
0    0  inf      0   0.0  34.0
1    0 -inf      0   1.0   inf
2    0  1.0      0  89.0  45.0
3    0  1.0      0  -inf  34.0


   one  two  three  four  five
0    0  NaN      0   0.0  34.0
1    0  NaN      0   1.0   NaN
2    0  1.0      0  89.0  45.0
3    0  1.0      0   NaN  34.0

Modified Dataframe

   one  two  three  four  five
2    0  1.0      0  89.0  45.0

Drop Infinite Values from dataframe using option_context()

We can drop infinite values by using pandas.opion_context() and dataframe.dropna() method. Call option_context(‘mode.use_inf_as_na’, True) to set infinite values as NaN. Then call the dropna() function to delete the NaN values. Eventually all the rows with infinite values will get deleted. Syntax is as follows,

# Changing option to use infinite as NaN and then 
# delete the NaN values i.e. infinite values
with pd.option_context('mode.use_inf_as_na', True):
    df = df.dropna()

The complete example is as follows,

import pandas as pd
import numpy as np

# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
                  'two':[np.inf, -np.inf,1,1],
                  'three':[0,0,0,0],
                  'four':[0,1,89, -np.inf],
                  'five':[34,np.inf,45,34]})

# Display the Dataframe
print(df)


# Changing option to use infinite as NaN and then 
# delete the NaN values i.e. infinite values
with pd.option_context('mode.use_inf_as_na', True):
    df = df.dropna()

print('Modified Dataframe')

# Display the Dataframe
print(df)

Output:

   one  two  three  four  five
0    0  inf      0   0.0  34.0
1    0 -inf      0   1.0   inf
2    0  1.0      0  89.0  45.0
3    0  1.0      0  -inf  34.0
Modified Dataframe
   one  two  three  four  five
2    0  1.0      0  89.0  45.0

Drop Infinite Values from dataframe using isin()

We can drop infinite values by using dataframe.isin() method. This operator will act as filter to drop the infinity values along with ~ operator. It will check the values which are infinity using isin() and select those indices. Then using ~ operator it will mark the values which are infinite in corresponding Boolean dataframe. Then pass the Boolean dataframe to [] operator, it will return a Dataframe that contains the NaN values instead of infinite values from original dataframe. Then using the dropna(0 function, delete the rows with NaN values. Checkout the complete example,

import pandas as pd
import numpy as np

# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
                  'two':[np.inf, -np.inf,1,1],
                  'three':[0,0,0,0],
                  'four':[0,1,89, -np.inf],
                  'five':[34,np.inf,45,34]})

# Display the Dataframe
print(df)

df = df[~df.isin([np.nan, np.inf, -np.inf])]
df.dropna(inplace=True)

print('Modified Dataframe')

# Display the Dataframe
print(df)

Output:

   one  two  three  four  five
0    0  inf      0   0.0  34.0
1    0 -inf      0   1.0   inf
2    0  1.0      0  89.0  45.0
3    0  1.0      0  -inf  34.0
Modified Dataframe
   one  two  three  four  five
2    0  1.0      0  89.0  45.0

Drop Infinite Values from dataframe using replace()

We can drop infinite values by using replace() method. We will first replace the infinity values with NA values using replace() methods and then drop the NA values. Syntax is as follows,

# Replace all Infinite values with NaN
# and drop all NaN values
df = df.replace([np.inf, -np.inf], np.nan).dropna()

Let’s see a complete example,

import pandas as pd
import numpy as np

# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
                  'two':[np.inf, -np.inf,1,1],
                  'three':[0,0,0,0],
                  'four':[0,1,89, -np.inf],
                  'five':[34,np.inf,45,34]})

# Display the Dataframe
print(df)

# Replace all Infinite values with NaN
# and drop all NaN values
df = df.replace([np.inf, -np.inf], np.nan).dropna()


print('Modified Dataframe')

# Display the Dataframe
print(df)

Output:

   one  two  three  four  five
0    0  inf      0   0.0  34.0
1    0 -inf      0   1.0   inf
2    0  1.0      0  89.0  45.0
3    0  1.0      0  -inf  34.0
Modified Dataframe
   one  two  three  four  five
2    0  1.0      0  89.0  45.0

Summary

In this article we discussed how to drop infinity values from the dataframe using set_option() , option_context(), using isin() operator and replace() methods.

Pandas Tutorials -Learn Data Analysis with Python

   

Are you looking to make a career in Data Science with Python?

Data Science is the future, and the future is here now. Data Scientists are now the most sought-after professionals today. To become a good Data Scientist or to make a career switch in Data Science one must possess the right skill set. We have curated a list of Best Professional Certificate in Data Science with Python. These courses will teach you the programming tools for Data Science like Pandas, NumPy, Matplotlib, Seaborn and how to use these libraries to implement Machine learning models.

Checkout the Detailed Review of Best Professional Certificate in Data Science with Python.

Remember, Data Science requires a lot of patience, persistence, and practice. So, start learning today.

Join a LinkedIn Community of Python Developers

Leave a Comment

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top