In this article, we will discuss different ways to Drop infinite values from a Pandas DataFrame.
Table of Contents
- Drop Infinite Values from dataframe using set_option()
- Drop Infinite Values from dataframe using option_context()
- Drop Infinite Values from dataframe using isin()
- Drop Infinite Values from dataframe using replace()
A DataFrame is a data structure that stores the data in rows and columns. We can create a DataFrame using pandas.DataFrame() method. Let’s create a dataframe with 4 rows and 5 columns with infinite values
In python , we can create infinite values using numpy module. Call the “numpy.inf” to get an infinite value. Let’s use this to create a dataframe with some infinite values,
import pandas as pd
import numpy as np
# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
'two':[np.inf, -np.inf,1,1],
'three':[0,0,0,0],
'four':[0,1,89, -np.inf],
'five':[34,np.inf,45,34]})
# Display the Dataframe
print(df)
Output:
one two three four five 0 0 inf 0 0.0 34.0 1 0 -inf 0 1.0 inf 2 0 1.0 0 89.0 45.0 3 0 1.0 0 -inf 34.0
Drop Infinite Values from dataframe using set_option()
We can drop infinite values by using set_option() method. This is used to set the values by using infinity values as NaN values . NaN values means not a number. So we have to convert the mode of operation. Lets see the syntax
pandas.set_option('mode.use_inf_as_na', True)
So the option used is – mode.use_inf_as_na. This will use infinity values as NaN values. After this line, contents of the dataframe will be ,
Frequently Asked:
- Replace NaN values with empty string in Pandas
- Add Column with random values in Pandas DataFrame
- Pandas: Select dataframe columns containing string
- Python: Find indexes of an element in pandas dataframe
one two three four five 0 0 NaN 0 0.0 34.0 1 0 NaN 0 1.0 NaN 2 0 1.0 0 89.0 45.0 3 0 1.0 0 NaN 34.0
And finally, we have to use the dropna() method to remove the rows that contains NaN values. Syntax is as follows:
df.dropna()
where, df is the input dataframe. . After this line, contents of the dataframe will be,
one two three four five 2 0 1.0 0 89.0 45.0
Let’s see the complete example. In this example, We are going to convert the infinity values to NA and then drop NA values from the dataframe. By this we can drop infinity values.
import pandas as pd
import numpy as np
# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
'two':[np.inf, -np.inf,1,1],
'three':[0,0,0,0],
'four':[0,1,89, -np.inf],
'five':[34,np.inf,45,34]})
# Display the Dataframe
print(df)
# Changing option to use infinite as NaN
pd.set_option('mode.use_inf_as_na', True)
print(df)
# Drop the rows with NaN values
df = df.dropna()
print('Modified Dataframe')
# Display the Dataframe
print(df)
Output:
one two three four five 0 0 inf 0 0.0 34.0 1 0 -inf 0 1.0 inf 2 0 1.0 0 89.0 45.0 3 0 1.0 0 -inf 34.0 one two three four five 0 0 NaN 0 0.0 34.0 1 0 NaN 0 1.0 NaN 2 0 1.0 0 89.0 45.0 3 0 1.0 0 NaN 34.0 Modified Dataframe one two three four five 2 0 1.0 0 89.0 45.0
Drop Infinite Values from dataframe using option_context()
We can drop infinite values by using pandas.opion_context() and dataframe.dropna() method. Call option_context(‘mode.use_inf_as_na’, True) to set infinite values as NaN. Then call the dropna() function to delete the NaN values. Eventually all the rows with infinite values will get deleted. Syntax is as follows,
# Changing option to use infinite as NaN and then
# delete the NaN values i.e. infinite values
with pd.option_context('mode.use_inf_as_na', True):
df = df.dropna()
The complete example is as follows,
import pandas as pd
import numpy as np
# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
'two':[np.inf, -np.inf,1,1],
'three':[0,0,0,0],
'four':[0,1,89, -np.inf],
'five':[34,np.inf,45,34]})
# Display the Dataframe
print(df)
# Changing option to use infinite as NaN and then
# delete the NaN values i.e. infinite values
with pd.option_context('mode.use_inf_as_na', True):
df = df.dropna()
print('Modified Dataframe')
# Display the Dataframe
print(df)
Output:
one two three four five 0 0 inf 0 0.0 34.0 1 0 -inf 0 1.0 inf 2 0 1.0 0 89.0 45.0 3 0 1.0 0 -inf 34.0 Modified Dataframe one two three four five 2 0 1.0 0 89.0 45.0
Drop Infinite Values from dataframe using isin()
We can drop infinite values by using dataframe.isin() method. This operator will act as filter to drop the infinity values along with ~ operator. It will check the values which are infinity using isin() and select those indices. Then using ~ operator it will mark the values which are infinite in corresponding Boolean dataframe. Then pass the Boolean dataframe to [] operator, it will return a Dataframe that contains the NaN values instead of infinite values from original dataframe. Then using the dropna(0 function, delete the rows with NaN values. Checkout the complete example,
import pandas as pd
import numpy as np
# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
'two':[np.inf, -np.inf,1,1],
'three':[0,0,0,0],
'four':[0,1,89, -np.inf],
'five':[34,np.inf,45,34]})
# Display the Dataframe
print(df)
df = df[~df.isin([np.nan, np.inf, -np.inf])]
df.dropna(inplace=True)
print('Modified Dataframe')
# Display the Dataframe
print(df)
Output:
one two three four five 0 0 inf 0 0.0 34.0 1 0 -inf 0 1.0 inf 2 0 1.0 0 89.0 45.0 3 0 1.0 0 -inf 34.0 Modified Dataframe one two three four five 2 0 1.0 0 89.0 45.0
Drop Infinite Values from dataframe using replace()
We can drop infinite values by using replace() method. We will first replace the infinity values with NA values using replace() methods and then drop the NA values. Syntax is as follows,
# Replace all Infinite values with NaN # and drop all NaN values df = df.replace([np.inf, -np.inf], np.nan).dropna()
Let’s see a complete example,
import pandas as pd
import numpy as np
# Create dataframe with few infinite values
df= pd.DataFrame({'one':[0,0,0,0],
'two':[np.inf, -np.inf,1,1],
'three':[0,0,0,0],
'four':[0,1,89, -np.inf],
'five':[34,np.inf,45,34]})
# Display the Dataframe
print(df)
# Replace all Infinite values with NaN
# and drop all NaN values
df = df.replace([np.inf, -np.inf], np.nan).dropna()
print('Modified Dataframe')
# Display the Dataframe
print(df)
Output:
one two three four five 0 0 inf 0 0.0 34.0 1 0 -inf 0 1.0 inf 2 0 1.0 0 89.0 45.0 3 0 1.0 0 -inf 34.0 Modified Dataframe one two three four five 2 0 1.0 0 89.0 45.0
Summary
In this article we discussed how to drop infinity values from the dataframe using set_option() , option_context(), using isin() operator and replace() methods.