This tutorial will discuss about different ways to replace NaN with preceding / previous values in pandas.
Table Of Contents
Introduction
Suppose we have a DataFrame with some NaN values i.e.
First Second 0 10.0 51.0 1 NaN 52.0 2 11.0 NaN 3 NaN 53.0 4 44.0 54.0 5 55.0 55.0
Now we want to replace all the NaN values in the DataFrame with the previous valid value from the same column. Like this,
First Second 0 10.0 51.0 1 10.0 52.0 2 11.0 52.0 3 11.0 53.0 4 44.0 54.0 5 55.0 55.0
The NaN value in column “Second” got replaced with the previous value in the same column i.e. 52. The NaN value in column “First” got replaced with the previous value in the same column i.e. 10 and 11.
Let’s see how to do that.
Preparing DataSet
First we will create a DataFrame with some NaN values.
Frequently Asked:
- How to replace a value in Pandas Column?
- Replace NaN with preceding/previous values in Pandas
- Replace NaN values with next values in Pandas
- Replace column values with regex in Pandas
import pandas as pd import numpy as np # Create a DataFrame df = pd.DataFrame( {'First': [10, np.NaN, 11, np.NaN, 44, 55], 'Second': [51, 52, np.NaN, 53, 54, 55]}) print(df)
Output
First Second 0 10.0 51.0 1 NaN 52.0 2 11.0 NaN 3 NaN 53.0 4 44.0 54.0 5 55.0 55.0
Replace NaN values with preceding values in DataFrame
Call the fillna() function on DataFrame, and pass the parameter method="ffill"
. It should fill each NaN value in DataFrame with the last valid value from the same column.
# Replace each NaN values with the previous valid value # in that column df.fillna(method="ffill", inplace=True) print(df)
Output
First Second 0 10.0 51.0 1 10.0 52.0 2 11.0 52.0 3 11.0 53.0 4 44.0 54.0 5 55.0 55.0
Replace NaN values in a column with preceding value
Select the column as Pandas Series object, and call fillna() function on that column/series with parameter method="ffill"
. It should fill all the NaNs in that column, with the previous value from the same column.
# Replace NaN values in column 'Second' with the previous valid value # in that column df['Second'].fillna(method="ffill", inplace=True) print(df)
Output
First Second 0 10.0 51.0 1 NaN 52.0 2 11.0 52.0 3 NaN 53.0 4 44.0 54.0 5 55.0 55.0
Summary
We learned how to replace NaN values with previous values in Pandas.