Replace NaN values with next values in Pandas

This tutorial will discuss about different ways to replace NaN values with next values in pandas.

Table Of Contents

Introduction

Suppose we have a DataFrame with some NaN values i.e.

   First  Second
0   10.0    51.0
1    NaN    52.0
2   11.0     NaN
3    NaN    53.0
4   44.0    54.0
5   55.0    55.0

Now we want to replace all the NaN values in the DataFrame with the next valid value from the same column. Like this,

   First  Second
0   10.0    51.0
1   11.0    52.0
2   11.0    53.0
3   44.0    53.0
4   44.0    54.0
5   55.0    55.0

The NaN value in column “Second” got replaced with the next value in the same column i.e. 53. The NaN value in column “First” got replaced with the next value in the same column i.e. 11 and 44.

Let’s see how to do that.

Preparing DataSet

First we will create a DataFrame with some NaN values.

import pandas as pd
import numpy as np

# Create a DataFrame
df = pd.DataFrame(
    {'First':  [10, np.NaN, 11, np.NaN, 44, 55],
     'Second': [51, 52, np.NaN, 53, 54, 55]})

print(df)

Output

   First  Second
0   10.0    51.0
1    NaN    52.0
2   11.0     NaN
3    NaN    53.0
4   44.0    54.0
5   55.0    55.0

Replace NaN values with next values in DataFrame

Call the fillna() function on DataFrame, and pass the parameter method="bfill". It should fill each NaN value in DataFrame with the next valid value from the same column.

# Replace each NaN values with the next valid value
# in that column
df.fillna(method="bfill", inplace=True)

print(df)

Output

   First  Second
0   10.0    51.0
1   11.0    52.0
2   11.0    53.0
3   44.0    53.0
4   44.0    54.0
5   55.0    55.0

Replace NaN values in a column with next value

Select the column as Pandas Series object, and call fillna() function on that column/series with parameter method="bfill". It should fill all the NaNs in that column, with the next value from the same column.

# Replace NaN values in column 'Second' with the next valid value
# in that column
df['Second'].fillna(method="bfill", inplace=True)

print(df)

Output

   First  Second
0   10.0    51.0
1    NaN    52.0
2   11.0    53.0
3    NaN    53.0
4   44.0    54.0
5   55.0    55.0

Summary

We learned how to replace NaN values with next values in Pandas.