Replace NaN with values from another DataFrame in Pandas

This tutorial will discuss about different ways to replace NaN with values from another DataFrame in pandas.

Table Of Contents

Introduction

Suppose we have two DataFrames with similar index and column names. Like,

First DataFrame:

   First  Second
0   10.0    51.0
1    NaN    52.0
2   11.0     NaN
3    NaN    53.0
4   44.0    54.0
5   55.0    55.0

Second DataFrame:

   First  Second
0     91      81
1     92      82
2     93      83
3     94      84
4     95      85
5     96      86

First DataFrame has certain NaN values. We want to replace those NaN values with the corresponding value from the second DataFrame. After replacement, the First DataFrame should be like,

   First  Second
0   10.0    51.0
1   92.0    52.0
2   11.0    83.0
3   94.0    53.0
4   44.0    54.0
5   55.0    55.0

Preparing DataSet

Let’s create the two DataFrames which we mentioned above.

import pandas as pd
import numpy as np

# First DataFrame with some NaN values
dfObj1 = pd.DataFrame( {'First':  [10, np.NaN, 11, np.NaN, 44, 55],
                        'Second': [51, 52, np.NaN, 53, 54, 55]})

print(dfObj1)

# Second DataFrame
dfObj2 = pd.DataFrame( {'First':  [91, 92, 93, 94, 95, 96],
                        'Second': [81, 82, 83, 84, 85, 86]})

print(dfObj2)

Output

   First  Second
0   10.0    51.0
1    NaN    52.0
2   11.0     NaN
3    NaN    53.0
4   44.0    54.0
5   55.0    55.0


   First  Second
0     91      81
1     92      82
2     93      83
3     94      84
4     95      85
5     96      86

Replace NaN with values from another DataFrame

Call the fillna() on first DataFrame, and pass the second dataframe as argument in it. It will replace all the NaN values in calling DataFrame object with the corresponding values from the another DataFrame (received as argument).

# Replace NaN values in dataframe dfObj1
# from the values from another dataframe dfObj2
dfObj1.fillna(dfObj2, inplace=True)

print(dfObj1)

Output

   First  Second
0   10.0    51.0
1   92.0    52.0
2   11.0    83.0
3   94.0    53.0
4   44.0    54.0
5   55.0    55.0

Summary

We learned how to replace all NaN values in a DataFrame by the corresponding values from another DataFrame. Thanks.