Replace NaN values with empty string in Pandas

In this article we will discuss different ways to replace NaN Values with empty strings in a specific column of Dataframe or in complete DataFrame in Python.

Table Of Contents

• Replace NaN values with empty string using fillna()
  • In a Column only
  • In entire DataFrame
• Replace NaN values with empty string using replace()
  • In a Column only
  • In entire DataFrame

A DataFrame is a data structure that stores the data the in tabular format i.e. in the format of rows and columns. We can create a DataFrame using pandas.DataFrame() method. In Python , we can create NaN values using the numpy module.. Let’s use this to create a dataframe of four rows and five columns with few NaN values.

import pandas as pd
import numpy as np

# Create dataframe with 4 rows and 5 columns
df= pd.DataFrame({'First'  :[0, 0, 0, 0],
                  'Second' :[np.nan, np.nan,1 ,1],
                  'Third' :[0, 0, 0, 0],
                  'Fourth' :[0, 1, 89, np.nan],
                  'Fifth'  :[34, np.nan,45,34]})

# Display the Dataframe
print(df)

Output:

   First  Second  Third  Fourth  Fifth
0      0     NaN      0     0.0   34.0
1      0     NaN      0     1.0    NaN
2      0     1.0      0    89.0   45.0
3      0     1.0      0     NaN   34.0

Replace NaN values with empty string using fillna()

In Pandas, both DataFrame and Series provides a member function fillna() to fill/replace NaN values with a specified value. Their Syntax are as follows,

Series.fillna(value) 

It replaces all the NaN values in the calling Series object with the specified value

DataFrame.fillna(value) 

It replaces all the NaN values in the calling DataFrame object with the specified value

Replace NaN values with empty string in a column using fillna()

We can select a single column of Dataframe as a Series object and then call the fillna(”) on that column to replace all NaN values with empty strings in that column. For example,

import pandas as pd
import numpy as np

# Create dataframe with 4 rows and 5 columns
df= pd.DataFrame({'First'  :[0, 0, 0, 0],
                  'Second' :[np.nan, np.nan,1 ,1],
                  'Third' :[0, 0, 0, 0],
                  'Fourth' :[0, 1, 89, np.nan],
                  'Fifth'  :[34, np.nan,45,34]})

# Display the Dataframe
print(df)

# Replace NaN with empty strings in column 'Second'
df['Second'] = df['Second'].fillna('')

# Display the Dataframe
print(df)

Output:

   First  Second  Third  Fourth  Fifth
0      0     NaN      0     0.0   34.0
1      0     NaN      0     1.0    NaN
2      0     1.0      0    89.0   45.0
3      0     1.0      0     NaN   34.0

   First Second  Third  Fourth  Fifth
0      0             0     0.0   34.0
1      0             0     1.0    NaN
2      0      1      0    89.0   45.0
3      0      1      0     NaN   34.0

Here, we selected the column ‘Second’ as a Series object and then called the fillna() function on that with an empty string as an argument. Therefore, it replaced all the NaN values in column ‘Second’ with the empty strings.

Replace NaN Values with empty strings entire dataframe using fillna()

Call the fillna() function of the DataFrame object with an empty string as argument. It will replace NaN values in the entire DataFrame with empty strings. For example,

import pandas as pd
import numpy as np

# Create dataframe with 4 rows and 5 columns
df= pd.DataFrame({'First'  :[0, 0, 0, 0],
                  'Second' :[np.nan, np.nan,1 ,1],
                  'Third' :[0, 0, 0, 0],
                  'Fourth' :[0, 1, 89, np.nan],
                  'Fifth'  :[34, np.nan,45,34]})

# Display the Dataframe
print(df)

# Replace NaN with empty strings in entire DataFrame
df = df.fillna('')

# Display the Dataframe
print(df)

Output:

   First  Second  Third  Fourth  Fifth
0      0     NaN      0     0.0   34.0
1      0     NaN      0     1.0    NaN
2      0     1.0      0    89.0   45.0
3      0     1.0      0     NaN   34.0

   First Second  Third Fourth Fifth
0      0             0      0    34
1      0             0      1      
2      0      1      0     89    45
3      0      1      0           34

Replace NaN values with empty string using replace()

In Pandas, both the Dataframe and series class provides a function replace() to change the contents. We are going to use these functions,

DataFrame.replace()

To replace all the occurrences of a value in the entire Dataframe, pass the item to be replaced and replacement value as arguments to the replace() function.

DataFrame.replace(to_replace, value)

Series.replace()

Series.replace(to_replace, value)

To replace the value to be changed with the given value.

Let’s use this to replace NaN values with empty strings.

Replace NaN Values with empty strings in a column using replace()

Select the column ‘Second’ as a Series object from the Dataframe and the call the replace() function to replace all NaN values in that column with empty strings. For example,

import pandas as pd
import numpy as np

# Create dataframe with 4 rows and 5 columns
df= pd.DataFrame({'First'  :[0, 0, 0, 0],
                  'Second' :[np.nan, np.nan,1 ,1],
                  'Third' :[0, 0, 0, 0],
                  'Fourth' :[0, 1, 89, np.nan],
                  'Fifth'  :[34, np.nan,45,34]})

# Display the Dataframe
print(df)

# Replace NaN with empty string in column 'Second'
df['Second'] = df['Second'].replace(np.NaN, '')

# Display the Dataframe
print(df)

Output:

   First  Second  Third  Fourth  Fifth
0      0     NaN      0     0.0   34.0
1      0     NaN      0     1.0    NaN
2      0     1.0      0    89.0   45.0
3      0     1.0      0     NaN   34.0

   First Second  Third  Fourth  Fifth
0      0             0     0.0   34.0
1      0             0     1.0    NaN
2      0      1      0    89.0   45.0
3      0      1      0     NaN   34.0

Replace NaN Values with empty strings in entire dataframe using replace()

Call the replace() function on DataFrame object with arguments NaN and ”. It will replace all occurrences of NaNs with empty strings in the entire DataFrame. For example,

import pandas as pd
import numpy as np

# Create dataframe with 4 rows and 5 columns
df= pd.DataFrame({'First'  :[0, 0, 0, 0],
                  'Second' :[np.nan, np.nan,1 ,1],
                  'Third' :[0, 0, 0, 0],
                  'Fourth' :[0, 1, 89, np.nan],
                  'Fifth'  :[34, np.nan,45,34]})

# Display the Dataframe
print(df)

# Replace NaN with empty strings in entore DataFrame
df = df.replace(np.NaN, '')

# Display the Dataframe
print(df)

Output:

   First  Second  Third  Fourth  Fifth
0      0     NaN      0     0.0   34.0
1      0     NaN      0     1.0    NaN
2      0     1.0      0    89.0   45.0
3      0     1.0      0     NaN   34.0

   First Second  Third Fourth Fifth
0      0             0      0    34
1      0             0      1      
2      0      1      0     89    45
3      0      1      0           34

Summary

In this article we learned about two different ways to replace NaN values with empty strings, either in a column or in entire dataframe.