In this article we will discuss how to remove rows from a dataframe with missing value or NaN in any, all or few selected columns.

DataFrame.dropna()

Python’s pandas library provides a function to remove rows or columns from a dataframe which contain missing values or NaN i.e.

Arguments :

  • axis:
    • 0 , to drop rows with missing values
    • 1 , to drop columns with missing values
  • how:
    • ‘any’ : drop if any NaN / missing value is present
    • ‘all’ : drop if all the values are missing / NaN
  • thresh: threshold for non NaN values
  • inplace: If True then make changes in the dataplace itself

It removes rows or columns (based on arguments) with missing values / NaN

Let’s use dropna() function to remove rows with missing values in a dataframe,

Suppose we have a dataframe i.e.

Contents of the dataframe,

Drop Rows with missing value / NaN in any column

Output:

It removed all the rows which had any missing value. It didn’t modified the original dataframe, it just returned a copy with modified contents.

How it worked ?
Default value of ‘how’ argument in dropna() is ‘any’ & for ‘axis’ argument it is 0. It means if we don’t pass any argument in dropna() then still it will delete all the rows with any NaN.

We can also pass the ‘how’ & ‘axis’ arguments explicitly too i.e.

It will work similarly i.e. it will remove the rows with any missing value.

P.S. It returned a copy of original dataframe with modified contents

Drop Rows in dataframe which has NaN in all columns

What if we want to remove rows in a dataframe, whose all values are missing i.e. NaN,

Output:

It removes the rows in which all values were missing i.e. all columns contains NaN. It didn’t modified the original dataframe, it just returned a copy with modified contents.

Drop Rows with any missing value in selected columns only

What if we want to remove rows in which values are missing in any of the selected column i.e. ‘Name’ & ‘Age’ columns

Output:

It removes the rows which contains NaN in either ‘Name’ or ‘Age’ column.

P.S. It returned a copy of original dataframe with modified contents

Drop Rows with missing values or NaN in all the selected columns

What if we want to remove rows in which values are missing in all of the selected column i.e. ‘Name’ & ‘Age’ columns

Output:

It removes the rows which contains NaN in both the ‘Name’ or ‘Age’ columns.

P.S. It returned a copy of original dataframe with modified contents

thresh Argument in the dropna() function

What if we want to remove the rows in a dataframe which contains less than n number of non NaN values ?
For this we can pass the n in thresh argument. For example,

Delete rows which contains less than 2 non NaN values

Output:

Drop Rows with missing values from a Dataframe in place

In the examples which we saw till now, dropna() returns a copy of the original dataframe with modified contents.
What if we want to drop rows with missing values in existing dataframe ?
Let’s see how to make changes in dataframe in place i.e.

Output:

As we passed the inplace argument as True. So, it modified the dataframe in place and removed rows from it which had any missing value.

Complete example is as follows,

Output:

Join LinkedIn Group of Python Professional Developers who wish to expand their network and share ideas.

You can also follow us On Twitter :

Click Here to Subscribe for more Articles / Tutorials like this.