In this article we will discuss how to read a CSV file line by line with or without header. Also select specific columns while iterating over a CSV file line by line.

Suppose we have a csv file students.csv and its contents are,

We want to read all the rows of this csv file line by line and process each line at a time.

Also note that, here we don’t want to read all lines into a list of lists and then iterate over it, because that will not be an efficient solution for large csv file i.e. file with size in GBs. We are looking for solutions where we read & process only one line at a time while iterating through all rows of csv, so that minimum memory is utilized.

Let’s see how to do this,

Python has a csv module, which provides two different classes to read the contents of a csv file i.e. csv.reader and csv.DictReader. Let’s discuss & use them one by one to read a csv file line by line,

Read a CSV file line by line using csv.reader

With csv module’s reader class object we can iterate over the lines of a csv file as a list of values, where each value in the list is a cell value. Let’s understand with an example,

Output:

It iterates over all the rows of students.csv file. For each row it fetched the contents of that row as a list and printed that list.

How did it work ?

It performed the following steps,

  1. Open the file ‘students.csv’ in read mode and create a file object.
  2. Create a reader object (iterator) by passing file object in csv.reader() function.
  3. Now once we have this reader object, which is an iterator, then use this iterator with for loop to read individual rows of the csv as list of values. Where each value in the list represents an individual cell.

This way only one line will be in memory at a time while iterating through csv file, which makes it a memory efficient solution.

Read csv file without header

In the previous example we iterated through all the rows of csv file including header. But suppose we want to skip the header and iterate over the remaining rows of csv file.
Let’s see how to do that,

Output:

It skipped the header row of csv file and iterate over all the remaining rows of students.csv file. For each row it fetched the contents of that row as a list and printed that list. In initially saved the header row in a separate variable and printed that in end.

How did it work ?

As reader() function returns an iterator object, which we can use with Python for loop to iterate over the rows. But in the above example we called the next() function on this iterator object initially, which returned the first row of csv. After that we used the iterator object with for loop to iterate over remaining rows of the csv file.

Read csv file line by line using csv module DictReader object

With csv module’s DictReader class object we can iterate over the lines of a csv file as a dictionary i.e.
for each row a dictionary is returned, which contains the pair of column names and cell values for that row.
Let’s understand with an example,

Output:

It iterates over all the rows of students.csv file. For each row it fetches the contents of that row as a dictionary and printed that list.

How did it work ?

It performed the following steps,

  1. Open the file ‘students.csv’ in read mode and create a file object.
  2. Create a DictReader object (iterator) by passing file object in csv.DictReader().
  3. Now once we have this DictReader object, which is an iterator. Use this iterator object with for loop to read individual rows of the csv as a dictionary. Where each pair in this dictionary represents contains the column name & column value for that row.

It is a memory efficient solution, because at a time only one line is in memory.

Get column names from header in csv file

DictReader class has a member function that returns the column names of the csv file as list.
let’s see how to use it,

Output:

Read specific columns from a csv file while iterating line by line

Read specific columns (by column name) in a csv file while iterating row by row

Iterate over all the rows of students.csv file line by line, but print only two columns of for each row,

Output:

DictReader returns a dictionary for each line during iteration. As in this dictionary keys are column names and values are cell values for that column. So, for selecting specific columns in every row, we used column name with the dictionary object.

Read specific columns (by column Number) in a csv file while iterating row by row

Iterate over all rows students.csv and for each row print contents of 2ns and 3rd column,

Output:

With csv.reader each row of csv file is fetched as a list of values, where each value represents a column value. So, selecting 2nd & 3rd column for each row, select elements at index 1 and 2 from the list.

The complete example is as follows,

Output:

Python Recommendations:

C++ & C++11 Recommendations:

If you didn't find what you were looking, then do suggest us in the comments below. We will be more than happy to add that.

Subscribe with us to join 1500+ Python & C++ developers, to get more Tips &  Tutorials like this.