In this article we will discuss different ways to read a file line by line in Python.
Suppose we have a file data.txt in same directory as our python script. Let’s see how to read it’s contents line by line.
Solution for Small Files : Get list of all lines in file using readlines()
First basic and inefficient solution is using function readlines().
If we have a small file, then we can call readlines() on the file handler, it reads the whole file content to memory, then splits it into seperate lines and returns a list of all lines in the file. All the lines in list except the last one, will contain new line character in end.
For example,
# Open file
fileHandler = open ("data.txt", "r")
# Get list of all lines in file
listOfLines = fileHandler.readlines()
# Close file
fileHandler.close()
# Iterate over the lines
for line in listOfLines:
print(line.strip())
Output:
sample message string. It is a text file. It contains three lines.
The readlines() function returns a list of lines in file. We can iterate over that list and strip() the new line character then print the line.
But if file size is large then it will consume a lot of memory, so better avoid this solution in case of large files.
Let’s look at some efficient solutions,
Read a file line by line using readline()
While Reading a large file, efficient way is to read file line by line instead of fetching all data in one go.
Let’s use readline() function with file handler i.e.
lineStr = fileHandler.readline()
readline() returns the next line in file which will contain the newline character in end. Also, if end of file is reached then it will return an empty string.
Now let’s see how to read contents of a file line by line using readline() i.e.
# Open file
fileHandler = open ("data.txt", "r")
while True:
# Get next line from file
line = fileHandler.readline()
# If line is empty then end of file reached
if not line :
break;
print(line.strip())
# Close Close
fileHandler.close()
Output:
sample message string. It is a text file. It contains three lines.
Read file line by line with context manager (with block)
When we open the file then we need to close that too. If we forget to close then it will be closed automatically when last reference to file handler is destroyed for example at the end of function. But what if we have a large function that is not going to end soon, even if file related work is complete. In that case we can use context manager to automatically cleanup the things like file closure etc.
For example,
# Open file
with open ("data.txt", "r") as fileHandler:
# Read each line in loop
for line in fileHandler:
# As each line (except last one) will contain new line character, so strip that
print(line.strip())
Output:
sample message string. It is a text file. It contains three lines.
In this case when control comes out of with block then file will be automatically closed. Even if it came out of block due to some exception.
Get List of lines in file with context manager (with block)
Let’s Iterate over all the lines in file and create a list of lines i.e.
# Get the all the lines in file in a list
listOfLines = list()
with open ("data.txt", "r") as myfile:
for line in myfile:
listOfLines.append(line.strip())
print(listOfLines)
Contents of the list listOfLines will be,
['sample message string.', 'It is a text file.', 'It contains three lines.']
Read contents of file line by line using with context manager and while loop
Let’s iterate over the lines in file with context manager and while loop i.e.
# Open file
with open("data.txt", "r") as fileHandler:
# Read next line
line = fileHandler.readline()
# check line is not empty
while line:
print(line.strip())
line = fileHandler.readline()
Contents of the list will be,
sample message string. It is a text file. It contains three lines.
Complete example to read a file line by lines is as follows
print("****Read all lines in file using readlines() *****")
# Open file
fileHandler = open ("data.txt", "r")
# Get list of all lines in file
listOfLines = fileHandler.readlines()
# Close file
fileHandler.close()
# Iterate over the lines
for line in listOfLines:
print(line.strip())
print("****Read file line by line and then close it manualy *****")
# Open file
fileHandler = open ("data.txt", "r")
while True:
# Get next line from file
line = fileHandler.readline()
# If line is empty then end of file reached
if not line :
break;
print(line.strip())
# Close Close
fileHandler.close()
print("****Read file line by line using with open() *****")
# Open file
with open ("data.txt", "r") as fileHandler:
# Read each line in loop
for line in fileHandler:
# As each line (except last one) will contain new line character, so strip that
print(line.strip())
print("****Read file line by line using with open *****")
# Get the all the lines in file in a list
listOfLines = list()
with open ("data.txt", "r") as myfile:
for line in myfile:
listOfLines.append(line.strip())
print(listOfLines)
print("****Read file line by line using with open() and while loop *****")
# Open file
with open("data.txt", "r") as fileHandler:
# Read next line
line = fileHandler.readline()
# check line is not empty
while line:
print(line.strip())
line = fileHandler.readline()
Output:
****Read all lines in file using readlines() ***** sample message string. It is a text file. It contains three lines. ****Read file line by line and then close it manualy ***** sample message string. It is a text file. It contains three lines. ****Read file line by line using with open() ***** sample message string. It is a text file. It contains three lines. ****Read file line by line using with open ***** ['sample message string.', 'It is a text file.', 'It contains three lines.'] ****Read file line by line using with open() and while loop ***** sample message string. It is a text file. It contains three lines.
Thank you! Very helpful.