Python: Read a file in reverse order line by line

In this article we will discuss an efficient solution to read the contents of a text or CSV file in reverse order i.e. either line by line or get them as a reversed list.

Read a file line by line in reversed order using python

An efficient solution to read a file in reverse order is,

Start reading the file from last and continue till the start of the file i.e. in reverse order. As soon as it encounter any ‘\n’ then it means, a complete line is read. Then yield that line and continue reading in the reverse direction until the top of the file is reached.

We have implemented this logic to a function,

import os

def read_reverse_order(file_name):
    # Open file for reading in binary mode
    with open(file_name, 'rb') as read_obj:
        # Move the cursor to the end of the file
        read_obj.seek(0, os.SEEK_END)
        # Get the current position of pointer i.e eof
        pointer_location = read_obj.tell()
        # Create a buffer to keep the last read line
        buffer = bytearray()
        # Loop till pointer reaches the top of the file
        while pointer_location >= 0:
            # Move the file pointer to the location pointed by pointer_location
            read_obj.seek(pointer_location)
            # Shift pointer location by -1
            pointer_location = pointer_location -1
            # read that byte / character
            new_byte = read_obj.read(1)
            # If the read byte is new line character then it means one line is read
            if new_byte == b'\n':
                # Fetch the line from buffer and yield it
                yield buffer.decode()[::-1]
                # Reinitialize the byte array to save next line
                buffer = bytearray()
            else:
                # If last read character is not eol then add it in buffer
                buffer.extend(new_byte)

        # As file is read completely, if there is still data in buffer, then its the first line.
        if len(buffer) > 0:
            # Yield the first line too
            yield buffer.decode()[::-1]

This function accepts the name of the file as an argument and then yields the lines of file from bottom to top.

How does this function work?

It opens the file in binary read mode and moves the cursor to the end of file using file.seek(). Then it starts reading each byte from the end of the file until the start of the file i.e. in reverse direction and save those bytes in a buffer. While reading each byte, as soon as it encounters a new line character ‘\n’, it means a line is read successfully. Then it reverses the buffer and yields the content of buffer i.e. a complete line. After that it re initializes the buffer and continue reading next bytes from the file in reverse direction till the top of the file is reached and keeps on yielding lines till top of file.

Let’s use this function,

Suppose we have a file ‘sample.txt’ and its contents are,

Hello this is a sample file
It contains sample text
Dummy Line A
Dummy Line B
Dummy Line C
This is the end of file

Read the contents of a ‘sample.txt’ in reverse order line by line,

# Iterate over the file in reverse order using for loop
for line in read_reverse_order('sample.txt'):
    print(line)

Output:

This is the end of file
Dummy Line C
Dummy Line B
Dummy Line A
It contains sample text
Hello this is a sample file

As we are reading one byte at a time and keeping only last line in buffer, therefore even if the file is large, our solution will be efficient.

Get a list of all lines of file in reversed order using python

Instead of reading line by line, suppose we want to get all the lines of a file as a list but in reverse order.
To do that we have created a function that reads all lines of a file in list and then returns a reversed list,

def read_reverse_order_2(file_name):
    """Read a file in reverse order line by line"""
    # Open file in read mode
    with open(file_name, 'r') as read_obj:
        # get all lines in a file as list
        lines = read_obj.readlines()
        lines = [line.strip() for line in lines]
        # reverse the list
        lines = reversed(lines)
        # Return the list with all lines of file in reverse order
        return lines

Let’s use this function to get a reversed list of all lines of a file ‘sample.txt’ and then iterate over that list,

# Get a list of lines in file as reverse order
lines_in_reverse_order = read_reverse_order_2('sample.txt')

# iterate over the lines in list
for line in lines_in_reverse_order:
    print(line)

Output:

This is the end of file
Dummy Line C
Dummy Line B
Dummy Line A
It contains sample text
Hello this is a sample file

Here we got all the lines of a file as a list in reverse order, then we iterated over that list.

If you just want to iterate over the lines of a file in reverse order then this is not an efficient solution, because it keeps all the lines in memory and if file is large like in GBs then it will create issues. So, use this solution with small files only and for large files prefer the first solution mentioned above. It was a little complex but was surely an efficient solution.

The complete example is as follows,

import os


def read_reverse_order_2(file_name):
    """Read a file in reverse order line by line"""
    # Open file in read mode
    with open(file_name, 'r') as read_obj:
        # get all lines in a file as list
        lines = read_obj.readlines()
        lines = [line.strip() for line in lines]
        # reverse the list
        lines = reversed(lines)
        # Return the list with all lines of file in reverse order
        return lines


def read_reverse_order(file_name):
    # Open file for reading in binary mode
    with open(file_name, 'rb') as read_obj:
        # Move the cursor to the end of the file
        read_obj.seek(0, os.SEEK_END)
        # Get the current position of pointer i.e eof
        pointer_location = read_obj.tell()
        # Create a buffer to keep the last read line
        buffer = bytearray()
        # Loop till pointer reaches the top of the file
        while pointer_location >= 0:
            # Move the file pointer to the location pointed by pointer_location
            read_obj.seek(pointer_location)
            # Shift pointer location by -1
            pointer_location = pointer_location -1
            # read that byte / character
            new_byte = read_obj.read(1)
            # If the read byte is new line character then it means one line is read
            if new_byte == b'\n':
                # Fetch the line from buffer and yield it
                yield buffer.decode()[::-1]
                # Reinitialize the byte array to save next line
                buffer = bytearray()
            else:
                # If last read character is not eol then add it in buffer
                buffer.extend(new_byte)

        # As file is read completely, if there is still data in buffer, then its the first line.
        if len(buffer) > 0:
            # Yield the first line too
            yield buffer.decode()[::-1]


def main():

    print('**** Read a file in reversed order line by line ****')

    # Iterate over the file in reverse order using for loop
    for line in read_reverse_order('sample.txt'):
        print(line)

    print('**** Get a list of all lines in file in reversed order****')

    # Get a list of lines in file as reverse order
    lines_in_reverse_order = read_reverse_order_2('sample.txt')

    # iterate over the lines in list
    for line in lines_in_reverse_order:
        print(line)


if __name__ == '__main__':
   main()

Output

**** Read a file in reversed order line by line ****
This is the end of file
Dummy Line C
Dummy Line B
Dummy Line A
It contains sample text
Hello this is a sample file
**** Get a list of all lines in file in reversed order****
This is the end of file
Dummy Line C
Dummy Line B
Dummy Line A
It contains sample text
Hello this is a sample file

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top