Python: How to Iterate over nested dictionary -dict of dicts

In this article, we will discuss different ways to iterate / loop over all key-value pairs of a nested dictionary.

A nested dictionary in python is a dictionary of dictionaries. It is a kind of dictionary which has another dictionary objects as values in the key-value pairs. These dictionary values can also have another dictionaries internally. The nested structure of dictionary can go on and on.

An example of nested dictionary in python,

# A Nested dictionary i.e. dictionaty of dictionaries
students = {
            'ID 1':    {'Name': 'Shaun', 'Age': 35, 'City': 'Delhi'},
            'ID 2':    {'Name': 'Ritika', 'Age': 31, 'City': 'Mumbai'},
            'ID 3':    {'Name': 'Smriti', 'Age': 33, 'City': 'Sydney'},
            'ID 4':    {'Name': 'Jacob', 'Age': 23, 'City': {'perm': 'Tokyo',
                                                             'current': 'London'}},
            }

Now we want to iterate over all key-value pairs of this dictionary i.e. including the values of internal dictionaries too. The expected result should be similar to this,

('ID 1', 'Name', 'Shaun')
('ID 1', 'Age', 35)
('ID 1', 'City', 'Delhi')
('ID 2', 'Name', 'Ritika')
('ID 2', 'Age', 31)
('ID 2', 'City', 'Mumbai')
('ID 3', 'Name', 'Smriti')
('ID 3', 'Age', 33)
('ID 3', 'City', 'Sydney')
('ID 4', 'Name', 'Jacob')
('ID 4', 'Age', 23)
('ID 4', 'City', 'perm', 'Tokyo')
('ID 4', 'City', 'current', 'London')

Let’s see how to do that.

Iterate over all values of a nested dictionary in python

For a normal dictionary, we can just call the items() function of dictionary to get an iterable sequence of all key-value pairs. But in a nested dictionary, a value can be an another dictionary object. For that we need to again call the items() function on such values and get another iterable sequence of pairs and then look for dict objects in those pairs too. We can achieve all this in a simple manner using recursion.

We have created a function that accepts a dictionary as an argument and yields all key-value pairs in it. Included the values of internal / nested dictionaries. For nested structure it returns a tuple that includes all keys for that value in hierarcy. Let’s use this function to iterate over all values of a dictionary of dictionaries,

# A Nested dictionary i.e. dictionaty of dictionaries
students = {
            'ID 1':    {'Name': 'Shaun', 'Age': 35, 'City': 'Delhi'},
            'ID 2':    {'Name': 'Ritika', 'Age': 31, 'City': 'Mumbai'},
            'ID 3':    {'Name': 'Smriti', 'Age': 33, 'City': 'Sydney'},
            'ID 4':    {'Name': 'Jacob', 'Age': 23, 'City': {'perm': 'Tokyo',
                                                             'current': 'London'}},
            }

def nested_dict_pairs_iterator(dict_obj):
    ''' This function accepts a nested dictionary as argument
        and iterate over all values of nested dictionaries
    '''
    # Iterate over all key-value pairs of dict argument
    for key, value in dict_obj.items():
        # Check if value is of dict type
        if isinstance(value, dict):
            # If value is dict then iterate over all its values
            for pair in  nested_dict_pairs_iterator(value):
                yield (key, *pair)
        else:
            # If value is not dict type then yield the value
            yield (key, value)


#Loop through all key-value pairs of a nested dictionary
for pair in nested_dict_pairs_iterator(students):
    print(pair)

Output:

('ID 1', 'Name', 'Shaun')
('ID 1', 'Age', 35)
('ID 1', 'City', 'Delhi')
('ID 2', 'Name', 'Ritika')
('ID 2', 'Age', 31)
('ID 2', 'City', 'Mumbai')
('ID 3', 'Name', 'Smriti')
('ID 3', 'Age', 33)
('ID 3', 'City', 'Sydney')
('ID 4', 'Name', 'Jacob')
('ID 4', 'Age', 23)
('ID 4', 'City', 'perm', 'Tokyo')
('ID 4', 'City', 'current', 'London')

Using the function nested_dict_pair_iterator() we iterated over all the values of a dictionary of dictionaries and printed each pair including the parent keys.

How it works ?

Inside the function, we iterated over all the key-value pairs of a given dictionary object and for each value object in the pair it checks if the value is of dict type or not. If not then it just yields the pair, but if value is dict type then it reccusrively calls itself with the dict value object as argument and then yields all its key-value pairs. The process goes on and on, till all the internal dictionary objects are not covered. This is how it yields all the key-value pairs of a dictionary of dictionaries.

Get a list of all key-value pairs of nested dictionary in python

We can also create a list of all key-value pairs of a dictionary of dictionaries, by passing the yielded tuples from function nested_dict_pair_iterator() to the list(). For example,

# A Nested dictionary i.e. dictionaty of dictionaries
students = {
            'ID 1':    {'Name': 'Shaun', 'Age': 35, 'City': 'Delhi'},
            'ID 2':    {'Name': 'Ritika', 'Age': 31, 'City': 'Mumbai'},
            'ID 3':    {'Name': 'Smriti', 'Age': 33, 'City': 'Sydney'},
            'ID 4':    {'Name': 'Jacob', 'Age': 23, 'City': {'perm': 'Tokyo',
                                                             'current': 'London'}},
            }

def nested_dict_pairs_iterator(dict_obj):
    ''' This function accepts a nested dictionary as argument
        and iterate over all values of nested dictionaries
    '''
    # Iterate over all key-value pairs of dict argument
    for key, value in dict_obj.items():
        # Check if value is of dict type
        if isinstance(value, dict):
            # If value is dict then iterate over all its values
            for pair in  nested_dict_pairs_iterator(value):
                yield (key, *pair)
        else:
            # If value is not dict type then yield the value
            yield (key, value)

# Get all key-value pairs of a nested dictionary as list
all_pairs = list(nested_dict_pairs_iterator(students))

for pair in all_pairs:
    print(pair)

Output:

('ID 1', 'Name', 'Shaun')
('ID 1', 'Age', 35)
('ID 1', 'City', 'Delhi')
('ID 2', 'Name', 'Ritika')
('ID 2', 'Age', 31)
('ID 2', 'City', 'Mumbai')
('ID 3', 'Name', 'Smriti')
('ID 3', 'Age', 33)
('ID 3', 'City', 'Sydney')
('ID 4', 'Name', 'Jacob')
('ID 4', 'Age', 23)
('ID 4', 'City', 'perm', 'Tokyo')
('ID 4', 'City', 'current', 'London')

Summary:

In this article we learned how to iterate over all pairs of a nested dictionary object i.e. a dictionary of dictionaries.