Pandas column of lists, create a row for each list element

How to Expand a Pandas Column of Lists

Are you struggling with a pandas DataFrame where some cells contain lists of multiple values? Are you looking for a way to expand the DataFrame so that each item in the list gets its own row? Look no further! In this quick and easy guide, we'll show you how to convert a pandas column of lists into a long-form representation where each list element has its own row.

Let's start with the initial DataFrame:

import pandas as pd
import numpy as np

df = pd.DataFrame(
    {'trial_num': [1, 2, 3, 1, 2, 3],
     'subject': [1, 1, 1, 2, 2, 2],
     'samples': [list(np.random.randn(3).round(2)) for i in range(6)]
    }
)

df

This code will give us the following output:

samples  subject  trial_num
0    [0.57, -0.83, 1.44]        1          1
1    [-0.01, 1.13, 0.36]        1          2
2   [1.18, -1.46, -0.94]        1          3
3  [-0.08, -4.22, -2.05]        2          1
4     [0.72, 0.79, 0.53]        2          2
5    [0.4, -0.32, -0.13]        2          3

The Problem

The current DataFrame structure is not ideal because the samples column contains lists of values, but we want each value to have its own row. Our goal is to convert this DataFrame into long form, with separate rows for each sample.

The Solution

To convert the DataFrame into long form, we can use the explode function in pandas. This function will create a new row for each element in the list, while keeping the values in other columns intact.

Here's how you can use explode to achieve our desired result:

df_exploded = df.explode('samples', ignore_index=True)
df_exploded['sample_num'] = df_exploded.groupby(['subject', 'trial_num']).cumcount()

df_exploded

The resulting DataFrame will look like this:

subject  trial_num  sample  sample_num
0        1          1    0.57           0
1        1          1   -0.83           1
2        1          1    1.44           2
3        1          2   -0.01           0
4        1          2    1.13           1
5        1          2    0.36           2
6        1          3    1.18           0
# etc.

As you can see, each sample from the original list now has its own row, with the corresponding subject, trial_num, and sample_num values.

Wrapping Up

Expanding a pandas column of lists into long form doesn't have to be a complex task. By using the explode function and a little bit of grouping, you can easily transform your DataFrame and work with the data in a more structured and convenient way.

Give it a try and let us know how it works for you! If you have any questions or suggestions, feel free to leave a comment below. Happy coding! 💻🚀

[Insert compelling call-to-action here, such as "Subscribe to our newsletter for more helpful pandas tips and tricks!" or "Join our community of pandas enthusiasts on our forum!"]