Selecting/excluding sets of columns in pandas

🐼 Selecting/Excluding Sets of Columns in Pandas 🐼

Do you ever find yourself wanting to create a new dataframe from an existing one, but only including or excluding specific columns? 🤔 Fear not, because pandas has got you covered! 🙌🐼

The Problem 😓

Let's say you have a dataframe called df1 with columns A, B, C, and D. You want to create a new dataframe called df2 that includes all the columns from df1, except for columns B and D. 📊

You might think that the following code would do the trick:

import numpy as np
import pandas as pd

# Create a dataframe with columns A, B, C, and D
df = pd.DataFrame(np.random.randn(100, 4), columns=list('ABCD'))

# Try to create a second dataframe df2 from df with all columns except 'B' and 'D'
my_cols = set(df.columns)
my_cols.remove('B').remove('D')

# This returns an error ("unhashable type: set")
df2 = df[my_cols]

Unfortunately, this approach will throw an error: "unhashable type: set". 😢

The Solution 🚀

To select or exclude columns in pandas, we have a couple of options: the loc and iloc selectors. These selectors allow us to specify the labels or positions of the columns we want to include/exclude. 🎯

Selecting Columns ✅

To select columns, we can use the loc selector and pass in a list of the column labels we want to keep. In our case, we want to keep columns A and C, so we can do:

df2 = df.loc[:, ['A', 'C']]  # Selecting columns A and C

This will create a new dataframe df2 with only columns A and C from df1. Easy peasy, right? 😄

Excluding Columns ✅

To exclude columns, we can use the loc selector as well, but this time we specify the labels of the columns we want to exclude. In our case, we want to exclude columns B and D, so we can do:

df2 = df.loc[:, ~df.columns.isin(['B', 'D'])]  # Excluding columns B and D

The ~df.columns.isin(['B', 'D']) part creates a Boolean mask that identifies the columns we want to exclude. The ~ operator negates the mask, effectively excluding those columns from the dataframe. 🙅‍♀️

Dive Deeper 🤿

Pandas also provides another selector called iloc, which allows selection based on column positions instead of labels. If you're interested in learning more about this topic, be sure to check out the official pandas documentation on Indexing and Selecting Data. 😉

Your Turn! ✏️

Now that you know how to select or exclude columns in pandas, put your newfound knowledge to the test! Try applying these techniques to your own dataframes and see the magic happen. ✨

Remember, practice makes perfect, so dive into your code and start creating beautiful new dataframes! 💻

Share your experience and any cool findings in the comments below. We would love to hear from you! 🗣️

Happy coding! 🐍💻

Did you find this guide helpful? Give it a 👍 and share it with your fellow pandas enthusiasts! 😉