7.2 Using the .apply() Method

Advanced DataFrames

Sometimes we need to apply more complex functions to our DataFrames that we cannot do with typical vectorized operations that exist for column operations. In this case, we can use the df.apply() method to apply a function along a particular axis of the DataFrame. These functions can be either built-in functions or user-defined functions. They allow us to perform advanced operations across the DataFrame. Better still, we are able to df.apply() a function either to a single column or to the entire DataFrame.

Using the .apply() method is most useful when you need to perform a custom operation on each element of a Series (single column) or each row/column of a DataFrame that cannot be easily accomplished with built-in vectorized functions.

For example, .apply() is helpful when:

Remember:

.apply() can be slower than vectorized operations, especially on large DataFrames. Always prefer built-in pandas functions when possible for better performance. Use .apply() when you need flexibility that vectorized methods can’t provide.

Applying Functions to a Single Column

The most direct usage of df.apply() is applying it to a single column (or a Series object). We can achieve this by choosing a column from a DataFrame and appending .apply() to it. The .apply() method in this instance accepts a function as its first argument (either a named function or a lambda). The parameter in the function will receive each element of the Series (i.e., each value in the column) as its input.

import pandas as pd
import random

# Set a seed for reproducibility
random.seed(42)

# Generate sample player names
players = ["Alice", "Bob", "Charlie", "David", "Eva"]

# Generate random stats for each player
data = {
    "player": players,
    "homeruns": [random.randint(0, 30) for _ in players],
    "at_bats": [random.randint(107, 450) for _ in players],
    "singles": [random.randint(20, 70) for _ in players],
    "doubles": [random.randint(2, 20) for _ in players],
    "triples": [random.randint(0, 5) for _ in players],
}

df = pd.DataFrame(data)
df
player homeruns at_bats singles doubles triples
0 Alice 20 232 25 4 4
1 Bob 3 221 57 8 1
2 Charlie 0 178 47 9 5
3 David 23 159 22 18 5
4 Eva 8 386 21 2 5

Now we have a DataFrame df with all the necessary player statistics to generate some insights, such as batting averages and slugging percentages.

For batting average, we can use basic column operations to calculate it as follows:

df["batting_average"] = (
    df["homeruns"] + df["singles"] + df["doubles"] + df["triples"]
) / df["at_bats"]

df.head()
player homeruns at_bats singles doubles triples batting_average
0 Alice 20 232 25 4 4 0.228448
1 Bob 3 221 57 8 1 0.312217
2 Charlie 0 178 47 9 5 0.342697
3 David 23 159 22 18 5 0.427673
4 Eva 8 386 21 2 5 0.093264

But other calculations might not be as simple as this.

apply() on the Entire DataFrame

For instance, if we wanted to calculate the slugging percentage, we would need to account for the different types of hits (singles, doubles, triples, and home runs) and their respective values. This is where the apply() method comes in handy.

def calculate_slugging_percentage(row):
    total_bases = (
        row["singles"] + 2 * row["doubles"] + 3 * row["triples"] + 4 * row["homeruns"]
    )
    return total_bases / row["at_bats"] if row["at_bats"] > 0 else 0

df["slugging_percentage"] = df.apply(calculate_slugging_percentage, axis=1)
df.head()
player homeruns at_bats singles doubles triples batting_average slugging_percentage
0 Alice 20 232 25 4 4 0.228448 0.538793
1 Bob 3 221 57 8 1 0.312217 0.398190
2 Charlie 0 178 47 9 5 0.342697 0.449438
3 David 23 159 22 18 5 0.427673 1.037736
4 Eva 8 386 21 2 5 0.093264 0.186528

The most important difference between applying a function to a single column and applying it to the entire DataFrame is the axis parameter. When applying a function to a single column, we do not need to specify the axis parameter because it is implicitly understood that we are working with a Series (which is one-dimensional). However, when applying a function to the entire DataFrame, we need to specify axis=1 to indicate that we want to apply the function to each row (as opposed to axis=0, which would apply the function to each column). The great part about apply() is that it is really clean and easy to read. Remember, good code is readable code. Using apply() allows us to write the function logic in a named function that we then pass to apply(), making it clear what the function does.