12.4 Project

Bivariate C2N Statitistics

Let’s use our data from our soccer players.

import pandas as pd
import scipy.stats as stats

url = "https://raw.githubusercontent.com/nkmwicz/data-for-students/refs/heads/main/soccer-players.csv"

df = pd.read_csv(url)
df.head()
player_id first_name last_name name last_season current_club_id player_code country_of_birth city_of_birth country_of_citizenship ... foot height_in_cm contract_expiration_date agent_name image_url url current_club_domestic_competition_id current_club_name market_value_in_eur highest_market_value_in_eur
0 10 Miroslav Klose Miroslav Klose 2015 398 miroslav-klose Poland Opole Germany ... right 184.0 NaN ASBW Sport Marketing https://img.a.transfermarkt.technology/portrai... https://www.transfermarkt.co.uk/miroslav-klose... IT1 Società Sportiva Lazio S.p.A. 1000000.0 30000000.0
1 26 Roman Weidenfeller Roman Weidenfeller 2017 16 roman-weidenfeller Germany Diez Germany ... left 190.0 NaN Neubauer 13 GmbH https://img.a.transfermarkt.technology/portrai... https://www.transfermarkt.co.uk/roman-weidenfe... L1 Borussia Dortmund 750000.0 8000000.0
2 65 Dimitar Berbatov Dimitar Berbatov 2015 1091 dimitar-berbatov Bulgaria Blagoevgrad Bulgaria ... NaN NaN NaN CSKA-AS-23 Ltd. https://img.a.transfermarkt.technology/portrai... https://www.transfermarkt.co.uk/dimitar-berbat... GR1 Panthessalonikios Athlitikos Omilos Konstantin... 1000000.0 34500000.0
3 77 NaN Lúcio Lúcio 2012 506 lucio Brazil Brasília Brazil ... NaN NaN NaN NaN https://img.a.transfermarkt.technology/portrai... https://www.transfermarkt.co.uk/lucio/profil/s... IT1 Juventus Football Club 200000.0 24500000.0
4 80 Tom Starke Tom Starke 2017 27 tom-starke East Germany (GDR) Freital Germany ... right 194.0 NaN IFM https://img.a.transfermarkt.technology/portrai... https://www.transfermarkt.co.uk/tom-starke/pro... L1 FC Bayern München 100000.0 3000000.0

5 rows × 23 columns

Let’s evaluate the various categories.

for col in df.columns:
    print(f'{col}: {df[col].isna().sum()}')
    if pd.api.types.is_numeric_dtype(df[col]):
        print(f'\t mean ({df[col].mean()}), std ({df[col].std()})')
    else:
        print(f'\t{col}: {df[col].nunique()}')
player_id: 0
     mean (344578.92571675463), std (280683.1398991373)
first_name: 2058
    first_name: 6966
last_name: 0
    last_name: 23701
name: 0
    name: 31708
last_season: 0
     mean (2019.3377773662933), std (3.965914065614451)
current_club_id: 0
     mean (4809.252044563775), std (11601.982977954953)
player_code: 0
    player_code: 31667
country_of_birth: 2803
    country_of_birth: 185
city_of_birth: 2461
    city_of_birth: 8537
country_of_citizenship: 380
    country_of_citizenship: 183
date_of_birth: 47
    date_of_birth: 9272
sub_position: 185
    sub_position: 13
position: 0
    position: 5
foot: 2593
    foot: 3
height_in_cm: 2344
     mean (182.29116071725608), std (6.9712243763986)
contract_expiration_date: 12131
    contract_expiration_date: 114
agent_name: 16015
    agent_name: 2833
image_url: 0
    image_url: 26561
url: 0
    url: 32403
current_club_domestic_competition_id: 0
    current_club_domestic_competition_id: 14
current_club_name: 0
    current_club_name: 437
market_value_in_eur: 1534
     mean (1611350.0599306747), std (6362017.247132169)
highest_market_value_in_eur: 1534
     mean (3675044.705043895), std (9653897.38659055)

Let’s get the birth months

from datetime import datetime
df = df.dropna(subset='date_of_birth')
df['b_month'] = df['date_of_birth'].apply(lambda x: datetime.strptime(x, '%Y-%m-%d %H:%M:%S').month)
df['b_month']
0        6
1        8
2        1
3        5
4        3
        ..
32398    2
32399    9
32400    2
32401    1
32402    6
Name: b_month, Length: 32356, dtype: int64

We can now run an ANOVA on the Positions.

df['position'].unique()

p1,p2,p3,p4,p5 = [df.loc[df['position']==i, 'b_month'] for i in df['position'].unique()]
f, p = stats.f_oneway(p1,p2,p3,p4,p5)

dfb = df['position'].nunique()- 1
dfw = len(df['position']) - df['position'].nunique()

critical = stats.f.ppf(1 - 0.05, dfb, dfw)

print(f'f: {f}, p: {p}, critical-f: {critical}')
f: 3.2134358565163734, p: 0.01202419637178675, critical-f: 2.372206772040356

The f-stat is 3.21, which is higher than the critical f, which is 2.37. This information tells us that there is a deviation in the mean between positions. Unfortunately, the ANOVA does not tell us what direction that it is in. In order to know where the deviation in mean is, we must run a loop over the positions and run a t-test against every other position.

positions = df["position"].unique()
num_pos = df["position"].nunique()

for pos in positions:
    for i in range(num_pos - 1, 0, -1):
        p1 = df.loc[df["position"] == pos, "b_month"]
        p2 = df.loc[df["position"] == positions[i], "b_month"]
        t_res = stats.ttest_ind(p1, p2)
        t, p = t_res
        d = t_res.df
        critical = stats.t.ppf(1-0.05/2, d)
        print(f"{pos}, {positions[i]}: t: {t}, p: {p}, critical: {critical}")
Attack, Missing: t: 1.3018933996037771, p: 0.19298606369161414, critical: 1.9602269330132052
Attack, Midfield: t: 2.430376280320745, p: 0.015092777291315614, critical: 1.960094675154383
Attack, Defender: t: 2.935473934765965, p: 0.0033343422037524057, critical: 1.9600878382903932
Attack, Goalkeeper: t: 2.7425321225077792, p: 0.006105392486458355, critical: 1.960152997933216
Goalkeeper, Missing: t: 0.6039000339664229, p: 0.5459453477422531, critical: 1.9605744804334273
Goalkeeper, Midfield: t: -0.8879167684936564, p: 0.37460192993300134, critical: 1.9601462452819385
Goalkeeper, Defender: t: -0.5752719319315933, p: 0.5651166337670872, critical: 1.9601332171941026
Goalkeeper, Goalkeeper: t: 0.0, p: 1.0, critical: 1.960283921416078
Defender, Missing: t: 0.7465039019615111, p: 0.45537980486190843, critical: 1.9601901561624726
Defender, Midfield: t: -0.4388844668147827, p: 0.6607500852760558, critical: 1.9600849027406235
Defender, Defender: t: 0.0, p: 1.0, critical: 1.9600790271210176
Defender, Goalkeeper: t: 0.5752719319315933, p: 0.5651166337670872, critical: 1.9601332171941026
Midfield, Missing: t: 0.8221523594899994, p: 0.4110108487174301, critical: 1.9602140444830949
Midfield, Midfield: t: 0.0, p: 1.0, critical: 1.9600914108392373
Midfield, Defender: t: 0.4388844668147827, p: 0.6607500852760558, critical: 1.9600849027406235
Midfield, Goalkeeper: t: 0.8879167684936564, p: 0.37460192993300134, critical: 1.9601462452819385
Missing, Missing: t: 0.0, p: 1.0, critical: 1.966612518845396
Missing, Midfield: t: -0.8221523594899994, p: 0.4110108487174301, critical: 1.9602140444830949
Missing, Defender: t: -0.7465039019615111, p: 0.45537980486190843, critical: 1.9601901561624726
Missing, Goalkeeper: t: -0.6039000339664229, p: 0.5459453477422531, critical: 1.9605744804334273

The output indicates that Attackers are the only position that has a statistically significant difference in birth month are the attackers. For each one of them, the attackers have a positive t-stat, which means that the first groups mean is greater than the second groups mean. In other words, attackers are on the whole born later in the year than (closer to December) than the rest of the positions.

We can make this loop more efficient.

pos = df["position"].unique()
for i in range(df["position"].nunique() - 1):
    for j in range(df["position"].nunique() - 1, 0 + i, -1):
        p1 = df.loc[df["position"] == pos[i], "b_month"]
        p2 = df.loc[df["position"] == pos[j], "b_month"]

        t_res = stats.ttest_ind(p1, p2)
        t, p = t_res
        d = t_res.df
        critical = stats.t.ppf(1 - 0.05/2, d)
        print(f"{pos[i]}, {pos[j]}: t: {t}, p: {p}, critical: {critical}")
Attack, Missing: t: 1.3018933996037771, p: 0.19298606369161414, critical: 1.9602269330132052
Attack, Midfield: t: 2.430376280320745, p: 0.015092777291315614, critical: 1.960094675154383
Attack, Defender: t: 2.935473934765965, p: 0.0033343422037524057, critical: 1.9600878382903932
Attack, Goalkeeper: t: 2.7425321225077792, p: 0.006105392486458355, critical: 1.960152997933216
Goalkeeper, Missing: t: 0.6039000339664229, p: 0.5459453477422531, critical: 1.9605744804334273
Goalkeeper, Midfield: t: -0.8879167684936564, p: 0.37460192993300134, critical: 1.9601462452819385
Goalkeeper, Defender: t: -0.5752719319315933, p: 0.5651166337670872, critical: 1.9601332171941026
Defender, Missing: t: 0.7465039019615111, p: 0.45537980486190843, critical: 1.9601901561624726
Defender, Midfield: t: -0.4388844668147827, p: 0.6607500852760558, critical: 1.9600849027406235
Midfield, Missing: t: 0.8221523594899994, p: 0.4110108487174301, critical: 1.9602140444830949