The Use and Misuse of Statistics

[Target-Man]

Did you know that Lucas is as good a player as Mascherano? I wasn't so sure, and I'm still not, but just the other day the OP of a thread claimed this as a practically indisputable fact based on a statistical breakdown of the performance of the two players over the course of last season.

The point has been made several times during the past 12 months, and indeed it is remarkable how similar the two players look when reduced to numbers and percentages. But while such stats can be illuminating, it might be time to consider, how much and exactly what you can actually prove by breaking down performances into numbers like this.

As the thread mentioned started with the claim that the stats offered "proof", it is safe to say that there is a school, which sees statistical analysis as offering incontrovertible evidence that can be used to establish a claim as an indisputable fact. Opposed to that is the notion that football cannot be effectively analyzed statistically in a way that sheds light on any deeper truths.

The blog entry, which the OP mentioned referred to, offered a "stat off" between Mascherano and Lucas and made some basic comments to the effect that the stats were remarkably similar, but made no far-reaching conclusions, leaving the statistics to speak for themselves. The question is: Can they really do that?

Let us take some examples. The stats mentioned include:

Tackles attempted: Masch 268, Lucas 285
Tackles won: Masch 181, Lucas 180
Success Percentage: Masch 67%, Lucas 63%

A similar method is applied to passing.

Indeed, on the basis of these numbers, you would conclude that the two players were just about equally good. The stats make a compelling case, and they are hard to argue with. But in all this, it is easy to forget an important question:

Are all tackles equal?
Are all passes equal?

In other words, what do these numbers really reveal when taken out of context.

This is an important question, because it is possible to perform a thorough statistical analysis even of relevant factors without revealing much of any value.

For example, few would argue against the ability to write complete, well-constructed sentences or conjugate verbs correctly being important parts of a writer's craft, but I doubt that anyone would find an argument over the greatness of Hemingway vs. Steinbeck based on a statistical breakdown of the number of well-constructed sentences or correctly conjugated verbs very illuminating.

The things that really matter in this comparison defy statistical analysis. We should make sure, that our statistical analysis of football players doesn't become equally nonsensical.

And here the problem is that passes are not equal, and neither are tackles for that matter.

The case of the passes is probably the most illustrative one. It should be safe to say that the concept of statistical analysis of sports is one that is imported from the US, where it plays an important part in sports coverage, and where it can indeed seem very illuminating at times. But when it comes for example analyzing completion percentages, there is a fundamental difference between NFL football and the game that the rest of the world calls football. When a quarterback completes a pass, it is practically always positive. It benefits his team every time he completes a pass, and so his completion percentage is a valid measure of his performance, even if it cannot necessarily stand alone.

Not so in "real" football. Here the players with the highest pass completion percentages are often players that only rarely make any key passes. The two CBs in a back four will often have very high completion percentages, simply because they play many passes without being under any kind of pressure, and their completion percentages say next to nothing about their ability to make a key pass to send a teammate clean through on goal.

In NFL football, passes are equal. In association football they are not.

And the reason why passes are not equal is, that the difficulty and value of completing it varies greatly with the position that it is played in and with its purpose. It also varies greatly with the quality of the opposition.

Even when you compare two players in similar positions, the all season breakdown doesn't take the quality of the opposition into account. It is possible to argue that there is a randomization factor, which for all practical purposes evens out this difference, but with teams rotating and resting key players for big matches, that is by no means certain. And numbers from a game against Reading are not really comparable to numbers from a game against Real Madrid.

In other words, two players can arrive at the same number in very different ways and under different circumstances, meaning that a 90% completion rate is not necessarily comparable to a 90% completion rate. Which obviously reduces the value of this number as "proof" of a player's quality vis-a-vis another player.

The same is true for tackles.

Completely lost in the statistical breakdown is, of course, the intangibles that do not lend themselves to easy quantification. The impact of a player on his teammates is hard to measure, but can be quite decisive in real match situations. Two CBs can have very similar numbers, but in a real situation the fact that one is very slow while the other is very fast might greatly affect the willingness of their teammates to push forward, depending on who is behind them.

All this limits the reach of the conclusions that you can draw from a statistical breakdown of players. Statistics can be highly illuminating in some cases, and can often function as a valuable corrective to deeply held prejudices, but as proof they are often more ambiguous than they seem.

To return to the question of the thread mentioned in the beginning, this means that the stats do not really "prove" that Lucas is as good as Mascherano. Nor do they prove the opposite. They strongly suggest that they were about equally effective in the matches they played during that season, which is not quite the same given that the quality of the opposition could have varied. But they don't prove it, because the impact on teammates is not quantified and taken into account.

So in determining the quality of two players compared to each other, the stats don't suffice, even if they can be helpful at times. There is, in the end, not really any way around the classic Eyeball Mk. I analysis of how the players perform over a number of matches against the best opposition available.

[ZouZizouZizou]

Said it before and I'll say it again you're the undisputed Sage of the boards.

However I disagree with you on one point, you wrote "Not so in "real" football. Here the players with the highest pass completion percentages are often players that only rarely make any key passes.", Xavi, Iniesta, Sniejder, Alonso, Pirlo and Scholes are examples of top players whose completion rates remain high even as they attempt more difficult and adventurous passes.

[Target-Man]

Said it before and I'll say it again you're the undisputed Sage of the boards.

However I disagree with you on one point, you wrote "Not so in "real" football. Here the players with the highest pass completion percentages are often players that only rarely make any key passes.", Xavi, Iniesta, Sniejder, Alonso, Pirlo and Scholes are examples of top players whose completion rates remain high even as they attempt more difficult and adventurous passes.

Which is exactly the point: A 90% completion rate for Lucas is not the same as 90% completion rate for Iniesta. The two numbers are not equal.

The word "often" is the key word here. It is not unusual for players in defensive positions to have higher completion rates than more creative passers in more creative positions. The fact that a few of the latter can match the former when it comes to completion rate is a testimony to their exceptional ability.

[-MabelMeister-]

Good post.

Stats don't prove anything really, they can be used to help prove a point, but they're not the be all end all of a discussion, unless that discussion is "Masch has a higher tackle completion percentage than Lucas".

You say in your OP that stats are taken out of context. That's very very true. Making a successful tackle on Drogba is a world different than making a successful tackle on Ebanks-Blake.

The only way to really know for sure who is the better player is to watch the football being played.

[Target-Man]

TM,you would make a top analyst.Your thoughts on football twist my head.

Thank you.

But sometimes it's not really a question of football as much logic and a bit of critical thinking, something which is unfortunately not always present among pundits and sports journalists.

I had a bit of fun last year with the journalists and pundits, who made a lot out of us conceding a relatively large proportion of our goals against on set pieces. Taking their logic to its ultimate consequence, I came up with a brilliant suggestion for improving our defense.

[Valmont]

i had this debate before. That anything half baked is worse than not having it at all.
I am an analyst in a financial company. And hence i live by stats so always get this tendency to defend statistics.

To take this lucas vs masch example. Yes just passes and tackles are not comparable and TM is perfectly right that 90% pass completion rate is not same for every player.

But that does not mean stats cannot tell you a better story. For example, let me put up some examples of what other stats i think should be included in a masch vs lucas debate.

1) Speed of each player.
2) Defensive aerial ability.
3) Time difference between the player getting the ball and the player making he pass.
5) Possession of our team( by region of the pitch) when one plays.
6) Possession of opposing team(by region) when one plays.
7) Interceptions.
8) Covering up of an opposition player to force him to play the ball back.
9) Winning the ball back(by region)
10) Shots and attempts at our goal when they play.

And most of all, before each match there should be a handicap given to the starter based on our starting team and the opposing team. So for example, spearing and darby starting for us will make the DM's job harder and hence the stats should be skewed that way.

Also something like a real time tool to decide the ranking of the best direct choices of a pass from the 10-15 odd choices he might have each time he gets possession. And then score them in a match on their choices made from those.

Also another graphical view to rate the position of the player receiving the ball from these two. This will be ranked on the receiving player's position in the pitch, players around him, his distance and orientation to the goal, the difficulty with which he had to control the pass and etc.

I can go on and on but my point is that if i add enough stats for each player, i will get the correct answer. What stats also do is remove the subjectivity of an opinion. For example, in a counter attack against us, masch running all the way and making a successful do or die sliding challenge to get the ball out of the play will remain in our memory for long.
But lucas just covering up the line and forcing the ball to be passed back while we recover will just be a simple play having similar results. And without a combination of correct stats, one would always skew towards the sliding challenge.

Football,i know is a much harder sport to statistically define, but as a proponent of the method, i have no doubt that it can be done to a very high degree of accuracy.

[kopitecrash]

Target-man, you are the most tactically knowledgeable poster on here. Just thought I should say that.

I would say though that the more stats you have on a player and an element of his play, the more of a picture you can form, through stats. Just basic one such as tackles attempted and succeeded don't cut it. But taking into account the position of the tackle and percentages of the time the tackle develops in a scrap for the ball rather then cleanly winning posession, and other elements and you could arguably form more of a picture. Thus the benefits of prozone.

[Target-Man]

i had this debate before. That anything half baked is worse than not having it at all.
I am an analyst in a financial company. And hence i live by stats so always get this tendency to defend statistics.

To take this lucas vs masch example. Yes just passes and tackles are not comparable and TM is perfectly right that 90% pass completion rate is not same for every player.

But that does not mean stats cannot tell you a better story. For example, let me put up some examples of what other stats i think should be included in a masch vs lucas debate.

1) Speed of each player.
2) Defensive aerial ability.
3) Time difference between the player getting the ball and the player making he pass.
5) Possession of our team( by region of the pitch) when one plays.
6) Possession of opposing team(by region) when one plays.
7) Interceptions.
8) Covering up of an opposition player to force him to play the ball back.
9) Winning the ball back(by region)
10) Shots and attempts at our goal when they play.

And most of all, before each match there should be a handicap given to the starter based on our starting team and the opposing team. So for example, spearing and darby starting for us will make the DM's job harder and hence the stats should be skewed that way.

Also something like a real time tool to decide the ranking of the best direct choices of a pass from the 10-15 odd choices he might have each time he gets possession. And then score them in a match on their choices made from those.

Also another graphical view to rate the position of the player receiving the ball from these two. This will be ranked on the receiving player's position in the pitch, players around him, his distance and orientation to the goal, the difficulty with which he had to control the pass and etc.

I can go on and on but my point is that if i add enough stats for each player, i will get the correct answer. What stats also do is remove the subjectivity of an opinion. For example, in a counter attack against us, masch running all the way and making a successful do or die sliding challenge to get the ball out of the play will remain in our memory for long.
But lucas just covering up the line and forcing the ball to be passed back while we recover will just be a simple play having similar results. And without a combination of correct stats, one would always skew towards the sliding challenge.

Football,i know is a much harder sport to statistically define, but as a proponent of the method, i have no doubt that it can be done to a very high degree of accuracy.

Now we're talking. Repped. (Except that I have apparently repped you too recently, so you'll have to do with the thought)

My argument is not with stats as such but rather with the way that a very limited set of statistics are used to make far-reaching conclusions that they simply do not justify. Part of the problem is that a few numbers taken out of context are often used uncritically by people with little training in using statistics.

While conclusive proof might be elusive, I certainly agree that the statistical analysis of player performance can be improved, possibly to a level where a fairly accurate abstract representation of a player's performance is possible. The question is then, whether the complexity of such a statistical representation might not limit its usefulness.

The scoring of passes has already been used by actual coaches, most notably Norway manager Egon "Drillo" Olsen.

As I said, stats can be used as a corrective to the kind of groupthink, which is fairly common among us football fans. And the statistics mentioned about Lucas and Masch have already stirred a debate, which has at least moved some people to reconsider their views concerning the respective abilities of the two players.

I would certainly not dismiss statistics per se. They have their use. Unfortunately, they also have their misuse, and that seems just as widespread. While some might reject the notion that stats can be used to decide the quality of a player categorically, I think we would need much better statistics before we can get anywhere near answering that question.

My point is essentially not to say that stats are useless, but rather to provide arguments to qualify the debate. Before we use stats, we need to be very clear about which conclusions can be drawn from which stats, and the football community as a whole is nowhere near this kind of clarity.

[HuytonHuyton]

Stats are facts on what they describe.

Pecentage of passers, number of passers. Stats do not prove who has the best passing ability etc...

If a player has 80%+ passing success it is fairly safe to say they are a decent passer considering thats in the top 5 passing success rates in the prem. Which Lucas and Masch had last season.

It proves they do not give the ball away alot, which helps the argument when people say mach and lucas's passing is terrible.

Take them with a pinch of salt and apply them where needed.

A pass in NFL is easy either. Say the passer had no one on him and had 10 seconds to pass it forward, compared to someone who had 2 men charging him with 2 seconds to get rid of it.

Stats are stats, not definitive facts on someones ability versus someone elses.

[Target-Man]

Stats are facts on what they describe.

Pecentage of passers, number of passers. Stats do not prove who has the best passing ability etc...

If a player has 80%+ passing success it is fairly safe to say they are a decent passer considering thats in the top 5 passing success rates in the prem. Which Lucas and Masch had last season.

It proves they do not give the ball away alot, which helps the argument when people say mach and lucas's passing is terrible.

Take them with a pinch of salt and apply them where needed.

A pass in NFL is easy either. Say the passer had no one on him and had 10 seconds to pass it forward, compared to someone who had 2 men charging him with 2 seconds to get rid of it.

Stats are stats, not definitive facts on someones ability versus someone elses.

Doesn't the NFL have stats on how quarterbacks pass when rushed? It's been a while since I've followed American football, but I seem to remember something about that.

But while there a several factors to take into account when trying to provide a statistical representation of a quarterback's passing ability, one thing is certain: It is always beneficial to his team when he completes a pass, and his passes are practically always offensive.

That is not the case in football. Many passes have little value, and many don't even really serve the purpose of maintaining possession. Indeed, many of the passes that are completed safely result in nothing except moving the ball from one unthreatening position to another.

The only safe conclusion that can be drawn from a high pass completion rate in association football is, that the player rarely loses the ball when making the kind of passes he makes.

[Jannno]

I've selected and copied the best 10 posts from T-Ms thread here - ie the ones that stuck to discussing the idea in general. Don't know if we would want to leave in the NFL post and replies to it...... and the debate is going on so there may be more good posts to come

[elasticman]

Did you know that Lucas is as good a player as Mascherano? I wasn't so sure, and I'm still not, but just the other day the OP of a thread claimed this as a practically indisputable fact based on a statistical breakdown of the performance of the two players over the course of last season.

The point has been made several times during the past 12 months, and indeed it is remarkable how similar the two players look when reduced to numbers and percentages. But while such stats can be illuminating, it might be time to consider, how much and exactly what you can actually prove by breaking down performances into numbers like this.

As the thread mentioned started with the claim that the stats offered "proof", it is safe to say that there is a school, which sees statistical analysis as offering incontrovertible evidence that can be used to establish a claim as an indisputable fact. Opposed to that is the notion that football cannot be effectively analyzed statistically in a way that sheds light on any deeper truths.

The blog entry, which the OP mentioned referred to, offered a "stat off" between Mascherano and Lucas and made some basic comments to the effect that the stats were remarkably similar, but made no far-reaching conclusions, leaving the statistics to speak for themselves. The question is: Can they really do that?

Let us take some examples. The stats mentioned include:

Tackles attempted: Masch 268, Lucas 285
Tackles won: Masch 181, Lucas 180
Success Percentage: Masch 67%, Lucas 63%

A similar method is applied to passing.

Indeed, on the basis of these numbers, you would conclude that the two players were just about equally good. The stats make a compelling case, and they are hard to argue with. But in all this, it is easy to forget an important question:

Are all tackles equal?
Are all passes equal?

In other words, what do these numbers really reveal when taken out of context.

This is an important question, because it is possible to perform a thorough statistical analysis even of relevant factors without revealing much of any value.

For example, few would argue against the ability to write complete, well-constructed sentences or conjugate verbs correctly being important parts of a writer's craft, but I doubt that anyone would find an argument over the greatness of Hemingway vs. Steinbeck based on a statistical breakdown of the number of well-constructed sentences or correctly conjugated verbs very illuminating.

The things that really matter in this comparison defy statistical analysis. We should make sure, that our statistical analysis of football players doesn't become equally nonsensical.

And here the problem is that passes are not equal, and neither are tackles for that matter.

The case of the passes is probably the most illustrative one. It should be safe to say that the concept of statistical analysis of sports is one that is imported from the US, where it plays an important part in sports coverage, and where it can indeed seem very illuminating at times. But when it comes for example analyzing completion percentages, there is a fundamental difference between NFL football and the game that the rest of the world calls football. When a quarterback completes a pass, it is practically always positive. It benefits his team every time he completes a pass, and so his completion percentage is a valid measure of his performance, even if it cannot necessarily stand alone.

Not so in "real" football. Here the players with the highest pass completion percentages are often players that only rarely make any key passes. The two CBs in a back four will often have very high completion percentages, simply because they play many passes without being under any kind of pressure, and their completion percentages say next to nothing about their ability to make a key pass to send a teammate clean through on goal.

In NFL football, passes are equal. In association football they are not.

And the reason why passes are not equal is, that the difficulty and value of completing it varies greatly with the position that it is played in and with its purpose. It also varies greatly with the quality of the opposition.

Even when you compare two players in similar positions, the all season breakdown doesn't take the quality of the opposition into account. It is possible to argue that there is a randomization factor, which for all practical purposes evens out this difference, but with teams rotating and resting key players for big matches, that is by no means certain. And numbers from a game against Reading are not really comparable to numbers from a game against Real Madrid.

In other words, two players can arrive at the same number in very different ways and under different circumstances, meaning that a 90% completion rate is not necessarily comparable to a 90% completion rate. Which obviously reduces the value of this number as "proof" of a player's quality vis-a-vis another player.

The same is true for tackles.

Completely lost in the statistical breakdown is, of course, the intangibles that do not lend themselves to easy quantification. The impact of a player on his teammates is hard to measure, but can be quite decisive in real match situations. Two CBs can have very similar numbers, but in a real situation the fact that one is very slow while the other is very fast might greatly affect the willingness of their teammates to push forward, depending on who is behind them.

All this limits the reach of the conclusions that you can draw from a statistical breakdown of players. Statistics can be highly illuminating in some cases, and can often function as a valuable corrective to deeply held prejudices, but as proof they are often more ambiguous than they seem.

To return to the question of the thread mentioned in the beginning, this means that the stats do not really "prove" that Lucas is as good as Mascherano. Nor do they prove the opposite. They strongly suggest that they were about equally effective in the matches they played during that season, which is not quite the same given that the quality of the opposition could have varied. But they don't prove it, because the impact on teammates is not quantified and taken into account.

So in determining the quality of two players compared to each other, the stats don't suffice, even if they can be helpful at times. There is, in the end, not really any way around the classic Eyeball Mk. I analysis of how the players perform over a number of matches against the best opposition available.

Stats need interpretation to have any meaning but of course they are useful. Successful passes - some of the spread betting companies only award a pt on a successful pass if it is in the oppo half for the reasons that you have stated. Clearly passes in the final 1/3 and key passes are the hardest of all. Statistics on all of these things are available through Opta, prozone, and others.

Where on the pitch is it easiest to make a tackle? I would presume in the defensive 1/3 rather than further up the pitch because a) attackers have less space and b) Players in the defensive areas seem to make more tackles than others.

If there is a whole in the picture it probably has much more to do with closing down that does not result in a tackle. This clearly has a massive tactical implication but doesnt show in the stats. Masch is the master at closing down and forcing attackers to be predictable.

[Target-Man]

For a more serious look at how to distort truth with numbers, here's a nice link, which will even teach you a neat new word: "proofiness"!

http://www.nytimes.com/2010/09/19/bo...tz-t.html?_r=1

[Liverdinner]

In NFL passes are equal?

[Target-Man]

In NFL passes are equal?

No, not completely, but in the NFL a completed pass is practically always beneficial to the team in that it, except in rare cases, will bring the team forward towards the opposition end zone. So in NFL you can with great certainty count on a high pass completion percentage being an indicator of good attacking play.

The difference between NFL and Association Football (i.e. our kind), when it comes to passing is, that in association football a completed pass is not necessarily good for the team, indeed it may often be either of no consequence or downright bad. Passing the ball from outside the opposition box and back to a centre back, who again passes it back to the keeper does not result in your team progressing towards a scoring opportunity. On the contrary. But it would still count as completed passes, even if the outcome is quite negative. That is why a player can have a high pass completion percentage without having any positive impact on your offense, something which is quite common among centre backs.

In the NFL, you can for all practical purposes count on every completed pass bringing you closer to scoring. That is why you can say that all completed passes are if not equal then at least comparable for statistical purposes.

Or to put in metaphorically:
In our kind of football, completed passes can be apples, oranges or lemons.
In the NFL, they can only be apples.

[Liverdinner]

No, not completely, but in the NFL a completed pass is practically always beneficial to the team in that it, except in rare cases, will bring the team forward towards the opposition end zone. So in NFL you can with great certainty count on a high pass completion percentage being an indicator of good attacking play.

The difference between NFL and Association Football (i.e. our kind), when it comes to passing is, that in association football a completed pass is not necessarily good for the team, indeed it may often be either of no consequence or downright bad. Passing the ball from outside the opposition box and back to a centre back, who again passes it back to the keeper does not result in your team progressing towards a scoring opportunity. On the contrary. But it would still count as completed passes, even if the outcome is quite negative. That is why a player can have a high pass completion percentage without having any positive impact on your offense, something which is quite common among centre backs.

In the NFL, you can for all practical purposes count on every completed pass bringing you closer to scoring. That is why you can say that all completed passes are if not equal then at least comparable for statistical purposes.

Or to put in metaphorically:
In our kind of football, completed passes can be apples, oranges or lemons.
In the NFL, they can only be apples.

Oh I see what you mean. Yeah stats there carry more weight in terms of evidence yeah.

[Simo429]

Stats can be used to say how good a player is at a certain aspect, they are better used to say how bad a player is at a certain aspect they cannot however be used to say how good a player is.

Taking Lucas as an example
Last year two of the main things used to criticise him was he was poor in the tackle and he gave the ball away too much.
Now the stats showed those ideas to be myths. That doens't then mean that Lucas is world class it just debunks those selected criteria.