Archive for January, 2006

The Dungy Index

I don’t know why this didn’t occur to me last year, but the methodology I applied to quarterbacks below makes as much sense — probably more sense — to apply to coaches rather than quarterbacks.

Since the methodology was described and discussed in the previous post, there’s not much to do here but post the list. So here goes.

                   expected actual Marginal
Coach                record record   wins
 Bill Belichick         7- 6  11- 2  +4.4
 Joe Gibbs             13-10  17- 6  +3.7
 Chuck Noll            12-12  16- 8  +3.9
 John Fox               3- 4   5- 2  +2.3
 Jimmy Johnson          6- 7   9- 4  +2.8
 Brian Billick          3- 4   5- 2  +2.2
*Tom Landry            16-15  19-12  +3.1
 Bill Walsh             8- 6  10- 4  +2.1
 Jerry Burns            2- 4   3- 3  +1.3
 Bill Parcells          9- 9  11- 7  +2.0
 Raymond Berry          2- 3   3- 2  +1.3
 Bum Phillips           3- 4   4- 3  +1.4
 Tom Flores             7- 4   8- 3  +1.5
 Barry Switzer          4- 3   5- 2  +0.9
 Don McCafferty         3- 2   4- 1  +0.7
*John Madden            7- 7   8- 6  +0.8
 John Robinson          3- 7   4- 6  +0.7
 Dan Reeves            10-10  11- 9  +0.8
 Mike Holmgren         10- 9  11- 8  +0.8
 Mike Shanahan          7- 6   8- 5  +0.7
 Herman Edwards         1- 4   2- 3  +0.5
 Jon Gruden             5- 3   5- 3  +0.4
 Jerry Glanville        3- 4   3- 4  +0.4
 Ray Malavasi           3- 3   3- 3  +0.4
 Marv Levy             11- 8  11- 8  +0.3
 Jeff Fisher            5- 4   5- 4  +0.4
 Sam Wyche              3- 2   3- 2  +0.2
 Andy Reid              7- 5   7- 5  +0.0
 Bill Cowher           11- 9  11- 9  -0.2
 George Seifert        10- 5  10- 5  -0.2
 Dick Nolan             2- 3   2- 3  -0.0
 Tom Coughlin           4- 5   4- 5  -0.3
 Ted Marchibroda        2- 4   2- 4  -0.2
 Dave Wannstedt         2- 3   2- 3  -0.1
*Bud Grant              9- 9   8-10  -0.7
 Bobby Ross             4- 4   3- 5  -0.6
 Chuck Knox             8-10   7-11  -1.0
 Art Shell              2- 3   2- 3  -0.4
 Steve Mariucci         4- 3   3- 4  -0.6
 Don Coryell            4- 5   3- 6  -0.8
 Tony Dungy             6- 7   5- 8  -1.1
 Dick Vermeil           7- 4   6- 5  -1.0
 Red Miller             2- 3   2- 3  -0.5
*Don Shula             19-12  17-14  -2.1
*George Allen           3- 4   2- 5  -1.0
 Mike Martz             4- 3   3- 4  -1.0
 Jim Fassel             3- 2   2- 3  -1.0
 Mike Ditka             7- 5   6- 6  -1.5
 Mike Sherman           3- 3   2- 4  -1.2
 Dennis Green           6- 6   4- 8  -2.0
 Wayne Fontes           2- 3   1- 4  -1.5
 Jack Pardee            3- 3   1- 5  -1.8
 Marty Schottenheimer   9- 8   5-12  -3.7
 Jim Mora               3- 3   0- 6  -3.3

[Technical note: this list includes all coaches who coached in at least five postseason games since 1970, and only includes games coached since 1970. The coaches marked with an asterisk coached in one or more postseason games prior to that, but those games are not counted.]

This measures a coach’s postseason performance relative to his postseason expectations. So if a coach, say Marty Schottenheimer, scores poorly here, it’s not clear whether he deserves blame for underachieving in the postseason or credit for overachieving during the regular season. With my next post, I’ll take a stab at ranking the coaches based on their regular season performance only.

The Manning Index Revisited

Last year in this space, I observed that Peyton Manning’s teams had won exactly as many playoff games as they had been the higher seed in. This fact, to my mind, ran contrary to the popular wisdom that Manning is a choker. So I came up with something I called the Manning Index, which essentially measures how many playoff games a quarterback has won compared to how many he “should have” won. Click the link above for more discussion; I won’t re-hash much of it here, but I will give a quick summary of the specifics.

Based on a logit regression of all playoff games during the past 30 years, I arrived at this formula for determining the probability of a given team winning a given playoff game:

Probability of winning = (1 + exp(-.43(windiff)-.24(homefield)))^(-1)

where windiff is the team’s wins minus the opponent’s wins and homefield is 1 if it’s a home game, -1 if it’s a road game, and 0 if it’s at a neutral site (i.e. a Super Bowl). For an example, let’s look at the Steelers’ and Seahawks’ 2005 playoff runs:

                                           prob of
Game                  windiff   homefield  winning
Steelers vs. Bengals      0        -1        44.1%
Steelers vs. Colts       -3        -1        18.1%
Steelers vs. Broncos     -2        -1        25.3%
Seahawks vs. Redskins    +3        +1        81.9%
Seahawks vs. Panthers    +2        +1        74.7%

Roethlisberger’s team won three games this postseason, when it should have been expected to win only about .88 games, so I’ll give Big Ben credit for 3 – .88 = 2.12 wins worth of clutchness. As I said last year, I think that awarding wins to quarterbacks is a suspect practice, but people are going to do it anyway. My only goal here is to put a quarterback’s postseason win-loss record into the proper perspective.

The main point of this post is to refresh the rankings with another year’s worth of data now in the books. So here they are. Some discussion follows.

                      Expected  Actual  Marginal
Quarterback            record   record    wins
Tom Brady               6- 5    10- 1     +4.4
Trent Dilfer            3- 3     5- 1     +2.2
Jake Delhomme           3- 4     5- 2     +2.3
Ben Roethlisberger      2- 3     4- 1     +1.6
Jeff Hostetler          2- 2     4- 1     +1.5
Mark Rypien             3- 5     5- 3     +1.6
Wade Wilson             2- 4     3- 3     +1.3
Joe Montana            14- 9    16- 7     +2.3
Troy Aikman             9- 7    11- 5     +1.8
John Elway             12-10    14- 8     +1.8
Jay Schroeder           2- 3     3- 2     +0.6
Drew Bledsoe            3- 4     4- 3     +0.7
Doug Williams           3- 4     4- 3     +0.6
Phil Simms              5- 5     6- 4     +0.6
Jim Everett             2- 3     2- 3     +0.4
Brad Johnson            3- 4     4- 3     +0.5
Brett Favre            10-10    11- 9     +0.8
Mark Brunell            4- 6     4- 6     +0.5
Jim Harbaugh            2- 3     2- 3     +0.3
Steve McNair            5- 4     5- 4     +0.4
Kurt Warner             5- 2     5- 2     +0.1
Rich Gannon             4- 4     4- 4     +0.1
Stan Humphries          3- 3     3- 3     +0.0
Donovan McNabb          7- 5     7- 5     +0.0
Jim Kelly               9- 7     9- 7     -0.3
Dave Krieg              3- 6     3- 6     -0.4
Kerry Collins           3- 3     3- 3     -0.3
Vinny Testaverde        2- 3     2- 3     -0.4
Bernie Kosar            3- 4     3- 4     -0.5
Jake Plummer            2- 4     2- 4     -0.5
Mike Tomczak            4- 2     3- 3     -0.6
Steve Young             9- 5     8- 6     -1.1
Dan Marino              9- 9     8-10     -1.4
Randall Cunningham      4- 6     3- 7     -1.2
Kordell Stewart         3- 2     2- 3     -1.0
Peyton Manning          5- 4     3- 6     -1.6
Jim McMahon             4- 2     3- 3     -1.2
Warren Moon             5- 5     3- 7     -1.9

First note that the records are all rounded to the nearest integer — records just don’t look right if they’re not integers — but the Marginal Wins column is not rounded (well, less rounded). Also, note that the list is sorted not by the Marginal Wins column, but by an approximation of the probability that an average quarterback would achieve the given record or a better one by sheer chance. For example, Joe Montana is +2.3 wins and Jeff Hostetler is +1.5, but Hoss rates higher than Joe because it’s less likely that random chance would produce a 4-1 record in the games Hostetler played in than that it would produce a 16-7 record in the games Montana played in. Incidentally, only one of the 38 guys on the list appears to be significantly better than chance, and none are significantly worse than chance. Make of that what you will.

Bonus fun fact I uncovered while running these numbers: according to the formula given at the top of this post, this season’s Steeler team is the most improbable Super Bowl team in history. Their estimated win probabilities were .441, .181, and .253, which means that their probability of winning all three (making all the usual incorrect assumptions about independence) was about .02, which is the lowest figure of any team to ever make a Super Bowl. Now that’s not too surprising, since they played three games and most Super Bowl teams only play two. But if you throw out the Cincinnati game, their probability would be .045, which would still be the lowest in history.

Most Improbable Super Bowl Teams

Team    Probability
pit 2005    2.0
nwe 1985    5.1
dal 1975    5.3
car 2003    7.6
ram 1979    8.6
bal 2000    9.9
oak 1980   10.9
ten 1999   11.2
sfo 1988   11.5
den 1997   12.3
buf 1992   12.8

Big Play Margin

How often have you heard, “the team that forces more turnovers will be the team that wins this game”?

Now how often have you heard, “the team that makes more big plays from scrimmage will be the team that wins this game”?

I haven’t kept an official tally, but I’d guess that the first assertion is made more often than the second. In fact, my impression is that when big plays are mentioned, it’s big plays on special teams that get more attention than big plays from scrimmage. It could be that people don’t mention big plays from scrimmage because it’s assumed that they’re important. Or it could be that people really believe that turnovers are more important.

I find this interesting becuase in many situations, a big play from scrimmage is exactly analagous to a turnover. If your running back rips off a 40-yard gain on 3rd-and-2, that’s the same result as if he had been stuffed, you punted, and then your defense forced a turnover on the next play. And we’ve all seen those interceptions 45 yards downfield that are the same as a punt. Granted, those are contrived examples and the correspondence isn’t always so clean, but it seems reasonable — at least to me — that the value of a big play from scrimmage is roughly comparable to the value of a turnover. Some theoretical evidence for this position can be found in The Hidden Game of Football, where the authors conclude that a turnover is, on average, worth about 40 yards of field position.

I decided to see how this plays out in practice. So I ran a regression with team wins as the output, and turnover margin and big play margin as the inputs. I defined a big play as any play from scrimmage that gained 30 or more yards. I didn’t include big plays on special teams simply because I don’t have that data in an easily-accessible format. This is based on data from 2003–2005. Here are the results:

Estimated Team Wins = 8 + .165 * (TurnoverMargin) + .163 * (BigPlayMargin)

What this says is that, over the course of a season, an increase of one in turnover margin will add about .165 to your win total, while an additional big play from scrimmage is worth .163 wins. To put it in less absurd terms, you can expect to gain an extra win in a 16-game season by forcing six extra turnovers. Or by allowing six fewer.

If you found the discussion above convincing, you shouldn’t be surprised to see that the two coefficients are so close. A big play is about as valuable as a turnover; no more, no less.

Or is it? It could be that big plays aren’t important at all. It could be that big plays are simply the byproduct of good offenses (and prevention of big plays is the byproduct of a good defense). In other words, it could be that good teams have good big play margins and good teams win games, but that big plays are incidental to the process. To check for this, I included the teams’ season yardage margin: total yards gained minus total yards allowed.

Estimated Team Wins = 8 + .163 * (TurnoverMargin) + .070 * (BigPlayMargin) + .0017 (YardageMargin)

The important thing to note here is that the BigPlayMargin coefficient is significant. That says that big plays do matter, even over and above the yardage they provide. A 50-yard play adds 50*.0017 + .070 = .155 to the win column, almost the same as a turnover.

Reputation and information

Who is a better football player: David Carr or Steve Hutchinson?

David Carr was the first overall pick in 2002, and the jury is still out on him. Some people believe he only needs a decent offensive line, a change of scenery, some better receivers, or some combination thereof; others believe he’s simply a bust. Steve Hutchinson, for you casual football fans who wandered in here not prepared for a quiz, plays guard for the Seahawks, and does so spectacularly. Let’s set aside for the moment the debate about the value of a mediocre quarterback versus a top-notch guard, and focus on who does his job better. You ask 100 knowledgeable football fans, and all 100 will tell you that Hutchinson is a much better guard than Carr is a quarterback.

That’s reputation.

Now let’s think about information. On what information are we intelligent football fans basing that judgement? As usual, this is easy for baseball fans. If we are debating baseball players, we can start by looking at the numbers. We may quibble about which numbers to look at or exactly what kind of adjustments we need to make to those numbers, but ultimately we feel pretty confident that the players’ statistics — if properly interpreted — tell us what we need to know.

It’s not so easy in football. There are, of course, plenty of stats on Carr. But everybody knows that they reflect not only his own performance but also his teammates’ performance, his coach’s offensive philosophy, and so forth. We know his numbers are tainted, but we’re willing to use them anyway, at least as a rough estimate.

For Hutchinson, we don’t even have that much. In fact, we’ve got nothing. Those of you who have broken down enough game film of Hutchinson, and have broken down enough game film on every other guard in the league, and who really know enough about guard play to know what you’re watching for, you can speak with authority. You form a miniscule fraction of the football watching population, though. For the rest of us, the only stat we have on offensive lineman is the number of pro bowls to which they’ve been named. Either directly or indirectly, that’s how most of us form our opinion of the quality of offensive linemen. Hutchinson has been to three straight pro bowls. He’s good. Who are the best offensive lineman in football? Jonathan Ogden? Nine straight pro bowls. Orlando Pace has been to seven straight. Willie Roaf? Eleven pro bowls in the last 12 years. Will Shields? Ten pro bowls in the last 11 years. Larry Allen likewise has gone to 10 in the last 11 years.

Do you know how many non offensive linemen have been to 10 pro bowls in 11 years, as Roaf, Shields, and Allen have? Concentrating for the moment just on the offensive side of the ball, here is the breakdown:

Number of players who have been to 10 pro bowls in an 11-year span
Offensive linemen – 9
Quarterbacks – 1
Running backs – 1
Wide Receivers – 1
Tight Ends – 0

In the history of the NFL, Johnny Unitas, Jerry Rice, and Barry Sanders are the only three players on the offensive side of the ball who played their position as consistently well as Will Shields, Willie Roaf, and Larry Allen have played theirs. Now, it is true that offensive linemen generally have longer careers than the so-called skill position players, but here is an equally revealing glimpse into the pro bowl voting:

Pro Bowl “retention rates”
Offensive line – 60%
Tight Ends – 52.6%
Wide Receivers – 45.5%
Running Backs – 43.4%
Quarterbacks – 43.2%

In other words, 60% of the offensive lineman who made the pro bowl in year N also made it in year N+1, while only 43% of quarterbacks were able to retain their pro bowl status from one year to the next. The contrast is even sharper when you realize that 19% of the league’s starting quarterbacks make the pro bowl in a given year but only 10% of the league’s starting offensive linemen do (6 of 32 QBs, 16 of 160 lineman), so it should be easier for quarterbacks to repeat. Of course it is possible that there is something inherent about the positions that makes skill position players much more volatile — pitchers are naturally more volatile than hitters, for example — but I’m suspicious.

If you ranked the positions on the offensive side of the ball in terms of how many statistics are available to describe the performance of players at that position, you would get a list that is in the exact opposite order of the above list. Quarterbacks are measured in several passing and rushing categories, including the complex passer rating formula. Running backs have rushing statistics and receiving statistics. Wide receivers have only receiving statistics. Tight ends also have only receiving statistics, but those receiving statistics measure a smaller part of the tight end’s job than of the wide receiver’s job.

Even when you include defensive players, the correspondence remains. Retention rates for linebackers, defensive backs, and defensive linemen — for whom we have a couple of statistics (sacks and interceptions), but not many — all fall between those of the offensive lineman and the wide receivers. Now that tackles are becoming a more standard statistic for defensive players, it will be interesting to see if their pro bowl retention rates fall. I bet they do.

The less information we have, the more we have to rely on reputation. We evaluate skill position players on some combination of information (stats) and reputation. For linemen, the information is the reputation, so we get a self-fulfilling prophecy. Offensive lineman make the pro bowl because they’re good. And we know they’re good because they make the pro bowl.

Sabernomics Super Bowl Extravaganza II

Very soon, Doug Drinen of Pro-Football-Reference will start posting on football related matters from now until the Super Bowl. If you missed it last year, here is a list of his posts.

Squares for Squares

Ranking the All-Time Great Wide Receivers
Joey Galloway?
The Manning Index

Welcome back, Doug. I look forward to reading your posts.

Pinto’s 2005 PMR

Dave Pinto is starting to post the results of his Probabilistic Model of Range over at Baseball Musings. People often e-mail me to ask what defensive metric I think is the best. The answer is easy: PMR.

Not only will Dave be rating players by positions, he occasionally adds a few other bits of relevant commentary. For example, he finds Horacio Ramirez to be the 8th-luckiest pitcher on balls in play in 2005.

Pro-Football-Reference Sponsorships

Pro-Football-Reference is now open for sponsorships. In order to get the ball rolling, Doug Drinen—P-F-R owner, Sabernomics contributor, and fellow Bobtown resident—is offering an exciting offer to Sabernomics readers.

For a limited time, if you open an account and then send Doug an e-mail letting him know that you heard about it on Sabernomics, Doug will match your initial donation dollar for dollar (e.g., put in $20, and he’ll put in another $20 so that you can sponsor $40 worth of pages). If you already have an account because you’ve sponsored some Baseball-Reference pages, then whatever you spend on P-F-R sponsorships in the next few days, Doug will put a matching amount into your account.

Pro-Football-Reference is the home for on-line football data. I have already taken the opportunity to sponsor a few pages of my own.

Jerry Richardson
Steve Smith
Julius Peppers

For those of you who enjoyed Doug’s Super Bowl Extravaganza last year, you will be happy to know that Doug will be doing the same thing again in just a few weeks.

Website Issues

I am almost certain that there will be significant disruptions with the site later today. This includes the possibility of the site going down for several days. I promise this is temporary, but I will be back up and running as quickly as possible.

Lucky Ellis

I see that Mark Ellis and the A’s are heading for aribitration. Using PrOPS I found that Ellis had the luckiest season in the majors last year. It’s the second luckiest season in the past four years (the luckiest was Scott Podsednik’s 2003). Read more about it in The Harball Times Baseball Annual 2006.

More Love for Murph

Frank Stephenson at Division of Labour pointed me to Kyle Wingfield’s column in yesterday’s Wall Street Journal. It’s a pay site, but I’ll post a brief quote.

…if voting patterns for the 1950s, ’60s and ’70s are any indication. In each of those decades, an average of 22 Hall of Famers played the most significant part of their careers — meaning a majority or near-majority of their statistical production came in that decade. …

So far, the corresponding number for the ’80s is only 13. Three more players — Tony Gwynn, Rickey Henderson and Cal Ripken Jr. — are sure to enter Cooperstown in the coming years. But that still leaves the ’80s six Hall of Famers short of the three previous, and largely comparable, decades.

Wingfield attributes the current “Steroid Era” as the cause of 80s players getting less respect. While I think the growing offensive numbers may be part of the reason, I don’t see much evidence that steroids is the cause of the rise in those numbers.