Assessing the quality of Brandon Beane’s drafts is a hotly debated topic among Buffalo Bills fans, and one to which I assume they ultimately want a definitive answer. I’ve set out to provide that answer, and it goes far beyond how I personally feel the draft work of the club’s president of football operations and general manager.
Let me explain the methodology of my study here, which at the outset, may seem complex but in reality is straightforward.
I have no problem admitting the awesome work done
by Leah Cammarano was the main inspiration for this piece, yet I wanted to conduct a slightly different study. If I understand her explanation correctly, she essentially found how much better a GM improved his respective team’s drafting compared to previous regimes…and ironically determined Beane was the best among NFL GMs.
My study’s focus was simply to determine how well Beane has drafted based on the expectation of NFL success for every single draft pick number.
To find a baseline expectation for each pick number, I used Pro Football Reference to snag 2003 – 2017 draft results for a clean 15-year sample to establish a “baseline” or expectation of NFL performance for each individual pick.
I utilized PFR’s Weighted Approximate Value (wAV) as the core metric to establish how good/bad a player has performed in the NFL.
PFR’s founder Doug Driner created AV, which, directly from the site is “an attempt to put a single number on the seasonal value of a player at any position from any year.”
In football analytics, AV is one of the foundational ways to compare player-vs-player at different positions in the NFL. For this study, I used Weighted AV because “it aims to measure how good a player was, rather than just how long they played. It rewards “peak” production. A player with three amazing seasons and two average ones will have a higher wAV than a player who was just “okay” for eight straight years.”
Here’s a chart that describes how wAV distinguishes players throughout their careers:
Now you have good perspective on wAV tiers. I will also note, I only used the Bills draft classes from 2018 – 2023, because even though this is a weighted metric, rookies and even second-year players simply don’t have a large enough sample to make a concrete determination on how good/bad they are as professionals just yet.
Whether you think that skews things or not is up to you. It just felt silly to include even the 2024 class in this study at this juncture, but of course, future updates will include those Bills players with three or more seasons in the NFL.
And now, for the study’s core findings.
(Before I start, I certainly don’t want to lose you, so feel free to skip down to the next paragraph now. For the data advocates out there, I did not simply use the wAV “average” for each individual draft pick number because I knew doing so would create noisy and unrealistic swings from pick to pick. I utilized a smoothed curve to establish more reasonable expected value for each draft pick number. For example, let’s say because three All-Pros were selected at No. 56, that pick carried a significantly higher wAV than pick No. 57. In actuality, the expected wAV for those two draft positions should be very close, hence the smoothed curve approach.)
Here are the Bills’ year-by-year results, with “Total Residual” on the right-hand column simply the sum of how much better or worse Buffalo’s draft picks have performed compared to what teams have historically gotten from that draft pick number. The formula is career wAV – expected wAV:
Rock solid across board from a yearly draft class perspective for Bills. The 2018 class stands out as especially spectacular, thanks in large part to the selection of Josh Allen. Then again, he wasn’t the only stellar pick in Beane’s first as Buffalo’s GM. Tremaine Edmunds, Harrison Phillips, and Taron Johnson have all outperformed their expected wAV based on draft pick position. Ironically though, the second-largest positive residual after Allen in the 2018 class didn’t play for the Bills very long — fifth-round selection Wyatt Teller.
While Beane hasn’t been able to rekindle the magic of 2018 — which set an enormously high bar — the collective draft class residuals have all been distinctly positive ever since, expect for 2023, a class dinged by Dalton Kincaid’s inability to stay healthy. Believe it or not, Kincaid’s wAV is actually currently negative (expected 30.87 vs. actual 17.0).
Here are the round-by-round results for Beane’s drafts from 2018 – 2023:
And my round-by-round takeaways:
Round 1 – As mentioned earlier, Allen significantly carries this group. His current wAV of 111 accounts for nearly 40% of the six total selections Beane made from 2018 – 2023 in Round 1, which is amazing. Allen’s +67.2 residual alone covers the collective negative residuals of Kaiir Elam (-23.13), and Kincaid (-13.87). Interestingly, Ed Oliver’s residual is almost exactly zero (+0.70) — which would indicate he’s performed exactly as expected.
Round 2 – Only slightly above average, which makes sense qualitatively too. Successful outcomes for second-round selections James Cook and O’Cyrus Torrence are balanced out by misses in Carlos Basham and Cody Ford, and A.J. Epenesa lands smack dab in the middle of those two pairs.
Round 3 – Regularly excellent, yet not a round that has featured tape-measure home runs. Think Harrison Phillips, Devin Singletary, Dawson Knox, Zack Moss, Terrel Bernard, Spencer Brown, and Dorian Williams.
Round 4 – Not many selections here, yet relative to expectation, these are where the home runs have resided with the like of Gabe Davis and Taron Johnson. Oddly enough, those two were the only fourth-round picks the Bills made in the timeframe used in this study.
Rounds 5-7 – Subtly above-average across the board. Quite naturally, history suggests players in this portion of the draft rarely amount to anything in the NFL, so the likes of Ray-Ray McCloud, Austin Proehl, Vosean Joseph, Jaquan Johnson, Marquez Stevenson, Rachad Wildgoose, Luke Tenuta, Justin Shorter, and Nick Broeker were slightly outweighed by big hits such as Wyatt Teller, Khalil Shakir, and Christian Benford.
There you have it — despite some indisputable early-round misses, when all rounds are considered, based on history, the numbers say Beane has done a rather marvelous job drafting for the Buffalo Bills.
