The KD Certified Elite Pack Drop Controversy Explained
by Chris Otis
2/7/2022
Last Thursday, February 3, a randomization error plagued one of NBA Top Shot’s most hyped pack drops to date — the KD Certified Elite Pack Drop. In the first Top Shot pack drop to include odds to pull a Rare of Legendary in packs, these coveted Moments were not distributed randomly, and soon after all pack purchases were refunded.
But what actually happened on Thursday’s pack drop, how do things currently stand, and where do we go from here? Let’s dive deep into what went down.
KD Certified Packs Background
Setting the stage for all this is the fact that Top Shot is planning to spend the month of February experimenting with a new type of pack drop system, which is explained in more detail in this blog. This experimentation would start with the KD Certified Packs and the Game Recognize Game set.
In short, the big change is that Rare and Legendary Moments will be available in all packs, as opposed to being confined to specific drops designed for those tiers, such as Holo Icon or MGLE drops.
Additionally, instead of priority queues only accessible to collectors with high collector scores, all collectors who meet a certain collector score could join the same queue. However, collectors who meet certain score thresholds would be able to purchase multiple packs.
These new drops will be two-pronged, with Standard and Elite variations being available. Elite packs will increase the likelihood to score a Rare or Legendary Moment. Here are the odds for the KD Elite Pack drop:
The specifics will vary by release, but the Elite packs will always be more expensive, contain greater odds at hitting Rare or Legendary Moments, and be gated behind a sizable drop-specific Collector Score requirement, whereas the Standard packs will be available for most, if not all, collectors.
The KD Certified Elite Pack Controversy
The first of these new drops were set to be the KD Certified Standard and Elite packs, built around the new Game Recognize Game set featuring Moments narrated by Kevin Durant. The Elite packs dropped first on Thursday, February 3.
However, the Elite pack drop went extremely poorly, with collectors discovered a bug which caused the limited quantity Holo Icon and MGLE packs to be purchased in a non-random order.
Rather than be distributed randomly, packs with Game Recognize Game Moments were distributed first, followed by packs with an MGLE Moment, followed by packs with a Holo Icon Moment. This effectively resulted in an extremely unfair distribution of the MGLE and Holo Icon Moments, causing an uproar amongst the NBA Top Shot community and forcing Top Shot to delay the Standard pack drop until Monday, February 7.
Let’s take a closer look at what actually happened based on everything we know so far.
What Happened?
A few hours after the drop, Dapper Labs released this blog post as part of their official response, which details some of what went wrong that led to the pack distribution error.
Basically, the unfortunate error boils down to how the packs were created, and bafflingly poor internal testing design decisions that caused them to miss this bug. Here’s the Top Shot blog’s explanation.
So in essence, there were three different pack types within the same drop. What should have happened, of course, is that they would be randomized, so that each pack type was distributed fairly throughout the queue. But as can be seen below, this did not happen, and the overwhelming majority of packs were purchased in order from GRG, to MGLE, to Holo, and then back to GRG at the end.
This uproar was especially amplified because some collectors could buy up to five packs (as planned), which could each contain one Holo Icon Moment based on where they were in the queue.
There’s a couple important things to note here before moving on to how this happened.
First, the queue itself was properly randomized. There isn’t any data to indicate that it was manipulated in any way, or that there were any issues regarding the blind, random placement of collectors in the queue.
Secondly, the packs were stuffed correctly, and the correct number of each type of Moment was present in this drop. There’s been a lot of chatter about “all the Holos being minted,” and rumors that some users pulled multiple Holo Icons from one pack — but that is not true. Thus, the amount of packs which contain MGLEs or Holo Icons for the Standard pack on Monday have not changed.
Lastly, the airdropped Elite packs for set completion were unaffected, and were randomized properly.
Despite all that, though, this is still an egregious error on Top Shot’s part, and a complete explanation as to just how it was allowed to happen is required. As mentioned above, internal testing prior to the drop played a role.
“The team had done several rounds of testing to prepare for today’s drop, but unfortunately, the focus of our team’s tests involved using Trade Tickets as the currency to purchase and did not involve Dapper Wallet,” Top Shot’s blog read after the randomization error. “The issues we experienced today were introduced as a result of the moving variables of today’s new packs. As a result, the bug in the pack purchase experience wasn’t discovered until today.”
Although I personally have no experience in these types of things, this statement left a few questions to be asked. Why test such a massive change using an entirely different system (Trade Tickets) than what will actually be used in reality (Dapper Wallet)? Why do a Stress Test that only tests one aspect of the major change as opposed to testing all of it?
Clearly, hindsight is 20/20, but after such a massive failure, it is necessary to ask about these sorts of things.
The KD Elite Pack Aftermath
As the drop was unfolding, it became clear that something was amiss as some community members began posting about receiving multiple Holo Icon-containing packs. Popular Top Shot streamer The Judge opened five KD Elite packs, all containing Holo Icon Moments.
Once everyone realized what was going on, the Top Shot community started losing their minds on Twitter and in the official Top Shot Discord. The Standard drop was swiftly delayed, and the wait began for an official explanation, which came later in the form of the aforementioned blog post.
Dapper Labs’ Response
In addition to the blog post explaining how the error occurred, Top Shot refunded each of the $49 packs that were purchased, regardless of whether they contained Holos, MGLEs, or Game Recognize Game Moments.
The blog also contained a few more crucial details as seen below, most importantly that they will not be able to be sold on the pack marketplace in the future.
One final thing to note about the blog post is that it is not visible on the blog home page, but only via the updates sub-section. It can be a bit hard to find this post without a direct link, which seems to be a telling decision on Dapper Labs’ part.
Top Shot posted one more blog on Monday which addressed the distribution of Holo Icon and MGLE Moments in the Elite packs.
“The key takeaways from the above table are that no cohort of specific individuals was statistically advantaged by the distribution,” the blog reads. “Additionally, only about 16% of the total supply of the new Holo Icon Moments ended up in the possession of people who got 3 or more in the drop.”
Top Shot Community Response
Prior to this controversy, there had been such a great run of high confidence in the product and market in recent weeks, mostly resulting from the hype around the Kevin Durant partnership and the fact that there hadn’t been any big technical errors in a while, so this really did happen at quite an unfortunate time.
As a whole, the Top Shot community was obviously very upset with what happened, with reactions varying from disappointment to fury and rage depending on who you asked. One common refrain, though, was that more important than any refund would be increased transparency going forward.
Top Shot provided additional transparency in a new blog post, with a deep dive into what collectors received Holo Icon and MGLE packs.
Some other points that have been raised specifically in regards to the refund include:
- The refund came in the form of non-withdrawable Dapper Balance, meaning that anyone who purchased with a debit/credit card would have to “wash” that money through the purchase and sale of a Moment, losing five percent in fees in the process.
- The refund doesn’t cover credit card fees for those who used cards instead of their Dapper Balance for purchasing.
One final sticking point for a lot of community members has to do with the upcoming Showcase Challenges related to these new Holo Icon and MGLE Moments. However, Top Shot cleared up any worried about challenges in Monday’s blog post.
“The Holo Icon Challenge originally planned to correspond with the Legendary Moments released with this drop will no longer happen,” the blog reads. The rewards will be minted and placed in packs in an upcoming drop.
What’s Next for Top Shot
So, where do we go from here? First, be sure to read responses to the randomization issue on the Top Shot blog here (first) and here (second).
For starters, I think it’s important to note that there’s chance that Dapper isn’t done trying to make this right. Airdrops or some other kind of further restitution very well may be on the way, especially if the community continues to make its voice heard.
Regardless of what else may or may not be coming, though, it’s crucial that some proper perspective is given to the situation, and we all look at what happened with a level head.
As much as it’s become an overstated punchline, Top Shot is still in beta, and mistakes are bound to happen. This mistake was a big one, no doubt about it, but it’s really important to keep it in perspective and not make it bigger than it really is. There’s no conspiracy here, just a really unfortunate and easily avoidable error that will hopefully lead to improvements and more transparency going forward.
As a community, we need to continue to hold Top Shot to a high standard we know they’re capable of reaching, but don’t forget to turn the page once the dust has settled. This ride is just beginning, and this rather large bump in the road isn’t going to slow anything down — as Top Shot community lead Jacob Eisenberg tweeted.
