Fall 2014 Lottery Results Survey Analysis – Part 2

August 15th, 2014 by Guy Forget 7 Comments

For every Phish fan I’ve ever known, I’ve probably heard 10 different theories about how the band’s mail order tickets are distributed to fans. One guy will be certain that he’s screwed tour after tour because Phish is trying to weed out old fans. Someone else will swear that the band will fill your entire order if you exclusively order Wednesday night shows, and 3 tickets for each. In the money-order days, it was the artwork with which you decorated your order envelope. And so on.

In an effort to prove or debunk some of these theories, I opened an online survey when this fall’s lottery results came back, and received over 2000 responses. In part 1 of my write-up of the results, I discussed what they told us about demand for individual shows on the upcoming tour.

In this part, I’ll take a broader look at what we can learn about which elements of Phish’s allocation of tickets are random — and which aren’t.

I will focus on only a few key areas, but will also open up the data, so that others can do more in-depth analysis if they choose.

A couple disclaimers, before we dive in. First, I am just a Phish fan with pretty good quantitative skills — not a trained statistician. Second, the data I collected was limited. Although I asked each respondent for their results on each show, as well as their region and whether they ordered a T-shirt, I did not ask how many tickets were ordered for each show, on what day they placed their orders, or any other potentially interesting information.

That said, let’s dig in.

Question 1: Does [Order] Size Matter?

In other words, does ordering the whole tour give you better odds than ordering a single show? To determine this, we can simply pick a single show — I’ll choose Halloween, since it was the most ordered, and thus gives us the most statistically significant results — and see how people of varying order sizes fared.

As you can see, there’s almost no difference between the success rate of those who ordered 11 shows vs. those who ordered only Halloween: they’re all right around 20%. (Note: I omitted orders for all 12 shows, because there were an improbable number of 0/12 and 12/12 orders, leading me to believe that some respondents clicked either “No” or “Yes” on every show in an effort to see the survey results.)

TAKE-AWAY: Order size has no influence on success for a given show.

Question 2: Are orders more likely to be filled for an entire run than a single show of the run?

In other words: suppose you ordered all 3 nights of Vegas. Were you more likely to get tickets to all 3 than to 1 or 2? To determine this, we can look at the 11/1 and 11/2 fulfillment rates for only those who got Halloween tickets, and compare it to the rate for everyone who ordered 11/1 and 11/2.

Mmm, pie charts. Sorry, what was I saying? Oh, right: math. The charts are pretty crystal clear: you were almost assured of getting 11/1 and 11/2 tickets if you got Halloween, while you had only a slim chance at them if, like most of us, you were rejected for Halloween.

What that tells us is that MusicToday most likely puts all the Vegas orders in a hat, randomly chooses entire orders (for Vegas only) to fill, and then distributes whatever single-night tickets remain to another randomly chosen pool.

TAKE-AWAY: If you get tickets to one night of a run, you’re likely to get tickets to all the nights of that run.

Question 3: Does that mean your best chance of getting Halloween tickets was to enter for the whole run, then?

No. That’s a separate question. Because remember, they’re putting all the Vegas orders into a hat, and the ones they pick get filled. Each order has the same likelihood of getting chosen, regardless of whether it’s for 1 requested show or 3.

To verify this, we can look at the 10/31 results for those who only requested that show versus those who requested the whole Vegas run.

Those who only requested 10/31 fared slightly better than the entire population; however, only 48 respondents were in this category. We would need a larger sample size to determine whether there is a true advantage.

TAKE-AWAY: Ordering only one show of a run definitely doesn’t dramatically increase your likelihood of getting tickets to that show. Whether it gives you any advantage at all can’t be concluded from these data.

Question 4: Does location influence your results?

I’ve heard a few theories here: some speculate that proximity to the show increases your odds. Others suggest the opposite: that willingness to travel is rewarded. Still others think that the East Coast always wins with Phish. Let’s see what the numbers say.

The variance is pretty small. Yes, the East Coast did better, but not by much. But also, Halloween in Vegas is about as location-neutral as it gets: probably most people will travel. So let’s look at one more show, and we’ll make it more of a local gig: Monday night in San Francisco.

To the extent that there’s a bias toward the East Coast, it ain’t showing up here. Nor is any preference for locals. Again, everyone hovered right around 22% odds.

TAKE-AWAY: Location has no impact on results.

Question 5: Does ordering a T-shirt help you get tickets?

You might laugh at this one, but I actually read a thread on the phish.net forums speculating that EVERY order was completely filled if it came with that T-shirt order. Any truth to that?

TAKE-AWAY: Nope.

Conclusion

Every conspiracy theory revolves around the idea of bias toward one group or another, whether it’s New Yorkers, tour rats, or what have you. Every conspiracy theory is wrong: the selection is random. Even the one result that, at first glance, looks non-random — Question 2, on multi-night runs — is in fact an instance in which orders are being selected randomly across an entire run, not an individual show.

There is, of course, more to be learned, both from the data collected in this survey and hopefully, more robust surveys in the future. I encourage everyone to look at my data and keep me and others informed if you do your own analysis.

« Fall 2014 Lottery Results Survey Analysis - Part 1         Dick's Adds to its Special Reputation »

Tags: , , ,

Posted in Featured

setlist provided by Phish.net