Pitch The Switch: And the Winners Are…

We’ve finally concluded the end of our month-long contest to discover who created the best video pitch in their own words for Infusionsoft. However, it was a tough one. First, I’ll announce the winners. Then, I’ll explain what we’ve learned from this contest — anyone considering to run an online contest needs to read this.

After tallying all 6,397 votes, here are the results of the Pitch the Switch contest:

FIRST PLACE, GRAND PRIZE
Scott Griggs, Trainz.com
$10,000 Cash
1 year of Infusionsoft

SECOND PLACE
Nathalie Lussier, Raw Foods Witch
Apple iPad

THIRD PLACE
Dustin Mathews
Apple iPod Touch

FOURTH PLACE
Marketing Ninjas
Flip HD Video Camera

Congratulations to the winners and thanks to everyone who participated. We saw some great activity and of course, it has been exciting to see how Infusionsoft has helped our customers grow their businesses and improve their lives.

Now that the winners have been announced, let me explain a few things that happened along the way and share a some lessons we learned.

Shortly after the second phase of the contest, where contestants were encouraged to drive votes on their videos, we noticed a couple of weird things. Some contestants were receiving far more votes than others. After some investigation, we discovered that those contestants had created bots (software programs) that would automatically submit votes. As soon as this activity was discovered, we issued a communication to all contestants stating that we would not tolerate cheating and adamantly stated that fraudulent votes would be disqualified and we may disqualify cheaters.

Amazingly, we received a formal apology from one of the contestants. He was unaware that one of his programmers had employed this tactic. He was sincere in apologizing and withdrew from the contest. Stand-up guy. I love our customers.

Our communication also mentioned methods that were okay for promoting their video, such as Twitter, blogging, emailing family and friends, etc. All of those would be what we label as “organic” traffic. However, we didn’t account for the possibility of paid traffic.

So, right before Thanksgiving, an email was sent from one customer (let’s call her Jill) informing us about another customer (let’s call him Jack). Jill showed us evidence of Jack paying for votes. Since it was the day before Thanksgiving, I was out of the office and not checking email. We didn’t see the email until early this week — which was a problem since the contest ended on Tuesday (yesterday).

We had some healthy debate (minor yelling and screaming) on whether or not allowing paid votes was acceptable or not. Initially, we leaned toward allowing those votes since we never explicitly stated that paid traffic was against the rules. In an effort to be as fair as possible, we decided that paying for votes was outside of the intent of the contest and we disqualified Jack’s paid votes.

You can imagine the quandary we were in. If we allowed the paid votes, then we would have a flurry of activity (which was already kicking up) on the social networks of people upset because we allowed votes that they considered fraudulent even though technically it wasn’t in the rules. We’d also have contestants that would feel disenfranchised because one customer won using a tactic that they felt was inappropriate (or that they would have used if they knew it was OK). On the other hand, if we chose to disqualify Jack because of his paid traffic, he could easily feel cheated (by us) for not holding to the letter of the law — our own rules didn’t explicitly ban paying for votes. Both arguments can be strongly justified and trust me, we had quite a debate here about it.

We were in a lose-lose situation.

Ultimately, we decided to disqualify Jack’s paid votes, but not disqualify him as a contestant since we had never had said specifically that paying for votes was against the rules.

As it turns out, Jack’s votes totaled only 8% of the total votes and didn’t skew the results at all. By the way, I called Jack this morning and had him show me through GoToMeeting the votes he paid for. He logged into the system he used to pay for the votes and showed me live. This verified the data we had on our side. With $10,000 on the line, we felt it was necessary to take every measure possible to ensure that we were counting votes appropriately and being as fair and equitable as possible.

In light of the above, here are the top five lessons we’ve learned from this contest. If you want to do a contest, please heed these pieces of advice.

1. Don’t allow anonymous votes. We probably should have required a valid email address to vote. This would have reduced the chance for fraud. It wouldn’t have eliminated the possibility of people creating bots or paying for votes, but we would have had one more data point to track.
2. A big cash prize makes the contest wonky and doesn’t produce results. People get very emotionally attached when they think they have a chance at winning a big cash sum. So, they do things they wouldn’t normally do and they get really let down if they see the prize getting further out of reach. The intent of our $10,000 prize was to encourage a ton of people to enter the contest. The reality was that we didn’t get any more entries to the contest than we’ve received for other contests where we didn’t offer a cash prize. So, the cash prize didn’t produce results and it caused the contest to get a little out of hand. The difference between $1000 and $10,000 didn’t necessarily render a better outcome in this contest.
3. Explicitly state every possibility in the rules. It’s impossible to think up all the possible ways that contestants might try and game the system. But, it’s worth the time and effort to really think through all the possibilities you can possibly imagine and explicitly state them in the rules. It will save you a ton of time, potentially money, and probably your reputation if you take the time to do this. Consider asking some people to look over the rules and listen to them if they can find loopholes in your policies in it. (We learned this by failing to mention paid traffic. We assumed everyone would drive organic traffic.)
   
4. Provide a sample entry. We realized after the fact that we probably would have had more entries if we had provided a sample entry for people to see what was required of them. We’ve seen contests be effective when a low quality sample entry is displayed as an example. The key is to provide a low quality entry. When the example is low quality, potential contestants automatically think, “I can do better than that”, and that mindset gets them motivated to enter. This resulted in a somewhat-uncomfortable delay until the first few videos were published.
5. Contests based on voting decrease participation. When winning a contest is based on voting, many people automatically discount themselves because they don’t have a big reach or the ability to find many people to vote for them. A contest where the winners are chosen at random or by the contest holder is more likely to encourage participation because potential contestants don’t immediately discount them self. This also limits the scientific approach for winning and enables people to become creative with their submissions.

Running the Pitch the Switch contest was a blast for us. I’m sorry that things got wonky in the end, but I feel like we made a good decision, and I’m even more glad that in the end the results were not skewed by questionable activity.

Congratulations to all the winners and everyone who participated. I hope our experience will help any others who are planning to run an online contest.

Oh, and by the way, watch their videos – they were all great!