statistics

I was on the "Ecosystem" track session selection team for Drupalcon London, which motivated me to finally do some more analysis on the traditional pre-selection session voting. Specifically, I wanted to compare the votes a session receives against the evaluations submitted after the conference.

By the way, if you have the opportunity, I highly suggest going to a Drupalcon; they are always great events.

Here are some conclusions based on analysis of the evaluation and voting data from DrupalCon Chicago:

• Voting was not a useful predictor of high quality sessions!
• The pre-selected sessions did not fare better in terms of evaluation than the other sessions (though they may have served a secondary goal of getting attendees to sign up earlier).
• We should re-evaluate how we do panels. They tend to get lower scores in the evaluation.
• The number of evaluations submitted increased 10% compared to San Francisco, which seems great (Larry Garfield theorizes it is related to the mobile app, I think there are a lot of factors involved)

Is voting a good way to judge conference session submissions?

Drupalcon has historically used a voting and committee system for session selection that is pretty common. This is also the default workflow for sites based on the Conference Organizing Distribution.

Typical system:

1. Users register on the site
2. They propose sessions (and usually there is a session submission cutoff date before voting)
3. Voting begins: people (sometimes registered users, sometimes limited to attendees) can vote on their favorite sessions
4. During steps 2 and 3, a session selection committee is encouraging submissions and contacting the session proposers to improve their session descriptions

Drupal is growing in complexity and growing simply in sheer numbers. We need more tools to help people manage the information overload and find the best voices in our community quickly. We should build dynamic tools to empower community members to join in and share their voices (if those voices are valuable) rather than walled gardens that keep people out. I believe voting, richer profiles, and the hot content are steps to help enable that vision. That said, the implementation has to match the community values. Below I've laid out the story of how some improvements to Groups.Drupal.org were made, provide data behind some of those improvements, and ask some questions so we can keep refining them.

At Drupalcon San Francisco there was a sprint for groups.drupal.org features where Josh Koenig and Brian Gilbert helped out add some new features. In particular we added voting on nodes & comments and we added a "hot" page which incorporates several elements to determine which content on the site is interesting in the last week.

I wanted to look back on the past year to think about these changes and whether or not they are an improvement.

Hot Content: G.d.o is a differentiated piece of $#!@$&