Posted by: bbannan | March 19, 2009

Wikis and Delphi Methods – Bill

The New Media Consortium releases the Horizon Report annually:
It’s not a survey per se, but a consensus report of experts, created on a wiki, using iterative cycles based on Delphi methodology. (Wikipedia has a nice definition of the Delphi Method. The Horizon Report methods section starts on page thirty.)

A survey would probably not be the best method for this type of research, as it deals with predictions of future behavior, rather than the record of past or current behaviors. The project’s Delphi methods include cycles of voting. I think that would invite group think and other forms of bias. But then, haven’t we all taken surveys where we felt those conducting the survey were attempting to obscure the true purposes of the study? To me, the complete transparency of the entire process seems like a refreshing alternative. I wonder how productive it might be to consider including carefully selected users in part of the survey creation process in a
similarly transparent way, rather than just inviting them to take an already prepared survey. You might be missing questions that are more relevant for users. In other words, do we always have to ask “Does this do what we want?” Shouldn’t we more often ask “What DO you want?”

By the way, the unfiltered wiki has many more interesting links for trend watchers:

My favorite among these was this suggestion that the internet will become “pragmatic,” rather than a “semantic” web:



  1. Although I have never participated on a research using Dephi method, I find that this approach is really useful specially when the experts are selected following a strict quality criteria. Sometimes experts are selected based on their work in research and not in their real-life experience. In this case, I find that some questions do not address the real-world needs but the research needs. Some how, we need to reduce this gap between research and industry to meet the users needs. You have very nice described this in the question “What do YOU want?”

  2. Bill:

    I put the wiki’s defintion of Delphi Method at the bottom of this comment. What I like about the DM in this instance is the people being questioned can stay focused and on topic. Sometimes, it’s best to look past (current) real-world parameters, because they’re entrenched in a broken situation. Stepping back, and examining in theory, can show how D will work, if you’re currently at A. Staying in real-world constraints, you may have to go A > B > C >D. Theoretically, you can study the value of D, because maybe you need to be at E.

    or 3.

    The Delphi method is a systematic, interactive forecasting method which relies on a panel of independent experts. The carefully selected experts answer questionnaires in two or more rounds. After each round, a facilitator provides an anonymous summary of the experts’ forecasts from the previous round as well as the reasons they provided for their judgments. Thus, experts are encouraged to revise their earlier answers in light of the replies of other members of their panel.

    tom sakell

  3. Thanks, Maricel and Tom.
    In this case (the New Horizons Report), they’re asking experts “What do YOU predict?” I have to admit, this topic isn’t about current user needs, so maybe my criticism was unfair. It was about instructional technology trends, so I thought some of you might be interested.

    As you mention, Tom, they’re looking past current parameters to predict the near future. (By the way, thanks for pasting the definition here. I should have recognized we’re all busy, and given the info, not just links.)

    Frankly, I prefer an even longer view predictions, like the “pragmatic web” that was in their early Delphi round, but not later. The idea is that by using more intelligent data mining, your computer will know what you want, perhaps even better than you do.

    For example, when you type “turkey” in Google, it might know you want the bird, not the country. And it might then know your food and spice preferences and suggest only the types of recipes you’re likely to enjoy, and those that don’t aggravate your known health conditions. Imagine that applied to learning situations. The web could deliver only those lessons in your own Zone of Proximal Development, that met your profile, and the search you typed. Individualized instruction, without the teacher. Some Brave New World!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: