Posted by: bbannan | March 22, 2010

Diaries – Shannon

You may be asking yourself how diaries have anything to do with design research. I know that when I think of them, one of the first thoughts that pops into my head is that of an angst-ridden teenager frantically scribbling in her diary about the trials of high school and how much she hates gym class. However, diaries can be so much more than day-to-day events captured in a notebook. They can be useful tools for helping researchers design all sorts of products, from better mobile phone interfaces to more comfortable office chairs at work. Best of all, they are relatively cheap to use and can reach an international target audience without the researcher ever having to leave the office.

Approaches to Diaries

According to Robert Gillham, of Amberlight Partners in London, there are two main approaches researchers take when using diaries for design research. “The diary study is a method that has its roots in both psychological and anthropological research” (p. 4). Each of these approaches tries to determine different information. Researchers use psychological diaries when it is important for participants to “record the frequency of events in their daily lives which are of interest to the researcher” (p. 4). Psychological diaries have been used to determine the frequency of mobile device use, “task switching and interruptions, and even the impact of different technology media on the frequency of lying” (p. 4).

Anthropological diaries on the other hand, allow participants to record any information that is relevant to their day-to-day lives, or that they feel is important to them. “This has also been called a ‘cultural probe’, and the subjects media for recording information may not be constrained to paper, but may include cameras or other devices in order to record anything which they may encounter which has salience” (p. 4). Anthropological diaries are good to use when researchers know little about the design project requirements.

Types of Diaries

Beyond approaches, there are many different types of diaries that you can use in your design research. The two main categories are unstructured and structured diaries.

Unstructured diaries aren’t as unstructured as they sound. Most of them have some guidelines attached to help researchers hone in on the information they are really after. According to Kuniavsky, unstructured diary entries should be “specific, brief, and complete. They should give the diarists a set of guiding principles about what kind of behavior should be recorded while encouraging active participation” (p. 371). Also be sure to provide a standardized form for the diary entries (p. 375). This makes the task less ambiguous for your participants.

Structured diaries are useful in helping researchers capture specific information about their study. Be careful, however, to not make your structured diary requirements so stringent that they leave no room for your participants to record other thoughts and observations about your product (Gillham, p. 6). According to Kuniavsky, there are three main types of structured diaries, survey-like diaries, usability test diaries, and problem report diaries (p. 375).

Survey-like diaries appear as an extension of a design research survey. They ask the participant to answer a series of questions about the product in question. These questions can be open-ended, to provide participants a chance to respond in more detail, or they can be close-ended (Kuniavsky, p. 377).

Usability test diaries ask “diarists to perform specific tasks that exercise certain features or expose them to certain content” (Kuniavsky, p. 377). This type of diary is useful when researchers want to study how a product and its features change over time.

Problem report diaries are only filled out when participants encounter a problem. This type of diary “minimizes the amount of bias that the study has on the users’ behavior, and it highlights the most painful parts of the interface” (Kuniavsky, p. 379).

Advantages of Diaries

  • • Diaries are more cost-efficient than traditional ethnographic design research, such as usability testing.
  • • Diaries allow researchers to study geographically dispersed individuals.
  • Studying a diverse target audience can show researchers how people of different ethnic and cultural backgrounds use their product.
  • • Diary studies are a good design research alternative to usability testing for small companies who may not have a lot of resources to bring to bear (Hagen, p. 5).
  • • Diary studies can be done the good old-fashioned way, using paper and pen, or researchers can take advantage of new technology, such as phones or cameras with voice and video capabilities, to record entries.

Disadvantages of Diaries

  • • For best results, diaries require multiple entries over a length of time determined by the researcher. If you don’t have the necessary time needed, diaries may not be the route you should take.
  • • Diaries can be time consuming for participants to complete, and it’s easy for participants to drop out of the study. To help combat this, provide incentives and send reminders and thank you notices to your participants.
  • • Managing responses can be time consuming and trends and patterns may not be easily identified. However, the key to a “successful diary study is the management of responses and respondents” (Kuniavsky, p. 381). For more information on how to manage and analyze responses, see chapter 12 of Kuniavsky’s book Observing the User Experience.

Tips for Using Diaries

  • • According to Kuniavsky, diaries are best used in conjunction with products that are fully functional (p. 369).
  • • Kuniavsky also suggests to offer some sort of incentive to get your audience to participate. Money and gift certificates seem to work well. However, Gillham suggests that you only make a monetary payment to participants after they have completed the diary study and returned any loaned out equipment (p. 6).
  • • Provide any necessary equipment to the target audience, such as voice recorders, cameras, or movie recording devices.
  • • Send reminders and thank you notices to participants in order to keep them engaged in the study.
  • • Provide contact information so that if participants have any questions they can easily contact you.
  • • Emphasize that the participants themselves are not being tested, the product is.
  • • Keep diary entry forms relatively short. It should only take 20-30 minutes max for your participants to fill them out (Kuniavsky, p. 375).
  • • Follow up diary research with personal interviews to discuss findings and further analyze patterns and trends.
  • • Organize diary responses! Code and tag them in whatever manner helps you make sense of the data. You can also create timelines with the information. This will help you recognize trends and patterns more readily. (Kuniavsky, p. 383).

Useful Links

Diary Studies as a Tool for Efficient Cross-Cultural Design by Robert Gillham

www.amber-light.co.uk/resources/whitepapers/international_diary_studies_amberlight.pdf

Engaging with Stakeholders: Mobile Diaries for Social Design by Penny Hagen

http://digitaleskimo.net/wp-content/uploads/2007/11/hagen_engaging-withstakeholdersmobilediaries.pdf

Using Diaries in Social Research by Louise Corti

http://sru.soc.surrey.ac.uk/sru2.html

Advertisements

Responses

  1. In my previous military career, we used diaries as a tool for evaluating and refining the use of night vision goggles (NVGs) for Coast Guard helicopter flight operations.

    During the two-week syllabus of NVG instructional flights, we directed our pilots and crews each night to record in structured diaries (we called them NVG flight diaries) their impressions, frustrations, difficulties and suggestions for using NVGs. As the pilots and crews grew more comfortable and proficient with NVGs, they provided insights for improving our instruction and developing operational parameters for NVG use. We reviewed the diaries at the end of each week.

    Although we instructor pilots were untrained in instructional design and development, we thought the NVG flight diaries were a good idea to improve our NVG program. Over a six-month span of NVG instruction, the diaries provided a lot of positive data on improving our NVG syllabus and NVG operational use.

  2. I enjoyed reading your post and found it very interesting. I’ve never been fortunate enough to participate or design a diary and never kept one when I was younger. So, although, I understand the concept, I’m wondering how it work in a real world environment and specifically when it comes to our project.

    “Diary studies are a good design research alternative to usability testing for small companies who may not have a lot of resources to bring to bear”

    I really like that you mentioned this in your post as it relates almost directly to our project. Since we are getting ready for Round 2 and while considering ideas as to what to ask in Round 2, one thing we have thought about is using diaries (per your suggestion). And the sentence above indicates that it would be a good way to approach our Round 2 phase since we will be evaluating usability. We will, of course, be using newer technologies rather than pen and paper, but I’m wondering if we could utilize a blog space so that users can contribute their daily thoughts online and make it easily accessible by all. As you mentioned in the disadvantage portion of your post, I’m wondering if there is adequate amount of time for this to be completed.

  3. Jeff, you provided a great example of how diaries can be applied. It seems like the military would be a good environment to use this appraoch since you can require the soldiers to comply. My concern with this approach is that it places a large burden upon the participant. Shannon mentioned in her post that using incentives is a good idea if you’re going to use this approach, and I can see why. It’s hard enough getting people to find time for an interview or to participate in a focus group. It seems like it would be very difficult to get a volunteer to commit to keeping a diary on a regular basis over a long period of time. To me, that might keep me from wanting to use this method.

  4. I love the idea of using Diary Studies as a method of user research. I think you’ve provided an excellent summary of the pros and cons as well as tips for a successful diary study.

    I think Bethany makes a valid point that the idea of completing a diary over a stated amount of time might seem daunting to participants and thereby deter them from wanting to participate in your study. I think one of the keys to easing these concerns is in the preparation for the study. If the researcher makes it very clear to the participants exactly what will be expected of them, it may help eliviate some of the worry about how much time they’ll have to invest in the study. I think another key to ease the burden of the participant is the diary design. As Shannon pointed out, even unstructured diaries should provide some guidlelines for the participants, so a) the researcher can “hone in on the information they are really after” and b) the participant is not faced with the fear of facing a blank sheet of paper with no real direction or idea of how to begin.

    At my workplace, we are piloting a 6-week virtual training course (which, a 6 week course done virtually, is relatively unprecedented, very little research is available and if companies are doing it they’re not talking about it). I like the idea for using a diary study to get students’ direct feedback regarding certain aspects of the training. However, something like that couldn’t really work, unless the students were given time out of their day to complete the diary entry or the company because by law we can’t require any more than 8 hours of work time. Still, though, in a perfect world, I think a diary study about the user experience throughout our virtual training course would be very interesting and that we could gain some valuable data.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: