Please purchase no more than one upgrade ticket.

Online Card Sorting & Analysis: How to organize menus intuitively using online research tools

Do you want input from your users about how the information and menus for your website or application should be organized to be intuitive to them?

Card sorting helps you understand the mental models for how users envision a structure for content.

Online card sorting tools, like WebSort, one of the tools demonstrated during this course, can help you gather information from dozens of users automatically online for analysis. With this input, you can more build a more effective information architecture.

Join us for a full day workshop with Carol Righi of Perficient as she presents the fundamentals of card sorting and analysis.

The session will not teach advanced statistics, though it will show how you can use particular tools to visualize trends in the data.

This tutorial has been successfully presented at UPA, CHI, and HFES conferences since 2005.

This tutorial, taught by one of the industry's most experienced trainers in user interface design, will be useful to marketing specialists, product planners, product managers, usability/UCD practitioners, interface designers, information architects, and technical communicators who would like to learn an end-to-end method for performing a card sort.

Attendees should have some background and familiarity with the concept of information architecture. They may or may not have performed a card sort in the past using other tools and methods. The tutorial is designed for both those new to the field of usability/UCD, as well as for advanced practitioners who are very familiar with the concept, but wish to learn a new tool and method.

Tutorial Structure

This tutorial will provide practical guidance and hands-on experience in the use of card sorting and cluster analysis methods and tools. This highly interactive tutorial will include lecture, discussion, and hands-on exercises in card sorting, cluster analysis, and the tools used to perform these methods. It will address issues surrounding card sorting and cluster analysis, and review actual projects in which these tools and methods were used.

In brief, this tutorial will address:

• What is a card sort and cluster analysis?
• How do I perform a card sort and cluster analysis (tools and methods)?
• What else do I need to know to be able to perform this method in practice?

Date:    Thursday, May 31^st, 2007

Time:   This is an all day event

Registration: 8:30am – 9:00am (refreshments served)

Morning Session: 9:00am to Noon (with a short break)

Lunch: Noon to 1:00pm (many places nearby)

Afternoon Session: 1:00pm to 6:00pm (with a short break)

Location: The New York Consumer Center, 28 West 44th Street

(Please Note: No walk ins for this event!) 

Sponsors: JPMorganChase for venue; Perficient for speaker. Thank you!

Cost:

$45 for NYC-UPA members
$55 for non-members
$30 for full-time students (please identify yourself as a student when pre-registering and provide valid ID at the event)

The event is limited to 48 participants.

There is a limit of 3 attendees per company due to class size and we can accept up to another 3 from the same company on the waiting list. 

Pre-payment is refundable until noon May 30, 2007 by sending a request to RSVP@nycupa.org. In addition, your ticket is transferable on or before May 29, 2007 by sending a request to RSVP@nycupa.org. You must notify us of this transfer since all names must be on building security list.

Course Outline

1. Introductions
2. Overview (session objectives)
3. What is a card sort?
4. Why are card sorts useful?
5. Mechanics of performing a card sort (Exercise)
6. Analysis of data (Exercise)
7. Group presentations (Audience participation)
8. Tools
9. Limitations and caveats

(Detailed tutorial description is further down on this page.)

Pre-workshop Assignment

Prior to the workshop, participants will participate in an online card sort, both to familiarize themselves with the concept and to provide a data set for the class to analyze during the workshop.

Registrants will be sent a link by Tuesday morning, May 22^nd. The link will include the card sort site, plus a download for files that will be used during the class. Please bring your laptop, if possible, though there may be power outlets for only one participant per table.

Food & Beverage

Workshop participants will have one hour between Noon and 1 PM to have lunch. Lunch is not provided as part of the workshop.  The midtown area has plenty of dining options for all price ranges.

There will be short breaks in the morning and afternoon. In the morning, breakfast breads and coffee/tea will be served. In the afternoon, fruit and cookies. and coffee/tea will be served.

Tutorial Detail

1. What is a card sort? This introductory segment will begin with a very brief demonstration of a manual card sort using index cards. A couple of attendees will be asked to sort items into categories, and name the categories. This exercise is simply intended to "get everyone on the same page" with regard to the essence of the card sort. We will then cover a brief history of the use of the card sort in information architecture. We will describe precursor methods to the method that will be described in this tutorial (such as Bridge, and physical card sorts using index cards or sticky notes). We will then complete this segment with a discussion of the use of card sorts in current practice.
2. Why are card sorts useful? In this segment, we will discuss the various applications of this methodology, including the following:
• Organization of menu structures for both web-based and desktop applications.
• Content organization for information-rich web sites and desktop applications.
• Organization of the content of documentation.

This section will include discussions of case studies of projects for which card sorts have been applied. The case studies will address the objectives of the web site, why a card sort was selected as a method to use to achieve those objectives, and at a high-level, how they were performed and applied.

1. Mechanics of performing a card sort. This segment covers the step-by-step method for performing a card sort. We will use the WebSort, a web-based tool, to give attendees hands-on experience with performing a card sort using WebSort and analyzing the data. We will provide the data that attendees will work with through this segment. The steps of the process will be clearly enumerated and explained in their proper sequence. These steps will include:

1.       Define the audience. We will discuss why it is critically important to define the audience for the web site or application, and address issues such as dealing with multiple audiences.

2.       Recruit. As with all aspects of UCD, it is necessary to find a pool of participants who are potential users of the resulting design. We will discuss issues such as how to recruit, how many to recruit, and reimbursing participants. We will show some examples of recruiting tools.

3.       Inventory the content. We will discuss the need to develop a complete inventory of a planned or existing web site or application, and show some examples of how best to develop the inventory. We will address issues such as how to avoid bias when providing descriptions for each of the items, how to determine the depth of content to include, and how to determine the appropriate terms to use for topics to avoid reflection of the current content for existing sites/application.

4.       Input the content into WebSort. We will demonstrate how the study administrator sets up a WebSort study. We will demonstrate how the "cards" are created, the mailing list is prepared, and other aspects from the point of view of the administrator.

5.       Use WebSort. We will engage the attendees in en exercise where they will be the participants in a card sorting study. We will have prepared a set of "cards" they are to sort. All attendees will perform this activity.

1. Analysis of data. This segment will address the second phase of this process: analysis of the data using cluster analysis. We will begin with a brief introduction to Cluster Analysis, which will include a conceptual discussion of the statistics behind the method. We will discuss how it has been used in the past, especially in marketing studies. We will contrast it to the "eyeball" method typically used with the data generated by manual card sorts.
   
   We will then return to our case study in progress. We will demonstrate the WebSort cluster analysis tool and explain how to use it. Attendees will then have an opportunity to use the WebSort tool to analyze the data. We will form teams of attendees. The teams will manipulate the tool to generate categories. Each team will be asked to create a basic architecture (categories and labels). We will have each team present their results, sharing the thought process that led them to their design. We will discuss how the same set of data can potentially generate multiple versions of an architecture. Other issues from the attendees' experience will be discussed.
2. Tools. This segment will cover some tools that are used to facilitate the process of the card sort and cluster analysis. We will discuss the use of:

• EZSort, WebSort, and others. An overview of the card sorting tools currently available.
• EZCalc and the WebSort cluster analysis tool. An overview of the cluster analysis tools currently available.
• Excel. How spreadsheets are used to organize and sort the information in a content inventory, track recruits, and display results.
• Visio or other graphics applications. Simple logic flow diagrams used to depict the site architecture.
• Wireframe. Interactive .html pages used to show the architecture "in action."
• Surveys. Recruiting, priority studies, etc., performed prior to a card sort can be performed using survey tools.
1. Limitations and caveats of card-sorting. The course will cover some of the current issues around card sorting, such as variations in the methodology, limitations to the current practice, and possible directions for the future. These include:

1. Open vs. closed card-sorts. Whether participants are allowed to introduce new items will change the manner in which results are interpreted.
2. Renaming content. Because terminology can affect usability, it is often valuable to have users name the content as well as the categories. We will discuss when this is and isn't recommended, and how to alter the methodology to accommodate this step (i.e., getting user feedback on the labels of the items after they've completed the sort).
3. Users' thought process. Asking users to articulate their rationales while or after they've completed the sort can provide more insight in to the user's mental model for the content, and can be helpful in developing the architecture.
4. Multi-level sorts. Currently, card sorts allow only one level of categorization. A variation in the methodology can help formulate lower-level categories into higher-level categories.
5. Card sorting and cluster analysis for large-scale information systems. In addition to "classic" web and application design, this method can and has been used for large systems (Dong, 2002)
6. Remote vs. face-to-face studies. We will discuss advantages and disadvantages to each.
7. Individual vs. group studies. We will discuss the implications to multiple individuals performing the sort to obtaining real-time group consensus by having groups perform the sort.
8. How many participants are needed? We will discuss a recent study by Tullis and Wood (2004) that provides guidance on this topic.

Tutorial Materials

The tutorial notes will include the workshop presentation slides, detailed descriptions of key content from the lecture portions of the tutorial, worksheets for the exercises, and a bibliography of recommended readings and tools to research.

Workshop Leader

Carol Righi, Director of User Experience, Perficient, Inc.

Carol Righi is Director of User Experience for Perficient. Carol has worked in the area of user experience and User-Centered Design for more than 20 years. Carol is widely considered a UCD thought leader, is published extensively, and has presented at numerous professional meetings worldwide. She is co-author of User-Centered Design: An Integrated Approach and is a featured columnist for TaskZ.com.

She recently published a second book, “User-Centered Design Stories: Real-World UCD Case Studies (Morgan Kaufmann Series in Interactive Technologies)” with Perficient colleague Janice James.

Carol has been both a manager and a practitioner throughout various point of her career. Her skills include interaction design, information architecture, usability engineering, and training. Carol has designed user interfaces for the Web, and for desktop, kiosk, and mainframe applications. She has planned and conducted extensive user research studies. She has served as manager for numerous design and evaluation efforts. She has helped lead the development of the UCD competency in various organizations, including IBM. Carol has also designed and developed many standup and e-learning courses in UCD. She has trained thousands of students.

Carol is a member of the Usability Professional's Association (UPA); SIGCHI (the human-factors special interest group of the Association for Computing Machinery). She serves on the editorial board of the peer-reviewed UPA online journal "Journal of Usability Studies," and is frequently sought out to review journal articles, conference submissions, and other scholarly and professional works. Carol was Usability Co-Chair for CHI 2007.

Carol has worked on projects across the industry for companies such as IBM, Google, Yahoo, Intuit, A. G. Edwards, Mapquest, Met Life, and many others. Past positions have included consultant for IBM's Global Services; Manager of Usability for IBM's Personal Software Products division; Manager of Academic User Services at Teachers College, Columbia University; Guest Lecturer at the Weizmann Institute of Science, Rehovot, Israel; and various other positions in consulting and educational services.

Carol received her undergraduate degree in psychology in 1981 and her Ph.D. in school/educational psychology in 1988, both from Fordham University in New York City with a concentration in computer applications to education.