CHI 97 Electronic Publications: Technical Notes

CHI 97 Electronic Publications: Technical Notes

Usability, Help Desk Calls, and Residential Internet Usage

Sara Kiesler, Robert Kraut, Vicki Lundmark, William Scherlis, Mukhopadhyay Tridas

Carnegie Mellon University
Pittsburgh, PA 15213, USA

kiesler@andrew.cmu.edu
robert.kraut@cmu.edu
lundmark@andrew.cmu.edu
scherlis@cs.cmu.edu
tridas@andrew.cmu.edu

ABSTRACT

For the average person, the Internet is still too hard to use. We report evidence from HomeNet, a field trial in Pittsburgh that tries to understand how people use the Internet. Despite our reducing technological and economic barriers to use, families had problems connecting and using the Internet. We show that help calls, however, are not a good indicator of usability, since it is the "enthusiasts" and people with instrumental tasks to accomplish who call.

INTRODUCTION

Like the telephone and television before it, household computing has the potential to change how average people live. In the field of HCI, it is now generally accepted that designers, developers, educators, and managers should understand how people use technology in the context of their daily lives. However, as yet there are few systematic reports in the literature about on-going residential use of home computing. HomeNet is a field trial at Carnegie Mellon University that tries to understand people's use of the Internet. We are carefully documenting how members of households use on-line services such as electronic mail, computerized bulletin boards, on-line chat groups, and the World Wide Web. This report describes usability problems we have observed in the first sample of 48 families with teenagers (133 individuals).

In the HomeNet field trial, we attempted to overcome economic and technological barriers to use. We provided each family with a Macintosh computer with 14.4 kbps modem connected to a dedicated telephone line. Each family member above age 8 received a full Internet account. All computers included a turnkey system for access to the entire Internet and included software for using electronic mail, newsgroups, the World Wide Web, MUDs, and special HomeNet chat newsgroups. Our software configuration allowed family members to use Internet services without learning the details of any operating system. They also received approximately three hours of training.

Support was provided through a help newsgroup, and a telephone help desk staffed by college students.

ABSTRACT
INTRODUCTION

Sample characteristics

WHAT WENT WRONG
RELATIONSHIP BETWEEN HELP CALLS AND USAGE
CONCLUSION
ACKNOWLEDGMENTS
REFERENCES

Sample characteristics

Our sample was recruited through the high schools of four demographically diverse neighborhoods in Pittsburgh, Pennsylvania. In each school, we approached students who worked on the school newspaper and their families, as well as at least one journalism teacher and that teacher's family. The common bond of journalism gave students from different schools something to discuss as we put the project on-line. The HomeNet sample, although not a representative sample of the US, is demographically more diverse than current Internet users. Fifty-seven percent are female; 42% are teens; 24% are members of minority groups; and 25% have household incomes less than the median US household income.

Our earlier research [1] documented that in 1995, before participants in our sample had actually used the Internet, they reported positive attitudes towards the Internet but only vague ideas of what it would be good for. A large minority did not know what downloading or email was. Given their vague beliefs and knowledge, it is probably not surprising that many had difficulty getting started. We expected that people with more computer skill and motivation would be likelier to overcome these difficulties, and would, in turn, be more likely to use the Internet frequently. However, we thought that as everyone learned how to use the computer and what the Internet could do for them, computer skill would predict Internet usage less well. We were wrong. Even after a year of experience with the Internet, computer skill still predicted Internet usage significantly. This result held true across different gender and age groups.

WHAT WENT WRONG

There is strong qualitative and quantitative evidence that some members of most of the families had problems using the Internet.

Over 70% of the households called the help desk. The kinds of problems logged by help desk staff included problems in installing phone service, configuring the telecommunication software, busy signals (users often blamed themselves!), buggy software, inexperience with mice, keyboards, scroll bars, terminology, radio buttons, and menus. Yet, in our home interviews, we noted there were many more problems participants had not called about. These included confusion using the Macintosh icons and features, trouble finding live sites on the Web with useful or fun information, and difficulties using email. Few participants were engineers--knowledgeable about how computers and telecommunications work. So, few understood much about the role of the modem or the way that software works (see examples in Table 1).

Examples: Error Samples
Symptom	Cause
Email freezes	Never installed the modem. Didn't know it was part of the computer
Computer keeps dialing the Giant Eagle Market	Typographical error in login script
I can't log in	Caps Lock for password not noticed because password is hidden
Netscape disappeared	User reformatted disk after advice from Apple's tech support line
No applications launch when clicked	User closed windows instead of quitting program; program doesn't open a window if already running
Modem won't dial	Someone else was using the phone
Can't find rabbit newsgroup	Didn't know how to use search function

Table 1. Examples of participant problems.

RELATIONSHIP BETWEEN HELP CALLS AND USAGE

A few months into the HomeNet field trial, we realized that calls to the help desk (along with email messages asking for help) did not reflect the full range of usability problems. Our model had been: Usability problems -> Calls for help + Lower usage. However, a different phenomenon seemed to be operating in which: Usability problems -> Calls for help + Higher usage. What seemed to be happening is that a group of the more skilled and enthusiastic participants was using the Internet, and they called the help desk to help them explore.

Another group was not necessarily as skilled, but had a purpose for using the Internet and motivation to seek help in increasing their skill. The correlation between calling the help desk and total Internet usage over a year was r = +.35, p < .001.

What measures of difficulty might predict not using the Internet? The HomeNet study has been mailing questionnaires to participants every two months that ask them to describe their last experience with a computer. Since they do this privately, we don't depend on their motivation to explore and learn about the computer to describe their problems. After a year, we had three measures of the difficulty each participant reported with using the Internet. The average of these measures is correlated r = -.11, p > .10 with usage, indicating a trend in the opposite direction to that of the help desk calls. That is, participants who reported difficulty tended not to use the Internet much.

CONCLUSION

Probably, every HCI professional has trouble using a computer now and then (encountering a "frozen" screen or sending email incorrectly, for example). Now that the Internet is migrating from workplaces that emphasize technology to the home, we need to reexamine such problems. The findings here suggest that simply looking at help desk logs may be misleading. Calls for assistance associated with continued usage rather than a tendency to give up. Although average persons may share similar Internet-related problems with professionals, they may not have acquired the same level of patience for solving them.

ACKNOWLEDGMENTS

With their skills, efforts, and enthusiasm, Jane Manning, project manager during the formative stages of the study, and Stewart Buskirk, who succeeded her, ensured that the project could move forward successfully HomeNet is funded by grants from Apple Computer, AT&T, Bell Atlantic, Bellcore, Hewlett Packard, Intel, Carnegie Mellon University's Information Networking Institute, Interval, The Markle Foundation, the National Science Foundation (Grant #IRI-9408271), the NPD Group, the US Postal Service, and US West. Farallon Computing and Netscape Communications contributed software.

REFERENCES

1. Kraut, Robert, Scherlis, William, Manning, Jane, Mudkophadhay, Tridas, and Kiesler, Sara. (1996). HomeNet: A Field Trial of Residential Internet Use. Proceedings of CHI '96.

CHI 97 Electronic Publications: Technical Notes