Navigating a Data Grid, With and Without a Visual Display

Jennifer Ockerman

Center for Human-Machine System Research
Georgia Institute of Technology
jojo@isye.isye.gatech.edu

CHI '97 Workshop: Speech User Interface Design Challenges

Short Biography

Jennifer J. Ockerman is currently working on a Ph.D. in human-machine systems research in the School of Industrial and Systems Engineering at the Georgia Institute of Technology. She received her B.S. in Industrial Engineering and Operations Research from Virginia Polytechnic Institute and State University in 1989. After four years as a manufacturing engineer at Westinghouse, she returned for her M.S. in human-machine systems research at Georgia Tech. She is currently designing and developing voice-activated performance support systems for wearable computers for Georgia Tech Research Institute. Her areas of research interest are computer-based decision and performance support systems which combine training with support in a speech-based environment.

Position Paper

In the development of a speech-based interface for quality inspectors in a poultry plant we have come across several problems in the design of the speech-based interface. The problem which seems most appropriate for this workshop is: What is the best method to support the navigation of a data grid, like a spreadsheet or a form, with and without a visual display? This involves both the input and output aspect of the speech-based interface; in other words, both the commands that the user uses and the feedback that the user receives from the system.

In our case, the quality inspectors have to notify a supervisor when one of their samples has a problem, then go back to the record of that sample and mark that they have notified the supervisor. Since the quality inspectors do not leave their work area, they do not notify a supervisor until the supervisor comes around to check on them. This usually means that they have to go back in their records and make a note. Currently they do this on paper by searching their hand written data for the piece of data that is out of limits and making a hand written note beside the out-of-limit data. However, the company is now going to an electronic data system and they will be entering the data by voice. The only way to distinguish the records from each other in the new electronic system is by the time they were recorded.

In work situations other than the notification of a supervisor, the navigation of a data grid can also be an issue. For example, in inventory control form-filling applications where the user might need to go back to make changes or add data. Navigating a data grid is an issue whenever a person needs to suspend an activity until a later time, but also continue working on other aspects of his or her job.

Why Is This a Problem?

This is a problem because it is difficult to navigate in a graceful way on a spreadsheet or form with voice commands. There are a few brute force methods but these appear to be tedious and lower the productivity of the user. They require the user to slowly work their way to the appropriate location. And also cause the user to switch between data entering and commanding.

It is often desirable, for practical and monetary reasons, to not have a visual display. This occurs when the user is mobile and also when there is limited space in the work area. When there is no visual display this problem of navigating a data grid is exacerbated. In these cases, it appears that the user needs to have a mental model of the data in order to navigate through it. This is not only difficult for people to do but in some cases may be impossible when there are other tasks involved.

Concrete Example

This is a simplified example of the system we are building for the poultry industry quality inspectors. In this example the inspector is recording the temperature of different kinds of poultry into the system. The computer automatically records the date and time of the data entry. If the temperature is above 40 degrees then the inspector needs to notify the supervisor and also tell the RicerS to put more dry ice on the chicken pieces in the collection bin. However, as mentioned above, the inspector must continue taking the temperature of samples even if a supervisor is not handy. The sample dialog is below:

Inspector               Computer
---------               --------
white meat              white meat
3 9                     three nine
tenderloins             tenderloins
3 8                     three eight
dark meat               dark meat
4 0                     four zero
white meat              white meat
3 8                     three eight
tenderloins             tenderloins
3 8                     three eight
dark meat               dark meat
4 1                     four one, this temperature is out-of-limits
white meat              white meat
3 9                     three nine
tenderloins             tenderloins
3 8                     three eight
dark meat               dark meat
4 0                     four zero
white meat              white meat
3 8                     three eight
tenderloins             tenderloins
3 8                     three eight
dark meat               dark meat
3 9                     three nine

***Supervisor comes by now, and the inspector informs him or her of the out-of-limit data and needs to mark that the supervisor knows. When they have a visual display they can see about 20 data records at a time, with the white meat, tenderloins, and dark meat on different displays.

In this next example the user is responsible for inputting data about incoming inventory. They enter the product name, product number, price, quantity, supplier, and warehouse location. The user knows all the data for a particular shipment except for the warehouse location which is relayed to him or her after the shipment has been placed in the warehouse. Once again the computer automatically records the data and time of the data entry.

User                            Computer
----                            --------
name                            product name
12 by 12 box                    12 by 12 box
number                          product number
6 7 5 8 4 9                     six seven five eight four nine
cost                            cost
point 2 3                       twenty-three cents
quantity                        quantity
2 0                             twenty
supplier                        supplier
Acme                            Acme
name                            product name
12 by 12 box tops               12 by 12 box tops
number                          product number
6 7 5 8 5 0                     six seven five eight five zero
cost                            cost
point 1 2                       twelve cents
quantity                        quantity
2 0                             twenty
supplier                        supplier
Acme                            Acme

***Now the user gets a message from the warehouse that the 12 by 12 boxes have been stored in row B on the third shelf. The user needs to amend the record for the 12 by 12 boxes to include the warehouse location.


CHI '97 Workshop: Speech User Interface Design Challenges