|
|
© 1997 Copyright on this material is held by the authors.
With recent advances in imaging technology, neuroscience has become highly image-intensive; in the last 3 years our lab alone has generated about 100k brain images which consume about 50 gigabytes. We must manage these images and use them in research (annotate, review, exchange between labs, and publish). Driven by this need, our team at the University of Minnesota is working on a viewing and database tool, called the Brain Neighborhood Viewer. We aim both to manage large volumes of data and to invent new modes of neuroscientific research.
For several reasons the Brain Neighborhood Viewer manipulates multiple stacks of 2D images, rather than a reconstructed 3D brain volume. First, it takes advantage of the neuroscientists� existing experience in working with 2D images at the light and confocal microscope. Second, the data we have is not evenly distributed. Rather, it is "globally sparse and locally dense," that is, in a single experiment a particular brain location, e.g., the habenula, may be heavily sampled while other parts of the brain are largely ignored. Third, neuroscientists need to explore pristine data, without the artifacts that can be introduced by reconstruction. Finally, the absence of the reconstruction stage allows easier exploration and comparison of data from different specimens, or from different imaging modalities.
A single 2D stack can be explored in two primary ways: the user can see the image at multiple resolutions and scroll images too large to fit on the screen (i.e., a pan/zoom interface); and the user can "rolodex" through a stack of images to view deeper or shallower slices of the particular brain. Even with these operations, though, exploration is severely hampered when only a single stack is visible. Identifying location is very difficult with only a single view, and most scientific exploration requires the comparison of multiple views of related phenomena.
A fundamental innovation in the Brain Neighborhood Viewer is the notion of brain neighborhood, a specific location of the brain that can be located and viewed in different image stacks. The viewer is used to see a set of coordinated views, from different image stacks, which may have been obtained from different cutting planes, on a particular neighborhood. Once the geometric relationships between stacks are established, the viewer tools can maintain coordination between views as a neuroscientist navigates through the brain. We call that "yoked" viewing.
For example, the figure below shows three orthogonal views of similar rat brains (referred to as coronal, saggital, and horizontal views). Once these views are yoked, any navigation in one view (e.g., rolodexing to a different image in the coronal view) results in corresponding navigation in the other views (i.e., panning in the sagittal and horizontal views). A user can unyoke stacks from a brain neighborhood, and explore them independently, and later reyoke again. A second common use of yoked viewing is to view stacks of images from differently treated brains (e.g., a treatment and a control brain) from the same orientation. In this case, yoking allows the user to find a location in only one stack and have the corresponding location selected in all related stacks.
Our experience is that using the neighborhood metaphor can help neuroscientists discover interesting features of the brain. However, establishing the yoking relationships between image stacks is a difficult task, and we are trying to build better tools into the Brain Neighborhood Viewer to support it.
Brain images are captured at different resolutions, and the most detailed ones are huge (10 - 100 MB), taking considerable time to load and manipulate even on fast workstations. Fortunately, neuroscientists do not need full-resolution images for all tasks. Lower resolution images work well enough for navigating to a neighborhood of interest. Once there, full-resolution images are sometimes needed to see interesting artifacts. We are exploring multiple zoom levels and incremental resolution in the Neighborhood Viewer. Incremental resolution allows us to quickly position to a brain neighborhood, and then explore fine features as resolution is increased.
Simply displaying images to a single user is not sufficient. Neuroscientists frequently return to images of interest, and often collaborate with others as they explore them. We are developing a concept of a Uniform Neighborhood Locator (UNL), a unique identifier for a neighborhood. We can then easily support annotations, history, and collaboration. The information about active image stacks, their zoom and resolution level can be attached to a UNL which is saved or passed along to a colleague. We should note that UNLs must be two-way�it is as important to easily find annotations that are relevant to a particular neighborhood or image as it is to find the images from the bookmark.
Since a single brain image can be large and neuroscientists want to simultaneously view a number of images, display space is at a premium. The Brain Neighborhood Viewer supports distributing its views among multiple displays and workstations. This gives the additional benefit of splitting expensive image manipulation between multiple processors, and thus improving responsiveness of the interface.
The distributed version of the Brain Neighborhood Viewer suggested to us the possibility of improving the interaction between neuroscientists in different labs by adding collaborative features to the viewer. A synchronous collaborative Brain Neighborhood Viewer is operational. Several neuroscientists can explore the same brain neighborhood; the software will keep their views coordinated. (We believe this can be particularly useful in a classroom situation.) The neuroscientists in our project are spread over seven time zones, so we are also exploring asynchronous collaboration, including history replay and voice annotations.
The Brain Neighborhood Viewer is a tool for browsing large databases of stacks of 2D images. We have found it to be useful for neuroscience research, and expect that the paradigm may apply to other applications as well. The key interface concepts are image stacks, yokings and neighborhoods; and we have found that practical concerns make it worthwhile to use multiple image resolutions, distributed viewers, and uniform locators to support for single- and multple-user viewing
We are grateful to John Riedl and Robert Elde for the cooperation on the project.
|
|