649week04 → Prominent Tools
I’ve shared my view that the history of InfoViz can be divided into two eras, the Big Viz era and the Populist Viz era. This distinction helps me to think about which changes over time are most important and where social and technical aspects of change meet. So I want to mention papers about the two most prominent tools from the Big Viz era still in commercial use today, Spotfire and Inxight, and I’d like to contrast them with some other tools more closely associated with the Populist Viz era, although some of these tools existed long before they were associated with InfoViz.
Two InfoViz Eras (picture links to 50MB presentation)
The academic papers I’d like to share about Spotfire and Inxight are Ahlberg (1996) and Rao et al. (1995). Contrast these with Shneiderman (1999) and an Inxight brochure for a sense of the path these two systems have followed. Both are underrepresented online in comparison with the other systems we’ll discuss today. Both fit very well with the model human processor studies following Card et al. (1983). They cater to highly skilled, articulated, dedicated users. These users do not merely face simple information glut. They possess sophisticated models of the information to which they attend while working. Their information is typically proprietary and obtained from a variety of sources with differing representation standards. A quintessential example is a pharmaceutical researcher examining information about drugs relevant an area of focus for a group within Pfizer. These specialists almost exclusively use desktop displays and may use the infoviz system for hours at a time.
What are the limitations of systems like this? How would you grow the user base and obtain help in using the system? Given that the information is usually proprietary, you will find few examples publicly available. Given that the users are engaged in highly competitive, secretive research, it seems that the adoption of these tools might proceed very slowly, based on competitive intelligence. Inxight was making about 14m USD when it was acquired by SAP, but almost the only publicly available material about it is sales-oriented. This suggests that these systems might have employed fairly sophisticated users as quasi-salespeople / roving tech support. Clients would need to trust such external helpers in at least two ways. First, clients need to believe that their secrets are safe from their competitors, who may also be InfoViz customers. Second, they need to trust that the helpers will not waste the time of their highly paid staff. The helpers must understand something of the client business. There may be a tendency for a symbiotic relationship to develop between such a helper and a group of clients. I have personally observed the case where the helper becomes a human interface to the system for a group that has come to trust this person.
Christopher Ahlberg, Spotfire: An Information Exploration Environment, SIGMOD Record, Vol. 25, No. 4, December 1996.
Rao, R., Pedersen, J. O., Hearst, M. A., Mackinlay, J. D., Card, S. K., Masinter, L., Halvorsen, P., and Robertson, G. C. 1995. Rich interaction in the digital library. Commun. ACM 38, 4 (Apr. 1995), 29-39. DOI= http://doi.acm.org/10.1145/205323.205326.
Card, S., Moran, T. & Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Erlbaum.
One of the thing I’m noting in these papers is the different interaction techniques that the various systems use. Spotfire requires the data to be in a formal database, Many Eyes lets you import data in a text format, and GGobi is accessible either from XML files or from an API. I’m wondering if this has any impact on how the tools are used. It seems that Many Eyes allows anyone to make a visualization, whereas GGobi and Spotfire have higher barriers to entry. However, GGobi’s API means that it can be embedded into other tools, making it more accessible to motivated programmers. Does the accessibility of Many Eyes and GGobi also explain their status as populist viz tools?
>>Does the accessibility of Many Eyes and GGobi also explain their status as populist viz tools?
I would say that’s a necessary condition, especially if accessibility includes ease of use. ManyEyes, I would say, takes the progress that GGobi, and similar tools took in lowering the technology barriers, and pushes those barriers even lower. With GGobi, and similar tools, you had to have a machine. ManyEyes, a rich internet app, lowers the barriers so that technology is almost no longer part of the equation: a visualizer no longer even needs a machine - anyone with access to a browser (i.e., a library card) can now visualize data.
It is not, however, sufficient. There are other issues that visualizers must contend with. The paper stated that most of the users of ManyEyes come from the visualization community (or are data junkies). The lack of a wide adoption (outside the viz community) implies a second problem that populist viz will have to deal with that is largely technology and tool independent: the perceived and real difficulty of creating quality visualizations that communicate their intended message.
The accessibility that GGobi (and ManyEyes, etc) displays is absolutely essential, whether they are populist tools or not. Jeremy brings up a good point about tools being traditionally seen as prohibitively difficult to use, and ManyEyes seems to be bringing down the entry barrier quite a bit. However, the Danis paper has a quote from one of their users complaining about not being able to keep datasets private until they were ready for public consumption. Only a data junkie would probably actually care, and since these tools have a low entry barrier, there is also great potential for inexperienced people to design bad visualizations.
This ties into Mick’s original point about proprietary data - many of these companies would not even consider using ManyEyes. I wonder if GGobi’s extendability has caused problems of adoption in corporate and other settings with proprietary data concerns. Page 433 of Swayne’s CS&D article hints that its extensible framework has not been widely adopted by the statistical computing world, but doesn’t really give much background as to why.
Mick says:
“There may be a tendency for a symbiotic relationship to develop between such a helper and a group of clients. I have personally observed the case where the helper becomes a human interface to the system for a group that has come to trust this person.”
With systems such as ManyEyes now available to the masses, I can’t help but think that over time the clunkier, less usable (although more powerful) tools like Spotfire and Inxight will necessarily evolve to be more accessible and intuitive as well. So that helper, or ‘human interface,’ may not be necessary as these systems evolve. What I’m imagining is some kind of ManyEyes for ‘competitive, secret research’ (as Mick put it). It seems like there would be a substantial market for such an easy-to-use-yet-powerful visualization tool.
>> The academic papers I’d like to share about Spotfire and Inxight are Ahlberg (1996) and Rao et al. (1995). Contrast these with Shneiderman (1999) and an Inxight brochure for a sense of the path these two systems have followed.
Are you saying that the information retrieval system for libraries discussed in Rao et al is what became Inxight? I wouldn’t have guessed it - but I guess it makes sense - what started as a project in academic was turned into a commercial product because companies had the need and would pay for it.
>> Both are underrepresented online in comparison with the other systems we’ll discuss today.
If I understand what you’re saying, you mean they are underrepresented because they are for a highly-skilled audience who uses the data internally. In contrast, ManyEyes is meant for lay users to share data publicly. It’s too bad, though, that we don’t have systems like Spotfire or Inxight available for public use (i.e. for free). They seem to be built for the lay user too, but have more sophisticated visualization techniques available. That would be nice to have for those of us who want to create more advanced visualization than what ManyEyes can offer, but our only other choice really is to program a visualization ourselves using something like Processing. So, are Spotfire and Inxight underrepresented online because the skill level of their users is different (which I’m guessing they’re not, as the Many Eyes study found that many users were experienced with data analysis), or just underrepresented because they have chosen to go after the business analytics market rather than the public market? I see now as I’m writing this that the needs of the two types of users differ - the business user is dealing with the data constantly and over time, and the public user creates quick visualizations occasionally. It seems also that the business user is the one who would most benefit from the community features that are not being used on ManyEyes, as the ManyEyes people found that people preferred to have conversations about their visualization with those in their community of practice.
Microsoft made WIMP the standard for interface design. It took many years to bring alternatives like multitouch to the market. Similarly Google has made text based search as the defacto method. Spotfire is a visual search interface that developed before Google became successful. But I think approaches like Spotfire will be useful in overcoming many limitations of text based search.
Searching is more effective if there is context while providing the search results. Approaching search results as an info-viz problem will make us provide context.
A personal finance tool like mint.com is a good candidate for trying an interface like Spotfire. I think prior or existing SI649 students have worked on personal finance.
Is it possible to privide the search criteria as information visulaization? In a way, Spotfire results pane is also used for specifying search criteria, zooming, panning are all search criteria specified visually.
With the gamut of existing infoviz tools in the market, there is an emerging need to categorize the various methods of visualizing data as well as the information visualization tools available with regards to the type of user. Reaching on common categorization is not very straightforward because of the difficulty in comparing these tools. The targeted audience for Many Eyes and Spotfire were very different. Spotfire was meant for professional analysts to visualize and interact with effectively leading to better decision-making and gaining insights. Many Eyes, on the other hand, placed emphasis on getting lay people familiar with creating, editing and discussing information visualization.
The subject of “closed” tools such as Inxight and Spotfire and the idea of populist infoviz immediately brought to mind Shneiderman’s description of the HomeFinder visualization. Its mashup of available housing information and a geographical layout may seem a little less “wow” today, but at the time it was considered “one of the most compelling and comprehensible” demos available. The key point here is that companies and even universities were unwilling to release housing data for this visualization, despite the fact that it was seen largely as a commercial solution. Fortunately, the creators manually gathered their data from the listings in the Washington Post. But that was about 1992.
This kind of gets into the “open/closed source” debate, but really companies will often lose major competitive advantages if some information is freely viewable…even though information is more freely available now than in 1992. But the populist viz movement seems to be catching up to the sophisticated tools used in these proprietary applications. If we go by Mick’s graphic, the Big Viz movement has been going on for decades, the populist movement has only taken off in the last 10-15. I think it’s also important to consider how ManyEyes is a sort of catalyst for collaboration (even if off the site) that may make the improvement process go even faster. So while Inxight and Spotfire have their specific market (and people are willing to pay for it!), hopefully those willing to be open about their information are not at a major disadvantage as far as tools and technology go.
Strict mathematical use of hyperbolic and perspective projections to highlight certain information while keeping other, peripheral information visible to provide context is an underused technique in the pop-viz movement, but a more abstract use of size to represent importance is very popular: tag clouds and tree maps (only recently entering the pop-viz space) being examples.
I suspect the pop-vizers don’t use hyperbolic projections and other mathematically complex visualization modes because the tools available to them are either straight graphics apps like Illustrator, or simple hierarchy and frequency analyzers like those available using more accessible tools like Many Eyes. As this toolset expands, I expect to see more complex visualizing algorithms becoming more popular.
Contrasting old viz and new viz in terms of accessibility of the technology and needs of the users yields some interesting thoughts. We have been moving towards more standardized ways of managing and gathering data due to the use of API’s on the web, while users’ needs have become less proprietary as more commonly users have access to data from the web. Viz technologies such as Many Eyes have become more accessible to end users who no longer need to construct their own proprietary means of storing and managing their data.
The Viz methods employed today however are quite similar to me in many ways to old viz. The common theme of “magic lens” to use context as a means of driving focus seems to be a common thread amongst users’ desires both old and new. Whereas in old viz these methods were utilized to view proprietary data, the same techniques are now being applied to operating systems and desktop software. While it is arguable if these new viz users are simply suffering from info glut, the projects viewable on Many Eyes and Visually Complex suggest that today’s end users are empowered by easier access to wealths of data and have viz needs beyond organizing their desktop space or contact lists.
There is a difference between the Big Viz tools and the pop-viz: Many Eyes in the sense of the needs of the user base, but also the capacities of the majority user base. I wonder what sort of trade offs it represents to try and place more math/processor intense analytical algorithms in the pop-viz user base? Does supporting more, and more powerful algorithms for modeling and manipulating data sets represent a change to the underlying application that might slow down the program? Of the population of users for Many Eyes are there a really large population of users who will benefit from the more powerful visualizing tools at this stage? I think there may be a time-to-user-capability critical mass that needs to made before more advanced VIZ tools can have an impact outside of generating a more complex user experience and maybe slowing down runtime, at least for the purposes of a user base that can get the most out of a very public tool like Many Eyes. There are reasons to believe that at least some portion of the users who would get the most of more complex viz algorithms may not use it over privacy issues: the Danis article mentions requests for a private mode just for this reason (Danis, 2008).
Mick brings up some Processor apps on a later comment on this topic. I think one thing they (Many Eyes and the Processor examples) share is relative accessibility in user control through simplicity. Limitations on what can be done, but that give some clarity to what capabilities are represented. This comment is just a wondering out loud if expanding the range of visualizing options would just create a “Photoshop-is-too-hard” opacity for a large part of the users, which would go against the very populist message of Many Eyes.
Obviously, I think this will change over time, as the reason for having a Many Eyes is to contribute to greater awareness in the making and the READING of information visualizations. As someone who is still having a bit of a time even making sense of some of the more advanced visualizations without an instructor to guide me through them, I’m not certain that advancing the range of capabilities of a populist visualization tool will necessarily help drive popular awareness to the field, at least not yet.
My impression is that while GGobi and Spotfire is mostly used as research tools combining with advanced statistic method, ManyEyes is kind of introductory and product-sharing tool. Although many people argued that they have different categories of targeted users, there might be possibility that the users of Many Eyes come from those who are skilled info viz developers who have used GGobi and Spotfire a lot. Skilled people are more likely to share their products and get comments for them, and they are more likely to understand the meaning of the graphs. It is said in the paper that the communication on ManyEyes are not as popular as in people’s personal blogs or their own physical enviroment-people don’t care if they do not understand. ManyEyes provide a platform for people to get connection to information visulization, but to further the understanding, people have to learn advanced technology, as well as statistic principles.
The component-based strategy of GGobi is very impressive.