Research

Summary of Research

The following is the introduction to my dissertation, I am here – Are you there? Sense of Presence and Implications for Virtual World Design (edited to eliminate some long trying-to-the-reader passages that consist entirely of authors’ names and semi-colons). The diagram below  represents the five conceptual frameworks for the research, the two theoretical foundations (collaboration and the sense of presence), and a general statement of the purpose of the research: to understand the human-computer interaction design of virtual learning environments (for higher education distance learning in particular), and the design elements that support the development of a sense of presence, that create and structure a virtual space that can become a “social place inhabited by ‘real’ communities of learners” (Cottone & Mantovani, 2003).

I developed a new construct for the sense of presence with four dimensions (sense of place, social presence, individual agency, and mediated collaborative action chains), where presence is defined as the ongoing result of a collaborative action-based process, in terms of contextualized human experience. I  also developed nine design principles, drawn from research on computer-supported collaborative learning, human-computer interaction design and work on the design of virtual worlds for education or entertainment. The construct has been operationalized both with respect to the four dimensions of the construct and with respect to the nine guidelines, in the Presence/Collaborative Learning in Virtual worlds matrix.

My research dealt specifically with the virtual world, Second

Life, evaluating the effect on the sense of collaborative learning spaces designed according to the sense of presence construct I proposed, using two of the nine design principles (wayfinding and annotation). I also looked for relationships, if any, among the four dimensions of presence. The major research findings were: (a) the construct appears robust, and future research in its application to other virtual worlds may be fruitful; (b) the experience of wayfinding (finding a path through virtual space) resulted overall in an observed pattern of a slightly stronger sense of place; (c) the experience of annotation (building) resulted overall in an observed pattern of a slightly stronger sense of agency; and (d) there is a positive association between sense of place and sense of agency.

Introduction

For the most part, use of new media has been considered second-best to actually being present in the original sense of the word. However, direct presence is not always possible. With the globalization of work, use of computer-supported collaborative work environments has become widespread. With the explosion in distance education and the adoption of hybrid or blended instruction, institutions of higher education have begun to depend much more heavily on virtual learning environments. Some of the causes for this trend include: (a) need to expand access to counter insufficient higher education infrastructure to accommodate enrollments; (b) students’ demands for courses that meet their schedules and circumstances; (c) increased competition from for-profit institutions of higher education and resulting change in the institutional landscape; (d) increases in costs (and tuitions) outpacing inflation; (e) increased emphasis on graduation requirements for technological fluency; and (f) improvements in the versatility and usability of technology and its potential to support new learning activities that cannot be offered in a face-to-face environment (e.g., simulations; Howell, Williams, & Lindsay, 2003). An emerging concern is the effect on the quality of education of this increased use of virtual environments. A challenging aspect for the design of online environments for computer-supported collaborative learning is the development and maintenance of the sense of presence (Kirschner, Strijbos, Kreijns, & Beers, 2004; Kreijns, Kirschner, & Jochems, 2002, 2003; Palloff & Pratt, 1999; Picciano, 2002; Rourke, Anderson, Garrison, & Archer, 2001; Whitelock, Romano, Jelfs, & Brna, 2004). Design elements and implementation practices can facilitate or hinder this development.

Next generation learners.  A new generation of learners is arriving at these institutions of higher education at the same time as the institutional landscape for higher education is changing. These learners are accustomed to operating in a personalized ubiquitous environment that integrates collaboration, cooperation, communication and the intense use of media-rich networked technologies for multitasking (Prensky, 2001). They expect a similar environment when they arrive at the university, as they should (Castronova, 2005) and find instead a deeply text-based culture of educators accustomed to generations of patient, passive listeners.

New genres of virtual environments.  Synchronously and perhaps serendipitously, a new genre of virtual environments is emerging and gaining considerable popular recognition and use, in the form of 3D multiuser virtual worlds. These worlds are compelling, engaging online spaces for entertainment, personal expression, commerce and social interaction. These worlds are also the native habitat (mostly as Massively Multiplayer Online Role Playing Games, or MMORPGs) for the new generation of learners. Designers of such commercial worlds have to attract and retain attention and motivation of players. This competitive pressure may account for the effectiveness of commercial designer practices compared to those of learning environment designers in higher education as well as those of virtual reality researchers, “not only in technical aspects such as graphics or networking, but also in how game designers have managed their online worlds as social environments” (B. Brown & Bell, 2006, p. 228).

Virtual worlds such as Second Life which were designed for entertainment may have new features that support formal collaborative learning. Virtual world design features of interest include 3D graphical interfaces, customizable avatars, synchronous and asynchronous communication, support for self-generated social structures such as groups, built-in infrastructures for world-building and creation and distribution of learner-created content (objects, simulations, documents), scripting for programming intelligent objects, customized application development, and integration with external web resources and learning management systems. The result of this wide range of features and the conceptualization of these virtual worlds as places with inhabitants or residents rather than members, users, or login accounts, may have advantages over traditional text-based online learning environments in creating a sense of presence. Earlier text-based virtual environments (Multi-user Dungeon/Dimension/Domain, or MUDs, and MUD Object-Oriented/Multi-user Object Oriented Systems, or MOOs) had many of the same capabilities; however, the influence the wide range of new design features might have on presence is not clear. In addition, the virtual worlds often privilege the sense of sight, allowing different views than are possible in real life, and make possible other manipulations such as “radical changes in the relative sizes of the participant and virtual objects [making it] possible for students to enter an atom . . . At the other extreme it is possible for students to get a sense of the relative sizes of and distances between planets of the solar system by flying from one to the other” (Winn, 1993, p. 9). In a virtual world, a resident can wear a “Heads-Up Display” (HUD) which provides “first person knowledge about objects and events that are accessible to them in the real world only as third-person descriptions” (p. 9).

Design and New Capabilities of Virtual World Technologies

Instructional design is based on an underlying theory of learning and the mind (whether the theory is implicit or explicit), and also on the capabilities of technologies and tools that learning environment designers have available to them. One way to view the range of instructional design approaches is to use the progression from deterministic, to systemic, to probabilistic described by Kirschner et al. (2004).

Deterministic and systemic instructional design.  Earlier generations of online learning environments were “deterministic” (supporting traditional instructional design, from a cognitive psychology perspective), in that they focused on “individual learning outcomes by influencing or controlling instructional variables to create a learning environment that supports the acquisition of specific skill” (Kirschner et al., 2004, p. 38). Subsequent development in technologies and constructivist theories of learning led to what Kirschner termed a more “systemic” design view focused on learning processes in individuals, where designers attempted to specify complex interdependencies in the learning processes, in advance. This approach shares the problems that arise when knowledge management systems attempt to codify expert tacit knowledge.

Virtual worlds and probabilistic design.  The new genre of open-ended, socially-oriented virtual worlds such as Active Worlds Educational Universe (AWEDU) and Second Life gives learners and learning designers “world-building” power by offering features that make the worlds into open design spaces. Learning designers (and learners) have control over the environment and the objects within it and thus can operate with a “probabilistic” design view, where complex interdependencies are “treated as unknowns and are not specified” (Kirschner et al., 2004, p. 48). In the probabilistic design approach, the emphasis is on interaction as well as learning processes, and the emergent, collective nature of learning is embraced.

The limit of the deterministic and systemic views is an implicit assumption that learner behavior will remain the same. The probabilistic approach accommodates change in user behavior and interaction (which occurs, one hopes, when they are learning). Kirschner et al. (2004) note: “The question is not what outcomes specific educational techniques and collaborative work forms cause, but rather what activities they actually afford” (p. 49).

Failure to capitalize on capabilities of virtual worlds. The full extent of the potential of the new generation of virtual is explored by few, as existing practice is maintained in the new environment resulting in little or no advantage over use of earlier technologies, a common response to emerging technologies. For example, in Second Life, it is the common practice of learning environment designers to design virtual copies of brick and mortar campuses and buildings without any particular learning design goal. When you can build just about anything you can imagine, why build real life replicas, use the environment for highly decorated chat or for “the simple transference of content from sequential media, [which] makes little sense” (Sherman & Craig, 2003, p. 419)? These applications don’t leverage the capabilities of the environment.

Creative uses of virtual worlds.  According to Sherman and Craig (2003), uses that leverage the capabilities of 3D virtual worlds are those that involve: (a) manipulating objects in a three-dimensional environment for “architectural walkthroughs, design spaces, virtual prototyping, scientific visualization, and medical research, training, and procedures” (p. 414); (b) using the extra dimensionality for representations of data over an x-y-z plot; (c) designing scenarios for “which the goal is to explore or familiarize oneself with a physical place” (p. 416). Further capabilities include a focus on:

1. Problems that cannot be tackled in the physical world (e.g., witnessing the birth of the universe).
2. Problems that cannot be studied safely (e.g., witnessing the turmoil within the funnel of a tornado).
3. Problems that cannot be experimented with due to cost constraints (e.g., let every student practice docking a billion dollar submarine).
4. Problems in “what if?” studies (where virtual exploration could lead to better understanding). (Sherman & Craig, 2003, p. 417)

Study environment. The sense of presence construct developed by the researcher is independent of any particular virtual world. For the purposes of the study, Second Life was chosen because of its nature: (a) as a 3D multiuser open-ended socially-oriented virtual world with a wide range of features and possibilities; (b) high level of accessibility, including use of an open source model for applications; (c) a business model that encourages content creation; and (d) the high level of adoption for development of learning environments for higher education use and the active community of practice maintained by these faculty and staff. Second Life has been variously described as a tool for social networking, for holding three-dimensional visual conversations, and for programming intelligent objects (Brogden, 2007). Most importantly for this study, it is, like Active Worlds Educational Universe (see http://www.activeworlds.com/edu/index.asp), one of a few “platform service[s] for the development [italics added] of shared three-dimensional environments that supports multiple users with real-time communication capabilities through both text and voice” (Rodriguez, 2006, p. 79).

Second Life is “resident-driven and self-evolving” (v3image, 2007, p. 10), in contrast to World of Warcraft, which is a virtual world fantasy game with a pre-established “back story” (that is, an integrated fantasy world, with built-in quests, internal plots and characters).

Second Life residents have unique representations (“avatars”) and can (a) create their own characters, surroundings, and objects; (b) have complete control over the appearance, clothing, behavior of their avatar; (c) make or acquire their own unique clothing; and (d) develop scripts for or acquire animations for avatar gestures and behavior. Ninety-nine percent of content is user-created (Ondrejka, 2004a) using the 3D modeling tool and a scripting language, Linden Scripting Language (LSL), to add behavior to objects. Content creation by residents is the basic world model. An open economy provides for sale and trade of content and resale of land, with a market that determines the value of the creations and real estate, and an exchange process that can convert Linden dollars, the currency of Second Life, into US dollars. The only back story for the world is that open economy. The intellectual property of “in-world” creations is owned expressly by the creator (even if exported elsewhere). The creation and sale of objects is a primary activity in the world.

The virtual world supports basic physics, although residents do have the magical power to fly, and imitates the physical world with sky, sun, moon, water, and land with highly varied terrain and, through animated objects, weather. Virtual land is divided into regions, which are “both geographical and administrative units” (Rymaszewski, Au, Wallace, Winters, Ondrejika, & Batstone-Cunninghma, 2007, p. 8). Landowners own part or all of a region. Groups of avatars can own land jointly. In the case of PepperdineUniversity, the Graduate School of Education and Psychology has purchased a private island for exploration and experimentation.

A large community of practice for Second Life educators (SLED) is very active and it (a) is supportive of teachers new to the environment; (b) offers free tutorials, workshops, seminars and regular in-world meetings; (c) sponsors a successful open source environment that results in many free educational objects and applications; and (d) maintains a web site and an electronic list. According to the web site maintained by the community, over one hundred universities, schools and colleges are using Second Life (see “Learning and teaching,” n.d.).

Design challenges inherent to open-ended, socially-oriented virtual worlds. In addition to the opportunities provided by 3D open-ended socially-oriented virtual worlds, new design issues are introduced, such as: (a) the lack of a back story and the challenge of providing an imperative to action in such an open-ended environment; (b) the confusion engendered by multiple user interfaces with arrays of buttons, menus, and heads-up displays; (c) the chaos and lack of structure which is introduced by the very flexibility and freedom to create that is a strength of such environments; and finally, (d) the learning curve to acquire mastery of such a robust environment, and of the scripting language necessary to develop new objects or interactive sites. The greatest disadvantage is that such environments may be more demanding of the learning designer, who, for effective design, may need to create the scaffolding and structure (or design activities by which the learners do so), and to design open-ended activities that include individual reflection and group dialogue about the experience.

Summary of the Problems to be Addressed by this Study

The changes in the landscape of higher education, increase in online offerings, nature of next generation learners and advancements in technology have converged to elevate the importance of the design of online learning environments for collaborative learning in higher education. Simultaneously, a new genre of virtual environments has emerged, designed for entertainment, personal expression, commerce and social interaction. Open-ended socially-oriented virtual worlds offer a wide range of new capabilities, balanced against the challenges that use of such worlds bring. Research indicates the importance of the sense of presence for computer-supported collaborative learning. To realize the potential of virtual worlds for learning, we need to understand the implications of design on the emergence of the sense of presence. Although adoption of the use of virtual worlds is increasing in higher education, absent a theory-based set of guidelines, most learning environment designers are not capitalizing effectively on the potential of these new virtual worlds. As one commentator noted, “We are like gods without a manual in Second Life” (J. B. Rhoads, personal communication, June 4, 2007).

As to application of previous studies to interaction design based on learning experience, most of the studies are not founded in any explicit theory of the mind, learning, and practice. To realize the potential of these kinds of virtual worlds as platforms for customized collaborative learning environments–that is, as open, world-building design spaces–we need to understand how they might invite the emergence of the sense of presence, and the intent of this study is to use a theory-based approach to extend the existing research.

Sense of Presence as a Multidimensional Construct

For the purposes of this study (understanding the sense of presence in virtual worlds used for formal collaborative learning environments in higher education), the sense of presence is defined as an collaborative action-based process (Spagnolli et al., 2003).

This study suggests we might learn something significant about the sense of presence and collaborative learning in virtual worlds if we include all four dimensions (sense of place, social presence, individual agency, and mediation of collaboration) in a multidimensional construct of the sense of presence, beginning with an assumption that each dimension is separate and logically orthogonal to the other.

To further operationalize the construct, a Presence/Collaborative Learning in Virtual Worlds Matrix was constructed by the researcher for use as a framework for exploring computer-supported collaborative learning and the development of presence in the virtual world, Second Life. The matrix has four columns, one each for the four broad dimensions of presence developed for the purposes of this study: sense of place, social presence, individual agency, and mediation of collaboration. The four columns are mapped against nine rows, each describing principles and guidelines for use of an open-ended, socially-oriented virtual world to create customized collaborative learning environments that invite the emergence of the sense of presence, as higher education students engage in formal collaborative learning activities in Second Life.

Significance of the Study

This study is significant, given: (a) the context of new social spaces with the potential for being harnessed as learning spaces; (b) the historical work that demonstrates the worth and nature of the sense of presence; (c) the gaps in theory-based design practice; (d) the limited implementations of online environments designed for learning as a social practice; and (e) the limitations of unidimensional definitions of presence. In response, the researcher has developed a new construct for the sense of presence with four dimensions (sense of place, social presence, individual agency, and mediated collaborative action chains), where presence is defined as the ongoing result of a collaborative action-based process, in terms of contextualized human experience.

The researcher has also developed nine design principles, drawn from research on computer-supported collaborative learning, human-computer interaction design and work on the design of virtual worlds for education or entertainment. The construct has been operationalized both with respect to the four dimensions of the construct and with respect to the nine guidelines, in the Presence/Collaborative Learning in Virtual worlds matrix.

Purpose of the Study

For the purposes of this study, the researcher has applied two of the nine design guidelines to explore the development of the sense of presence across all four dimensions of presence. The study utilized activities that have been used in other research on presence and collaboration, and was carried out under three conditions to compare two design guidelines, as follows: (a) where wayfinding is foregrounded, (b) where annotation is foregrounded, and (c) where both wayfinding and annotation are implemented together to control for order effects. The study explored to what extent the subjective report of the experience of presence aligns with the hypothesized effect of designed-presence.

The guiding question for inquiry was: What is the effect on the sense of presence in collaborative learning spaces designed according to the sense of presence construct proposed, under three design conditions (wayfinding, annotation, and wayfinding and annotation together) in the 3D open-ended, socially-oriented virtual world, Second Life? Another question of interest: what are the relationships (if any) among the four dimensions of presence described by the construct?

It is hoped that the research can begin to bridge the gap between abstract theory and practice, by providing a theory-based and validated set of guidelines for virtual world design to create customized collaborative learning environments (for higher education students) that invite the emergence of the sense of presence. To the extent the construct has been validated, the design principles based upon it will be useful to learning environment designers for leveraging the capabilities of Second Life, and for addressing the issues and challenges that this new platform for designing learning environments introduces.

Research Methods and Design of Study

Multiple sources for data informed measurement for the proposed construct. Analyses from Second Life interactions was conducted to validate the construct and two principles from the set of theory-based design guidelines based upon it. Twenty learners recruited from the Graduate School of Education and Psychology at PepperdineUniversity carried out assigned collaborative activities under three conditions: where wayfinding was foregrounded, where annotation was foregrounded, and where both wayfinding and annotation were foregrounded.

Experienced-Presence Online Surveys

After each learning activity, learners completed an online survey concerning their subjective experience of presence during the activity. The survey is a combination of three experienced-presence questionnaires developed by other researchers, to test sense of place (Slater et al., 1994; Usoh, Catena, Arman & Slater, 2000), social presence (Biocca, Harms, et al., 2001), and individual agency (Witmer & Singer, 1998). The surveys were elected on the basis of (a) match to the definitions being used in this study for the first three dimensions of presence, (b) on the extent of the surveys use in prior studies, and (c) on external reviews by other researchers as to the validity, reliability, and sensitivity of the instruments. The research examined the means and standard deviations obtained concerning the survey questions, created and evaluated summary statistics, and evaluated the quantitative results for correlations across the dimensions of presence.

Qualitative Data Analysis

General questions of an exploratory nature were also be pursued through: (a) researcher and trained second rater’s open-ended observations of learners carrying out the assigned collaborative tasks under the three design conditions, (b) clarifying and confirmatory interviews with a sample of learners after completion of the collaborative learning activities, and (c) a focus session with expert group of faculty and staff using Second Life for teaching and learning.

Content validation of the construct was established through a semi-structured focus group session with a three-person group of experienced, exemplar members of the Second Life Educators community of practice. Focus group members were asked for feedback on the clarity, utility, and theoretical soundness on the design principles of the Presence/Collaborative Learning in Virtual Worlds model.

An adaptation of Mwanza’s “Eight-Step Process” in her Activity-Oriented Design Model (Mwanza, 2002) was used as the organizing framework for data analysis of the fourth independent variable, mediated collaborative actions/operations chains. A prospective mapping of the nodes of a collaborative learning activity system was performed prior to the experiments to prepare for data collection, and then was revisited given the action and operation chains actually observed during the learning activities.

Mwanza’s “Activity Notation” (Mwanza, 2002), was used to decompose the situation’s activity system into “manageable constitutive units or sub-activity systems…linked together through the shared object of the main activity system” (p. 191).

Interaction analysis was conducted on the qualitative data collected during the collaborative learning activities (observation notes, chat transcripts) to discover occurrences of or references to the phenomena of interest (collaboration and the sense of presence). These were coded according to the Presence/Collaborative Learning in Virtual Worlds matrix, identifying design attributes or tools that supported the phenomenon (whether as designed or in new ways), or for gaps and unmet needs (additional features that might address problems observed in supporting sense of presence in the environment).

REFERENCE LIST

Biocca, F., Burgoon, J. K., Harms, C., & Stoner, M. (2001). Criteria and scope conditions for a theory and measure of social presence, PRESENCE 2001: Fourth Annual International Workshop on Presence. Philadelphia, PA at Temple University: Institute for Presence Research.

Brogden, W. (2007). Web services and second life.  Retrieved September 11, 2007, from http://searchsoa.techtarget.com/tip/0,289483,sid26_gci1255023,00.html

Brown, B., & Bell, M. (2006). Play and sociability in there: Some lessons from online games for collaborative virtual environments. In R. Schroeder & A. S. Axelsson (Eds.), Avatars at work and play: Collaboration and interaction in shared virtual environments (Vol. 34, pp. 228–245). Dordrecht, The Netherlands: Springer.

Castronova, E. (2005). Synthetic worlds: The business and culture of online games. Chicago: University of Chicago Press.

Cottone, P., & Mantovani, G. (2003). Grounding “subjective views:” situation awareness and co-reference in distance learning. In G. Riva, F. Davide & W. A. IJsselsteijn (Eds.), Being there: Concepts, effects and measurements of user presence in synthetic environments (Vol. 5, pp. 249–260). Amsterdam, The Netherlands: IOS Press.

Kirschner, P., Strijbos, J.-W., Kreijns, K., & Beers, P. J. (2004). Designing electronic collaborative learning environments. Educational Technology Research and Development, 52(3), 47–66. doi: 10.1007/BF02504675

Kreijns, K., Kirschner, P. A., & Jochems, W. (2002). The sociability of computer-supported collaborative learning environments. Educational Technology and Society, 5(1).

Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior, 19(3), 335–353. doi:10.1016/S0747-5632(02)00057-2

Mwanza, D. (2002). Toward an activity-oriented design method for HCI research and practice (Doctoral dissertation, The Open University). Retrieved from http://iet-staff.open.ac.uk/d.mwanza/Phd.cfm

Ondrejka, C. (2004a, May 18). Escaping the gilded cage: User created content and building the metaverse. SSRN Accepted Paper Series. Retrieved July 10, 2006 from http://ssrn.com/abstract=538362

Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace: Effective strategies for the online classroom. San Francisco: Jossey-Bass.

Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence, and performance in an online course. Journal of Asynchronous Learning Networks, 6(1), 21-40.

Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5). Retrieved from http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf

Rodriguez, A. M. (2006). Framework for designing and evaluating three-dimensional multi-user virtual environments for effective system and task performances with further applications for second language learning (Doctoral dissertation). Retrieved from Dissertation Abstracts International (AAT No. 3267116).

Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2). Retrieved June, 2008 from http://auspace.athabascau.ca:8080/ dspace/bitstream/2149/732/1/Assessing%20Social%20Presence%20In%20Asynchronous%20Text-based%20Computer%20Conferencing.pdf

Rymaszewski, M., Au, W. J., Wallace, M., Winters, C., Ondrejika, C., & Batstone-Cunninghma, B. (2007). Second life: The official guide. Hoboken, NJ: John Wiley & Sons.

Sherman, W. R., & Craig, A. (2003). Understanding virtual reality: Interface, application, and design. San Francisco, CA: Elsevier Science.

Slater, M., Usoh, M., & Steed, A. (1994). Depth of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 3, 130–144. Retrieved from http://publicationslist.org/data/melslater/ref-24/depth%20of%20presence.pdf

Spagnolli, A., Varotto, D., & Mantovani, G. (2003). An ethnographic, action-based approach to human experience in virtual environments. International Journal of Human-Computer Interaction, 59, 797–822.

Usoh, M., Catena, E., Arman, S., & Slater, M. (2000). Using presence questionnaires in reality. Presence: Teleoperators and Virtual Environments, 9(5), 497–503. doi:10.1162/105474600566989

Whitelock, D., Romano, D., Jelfs, A., & Brna, P. (2004). Perfect presence: What does this mean for the design of virtual learning environments? Education and Information Technologies, 5(4), 277–289.

Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments, 7(3), 225–240. doi:10.1162/105474698565686

Winn, W. (1993). A conceptual basis for educational applications of virtual reality. (Report No. TR-93-9). Retrieved from Human Interface Technology Laboratory, Washington Technology Center, University of Washington, September 26, 2005, from http://www.hitl.washington.edu/publications/r-93-9/