M.U.S.C.L.E Project

Date: 2003 Site: Centre Pompidou Paris Design architect: Prof ir Kas Oosterhuis

Design team: Kas Oosterhuis, Ilona Lénárd, Bert Bongers, Chris Kievid, Laura Aquili, Remko Siemerink, Sven Blokker

Engineering: onl, d3bn, Festo, buitink Client: Biennale 2000 Venice, mnam/cci centre pompidou paris

This project utilizes an interface of sensors distributed throughout the inside of the structure. The flexible mesh is suspended between Festo actuated pistons which expand and contract to reshape the structure. The structure reacts to the proximity of the user and becomes a constantly shifting object of space.

Code written into the piston funcitions allow the project to set up a two-way interaction; if the MUSCLE Project approaches a tipping angle, it will contract back to the user and right itself. This sets up an interaction between the user that is not necessarily predictable, and relies upon an external and internal feedback system. This comes in the form of a virtual interface set up through VirTools.

The MUSCLE Project has a manual operation that allows users to manipulate individual pistons on a computer screen for direct effect. The object is reconstructed in a virtual space called VirTools and passerby outside the MUSCLE object can use a touch screen to directly activate different pistons to change form of the object. As this object and its causal chain extend into the virtual, its interface changes and the user may understand the object differently and thus react differently to its dynamics. Users on the interior of the object may also directly activate the pistons by proximity which sets up a three-way dynamic between the outside users, the MUSCLE object, and the users on the interior. This is an excellent example of how the virtual may enable a different type of interaction to occur (in this case a three way interaction). As SecondLife architects, it is not difficult to see how we could produce a similar kinetic effect between SL and RL.

MIT Dual Reality Lab - Innovation in cross spatial communication

This link was passed to me by Keystone Bouchard. This group (ResEnv or Responsive Environments Group) works closely with the MIT media lab group and focuses on devices and systems that bridge the virtual and the physical.

.

The above example is called their 'shadow office' which is linked real-time with their physical office. The description labels the design as a two-way interaction, but as far as I can tell the sensors pull information from the physical and display it in the virtual. While this group sees the virtual and physical as two distinct environments (hence the name Dual Reality Lab) I prefer to view this as a singular architecture that extends between the two types of space.

The form that this architecture takes is dependent, at this point, upon the technology that can bridge this gap. For this reason, the MIT team has created a custom framework for this interface in the form of distributed sensors in the physical environment. They call this the PLUG system and as you can see here, consists of various cross platform MIDI devices as well as custom scripting.

http://www.media.mit.edu/resenv/plug/




Much like the physical laws of space govern the structural form of traditional architecture, so technology constraints shape and limit the form of transArchitectures.

This group has recently published a series of papers tracking this progress and their thesis are freely available through the site. They have also branced into various devices and products that take advantage of distributed sensor systems including a Star Trek inspired 'tricorder' device that can see through walls. Some of their other projects can be seen here

http://www.media.mit.edu/resenv/projects.html

I applaud the efforts of the MIT group and I plan to visit with them in my move to the Northeast. While they have some very bright and well connected individuals working with Linden Labs on new forms of interface, this group does not necessarily consist of all architects. For this reason, I think it is our responsibility as virtual architects to learn from the advances that they have made and begin to apply these innovations to our virtual creations. This way we can begin to refine our virtual vocabulary and move toward a set of virtual principals that Keystone and others have begun to identify.

Virtual Space: A New Categorization

Description of Virtual Space
Mixed Reality is ultimately a combination of what we think of as a perceived combination or hybrid of virtual space and physical space. Through extensive personal experience and research on the topic, I have been able to identify three main categories which virtual space may fall into. These may be known as the Virtually Recreated Environment, the Virtual Fantastical Environment, and Virtually Collated Space.


Virtual Recreated Environment
A virtual recreation is constructed from information directly pulled from the physical environment. This includes visual geographical data, 3D computer models of buildings, or processes that occur in such environments. This type of virtual environment is usually employed for the purpose of research, design, or a sense of nostalgia or familiarity. It correlates as closely as possible to physical space which makes it useful for operations such as simulation and rehearsal. These environments are usually constructed to transmit information or to test hypothesis about physical processes or materials. For example, a virtual city may be reconstructed to allow visitors to ease navigation and plan routs before visiting the city in physical space. Another example of this is the use of 3D modeling software by companies to virtually construct a product for pre-production tests ranging from the physical properties of the product in certain conditions to how that product might fit into a human hand.

The idea here is that Virtual Recreation exists on a 1:1 correlation with physical space meaning that it resembles, reflects, and reacts in a similar manner to what we think of as our traditional physical environment. Events in Virtual Recreated Environment (VRE) are predictable because they operate through the same (or similar) physical properties (enabled by algorithmic programming) that we experience in our everyday lives. As a recent example, this seems to have become an ideal for many recent war video games that try as closely as possible to reproduce the extreme conditions, environments, and even sounds experienced by soldiers of war. In fact, many of these game production companies actually conduct hundreds of hours of interviews as well as land surveys to recreate historical environments for the purpose of online virtual battle. Another clear example is the recent development of programs such as Google Earth which I believe will become ever more refined, detailed and interactive as years pass, data accumulates, and the technology continually develops. In a way a recreation of our physical environment is to understand it by capturing it in a sense - we can now hold a virtual earth in the palm of our hand.


Virtual Fantastical Environment
Virtual fantasy exists as a type of space that follows select rules of physical space while also introducing outside elements and processes. When we explore a fantastical world through a created character, we might be able to fly, fight dragons, or listen to animals talk; but some elements (be it gravity, anthropomorphized characters, a narrative, or familiar objects) always ground it to some element of our familiar environment. This grounding becomes the common language which allows different users to interact with the environment. For example, to talk to another person most Virtual Fantasy environments require the user to be within a certain physical proximity of another avatar or character. This is quite intuitive and I have observed many new characters that immediately approach another character to speak with them (through a textual interface). So while the user's avatar might be a dragon flying through the clouds, they are still required to be in earshot of another character to interact with them. This is an example of the realistic grounding that creates the common language for communication. In other words, elements of the Virtual Fantasy environment may be out of place or unfamiliar, but for the environment to work as a common communication platform for a individual in physical space to interact with meaningfully, it must have some familiar elements of physical reality written into it.

Inhabitable virtual environments such as Second Life begin approach virtual recreation in places like Amsterdam (10 city blocks of Amsterdam virtually recreated in virtual space), but the ability for avatars to fly and the lack of complete construction methods (usually just textures plastered onto solid blocks) pushes this into the territory of Virtual Fantasy. Other environments such as SimCity approach Virtual Recreation in another direction by concentrating more upon the inner workings and complexities of a physical city. While this may be the case, these games remain in the realm of Virtual Fantasy for the reason that many enter these environments for a form of escapism or exploration. For this reason, Virtual Fantasy is usually employed for such environments for the purpose of making it different from, but still tied to reality.

Virtual Collated Space
Virtual Collation seems to be reserved for special use and is generally the medium of digital artists or theorists such as Mark Novak's Liquid Architecture. Virtual Collation is the compiling of different layers of information, input, or measured readings layered together either spatially or programmatically (code) in a meaningful way. The resultant combination may have a spatial quality, or may simply be represented by an object or even an individual. The information or input may be directly translated into graphical representations (2D - 4D) or is usually translated somehow into a common format to be interpreted by the inhabitants of this environment. Most of the recent interactive architectural projects utilize this type of virtual environment as it tends to provide insight or meaning into physical phenomena.

Physical phenomena utilized by this type of space can include light, sound, human input, environmental conditions, weather, or data. These are usually translated from local sensory equipment directly into a virtual spatial form. A great example of this is the project Synthecology which takes a musician's music (being played on a physical stage) and translates it into virtual sculptures which grow in a virtual garden. As the musician's change chord, tempo, or cadence, the virtual environment receives this information and incorporates it into the virtual sculptures accordingly. Thus, the sculpture becomes a virtual collation of information received and translated from the physical environment.

Other projects take this to a higher degree and actually produce types of space that are formed or changed spatially due to direct input from people in the local environment. An example of this type of spatial condition is the (Artist Studio) project that takes input from an art show (provided by the visitors to the show) and collates these physical experiences of the art into a virtually created environment. Thus the artwork is represented in a different form to be consumed differently and the visitor gets to experience the art show in an entirely different way (that still somewhat follows the meaning of the show). This way, the virtual environment becomes dependent upon the physical environment and this sets up a dynamic interaction between the two.

Simulation Space
Finally, what I am calling Simulation Space takes advantage of the virtual recreation category as it benefits most from a 1:1 correlation to physical space. Simulation space recreates physical space to a degree that it becomes useful as a practical tool or utility. Examples of this include flight simulators for pilots, software for 3D physical object development, and even some types of video games. Simulation space is the most direct form of Virtual Spatial Recreation as it is literally programmed with the Newtonian properties of physical space. This is addressed more comprehensively in a separate paper 'The Gamer's Rationale' that discusses the topic of simulation and culture in depth.

http://www.evokestudio8.com/Gamersrationale.html

Mixed Reality: Definitions and clarifications

Mixed Reality as a popular term tends to confuse a few ideas and it is necessary to examine what this specifically refers to as well as which portion this proposal will focus on.

With the proliferation of Virtual Reality over the last 30 years or so, there has been a recent emergence and a variety of projects that have begun to explore the fusion of the virtual and physical space. As explained previously, this has become commonly referred to as Mixed Reality. Mixed Reality has been defined by Milgram & Kishino 1994 as that which, "Joins or overlays physical and virtual environments to varying degrees, using a number of different approaches, technologies, and interaction paradigms." Another definition by Benford et.al.1998 describes Mixed Reality as that which "Link(s) and overlay(s) multiple physical and virtual spaces that have three spatial dimensions and one temporal dimension." Milgram and Kishino seem to encompass the general scope of Mixed Reality spaces defining it as both an approach as well as an application while Benford et. al. identifies the spatial characteristics and most importantly the common temporal dimension. Another definition from (Canon Technology 2001 citation and definition here). This project will focus on the perceptual characteristics of mixed reality space and will refer often to the latter definition.

One of the latter studies sought to divide Mixed Reality into two components along a socio-perceptual scale. This study differentiated perceptual Mixed Reality (visual construction and cohesion) from socially based Mixed Reality (or the ability to communicate). This particular project defined MR as a form of social consciousness where the Mixed Reality only existed to the degree that there were conscious beings perceiving the space in ‘real-time.’ Consistency of social interaction (as defined by Inga Tomic-Koludrovic, Mirko Petric and Ivica Mitrovic (2002) Mixed Reality or One Reality: A Social-Semiotic Approach to Hybrid Multiagent Environments, Journal of Artificial Societies and Social Simulation vol 5 no.1 published 2002). This study introduces the term, "hybrid multi-agent environments" which refer to cross social communication between virtual and actual which is not necessarily the focus of this project. This proposal will take into account these ideas, but will primarily focus on the perceptual characteristics of MR as it applies to spatial inhabitation.

Perceptual MR encompasses two simultaneous perceptual realities; the virtual and the physical. These projects usually fall along a sliding scale between projects primarily based in physical or virtual reality. If we look at the example of a music concert, there have been examples of bands creating avatars and piping their music into a virtual recreation of a stadium or coffee shop. There have also been examples of concerts that simply place screens or portals into virtual space to allow virtual viewers to see the actual concert through a screen from a virtual location.
Projects which are primarily based in physical reality with minimal MR intrusion I have labeled ‘Pbased’ projects as these tend to focus on interface screens or digitally integrated objects into the physical environment. These projects focus on an event or an environment that is based in physical reality where the connection to the virtual is simplified or minimized spatially or programmatically. The physical becomes the primary medium and is required for navigational cues.

Virtual based or Vbased projects are primarily experienced though an avatar or virtual representation of the self. Acting though the avatar brings with it the possibility to focus and respond to virtually constructed events and environments. Vbased projects usually include concerts or sports events held in SL where reality is pumped in and experienced through the virtual medium. A recent example of this is a Wimbledon match that was recreated real-time in a virtual environment. Attendees are able to see and track the movement of the players and the ball which are constantly being updated with actual information from the live event. Another example is the recent creation of a landscape consisting of boxes representing the business of the S&P 500. As stock prices change, the boxes change from red to green and size themselves according to their current price. The result is a constantly shifting virtual landscape which is being fed with real-time data from a physical space.

What is transArchitectural Topography?

I will define transArchitectural Topography as the substrata or virtual/physical topography which underlie all connected virtual and physical spaces. transArchitectural Topography encompasses a physical location, virtual location(s), and any infrastructure that connects the two. Specifically, the transArchitectural Topography extends from a single physical space, into a single virtual space uniting the two. As Marcos Novak describes the idea of transArchitecture, he speaks about both a visible and invisible element to it. The invisible is referred to as the underlying structural system or infrastructural system which supports the architecture in either case. Novak sees this architecture as one which exists across spatial conditions.

When Marcos Novak continues to describe the virtual, he describes it as a set of possible spatial iterations which exists within any given physical space. If we think of the dimensional topography which underlies these two conditions of the physical and the virtual it shifts from a fixed to an elusive position (in other word from the actual to the possible). As a result, the topography must move from a fixed position (or a defined set of fixed coordinate points) to a non-position or set of possible positions (range of Cartesian coordinate points).

If we examine this scenario with a more concrete example, we can look at a case of two rooms that might be positioned adjacently. One of these is fixed in space and the other room is a virtual space that has no set location, but may exist within a range of locations. Much like the Heidegger principal in quantum mechanics, the instant we attempt locate the second space, it becomes dimensionally fixed in space and loses its status as a virtual object. It is easier to think of the second space as simply existing as a virtual set of possible locations. The figure below displays how this might be spatially considered. The single grouped piece at the bottom represents the addition of all the possible ways two spaces may connect given a common boundary. The blue represents the physical while the gold represents the virtual possibilities for combination and placement. If we try to fix the virtual space in definitive coordinates (for example if we were to superimpose holographic elements into a fixed room) then that space becomes a physical space as it is now defined and bounded by a set of fixed dimensional coordinate points.


To further explain this paradox, we can imagine drawing a straight line from a physical space into a virtual space. The line remains straight until it arrives at the boundary between the physical and the virtual, and then breaks off into an infinite number of possible lines. The physically drawn line is the physical manifestation of all possible lines for a given context which then become visualized as we enter the virtual.

It is for this reason that the virtual may be called liquid, as its underlying construct is constantly in flux until it crystallizes into actual form - much like water and ice. When water is exposed to a particular condition, it hardens and takes on a fixed dimensional form. Any of the H2O particles of water can become part of the ice, but are not considered ice until they become restructured and defined in a fixed position. The same analogy may be used for the transArchitectural Topography. The physical architecture that we see and experience is a crystallization of all possible forms that may occur in that space. Another analogy is that of kinetic and potential energy. Potential energy stores within it the capability to be converted to kinetic energy (movement) at any given time. While we do not say that the potential energy exists necessarily as a tactile property of the object, it is a way to understand the possible physical actions that an object might take. As a result, the physical form or physical space contains within it all possible forms or spaces that may occur, but only the physical may be occupied due the physical nature of our bodies (also fixed dimensional physical elements).

As the physical structure approaches the virtual it will begin to represent all possible spatial iterations contained within it.

Welcome

Greetings all,

After recently completing my thesis and gaining the opportunity to research various projects, proposals, and designs regarding the virtual/physical divide, I thought it might be a good idea to apply that knowledge to Second Life designs and record it in this blog. The most frequently used categories I have been able to identify regarding this type of project is 'Hybrid Spatial Environments,' 'Mixed Reality Architecture,' and/or 'transArchitecture' (the last coined by Marcos Novak). This encompasses a very large and diverse group of projects and I think it is necessary to begin to find consistency and continuity for this group which will only lend credibility to its development.

With some of the recent projects I have examined and participated in, there is usually an element of recreation, duplication, or the idea of a physical/virtual duality. Examples of this are recent concerts or athletic events 'piped' into SecondLife with the suggestion that one can attend the physical concert or have the opportunity to 'attend' the concert in a virtual environment. I propose instead that we think of a singular structure or event that exists along the
virtual<--->physical continuum. If we can think of it in terms of architecture, it might be a structure which has one end planted in the physical realm and one in the virtual.

Because these two environments (the physical and the virtual) are qualitatively different in terms of physical forces, scripting possibilities etc., an architecture or an environment which can exist across these two realms must contain an idea of consistency or continuity between the two spaces. This blog is dedicated to the identification of those traits as well as architectural proposals that stem from this investigation.