Tuesday, February 5, 2008

CoSpaces-Augmented Reality and Collaborative Environments

Wow! Have a look at this organization. The 'CoSpaces' group describes themselves as striving toward, "innovative collaborative workspaces for individuals and project teams within distributed virtual manufacturing enterprises." No kidding. As you can see in the introductory video at the top of their site... they mean business.

I was working on something similar in my own proposals (which I will present in an upcoming post) but it seems this group is well on top of it. The application toward the architectural and construction industries is clear. One of the lead innovators of this group is Holger Schnadelbach who's work I will be reviewing over the next couple weeks. He has most recently been working out of the Mixed Reality Laboratory located within the University of Nottingham.

It is not difficult to see the benefit here of reconstructing exact virtual replications of our RL structures. While this may not represent the full potential of transTopographical representation, it certainly has its practical uses. For example, municipalities currently require land surveys and blueprints of our built environment for reference in matters of safety and security, fire protection, zoning, etc. One can imagine the eventual requirement of 3D digital models as augmented reality technologies become more prevalent in the business sector, search and rescue, security, and construction industries.

Sunday, January 13, 2008

The Gate - social communication and the mixed reality boundary

I recently viewed a demo video of 'The Gate' project at the at the inaugural exhibition for the opening of iMAL new venue, 4-7 OCT 2007 in Brussels. This installation utilizes a single mixed reality boundary (MRB) to connect physical space with the virtual world known as Second Life. The project shows some interesting interactions that take place between the two interconnected spaces including spontaneous dancing (by both physical occupants and avatars) as well as some attempts at textual communication. The participants at both ends seem enthusiastic about expression and communication across spatial types and more importantly- having fun.

The installation makes an attempt at reaching a 1 to 1 scale and the placement of the mixed reality boundary (and its camera) reinforces this. I think this is crucial as this seems the best way to maintain consistency through heterogeneous environments viewed through a screen or display. Thus, when the camera angle changes or positions itself away from user eye-height or 1:1 scale, the user becomes conscious of being an observer and the immersion factor is lost. For this reason, 'life-size’ connection between spaces must be maintained and the MRB scale of this project successfully reflects this.

The Gate project also adheres to the 'consistency of information' concept as an attempt is made to match similar structures in both environments. For example, the Second Life columnar portal echoes the column/beam structure housing the screen in physical space. This seems to be a minor aesthetic consideration as the main focus is on the content provided through the projection screen itself. This approach of minimizing the structural characteristics of the MRB and attempting to work within existing architectural elements serves to reinforce the overall focus on content over structural expression. We may begin to see other architectural opportunities for aesthetic incorporation of the MRB as we begin to see products such as translucent projection screens and holoscreens nearing market release.

I did not see specific communication across the MRB in this case, but this might be a consideration for future projects. In my own thesis ruminations, I began to propose an MRB allowing for textual communication between spaces.

Specifically, I was proposing installation of a keyboard or other interface device to allow people in physical space to communicate textually with avatars and vice versa. With the introduction of VOIP and voice recognition software, we may soon develop some exciting new forms of cross boundary communication. One of the main purposes of these types of installations, besides a questioning of spatial inhabitation, is social communication. New technologies and interfaces adopted by the MRB can only serve to facilitate this into a seamless transtopographical experience.
Up next... and examination of the hybrid spatial pocket.

Wednesday, January 2, 2008

Application of the Mixed Reality Boundary - Liberate Your Avatar

This project was displayed during the Urban Screens Manchester 2007 festival. It is aptly called 'Liberate Your Avatar' and has a dedicated web page here. Paul Sermon has created a space where people can interact with avatars through a single Mixed Reality Boundary. As you can see, this boundary exists in both physical space and virtual space simulatneously. He creates a composite image that allows people and avatars to 'inhabit' the same space while simultaneously (bodily) occupying their own space.
As described on the website:
"The merged realities of ‘All Saints Gardens’ on Oxford Road, and its online three-dimensional counterpart in ‘Second Life’ will, for the first time, allow ‘first life’ visitors and ‘second life’ avatars to coexist and share the same park bench in a live interactive public video installation. Entering into this feedback loop through a portal between these two parallel worlds this event exposes the identity paradox in Second Life."
I analyze this relationship in my thesis on Transarchitecture but Paul has done a wonderful job of excecuting this interaction. Bravo!

Mixed Realities Competition

Great event coming up if anyone can attend. A symposium and exhibition called Mixed Realities will showcase the winners of the competition run by Turbulence.org in 2007. The exhibition of the chosen 5 entries opens on Feb. 7th 2008 at Emerson College in Boston. They will also be conducting some sort of workshop on Feb. 8th and 9th.

These winning entries, "explore the convergence—through cyberspace—of real and synthetic places made possible by computers and networks."

It sounds like these artistic works utilize Second Life to experiment with a hybrid spatial condition between the virtual and physical. This is enabled through real-time communication between physical and virtual objects.

Hope to see you there. I'm registering for Saturday.

Wednesday, October 24, 2007

TransTopography New York City- CSI in Second Life

Hello all. I saw this little blurb on the Gateway homepage recently. Looks interesting but I haven't seen the episode yet. Anyone who gets a chance to check this out please report back..

Wednesday, October 24 10-11PM CBS, TV-14-VINSIDE LOOK CSI: NY It starts with a dead woman dressed like an avatar - but where it goes from there is really odd, says CSI: NY executive producer Anthony E. Zuiker. The show moves into the virtual world Second Life, where ''we're going to re-create New York City,'' says Zuiker, who's also setting up other crimes viewers can solve online. Tonight's avatar-murder case will continue on the Web through February, then jump back to TV just in time for sweeps - not that the tech-unsavvy will miss anything. ''If you don't have a computer you're not behind,'' promises Zuiker. ''We refuse to punish the viewer for not logging into Second Life.'' Unless, that is, you look like an avatar.

Monday, July 16, 2007

M.U.S.C.L.E Project

Date: 2003 Site: Centre Pompidou Paris Design architect: Prof ir Kas Oosterhuis

Design team: Kas Oosterhuis, Ilona Lénárd, Bert Bongers, Chris Kievid, Laura Aquili, Remko Siemerink, Sven Blokker

Engineering: onl, d3bn, Festo, buitink Client: Biennale 2000 Venice, mnam/cci centre pompidou paris

This project utilizes an interface of sensors distributed throughout the inside of the structure. The flexible mesh is suspended between Festo actuated pistons which expand and contract to reshape the structure. The structure reacts to the proximity of the user and becomes a constantly shifting object of space.

Code written into the piston funcitions allow the project to set up a two-way interaction; if the MUSCLE Project approaches a tipping angle, it will contract back to the user and right itself. This sets up an interaction between the user that is not necessarily predictable, and relies upon an external and internal feedback system. This comes in the form of a virtual interface set up through VirTools.

The MUSCLE Project has a manual operation that allows users to manipulate individual pistons on a computer screen for direct effect. The object is reconstructed in a virtual space called VirTools and passerby outside the MUSCLE object can use a touch screen to directly activate different pistons to change form of the object. As this object and its causal chain extend into the virtual, its interface changes and the user may understand the object differently and thus react differently to its dynamics. Users on the interior of the object may also directly activate the pistons by proximity which sets up a three-way dynamic between the outside users, the MUSCLE object, and the users on the interior. This is an excellent example of how the virtual may enable a different type of interaction to occur (in this case a three way interaction). As SecondLife architects, it is not difficult to see how we could produce a similar kinetic effect between SL and RL.

Friday, July 13, 2007

MIT Dual Reality Lab - Innovation in cross spatial communication

This link was passed to me by Keystone Bouchard. This group (ResEnv or Responsive Environments Group) works closely with the MIT media lab group and focuses on devices and systems that bridge the virtual and the physical.
The above example is called their 'shadow office' which is linked real-time with their physical office. The description labels the design as a two-way interaction, but as far as I can tell the sensors pull information from the physical and display it in the virtual. While this group sees the virtual and physical as two distinct environments (hence the name Dual Reality Lab) I prefer to view this as a singular architecture that extends between the two types of space.
The form that this architecture takes is dependent, at this point, upon the technology that can bridge this gap. For this reason, the MIT team has created a custom framework for this interface in the form of distributed sensors in the physical environment. They call this the PLUG system and as you can see here, consists of various cross platform MIDI devices as well as custom scripting.


Much like the physical laws of space govern the structural form of traditional architecture, so technology constraints shape and limit the form of transArchitectures.

This group has recently published a series of papers tracking this progress and their thesis are freely available through the site. They have also branced into various devices and products that take advantage of distributed sensor systems including a Star Trek inspired 'tricorder' device that can see through walls. Some of their other projects can be seen here


I applaud the efforts of the MIT group and I plan to visit with them in my move to the Northeast. While they have some very bright and well connected individuals working with Linden Labs on new forms of interface, this group does not necessarily consist of all architects. For this reason, I think it is our responsibility as virtual architects to learn from the advances that they have made and begin to apply these innovations to our virtual creations. This way we can begin to refine our virtual vocabulary and move toward a set of virtual principals that Keystone and others have begun to identify.

Saturday, July 7, 2007

Virtual Space: A New Categorization

Description of Virtual Space
Mixed Reality is ultimately a combination of what we think of as a perceived combination or hybrid of virtual space and physical space. Through extensive personal experience and research on the topic, I have been able to identify three main categories which virtual space may fall into. These may be known as the Virtually Recreated Environment, the Virtual Fantastical Environment, and Virtually Collated Space.

Virtual Recreated Environment
A virtual recreation is constructed from information directly pulled from the physical environment. This includes visual geographical data, 3D computer models of buildings, or processes that occur in such environments. This type of virtual environment is usually employed for the purpose of research, design, or a sense of nostalgia or familiarity. It correlates as closely as possible to physical space which makes it useful for operations such as simulation and rehearsal. These environments are usually constructed to transmit information or to test hypothesis about physical processes or materials. For example, a virtual city may be reconstructed to allow visitors to ease navigation and plan routs before visiting the city in physical space. Another example of this is the use of 3D modeling software by companies to virtually construct a product for pre-production tests ranging from the physical properties of the product in certain conditions to how that product might fit into a human hand.

The idea here is that Virtual Recreation exists on a 1:1 correlation with physical space meaning that it resembles, reflects, and reacts in a similar manner to what we think of as our traditional physical environment. Events in Virtual Recreated Environment (VRE) are predictable because they operate through the same (or similar) physical properties (enabled by algorithmic programming) that we experience in our everyday lives. As a recent example, this seems to have become an ideal for many recent war video games that try as closely as possible to reproduce the extreme conditions, environments, and even sounds experienced by soldiers of war. In fact, many of these game production companies actually conduct hundreds of hours of interviews as well as land surveys to recreate historical environments for the purpose of online virtual battle. Another clear example is the recent development of programs such as Google Earth which I believe will become ever more refined, detailed and interactive as years pass, data accumulates, and the technology continually develops. In a way a recreation of our physical environment is to understand it by capturing it in a sense - we can now hold a virtual earth in the palm of our hand.

Virtual Fantastical Environment
Virtual fantasy exists as a type of space that follows select rules of physical space while also introducing outside elements and processes. When we explore a fantastical world through a created character, we might be able to fly, fight dragons, or listen to animals talk; but some elements (be it gravity, anthropomorphized characters, a narrative, or familiar objects) always ground it to some element of our familiar environment. This grounding becomes the common language which allows different users to interact with the environment. For example, to talk to another person most Virtual Fantasy environments require the user to be within a certain physical proximity of another avatar or character. This is quite intuitive and I have observed many new characters that immediately approach another character to speak with them (through a textual interface). So while the user's avatar might be a dragon flying through the clouds, they are still required to be in earshot of another character to interact with them. This is an example of the realistic grounding that creates the common language for communication. In other words, elements of the Virtual Fantasy environment may be out of place or unfamiliar, but for the environment to work as a common communication platform for a individual in physical space to interact with meaningfully, it must have some familiar elements of physical reality written into it.

Inhabitable virtual environments such as Second Life begin approach virtual recreation in places like Amsterdam (10 city blocks of Amsterdam virtually recreated in virtual space), but the ability for avatars to fly and the lack of complete construction methods (usually just textures plastered onto solid blocks) pushes this into the territory of Virtual Fantasy. Other environments such as SimCity approach Virtual Recreation in another direction by concentrating more upon the inner workings and complexities of a physical city. While this may be the case, these games remain in the realm of Virtual Fantasy for the reason that many enter these environments for a form of escapism or exploration. For this reason, Virtual Fantasy is usually employed for such environments for the purpose of making it different from, but still tied to reality.

Virtual Collated Space
Virtual Collation seems to be reserved for special use and is generally the medium of digital artists or theorists such as Mark Novak's Liquid Architecture. Virtual Collation is the compiling of different layers of information, input, or measured readings layered together either spatially or programmatically (code) in a meaningful way. The resultant combination may have a spatial quality, or may simply be represented by an object or even an individual. The information or input may be directly translated into graphical representations (2D - 4D) or is usually translated somehow into a common format to be interpreted by the inhabitants of this environment. Most of the recent interactive architectural projects utilize this type of virtual environment as it tends to provide insight or meaning into physical phenomena.

Physical phenomena utilized by this type of space can include light, sound, human input, environmental conditions, weather, or data. These are usually translated from local sensory equipment directly into a virtual spatial form. A great example of this is the project Synthecology which takes a musician's music (being played on a physical stage) and translates it into virtual sculptures which grow in a virtual garden. As the musician's change chord, tempo, or cadence, the virtual environment receives this information and incorporates it into the virtual sculptures accordingly. Thus, the sculpture becomes a virtual collation of information received and translated from the physical environment.

Other projects take this to a higher degree and actually produce types of space that are formed or changed spatially due to direct input from people in the local environment. An example of this type of spatial condition is the (Artist Studio) project that takes input from an art show (provided by the visitors to the show) and collates these physical experiences of the art into a virtually created environment. Thus the artwork is represented in a different form to be consumed differently and the visitor gets to experience the art show in an entirely different way (that still somewhat follows the meaning of the show). This way, the virtual environment becomes dependent upon the physical environment and this sets up a dynamic interaction between the two.

Simulation Space
Finally, what I am calling Simulation Space takes advantage of the virtual recreation category as it benefits most from a 1:1 correlation to physical space. Simulation space recreates physical space to a degree that it becomes useful as a practical tool or utility. Examples of this include flight simulators for pilots, software for 3D physical object development, and even some types of video games. Simulation space is the most direct form of Virtual Spatial Recreation as it is literally programmed with the Newtonian properties of physical space. This is addressed more comprehensively in a separate paper 'The Gamer's Rationale' that discusses the topic of simulation and culture in depth.


Mixed Reality: Definitions and clarifications

Mixed Reality as a popular term tends to confuse a few ideas and it is necessary to examine what this specifically refers to as well as which portion this proposal will focus on.

With the proliferation of Virtual Reality over the last 30 years or so, there has been a recent emergence and a variety of projects that have begun to explore the fusion of the virtual and physical space. As explained previously, this has become commonly referred to as Mixed Reality. Mixed Reality has been defined by Milgram & Kishino 1994 as that which, "Joins or overlays physical and virtual environments to varying degrees, using a number of different approaches, technologies, and interaction paradigms." Another definition by Benford et.al.1998 describes Mixed Reality as that which "Link(s) and overlay(s) multiple physical and virtual spaces that have three spatial dimensions and one temporal dimension." Milgram and Kishino seem to encompass the general scope of Mixed Reality spaces defining it as both an approach as well as an application while Benford et. al. identifies the spatial characteristics and most importantly the common temporal dimension. Another definition from (Canon Technology 2001 citation and definition here). This project will focus on the perceptual characteristics of mixed reality space and will refer often to the latter definition.

One of the latter studies sought to divide Mixed Reality into two components along a socio-perceptual scale. This study differentiated perceptual Mixed Reality (visual construction and cohesion) from socially based Mixed Reality (or the ability to communicate). This particular project defined MR as a form of social consciousness where the Mixed Reality only existed to the degree that there were conscious beings perceiving the space in ‘real-time.’ Consistency of social interaction (as defined by Inga Tomic-Koludrovic, Mirko Petric and Ivica Mitrovic (2002) Mixed Reality or One Reality: A Social-Semiotic Approach to Hybrid Multiagent Environments, Journal of Artificial Societies and Social Simulation vol 5 no.1 published 2002). This study introduces the term, "hybrid multi-agent environments" which refer to cross social communication between virtual and actual which is not necessarily the focus of this project. This proposal will take into account these ideas, but will primarily focus on the perceptual characteristics of MR as it applies to spatial inhabitation.

Perceptual MR encompasses two simultaneous perceptual realities; the virtual and the physical. These projects usually fall along a sliding scale between projects primarily based in physical or virtual reality. If we look at the example of a music concert, there have been examples of bands creating avatars and piping their music into a virtual recreation of a stadium or coffee shop. There have also been examples of concerts that simply place screens or portals into virtual space to allow virtual viewers to see the actual concert through a screen from a virtual location.
Projects which are primarily based in physical reality with minimal MR intrusion I have labeled ‘Pbased’ projects as these tend to focus on interface screens or digitally integrated objects into the physical environment. These projects focus on an event or an environment that is based in physical reality where the connection to the virtual is simplified or minimized spatially or programmatically. The physical becomes the primary medium and is required for navigational cues.

Virtual based or Vbased projects are primarily experienced though an avatar or virtual representation of the self. Acting though the avatar brings with it the possibility to focus and respond to virtually constructed events and environments. Vbased projects usually include concerts or sports events held in SL where reality is pumped in and experienced through the virtual medium. A recent example of this is a Wimbledon match that was recreated real-time in a virtual environment. Attendees are able to see and track the movement of the players and the ball which are constantly being updated with actual information from the live event. Another example is the recent creation of a landscape consisting of boxes representing the business of the S&P 500. As stock prices change, the boxes change from red to green and size themselves according to their current price. The result is a constantly shifting virtual landscape which is being fed with real-time data from a physical space.