Augmented Reality- HoloLens

By Eric Sauda and Alireza Karduni

Research has been conducted on augmented reality (AR) for several decades. Early work by Azuma helped defined the field [1], and more recent research by Bolter and Barfield has investigated strategies for engaging users[2, 3]. Recent advances in head mounted hardware and software by Microsoft and Google have poised AR for widespread use.

In architectural design, AR is of particular interest because it is inherently spatial and can lead to an understanding of architecture as an interface, marrying space and information.  The release of development versions of Microsoft Hololens has provided us with the opportunity to study how AR may intersect with the architectural setting, which we call the architecture interface.

There is a strong inclination within computer science to view AR as a technology that can adapt to existing spatial settings; technology is developed and dropped into an architectural setting. However, designing AR applications and architectural settings together will allow for exploration of a full range of affordances, feedback mechanism and output/display options, leading to generalized set of usability heuristics for AR. It can become one example of Dourish’s call for embodied interaction[4].

We chose to design for a setting that would present considerable spatial and architectural challenges as well as AR challenges centering on access to large amounts of data. The architectural brief was the Mount Zion Archaeological Park in Jerusalem that has been the site of on-going excavations by archeology researchers. Archeological work on the site is close to completion and a design for a museum and park preserving and explain the site is being contemplated. Huge amount of data have been generated during the eight years of the excavation (type of object, location, relationship to other objects) as well as possible connection to the larger archeological record.

We employ a design science research method that generates knowledge and insight of complex problems by the process of design [5]; further, within this knowledge paradigm, multiple designs can serve to uncover an expanded range of possibilities. Our research was conducted by a team of twelve architectural designers and a team of three Hololens developers. This resulted in the design of 12 distinct designs, each of which proposed a specific architectural form and a specific use of Hololens technology.

We evaluated designs using both normative architectural evaluations methods (geometric clarity, circulation, views, lighting and enclosure) as well as interaction evaluation (digital building models and interaction simulation). The programming of the Hololens was conducted in Unity 3D using C# programming language. Each designer created an architectural design that could be tested, changed and refined as the potentials of the Hololens was explored.

In order to better explore and refine the creations of our team of designers. We created an interface for exploration of their architectural and AR interaction designs. The interface would enable us to import 3D models of buildings and create a walkthrough in the proposed space, as well as to embed the proposed interactions. Our interface allows users to view the space in both perspective and plan view and interact with proposed AR objects and information. It also consists of complimentary views that show a user’s interaction log and finally a view highlighting different AR interactions as a user experiences a proposed intersection of AR and architectural space. Experiments with the interface allowed our team to refine the different affordances to fit best with a designer’s vision and to easily implement the proposed interaction affordances for Microsoft Hololens. A taxonomy was developed from these experiments and became available to all designer

3 FINDINGS

Based on the 12 architectural designs, we developed a taxonomy based on inputs (afforded and interactive) and outputs, (display and physical). Investigations involved the creation of specific architectural paired with the use of AR to deliver supplemental data and perspective. As we proceeded, each designer evolved a set of connections between the architecture interface and AR. We compiled these examples into a taxonomy of possibilities.

Afforded inputs are those that because of physical or cultural factors are understood without need for visual feedback. These include:

• proximity, moving closer to or further from an object in the model was understood without the need for feedback.
• gaze target, the direction of a user’s gaze was immediately apparent to user of the system
• gaze duration, because obvious after a few minutes of use by the Hololens
• the orientation that a user faces is apparent in at least two ways; inside/outside can be understood wherever the architecture makes it obvious; geometric orientation is made obvious when the architectural arrangement is strongly delineated.

Interactive inputs require some visual feedback from the Hololens to make it cognitively present:

• angle of view: head motion can be understood as an input in either the vertical or horizontal direction, but require visual feedback to be understood.
• hand gesture: the standard interface in Hololens of finger or hand gesture require  significant feedback to be obvious
• voice: the use of  voice recognition can be used to provide rich input to the system; feedback verifies the system is engaged.

The output of the architectural/AR system can assume either display or physical aspects.

• physical computing: the rearrangement of physical objects base on a user’s position
• overview/detail: details can be made to appear to come closer to user and be available for inspection
• transparent/opaque: walls can be selectively closed and opened
• “x ray vision”: one can creates display that appear to allow users to see into other rooms or into the city beyond
• virtual space: can be generated around a user as they move through space, guiding or circumscribing movement
• a heat map can show the locations of classes of objects
• cognitive maps can capture the interest of an individual that later guide a customized tour of the site
• explanatory text can appear at appropriate locations, and can become more detailed as one approaches
• virtual objects can appear in the space to connect other objects from the site or from a larger corpus)
• a direction path can be created to guide users

Our work has used Holoens technology but it applies more broadly to any form of AR. For, if AR aims to create new forms of interactions that engage both technological and social factors, design methods must be able to represent both the architectural setting and information interaction simultaneously. Otherwise it will remain separate from the life of the places it occupies.

REFERENCES

[1] Azuma, R. T. A survey of augmented reality. Presence: Teleoperators and virtual environments, 6, 4 (1997), 355-385.
[2] Barfield, W. Fundamentals of wearable computers and augmented reality. CRC Press, 2015.
[3] Bolter, J. D., Engberg, M. and MacIntyre, B. Media studies, mobile augmented reality, and interaction design. interactions, 20, 1 (2013), 36-45.
[4] Dourish, P. Where the action is: the foundations of embodied interaction. MIT press, 2004.
[5] Hevner, A. and Chatterjee, S. Design science research in information systems. Springer, 2010.