Abstract

This thesis explores the potential for creation of three-dimensional, parametric computer aided design (CAD) models using augmented reality technology. The current standard – a flat screen – can only display two dimensions of information, which constrains the display of three-dimensional geometry as either perspective, parallel, or orthographic projection. Each projection type has trade-offs, resulting in distortion, inaccurate scale, or the need of multiple planar projections to capture three-dimensional space. Augmented reality technology overcomes the limitations of the flat screen – for the first time, holographic, truly three-dimensional display of geometry is possible and can be integrated into a real scene by using a head-mounted AR/Mixed Reality headset, affording a CAD display that imparts better understanding and accuracy during the entirety of the product development process. Using current devices and adapted practices, I developed a prototype mixed reality solid modeling environment to simulate the experience of using such a tool, and test how we might interact with it. This thesis documentation reflects the process of creating the interactive demonstration and reflects on the lessons and feedback gained from the students, enthusiasts, and industry professionals who engaged with it.

In short, what if we could use Augmented Reality technology to go beyond merely inspecting 3-D models as holograms and actually create professional-grade, solid models in mixed reality with the precision we’ve come to expect from current CAD tools?

To explore and study this idea, I created an interactive HoloLens demonstration of what that system might feel like.

Process:

Problem Definition ⭢ Background Research ⭢ User Information Gathering ⭢ Systems Information Gathering ⭢ Synthesis ⭢ User Testing ⭢ Video Documentation ⭢ Written Documentation

Underlying Problem

Our current flat screens do a disservice to our creative process in representing three-dimensional data on two-dimensional screens.

Mixed Reality (the entire spectrum of virtual and augmented reality) is here now and is typically powered by 3-D models that were originally designed on screen-based CAD systems and ported over to synthetic environments. While experiencing three-dimensional CAD models in these Mixed Reality environments we are afforded a much more natural understanding of their attributes because they are presented in a way that is similar to physical objects in the real world.

Opportunity

My hypothesis was that, for the sake of creation:

“Designers who develop three-dimensional products in simulated environments could instead use mixed reality to better understand and interact with their creations—experiencing virtual designs blended into real environments at full scale and with realistic perspective change relative to gaze and point of view.”

-from ARCAD Thesis, 2018

1. History

Digital drafting has existed since Ivan Sutherland created the first interactive “graphical communication system” Sketchpad as part of his PhD work in 1963.

We have since pushed this technology to move from flat, blueprint-like drawings to three-dimensional models with increasingly high resolution and precision.

Beyond single models, we’ve since created exacting and uncanny environmental models in which to immerse ourselves.

Currently though, most all these models and environments are first created on a 2-D screen, not all that dissimilar from the Sketchpad of 1963.

Creating these models is an integral part of the daily lives of people in the design and entertainment industries now. So much so, there was a large pool of users from which to learn during this formative period, many with insightful and interesting opinions.

2. Users: Student and Professional

In studying the potential end users, I observed and interviewed students in an undergraduate 3-D modeling class. I analyzed their curriculum, watched as they struggled to overcome the barriers of the screen, and discussed their frustrations and desires.

“Gah, this view is confusing. Why can’t I click there?!”

“I wish I knew which tool could do that. It’s so easy in real life.”

I interviewed faculty in both Industrial Design and Interaction Design at the University of Washington.

“Sometimes, in certain programs, we just get lost. Which way is up?”

“A 3-D model affords infinite views, but in AR you become the camera. You could walk around the model instead of spinning the environment.”

I met with top-tier professionals in the fields of Engineering, Industrial Design and Interaction Design; discussing frustrations and perceived limitations of 3-D modeling and VR sketching, as well as brainstorming applications for parametric modeling in Mixed Reality.

“I’d love to see how a CAD model looks in an actual environment. Context for our work is so important and you can never really know until you’re done.”

“There are projects I’ve worked on that don’t allow for full-scale models. I don’t really get to see those until 6 months after all the work is done.”

“[Prototyping in AR] is absolutely where we’re going. There’s no doubt.”

As informed by these interviews and discussions, I created three guiding, representational personas as the target users for an ARCAD environment:

3. Survey of Existing Solutions

In the existing 3-D modeling programs, I studied their workflows, iconography, visual attributes, signifiers, and features. There are many overlaps in their primary goals but to illustrate some of their essential strengths and weaknesses, a competitive analysis was performed.

Synthesis

Through the research and observation, I distilled down some of the essential features that an AR CAD system should have. To effectively demonstrate the ways in which this system might behave, I developed a workflow from which to build.

Reception

Over the course of its construction and refinement, the ARCAD system was experienced by several dozen people from a wide range of backgrounds. For many, this was their first foray into an augmented reality experience, so wonderment at the AR illusion was typically the first reaction. As the novelty wore off, any confusion and awkwardness made it clear how important the interactions were. To quote one professor “the model is intuitive… engaging with it is the trick.”

Each end of the spectrum of mixed reality represents its own distinct opportunities. For one industrial design professor, trying to use the system in his office was wholly distracting. He could see all of the elements in AR, but also all of the books in the physical world behind it. He thought that he might like it better in VR where he could focus on the task at hand. The owner of an exhibit design company in Seattle Washington felt the same. He sculpts entire exhibits using MasterpieceVR and will 3-D print them directly from the files created in VR. He said he “rather likes putting on a headset and headphones and blocking everything out, so he can work.”

After experiencing ARCAD, a professional industrial designer wished he had had this a few months ago, explaining that he was designing a crane truck that was so large that there was no way to physically model it. He spent months heavily focused on designing a form and had resigned himself to the fact that building a model at full scale, while possible, was quite impractical. He imagined bringing the system to a parking lot next to a building and being able to position and extend the crane arm, adjusting his parameters to get the assembly’s geometry perfect.

An undergraduate student learning CAD as part of her Industrial Design training was excited to try “a different way of doing it, because the screens seem so small.” While in the system, she said “wow, this just makes more sense. I don’t like the gestures, but this could definitely be a thing.”

A motorcycle builder stared at the rear suspension for some time during a demonstration. When I asked what he was looking at he said that he wished it was a fully functioning assembly with mates, as in SolidWorks, so that “he could adjust some of the rear suspension geometry and test the swingarm travel. Or maybe raise the handle bars and check the seating position.”

A cohort of mine, who had not yet experienced AR, tested ARCAD during the first version. She approached it with trepidation, but her eyes widened when I started the demonstration. She walked from one end of the room to the other trying to manipulate the models. “I get it now; this hype makes sense. It could change everything”

Professional Interaction Designers were very helpful, too. One searched around for the usual spatial indicators from a CAD program and exclaimed “Oh, the orientation signifiers are moot, it is natural!” A design director from a Seattle-based consultancy helped by finding a few behavioral and graphic elements he would recommend modifying, but not before throwing his leg over the holographic motorcycle and looking down at his hands to check the span of the handlebars.

I presented the system to a design team from a Seattle consultancy who had not yet included mixed reality in their process but thought that they might in the future. While demonstrating, their President – also an active designer – excitedly imagined the prospect of being able to create a product in the context of the environment in which it will eventually fit; like designing a medical device in a real hospital room. Building on that, another designer imagined being able to accurately arrange holographic versions of internal componentry within an existing physical enclosure; or conversely, testing multiple holographic enclosures around an existing internal structure. Was there a future for designing with this technology? “No doubt” he said.

Documentation

After months spent researching and creating the demonstration, many of the assets from the Unity scene were ported over to After Effects with Element 3D and Cinema 4D and layered atop external footage of me operating the demonstration. This footage was combined with on-board footage recorded by the HoloLens and edited in Premiere to create the video that appears at the top of this page.

Finally, the thesis documentation was written to compile and publish the various lessons gleaned from the research done. My thesis chair requested a document that would provide an introduction and head start to students interested in Augmented Reality prototyping; so that is what was written. This thesis taught me a lot about translating professional-caliber CAD models into holograms, working with new input methods, and creating interactions for a new spacial-computing approach. An extended description of the background, processes, and research findings can be read in my ARCAD thesis documentation. For a copy of the entire document, contact me through the ‘About’ button below.

Conclusion

Computer aided design has long been subjected to the inherent hindrances of the technology on which it is viewed. The designers working in these simulated two-dimensional environments are incapable of truly designing in a real-world context, instead, employing a familiar but flawed flat screen to achieve their goals. To quote James Gibson “The very notion of an image as a flattened-out object, a sort of pancake of solid body, is … misleading” (Gibson, 2015). But without an effective alternative, CAD has flourished despite the visual, and thus experiential, incongruities.

The Augmented Reality Computer Aided Design system is a response to these drawbacks and offers an alternative method for designing through a new visual modality in computing. Ivan Sutherland’s Sketchpad was introduced as a way of “talking to the computer using a graphical language,” a language now spoken daily by designers. New mixed reality tools present the potential to elevate the dialogue between designer and computer, and by extension the designs being created with those systems. This thesis work demonstrates a potential form upon which this new dialogue could be conceived of and built.

To achieve this, I thoroughly surveyed precedents upon which our current tools were built, the functional operation that resulted in the various approaches to modeling, the current generation of screen-based CAD systems common to industry, and the new generation of virtual reality modeling. From there, I created a workflow for importing professional-grade CAD models into the Unity game engine, and open-source processes from the augmented reality development community were used to experience models through a HoloLens. Next, based on a cross-pollination of best practices in virtual reality coupled with teachings from the current fields of design, an interactive workflow was built to simulate a CAD design environment in augmented reality. This demonstration was shown to a variety of interested parties, the results of which I incorporated and documented into the overall design.

I designed and built the ARCAD system to be an incremental improvement to the digital tools that facilitate 3-D creation now, brought to a logical realm of mixed reality. Tools are usually created for one of two reasons; as response to a need in a process, or as a technology matures enough to uncover a use not yet realized. Technological tools, like the computer, can have a plethora of applications and can grow to be an indispensable part of a practice, like that of design. As in the transition to digital drafting, evolutionary tools can come to define a time frame’s identity because they assimilate into the process of which they are applied, drastically affecting the results. As a tool that can more accurately communicate form in an intuitive way, 3-D modeling in a truly 3-D environment heightens a designer’s cognition by adding visual cues for dynamic spatial perception – a significant addition to the visual representation of spatial geometry as a naturalistic 3-D display that could radically change the design process for those who create content in three dimensions.

Many thanks to Ronin Motorworks for providing the motorcycle CAD file.