Games of Deletion
Alessio Grancini & Runze Zhang
“Games of Deletion” or GoD is an Augmented Reality game based on the use of virtual deletion in the field of design and architecture. The game is inspired by the semi-fictional novel “Games of Deletion” that will be released beside the game on the next 09.09.18 at Southern California Institute of Architecture in Los Angeles.
The game is supported by Android Google Pixel 2, iOS I phone X and in the future it could be adapted for some of the main AR headset as Holo Lens or Mira Prism.
GoD is a one VS one real scale design visual battle. The player A, located in a certain Location A plays against Player B in a certain location B. The game for being activated needs the Smartphone camera pointing at a Murals that has been previously associated with the game. In this way the game is constantly associated to real existing space.
GoD challenges the following themes in AR developments and Architecture.
– Vertical plane detections
– AR Cloud multi-player experience
– Player location exchange of information
– 6 Degree of Freedom
– Hologram resolution in daylight through further headset developments.
– Education in the field of Design and Architecture
– Collection of Data users
– Creation of a shared expandable AR platform
– Cinematic real scale augmentation
“A Game of Deletion” is a Real Scale Mixed Reality Game or RSMR based on additive and subtractive visual layer of information mapped on a determined urban environment. The first version of the game happen to be developed in the city of Los Angeles because of the large quantity of Murals. Murals in the city of Los Angeles have a different treatment from any other place. Indeed the DCA [ Department of Cultural Affairs reference ] regulates the mural’s artists through the Citywide Mural Program establishing a comprehensive network of mural activity and engagement by muralists, property owners, community stakeholders, educators, technicians, technologists, and preservationists.
Murals can be considered powerful Augmented Reality markers. A marker is any object that can be placed in a scene to provide a fixed point of reference of position or scale. In AR, these markers can provide an interface between the physical world and the augmented reality content, such as 3D models or videos. At their core these markers allow the device which is generating the AR content to calculate the position and orientation of its camera. When this calculation is done in real time, this process is known as tracking.
Game and Representation
Spatial representation through flat images brings to think about perspective as a human effort to perceive the space. Sarah Oppenheimer is a Visual artist based in New York that extends any kind of boundary dealing with perception of depth in space. Sarah connects different point of view, physically deleting partitions’s parts. Her design goes beyond any conventional perception of architectural construction. The artist’s design is a visionaire interpretation of new configuration in architecture programs.
Indeed Game of Deletion embraces Sarah’s vision in a larger urban scale. Also, the game tries to categorize and standardize deletion’s primary shapes.
Connection between spaces, is the main concern of every designer, when it comes to layout a living space, of any scale. The addition of a virtual information into this design process leaves room for many applications.
GOD is a Urban Scale S.Oppenheimer virtual installation. The difference between the artist work and the game itself is the wish to network the murals images between them. Images becomes portals and the game main speculation moves from augmentation to visual connection.
Nevertheless, The story and the game itself deal with the limits of the application of Augmented Reality in this current scenario, and let the reader absorbing the use of the technology for design purposes.
The narrative of the story reflects how the society is integrating a new media of visualization and imaging all the consequences of it.
The representation in the game is a mix between physical and virtual elements. The final picture we are going to experience is a virtually broken piece of reality that can be showing any kind of content. We defined deletion as communication and diminishing reality. Indirectly “Deletion” becomes a generative tool.
Architecture and Technology
Billions of dollars have been invested in XR between 2017 and 2018. The market between the main tech firms is not being defined yet and every year new products keep being testing what consumers need the most.
For examples, The Oculus headset was supposed to be the new smartphone device for everyone, with many features included into the headset. Unfortunately, the smartphone remains still one of the most technological inventions because its including endless applications in a very minimized envelope. Smartphones are still very hard to be beaten. Currently , Oculus like many other minor VR headset are being used mainly for gaming and particular design processes.
Headsets are dealing with our image and senses. Currently , looks like, we still can’t include a CPU and wireless connection into a comfortable pair of contact lenses as many movies speculates on. GOD is a reaction to the conventional use of this technology as a pure entertainment application.
A common scenario in architecture school is the comparison between new techniques and old techniques. Any architecture school complete cycle including undergraduate studies and master studies could take between 7-8 years.
Since the design process has been virtualized through the use of software, students are indirectly required to learn a large amount of different design platforms that can radically change during 7-8 years.
Between the most common uses we can name.
Sketch up, Rhinoceros and Grasshopper, Maya, Revit and Dynamo, Archicad, Autocad, Keyshot, Cinema 4D, 3Dmax, 3Dexperience, Unreal Engine, Unity, Processing, Monolith and many others for more specific purposes.
These are just some of the commonly used software that belongs to this period in the history, but any architecture student can confirm how fast these software evolve and how many skills have to be continuously improved for keeping up with the avantgarde design process. Undoubtedly, as the design process is changing, also real buildings will be changing in the future.
We often describe modern architecture as complex freeform shape or elaborate tectonic system, but not necessarily a complex smooth design means technologic design.
GOD story doesn’t argue the fact that technology will improve how buildings will be configured in the future, but at the same moment points out that the bigger change in design possibly will be no longer about aesthetic.
Technology could drive architecture aesthetic but in a very small part of it.
Patrick Schumacher in his “Parametricism” manifesto tried to associated a new taste in design with a parametric design process. Even if parametric approach is still today one the strongest point of Zaha Hadid architecture studio, his effort in defining a new aesthetic out of a software failed.
This phenomenon is affecting even different design fields. Nike is a fashion sport brand based on computational design and strong parametric process. However, during the last years Nike is changing its approach to the fashion design process through the implementation of departments focusing on design studios and conceptual phase in all its campus around the world.
The city of Los Angeles, is a perfect example proving an undefined aesthetic in architecture. The endless different cultures coexisting in one spread urban development show us how during the time we keep rejecting any imposition in terms of style.
We could run into a new real estate development showing a coherent aesthetic value for 5000 sq ft. After we crossed the development site, we’ll be seeing something else without any relation with the previous site. Building features are not necessary related to a defined period in the history, and the technology included in the building neither.
Architecture is a slow practice. Buildings are tangible element. They need to be designed, built and maintained. If we are hoping to see Tron legacy downtown in a close future, we are on the wrong path for understanding how technology will affect architecture in this era.
What prompted the project?
“Games of Deletions” is without any doubt a consequence of one-full year of research in Augmented Reality applied to “ARchitecture”.
There are three main chapters that brought this project to be what it is today.
My first contact with Augmented Reality happened unexpectedly. I was working in my summer-internship at Super78 in August 2017. Super78 is a small media-production studio based in Silverlake, Los Angeles.
The studio was focusing on the making a VR game development and some Design Development Drawings for the Walt Disney’s Birthplace. The CEO, Dina Benadon and the Creative Director, Brent Young, are part of the committee in charge of the house restoration in Chicago.
One day, Brent asked me to prepare a visualization for the day after, since the team was having a panel at Pixar studios for showcasing the Disney project. All we know that architects are good at deadlines, but… that amount of time was insanely short for maturing any kind of compelling representation of the project in front of such a demanding audience as the Pixar media department.
Some days before I downloaded, for fun, from the Apple store an App called “Augment”. “Augment” it is a basic AR platform that displays any kind of 3D model ; you just need to upload it on their server, making an account on their website.
I enjoyed animating game characters/any other thing for dropping them on the Augment App and visualizing it in AR on the table. Given the circumstances, I thought it would have been interesting showcasing a sort of “motion” model of the Disney project in progress.
It was a very small scale panel meeting between the studio, and Pixar. The last thing I wanted, was that the Pixar’s director taking a seat and keep watching a very long animation that documents how the project has been coming along until that moment. Designer’s client are not designer. Also, many things in the model were not complete enough for being proudly showcased to someone.
The good or bad thing about AR is that when you are experiencing any kind of visualization through its technology, the meaning of the showcased content shifts its meaning. In other words, I feel that we are still living now in a “wow effect” transition period with AR and VR where sometimes the content comes after, because the experiencing process takes over. This will be ending with the time, for sure when we will get used to it.
I see Augmented Reality as a double edged weapon. On one hand it is an impressive tool, on the other its utilization has so many limitation that seems to be reducing the quality of what we know how to do yet.
This sort of active visualization, is closer to us as human being. Experiencing directly the space through motion is a more familiar activity than visualizing a precise target image. [AR vs 2D render representation].
>Implementation of AR in Vertical studios at SCI-Arc
The Fall semester 2017, after my internship, I have been asked to TA in a Visual Studies Class taught by Devyn Weiser. Devyn is the most technology speculative instructor at SCI-Arc with Peter Testa. They both are passionate about distorting the conventional use of technology, in order to make new uses of it in the field of Design. Talking with Devyn about my internship experience, we started to collect information about visualization in the game industry, focusing on Augmented Reality as an emergent tool.
Devyn, during the summer asked me to collaborate at the preparation of the Visual Studies class and at the same moment she introduced me Peter Biggart, a thesis student that wanted to implement Augmented Reality in his project as a extended visual representation.
This scenario created the foundation for my initial learning process about AR.
Devyn, connected me with Runze Zhang, my current thesis partner. Runze, beside his high-level design capabilities, has a strong background in math and basics of Computer Science. This union will let us be considered with the time “Designers-Developers” around the SCI-Arc community, even if I personally think that for being considered a developer you need some more knowledge. We are working toward that direction on a daily base beside our daily design feed.
During Devyn’s Vertical Studio and Testa’s vertical studio we only spent time on studying how Augmented Reality can be or not be a game changer for architecture.
In “Physical Figures” by Peter Testa we treated AR not only as a process, but also as a result. The final Model was showing the super imposition of AR volumes on 3D printed Physical volumes arguing that if we actually do not consider that part of the architecture as tangible, we might be need just a visual information that can be provided virtually rather than physically. In this way, as architects we balanced contents between physical and digital information using the same scale for both the media.
In “Part to Play” taught by Devyn Weiser, we focused on how AR can be displayed as architectural information and if there is any kind of aesthetic imposed by the tool itself or not.
> Games of Deletion
After two semester, Me and Runze had an hard time figuring out what to do with all the information about Augmented Reality. One thing was certain. We wanted to bring the use of Augmented Reality at the next level within the Design field.
Using Vuforia, the most famous AR plugin for game engines now powered by Unity, we kept relying on a physical component that identifies a plane for augmenting any content we were pre-producing.
Right. AR primarily needs planes recognition. There are two ways for recognizing a plane. The first one is identifying an image 2D with a normal camera acting as a plane and the second one, more sophisticated, is tracking a plane through the use of a depth sensor that excludes the need of any image marker.
However, it is funny how a 2D physical flat Image is needed in order to activate a 3D virtual experience correlated to our world perception.
We thought to bring the AR experience to the the Real Scale through the use of Murals acting as huge markers. Los Angeles is the murals city for excellence.
We realized that someone else had this idea already, but we kept going since our goal was different from any other precedent we were taking in consideration.
Games of deletion is a basic framework for building an urban platform for connecting players and locations at the same moments. The name “deletion” becomes almost an overused terms, since the player experiences many values in the game beside the physical deletion of the building.
The connection between urban spaces through the portal is what makes the game being different from the conception of AR portal generally. Indeed, usually applications aims to bring the player somewhere else, where a fictional aspect enhance the player experience.
Games of deletions denies the creation of a new world, typical aspect of video games. The game relying on our world and promote the competition and cooperation of the players.
What drew you to explore and reflect upon the medium of augmented reality?
[ Runze ]:
We were exposed to augmented reality at a relatively good time. Although we spent most of our time solving technical problems, with the open source and advancing ecosystem, most of the problems were solved efficiently. As we became more and more skilled in C# programming and with all the architecture design knowledge, we slowly began to make more interesting scenes and mature applications.
SCIArc has a very good academic environment to discuss on and practice with digital tools, and California has naturally provided a good ecosystem of science and technology. Many of SCIArc studios and electives are keen to explore the possibilities brought about by advanced digital tools in architectural design. We are provided a very large platform to do all these experimentation. In California, there are many activities such as AWE, XRLA, allowing people communicate and exchange ideas constantly.
It is very important that creating AR scenes based on real scale involves many architecture discourse, and in fact, interaction design of AR scenes requires stronger three-dimensional imagination. For example, where does the user look from, what is the spatial configuration of the real scene and the virtual scene, how the depth mask of the real scene is arranged, what is the render order of the camera, what is the blocking relationship between the objects, etc. Many of these problems are often encountered during the architectural design process as well. And considering the application of AR at the actual city scale, for instance, how to provide a safe and stable place for people to experience AR in the real world, how to take the use of public space, what is the relation to the behavior of and space for user, audience and non-user, etc., they have become problems of architecture derived from AR. This allows us to broaden the research and discussion of architecture and AR. It also brings us more interest in AR.
In addition, I have always thought that there is much fun in architecture design that is worthy of public engagement and recognition. Therefore, I think that as an architecture student, I feel it is a good opportunity to let the public understand more about the art, politics, philosophies, and other values out of architectural design. Augmented reality is, in effect, capable of this intervention., because people are willing to be and are naturally exposed to these 3D experiences and can actively think and be ready to contribute their creativity out of immersive interactions.
How and to what extent does designing with AR/VR have the possibility to revolutionize how we conceive and shape space? [AR/VR?]
Some days ago, me and my thesis partner have been invited at Locate Mapbox in San Francisco. During the conference there has been a panel about the use of AR in social platform with Adam Debreczeni, AR director at Mapbox, Jon Lax, head of AR/VR Design from Facebook and Timoni West, director of XR research at Unity.
It’s been the first time I have heard someone mentioning specifically “Designers and architect are called to rethink every virtual platform contents in relation with the real space”. The statement by Jon Lax underlined that with the use AR/VR the word “3D” now tastes different.
Someone could think that there is no difference between a modeling software as Rhinoceros and an immersive modeling platform as Oculus Medium. I do believe that the significant shift is in our personal scale as creators in relation with the object. Both AR and VR impress us for the immersive experience due to the “immersive scale” of the modeling environment or simply the model itself around us.
AR particularly can influence us in four different ways.
– Through our design process
AR can be a great tool for let us see something we didn’t see “in the box”. For example, In our specific case we used AR as photogrammetry tool. Virtual objects do not need any physics system. We took advantage of this feature letting some architectural objects floating in front of the camera and obtaining a perfect spherical capture. It would be impossible with physical objects.
– Through our design result
AR can be considered as Virtual Property in our real world. At AWE in San Francisco some days ago, we found a startup called “YouAR” from Portland . YouAR is trying to organize a sort of AR property content through channels [as You Tube] improving GPS localization of content and flattening the difference between AR media as Vuforia, ARKit, ARCore, Wikitude etc.
We passed the last year talking about this, and we have been not surprised in finding several startups going in this precise directions. In other words, cities are empty canvas for any kind of virtual content. AR now is a decentralize system. The AR cloud doesn’t exist at all and it will struggle to be accepted in the future. There will be many diversified offers, and the more successful one will take over and being monetized, as today we daily reserve a large amount of our time living through our socials.
So, if AR is experience, AR is also architecture. In Peter Testa Vertical Studio we tried to create a conceptual model that is 50% Physical and 50% AR oriented.
The physical architecture side of the model works on its own but predisposes spaces for augmentation.
– Through our design representation
AR can be used as a pure representation tool. Believe it or not, the every design that is displayed in AR will be strongly influenced by this media.
In fact AR, is still a “in progress” technology, and requires some level of knowledge for being used. Especially, AR representation are often associated to devices as IPad, IPhone, Smartphones in general.
The performance threshold of our device will be influencing our visualization, and with it also the final idea of the project.
However, there are valid optimization processes bring to amazing result beyond every limit.
– All the previous four together
If we collect the three previous approaches of AR, we could argue that indirectly there is an existing AR oriented model in the design process. This is exciting for me, and motivate me every day in looking for something more relevant to include in this analysis.
Also, it is interesting to see how architectural software evolve and start merging with Computer Science focuses and Game Engines platforms.
Eventually Augmented Reality will be the media to connect our GPS superimposition on the real world with our physical first person experience. In fact, the overlaying of virtual information in our surrounding generates more issues than before we were not experiencing.
The most important part of Augmented Reality is the stability of the tracking. [ What I see shouldn’t be moving ] Not many companies are able to provide a perfectly stable tracking, and usually when they do, they are relying on a very expensive and powerful hardware.
Should we design only in terms of experience through sight or should we also think of the other senses? To what extent is virtual reality lacking behind on this? What defined the absence of sound in your video?
[ Alessio ]
I would start answering to this question starting from commenting the difference between AR and VR. Generally, Augmented Reality is the overlaying of Virtual information on our physical world, meanwhile Virtual Reality deals with pure isolation from the real physical world providing an new understanding of the context.
They both belong to the XR categories and with the time probably they will be less and less different, since a total overlaying of AR info, is a sort of VR matching our physical environment.
In both cases, we are fully immersed in the experience. Sight is not the only sense involved yet. However, all we know that we can’t touch or smell the projected content. This limitation leaves room for any kind of improvement, but personally I think there are two main ways to “solve” the problem.
The first solution is human-oriented and the second one is machine-oriented. The human oriented solution puts the content outside of any headset and tries to build devices for sharing a content without the use of any bridge device. For example, 3D motion screens are something existing since many years and I still don’t understand why at the theatre they don’t want to upgrade their screens yet and we are always required to use glasses supporting the projected 3D format video.
MIT media lab built in 2016 “Materiable”, a sort of holo-physical reactor that aim to prove a connection between physical and digital information without any in-between.
The machine oriented model implies a bridge device, as headset or in the future glasses or lenses. And here everything gets easier, more artificial and marketable.
VR vibration Suits, eventual “smells-plugin” for it and all what comes after for any other specific needs.
So, generally my answer is that we absolutely should think of other senses beside the faculty of seeing. We also should be very clear about what kind of solution-model we would like to pursue. be very clear about
Virtual Reality is potentially a very complete tool for designers, but it is still putting us at the center of the platform. I see the future of design somewhere else, where we are cooperating with the tool rather than fully controlling it.
What are/is for you the most ‘important’ software available to you as a student of architecture? how has/have these helped you shape and challenge your approach to the discipline?
[ Runze ]
In the beginning we used Unity, the game development platform, with Vuforia to drive AR. At this stage, we tried to find out whether AR can bring useful contingencies to our design. However, we encountered many bottlenecks in this process because there are many restrictions on the Vuforia version provided for developers, such as the inability to fully activate the full resolution of the camera. The result of these technical problems is that during our semester reviews, there are always questionings, such as “Your live demonstration is very interesting, but if you cannot achieve the same quality as what we can get by the use of existing modeling software and traditional imagery, then your particularity and technicality are less convincing”, ”The modeling apparatus you build with AR is interesting, but visual and perceptual experience with these mixed reality varies from person to person. Therefore, even if you are really interested in the results of your design through the process of human perception, this process cannot be objectively perceived by outsiders through your final demonstration, so as not to conduct objective and effective conversations”, etc.
Therefore, our reflection on the first stage is that AR is, from a role point of view, an experience and a medium for interface more than a tool. For the problems that AR can solve, traditional computer technology and mobile technology can tackle as well. If the representation of the technology itself is higher than the intuitive visual experience it brings, then the “augmented” in the AR seems unconvincing. It can be seen that the fun of the AR experience is an important thing that can reduce the cost of promotion and explanation. Therefore, how to make people experience the uniqueness of experience becomes a problem that must be solved first.
Therefore, in the second phase, we divided our focus into two directions: One is still relying on Unity and various plug-ins such as ARcore, ARkit, Vuforia, etc. for exploration and experimentation. The other is through some powerful software with video tracking technology such as Cinema4D to enhance people’s perception of what we ultimately want to present to promote communication during the review. The smaller the gap between the two, the closer we are to the ultimate goal. And at this stage, we put the emphasis on the AR game because we think the fun of the game is a good breakthrough and starting point for letting more people get involved in the conversation.
Where do you see the dangers of a time where software and hardware is continuously evolving and developing? how can we collect and archive contemporary materials and reflections on architecture practice?
[ Alessio ]
I have never seen dangers in using software. I have rather seen fear around designers communities.
I am a graduate student and my academic career will be officially ending in the next september 2018. At the same moment, I feel like I need to restart a design program because of the new tools coming up.
Design, as many other professions requires a insatiable wish to learn and avoiding to be out the game. I do believe, that good designers can use any sort of tool for generating design but at the same moment it would be hypocrite thinking that the tool is not affecting you as designer.
However, how could I deal with a 16 years old art school student that knows how to manage Kuka’s robot and has a solid understanding of C+ ?
The designer’s upgrade needs to be targeting our needs first as thinkers. If I am a licensed architect aiming primarily to “build stuff”, make sense that I get into any kind of major BIM technology. If I am a designer in charge of designing installations, maybe some parametric/interactive tool would make my work more interesting and competitive.
For visualization designer, interactive designers will be the same process as long as it is suiting their needs.
Archiving is very important. I love documenting my work in every format, from physical model, writings to files, I always overdo this passage.
I think archiving defines you as a person more than a good worker. If we take photos to remember, we archive for understanding more of ourselves.
Now as never, in this new XR era, archiving and data administration is for sure the most important factor influencing our daily life.
For instance, Gensler is the most productive and profitable architecture firm in the world with hundreds and hundreds of project in more than 16 different countries.
If we need to make a 3D accurate world model for properly starting this XR overlaying of information, Gensler, with many other architectural firms would be between the primary sources. Google 3D maps is unfortunately not enough when it comes to 1 to 1 scale and specifically architects are needed.
If architecture is a multi-disciplinary subject as it claims to be, I do believe the future of the archiving is in the scientific appropriation of any other inspirational tool. With this I mean that the profession of the architects in the future, will be highly diversified, because our background could be not necessarily only dealing with design but also with medicine, computer science, physics, math or other correlated subjects.
What is your take on the analogue site and context of the paper and pen?
To me it is a take on the difference between tooling and how to test tools. If digital tools help designers to finish drawings, images better, accurately express what they could not express through pens and papers, or improve their efficiency, then it is naturally good. But when you want some non-existing design goals and try to use and test the potential and application prospects of a certain tool, it becomes an experiment, then the method becomes very important, otherwise it will be a cart before the horse.
This is also true for AR technology. For example, when we try to use AR to do something for architectural design, our purpose becomes to merely test and use the tool. The hierarchy of the tool is higher than any architectural core thinking. Without self-awareness of this different nature, blindness and lack of diversity in the operation process can result, leading to inefficiency of trial and error without any useful contingency. In the process of discussing with Peter Testa, we discovered that since the innovation of the tool is inevitable, and it is impractical for people in the post-digital era to completely avoid involvement of the digital in thinking and practicing, then in order to enable architectural design elements to all participate in the project effectively, it is necessary to look at each element of the design process in a parallel or looping manner. In this way, many interesting combinations will be discovered and interesting discussions will be held. So when we move from the linear thinking as a design tool to the establishment of a game mechanic, all the necessary elements can act in the same direction, which in turn stimulates more architectural problems, such as the urban image and urban site, for example, where does the game take place, whether real-time architectural scenes created by people in urban AR games are of practical significance, how to design game methods so that users can intuitively experience architectural design processes with fun, and so on so forth.
In general, I believe in the post-digital, simply distinguishing between the digital and the physical cannot easily help us find innovative breakthrough and effective representation any more. Instead, we need to be aware of our roles in the process and how to choose the right thing at the moment.