Real-Time place-based Storytelling + Service Learning

How can we use Unreal Engine to tell compelling place-based stories and build preferable futures through immersive and accessible online content? How can Architecture + XR create new kinds of monuments and memorials? How will the sharing of this tool and methodology with future generations of designers, storytellers, worldbuilders, and activists co-create new possibilities for our civic imagination and legal and policy frameworks? Using Unreal Engine, we will develop a model pedagogy for project-based service-learning that can be replicated at other educational organizations and institutions for immersive learning, storytelling, and activism. 

We kicked off the pilot inter-institutional course between the Johnny Carson Center for Emerging Media Arts and The School of Architecture this fall which brings together students in filmmaking, emerging media arts, design, and architecture for a collaborative project-based learning experience using Unreal Engine that is focused on the historic black-owned beach community in Seabreeze, NC. 

We are working to create a set of modules that ultimately we would like to record as tutorials to support future iterations of the course. The course is designed to deliver a multidisciplinary set of skills that blends traditional storytelling with emerging and immersive forms.

Current Workflows and workshops under development include: Intro to Unreal Engine, Worldbuilding workshop (when fact meets fiction) 360-degree video into Unreal, Spatialized Audio recording workshop on-site then brought into Unreal, Cesium into Unreal, Locative Media Workshop that uses GPS, Volumetric Capture workshop that considers drone and photogrammetry into Reality capture, Real-time Architectural Design and Visualization: architectural model in Rhino 3D and Blender exported to Twinmotion for real-time rendering within photogrammetric site data.

We are excited about future potentialities for this curriculum to integrate real-time data into Unreal as well as consider how blockchain technologies and NFT’s are integrating with physical land ownership and artistic prodcution.  We hope this curriculum is for anyone interested in shaping the future of the “phygital”, the hybrid realm where the digital and physical come together.

Menu

Team

 

We come to this project with backgrounds in journalism, filmmaking, architecture, emerging media arts and performance and aim to create 

dynamic curriculum that can be shared with other teachers and students around the globe.

 

Co-lead Instructor, The Johnny Carson School for Emerging Media Arts

Ash Eliza Smith, is from North Carolina and runs the Storytelling + Worldbuilding lab within The Johnny Carson Center for Emerging Media Arts, a new interdisciplinary program at the University of Nebraska-Lincoln. Its students and faculty use technology to innovate, to solve human-scale problems, to entertain audiences, and to tell breathtaking stories that stimulate, provoke and inspire. Ash has worked as a producer and director in the industry before joining the team at UNL.

 

Southern Devices

A place-based film shot in North Carolina, composed with aerial drone footage and edited with a machine learning algorithm. The film is a docu-fiction filmed in Smith’s hometown about data centers, mineral mining and the digital afterlife. short clip

OH HEY! HAYMARKET

In Fall 2021, while working remotely with students, we used a social VR platform, Mozilla Hubs, to reimagine a local landmark, The Haymarket.  For most of my students, this was the first time they had used social VR, but they all knew the local landmark. The platform became integral to our individual and collaborative writing and storytelling process through the ideation phase, design fiction prototyping phase, character phase, and ultimately doing a live-action role-play to test the story ideas and write further.

Artist-Designer Fellowship at Birch Aquarium at Scripps Institution of Oceanography

Can we think beyond human-centered design? I worked on answering these questions side-by-side scientists Here we also developed many applications in VR/AR and media design. I was also able to procure an Omni360 camera from a partnership that I created with GoPro. We began doing multiple experiments with the camera that challenged notions of understanding and protecting the planet.

The above is an image of a 3D scan of a kelp tank used to develop AR and VR applications and to create a curriculum from the research. 

Co-lead Instructor, The School of Architecture

Chris Lasch, is president of the School of Architecture, Founded by Frank Lloyd Wright. The School teaches and practices learning by doing, new ways of looking, an ability to honor and build with the landscape, and experimentation; above all, the School aims to serve its diverse communities by making its environment more sustainable, open to all, and beautiful. The School aims to change the world through architecture.

Shelter World: Online VR Exhibition Space

Shelter World is an online exhibition and social space to display and discuss student work associated with the TSOA’s 2021 student thesis design projects. Created in Mozilla Hubs, the environment includes mixes of the surrounding landscape, including student shelter construction sites, with 3D models and multi-media project presentation material. Created by TSOA faculty members Leah Wulffman, Stephanie Lin with Bika Rebek.

chris 3 LRG

Service-learning: Bio-Materials Design-Build, Arcosanti AZ

Design-build project to provide temporary housing for volunteers in Arcosanti’s Food Forest project. The building features an innovative mycelium (mushroom) insulation material prototyped in collaboration with Ecovative.

Student lead: Michele Yeeles, 2021

SOAT Presentation Image 2

Service-learning: Design Studio, Globe-Miami, AZ

School of Architecture Student-led Community Design Workshop, 2019, San Carlos Apache Reservation, Gila County, AZ

Student lead: Lorraine Etchell

Creative Technologist, The Carson School for Emerging Media Arts

Ben Kreimer is an artistic researcher and technologist-in-residence at the Carson Center. He has been recognized by Forbes 30-Under-30 for his international work developing enduring applications for immersive and other emerging and emerged media technologies through collaborations with journalists, scholars, students, artists, mission-driven organizations and businesses.

3D Virtual Reality From Photos Captured by a DJI Phantom 2 and Phantom 3 Drones

Aerial images and video of the Antiochia ad Cragum Archaeological Research Project, Turkey excavations on the Mediterranean coast near the small city of Gazipaşa.

Aerial Tour of the Amboseli Ecosystem

Produced in collaboration with the African Wildlife Foundation and Big Life Foundation, this aerial tour shows off the beautiful landscapes and wildlife found in the Amboseli Ecosystem of southern Kenya. Together, AWF and Big Life Foundation support and train community scouts who monitor and protect the area’s wildlife.

3D Reconstructing a UNESCO World Heritage Site: The Church of St. George in Lalibela, Ethiopia

The Church of St. George is located in Lalibela, a town in northern Ethiopia. The church dates back to the 12th or 13th century, and was built/carved from the top down, into the ground, from a single piece of rock. Visitors can also go inside the church. It is one of eleven rock-hewn churches making up the Lalibela UNESCO World Heritage Site.

Technical & Creative Consultant

Seth Winslow joins the team as a 3D artist and Unreal Engine developer working in the Arcosanti Archives. He will be integral to teaching and learning components across both campuses.

Arcosanti VR

Digital Reconstruction. A work in progress 3D recreation of Arcosanti Arizona for the Cosanti Foundation. Made with Unreal Engine.

Arcosanti Archives: Speculative Arcologies

VR exploration of unrealized designs by visionary architect Paolo Soleri. 3D models are created from archival drawings and embodied through online VR within a digital reconstruction of the existing Arcosanti landscape. Made with Unreal Engine.

Community Partner

Bob Wynn is a North Carolina native and the founder of Land Rich. This organization works on land loss issues and heirs property connected to historically African American family property throughout the Southeast. Land Rich helps low-wealth African American landowners navigate the legal and development process to realize the potential value of their heirs’ property and to begin building the kind of generational wealth that has been inaccessible to these vulnerable communities for decades.

Seabreeze Bop City

The focus of this year’s Course is Seabreeze, North Carolina, a beachside community first established by formerly enslaved Alexander and Charity Freeman in 1855. Overtime, Seabreeze grew to become a thriving beach resort and an extraordinary example of a Jim Crow-era community composed entirely of black-owned land and businesses, giving people of color a rare chance to celebrate and relax at the seashore during segregation.

Once an incubator of African American music and culture, the community is now fading. This is due in part to the end of segregation and natural disasters amplified by climate change; however, most notably, Seabreeze is one of many examples of the scourge of Black land loss across the Southeast. Seabreeze is in a prolonged struggle with legal difficulties stemming from heirs’ property issues, a legal status for property that results from the informal transfer of land from generation to generation, clouding ownership and making the property vulnerable to predatory speculation. In recent years, the community has lost hundreds of acres of beachfront property with very little compensation, representing a pattern that has repeated over and over in the US throughout the last century, limiting the ability of Black Americans to build generational wealth and contributing significantly to today’s racial wealth gap.

Over the next year, the team will work to create an innovative place-based extended reality documentary project that will reveal the remarkable history of Seabreeze while helping community members to document their historic claims to the land. In addition, the Team is helping the community to realize their vision for “Seabreeze Village,” a new neighborhood development that will include commercial and event facilities along with a collection of affordable tiny homes and a museum or interpretive center that will memorialize and communicate the community’s history while serving as a gateway to the larger augmented reality project and tour of the surrounding landscape.

PDF Project Presentation Link

Current Work and Prototypes

Pilot Project Preparation: Asset Gathering

In June, the team with a group of students traveled to Seabreeze NC to gather assets for our pilot project. The team conducted photogrammetric mapping and produced original video, images, and audio through site documentation and interviews in the Seabreeze community.

Site Documentation and Asset Gathering – Audio, Video and Interviews

                        The Barrel at Seabreeze                                                                                                                                              Billy and Jameel Freeman 

Initial Seabreeze Photogrammetric Mapping and Point Clouds in Reality Capture

                             Point cloud of a former Seabreeze juke joint                                                                                     Point Cloud of Seabreeze                                                                                                             3D Mesh of a former juke joint

Community Design Visualization in Twinmotion

 

Freeman Family Gathering Space Sketch modeled in Rhino 3D, experienced in Twinmotion (Play Video)