What is photogrammetry?
Photogrammetry involves extracting three-dimensional (3D) information from photographs. This technique encompasses capturing overlapping images of an object, person, structure, or space and converting them into 3D digital models. Aerial and close-range photogrammetry represent the two primary categories within this field.
Aerial photogrammetry entails utilising aircraft to capture aerial photographs that can subse- quently be transformed into 3D models or digitally mapped. Today, this process can also be accomplished using drones, which have significantly simplified the safe acquisition of data from challenging-to-reach or hazardous locations, where conventional surveying methods may be impractical or perilous.
Close-range photogrammetry, on the other hand, involves capturing images using a hand-held camera or a camera affixed to a tripod. The objective of this approach is not to generate topographic maps but rather to create 3D models of relatively smaller objects. The images are usually captured sequentially.
DSLR array scan, captures a object or person by taking multiple photo- graphs at once. An array of lights and cameras are triggered simultaneously, giving in a quick capture and consistent results.
What are the potential use cases for theatre companies?
Photogrammetry offers numerous commercial and archiving uses for museums, galleries and Arts Institutions such as the RSC. Here are some potential applications demonstrating how photogrammetry can revolutionise the way institutions engage with audiences, preserve cultural heritage, and facilitate research and education in the digital age.:
Virtual Exhibitions: Photogrammetry enables the creation of immersive virtual exhibitions. Visitors can explore digital replicas of artworks and artefacts, providing access to a broader audience worldwide. This technology enhances outreach, education, and engagement, espe- cially for individuals who cannot physically visit the museum or gallery.
Online Collections: Museums can utilise photogrammetry to digitize their entire collection and create online databases. This allows users to browse and search through the collection remote- ly, providing a comprehensive digital archive for researchers, students, and enthusiasts.
Conservation and Restoration: Photogrammetry can aid in the documentation, conserva- tion, and restoration of artworks. High-resolution 3D models can capture fine details, which can be studied to better understand the piece, plan restoration processes, and track changes over time.
Reproduction and Merchandising: Photogrammetry can be used to create accurate replicas of artworks and artefacts for reproduction and merchandising purposes. These replicas can be sold in museum shops or used for educational purposes, enabling broader dissemination of cultural heritage.
Archival Preservation: 3D models created through photogrammetry can serve as archival records of valuable objects. In the event of damage, loss, or destruction, these models can act as references for restoration or reconstruction efforts.
Interactive Installations: Photogrammetry can support the creation of interactive installations within museums and galleries. Visitors can manipulate and explore 3D models through touch screens or virtual reality (VR) experiences, enhancing the overall visitor experience and offer- ing new perspectives on artworks and artefacts.
Research and Education: Photogrammetry enables detailed analysis and research on cultural objects without physically handling them. Scholars and researchers can study objects remotely, examining their intricate details, materials, and structures. Additionally, educational institutions can utilise 3D models for teaching purposes, providing students with hands-on experiences even when the objects are not physically accessible.
Documentation and Cataloguing: Photogrammetry simplifies the process of documenting and cataloguing objects in a museum or gallery. Instead of traditional manual methods, the creation of 3D models can streamline the inventory management process and improve the accuracy and efficiency of record-keeping.
Selling 3D Models on Game Marketplaces: Photogrammetry-generated 3D models have great potential for commercialisation in the gaming industry. Game developers and designers are constantly seeking high-quality assets to enhance their virtual worlds and provide realistic experi- ences to players. By selling 3D models on game marketplaces like Epic’s Unreal Engine Market- place, artists and museums can tap into this demand and monetize their digital replicas.
Asset Creation for Games: Photogrammetry allows for the creation of highly detailed and accurate 3D models that can be used as assets in video games. These models can be textures, characters, objects, or environments, providing game developers with ready-to-use content that saves them time and resources in the asset creation pipeline.
Licensing Opportunities: Museums and galleries can enter into licensing agreements with game developers, granting them the rights to use their 3D models in commercial game projects. This presents a unique revenue stream for cultural institutions, where they can earn royalties based on the sales and usage of their licensed assets.
Collaborations with Game Studios: Cultural institutions can also explore collaborations with game studios to create virtual experiences or games that incorporate their collections. These collaborations can result in innovative and immersive projects that blend art, history, and gaming, reaching a broader audience and generating new revenue streams.
Integration with Virtual Reality (VR): With the rise of virtual reality, the demand for realistic and immersive experiences has increased. Photogrammetry-generated 3D models can be integrated into VR applications and games, offering users the opportunity to explore virtual museums and interact with digital replicas of historical artifacts and artworks. This can open up partnerships with VR developers and platforms, creating new avenues for revenue generation.
Selling 3D models on game marketplaces not only provides financial benefits but also promotes the cultural heritage represented by the objects. By incorporating these models into video games and virtual experiences, museums and galleries can engage younger audiences and introduce them to art and history in an interactive and entertaining manner.
Process
The technical pipeline of photogrammetry typically involves several steps:
Image Acquisition:
Capture: The first step in photogrammetry is to capture a series of photographs of the object or scene from different angles. These images should cover the entire object or scene from various viewpoints, ensuring proper overlap between images.
Image Preprocessing and Orientation: Before the images can be used for photogrammetry, they often require preprocessing to enhance their quality and remove any distortions or artefacts. This may involve tasks such as lens correction, noise reduction, colour balancing, and image alignment.
Processing (Automated in software):
Feature Extraction: Once the images are properly oriented, distinctive features or keypoints are identi- fied in each image. These features can be corners, edges, or other identifiable points that can be reliably tracked across multiple images. The extraction of features is important for establishing correspondences between images.
Image Matching: In this step, the extracted features from different images are matched to find correspondences. The goal is to identify which key points in one image correspond to the same physical point or feature in another image. These matches are used to establish the relationships between different images and further refine the 3D reconstruction.
Dense Point Cloud Generation: Using the correspondences established in the previous step, a dense point cloud is generated. This point cloud represents a set of 3D points in the scene, with each point having corresponding 2D image coordinates in multiple images. There are various algorithms and techniques for generating dense point clouds, such as multi-view stereo algorithms.
Surface Reconstruction: Once the dense point cloud is generated, a surface reconstruction algorithm is applied to create a continuous 3D surface model. This can be done using techniques like triangulation or volumetric reconstruction. The result is a digital representation of the object’s surface, often represented as a triangular mesh.
Texture Mapping: To enhance the visual appearance of the 3D model, texture mapping is performed. The original input images are projected onto the reconstructed surface, assigning colour information to the corresponding 3D points. This process adds realistic texture and details to the model.
Processing (Manual):
Model Refinement: After the initial reconstruction, various refinement techniques can be applied to improve the accuracy and quality of the 3D model. This may involve processes like noise reduction, hole filling, mesh smoothing, or texture optimization.
Output: The final output of the photogrammetry pipeline is a complete and accurate 3D model of the object or scene. This model can be exported in various file formats, such as OBJ, FBX, or USDz, and can be used for visualisation, analysis, virtual reality, or other applications.
It’s important to note that the specific techniques and algorithms used in each step may vary depending on the software or photogrammetry pipeline being employed. Additionally, the complexity and accuracy of the pipeline can vary depending on the desired output and the quality of the input images.
File formatting
Using the USDz file system, we are able to attach corresponding metadata to each object. This enables us to curate a more holistic narrative around the object, centring craft and creation to bring the object alive. When packaged as USDz, the metadata becomes available to users as they explore the object. This metadata can include:
- Collection, storage, conservation and museum item referencing (via MODES)
- Artist & craft record – designer, maker, team, process
- Live performance media & supporting media (process video, BTS, audio, photography) * Curatorial, EDJI & relevant social context tags
| USE CASE EXAMPLES RESOURCES OUTCOMES & REQUIREMENTS | |||
| Artistic & Creative: Hybrid Productions Digitally Originated projects Technology partnerships Community creative & Placemaking | Transposing physical props, costume, set from stage to digital platforms AR/mapped overlay or accompanying standalone digital product Game, VR/AR products, installation, online. Outreach projects led by CPPP team | * Scaled Lightbox * Camera rig * Capture team (in house) * Post processing (in house) * Application development or device (commission or partnership) inc UX to access metadata Access to objects post build, pre stage production use (2-6 hrs per object) | Shared aesthetic, look and feel between live perfor- mance and digital offer. Potential of digital overlay during performance Use within production video to map / augment physical objects Extends audience engage- ment in performance beyond the stage. Transposing RSC physical craft skills into our digital products, including design, technical build/making. |
| RSC Collection | 3D capture of objects as they enter the collection Enabling digital loans to other museum/institutions Digital preservation of items otherwise unable to be inducted into collection | * Rig install at Arden Street * Training of collections teams to capture * Post processing pipeline & clean up (in house) * Collation of metadata (DD Archive Administrator) * Licence review – including IP, rights (redraft- ing of collection ownership model & contracts) | Enabling use of delicate objects, otherwise unable to leave the collection stores Opening access to the collection, sharing our assets with other museums, partners and academic institutions |
| RSC commercial & Brand | Licensing sets of objects and skins/wearables as inventory for third party platforms (such as in game, in metaverse) Minting of NFT versions of collec- tion objects for sale. | * License agreement with owners of objects, poten- tial renegotiation with designers for commercial use * Commercial partnership with platform owners / developers * Legal framework for use & to negotiate terms/in- come | Brand reach to game/digital audiences Marketing activations for major titles/commercial products Income generation (via marketplaces) Individual Giving (HNW individuals) |