Introduction

Trying to get started with AWS IoT TwinMaker and need to convert your OBJ file to glTF? Perhaps you have performed a point cloud scan of your environment with Matterport and it’s not clear how you can import the Matterpak bundle into AWS IoT TwinMaker. In this blog, you will apply a model conversion pipeline to compress and convert a Matterpak bundle into glTF format. This approach will provide up to date 3D models and improved scene load times in AWS IoT TwinMaker.

Overview

In this blog post, several file extensions and model formats are referenced. Before getting started, it is good to understand the following:

• OBJ – Object file, a standard 3D image format that can be exported and opened by various 3D image editing programs.
• MTL – Material library file, contains one or more material definitions, each of which includes the color, texture, and reflection map of individual materials for objects in an OBJ model
• glTF – Graphics Language Transmission Format, a standard file format for three-dimensional scenes and models. A glTF file uses one of two possible file extensions: .gltf or .glb
• Draco Compression – A glTF extension for mesh compression. This Cesium library will compress and decompress 3D meshes to significantly reduce the size of 3D content. It compresses vertex positions, normals, colors, texture coordinates, and any other generic vertex attributes, improving the efficiency and speed of transmitting 3D content over the web.
• Point Cloud Scans – A large collection of individual points (x, y, z coordinates) within a 3D space, captured using a 3D laser scanner and stored in ASCII (.xyz) or binary format.

AWS IoT TwinMaker supports 3D assets in the glTF format, which is a 3D file format that stores 3D model information in JSON format or binary and enables efficient transmission and loading of 3D models in applications. The glTF format minimizes the size of 3D assets and the runtime processing needed to unpack and use them. The 3D models from traditional CAD applications, as well as point cloud scans, can be converted to glTF using AWS Partner solutions, such as those from Pixyz. In this blog, you will explore an alternative server-less approach to model conversion of Matterpak bundles to glTF using open source libraries by Cesium, including obj2gltf and gltf-pipeline.

In the architecture below, you will see how AWS Lambda can be used to detect a Matterpak zip bundle uploaded to an Amazon S3 bucket. This will trigger the conversion to glTF within a long running Lambda execution. The zipped file may contain OBJ, MTL, and JPG files.

Within a Matterpak bundle, there are several files including an OBJ, MTL, point cloud scan (xyz), and possibly many JPG files. Matterport in this example has converted the point cloud scan to an object mesh format, OBJ. The MTL and JPG files together provides colored texturing over the objects within the OBJ model. The xyz file will not be used in this conversion process as this has already been converted to OBJ in the Matterpak.

Model Conversion Pipeline Architecture

When working with point cloud scans such as Matterport, high resolution JPG textures are captured throughout the scan. Doing a simple conversion of the OBJ to glTF will still be quite large. To improve this, the Lambda function in this blog will first compress all JPG images prior to converting to glTF. In addition, this model will be compressed even further by using Draco Compression. As a result, the conversion will produce a much smaller glTF file as seen in this AWS IoT TwinMaker Scene below. Note, a glTF file uses one of two possible file extensions: .gltf or .glb. The glTF extension will be used in this blog.

Example Matterport Scan in AWS IoT TwinMaker

Prerequisites

An AWS account will be required to setup and execute the steps in this blog. An AWS Cloudformation template will configure and install the necessary AWS Lambda Function, IAM roles, and Amazon S3 bucket. It is recommended that you work in the Virginia region (us-east-1). You may incur cost on some of the following services:

• Amazon Simple Storage Service (S3) Storage costs
• AWS Lambda Model Convert Function

Steps

Download Matterpak Sample Bundle

Download one of the Matterpak Bundles. Select one of the bundles, such as Pro2. This available list of bundles may change. The approximate file size for the Pro2 sample bundle is 178MB.

Install Model Convert Lambda Function

1. Download the sample Lambda Model Convert deployment package. The function code within this package will perform the following:
   – Download Matterpak bundle from S3
   – Extract to the Lambda /tmp directory
   – Compress all JPG images
   – Convert OBJ files to glTF
   – Convert glTF to Draco glTF
   – Upload Draco glTF model back to the S3 Bucket.
2. Log into the Amazon S3 console
3. Create an S3 bucket or choose an existing one where you will upload the Lambda function you downloaded. Leave the file zipped as is.
4. Once the Lambda function has been placed in S3, launch this CloudFormation template
5. Change the LambdaArtifactBucketName parameter value to the name of the bucket you uploaded the Lambda function to
6. Change the S3BucketName parameter value to the name of a new bucket that will host your model files. This will be created for you. Be sure to select a name that is globally unique as it will fail during the creation of the stack otherwise.
7. Click on Create Stack to set up the model conversion pipeline
8. Once complete, navigate to the new S3 bucket. A link can be found under the Resources tab
   
9. Create a folder in this bucket and name it paks
10. Upload the Matterpak bundle that was downloaded in step 1 to the paks folder. Be sure to keep it zipped as the Lambda function will unzip it during processing. The conversion process will begin automatically and may take a few minutes.
11. If the model is converted successfully, you will see a glTF file in the root of the S3 Bucket. If not, check Amazon CloudWatch for any logs from the Lambda function.
    

Add Model to Scene (Optional)

To recap, you have successfully compressed and converted a nearly 180MB point cloud scan by Matterport to an 8MB glTF file. With the model converted, you can try to load this in your IoT TwinMaker workspace. Note that any Mattertags you have created in Matterport are not transferrable in this process. This must be recreated using IoT TwinMaker Tags in the Scene composer.

1. In your IoT TwinMaker Workspace, upload the glTF model in the Resources section. If you haven’t already created a workspace, please follow the steps at Getting Started with AWS IoT TwinMaker.
   
2. Add this model to your scene or create one if it doesn’t already exist. If you need guidance on this process, the documentation is available here. Don’t forget to set environmental lighting as the model will appear all black.

Clean Up

Be sure to clean up the work in this blog to avoid charges. Delete the following resources when finished in this order

1. Delete the object files in the Lambda and Model S3 Buckets. Note, this is not the IoT TwinMaker Workspace bucket but rather the buckets created for this blog
2. Delete the CloudFormation Stack
3. Delete the model from your TwinMaker workspace

Conclusion

In this blog, you created a model conversion pipeline to compress and convert a Matterpak bundle into glTF format.  This includes generic conversion of OBJ from other systems as well. With this pipeline, you will be able to reduce Scene load times and streamline 3D model updates directly to your IoT TwinMaker workspace.

Updates

• 9/1/2022: Added Draco Compression to reduce model size

About the author

Chris Azer is a Principal IoT Specialist Solutions Architect helping customers with their digital twin initiatives. Chris has worked in various roles at AWS since 2017 supporting partners and customers with architecting IoT solutions. This includes a broad set of use cases covering the DoD, Manufacturing, State and Local Government, Federal and Civilian, Smart Cities, Partners, and others. His career in Industrial Automation dates back to 2004 where he continues to assist enterprises today with their smart manufacturing journey.