Unreal Engine Pixel Streaming in AWS with Ubuntu OS

This blog was written by Gena Gizzi and Noor Fairoza.

A 2020 Global Market Insights report estimates that the 3D rendering market will hit $9 billion by 2026, especially with the rise of 3D enabling devices, high-speed internet and cloud technology. There is an increasing adoption of 3D technology across multiple industry verticals where the goal is to provide customers with interactive experiences where they can visualize and engage with products, training, immersive games, illustrations etc.

Unreal Engine is one of the most advanced real-time 3D creation and rendering tools for photo-real visuals and immersive experiences. Pixel Streaming by Epic Games is a solution that allows you to stream Unreal Engine projects from any device that has a screen and connection to the internet such as computers, cellphones, tablets and so on. As the availability of streaming technology matures, businesses of all sizes can take advantage of delivering high-quality 3D rendering for interactive entertainment, architectural visualization, high fidelity car configurators, or any high quality interactive content over thin clients such as web browsers, regardless of local GPU power.

Using AWS and Unreal Engine’s Pixel Streaming solution, developers can create content with Unreal Engine (UE) and deploy it on Amazon Elastic Compute Cloud (EC2) so users can engage with the content. You can use EC2 instances with either Windows or Linux OS to deploy your Unreal Engine application. This tutorial will use a Linux OS, but you can find documentation on how to do this with Windows here. This documentation will describe step-by-step how to setup Pixel Streaming over an Ubuntu server on AWS and connect to the server. You can learn more about Pixel Streaming here.

We are going to use a G4dn EC2 instance to build an Unreal Engine application and run the build. G4 instances are GPU instances that are designed for graphics-intensive workloads and offer a powerful, low-cost, pay-as-you-go model which is ideal for on-demand interactive content.

Step 1: Create an EC2 instance and connect to your instance

The first step is to login to the AWS Management Console and create an EC2 Instance and an AMI with Ubuntu OS. If you do not have an AWS account, you can do so following these instructions.

1. In the AWS Management Console, search for EC2 and select “Launch instance”
2. Give your instance a name, such as “Pixel Streaming Server“ and select an Amazon Machine Image (AMI). For graphics instances we want to ensure that the appropriate GPU drivers are installed for video encoding. I have used Ubuntu 20.04 LTS AMI, as this AMI come pre-installed with NVIDIA CUDA drivers. Or you could choose to install the drivers by following instructions in this documentation after creating your instance.
   

   Image 1: Creating EC2 instance: choose appropriate AMI

   

3. Choose the instance type to be at-least G4dn.Xlarge
4. Create a new key pair, unless you have an existing one that you want to use, and download it for future use.
5. Make sure you add the following rules to your security group.
   TCP/UDP port 8443 is configured here because NICE DCV, the service used to access the instance, operates on that port. HTTP port 80 is opened because users access pixel streaming on the instance through a web browser on that port. You can set the IP range to 0.0.0.0/0 which is open to the world, just for demo purposes. The setup still has a layer of security, because we need a key-pair to SSH and a username and password to access our EC2 instance through NICE DCV. In a production environment however, you’ll want to operate by the least privileged principle to reduce your blast radius and set the IP range to your IP address or a range that is confined to your developers.
6. Add at least 250 GiB storage on the root EBS volume.
   In this guide, we use Nice DCV for connecting to the EC2 instance and deploying the Unreal Engine project. NICE DCV is a high-performance remote display protocol that provides customers with a secure way to deliver remote desktops and application streaming. Nice DCV is free to use on Amazon EC2 and it provides high performance and low latency for demanding 2D and 3D applications. By using NICE DCV with Amazon EC2, you can run graphics-intensive applications remotely on Amazon EC2 instances. You can then stream them to your machines via the client, DCV Viewer.
7. Download the client (DCV Viewer) here. Make sure to download the client version that works on your own computer that you are building this demo on. Alternatively, you could also use AWS Marketplace AMI with DCV server and Nvidia drivers pre-configured. If you used the marketplace AMI, You can skip to ‘Connect to your EC2 instance via DCV Viewer’.NICE DCV operates on TCP/UDP port 8443, which is why it is open in our security group. By default, NICE DCV uses the WebSocket protocol, which is based on TCP, for data transport. You can configure NICE DCV to use the QUIC protocol for data transport, which is based on UDP. If your network experiences high latency and packet loss, using QUIC might improve performance. You can find more information here.Before you launch the instance, please note that, the cost estimate for using G4dn.Xlarge instance with 250 GB EBS in us-east-1 region is $408 USD/Month. You can use AWS Cost Calculator to get a cost estimate with different configurations.
   

   Image 2: G4dn Instances Cost in us-east-1

8. Keep all other configurations as default. Finally, launch the instance.
9. Connect to the EC2 instance once it has been launched and run the following commands. You will need to add an IAM role to the EC2 instance that allows S3 Read permissions
   #!/bin/bash
#Installing the Ubuntu desktop manager
sudo apt update -y
sudo apt install -y ubuntu-desktop
sudo apt install x11-xserver-utils
sudo apt install awscli -y
sudo apt install -y dpkg-dev
sudo systemctl restart gdm3
sudo apt-get install xorg-dev -y
#............................
# Disable the Wayland protocol. NICE DCV doesn't support the Wayland protocol.
sudo sed -i '/WaylandEnable/s/^#//g' /etc/gdm3/custom.conf
# .............................
# Install NVIDIA Drivers
#..............................
sudo apt-get install -y unzip gcc make linux-headers-$(uname -r)
cat << EOF | sudo tee --append /etc/modprobe.d/blacklist.conf
blacklist vga16fb
blacklist nouveau
blacklist rivafb
blacklist nvidiafb
blacklist rivatv
EOF
# ........
sudo sed -i -e '$a GRUB_CMDLINE_LINUX="rdblacklist=nouveau"' /etc/default/grub
sudo update-grub
# Make sure to give the EC2 instance S3 read permissions, otherwise this will fail
sudo aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ .
sudo chmod +x NVIDIA-Linux-x86_64*.run
sudo /bin/sh ./NVIDIA-Linux-x86_64*.run
nvidia-smi -q | head# Configure the X Server
sudo systemctl set-default graphical.target
sudo systemctl isolate graphical.target
sudo apt install -y mesa-utils
sudo init 3
sudo init 5
sudo nvidia-xconfig --preserve-busid --enable-all-gpus
sudo DISPLAY=:0 XAUTHORITY=$(ps aux | grep "X.*\-auth" | grep -v grep \
| sed -n 's/.*-auth \([^ ]\+\).*/\1/p') glxinfo | grep -i "opengl.*version"
sudo systemctl isolate multi-user.target
sudo systemctl isolate graphical.target# Install DCV
wget https://d1uj6qtbmh3dt5.cloudfront.net/NICE-GPG-KEY
gpg --import NICE-GPG-KEY
sudo rm NICE-GPG-KEY
sudo dcvstartx &
sudo wget https://d1uj6qtbmh3dt5.cloudfront.net/2022.0/Servers/nice-dcv-2022.0-12123-ubuntu2004-x86_64.tgz
sudo tar xvzf nice-dcv-*ubun*.tgz && cd nice-dcv-*64
sudo apt install -y ./nice-dcv-ser*.deb
sudo apt install -y ./nice-x*.deb
sudo apt install -y ./nice-dcv-web*.deb
sudo usermod -aG video ubuntu
sudo usermod -aG video dcv# Start the DCV Server and DCV session
sudo systemctl enable dcvserver
sudo systemctl start dcvserver
sudo dcv create-session --type=console --owner=ubuntu newsession
sudo dcv list-sessions
#sudo dcv close-session newsession
# Create random password for the user Ubuntu
PASS=$(aws secretsmanager get-random-password --require-each-included-type --password-length 20 --query RandomPassword)
INSTANCE=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/public-hostname)
aws secretsmanager create-secret --name DCV/$INSTANCE --description "Credentials for $INSTANCE." --secret-string "{\"user\":\"ubuntu\",\"password\":$PASS}"
sudo systemctl isolate multi-user.target
sudo systemctl isolate graphical.target

Connect to your EC2 instance via DCV Viewer.

10. Connect to the instance by using the public ip address from EC2 console and port number 8443 and session #newsession. You specify the session details to connect in the following way:

server_hostname_or_IP:port#session_id

So, if will look like:

server_hostname_or_IP:8443#newsession

You can use username ‘Ubuntu’ and the random password is stored in SecretsManager and copy it from there to update the workstation password.

Image 6: Connect to Ec2 via DCV Viewer

If you can’t connect to the instance immediately, that is likely because the instance is still initializing and the user data is still executing. Give it a few minutes to complete and try again.

Step 2: Install Unreal Engine Editor in your Ubuntu Instance

If you already have your Unreal Engine application with the Pixel Streaming plugin enabled and which was packaged for the Linux OS, you can skip to step ‘Setup Pixel Streaming server’.

Unfortunately, you cannot install Unreal Engine on Linux OS with a binary installer. Instead, you will need to compile a binary of Unreal Engine from the source code in the Epic Games GitHub repository. You will need to first connect your Epic Games account to your Github account to get access to the repository.

11. Follow the instructions here after creating and connecting your EC2 instance to have Unreal Engine running on your EC2 machine.
12. Once you have access to the Unreal Engine Github repository found here, https://github.com/EpicGames/UnrealEngine then open the terminal from your EC2 by selecting the “Connect” option and clone the git repo in your machine using the command below. Make sure you have setup an ssh key in your Github. Please refer here for more information to generate an ssh key and add it to your GitHub profile.

git clone -b 4.27 git@github.com:EpicGames/UnrealEngine.git

Image 7: Cloning the Unreal Engine git repository

13. The Unreal Engine repository consists of scripts to setup and build the Unreal Engine Editor. Run the following commands

cd UnrealEngine/
./Setup.sh
./GenerateProjectFiles.sh
make

Image 8: Unreal Engine setup scripts

Image 9: Generating the Project files script

Image 10: building Unreal Engine Editor

14. Once the build is successful, you can find the editor’s binary in the following path. Run the commands to launch Unreal Engine

cd Engine/Binaries/Linux/
./UE4Editor

Image 11: Starting Unreal Engine Editor from terminal

Image 12: Unreal Engine editor

Step 3: Setup your Unreal Engine Application for Pixel Streaming

15. Go back to the NICE DCV viewer and you should see Unreal engine opening. It might take a bit to open as it has to compile shaders and initialize.

Image 13: Create a new project in Unreal Engine Editor

16. Create or open an Unreal Engine project. If you don’t have a project, you can use a sample game provided by Unreal Engine.
17. After creating your project, you need to prepare your Unreal Engine application for Pixel Streaming. For detailed instructions follow the guide here. To start, enable the Pixel Streaming plugin by going to Edit > Plugins, search for Pixel Streaming, and check the Enable box and restart Unreal Engine.
18. Add the following additional launch parameters by opening Edit > Editor Preferences > Level Editor > Play category and putting the following in the Additional Launch Parameters section:

-AudioMixer -PixelStreamingIP=localhost -PixelStreamingPort=8888

19. Package your Project for Linux. From the main menu in the Unreal Editor, choose Files > Package Project > Linux>Linux

Image 14: Package Project in Linux

Depending on the project and size of the instance, this step might take few minutes to an hour to finish.

Image 15: Packaging prompt with output log

You can take a look at the logs by clicking ‘Show Output Log’ link the popup.

Image 15: Logs showing successful build

20. After a successful build, navigate to the folder LinuxNoEditor to find the packaged Unreal Engine application build. Here, it is fps.sh or <Project-Name>.sh

Image 16: Built Unreal Engine application in LinuxNoEditor Directory

Step 4: Setup signalling server for Pixel Streaming

To setup the Pixel Streaming server, Unreal Engine has provided the scripts to start the signaling server, web server, and matchmaker server under in the Samples/PixelStreaming folder. You can find the Samples in the Unreal Engine installation folder as well as in your packaged Unreal Engine application folder.

A Signalling server is a WebRTC server that manages the connections between the browser and Unreal Engine Pixel Streaming application in the EC2 instance. The signalling and webserver is a node application cirrus.js which intakes the parameters for configuration from the json document config.json in the PixelStreaming folder.

21. To prepare the signaling server, change directory to the SignallingWebServer > platform_scripts > bash folder as shown below and run the setup script ./setup.sh

Image 17: Signalling Web server

Image 18: Setup pre-reqs for pixel streaming

22. You can now start the signalling server with the command ./Start_SignallingServer.sh

This script in-turn runs the Start_Common.sh script which sets up the signalling server’s peerConnectionOptions (hostnames and ip addresses of STUN and TURN servers) and the public ip parameters. You can change the default values of the port number, publicConnectionOptio, etc. Please refer to the SIgnalling Server configuration parameters section this reference guide.

Image 20: Start Unreal Engine application with Pixel Streaming

23. From a separate terminal window, navigate to your packaged project directory. You can now start your Pixel Streaming application with Renderoffscreen flag. In this tutorial, I have created a first person shooter Unreal Engine application ‘fps.sh’.

./fps.sh -RenderOffScreen -PixelStreamingIP=127.0.0.1 -PixelStreamingPort=8888

After you start your application, if Pixel Streaming is setup correctly, you can see the signalling server logs ‘Streamer Connected ::ffff’

Image 21: Unreal Engine app connected to Signalling Server

Step 5: Connect to your stream

24.  You can now browse to the local host within the same instance or use the public ip of the EC2 instance to access your UnrealEngine Pixel Streaming application on any browser on a Laptop or mobile device.
25. Click on the Play button to start your stream.

Image 22: Connect to Streaming App via browser

Step 6: Environment Cleanup

Once you are done the tutorial, make sure to terminate your instances so you are not being charged if they are not needed.

26. Login to your AWS console and navigate to EC2 Dashboard. Choose the EC2 instance you have created earlier by selecting the checkbox. Click on Instance state dropdown and click on ‘Terminate Instance’.

Conclusion

In this tutorial, you have created a G4dn EC2 instance, set up Nice DCV server and connected to your Ubuntu-based EC2 instance via DCV viewer, setup and built your UnrealEngine Editor on Linux, created and configured your application to enable Pixel Streaming, packaged your application to run on linux, setup and run a signalling server, started and got access to your Pixel Streaming application via web browser.