AWS Partner Network (APN) Blog
Monitoring Your Linux Workloads on AWS Graviton with Splunk Cloud
By Karsten Ploesser, Sr. ISV Solutions Architect at AWS
By Igor Alekseev, Sr. Partner Solutions Architect at AWS
 |
 |
 |
Splunk recently announced the availability of Armv8 64-bit architecture support for the Splunk Universal Forwarder. This allows AWS Graviton2 customers with Linux workloads to collect and forward machine data to their Splunk environment.
In this post, we provide a step-by-step guide to help you set up a Universal Forwarder on a Graviton2 instance running Linux.
You’ll also learn how to configure the instance and the Universal Forwarder to forward data to Splunk Cloud.
Splunk is an AWS Advanced Technology Partner with AWS Competencies in Data & Analytics, DevOps, Security, and other key areas. Organizations use Splunk’s Data-to-Everything Platform to solve their toughest IT and security challenges by turning their data into doing.
Background
AWS Graviton2 processors add even more choice to help customers optimize performance and cost for their workloads.
They are custom built by Amazon Web Services (AWS) using 64-bit Arm Neoverse cores to deliver the best price performance for your cloud workloads running in Amazon Elastic Compute Cloud (Amazon EC2).
Graviton2 has broad applicability for a wide variety of cloud workloads, including application servers, microservices, high performance computing (HPC), electronic design automation, gaming, open-source databases, and in-memory caches.
Splunk Universal Forwarders are designed to be extremely lightweight Splunk instances whose primary purpose is to collect machine data from a remote system and forward it to a Splunk index on another machine.
They can be deployed in the tens of thousands, reporting data in near real-time, providing reliable and secure data collection from remote sources sending data into Splunk software.
For more information about Universal Forwarders and supported platforms, check out Splunk’s documentation.
Use Case Scenario
In our scenario, we’ll configure a single Universal Forwarder installed on a 64-bit Arm version of Amazon Linux that’s running on a Graviton2 instance. This deployment maps to the data consolidation topology described in the Splunk Forwarder Manual.
A typical deployment consists of multiple forwarders sending data to a single Splunk instance (in our case, an indexer in Splunk Cloud).
For the purpose of illustration, we’ll configure a single Universal Forwarder and monitor events in the Linux authorization systems, such as authentication failures for indexing and consolidation in Splunk Cloud.
Figure 1 – Splunk Universal Forwarder Deployment Topology.
Download Splunk Universal Forwarder Binaries for Armv8
Start by getting a download link that points to the Armv8 binaries for the Universal Forwarder, which supports multiple platforms including but not limited to Armv8.
Navigate to the Splunk download page and select Universal Forwarder to begin. You’ll need to sign in to access a download link.
Once you’re on the download page for Universal Forwarder, select Linux and then Armv8. Accept the license agreement and proceed to the download page for your installation package.
Figure 2 – Splunk Universal Forwarder download page.
Next, copy the command line instructions for downloading the Universal Forwarder from your Linux instance, as you will need it in the next step. They will look similar to the code snippet below (assuming your link references the .tgz package).
wget -O splunkforwarder-<YOUR_VERSION>-Linux-armv8.tgz 'https://www.splunk.com/<PATH_TO_DOWNLOAD>'
Launch AWS Graviton2 Instance
Now that you have your Universal Forwarder download link, it’s time to launch a Graviton2 instance to install it on.
You can either use an existing instance or spin up a new one. If you’re unsure how to do that, check out our AWS Knowledge Base.
Splunk’s documentation recommends the Universal Forwarder run on a dual core processor with at least 1GB of RAM. We recommend you test this setup on an M6g instance (such as the instance size m6g.large).
Make sure to open port 22 for Secure Shell (SSH) if you want to connect to this instance using EC2 Instance Connect or SSH.
Figure 3 – Amazon EC2 console.
Make sure you select an Armv8-compatible operating system such as Amazon Linux 2 to run your Universal Forwarder on. Select the corresponding radio button for 64-bit (Arm) and click Select.
Figure 4 – Amazon EC2 launch wizard.
Complete the remaining configuration steps to launch your Graviton2 instance.
Install and Configure the Universal Forwarder
After spinning up your new Graviton2 instance, log into Linux to install and configure the Universal Forwarder software. Use your preferred way of connecting.
In the example below, we use EC2 Instance Connect to establish a browser-based SSH session with instances for setup and configuration tasks.
Figure 5 – Amazon EC2 Instance Connect dialog.
Download and install the Universal Forwarder binaries on your Linux system in accordance with Splunk’s documentation.
If installing manually, use the wget command you saved from above. Below is an example of the commands you can run to download and unpack the software, assuming your link references the .tgz package.
Other package types are also available. You may also want to replace variables such as the download path and forwarder version with your preferences.
Next, set the SPLUNK_HOME environment variable to the location you installed the package (typical default is /opt/splunkforwarder) and start the Universal Forwarder to set an admin user and password. Details can be found in Splunk documentation.
You’ll then be prompted to enter credentials. Make a note of them as you will require them in later steps.
Download Universal Forwarder Credentials Package from Splunk Cloud
Note that to complete this step, you must download the credentials file via your web browser and upload it to your Graviton2 instance. Downloading the file via wget does not work.
Log into your Splunk Cloud environment to download the Universal Forwarder credentials package. Once this package is installed on your Universal Forwarder, it allows you to securely forward data to your Splunk Cloud environment.
If you do not already have a Splunk Cloud environment, it’s easy to set one up by requesting a trial.
Once logged in an on the Splunk Cloud home page, navigate to the Universal Forwarder app. Click on Download to get the customized credentials package for your Splunk Cloud instance.
Figure 6 – Universal Forwarder credentials package download page.
Move the credentials package from your local compute to your Graviton2 instance by opening a terminal and running the following command. This will move the credentials packaged in the /tmp folder on your instance.
scp -i "YOUR_PEM_KEY" splunkclouduf.spl <YOUR_EC2_INSTANCE_URL>:/tmp
Next, switch back to your SSH session and copy the commands below. This moves the credentials package to the Splunk apps folder on your Linux system and installs it in your Universal Forwarder environment.
This completes the basic installation of the Universal Forwarder for Linux.
Send Logs to Your Indexer
You can now add log sources to monitor and configure forwarding to your Splunk Cloud instance. Below is an example of configuring monitoring and outputs for a few common Linux log files.
Start by adding Linux folders and system files to monitor. In this example, we configure the Universal Forwarder to track the usage of authorization systems and forward security-related messages such as authentication failures for indexing by Splunk Cloud.
Next, configure your Universal Forwarder to forward logs to your Splunk Cloud instance. Replace any variables such as your Splunk Cloud instance name with your settings.
Make sure to prepend your Splunk Cloud instance URL with input. Otherwise, you will not be able to ingest logs that are forwarded to Splunk Cloud.
$SPLUNK_HOME/bin/splunk set deploy-poll input-<YOUR_SPLUNK_CLOUD_INST>.cloud.splunk.com:8089
Note that for a production system you may wish to install and configure the *.nix Splunk Add-on for comprehensive Linux monitoring. More about installation of that Add-on can be found on splunkbase.com.
Validate in Splunk Cloud
This completes our step-by-step guide to monitoring Linux workloads on Graviton2 with Splunk Cloud.
In this final step, validate your setup by running a Splunk search against the main index. Wait a few minutes in case no entries are visible. Filter by host name by setting host to the internal DNS name of your Graviton2 instance.
Figure 7 – Splunk search app.
Conclusion
This post demonstrated the ease with which you can install and configure Splunk Universal Forwarders on AWS Graviton2 instances. Doing so gives you access to the better price performance of AWS Graviton2 over comparable x86 instances without compromising your security and operational excellence needs.
Get started today by installing Universal Forwarders on your AWS Graviton2 instances to automate data collection and indexing of machine data in your Splunk environment.
AWS Graviton2 is available in most global AWS regions with additional regions being added continuously.
.
.
Splunk – AWS Partner Spotlight
Splunk is an AWS Competency Partner. Their software and cloud services enable customers to search, monitor, analyze, and visualize machine-generated big data from websites, applications, servers, networks, IoT, and mobile devices.
Contact Splunk | Partner Overview | AWS Marketplace
*Already worked with Splunk? Rate the Partner
*To review an AWS Partner, you must be a customer that has worked with them directly on a project.