Customers build, deploy, and maintain millions of web applications on AWS and many customers deploy these applications using the Apache web application server. Web application performance is a key metric in modern enterprise applications. On AWS customers leverage Amazon CloudWatch to monitor response times, uptime, and provide SLAs. Engineering teams that run large scale applications on the Apache web application server also track data points and metrics such as server performance within Apache web application server logs. Engineering teams know that subpar performance will degrade customer experience that can lead to underperformance in revenue, possibly affecting compliance, and possibly operational downtime.

Customers can leverage Contributor Insights rules, a feature of Amazon CloudWatch, for web application log analysis. Customers can evaluate patterns in structured log events as they are streamed to CloudWatch Logs, including any custom logs sent by applications in the cloud or from on-premises servers, such as Apache web server application logs. CloudWatch Contributor Insights enable product and engineering teams to view, investigate, and remediate issues that occur in web applications. Once configured, Contributor Insights runs continuously without manual intervention aiding operators isolate, diagnose, understand, and remediate issues during an operational event.

This post will show you how to monitor and perform analysis of Apache logs using CloudWatch Contributor Insights. Apache logs provide visibility on web applications such as errors, response times, and performance of web applications. In addition, to CloudWatch Contributor Insights, customers can then extend their monitoring to CloudWatch Metrics, dashboards, and alarms for a complete monitoring system.

Prerequisites

For this walkthrough, you should have the following prerequisites:

1. Enable Apache server to run in an AWS account by using the included CloudFormation template that spins up a sample web application running behind an Apache web application server.
2. Apache server running in an AWS account with web logs enabled in JSON format (see below for Apache configuration).

Solution Overview

Figure1: Solution Architecture

Preceding figure shows how Apache web server application logs are sent to CloudWatch through the pre-installed CloudWatch agent on the instances. All Amazon Linux 2 AMIs include CloudWatch agent. These logs are analyzed by Contributor Insights rules and report is displayed on CloudWatch dashboard.

Walkthrough

From the AWS Management Console, navigate to CloudWatch to create a log group.

Step 1: Select Create log group.

Figure 2: Creating log group

In CloudWatch navigate to Contributor Insights. On the Contributor Insights home page, select Create a Rule.

1. Select Apache log group by name from the drop down.
2. Select Custom rule for Rule type.
3. In Contribution section – enter the unique keys ‘remote_addr’ and ‘status’ to extract remote address of the request and the status of the request. These will be visualized in the CloudWatch dashboard.
4. Give Rule name.
5. Select Create.

Figure 3: Creating rule definition

Once you create a rule, it may take up to 5 minutes before you see any data being reported. At this point, let’s deploy the CloudFormation Stack that will create an EC2 Instance running Apache and a Node Express Web Application w/ two API endpoints to simulate a web application and API example.

Step 2: Deploy the following CFT template:

https://apache-app-server-cw-00.s3.amazonaws.com/Apache-Web-App-Server-Express-App-Monitor-CloudWatch.yaml

Figure 4: Select CloudFormation

Figure 5: Specify parameters

This will take 5 – 10 minutes. You can view the status of the deployment.

Figure 6: CloudFormation events screen

Once the stack is deployed check the outputs tab for the Apache URL:

Figure 7: CloudFormation output parameters

Step 3: Once complete click on the WebsiteURL to see the following monitoring application running with two API endpoints FAILED and OK. The application is running and logging to Apache web logs in JSON format. These logs are streamed to CloudWatch in the access_json_log log group for analysis.

Click continue to HTTP Site when prompted with the following page:

Figure 8: View web application

Step 4: View the Apache Web application server status and then click on /admin/health/x1 and /return-status/200 to open in new tabs (open multiple times to simulate API access):

Figure 9: Apache app server status

Step 5: Now that the example web application and API is running, we will create a CloudWatch Dashboard to display 5XX, 4XX, 3XX response codes from the application. Then we will create CloudWatch Metrics for 5XX 4XX 3XX response codes that will be used for CloudWatch Alarms.

Figure 10: CloudWatch log group

Let’s filter for log messages with 3XX response code.

Figure 11: 3XX response code

Let’s filter for log messages with 4XX response code.

Figure 12: 4XX response code

Let’s filter for log messages with 5XX response code.

Figure 13: 5XX response code

Step 6: Now let’s create a Metric Filter that will be used for visualization on the CloudWatch dashboard and for a CloudWatch Alarm.

Figure 14: Creating metric filter

Figure 15: Assigning metric filter

Figure 16: Metric filter review screen

Similarly, for 2XX, 4XX, 5XX status codes

Figure 17: CloudWatch metric filter screen

Step 7: Once these metrics are created the aggregated data will aid with the reporting of the contributions in Contributor Insights. Now let’s add the Contributor Insight rule to the CloudWatch Dashboard. Select on Add to dashboard to add to existing dashboard or create a new dashboard. Here we’ll add the CI rules as well the Alarm created previously to show a single pane of glass for 5XX.

Figure 18: Contributor Insight rule 1

Figure 19: Contributor Insight rule 2

Figure 20: Contributor Insight rule – adding to dashboard

Figure 21: Add to dashboard

Figure 22: Dashboard view 1

Figure 23: Dashboard view 2

Summary

In this blog we explained how to stream Apache logs from an EC2 instance to CloudWatch, parse the logs with filters, create a graph to visualize Apache status codes, and create alarms for status codes all within a single pane of glass.

Web application/API monitoring and observability are instrumental to enterprises and their business. Product and engineering teams can view, investigate, and remediate issues that occur in web applications when web server logs (Apache/Nginx) are streamed to CloudWatch using the broad capabilities of CloudWatch Contributor Insights, CloudWatch Logs, CloudWatch Metrics, and CloudWatch Alarms.

About the Author