Introduction

Over the years, enterprises have enjoyed the power of in-memory SAP HANA databases to help with the performance of critical business processes. However, as their usage and data footprint increase, one operational challenge is the startup time required for data to be loaded into memory following an application restart. This requirement results in longer system outages for some patching and SAP HANA based failure scenarios.

As an example, SAP Administrators are often asked by the business – “How much downtime do you need to apply a particular SAP HANA SP Stack upgrade?” In the breakdown of the downtime activities, the SAP HANA startup time could consume a significant portion of the downtime window, often in the range of 10-30 minutes or more depending on the database size.

With SAP HANA 2.0 SPS04+, SAP introduced the Fast Restart Option, leveraging the native Linux tmpfs feature, to help enterprises reduce the business downtime by significantly reducing startup times.  The feature is quick to implement and requires no additional resources, so should be evaluated for all SAP HANA based workloads where startup time is important.

In this blog, we will explain the concept, demonstrate the difference in startup times and provide details for implementation. The SAP HANA Fast Restart feature can be implemented for SAP HANA databases running in AWS on SAP certified Amazon Elastic Compute Cloud (EC2) instances.

Solution Overview

To understand the solution, let’s first look at SAP HANA Memory Management.

The SAP HANA database supports two types of table storage, column store and row store. SAP have optimized HANA to use column store, and this is the default.

The column store is made up of two data structures – MAIN and DELTA. MAIN storage is compressed and optimized for read operations whereas DELTA storage is the target for all write operations. Data changes are moved from DELTA to MAIN storage via the delta merge operation. More details can be found in the SAP Documentation: Memory Management in the Column Store

The SAP HANA Fast Restart option uses tmpfs storage, a volatile temporary file system that resides in RAM, to preserve and reuse MAIN data fragments. This is effective in minimising memory load times in cases where the operating system is not restarted, applicable to the following scenarios.

●       SAP HANA Restart

●       SAP HANA Service Restart, including Index Server Crash.

●       SAP HANA Upgrade/Service Pack

Fig. 1 shows the memory use case as well as how tmpfs can grow and shrink dynamically. The following three parameters are important for the setup:

1)    basepath_persistent_memory_volumes: The location of the tmpfs filesystems.

2)    persistent_memory_global_allocation_limit: By default, no limit is specified. You have the option to limit the maximum size of the persistent memory on a host.

3)    table_default: By default, the value is ON. You have the option to set it to OFF and manually control the persistent memory usage at any of the three levels – table, partition or column using the PERSISTENT MEMORY switch.

More information on the above parameters and their usage can be found in SAP Documentation: Persistent Memory

Fig. 1 SAP HANA Fast Restart

Every customer on SAP HANA 2.0 SP4 or higher should consider implementing this feature as there is no impact to SAP HANA online performance and sizing KPIs. Also, to have consistency across the landscape, enable SAP HANA Fast Restart across your system environments – non-production and production.

Test Results with and without HANA FAST Restart Enabled

For testing purposes, I created multiple tables with test data in the DBACOCKPIT schema. Let’s look at the restart times for a database size of 1,062 GB on an x2idn.32xlarge EC2 instance with 128 vCPUs, and 2048 (GiB) of memory with and without SAP HANA Fast Restart.

Operating System: SUSE Linux Enterprise Server 12 SP4
SAP HANA Version: 2.00.050.00.1592305219 (fa/hana2sp05)
HANA DB Size: 1,062 GB
Instance Type: x2idn.32xlarge
Storage Type: gp2/gp3 (configuration based on SAP HANA on AWS storage configuration guide)

The data size on disk and the memory can be seen below:

Startup load time without Fast Restart:

Index Server Trace:

[57833]{-1}[12/-1] 2022-08-03 08:53:44.790361 i TableReload      TRexApiSystem.cpp(00376) : Starting preload of table HDB::DBACOCKPIT:LOADGENen
[57831]{-1}[-1/-1] 2022-08-03 09:16:28.450727 i Service_Startup  TREXIndexServer.cpp(02059) : Pre-/Re-Loading of column store tables finished.

It took 23 minutes to completely load the column store.

Startup load time with Fast Restart:

Index Server Trace:

[77218]{-1}[13/-1] 2022-08-03 09:25:26.339358 i TableReload      TRexApiSystem.cpp(00376) : Starting preload of table HDB::DBACOCKPIT:LOADGENen
[79544]{-1}[-1/-1] 2022-08-03 09:26:28.037447 i Service_Startup  TREXIndexServer.cpp(02059) : Pre-/Re-Loading of column store tables finished.

It took 1 minute to completely load the column store.

Result: Using Fast Restart Option, HANA STARTUP load time reduced significantly. In this case the startup time reduced from 23 minutes without Fast Restart to 1 minute with Fast Restart.

Implementation Steps

Often greater benefits come with greater complexities and cost, however in the case of SAP HANA Fast Restart that’s not the case. Fast Restart offers easy implementation and is included with SAP HANA 2.0 SPS04 and higher at no additional cost.

Steps to implement SAP HANA Fast Restart on an SAP HANA certified EC2 instance:

Step 1 – Determine how much memory each CPU socket has:

cat /sys/devices/system/node/node*/meminfo | grep MemTotal | awk 'BEGIN {printf "%10s | %20s\n", "NUMA NODE", "MEMORY GB"; while (i++ < 33) printf "-"; printf "\n"} {printf "%10d | %20.3f\n", $2, $4/1048576}'

Example output:

NUMA NODE |              MEMORY GB
--------------------------------------------
        0 |               1000.034
        1 |               1000.067

Step 2 – Create the mount points. Create 1 tmpfs per NUMA node. I am creating 2 mount points as x2idn.32xlarge has 2 NUMA nodes with 1000GB memory each.

mkdir -p /hana/tmpfs0/<SID>
mkdir -p /hana/tmpfs1/<SID>
chown -R <sid>adm:sapsys /hana/tmpfs*/<SID>
chmod 777 -R /hana/tmpfs*/<SID>

Step 3 – Add the following lines to /etc/fstab

tmpfs<SID>0 /hana/tmpfs0/<SID> tmpfs rw,relatime,mpol=prefer:0
tmpfs<SID>1 /hana/tmpfs1/<SID> tmpfs rw,relatime,mpol=prefer:1

Step 4 (Optional) – To limit the memory allocated to the TMPFS filesystems, it is possible by passing the parameter “size”. In the below example I am limiting the memory to 250G.

tmpfs<SID>0 /hana/tmpfs0/<SID> tmpfs rw,relatime,mpol=prefer:0,size=250G
tmpfs<SID>1 /hana/tmpfs1/<SID> tmpfs rw,relatime,mpol=prefer:1,size=250G

Step 5 – Mount the filesystems you have just added to /etc/fstab

mount -a

Step 6 – Alter the following parameters using HANA Studio, or hdbsql (run as the user <dbsid>adm).

hdbsql -u system -p <password> "ALTER SYSTEM ALTER CONFIGURATION ('global.ini','SYSTEM') set ('persistence','basepath_persistent_memory_volumes') = '/hana/tmpfs0/<SID>;/hana/tmpfs1/<SID>' with reconfigure;"

hdbsql -u system -p <password> "ALTER SYSTEM ALTER CONFIGURATION ('indexserver.ini', 'SYSTEM') SET ('persistent_memory', 'table_default') = 'on' WITH RECONFIGURE;"

Step 7 (Optional) – Alter (or use default) parameters related to persistent_memory_global_allocation_limit and table_default. I am using default values for both the parameters.

persistent_memory_global_allocation_limit = Max Size (no limit is specified – default)
table_default = ON (default)

Step 8 – Restart SAP HANA in order to activate the changes

HDB start

Step 9 – Check tmpfs consumption.

Note: The first restart after implementing Steps 1 to 6 will take the same amount of time as without tmpfs. The subsequent restarts will be faster.

df -h | head -1; df -h| grep tmpfs<SID>

Step 10 – Check table consistency (optional)

CALL CHECK_TABLE_CONSISTENCY('CHECK_PERSISTENT_MEMORY_CHECKSUM', NULL, NULL);

Additional Information:

The setup of SAP HANA FSO creates an association between the tmpfs configuration of your SAP HANA system and the CPU and Memory specifications of the EC2 instance. If you wish to maintain the flexibility to change instance sizes, consider setting up a systemd service to re-size tmpfs. The following steps provide sample guidance for doing this.

Step 1 – Navigate to /etc/systemd/system

Step 2 – Create your service. E.g. sap-hana-tmpfs.service. You can use this script as an example to create your service:

https://github.com/aws-samples/aws-sap-hana-fast-restart-scripts/blob/main/sap-hana-tmpfs.service

Step 3 – Create the script which is called by the service created in Step 2. E.g. sap-hana-tmpfs.sh script at /etc/rc.d/sap-hana-tmpfs.sh. You can use this script as an example:

https://github.com/aws-samples/aws-sap-hana-fast-restart-scripts/blob/main/sap-hana-tmpfs.sh

Step 4 – Reload the service files to include the new service:

sudo systemctl daemon-reload

Step 5 – Start the service:

sudo systemctl start <your_service_name>

Step 6 – Check status of your service

sudo systemctl status <your_service_name>

Note: After changing the instance type, check the HANA parameters manually to ensure the configuration is according to your requirements after changing the instance type.

Conclusion

For enterprises running their mission critical applications on SAP HANA Databases, minimizing downtime is key and leveraging the SAP HANA Fast Restart Option can help to reduce the business downtime without modifying the underlying infrastructure.

Join the SAP on AWS Discussion

In addition to your customer account team and AWS Support channels, we have recently launched re:Post – A Reimagined Q&A Experience for the AWS Community. Our SAP on AWS Solution Architecture team regularly monitor the SAP on AWS topic for discussion and questions that could be answered to assist our customers and partners. If your question is not support-related, consider joining the discussion over at re:Post and adding to the community knowledge base.

To learn why AWS is the platform of choice and innovation for more than 5000 active SAP customers, visit the SAP on AWS page.