AWS Developer Tools Blog

Utilizing Amazon ElastiCache Auto Discovery in .NET Through Enyim

Today, we released a new library, Amazon ElastiCache Cluster Configuration, that allows .NET applications to easily leverage ElastiCache features. This post explains why a programmer would want to use this library and offers a quick and easy way to try it yourself.

What is Memcached?

Memcached provides a way to easily avoid some of the latency that comes with using a database, and it can also help applications at scale by removing some of the strain that can be placed on databases. This is accomplished by having Memcached servers be an intermediary, in-memory cache that can return results much faster than a normal database. In a typical program flow, this is accomplished by requesting a key from a group of cache servers and, if a value is retrieved, no database query is needed. Below is a simple diagram of what this function normally looks like: Memcached Diagram

Why ElastiCache?

ElastiCache provides a way to dynamically add and remove Memcached servers inside of a cache cluster. All servers are completely managed, which means that when servers are added they are automatically configured for Memcached. They are also added to the cluster and, when they are deleted, the cluster is updated. This means you spend less time configuring Memcached servers and more time working on things that matter. Being able to add and remove these nodes dynamically also means your application can easily scale whenever necessary through the AWS Management Console or through one of the many AWS APIs.

Using ElastiCache in .NET

Many .NET developers leverage ElastiCache through the Enyim framework. Enyim provides a client that manages server connections as well as what server your cache data should be stored on. To be aware of the Memcached servers, the Enyim client is configured on instantiation with IPs and ports for all of the servers. When the server information changes, the client must be disposed and re-instantiated with the new server information. The re-instantiation of the client tends to be tedious and can also cause issues if you update your configuration incorrectly when nodes change. One feature of ElastiCache that helps avoid this issue is Auto Discovery. This feature allows clients to find out the cluster configuration through an endpoint URL. The endpoint URL is sort of like an alias that just points to one of the servers in the cluster. Each server holds information about the configuration such as how many times the configuration has changed and the hostname, ip, and port of each server in the cluster. For more information on how Auto Discovery works, visit this page.

ElastiCache Cluster Configuration

Although Auto Discovery is useful, it is not accessible through Enyim’s client because it is not something that is found in standard Memcached clusters. Currently, Amazon has released clients for Java and PHP that extend Memcached clients in order to take advantage of Auto Discovery. With today’s release of the ElastiCache Cluster Configuration library, any .NET application using framework 3.5 or higher can now take full advantage of this great feature. All that is required is to add the clusterclient section to your App.config or instantiate the configuration object through parameters. After that, pass it as the configuration for the Enyim MemcachedClient, and you now have a MemcachedClient functioning through ElastiCache Auto Discovery.

Here is a sample App.config that shows how to specify the Amazon ElastiCache cluster.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <configSections>
      <section name="clusterclient" type="Amazon.ElastiCacheCluster.ClusterConfigSettings, Amazon.ElastiCacheCluster" />
    </configSections>

    <clusterclient>
      <endpoint hostname="my-configuruation-url.cache.amazonaws.com" port="11211" />
      <poller intervalDelay="60000" />
    </clusterclient>

</configuration>

 

Try it now!

For this walkthrough, we assume you have an AWS account with the ability to control Amazon EC2 and ElastiCache services, internet access, and the AWS Tools for Windows PowerShell installed and configured with an API key and secret key.

Create AWS Services using PowerShell

Start a PowerShell instance as Administrator. Below, you can edit these variable values to change property names, and then copy and paste the rest.

$ec2SecurityGroupName = "myElastiCacheGroup"
$cacheGroupName = "myElastiCacheSecGroup"
$keyPairName = "myConfigKeyPair"
$cacheClusterName = "demoCluster"

To use ElastiCache, you must create an ElastiCache cluster and an EC2 instance to access it. First, we create a key pair and an EC2 security group, and then we create an EC2 instance based on those values.

$myConfigKeyPair = New-EC2KeyPair -KeyName $keyPairName
$myConfigKeyPair.KeyMaterial | Out-File -Encoding ascii C:$keyPairName.pem

New-EC2SecurityGroup -GroupName $ec2SecurityGroupName -GroupDescription "ElastiCache Config Demo"

$cidrBlocks = @("0.0.0.0/0")
$ipPermissions = New-Object Amazon.EC2.Model.IpPermission -Property @{IpProtocol = "tcp"; FromPort = 11211; ToPort = 11211; IpRanges = $cidrBlocks}
Grant-EC2SecurityGroupIngress -GroupName $ec2SecurityGroupName -IpPermissions $ipPermissions
$ipPermissions = New-Object Amazon.EC2.Model.IpPermission -Property @{IpProtocol = "tcp"; FromPort = 3389; ToPort = 3389; IpRanges = $cidrBlocks}
Grant-EC2SecurityGroupIngress -GroupName $ec2SecurityGroupName -IpPermissions $ipPermissions

$image = Get-EC2ImageByName -Names WINDOWS_2012R2_BASE
if($image -is [system.array]) {$image = $image[0]}

$reservation = New-EC2Instance -ImageId $image.ImageId -KeyName $keyPairName -SecurityGroups $ec2SecurityGroupName -InstanceType t1.micro

After that is complete, we create a new ElastiCache cluster with three nodes and add the EC2 security group to its policy.

New-ECCacheSecurityGroup -CacheSecurityGroupName $cacheGroupName -Description "Demo for ElastiCache Config"
$secGroup = Get-EC2SecurityGroup -GroupNames $ec2SecurityGroupName
Approve-ECCacheSecurityGroupIngress -CacheSecurityGroupName $cacheGroupName -EC2SecurityGroupName $ec2SecurityGroupName -EC2SecurityGroupOwnerId $secGroup.OwnerId

New-ECCacheCluster -CacheNodeType cache.t1.micro -CacheClusterId $cacheClusterName -CacheSecurityGroupNames $cacheGroupName -Engine memcached -EngineVersion 1.4.14 -NumCacheNodes 3 -Port 11211

Create the Application

To demonstrate how to use ElastiCache Cluster Configuration, we’ll make a quick console application. From the start page of Visual Studio 2010 or higher, click “New Project…”, and in the new project dialog, create a new Visual C# Console Application named “Cluster Config Demo”. After the project is created, in Solution Explorer, right-click on the “References” section, and in the drop-down menu, click “Manage NuGet Packages…”. After that, we’re going to search for “ElastiCacheClusterConfig” and then install it into our current project. When you install this package, Enyim is also installed on the project. Now that we have the project configured, let’s write the code. First, add the packages to the code by pasting this code into the top of the file “Program.cs”.

using Enyim.Caching;
using Amazon.ElastiCacheCluster;

Next, copy the code below and paste it into the Main function in the “Program.cs” file. This snippet creates an ElastiCacheClusterConfig object using the hostname and port specified in the parameters, and then defaults the rest of the settings. It then creates a MemcachedClient through the Enyim framework by passing in the ElastiCacheClusterConfig as an IMemcachedClientConfiguration. The program then attempts to store a value to the cache followed by trying to retrieve a value in the cache.

Console.WriteLine("Creating config...");
ElastiCacheClusterConfig config = new ElastiCacheClusterConfig("YOUR-URL-HERE", 11211);
    
Console.WriteLine("Creating client...");
MemcachedClient client = new MemcachedClient(config);
    
if (client.Store(Enyim.Caching.Memcached.StoreMode.Set, "Demo", "Hello World"))
{
    Console.WriteLine("Stored to cache successfully");
}
else
{
   Console.WriteLine("Did not store to cache successfully");
}

Object value;
if (client.TryGet("Demo", out value))
{
   Console.WriteLine("Got the value: " + (value as string));
}
else
{
   // Search Database if the get fails
   Console.WriteLine("Checking database because get failed");
}

Console.Read();

Be sure to replace “YOUR-URL-HERE” with the endpoint URL of your cluster. You can find this URL by setting your variables from earlier if you closed out of PowerShell, and then running the following:

(Get-ECCacheCluster -CacheCluster $cacheClusterName).ConfigurationEndpoint.Address

Now, go ahead and build your project by right-clicking on the project in Solution Explorer and clicking “Build”. If you run this code on your local machine, it will throw an error because you can only connect to ElastiCache inside of an EC2 instance. That’s why we need to transfer it to the EC2 instance we created earlier.

Upload it to the EC2 instance and test it

There are many ways to access and upload your ElastiCache application to EC2 such as the Visual Studio Toolkit, opening PowerShell access remotely on the instance and downloading it from a URL, or using remote desktop into the instance. Today, we’ll use remote desktop just for simplicity, even though there are much better ways to do this in a development stack. Run the following cmdlet to open a remote desktop connection to the instance. If you closed PowerShell earlier, be sure to copy in the predefined variables. If login fails, which is caused by a changed registry value, simply copy the value from $pass, paste it in as the password, and then login.

$secGroup = Get-EC2SecurityGroup -GroupNames $ec2SecurityGroupName
$groupArray = @($secGroup.GroupId)
$filter_groupId= New-Object Amazon.EC2.Model.Filter -Property @{Name = "group-id"; Values = $groupArray}
$instances = (Get-EC2Instance -Filter $filter_groupId).Instances
$pass = Get-EC2PasswordData -InstanceId $instances.InstanceId -PemFile C:$keyPairName.pem
$dns = $instances.PublicDnsName
cmdkey /generic:$dns /user:administrator /pass:$pass
mstsc /v:$dns

If you would rather use the console, you can find the .pem file for the password in the base directory of C:. Now that we have our connection open, go ahead and copy the executable and .dlls we created earlier and paste them into the instance. Run it and you should see the following output:

Creating config...
Creating client...
Stored to cache successfully
Got the value: Hello World

Delete Demo Services

Once you’ve successfully run your application, you can then delete the resources we created from AWS using the cmdlets below. Note: Be sure to copy the variables from earlier if you’ve closed the PowerShell window before now.

$secGroup = Get-EC2SecurityGroup -GroupNames $ec2SecurityGroupName
$groupArray = @($secGroup.GroupId)
$filter_groupId= New-Object Amazon.EC2.Model.Filter -Property @{Name = "group-id"; Values = $groupArray}
$instances = (Get-EC2Instance -Filter $filter_groupId).Instances
Stop-EC2Instance -Instance $instances -Terminate -Force

Remove-EC2KeyPair -KeyName $keyPairName -Force

Remove-ECCacheCluster -CacheClusterId $cacheClusterName -Force

And delete the policies when both services have finished terminating the resources.

$secGroup = Get-EC2SecurityGroup -GroupNames $ec2SecurityGroupName
Remove-EC2SecurityGroup -GroupId $secGroup.GroupId -Force
Remove-ECCacheSecurityGroup -CacheSecurityGroupName $cacheGroupName -Force

And that’s it for using Amazon’s new ElastiCache Cluster Configuration library for .NET. If you’d like to find out more, visit this wiki or fork the code at our Github repository.