Microsoft Workloads on AWS

How to load .NET configuration from AWS Secrets Manager

Every application has secrets – database credentials, API keys used to call external services, or private encryption keys needed to secure data. In this blog post, I will show you how you can load and use secrets using .NET’s configuration system and AWS Secrets Manager (Secrets Manager).

Keeping your sensitive data outside of your code is crucial, but reducing the risk of compromise is not always a simple task. Many companies find themselves inventing complex and difficult-to-implement techniques, which result in a less streamlined development experience. This might lead developers to find simpler, less secure solutions for storing the application’s sensitive data, such as using hard-coded strings in the application’s code or configuration files. This may lead to a security breach when code that contains sensitive data is then saved in the company’s source control, where multiple parties can access it without proper auditing or other security safeguards. In addition, an organization needs to keep track of multiple versions of the same secret and different values depending on the deployed environment (such as dev, staging, or production). This can become difficult when those secrets are tightly coupled with the deployed code.

Secrets Manager helps you protect secrets needed to access your applications, services, and IT resources. It enables you to easily rotate, manage, and retrieve secrets used by your application, eliminating the need to hard-code sensitive information in plain text. You can use the Secrets Manager client to retrieve secrets using AWS SDK for .NET. However, this would require code changes and add to the complexity of your code, as you need to invoke the client whenever you need to read data stored in Secrets Manager. Instead, you can use the .NET configuration system – an extensible API used to read and manage application secrets. This lets developers use a familiar API to access secrets in secure storage and reduce complexity by using a single code path for all environments. Additionally, the provider lets existing applications move to Secrets Manager without making any code changes.

In this blog post, I will show you how to create a custom configuration provider that loads sensitive data from Secrets Manager and makes those secrets available to your application.

Walkthrough

To load values from Secrets Manager to your .NET configuration, you will need to complete the following steps:

  • Create a custom configuration provider
  • Create a configuration source to initialize the new provider
  • Create a new class to pass the secret’s data to your code
  • Update your code to use the new configuration source
  • Optional: enable secrets reloading

Prerequisites

This example will use credentials needed to connect to a 3rd-party API. The credentials will include an API key, user ID, and password — all stored in Secrets Manager.

Create a secret to use in your application
First, you’ll need to add a new secret to load from your code.

  1. Log in to the Secrets Manager console
  2. Click on the Store a new secret to create your new secret value.
  3. Choose Other type of secret and add the following key/value pairs: 
    {
  "ApiKey": "key1",
  "UserId": "User1",
  "Password": "12345",
}

    AWS Secrets Manager - Choose Secret Type

  4. Choose Next
  5. Fill the secret’s name, description, add tags and then choose Next
  6. On the next screen, choose Next on the Secret rotation page (automatic rotation disabled)
  7. On the last screen, choose Store once you’re done reviewing your changes.

Create an ASP.NET Core Web API project

Although the .NET configuration system can be used by different project types and different versions of .NET, I am using the .NET 6 ASP.NET Core Web API project template (using Visual Studio 2022 or later).

  1. Create a new project of type ASP.NET Core Web API
  2. Fill your project name and choose Next
  3. On the next screen, make sure .NET 6.0 is selected, and choose Create

Step 1: Create a custom configuration provider

  1. Use NuGet to add AWS.SecretsManager as a dependency in your project.
  2. Create a new class named AmazonSecretsManagerConfigurationProvider that inherits from the ConfigurationProvider abstract class. The class constructor will receive two string parameters, region and secretName, and store them as fields in the class.
  3. Override the Load method and add code to retrieve your secret from Secrets Manager as JSON, then deserialize the result as a Dictionary<string, string>. Store the result in Data property (inherited from ConfigurationProvider): 
    public class AmazonSecretsManagerConfigurationProvider : ConfigurationProvider
{
    private readonly string _region;
    private readonly string _secretName;
    
    public AmazonSecretsManagerConfigurationProvider(string region, string secretName)
    {
        _region = region;
        _secretName = secretName;
    }

    public override void Load()
    {
        var secret = GetSecret();

        Data = JsonSerializer.Deserialize<Dictionary<string, string>>(secret);
    }

   private string GetSecret()
    {
        var request = new GetSecretValueRequest
        {
            SecretId = _secretName,
            VersionStage = "AWSCURRENT" // VersionStage defaults to AWSCURRENT if unspecified.
        };

        using (var client = 
		new AmazonSecretsManagerClient(RegionEndpoint.GetBySystemName(_region)))
        {
            var response = client.GetSecretValueAsync(request).Result;

            string secretString;
            if (response.SecretString != null)
            {
                secretString = response.SecretString;
            }
            else
            {
                var memoryStream = response.SecretBinary;
                var reader = new StreamReader(memoryStream);
                secretString = 
		System.Text.Encoding.UTF8
			.GetString(Convert.FromBase64String(reader.ReadToEnd()));
            }

            return secretString;
        }
    }
}

Step 2: Create a configuration source to initialize the new provider

Create a class that implements IConfigurationSource and creates a new instance of AmazonSecretsManagerConfigurationProvider:

public class AmazonSecretsManagerConfigurationSource : IConfigurationSource
{
    private readonly string _region;
    private readonly string _secretName;

    public AmazonSecretsManagerConfigurationSource(string region, string secretName)
    {
        _region = region;
        _secretName = secretName;
    }

    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        return new AmazonSecretsManagerConfigurationProvider(_region, _secretName);
    }
}

Step 3: Create a new class to pass the secret’s data to your code

Create a new class that has properties with the same names as the secret’s keys:

public class MyApiCredentials
{
    public string ApiKey { get; set; }
    public string UserId { get; set; }
    public string Password { get;set; }
}

Step 4: Update your code to use the new configuration source

In this example, I’m using the Visual Studio 2022 .NET project templates. In the new templates, the service’s configuration is in the Program.cs file. Older IDEs (such as Visual Studio 2019 or prior) have similar code in the Startup.cs file.

  1. Create a new static class with an extension method to initialize the new configuration source and add it to the configuration builder: 
    public static void AddAmazonSecretsManager(this IConfigurationBuilder configurationBuilder, 
    					string region,
    					string secretName)
{
    var configurationSource = 
			new AmazonSecretsManagerConfigurationSource(region, secretName);

    configurationBuilder.Add(configurationSource);
}
  2. Add the new configuration provider to the existing list of configuration providers: 
    builder.Host.ConfigureAppConfiguration(((_, configurationBuilder) =>
{
    configurationBuilder.AddAmazonSecretsManager("<your region>", "<secret name>");
}));
  3. Make sure that your new configuration class is registered as the target of your service configuration by adding a call to builder.Services.Configure: 
    builder.Services.AddControllers();

...

builder.Services.Configure<MyApiCredentials>(builder.Configuration);

...

var app = builder.Build();
  4. Now, use the built-in configuration IOption interface to inject the credentials into your application: 
    private readonly MyApiCredentials _myApiCredentials;

public MyService(IOptions<MyApiCredentials> options)
{
    _myApiCredentials = options.Value;
}

Step 5: Reloading secrets

If your application requires configuration updates without restarting – you will need to add functionality to your configuration provider to refresh the configuration data.

  1. Create a new class that will hold the logic needed to refresh your configuration values.This class must have a method/property that returns IChangeToken: 
    public class PeriodicWatcher : IDisposable
{
    private readonly TimeSpan _refreshInterval;
    private IChangeToken _changeToken;
    private readonly Timer _timer;
    private CancellationTokenSource _cancellationTokenSource;

    public PeriodicWatcher(TimeSpan refreshInterval)
    {
        _refreshInterval = refreshInterval;
        _timer = new Timer(OnChange, null, TimeSpan.Zero, _refreshInterval);
    }

    private void OnChange(object? state)
    {
        _cancellationTokenSource?.Cancel();
    }
    
    public IChangeToken Watch()
    {
        _cancellationTokenSource = new CancellationTokenSource();
        _changeToken =  new CancellationChangeToken(_cancellationTokenSource.Token);

        return _changeToken;
    }

    public void Dispose()
    {
        _timer?.Dispose();
        _cancellationTokenSource?.Dispose();
    }
}

    Here I’m using the built-in CancellationChangeToken class to create a change token from CancellationTokenSource.

  2. Pass an instance of that class to the configuration provider and call ChangeToken.OnChange to tie the change token to a method that will be called on each change – in this case Load: 
    public AmazonSecretsManagerConfigurationProvider(
string region, string secretName, PeriodicWatcher watcher)	
{
    _region = region;
    _secretName = secretName;

    _changeTokenRegistration = 
			ChangeToken.OnChange(() => source.PeriodicWatcher.Watch(), Load);

}
  3. Update your code to use IOptionsSnapshot instead of IOption. IOption<T> caches the configuration read from Data once for the entire application lifecycle, but IOptionsSnapshot<T> reads Data on each HTTP request, making sure that the provider gets the latest configuration values: 
    private readonly string _myApiCredentials;

public MyService(IOptionsSnapshot<MyApiCredentials> options)
{
    _myApiCredentials= options.Value;
}

Conclusion

Every application has sensitive data it needs to store in a secure location. Keeping sensitive data in your application code can cause a security breach. On the other hand, you do not want to slow down developers by adding hard-to-follow steps to run and debug your application.

In this post, I showed you how to use Secrets Manager to store your application’s secrets securely and easily retrieve them using the .NET configuration system. This solution will help you reduce the complexity of using and storing your application’s sensitive data without compromising security.

To learn more about how to use Secrets Manager, read How to Store, Distribute, and Rotate Credentials Securely with Secrets Manager or refer to the Secrets Manager documentation.

And if you need to store your .NET application’s properties in an easy to manage, centralized service, read about how to use the .NET Core configuration provider for AWS Systems Manager.

AWS can help you assess how your company can get the most out of cloud. Join the millions of AWS customers that trust us to migrate and modernize their most important applications in the cloud. To learn more on modernizing Windows Server or SQL Server, visit Windows on AWS. Contact us to start your modernization journey today.

TAGS: ,
Dror Helper

Dror Helper

Dror is a Sr. Microsoft Specialist Solutions Architect at AWS. He has been developing and architecting software for the last 20 years and when not writing code has worked with software companies on modernizing their existing legacy code while improving the way they develop software. In his current role Dror assist organizations to migrate and modernize their existing applications.