AWS Cloud Financial Management

Automating tagging for resources created by AWS Service Catalog

AWS Cloud Financial Management services provide cost management and optimization capabilities. You can utilize resource tagging for resource-level cost and usage analytics using cost management tools. However, you may not always adopt a consistent tagging strategy. Customers tell us that the effort to ensure tagging consistency grows with the number of resources utilized across their Amazon Web Services (AWS) accounts. Tagging inconsistencies limit the insights gained from cloud financial management solution.

This blog shows how you can automatically propagate account-level tags to AWS resources created by AWS Service Catalog. Service Catalog allows sharing of portfolios across AWS accounts and provides a TagOption library to manage tags on provisioned AWS resources. Resource tagging varies by account, so it is not part of the portfolio product configurations. We designed the solution to reduce the burden on users to a minimum, while also adopting cloud best practices such as infrastructure automation. We suggest that this solution is deployed by a central team, with permissions to access the AWS Organizations data and to share service catalog portfolios across accounts.

Benefits to tagging resources with account tags

  • Simple and consistent tagging by centralizing tagging operations across multiple AWS accounts
  • Reduced maintenance overhead by adopting serverless-first design principle
  • Reliable tagging operations using event-driven design principle
  • Improved user experience as tagging compliance is ensured automatically

Prerequisites

  • An AWS account with appropriate permissions to create, launch, and update AWS CloudFormation stacks, Amazon EventBridge rules, AWS Lambda functions, and AWS Service Catalog resources (including at least one shared portfolio)
  • Cross-account access to the Management account in order to retrieve account-level tags
  • Administrator privileges to manage AWS Service Catalog including updating portfolios, TagOptions, and associating TagOptions with resources
  • Use AWS Service Catalog to manage the configuration of AWS resources

Overview of the solution

We design an event-driven workflow which automatically updates resource tagging across AWS accounts and associates account-level tags with each AWS resource launched through AWS Service Catalog in four steps (Figure 1).

Figure 1. Automated tagging for Service Catalog portfolios and products

Figure 1. Automated tagging for Service Catalog portfolios and products

  1. The Service Catalog Administrator shares AWS Service Catalog portfolio(s), usually with newly created application environment account(s)
  2. This invokes an Amazon EventBridge rule with event matching pattern on AcceptPortfolioShare and CreatePortfolio with a Lambda function as target
  3. The invoked Lambda function assumes an IAM role with permissions to list account-level tags and retrieves the account-specific tags from the Management account using the AWS Organizations API
  4. The Lambda function updates the TagOption association with the Service Catalog portfolios and products in the respective AWS account.

This ensures all account tags are associated with all products launched from the Service Catalog portfolios.

Walkthrough

We’ve prepared two AWS CloudFormation templates which you can launch in the Management account and the delegated administrator accounts where your service catalog administrators publish service catalog portfolios. The templates provision the following resources (Figure 2):

Figure 2. Resources provisioned with the CloudFormation templates

Figure 2. Resources provisioned with the CloudFormation templates

  1. AWS Lambda function activating the cost allocation tag status for account-level tags in AWS Cost Explorer
  2. AWS IAM Role allowing the cost allocation function to list resource tags, update cost allocation tag status, create log group/stream and put log events
  3. Amazon EventBridge rule invoking the Lambda function on a schedule every day
  4. AWS IAM Role allowing cross-account access to be assumed by the Lambda function to list account-level tags (ListTagsForResource API call).
  5. Amazon EventBridge rule with event matching pattern listening on AcceptPortfolioShare and CreatePortfolio and with a Lambda function as target
  6. AWS Lambda function handling tag association with service catalog portfolios using Python/Boto3
  7. AWS IAM Role allowing the tag association function to list resource tags; perform Service Catalog operations such as updating tag options; and to create log group/stream and put log events
  8. Ancillary function including a Lambda trigger to invoke the tag association Lambda function
  9. IAM Role for the ancillary function which manages the invocation of the tag association function.

Deploying the solution

To launch the CloudFormation stack in your Management account

  1. Log in to the CloudFormation console in the Management account and start the Create Stack wizard.
  2. Select and upload the stack template 01-mgmt_account_setup_stack.yaml.
  3. Specify stack name and parameters, in this case the organizational unit ID(s) and optionally specific member account IDs.
  4. Review and monitor progress until stack status is CREATE_COMPLETE.

To launch the CloudFormation stackset for provisioning into multiple member accounts

  1. Switch to the delegated administrator account where your service catalog administrators are developing and publishing portfolios. Start the Create StackSet wizard.
  2. Select Self-service permissions and the stack template 02-account-tag-association-stackset.yaml.
  3. Specify stack name and parameters. This stack has two parameters: 1) OrganizationCrossAccountTrusteeRole for the role name in the Management account, and 2) the account ID of the Management account.
  4. Set deployment options to either Deploy stack in account and add individual account IDs, or Deploy stack to all accounts in an organizational unit and list the ID of the organizational unit.
  5. Specify regions where the stack resources should get propagated.
  6. Review and monitor progress to confirm successful creation (CREATE_COMPLETE status).

Your service catalog portfolio should now have the account-level tags. In addition, you should be able to see the tags in the AWS Cost Explorer console with the next update cycle (updated at least once every 24 hours). View Cost allocation tags on the AWS Billing console to confirm that the account-level tags appear as User-defined cost allocation tags.

Understanding what you have built

The Lambda function handling tag associations is replicated to each member account via the CloudFormation stackset. It has the following functionality:

  1. Retrieve tags for the member accounts in Figure 1 (list_tags_for_resource)
  2. Retrieve Service Catalog TagOptions (list_tag_options) or create new set (create_tag_option)
  3. List portfolios for which organizational shares were accepted (list_accepted_portfolio_shares)
  4. Check if TagOptions are already associated with each portfolio (list_resources_for_tag_option). Else proceed with the association using the associate_tag_option_with_resource operation.

The tag association function is invoked automatically by the ancillary Lambda function upon completion of stack provisioning and every time a Service Catalog portfolio is shared with the AWS member account. The ancillary Lambda function checks which AWS region the portfolio is in and launches the tagging association Lambda function in the respective region. Each Lambda function has its own log stream in Amazon CloudWatch where you can access all logs. The workflow is based on Service Catalog events as we described in Figure 1. Subsequent updates to the account tags would require to manually re-run the tag association function.

The logic to activate cost allocation status for account-level tags is handled by the Lambda function in the Management account. It is invoked daily by the Amazon EventBridge rule and runs in three steps:

  1. List all member accounts in scope (list_accounts).
  2. List tags for each account (list_tags_for_resource).
  3. Activate the newly associated tags in AWS Billing (update_cost_allocation_tags_status). This will ensure they appear on AWS CUR and Cost Explorer.

Cleaning up

To avoid incurring future charges, remove all resources created with the CloudFormation templates:

  1. Access the AWS CloudFormation service in the AWS Management Console.
  2. Go to StackSets, click on Actions, then Delete stacks from StackSet, and provide all accounts and OUs you have shared. You can find the details of the AWS accounts in StackInstances tab.
  3. Wait until all accounts under Stack instances are deleted. Next, click on Actions and Delete StackSet.
  4. Wait until the stackset is deleted. Log in to member accounts to validate that the stacks are deleted successfully.
  5. Go to the Management account, select the stack in the CloudFormation console and click Delete. Wait until status is DELETE_COMPLETE for all stack resources.

This will delete all resources including

  1. AWS EventBridge rules
  2. AWS Lambda functions along with their configuration
  3. AWS Lambda custom resource which invokes the Lambda function and handles the response
  4. AWS IAM Roles in the Management account and delegated administrator account including cross-account role and custom resource roles

Conclusion

Tagging inconsistencies limit the insights gained from your cloud financial management solution. This post showed how you can automate account-level tag associations for resources in your AWS accounts. We used AWS Service Catalog portfolios to manage resource configurations. The Service Catalog TagOption library made it easy to enforce a consistent tagging strategy across all provisioned resources. As the number of AWS resources in your organization grows, we recommend you evaluate an automated approach such as the one presented in this post instead of shifting the maintenance effort to your end users.

Rostislav Markov

Rostislav Markov

Rostislav is principal architect with AWS Professional Services. As technical leader in AWS Industries, he works with AWS customers and partners on their cloud transformation programs. Outside of work, he enjoys spending time with his family outdoors, playing tennis, and skiing.

Shydhanya Dharmalingaraja

Shydhanya Dharmalingaraja

Shylu is an AWS Professional Services DevOps Consultant from Connecticut. She enjoys to explore and handle challenging solutions. Continuous learner and believe in “Never stop learning, as life never stops teaching” – Buddha. Outside of office work, she likes to workout, dance, and enjoys driving with family.