Reduce project delays with a docs-as-code solution

Are your projects often delayed because of documentation? The code is complete, testing checks have all passed, and key stakeholders have approved the project for the official launch. Everything is in order for the big day, except for one piece—the documentation. With no estimated completion time in sight, you have no choice but to wait for the writing team to finalize and publish the content.

Unfortunately, this scenario plays out over and over again when the tools and processes used by writing teams aren’t integrated with those used by development teams. In a typical workflow, developers finish building the code in their own version control systems and then pass the documentation to a team of writers. The writers use their own tools, workflows, and pipelines to review and edit the content, convert it into a new format, and then publish it to the company site.

Despite being part of the same project, the developers and writers work independently using siloed processes to complete their tasks, while stakeholders observe from the sidelines, hoping that the two efforts sync up in time for the official launch.

In this post, we explain how the Amazon Web Services (AWS) Quick Start team solved our documentation delays by using a documentation-as-code (docs-as-code) solution based on GitHub and AsciiDoc. We take you through the initial challenges that led to the changes followed by the new solution and its benefits. Finally, we provide a set of steps that you can follow to explore the source code of our solution and decide whether it might be a potential fit for your organization.

About the AWS Quick Start program

Launched in 2016, the AWS Quick Start program provided partners with a secure environment for building and delivering production-ready, automated reference deployments of their products on the AWS Cloud. Partners collaborated with AWS solutions architects to develop code and assets using a variety of components, including AWS CloudFormation templates, Bash and PowerShell scripts, AWS Lambda functions, and proprietary installation media—all bundled together in open-source GitHub repositories in the AWS Quick Start GitHub organization.

Figure 1 shows an example of a typical Quick Start architecture.

Figure 1: Quick Start architecture diagram

Each Quick Start provided customers with a detailed deployment guide with step-by-step instructions for configuring parameters, deploying to the cloud, and troubleshooting errors. Originally, Microsoft Word was the authoring application of choice due to its support for custom templates, built-in tracking features, and wide availability both within and outside of AWS. The architects and partners passed these files through a series of content reviews before they eventually handed the files off to a small team of technical writers for a final edit and publication.

Documentation becomes a dreaded bottleneck

As the Quick Start program gained momentum throughout the partner community, the catalog of reference deployments steadily grew into the hundreds with an average of 250,000 deployments annually. However, despite the positive reactions from partners and AWS leadership, the technical writing team quickly lost pace with the demands of the program, often causing delays in publishing. These delays were especially frustrating to partners who were trying to coordinate dates with a company announcement or event.

Tooling and processes create headaches

With the explosion in popularity of the Quick Start program, the processes and tools used to create and maintain the deployment guides in the early days quickly became the team’s biggest obstacle to publishing on time. Partners and architects continued to pass Microsoft Word files back and forth as they worked through multiple iterations of feedback and reviews. Documents with various file names (for example, document-v1.docx, document-v2.docx, and document-v2-bob-edit.docx) eventually piled up in inboxes, leading to bloated file directories and a confusing mix of current and obsolete review comments.

By the time the Microsoft Word files were passed to the team of technical writers for formal editing, the documents almost always contained formatting issues. These issues were typically caused by a lack of consistency across contributors combined with conflicting conventions and settings used in operating systems and word processing programs such as Microsoft Word, OpenOffice, and LibreOffice. According to the Quick Start writers, they spent an average of 6-8 hours on each deployment guide addressing formatting issues before they even started reviewing the text.

Publishing multiple formats is time-consuming

When the Microsoft Word files were finally edited and formatting issues resolved, the publishing process began. First, the writers had to convert the files to both PDF and HTML format. Next, they published the converted files to multiple locations. The PDFs and images were published both to Amazon Simple Storage Service (Amazon S3) and an internal GitHub repository, and the HTML files were published to the AWS public-facing documentation site (AWS Docs). Even one small change to the content meant repeating the entire process of converting, storing, and publishing the files to all three locations.

To add even more complexity, deployment guides often went through revisions due to new software versions of partner products, AWS service updates, and bug fixes. The same confusion of mismatched documents, formatting issues, and publishing woes that plagued the initial launch of the Quick Start continued any time the guide needed an update. Even minor updates to a deployment guide could take weeks to complete.

The Quick Start team knew they needed to solve these problems quickly and permanently if they wanted to continue supporting partners and their customers. They needed a scalable and efficient solution that would keep projects moving forward and on schedule. At a minimum, they needed new tooling and processes to help simplify and streamline the end-to-end workflow of creating, updating, publishing, and maintaining deployment guides.

A new docs-as-code solution

After weeks of gathering data, the Quick Start team finally decided upon a docs-as-code solution. Internally referred to as Docs 2.0, this new model was a natural choice for the team, especially considering that they were already collaborating with partners on Quick Start code using GitHub, a well-known and widely used version control system. If GitHub was well suited for storing templates, scripts, source code, submodules, and other code artifacts in plain text, then surely it would suffice for collaborating on and storing text files and related image artifacts such as screenshots, AWS logos, and partner logos.

After weighing the pros and cons between markdown and AsciiDoc, the team decided upon AsciiDoc as the authoring format due to its powerful and standardized toolset (see Figure 2). With the docs-as-code solution, architects, partners, and writers use well-known editing software such as Visual Studio Code to edit the content and commit their changes to a dedicated, versioned GitHub repository. Here, they collaborate, review, and approve, and eventually merge content into a gh-pages branch dedicated to documentation. All changes are now tracked to an author and timestamp.

Figure 2: AsciiDoc file in Visual Studio Code

To keep the structure and formatting of the deployment guides consistent across projects, architects prepopulate each documentation repository with a set of AsciiDoc files, images, style sheets, and index files at the start of each project. These files control the structure and table of contents of each guide. Boilerplate text and images that are common across all deployment guides are single-sourced and maintained in a separate, non-editable submodule that’s pulled in dynamically when the guides are built and converted to HTML.

The HTML output of the deployment guides is published directly out of the repository using GitHub Pages, GitHub’s hosting service for building and publishing websites (see Figure 3). Using simple commands, writers can build the guides locally on their machines and publish to a staging site to check for UI issues such as broken links and images before publishing to the live server. Now, writers could push either new or updated content live into production with only a few steps.

Figure 3: Published deployment guide

With the docs-as-code solution, the days of juggling multiple files, sifting through obsolete content, and managing siloed feedback are over. Documentation is stored in a single repository, and content reviews between AWS and partners are organized and trackable. Using the open-source tool AsciiDoctor, the published HTML is well-structured, accessible, and free of formatting issues. According to the writers on the Quick Start team, eliminating formatting issues saved them approximately 50% of editing time, a massive improvement over the previous process.

Docs-as-code gives the architects more time to spend with partners on the design side of their projects, and writers have more time to propose and implement improvements in the overall process. Writers can also revisit previously published deployment guides and migrate them to the new process. Docs-as-code sets writers up for success when they have to update documentation down the road. Managers are thrilled, because they no longer need to request more resources and train additional employees.

Explore for yourself

With so many improvements and positive feedback from partners, the Quick Start team transitioned its docs-as-code framework to an open-source model and made it available to the wider community both within and outside of AWS.

To implement a docs-as-code solution in your environment, complete the following tasks:

1. Navigate to the docs-as-code boilerplate repository we’ve provided. No credentials are required.
2. Clone an existing GitHub repository that you can use to explore and experiment with the boilerplate code.
3. In your cloned repository, run the following commands to add the boilerplate repository as a submodule to your environment:

   git submodule add https://github.com/aws-ia/aws-ia-documentation-base-common.git docs/boilerplate

   Now you should see a new docs/boilerplate/ directory with content from the boilerplate repository in your cloned environment.

4. To build the folder structure for a CloudFormation-based deployment guide, run the following command:

   docs/boilerplate/.utils/create_repo_structure.sh -d -c

   The script also contains options for other types of documents such as migration and operational guides.

5. With AsciiDoctor installed, run the following command to generate the HTML version of the deployment guide:

   docs/boilerplate/.utils/build_docs.sh

6. In GitHub, choose Settings,pages to allow GitHub Pages to associate the HTML file with your repository and add it to the root of your repository in the main branch (or another branch that you designate).

Conclusion

The AWS Quick Start team saw massive improvements with their docs-as-code process for building and publishing deployment guides. By integrating workflows and toolsets, they simplified the content review cycles and streamlined the publishing process. The end result was a scalable solution that eliminated obsolete processes, complicated workflows, and inefficient publishing pipelines. The solution gave the writing team a scalable, integrated solution that allowed them to complete work requests on time, even during times of heavy workloads.

Next steps

Check out the Taskcat CloudFormation testing application’s documentation site for an example of a guide published using the AWS Quick Start team’s docs-as-code process. You can find the code for this publication in the main branch of the Taskcat GitHub repository.

About the authors