AWS Open Source Blog
Integrating EC2 macOS workers with EKS and GitLab
At our annual re:Invent conference in December 2020 we announced an all new macOS-based Amazon Elastic Compute Cloud (Amazon EC2) instance. This new instance allows developers to build, test, and package their applications for all Apple platforms, such as macOS, iOS, iPadOS, tvOS, and watchOS. Customers have been asking us for ways to integrate their native build tools, such as Jenkins and GitLab, with this new instance type. We published a blog post in late December that walks through integrating Jenkins with the AWS Cloud Development Kit (AWS CDK).
In this post, we will walk through configuring GitLab with a mac1.metal Amazon EC2 instance using another open source infrastructure as code (IaC) automation tool, Pulumi.
Deployment overview
We’re seeing a new infrastructure automation trend emerging. Historically, developers have been using AWS CloudFormation or Terraform to automate AWS infrastructure deployments. Both of these tools have their respective strengths, but they require learning their own domain-specific language or writing “code” in YAML/JSON. I don’t know about you, but I prefer to write my automation using languages I’m already familiar with, such as Python or JavaScript/TypeScript. The trend we’re seeing is that developers are now using their programming language of choice with tools such as Pulumi or AWS Cloud Development Kit to build infrastructure.
Pulumi is an AWS partner that allows developers to build their cloud-native infrastructure using a familiar programming language. In addition to building infrastructure, Pulumi also has providers for integrating common tooling, such as Datadog or PagerDuty, via code.
For this post, we’ll use Pulumi’s native AWS resources to:
- Create a new Amazon Virtual Private Cloud (Amazon VPC) specifically for this build environment.
- Create a new Amazon Elastic Kubernetes Service (Amazon EKS) cluster to manage builds.
- Create an Amazon EC2 macOS instance that automatically becomes a GitLab worker.
- Create an Amazon S3 bucket that is following a least privilege security model so that only the worker can access the bucket to store build artifacts.
- Create an Amazon Route 53 public hosted zone for a domain that will direct traffic to our GitLab instance fronted by an Elastic Load Balancer (ELB).
We’ll also use the Kubernetes package manager, Helm, to deploy GitLab to our Amazon EKS cluster. The entire process for getting everything set up will take about 45 minutes. Please note that this is an example and should be used as a guide for your site reliability engineering teams to develop their own production deployment.
Let’s get started!
Requirements
To deploy this environment, you will need to install Pulumi and have the AWS CLI installed and configured. Additionally, ensure that you have kubectl installed to work with your EKS cluster. Also, make sure the Kubernetes package manager, Helm, is installed.
Note: This project will be using Route 53 to host DNS for a domain we’re using with GitLab. If you don’t have an existing GitLab server, please have a domain that you want to use (this is required for TLS certificate generation).
Cloning the repo
First, clone the code from GitHub to get the project started.
bash$ git clone https://github.com/aws-samples/eks-mac1-gitlab.git
bash$ cd eks-mac1-gitlab/Provisioning EKS, GitLab, macOS, Route 53, and an S3 Bucket
Now that we have configured the AWS CLI and generated an SSH key, we are ready to begin launching the build environment.
We will deploy multiple Pulumi stacks. Let’s take a quick look at what stacks are available in the repo.
bash$ ls
eks-vpc
mac-worker
r53-conf
s3-iamIn this example, we will use the AWS region Oregon or us-west-2. To use a different region, update the Pulumi configuration parameters in each stack.
bash$ cd eks-vpc
bash$ pulumi config set aws:region us-west-2
bash$ cd ../mac-worker
bash$ pulumi config set aws:region us-west-2
bash$ cd ../r53-conf
bash$ pulumi config set aws:region us-west-2
bash$ cd ../s3-iam
bash$ pulumi config set aws:region us-west-2Creating an S3 Bucket for artifacts
We want to have an S3 bucket to store artifacts from our GitLab runners. We also want to lock this bucket down so there is no public access; we only want the runners themselves to be able to read and write to this bucket. This is a best practice that follows a least privilege security model.
bash$ cd s3-iam
bash$ npm i
bash$ pulumi up -yCreating a new VPC and EKS cluster
Next, let’s deploy a new Amazon VPC and an Amazon EKS cluster. Provisioning an EKS cluster takes about 20 minutes.
bash$ cd ../eks-vpc
bash$ npm i
bash$ pulumi up -y
bash$ pulumi stack output kubeconfig > kubeconfig
bash$ export KUBECONFIG=`pwd`/kubeconfigNext, let’s run the aws eks update-kubeconfig and aws eks get-token commands to properly set up kubeconfig, or we can configure the KUBECONFIG environment variable as we have done in the previous command.
Installing GitLab with Helm
To install GitLab, update the internet-accessible domain (that you own) and email address in the following command:
bash$ helm install gitlab gitlab/gitlab \
--set global.hosts.domain=yourdomain.com \
--set certmanager-issuer.email=admin@yourdomain.comThe installation takes about five minutes to complete, and we can check the status with the following command:
bash$ kubectl get pods
NAME READY STATUS RESTARTS AGE
gitlab-cainjector-67dbdcc896-wlxbl 1/1 Running 0 47s
gitlab-cert-manager-564fc9d7f5-ctdqw 1/1 Running 0 46s
gitlab-gitaly-0 1/1 Running 0 46s
gitlab-gitlab-exporter-5c4b45cff8-hw7bx 1/1 Running 0 47s
gitlab-gitlab-runner-6bbf8c5fcf-9dqmb 0/1 Running 0 47s
gitlab-gitlab-shell-df87cf6c-v5rrv 1/1 Running 0 32s
gitlab-gitlab-shell-df87cf6c-whpvt 1/1 Running 0 47s
gitlab-issuer-1-4htcq 0/1 Completed 0 47s
gitlab-migrations-1-92h6b 1/1 Running 0 47s
gitlab-minio-56667f8cb4-cglkv 1/1 Running 0 47s
gitlab-minio-create-buckets-1-5lvf4 0/1 Completed 0 47s
gitlab-nginx-ingress-controller-5d475855bb-df2pj 1/1 Running 0 46s
gitlab-nginx-ingress-controller-5d475855bb-hbvxj 1/1 Running 0 46s
gitlab-nginx-ingress-default-backend-658cc89589-zhmk8 1/1 Running 0 47s
gitlab-postgresql-0 2/2 Running 0 46s
gitlab-prometheus-server-768cd8f69-fv828 1/2 Running 0 46s
gitlab-redis-master-0 2/2 Running 0 46s
gitlab-registry-9ddb975d6-frg58 1/1 Running 0 47s
gitlab-registry-9ddb975d6-tpglb 1/1 Running 0 47s
gitlab-sidekiq-all-in-1-v1-5b57f74dc4-swstj 0/1 Init:2/3 0 46s
gitlab-task-runner-95499488b-q48db 1/1 Running 0 47s
gitlab-webservice-default-565765b8c-8qfwf 0/2 Init:2/3 0 46s
gitlab-webservice-default-565765b8c-zgkk9 0/2 Init:2/3 0 47sWhen all of the Kubernetes pods have transitioned to Running or Completed, the installation is done.
Next, we need to get the Elastic Load Balancer hostname. Copy this hostname because we will use it to configure Route 53.
bash$ kubectl get ingress/gitlab-webservice-default -ojsonpath='{.status.loadBalancer.ingress[0].hostname}'
123456790281923943823-8675309---us-west-2---elb.amazonaws.com.rproxy.goskope.comConfiguring Route 53 to use your domain
Next, we will configure Route 53. First, we are going to create a new hosted zone for your domain. Second, we will create a Route 53 ALIAS DNS record entry that maps gitlab.yourdomain.com (or whatever your actual domain name is) to the load balancer that was previously configured via Helm.
bash$ cd ../r53-conf
bash$ npm i
bash$ pulumi config set elb_hostname 123456790281923943823-8675309.us-west-2.elb.amazonaws.com
bash$ pulumi config set domain_name yourdomain.com
bash$ pulumi up -yNote the outputs that Pulumi generates:
domain_name : "yourdomain.com"
elb_hostname: "123456790281923943823-8675309.us-west-2.elb.amazonaws.com"
ns1 : "ns-1463.awsdns-55.org"
ns2 : "ns-1996.awsdns-58.co.uk"
ns3 : "ns-296.awsdns-38.com"
ns4 : "ns-790.awsdns-32.net"You will need to update your domain’s DNS to use Route 53’s nameservers. If your domain is hosted by GoDaddy, an example of how to update your nameservers can be found in the blog post Domain by GoDaddy, DNS by Route53.
Note that when you update the nameservers and configure Route 53, DNS propagation can take up to 15 minutes.
Logging into GitLab
Now that we have updated Route 53, let’s log in to GitLab. The GitLab Helm chart has created a Kubernetes secret containing the root password to log in. Use the following command to capture the password:
bash$ kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo
1234567890abcdefghijklmnopqrstuvwxyzIn a web browser, navigate to https://gitlab.yourdomain.com (or whatever domain you chose). The login username is root and use the password from the kubectl get secret command output.
Now that we have logged in, we want to get the runner registration token to add Amazon EC2 workers to GitLab.
Select the wrench icon (Admin Area), found on the top navigation bar.
Next, select Runners.
Copy the registration token, which we will use for our Amazon EC2 macOS instance configuration.
Creating an EC2 macOS GitLab runner
We need to get the current organization name that we configured when installing Pulumi. To do this, run the following command and set a configuration parameter. We are setting this parameter because we are referencing outputs from our Amazon EKS, S3, and Route 53 stacks.
bash$ cd ../mac-worker
bash$ npm i
bash$ pulumi whoami
my-org-name
bash$ pulumi config set pulumi_org my-org-nameNext, create an SSH key in the AWS console if you don’t already have one. Name it pulumi-key, for example.
We now will use the registration token for the GitLab runner. We also will use the SSH key we created in the console in the first step to allow secure logins. An inbound EC2 security group is also automatically provisioned, which only allows port 22 inbound access.
bash$ cd ../mac-worker
bash$ pulumi config set --secret reg_token 123456789-my-token-goes-here-987654321
bash$ pulumi config set ssh_keyname pulumi-key
bash$ pulumi up -yNote that it takes about 15 minutes to create and configure the macOS GitLab runner.
After about 15 minutes, we can log back into our GitLab instance (https://gitlab.yourdomain.com/admin/runners) and validate that the runner has successfully registered.
Now that we have successfully added a runner, we need to create a project to kick off any automated builds. Our GitLab project will require a .gitlab-ci.yml file that will tell the runner what to do when code has been committed to the repo.
Conclusion
Configuring GitLab in a reliable way can be challenging. Using automation tooling, such as Pulumi, can reduce the time it takes to build and deploy complex infrastructure and application environments. This post has provided a guide on how to configure GitLab with our new mac1.metal instance. If you run into any problems, feel free to open a GitHub issue and we’ll respond. Happy building!