By in large there are three paths you can take to build your platforms, you can combine 2 and 3 and the first is not recommended for any production level workloads.
- AWS Web Console has many wizards available crafted to make it easy building the different pieces of your platform, unfortunately, there is no way to ensure your configuration state which means if you want to create the same platform on another environment/account the two will be different, only recommended for very simple and quick setups don’t use for workloads which must go into production.
- AWS CLI tools allow you to script platform build and excellent to pull information from your AWS account, you can use these tools for automation but it will require you to “invent the wheel” compared to option 3, there is a place to use them for your own application code or very “tailored” automation.
- AWSCli v1/2 command-line tool available for most OS’s.
- AWS SDK’s to integrate in your application code etc.
- AWS Infrastructure-As-Code (IaC) and SDKs will allow you to build the platform with code that will allow you to manage state and achieve automation, we will be focussing on this category because it allows for operational state management and automation, some of these tools are:
- AWS Clouformation/CDK is maintained by AWS and can only be used on AWS.
- Hashicorp Terraform/CDK is a third party tool widely used because of its extensibility, ex. you can use Terraform for AWS or other clouds.
- Ansible; Chef; Puppet are today used more in legacy systems, more focussed on maintaining state for legacy applications.
- Gruntwork Terragrunt is actually a wrapper on Terraform extending its capabilities formatting and testing.
You must go with option 3 (and 2) for your AWS implementation, if you decide to go with option 1 or 2 alone your project might advance faster initially but the technical debt will “pile-up” and slowly but surely slow you down which will cost you time and as a consequence money.
Bottom line, start your AWS platform build and account setup using code and manually configure only the most basic and necessary tasks, for example, creating your first AWS account.
Now that you are on the path of using code you also have to store the code in a way it is possible to share author version and develop in parallel, and, this is why using an SCM (Software Configuration Management) platform such as Github; GitLab; Bitbucket etc is crucial to provide these and more features for your code development success.
After you have set up your SCM solution we are on the path of implementing a CI/CD or GitOps strategy to automate the management and release of your platforms on AWS.
Some advice from field experience:
- Never store secrets or potentially compromising configurations in code, only the references to secrets or configuration stored on different systems, here’s why:
- Code will be frequently accessed/changed by many developers (some even temporary) and storing these values in code will expose you to security compromises and exploits if not handled properly, some possible alternatives are AWS Secrets Manager; AWS SSM Parameter Store or a 3rd party such as Hashicorp Vault for secret and configuration authoring and versioning.
- Best practices monitoring reporting and enforcement through automation, you can now set up scanning and verification of the code even before the code is committed to the main branches, or, applied/deployed to your platform, this will work as an early detection system preventing issues being detected too late.
- Whatever SCM platform you chose, if you go for the Hosted (SaaS) option always create an organisation account, users change but your organisation will stay the same.
- Filter access control at the group level and assign the groups to projects or even to the repository itself.
- Guard the branches that trigger any automation by setting rules on how you can commit to those branches and who approves changes to those branches, these rules you set up will protect you from committing the wrong code or configurations and allow for proper review from peers.
Getting the right strategy and processes to use to develop your platform’s code is the critical step on your AWS journey success, we’ve seen too many projects trying to catch up and taking a long time to delivery specifically because they didn’t get this step right from the start.
In Part 1 we presented the AWS Organisation with multiple accounts model and here the SCM and CI/CD or GitOps enabled by a good SCM implementation will enable the automation and management of the multiple accounts and environments through code that can be re-used and redeployed to different accounts or environments, we can’t emphasize enough how crucial it is to get this right from day 1.
