For those that know me, they know I have been a huge fan of HashiCorp Nomad for the past year or so. I have been very open with my thoughts that I don’t believe that sweetspot of the tool is for scheduling containers. I believe the ship has sailed in the, very fashionable, direction of Kubernetes for container scheduling. I have been using Nomad in a similar vein to how others are supervising their services with tooling like systemd.
Following my pattern of building AMIs for applications, I create my Apache Zookeeper cluster with Packer for my AMI and Terraform for the infrastructure. This Zookeeper cluster is auto-discovering of the other nodes that are determined to be in the cluster
###Building Zookeeper AMIs with Packer
The packer template looks as follows:
{ "variables": { "ami_id": "", "private_subnet_id": "", "security_group_id": "", "packer_build_number": "", }, "description": "Zookeeper Image", "builders": [ { "ami_name": "zookeeper-{{user `packer_build_number`}}", "availability_zone": "eu-west-1a", "iam_instance_profile": "app-server", "instance_type": "t2.
The instances that run in my infrastructure get a lifespan of 14 days. This allows me to continually test that I can replace my environment at any point. People always ask me if I follow the same principal for data nodes. I posted previously about replacing nodes is an ElasticSearch cluster, this post will detail how I replace nodes in a Riak cluster
NOTE: This post assumes that you have the Riak Control console enabled for Riak.
Following my pattern of building AMIs for applications, I create my riak cluster with Packer for my AMI and Terraform for the infrastructure
###Building Riak AMIs with Packer
{% raw %}
{ "variables": { "ami_id": "", "private_subnet_id": "", "security_group_id": "", "packer_build_number": "", }, "description": "Riak Image", "builders": [ { "ami_name": "riak-{{user `packer_build_number`}}", "availability_zone": "eu-west-1a", "iam_instance_profile": "app-server", "instance_type": "t2.small", "region": "eu-west-1", "run_tags": { "role": "packer" }, "security_group_ids": [ "{{user `security_group_id`}}" ], "source_ami": "{{user `ami_id`}}", "ssh_timeout": "10m", "ssh_username": "ubuntu", "subnet_id": "{{user `private_subnet_id`}}", "tags": { "Name": "riak-packer-image" }, "type": "amazon-ebs" } ], "provisioners": [ { "type": "shell", "inline": [ "sleep 10" ] }, { "type": "shell", "script": "install_dependencies.
In a previous post, I talked about how I have tended towards the philosophy of ‘Immutable Infrastructure’. As part of that philospohy, when a box is created in my environment, it has a lifespan of 14 days. On the 14th day, I get a notification to tell me that the box is due for renewal. When it comes to ElasticSearch nodes, there is a process I follow to renew a box.
In my last post, I described how I use Packer and Terraform to deploy an ElasticSearch cluster. In order to make the logs stored in ElasticSearch searchable, I use Kibana. I follow the previous pattern and deploy Kibana using Packer to build an AMI and then create the infrastructure using Terraform. The Packer template has already taken into account that I want to use nginx as a proxy.
###Building Kibana AMIs with Packer and Ansible
As discussed in a previous post, I like to build separate AMIs for each of my systems. This allows me to scale up and recycle nodes easily. I have been doing this with ElasticSearch for a while now. I usually build an AMI with Packer and Ansible and I use Terraform to roll out the infrastructure
###Building ElasticSearch AMIs with Packer
The packer template looks as follows:
{% raw %}
I use Autoscaling Groups in AWS for all of my systems. The main benefit for me here was to make sure that when a node died in AWS, the Autoscaling Group policy made sure that the node was replaced. I wanted to get some visibility of when the Autoscaling Group was launching and terminating nodes and decided that posting notifications to Slack would be a good way of getting this. With Terraform and AWS Lambda, I was able to make this happen.
I’ve long been a configuration management tool fan. I have blogged, spoken at conferences and used Puppet as well as Chef and Ansible. The more I use these tools now, the more I realise I’m actually not making my life any easier
Currently, the infrastructure I manage is 100% AWS Cloud based. This has actually changed how I work:
I have learned to always expect problems so I therefore should have everything 100% automated.
So I just tweeted the following,
Firstly, I’d like to say that this is not about naming and shaming. Secondly, I am not annoyed with the conference at all about the response. The conference I spoke to advertises itself as “engineering talks only” so I wanted to post a few things about that.
In my opinion, the writing of code and the ecosystem of a specific platform is only 10% (or rather a small portion) of what we need to be aware of as software engineers.