Blogs

Running Cockroach DB in Production with Nomad

For those that know me, they know I have been a huge fan of HashiCorp Nomad for the past year or so. I have been very open with my thoughts that I don’t believe that sweetspot of the tool is for scheduling containers. I believe the ship has sailed in the, very fashionable, direction of Kubernetes for container scheduling. I have been using Nomad in a similar vein to how others are supervising their services with tooling like systemd.

Building an Autodiscovering Apache Zookeeper Cluster in AWS using Packer, Ansible and Terraform

Following my pattern of building AMIs for applications, I create my Apache Zookeeper cluster with Packer for my AMI and Terraform for the infrastructure. This Zookeeper cluster is auto-discovering of the other nodes that are determined to be in the cluster ###Building Zookeeper AMIs with Packer The packer template looks as follows: { "variables": { "ami_id": "", "private_subnet_id": "", "security_group_id": "", "packer_build_number": "", }, "description": "Zookeeper Image", "builders": [ { "ami_name": "zookeeper-{{user `packer_build_number`}}", "availability_zone": "eu-west-1a", "iam_instance_profile": "app-server", "instance_type": "t2.

Replacing a node in a Riak Cluster

The instances that run in my infrastructure get a lifespan of 14 days. This allows me to continually test that I can replace my environment at any point. People always ask me if I follow the same principal for data nodes. I posted previously about replacing nodes is an ElasticSearch cluster, this post will detail how I replace nodes in a Riak cluster NOTE: This post assumes that you have the Riak Control console enabled for Riak.

Building a Riak Cluster in AWS with Packer and Terraform

Following my pattern of building AMIs for applications, I create my riak cluster with Packer for my AMI and Terraform for the infrastructure ###Building Riak AMIs with Packer {% raw %} { "variables": { "ami_id": "", "private_subnet_id": "", "security_group_id": "", "packer_build_number": "", }, "description": "Riak Image", "builders": [ { "ami_name": "riak-{{user `packer_build_number`}}", "availability_zone": "eu-west-1a", "iam_instance_profile": "app-server", "instance_type": "t2.small", "region": "eu-west-1", "run_tags": { "role": "packer" }, "security_group_ids": [ "{{user `security_group_id`}}" ], "source_ami": "{{user `ami_id`}}", "ssh_timeout": "10m", "ssh_username": "ubuntu", "subnet_id": "{{user `private_subnet_id`}}", "tags": { "Name": "riak-packer-image" }, "type": "amazon-ebs" } ], "provisioners": [ { "type": "shell", "inline": [ "sleep 10" ] }, { "type": "shell", "script": "install_dependencies.

Replacing the Nodes in an AWS ElasticSearch Cluster

In a previous post, I talked about how I have tended towards the philosophy of ‘Immutable Infrastructure’. As part of that philospohy, when a box is created in my environment, it has a lifespan of 14 days. On the 14th day, I get a notification to tell me that the box is due for renewal. When it comes to ElasticSearch nodes, there is a process I follow to renew a box.

Deploying Kibana Using Nginx as an SSL Proxy

In my last post, I described how I use Packer and Terraform to deploy an ElasticSearch cluster. In order to make the logs stored in ElasticSearch searchable, I use Kibana. I follow the previous pattern and deploy Kibana using Packer to build an AMI and then create the infrastructure using Terraform. The Packer template has already taken into account that I want to use nginx as a proxy. ###Building Kibana AMIs with Packer and Ansible

Building an ElasticSearch cluster in AWS with Packer and Terraform

As discussed in a previous post, I like to build separate AMIs for each of my systems. This allows me to scale up and recycle nodes easily. I have been doing this with ElasticSearch for a while now. I usually build an AMI with Packer and Ansible and I use Terraform to roll out the infrastructure ###Building ElasticSearch AMIs with Packer The packer template looks as follows: {% raw %}

Autoscaling Group Notifications with Terraform and AWS Lambda

I use Autoscaling Groups in AWS for all of my systems. The main benefit for me here was to make sure that when a node died in AWS, the Autoscaling Group policy made sure that the node was replaced. I wanted to get some visibility of when the Autoscaling Group was launching and terminating nodes and decided that posting notifications to Slack would be a good way of getting this. With Terraform and AWS Lambda, I was able to make this happen.

The Quest for Infrastructure Management 2.0

I’ve long been a configuration management tool fan. I have blogged, spoken at conferences and used Puppet as well as Chef and Ansible. The more I use these tools now, the more I realise I’m actually not making my life any easier Currently, the infrastructure I manage is 100% AWS Cloud based. This has actually changed how I work: I have learned to always expect problems so I therefore should have everything 100% automated.

DevOps and .Net Conference

So I just tweeted the following, Firstly, I’d like to say that this is not about naming and shaming. Secondly, I am not annoyed with the conference at all about the response. The conference I spoke to advertises itself as “engineering talks only” so I wanted to post a few things about that. In my opinion, the writing of code and the ecosystem of a specific platform is only 10% (or rather a small portion) of what we need to be aware of as software engineers.