Right so we have our handy services Virtual Private Cloud (VPC) with access via OpenVPN (and the awesome Viscosity OSX VPN Client) now we need to start adding useful things into it.
(See this blog post for info)
For me the next step was looking at how we could automate deployments using our own tool chain, part of the reason we are looking at AWS is to get a bit more flexibility and also the benefits of greater automation. We’ve already had success using BitBucket -> Codeship -> Heroku as a work flow to make our code visible and available in readily shareable environment, and it took < 5 minutes to get it up and running ;-)
Certainly easier than expecting a non-developer to checkout the code and run a whole load of NPM / Bower / Gulp commands to review progress, so much easier to just send through a link using Slack (other IM services are available) to a server that always has the latest “working” code in place ;-)
But moving on we need to smarten things up a bit and be a bit more clever, to get things deploying correctly on Heroku we had to use a custom build back. This is because we are using Yeoman as the basis of the code and didn’t want to commit “built” files into the repo, but we had to move all of the “devDependencies” into just “dependencies” and then build in production, oh and also build in Codeship so that the tests could run. This delays the build going live and is duplicated work.
Really what we wanted and needed is to test, build, deploy (repeat). Hence rolling our own.
So now that we have a VPC with NAT and VPN access running in AWS it made sense to get Jenkins running in our services VPC as well.
Getting Jenkins installed was relatively painless, build a new instance in the VPC (using Amazon Linux) add in the Jenkins repo and install, check the service is setup to run at the correct run levels and finally start Jenkins. Sometimes stuff just works.
$ sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo $ sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key $ sudo yum install jenkins $ chkconfig --list jenkins $ sudo service jenkins start
Give it a few minutes and with a bit of luck you’ll have Jenkins installed. You’ll need to check your firewall rules for the EC2 instance to ensure you can access port 8080
Created a new ssh key for the jenkins user and did a quick test of a git clone to get Github added to ~/.ssh/known_hosts and ensure the ssh key is working.
$ sudo su jenkins $ cd ~/ $ ssh-keygen -t rsa -b 4096 -C "your_email@example.com" $ more ~/.ssh/id_rsa.pub ssh-rsa .... hey stuff in here which you need to copy and add into Github or SCM repo of choice. $ git clone git@bitbucket.org:team/repo.git test_clone
Add in some plugins:
- Bitbucket OAuth Plugin
- Bitbucket Plugin
- NodeJS Plugin
- (if the auto-installers aren’t available this will fix it)
- Poll SCM plugin (Jenkins is in a VPC and not accessible from Bitbucket so webhooks are no good)
- SCM Sync Configuration Plugin
- Slack Notification Plugin
- Wall Display Master Project
I had an issue with the build I was running whilst it was trying to pull down packages from GitHub, the error message was pointing to a timeout which was surprising however the clue was in the protocol being used:
git://github.com/....
So changing /etc/gitconfig to:
[url "https://github"]
insteadOf = git://github
Solved the issue, there is a chance I’ll have to change it for “git://” rather than just Github but at least now my build is able to pull in the dependancies and then fail further down the process.
Automation comes later.