Friday, 21 November 2014

Version control: the essentials in five minutes

So, everyone keeps talking to me about this version control thing. What's the beef?

Version control is really just keeping a history of all the changes you made in a set of documents.

Let's imaging you're working on a project right now, and all the code and configs are stored in a directory. The basic version functions, but you want to add more features. You make some changes, and now things are broken.

However, your directory is under version control, so you can compare what the code looks like now with what the code looked like when it last worked, and see what you changed.
If you want you can just revert to a previous version, undoing as many stages as you need.

Right now I just copy myfile.txt to myfile.bak, that lets me keep as many versions of my document as I like. And we have daily backups, so I won't lose anything.

And what happens when you make the next version? myfile.bak.2?


And in three years' time when you have made 1,463 changes to the 14 files that make up your project and you need to know when you introduced the bug you just found?

Sometimes I put the date after .bak...

And how do you communicate these changes and what they mean to other members of your team? Especially if they work remotely?

Post-it notes?

They might kill you after post-it note #1,462.

Ok so tell me a better way.

If you keep your project under version control, then every time you reach a moment where you want to record the exact state you can 'commit' the current state of all the tracked files.
The commit will have a unique identifier, and with a nice comment so you can browse through your history and see the project evolution, and so that your collaborators in a different timezone can see what you changed.

How do I share this project with them?

Usually you'd have a copy of the project on a central repository that everyone can access.

Each team member takes a copy, works on it locally, and then uploads their changes back to the central repo when they're ready.

What happens if two people change the same file at the same time?

Then you'll get a conflict, which might be automatically resolved if the changes don't overlap, or might need you to help it decide which revisions are kept.

Sounds complicated.

It doesn't have to be. Projects can get more complicated if there are lots of contributors and lots of interdependent files, but for a simple project - maybe just a document or a couple of scripts in the beginning - it's no more complicated than the revision history feature of many word processing tools.

How about if I have a working piece of software and I want to try out adding a new feature?

Then you probably want to branch the project, where the working software continues along the track it was on, and you go off and work on your branch. When the feature is ready, you can merge the two branches together again.

I hear a lot of chatter about which software I ought to use, but all these three-letter names mean nothing to me.

In the beginning, there was CVS. Then SVN more or less replaced it by making a better version of the concept. Then git turned up and did everything completely differently. Right now, git and SVN have comparable market shares, but git's popularity is accelerating.

SVN (like CVS) uses a client-server model, where the central server houses the 'definitive' repository, and your laptop is merely a client. Every time a developer makes a change they want to save, this must be written back to the central server, and then updated on
each collaborator's computer.

You can use the same workflow in git, but you don't have to. Git is technically a distributed-model system, which means that everywhere the project lives is a fully-formed repository: your laptop, your colleague's pc, the central git server.

Why would I want my own repository on my laptop?

Well, say you're working on your own branch of a project and you go away for a few days to a conference, and the internet is so bad you can't reach the central repository. 
If you're using git you can continue to work on your code offline, commit your changes to your local repo, and when you get back to the office next week you can upload everything you've done back to the central storage area.

The flexibile workflow and clever branching and merging features that git offers are some of the reasons it's become so popular. That
 and Github.


You know, where all the cool kids put their open source stuff.

I'll have to check that out...

Do. Git has a bit of a reputation for having a steep learning curve, but if you're starting out with a straightforward project of a couple of files and users it's probably no harder than learning SVN. And frankly about a million times better than having folders full of .bak files. 

The sixty-four-thousand dollar question: will version control stop my blood running cold when I realise I've messed up all the configs and now everything is on fire?

* Probably.

Wednesday, 23 July 2014

Creating a CentOS base box for Vagrant

I've been using Vagrant for a while, but I recently decided to start making my own base boxes for various reasons (curiosity and paranoia mainly). My Debian base box pretty much 'just worked' thanks to some nice instructions here.

However my CentOS (6.5) base box proved a little trickier. This how I eventually got it working.

1. Make the VM in VirtualBox

  • In setup wizard
    • Name: whatever takes your fancy
    • RAM: we can set this small, say 512MB, and increase it later
    • Hard disk: VMDK, dynamically allocated, again start small, say 10GB
  • In settings
    • CPUs: give it a couple if you have the resources
    • Disable audio and USB
    • Network adapter 1 set to NAT, cable connected, 
    • Port forward SSH (TCP) from host port 2222 to guest port 22

2. Install the operating system

  • Download the latest iso and blast through the install. I didn't do anything fancy here.
  • The root password needs to be 'vagrant'
  • There also needs to be a user called 'vagrant' with the password 'vagrant'

3. Log in as root and update the packages

  • yum update
  • if the following are not already installed, yum install them:
    • sudo
    • ssh-server
    • curl
    • vim (if you are so inclined, then do update-alternatives --config editor)
    • ntp
  • Set sshd and ntp to auto-start on boot
    • chkconfig <servicename> on

4. Give vagrant user full access to all the things

Run visudo (as root):
  • add this line to give vagrant passwordless sudo:
    • vagrant ALL=(ALL) NOPASSWD:ALL
  • edit this line to allow sudo without tty duing vagrant up:
    • Defaults !requiretty
  • NB I read a lot of advice about just commenting this out, but that didn't work for me - I had to explicity set it to not require tty

5. Install Guest Additions

I did the initial install using the GUI, as that seems to make it a bit easier to get the Guest Additions to work / mount / run
  • Install the dependencies first:
    • yum --enablerepo rpmforge
    • yum install dkms kernel-devel kernel-headers linux-headers-server
    • yum groupinstall "Development Tools"
      • or you can try limiting this to just gcc and make
  • In the VM window, select Devices, Install Guest Additions to mount the virtual CD
  • Click on the icon and let it autorun (or mount and run if you're not using the GUI)
  • I got some errors to begin with; these were fixed by doing this patch for 6.5
    • cd /usr/src/kernels/<kernel_release>/include/drm/
    • ln -s /usr/include/drm/drm.h drm.h
    • ln -s /usr/include/drm/drm_sarea.h drm_sarea.h
    • ln -s /usr/include/drm/drm_mode.h drm_mode.h
    • ln -s /usr/include/drm/drm_fourcc.h drm_fourcc.h

6. Set up ssh access for the vagrant user

  • Download the standard vagrant public key
    • wget --no-check-certificate
  • Save it to .ssh/authorized_keys
  • Make sure the appropriate permissions are set
    • 700 for .ssh
    • 600 for .ssh/authorized_keys
  • And that the whole lot is owned by vagrant

7. Disable the GUI

  • In /etc/inittab, set run level to '3' (GUI is '5')

8. Networking

Right now my VM only has loopback and eth0 interfaces; this is fine. I want to set eth0 to get assigned an IP by DHCP, and I'll leave it as that for the base box. When I bring up a new vagrant instance I want to be able to assign a static IP on eth1.

This is where I started to run into difficulties. No matter what I set in my ifcfg-eth0/eth1 files, it was completely ignored, and I was unable to stop the VM bringing up eth1 as DHCP and auto-assigning an IP from the VirtualBox DHCP range (, since you ask).

I could probably have coped with the automatically assigned ip address, but it was a matter of principle, plus it annoyed me to have some of my VMs on different subnets.

What it turned out to be was, since I'd installed the OS using a GUI, CentOS had automagically set NetworkManager to 'on' and 'network' to 'off'. (chkconfig --list | grep etwork). This has the effect of disregarding anything set in /etc/sysconfig/network-scripts/.

To get things back to the way I like them, I did:

  • service NetworkManager stop
  • chkconfig NetworkManager off
  • service network on
  • chkconfig network on
  • /etc/sysconfig/network-scripts/
    • DEVICE=eth0
    • TYPE=Ethernet
    • ONBOOT=yes
    • BOOTPROTO=dhcp
And just to be on the safe side (I was sort of bored with fighting with networks by this point)
  • /etc/udev/rules.d/70-persistent-*
  • /etc/selinux/config 
    • SELINUX=permissive
  • chkconfig iptables off
  • chkconfig ip6tables off

9. Check stuff

  • Reboot
  • check things are on / off / persistent / running as appropriate
  • delete history if you like things tidy
  • shutdown the machine.

10. Package as a Vagrant box

Within the folder containing all the vbox files etc, package up your new VM as a Vagrant box:

  • vagrant package --base centos_6_64 --output
  • vagrant box add centos6

11. Make a test Vagrant VM

Now let's make a test Vagrant VM to ensure everything is working ok:

  • vagrant init
  • Vagrantfile:
# vi: set ft=ruby :
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| = "centos6" "private_network", ip: ""
        config.vm.hostname = "cirque-centos.vm"
  • vagrant up
  • vagrant ssh

All things being well, you should now be able to ssh into the box, and check things look ok.
In particular, check the output of ifconfig and be sure that lo, eth0 and eth1 are configured as required, and that the shared folder mounts correctly under /vagrant.


Thursday, 30 January 2014

Translating user security for PCI compliance into configs

As part of our becoming-PCI-compliant project, there are a list of requirements about user security that needed translating from legal-speak into practical actions and ways to implement these actions for our Linux servers. We're using Debian, your mileage may vary.

"The last 4 passwords must be different"

This is controlled by PAM using the (or module, which ships as default in Debian. In the config file
the line containing "" needs to include "remember-4", eg
password requisite obscure use_authtok try_first_pass sha512 remember=4

"Passwords must contain more than 7 characters, and must be a mixture of upper and lowercase letters, and numbers"

Passwords can be tested with the PAM module
On debian/ubuntu, this can be installed with
# apt-get install libpam-cracklib
and this will generate an entry in the config file found at /etc/pam.d/common-password along the lines of
password requisite retry=3 minlen=11 lcredit=1 ucredit=1 dcredit=1 ocredit=1
What we're interested in here is primarily minlen - but it's not exactly the minumum length of the password as you might expect. Rather, it's a total of the number of characters in the password, plus scores from lcredit, ucredit, dcredit and ocredit, where the parameters mean
  • lcredit
    • maximum credit allowed from required lower-case characters
  • ucredit
    • number of required upper-case characters
  • dcredit
    • number of required digits
  • ocredit
    • number of required other characters (non-alphanumeric)
So, if we have the requirement of 8 or more characters including numbers, and upper and lowercase letters, then we can set lcredit, ucredit, and dcredit all to 1, and require minlen = (8+1+1+1) = 11 to ensure this policy is enforced.

"Passwords must be changed every 45 days"

Aha, an easy one. In /etc/login.defs , set

"Accounts are made inactive 90 days after last login"

Also straightforwards. In /etc/default/useradd , set

 "Sessions timeout after 15 mins inactivity"

Within /etc/bash.bashrc , set
export TMOUT=900

"Account locks out for 30 mins after 6 failed login attempts"

For this one, we need to install fail2ban (apt-get install fail2ban), and then create a file at /etc/fail2ban/jail.local which contains the following:
bantime = 1800
maxretry =6
Remember to restart fail2ban after you've made the config change.


Saturday, 11 January 2014

On defensiveness

We've all been there: you spend hours thinking about the best way to solve a problem, days or weeks setting up the basics and getting something functional working, and then at the moment you demo the prototype to your peers they point out a fatal flaw that you'd somehow overlooked. You're naturally disappointed, having invested time and effort in your project, but you also find yourself angrily fighting your corner, entrenching yourself and not listening. A few hours or days later you have enough clearance from the exchange to look at it objectively and concede that they might have a point. So why did you get so defensive?

Defensiveness is a reaction to threat. If you're out for a drive in your car and someone else is acting like a bit of a lunatic, accelerating hard and beeping their horn, you're likely to perceive that as a threat, and will probably modify your own driving behaviour to try and stay safe. But when your colleague criticises your idea, you have a similarly defensive reaction because you perceive the criticism as a threat to your self-image.

Self-image - ego, if you like - is the way we see ourselves, and has a huge influence on our behaviour. Do we think we're smart, accomplished at our job, funny, a good friend, an accomplished musician or a genius programmer? Self-image is so important to us that we will - consciously or unconsciously - seek to protect it even at enormous cost. If you're having a bad day you might be more defensive than usual because adding to your stack of annoyances seems unbearable.

People working in technical areas like computing and science have a natural tendency to like hard facts and best practice. Sometimes there IS a "right way" of doing something, and that makes us feel secure and comfortable. But if there's a grey area we have to start evaluating the pros and cons of each idea, and maybe we'll have to choose a working but imperfect solution. And if the issue starts to incorporate the messy world of real people, then we start having to face the possibility that rather than an either/or diagnosis, and/both might be more appropriate.

And/both means that sometimes conflicting realities can exist and be equally valid. You think that comedian is hilarious, but your friend thinks she's rubbish. You think the cashier was rude to you, but he thinks he was polite and efficient. And when self-image is involved it's even harder to be objective. You think your idea is amazeballs but your colleague points out the problems with it. Now you have to decide what to do with their opinion: do you keep theirs and throw out your idea? Or dismiss theirs and get defensive? Or, do you try and hold both in your mind together, which is uncomfortable at the best of times and nigh on impossible if you feel threatened.

Reacting defensively might mean you come across uncooperative or arrogant, which tends not to go down terribly well with people. But more than that, defensive behaviour means a missed opportunity. Paraphrasing from [1],
a person may disidentify with or downplay the personal importance of domains in which they are failing to sustain their sense of self-worth, but in doing so they preclude the opportunity for improvement. 
"Work your weaknesses" is easier said than done, but before you can work them you need to acknowledge them. Getting defensive about an issue might not be something you can do anything about in the heat of the moment, but it does provide you with a huge flashing arrow pointing towards a self-belief you might not even realise you have. And then maybe next time, or the time after that, when your colleague shoots a hole in of utter genius you might be able to agree use their point to make your creation even better, and then go about your day.

[1] The psychology of self-defense, Sherman and Cohen