Automated regression testing of JIRA stories

Screen Shot 2014-05-23 at 2.44.19 am


When a story is ‘done’ and ‘on the bus’, it’s expected to stay that way until go live. When working in a large team that share a code base, it’s common to break each others stuff.

Automated regression testing has long existed in most applications but for many web sites because it’s impossible to write tests to verify if a website was being displayed correctly. Today opensource tools like Wraith, Huxley, PhantomCSS, PhotoBox etc solve this issue using screencapture comparison techneques.

What will this post help me do?

Automate website regression testing for agile teams that use JIRA. This will prevent you from committing code that can undo previously completed stories.

JIRA stories are used to add test URLs. Every team member can view them which help increase test coverage.

Why do this?

1. Early discovery of regression issues leads to immediate fixes, saving time and effort otherwise wasted on raising bug report, context switching etc.

2. This will lead to a happier team.

3. No Less nasty surprises/bugs the week before going live.

Technologies Used

JIRA – Let’s anyone in the team add/manage test URLs

PhantomCSS – For screen capture bases visual regression.

Git Hooks – Used to force running of tests before commiting.

Basic setup

1 Create a custom field named “Test URL” in JIRA.

2. Download and install PhanthomCSS

3. Get test URLs of stories in the ‘Done’ column from the JIRA REST API.
curl -D- -u jira_username:password -X GET AND status=Resolved OR status=Closed

4. Use git pre-commit hook to run PhanthomCSS test suites on the URLs before each commit.

Now all the tests will have to pass before being allowed to commit.

The reusable and maintainable setup

Next time you start a project, follow these 4 steps.

1. Install grunt plugin for PhantomCSS. npm install grunt-phantomcss --save-dev

2. Add the following to your grunt file.

3. Create a test suite file with your JIRA project information.

Now you can run your tests manually using grunt phantomcss

Finally, sharing and maintaining git hooks with the team can get messy because git hooks are not a part of source control, so we use grunt git hooks module to manage it.

4. Add grunt-githooks to your project npm install grunt-githooks --save-dev

5. Add the following to your grunt file.

6. Use grunt githooks to add git hooks into your local git.

Now every time you commit, the tests will be run from JIRA story URLs and the commit will be rejected if tests fail.