Moving on…

I’m relocating this blog to a new location and also taking time off to do some travelling which will be covered on my personal blog.  So this blog will no longer be maintained…

Setting up an Automated Build System

Many teams apply continuous integration as part of their software development process.  This means that each team member integrates their work regularly.  To facilitate doing this, you first need an automated build script that each developer can use to compile and build the code locally.  If there are no issues, then the developer can check in their code.  At this point you want an automated build system that takes the latest code from source control and builds it, notifying the team if there are any errors.  The advantage of such an automated build system is that teams are notified early on if there are any integration issues rather than just before a  release deadline.  The build machine also serves as a fully deployed example of the latest version.  When a team member says that a piece of work is completed it should be verified on this machine – not their own.

Cruisecontrol is an open source continuous integration tool.  It allows you to monitor a project, building it when files are updated and notifying you if any part of the build process fails.  There are a lot of reporting tools that can be run on your build.  Unit tests, cover coverage and analysis tools and java documentation are useful to add to your build and make available in your reporting application.  These can be integrated into cruisecontrol.

This post assumes that you are installing cruisecontrol on a redhat machine and that you deploy your application to JBoss.  Any necessary databases have been installed and configured, and you have an ant build script for building and deploying the application.  You have also created a user called cruise that will run the automated build.

Install Java
Download jdk-6u13-linux-i586.bin to /home/cruise
Type sh jdk-6u13-linux-i586.bin
Accept the license agreement
Java will install into the cruise home directory
JAVA_HOME = /home/cruise/jdk1.6.0_13

Install JBoss
Download jboss-4.2.2.GA.zip to /home/cruise
unzip jboss-4.2.2.GA.zip
To avoid any permgen memory errors edit JBOSS_HOME/bin/run.conf and set the following Xms, Xmx, PermSize and MaxPermSize values

JAVA_OPTS=”-Xms512m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512 -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000″

Install Ant
Download apache-ant-1.7.1-bin.tar.gz to /home/cruise
gunzip apache-ant-1.7.1-bin.tar.gz
tar -xvf  apache-ant-1.7.1-bin.tar

Configure paths and settings for the user

You need to configure the .bashrc for the cruise user (assuming the user shell is bash)

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions
export JAVA_HOME=/home/cruise/jdk1.6.0_13
export ANT_HOME=/home/cruise/apache-ant-1.7.1
export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server
export JBOSS_HOME=/home/cruise/jboss-4.2.2.GA

export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$ORACLE_HOME/bin:$JBOSS_HOME/bin:$PATH

Install subversion
Subversion should already be available, check this by running ‘rpm -q subversion’
and making sure it returns a value

Install cruisecontrol
Download cruisecontrol-src-2.8.2.zip to /home/cruise
unzip cruisecontrol-src-2.8.2.zip
Type ant in /home/cruise/cruisecontrol-2.8.2/main

Configure the build projects
mkdir /home/cruise/build
mkdir /home/cruise/build/artifacts
mkdir /home/cruise/build/logs
mkdir /home/cruise/build/projects
Configure a new project

For a project called ‘projectname’, in /home/cruise/build/projects/projectname you should check out the source code eg.
svn checkout http://svn.company.com/project/trunk
Configure the project settings and test the build script to verify that it works without errors.

Configure CruiseControl

You will need to create a file called build-projectname.xml

<!– Delegating build script, used by cruisecontrol to build the project.
Note that the basedir is set to the checked out project –>
<project name=”build-projectname“
default=”automatedbuild”
basedir=”/home/cruise/build/projects/projectname“>
<target name=”automatedbuild”>
<!– Get the latest from SVN –>
<exec executable=”svn”>
<arg line=”up –username user –password pass“/>
</exec>
<!– Call the target that does everything –>
<ant antfile=”build.xml” target=”clean-deploy“/>
</target>
</project>

Edit this file to match your own settings and set the correct username/password for subversion.

Now you need to create the config.xml that is used by cruisecontrol:

<cruisecontrol>

<!– Schedule each project to check for modifications every 15 mins and build if modifications have been made –>

<project name=”projectname” buildafterfailed=”true” requiremodification=”true”>
<plugin name=”svn” classname=”net.sourceforge.cruisecontrol.sourcecontrols.SVN”/>
<listeners>
<currentbuildstatuslistener file=”logs/projectname/status.txt”/>
</listeners>
<modificationset quietperiod=”180″ ignoreFiles=”*/projectname/db/**”>
<svn localworkingcopy=”projects/projectname“/>
</modificationset>
<schedule interval=”900″>
<ant anthome=”/home/cruise/apache-ant-1.7.1″
buildfile=”build-projectname.xml”
target=”automatedbuild”
uselogger=”true”
usedebug=”false”/>
</schedule>
<log logdir=”logs/projectname“>
<delete every=”1″ unit=”MONTH”/>
<merge dir=”projects/projectname/reports/junit/”/>
<merge file=”projects/projectname/reports/checkstyle/cs_errors.xml”/>
<merge dir=”projects/projectname/reports/metrics/”/>
<merge dir=”projects/projectname/reports/cobertura/”/>
</log>
<publishers>
<htmlemail mailhost=”xxx.xxx.x.x“
username=”emailuser“
password=”emailpass“
returnname=”CruiseControl”
returnaddress=”user@company.com“
reportsuccess=”always”
subjectprefix=”CruiseControl: Projectname - “
buildresultsurl=”http://xxx.xxx.x.x:8080/cruisecontrol/buildresults/projectname”
skipusers=”true” spamwhilebroken=”false”
logdir=”logs/projectname“
xsldir=”/home/cruise/cruisecontrol-2.8.2/reporting/jsp/webcontent/xsl”
css=”/home/cruise/cruisecontrol-2.8.2/reporting/jsp/webcontent/css/cruisecontrol.css”>
<always address=”user@company.com” />
<failure address=”teamlist@company.com” reportWhenFixed=”true”/>
</htmlemail>

</publishers>

</project>

</cruisecontrol>

Edit this again to suit your own settings.  Refer back to the cruisecontrol documentation for further information on the settings above, but this will check for updates to the source code every 15 mins.  If there are changes then the build will be run.  If the build fails then the team is emailed.  If it’s successful then someone who monitors the build system can be emailed.

Start CruiseControl

To test the cruisecontrol configuration you need to execute cruisecontrol.sh from /home/cruise/build
First set the permissions with chmod a+x /home/cruise/cruisecontrol-2.8.2/main/bin/cruisecontrol.sh
Then run
/home/cruise/cruisecontrol-2.8.2/main/bin/cruisecontrol.sh

Configure Reporting
Type ant in
/home/cruise/cruisecontrol-2.8.2/reporting/jsp

You will be asked for these parameters:
log.dir: /home/cruise/build/logs
status.file: status.txt
artifacts.dir: /home/cruise/build/artifacts

Copy /home/cruise/cruisecontrol-2.8.2/reporting/jsp/dist/cruisecontrol.war to the jboss deploy directory

Start up jboss and you should be able to access cruisecontrol at http://xxx.xxx.x.x:8080/cruisecontrol/

Applying Scrum – Where to Start?

No one ever said it was going to be easy taking on the role of scrum master.  One of the best things about scrum is also the worst thing about it.  There’s a lot of room for interpretation.  Rather than being a detailed framework, scrum is like a list of guidelines or best practices for developing software.  The purist approach to scrum can be too rigid to flourish in some environments, and after all – scrum is supposed to be about adapting and improving.  I think it’s more important to focus on the spirit of scrum and try and implement the best practices described as best you can.

A large number of companies out there now say that they ‘do scrum’.  This can be a bone of contention, especially for those who have a very strict interpretation of what scrum is, and what it isn’t.  It also presents the problem of taking on the role of scrum master with a team that has been ‘doing scrum’ for a period of time, but not necessarily successfully.  It can be hard to sell a new style of managing software development that comes with the same label on the tin as what’s been done before.  People who have had a negative experience of scrum may be extremely resistent to continuing to use it, even in a different form.

It’s a daunting task to take responsibility for a team. When you’re inheriting a development process that has a list of ‘issues’ so long you can’t see the end of it, it’s easy to get lost just trying to figure out where to start. In the mean time, you have a team that needs to be kept productively busy while you find a way to start progressing in the direction you want to go.  Really, the only place to start with is the Product Backlog.  If that’s not right, everything is going to fall apart.  This is something to tackle first with the Product Owner, and then with the team.

The scrum master and product owner must commit to working on the backlog and producing something useful for the team.  It can be a good idea to put this as a task in the current Sprint – even as one with 0 Story Points.  If the team is to commit to delivering what it promises, the Product Owner and Scrum Master must lead by example and do the same.  Inevitably, the team will need to get involved at a certain point to assist with breaking down the backlog items and assigning story point estimates, but there needs to be something there to start with.

To start generating the Product Backlog, a number of meetings were needed with the Product Owner.  We needed a clear understanding of what projects were potentially in the pipeline, what the deadlines were, and how to prioritise these things.  We started with an extremely high level list of about 10 items that was loosely prioritised.  For each of the items a document was created with a broad description of the item and what it involved.  I then split some of these stories into smaller ones and added some of the tasks that might be needed to complete them.  We reviewed what we had at that point and were happy that the list could be broken down into a Product Backlog that would at least cover a couple of months for the development team.  The team could now provide some input and take the first steps into being less dependent on having the Product Owner present for every single meeting.  This had been a must when requirements were only generated in the Planning meetings at the start of every week.

Implementing Scrum

Scrum is an iterative and incremental software development framework.  It’s better described as a set of best practices or guidelines, rather than a detailed step-by-step software development process.  The main focus is to increase communication and help teams to empower themselves, so that they can resolve any issues that are preventing them from working to the best of their ability.  Engineering practices like XP work well within Scrum.  Scrum manages the product development side of things and XP can be used to help improve quality and productivity.

There are two types of roles defined in Scrum.  Pigs are committed to the project – they are members of the Scrum Team.  Within a Scrum Team you have the Scrum Master who ensures that the process is being followed and performs administrative functions.  The Product Owner represents the project stakeholders.  And the remainder of the team is a cross-functional group of people that are needed to work on the project.  Any other roles are chicken roles.  These people are not part of the process and are not committed to delivering the product, but they provide input.  Chicken roles include stakeholders (customers and vendors), end users, and managers.

Each time-boxed iteration is known as a Sprint.  This is usually 1-4 weeks long.  The less confident a team is, the shorter the iterations should be.  At the end of each sprint the team is expected to deliver a potentially shippable piece of software.  The team takes sets of requirements for each sprint from the top of a prioritised list of requirements called the Product Backlog that is maintained by the Product Owner.

To start using the scrum process you first need to define a Product Vision describing the goal of the project.  You then need to generate a Product Backlog to start working from and assign complexity estimates for each item (this may require a Team Estimation Meeting).  There are a number of meetings that are held during each sprint.  In the Sprint Planning Meeting the team discusses requirements in the product backlog, getting as much information as they need from the Product Owner.  The team then selects the requirements (called stories) they think they can commit to delivering at the end of the iteration.  The team breaks down each story into tasks and begins to assign them.  The stories and tasks for the Sprint are put into a Sprint Backlog.

Every day of the Sprint the team has a Daily Scrum Meeting to synch up, update the Sprint Backlog and make sure everyone knows how the work is progressing.  On the last day of the sprint a Sprint Review Meeting is held to demonstrate only completed stories to the Product Owner who will give feedback.  Directly after the Review meeting the team holds a Sprint Retrospective to discuss how the process can be improved.  The Sprint has now ended.  The Product Owner should make sure that the Product Backlog has been updated and reprioritised before a new sprint begins.  A Team Estimation Meeting may be required to do this.

Scrum builds cross-functional, self-organised teams that communicate as much as possible.  Teams are small with 4-9 members.  It allows for clients to make changes to the requirements and reprioritise them – but only between sprints.  The work selected for each sprint is frozen and can only be changed by the team in consultation with the Product Owner.  Visibility is high throughout the process and there are no penalties for identifying a problem. To successfully implement Scrum within an organisation, both management and team members need to buy into the idea.  It’s also vitally important to find a Scrum Master that will do a good job in coaching the team to reach their full potential.  This is a difficult job to do, as often the person who enforces change is not popular.  Someone who is a poor team lead, is not going to be a good Scrum Master.

When working well, the Scrum process is a nice one to use.  It keeps all the team members heavily involved in the project’s success and engaged in constantly making improvements.  However it can also be a difficult process to bring into a company as it doesn’t solve problems that already exist in a company – it only helps identify the issues.  Ultimately it’s up to the team to find a way to resolve any issues rather than changing the process to accommodate them.  So if input into the scrum is poor (bad requirements, individuals unwilling to work as a team), then the output will also be poor.

“I estimate that 75% of those organizations using Scrum will not succeed in getting the benefits that they hope for from it.”  Ken Schwaber

That’s the theory behind scrum. In practise, it’s not quite as easy as it may sound. Over the next few months I’ll be trying out some more practical approaches to bringing scrum into an organisation.  It seems that every organisation has its own unique approach to using scrum – what works well for one team doesn’t necessarily translate to another.  It’s interesting to see how other teams tackle particular problems and evaluate whether some of those tactics can be employed or altered to work well within your own team.

Scrum – Release Plan

The Release Plan forecasts future velocity based on historical data.  It’s created by the Product Owner who usually waits 2-3 sprints until some historical data has been generated.  The input is the total effort in the product backlog, the team’s availability, sprint length and velocity.  From this you can determine how many sprints are needed in total to complete the backlog.   The Product Owner uses this information to update the backlog items so they will get assigned to Sprints according to priority, capacity and building shippable increments.

The Product Owner updates the Release Plan at the end of each Sprint to include current velocity, changes in capacity or capability, and changes in the Product Backlog.  The data can also be shown as a Release Burndown Chart plotting the effort left in the Product Backlog vs time.

The Release Plan should show:

Sprint numbers
Start and End Dates
Velocity (guess)
Comments