Perform Design Reviews Continuously

Either an object has responsibility to too many other objects (highly afferent, A) or the object isn’t sufficiently independent of other objects (highly efferent, E).

I don’t agree with the use of the formula that the text presents, which attempts to construct a ratio to summarize how likely it is that a change will impact an object (instability, I).

Why do additional responsibilities to other objects (higher A) mitigate the risk of change caused by dependence on other objects? I would expect any kind of coupling to increase instability linearly (or exponentially, depending on the kind of dependencies). If we want to essentially just consider E, but scale it against A, then maybe this measure would be better than the ratio.

where I=E if A=0.

I do think that there’s an argument to make for considering A in an object’s instability, in a system where objects’ designers are aware of each others’ needs, and attempt to accommodate them. Responsibility to other objects increases instability of the base object because of the varying design influence of each dependent object.

Assemblies with high afferent coupling should have a high degree of associated tests because, of course, with so much code dependent on that assembly, you want to guarantee it is reliable.  Break assemblies into smaller, more flexible chunks of code.

For high efferent coupling, ensure good tests, code coverage, refactoring, and consider long-term implications of instability.

[Duvall et al. 2007, “Continuous Inspection"]

Label a Repository’s Assets

A *.00 baseline label.

Creating a repository label clearly delineates a group of files as belonging together. Labeling repository versions enables a smooth transition to newer versions of code by creating snapshots in time.

Parallel branches handle multiple development lines.  Developers can add the required fixes and produce a stable build that doesn’t necessarily contain new features that exist on the trunk [mainline].

The text credits labels rather than branches.  This is true for a one-off patch, but truly parallel development requires branches, since we cannot persistently fork using only labels.  What does working in parallel mean, if not creating permanent versions from a label on a branch?

Once a release has been labeled, generate that same release by specifying the label identifier, which can follow many different styles.

[Duvall et al. 2007, “Continuous Deployment"]

The Future of CI

Before the repository accepts the code, it runs an integration build on a separate machine. Only if the integration build is successful will it commit the code to the repository.

Is this worth the extra effort?   With frequent and appropriately high-visibility builds, no build series will remain broken for long.  Release managers may just ignore builds which fail, since enough succeed to meet the test and delivery schedules.

You integrate all your changes and run an integration build on a separate machine before committing your changes to the repository.

Our practice: anyone can build with our build systems, on any machine with the appropriate OS, since all the unix build machines are centrally managed.  Developers should reverse-merge (or see the latest integration-branch changes with an “agile” config spec), build, check in, and merge to the integration branch.

Expect more parallelization, and other capabilities, to leverage additional hardware and software resources, to speed up builds.

For example, ElectricAccelerator.

[Duvall et al. 2007, "The Future of CI"]

Automate Database Integration

Automate database integration activities: drop database; create database; insert system data; insert test data; migrate database and data; set up database instances in multiple environments; modify column attributes and constraints; modify test data; modify stored procedures, functions, and triggers; obtain access to different environments; and back up and restore large data sets.

Execute SQL scripts to define and manipulate a database with a single instruction from the command line, which uses attributes defined in a single file.  Organize your targets and scripts by database definition type or by subsystem.

Drop and recreate the database, enforce data integrity through constraints and triggers, and define database behavior through stored procedures or functions.  Create database and permissions, tables, sequences, views, stored procedures and functions, and triggers, ordered by database object dependencies.

Provide initial data and test data for testing code that relies on the database.  Use different SQL data files to support development, test, quality assurance (QA), and production environments.  Provide scripts for inserting, updating, and deleting data.

Combine Data Definition Language (DDL) and Data Manipulation Language (DML) statements, in a database integration orchestration script, for the higher-level build and integration processes.

[Duvall et al. 2007, “Continuous Database Integration”]

Continuous Database Integration

Rebuild your database and test data any time a change is applied to a project’s version control repository.

Store artifacts related to database integration in a version control system, trigger an integration build when they change, generate them with build scripts, test them for correctness, and inspect them for policy compliance.

[Duvall et al. 2007, “Continuous Database Integration”]

Stage Builds

Run lightweight “commit” builds that perform compile, unit test execution, and deployment followed by heavyweight “secondary” builds that include component, system, and other slower-running tests and inspections.

First run an initial integration “commit” or lightweight build that integrates the software components and runs unit tests to root out any obvious problems. After this lightweight build is successful, a more exhaustive integration build is run to include component tests or system tests, inspections, and deployment.

[Duvall et al. 2007, “Building Software at Every Change”]

Run Fast Builds

Try to get your integration builds down to ten minutes by increasing computing resources, offloading slower tests, offloading or reducing inspections, and running staged builds.

Build scalability indicates how capable the build system is of handling an increase in the amount of code that it integrates and analyzes. Build performance refers to the duration of the build.

Developers do not move on to the next task until their most recent check-in integrates successfully. Therefore, builds taking longer than ten minutes can interrupt their flow.

Prioritize improvement tactics, for build duration reduction, by scalability, performance, and difficulty in implementation.

Integration Build Metrics

Compilation time. The time it takes to compile the software, and how it compares to your past compile times. [Recorded by daily builds.]

Number of source lines of code (SLOC). This indicates the system’s size or at least what needs to be compiled. [Calculated by SSMT.]

Number and types of inspections. The number of different types of inspections you are performing. Consider eliminating any redundancy.

Average assembly generation time. The time it takes to generate the assembly, archive, or however you are packaging the software. [Recorded by daily builds.]

Test execution time (based on category). The time it takes to perform testing at each level: unit, component, and system.

Ratio between successful and unsuccessful builds. Divide the number of failed builds into the number of total builds to determine the ratio between successful and unsuccessful builds. [Calculable from daily build archive.]

Inspection time. The time it takes to perform all of the automated inspections. [Timeable from a Klocwork or Purify run.]

Deployment time. The time it takes to deploy the software into the target environment from the integration build.

Database rebuild time. The time it takes to rebuild your database.

Integration build machine system resources and usage. Improving the memory, disk speed, and/or processor can improve the performance of your integration builds. This helps determine whether an integration build machine has an application server or database server or some other process that is using up memory or processor speed.

Version control system load. Helps determine the version control system’s peak-time load, how long it takes to check out/update your project from the integration build machine, and if the network bandwidth, processor, memory, or disk drives are adequate.

Integration Build Duration Improvements

Use a dedicated integration build machine. Reduce false positive or negative builds, and enable faster builds.

Increase integration build machine(s) hardware capacity. Perform available upgrades to CPU, disk, or memory. Offload processes to other systems. Eliminate unnecessary system processes.

Improve test performance. Separate automated tests by category—unit, component, and system—and run these tests at different times (e.g., unit tests at every commit, component/system tests in a secondary build). Refactor your tests based on the results of the inspection tools. Use mocks/stubs for components that may otherwise be too difficult or complex to use in the unit-testing environment. For example, a common implementation of a mock object is mocking the interface for data access. Separate long-running integration tests into separate specialized test suites. Execute your tests in parallel. Run different types of tests based on build type: a commit build is followed by secondary builds, a full integration build, or release build.

Streamline integration builds. See the practice Stage Builds.

Examine and optimize infrastructure. Investigate and improve any infrastructure resources to reduce the build duration.

Optimize the build process. An incremental build will compile and/or regenerate only the files that have changed. Ideally, an integration environment should be cleaned by removing old files and then compiling/regenerating the code to effectively determine if anything has broken. Might be reasonable to only rebuild once a day, objects that rarely change.

Build system components separately. Break apart the software into smaller subsystems (modules) and build each of the subsystems separately. If there are any changes to one project based on the dependencies, the other projects are rebuilt as well.

Improve software inspection performance. Which metrics are used? Does each metric provide tangible value? Are there two or more tools providing the same metrics, which may decrease your build performance? Are you running automated inspections with every build? Are there certain analyses that can run as part of a secondary or periodic build? Are there inspections that you can run on specific subsystems rather than the entire code base?

Perform distributed integration builds. Moving part of the build to another machine may mean copying large files around as part of the build process, which has the potential to slow things down even more.

[Duvall et al. 2007, “Building Software at Every Change”]

Run Manual Integration Builds

Run a sequential integration build manually using an automated build as an approach to reduce integration build errors. Some use this approach as an alternative to a CI server.

Developers manually run an integration build on a separate integration machine to ensure the build always stays in the green. Doesn’t scale particularly well for larger teams, who may stack up many changes for larger and less frequent integrations. So no guarantee the practice will be followed.

Combine, with automated CI, the approach of separate private builds by each developer, to prevent broken integration builds.

[Duvall et al. 2007, “Building Software at Every Change”, optional]

Use a CI Server

In addition to or as an alternative to running manual integration builds, use a CI server to automatically poll for version control changes and run an integration build on a separate machine.

Poll for changes in the version control repository on a specified time interval. Perform certain actions on a scheduled basis, such as hourly or daily. Identify a “quiet period” during which no integration builds are performed for the project. Support for different build scripting tools including command-line tools such as Rake, make, Ant, or NAnt. Send e-mails to the concerned parties. Display a history of previous builds. Display a dashboard that is Web accessible so that everyone can review integration build information. Support multiple version control systems for your different projects.

Manually perform integrations [to compensate for] minimal tool support for preventing broken code from entering the repository.

[Duvall et al. 2007, “Building Software at Every Change”, optional]

Use a Dedicated Integration Build Machine

Use one machine dedicated to running your builds. Ensure that the integration location is free of old build artifacts.

Run on a separate machine, with application and database servers into known states, to find [process] omissions. Remove any code dependencies on the integration environment, remove all of the source code and binaries from the previous integration build (to baseline the environment), and set test data and any other configuration elements to a known state.

When individuals know that the latest integration build failed, they can avoid getting the bad source code from the version control repository. This includes source code, build scripts, configuration files, tools (such as application server, database server, and static analysis tools), test code, and database scripts/files.

“The reason we had to come in last Saturday was because of integration problems.” It is worth the money to increase hardware resources for an integration build machine rather than wasting time waiting for slow builds. Efficient builds run often, made repeatable by reducing environmental assumptions.

Script most dependencies into your build and include the dependency in your repository. Create an image of certain dependencies. Refactor your build scripts to remove many of the hard-coded dependencies, and make relative references instead.

[Duvall et al. 2007, “Building Software at Every Change”]