CircleCI 60% faster builds: use Xcode DerivedData for caching!

When working on large projects you have to constantly take care of the CI and build time condition, especially when you time have more than 5 members. In team where i am currently working we reach the point where our current app build time take 25 minutes. We were thinking how we could reduce build time and CI maintenance cost.

The main idea was to cache DerivedData from develop where we are doing clean build, and reuse it on each feature branch. The most optimal solution would be to take commit from where feature branch started and use develop cache, based on SHA-1.

Xcode stores all build data related cache files and misc data in derived data folder (~/Library/Developer/Xcode/DerivedData). Caching and usage of DerivedData depends on many properties like file modification times, content, inode changes etc.

When you checkout your repository on CircleCI all files will have modification times to be set on current time (the time of git clone), so all your files will be considered new for every new build, even if most files’ content didn’t change at all between builds. Xcode uses modification time (mtime) as one of its signals for incremental builds. To be able to fake Xcode we will have to sets mtime to time when file was modified in git.

setup:
  steps:
    - checkout

Luckily we will be able to use git-tools/git-restore-mtime script written by Rodrigo Silva (MestreLion) to change mtime of files based on commit date of last change. So let’s write action called restore_mtime :

restore_mtime:
    steps:
      - run:
          name: Update mtime for incremental builds
          command: |
            python3 git-restore-mtime --force

Now we need to create action which will allows us to cache DerivedData base on commit hash, which we will be calling after develop build. Important note is that we need to tar/untar data in as a own step to preserve persmissions, we will use posix notation for higher-resolution timestamps. This preserves various attributes used by xcodebuild.

save_derived_data CircleCI step

We already createdsave_derived_data step, but we still need step where we will resolve SHA and restore derived data from develop

Where nearest_develop_SHA.sh is:

As a last very important step which need to be called before each build is telling xcode to ignore inode changes IgnoreFileSystemDeviceInodeChanges=YESon Circle-CI, because default behavior may not be modification times, but rather the device inode changes that are causing the derived data to be considered out of date.

Apple Developer Relations Dec 20 Engineering has the following feedback for you:

It is expected behavior that file modification times will cause rebuilds. In Travis-CI, however, it may not be the modification times, but rather the device inode changes that are causing the derived data to be considered out of date. Try setting:

defaults write com.apple.dt.XCBuild IgnoreFileSystemDeviceInodeChanges -bool YES

Or pass it on the xcodebuild command-line:

IgnoreFileSystemDeviceInodeChanges=1 xcodebuild …

We are now closing this bug report.

If you have questions or comments about the resolution, please update your bug report with that information so we can respond.

So this is how complete example look like:

jobs:
  build_and_test:
    steps:
      - checkout
      - restore_derived_data_from_develop
      - restore_mtime
# Fastlane
      - run:
          name: Run fastlane
          command: bundle exec fastlane test clean:false
  test_develop:
    steps:
      - checkout
      - restore_mtime
       - run:
          name: Set IgnoreFileSystemDeviceInodeChanges flag
          command: defaults write com.apple.dt.XCBuild IgnoreFileSystemDeviceInodeChanges -bool YES
      - run:
          name: Run fastlane
          command: bundle exec fastlane test clean:true
      - save_derived_data

And fastlane lane:

desc 'Runs the tests'
  lane :test do |options|
scan(scheme: ENV['SCHEME'],
     device: ENV['TEST_DEVICE'],
     clean: options[:clean] == false ? false : true,
     code_coverage: true
     )
end

By sharing Xcode derived data folder across jobs we were able to reduce build time on each feature branch to 15 minutes which gave us around 60% faster builds.