Apple CI / Xcode Service and Jenkins

Question

Is there a way/plugin to integrate the new Xcode service and/or the new Apple CI with Jenkins?

Why?

A main issue with having a Jenkins server + an OSX build slave connected via ssh is that Unit Tests do not work, as the iOS Simulator needs a graphical environment which is not present in this configuration.

I hope that it is possible to integrate the Xcode service (which supports Unit Testing) with Jenkins.

It could be that using the Apple CI will be enough for my needs, but this question aims at the integration of the Xcode service with Jenkins.

What I do already know

I have experience with the existing Xcode Jenkins plugin, but it seems not to support the brand new Xcode service or the new Apple CI. I'm especially keen on unit testing via CI (which did not work properly over a ssh session with the old way).

What I want to know

I'd like info on the following issues currently not working with Jenkins and an ssh connected build slave:

• Unit Tests on a headless system
• Acceptance tests with Frank or similar
• Automatic Provisioning Profile updating (Apple CI does that)

And info on things that currently do work fine with Jenkins and an ssh connected build slave and still should work with an Xcode service integration:

• Builds of different build configurations (Release, Debug, TestFlight) / schemes
• Automatic Build number increment
  • (With Jenkins I can set the build number in my project to ${BUILD_NUMBER}, and Jenkins sets this environment variable according to its build number. When the Apple CI does the builds most probably it will set the build number instead.)
  • Handling/synchronization of Build number between Jenkins & Apple CI
• Accessing build products of the Apple CI from different Jenkins Jobs
  • e.g. for a Job to upload to TestFlight
• Backup of builds
• Automatic builds on git push to a specific branch
• E-Mail notifications

Some additional questions/hints

• I'm not sure whether the Apple CI == Xcode service or if the Apple CI just uses the Xcode service. In the latter case the Xcode service just would be like an intelligent build slave, and Jenkins maybe could use that to do builds and tests, but manage build numbers and products by itself.
• I'm aware that the Apple CI is an separate CI, and integrating several CIs with each other is not the most easy or useful way to go. I just fear that the Apple CI is not flexible enough for my needs (see above), and that the old way with Jenkins bears some problems (see above).

Solution 1

I believe you are going to have to choose either jenkins or xcode server, not both. I don't know much about xcode server, but I do know about jenkins and xcode 5.

Builds with different configurations:

In the xcode plugin, you can set the scheme to use.

Automatic Build Number Increment

I added a parameter to my jenkins job called XCODEBUILDNUMBER. And whenever I start a build, I simply copy the build number out of my xcode project (I increment it manually. Mine looks like 080813A) and paste it into the XCODEBUILD parameter. I use this to name my output files, etc. There are plugins for jenkins that can automatically increment your build number, but they don't integrate, or sync with xcode.

Handling/synchronization of Build number between Jenkins & Apple CI

As I said before, I don't know of a way to sync the build numbers, but I just thought of a possible solution. You could use the command line tool plistbuddy, to set the build number in your info.plist, as a build step in your jenkins job.

Unit Tests

I have not successfully made unit tests work with Xcode5, but I know that the xcode plugin for jenkins supports it. I believe that the absence of the "Test After Build" key in the project settings may have something to do with it. If you make it work, i'd love to know. (I am also keen on making this work)

Acceptance Tests

From what I can tell, Frank is a command line tool. You can easily integrate it into your Jenkins job, and I believe that it will fail the build if your tests don't pass.

Accessing build products of the Apple CI from different Jenkins Jobs

Not completely sure what you mean, but with jenkins you can archive your build product (a .ipa), for later download and upload to a service like testflight. Again, I don't know much about Xcode Server (CI).

Backup of builds

As I said before, jenkins can archive your build product. Also, I use the the ${BUILD_NUMBER} variable in my build products directory, so I have a different directory for each build. This directory is also backed up to my Time Machine, and important builds copied to my web directory.

Automatic builds on git push to a specific branch

With the jenkins git plugin, you can make jenkins poll your scm in a interval specified by you, and can trigger a build on a change.

E-Mail notifications

I am sure that there is a plugin for this. (that emails you when a build failed/succeeded. in fact, this may be built-in)

In Closing

The xcode CI is a full independent CI, that may be hard to integrate with jenkins. Personally, I would recommend jenkins simply due to its extendability. Sorry I don't know much about Xcode Server.

Solution 2

I've got unit tests running in Jenkins with Xcode 5 on my OS X build slave. Instead of using the Xcode plugin, I run as an execute shell build step:

xcodebuild test -scheme <scheme> -configuration Coverage -sdk iphonesimulator7.0 -destination OS=7.0,name="iPhone Retina (4-inch)"

My coverage configuration is the exact same as my Debug config, except Generate Test Coverage Reports is set to YES, and Instrument Program Flow is set to YES. This is done so test coverage files are created. Due to a bug in Xcode 5, I call __gcov_flush(); in the tearDown of all my tests. I pipe the output of this xcodebuild command into ocunit2junit to get test reports in Jenkins.