Submit Your Ideas

We want to hear from you - vote for the features and improvements you'd most like to see, or submit your own ideas if you don't find them already listed.

Docker for MacOS Builders to Manage Test Dependencies

I know this has been brought up before:

However, another angle to bring up is the prospect of using Docker to manage test dependencies for iOS builds. Of course, we're highly unlikely to ever see a fully Dockerized MacOS environment due to the inability to run MacOS in a docker container, but that doesn't mean Docker has no usefulness for an iOS CI pipeline. 

Consider this scenario: an iOS app wishes to use XCTest UI to write integration tests. Compared to other options like EarlGrey, XCTest has the advantage of being Apple-supported (ships with Xcode) and having an assertion syntax more likely to be in sync with what developers are already using for unit tests. However, XCTest UI has the disadvantage of being a black-box framework: it cannot directly mock the app code for things like external API dependencies as a unit test would be able to (note also that Espresso for Android is grey box and does not have this limitation). A docker-driven environment for running an iOS app's API dependencies locally for development and testing purposes is viable solution to this problem - assuming the org commits fully to docker-driven development, an iOS developer could bring up the necessary pieces of the backend system locally on their machine and know that their work is compatible with all the latest versions of the various APIs the app depends on. And given this setup, the UI tests could simply run against this docker-driven local environment.

Unfortunately, the developer could not, given current capabilities of CircleCI MacOS builders, use this kind of setup for running their tests in CI. This, to me, would be the main reason for supporting Docker for MacOS builders.

The fact that this capability does not currently exist isn't a dealbreaker necessarily, but it does create a disconnect between what we're able to do in CI on other platforms and the iOS-only workarounds we're forced to employ. For instance, our iOS UI tests currently run against a manually-maintained fake HTTP server that serves up mock JSON files that we must also manually maintain, and we run this server in CI using Supervisor. Now this approach could be refined - we could keep our mocks up to date by using our docker setup locally to record new versions of the mocks periodically. It's just that we're only having to do that work because we don't have the ability to run docker on our CI builders for MacOS - it's not a concern for our apps on other platforms that can use Docker in CI.

I'm aware that Docker for MacOS can be a bit of a drain on computing resources, and given the size of the MacOS VMs (I have another idea about that, incidentally), it would be nice to know if computing resources was a motivating concern behind not enabling Docker for MacOS builders, or if there are other reasons.

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • Jul 19 2018
  • Future consideration
  • Attach files