Dirk Eddelbuettel

Sat, 26 Sep 2020

#30: Easy, Reliable, Fast and Portable Linux and macOS Continuous Integration

Welcome to the 30th post in the rarified R recommendation resources series or R4 for short. The last post introduced BSPM. In the four weeks since, we have worked some more on BSPM to bring it to the point where it is ready for use with continuous integration. Building on this, it is now used inside the run.sh script that driven our CI use for many years (via the r-travis repo).

Which we actually use right now on three different platforms:

All three use the exact same script facilitating this, and run a ‘matrix’ over Linux and macOS. You read this right: one CI setup that is portable and which you can take to your CI provider of choice. No lock-in or tie-in. Use what works, change at will. Or run on all three if you like burning extra cycles.

This is already used by handful of my repos as well as by at least two repos of friends also deploying r-travis. How does it work? In a nutshell we are

  • downloading run.sh via curl and changing its mode;
  • running run.sh bootstrap which sets the operating system default:
    • on Linux we use Ubuntu,
      • add two PPAs repos for R itself and over 4600 r-cran-* binaries,
      • and enable BSPM to use these from install.packages()
    • on macOS we use the standard setup also used on Travis, GitHub Actions and elsewhere;
    • this provides us with fast, reliable, easy, and portable access to binaries on two OSs under dependency resolution;
  • running run.sh install_deps to install just the requireded Depends:, Imports: and LinkingTo:
  • running run.sh tests to build the tarball and test it via R CMD check --as-cran.

There are several customizations that are possible via environment variables

  • additional PPAs or drat repos can be added to offer even more package choice;
  • alternatively one could run run.sh install_all to also install Suggests:;
  • optionally one could run run.sh install_r pkgA pkgB ... to install packages explicitly listed;
  • optionally one could also run run.sh install_aptget r-cran-pkga r-cran-pkgb otherpackage to add more Ubuntu binaries.

We find this setup compelling. The scheme is simple: there really is just one shell script behind it which can also be downloaded and altered. The scheme is also portable as we can (as shown) rotate between CI provides. The scheme is also more flexible: in case of debugging needs one can simply run the script on a local Docker or VM instance. Lastly, the scheme moves away from single points of failure or breakage.

Currently the script uses only BSPM as I had the hunch that it would a) work and b) be compelling. Adding support for RSPM would be equally easy, but I have no immediate need to do so. Adding BioConductor installation may be next. That is easy when BioConductor uses r-release; it may be little more challenging under r-devel to but it should work too. Stay tuned.

In the meantime, if the above sounds compelling, give run.sh from r-travis a go!

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link