RcppArmadillo 15 CRAN Transition: Offering Office Hours
Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1273 other packages on CRAN, downloaded 41.8 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 651 times according to Google Scholar.
Armadillo 15 brought changes. We mentioned these in the 15.0.2-1 and 15.0.2-1 release blog posts:
- Minimum C++ standard of C++14
- No more suppression of deprecation notes
(The second point is a consequence of the first. Prior to C++14, deprecation notes were issue via a macro, and the macro was set up by Conrad in the common way of allowing an override, which we took advantage of in RcppArmadillo effectively shielding downstream package. In C++14 this is now an attribute, and those cannot be suppressed.)
We tested this then-upcoming change extensively: Thirteen reverse dependency runs expoloring different settings and leading to the current package setup where an automatic fallback to the last Armadillo 14 release offers fallback for hardwired C++11 use and Armadillo 15 others. Given the 1200+ reverse deoendencies, this took considerable time. All this was also quite extensively discussed with CRAN (especially Kurt Hornik) and documented / controlled via a series of issue tickets starting with overall issue #475 covering the subissues:
- open issue
#475 describes the version selection between Armadillo 14 and 15 via
#define - open issue #476 illustrates how package without deprecation notes are already suitable for Armadillo 15 and C++14
- open issue #477 demonstrates how a package with a simple deprecation note can be adjusted for Armadillo 15 and C++14
- closed issue #479 documents a small bug we created in the initial transition package RcppArmadillo 15.0.1-1 and fixed in the 15.0.2-1
- closed issue #481 discusses removal of the check for insufficient LAPACK routines which has been removed given that R 4.5.0 or later has sufficient code in its fallback LAPACK (used e.g. on Windows)
- open issue #484 offering help to the (then 226) packages needing help transitioning from (enforced) C++11
- open issue #485 offering help to the (then 135) packages needing help with deprecations
- open issue #489 coordinating pull requests and patches to 35 packages for the C++11 transition
- open issue #491 coordinating pull requests and patches to 25 packages for deprecation transition
The sixty pull requests (or emailed patches) followed a suggestion by CRAN to rank-order packages affected by their reverse dependencies sorted in descending package count. Now, while this change from Armadillo 14 to 15 was happening, CRAN also tightened the C++11 requirement for packages and imposed a deadline for changes. In discussion, CRAN also convinced me that a deadline for the deprecation warning, now unmasked, was viable (and is in fairness commensurate with similar, earlier changes triggered by changes in the behaviour of either gcc/g++ or clang/clang++). So we now have two larger deadline campaigns affecting the package (and as always there are some others).
These deadlines are coming close: October 17 for the C++11
transition, and October 23 for the deprecation warning. Now, as became
clear preparing the sixty pull requests and patches, these changes are
often relatively straightforward. For the former, remove the C++11
enforcement and the package will likely build without changes. For the
latter, make the often simple (e.g. swith from
arma::is_finite to std::isfinite) change. I
did not encounter anything much more complicated yet.
The number of affected packages—approximated by looking at all packages with a reverse dependency on RcppArmadillo and having a deadline–can be computed as
suppressMessages(library(data.table))
D <- setDT(tools::CRAN_package_db())
P <- data.table(Package=tools::package_dependencies("RcppArmadillo", reverse=TRUE, db=D)[[1]])
W <- merge(P, D, all.x=TRUE)[is.na(Deadline)==FALSE,c(1:2,38,68)]
W
W[, nrevdep := length(tools::package_dependencies(Package, reverse=TRUE, recursive=TRUE, db=D)[[1]]), by=Package]
W[order(-nrevdep)]and has been declining steadily from over 350 to now under 200. For that a big and heartfelt Thank You! to all the maintainers who already addressed their package and uploaded updated packages to CRAN. That rocks, and is truly appreciated.
Yet the number is still large. And while issues #489 and #491 show a number of ‘pending’ packages that have merged but not uploaded (yet?) there are also all the other packages I have not been able to look at in detail. While preparing sixty PRs / patches was viable over a period of a good week, I cannot create these for all packages. So with that said, here is a different suggestion for help: All of next week, I will be holding open door ‘open source’ office hours online two times each day (11:00h to 13:00h Central, 16:00h to 18:00h Central) which can be booked via this booking link for Monday to Friday next week in either fifteen or thirty minutes slots you can book. This should offer Google Meet video conferencing (with jitsi as an alternate, you should be able to control that) which should allow for screen sharing. (I cannot hookup Zoom as my default account has organization settings with a different calendar integration.)
If you are reading this and have a package that still needs helps, I hope to see you in the Open Source Office Hours to aid in the RcppArmadillo package updates for your package. Please book a slot!
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.
