Sat, 09 Dec 2023

RcppInt64 0.0.4 on CRAN: Minor Bugfix

The new-ish package RcppInt64 (announced earlier this fall in this post, with two small updates following) arrived on CRAN minutes ago as relase 0.0.4. RcppInt64 collects some of the previous conversions between 64-bit integer values in R and C++, and regroups them in a single package. It offers two interfaces: both a more standard as<>() converter from R values along with its companions wrap() to return to R, as well as more dedicated functions ‘from’ and ‘to’.

This release addresses an issues Sebastian reported a few hours and which is reported by newer, pickier compilers: We need to include <cstdint> so that int64_t is declared. CRAN was at its usual best processing this efficiently including tests of the by now two reverse dependencies. Twenty two minutes total, all automated:

The brief NEWS entry follows:

Changes in version 0.0.4 (2023-12-09)

  • The cstdint header is now included (closes #1).

Courtesy of my CRANberries, there is a diffstat report relative to previous release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 04 Dec 2023

RcppArmadillo 0.12.6.6.1 on CRAN: No More Deprecation

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1126 other packages on CRAN, downloaded 31.7 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 569 times according to Google Scholar.

This release ends the practice on asking Armadillo to suppress deprecation warnings. RcppArmadillo, as noted, has a large user base. Sometimes Conrad sometimes made changes without too much of a heads-up so at times it was opportune to not bring those warnings to dozens (or maybe hundreds) of packages at CRAN. Yet we need to balance this with the demonstrable need to call out older deprecated code use. So sixteen months ago, with GitHub issue #391, we started to alert author of 30+ affected packages and supplied either pull requests or emailed patches to all. Eleven months ago GitHub issues #402 was added for a second deprecation.

And the time of making the switch has come. Release 0.12.6.6.1 no longer defines ARMA_IGNORE_DEPRECATED_MARKER. So among the over 1100 packages using RcppArmadillo at CRAN, around a good dozen or so were flagged in the upload – but CRAN concurred and let the package migrate to CRAN.

If you maintain an affected package, consider applying the patch or pull request now. A simple stop-gap measure also exists by adding -DARMA_IGNORE_DEPRECATED_MARKER to src/Makevars as either PKG_CPPFLAGS or PKG_CXXFLAGS to reactivate it. But a proper code update, which is generally simple, may be better. If you are unsure, do not hesitate to get in touch.

The set of changes since the last CRAN release follows.

Changes in RcppArmadillo version 0.12.6.6.1 (2023-12-03)

  • Following the extendeded transition in #391 and #402, this release no longer sets ARMA_IGNORE_DEPRECATED_MARKER. Maintainers of affected packages have received pull requests or patches and can set -DARMA_IGNORE_DEPRECATED_MARKER as PKG_CPPFLAGS.

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 02 Dec 2023

dang 0.0.16: New Features, Some Maintenance

A new release of my mixed collection of things package dang package arrived at CRAN a little while ago. The dang package regroups a few functions of mine that had no other home as for example lsos() from a StackOverflow question from 2009 (!!), the overbought/oversold price band plotter from an older blog post, the market monitor blogged about as well as the checkCRANStatus() function tweeted about by Tim Taylor. And more so take a look.

This release brings a number of updates, including a rather nice improvement to the market monitor making updates buttery smooth and not flickering (with big thanks to Paul Murrell who calmly pointed out once again … that base R does of course have the functionality I was seeking) as well as three new functions (!!) and then a little maintenance on the -Wformat print format string issue that kept ‘everybody’ busy this week.

The NEWS entry follows.

Changes in version 0.0.16 (2023-12-02)

  • Added new function str.language() based on post by Bill Dunlap

  • Added new argument sleep in intradayMarketMonitor

  • Switched to dev.hold() and dev.flush() in intradayMarketMonitor with thanks to Paul Murrell

  • Updated continued integration setup, twice, and package badges

  • Added new function shadowedPackages

  • Added new function limitDataTableCores

  • Updated two error() calls to updated tidyCpp signature to not tickle -Wformat warnings under R-devel

  • Updated two URL to please link checks in R-devel

  • Switch two tests for variable of variable to is.* and inherits(), respectively

Courtesy of my CRANberries, there is a comparison to [the previous release][previous releases]. For questions or comments use the the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/dang | permanent link

tidyCpp 0.0.7 on CRAN: Maintenance

A maintenance release of the tidyCpp package arrived on CRAN this morning. The packages offers a clean C++ layer (as well as one small C++ helper class) on top of the C API for R which aims to make use of this robust (if awkward) C API a little easier and more consistent. See the vignette for motivating examples.

This release follows a number of other releases this week in accomodating the recently added -Wformat -Wformat-security from the development branch of R. It also includes a new example snippet illustrating creation of a numeric matrix.

The NEWS entry follows.

Changes in tidyCpp version 0.0.7 (2023-11-30)

  • Add an example for a numeric matrix creator

  • Update the continuous integration setup

  • Accomodate print format warnings from r-devel

Thanks to my CRANberries, there is also a diffstat report for this release. For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/tidycpp | permanent link

Thu, 30 Nov 2023

RcppClassicExamples 0.1.3 on CRAN: Maintenance

Another upgrade triggered solely by changing CRAN standards (see previous one from five years ago). This time it concerns warnings under r-devel with -Wformat -Wformat-security so we injected a number of "%s" into Rf_error() calls.

No new code or features. Full details below. And as a reminder, don’t use the old RcppClassic – use Rcpp instead.

Changes in version 0.1.3 (2023-11-30)

  • Update Rf_error() call to not tickle -Wformat

  • Minor other packaging and continuous integration tweaks

Thanks to CRANberries, you can also look at a diff to the previous release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 29 Nov 2023

RcppSpdlog 0.0.15 on CRAN: Maintenance

Version 0.0.15 of RcppSpdlog is now on CRAN and will be uploaded to Debian. RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich. You can learn more at the nice package documention site.

This releases updates the code to address warning now shown under R-devel when -Wformat -Wformat-security are enabled. This amounted to re-generating RcppExports.cpp under an updated [Rcpp][rcpp] version. It also updates the package default help page by removing some stubs that were not filled in.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.15 (2023-11-29)

  • Correct default package help page slighly

  • RcppExports.cpp has been regenerated under an updated Rcpp to address a format string warning under R-devel

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page, or the package documention site.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

qlcal 0.0.9 on CRAN: Maintenance

The ninth release of the qlcal package arrivied at CRAN today.

qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more.

This releases updates the code to address warning now shown under R-devel when -Wformat -Wformat-security are enabled. This amounted to re-generating RcppExports.cpp under an updated Rcpp version. We also no longer set C++14 explicitly as a compilation standard but rather determine at build time if it is needed or not.

Changes in version 0.0.9 (2023-11-29)

  • configure now uses a new helper script to only set a compilation standard when needed for R versions older than 4.2.0

  • The file RcppExports.cpp was regenerated to avoid a string format warning from R-devel

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/qlcal | permanent link

RcppQuantuccia 0.1.1 on CRAN: Maintenance

A minor release of RcppQuantuccia arrived on CRAN today. RcppQuantuccia started from the Quantuccia header-only subset / variant of QuantLib which it brings it to R. This project validated the idea of making the calendaring functionality of QuantLib available in a more compact and standalone project – which we now do with qlcal which can be seen as a successor to this.

This releases updates the code to address warning now shown under R-devel when -Wformat -Wformat-security are enabled. This amounted to re-generating RcppExports.cpp under an updated Rcpp version. We also no longer set C++14 explicitly as a compilation standard.

The complete list changes for this release follows.

Changes in version 0.1.2 (2023-11-29)

  • RcppExports.cpp has been regenerated under an updated Rcpp to address a format string warning under R-devel

  • The compilation standard is no longer set to C++14

Courtesy of CRANberries, there is also a diffstat report relative to the previous release. More information is on the RcppQuantuccia page. Issues and bugreports should go to the GitHub issue tracker.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 28 Nov 2023

RcppSimdJson 0.1.11 on CRAN: Maintenance

A new maintenance release 0.1.11 of the RcppSimdJson package is now on CRAN.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon.

This release responds to a CRAN request to address issues now identified by -Wformat -Wformat-security. These are frequently pretty simple changes as it was here: all it took was an call to compileAttributes() from an updated Rcpp version which now injects "%s" as a format string when calling Rf_error().

The (very short) NEWS entry for this release follows.

Changes in version 0.1.11 (2023-11-28)

  • RcppExports.cpp has been regenerated under an update Rcpp to address a print format warning (Dirk in #88).

Courtesy of my CRANberries, there is also a diffstat report for this release. For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 27 Nov 2023

RcppCNPy 0.2.12 on CRAN: More Maintenance

A new (and again somewhat minor) maintenance release of the RcppCNPy package arrived on CRAN earlier today.

RcppCNPy provides R with read and write access to NumPy files thanks to the cnpy library by Carl Rogers along with Rcpp for the glue to R.

Recent changes in r-devel hone in on issues concerning printf format string inaccuracies the compiler can detect via the -Wformat -Wformat-security flags. Two fairly simplye ones were present here and have been addressed. In the time since the last release about twenty months ago two or three other minor packaging and setup details have also been taken care of, details are below.

Changes in version 0.2.12 (2022-11-27)

  • The continuous integration workflow received a trivial update, twice.

  • The C++ compilation standard is now implicit per CRAN and R preference.

  • The CITATION file format has been updated for the current usage.

  • Two print format string issues reported by current R-devel have been addressed.

CRANberries also provides a diffstat report for the latest release. As always, feedback is welcome and the best place to start a discussion may be the GitHub issue tickets page.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 26 Nov 2023

RQuantLib 0.4.20 on CRAN: More Maintenance

A new release 0.4.20 of RQuantLib arrived at CRAN earlier today, and has already been uploaded to Debian as well.

QuantLib is a rather comprehensice free/open-source library for quantitative finance. RQuantLib connects (some parts of) it to the R environment and language, and has been part of CRAN for more than twenty years (!!) as it was one of the first packages I uploaded there.

This release of RQuantLib brings a few more updates for nags triggered by recent changes in the upcoming R release (aka ‘r-devel’, usually due in April). The Rd parser now identifies curly braces that lack a preceding macro, usually a typo as it was here which affected three files. The printf (or alike) format checker found two more small issues. The run-time checker for examples was unhappy with the callable bond example so we only run it in interactive mode now. Lastly I had alread commented-out the setting for a C++14 compilation (required by the remaining Boost headers) as C++14 has been the default since R 4.2.0 (with suitable compilers, at least). Those who need it explicitly will have to uncomment the line in src/Makevars.in. Lastly, the expand printf format strings also found a need for a small change in Rcpp so the development version (now 1.0.11.5) has that addressed; the change will be part of Rcpp 1.0.12 in January.

Changes in RQuantLib version 0.4.20 (2023-11-26)

  • Correct three help pages with stray curly braces

  • Correct two printf format strings

  • Comment-out explicit selection of C++14

  • Wrap one example inside 'if (interactive())' to not exceed total running time limit at CRAN checks

Courtesy of my CRANberries, there is also a diffstat report for the this release 0.4.20. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rquantlib | permanent link

Fri, 03 Nov 2023

RcppEigen 0.3.3.9.4 on CRAN: Maintenance, Matrix Changes

A new release 0.3.3.9.4 of RcppEigen arrived on CRAN yesterday, and went to Debian today. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.

This update contains a small amount of the usual maintenance (see below), along with a very nice pull request by Mikael Jagan which simplifies to interface with the Matrix package and inparticular the CHOLMOD library that is part of SuiteSparse. This release is coordinated with lme4 and OpenMx which are also being updated.

The complete NEWS file entry follows.

Changes in RcppEigen version 0.3.3.9.4 (2023-11-01)

  • The CITATION file has been updated for the new bibentry style.

  • The package skeleton generator has been updated and no longer sets an Imports:.

  • Some README.md URLs and badged have been updated.

  • The use of -fopenmp has been documented in Makevars, and a simple thread-count reporting function has been added.

  • The old manual src/init.c has been replaced by an autogenerated version, the RcppExports file have regenerated

  • The interface to package Matrix has been updated and simplified thanks to an excllent patch by Mikael Jagan.

  • The new upload is coordinated with packages lme4 and OpenMx.

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 31 Oct 2023

RcppArmadillo 0.12.6.6.0 on CRAN: Bugfix, Thread Throttling

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1110 other packages on CRAN, downloaded 31.2 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 563 times according to Google Scholar.

This release brings upstream bugfix releases 12.6.5 (sparse matrix corner case) and 12.6.6 with an ARPACK correction. Conrad released it this this morning, I had been running reverse dependency checks anyway and knew we were in good shape so for once I did not await a full run against the now over 1100 (!!) packages using RcppArmadillo.

This release also contains a change I prepared on Sunday and which helps with much-criticized (and rightly I may add) insistence by CRAN concerning ‘throttling’. The motivation is understandable: CRAN tests many packages at once on beefy servers and can ill afford tests going off and requesting numerous cores. But rather than providing a global setting at their end, CRAN insists that each package (!!) deals with this. The recent traffic on the helpful-as-ever r-pkg-devel mailing clearly shows that this confuses quite a few package developers. Some have admitted to simply turning examples and tests off: a net loss for all of us. Now, Armadillo defaults to using up to eight cores (which is enough to upset CRAN) when running with OpenMP (which is generally only on Linux for “reasons” I rather not get into…). With this release I expose a helper functions (from OpenMP) to limit this. I also set up an example package and repo RcppArmadilloOpenMPEx detailing this, and added a demonstration of how to use the new throttlers to the fastLm example. I hope this proves useful to users of the package.

The set of changes since the last CRAN release follows.

Changes in RcppArmadillo version 0.12.6.6.0 (2023-10-31)

  • Upgraded to Armadillo release 12.6.6 (Cortisol Retox)

    • Fix eigs_sym(), eigs_gen() and svds() to generate deterministic results in ARPACK mode
  • Add helper functions to set and get the number of OpenMP threads

  • Store initial thread count at package load and use in thread-throttling helper (and resetter) suitable for CRAN constraints

Changes in RcppArmadillo version 0.12.6.5.0 (2023-10-14)

  • Upgraded to Armadillo release 12.6.5 (Cortisol Retox)

    • Fix for corner-case bug in handling sparse matrices with no non-zero elements

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 21 Oct 2023

qlcal 0.0.8 on CRAN: QuantLib 1.32 Updates

The eighth release of the still fairly new qlcal package arrivied at CRAN today.

qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more.

This release brings updates from the just-released QuantLib 1.32 version. It also avoids a nag from R during build (“only specify C++14 if you really need it”) but switching to a versioned depends on R 4.2.0 or later. This implies C++14 or later as the default. If you need qlcal on an older R, grab the sources, edit DESCRIPTION to remove this constraint and set the standard as before in src/Makevars (or src/Makevars.win).

Changes in version 0.0.8 (2023-10-21)

  • A small set of updates from QuantLib 1.32 have been applied

  • The explicit C++14 compilation standard has been replaced with an implicit one by relying on R (>= 4.2.0)

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/qlcal | permanent link

Mon, 09 Oct 2023

drat 0.2.4 on CRAN: Improved macOS Support, General Updates

drat user

A new minor release of the drat package arrived on CRAN today making it the first release in one and a half years. drat stands for drat R Archive Template, and helps with easy-to-create and easy-to-use repositories for R packages. Since its inception in early 2015 it has found reasonably widespread adoption among R users because repositories with marked releases is the better way to distribute code.

Because for once it really is as your mother told you: Friends don’t let friends install random git commit snapshots. Properly rolled-up releases it is. Just how CRAN shows us: a model that has demonstrated for two-plus decades how to do this. And you can too: drat is easy to use, documented by six vignettes and just works. Detailed information about drat is at its documentation site. Two more blog posts using drat from GitHub Actions were just added today showing, respectively, how to add to a drat repo in either push or pull mode.

This release contains two extended PRs contributed by drat users! Both extended support for macOS: Joey Reid extended M1 support to pruning and archival, and Arne Johannes added bug-sur support. I polished a few more things around the edges, mostly documentation or continuos-integrations related.

The NEWS file summarises the release as follows:

Changes in drat version 0.2.4 (2023-10-09)

  • macOS Arm M1 repos are now also supported in pruning and archival (Joey Reid in #135 fixing #134)

  • A minor vignette typo was fixed (Dirk)

  • A small error with setwd() in insertPackage() was corrected (Dirk)

  • macOS x86_64 repos (on big-sur) are now supported too (Arne Johannes Holmin in #139 fixing #138)

  • A few small maintenance tweaks were applied to the CI setup, and to the main README.md

Courtesy of my CRANberries, there is a comparison to the previous release. More detailed information is on the drat page as well as at the documentation site.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/drat | permanent link

Tue, 19 Sep 2023

RcppInt64 0.0.3 on CRAN: Now with nanotime Support

The still new package RcppInt64 (announced two weeks ago in this post, with this followup last week) arrived on CRAN earlier today in its second update and relase 0.0.3. RcppInt64 collects some of the previous conversions between 64-bit integer values in R and C++, and regroups them in a single package by providing a single header. It offers two interfaces: both a more standard as<>() converter from R values along with its companions wrap() to return to R, as well as more dedicated functions ‘from’ and ‘to’.

This release adds support for the corresponding nanotime conversion between R and C++. nanotime is leveraging the same bit64-based reprensentation of 64-bit integers for nanosecond resolution timestamps. A thorough S4 wrapping the offers R based access for convenient and powerful operations at nanosecond resolution. And as tweeted (here and here), tooted (here and here), and skeeted (here and here) in a quick preview last Sunday, it makes for easy and expressive code.

The brief NEWS entry follows:

Changes in version 0.0.3 (2023-09-19)

  • The as<>() and wrap() converters are now declared inline.

  • Conversion to and from nanotime has been added.

Courtesy of my CRANberries, there is a diffstat report relative to previous release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 12 Sep 2023

RcppInt64 0.0.2 on CRAN: Small Update

The still very new package RcppInt64 (announced a week ago in this post) arrived on CRAN earlier today in its first update, now at 0.0.2. RcppInt64 collects some of the previous conversions between 64-bit integer values in R and C++, and regroups them in a single package by providing a single header. It offers two interfaces: both a more standard as<>() converter from R values along with its companions wrap() to return to R, as well as more dedicated functions ‘from’ and ‘to’.

The package by now has its first user as we rearranged RcppFarmHash to use it. The change today makes bit64 a weak rather than strong dependency as we use it only for tests and illustrations. We also added two missing fields to DESCRIPTION and added badges to README.md.

The brief NEWS entry follows:

Changes in version 0.0.2 (2023-09-12)

  • DESCRIPTION has been extended, badges have been added to README.md

  • Package bit64 is now a Suggests:

Courtesy of my CRANberries, there is a [diffstat report relative to previous release][this release].

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 10 Sep 2023

RcppArmadillo 0.12.6.4.0 on CRAN: Another Upstream Bugfix

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1096 other packages on CRAN, downloaded 30.5 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 552 times according to Google Scholar.

This release brings bugfix upstream release 12.6.4. Conrad prepared this a few days ago; it takes me the usual day or so to run reverse-dependency check against the by-now almost 1100 CRAN packages using RcppArmadillo. And this time, CRAN thought it had found two issues when I submitted and it took two more days til we were all clear about those two being false positives (as can, and does, happen). So today it reached CRAN.

The set of changes follows.

Changes in RcppArmadillo version 0.12.6.4.0 (2023-09-06)

  • Upgraded to Armadillo release 12.6.4 (Cortisol Retox)

    • Workarounds for bugs in Apple accelerate framework

    • Fix incorrect calculation of rcond for band matrices in solve()

    • Remove expensive and seldom used optimisations, leading to faster compilation times

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 09 Sep 2023

Carmageddon by Daniel Knowles: A Brief Review

Carmageddon

Daniel Knowles’ Carmageddon: How Cars Make Life Worse and What to Do About It is an entertaining, lucid, and well-written “manifesto” (to borrow a term from the author) aiming to get us all thinking a bit more about what cars do to society, and how to move on to a better outcome for all.

The book alternates between historical context and background, lived experience (as the author is a foreign correspondent who had the opportunity to travel), and researched content. It is refreshingly free of formalities (no endless footnotes or endnotes with references, though I would have liked occassional references but hey we all went to school long enough to do a bit of research given a pointer or two). I learned or relearned a few things as I was for example somewhat unaware of the air pollution (micro-particle) impact stemming from tires and brake abrasions—for which electronic vehicles do zilch, and for which the auto-obesity of ever larger and heavier cars is making things much worse. And some terms (even when re-used by Knowles) are clever such bionic duckweed. But now you need to read the book to catch up on it.

Overall, the book argues its case rather well. The author brings sufficient evidence to make the formal ‘guilty’ charge quite convincing. It is also recent having come out just months ago, making current figures even more relevant.

I forget the exact circumstance but I think I came across the author in the context of our joint obsession with both Chicago and cycling (as there may have been a link from a related social media post) and/or the fact that I followed some of his colleagues at The Economist on social media. Either way, the number of Chicago and MidWest references made for some additional fun when reading the book over a the last few days. And for me another highlight was the ode to Tokyo which I wholeheartedly agree with: on my second trip to Japan I spent a spare day cycling across the city as the AirBnB host kindly gave me access to his bicycles. Great weather, polite drivers, moderate traffic, and just wicked good infrastructure made me wonder why I did not see more cyclists.

I have little to criticize beyond the lack of any references. The repeated insistence on reminding us that Knowles comes from Birmingham gets a little old by the fifth or sixth repetition. It is all a wee bit anglo- or UK-centric. It obviously has a bit on France, Paris, and all the recent success of Anne Hidalgo (who, when I was in graduate school in France, was still a TV person rather than the very successful mayor she is now) but then does not mention the immense (and well known) success of the French train system which lead to a recent dictum to no longer allow intra-Frace air travel if train rides of under 2 1/2 hours are available which is rather remarkable. (Though in fairness that may have been enacted once the book was finished.)

Lastly, the book appears to have a few sections available via Google Books. My copy will good back from one near-west suburban library to the neighbouring one.

Overall a strong recommendation for a very good and timely book.

/books | permanent link

RcppFarmHash 0.0.3 on CRAN: Small Update

A minor maintenance release of the RcppFarmHash package is now on CRAN as version 0.0.3.

RcppFarmHash wraps the Google FarmHash family of hash functions (written by Geoff Pike and contributors) that are used for example by Google BigQuery for the FARM_FINGERPRINT digest.

This releases farms out the conversion to the integer64 add-on type in R to the new package RcppInt64 released a few days ago and adds some minor maintenance on continuous integration and alike.

The brief NEWS entry follows:

Changes in version 0.0.3 (2023-09-09)

  • Rely on new RcppInt64 package and its header for conversion

  • Minor updates to continuous integration and README.md

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 07 Sep 2023

x13binary 1.1.57-4 on CRAN: Minor Update

Release 1.1.57-4 of the x13binary package providing the X-13ARIMA-SEATS program by the US Census Bureau arrived earlier on CRAN.

This release sets an explicit download timeout option value at the request of CRAN who, just like everybody else, were bitten a little by the instability at GitHub this week.

Courtesy of my CRANberries, there is also a diffstat report for this release showing changes to the previous release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/x13binary | permanent link

Tue, 05 Sep 2023

RcppInt64 0.0.1 on CRAN: New Package!

Happy to share that a new package RcppInt64 arrived on CRAN earlier today after a brief one-day inspection round. RcppInt64 collects some of the previous conversions between 64-bit integer values in R and C++, and regroups them in a single package.

A single header is provided. It actually offers two interfaces: both a more standard as<>() converter from R values along with its companions wrap() to return to R, as well as more dedicated functions ‘from’ and ‘to’. A key difficulty faced when supporting 64 bit integer payloads is that R has no proper type for it so the standard template-based approaches use by Rcpp do not apply. To ‘carry’ 64 bit integers, the clever approach by Jens Oehlschlägel and his bit64 package is used. However, its use of a double to transport the int64 payload means we must take care to not uninentionally interpret the double variables as, well, double. So we use an simple S3 class in R, and check for it. With some care (as provided by these helper functions) this works well.

The RcppInt64 packages contains both an example function, as well as an entire example package to demonstrate how to use these facilities in your package. We hope others will find this useful.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 29 Aug 2023

RcppArmadillo 0.12.6.3.0 on CRAN: New Upstream Bugfix

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1092 other packages on CRAN, downloaded 30.3 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 549 times according to Google Scholar.

This release brings bugfix upstream release 12.6.3. We skipped 12.6.2 at CRAN (as discussed in the previous release notes) as it only affected Armadillo-internal random-number generation (RNG). As we default to supplying the RNGs from R, this did not affect RcppArmadillo. The bug fixes in 12.6.3 are for csv reading which too will most likely be done by R tools for R users, but given two minor bugfix releases an update was in order. I ran the full reverse-depenency check against the now more than 1000 packages overnight: no issues. armadillo processing CRAN processed the package fully automatically as it has no issues, and nothing popped up in reverse-dependency checking.

The set of changes for the last two RcppArmadillo releases follows.

Changes in RcppArmadillo version 0.12.6.3.0 (2023-08-28)

  • Upgraded to Armadillo release 12.6.3 (Cortisol Retox)

    • Fix for corner-case in loading CSV files with headers

    • For consistent file handling, all .load() functions now open text files in binary mode

Changes in RcppArmadillo version 0.12.6.2.0 (2023-08-08)

  • Upgraded to Armadillo release 12.6.2 (Cortisol Retox)

    • use thread-safe Mersenne Twister as the default RNG on all platforms

    • use unique RNG seed for each thread within multi-threaded execution (such as OpenMP)

    • explicitly document arma_rng::set_seed() and arma_rng::set_seed_random()

  • None of the changes above affect R use as RcppArmadillo connects the RNGs used by R to Armadillo

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 20 Aug 2023

RcppRedis 0.2.4 on CRAN: Maintenance

Another minor release, now at 0.2.4, of our RcppRedis package arrived on CRAN yesterday. RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has carried production loads on a trading floor for several years. It also supports pub/sub dissemination of streaming market data as per this earlier example.

This update is (just like the previous one) fairly mechanical. CRAN noticed a shortcoming of the default per-package help page in a number of packages, in our case it was matter of adding one line for a missing alias to the Rd file. We also demoted the mention of the suggested (but retired) rredis package to a mere mention in the DESCRIPTION file as a formal Suggests: entry, even with an added Additional_repositories, create a NOTE. Life is simpler without those,

The detailed changes list follows.

Changes in version 0.2.4 (2023-08-19)

  • Add missing alias for ‘RcppRedis-package’ to rhiredis.Rd.

  • Remove Suggests: rredis which triggers a NOTE nag as it is only on an ‘Additional_repositories’.

Courtesy of my CRANberries, there is also a diffstat report for this this release. More information is on the RcppRedis page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 17 Aug 2023

#43: r2u Faster Than the Alternatives

Welcome to the 43th post in the $R^4 series.

And with that, a good laugh. When I set up Sunday’s post, I was excited enough about the (indeed exciting !!) topic of r2u via browser or vscode that I mistakenly labeled it as the 41th post. And overlooked the existing 41th post from July! So it really is as if Douglas Adams, Arthur Dent, and, for good measure, Dirk Gently, looked over my shoulder and declared there shall not be a 42th post!! So now we have two 41th post: Sunday’s and July’s.

Back the current topic, which is of course r2u. Earlier this week we had a failure in (an R based) CI run (using a default action which I had not set up). A package was newer in source than binary, so a build from source was attempted. And of course failed as it was a package needing a system dependency to build. Which the default action did not install.

I am familiar with the problem via my general use of r2u (or my r-ci which uses it under the hood). And there we use a bspm variable to prefer binary over possibly newer source. So I was curious how one would address this with the default actions. It so happens that the same morning I spotted a StackOverflow question on the same topic, where the original poster had suffered the exact same issue!

I offered my approach (via r2u) as a comment and was later notified of a follow-up answer by the OP. Turns our there is a new, more powerful action that does all this, potentially flipping to a newer version and building it, all while using a cache.

Now I was curious, and in the evening cloned the repo to study the new approach and compare the new action to what r2u offers. In particular, I was curious if a use of caches would be benficial on repeated runs. A screenshot of the resulting Actions and their times follows.

Turns out maybe not so much (yet ?). As the actions page of my cloned ‘comparison repo’ shows in this screenshot, r2u is consistently faster at always below one minute compared to new entrant at always over two minutes. (I should clarify that the original actions sets up dependencies, then scrapes, and commits. I am timing only the setup of dependencies here.)

We can also extract the six datapoints and quickly visualize them.

Now, this is of course entirely possibly that not all possible venues for speedups were exploited in how the action setup was setup. If so, please file an issue at the repo and I will try to update accordingly. But for now it seems that a default of setup r2u is easily more than twice as fast as an otherwise very compelling alternative (with arguably much broader scope). However, where r2u choses to play, on the increasingly common, popular and powerful Ubuntu LTS setup, it clearly continues to run circles around alternate approaches. So the saying remains:

r2u: fast, easy, reliable.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Originally posted 2023-08-13, minimally edited 2023-08-15 which changed the timestamo and URL.

/code/r4 | permanent link

Tue, 15 Aug 2023

#41: Using r2u in Codespaces

Welcome to the 41th post in the $R^4 series. This post draws on joint experiments first started by Grant building on the lovely work done by Eitsupi as part of our Rocker Project. In short, r2u is an ideal match for Codespaces, a Microsoft/GitHub service to run code ‘locally but in the cloud’ via browser or Visual Studio Code. This posts co-serves as the README.md in the .devcontainer directory as well as a vignette for r2u.

So let us get into it. Starting from the r2u repository, the .devcontainer directory provides a small self-containted file devcontainer.json to launch an executable environment R using r2u. It is based on the example in Grant McDermott’s codespaces-r2u repo and reuses its documentation. It is driven by the Rocker Project’s Devcontainer Features repo creating a fully functioning R environment for cloud use in a few minutes. And thanks to r2u you can add easily to this environment by installing new R packages in a fast and failsafe way.

Try it out

To get started, simply click on the green “Code” button at the top right. Then select the “Codespaces” tab and click the “+” symbol to start a new Codespace.

The first time you do this, it will open up a new browser tab where your Codespace is being instantiated. This first-time instantiation will take a few minutes (feel free to click “View logs” to see how things are progressing) so please be patient. Once built, your Codespace will deploy almost immediately when you use it again in the future.

After the VS Code editor opens up in your browser, feel free to open up the examples/sfExample.R file. It demonstrates how r2u enables us install packages and their system-dependencies with ease, here installing packages sf (including all its geospatial dependencies) and ggplot2 (including all its dependencies). You can run the code easily in the browser environment: Highlight or hover over line(s) and execute them by hitting Cmd+Return (Mac) / Ctrl+Return (Linux / Windows).

(Both example screenshots reflect the initial codespaces-r2u repo as well as personal scratchspace one which we started with, both of course work here too.)

Do not forget to close your Codespace once you have finished using it. Click the “Codespaces” tab at the very bottom left of your code editor / browser and select “Close Current Codespace” in the resulting pop-up box. You can restart it at any time, for example by going to https://github.com/codespaces and clicking on your instance.

Extend r2u with r-universe

r2u offers “fast, easy, reliable” access to all of CRAN via binaries for Ubuntu focal and jammy. When using the latter (as is the default), it can be combined with r-universe and its Ubuntu jammy binaries. We demontrates this in a second example file examples/censusExample.R which install both the cellxgene-census and tiledbsoma R packages as binaries from r-universe (along with about 100 dependencies), downloads single-cell data from Census and uses Seurat to create PCA and UMAP decomposition plots. Note that in order run this you have to change the Codespaces default instance from ‘small’ (4gb ram) to ‘large’ (16gb ram).

Local DevContainer build

Codespaces are DevContainers running in the cloud (where DevContainers are themselves just Docker images running with some VS Code sugar on top). This gives you the very powerful ability to ‘edit locally’ but ‘run remotely’ in the hosted codespace. To test this setup locally, simply clone the repo and open it up in VS Code. You will need to have Docker installed and running on your system (see here). You will also need the Remote Development extension (you will probably be prompted to install it automatically if you do not have it yet). Select “Reopen in Container” when prompted. Otherwise, click the >< tab at the very bottom left of your VS Code editor and select this option. To shut down the container, simply click the same button and choose “Reopen Folder Locally”. You can always search for these commands via the command palette too (Cmd+Shift+p / Ctrl+Shift+p).

Use in Your Repo

To add this ability of launching Codespaces in the browser (or editor) to a repo of yours, create a directory .devcontainers in your selected repo, and add the file .devcontainers/devcontainer.json. You can customize it by enabling other feature, or use the postCreateCommand field to install packages (while taking full advantage of r2u).

Acknowledgments

There are a few key “plumbing” pieces that make everything work here. Thanks to:

Colophon

More information about r2u is at its site, and we answered some question in issues, and at stackoverflow. More questions are always welcome!

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Originally posted 2023-08-13, minimally edited 2023-08-15 which changed the timestamo and URL.

/code/r4 | permanent link

Thu, 10 Aug 2023

RcppArmadillo 0.12.6.1.0 on CRAN: New Upstream

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1092 other packages on CRAN, downloaded 30.1 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 545 times according to Google Scholar.

This release brings bugfix upstream release 12.6.1. Conrad release 12.6.0 when CRAN went on summer break. I rolled it up ran the full reverse-depenency check against the now more than 1000 packages. And usage from one those revealed a corner-case bug (of not always ‘flattening’ memory for sparse matrices to zero values) so 12.6.1 followed. This is what was uploaded today. And as I prepared it earlier in the week as CRAN reopened, Conrad released a new 12.6.2. However, its changes are only concerned with settings for Armadillo-internal use of its random number generators (RNGs). And as RcppArmadillo connects Armadillo to the RNGs provided by R, the upgrade does not affect R users at all. However it is available in the github repo, in the Rcpp drap repo and at r-universe.

The set of changes for this RcppArmadillo release follows.

Changes in RcppArmadillo version 0.12.6.1.0 (2023-07-26)

  • Upgraded to Armadillo release 12.6.1 (Cortisol Retox)

    • faster multiplication of dense vectors by sparse matrices (and vice versa)

    • faster eigs_sym() and eigs_gen()

    • faster conv() and conv2() when using OpenMP

    • added diags() and spdiags() for generating band matrices from set of vectors

Courtesy of my CRANberries, there is a [diffstat report relative to previous release]. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 08 Aug 2023

dtts 0.1.1 on CRAN: Enhancements

Leonardo and I are happy to announce the release of a first follow-up release 0.1.1 of our dtts package which got to [CRAN][cran] in its initial upload last year.

dtts builds upon our nanotime package as well as the beloved data.table to bring high-performance and high-resolution indexing at the nanosecond level to data frames. dtts aims to bring the time-series indexing versatility of xts (and zoo) to the immense power of data.table while supporting highest nanosecond resolution.

This release fixes a bug flagged by valgrind and brings several internal enhancements.

Changes in version 0.1.1 (2023-08-08)

  • A simplifcation was applied to the C++ interface glue code (#9 fixing #8)

  • The package no longer enforces the C++11 compilation standard (#10)

  • An uninitialized memory read has been correct (#11)

  • A new function ops has been added (#12)

  • Function names no longer start with a dot (#13)

  • Arbitrary index columns are now supported (#13)

Courtesy of my CRANberries, there is also a diffstat report for the this release this release. Questions, comments, issue tickets can be brought to the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/dtts | permanent link

Mon, 07 Aug 2023

RQuantLib 0.4.19 on CRAN: More Maintenance

A new release 0.4.19 of RQuantLib arrived at CRAN earlier today, and has already been uploaded to Debian too.

QuantLib is a rather comprehensice free/open-source library for quantitative finance. RQuantLib connects it to the R environment and language, and has been part of CRAN for more than twenty years (!!)

This release of RQuantLib brings a small update to three unit tests as very recent 1.31 release QuantLib brought a subtle change to some fixed income payment schedules and dates. On a sadder note, as CRAN now checks the ratio of ‘user time’ over ‘elapsed time’, excessive threading was inferred for five examples. As we seemingly cannot limit std::thread here, I opted to park these examples behind a \dontrun{}. Not ideal. Lastly, a few version checks in configure were updated.

Changes in RQuantLib version 0.4.19 (2023-08-07)

  • Three calendaring / schedule tests were adjusted for slightly changed values under QuantLib 1.31

  • Several checks in the configure script have been updated to reflect current versions of packages.

  • Five examples no longer run because, even while extremely short, use of (too many default) threads was seen.

Courtesy of my CRANberries, there is also a diffstat report for the this release 0.4.19. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rquantlib | permanent link

Sun, 23 Jul 2023

#41: Another r2u Example – Really Simple CI

Welcome to the 41th post in the $R^4 series. Just as the previous post illustrated r2u use to empower interactive Google Colab sessions, today we want to look at continuous integration via GitHub Actions.

Actions are very powerful, yet also intimidating and complex. How does one know what to run? How does ensure requirements are installed? What does these other actions do?

Here we offer a much simpler yet fully automatic solution. It takes advantage of the fact that r2u integrates fully and automatically with the system, here apt, without us having to worry about the setup. One way to make this very easy is the use of the Rocker containers for r2u. They already include the few lines of simple (and scriptable) setup, and have bspm setup so that R commands to install packages dispatch to apt and will bring all required dependencies automatically and easily.

With that the required yaml file for an action can be as simple as this:

name: r2u

on:
  push:
  pull_request:
  release:

jobs:
  ci:
    runs-on: ubuntu-latest
    container:
      image: rocker/r2u:latest
    steps:
      - uses: actions/checkout@v3
      - name: SessionInfo
        run: R -q -e 'sessionInfo()'
      #- name: System Dependencies
      #  # can be used to install e.g. cmake or other build dependencies
      #  run: apt update -qq && apt install --yes --no-install-recommends cmake git
      - name: Package Dependencies
        run: R -q -e 'remotes::install_deps(".", dependencies=TRUE)'
      - name: Build Package
        run: R CMD build --no-build-vignettes --no-manual .
      - name: Check Package
        run: R CMD check --no-vignettes --no-manual $(ls -1tr *.tar.gz | tail -1)

There are only a few key components here.

First, we have the on block where for simplicity we select pushes, pull requests and releases. One could reduce this to just pushes by removing or commenting out the next two lines. Many further refinements are possible and documented but not reqired.

Second, the jobs section and its sole field ci saythat we are running this CI on Ubuntu in its latest release. Importantly we then also select the rocker container for r2 meaning that we explicitly select running in this container (which happens to be an extension and refinement of ubuntu-latest). The latest tag points to the most recent LTS release, currently jammy aka 22.04. This choice also means that our runs are limited to Ubuntu and exclude macOS and Windows. That is a choice: not every CI task needs to burn extra (and more expensive) cpu cycles on the alternative OS, yet those can always be added via other yaml files possibly conditioned on fewer runs (say: only pull requests) if needed.

Third, we have the basic sequence of steps. We check out the repo this file is part of (very standard). After that we ask R show the session info in case we need to troubleshoot. (These two lines could be commented out.) Next we show a commented-out segment we needed in another repo where we needed to add cmake and git as the package in question required local compilation during build. Such a need is fairly rare, but as shown can be be accomodated easily while taking advantage of the rich development infrastructure provided by Ubuntu. But the step should be optional for most R packages so it is commented out here. The next step uses the remotes package to look at the DESCRIPTION file and install all dependencies which, thanks to r2u and bspm, will use all Ubuntu binaries making it both very fast, very easy, and generally failsafe. Finally we do the two standard steps of building the source package and checking it (while omitting vignettes and the (pdf) manual as the container does not bother with a full texlive installation—this could be altered if desired in a derived container).

And that’s it! The startup cost is a few seconds to pull the container, plus a few more seconds for dependencies – and let us recall that e.g. the entire tidyverse installs all one hundred plus packages in about twenty seconds as shown in earlier post. After that the next cost is generally just what it takes to build and check your package once all requirements are in.

To use such a file for continuous integration, we can install it in the .github/workflows/ directory of a repository. One filename I have used is .github/workflows/r2u.yaml making it clear what this does and how.

More information about r2u is at its site, and we answered some question in issues, and at stackoverflow. More questions are always welcome!

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Wed, 19 Jul 2023

qlcal 0.0.7 on CRAN: QuantLib 1.31 Updates

The seventh release of the still pretty new qlcal package arrivied at CRAN today.

qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more.

This release brings updates from the just-released QuantLib 1.31 version.

Changes in version 0.0.7 (2023-07-19)

  • Updates, extensions and corrections to calendars from South Korea, Hong Kong, Singapore, India, Taiwan, South Africa, Denmark and Finland from QuantLib 1.31

  • Added support for UnitedStates/SOFR calendar

As the update comprises a number of small-but-important updates to eight different country calendars as well as a new US calendar (in a slight variation on the government bond calendar) we can also update the little demo highlighting all calendars within the US for the given year:

> ## see https://github.com/qlcal/qlcal-r/blob/master/demo/allUScalendars.R
> ## see R/calendars.R in qlcal, we prepend 'UnitedStates/' in makeHol()
> cals <- c("LiborImpact", "NYSE", "GovernmentBond", "NERC", "FederalReserve", "SOFR")
> print(Reduce(cbind, Map(makeHol, cals)))
           LiborImpact NYSE GovernmentBond NERC FederalReserve SOFR
2023-01-02        TRUE TRUE           TRUE TRUE           TRUE TRUE
2023-01-16        TRUE TRUE           TRUE   NA           TRUE TRUE
2023-02-20        TRUE TRUE           TRUE   NA           TRUE TRUE
2023-04-07          NA TRUE             NA   NA             NA TRUE
2023-05-29        TRUE TRUE           TRUE TRUE           TRUE TRUE
2023-06-19        TRUE TRUE           TRUE   NA           TRUE TRUE
2023-07-04        TRUE TRUE           TRUE TRUE           TRUE TRUE
2023-09-04        TRUE TRUE           TRUE TRUE           TRUE TRUE
2023-10-09        TRUE   NA           TRUE   NA           TRUE TRUE
2023-11-10        TRUE   NA             NA   NA             NA   NA
2023-11-23        TRUE TRUE           TRUE TRUE           TRUE TRUE
2023-12-25        TRUE TRUE           TRUE TRUE           TRUE TRUE
> 

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/qlcal | permanent link

Mon, 10 Jul 2023

RcppSpdlog 0.0.14 on CRAN: Upstream Update

Version 0.0.14 of RcppSpdlog is now on CRAN and has just been uploaded to Debian. RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich. You can learn more at the nice package documention site.

This release simply brings an update to the just release spdlog 1.12.0 from a few days ago.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.14 (2023-07-09)

  • Added new badge to README.md

  • Upgraded to upstream releases spdlog 1.12.0

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page, or the package documention site.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 08 Jul 2023

#40: Another r2u Example – Making Colab Easier

Welcome to the 40th post in the $R^4 series. This one will just be a very brief illustration of r2u use in what might be an unexpected place: Google Colab. Colab has a strong bent towards Jupyter and Python but has been supporting R compute kernels for some time (by changing what they call the ‘runtime’). And with a little exploration one can identify these are (currently, as of July 2023) running Ubuntu 20.04 aka ‘focal’.

Which is of course one of two system supported by our lovely r2u project (with the other being the newer 22.04 aka ‘jammy’). And I mostly tweeted / tooted about r2u since the its introduction in #37. And gave basically just a mention in passing in ‘faster feedback’ post #38 as well as the ‘faster feedback in ci’ post #39). So a brief recap may be in order. In essence, r2u makes all of CRAN available as full-fledged Ubuntu binaries with complete and full dependencies which are then installed directly and quickly via apt. Which, to top it of, are accessed directly from R via install.packages() so no special knowledge or sauce needed. We often summarize it as “fast, easy, reliable: what is not to like”.

And, as we established in a few minutes of probing, it also works in the ‘focal’-based Colab session. The screen shot shows the basic step of fetching the setup script (for ‘plain’ Ubuntu focal system) from r2u, making it executable and running it. Total time: 34 seconds. And after that we see the pure magic of install.packages("tidyverse") installing all of it in nine seconds. Additionally, we add the brms package in thirty-one seconds cia install.packages("brms"). Both load just fine and echo their current values.

r2u on colab focal

The commands that are executed in that R session are just

download.file("https://github.com/eddelbuettel/r2u/raw/master/inst/scripts/add_cranapt_focal.sh",
              "add_cranapt_focal.sh")
Sys.chmod("add_cranapt_focal.sh", "0755")
system("./add_cranapt_focal.sh")
install.packages("tidyverse")
library(tidyverse)
install.packages("brms")
library(brms)

The timings are the Colab notebook are visible in the left margin. The lack of output makes debugging a little trickier so I still recommend to use r2u for first expploration via a Docker container as e.g. rocker/r2u:jammy.

More information about r2u is at its site, and we answered some question in issues, and at stackoverflow. More questions are always welcome!

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Fri, 07 Jul 2023

digest 0.6.33 on CRAN: Bugfix

Release 0.6.33 of the digest package arrived at CRAN today and has been uploaded to Debian already.

digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, blake3, and crc32c algorithms) permitting easy comparison of R language objects. It is a mature and widely-used (with 58.6 million downloads just on the partial cloud mirrors of CRAN which keep logs) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation to quickly identify the various objects.

This release corrects a bad bug we accidentally introduced in release 0.6.32 a little under two weeks ago. As described we added crc32c and aimed for the simple ‘portable’ approach (given that the accelerated version is now availble in our package crc32c) but overlooked one setting which promptly broked compilation on M1/M2/Arm64 (which we do not have access too, sadly). This was actually reported in issue #189 on the very day 0.6.32 was released and even before we got to blog about its announcement so that we could even mention it in the third paragraph there.

This fix was a one-liner, and was committed hours later. Now, as it sometimes happens, processing the new upload at CRAN took some time. They appeared to be down the next day not processing anything. Once opened again, they reported ‘changes to worse’ as can be seen in reverse-dependency checks. Turns out those were false positive (“it happens”) but then the package lingered a little longer and quite a lot of folks chimed in at in issue #189. I have now learned that there were in fact some temporary hardware issues in one of the CRAN locations, so it all had a reason—but most importantly it is all better now. My apologies for the temporary breakage, we attempted to address it swiftly but sometimes things beyond our control (and reach) do slow things down. Now new and fixed binaries should appear in the next few days.

My CRANberries provides the usual summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/digest | permanent link

Thu, 06 Jul 2023

Rcpp 1.0.11 on CRAN: Updates and Maintenance

rcpp logo

The Rcpp Core Team is delighted to announce that the newest release 1.0.11 of the Rcpp package arrived on CRAN and in Debian earlier today. Windows and macOS builds should appear at CRAN in the next few days, as will builds in different Linux distribution–and of course at r2u. The release was finalized three days ago, but given the widespread use and extended reverse dependencies at CRAN it usually takes a few days to be processed. This release continues with the six-months January-July cycle started with release 1.0.5 in July 2020. As a reminder, we do of course make interim snapshot ‘dev’ or ‘rc’ releases available via the Rcpp drat repo and strongly encourage their use and testing—I run my systems with these versions which tend to work just as well, and are also fully tested against all reverse-dependencies.

Rcpp has long established itself as the most popular way of enhancing R with C or C++ code. Right now, 2720 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 251 in BioConductor. On CRAN, 13.7% of all packages depend (directly) on Rcpp, and 59.6% of all compiled packages do. From the cloud mirror of CRAN (which is but a subset of all CRAN downloads), Rcpp has been downloaded 72.5 million times. The two published papers (also included in the package as preprint vignettes) have, respectively, 1678 (JSS, 2011) and 259 (TAS, 2018) citations, while the the book (Springer useR!, 2013) has another 588.

This release is incremental as usual, generally preserving existing capabilities faithfully while smoothing our corners and / or extending slightly, sometimes in response to changing and tightened demands from CRAN or R standards.

The full list below details all changes, their respective PRs and, if applicable, issue tickets. Big thanks from all of us to all contributors!

Changes in Rcpp version 1.0.11 (2023-07-03)

  • Changes in Rcpp API:

    • Rcpp:::CxxFlags() now quotes only non-standard include path on linux (Lukasz in #1243 closing #1242).

    • Two unit tests no longer accidentally bark on stdout (Dirk and Iñaki in #1245).

    • Compilation under C++20 using clang++ and its standard library is enabled (Dirk in #1248 closing #1244).

    • Use backticks in a generated .Call() statement in RcppExports.R (Dirk #1256 closing #1255).

    • Switch to system2() to capture standard error messages in error cases (Iñaki in #1259 and #1261 fixing #1257).

  • Changes in Rcpp Documentation:

    • The CITATION file format has been updated (Dirk in #1250 fixing #1249).
  • Changes in Rcpp Deployment:

    • A test for qnorm now uses the more accurate value from R 4.3.0 (Dirk in #1252 and #1260 fixing #1251).

    • Skip tests with path issues on Windows (Iñaki in #1258).

    • Container deployment in continuous integrations was improved. (Iñaki and Dirk in #1264, Dirk in #1269).

    • Several files receives minor edits to please R CMD check from r-devel (Dirk in #1267).

Thanks to my CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2994 previous questions.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 02 Jul 2023

RcppAnnoy 0.0.21 on CRAN: Upstream Update

annoy image

Another minor maintenance release, now at version 0.0.20, of RcppAnnoy has arrived on CRAN.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the Spotify music discovery algorithm.

This release mainly updates the included copy of Annoy to version 1.7.3 which now provides a C++ namespace in the header API. So I sent a pair of PRs to uwot to update its (compiled) use, which James promptly merged. With that the CRAN update was once again seamless. Otherwise the package follows current custom by moving away from C++11 as a set compilation standard to be open for more modern C++ idioms. Here, this allows in particular for possible multithreaded indexing for which everything needed is in C++17. However, we did not turn multithreaded indexing on as the RNG use is such that the streams would differ leading to slightly different (if of course equivalent) results. Users can enable this by recompiling with the switch flipped, see src/Makevars.

Changes in version 0.0.21 (2023-07-02)

  • The build setup switched from C++11 to C++17 which offers threading support (which remains off by default to ensure consistent results)

  • Upstream code was update to Annoy 1.17.3, the switch to an explicit C++ namespace has been accomodated (Dirk in #75)

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 01 Jul 2023

New Achievement Unlocked: Bugfix Patch into Stable Release Update

Last weekend was the bi-annual time to roll the main machine and server to the current Ubuntu release, now at 23.04. It must now have been fifteen or so years that I have used Ubuntu for my desktop / server (for “reasons” I may write about another time). And of course it all passed swimmingly as usual.

[ And a small aside, if I may. Among all these upgrades, one of my favourite piece of tech trivia that may well be too little known remains the dedication of the PostgreSQL maintainers installing the new version, now PostgreSQL 15, seamlessly in parallel with the existing one, in my case PostgreSQL 14, keeping both running (!!) on two neighbouring ports (!!) so that there is no service disruption. So at some point, maybe this weekend, I will run the provided script to dump-and-restore to trigger the database migration at my convenience. Happy PostgreSQL on Debian/Ubuntu user since the late 1990s. It. Just. Works. ]

[ Similarly, it is plainly amazeballs how apt orders and runs package updates to service to keep running for a maximum amount of time. This machine acts as e.g. a web server and it was up and running (as were other services) while thousands of package got updated/replaced. It is pretty amazing. Whereas on other platforms people still maintain the “do not ever update anything” we demonstrably offer the opposite here. Really not too shabby. ]

This time, I had one small hickup. Emacs, now at version 28 bringing loads of niceties along, would not load. And the error soon lead to a post on the magit list where its indefatigable author Jonas Bernoulli suggested a rebuild (and hence re-compilation of the elisp files). Which I did, and which allowed a start of VM inside Emacs. So I was happy. But it allowed it only once for each VM package reinstall. Not good, and I remained curious.

Some more digging lead to a breakthrough. A post and commit at the Fedora Project indicated that for just VM within Emacs, byte-compilation throws a spanner. Which one can work around by telling Emacs not to compile the files in the VM folder.

So I applied that patch to the VM package in a local build et voilà we have working VM. The world is clearly better when your email client of 25+ years just works. And feels snappier because everything under Emacs28 feels snappier! So I set this up properly and filed Debian Bug Report #1039105.

To which Ian Jackson, the maintainer, replied a few days later nodding that he could reproduce. And that he concurred with the bug report, and was planning to update throughout. And lo and behold this morning’s update reveals that this made into an update for the just-released Debian Bookworm.

So yay. In all these years of Debian maintainership (somewhere between twentyfive and thirty) this may be my first bug report with patch going straight into a stable release. But of course, true and full credit goes of course to Göran Uddeborg for putting it up first for Fedora. Lovely how Open Source can work together. We really should do more, not less, of that. But I digress…

Anyway, in sum: If you try to use VM under the lovely Emacs 28, there is a fix, and if you use it with Debian Bookworm the fix should hit your mirrors soons. Ditto, methinks, for the next Ubuntu release. If you use it under Ubuntu now, the package is (elisp) text-only and can be safely installed on a derivative (which we do not enjoy in general but which is fine here). So enjoy!

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/computers/linux/debian/misc | permanent link

Mon, 26 Jun 2023

digest 0.6.32 on CRAN: CRC32C and AES Updates

Release 0.6.32 of the digest package arrived at CRAN this morning, and will be uploaded to Debian as well.

digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, blake3, and crc32c algorithms) permitting easy comparison of R language objects. It is a mature and widely-used (with 58.3 million downloads just on the partial cloud mirrors of CRAN which keep logs) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation to quickly identify the various objects.

This release brings two changes.

First, we added crc32c as a new hashing algorithm. And we did so in a ‘portable’ minimal fashion—while also adding a new CRAN crc32c package with the full hardware-optimised support form x86_64 and Arm64 (M1/M2) chips. Fully integrating the optional added package is still work in progress we may refine. (Now, as it turns out, a first bug report that this is not as portable as we hoped. But it also looks like we already have a fix. So a quick follow-up release is likely.) Second, Dean Attali had looked into AES digests and cyphers using the CBC mode and noticed what we needed padding which he kindly contributed in PR #186.

My CRANberries provides the usual summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/digest | permanent link

Mon, 19 Jun 2023

RcppArmadillo 0.12.4.1.0 on CRAN: New Upstream Bugfix

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1079 other packages on CRAN, downloaded 29.6 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper) (preprint / vignette) by Conrad and myself has been cited 543 times according to Google Scholar.

This release brings bugfix upstream release 12.4.1 made by Conrad at the end of last week. As usual, I prepared the usual release candidate, tested on the over 1000 reverse depends (which sadly takes a long time on old hardware), found no issues and sent it to CRAN. Where it got tested again and was by a stroke of bad luck upheld for two unrelated issue (one package fell over one of its other dependencies changing a data representation, another fell afoul of a tightened test on total test time) so this awaited the usual email handshake with the CRAN maintainers … and the weekend got in the way. The release also contains a PR kindly provided by Mikael Jagan for an upcoming change in package Matrix.

As a bugfix release, the set of changes is fairly small.

Changes in RcppArmadillo version 0.12.4.1.0 (2023-06-17)

  • Upgraded to Armadillo release 12.4.1 (Cortisol Profusion Redux)

    • fix bug in SpMat::shed_cols()

    • functions such as .is_finite() and find_nonfinite() will now emit a runtime warning when compiled in fast math mode; such compilation mode disables detection of non-finite values

  • Accommodate upcoming change in package Matrix (Mikael Jagan in #417 addressing #415)

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 18 Jun 2023

spdl 0.0.5 on CRAN: Small Extension

Another quick update to the still somewhat new package spdl is now om CRAN, and will go to Debian soon too. The key focus of spdl is to offer the exact same interface to logging from both R and C++ by relying on spdlog via my RcppSpdlog package. Usage examples are shown on the RcppSpdlog docs page.

This release add support for the wrappers init() and log() wrapping the existing setup() function but requiring only the level argument. This requires version 0.0.13 of RcppSpdlog which was released to CRAN yesterday.

The short NEWS entry follows.

Changes in spdl version 0.0.5 (2023-06-18)

  • Add simple aliases init() and log() wrapping setup() but requiring only the logging level argument

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the spdl page

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/spdl | permanent link

Sat, 17 Jun 2023

RcppSpdlog 0.0.13 on CRAN: Small Extensions

Version 0.0.13 of RcppSpdlog is now on CRAN and will be soon be uploaded to Debian too. RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich. You can learn more at the package documention site.

This release adds a small (but handy) accessor generalisation: Instead of calling setup() with two arguments for a label and the logging level we now only require the desired level. We also cleaned up one implementation detail for the stopwatch feature added in January, and simplified the default C++ compilation standard setting.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.13 (2023-06-17)

  • Minor tweak to stopwatch setup avoids pulling in fmt

  • No longer set a C++ compilation standard as the default choices by R are sufficient for the package

  • Add convenience wrapper log_init omitting first argument to log_setup while preserving the interface from the latter

  • Add convenience setup wrappers init and log to API header file spdl.h

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page, or the package documention site.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 11 Jun 2023

sanitizers 0.1.1 on CRAN: Updated and Expanded

bleach

The second release of the sanitizers package is now on CRAN. sanitizers provides ‘true positives’ for programming errors detected by Address Sanitizers and friends. This permits validation of the setup when chasing such bug reports: it allows us to ascertain that the compiler (and instrumented R version) are correctly set up and the errors we expect to be reported are in fact reported.

Almost nine years (!!) since the first release, this update brings an added integer overflow sanitizer contributed by Greg Jeffries so long ago that I had thought it was part of the CRAN releases—my bad for delaying this. It also updates programming practices by switching to symbol registration for the compiled functions. And of course several R packaging best practices have improved since the initial release so we updated a few small things throughout.

A very good resources for all things sanitizers is the Google repo at GitHub and especially its wiki.

The brief NEWS entry follows.

Changes in version 0.1.1 (2023-06-11)

  • Added integer overflow example kindly contributed by Greg Jeffries

  • Added continuous integration and badges

  • Updated package to use symbol registration for compiled code

  • Updated and edited DESCRIPTION, README and help pages for current packaging standards

  • Expanded README with usage example via r-devel-san Rocker container

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page, the github repo, and the package documentation for more details.

If you like the open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/sanitizers | permanent link

Sat, 27 May 2023

RcppArmadillo 0.12.4.0.0 on CRAN: New Upstream Minor

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1074 other packages on CRAN, downloaded 29.3 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 535 times according to Google Scholar.

This release brings a new upstream release 12.4.0 made by Conrad a day or so ago. I prepared the usual release candidate, tested on the over 1000 reverse depends (which sadly takes almost a day on old hardware), found no issues and sent it to CRAN. Where it got tested again and was once again auto-processed smoothly by CRAN within a few hours on a Friday night which is just marvelous. So this time I tweeted about it too.

The releases actually has a relatively small set of changes as a second follow-up release in the 12.* series.

Changes in RcppArmadillo version 0.12.4.0.0 (2023-05-26)

  • Upgraded to Armadillo release 12.4.0 (Cortisol Profusion Redux)

    • Added norm2est() for finding fast estimates of matrix 2-norm (spectral norm)

    • Added vecnorm() for obtaining the vector norm of each row or column of a matrix

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like my open-source work, you may consider sponsoring me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 25 May 2023

qlcal 0.0.6 on CRAN: More updates from QuantLib

The sixth release of the still new-ish qlcal package arrivied at CRAN today.

qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more.

This release brings updates to a few calendars which happened since the QuantLib 1.30 release, and also updates a several of the (few) non-calendaring functions.

Changes in version 0.0.6 (2023-05-24)

  • Several calendars (India, Singapore, South Africa, South Korea) updated with post-QuantLib 1.3.0 changes (Sebastian Schmidt in #6)

  • Three now-used scheduled files were removed (Dirk in #7))

  • A number of non-calendaring files used were synchronised with the current QuantLib repo (Dirk in #8)

Last release, we also added a quick little demo using xts to column-bind calendars produced from each of the different US sub-calendars. This is a slightly updated version of the sketch we tooted a few days ago. The output now is

> print(Reduce(cbind, Map(makeHol, cals)))
           LiborImpact NYSE GovernmentBond NERC FederalReserve
2023-01-02        TRUE TRUE           TRUE TRUE           TRUE
2023-01-16        TRUE TRUE           TRUE   NA           TRUE
2023-02-20        TRUE TRUE           TRUE   NA           TRUE
2023-04-07          NA TRUE             NA   NA             NA
2023-05-29        TRUE TRUE           TRUE TRUE           TRUE
2023-06-19        TRUE TRUE           TRUE   NA           TRUE
2023-07-04        TRUE TRUE           TRUE TRUE           TRUE
2023-09-04        TRUE TRUE           TRUE TRUE           TRUE
2023-10-09        TRUE   NA           TRUE   NA           TRUE
2023-11-10        TRUE   NA             NA   NA             NA
2023-11-23        TRUE TRUE           TRUE TRUE           TRUE
2023-12-25        TRUE TRUE           TRUE TRUE           TRUE
> 

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/qlcal | permanent link

Sun, 14 May 2023

RcppSimdJson 0.1.10 on CRAN: New Upstream

We are happy to share that the RcppSimdJson package has been updated to release 0.1.10.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon.

This release updates the underlying simdjson library to version 3.1.8 (also made today). Otherwise we only made a minor edit to the README and adjusted one tweek for code coverage.

The (very short) NEWS entry for this release follows.

Changes in version 0.1.10 (2023-05-14)

  • simdjson was upgraded to version 3.1.8 (Dirk in #85).

Courtesy of my CRANberries, there is also a diffstat report for this release. For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 11 May 2023

crc32c 0.0.2 on CRAN: Build Fixes

A first follow-up to the initial announcement just days ago of the new crc32c package. The package offers cyclical checksum with parity in hardware-accelerated form on (recent enough) intel cpus as well as on arm64.

This follow-up was needed because I missed, when switching to a default static library build, that newest compilers would complain if -fPIC was not set. gcc-12 on my box was happy, gcc-13 on recent Fedora as used at CRAN was not. A second error was assuming that saying SystemRequirements: cmake would suffice. But hold on whippersnapper: macOS always has a surprise for you! As described at the end of the appropriate section in Writing R Extensions, on that OS you have to go the basement, open four cupboards, rearrange three shelves and then you get to use it. And then in doing so (from an added configure script) I failed to realize Windows needed a fallback. Gee.

The NEWS entry for this (as well the initial release) follows.

Changes in version 0.0.2 (2023-05-11)

  • Explicitly set cmake property for position-independent code

  • Help macOS find its cmake binary as detailed also in WRE

  • Help Windows with a non-conditional Makevars.win pointing at cmake

  • Add more badges to README.md

Changes in version 0.0.1 (2023-05-07)

  • Initial release version and CRAN upload

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/crc32c | permanent link

Mon, 08 May 2023

crc32c 0.0.1 on CRAN: New Package

Happy to announce a new package: crc32c. This arose out of a user request to add crc32c (which is related to but differnt from crc32 without the trailing c) to my digest package. Which I did (for now in a branch), using the software-fallback version of crc32c from the reference implementation by Google at their crc32c repo.

However, the Google repo also offers hardware-accelerated versions and switches at run-time. So I pondered a little about how to offer the additional performance without placing a dependency burden on all users of digest.

Lo and behold, I think I found a solution by reusing what R offers. First off, the crc32c package wraps the Google repo cleanly and directly. We include all the repo code – but not the logging or benchmarking code. This keeps external dependencies down to just cmake. Which while still rare in the CRAN world is at least not entirely uncommon. So now each time you build the crc32c R package, the upstream hardware detection is added transparently thanks in part to cmake. We build libcrc32c.a as a static library and include it in the R package and its own shared library.

And we added exporting of three key functions, directly at the C level, along with exporting one C level function of package that other packages can call. The distinction matters. The second step of offering a function R can call (also from other packages) is used and documented. By means of an Imports: statement to instantiate the package providing the functionality, the client package obtains access to a compiled functions its R code can then call. A number of other R packages use this.

But what is less well known is that we can do the same with C level functions. Again, it takes an imports statement but once that is done we can call ‘C to C’. Which is quite nice. I am working currently on the branch in digest which motivated this, and it can import the automagically hardware-accelerated functionality on the suitable hardware. Stay tuned for a change in digest.

I also won and lost the CRAN lottery for once: the package made it through the ‘new package’ checks without any revisions. Only to then immediately fail on the CRAN machines using gcc-13 as a -fPIC was seen as missing when making the shared library. Even though both my CI and the r-universe builds were all green. Ah well. So a 0.0.2 release will be coming in a day or two.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/crc32c | permanent link

Mon, 01 May 2023

RQuantLib 0.4.18 on CRAN: Maintenance

A new release 0.4.18 of RQuantLib arrived at CRAN earlier today, and will be uploaded to Debian as well.

QuantLib is a very comprehensice free/open-source library for quantitative finance; RQuantLib connects it to the R environment and language.

This release of RQuantLib comes about six months after the previous maintenance release. It brings a few small updates triggered by small changes in the QuantLib releases 1.29 and 1.30. It also contains updates reflecting changes in the rgl package kindly contributed by Duncan Murdoch. Last but not least, Jeroen Ooms helped with two pull requests updating builds on, repspectively, macOS and Windows by updating the pre-made libraries of QuantLib.

Changes in RQuantLib version 0.4.18 (2023-05-01)

  • Use of several rgl functions was updated to a new naming scheme in the package (kindly contributed by Duncan Murdoch in #174)

  • A default argument is now given for option surface plots

  • Changed call from SwaptionVolCube1 to SabrSwaptionVolatilityCube (conditional on using QuantLib 1.30 or later)

  • Some other deprecation warnings were tweaked as in QL test file

  • Builds for macOS and Windows were updated with more library build (changes kindly contributed by Jeron Ooms in #176 and #175)

  • Some remaining throw calls were replace by Rcpp::stop

Courtesy of my CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the new rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rquantlib | permanent link

Thu, 20 Apr 2023

qlcal 0.0.5 on CRAN: Updates from QuantLib 1.3.0

The fifth release of the still new-ish qlcal package arrivied at CRAN just now.

qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more.

This release brings updates to five calendars from the QuantLib 1.30 release from this week.

Changes in version 0.0.5 (2023-04-19)

  • Calendars routines for Australia, Denmark, New Zealand, Turkey and the US have been updated from QuantLib 1.30.

  • Support for 'Australia/ASX' has been added.

  • Added demo showing all US holidays in current year

We also added a quick little demo using xts to column-bind calendars produced from each of the different US sub-calendars. This is a slightly updated version of the sketch we tooted a few days ago. The output now is

> print(Reduce(cbind, Map(makeHol, cals)))
           LiborImpact NYSE GovernmentBond NERC FederalReserve
2023-01-02        TRUE TRUE           TRUE TRUE           TRUE
2023-01-16        TRUE TRUE           TRUE   NA           TRUE
2023-02-20        TRUE TRUE           TRUE   NA           TRUE
2023-04-07          NA TRUE             NA   NA             NA
2023-05-29        TRUE TRUE           TRUE TRUE           TRUE
2023-06-19        TRUE TRUE           TRUE   NA           TRUE
2023-07-04        TRUE TRUE           TRUE TRUE           TRUE
2023-09-04        TRUE TRUE           TRUE TRUE           TRUE
2023-10-09        TRUE   NA           TRUE   NA           TRUE
2023-11-10        TRUE   NA             NA   NA             NA
2023-11-23        TRUE TRUE           TRUE TRUE           TRUE
2023-12-25        TRUE TRUE           TRUE TRUE           TRUE
> 

(and we just discovered a tiny h -> hols bug in the demo so see the git repo – sorry!).

The release was finalized uploaded yesterday morning. But because we have to set CXX_STD=CXX14 as satisfy requirements of some of the Boost headers, we get ourselves a NOTE and with that a manual inspection … and a delay of 1 1/2 days. Not really all that meaningful in the grand scheme of things but still suboptimal relative to the fully automated passage this release should have gotten. Oh well.

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/qlcal | permanent link

Wed, 05 Apr 2023

RcppArmadillo 0.12.2.0.0 on CRAN: New Upstream Minor

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1052 other packages on CRAN, downloaded 28.6 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 522 times according to Google Scholar.

This release brings a new upstream release 12.2.0 made by Conrad a day or so ago. We prepared the usual release candidate, tested on the over 1000 reverse depends, found no issues and sent it to CRAN. Where it got tested again and was auto-processed smoothly by CRAN.

The releases actually has a relatively small set of changes as a first follow-up release in the 12.2.* series.

Changes in RcppArmadillo version 0.12.2.0.0 (2023-04-04)

  • Upgraded to Armadillo release 12.2.0 (Cortisol Profusion Deluxe)

    • more efficient use of FFTW3 by fft() and ifft()

    • faster in-place element-wise multiplication of sparse matrices by dense matrices

    • added spsolve_factoriser class to allow reuse of sparse matrix factorisation for solving systems of linear equations

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 26 Mar 2023

littler 0.3.18 on CRAN: New and Updated Scripts

max-heap image

The nineteenth release of littler as a CRAN package landed this morning, following in the now seventeen year history (!!) as a package started by Jeff in 2006, and joined by me a few weeks later.

littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only began to do in recent years.

littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default were a good idea?) and simply does not exist on Windows (yet – the build system could be extended – see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH. A few examples are highlighted at the Github repo, as well as in the examples vignette.

This release is somewhat unique is not having any maintenance of the small core but rather focussing exclusively on changes to the (many !!) scripts collected in examples/scripts/. We added new ones and extended several existing ones including install2.r thanks to patches by @eitsupi. Of the new scripts, I already use tttl.r quite a bit for tests on co-maintained packages that use testthat (but as I have made known before my heart belongs firmly to tinytest as nobody should need thirty-plus recursive dependencies to run a few test predicates) as well as the very new installRub.r to directly add r-universe Ubuntu binaries onto an r2u systems as demonstrated in a first and second tweet (and corresponding toot) this week.

The full change description follows.

Changes in littler version 0.3.18 (2023-03-25)

  • Changes in examples scripts

    • roxy.r can now set an additional --libpath

    • getRStudioDesktop.r and getRStudioServer.r have updated default download file

    • install2.r and installGithub.r can set --type

    • r2u.r now has a --suffix option

    • tt.r removes a redundant library call

    • tttl.r has been added for testthat::test_local()

    • installRub.r has been added to install r-universe binaries on Ubuntu

    • install2.r has updated error capture messages (Tatsuya Shima and Dirk in #104)

My CRANberries service provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page, and also on the package docs website. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian (currently in ‘experimental’ due to a release freeze) as well as (in a day or two) Ubuntu binaries at CRAN thanks to the tireless Michael Rutter — and of course via r2u following its next refresh.

Comments and suggestions are welcome at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/littler | permanent link

Wed, 22 Mar 2023

RcppSMC 0.2.7 on CRAN: Extensions and Update

A new release 0.2.7 of our RcppSMC package arrived at CRAN earlier today. It contains several extensions added by team member (and former GSoC student) Ilya Zarubin since the last release. We were a little slow to release those—but “one of those CRAN emails” forced our hand for a release now. The updated ‘uninitialized variable’ messages in clang++-16 have found a fan in Brian Ripley, and so he sent us a note. And as the issue was trivially reproducible with clang++-15 here too I had it fixed in no time. And both changes taken together form the incremental 0.2.7 release.

RcppSMC provides Rcpp-based bindings to R for the Sequential Monte Carlo Template Classes (SMCTC) by Adam Johansen described in his JSS article. Sequential Monte Carlo is also referred to as Particle Filter in some contexts. The package now also features the Google Summer of Code work by Leah South in 2017, and by Ilya Zarubin in 2021.

The release is summarized below.

Changes in RcppSMC version 0.2.7 (2023-03-22)

  • Extensive extensions for conditional SMC and resample, updated hello_world example, added skeleton function for easier package creation (Ilya in #67,#72)

  • Small package updates (Dirk in #75 fixing #74)

Courtesy of my CRANberries, there is a diffstat report for this release.

More information is on the RcppSMC page. Issues and bugreports should go to the GitHub issue tracker.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 11 Mar 2023

pkgKitten 0.2.3 on CRAN: Minor Update

kitten

A new release 0.2.3 of pkgKitten arrived on CRAN earlier, and will be uploaded to Debian. pkgKitten makes it simple to create new R packages via a simple function invocation. A wrapper kitten.r exists in the littler package to make it even easier.

This release improves the created ‘Description:’, and updated some of the continuous integration.

Changes in version 0.2.3 (2023-03-11)

  • Small improvement to generated Description: field and Title:

  • Maintenance for continuous integration setup

More details about the package are at the pkgKitten webpage, the pkgKitten docs site, and the pkgKitten GitHub repo.

Courtesy of my CRANberries site, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/pkgkitten | permanent link

Wed, 08 Mar 2023

RcppRedis 0.2.3 on CRAN: Maintenance

A new minor release 0.2.3 of our RcppRedis package arrived on CRAN today. RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has carried production loads on a trading floor for several years.

This update is fairly mechanical. CRAN wants everybody off the C++11 train which is fair game given that it 2023 and most sane and lucky people are facing sane and modern compilers so this makes sense. (And I raise a toast to all those poor souls facing RHEL 7 / CentOS 7 with a compiler from many moons ago: I hear it is a vibrant job market out there so maybe time to make a switch…). As with a few of my other packages, this release simply does away with the imposition of C++11 as the package will compile just fine under C++14 or C++17 (as governed by your version of R).

The detailed changes list follows.

Changes in version 0.2.3 (2023-03-08)

  • No longer set a C++ compilation standard as the default choices by R are sufficient for the package

  • Switch include to Rcpp/Rcpp which signals use of all Rcpp features including Modules

Courtesy of my CRANberries, there is also a diffstat report for this release. More information is on the RcppRedis page.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 06 Mar 2023

RcppFastAD 0.0.1 and 0.0.2: New Package on CRAN!

James Yang and I are thrilled to announce the new CRAN package RcppFastAD which arrived at CRAN last Monday as version 0.0.1, and is as of today at version 0.0.2 with a first set of small updates.

It is based on the FastAD header-only C++ library by James which provides a C++ implementation of both forward and reverse mode of automatic differentiation in an easy-to-use header library (which we wrapped here) that is both lightweight and performant. With a little of bit of Rcpp glue, it is also easy to use from R in simple C++ applications. Included in the package are three example: a simple quadratic expression evaluating x' S x for given x and S return the expression value with a gradient, a linear regression example generalising this and using the gradient to derive to arrive at the least-squares minimizing solution, as well as the well-known Black-Scholes options pricer and its important partial derivatives delta, rho, theta and vega derived via automatic differentiation.

The NEWS file for these two initial releases follows.

Changes in version 0.0.2 (2023-03-05)

  • One C++ operation is protected from operating on a nullptr

  • Additional tests have been added, tests now cover all three demo / example functions

  • Return values and code for the examples linear_regression and quadratic_expression have been adjusted

Changes in version 0.0.1 (2023-02-24)

  • Initial release version and CRAN upload

Courtesy of my CRANberries, there is also a diffstat report for the most recent release. More information is available at the repository or the package page.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 24 Feb 2023

ttdo 0.0.9 on CRAN: Small Update

A new minor release of our ttdo package arrived on CRAN a few days ago. The ttdo package extends the excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam to give us test results with visual diffs (as shown in the screenshot below) which seemingly is so compelling an idea that it eventually got copied by another package which shall remain unnamed…

ttdo screenshot

This release adds a versioned dependency on the just released tinytest version 1.4.1. As we extend tinytest (for use in the autograder we deploy within the lovely PrairieLearn framework) by consuming the tinytest code we have to update in sync.

There were no other code changes in the package beside the usual maintenance of badges and continuous integration setup.

As usual, the NEWS entry follows.

Changes in ttdo version 0.0.9 (2023-02-21)

  • Minor cleanup in README.md

  • Minor continuous integration update

  • Updated (versioned) depends on tinytest to 1.4.1

My CRANberries provides the usual summary of changes to the previous version. Please use the GitHub repo and its issues for any questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/ttdo | permanent link

Wed, 22 Feb 2023

RcppArmadillo 0.12.0.1.0 on CRAN: New Upstream, New Features

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1042 other packages on CRAN, downloaded 28.1 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 513 times according to Google Scholar.

This release brings a new upstream release 12.0.1. We found a small regression with the 12.0.0 release when we tested prior to a CRAN upload. Conrad very promptly fixed this with a literal one liner and made it 12.0.1 which we wrapped up as 0.12.0.1.0. Subsequent testing revealed no issues for us, and CRAN autoprocessed it as I tweeted earlier. This is actually quite impressive given the over 1000 CRAN packages using it all of which got tested again by CRAN. All this is testament to the rigour, as well as the well-oiled process at the repository. Our thanks go to the tireless maintainers!

The releases actually has a rather nice set of changes (detailed below) to which we added one robustification thanks to Kevin.

The full set of changes follows. We include the previous changeset as we may have skipped the usual blog post here.

Changes in RcppArmadillo version 0.12.0.1.0 (2023-02-20)

  • Upgraded to Armadillo release 12.0.1 (Cortisol Profusion)

    • faster fft() and ifft() via optional use of FFTW3

    • faster min() and max()

    • faster index_min() and index_max()

    • added .col_as_mat() and .row_as_mat() which return matrix representation of cube column and cube row

    • added csv_opts::strict option to loading CSV files to interpret missing values as NaN

    • added check_for_zeros option to form 4 of sparse matrix batch constructors

    • inv() and inv_sympd() with options inv_opts::no_ugly or inv_opts::allow_approx now use a scaled threshold similar to pinv()

    • set_cout_stream() and set_cerr_stream() are now no-ops; instead use the options ARMA_WARN_LEVEL, or ARMA_COUT_STREAM, or ARMA_CERR_STREAM

    • fix regression (mis-compilation) in shift() function (reported by us in #409)

  • The include directory order is now more robust (Kevin Ushey in #407 addressing #406)

Changes in RcppArmadillo version 0.11.4.4.0 (2023-02-09)

  • Upgraded to Armadillo release 11.4.4 (Ship of Theseus)

    • extended pow() with various forms of element-wise power operations

    • added find_nan() to find indices of NaN elements

    • faster handling of compound expressions by sum()

  • The package no longer sets a compilation standard, or progagates on in the generated packages as R ensures C++11 on all non-ancient versions

  • The CITATION file was updated to the current format

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 01 Feb 2023

RInside 0.2.18 on CRAN: Maintenance

A new release 0.2.18 of RInside arrived on CRAN and in Debian today. This is the first release in ten months since the 0.2.17 release. RInside provides a set of convenience classes which facilitate embedding of R inside of C++ applications and programs, using the classes and functions provided by Rcpp.

This release brings a contributed change to how the internal REPL is called: Dominick found the current form more reliable when embedding R on Windows. We also updated a few other things around the package.

The list of changes since the last release:

Changes in RInside version 0.2.18 (2023-02-01)

  • The random number initialization was updated as in R.

  • The main REPL is now running via 'run_Rmainloop()'.

  • Small routine update to package and continuous integration.

My CRANberries also provides a short report with changes from the previous release. More information is on the RInside page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page, or to issues tickets at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rinside | permanent link

Mon, 30 Jan 2023

#39: Faster Feedback Systems – A Continuous Integration Example

Welcome to the 39th post in the relatively randomly recurring rants, or R4 for short. Today’s post picks up where the previous post #38: Faster Feedback Systems started. As we argued in #38, the need for fast feedback loops is fairly universal and widespread. Fairly shortly after we posted #38, useR! 2022 happened and one presentation had the key line

Waiting 1-24 minutes for a build to finish can be a massive time suck.

which we would like to come back to today. Furthermore, the unimitable @b0rk had a fabulous tweet just weeks later stating the same point debugging strategy: shorten your feedback loop as a key in a successful debugging strategy.

So in sum: shorter is better. Nobody likes to wait. And shorter i.e. faster is a key and recurrent theme in the R4 series. Today we have a fairly nice illustration of two aspects we have stressed before:

  • Fewer dependencies makes for faster installation time (apart from other desirable robustness aspects); and

  • Using binaries makes for faster installation time as it removes the need for compilations.

The combined effects can be staggering as we show below. The example is motivated by a truly “surprising” (we are being generous here) comment we received as an aside when discussing the eternal topic of whether R users do, or do not, have a choice when picking packages, or approaches, or verses. To our surprise, we were told that “packages are not substitutable”. Which is both demonstrably false (see below) and astonishing as it came from an academic. I.e. someone trained and paid to abstract superfluous detail away and recognise and compare ‘core’ features of items under investigation. Truly stunning. But I digress.

CRAN by now has many packages, slowly moving in on 20,000 in total, and is a unique success we commented-on time and time before. By now many packages shadow or duplicate each other, reinvent one another, or revise/review. One example is the pair of packages accessing PostgreSQL databases. There are several, but two are key. The older one is RPostgreSQL which has been around since Sameer Kumar Prayaga wrote it as a Google Summer of Code student in 2008 under my mentorship. The package has now been maintained well (if quietly) for many years by Tomoaki Nishiyama. The other entrant is more recent, though not new, and is RPostgres by Kirill Müller and others. We will for the remainder of this post refer to these two as the tiny and the tidy version as either can been as being a representative of a certain ‘verse’.

The aforementioned comment on non-substitutability struck us as eminently silly, so we set out to prove just how absurd it really is. So about a year ago we set up pair of GitHub repos with minimal code in a pair we called lim-tiny and lim-tidy. Our conjecture was (and is!) that less is more – just as post #34 titled Less Is More argued with respect to package dependencies. Each of the repos just does one thing: a query to a (freely accessible but remote) PostgreSQL database. The tiny version just retrieves a data frame using only the dependencies needed for RPostgreSQL namely DBI and nothing else. The tidy version retrieves a tibble and has access to everything else that comes when installing RPostgres: DBI, dplyr, and magrittr – plus of course their respective dependencies. We were able to let the code run in (very default) GitHub Actions on a weekly schedule without intervention apart from one change to the SQL query when the remote server (providing public bioinformatics data) changed its schema slighly, plus one update to the action yaml code version. No other changes.

We measure the time a standard continuous integration run takes in total using the default GitHub Action setup in the tidy case (which relies on RSPM/PPM binaries, caching, …, and does not rebuild from source each time), and our own r-ci script (introduced in #32 for CI with R. It switched to using r2u during the course of this experiment but already had access to binaries via c2d4u – so it is always binaries-based (see e.g. #37 or #13 for more). The chart shows the evolution of these run-times over the course of the year with one weekly run each.

tiny vs tidy ci timing impact chart

This study reveals a few things:

  • “Yes, Veronica, we have a choice”: one year of identical results retrieving a data set via SQL from a PostgreSQL server. We can often choose between alternative packages to get (or process) our data.
  • There is a Free Lunch (TM) here: RPostgreSQL consistently dominates RPostgres when measured in CI run-time for the same task.
  • While there is (considerable) variability (likely stemming from heterogenous setups at GitHub Action) the tiny approach is on average about twice as fast as the tidy approch.
  • You do have a choice. Would you rather wait x minutes for CI, or 2x?

The key point there is that while the net-time to fire off a single PostgreSQL is likely (near-)identical, the net cost of continuous integration is not. In this setup, it is about twice the run-time simply because ‘Less Is More’ which (in this setup) comes out to about being twice as fast. And that is a valid (and concrete, and verifiable) illustration of the overall implicit cost of adding dependencies to creating and using development, test, or run-time environments.

Faster feedback loops make for faster builds and faster debugging. Consider using fewer dependencies, and/or using binaries as provided by r2u.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Sun, 29 Jan 2023

RcppTOML 0.2.2 on CRAN: Now with macOS-on-Intel Builds

Just days after a build-fix release (for aarch64) and still only a few weeks after the 0.2.0 release of RcppTOML and its switch to toml++, we have another bugfix release 0.2.2 on CRAN also bringing release 3.3.0 of toml++ (even if we had large chunks of 3.3.0 already incorporated).

TOML is a file format that is most suitable for configurations, as it is meant to be edited by humans but read by computers. It emphasizes strong readability for humans while at the same time supporting strong typing as well as immediate and clear error reports. On small typos you get parse errors, rather than silently corrupted garbage. Much preferable to any and all of XML, JSON or YAML – though sadly these may be too ubiquitous now. TOML is frequently being used with the projects such as the Hugo static blog compiler, or the Cargo system of Crates (aka “packages”) for the Rust language.

The package was building fine on Intel-based macOS provided the versions were recent enough. CRAN, however, aims for the broadest possibly reach of binaries and builds on a fairly ancient macOS 10.13 with clang version 10. This confused toml++ into (wrongly) concluding it could not build when it in fact can. After a hint from Simon that Apple in their infinite wisdom redefines clang version ids, this has been reflected in version 3.3.0 of toml++ by Mark so we should now build everywhere. Big thanks to everybody for the help.

The short summary of changes follows.

Changes in version 0.2.2 (2023-01-29)

  • New toml++ version 3.3.0 with fix to permit compilation on ancient macOS systems as used by CRAN for the Intel-based builds.

Courtesy of my CRANberries, there is a diffstat report for this release. More information is on the RcppTOML page page. Please use the GitHub issue tracker for issues and bugreports.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link