Sat, 17 Apr 2021

RcppAPT 0.0.7: Micro Update

A new version of the RcppAPT package interfacing from R to the C++ library behind the awesome apt, apt-get, apt-cache, … commands and their cache powering Debian, Ubuntu and the like arrived on CRAN yesterday. This comes a good year after the previous maintenance update for release 0.0.6.

RcppAPT allows you to query the (Debian or Ubuntu) package dependency graph at will, with build-dependencies (if you have deb-src entries), reverse dependencies, and all other goodies. See the vignette and examples for illustrations.

The maintenance release responds to call for updates from CRAN desiring that make all implicit dependencies on packages markdown and rmarkdown explicit via a Suggests: entry. Two of the many packages I maintain were part of the (large !!) list in the CRAN email, and this is one of them. While making the update, we refreshed two other packaging details.

Changes in version 0.0.7 (2021-04-16)

  • Add rmarkdown to Suggests: as an implicit conditional dependency

  • Switch vignette to minidown and its water framework, add minidown to Suggests as well

  • Update two URLs in the README.md file

Courtesy of my CRANberries, there is also a diffstat report for this release. A bit more information about the package is available here as well as as the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 13 Apr 2021

RcppArmadillo 0.10.4.0.0 on CRAN: New Upstream ‘Plus’

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 852 other packages on CRAN.

This new release brings us the just release Armadillo 10.4.0. Upstream moves at a speed that is a little faster than the cadence CRAN likes. We release RcppArmadillo 0.10.2.2.0 on March 9; and upstream 10.3.0 came out shortly thereafter. We aim to accomodate CRAN with (roughly) monthly (or less frequent) releases) so by the time we were ready 10.4.0 had just come out.

As it turns, the full testing had a benefit. Among the (currently) 852 CRAN packages using RcppArmadillo, two were failing tests. This is due to a subtle, but important point. Early on we realized that it would be beneficial if the standard R control over random-number creation and seeding affected Armadillo too, which Conrad accomodated kindly with an optional RNG interface—which RcppArmadillo supplies. With recent changes he made, the R side saw normally-distributed draws (via the Armadillo interface) changed, which lead to the two changes. All hail unit tests. So I mentioned this to Conrad, and with the usual Chicago-Brisbane time difference late my evening a fix was in my inbox. The CRAN upload was then halted as I had missed that due to other changes he had made random draws from a Gamma would now call std::rand() which CRAN flags. Another email to Brisbane, another late (one-line) fix back and all was good. We still encountered one package with an error but flagged this as internal to that package’s setup, so Uwe let RcppArmadillo onto CRAN, I contacted that package’s maintainer—who was very receptive and a change should be forthcoming. So with all that we have 0.10.4.0.0 on CRAN giving us Armadillo 10.4.0.

The full set of changes follows. As Armadillo 10.3.0 was not uploaded to CRAN, its changes are included too.

Changes in RcppArmadillo version 0.10.4.0.0 (2021-04-12)

  • Upgraded to Armadillo release 10.4.0 (Pressure Cooker)

    • faster handling of triangular matrices by log_det()

    • added log_det_sympd() for log determinant of symmetric positive matrices

    • added ARMA_WARN_LEVEL configuration option, to control the degree of emitted warning messages

    • reduced the default degree of warning messages, so that failed decompositions, failed saving/loading, etc, no longer emit warnings

  • Apply one upstream corrections for arma::randn draws when using alternative (here R) generator, and arma::randg.

Changes in RcppArmadillo version 0.10.3.0.0 (2021-03-10)

  • Upgraded to Armadillo release 10.3 (Sunrise Chaos)

    • faster handling of symmetric positive definite matrices by pinv()

    • expanded .save() / .load() for dense matrices to handle coord_ascii format

    • for out of bounds access, element accessors now throw the more nuanced std::out_of_range exception, instead of only std::logic_error

    • improved quality of random numbers

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 31 Mar 2021

Rcpp now used by 2250 CRAN packages!

2250 Rcpp packages

As of today, Rcpp stands at 2255 reverse-dependencies on CRAN. The graph on the left depicts the growth of Rcpp usage (as measured by Depends, Imports and LinkingTo, but excluding Suggests) over time. We actually crossed 2250 once a week ago, but “what CRAN giveth, CRAN also taketh” and counts can fluctuate. It had dropped back to 2248 a few days later.

Rcpp was first released in November 2008. It probably cleared 50 packages around three years later in December 2011, 100 packages in January 2013, 200 packages in April 2014, and 300 packages in November 2014. It passed 400 packages in June 2015 (when I tweeted about it), 500 packages in late October 2015, 600 packages in March 2016, 700 packages last July 2016, 800 packages last October 2016, 900 packages early January 2017, 1000 packages in April 2017, 1250 packages in November 2017, 1500 packages in November 2018, 1750 packages in August 2019, and then the big 2000 packages (as well as one in eight) in July 2020. The chart extends to the very beginning via manually compiled data from CRANberries and checked with crandb. The next part uses manually saved entries. The core (and by far largest) part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of packages using Rcpp is available too.

Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited keynote. We passed five percent in December of 2014, six percent July of 2015, seven percent just before Christmas 2015, eight percent in the summer of 2016, nine percent mid-December 2016, cracked ten percent in the summer of 2017 and eleven percent in 2018. Last year, along with passing 2000 package, we also passed 12.5 percent—so one more than in every eight CRAN packages depends on Rcpp. Stunning. There is more detail in the chart: how CRAN seems to be pushing back more and removing more aggressively (which my CRANberries tracks but not in as much detail as it could), how the growth of Rcpp seems to be slowing somewhat outright and even more so as a proportion of CRAN – as one would expect a growth curve to.

2250 user packages, and the continued growth, is truly mind-boggling. We can use the progression of CRAN itself compiled by Henrik in a series of posts and emails to the main development mailing list. Not that long ago CRAN itself did have only 1000 packages, then 5000, 10000, and here we are at over 17300 with Rcpp now at nearly 13.0% and still growing. Amazeballs.

The Rcpp team, recently grown in strength with the addition of Iñaki, continues to aim for keeping Rcpp as performant and reliable as it has been. A really big shoutout and Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 28 Mar 2021

RcppSpdlog 0.0.5 on CRAN: New upstream versions

About three months after the last update, we can announce a new version 0.0.5 of RcppSpdlog. It contains releases 1.8.3, 1.8.4 and 1.8.5 of spdlog which were made in quick succession mid-week (while we were waiting on an update of CRAN’s own machinery) and was processed yesterday and overnight.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.5 (2020-12-11)

  • Upgraded to upstream release spdlog 1.8.5 (and 1.8.4 and 1.8.3)

  • Small enhancements to DESCRIPTION files

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 09 Mar 2021

RcppArmadillo 0.10.2.2.0 on CRAN: New Upstream Patch Release

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 841 other packages on CRAN.

This release brings us a second update within the Armadillo 10.2 series, one month after the initial release. Upstream work has commenced at a 10.3 series with first pre-releases which we are currently testing. If possible, CRAN release will continue to be at least one month apart (such as this time) but we continue to make interim releases available on the Rcpp drat repo repo.

The full set of changes follows.

Changes in RcppArmadillo version 0.10.2.2.0 (2021-03-09)

  • Upgraded to Armadillo release 10.2.2 (Cicada Swarm)

    • faster handling of subcubes

    • added tgamma()

    • added .brief_print() for abridged printing of matrices & cubes

    • expanded forms of trimatu() and trimatl() with diagonal specification to handle sparse matrices

    • expanded eigs_sym() and eigs_gen() with optional shift-invert mode

  • Removed debian/ directory from repository as packaging is on salsa.debian.org.

  • Relaxed tolerance on two cube tests on Windows to accomodate new 'gcc10-UCRT' builder.

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 24 Feb 2021

RcppSimdJson 0.1.4 on CRAN: Small Bugfix

A quick note to say that we finalized a bugfix release 0.1.5 of RcppSimdJson yesterday which got onto CRAN earlier today. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

This version brings a small fix to Windows where temporary filenames constructed from URLs (as for example constructed by our td package) need an extra sanization for a possible ‘?’ character. We will be addressing that properly “in due course” but for now a simple gsub() will do. The NEWS entry follows.

Changes in version 0.1.5 (2021-02-23)

  • Temporary filenames from request URLs need an extra path sanitization on Windows as seen with package td.

  • A few #nocov tags were added to the code.

Courtesy of my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 13 Feb 2021

RcppFastFloat 0.0.2: New Function

The second release of RcppFastFloat is now on CRAN. The package wraps fastfloat, another nice library by Daniel Lemire who showed in a recent arXiv paper that one can convert character representations of ‘numbers’ into floating point at rates at or exceeding one gigabyte per second.

Thanks to Brendan, this release adds a helper function as.double2() modeled after the base R function but using, of course, the features from fast_float in RcppFastFloat.

Release notes follow.

Changes in version 0.0.2 (2021-02-13)

  • New function as.double2() demonstrating fast_float (Brendan in #1)

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 12 Feb 2021

RcppSimdJson 0.1.4 on CRAN: New Improvements

Brendan and I are happy to share that a new RcppSimdJson release 0.1.4 arrived on CRAN earlier today. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

This version brings a new option to always return list types, tweaks to setting option in the the request and other small improvements. The NEWS entry follows.

Changes in version 0.1.4 (2021-02-12)

  • Support additional headers in fload (Dirk in #60).

  • Enable continuous integration via GitHub Actions using run.sh from r-ci repo (Dirk in #61, #62).

  • Add option to always return list to fparse()/fload() (Brendan in #65 closing #64).

Courtesy of my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 10 Feb 2021

RcppSMC 0.2.3 on CRAN: Updated Snapshot

A new release 0.2.3 of the RcppSMC package arrived on CRAN earlier today. Once again it progressed as a very quick pretest-publish within minutes of submission—thanks CRAN!

RcppSMC provides Rcpp-based bindings to R for the Sequential Monte Carlo Template Classes (SMCTC) by Adam Johansen described in his JSS article. Sequential Monte Carlo is also referred to as Particle Filter in some contexts.

This release somewhat belatedly merges a branch Leah had been working on and which we all realized “is ready”. We now have a good snapshot to base new work on, as maybe with the Google Summer of Code 2021.

Changes in RcppSMC version 0.2.3 (2021-02-10)

  • Addition of a Github Action CI runner (Dirk)

  • Switching to inheritance for the moveset rather than pointers to functions (Leah in #45).

Courtesy of my CRANberries, there is a diffstat report for this release.

More information is on the RcppSMC page. Issues and bugreports should go to the GitHub issue tracker.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 09 Feb 2021

RcppArmadillo 0.10.2.1.0: New Upstream Release

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 823 other packages on CRAN.

This release brings us Armadillo 10.2 with a few updates as detailed below in the list of changes. Upstream release 10.2 was made a couple of days ago, but we need to balance new upstream updates with a responsible release cadence at CRAN. As we needed a maintenance release in early January, I opted to wait four weeks with this one which hence gets us 10.2.0 and 10.2.1 at once. As tweeted (with a follow-up) it had yet another very smooth passage at CRAN so we again appreciate the excellent work of the CRAN maintainers and say Thank You!.

Anybody who desires more frequent updates show look at the RcppCore drat repo which provides more frequent interim updates. Here for example we also had 0.10.2.0.0 available for your testing pleasure.

Also of note is that here is now a Python variant pyarma for those who might want enjoy Armadillo with Python.

The full set of changes follows.

Changes in RcppArmadillo version 0.10.2.1.0 (2021-02-09)

  • Upgraded to Armadillo release 10.2.1 (Cicada Swarm)

    • faster handling of subcubes

    • added tgamma()

    • added .brief_print() for abridged printing of matrices & cubes

    • expanded forms of trimatu() and trimatl() with diagonal specification to handle sparse matrices

    • expanded eigs_sym() and eigs_gen() with optional shift-invert mode

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 05 Feb 2021

RcppFastFloat 0.0.1: New Package, Already on CRAN

A new package, once again based on wonderful library by Daniel Lemire, is now on CRAN in its initial version 0.0.1. Daniel, in a recent arXiv paper shows that one can convert character representations of ‘numbers’ into floating point at rates at or exceeding one gigabyte per second. His tests show a fourfold gain over library functions such as strtod.

We put a simply package together showing use of the example parser, and containing a simple ‘all-in’ comparison benchmark (where we time the function call overhead as well) and get roughly 3x. See the repo for details; we are borring the table and figure here:

> source("comparison.R")
Unit: milliseconds
      expr      min       lq     mean   median       uq      max neval cld
     scanf 218.8936 224.1223 238.5650 227.1901 229.9116 1343.433   100   c
      atof 124.8087 127.3274 129.4104 128.5858 130.9138  146.334   100  b 
    strtod 124.5705 127.2157 129.1238 129.1042 130.7504  137.143   100  b 
      stod 127.1751 129.7343 131.7339 131.4854 133.1425  147.763   100  b 
 fastfloat  40.6219  41.3042  42.5729  42.3209  43.1738   57.788   100 a  
> 

Or in chart form:

Not much to say yet for the initial release:

Changes in version 0.0.1 (2021-01-31)

  • Initial version and CRAN upload

While the package was waiting to be added to CRAN, Brendan already added a potential as.double() replacement which will be in the next version.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 30 Jan 2021

RcppDate 0.0.2: Regular Update

RcppDate wraps the date library by Howard Hinnant for use with R. This header-only modern C++ library has been in pretty wide-spread use for a while now, and adds to C++11/C++14/C++17 what will be (with minor modifications) the ‘date’ library in C++20. Since the original 0.0.1 CRAN release I have also added this package along with RcppCCTZ and nanotime (which uses / requires both) to Debian so an apt based install is also possible for some.

Release 0.0.2 arrived on CRAN yesterday. It simply updates the included library to version upstream version 3.0.0, and touches up some packaging internals for continued integration.

Changes in version 0.0.2 (2021-01-29)

  • Added GitHub Actions CI via run.sh from r-ci

  • Updated to upstream version 3.0.0 (plus newer commits)

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 15 Jan 2021

Rcpp 1.0.6: Some Updates

rcpp logo

The Rcpp team is proud to announce release 1.0.6 of Rcpp which arrived at CRAN earlier today, and has been uploaded to Debian too. Windows and macOS builds should appear at CRAN in the next few days. This marks the first release on the new six-months cycle announced with release 1.0.5 in July. As reminder, interim ‘dev’ or ‘rc’ releases will often be available in the Rcpp drat repo; this cycle there were four.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 2174 packages on CRAN depend on Rcpp for making analytical code go faster and further (which is an 8.5% increase just since the last release), along with 207 in BioConductor.

This release features six different pull requests from five different contributors, mostly fixing fairly small corner cases, plus some minor polish on documentation and continuous integration. Before releasing we once again made numerous reverse dependency checks none of which revealed any issues. So the passage at CRAN was pretty quick despite the large dependency footprint, and we are once again grateful for all the work the CRAN maintainers do.

Changes in Rcpp patch release version 1.0.6 (2021-01-14)

  • Changes in Rcpp API:

    • Replace remaining few uses of EXTPTR_PTR with R_ExternalPtrAddr (Kevin in #1098 fixing #1097).

    • Add push_back and push_front for DataFrame (Walter Somerville in #1099 fixing #1094).

    • Remove a misleading-to-wrong comment (Mattias Ellert in #1109 cleaning up after #1049).

    • Address a sanitizer report by initializing two private bool variables (Benjamin Christoffersen in #1113).

    • External pointer finalizer toggle default values were corrected to true (Dirk in #1115).

  • Changes in Rcpp Documentation:

    • Several URLs were updated to https and/or new addresses (Dirk).
  • Changes in Rcpp Deployment:

    • Added GitHub Actions CI using the same container-based setup used previously, and also carried code coverage over (Dirk in #1128).
  • Changes in Rcpp support functions:

    • Rcpp.package.skeleton() avoids warning from R. (Dirk)

Thanks to my CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2616 previous questions.

If you like this or other open-source work I do, you can sponsor me at GitHub. My sincere thanks to my current sponsors for me keeping me caffeinated.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 10 Jan 2021

RcppArmadillo 0.10.1.2.2: Minor update

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 802 other packages on CRAN.

This release was needed because we use the Matrix package for some (optional) tests related to sparse matrices, and a small and subtle change and refinement in the recent 1.3.0 release of Matrix required us to make an update for the testing. Nothing has changed in how we set up, or operate on, sparse matrices. My thanks to Binxiang and Martin Maechler for feedback and suggestion on the initial fix both Binxiang and I set up independently. At the same time we upgrade some package internals related to continuous integration (for that, also see my blog post and video from earlier this week). Lastly Conrad sent in a one-line upstream fix for dealing with NaN in sign().

The full set of changes follows.

Changes in RcppArmadillo version 0.10.1.2.2 (2021-01-08)

  • Correct one unit test for Matrix 1.3.0-caused changed (Binxiang in #319 and Dirk in #322).

  • Suppress one further warning from Matrix (Dirk)

  • Apply an upstream NaN correction (Conrad in #321)

  • Added GitHub Actions CI using run.sh from r-ci (Dirk)

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 17 Dec 2020

RcppEigen 0.3.3.9.1: New upstream

A new release 0.3.3.9.1 of RcppEigen arrived on CRAN today (and just went to Debian too) bringing support for Eigen 3.3.9 to R. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.

The release started with a gentle nudge from the Stan team to package Eigen 3.3.8. And once I got around to it and sat down, Eigen 3.3.9 had just been released the previous day. The freshly prepared release 0.3.3.9.0 was then announced on Twitter (and the rcpp-devel mailing list) along with a request for testing alongside the usual extended reverse dependency checking we do (just like CRAN asks us to). And it turns out that Eigen 3.3.9 required an update to StanHeaders, a CRAN package that is itself quite widely used. This took a couple of days and its round of testing, but Ben Goodrich shipped it to CRAN last night, so we were able to ship RcppEigen in turn today.

So this release once again bring few required changes to Eigen (see this directory for details). As we said for the previous three releases:

One additional and recent change was the accomodation of a recent CRAN Policy change to not allow gcc or clang to mess with diagnostic messages. A word of caution: this may make your compilation of packages using RcppEigen very noisy so consider adding -Wno-ignored-attributes to the compiler flags added in your ~/.R/Makevars.

I actually find this requirement rather annoying. Eigen is then only usable if you set, say,

-Wno-deprecated-declarations -Wno-parentheses -Wno-ignored-attributes -Wno-unused-function

as g++ options in ~/.R/Makevars. But CRAN makes the rules. Maybe if a few of us gently and politely nudge them they may relent one day. One can only hope.

The complete NEWS file entry follows.

Changes in RcppEigen version 0.3.3.9.1 (2020-12-17)

  • Upgraded to Eigen 3.3.9 (Dirk in #92 fixing #91).

  • Added GitHub Actions CI using run.sh from r-ci (Dirk)

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 15 Dec 2020

RcppAnnoy 0.0.18: Tweaks

annoy image

A new maintenance release, now at version 0.0.18, of RcppAnnoy is now on CRAN. RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release is follow-up to release 0.0.17 which was made four weeks ago, and which brought the new upstream release 1.17 of Annoy. We now have an updated upstream with a PR by Aaron aiming for improved control of RNG seeding to ensure better reproducibility, along with extended tests and new helpers for RcppAnnoy and Annoy version numbers. The release was once again coordinated with Annoy and James whose BiocNeighbors and uwot packages both consume the Annoy header library shipped here.

Detailed changes follow below.

Changes in version 0.0.18 (2020-12-15)

  • Small tweaks to threading policy header defines (Dirk closing #65)

  • Vignette code is again compiled during testing (Aaron Lum and Dirk in #66 addressing #64)

  • Upstream code (with Aaron's PR) was synchronized once more (Dirk in #67)

  • A new helper function was added to report the Annoy version (Aaron in #68)

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 11 Dec 2020

RcppSpdlog 0.0.4: New upstream

Not quite two months after the last release, we are happy to announce version 0.0.4 of RcppSpdlog. It contains release 1.8.2 of spdlog made today, along with version 7.1.3 of fmt.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.4 (2020-12-11)

  • Upgraded to upstream release spdlog 1.8.2

  • Added GitHub Actions CI using run.sh from r-ci

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 02 Dec 2020

RcppTOML 0.1.7: Support for g++-11, Minor Updates

A new RcppTOML release arrived on CRAN earlier today evening. RcppTOML brings TOML to R.

TOML is a file format that is most suitable for configurations, as it is meant to be edited by humans but read by computers. It emphasizes strong readability for humans while at the same time supporting strong typing as well as immediate and clear error reports. On small typos you get parse errors, rather than silently corrupted garbage. Much preferable to any and all of XML, JSON or YAML – though sadly these may be too ubiquitous now. TOML has been making inroads with projects such as the Hugo static blog compiler, or the Cargo system of Crates (aka “packages”) for the Rust language.

CRAN had sent us a note that the package no longer compiled under the [unreleased, of course, never change, BDR ;-) ] g++-11 compiler, but were kind enough to hint that it was only lacking an #include <limits>. These things happen: newer compilers are generally more strict, and that is generally a good things. (Last year this time we prepped code for the more stringent view on global variables under gcc-10. Earlier g++ version had similar demands to clarify include headers.) I set up a simple Docker contain with on Ubuntu 21.04 with g++-11, R, and Rcpp to build the package and make this change (which was of course also PR’ed upstream at cpptoml), plus some other small ones that update the package since the last release roughly 18 months ago. We also switched CI use to the r-ci setup I should blog about a little more, removed a bashism and updated a few URLs. The bulleted list of changes in this version follows.

Changes in version 0.1.7 (2020-12-01)

  • Add #include <limits> to header file, also contributed upstream, to permit compilation under the (unreleased) g++-11.

  • Switch the simple cleanup script to sh.

  • Switch CI use to r-ci for focal and bspm.

  • Update several TOML URLs to https://toml.io/en/.

Courtesy of my CRANberries, there is a diffstat report for this release. More information is on the RcppTOML page page. Please use the GitHub issue tracker for issues and bugreports.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 16 Nov 2020

RcppArmadillo 0.10.1.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 779 other packages on CRAN.

This release ties up a few loose ends from the recent 0.10.1.0.0.

Changes in RcppArmadillo version 0.10.1.2.0 (2020-11-15)

  • Upgraded to Armadillo release 10.1.2 (Orchid Ambush)

  • Remove three unused int constants (#313)

  • Include main armadillo header using quotes instead of brackets

  • Rewrite version number use in old-school mode because gcc 4.8.5

  • Skipping parts of sparse conversion on Windows as win-builder fails

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

RcppAnnoy 0.0.17

annoy image

A new release 0.0.17 of RcppAnnoy is now on CRAN. RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release brings a new upstream version 1.17, released a few weeks ago, which adds multithreaded index building. This changes the API by adding a new ‘threading policy’ parameter requiring code using the main Annoy header to update. For this reason we waited a little for the dust to settle on the BioConductor 3.12 release before bringing the changes to BiocNeighbors via this commit and to uwot via this simple PR. Aaron and James updated their packages accordingly so by the time I uploaded RcppAnnoy it made for very smooth sailing as we all had done our homework with proper conditional builds, and the package had no other issue preventing automated processing at CRAN. Yay. I also added a (somewhat overdue one may argue) header file RcppAnnoy.h regrouping defines and includes which should help going forward.

Detailed changes follow below.

Changes in version 0.0.17 (2020-11-15)

  • Upgrade to Annoy 1.17, but default to serial use.

  • Add new header file to regroup includes and defines.

  • Upgrade CI script to use R with bspm on focal.

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 02 Nov 2020

RcppSimdJson 0.1.2: New Upstream, New Utilities

A new RcppSimdJson release arrived on CRAN late yesterday bringing along the one recently updated simdjson release 0.6.0.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

Other than the upstream update, Brendan added some new utilities to check for valid utf-8 or json format, and to minify json plus a small workaround for a clang-9 bug we encountered. We can confirm Daniel’s statement on ridiculously fast utf-8 validattion. It is so cool to work with amazing tools.

The NEWS entry follows.

Changes in version 0.1.3 (2020-11-01)

  • Added URLs to DESCRIPTION (Dirk closing #50).

  • Upgraded to simdjson 0.6.0 (Dirk in #52).

  • New policy option to always convert integers to int64_t (Brendan in #55 closing #54).

  • Added workaround for odd clang-9 bug (Brendan in #57).

  • New utility functions is_valid_utf8(), is_valid_json() and fminify() (Brendan in #58).

Courtesy of my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 23 Oct 2020

RcppSpdlog 0.0.3: New features and much more docs

A good month after the initial two releases, we are thrilled to announce relase 0.0.3 of RcppSpdlog. This brings us release 1.8.1 of spdlog as well as a few local changes (more below).

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovic.

This version of RcppSpdlog brings a new top-level function setLogLevel to control what events get logged, updates the main example to show this and to also make the R-aware logger the default logger, and adds both an extended vignette showing several key features and a new (external) package documentation site.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.3 (2020-10-23)

  • New function setLogLevel with R accessor in exampleRsink example

  • Updated exampleRsink to use default logger instance

  • Upgraded to upstream release 1.8.1 which contains finalised upstream use to switch to REprintf() if R compilation detected

  • Added new vignette with extensive usage examples, added compile-time logging switch example

  • A package documentation website was added

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 21 Oct 2020

RcppZiggurat 0.1.6

ziggurats

A new release, now at version 0.1.6, of RcppZiggurat is now on the CRAN network for R.

The RcppZiggurat package updates the code for the Ziggurat generator by Marsaglia and other which provides very fast draws from a Normal distribution. The package provides a simple C++ wrapper class for the generator improving on the very basic macros, and permits comparison among several existing Ziggurat implementations. This can be seen in the figure where Ziggurat from this package dominates accessing the implementations from the GSL, QuantLib and Gretl—all of which are still way faster than the default Normal generator in R (which is of course of higher code complexity).

This release brings a corrected seed setter and getter which now correctly take care of all four state variables, and not just one. It also corrects a few typos in the vignette. Both were fixed quite a while back, but we somehow managed to not ship this to CRAN for two years.

The NEWS file entry below lists all changes.

Changes in version 0.1.6 (2020-10-18)

  • Several typos were corrected in the vignette (Blagoje Ivanovic in #9).

  • New getters and setters for internal state were added to resume simulations (Dirk in #11 fixing #10).

  • Minor updates to cleanup script and Travis CI setup (Dirk).

Courtesy of my CRANberries, there is a diffstat report for this release. More information is on the RcppZiggurat page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 20 Oct 2020

RcppArmadillo 0.10.1.0.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 786 other packages on CRAN.

A little while ago, Conrad released version 10.1.0 of Armadillo, a a new major release. As before, given his initial heads-up we ran two full reverse-depends checks, and as a consequence contacted four packages authors (two by email, two via PR) about a miniscule required change (as Armadillo now defaults to C++11, an old existing setting of avoiding C++11 lead to an error). Our thanks to those who promptly update their packages—truly appreciated. As it turns out, Conrad also softened the error by the time the release ran around.

But despite our best efforts, the release was delayed considerably by CRAN. We had made several Windows test builds but luck had it that on the uploaded package CRAN got itself a (completely spurious segfault—which can happen on a busy machine building machine things at once). Sadly it took three or four days for CRAN to reply our email. After which it took another number of days for them to ponder the behaviour of a few new ‘deprecated’ messaged tickled by at the most ten or so (out of 786) packages. Oh well. So here we are, eleven days after I emailed the rcpp-devel list about the new package being on CRAN but possibly delayed (due to that seg.fault). But during all that time the package was of course available via the Rcpp drat.

The changes in this release are summarized below as usual and are mostly upstream along with an improved Travis CI setup due to the aforementioned use of the bspm package for binaries at Travis.

Changes in RcppArmadillo version 0.10.1.0.0 (2020-10-09)

  • Upgraded to Armadillo release 10.1.0 (Orchid Ambush)

    • C++11 is now the minimum required C++ standard

    • faster handling of compound expressions by trimatu() and trimatl()

    • faster sparse matrix addition, subtraction and element-wise multiplication

    • expanded sparse submatrix views to handle the non-contiguous form of X.cols(vector_of_column_indices)

    • expanded eigs_sym() and eigs_gen() with optional fine-grained parameters (subspace dimension, number of iterations, eigenvalues closest to specified value)

    • deprecated form of reshape() removed from Cube and SpMat classes

    • ignore and warn on use of the ARMA_DONT_USE_CXX11 macro

  • Switch Travis CI testing to focal and BSPM

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 08 Oct 2020

RcppSimdJson 0.1.2: Upstream update

A new RcppSimdJson release arrived on CRAN yesterday bringing along the simdjson 0.5.0 release that happened a few weeks.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

Beside the upstream update, not too much happened to our package itself since 0.1.1 though Brandon did help one user to seriously speed up his JSON processing. The (this time very short) NEWS entry follows.

Changes in version 0.1.2 (2020-10-07)

  • Upgraded to simdjson 0.5.0 (Dirk #49)

Courtesy of my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 20 Sep 2020

RcppSpdlog 0.0.2: New upstream, awesome new stopwatch

Following up on the initial RcppSpdlog 0.0.1 release earlier this week, we are pumped to announce release 0.0.2. It contains upstream version 1.8.0 for spdlog which utilizes (among other things) a new feature in the embedded fmt library, namely completely automated formatting of high resolution time stamps which allows for gems like this (taken from this file in the package and edited down for brevity):

    spdlog::stopwatch sw;                                   // instantiate a stop watch

    // some other code

    sp->info("Elapsed time: {}", sw);

What we see is all there is: One instantiates a stopwatch object, and simply references it. The rest, as they say, is magic. And we get tic / toc alike behaviour in modern C++ at essentially no cost us (as code authors). So nice. Output from the (included in the package) function exampleRsink() (again edited down just a little):

R> RcppSpdlog::exampleRsink()
[13:03:23.845114] [fromR] [I] [thread 793264] Welcome to spdlog!
[...]
[13:03:23.845205] [fromR] [I] [thread 793264] Elapsed time: 0.000103611
[...]
[13:03:23.845281] [fromR] [I] [thread 793264] Elapsed time: 0.00018203
R> 

We see that the two simple logging instances come 10 and 18 microseconds into the call.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovic.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.2 (2020-09-17)

  • Upgraded to upstream release 1.8.0

  • Switched Travis CI to using BSPM, also test on macOS

  • Added 'stopwatch' use to main R sink example

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppSpdlog page.

The only sour grapes, again, are once more over the CRAN processing. And just how 0.0.1 was delayed for no good reason for three weeks, 0.0.2 was delayed by three days just because … well that is how CRAN rules sometimes. I’d be even more mad if I had an alternative but I don’t. We remain grateful for all they do but they really could have let this one through even at one-day update delta. Ah well, now we’re three days wiser and of course nothing changed in the package.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 16 Sep 2020

RcppSpdlog 0.0.1: New and Exciting Logging Package

Very thrilled to announce a new package RcppSpdlog which is now on CRAN in its first release 0.0.1. We had tweeted once about the earliest version which had already caught the eyes of Gabi upstream.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovic.

I had meant to package this for a few years now but didn’t find an (easy, elegant) way to completely lift stdout / stderr and related uses which R wants us to remove for smoother operations from R itself including synchronized input/output. It was only a few weeks ago that I realized I should subclass a logger (or, more concretely, a sink for a logger) which could then use R input/output. With the right idea the implementaton was easy, and [Gabi]((https://github.com/gabime) was most helpful in making sure R CMD check would not see one or two remaining C++ i/o operations (which we currently do by not activating a default logger, and substituing REprintf() in one call). So this is now clean and sween and a simple use is included in an example in the package we can show here too (in slightly shorter form minus the documentation header):

// this portmanteau include also defines the r_sink we use below, and which
// diverts all logging to R via the Rcpp::Rcout replacement for std::cout
#include <RcppSpdlog>

// [[Rcpp::export]]
void exampleRsink() {

    std::string logname = "fromR";                          // fix a name for this logger
    auto sp = spdlog::get(logname);                         // retrieve existing one
    if (sp == nullptr) sp = spdlog::r_sink_mt(logname);     // or create new one if needed

    // change log pattern (changed from [%H:%M:%S %z] [%n] [%^---%L---%$] )
    sp->set_pattern("[%H:%M:%S.%f] [%n] [%^%L%$] [thread %t] %v");

    sp->info("Welcome to spdlog!");
    sp->error("Some error message with arg: {}", 1);

    sp->warn("Easy padding in numbers like {:08d}", 12);
    sp->critical("Support for int: {0:d};  hex: {0:x};  oct: {0:o}; bin: {0:b}", 42);
    sp->info("Support for floats {:03.2f}", 1.23456);
    sp->info("Positional args are {1} {0}..", "too", "supported");
    sp->info("{:<30}", "left aligned");

}

The NEWS entry for the first release follows.

Changes in RcppSpdlog version 0.0.1 (2020-09-08)

  • Initial release with added R/Rcpp logging sink example

The only sour grapes, if any, are over the CRAN processing. This was originally uploaded three weeks ago. As a new package, it got extra attention and some truly idiosyncratic attention to two details that were already supplied in the first uploaded version. Yet it needed two rounds of going back and forth for really no great net gain, yet wasting a week each time. I am not all that impressed by this, and not particularly pleased either, but I presume is the “tax” we all pay in order to enjoy the unsurpassed richness of the CRAN repository system which continues to work just flawlessly.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 03 Sep 2020

RcppArmadillo 0.9.900.3.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 769 other packages on CRAN.

A few days ago, Conrad released a new minor version 9.900.3 of Armadillo which we packaged and tested as usual. Given the incremental release character, we only tested the release and not candidate release. No regressions were found, and, as usual, logs from reverse-depends runs are in the rcpp-logs repo.

All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.3.0 (2020-09-02)

  • Upgraded to Armadillo release 9.900.3 (Nocturnal Misbehaviour)

    • More efficient code for initialising matrices with fill::zeros

    • Fixes for various error messages

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 30 Aug 2020

RcppCCTZ 0.2.9: API Header Added

A new minor release 0.2.9 of RcppCCTZ is now on CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do—using copies in their packages which remains less than ideal.

This version adds a header file for the recently-exported three functions.

Changes in version 0.2.9 (2020-08-30)

  • Provide a header RcppCCZT_API.h for client packages.
  • Show a simple example of parsing a YYYYMMDD HHMMSS.FFFFFF date.

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

RcppSMC 0.2.2: Small updates

A new release 0.2.2 of the RcppSMC package arrived on CRAN earlier today (and once again as a very quick pretest-publish within minutes of submission).

RcppSMC provides Rcpp-based bindings to R for the Sequential Monte Carlo Template Classes (SMCTC) by Adam Johansen described in his JSS article. Sequential Monte Carlo is also referred to as Particle Filter in some contexts.

This releases contains two fixes from a while back that had not been released, a CRAN-requested update plus a few more minor polishes to make it pass R CMD check --as-cran as nicely as usual.

Changes in RcppSMC version 0.2.2 (2020-08-30)

  • Package helper files .editorconfig added (Adam in #43).

  • Change const correctness and add return (Leah in #44).

  • Updates to continuous integration and R versions used (Dirk)

  • Accomodate CRAN request, other updates to CRAN Policy (Dirk in #49 fixing #48).

Courtesy of CRANberries, there is a diffstat report for this release.

More information is on the RcppSMC page. Issues and bugreports should go to the GitHub issue tracker.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 11 Aug 2020

RcppSimdJson 0.1.1: More Features

A first update following for the exciting RcppSimdJson 0.1.0 release last month is now on CRAN. Version 0.1.1 brings further enhancements such direct parsing of raw chars, working with compressed files as well as much expanded querying ability all thanks to Brendan, some improvements to our demos thanks to Daniel as well as a small fix via a one-liner borrowed from upstream for a reported UBSAN issue.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

The detailed list of changes follows.

Changes in version 0.1.1 (2020-08-10)

  • Corrected incorrect file deletion when mixing local and remote files (Brendan in #34) closing #33.

  • Added support for raw vectors, compressed files, and compressed downloads (Dirk and Brendan in #36, #39, and #45 closing #35 and addressing issues raised in #40 and #44).

  • Examples in two demos are now more self-sufficient (Daniel Lemire and Dirk in #42).

  • Expanded query functionality to include single, flat, and nested queries (Brendan in #45 closing #43).

  • Split error handling parameters from error_ok/on_error into parse_error_ok/on_parse_error and query_error_ok/on_query_error (Brendan in #45).

  • One-line upstream change to address sanitizer error on cast.

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 04 Aug 2020

RcppCCTZ 0.2.8: Minor API Extension

A new minor release 0.2.8 of RcppCCTZ is now on CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do—using copies in their packages which remains less than ideal.

This version adds three no throw variants of three existing functions, contributed again by Leonardo. This will be used in an upcoming nanotime release which we are finalising now.

Changes in version 0.2.8 (2020-08-04)

  • Added three new nothrow variants (for win32) needed by the expanded nanotime package (Leonardo in #37)

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 17 Jul 2020

RcppArmadillo 0.9.900.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 757 other packages on CRAN.

Conrad just released a new minor upstream version 9.900.2 of Armadillo which we packaged and tested as usual first as a ‘release candidate’ build and then as the release. As usual, logs from reverse-depends runs are in the rcpp-logs repo.

All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.2.0 (2020-07-17)

  • Upgraded to Armadillo release 9.900.2 (Nocturnal Misbehaviour)

    • In sort(), fixes for inconsistencies between checks applied to matrix and vector expressions

    • In sort(), remove unnecessary copying when applied in-place to vectors function when applied in-place to vectors

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 07 Jul 2020

RcppSimdJson 0.1.0: Now on Windows, With Parsers and Faster Still!

A smashing new RcppSimdJson release 0.1.0 containing several small updates to upstream simdjson (now at 0.4.6) in part triggered by very excisting work by Brendan who added actual parser from file and string—and together with Daniel upstream worked really hard to make Windows builds as well as complete upstream tests on our beloved (ahem) MinGW platform possible. So this version will, once the builders have caught up, give everybody on Windows a binary—with a JSON parser running circles around the (arguably more feature-rich and possibly easier-to-use) alternatives. Dave just tweeted a benchmark snippet by Brendan, the full set is at the bottom our issue ticket for this release.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators, which in its upstream release 0.4.0 improved once more (also see the following point releases). Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

As mentioned, this release expands the reach of the package to Windows, and adds new user-facing functions. A big thanks for most of this is owed to Brendan, so buy him a drink if you run across him. The full NEWS entry follows.

Changes in version 0.1.0 (2020-07-07)

  • Upgraded to simdjson 0.4.1 which adds upstream Windows support (Dirk in #27 closing #26 and #14, plus extensive work by Brendan helping upstream with mingw tests).

  • Upgraded to simdjson 0.4.6 with further upstream improvements (Dirk in #30).

  • Change Travis CI to build matrix over g++ 7, 8, 9, and 10 (Dirk in #31; and also Brendan in #32).

  • New JSON functions fparse and fload (Brendan in #32) closing #18 and #10).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 06 Jul 2020

Rcpp 1.0.5: Several Updates

rcpp logo

Right on the heels of the news of 2000 CRAN packages using Rcpp (and also hitting 12.5 of CRAN package, or one in eight), we are happy to announce release 1.0.5 of Rcpp. Since the ten-year anniversary and the 1.0.0 release release in November 2018, we have been sticking to a four-month release cycle. The last release has, however, left us with a particularly bad taste due to some rather peculiar interactions with a very small (but ever so vocal) portion of the user base. So going forward, we will change two things. First off, we reiterate that we have already made rolling releases. Each minor snapshot of the main git branch gets a point releases. Between release 1.0.4 and this 1.0.5 release, there were in fact twelve of those. Each and every one of these was made available via the drat repo, and we will continue to do so going forward. Releases to CRAN, however, are real work. If they then end up with as much nonsense as the last release 1.0.4, we think it is appropriate to slow things down some more so we intend to now switch to a six-months cycle. As mentioned, interim releases are always just one install.packages() call with a properly set repos argument away.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 2002 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 203 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steady at around one millions downloads per month.

This release features again a number of different pull requests by different contributors covering the full range of API improvements, attributes enhancements, changes to Sugar and helper functions, extended documentation as well as continuous integration deplayment. See the list below for details.

Changes in Rcpp patch release version 1.0.5 (2020-07-01)

  • Changes in Rcpp API:

    • The exception handler code in #1043 was updated to ensure proper include behavior (Kevin in #1047 fixing #1046).

    • A missing Rcpp_list6 definition was added to support R 3.3.* builds (Davis Vaughan in #1049 fixing #1048).

    • Missing Rcpp_list{2,3,4,5} definition were added to the Rcpp namespace (Dirk in #1054 fixing #1053).

    • A further updated corrected the header include and provided a missing else branch (Mattias Ellert in #1055).

    • Two more assignments are protected with Rcpp::Shield (Dirk in #1059).

    • One call to abs is now properly namespaced with std:: (Uwe Korn in #1069).

    • String object memory preservation was corrected/simplified (Kevin in #1082).

  • Changes in Rcpp Attributes:

    • Empty strings are not passed to R CMD SHLIB which was seen with R 4.0.0 on Windows (Kevin in #1062 fixing #1061).

    • The short_file_name() helper function is safer with respect to temporaries (Kevin in #1067 fixing #1066, and #1071 fixing #1070).

  • Changes in Rcpp Sugar:

    • Two sample() objects are now standard vectors and not R_alloc created (Dirk in #1075 fixing #1074).
  • Changes in Rcpp support functions:

    • Rcpp.package.skeleton() adjusts for a (documented) change in R 4.0.0 (Dirk in #1088 fixing #1087).
  • Changes in Rcpp Documentation:

    • The pdf file of the earlier introduction is again typeset with bibliographic information (Dirk).

    • A new vignette describing how to package C++ libraries has been added (Dirk in #1078 fixing #1077).

  • Changes in Rcpp Deployment:

    • Travis CI unit tests now run a matrix over the versions of R also tested at CRAN (rel/dev/oldrel/oldoldrel), and coverage runs in parallel for a net speed-up (Dirk in #1056 and #1057).

    • The exceptions test is now partially skipped on Solaris as it already is on Windows (Dirk in #1065).

    • The default CI runner was upgraded to R 4.0.0 (Dirk).

    • The CI matrix spans R 3.5, 3.6, r-release and r-devel (Dirk).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2455 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 04 Jul 2020

Rcpp now used by 2000 CRAN packages–and one in eight!

2000 Rcpp packages

As of yesterday, Rcpp stands at exactly 2000 reverse-dependencies on CRAN. The graph on the left depicts the growth of Rcpp usage (as measured by Depends, Imports and LinkingTo, but excluding Suggests) over time.

Rcpp was first released in November 2008. It probably cleared 50 packages around three years later in December 2011, 100 packages in January 2013, 200 packages in April 2014, and 300 packages in November 2014. It passed 400 packages in June 2015 (when I tweeted about it), 500 packages in late October 2015, 600 packages in March 2016, 700 packages last July 2016, 800 packages last October 2016, 900 packages early January 2017, 1000 packages in April 2017, 1250 packages in November 2017, 1500 packages in November 2018 and then 1750 packages last August. The chart extends to the very beginning via manually compiled data from CRANberries and checked with crandb. The next part uses manually saved entries. The core (and by far largest) part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of packages using Rcpp is available too.

Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent July of 2015, seven percent just before Christmas 2015, eight percent in the summer of 2016, nine percent mid-December 2016, cracked ten percent in the summer of 2017 and eleven percent in 2018. We now passed 12.5 percent—so one in every eight CRAN packages dependens on Rcpp. Stunning. There is more detail in the chart: how CRAN seems to be pushing back more and removing more aggressively (which my CRANberries tracks but not in as much detail as it could), how the growth of Rcpp seems to be slowing somewhat outright and even more so as a proportion of CRAN – as one would expect a growth curve to.

To mark the occassion, I sent out two tweets yesterday: first a shorter one with “just the numbers”, followed by a second one also containing the few calculation steps. The screenshot from the second one is below.

2000 Rcpp packages

2000 user packages is pretty mind-boggling. We can use the progression of CRAN itself compiled by Henrik in a series of posts and emails to the main development mailing list. Not that long ago CRAN itself did have only 1000 packages, then 5000, 10000, and here we are at just over 16000 with Rcpp at 12.5% and still growing (though maybe more slowly). Amazeballs.

The Rcpp team continues to aim for keeping Rcpp as performant and reliable as it has been. A really big shoutout and Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 25 Jun 2020

RcppSimdJson 0.0.6: New Upstream, New Features!

A very exciting RcppSimdJson release with the updated upstream simdjson release 0.4.0 as well as a first set of new JSON parsing functions just hit CRAN. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The very recent 0.4.0 release further improves the already impressive speed.

And this release brings a first set of actually user-facing functions thanks to Brendan which put in a series of PRs! The full NEWS entry follows.

Changes in version 0.0.6 (2020-06-25)

  • Created C++ integer-handling utilities for safe downcasting and integer return (Brendan in #16 closing #13).

  • New JSON functions .deserialize_json and .load_json (Brendan in #16, #17, #20, #21).

  • Upgrade Travis CI to 'bionic', extract package and version from DESCRIPTION (Dirk in #23).

  • Upgraded to simdjson 0.4.0 (Dirk in #25 closing #24).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 21 Jun 2020

RcppGSL 0.3.8: More fixes and polish

Release 0.3.8 of RcppGSL is now getting onto CRAN. The RcppGSL package provides an interface from R to the GNU GSL using the Rcpp package.

Peter Carbonetto let us know in issue #25 that the included example now showed linker errors on (everybody’s favourite CRAN platform) Slowlaris. Kidding aside, the added compiler variety really has benefits because we were indeed missing a good handful or two of inline statements in the headers—which our good friends g++ and clang++ apparently let us get away with. This has been fixed, and a little bit of the usual package polish and cleanup has been added; see the list of detailed changes below.

Changes in version 0.3.8 (2020-06-21)

  • A few missing inline statements were added to the headers fixing a (genuine) error that was seen only on Solaris (Dirk).

  • The nice colNorm example is now in a file by itself, the previous versions are off in a new file colNorm_old.cpp (Dirk).

  • The README.me now sports two new badges (Dirk).

  • Travis CI was updated to 'bionic' and R 4.0 (Dirk).

Special thanks also to CRAN for a super-smooth and fully automated processing of a package with both compiled code and two handful of reverse dependencies.

Courtesy of CRANberries, a summary of changes to the most recent release is also available.

More information is on the RcppGSL page. Questions, comments etc should go to the issue tickets at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 15 Jun 2020

Rcpp 1.0.5 in two+ weeks: Please help test

rcpp logo

With the current four-month release cycle, the next Rcpp release is due in July following the 1.0.4 release in March. Just prior to the 1.0.4 release I had asked this:

It would be particularly beneficial if those with “unsual” build dependencies tested it as we would increase overall coverage beyond what I get from testing against 1800+ CRAN packages. BioConductor would also be welcome.

but only on the rcpp-devel list, and only about a good week prior to the release.

I remain rather disappointed and disillusioned about what happened after 1.0.4 was released. Two PRs in that release were soon seen to have side effects on more ‘marginal’ test systems, precisely what added testing could have revealed. An additional issue arose from changes in R’s make system, which is harder to anticipate or test. Each and every infelicity was fixed within a day or so, and we always make candidate releases available—the current Rcpp as of this writing is 1.0.4.12 meaning twelve microreleases were made since 1.0.4. And those microreleases are always available for normal download and install.packages use via the Rcpp drat repository accessible to all. So it was truly troubling to see some, especially those with experience in setting up or running testing / ci platforms, pretend to be unable to access, install, and provide these for their own tests, or the tests of their users. It just doesn’t pass a basic logic test: it takes a single call to install.packages(), or, even more easily, a single assignment of an auxiliary repo. All told this was a rather sad experience.

So let’s try to not repeat this. If you, or maybe users of a build or ci system you maintain, rely on Rcpp, and especially if you do so on systems outside the standard CRAN grid of three OSs and the triplet of “previous, current, next” releases of R, then please help by testing. I maitain these release as a volunteer, unpaid at that, and I simply cannot expand to more systesm. We take reverse dependency check seriously (and I just run two taking about a day each) but if you insist on building on stranger hardware or much older releases it will be up to you to ensure Rcpp passes. We prep for CRAN, and try our best to pass at CRAN. For nearly a dozen years.

To install the current microrelease from the Rcpp drat repository, just do

install.packages("Rcpp", repos="https://rcppcore.github.io/drat")

That is all there is to it. You could even add the Rcpp drat repository to your repository list.

Rcpp has become successful because so many people help with suggestions, documentation, and code. It is used by (as of today) 1958 CRAN packages, 205 BioConductor packages, and downloaded around a million times per month. So if you can, please help now with some more testing.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 09 Jun 2020

RcppArmadillo 0.9.900.1.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 727 other packages on CRAN.

Conrad recently released a new upstream version 9.900.1 of Armadillo which we packaged and tested as usual first as a ‘release candidate’ build and then as the release. As usual, logs from reverse-depends runs are in the rcpp-logs repo.

Apart from the new upstream release, we updated Travis use, ornamented the README a little, and smoothed over a rough corner from the recent R 4.0.0 release. All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.1.0 (2020-06-08)

  • Upgraded to Armadillo release 9.900.1 (Nocturnal Misbehaviour)

    • faster solve() for under/over-determined systems

    • faster eig_gen() and eig_pair() for large matrices

    • expanded eig_gen() and eig_pair() to optionally provide left and right eigenvectors

  • Switch Travis CI testing to R 4.0.0, use bionic as base distro and test R 3.6.3 and 4.0.0 in a matrix (Dirk in #298).

  • Add two badges to README for indirect use and the CSDA paper.

  • Adapt RcppArmadillo.package.skeleton() to a change in R 4.0.0 affecting what it exports in NAMESPACE.

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Rcpp Webinar Recording Available

As announced in a few tweets leading up to it, I took the date of what would have been the annual R/Finance conference as an opportunity to hold the one-hour tutorial / workshop with introductory Rcpp material which I often present on the first morning preceding the conference as a self-organized webinar. The live-streaming worked actually reasonably well via obs to youtube (even though the comprehensive software by the latter complained at times about insufficient bitstream rates–the joys of living with a (near) monopolistic broadband provider whom I should leave for fiber…). Apparently around seventy people connected to the stream—which is more than we usually have in the seminar room at UIC for the R/Finance morning.

The recording is now available here, and has already been seen over 200 times:

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 23 May 2020

RcppSimdJson 0.0.5: Updated Upstream

A new RcppSimdJson release with updated upstream simdjson code just arrived on CRAN. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

This release brings updated upstream code (thanks to Brendan Knapp) plus a new example and minimal tweaks. The full NEWS entry follows.

Changes in version 0.0.5 (2020-05-23)

  • Add parseExample from earlier upstream announcement (Dirk).

  • Synced with upstream (Brendan in #12) closing #11).

  • Updated example parseExample to API changes (Brendan).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 17 May 2020

RcppArmadillo 0.9.880.1.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 719 other packages on CRAN.

Conrad released a new upstream version 9.880.1 of Armadillo on Friday which I packaged and tested as usual (result log here in the usual repo). The R package also sports a new OpenMP detection facility once again motivated by macOS which changed its setup yet again.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.880.1.0 (2020-05-15)

  • Upgraded to Armadillo release 9.880.1 (Roasted Mocha Detox)

    • expanded qr() to optionally use pivoted decomposition

    • updated physical constants to NIST 2018 CODATA values

    • added ARMA_DONT_USE_CXX11_MUTEX confguration option to disable use of std::mutex

  • OpenMP capability is tested explicitly (Kevin Ushey and Dirk in #294, #295, and #296 all fixing #290).

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 27 Apr 2020

RcppArmadillo 0.9.870.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 705 other packages on CRAN.

A new upstream release 9.870.2 of Armadillo was released a few days ago. We had tested two release candidates, and this caught one bug. The release was held up at CRAN for a few days as one package had an overly sensitive test depending on random input data; eventually we all convinced ourselves that there was no (Rcpp)Armadillo issue here. So morale: friends don’t let friends have tests depend on random behavior.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.870.2.0 (2020-04-24)

  • Upgraded to Armadillo release 9.870.2 (Roasted Mocha Retox)

    • faster handling of matrix multiplication expressions by diagvec() and diagmat()

    • added trimatu_ind() and trimatl_ind()

    • more consistent detection of sparse vector expressions

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 14 Apr 2020

RcppArmadillo 0.9.860.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 706 other packages on CRAN.

A new upstream release 9.860.2 of Armadillo was just released. The theme of “convergence” continues; the previous release increased the minor from 800 to 850, now we are at 860. We first wrapped this up as version 0.9.859.1.0, but it turned out to have been held back by a buglet between R 4.0.0 and Rcpp which the recent patch release fixed (along with other woes on old R or non-CRAN-alike macOS). It then turns out that the new (upstream) version 9.860.1 had a minor bug which I missed as I reverse-depends checked the prior version. Doh. My thanks, as always, to CRAN for spotting this. The fix was added upstream and we have 9.860.2 as RcppArmadillo 0.9.860.2.0.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.860.2.0 (2020-04-13)

  • Upgraded to Armadillo release 9.860.2 (Roasted Mocha Fix)

    • Added powmat()

    • Faster access to columns in sparse submatrix views

    • Faster handling of relational expressions by accu()

    • Faster handling of sympd matrices by expmat(), logmat(), sqrtmat()

    • Workaround for save/load issues with HDF5 v1.12

  • Vignettes are now pre-made and include (#285)

  • Two test files are now skipped on 32-bit Windows

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 09 Apr 2020

Rcpp 1.0.4.6: Bug fix interim version

rcpp logo

Rcpp 1.0.4 was released on March 17, following the usual sequence of fairly involved reverse-depends check along with a call for community testing issued weeks before the release. In that email I specifically pleaded with folks to pretty-please test non-standard setups:

It would be particularly beneficial if those with “unsual” build dependencies tested it as we would increase overall coverage beyond what I get from testing against 1800+ CRAN packages. BioConductor would also be welcome.

Alas, you can’t always get what you want. Shortly after the release we were made aware that the two (large) pull request at the book ends of the 1.0.3 to 1.0.4 release period created trouble. Of these two, the earliest PR in the 1.0.4 release upset older-than-CRAN-tested installation, i.e. R 3.3.0 or before. (Why you’d want to run R 3.3.* when R 3.6.3 is current is something I will never understand, but so be it.) This got addressed in two new PRs. And the matching last PR had a bit of sloppyness leaving just about everyone alone, but not all those macbook-wearing data scientists when using newer macOS SDKs not used by CRAN. In other words, “unsual” setups. But boy, do those folks have an ability to complain. Again, two quick PRs later that was addressed. Along came a minor PR with two more Rcpp::Shield<> uses (as life is too short to manually count PROTECT and UNPROTECT). And then a real issue between R 4.0.0 and Rcpp first noticed with RcppParallel builds on Windows but then also affecting RcppArmadillo. Another quickly issued fix. So by now the count is up to six, and we arrived at Rcpp 1.0.4.6.

Which is now on CRAN, after having sat there for nearly a full week, and of course with no reason given. Because the powers that be move in mysterious ways. And don’t answer to earthlings like us.

As may transpire here, I am little tired from all this. I think we can do better, and I think we damn well should, or I may as well throw in the towel and just release to the drat repo where each of the six interim versions was available for all to take as soon as it materialized.

Anyway, here is the state of things. Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1897 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 191 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steasy at one millions downloads per month.

The changes for this interim version are summarized below.

Changes in Rcpp patch release version 1.0.4.6 (2020-04-02)

  • Changes in Rcpp API:

    • The exception handler code in #1043 was updated to ensure proper include behavior (Kevin in #1047 fixing #1046).

    • A missing Rcpp_list6 definition was added to support R 3.3.* builds (Davis Vaughan in #1049 fixing #1048).

    • Missing Rcpp_list{2,3,4,5} definition were added to the Rcpp namespace (Dirk in #1054 fixing #1053).

    • A further updated corrected the header include and provided a missing else branch (Mattias Ellert in #1055).

    • Two more assignments are protect with Rcpp::Shield (Dirk in #1059)

  • Changes in Rcpp Attributes:

    • Empty strings are not passed to R CMD SHLIB which was seen with R 4.0.0 on Windows (Kevin in #1062 fixing #1061).
  • Changes in Rcpp Deployment:

    • Travis CI unit tests now run a matrix over the versions of R also tested at CRAN (rel/dev/oldrel/oldoldrel), and coverage runs in parallel for a net speed-up (Dirk in #1056 and #1057).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2356 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 03 Apr 2020

RcppSimdJson 0.0.4: Even Faster Upstream!

A new (upstream) simdjson release was announced by Daniel Lemire earlier this week, and my Twitter mentions have been running red-hot ever since as he was kind enough to tag me. Do look at that blog post, there is some impressive work in there. We wrapped up the (still very simple) rcppsimdjson around it last night and shipped it this morning.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. For illustration, I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release brings upstream 0.3 (and 0.3.1) plus a minor tweak (also shipped back upstream). Our full NEWS entry follows.

Changes in version 0.0.4 (2020-04-03)

  • Upgraded to new upstream releases 0.3 and 0.3.1 (Dirk in #9 closing #8)

  • Updated example validateJSON to API changes.

But because Daniel is such a fantastic upstream developer to collaborate with, he even filed a full feature-request ‘maybe you can consider upgrading’ as issue #8 at our repo containing the fully detailed list of changes. As it is so impressive I will simple quote the upper half of just the major changes:

Highlights

  • Multi-Document Parsing: Read a bundle of JSON documents (ndjson) 2-4x faster than doing it individually. API docs / Design Details
  • Simplified API: The API has been completely revamped for ease of use, including a new JSON navigation API and fluent support for error code and exception styles of error handling with a single API. Docs
  • Exact Float Parsing: Now simdjson parses floats flawlessly without any performance loss (https://github.com/simdjson/simdjson/pull/558). Blog Post
  • Even Faster: The fastest parser got faster! With a shiny new UTF-8 validator and meticulously refactored SIMD core, simdjson 0.3 is 15% faster than before, running at 2.5 GB/s (where 0.2 ran at 2.2 GB/s).

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 24 Mar 2020

New package RcppDate 0.0.1 now on CRAN!

A new small package with a new C++ header library is now on CRAN. It brings the date library by Howard Hinnant to R. This library has been in pretty wide-spread use for a while now, and adds to C++11/C++14/C++17 what will be (with minor modifications) the ‘date’ library in C++20. I had been aware of it for a while, but not needed thanks to CCTZ library out of Google and our RcppCCTZ package. And like CCTZ, it builds upon std::chron adding a whole lot of functionality and useability enhancement. But a some upcoming (and quite exciting!) changes in nanotime required it, I had a reason to set about packaging it as RcppDate. And after a few days of gestation and review it is now available via CRAN.

Two simple example files are included and can be accessed by Rcpp::sourceCpp(). Some brief excerpts follow.

The first example shows three date constructors. Note how the month (and the leading digits) are literals. No quotes for strings anywhere. And no format (just like our anytime package for R).

  constexpr auto x1 = 2015_y/March/22;
  constexpr auto x2 = March/22/2015;
  constexpr auto x3 = 22_d/March/2015;

Note that these are constexpr that resolve at compile-time, and that the resulting year_month_day type is inferred via auto.

A second example constructs the last day of the months similarly:

  constexpr auto x1 = 2015_y/February/last;
  constexpr auto x2 = February/last/2015;
  constexpr auto x3 = last/February/2015;

For more, see the copious date.h documentation.

The (very bland first) NEWS entry (from a since-added NEWS file) for the initial upload follows.

Changes in version 0.0.1 (2020-01-17)

  • Initial CRAN upload of first version

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 18 Mar 2020

RcppCCTZ 0.2.7

A new release 0.2.7 of RcppCCTZ is now at CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do—using copies in their packages which remains less than ideal.

This version adds internal extensions, contributed by Leonardo, which support upcoming changes to the nanotime package we are working on.

Changes in version 0.2.7 (2020-03-18)

  • Added functions _RcppCCTZ_convertToCivilSecond that converts a time point to the number of seconds since epoch, and _RcppCCTZ_convertToTimePoint that converts a number of seconds since epoch into a time point; these functions are only callable from C level (Leonardo in #34 and #35).

  • Added function _RcppCCTZ_getOffset that returns the offset at a speficied time-point for a specified timezone; this function is only callable from C level (Leonardo in #32).

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 17 Mar 2020

Rcpp 1.0.4: Lots of goodies

rcpp logo

The fourth maintenance release 1.0.4 of Rcpp, following up on the 10th anniversary and the 1.0.0. release sixteen months ago, arrived on CRAN this morning. This follows a few days of gestation at CRAN. To help during the wait we provided this release via drat last Friday. And it followed a pre-release via drat a week earlier. But now that the release is official, Windows and macOS binaries will be built by CRAN over the next few days. The corresponding Debian package will be uploaded as a source package shortly after which binaries can be built.

As with the previous releases Rcpp 1.0.1, Rcpp 1.0.2 and Rcpp 1.0.3, we have the predictable and expected four month gap between releases which seems appropriate given both the changes still being made (see below) and the relative stability of Rcpp. It still takes work to release this as we run multiple extensive sets of reverse dependency checks so maybe one day we will switch to six month cycle. For now, four months still seem like a good pace.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1873 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 191 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steasy at one millions downloads per month.

This release features quite a number of different pull requests by seven different contributors as detailed below. One (personal) highlight is the switch to tinytest.

Changes in Rcpp version 1.0.4 (2020-03-13)

  • Changes in Rcpp API:

    • Safer Rcpp_list*, Rcpp_lang* and Function.operator() (Romain in #1014, #1015).

    • A number of #nocov markers were added (Dirk in #1036, #1042 and #1044).

    • Finalizer calls clear external pointer first (Kirill Müller and Dirk in #1038).

    • Scalar operations with a rhs matrix no longer change the matrix value (Qiang in #1040 fixing (again) #365).

    • Rcpp::exception and Rcpp::stop are now more thread-safe (Joshua Pritikin in #1043).

  • Changes in Rcpp Attributes:

    • The cppFunction helper now deals correctly with mulitple depends arguments (TJ McKinley in #1016 fixing #1017).

    • Invisible return objects are now supported via new option (Kun Ren in #1025 fixing #1024).

    • Unavailable packages referred to in LinkingTo are now reported (Dirk in #1027 fixing #1026).

    • The sourceCpp function can now create a debug DLL on Windows (Dirk in #1037 fixing #1035).

  • Changes in Rcpp Documentation:

    • The .github/ directory now has more explicit guidance on contributing, issues, and pull requests (Dirk).

    • The Rcpp Attributes vignette describe the new invisible return object option (Kun Ren in #1025).

    • Vignettes are now included as pre-made pdf files (Dirk in #1029)

    • The Rcpp FAQ has a new entry on the recommended importFrom directive (Dirk in #1031 fixing #1030).

    • The bib file for the vignette was once again updated to current package versions (Dirk).

  • Changes in Rcpp Deployment:

    • Added unit test to check if C++ version remains remains aligned with the package number (Dirk in #1022 fixing #1021).

    • The unit test system was switched to tinytest (Dirk in #1028, #1032, #1033).

Please note that the change to execptions and Rcpp::stop() in pr #1043 has been seen to have a minor side effect on macOS issue #1046 which has already been fixed by Kevin in pr #1047 for which I may prepare a 1.0.4.1 release for the Rcpp drat repo in a day or two.

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2356 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 14 Mar 2020

RcppAPT 0.0.6

A new version of RcppAPT – our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, … commands and their cache powering Debian, Ubuntu and the like – is now on CRAN.

RcppAPT allows you to query the (Debian or Ubuntu) package dependency graph at will, with build-dependencies (if you have deb-src entries), reverse dependencies, and all other goodies. See the vignette and examples for illustrations.

This new version corrects builds failures under the new and shiny Apt 2.0 release (and the pre-releases like the 1.9.* series in Ubuntu) as some header files moved around. My thanks to Kurt Hornik for the heads-up. I accomodated the change in the (very simple and shell-based) configure script by a) asking pkg-config about the version of pkg-apt and then using that to b) compare to a ‘threshold value’ of ‘1.9.0’ and c) setting another compiler #define if needed so that d) these headers could get included if defined. The neat part is that a) and b) are done in an R one-liner, and the whole script is still in shell. Now, CRAN being CRAN, I now split the script into two: one almost empty one not using bash that passes the ‘omg but bash is not portable’ test, and which calls a second bash script doing the work. Fun and games…

The full set of changes follows.

Changes in version 0.0.6 (2020-03-14)

  • Accomodate Apt 2.0 code changes by including more header files

  • Change is backwards compatible and conditional

  • Added configure call using pkg-config and package version comparison (using R) to determine if the define is needed

  • Softened unit tests as we cannot assume optional source deb information to be present, so demo code runs but zero results tolerated

Courtesy of CRANberries, there is also a diffstat report for this release.

A bit more information about the package is available here as well as as the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 08 Mar 2020

RcppAnnoy 0.0.16

annoy image

A new release 0.0.16 of RcppAnnoy is available via the Github-hosted R Repository (ghrr). Edit a good two hours later: And wonder of wonders, now also on CRAN.

It remains in limbo at CRAN for no apparent reason. No change appears to be imminent either as the CRAN maintainers continue to play a passive-aggressive game of no communication for any reason. Which is a genuine shame as e [E]verbody involved in the package, i.e. Erik (upstream) and myself but also Aaron (downstream) worked pretty hard and well last weekend (while I was traveling / attending the wonderful celebRtion 2020 conference for the 20th anniversary of the R 1.0.0) to iron out all remaining issues. Installation is pretty flawless and silent as all compiler warnings have been takeb care of even under -pedantic on a recent version, and the last remaining UBSAN issue is also fixed.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

As I wrote in the announcement for 0.0.15 which fixed many-but-not-all issues:

The other important issue is that there will be another 0.0.16 release real soon to incorporate three more small upstream PRs driven by these discussions which kept going on post-release while I was conferencing (or traveling), and which should fix things for good, or so we hope. This should go out probably by the end of the week to not exceed a weekly upload cadence; if you want to see more, or get earlier access, see the git repo which is in fine shape. If you want to see a pre-release on the ghrr drat drop me a line.

As CRAN is holding the package hostage, all I can do now is to release to the Github-hosted R Repository (ghrr) from where you can install it via a simple install.packages("RcppAnnoy", repos="https://ghrr.github.io/drat") (or any of the other drat supported commands, see the ghrr page for more). Or wait and wait and wait … until CRAN graces us with a manual admission (given that the previous upload left one small UBSAN issue to fix). One day. Hopefully. The package is now on CRAN.

Detailed changes follow below.

Changes in version 0.0.16 (2020-03-06)

  • Use int in two interfaces (Dirk in #59 for upstream PR 460 and closing #56).

  • Use inline for two helper functions (Dirk in #59 for upstream PR 461 and closing #57; also Aaron in #58 after earlier discussion).

  • Removed a noisy pragma (Dirk in #60 for upstream PR 462).

  • Add a simple helper function displaying compiler status.

Courtesy of CRANberries, there will also be diffstat report if and when the package ever makes it to CRAN. is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 04 Mar 2020

RcppSimdJson 0.0.3: Second Update!

Following up on both the initial RcppSimdJson release and the first update, the second update release 0.0.3 arrived on CRAN yesterday.

RcppSimdJson wraps the fantastic simdjson library by Daniel Lemire which is truly impressive. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. For illustration, I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release once again syncs the simdjson headers with upstream, and strengthens the build setup a little bit more. We only turn C++17 (which is needed) on when R knows it (from its builds), report the architecture status at package load (in a suppressable message), and only attempt to parse in examples and unit tests when know that we are on a sufficient platform. The full NEWS entry follows.

Changes in version 0.0.3 (2020-03-03)

  • Sychronized once more with upstream.

  • Created new C++ function to check for unsupported architecture, and report the status on package load.

  • Only run example and unit tests if supported architecture is found.

  • Created small configure script to see if R was built with C++17 support, and record it in src/Makevars.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 02 Mar 2020

RcppAnnoy 0.0.15

annoy image

A few days ago, a new release 0.0.15 of RcppAnnoy got onto CRAN while I was traveling / attending the wonderful celebRtion 2020 for the 20th anniversary of the R 1.0.0 release.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This releases makes great strides towards avoiding long-standing SAN/UBSAN issues. Upstream author Erik has been most helpful, as has been the feedback and input from two downstream users of RcppAnnoy, namely Aaron and James. This 0.0.15 release addresses one key, and longstanding, SAN/UBSAN issue. It is actually rather tricky as the code, for efficiency reason, bounces at the edge of what can be done. But a small rearrangement suppresses one such message which is good. We also got a hint from CRAN (thanks for that as always) to re-read one section of Writing R Extensions to make alloca more portable so that Solaris does not have to cry, and Bill Venables kindly helped with a small correction to the docs.

The other important issue is that there will be another 0.0.16 release real soon to incorporate three more small upstream PRs driven by these discussions which kept going on post-release while I was conferencing (or traveling), and which should fix things for good, or so we hope. This should go out probably by the end of the week to not exceed a weekly upload cadence; if you want to see more, or get earlier access, see the git repo which is in fine shape. If you want to see a pre-release on the ghrr drat drop me a line.

Detailed changes follow below.

Changes in version 0.0.15 (2020-02-25)

  • RcppAnnoy synchronized with upstream PR 455 (Dirk in #55).

  • The help page has a small correction thanks to Bill1 Venables.

  • The alloca() function is now declared portably thanks to a working example in Writing R Extensions.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 22 Feb 2020

RcppSimdJson 0.0.2: First Update!

Following up on the initial RcppSimdJson release, a first updated arrived on CRAN yesterday.

RcppSimdJson wraps the fantastic simdjson library by Daniel Lemire which truly impressive. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release syncs the simdjson headers with upstream, and polishes the build a little by conditioning on actually having a C++17 compiler rather than just suggesting it. The NEWS entry follows.

Changes in version 0.0.2 (2020-02-21)

  • Sychronized with upstream (Dirk in #4 and #5).

  • The R side of validateJSON now globs the file argument, expanding symbols like ~ appropriately.

  • C++ code in validateJSON now conditional on C++17 allowing (incomplete) compilation on lesser systems.

  • New helper function returning value of __cplusplus macro, used in package startup to warn if insufficient compiler used.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 13 Feb 2020

RcppSimdJson 0.0.1 now on CRAN!

A fun weekend-morning project, namely wrapping the outstanding simdjson library by Daniel Lemire (with contributions by Geoff Langdale, John Keiser and many others) into something callable from R via a new package RcppSimdJson lead to a first tweet on January 20, a reference to the brand new github repo, and CRAN upload a few days later—and then two weeks of nothingness.

Well, a little more than nothing as Daniel is an excellent “upstream” to work with who promptly incorporated two changes that arose from preparing the CRAN upload. So we did that. But CRAN being as busy and swamped as they are we needed to wait. The ten days one is warned about. And then some more. So yesterday I did a cheeky bit of “bartering” as Kurt wanted a favour with an updated digest version so I hinted that some reciprocity would be appreciated. And lo and behold he admitted RcppSimdJson to CRAN. So there it is now!

We have some upstream changes already in git, but I will wait a few days to let a week pass before uploading the now synced upstream code. Anybody who wants it sooner knows where to get it on GitHub.

simdjson is a gem. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

The NEWS entry (from a since-added NEWS file) for the initial RcppSimdJson upload follows.

Changes in version 0.0.1 (2020-01-24)

  • Initial CRAN upload of first version

  • Comment-out use of stdout (now updated upstream)

  • Deactivate use computed GOTOs for compiler compliance and CRAN Policy via #define

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 09 Feb 2020

RcppArmadillo 0.9.850.1.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 685 other packages on CRAN.

A new upstream release 9.850.1 of Armadillo was just released. And as some will undoubtedly notice, Conrad opted for an increment of 50 rather 100. We wrapped this up as version 0.9.850.1.0, having prepared a full (github-only) tarball and the release candidate 9.850.rc1 a few days ago. Both the release candidate and the release got the full reverse depends treatment, and no issues were found.

Changes in the new release below.

Changes in RcppArmadillo version 0.9.850.1.0 (2020-02-09)

  • Upgraded to Armadillo release 9.850.1 (Pyrocumulus Wrath)

    • faster handling of compound expressions by diagmat()

    • expanded .save() and .load() to handle CSV files with headers via csv_name(filename,header) specification

    • added log_normpdf()

    • added .is_zero()

    • added quantile()

  • The sparse matrix test using scipy, if available, is now simplified thanks to recently added reticulate conversions.

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 24 Jan 2020

RcppArmadillo 0.9.800.4.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 680 other packages on CRAN.

A second small Armadillo bugfix upstream update 9.800.4 came out yesterday for the 9.800.* series, following a similar bugfix release 9.800.3 in December. This time just one file was changed (see below).

Changes in RcppArmadillo version 0.9.800.4.0 (2020-20-24)

  • Upgraded to Armadillo release 9.800.4 (Horizon Scraper)

    • fixes for incorrect type promotion in normpdf()

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 16 Jan 2020

RcppRedis 0.1.10: Switch to tinytest

Another minor release of RcppRedis just arrived on CRAN, following a fairly long break since the last release in October 2018.

RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has carried production loads for several years now.

This release switches to the fabulous tinytest package, allowing for very flexible testing during development and deployment—three cheers for easily testing installed packages too.

Changes in version 0.1.10 (2020-01-16)

  • The package now uses tinytest for unit tests (Dirk in #41).

Courtesy of CRANberries, there is also a diffstat report for this release. More information is on the RcppRedis page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 09 Dec 2019

RcppClassic 0.9.12

A maintenance release 0.9.12 of the RcppClassic package arrived earlier today on CRAN. This package provides a maintained version of the otherwise deprecated initial Rcpp API which no new projects should use as the normal Rcpp API is so much better.

Changes are all internal. Testing is now done via tinytest, vignettes are now pre-built and at the request of CRAN we no longer strip the resulting library. No other changes were made.

CRANberries also reports the changes relative to the previous release from July of last year.

Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link