Thinking inside the box

Thu, 17 Dec 2020

RcppEigen 0.3.3.9.1: New upstream

A new release 0.3.3.9.1 of RcppEigen arrived on CRAN today (and just went to Debian too) bringing support for Eigen 3.3.9 to R. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.

The release started with a gentle nudge from the Stan team to package Eigen 3.3.8. And once I got around to it and sat down, Eigen 3.3.9 had just been released the previous day. The freshly prepared release 0.3.3.9.0 was then announced on Twitter (and the rcpp-devel mailing list) along with a request for testing alongside the usual extended reverse dependency checking we do (just like CRAN asks us to). And it turns out that Eigen 3.3.9 required an update to StanHeaders, a CRAN package that is itself quite widely used. This took a couple of days and its round of testing, but Ben Goodrich shipped it to CRAN last night, so we were able to ship RcppEigen in turn today.

So this release once again bring few required changes to Eigen (see this directory for details). As we said for the previous three releases:

One additional and recent change was the accomodation of a recent CRAN Policy change to not allow gcc or clang to mess with diagnostic messages. A word of caution: this may make your compilation of packages using RcppEigen very noisy so consider adding -Wno-ignored-attributes to the compiler flags added in your ~/.R/Makevars.

I actually find this requirement rather annoying. Eigen is then only usable if you set, say,

-Wno-deprecated-declarations -Wno-parentheses -Wno-ignored-attributes -Wno-unused-function

as g++ options in ~/.R/Makevars. But CRAN makes the rules. Maybe if a few of us gently and politely nudge them they may relent one day. One can only hope.

The complete NEWS file entry follows.

Changes in RcppEigen version 0.3.3.9.1 (2020-12-17)

Upgraded to Eigen 3.3.9 (Dirk in #92 fixing #91).

Added GitHub Actions CI using run.sh from r-ci (Dirk)

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 15 Dec 2020

RcppAnnoy 0.0.18: Tweaks

annoy image

A new maintenance release, now at version 0.0.18, of RcppAnnoy is now on CRAN. RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release is follow-up to release 0.0.17 which was made four weeks ago, and which brought the new upstream release 1.17 of Annoy. We now have an updated upstream with a PR by Aaron aiming for improved control of RNG seeding to ensure better reproducibility, along with extended tests and new helpers for RcppAnnoy and Annoy version numbers. The release was once again coordinated with Annoy and James whose BiocNeighbors and uwot packages both consume the Annoy header library shipped here.

Detailed changes follow below.

Changes in version 0.0.18 (2020-12-15)

Small tweaks to threading policy header defines (Dirk closing #65)

Vignette code is again compiled during testing (Aaron Lum and Dirk in #66 addressing #64)

Upstream code (with Aaron's PR) was synchronized once more (Dirk in #67)

A new helper function was added to report the Annoy version (Aaron in #68)

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 11 Dec 2020

RcppSpdlog 0.0.4: New upstream

Not quite two months after the last release, we are happy to announce version 0.0.4 of RcppSpdlog. It contains release 1.8.2 of spdlog made today, along with version 7.1.3 of fmt.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.4 (2020-12-11)

Upgraded to upstream release spdlog 1.8.2

Added GitHub Actions CI using run.sh from r-ci

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 03 Dec 2020

#31: Test your R package against bleeding-edge gcc

Welcome to the 31th post in the rapturously rampant R recommendations series, or R⁴ for short. This post will once again feature Docker for use with R.

Earlier this week, I received a note from CRAN about how my RcppTOML package was no longer building with the (as of right now of course unreleased) version 11 of the GNU C++ compiler, i.e. g++-11. And very kindly even included a hint about the likely fix (which was of course correct). CRAN, and one of its maintainers in particular, is extremely forward-looking in terms of toolchain changes. A year ago we were asked to updated possible use of global variables in C code as gcc-10 tightened the rules. This changes is a C++ one, and a fairly simple one of simply being more explicit with include headers. Previous g++ release had done the same.

The question now was about the least painful way to get g++-11 onto my machine, with the least amount of side-effects. Regular readers of this blog will know where this is headed, but even use of Docker requires binaries. A look at g++-11 within packages.debian.org comes up empty. No Debian means no Ubuntu. But … there is a PPA for Ubuntu with toolchain builds we have used before. And voilà there we have it: within the PPA for Ubuntu Toolchain repository is the volatile packages PPA with both g++-10 and g++-11. Here Ubuntu 20.10 works with g++-10, but g++-11 requires Ubuntu 21.04. Docker containers are there for either. So with the preliminaries sorted out, the key steps are fairly straightforward:

start from ubuntu:21.04 to be able to install g++-11 later
install the software-properties-common package to be able to add a PPA
(plus a few more packages to deal with the repository signing key)
run the sudo add-apt-repository ppa:ubuntu-toolchain-r/volatile command to add the volatile packages PPA
install g++-11 (along with, for good measure) gcc-11 and gfortran-11
use update-alternative (a clever Debian/Ubuntu command) to make version ‘11’ the default
install R itself (via r-base-core) which we simply take from the distro as 21.04 is by construction very recent
install Rcpp via the r-cran-rcpp binary which covers all dependencies for the package in question

And that is it! RcppTOML is fairly minimal and could be a member of the tinyverse so no other dependencies are needed—if your package has any you could just use the standard steps to install from source, or binary (including using RSPM or bspm). You can see the resulting Dockerfile which contains a minimal amount of extra stuff to deal with some environment variables and related settings. Nothing critical, but it smoothes the experience somewhat.

This container is now built (under label rocker/r-edge with tags latest and gcc-11), and you can download it from Docker Hub. With that the ‘proof’ of the (now fixed and uploaded) package building becomes as easy as

edd@rob:~/git/rcpptoml(master)$ docker run --rm -ti -v $PWD:/mnt -w /mnt rocker/r-edge:gcc-11 g++ --version
g++ (Ubuntu 11-20201128-0ubuntu2) 11.0.0 20201128 (experimental) [master revision fb6b29c85c4:a331ca6194a:e87559d202d90e614315203f38f9aa2f5881d36e]
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

edd@rob:~/git/rcpptoml(master)$ 
edd@rob:~/git/rcpptoml(master)$ docker run --rm -ti -v $PWD:/mnt -w /mnt rocker/r-edge:gcc-11 R CMD INSTALL RcppTOML_0.1.7.tar.gz
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘RcppTOML’ ...
** using staged installation
** libs
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I../inst/include/ -DCPPTOML_USE_MAP -I'/usr/lib/R/site-library/Rcpp/include'    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-Fuvi9C/r-base-4.0.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I../inst/include/ -DCPPTOML_USE_MAP -I'/usr/lib/R/site-library/Rcpp/include'    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-Fuvi9C/r-base-4.0.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c parse.cpp -o parse.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o RcppTOML.so RcppExports.o parse.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-RcppTOML/00new/RcppTOML/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppTOML)
edd@rob:~/git/rcpptoml(master)$

I hope both the availability of such a base container with gcc-11 (and g++-11 and gfortran-11) as well as a “recipe” for building similar containers with newer clang version will help other developers.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Wed, 02 Dec 2020

RcppTOML 0.1.7: Support for g++-11, Minor Updates

A new RcppTOML release arrived on CRAN earlier today evening. RcppTOML brings TOML to R.

TOML is a file format that is most suitable for configurations, as it is meant to be edited by humans but read by computers. It emphasizes strong readability for humans while at the same time supporting strong typing as well as immediate and clear error reports. On small typos you get parse errors, rather than silently corrupted garbage. Much preferable to any and all of XML, JSON or YAML – though sadly these may be too ubiquitous now. TOML has been making inroads with projects such as the Hugo static blog compiler, or the Cargo system of Crates (aka “packages”) for the Rust language.

CRAN had sent us a note that the package no longer compiled under the [unreleased, of course, never change, BDR ;-) ] g++-11 compiler, but were kind enough to hint that it was only lacking an #include <limits>. These things happen: newer compilers are generally more strict, and that is generally a good things. (Last year this time we prepped code for the more stringent view on global variables under gcc-10. Earlier g++ version had similar demands to clarify include headers.) I set up a simple Docker contain with on Ubuntu 21.04 with g++-11, R, and Rcpp to build the package and make this change (which was of course also PR’ed upstream at cpptoml), plus some other small ones that update the package since the last release roughly 18 months ago. We also switched CI use to the r-ci setup I should blog about a little more, removed a bashism and updated a few URLs. The bulleted list of changes in this version follows.

Changes in version 0.1.7 (2020-12-01)

Add #include <limits> to header file, also contributed upstream, to permit compilation under the (unreleased) g++-11.

Switch the simple cleanup script to sh.

Switch CI use to r-ci for focal and bspm.

Update several TOML URLs to https://toml.io/en/.

Courtesy of my CRANberries, there is a diffstat report for this release. More information is on the RcppTOML page page. Please use the GitHub issue tracker for issues and bugreports.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 30 Nov 2020

inline 0.3.17: Refactored and New Tests

A new release of the inline package arrived on CRAN this evening and has already been shipped to Debian as well. inline facilitates writing code in-line in simple string expressions or short files. The package was used quite extensively by Rcpp in the days before Rcpp Attributes arrived on the scene proving an even better alternative for its use cases. inline is still use by rstan and a number of other packages.

One of those other packages is mkin, and its author Johannes Ranke overhauled the saving and re-loading of C functions part with a really well-done set of contributions. In the process we also added unit testing via the lovely tinytest, and changed to continuous integration setup to r-ci.

See below for a detailed list of changes extracted from the NEWS file.

Changes in inline version 0.3.17 (2020-11-30)

Unit testing is now supported via tinytest (Johannes in #15 addressing #14).

CI was updated to use focal and run.sh from r-ci on Travis and GitHub Actions (Dirk)

The writing and reading of compiled code was refactored and extended (Johannes in #16 fixing #13).

Some minor problems related to CRAN checks and tests were corrected (Johannes and Dirk in #17, Johannes in #18, #19, #20).

Small stylistic updates have been applied to some R and Rd files (Dirk).

Courtesy of my CRANberries, there is a comparison to the previous release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/inline | permanent link

Mon, 16 Nov 2020

RcppArmadillo 0.10.1.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 779 other packages on CRAN.

This release ties up a few loose ends from the recent 0.10.1.0.0.

Changes in RcppArmadillo version 0.10.1.2.0 (2020-11-15)

Upgraded to Armadillo release 10.1.2 (Orchid Ambush)

Remove three unused int constants (#313)

Include main armadillo header using quotes instead of brackets

Rewrite version number use in old-school mode because gcc 4.8.5

Skipping parts of sparse conversion on Windows as win-builder fails

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

RcppAnnoy 0.0.17

annoy image

A new release 0.0.17 of RcppAnnoy is now on CRAN. RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release brings a new upstream version 1.17, released a few weeks ago, which adds multithreaded index building. This changes the API by adding a new ‘threading policy’ parameter requiring code using the main Annoy header to update. For this reason we waited a little for the dust to settle on the BioConductor 3.12 release before bringing the changes to BiocNeighbors via this commit and to uwot via this simple PR. Aaron and James updated their packages accordingly so by the time I uploaded RcppAnnoy it made for very smooth sailing as we all had done our homework with proper conditional builds, and the package had no other issue preventing automated processing at CRAN. Yay. I also added a (somewhat overdue one may argue) header file RcppAnnoy.h regrouping defines and includes which should help going forward.

Detailed changes follow below.

Changes in version 0.0.17 (2020-11-15)

Upgrade to Annoy 1.17, but default to serial use.

Add new header file to regroup includes and defines.

Upgrade CI script to use R with bspm on focal.

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 12 Nov 2020

tidyCpp 0.0.2: More documentation and features

A first update of the still fairly new package tidyCpp is now on CRAN. The packages offers a C++ layer on top of the C API for R which aims to make its use a little easier and more consistent.

The vignette has been extended with a new examples, a new section and some general editing. A few new defines have been added mostly from the Rinternals.h header. We also replaced the Shield class with a simpler yet updated version class Protect. The name better represent the core functionality of offering a simpler alternative to the PROTECT and UNPROTECT macro pairing. We also added a short discussion to the vignette of a gotcha one has to be mindful of, and that we fell for ourselves in version 0.0.1. We also added a typedef so that code using Shield can still be used.

The NEWS entry follows.

Changes in tidyCpp version 0.0.2 (2020-11-12)

Expanded definitions in internals.h to support new example.

The vignette has been extended with an example based on package uchardet.

Class Shield has been replaced by an new class Protect; a compatibility typdef has been added.

The examples and vignette have been clarified with respect to proper ownership of protected objects; a new vignette section was added.

Thanks to my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/tidycpp | permanent link

Mon, 02 Nov 2020

RcppSimdJson 0.1.2: New Upstream, New Utilities

A new RcppSimdJson release arrived on CRAN late yesterday bringing along the one recently updated simdjson release 0.6.0.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

Other than the upstream update, Brendan added some new utilities to check for valid utf-8 or json format, and to minify json plus a small workaround for a clang-9 bug we encountered. We can confirm Daniel’s statement on ridiculously fast utf-8 validattion. It is so cool to work with amazing tools.

The NEWS entry follows.

Changes in version 0.1.3 (2020-11-01)

Added URLs to DESCRIPTION (Dirk closing #50).

Upgraded to simdjson 0.6.0 (Dirk in #52).

New policy option to always convert integers to int64_t (Brendan in #55 closing #54).

Added workaround for odd clang-9 bug (Brendan in #57).

New utility functions is_valid_utf8(), is_valid_json() and fminify() (Brendan in #58).

Courtesy of my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 24 Oct 2020

digest 0.6.27: Build fix

Exactly one week after the previous release 0.6.26 of digest, a minor cleanup release 0.6.27 just arrived on CRAN and will go to Debian shortly.

digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, and blake3 algorithms) permitting easy comparison of R language objects. It is a fairly widely-used package (currently listed at one million monthly downloads, 282 direct reverse dependencies and 8068 indirect reverse dependencies, or just under half of CRAN) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation.

Release 0.6.26 brought support for the (nice, even cryptographic) blake3 hash algorithm. In the interest of broader buildability we had already (with a sad face) disabled a few very hardware-specific implementation aspects using intrinsic ops. But to our chagrin, we left one #error define that raised its head on everybody’s favourite CRAN build platform. Darn. So 0.6.27 cleans that up and also removes the check and #error as … all the actual code was already commented out. If you read this and tears start running down your cheeks, then by all means come and help us bring blake3 to its full (hardware-accelerated) potential. This (probably) only needs a little bit of patient work with the build options and configurations. You know where to find us…

My CRANberries provides the usual summary of changes to the previous version.

For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/digest | permanent link

Fri, 23 Oct 2020

RcppSpdlog 0.0.3: New features and much more docs

A good month after the initial two releases, we are thrilled to announce relase 0.0.3 of RcppSpdlog. This brings us release 1.8.1 of spdlog as well as a few local changes (more below).

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovic.

This version of RcppSpdlog brings a new top-level function setLogLevel to control what events get logged, updates the main example to show this and to also make the R-aware logger the default logger, and adds both an extended vignette showing several key features and a new (external) package documentation site.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.3 (2020-10-23)

New function setLogLevel with R accessor in exampleRsink example

Updated exampleRsink to use default logger instance

Upgraded to upstream release 1.8.1 which contains finalised upstream use to switch to REprintf() if R compilation detected

Added new vignette with extensive usage examples, added compile-time logging switch example

A package documentation website was added

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 21 Oct 2020

RcppZiggurat 0.1.6

ziggurats

A new release, now at version 0.1.6, of RcppZiggurat is now on the CRAN network for R.

The RcppZiggurat package updates the code for the Ziggurat generator by Marsaglia and other which provides very fast draws from a Normal distribution. The package provides a simple C++ wrapper class for the generator improving on the very basic macros, and permits comparison among several existing Ziggurat implementations. This can be seen in the figure where Ziggurat from this package dominates accessing the implementations from the GSL, QuantLib and Gretl—all of which are still way faster than the default Normal generator in R (which is of course of higher code complexity).

This release brings a corrected seed setter and getter which now correctly take care of all four state variables, and not just one. It also corrects a few typos in the vignette. Both were fixed quite a while back, but we somehow managed to not ship this to CRAN for two years.

The NEWS file entry below lists all changes.

Changes in version 0.1.6 (2020-10-18)

Several typos were corrected in the vignette (Blagoje Ivanovic in #9).

New getters and setters for internal state were added to resume simulations (Dirk in #11 fixing #10).

Minor updates to cleanup script and Travis CI setup (Dirk).

Courtesy of my CRANberries, there is a diffstat report for this release. More information is on the RcppZiggurat page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 20 Oct 2020

RcppArmadillo 0.10.1.0.0

armadillo image

A little while ago, Conrad released version 10.1.0 of Armadillo, a a new major release. As before, given his initial heads-up we ran two full reverse-depends checks, and as a consequence contacted four packages authors (two by email, two via PR) about a miniscule required change (as Armadillo now defaults to C++11, an old existing setting of avoiding C++11 lead to an error). Our thanks to those who promptly update their packages—truly appreciated. As it turns out, Conrad also softened the error by the time the release ran around.

But despite our best efforts, the release was delayed considerably by CRAN. We had made several Windows test builds but luck had it that on the uploaded package CRAN got itself a (completely spurious segfault—which can happen on a busy machine building machine things at once). Sadly it took three or four days for CRAN to reply our email. After which it took another number of days for them to ponder the behaviour of a few new ‘deprecated’ messaged tickled by at the most ten or so (out of 786) packages. Oh well. So here we are, eleven days after I emailed the rcpp-devel list about the new package being on CRAN but possibly delayed (due to that seg.fault). But during all that time the package was of course available via the Rcpp drat.

The changes in this release are summarized below as usual and are mostly upstream along with an improved Travis CI setup due to the aforementioned use of the bspm package for binaries at Travis.

Changes in RcppArmadillo version 0.10.1.0.0 (2020-10-09)

Upgraded to Armadillo release 10.1.0 (Orchid Ambush)

C++11 is now the minimum required C++ standard

faster handling of compound expressions by trimatu() and trimatl()

faster sparse matrix addition, subtraction and element-wise multiplication

expanded sparse submatrix views to handle the non-contiguous form of X.cols(vector_of_column_indices)

expanded eigs_sym() and eigs_gen() with optional fine-grained parameters (subspace dimension, number of iterations, eigenvalues closest to specified value)

deprecated form of reshape() removed from Cube and SpMat classes

ignore and warn on use of the ARMA_DONT_USE_CXX11 macro

Switch Travis CI testing to focal and BSPM

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 17 Oct 2020

digest 0.6.26: Blake3 and Tuning

And a new version of digest is now on CRAN will go to Debian shortly.

digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, and blake3 algorithms) permitting easy comparison of R language objects. It is a fairly widely-used package (currently listed at 896k monthly downloads, 279 direct reverse dependencies and 8057 indirect reverse dependencies, or just under half of CRAN) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation.

This release brings two nice contributed updates. Dirk Schumacher added support for blake3 (though we could probably push this a little harder for performance, help welcome). Winston Chang benchmarked and tuned some of the key base R parts of the package. Last but not least I flipped the vignette to the lovely minidown, updated the Travis CI setup using bspm (as previously blogged about in r⁴ #30), and added a package website using Matertial for MkDocs.

My CRANberries provides the usual summary of changes to the previous version.

For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/digest | permanent link

Wed, 14 Oct 2020

dang 0.0.12: Two new functions

A new release of the dang package is now on CRAN, roughly one year after the last release. The dang package regroups a few functions of mine that had no other home as for example lsos() from a StackOverflow question from 2009 (!!) is one, this overbought/oversold price band plotter from an older blog post is another. More recently added were helpers for data.table to xts conversion and a git repo root finder.

This release adds two functions. One was mentioned just days ago in a tweet by Nathan and is a reworked version of something Colin tweeted about a few weeks ago: a little data wrangling off the kewl rtweet to find maximally spammy accounts per search topic. In other words those who include more than ‘N’ hashtags for given search term. The other is something I, if memory serves, picked up a while back on one of the lists: a base R function to identify non-ASCII characters in a file. It is a C function that is not directly exported by and hence no accessible, so we put it here (with credits, of course). I mentioned it yesterday when announcing tidyCpp as I this C function was the starting point for the new tidyCpp wrapper around some C API of R functions.

The (very short) NEWS entry follows.

Changes in version 0.0.12 (2020-10-14)

New functions muteTweets and checkPackageAsciiCode.

Updated CI setup.

Courtesy of CRANberries, there is a comparison to the previous release. For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/dang | permanent link

Tue, 13 Oct 2020

tidyCpp 0.0.1: New package

A new package arrived on CRAN a few days ago. It offers a few headers files which wrap (parts) of the C API for R, but in a form that may be a little easier to use for C++ programmers. I have always liked how in Rcpp we offer good parts of the standalone R Math library in a namespace R::. While working recently with a particular C routine (for checking non-ASCII characters that will be part of the next version of the dang package which collecting various goodies in one place), I realized there may be value in collecting a few more such wrappers. So I started a few simple ones starting from simple examples.

Currently we have five headers defines.h, globals.h, internals.h, math.h, and shield.h. The first four each correpond to an R header file of the same or similar name, and the last one brings a simple yet effective alternative to PROTECT and UNPROTECT from Rcpp (in a slightly simplified way). None of the headers are “complete”, for internals.h in particular a lot more could be added (as I noticed today when experimenting with another source file that may be converted). All of the headers can be accessed with a simple #include <tidyCpp> (which, following another C++ convention, does not have a .h or .hpp suffix). And a the package ships these headers, packages desiring to use them only need LinkingTo: tidyCpp.

As usage examples, we (right now) have four files in the snippets/ directory of the package. Two of these, convolveExample.cpp and dimnamesExample.cpp both illustrate how one could change example code from Writing R Extensions. Then there are also a very simple defineExample.cpp and a shieldExample.cpp illustrating how much easier Shield() is compared to PROTECT and UNPROTECT.

Finally, there is a nice vignette discussing the package motivation with two detailed side-by-side ‘before’ and ‘after’ examples that are the aforementioned convolution and dimnames examples.

Over time, I expect to add more definitions and wrappers. Feedback would be welcome—it seems to hit a nerve already as it currently has more stars than commits even though (prior to this post) I had yet to tweet or blog about it. Please post comments and suggestions at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/tidycpp | permanent link

Mon, 12 Oct 2020

GitHub Streak: Round Seven

Six years ago I referenced the Seinfeld Streak used in an earlier post of regular updates to to the Rcpp Gallery:

This is sometimes called Jerry Seinfeld’s secret to productivity: Just keep at it. Don’t break the streak.

and then showed the first chart of GitHub streaking 366 days:

github activity october 2013 to october 2014

And five years ago a first follow-up appeared in this post about 731 days:

github activity october 2014 to october 2015

And four years ago we had a followup at 1096 days

github activity october 2015 to october 2016

And three years ago we had another one marking 1461 days

github activity october 2016 to october 2017

And two years ago another one for 1826 days

github activity october 2017 to october 2018

And last year another one bringing it to 2191 days

github activity october 2018 to october 2019

And as today is October 12, here is the newest one from 2019 to 2020 with a new total of 2557 days:

github activity october 2018 to october 2019

Again, special thanks go to Alessandro Pezzè for the Chrome add-on GithubOriginalStreak.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/computers/misc | permanent link

Thu, 08 Oct 2020

RcppSimdJson 0.1.2: Upstream update

A new RcppSimdJson release arrived on CRAN yesterday bringing along the simdjson 0.5.0 release that happened a few weeks.

Beside the upstream update, not too much happened to our package itself since 0.1.1 though Brandon did help one user to seriously speed up his JSON processing. The (this time very short) NEWS entry follows.

Changes in version 0.1.2 (2020-10-07)

Upgraded to simdjson 0.5.0 (Dirk #49)

Courtesy of my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 07 Oct 2020

littler 0.3.12: Exciting updates

max-heap image

The thirteenth release of littler as a CRAN package became available today (after a three day ‘rest’ at CRAN for no real reason), following in the fourteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later.

littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do in recent years.

littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default where a good idea?) and simply does not exist on Windows (yet – the build system could be extended – see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH.

A few examples are highlighted at the Github repo, as well as in the examples vignette.

This release brings five new example scripts and command wrappers:

installDeps.r installs all dependencies of package (directory or tarball)
installRSPM.r relies on RSPM to install binary packages
installBSPM.r relies on BSPM to install binary packages (esp. on Linux)
cranIncomimg.r checks the incoming queue for one or more packages
urlUpdates.r checks and/or updates stale URLs leading to redirects

A number of commands were also extended or updated (see below for more). We have a new and very slick documentation website once again utilising Material for MkDocs. Last but not least the two included vignettes now use minidown and the fabulous water css theme—which reduced the file sizes of the two vignettes from, respectively, 884kb and 873kb to 47kb and 15kb. Yes, that is correct. That alone brought the package file size down from 641kb to 116kb. Incredible.

The NEWS file entry is below.

Changes in littler version 0.3.12 (2020-10-04)

Changes in examples

Updates to scripts tt.r, cos.r, cow.r, c4r.r, com.r

New script installDeps.r to install dependencies

Several updates tp script check.r

New script installBSPM.r and installRSPM.r for binary package installation (Dirk and Iñaki in #81)

New script cranIncoming.r to check in Incoming

New script urlUpdate.r validates URLs as R does

Changes in package

Travis CI now uses BSPM

A package documentation website was added

Vignettes now use minidown resulting in much reduced filesizes: from over 800kb to under 50kb (Dirk in #83)

My CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page, and now also on the new package docs website. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter.

Comments and suggestions are welcome at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/littler | permanent link

Sat, 03 Oct 2020

pinp 0.0.10: More Tweaks

A new version of our pinp package arrived on CRAN two days ago, roughly one year after the previous release. The pinp package allows for snazzier one or two column Markdown-based pdf vignettes, and is now used by a few packages. A screenshot of the package vignette can be seen below. Additional screenshots are at the pinp page.

pinp vignette

This release adds another option for a customized date YAML variable suitable for e.g. a published at date thanks to Ilya Kashnitsky, has some tweaks to the README.md as well as support for pandoc columns mode (as a small extension of code from the nice repo by Grant McDermott).

The NEWS entry for this release follows.

Changes in pinp version 0.0.10 (2020-10-01)

New document_date YAML variable to optionally set a 'Published at' or alike date (Ilya Kashnitsky in #85).

Small tweaks to README.md (Dirk)

Support pandoc columns mode (Dirk in #88)

Courtesy of my CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/pinp | permanent link

Sun, 27 Sep 2020

pkgKitten 0.2.0: Now with tinytest and new docs

kitten

A new release 0.2.0 of pkgKitten just hit on CRAN today, or about eleven months after the previous release.

This release brings support for tinytest by having pkgKitten::kitten() automagically call tinytest::puppy() if the latter package is installed (and the user did not opt out of calling it). So your newly created minimal package now also uses a wonderful yet tiny testing framework. We also added a new documentation site using the previously tweeted-about wrapper for Material for MkDocs I really dig. And last but not least we switched to BSPM-based Continued Integration (which I wrote about yesterday in R⁴ #30) and fixed one bug regarding the default NAMESPACE file.

Changes in version 0.2.0 (2020-09-27)

Continuous Integration uses the updated BSPM-based script on Travis and with GitHub Actions (Dirk in #11 plus earlier commits).

A new default NAMESPACE file is now installed (Dirk in #12).

A package documentation website was added (Dirk in #13).

Call tinytest::puppy if installed and not opted out (Dirk in #14).

More details about the package are at the pkgKitten webpage, the (new) pkgKitten docs site, and the pkgKitten GitHub repo.

Courtesy of my CRANberries site, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/pkgkitten | permanent link

Sat, 26 Sep 2020

#30: Easy, Reliable, Fast and Portable Linux and macOS Continuous Integration

Welcome to the 30th post in the rarified R recommendation resources series or R⁴ for short. The last post introduced BSPM. In the four weeks since, we have worked some more on BSPM to bring it to the point where it is ready for use with continuous integration. Building on this, it is now used inside the run.sh script that driven our CI use for many years (via the r-travis repo).

Which we actually use right now on three different platforms:

Travis CI as for example seen in this .travis.yml file (as well as a half dozen more);
Azure Pipelines for example seen in this azure-pipelines.yml;
GitHub Actions as for example seen in this R-CMD-check.yaml file.

All three use the exact same script facilitating this, and run a ‘matrix’ over Linux and macOS. You read this right: one CI setup that is portable and which you can take to your CI provider of choice. No lock-in or tie-in. Use what works, change at will. Or run on all three if you like burning extra cycles.

This is already used by handful of my repos as well as by at least two repos of friends also deploying r-travis. How does it work? In a nutshell we are

downloading run.sh via curl and changing its mode;
running run.sh bootstrap which sets the operating system default:
- on Linux we use Ubuntu,
  - add two PPAs repos for R itself and over 4600 r-cran-* binaries,
  - and enable BSPM to use these from install.packages()
- on macOS we use the standard setup also used on Travis, GitHub Actions and elsewhere;
- this provides us with fast, reliable, easy, and portable access to binaries on two OSs under dependency resolution;
running run.sh install_deps to install just the requireded Depends:, Imports: and LinkingTo:
running run.sh tests to build the tarball and test it via R CMD check --as-cran.

There are several customizations that are possible via environment variables

additional PPAs or drat repos can be added to offer even more package choice;
alternatively one could run run.sh install_all to also install Suggests:;
optionally one could run run.sh install_r pkgA pkgB ... to install packages explicitly listed;
optionally one could also run run.sh install_aptget r-cran-pkga r-cran-pkgb otherpackage to add more Ubuntu binaries.

We find this setup compelling. The scheme is simple: there really is just one shell script behind it which can also be downloaded and altered. The scheme is also portable as we can (as shown) rotate between CI provides. The scheme is also more flexible: in case of debugging needs one can simply run the script on a local Docker or VM instance. Lastly, the scheme moves away from single points of failure or breakage.

Currently the script uses only BSPM as I had the hunch that it would a) work and b) be compelling. Adding support for RSPM would be equally easy, but I have no immediate need to do so. Adding BioConductor installation may be next. That is easy when BioConductor uses r-release; it may be little more challenging under r-devel to but it should work too. Stay tuned.

In the meantime, if the above sounds compelling, give run.sh from r-travis a go!

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Sun, 20 Sep 2020

RcppSpdlog 0.0.2: New upstream, awesome new stopwatch

Following up on the initial RcppSpdlog 0.0.1 release earlier this week, we are pumped to announce release 0.0.2. It contains upstream version 1.8.0 for spdlog which utilizes (among other things) a new feature in the embedded fmt library, namely completely automated formatting of high resolution time stamps which allows for gems like this (taken from this file in the package and edited down for brevity):

    spdlog::stopwatch sw;                                   // instantiate a stop watch

    // some other code

    sp->info("Elapsed time: {}", sw);

What we see is all there is: One instantiates a stopwatch object, and simply references it. The rest, as they say, is magic. And we get tic / toc alike behaviour in modern C++ at essentially no cost us (as code authors). So nice. Output from the (included in the package) function exampleRsink() (again edited down just a little):

R> RcppSpdlog::exampleRsink()
[13:03:23.845114] [fromR] [I] [thread 793264] Welcome to spdlog!
[...]
[13:03:23.845205] [fromR] [I] [thread 793264] Elapsed time: 0.000103611
[...]
[13:03:23.845281] [fromR] [I] [thread 793264] Elapsed time: 0.00018203
R>

We see that the two simple logging instances come 10 and 18 microseconds into the call.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovic.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.2 (2020-09-17)

Upgraded to upstream release 1.8.0

Switched Travis CI to using BSPM, also test on macOS

Added 'stopwatch' use to main R sink example

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppSpdlog page.

The only sour grapes, again, are once more over the CRAN processing. And just how 0.0.1 was delayed for no good reason for three weeks, 0.0.2 was delayed by three days just because … well that is how CRAN rules sometimes. I’d be even more mad if I had an alternative but I don’t. We remain grateful for all they do but they really could have let this one through even at one-day update delta. Ah well, now we’re three days wiser and of course nothing changed in the package.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 16 Sep 2020

RcppSpdlog 0.0.1: New and Exciting Logging Package

Very thrilled to announce a new package RcppSpdlog which is now on CRAN in its first release 0.0.1. We had tweeted once about the earliest version which had already caught the eyes of Gabi upstream.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovic.

I had meant to package this for a few years now but didn’t find an (easy, elegant) way to completely lift stdout / stderr and related uses which R wants us to remove for smoother operations from R itself including synchronized input/output. It was only a few weeks ago that I realized I should subclass a logger (or, more concretely, a sink for a logger) which could then use R input/output. With the right idea the implementaton was easy, and [Gabi]((https://github.com/gabime) was most helpful in making sure R CMD check would not see one or two remaining C++ i/o operations (which we currently do by not activating a default logger, and substituing REprintf() in one call). So this is now clean and sween and a simple use is included in an example in the package we can show here too (in slightly shorter form minus the documentation header):

// this portmanteau include also defines the r_sink we use below, and which
// diverts all logging to R via the Rcpp::Rcout replacement for std::cout
#include <RcppSpdlog>

// [[Rcpp::export]]
void exampleRsink() {

    std::string logname = "fromR";                          // fix a name for this logger
    auto sp = spdlog::get(logname);                         // retrieve existing one
    if (sp == nullptr) sp = spdlog::r_sink_mt(logname);     // or create new one if needed

    // change log pattern (changed from [%H:%M:%S %z] [%n] [%^---%L---%$] )
    sp->set_pattern("[%H:%M:%S.%f] [%n] [%^%L%$] [thread %t] %v");

    sp->info("Welcome to spdlog!");
    sp->error("Some error message with arg: {}", 1);

    sp->warn("Easy padding in numbers like {:08d}", 12);
    sp->critical("Support for int: {0:d};  hex: {0:x};  oct: {0:o}; bin: {0:b}", 42);
    sp->info("Support for floats {:03.2f}", 1.23456);
    sp->info("Positional args are {1} {0}..", "too", "supported");
    sp->info("{:<30}", "left aligned");

}

The NEWS entry for the first release follows.

Changes in RcppSpdlog version 0.0.1 (2020-09-08)

Initial release with added R/Rcpp logging sink example

The only sour grapes, if any, are over the CRAN processing. This was originally uploaded three weeks ago. As a new package, it got extra attention and some truly idiosyncratic attention to two details that were already supplied in the first uploaded version. Yet it needed two rounds of going back and forth for really no great net gain, yet wasting a week each time. I am not all that impressed by this, and not particularly pleased either, but I presume is the “tax” we all pay in order to enjoy the unsurpassed richness of the CRAN repository system which continues to work just flawlessly.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 06 Sep 2020

inline 0.3.16: Now with system2()

A new minor release of the inline package just arrived on CRAN. inline facilitates writing code in-line in simple string expressions or short files. The package is mature and stable, and can be considered to be in maintenance mode: Rcpp used it extensively in the vrey early days before Rcpp Attributes provided an even better alternative. Seveal other package still rely on inline.

One of these package is rstan, and Ben Goodrich updated our use of system() to system2() allowing for better error diagnostics. We also did a bit of standard maintenance to Travis CI and the README.md file.

See below for a detailed list of changes extracted from the NEWS file.

Changes in inline version 0.3.16 (2020-09-06)

Maintenance updates to README.md standardizing badges (Dirk).

Maintenance update to Travis CI setup (Dirk).

Switch to using system2() for better error diagnostics (Ben Goodrich in #12).

Courtesy of CRANberries, there is a comparison to the previous release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/inline | permanent link

Fri, 04 Sep 2020

nanotime 0.3.2: Tweaks

Another (minor) nanotime release, now at version 0.3.2. This release brings an endianness correction which was kindly contributed in a PR, switches to using the API header exported by RcppCCTZ, and tweaks test coverage a little with respect to r-devel.

nanotime relies on the RcppCCTZ package for (efficient) high(er) resolution time parsing and formatting up to nanosecond resolution, and the bit64 package for the actual integer64 arithmetic. Initially implemented using the S3 system, it has benefitted greatly from work by co-author Leonardo who not only rejigged nanotime internals in S4 but also added new S4 types for periods, intervals and durations.

The NEWS snippet adds full details.

Changes in version 0.3.2 (2020-09-03)

Correct for big endian (Elliott Sales de Andrade in #81).

Use the RcppCCTZ_API.h header (Dirk in #82).

Conditionally reduce test coverage (Dirk in #83).

Thanks to CRANberries there is also a diff to the previous version. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/nanotime | permanent link

Thu, 03 Sep 2020

RcppArmadillo 0.9.900.3.0

armadillo image

A few days ago, Conrad released a new minor version 9.900.3 of Armadillo which we packaged and tested as usual. Given the incremental release character, we only tested the release and not candidate release. No regressions were found, and, as usual, logs from reverse-depends runs are in the rcpp-logs repo.

All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.3.0 (2020-09-02)

Upgraded to Armadillo release 9.900.3 (Nocturnal Misbehaviour)

More efficient code for initialising matrices with fill::zeros

Fixes for various error messages

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 30 Aug 2020

RcppCCTZ 0.2.9: API Header Added

A new minor release 0.2.9 of RcppCCTZ is now on CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do—using copies in their packages which remains less than ideal.

This version adds a header file for the recently-exported three functions.

Changes in version 0.2.9 (2020-08-30)

Provide a header RcppCCZT_API.h for client packages.

Show a simple example of parsing a YYYYMMDD HHMMSS.FFFFFF date.

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

RcppSMC 0.2.2: Small updates

A new release 0.2.2 of the RcppSMC package arrived on CRAN earlier today (and once again as a very quick pretest-publish within minutes of submission).

RcppSMC provides Rcpp-based bindings to R for the Sequential Monte Carlo Template Classes (SMCTC) by Adam Johansen described in his JSS article. Sequential Monte Carlo is also referred to as Particle Filter in some contexts.

This releases contains two fixes from a while back that had not been released, a CRAN-requested update plus a few more minor polishes to make it pass R CMD check --as-cran as nicely as usual.

Changes in RcppSMC version 0.2.2 (2020-08-30)

Package helper files .editorconfig added (Adam in #43).

Change const correctness and add return (Leah in #44).

Updates to continuous integration and R versions used (Dirk)

Accomodate CRAN request, other updates to CRAN Policy (Dirk in #49 fixing #48).

Courtesy of CRANberries, there is a diffstat report for this release.

More information is on the RcppSMC page. Issues and bugreports should go to the GitHub issue tracker.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 28 Aug 2020

anytime 0.3.9: More Minor Maintenance

A new minor release of the anytime package arrived on CRAN yesterday. This is the twentieth release, but sadly we seem to be spinning our wheels just accomodating CRAN (which the two or three last releases focused on). Code and functionality remain mature and stable, of course.

anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, … format to either POSIXct or Date objects – and to do so without requiring a format string as well as accomodating different formats in one input vector. See the anytime page, or the GitHub README.md for a few examples.

This release once again has to play catchup with CRAN as r-devel now changes how tzone propagates when we do as.POSIXct(as.POSIXlt(Sys.time()) — which is now no longer “equal” to as.POSIXct(Sys.time()) even for a fixed, stored Sys.time() call result. Probably for the better, but an issue for now so we … effectively just reduced test coverage. Call it “progress”.

The full list of changes follows.

Changes in anytime version 0.3.9 (2020-08-26)

Skip one test file that is impossible to run across different CRAN setups, and life is definitely too short for these games.

Change remaining http:// to https:// because, well, you know.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. The issue tracker tracker off the GitHub repo can be use for questions and comments.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/anytime | permanent link

Wed, 26 Aug 2020

#29: Easy, Reliable, Fast Linux CRAN Binaries via BSPM

Welcome to the 29th post in the randomly repeating R recommendations series or R⁴ for short. Our last post #28 introduced RSPM, and just before that we also talked in #27 about binary installations on Ubuntu (which was also a T⁴ video). This post is joined with Iñaki Ucar and mainly about work we have done with his bspm package.

Background

CRAN has been a cornerstone of the success of R in recent years. As a well-maintained repository with stringent quality control, it ensures users have access to highest-quality statistical / analytical software that “just works”. Users on Windows and macOS also benefit from faster installation via pre-compiled binary packages.

Linux users generally install from source, which can be more tedious and, often, much slower. Those who know where to look have had access to (at least some) binaries for years as well (and one of us blogged and vlogged about this at length). Debian users get close to 1000 CRAN and BioConductor packages (and, true to Debian form, for well over a dozen hardware platforms). Michael Rutter maintains a PPA with 4600 binaries for three different Ubuntu flavors (see c2d4u4.0+). More recently, Fedora joined the party with 16000 (!!) binaries, essentially all of CRAN, via a Copr repository (see iucar/cran).

The buzz currently is however with RSPM, a new package manager by RStudio. An audacious project, it provides binaries for several Linux distributions and releases. It has already been tested in many RStudio Cloud sessions (including with some of our students) as well as some CI integrations.

RSPM cuts “across” and takes the breadth of CRAN across several Linux distributions, bringing installation of pre-built CRAN packages a binaries under their normal CRAN package names. Another nice touch is the integration with install.packages(): these binaries are installed in a way that is natural for R users—but as binaries. It is however entirely disconnected from the system package management. This means that the installation of a package requiring an external library may “succeed” and still fail, as a required library simply cannot be pulled in directly by RSPM.

So what is needed is a combination. We want binaries that are aware of their system dependencies but accessible directly from R just like RSPM offers it. Enter BSPM—the Bridge to System Package Manager package (also on CRAN).

The first illustration (using Ubuntu 18.04) shows RSPM on the left, and BSPM on the right, both installing the graphics package Cairo (and both using custom Rocker containers).

This fails for RSPM as no binary is present and a source build fails for the familiar lack of a -dev package. It proceeds just fine on the right under BSPM.

A second illustration shows once again RSPM on the left, and BSPM on the right (this time on Fedora), both installing the units package without a required system dependency.

The installation of units works for BSPM as the dependency libudunits is brought in, but fails under RSPM. The binary installation succeeds in both cases, but the missing dependency (the UDUNITS2 library) is brought in only by BSPM. Consequently, the package fails to load under RSPM.

Summary

To conclude, highlights of BSPM are:

direct installation of binary packages from R via R commands under their normal CRAN names (just like RSPM);
full integration with the system package manager, delivering system installations (improving upon RSPM);
full dependency resolution for R packages, including system requirements (improving upon RSPM).

This offers easy, reliable, fast installation of R packages, and we invite you to pick all three. We recommend usage with either Ubuntu with the 4.6k packages via the Rutter PPA, or Fedora via the even more complete Copr repository (which already includes a specially-tailored version of BSPM called CoprManager).

We hope this short note wets your appetite to learn more about bspm (which is itself on CRAN) and the two sets of Rocker containers shown. The rocker/r-rspm container comes in two two flavours for Ubuntu 18.04 and 20.04. Similarly, the rocker/r-bspm container comes in the same two two flavours for Ubuntu 18.04 and 20.04, as well as in a Debian testing variant.

Feedback is appreciated at the bspm or rocker issue trackers.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Tue, 11 Aug 2020

RcppSimdJson 0.1.1: More Features

A first update following for the exciting RcppSimdJson 0.1.0 release last month is now on CRAN. Version 0.1.1 brings further enhancements such direct parsing of raw chars, working with compressed files as well as much expanded querying ability all thanks to Brendan, some improvements to our demos thanks to Daniel as well as a small fix via a one-liner borrowed from upstream for a reported UBSAN issue.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

The detailed list of changes follows.

Changes in version 0.1.1 (2020-08-10)

Corrected incorrect file deletion when mixing local and remote files (Brendan in #34) closing #33.

Added support for raw vectors, compressed files, and compressed downloads (Dirk and Brendan in #36, #39, and #45 closing #35 and addressing issues raised in #40 and #44).

Examples in two demos are now more self-sufficient (Daniel Lemire and Dirk in #42).

Expanded query functionality to include single, flat, and nested queries (Brendan in #45 closing #43).

Split error handling parameters from error_ok/on_error into parse_error_ok/on_parse_error and query_error_ok/on_query_error (Brendan in #45).

One-line upstream change to address sanitizer error on cast.

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 10 Aug 2020

nanotime 0.3.1: Misc Build Fixes for Yuge New Features!

The nanotime 0.3.0 release four days ago was so exciting that we decided to do it again! Kidding aside, and fairly extensive tests notwithstanding we were bitten by a few build errors: who knew clang on macOS needed extra curlies to be happy, another manifestation of Solaris having no idea what a timezone setting “America/New_York” is, plus some extra pickyness from the SAN tests and whatnot. So Leonardo and I gave it some extra care over the weekend, uploaded it late yesterday and here we are with 0.3.1. Thanks again to CRAN for prompt processing even though they are clearly deluged shortly before their (brief) summer break.

nanotime relies on the RcppCCTZ package for (efficient) high(er) resolution time parsing and formatting up to nanosecond resolution, and the bit64 package for the actual integer64 arithmetic. Initially implemented using the S3 system, it has benefitted greatly from work by Leonardo Silvestri who rejigged internals in S4—and now added new types for periods, intervals and durations.

The NEWS snippet adds full details.

Changes in version 0.3.1 (2020-08-09)

Several small cleanups to ensure a more robust compilation (Leonardo and Dirk in #75 fixing #74).

Show Solaris some extra love by skipping tests and examples with a timezone (Dirk in #76).

Thanks to CRANberries there is also a diff to the previous version. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/nanotime | permanent link

Fri, 07 Aug 2020

RVowpalWabbit 0.0.15: Some More CRAN Build Issues

Another maintenance RVowpalWabbit package update brought us to version 0.0.15 earlier today. We attempted to fix one compilation error on Solaris, and addressed a few SAN/UBSAN issues with the gcc build.

As noted before, there is a newer package rvw based on the excellent GSoC 2018 and beyond work by Ivan Pavlov (mentored by James and myself) so if you are into Vowpal Wabbit from R go check it out.

CRANberries provides a summary of changes to the previous version. More information is on the RVowpalWabbit page. Issues and bugreports should go to the GitHub issue tracker.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rvowpalwabbit | permanent link

Thu, 06 Aug 2020

nanotime 0.3.0: Yuge New Features!

A fresh major release of the nanotime package for working with nanosecond timestamps is hitting CRAN mirrors right now.

nanotime relies on the RcppCCTZ package for (efficient) high(er) resolution time parsing and formatting up to nanosecond resolution, and the bit64 package for the actual integer64 arithmetic. Initially implemented using the S3 system, it has benefitted greatly from work by Leonardo Silvestri who rejigged internals in S4—and now added new types for periods, intervals and durations. This is what is commonly called a big fucking deal!! So a really REALLY big thank you to my coauthor Leonardo for all these contributions.

With all these Yuge changes patiently chisseled in by Leonardo, it took some time since the last release and a few more things piled up. Matt Dowle corrected something we borked for integration with the lovely and irreplacable data.table. We also switched to the awesome yet minimal tinytest package by Mark van der Loo, and last but not least we added the beginnings of a proper vignette—currently at nine pages but far from complete.

The NEWS snippet adds full details.

Changes in version 0.3.0 (2020-08-06)

Use tzstr= instead of tz= in call to RcppCCTZ::parseDouble()) (Matt Dowle in #49).

Add new comparison operators for nanotime and charcters (Dirk in #54 fixing #52).

Switch from RUnit to tinytest (Dirk in #55)

Substantial functionality extension in with new types nanoduration, nanoival and nanoperiod (Leonardo in #58, #60, #62, #63, #65, #67, #70 fixing #47, #51, #57, #61, #64 with assistance from Dirk).

A new (yet still draft-ish) vignette was added describing the four core types (Leonardo and Dirk in #71).

A required compilation flag for Windows was added (Leonardo in #72).

RcppCCTZ function are called in new 'non-throwing' variants to not trigger exeception errors (Leonardo in #73).

We also have a diff to the previous version thanks to CRANberries. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/nanotime | permanent link

Tue, 04 Aug 2020

RcppCCTZ 0.2.8: Minor API Extension

A new minor release 0.2.8 of RcppCCTZ is now on CRAN.

This version adds three no throw variants of three existing functions, contributed again by Leonardo. This will be used in an upcoming nanotime release which we are finalising now.

Changes in version 0.2.8 (2020-08-04)

Added three new nothrow variants (for win32) needed by the expanded nanotime package (Leonardo in #37)

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 28 Jul 2020

Installing and Running Ubuntu on a 2015-ish MacBook Air

So a few months ago kiddo one dropped an apparently fairly large cup of coffee onto her one and only trusted computer. With a few months (then) to graduation (which by now happened), and with the apparent “genuis bar” verdict of “it’s a goner” a new one was ordered. As it turns out this supposedly dead one coped well enough with the coffee so that after a few weeks of drying it booted again. But give the newer one, its apparent age and whatnot, it was deemed surplus. So I poked around a little on the interwebs and conclude that yes, this could work.

Fast forward a few months and I finally got hold of it, and had some time to play with it. First, a bootable usbstick was prepared, and the machine’s content was really (really, and check again: really) no longer needed, I got hold of it for good.

tl;dr It works just fine. It is a little heavier than I thought (and isn’t “air” supposed to be weightless?) The ergonomics seem quite nice. The keyboard is decent. Screen-resolution on this pre-retina simple Air is so-so at 1440 pixels. But battery live seems ok and e.g. the camera is way better than what I have in my trusted Lenovo X1 or at my desktop. So just as a zoom client it may make a lot of sense; otherwise just walking around with it as a quick portable machine seems perfect (especially as my Lenovo X1 still (ahem) suffers from one broken key I really need to fix…).

Below are some lightly edited notes from the installation. Initial steps were quick: maybe an hour or less? Customizing a machine takes longer than I remembered, this took a few minutes here and there quite a few times, but always incremental.

Initial Steps

Download of Ubuntu 20.04 LTS image: took a few moments, even on broadband, feels slower than normal (fast!) Ubuntu package updates, maybe lesser CDN or bad luck
Startup Disk Creator using a so-far unused 8gb usb drive
Plug into USB, recycle power, press “Option” on macOS keyboard: voila
After a quick hunch… no to ‘live/test only’ and yes to install, whole disk
install easy, very few questions, somehow skips wifi
so activate wifi manually — and everythings pretty much works

Customization

First deal with ‘fn’ and ‘ctrl’ key swap. Install git and followed this github repo which worked just fine. Yay. First (manual) Linux kernel module build needed need in … half a decade? Longer?
Fire up firefox, go to ‘download chrome’, install chrome. Sign in. Turn on syncing. Sign into Pushbullet and Momentum.
syncthing which is excellent. Initially via apt, later from their PPA. Spend some time remembering how to set up the mutual handshakes between devices. Now syncing desktop/server, lenovo x1 laptop, android phone and this new laptop
keepassx via apt and set up using Sync/ folder. Now all (encrypted) passwords synced.
Discovered synergy now longer really free, so after a quick search found and installed barrier (via apt) to have one keyboard/mouse from desktop reach laptop.
Added emacs via apt, so far ‘empty’, so config files yet
Added ssh via apt, need to propagate keys to github and gitlab
Added R via add-apt-repository --yes "ppa:marutter/rrutter4.0" and add-apt-repository --yes "ppa:c2d4u.team/c2d4u4.0+". Added littler and then RStudio
Added wajig (apt frontend) and byobu, both via apt
Created ssh key, shipped it to server and github + gitlab
Cloned (not-public) ‘dotfiles’ repo and linked some dotfiles in
Cloned git repo for nord-theme for gnome terminal and installed it; also added it to RStudio via this repo
Emacs installed, activated dotfiles, then incrementally install a few elpa-* packages and a few M-x package-install including nord-theme, of course
Installed JetBrains Mono font from my own local package; activated for Gnome Terminal and Emacs
Install gnome-tweak-tool via apt, adjusted a few settings
Ran gsettings set org.gnome.desktop.wm.preferences focus-mode 'sloppy'
Set up camera following this useful GH repo
At some point also added slack and zoom, because, well, it is 2020
STILL TODO:
- docker
- bother with email setup?,
- maybe atom/code/…?

/computers/misc | permanent link

ttdo 0.0.6: Bugfix

A bugfix release of our (still small) ttdo package arrived on CRAN overnight. As introduced last fall, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam to give us test results with visual diffs:

ttdo screenshot

This release corrects a minor editing error spotted by the ever-vigilant John Blischak.

The NEWS entry follow.

Changes in ttdo version 0.0.6 (2020-07-27)

Correct a minor editing mistake spotted by John Blischak.

CRANberries provides the usual summary of changes to the previous version. Please use the GitHub repo and its issues for any questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/ttdo | permanent link

Fri, 24 Jul 2020

anytime 0.3.8: Minor Maintenance

A new minor release of the anytime package arrived on CRAN overnight. This is the nineteenth release, and it comes just over six months after the previous release giving further indicating that we appear to have reached a nice level of stability.

anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, … format to either POSIXct or Date objects – and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples.

This release mostly plays games with CRAN. Given the lack of specification for setups on their end, reproducing test failures remains, to put it mildly, “somewhat challenging”. So we eventually gave up—and weaponed up once more and now explicitly test for the one distribution where tests failed (when they clearly passed everywhere else). With that we now have three new logical predicates for various Linux distribution flavours, and if that dreaded one is seen in one test file the test is skipped. And with that we now score twelve out of twelve OKs. This being a game of cat and mouse, I am sure someone somewhere will soon invent a new test…

The full list of changes follows.

Changes in anytime version 0.3.8 (2020-07-23)

A small utility function was added to detect the Linux distribution used in order to fine-tune tests once more.

Travis now uses Ubuntu 'bionic' and R 4.0.*.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. The issue tracker tracker off the GitHub repo can be use for questions and comments.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/anytime | permanent link

Sat, 18 Jul 2020

tint 0.1.3: Fixes for html mode, new demo

A new version 0.1.3 of the tint package arrived at CRAN today. It corrects some features for html output, notably margin notes and references. It also contains a new example for inline references.

The full list of changes is below.

Changes in tint version 0.1.3 (2020-07-18)

A new minimal demo was added showing inline references (Dirk addressing #42).

Code for margin notes and reference in html mode was updated with thanks to tufte (Dirk in #43 and #44 addressing #40).

The README.md was updated with a new 'See Also' section and a new badge.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page.

For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/tint | permanent link

drat 0.1.8: Minor test fix

drat user

A new version of drat arrived on CRAN today. This is a follow-up release to 0.1.7 from a week ago. It contains a quick follow-up by Felix Ernst to correct on of the tests which misbehaved under the old release of R still being tested at CRAN.

drat stands for drat R Archive Template, and helps with easy-to-create and easy-to-use repositories for R packages. Since its inception in early 2015 it has found reasonably widespread adoption among R users because repositories with marked releases is the better way to distribute code.

As your mother told you: Friends don’t let friends install random git commit snapshots. Rolled-up releases it is. drat is easy to use, documented by five vignettes and just works.

The NEWS file summarises the release as follows:

Changes in drat version 0.1.8 (2020-07-18)

The archive pruning test code was corrected for r-oldrel (Felix Ernst in #105 fixing #104).

Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/drat | permanent link

Fri, 17 Jul 2020

RcppArmadillo 0.9.900.2.0

armadillo image

Conrad just released a new minor upstream version 9.900.2 of Armadillo which we packaged and tested as usual first as a ‘release candidate’ build and then as the release. As usual, logs from reverse-depends runs are in the rcpp-logs repo.

All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.2.0 (2020-07-17)

Upgraded to Armadillo release 9.900.2 (Nocturnal Misbehaviour)

In sort(), fixes for inconsistencies between checks applied to matrix and vector expressions

In sort(), remove unnecessary copying when applied in-place to vectors function when applied in-place to vectors

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 11 Jul 2020

drat 0.1.7: New functionality

drat user

A new version of drat arrived on CRAN yesterday. Once again, this release is mostly the work of Felix Ernst who extended some work from the previous release, and added support for repository updates (outside of package insertion) and more.

As your mother told you: Friends don’t let friends install random git commit snapshots. Rolled-up releases it is. drat is easy to use, documented by five vignettes and just works.

The NEWS file summarises the release as follows:

Changes in drat version 0.1.7 (2020-07-10)

Functions insertPackages, archivePackages and prunePackages are now vectorised (Patrick Schratz and Felix Ernst in #93, #100).

The new functionality is supported by unit tests (Felix Ernst in #93, and #102 fixing #101).

Added new function updateRepo (Felix Ernst in #95, #97).

Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/drat | permanent link

Tue, 07 Jul 2020

RcppSimdJson 0.1.0: Now on Windows, With Parsers and Faster Still!

A smashing new RcppSimdJson release 0.1.0 containing several small updates to upstream simdjson (now at 0.4.6) in part triggered by very excisting work by Brendan who added actual parser from file and string—and together with Daniel upstream worked really hard to make Windows builds as well as complete upstream tests on our beloved (ahem) MinGW platform possible. So this version will, once the builders have caught up, give everybody on Windows a binary—with a JSON parser running circles around the (arguably more feature-rich and possibly easier-to-use) alternatives. Dave just tweeted a benchmark snippet by Brendan, the full set is at the bottom our issue ticket for this release.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators, which in its upstream release 0.4.0 improved once more (also see the following point releases). Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

As mentioned, this release expands the reach of the package to Windows, and adds new user-facing functions. A big thanks for most of this is owed to Brendan, so buy him a drink if you run across him. The full NEWS entry follows.

Changes in version 0.1.0 (2020-07-07)

Upgraded to simdjson 0.4.1 which adds upstream Windows support (Dirk in #27 closing #26 and #14, plus extensive work by Brendan helping upstream with mingw tests).

Upgraded to simdjson 0.4.6 with further upstream improvements (Dirk in #30).

Change Travis CI to build matrix over g++ 7, 8, 9, and 10 (Dirk in #31; and also Brendan in #32).

New JSON functions fparse and fload (Brendan in #32) closing #18 and #10).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

AsioHeaders 1.16.1-1 on CRAN

An updated version of the AsioHeaders package arrived on CRAN today (after a we days of “rest” in the incoming directory of CRAN). Asio provides a cross-platform C++ library for network and low-level I/O programming. It is also included in Boost – but requires linking when used as part of Boost. This standalone version of Asio is a header-only C++ library which can be used without linking (just like our BH package with parts of Boost).

This release brings a new upstream version. Its changes required a corresponding change in one of (only) three reverse depends which delayed the CRAN admisstion by a few days.

Changes in version 1.16.1-1 (2020-06-28)

Upgraded to Asio 1.16.1 (Dirk in #5).

Updated README.md with standard set of badges

Via CRANberries, there is a diffstat report relative to the previous release.

Comments and suggestions about AsioHeaders are welcome via the issue tracker at the GitHub GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/asioheaders | permanent link

Mon, 06 Jul 2020

Rcpp 1.0.5: Several Updates

Right on the heels of the news of 2000 CRAN packages using Rcpp (and also hitting 12.5 of CRAN package, or one in eight), we are happy to announce release 1.0.5 of Rcpp. Since the ten-year anniversary and the 1.0.0 release release in November 2018, we have been sticking to a four-month release cycle. The last release has, however, left us with a particularly bad taste due to some rather peculiar interactions with a very small (but ever so vocal) portion of the user base. So going forward, we will change two things. First off, we reiterate that we have already made rolling releases. Each minor snapshot of the main git branch gets a point releases. Between release 1.0.4 and this 1.0.5 release, there were in fact twelve of those. Each and every one of these was made available via the drat repo, and we will continue to do so going forward. Releases to CRAN, however, are real work. If they then end up with as much nonsense as the last release 1.0.4, we think it is appropriate to slow things down some more so we intend to now switch to a six-months cycle. As mentioned, interim releases are always just one install.packages() call with a properly set repos argument away.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 2002 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 203 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steady at around one millions downloads per month.

This release features again a number of different pull requests by different contributors covering the full range of API improvements, attributes enhancements, changes to Sugar and helper functions, extended documentation as well as continuous integration deplayment. See the list below for details.

Changes in Rcpp patch release version 1.0.5 (2020-07-01)

Changes in Rcpp API:

The exception handler code in #1043 was updated to ensure proper include behavior (Kevin in #1047 fixing #1046).

A missing Rcpp_list6 definition was added to support R 3.3.* builds (Davis Vaughan in #1049 fixing #1048).

Missing Rcpp_list{2,3,4,5} definition were added to the Rcpp namespace (Dirk in #1054 fixing #1053).

A further updated corrected the header include and provided a missing else branch (Mattias Ellert in #1055).

Two more assignments are protected with Rcpp::Shield (Dirk in #1059).

One call to abs is now properly namespaced with std:: (Uwe Korn in #1069).

String object memory preservation was corrected/simplified (Kevin in #1082).

Changes in Rcpp Attributes:

Empty strings are not passed to R CMD SHLIB which was seen with R 4.0.0 on Windows (Kevin in #1062 fixing #1061).

The short_file_name() helper function is safer with respect to temporaries (Kevin in #1067 fixing #1066, and #1071 fixing #1070).

Changes in Rcpp Sugar:

Two sample() objects are now standard vectors and not R_alloc created (Dirk in #1075 fixing #1074).

Changes in Rcpp support functions:

Rcpp.package.skeleton() adjusts for a (documented) change in R 4.0.0 (Dirk in #1088 fixing #1087).

Changes in Rcpp Documentation:

The pdf file of the earlier introduction is again typeset with bibliographic information (Dirk).

A new vignette describing how to package C++ libraries has been added (Dirk in #1078 fixing #1077).

Changes in Rcpp Deployment:

Travis CI unit tests now run a matrix over the versions of R also tested at CRAN (rel/dev/oldrel/oldoldrel), and coverage runs in parallel for a net speed-up (Dirk in #1056 and #1057).

The exceptions test is now partially skipped on Solaris as it already is on Windows (Dirk in #1065).

The default CI runner was upgraded to R 4.0.0 (Dirk).

The CI matrix spans R 3.5, 3.6, r-release and r-devel (Dirk).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2455 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 04 Jul 2020

Rcpp now used by 2000 CRAN packages–and one in eight!

2000 Rcpp packages

As of yesterday, Rcpp stands at exactly 2000 reverse-dependencies on CRAN. The graph on the left depicts the growth of Rcpp usage (as measured by Depends, Imports and LinkingTo, but excluding Suggests) over time.

Rcpp was first released in November 2008. It probably cleared 50 packages around three years later in December 2011, 100 packages in January 2013, 200 packages in April 2014, and 300 packages in November 2014. It passed 400 packages in June 2015 (when I tweeted about it), 500 packages in late October 2015, 600 packages in March 2016, 700 packages last July 2016, 800 packages last October 2016, 900 packages early January 2017, 1000 packages in April 2017, 1250 packages in November 2017, 1500 packages in November 2018 and then 1750 packages last August. The chart extends to the very beginning via manually compiled data from CRANberries and checked with crandb. The next part uses manually saved entries. The core (and by far largest) part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of packages using Rcpp is available too.

Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent July of 2015, seven percent just before Christmas 2015, eight percent in the summer of 2016, nine percent mid-December 2016, cracked ten percent in the summer of 2017 and eleven percent in 2018. We now passed 12.5 percent—so one in every eight CRAN packages dependens on Rcpp. Stunning. There is more detail in the chart: how CRAN seems to be pushing back more and removing more aggressively (which my CRANberries tracks but not in as much detail as it could), how the growth of Rcpp seems to be slowing somewhat outright and even more so as a proportion of CRAN – as one would expect a growth curve to.

To mark the occassion, I sent out two tweets yesterday: first a shorter one with “just the numbers”, followed by a second one also containing the few calculation steps. The screenshot from the second one is below.

2000 Rcpp packages

2000 user packages is pretty mind-boggling. We can use the progression of CRAN itself compiled by Henrik in a series of posts and emails to the main development mailing list. Not that long ago CRAN itself did have only 1000 packages, then 5000, 10000, and here we are at just over 16000 with Rcpp at 12.5% and still growing (though maybe more slowly). Amazeballs.

The Rcpp team continues to aim for keeping Rcpp as performant and reliable as it has been. A really big shoutout and Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 03 Jul 2020

#28: Welcome RSPM and test-drive with Bionic and Focal

Welcome to the 28th post in the relatively random R recommendations series, or R⁴ for short. Our last post was a “double entry” in this R⁴ series and the newer T⁴ video series and covered a topic touched upon in this R⁴ series multiple times: easy binary install, especially on Ubuntu.

That post already previewed the newest kid on the block: RStudio’s RSPM, now formally announced. In the post we were only able to show Ubuntu 18.04 aka bionic. With the formal release of RSPM support has been added for Ubuntu 20.04 aka focal—and we are happy to announce that of course we added a corresponding Rocker r-rspm container. So you can now take full advantage of RSPM either via docker pull rocker/r-rspm:18.04 or via docker pull rocker/r-rspm:20.04 covering the two most recent LTS releases.

RSPM is a nice accomplishment. Covering multiple Linux distributions is an excellent achievement. Allowing users to reason in terms of the CRAN packages (i.e. installing xml2, not r-cran-xml2) eases use. Doing it from via the standard R command install.packages() (or wrapper around it like our install.r from littler package) is very good too and an excellent technical achievement.

There is, as best as I can tell, only one shortcoming, along with one small bit of false advertising. The shortcoming is technical. By bringing the package installation into the user application domain, it is separated from the system and lacks integration with system libraries. What do I mean here? If you were to add R to a plain Ubuntu container, say 18.04 or 20.04, then added the few lines to support RSPM and install xml2 it would install. And fail. Why? Because the system library libxml2 does not get installed with the RSPM package—whereas the .deb from the distribution or PPAs does. So to help with some popular packages I added libxml2, libunits and a few more for geospatial work to the rocker/r-rspm containers. Being already present ensures packages xml2 and units can run immediately. Please file issue tickets at the Rocker repo if you come across other missing libraries we could preload. (A related minor nag is incomplete coverage. At least one of my CRAN packages does not (yet?) come as a RSPM binary. Then again, CRAN has 16k packages, and the RSPM coverage is much wider than the PPA one. But completeness would be neat. The final nag is lack of Debian support which seems, well, odd.)

So what about the small bit of false advertising? Well it is claimed that RSPM makes installation “so much faster on Linux”. True, faster than the slowest possible installation from source. Also easier. But we had numerous posts on this blog showing other speed gains: Using ccache. And, of course, using binaries. And as the initial video mentioned above showed, installing from the PPAs is also faster than via RSPM. That is easy to replicate. Just set up the rocker/r-ubuntu:20.04 (or 18.04) container alongside the rocker/r-rspm:20.04 (or also 18.04) container. And then time install.r rstan (or install.r tinyverse) in the RSPM one against apt -y update; apt install -y r-cran-rstan (or ... r-cran-tinyverse). In every case I tried, the installation using binaries from the PPA was still faster by a few seconds. Not that it matters greatly: both are very, very quick compared to source installation (as e.g. shown here in 2017 (!!)) but the standard Ubuntu .deb installation is simply faster than using RSPM. (Likely due to better CDN usage so this may change over time. Neither method appears to do downloads in parallel so there is scope for both for doing better.)

So in sum: Welcome to RSPM, and nice new tool—and feel free to “drive” it using rocker/r-rspm:18.04 or rocker/r-rspm:20.04.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Fri, 26 Jun 2020

littler 0.3.11: docopt updates

max-heap image

The twelveth release of littler as a CRAN package is now available, following in the fourteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later.

A few examples are highlighted at the Github repo, as well as in the examples vignette.

This release mostly responds to the recent docopt release 0.7.0 which brought a breaking change for quoted arguments. In short, it is for the better because an option --as-cran is now available parsed as opt$as_cran which is easier than the earlier form where we needed to back-tick protect as-cran containing an underscore. We also added a new portmanteau-ish option to roxy.r.

The NEWS file entry is below.

Changes in littler version 0.3.11 (2020-06-26)

Changes in examples

Scripts check.r and rcc.r updated to reflect updated docopt 0.7.0 behaviour of quoted arguments

The roxy.r script has a new ease-of-use option -f | --full regrouping two other options.

CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter.

Comments and suggestions are welcome at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/littler | permanent link

Thu, 25 Jun 2020

RcppSimdJson 0.0.6: New Upstream, New Features!

A very exciting RcppSimdJson release with the updated upstream simdjson release 0.4.0 as well as a first set of new JSON parsing functions just hit CRAN. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The very recent 0.4.0 release further improves the already impressive speed.

And this release brings a first set of actually user-facing functions thanks to Brendan which put in a series of PRs! The full NEWS entry follows.

Changes in version 0.0.6 (2020-06-25)

Created C++ integer-handling utilities for safe downcasting and integer return (Brendan in #16 closing #13).

New JSON functions .deserialize_json and .load_json (Brendan in #16, #17, #20, #21).

Upgrade Travis CI to 'bionic', extract package and version from DESCRIPTION (Dirk in #23).

Upgraded to simdjson 0.4.0 (Dirk in #25 closing #24).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 22 Jun 2020

#27: R and CRAN Binaries for Ubuntu

Welcome to the 27th post in the rationally regularized R revelations series, or R⁴ for short. This is a edited / updated version of yesterday’s T^4 post #7 as it really fits the R⁴ series as well as it fits the T⁴ series.

A new video in both our T^4 series of video lightning talks with tips, tricks, tools, and toys is also a video in the R^4 series as it revisits a topic previously covered in the latter: how to (more easily) get (binary) packages onto your Ubuntu system. In fact, we show it in three different ways.

The slides are here.

This repo at GitHub support the series: use it to open issues for comments, criticism, suggestions, or feedback.

Thanks to Iñaki Ucar who followed up on twitter with a Fedora version. We exchanged some more message, and concluded that complete comparison (from an empty Ubuntu or Fedora container) to a full system takes about the same time on either system.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Sun, 21 Jun 2020

T^4 #7 and R^4 #5: R and CRAN Binaries for Ubuntu

The slides are here.

This repo at GitHub support the series: use it to open issues for comments, criticism, suggestions, or feedback.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

RcppGSL 0.3.8: More fixes and polish

Release 0.3.8 of RcppGSL is now getting onto CRAN. The RcppGSL package provides an interface from R to the GNU GSL using the Rcpp package.

Peter Carbonetto let us know in issue #25 that the included example now showed linker errors on (everybody’s favourite CRAN platform) Slowlaris. Kidding aside, the added compiler variety really has benefits because we were indeed missing a good handful or two of inline statements in the headers—which our good friends g++ and clang++ apparently let us get away with. This has been fixed, and a little bit of the usual package polish and cleanup has been added; see the list of detailed changes below.

Changes in version 0.3.8 (2020-06-21)

A few missing inline statements were added to the headers fixing a (genuine) error that was seen only on Solaris (Dirk).

The nice colNorm example is now in a file by itself, the previous versions are off in a new file colNorm_old.cpp (Dirk).

The README.me now sports two new badges (Dirk).

Travis CI was updated to 'bionic' and R 4.0 (Dirk).

Special thanks also to CRAN for a super-smooth and fully automated processing of a package with both compiled code and two handful of reverse dependencies.

Courtesy of CRANberries, a summary of changes to the most recent release is also available.

More information is on the RcppGSL page. Questions, comments etc should go to the issue tickets at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 15 Jun 2020

Rcpp 1.0.5 in two+ weeks: Please help test

With the current four-month release cycle, the next Rcpp release is due in July following the 1.0.4 release in March. Just prior to the 1.0.4 release I had asked this:

It would be particularly beneficial if those with “unsual” build dependencies tested it as we would increase overall coverage beyond what I get from testing against 1800+ CRAN packages. BioConductor would also be welcome.

but only on the rcpp-devel list, and only about a good week prior to the release.

I remain rather disappointed and disillusioned about what happened after 1.0.4 was released. Two PRs in that release were soon seen to have side effects on more ‘marginal’ test systems, precisely what added testing could have revealed. An additional issue arose from changes in R’s make system, which is harder to anticipate or test. Each and every infelicity was fixed within a day or so, and we always make candidate releases available—the current Rcpp as of this writing is 1.0.4.12 meaning twelve microreleases were made since 1.0.4. And those microreleases are always available for normal download and install.packages use via the Rcpp drat repository accessible to all. So it was truly troubling to see some, especially those with experience in setting up or running testing / ci platforms, pretend to be unable to access, install, and provide these for their own tests, or the tests of their users. It just doesn’t pass a basic logic test: it takes a single call to install.packages(), or, even more easily, a single assignment of an auxiliary repo. All told this was a rather sad experience.

So let’s try to not repeat this. If you, or maybe users of a build or ci system you maintain, rely on Rcpp, and especially if you do so on systems outside the standard CRAN grid of three OSs and the triplet of “previous, current, next” releases of R, then please help by testing. I maitain these release as a volunteer, unpaid at that, and I simply cannot expand to more systesm. We take reverse dependency check seriously (and I just run two taking about a day each) but if you insist on building on stranger hardware or much older releases it will be up to you to ensure Rcpp passes. We prep for CRAN, and try our best to pass at CRAN. For nearly a dozen years.

To install the current microrelease from the Rcpp drat repository, just do

install.packages("Rcpp", repos="https://rcppcore.github.io/drat")

That is all there is to it. You could even add the Rcpp drat repository to your repository list.

Rcpp has become successful because so many people help with suggestions, documentation, and code. It is used by (as of today) 1958 CRAN packages, 205 BioConductor packages, and downloaded around a million times per month. So if you can, please help now with some more testing.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 14 Jun 2020

T^4 #6: Byobu Sessions

The next video in our T^4 series of video lightning talks with tips, tricks, tools, and toys (where we had seen the announcement, shells sessions one, two, and three, as well as byoby sessions one and two) is now up at YouTube. It covers session management for the wonderful byobu tool that is both a ‘text-based window manager’ and a ‘terminal multiplexer’:

The slides are here.

This repo at GitHub support the series: use it to open issues for comments, criticism, suggestions, or feedback.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

RVowpalWabbit 0.0.14: More Keeping CRAN happy

Another maintenance RVowpalWabbit package update brings us to version 0.0.14. This time CRAN asked us to replace the (long obsoleted C-library) function ftime(). Along the way, we also updated links in the DESCRIPTION file to the (spiffy!!) new vowpalwabbit.org website, updated Travis use and fine-tuned some autoconf code in configure.ac.

There is a newer package rvw based on the excellent GSoC 2018 and beyond work by Ivan Pavlov (mentored by James and myself) so if you are into Vowpal Wabbit from R go check it out. It should go to CRAN “eventually” once we have better mechanisms to support external libraries.

CRANberries provides a summary of changes to the previous version. More information is on the RVowpalWabbit page. Issues and bugreports should go to the GitHub issue tracker.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rvowpalwabbit | permanent link

Wed, 10 Jun 2020

binb 0.0.6: Small enhancements

The sixth release of the binb package is now on CRAN. binb regroups four rather nice themes for writing LaTeX Beamer presentations much more easily in (R)Markdown. As a teaser, a quick demo combining all four themes follows; documentation and examples are in the package.

Via two contributed PRs, this releases adds titlepage support via the YAML header for Metropolis, and suppresses nags about the changed natbib default. A little polish on the README and Travis rounds everything off.

Changes in binb version 0.0.6 (2020-06-10)

Support for YAML option titlegraphic was added in Metropolis (Andras Scraka in #23).

The README.md file received another badge (Dirk).

The natbib default value was updated to accomodate rmarkdown (Joseph Stachelek in #26).

Travis now uses R 4.0.0 and 'bionic' (Dirk).

CRANberries provides a summary of changes to the previous version. For questions or comments, please use the issue tracker at GitHub.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/binb | permanent link

Tue, 09 Jun 2020

RcppArmadillo 0.9.900.1.0

armadillo image

Conrad recently released a new upstream version 9.900.1 of Armadillo which we packaged and tested as usual first as a ‘release candidate’ build and then as the release. As usual, logs from reverse-depends runs are in the rcpp-logs repo.

Apart from the new upstream release, we updated Travis use, ornamented the README a little, and smoothed over a rough corner from the recent R 4.0.0 release. All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.1.0 (2020-06-08)

Upgraded to Armadillo release 9.900.1 (Nocturnal Misbehaviour)

faster solve() for under/over-determined systems

faster eig_gen() and eig_pair() for large matrices

expanded eig_gen() and eig_pair() to optionally provide left and right eigenvectors

Switch Travis CI testing to R 4.0.0, use bionic as base distro and test R 3.6.3 and 4.0.0 in a matrix (Dirk in #298).

Add two badges to README for indirect use and the CSDA paper.

Adapt RcppArmadillo.package.skeleton() to a change in R 4.0.0 affecting what it exports in NAMESPACE.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Rcpp Webinar Recording Available

As announced in a few tweets leading up to it, I took the date of what would have been the annual R/Finance conference as an opportunity to hold the one-hour tutorial / workshop with introductory Rcpp material which I often present on the first morning preceding the conference as a self-organized webinar. The live-streaming worked actually reasonably well via obs to youtube (even though the comprehensive software by the latter complained at times about insufficient bitstream rates–the joys of living with a (near) monopolistic broadband provider whom I should leave for fiber…). Apparently around seventy people connected to the stream—which is more than we usually have in the seminar room at UIC for the R/Finance morning.

The recording is now available here, and has already been seen over 200 times:

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 07 Jun 2020

T^4 #5: More About Byobu

Another video in our T^4 series of video lightning talks with tips, tricks, tools, and toys (where we had seen the announcement, shells sessions one, two, and three, as well as one byobu session) is now up at YouTube. It goes a little deeper in the wonderful byobu and persistent sessions with multiple concurrent accesses illustrating why it is called both a ‘text-based window manager’ and a ‘terminal multiplexer’:

The slides are here.

This repo at GitHub support the series: use it to open issues for comments, criticism, suggestions, or feedback.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

Fri, 05 Jun 2020

corels 0.0.2 on CRAN: Initial upload!

Corels is now on CRAN! The package was introduced with an initial tweet which pointed to the GitHub repo.

The source code repo has since been relocated from my account to the (upstream) corels org in GitHub. And renamed: as the upstream (C++) repo as well as the existing Python package simply call is corels we now do too. The repo, to be distinguishable as a directory, will remain named rcppcorels.

We also describe the package a little on the corels package page. Some more work should go into along with work in the upstream repos, so please follow whichever GitHub repo you are interested in.

The work on corels is also what is described in the recent arXiv paper on Rcpp and libraries:

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/corels | permanent link

Wed, 03 Jun 2020

littler 0.3.10: Some more updates

max-heap image

The eleventh release of littler as a CRAN package is now available, following in the fourteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later.

A few examples are highlighted at the Github repo, as well as in the examples vignette.

This release adds a new helper / example script installBioc.r for BioConductor package installation, generalizes the roxygenize() wrapper roxy.r a little, and polished a couple of other corners.

The NEWS file entry is below.

Changes in littler version 0.3.10 (2020-06-02)

Changes in examples

The update.r script only considers writeable directories.

The rcc.r script tries to report full logs by setting _R_CHECK_TESTS_NLINES_=0.

The tt.r script has an improved ncpu fallback.

Several installation and updating scripts set _R_SHLIB_STRIP_ to TRUE.

A new script installBioc.r was added.

The --error option to install2.r was generalized (Sergio Oller in #78).

The roxy.r script was extended a little.

Changes in package

Travis CI now uses R 4.0.0 and the bionic distro

Comments and suggestions are welcome at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/littler | permanent link

Sun, 31 May 2020

T^4 #4: Introducing Byobu

The next video (following the announcement, and shells sessions one, two, and three) is up in the T^4 series of video lightning talks with tips, tricks, tools, and toys. This time we introduce the wonderful byobu tool which is called both a ‘text-based window manager’ and a ‘terminal multiplexer’:

The slides are here.

This repo at GitHub support the series: use it to open issues for comments, criticism, suggestions, or feedback.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

Sat, 30 May 2020

drat 0.1.6: Rewritten macOS binary support

drat user

A new version of drat arrived on CRAN overnight, once again taking advantage of the fully automated process available for such packages with few reverse depends and no open issues. As we remarked at the last release fourteen months ago when we scored the same nice outcome: Being a simple package can have its upsides…

This release is mostly the work of Felix Ernst who took on what became a rewrite of how binary macOS packages are handled. If you need to distribute binary packages for macOS users, this may help. Two more small updates were made, see below for full details.

As your mother told you: Friends don’t let friends install random git commit snapshots. Rolled-up releases it is. drat is easy to use, documented by five vignettes and just works.

The NEWS file summarises the release as follows:

Changes in drat version 0.1.6 (2020-05-29)

Changes in drat functionality

Support for the various (current) macOS binary formats was rewritten (Felix Ernst in #89 fixing #88).

Travis CI use was updated to R 4.0.0 and bionic (Dirk).

A drat repo was added to the README (Thomas Fuller in #86)

Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/drat | permanent link

Sun, 24 May 2020

#3 T^4: Customizing The Shell

The third video (following the announcement, the shell colors) one as well as last week’s shell prompt one, is up in the stil new T^4 series of video lightning talks with tips, tricks, tools, and toys. Today we cover customizing the shell some more.

The slides are here.

This repo at GitHub support the series: use it to open issues for comments, criticism, suggestions, or feedback.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

Sat, 23 May 2020

RcppSimdJson 0.0.5: Updated Upstream

A new RcppSimdJson release with updated upstream simdjson code just arrived on CRAN. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

This release brings updated upstream code (thanks to Brendan Knapp) plus a new example and minimal tweaks. The full NEWS entry follows.

Changes in version 0.0.5 (2020-05-23)

Add parseExample from earlier upstream announcement (Dirk).

Synced with upstream (Brendan in #12) closing #11).

Updated example parseExample to API changes (Brendan).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 17 May 2020

#2 T^4: Customizing The Shell Prompt

The second video (following the announcement and last week’s shell colors) is up in the stil new T^4 series of video lightning talks with tips, tricks, tools, and toys. Today we cover customizing shell prompts.

The slides are available here. Next week we likely continue on shell customization with aliases.

This repo at GitHub support the series: use it to open issues for comments, criticism, suggestions, or feedback.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

RcppArmadillo 0.9.880.1.0

armadillo image

Conrad released a new upstream version 9.880.1 of Armadillo on Friday which I packaged and tested as usual (result log here in the usual repo). The R package also sports a new OpenMP detection facility once again motivated by macOS which changed its setup yet again.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.880.1.0 (2020-05-15)

Upgraded to Armadillo release 9.880.1 (Roasted Mocha Detox)

expanded qr() to optionally use pivoted decomposition

updated physical constants to NIST 2018 CODATA values

added ARMA_DONT_USE_CXX11_MUTEX confguration option to disable use of std::mutex

OpenMP capability is tested explicitly (Kevin Ushey and Dirk in #294, #295, and #296 all fixing #290).

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 15 May 2020

Let’s celebrate Anna!

Today is graduation at Washington University, and just like many other places, the ceremonies are a lot more virtual and surreal that in other years. For Anna today marks the graduation from Biomedical Engineering with a BSc. The McKelvey School of Engineering put a Zoom meeting together yesterday which was nice, and there is something more virtual here. Hopefully a real-life commencenment can take place in a year—the May 30, 2021, date has been set. The university also sent out a little commencement site/video which was cute. But at end of the day online-only still falls short of the real deal as we all know too well by now.

During those years, just about the only thing really I ever tweeted about appears to be soccer related. As it should because ball is life, as we all know. Here is one from 1 1/2 years ago when her Club Team three-peated in their NIRSA division:

And that opens what may be the best venue for mocking Anna: this year, which her a senior and co-captain, the team actually managed to loose a league game (a shocking first in these years) and to drop the final. I presume they anticipated that all we would all talk about around now is The Last Dance and three-peats, and left it at that. Probably wise.

Now just this week, and hence days before graduating with her B.Sc., also marks the first time Anna was addressed as Dr Eddelbuettel. A little prematurely I may say, but not too shabby to be in print already!

But on the topic of gratulations and what comes next, this tweet was very sweet:

As was this, which marked another impressive score:

So big thanks from all of us to WashU for being such a superb environment for Anna for those four years, and especially everybody at the Pappu Lab for giving Anna a home and base to start a research career.

And deepest and most sincere congratulations to Anna before the next adventure starts….

/misc | permanent link

Sun, 10 May 2020

#1 T^4: Adding Some Color to the Shell

The first proper video (following last week’s announcement) is up for new T^4 series of video lightning talks with tips, tricks, tools, and toys. Today we just to a little enhancement for the shell enabled color output (if not already on by default).

The slides are available here.

Next week we continue on shell customization by looking at the prompt.

Also of note, a new repo at GitHub to support the series: use it to open issues for comments, criticism, suggestions, or feedback.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

Sat, 09 May 2020

ttdo 0.0.5: Reflect tinytest update

A maintenance release of our (still small) ttdo package just arrived on CRAN. As introduced last fall, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam to give us test results with visual diffs:

ttdo screenshot

tinytest has an extension mechanism we use, and as tinytest was just upgraded to version 1.2.0 changing, among other nice extensions, one interface by allowing for a new error class argument, we had to rebuild as well in order to document the new argument.

The release was actually prepared three days ago when tinytest itself was updated, but we waited for the binaries at CRAN to be updated and rebuilt to take advantage of the fully automated submission and test process at CRAN.

The NEWS entry follow.

Changes in ttdo version 0.0.5 (2020-05-06)

Rebuilt under tinytest 1.2.0 to add support for class argument in error-code test predicates

CRANberries provides the usual summary of changes to the previous version. Please use the GitHub repo and its issues for any questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/ttdo | permanent link

Thu, 07 May 2020

Rcpp Virtual Talk on June 5

We had to cancel R/Finance 2020 due to what is happening all around us. But I plan to present the one-hour workshop I often give in the tutorial session preceding the first day—but this time online!

To keep it simple, we will stick with the same day, and possibly the same time: Friday morning at 8:00am! So that makes Friday, June 5, at 08:00h Central time.

This YouTube! link should then provide the stream, I reckon there may also be a recording afterwards.

The talk / demo / presentation will be about an hour long, and material should be similar to the previous ones (of the same length) still available at the talks page (which also has longer talks all the way to the two-day workshops).

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/misc | permanent link

Sun, 03 May 2020

#0: Introducing T^4: Tips, Tricks, Tools, and Toys

For way too long now something I had meant to start was a little series about tips, tricks, tools, and toys. I had mentioned the idea a few times to a friend or two, and generally received a thumbs up or a ‘go for it’. But it takes a little to get over the humb and get going. And it turns out that last week’s r^4 talk on upgrading to R 4.0.0 hit some latent demand as we are now at 1400 views on YouTube. Wowser.

So hence without further ado, let’s kick off T^4. Similar in spirit to R^4, but broader in scope and going beyond R. The opening slides explaining what we plan to do are here, and the video link follows below:

With some luck we should have the first actual talk next week. See you then!

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/t4 | permanent link

Mon, 27 Apr 2020

RcppArmadillo 0.9.870.2.0

armadillo image

A new upstream release 9.870.2 of Armadillo was released a few days ago. We had tested two release candidates, and this caught one bug. The release was held up at CRAN for a few days as one package had an overly sensitive test depending on random input data; eventually we all convinced ourselves that there was no (Rcpp)Armadillo issue here. So morale: friends don’t let friends have tests depend on random behavior.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.870.2.0 (2020-04-24)

Upgraded to Armadillo release 9.870.2 (Roasted Mocha Retox)

faster handling of matrix multiplication expressions by diagvec() and diagmat()

added trimatu_ind() and trimatl_ind()

more consistent detection of sparse vector expressions

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 26 Apr 2020

#26: Upgrading to R 4.0.0

Welcome to the 26th post in the rationally regularized R revelations series, or R⁴ for short.

R 4.0.0 was released two days ago, and a casual glance at some social media conversations appears to suggest quite some confusion, almost certainly some misunderstandings, and possibly also a fair amount of fear, uncertainty, and doubt about the process. So I thought I could show how I upgrade my own main workstation, live and in colour without a safety net. (Almost: I did upgrade my laptop yesterday which went swimmingly, if more slowly.) So here is a fresh video about upgrading to R 4.0.0, with some support slides as usual:

The slides used in the video are at this link.

A few quick follow-ups to the ‘live’ nature of this. The pbdZMQ package did in fact install smoothly once the (Ubuntu) -dev packages for Zero MQ were (re-)installed; then IRkernel also followed. BioConductor completed once I realized that GOSemSim needed the annotation package GO.db to be updated, that allowed MNF to install. So the only bug, really, was the circular depdency between pkgload and testthat. Overall, not bad at all for a quick afternoon session!

And as mentioned, if you are interested and have questions concerning use of R on a .deb based system like Debain or Ubuntu (or Mint or …), the r-sig-debian list is a very good and friendly place to ask them.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Sun, 19 Apr 2020

prrd 0.0.3: More improvements

Back in early 2018, the prrd package was introduced as release 0.0.1, uploaded to CRAN, and updated once as release 0.0.2. I have used it extensively for every CRAN release of Rcpp, RcppArmadillo, RcppEigen, BH, and possibly others.

The idea of prrd is simple, and described in some more detail on its webpage and its GitHub repo. Reverse dependency checks are an important part of package development and is easily done in a (serial) loop. But these checks are also generally embarassingly parallel as there is no or little interdependency between them (besides maybe shared build depedencies). See the following (dated) screenshot (running six parallel workers, arranged in split byobu session).

Since the two early releases, I made a few changes to e.g. the analysis of package checks and their summaries (but never pushed these) as well as a few usability enhancements. This release regroups these.

The release is summarised in the NEWS entry:

Changes in prrd version 0.0.3 (2020-04-19)

Extended functionality summarise() with optional use of package foghorn (Dirk)

New scripts enqueueDepends, dequeueDepends and summmariseJobs (Dirk)

Added tweaks and enhancements throughout (Dirk)

The default queue location can now be set via options (Dirk)

Meta-data on package, version and date is now stored, and shown on dequeue startup (Dirk)

The README.md was clarified and improved (Dirk, Avraham Adler in #8, Robert Nuske in #10)

CRANberries provides the usual summary of changes to the previous version. See the aforementioned webpage and its repo for details. For more questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/prrd | permanent link

Tue, 14 Apr 2020

RcppArmadillo 0.9.860.2.0

armadillo image

A new upstream release 9.860.2 of Armadillo was just released. The theme of “convergence” continues; the previous release increased the minor from 800 to 850, now we are at 860. We first wrapped this up as version 0.9.859.1.0, but it turned out to have been held back by a buglet between R 4.0.0 and Rcpp which the recent patch release fixed (along with other woes on old R or non-CRAN-alike macOS). It then turns out that the new (upstream) version 9.860.1 had a minor bug which I missed as I reverse-depends checked the prior version. Doh. My thanks, as always, to CRAN for spotting this. The fix was added upstream and we have 9.860.2 as RcppArmadillo 0.9.860.2.0.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.860.2.0 (2020-04-13)

Upgraded to Armadillo release 9.860.2 (Roasted Mocha Fix)

Added powmat()

Faster access to columns in sparse submatrix views

Faster handling of relational expressions by accu()

Faster handling of sympd matrices by expmat(), logmat(), sqrtmat()

Workaround for save/load issues with HDF5 v1.12

Vignettes are now pre-made and include (#285)

Two test files are now skipped on 32-bit Windows

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

gettz 0.0.4

A minor routine update 0.0.4 of gettz arrived on CRAN overnight.

gettz provides a possible fallback in situations where Sys.timezone() fails to determine the system timezone. That happened when e.g. the file /etc/localtime somehow is not a link into the corresponding file with zoneinfo data in, say, /usr/share/zoneinfo. Since the package was written (in the fall of 2016), R added a similar extended heuristic approach itself.

This release adds registration of the compiled routine via R_registerRoutines() and R_useDynamicSymbols(), adds .registration=TRUE to useDynLib() in NAMESPACE, and uses an unquoted symbol in .Call(). Two new badges were added to the README.md as well. And as in the previous release in 2016: No new code, or new features.

Courtesy of CRANberries, there is a comparison to the previous release.

More information is on the gettz page. For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/gettz | permanent link

Sun, 12 Apr 2020

#25: Test, test, test, … those R 4.0.0 binaries with Ubuntu 20.04 and Rocker

Welcome to the 25nd post in the randomly recurring R recitations series, or R⁴ for short.

Just yesterday, we posted a short post along with a video and supporting slides. It covered how to test the soon-to-be-released R 4.0.0 on a custom Ubuntu 18.04 Rocker container.

A container for Ubuntu 20.04, which is itself in final beta stages, was being built while the video was made. As it is available now, we created a quick follow-up video showing the use under Ubuntu 20.04:

The updated supporting slides from the video are still at this link.

What we showed in both videos does of course also work directly on Ubuntu (or Debian, using those source repos) installations; the commands shown in the Rocker use case generally apply equally to a normal installation.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Sat, 11 Apr 2020

#24: Test, test, test, … those R 4.0.0 binaries with Ubuntu and Rocker

Welcome to the 24nd post in the relentlessly regular R ravings series, or R⁴ for short.

R 4.0.0 will be released in less than two weeks, and testing is very important. I had uploaded two alpha release builds (at the end of March and a good week ago) as well as a first beta release yesterday, all to the Debian ‘experimental’ distribution (as you can see here) tracking the release schedule set by Peter Dalgaard. Because R 4.0.0 will require reinstallation of all packages, it makes some sense to use a spare machine. Or a Docker container. So to support that latter mode, I have now complemented the binaries created from the r-base source package with all base and recommended packages, providing a starting point for actually running simple tests. Which is what we do in the video, using again the ‘R on Ubuntu (18.04)’ Rocker container:

Slides from the video are at this link.

This container based on 18.04 is described here on the Docker Hub; a new 20.04 container with the pre-release of the next Ubuntu LTS should be there shortly once it leaves the build queue.

What we showed does of course also work on direct Ubuntu (or Debian, using those source repos) installations; the commands shown in the Rocker use case generally apply equally to a normal installation.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/r4 | permanent link

Thu, 09 Apr 2020

Rcpp 1.0.4.6: Bug fix interim version

Rcpp 1.0.4 was released on March 17, following the usual sequence of fairly involved reverse-depends check along with a call for community testing issued weeks before the release. In that email I specifically pleaded with folks to pretty-please test non-standard setups:

It would be particularly beneficial if those with “unsual” build dependencies tested it as we would increase overall coverage beyond what I get from testing against 1800+ CRAN packages. BioConductor would also be welcome.

Alas, you can’t always get what you want. Shortly after the release we were made aware that the two (large) pull request at the book ends of the 1.0.3 to 1.0.4 release period created trouble. Of these two, the earliest PR in the 1.0.4 release upset older-than-CRAN-tested installation, i.e. R 3.3.0 or before. (Why you’d want to run R 3.3.* when R 3.6.3 is current is something I will never understand, but so be it.) This got addressed in two new PRs. And the matching last PR had a bit of sloppyness leaving just about everyone alone, but not all those macbook-wearing data scientists when using newer macOS SDKs not used by CRAN. In other words, “unsual” setups. But boy, do those folks have an ability to complain. Again, two quick PRs later that was addressed. Along came a minor PR with two more Rcpp::Shield<> uses (as life is too short to manually count PROTECT and UNPROTECT). And then a real issue between R 4.0.0 and Rcpp first noticed with RcppParallel builds on Windows but then also affecting RcppArmadillo. Another quickly issued fix. So by now the count is up to six, and we arrived at Rcpp 1.0.4.6.

Which is now on CRAN, after having sat there for nearly a full week, and of course with no reason given. Because the powers that be move in mysterious ways. And don’t answer to earthlings like us.

As may transpire here, I am little tired from all this. I think we can do better, and I think we damn well should, or I may as well throw in the towel and just release to the drat repo where each of the six interim versions was available for all to take as soon as it materialized.

Anyway, here is the state of things. Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1897 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 191 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steasy at one millions downloads per month.

The changes for this interim version are summarized below.

Changes in Rcpp patch release version 1.0.4.6 (2020-04-02)

Changes in Rcpp API:

The exception handler code in #1043 was updated to ensure proper include behavior (Kevin in #1047 fixing #1046).

A missing Rcpp_list6 definition was added to support R 3.3.* builds (Davis Vaughan in #1049 fixing #1048).

Missing Rcpp_list{2,3,4,5} definition were added to the Rcpp namespace (Dirk in #1054 fixing #1053).

A further updated corrected the header include and provided a missing else branch (Mattias Ellert in #1055).

Two more assignments are protect with Rcpp::Shield (Dirk in #1059)

Changes in Rcpp Attributes:

Empty strings are not passed to R CMD SHLIB which was seen with R 4.0.0 on Windows (Kevin in #1062 fixing #1061).

Changes in Rcpp Deployment:

Travis CI unit tests now run a matrix over the versions of R also tested at CRAN (rel/dev/oldrel/oldoldrel), and coverage runs in parallel for a net speed-up (Dirk in #1056 and #1057).

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 03 Apr 2020

RcppSimdJson 0.0.4: Even Faster Upstream!

A new (upstream) simdjson release was announced by Daniel Lemire earlier this week, and my Twitter mentions have been running red-hot ever since as he was kind enough to tag me. Do look at that blog post, there is some impressive work in there. We wrapped up the (still very simple) rcppsimdjson around it last night and shipped it this morning.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. For illustration, I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release brings upstream 0.3 (and 0.3.1) plus a minor tweak (also shipped back upstream). Our full NEWS entry follows.

Changes in version 0.0.4 (2020-04-03)

Upgraded to new upstream releases 0.3 and 0.3.1 (Dirk in #9 closing #8)

Updated example validateJSON to API changes.

But because Daniel is such a fantastic upstream developer to collaborate with, he even filed a full feature-request ‘maybe you can consider upgrading’ as issue #8 at our repo containing the fully detailed list of changes. As it is so impressive I will simple quote the upper half of just the major changes:

Highlights

Multi-Document Parsing: Read a bundle of JSON documents (ndjson) 2-4x faster than doing it individually. API docs / Design Details

Simplified API: The API has been completely revamped for ease of use, including a new JSON navigation API and fluent support for error code and exception styles of error handling with a single API. Docs

Exact Float Parsing: Now simdjson parses floats flawlessly without any performance loss (https://github.com/simdjson/simdjson/pull/558). Blog Post

Even Faster: The fastest parser got faster! With a shiny new UTF-8 validator and meticulously refactored SIMD core, simdjson 0.3 is 15% faster than before, running at 2.5 GB/s (where 0.2 ran at 2.2 GB/s).

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 02 Apr 2020

RQuantLib 0.4.12: Small QuantLib 1.18 update

A new release 0.4.12 of RQuantLib arrived on CRAN today, and was uploaded to Debian as well.

QuantLib is a very comprehensice free/open-source library for quantitative finance; RQuantLib connects it to the R environment and language.

This version does relatively little. When QuantLib 1.18 came out, I immediately did my usual bit of packaging it for Debian as well creating binaries via my Ubuntu PPA so that I could test the package against it. And a few call from RQuantLib are now hitting interface functions marked as ‘deprecated’ leading to compiler nags. So I fixed that in PR #146. And today CRAN sent me email to please fix in the released version—so I rolled this up as 0.4.12. Not other changes.

Changes in RQuantLib version 0.4.12 (2020-04-01)

Changes in RQuantLib code:

Calls deprecated-in-QuantLib 1.18 were updated (Dirk in #146).

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the new rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rquantlib | permanent link

Sat, 28 Mar 2020

RProtoBuf 0.4.17: Robustified

A new release 0.4.17 of RProtoBuf is now on CRAN. RProtoBuf provides R with bindings for the Google Protocol Buffers (“ProtoBuf”) data encoding and serialization library used and released by Google, and deployed very widely in numerous projects as a language and operating-system agnostic protocol.

This release contains small polishes related to the release 0.4.16 which added JSON support for messages, and switched to ByteSizeLong. This release now makes sure JSON functionality is only tested where available (on version 3 of the Protocol Buffers library), and that ByteSizeLong is only called where available (version 3.6.0 or later). Of course, older versions build as before and remain fully supported.

Changes in RProtoBuf version 0.4.17 (2020-03-xx)

Condition use of ByteSizeLong() on building with ProtoBuf 3.6.0 or later (Dirk in #71 fixing #70).

The JSON unit tests are skipped if ProtoBuf 2.* is used (Dirk, also #71).

The configure script now extracts the version from the DESCRIPTION file ( (Dirk, also #71).

CRANberries provides the usual diff to the previous release. The RProtoBuf page has copies of the (older) package vignette, the ‘quick’ overview vignette, and the pre-print of our JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rprotobuf | permanent link

Tue, 24 Mar 2020

New package RcppDate 0.0.1 now on CRAN!

A new small package with a new C++ header library is now on CRAN. It brings the date library by Howard Hinnant to R. This library has been in pretty wide-spread use for a while now, and adds to C++11/C++14/C++17 what will be (with minor modifications) the ‘date’ library in C++20. I had been aware of it for a while, but not needed thanks to CCTZ library out of Google and our RcppCCTZ package. And like CCTZ, it builds upon std::chron adding a whole lot of functionality and useability enhancement. But a some upcoming (and quite exciting!) changes in nanotime required it, I had a reason to set about packaging it as RcppDate. And after a few days of gestation and review it is now available via CRAN.

Two simple example files are included and can be accessed by Rcpp::sourceCpp(). Some brief excerpts follow.

The first example shows three date constructors. Note how the month (and the leading digits) are literals. No quotes for strings anywhere. And no format (just like our anytime package for R).

  constexpr auto x1 = 2015_y/March/22;
  constexpr auto x2 = March/22/2015;
  constexpr auto x3 = 22_d/March/2015;

Note that these are constexpr that resolve at compile-time, and that the resulting year_month_day type is inferred via auto.

A second example constructs the last day of the months similarly:

  constexpr auto x1 = 2015_y/February/last;
  constexpr auto x2 = February/last/2015;
  constexpr auto x3 = last/February/2015;

For more, see the copious date.h documentation.

The (very bland first) NEWS entry (from a since-added NEWS file) for the initial upload follows.

Changes in version 0.0.1 (2020-01-17)

Initial CRAN upload of first version

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 19 Mar 2020

RProtoBuf 0.4.16: Now with JSON

A new release 0.4.16 of RProtoBuf is now on CRAN. RProtoBuf provides R with bindings for the Google Protocol Buffers (“ProtoBuf”) data encoding and serialization library used and released by Google, and deployed very widely in numerous projects as a language and operating-system agnostic protocol.

This release contains a PR contributed by Siddhartha Bagaria which adds JSON support for messages, which had been an open wishlist item. I also appeased a clang deprecation warning that had come up on one of the CRAN test machines.

Changes in RProtoBuf version 0.4.16 (2020-03-19)

Added support for parsing and printing JSON (Siddhartha Bagaria in #68 closing wishlist #61).

Switched ByteSize() to ByteSizeLong() to appease clang (Dirk).

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rprotobuf | permanent link

Wed, 18 Mar 2020

RcppCCTZ 0.2.7

A new release 0.2.7 of RcppCCTZ is now at CRAN.

This version adds internal extensions, contributed by Leonardo, which support upcoming changes to the nanotime package we are working on.

Changes in version 0.2.7 (2020-03-18)

Added functions _RcppCCTZ_convertToCivilSecond that converts a time point to the number of seconds since epoch, and _RcppCCTZ_convertToTimePoint that converts a number of seconds since epoch into a time point; these functions are only callable from C level (Leonardo in #34 and #35).

Added function _RcppCCTZ_getOffset that returns the offset at a speficied time-point for a specified timezone; this function is only callable from C level (Leonardo in #32).

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 17 Mar 2020

Rcpp 1.0.4: Lots of goodies

The fourth maintenance release 1.0.4 of Rcpp, following up on the 10th anniversary and the 1.0.0. release sixteen months ago, arrived on CRAN this morning. This follows a few days of gestation at CRAN. To help during the wait we provided this release via drat last Friday. And it followed a pre-release via drat a week earlier. But now that the release is official, Windows and macOS binaries will be built by CRAN over the next few days. The corresponding Debian package will be uploaded as a source package shortly after which binaries can be built.

As with the previous releases Rcpp 1.0.1, Rcpp 1.0.2 and Rcpp 1.0.3, we have the predictable and expected four month gap between releases which seems appropriate given both the changes still being made (see below) and the relative stability of Rcpp. It still takes work to release this as we run multiple extensive sets of reverse dependency checks so maybe one day we will switch to six month cycle. For now, four months still seem like a good pace.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1873 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 191 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steasy at one millions downloads per month.

This release features quite a number of different pull requests by seven different contributors as detailed below. One (personal) highlight is the switch to tinytest.

Changes in Rcpp version 1.0.4 (2020-03-13)

Changes in Rcpp API:

Safer Rcpp_list*, Rcpp_lang* and Function.operator() (Romain in #1014, #1015).

A number of #nocov markers were added (Dirk in #1036, #1042 and #1044).

Finalizer calls clear external pointer first (Kirill Müller and Dirk in #1038).

Scalar operations with a rhs matrix no longer change the matrix value (Qiang in #1040 fixing (again) #365).

Rcpp::exception and Rcpp::stop are now more thread-safe (Joshua Pritikin in #1043).

Changes in Rcpp Attributes:

The cppFunction helper now deals correctly with mulitple depends arguments (TJ McKinley in #1016 fixing #1017).

Invisible return objects are now supported via new option (Kun Ren in #1025 fixing #1024).

Unavailable packages referred to in LinkingTo are now reported (Dirk in #1027 fixing #1026).

The sourceCpp function can now create a debug DLL on Windows (Dirk in #1037 fixing #1035).

Changes in Rcpp Documentation:

The .github/ directory now has more explicit guidance on contributing, issues, and pull requests (Dirk).

The Rcpp Attributes vignette describe the new invisible return object option (Kun Ren in #1025).

Vignettes are now included as pre-made pdf files (Dirk in #1029)

The Rcpp FAQ has a new entry on the recommended importFrom directive (Dirk in #1031 fixing #1030).

The bib file for the vignette was once again updated to current package versions (Dirk).

Changes in Rcpp Deployment:

Added unit test to check if C++ version remains remains aligned with the package number (Dirk in #1022 fixing #1021).

The unit test system was switched to tinytest (Dirk in #1028, #1032, #1033).

Please note that the change to execptions and Rcpp::stop() in pr #1043 has been seen to have a minor side effect on macOS issue #1046 which has already been fixed by Kevin in pr #1047 for which I may prepare a 1.0.4.1 release for the Rcpp drat repo in a day or two.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 14 Mar 2020

RcppAPT 0.0.6

A new version of RcppAPT – our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, … commands and their cache powering Debian, Ubuntu and the like – is now on CRAN.

RcppAPT allows you to query the (Debian or Ubuntu) package dependency graph at will, with build-dependencies (if you have deb-src entries), reverse dependencies, and all other goodies. See the vignette and examples for illustrations.

This new version corrects builds failures under the new and shiny Apt 2.0 release (and the pre-releases like the 1.9.* series in Ubuntu) as some header files moved around. My thanks to Kurt Hornik for the heads-up. I accomodated the change in the (very simple and shell-based) configure script by a) asking pkg-config about the version of pkg-apt and then using that to b) compare to a ‘threshold value’ of ‘1.9.0’ and c) setting another compiler #define if needed so that d) these headers could get included if defined. The neat part is that a) and b) are done in an R one-liner, and the whole script is still in shell. Now, CRAN being CRAN, I now split the script into two: one almost empty one not using bash that passes the ‘omg but bash is not portable’ test, and which calls a second bash script doing the work. Fun and games…

The full set of changes follows.

Changes in version 0.0.6 (2020-03-14)

Accomodate Apt 2.0 code changes by including more header files

Change is backwards compatible and conditional

Added configure call using pkg-config and package version comparison (using R) to determine if the define is needed

Softened unit tests as we cannot assume optional source deb information to be present, so demo code runs but zero results tolerated

Courtesy of CRANberries, there is also a diffstat report for this release.

A bit more information about the package is available here as well as as the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 12 Mar 2020

RInside 0.2.16

A new release 0.2.16 of RInside arrived on CRAN and in Debian today. This comes almost exactly one year after the previous release 0.2.15. RInside provides a set of convenience classes which facilitate embedding of R inside of C++ applications and programs, using the classes and functions provided by Rcpp.

This release brings one new feature, contributed by Lance Bachmeier (with some additional post-processing by me). It adds the ability to embed and call R from C programs and applications. The interface is more limited as we do not get Rcpp for automagic conversion. But this offers the door to a number of applications supporting plain C interface, and the new examples directory for example shows one for ruby. We may add others.

The short list of changes since the last release were:

Changes in RInside version 0.2.16 (2020-03-12)

RInside is now embeddable (with a reduced interface) from C applications thanks to Lance Bachmeier (who is now co-author) plus some polish by Dirk in #43)

Added R_SESSION_INIIALIZED to list of excluded variables.

Added simple diagnostics function to have a registered function.

CRANberries also provides a short report with changes from the previous release. More information is on the RInside page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page, or to issues tickets at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rinside | permanent link

Wed, 11 Mar 2020

AsioHeaders 1.12.2-1

An updated minor version of the AsioHeaders package arrived on CRAN today. Asio provides a cross-platform C++ library for network and low-level I/O programming. It is also included in Boost – but requires linking when used as part of Boost. This standalone version of Asio is a header-only C++ library which can be used without linking (just like our BH package with parts of Boost).

This release corresponds to a minor upstream update, and is only the second update ever. It may help overcome one santizer warning which David Hall brought to my attention. We tested this version against all reverse depends (which was easy enough as there are only three).The NEWS entry follows.

Changes in version 1.12.2-1 (2020-03-11)

Upgraded to Asio 1.12.2 (Dirk in #4 fixing #3)

Via CRANberries, there is a diffstat report relative to the previous release.

Comments and suggestions about AsioHeaders are welcome via the issue tracker at the GitHub GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/asioheaders | permanent link

Sun, 08 Mar 2020

RcppAnnoy 0.0.16

annoy image

A new release 0.0.16 of RcppAnnoy is available via the Github-hosted R Repository (ghrr). Edit a good two hours later: And wonder of wonders, now also on CRAN.

It remains in limbo at CRAN for no apparent reason. No change appears to be imminent either as the CRAN maintainers continue to play a passive-aggressive game of no communication for any reason. Which is a genuine shame as e [E]verbody involved in the package, i.e. Erik (upstream) and myself but also Aaron (downstream) worked pretty hard and well last weekend (while I was traveling / attending the wonderful celebRtion 2020 conference for the 20th anniversary of the R 1.0.0) to iron out all remaining issues. Installation is pretty flawless and silent as all compiler warnings have been takeb care of even under -pedantic on a recent version, and the last remaining UBSAN issue is also fixed.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

As I wrote in the announcement for 0.0.15 which fixed many-but-not-all issues:

The other important issue is that there will be another 0.0.16 release real soon to incorporate three more small upstream PRs driven by these discussions which kept going on post-release while I was conferencing (or traveling), and which should fix things for good, or so we hope. This should go out probably by the end of the week to not exceed a weekly upload cadence; if you want to see more, or get earlier access, see the git repo which is in fine shape. If you want to see a pre-release on the ghrr drat drop me a line.

As CRAN is holding the package hostage, all I can do now is to release to the Github-hosted R Repository (ghrr) from where you can install it via a simple install.packages("RcppAnnoy", repos="https://ghrr.github.io/drat") (or any of the other drat supported commands, see the ghrr page for more). Or wait and wait and wait … until CRAN graces us with a manual admission (given that the previous upload left one small UBSAN issue to fix). One day. Hopefully. The package is now on CRAN.

Detailed changes follow below.

Changes in version 0.0.16 (2020-03-06)

Use int in two interfaces (Dirk in #59 for upstream PR 460 and closing #56).

Use inline for two helper functions (Dirk in #59 for upstream PR 461 and closing #57; also Aaron in #58 after earlier discussion).

Removed a noisy pragma (Dirk in #60 for upstream PR 462).

Add a simple helper function displaying compiler status.

Courtesy of CRANberries, there ~~will also be diffstat report if and when the package ever makes it to CRAN.~~ is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 04 Mar 2020

RcppSimdJson 0.0.3: Second Update!

Following up on both the initial RcppSimdJson release and the first update, the second update release 0.0.3 arrived on CRAN yesterday.

RcppSimdJson wraps the fantastic simdjson library by Daniel Lemire which is truly impressive. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. For illustration, I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release once again syncs the simdjson headers with upstream, and strengthens the build setup a little bit more. We only turn C++17 (which is needed) on when R knows it (from its builds), report the architecture status at package load (in a suppressable message), and only attempt to parse in examples and unit tests when know that we are on a sufficient platform. The full NEWS entry follows.

Changes in version 0.0.3 (2020-03-03)

Sychronized once more with upstream.

Created new C++ function to check for unsupported architecture, and report the status on package load.

Only run example and unit tests if supported architecture is found.

Created small configure script to see if R was built with C++17 support, and record it in src/Makevars.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 02 Mar 2020

RcppAnnoy 0.0.15

annoy image

A few days ago, a new release 0.0.15 of RcppAnnoy got onto CRAN while I was traveling / attending the wonderful celebRtion 2020 for the 20th anniversary of the R 1.0.0 release.

This releases makes great strides towards avoiding long-standing SAN/UBSAN issues. Upstream author Erik has been most helpful, as has been the feedback and input from two downstream users of RcppAnnoy, namely Aaron and James. This 0.0.15 release addresses one key, and longstanding, SAN/UBSAN issue. It is actually rather tricky as the code, for efficiency reason, bounces at the edge of what can be done. But a small rearrangement suppresses one such message which is good. We also got a hint from CRAN (thanks for that as always) to re-read one section of Writing R Extensions to make alloca more portable so that Solaris does not have to cry, and Bill Venables kindly helped with a small correction to the docs.

The other important issue is that there will be another 0.0.16 release real soon to incorporate three more small upstream PRs driven by these discussions which kept going on post-release while I was conferencing (or traveling), and which should fix things for good, or so we hope. This should go out probably by the end of the week to not exceed a weekly upload cadence; if you want to see more, or get earlier access, see the git repo which is in fine shape. If you want to see a pre-release on the ghrr drat drop me a line.

Detailed changes follow below.

Changes in version 0.0.15 (2020-02-25)

RcppAnnoy synchronized with upstream PR 455 (Dirk in #55).

The help page has a small correction thanks to Bill1 Venables.

The alloca() function is now declared portably thanks to a working example in Writing R Extensions.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 22 Feb 2020

digest 0.6.25: Spookyhash bugfix

And a new version of digest is getting onto CRAN now, and to Debian shortly.

digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, and spookyhash algorithms) permitting easy comparison of R language objects. It is a fairly widely-used package (currently listed at 889k monthly downloads with 255 direct reverse dependencies and 7340 indirect reverse dependencies) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation.

This release is a one issue fix. Aaron Lun noticed some issues when spookyhash is used in streaming mode. Kendon Bell, who also contributed spookyhash quickly found the issue which is a simple oversight. This was worth addressing in new release, so I pushed 0.6.25.

CRANberries provides the usual summary of changes to the previous version.

For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/digest | permanent link

RcppSimdJson 0.0.2: First Update!

Following up on the initial RcppSimdJson release, a first updated arrived on CRAN yesterday.

RcppSimdJson wraps the fantastic simdjson library by Daniel Lemire which truly impressive. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release syncs the simdjson headers with upstream, and polishes the build a little by conditioning on actually having a C++17 compiler rather than just suggesting it. The NEWS entry follows.

Changes in version 0.0.2 (2020-02-21)

Sychronized with upstream (Dirk in #4 and #5).

The R side of validateJSON now globs the file argument, expanding symbols like ~ appropriately.

C++ code in validateJSON now conditional on C++17 allowing (incomplete) compilation on lesser systems.

New helper function returning value of __cplusplus macro, used in package startup to warn if insufficient compiler used.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 13 Feb 2020

RcppSimdJson 0.0.1 now on CRAN!

A fun weekend-morning project, namely wrapping the outstanding simdjson library by Daniel Lemire (with contributions by Geoff Langdale, John Keiser and many others) into something callable from R via a new package RcppSimdJson lead to a first tweet on January 20, a reference to the brand new github repo, and CRAN upload a few days later—and then two weeks of nothingness.

Well, a little more than nothing as Daniel is an excellent “upstream” to work with who promptly incorporated two changes that arose from preparing the CRAN upload. So we did that. But CRAN being as busy and swamped as they are we needed to wait. The ten days one is warned about. And then some more. So yesterday I did a cheeky bit of “bartering” as Kurt wanted a favour with an updated digest version so I hinted that some reciprocity would be appreciated. And lo and behold he admitted RcppSimdJson to CRAN. So there it is now!

We have some upstream changes already in git, but I will wait a few days to let a week pass before uploading the now synced upstream code. Anybody who wants it sooner knows where to get it on GitHub.

simdjson is a gem. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

The NEWS entry (from a since-added NEWS file) for the initial RcppSimdJson upload follows.

Changes in version 0.0.1 (2020-01-24)

Initial CRAN upload of first version

Comment-out use of stdout (now updated upstream)

Deactivate use computed GOTOs for compiler compliance and CRAN Policy via #define

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 12 Feb 2020

digest 0.6.24: Some more refinements

Another new version of digest arrived on CRAN (and also on Debian) earlier today.

This release comes a few month after the previous release. It contains a few contributed fixes, some of which prepare for R 4.0.0 in its current development. This includes a testing change to the matrix/array class, and corrects the registration for the PMurHash routine as pointed out by Tomas Kalibera and Kurt Hornik (who also kindly reminded me to finally upload this as I had made the fix already in December). Moreover, Will Landau sped up one operation affecting his popular drake pipeline toolkit. Lastly, Thierry Onkelinx corrected one more aspect related to sha1.

CRANberries provides the usual summary of changes to the previous version.

For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/digest | permanent link

Sun, 09 Feb 2020

RcppArmadillo 0.9.850.1.0

armadillo image

A new upstream release 9.850.1 of Armadillo was just released. And as some will undoubtedly notice, Conrad opted for an increment of 50 rather 100. We wrapped this up as version 0.9.850.1.0, having prepared a full (github-only) tarball and the release candidate 9.850.rc1 a few days ago. Both the release candidate and the release got the full reverse depends treatment, and no issues were found.

Changes in the new release below.

Changes in RcppArmadillo version 0.9.850.1.0 (2020-02-09)

Upgraded to Armadillo release 9.850.1 (Pyrocumulus Wrath)

faster handling of compound expressions by diagmat()

expanded .save() and .load() to handle CSV files with headers via csv_name(filename,header) specification

added log_normpdf()

added .is_zero()

added quantile()

The sparse matrix test using scipy, if available, is now simplified thanks to recently added reticulate conversions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 08 Feb 2020

RProtoBuf 0.4.15: One fix, some updates, depcrecation coming

A new release 0.4.15 of RProtoBuf just arrived at CRAN. RProtoBuf provides R with bindings for the Google Protocol Buffers (“ProtoBuf”) data encoding and serialization library used and released by Google, and deployed very widely in numerous projects as a language and operating-system agnostic protocol.

This release contains a small bug fix for repeated messages and groups. While making changes, I used the opportunity to change the unit testing framework to the excellent and lightweight tinytest package permitting, among other things, tests of the installed package, and also simplified the build by using pre-made pdf vignettes. A list of changes follows below.

As one heads-up and deprecation message, we are planning to remove the (entirely unused as best as we can tell, and minimal) remote procedure call feature. Protocol Buffers itself always lacked this, but eventually gRPC arrived to fill that void. All use cases should rely on it. So we prepared a test release 0.14.15.1 corresponding to the feature/retire_minimal_rpc branch. You can install this dev release from the ghrr drat repo via one of the drat-assisted commands, or directly via install.packages("RProtoBuf", repos="https://ghrr.github.io/drat"). Please do so and test if you suspect that the change may affect you. Otherwise the removal is likely to happen in the next release (but we will strive to wait a couple of months before doing so).

Changes in RProtoBuf version 0.4.15 (2020-02-08)

Repeated Message and Group objects are now returned correctly (Dirk in #64 fixing #62).

The unit tests are now run by tinytest (Dirk in #65).

The vignettes are now included pre-made (Dirk in #67).

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rprotobuf | permanent link

Fri, 24 Jan 2020

RcppArmadillo 0.9.800.4.0

armadillo image

A second small Armadillo bugfix upstream update 9.800.4 came out yesterday for the 9.800.* series, following a similar bugfix release 9.800.3 in December. This time just one file was changed (see below).

Changes in RcppArmadillo version 0.9.800.4.0 (2020-20-24)

Upgraded to Armadillo release 9.800.4 (Horizon Scraper)

fixes for incorrect type promotion in normpdf()

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 20 Jan 2020

anytime 0.3.7

A fresh minor release of the anytime package is arriving on CRAN right now. This is the eighteenth release, and it comes roughly five months after the previous showing the relative feature-stability we have now.

anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, … format to either POSIXct or Date objects – and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples.

This release brings a clever new option, thanks to Stephen Froehlich. If you know your input has (lots) of duplicates you can now say so and anytime() (and the other entry points for times and dates, UTC or not) will only parse the unique entries leading to potentially rather large speed gains (as in Stephen’s case where he often has more than 95% of the data as duplicates). We also tweaked the test setup some more, but as we are still unable to replicate what is happening with the Fedora test boxen at CRAN due to the non-reproducible setup so this remains a bit of guess work. Lastly, I am making use of a new Rcpp #define to speed up compilation a little bit too.

The full list of changes follows.

Changes in anytime version 0.3.7 (2019-01-20)

Test and possibly condition away one more test file.

Small enhancement for compilation by setting no-rtti define via Rcpp.

New option calcUnique for speed-up by parseing only unique timestamps (Stephen Froehlich in #110 fixing #109).

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. The issue tracker tracker off the GitHub repo can be use for questions and comments.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/anytime | permanent link

Sun, 19 Jan 2020

RPushbullet 0.3.3

RPpushbullet demo

Release 0.3.3 of the RPushbullet package just got to CRAN. RPushbullet offers an interface to the neat Pushbullet service for inter-device messaging, communication, and more. It lets you easily send (programmatic) alerts like the one to the left to your browser, phone, tablet, … – or all at once.

This release further robustifies operations via two contributed PRs. The first by Chan-Yub ensures we set UTF-8 encoding on pushes. The second by Alexandre permits to downgrade from http/2 to http/1.1 which he needed for some operations with a particular backend. I made that PR a bit more general by turning the downgrade into one driven by a new options() toggle. Special thanks also to Jeroen in help debugging this issue. See below for more details.

Changes in version 0.3.3 (2020-01-18)

UTF-8 encoding is now used (Chan-Yub Park in #55).

Curl can use HTTP/1.1 (Alexandre Shannon in #59 fixing #57, plus Dirk in #60 making it optional).

Courtesy of CRANberries, there is also a diffstat report for this release. More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rpushbullet | permanent link

Thu, 16 Jan 2020

RcppRedis 0.1.10: Switch to tinytest

Another minor release of RcppRedis just arrived on CRAN, following a fairly long break since the last release in October 2018.

RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has carried production loads for several years now.

This release switches to the fabulous tinytest package, allowing for very flexible testing during development and deployment—three cheers for easily testing installed packages too.

Changes in version 0.1.10 (2020-01-16)

The package now uses tinytest for unit tests (Dirk in #41).

Courtesy of CRANberries, there is also a diffstat report for this release. More information is on the RcppRedis page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 15 Jan 2020

RQuantLib 0.4.11: More polish

New year, new RQuantLib! A new release 0.4.11 of RQuantLib arrived overnight on CRAN; and a Debian upload will follow shortly.

QuantLib is a very comprehensice free/open-source library for quantitative finance; RQuantLib connects it to the R environment and language.

This version does three new things. First, we fixed an oversight on our end and now allow a null calendar (as the C++ API). Second, the package switched to tinytest as a few of my other packages have done, allowing for very flexible testing during development and deployment—three cheers for easily testing installed packages too. Third, and per a kind nag from Kurt Hornik I updated a few calls which the current QuantLib 1.17 marks as deprecated. That lead to a compile issue with 1.16 so the change is conditional in one part. The complete set of changes is listed below:

Changes in RQuantLib version 0.4.11 (2020-01-15)

Changes in RQuantLib code:

The 'Null' calendar without weekends or holidays is now recognized.

The package now uses tinytest for unit tests (Dirk in #140).

Calls deprecated-in-QuantLib 1.17 were updated (Dirk in #144).

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rquantlib | permanent link

Thu, 09 Jan 2020

rfoaas 2.1.0: New upstream so new access point!

rfoaas greed example

FOAAS, having been resting upstream for some time, released version 2.1.0 of its wonderful service this week! So without too much further ado we went to work and added support for it. And now we are in fact thrilled to announce that release 2.1.0 of rfoaas is now on CRAN as of this afternoon (with a slight delay as yours truly managed to state the package release date as 2019-01-09 which was of course flagged as ‘too old’).

The new 2.1.0 release of FOAAS brings a full eleven new REST access points, namely even(), fewer(), ftfty(), holygrail(), idea(), jinglebells(), legend(), logs(), ratsarse(), rockstar(), and waste(). On our end, documentation and tests were updated.

As usual, CRANberries provides a diff to the previous CRAN release. Questions, comments etc should go to the GitHub issue tracker. More background information is on the project page as well as on the github repo

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rfoaas | permanent link

Wed, 08 Jan 2020

BH 1.72.0-3 on CRAN

Boost

The BH 1.72.0-1 release of BH required one update 1.72.0-2 when I botched a hand-edited path (to comply with the old-school path-length-inside-tar limit).

Turns out another issue needed a fix. This release improved on prior ones by starting from a pristine directory. But as a side effect, Boost Accumulators ended up incomplete with only the dependented-upon-by-others files included (by virtue of the bcp tool). So now we declared Boost Accumulators a full-fledged part of BH ensuring that bcp copies it “whole”. If you encounter issues with another incomplete part, please file an issue ticket at the GitHub repo.

No other changes were made.

Also, this fix was done initially while CRAN took a well-deserved winter break, and I had tweeted on Dec 31 about availability via drat and may use this more often for pre-releases. CRAN is now back, and this (large !!) package is now processed as part of the wave of packages that were in waiting (and Henrik got that right yesterday…).

Via CRANberries, there is a diffstat report relative to the previous release.

Comments and suggestions about BH are welcome via the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/bh | permanent link