Tue, 07 Jul 2020

RcppSimdJson 0.1.0: Now on Windows, With Parsers and Faster Still!

A smashing new RcppSimdJson release 0.1.0 containing several small updates to upstream simdjson (now at 0.4.6) in part triggered by very excisting work by Brendan who added actual parser from file and string—and together with Daniel upstream worked really hard to make Windows builds as well as complete upstream tests on our beloved (ahem) MinGW platform possible. So this version will, once the builders have caught up, give everybody on Windows a binary—with a JSON parser running circles around the (arguably more feature-rich and possibly easier-to-use) alternatives. Dave just tweeted a benchmark snippet by Brendan, the full set is at the bottom our issue ticket for this release.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators, which in its upstream release 0.4.0 improved once more (also see the following point releases). Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

As mentioned, this release expands the reach of the package to Windows, and adds new user-facing functions. A big thanks for most of this is owed to Brendan, so buy him a drink if you run across him. The full NEWS entry follows.

Changes in version 0.1.0 (2020-07-07)

  • Upgraded to simdjson 0.4.1 which adds upstream Windows support (Dirk in #27 closing #26 and #14, plus extensive work by Brendan helping upstream with mingw tests).

  • Upgraded to simdjson 0.4.6 with further upstream improvements (Dirk in #30).

  • Change Travis CI to build matrix over g++ 7, 8, 9, and 10 (Dirk in #31; and also Brendan in #32).

  • New JSON functions fparse and fload (Brendan in #32) closing #18 and #10).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 06 Jul 2020

Rcpp 1.0.5: Several Updates

rcpp logo

Right on the heels of the news of 2000 CRAN packages using Rcpp (and also hitting 12.5 of CRAN package, or one in eight), we are happy to announce release 1.0.5 of Rcpp. Since the ten-year anniversary and the 1.0.0 release release in November 2018, we have been sticking to a four-month release cycle. The last release has, however, left us with a particularly bad taste due to some rather peculiar interactions with a very small (but ever so vocal) portion of the user base. So going forward, we will change two things. First off, we reiterate that we have already made rolling releases. Each minor snapshot of the main git branch gets a point releases. Between release 1.0.4 and this 1.0.5 release, there were in fact twelve of those. Each and every one of these was made available via the drat repo, and we will continue to do so going forward. Releases to CRAN, however, are real work. If they then end up with as much nonsense as the last release 1.0.4, we think it is appropriate to slow things down some more so we intend to now switch to a six-months cycle. As mentioned, interim releases are always just one install.packages() call with a properly set repos argument away.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 2002 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 203 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steady at around one millions downloads per month.

This release features again a number of different pull requests by different contributors covering the full range of API improvements, attributes enhancements, changes to Sugar and helper functions, extended documentation as well as continuous integration deplayment. See the list below for details.

Changes in Rcpp patch release version 1.0.5 (2020-07-01)

  • Changes in Rcpp API:

    • The exception handler code in #1043 was updated to ensure proper include behavior (Kevin in #1047 fixing #1046).

    • A missing Rcpp_list6 definition was added to support R 3.3.* builds (Davis Vaughan in #1049 fixing #1048).

    • Missing Rcpp_list{2,3,4,5} definition were added to the Rcpp namespace (Dirk in #1054 fixing #1053).

    • A further updated corrected the header include and provided a missing else branch (Mattias Ellert in #1055).

    • Two more assignments are protected with Rcpp::Shield (Dirk in #1059).

    • One call to abs is now properly namespaced with std:: (Uwe Korn in #1069).

    • String object memory preservation was corrected/simplified (Kevin in #1082).

  • Changes in Rcpp Attributes:

    • Empty strings are not passed to R CMD SHLIB which was seen with R 4.0.0 on Windows (Kevin in #1062 fixing #1061).

    • The short_file_name() helper function is safer with respect to temporaries (Kevin in #1067 fixing #1066, and #1071 fixing #1070).

  • Changes in Rcpp Sugar:

    • Two sample() objects are now standard vectors and not R_alloc created (Dirk in #1075 fixing #1074).
  • Changes in Rcpp support functions:

    • Rcpp.package.skeleton() adjusts for a (documented) change in R 4.0.0 (Dirk in #1088 fixing #1087).
  • Changes in Rcpp Documentation:

    • The pdf file of the earlier introduction is again typeset with bibliographic information (Dirk).

    • A new vignette describing how to package C++ libraries has been added (Dirk in #1078 fixing #1077).

  • Changes in Rcpp Deployment:

    • Travis CI unit tests now run a matrix over the versions of R also tested at CRAN (rel/dev/oldrel/oldoldrel), and coverage runs in parallel for a net speed-up (Dirk in #1056 and #1057).

    • The exceptions test is now partially skipped on Solaris as it already is on Windows (Dirk in #1065).

    • The default CI runner was upgraded to R 4.0.0 (Dirk).

    • The CI matrix spans R 3.5, 3.6, r-release and r-devel (Dirk).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2455 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 04 Jul 2020

Rcpp now used by 2000 CRAN packages–and one in eight!

2000 Rcpp packages

As of yesterday, Rcpp stands at exactly 2000 reverse-dependencies on CRAN. The graph on the left depicts the growth of Rcpp usage (as measured by Depends, Imports and LinkingTo, but excluding Suggests) over time.

Rcpp was first released in November 2008. It probably cleared 50 packages around three years later in December 2011, 100 packages in January 2013, 200 packages in April 2014, and 300 packages in November 2014. It passed 400 packages in June 2015 (when I tweeted about it), 500 packages in late October 2015, 600 packages in March 2016, 700 packages last July 2016, 800 packages last October 2016, 900 packages early January 2017, 1000 packages in April 2017, 1250 packages in November 2017, 1500 packages in November 2018 and then 1750 packages last August. The chart extends to the very beginning via manually compiled data from CRANberries and checked with crandb. The next part uses manually saved entries. The core (and by far largest) part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of packages using Rcpp is available too.

Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent July of 2015, seven percent just before Christmas 2015, eight percent in the summer of 2016, nine percent mid-December 2016, cracked ten percent in the summer of 2017 and eleven percent in 2018. We now passed 12.5 percent—so one in every eight CRAN packages dependens on Rcpp. Stunning. There is more detail in the chart: how CRAN seems to be pushing back more and removing more aggressively (which my CRANberries tracks but not in as much detail as it could), how the growth of Rcpp seems to be slowing somewhat outright and even more so as a proportion of CRAN – as one would expect a growth curve to.

To mark the occassion, I sent out two tweets yesterday: first a shorter one with “just the numbers”, followed by a second one also containing the few calculation steps. The screenshot from the second one is below.

2000 Rcpp packages

2000 user packages is pretty mind-boggling. We can use the progression of CRAN itself compiled by Henrik in a series of posts and emails to the main development mailing list. Not that long ago CRAN itself did have only 1000 packages, then 5000, 10000, and here we are at just over 16000 with Rcpp at 12.5% and still growing (though maybe more slowly). Amazeballs.

The Rcpp team continues to aim for keeping Rcpp as performant and reliable as it has been. A really big shoutout and Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 25 Jun 2020

RcppSimdJson 0.0.6: New Upstream, New Features!

A very exciting RcppSimdJson release with the updated upstream simdjson release 0.4.0 as well as a first set of new JSON parsing functions just hit CRAN. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The very recent 0.4.0 release further improves the already impressive speed.

And this release brings a first set of actually user-facing functions thanks to Brendan which put in a series of PRs! The full NEWS entry follows.

Changes in version 0.0.6 (2020-06-25)

  • Created C++ integer-handling utilities for safe downcasting and integer return (Brendan in #16 closing #13).

  • New JSON functions .deserialize_json and .load_json (Brendan in #16, #17, #20, #21).

  • Upgrade Travis CI to 'bionic', extract package and version from DESCRIPTION (Dirk in #23).

  • Upgraded to simdjson 0.4.0 (Dirk in #25 closing #24).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 21 Jun 2020

RcppGSL 0.3.8: More fixes and polish

Release 0.3.8 of RcppGSL is now getting onto CRAN. The RcppGSL package provides an interface from R to the GNU GSL using the Rcpp package.

Peter Carbonetto let us know in issue #25 that the included example now showed linker errors on (everybody’s favourite CRAN platform) Slowlaris. Kidding aside, the added compiler variety really has benefits because we were indeed missing a good handful or two of inline statements in the headers—which our good friends g++ and clang++ apparently let us get away with. This has been fixed, and a little bit of the usual package polish and cleanup has been added; see the list of detailed changes below.

Changes in version 0.3.8 (2020-06-21)

  • A few missing inline statements were added to the headers fixing a (genuine) error that was seen only on Solaris (Dirk).

  • The nice colNorm example is now in a file by itself, the previous versions are off in a new file colNorm_old.cpp (Dirk).

  • The README.me now sports two new badges (Dirk).

  • Travis CI was updated to 'bionic' and R 4.0 (Dirk).

Special thanks also to CRAN for a super-smooth and fully automated processing of a package with both compiled code and two handful of reverse dependencies.

Courtesy of CRANberries, a summary of changes to the most recent release is also available.

More information is on the RcppGSL page. Questions, comments etc should go to the issue tickets at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 15 Jun 2020

Rcpp 1.0.5 in two+ weeks: Please help test

rcpp logo

With the current four-month release cycle, the next Rcpp release is due in July following the 1.0.4 release in March. Just prior to the 1.0.4 release I had asked this:

It would be particularly beneficial if those with “unsual” build dependencies tested it as we would increase overall coverage beyond what I get from testing against 1800+ CRAN packages. BioConductor would also be welcome.

but only on the rcpp-devel list, and only about a good week prior to the release.

I remain rather disappointed and disillusioned about what happened after 1.0.4 was released. Two PRs in that release were soon seen to have side effects on more ‘marginal’ test systems, precisely what added testing could have revealed. An additional issue arose from changes in R’s make system, which is harder to anticipate or test. Each and every infelicity was fixed within a day or so, and we always make candidate releases available—the current Rcpp as of this writing is 1.0.4.12 meaning twelve microreleases were made since 1.0.4. And those microreleases are always available for normal download and install.packages use via the Rcpp drat repository accessible to all. So it was truly troubling to see some, especially those with experience in setting up or running testing / ci platforms, pretend to be unable to access, install, and provide these for their own tests, or the tests of their users. It just doesn’t pass a basic logic test: it takes a single call to install.packages(), or, even more easily, a single assignment of an auxiliary repo. All told this was a rather sad experience.

So let’s try to not repeat this. If you, or maybe users of a build or ci system you maintain, rely on Rcpp, and especially if you do so on systems outside the standard CRAN grid of three OSs and the triplet of “previous, current, next” releases of R, then please help by testing. I maitain these release as a volunteer, unpaid at that, and I simply cannot expand to more systesm. We take reverse dependency check seriously (and I just run two taking about a day each) but if you insist on building on stranger hardware or much older releases it will be up to you to ensure Rcpp passes. We prep for CRAN, and try our best to pass at CRAN. For nearly a dozen years.

To install the current microrelease from the Rcpp drat repository, just do

install.packages("Rcpp", repos="https://rcppcore.github.io/drat")

That is all there is to it. You could even add the Rcpp drat repository to your repository list.

Rcpp has become successful because so many people help with suggestions, documentation, and code. It is used by (as of today) 1958 CRAN packages, 205 BioConductor packages, and downloaded around a million times per month. So if you can, please help now with some more testing.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 09 Jun 2020

RcppArmadillo 0.9.900.1.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 727 other packages on CRAN.

Conrad recently released a new upstream version 9.900.1 of Armadillo which we packaged and tested as usual first as a ‘release candidate’ build and then as the release. As usual, logs from reverse-depends runs are in the rcpp-logs repo.

Apart from the new upstream release, we updated Travis use, ornamented the README a little, and smoothed over a rough corner from the recent R 4.0.0 release. All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.1.0 (2020-06-08)

  • Upgraded to Armadillo release 9.900.1 (Nocturnal Misbehaviour)

    • faster solve() for under/over-determined systems

    • faster eig_gen() and eig_pair() for large matrices

    • expanded eig_gen() and eig_pair() to optionally provide left and right eigenvectors

  • Switch Travis CI testing to R 4.0.0, use bionic as base distro and test R 3.6.3 and 4.0.0 in a matrix (Dirk in #298).

  • Add two badges to README for indirect use and the CSDA paper.

  • Adapt RcppArmadillo.package.skeleton() to a change in R 4.0.0 affecting what it exports in NAMESPACE.

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Rcpp Webinar Recording Available

As announced in a few tweets leading up to it, I took the date of what would have been the annual R/Finance conference as an opportunity to hold the one-hour tutorial / workshop with introductory Rcpp material which I often present on the first morning preceding the conference as a self-organized webinar. The live-streaming worked actually reasonably well via obs to youtube (even though the comprehensive software by the latter complained at times about insufficient bitstream rates–the joys of living with a (near) monopolistic broadband provider whom I should leave for fiber…). Apparently around seventy people connected to the stream—which is more than we usually have in the seminar room at UIC for the R/Finance morning.

The recording is now available here, and has already been seen over 200 times:

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 23 May 2020

RcppSimdJson 0.0.5: Updated Upstream

A new RcppSimdJson release with updated upstream simdjson code just arrived on CRAN. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed; see the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

This release brings updated upstream code (thanks to Brendan Knapp) plus a new example and minimal tweaks. The full NEWS entry follows.

Changes in version 0.0.5 (2020-05-23)

  • Add parseExample from earlier upstream announcement (Dirk).

  • Synced with upstream (Brendan in #12) closing #11).

  • Updated example parseExample to API changes (Brendan).

Courtesy of CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 17 May 2020

RcppArmadillo 0.9.880.1.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 719 other packages on CRAN.

Conrad released a new upstream version 9.880.1 of Armadillo on Friday which I packaged and tested as usual (result log here in the usual repo). The R package also sports a new OpenMP detection facility once again motivated by macOS which changed its setup yet again.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.880.1.0 (2020-05-15)

  • Upgraded to Armadillo release 9.880.1 (Roasted Mocha Detox)

    • expanded qr() to optionally use pivoted decomposition

    • updated physical constants to NIST 2018 CODATA values

    • added ARMA_DONT_USE_CXX11_MUTEX confguration option to disable use of std::mutex

  • OpenMP capability is tested explicitly (Kevin Ushey and Dirk in #294, #295, and #296 all fixing #290).

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 27 Apr 2020

RcppArmadillo 0.9.870.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 705 other packages on CRAN.

A new upstream release 9.870.2 of Armadillo was released a few days ago. We had tested two release candidates, and this caught one bug. The release was held up at CRAN for a few days as one package had an overly sensitive test depending on random input data; eventually we all convinced ourselves that there was no (Rcpp)Armadillo issue here. So morale: friends don’t let friends have tests depend on random behavior.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.870.2.0 (2020-04-24)

  • Upgraded to Armadillo release 9.870.2 (Roasted Mocha Retox)

    • faster handling of matrix multiplication expressions by diagvec() and diagmat()

    • added trimatu_ind() and trimatl_ind()

    • more consistent detection of sparse vector expressions

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 14 Apr 2020

RcppArmadillo 0.9.860.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 706 other packages on CRAN.

A new upstream release 9.860.2 of Armadillo was just released. The theme of “convergence” continues; the previous release increased the minor from 800 to 850, now we are at 860. We first wrapped this up as version 0.9.859.1.0, but it turned out to have been held back by a buglet between R 4.0.0 and Rcpp which the recent patch release fixed (along with other woes on old R or non-CRAN-alike macOS). It then turns out that the new (upstream) version 9.860.1 had a minor bug which I missed as I reverse-depends checked the prior version. Doh. My thanks, as always, to CRAN for spotting this. The fix was added upstream and we have 9.860.2 as RcppArmadillo 0.9.860.2.0.

Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.860.2.0 (2020-04-13)

  • Upgraded to Armadillo release 9.860.2 (Roasted Mocha Fix)

    • Added powmat()

    • Faster access to columns in sparse submatrix views

    • Faster handling of relational expressions by accu()

    • Faster handling of sympd matrices by expmat(), logmat(), sqrtmat()

    • Workaround for save/load issues with HDF5 v1.12

  • Vignettes are now pre-made and include (#285)

  • Two test files are now skipped on 32-bit Windows

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 09 Apr 2020

Rcpp 1.0.4.6: Bug fix interim version

rcpp logo

Rcpp 1.0.4 was released on March 17, following the usual sequence of fairly involved reverse-depends check along with a call for community testing issued weeks before the release. In that email I specifically pleaded with folks to pretty-please test non-standard setups:

It would be particularly beneficial if those with “unsual” build dependencies tested it as we would increase overall coverage beyond what I get from testing against 1800+ CRAN packages. BioConductor would also be welcome.

Alas, you can’t always get what you want. Shortly after the release we were made aware that the two (large) pull request at the book ends of the 1.0.3 to 1.0.4 release period created trouble. Of these two, the earliest PR in the 1.0.4 release upset older-than-CRAN-tested installation, i.e. R 3.3.0 or before. (Why you’d want to run R 3.3.* when R 3.6.3 is current is something I will never understand, but so be it.) This got addressed in two new PRs. And the matching last PR had a bit of sloppyness leaving just about everyone alone, but not all those macbook-wearing data scientists when using newer macOS SDKs not used by CRAN. In other words, “unsual” setups. But boy, do those folks have an ability to complain. Again, two quick PRs later that was addressed. Along came a minor PR with two more Rcpp::Shield<> uses (as life is too short to manually count PROTECT and UNPROTECT). And then a real issue between R 4.0.0 and Rcpp first noticed with RcppParallel builds on Windows but then also affecting RcppArmadillo. Another quickly issued fix. So by now the count is up to six, and we arrived at Rcpp 1.0.4.6.

Which is now on CRAN, after having sat there for nearly a full week, and of course with no reason given. Because the powers that be move in mysterious ways. And don’t answer to earthlings like us.

As may transpire here, I am little tired from all this. I think we can do better, and I think we damn well should, or I may as well throw in the towel and just release to the drat repo where each of the six interim versions was available for all to take as soon as it materialized.

Anyway, here is the state of things. Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1897 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 191 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steasy at one millions downloads per month.

The changes for this interim version are summarized below.

Changes in Rcpp patch release version 1.0.4.6 (2020-04-02)

  • Changes in Rcpp API:

    • The exception handler code in #1043 was updated to ensure proper include behavior (Kevin in #1047 fixing #1046).

    • A missing Rcpp_list6 definition was added to support R 3.3.* builds (Davis Vaughan in #1049 fixing #1048).

    • Missing Rcpp_list{2,3,4,5} definition were added to the Rcpp namespace (Dirk in #1054 fixing #1053).

    • A further updated corrected the header include and provided a missing else branch (Mattias Ellert in #1055).

    • Two more assignments are protect with Rcpp::Shield (Dirk in #1059)

  • Changes in Rcpp Attributes:

    • Empty strings are not passed to R CMD SHLIB which was seen with R 4.0.0 on Windows (Kevin in #1062 fixing #1061).
  • Changes in Rcpp Deployment:

    • Travis CI unit tests now run a matrix over the versions of R also tested at CRAN (rel/dev/oldrel/oldoldrel), and coverage runs in parallel for a net speed-up (Dirk in #1056 and #1057).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2356 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 03 Apr 2020

RcppSimdJson 0.0.4: Even Faster Upstream!

A new (upstream) simdjson release was announced by Daniel Lemire earlier this week, and my Twitter mentions have been running red-hot ever since as he was kind enough to tag me. Do look at that blog post, there is some impressive work in there. We wrapped up the (still very simple) rcppsimdjson around it last night and shipped it this morning.

RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. For illustration, I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release brings upstream 0.3 (and 0.3.1) plus a minor tweak (also shipped back upstream). Our full NEWS entry follows.

Changes in version 0.0.4 (2020-04-03)

  • Upgraded to new upstream releases 0.3 and 0.3.1 (Dirk in #9 closing #8)

  • Updated example validateJSON to API changes.

But because Daniel is such a fantastic upstream developer to collaborate with, he even filed a full feature-request ‘maybe you can consider upgrading’ as issue #8 at our repo containing the fully detailed list of changes. As it is so impressive I will simple quote the upper half of just the major changes:

Highlights

  • Multi-Document Parsing: Read a bundle of JSON documents (ndjson) 2-4x faster than doing it individually. API docs / Design Details
  • Simplified API: The API has been completely revamped for ease of use, including a new JSON navigation API and fluent support for error code and exception styles of error handling with a single API. Docs
  • Exact Float Parsing: Now simdjson parses floats flawlessly without any performance loss (https://github.com/simdjson/simdjson/pull/558). Blog Post
  • Even Faster: The fastest parser got faster! With a shiny new UTF-8 validator and meticulously refactored SIMD core, simdjson 0.3 is 15% faster than before, running at 2.5 GB/s (where 0.2 ran at 2.2 GB/s).

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 24 Mar 2020

New package RcppDate 0.0.1 now on CRAN!

A new small package with a new C++ header library is now on CRAN. It brings the date library by Howard Hinnant to R. This library has been in pretty wide-spread use for a while now, and adds to C++11/C++14/C++17 what will be (with minor modifications) the ‘date’ library in C++20. I had been aware of it for a while, but not needed thanks to CCTZ library out of Google and our RcppCCTZ package. And like CCTZ, it builds upon std::chron adding a whole lot of functionality and useability enhancement. But a some upcoming (and quite exciting!) changes in nanotime required it, I had a reason to set about packaging it as RcppDate. And after a few days of gestation and review it is now available via CRAN.

Two simple example files are included and can be accessed by Rcpp::sourceCpp(). Some brief excerpts follow.

The first example shows three date constructors. Note how the month (and the leading digits) are literals. No quotes for strings anywhere. And no format (just like our anytime package for R).

  constexpr auto x1 = 2015_y/March/22;
  constexpr auto x2 = March/22/2015;
  constexpr auto x3 = 22_d/March/2015;

Note that these are constexpr that resolve at compile-time, and that the resulting year_month_day type is inferred via auto.

A second example constructs the last day of the months similarly:

  constexpr auto x1 = 2015_y/February/last;
  constexpr auto x2 = February/last/2015;
  constexpr auto x3 = last/February/2015;

For more, see the copious date.h documentation.

The (very bland first) NEWS entry (from a since-added NEWS file) for the initial upload follows.

Changes in version 0.0.1 (2020-01-17)

  • Initial CRAN upload of first version

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 18 Mar 2020

RcppCCTZ 0.2.7

A new release 0.2.7 of RcppCCTZ is now at CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do—using copies in their packages which remains less than ideal.

This version adds internal extensions, contributed by Leonardo, which support upcoming changes to the nanotime package we are working on.

Changes in version 0.2.7 (2020-03-18)

  • Added functions _RcppCCTZ_convertToCivilSecond that converts a time point to the number of seconds since epoch, and _RcppCCTZ_convertToTimePoint that converts a number of seconds since epoch into a time point; these functions are only callable from C level (Leonardo in #34 and #35).

  • Added function _RcppCCTZ_getOffset that returns the offset at a speficied time-point for a specified timezone; this function is only callable from C level (Leonardo in #32).

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 17 Mar 2020

Rcpp 1.0.4: Lots of goodies

rcpp logo

The fourth maintenance release 1.0.4 of Rcpp, following up on the 10th anniversary and the 1.0.0. release sixteen months ago, arrived on CRAN this morning. This follows a few days of gestation at CRAN. To help during the wait we provided this release via drat last Friday. And it followed a pre-release via drat a week earlier. But now that the release is official, Windows and macOS binaries will be built by CRAN over the next few days. The corresponding Debian package will be uploaded as a source package shortly after which binaries can be built.

As with the previous releases Rcpp 1.0.1, Rcpp 1.0.2 and Rcpp 1.0.3, we have the predictable and expected four month gap between releases which seems appropriate given both the changes still being made (see below) and the relative stability of Rcpp. It still takes work to release this as we run multiple extensive sets of reverse dependency checks so maybe one day we will switch to six month cycle. For now, four months still seem like a good pace.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1873 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 191 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steasy at one millions downloads per month.

This release features quite a number of different pull requests by seven different contributors as detailed below. One (personal) highlight is the switch to tinytest.

Changes in Rcpp version 1.0.4 (2020-03-13)

  • Changes in Rcpp API:

    • Safer Rcpp_list*, Rcpp_lang* and Function.operator() (Romain in #1014, #1015).

    • A number of #nocov markers were added (Dirk in #1036, #1042 and #1044).

    • Finalizer calls clear external pointer first (Kirill Müller and Dirk in #1038).

    • Scalar operations with a rhs matrix no longer change the matrix value (Qiang in #1040 fixing (again) #365).

    • Rcpp::exception and Rcpp::stop are now more thread-safe (Joshua Pritikin in #1043).

  • Changes in Rcpp Attributes:

    • The cppFunction helper now deals correctly with mulitple depends arguments (TJ McKinley in #1016 fixing #1017).

    • Invisible return objects are now supported via new option (Kun Ren in #1025 fixing #1024).

    • Unavailable packages referred to in LinkingTo are now reported (Dirk in #1027 fixing #1026).

    • The sourceCpp function can now create a debug DLL on Windows (Dirk in #1037 fixing #1035).

  • Changes in Rcpp Documentation:

    • The .github/ directory now has more explicit guidance on contributing, issues, and pull requests (Dirk).

    • The Rcpp Attributes vignette describe the new invisible return object option (Kun Ren in #1025).

    • Vignettes are now included as pre-made pdf files (Dirk in #1029)

    • The Rcpp FAQ has a new entry on the recommended importFrom directive (Dirk in #1031 fixing #1030).

    • The bib file for the vignette was once again updated to current package versions (Dirk).

  • Changes in Rcpp Deployment:

    • Added unit test to check if C++ version remains remains aligned with the package number (Dirk in #1022 fixing #1021).

    • The unit test system was switched to tinytest (Dirk in #1028, #1032, #1033).

Please note that the change to execptions and Rcpp::stop() in pr #1043 has been seen to have a minor side effect on macOS issue #1046 which has already been fixed by Kevin in pr #1047 for which I may prepare a 1.0.4.1 release for the Rcpp drat repo in a day or two.

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2356 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 14 Mar 2020

RcppAPT 0.0.6

A new version of RcppAPT – our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, … commands and their cache powering Debian, Ubuntu and the like – is now on CRAN.

RcppAPT allows you to query the (Debian or Ubuntu) package dependency graph at will, with build-dependencies (if you have deb-src entries), reverse dependencies, and all other goodies. See the vignette and examples for illustrations.

This new version corrects builds failures under the new and shiny Apt 2.0 release (and the pre-releases like the 1.9.* series in Ubuntu) as some header files moved around. My thanks to Kurt Hornik for the heads-up. I accomodated the change in the (very simple and shell-based) configure script by a) asking pkg-config about the version of pkg-apt and then using that to b) compare to a ‘threshold value’ of ‘1.9.0’ and c) setting another compiler #define if needed so that d) these headers could get included if defined. The neat part is that a) and b) are done in an R one-liner, and the whole script is still in shell. Now, CRAN being CRAN, I now split the script into two: one almost empty one not using bash that passes the ‘omg but bash is not portable’ test, and which calls a second bash script doing the work. Fun and games…

The full set of changes follows.

Changes in version 0.0.6 (2020-03-14)

  • Accomodate Apt 2.0 code changes by including more header files

  • Change is backwards compatible and conditional

  • Added configure call using pkg-config and package version comparison (using R) to determine if the define is needed

  • Softened unit tests as we cannot assume optional source deb information to be present, so demo code runs but zero results tolerated

Courtesy of CRANberries, there is also a diffstat report for this release.

A bit more information about the package is available here as well as as the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 08 Mar 2020

RcppAnnoy 0.0.16

annoy image

A new release 0.0.16 of RcppAnnoy is available via the Github-hosted R Repository (ghrr). Edit a good two hours later: And wonder of wonders, now also on CRAN.

It remains in limbo at CRAN for no apparent reason. No change appears to be imminent either as the CRAN maintainers continue to play a passive-aggressive game of no communication for any reason. Which is a genuine shame as e [E]verbody involved in the package, i.e. Erik (upstream) and myself but also Aaron (downstream) worked pretty hard and well last weekend (while I was traveling / attending the wonderful celebRtion 2020 conference for the 20th anniversary of the R 1.0.0) to iron out all remaining issues. Installation is pretty flawless and silent as all compiler warnings have been takeb care of even under -pedantic on a recent version, and the last remaining UBSAN issue is also fixed.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

As I wrote in the announcement for 0.0.15 which fixed many-but-not-all issues:

The other important issue is that there will be another 0.0.16 release real soon to incorporate three more small upstream PRs driven by these discussions which kept going on post-release while I was conferencing (or traveling), and which should fix things for good, or so we hope. This should go out probably by the end of the week to not exceed a weekly upload cadence; if you want to see more, or get earlier access, see the git repo which is in fine shape. If you want to see a pre-release on the ghrr drat drop me a line.

As CRAN is holding the package hostage, all I can do now is to release to the Github-hosted R Repository (ghrr) from where you can install it via a simple install.packages("RcppAnnoy", repos="https://ghrr.github.io/drat") (or any of the other drat supported commands, see the ghrr page for more). Or wait and wait and wait … until CRAN graces us with a manual admission (given that the previous upload left one small UBSAN issue to fix). One day. Hopefully. The package is now on CRAN.

Detailed changes follow below.

Changes in version 0.0.16 (2020-03-06)

  • Use int in two interfaces (Dirk in #59 for upstream PR 460 and closing #56).

  • Use inline for two helper functions (Dirk in #59 for upstream PR 461 and closing #57; also Aaron in #58 after earlier discussion).

  • Removed a noisy pragma (Dirk in #60 for upstream PR 462).

  • Add a simple helper function displaying compiler status.

Courtesy of CRANberries, there will also be diffstat report if and when the package ever makes it to CRAN. is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 04 Mar 2020

RcppSimdJson 0.0.3: Second Update!

Following up on both the initial RcppSimdJson release and the first update, the second update release 0.0.3 arrived on CRAN yesterday.

RcppSimdJson wraps the fantastic simdjson library by Daniel Lemire which is truly impressive. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. For illustration, I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release once again syncs the simdjson headers with upstream, and strengthens the build setup a little bit more. We only turn C++17 (which is needed) on when R knows it (from its builds), report the architecture status at package load (in a suppressable message), and only attempt to parse in examples and unit tests when know that we are on a sufficient platform. The full NEWS entry follows.

Changes in version 0.0.3 (2020-03-03)

  • Sychronized once more with upstream.

  • Created new C++ function to check for unsupported architecture, and report the status on package load.

  • Only run example and unit tests if supported architecture is found.

  • Created small configure script to see if R was built with C++17 support, and record it in src/Makevars.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 02 Mar 2020

RcppAnnoy 0.0.15

annoy image

A few days ago, a new release 0.0.15 of RcppAnnoy got onto CRAN while I was traveling / attending the wonderful celebRtion 2020 for the 20th anniversary of the R 1.0.0 release.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This releases makes great strides towards avoiding long-standing SAN/UBSAN issues. Upstream author Erik has been most helpful, as has been the feedback and input from two downstream users of RcppAnnoy, namely Aaron and James. This 0.0.15 release addresses one key, and longstanding, SAN/UBSAN issue. It is actually rather tricky as the code, for efficiency reason, bounces at the edge of what can be done. But a small rearrangement suppresses one such message which is good. We also got a hint from CRAN (thanks for that as always) to re-read one section of Writing R Extensions to make alloca more portable so that Solaris does not have to cry, and Bill Venables kindly helped with a small correction to the docs.

The other important issue is that there will be another 0.0.16 release real soon to incorporate three more small upstream PRs driven by these discussions which kept going on post-release while I was conferencing (or traveling), and which should fix things for good, or so we hope. This should go out probably by the end of the week to not exceed a weekly upload cadence; if you want to see more, or get earlier access, see the git repo which is in fine shape. If you want to see a pre-release on the ghrr drat drop me a line.

Detailed changes follow below.

Changes in version 0.0.15 (2020-02-25)

  • RcppAnnoy synchronized with upstream PR 455 (Dirk in #55).

  • The help page has a small correction thanks to Bill1 Venables.

  • The alloca() function is now declared portably thanks to a working example in Writing R Extensions.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 22 Feb 2020

RcppSimdJson 0.0.2: First Update!

Following up on the initial RcppSimdJson release, a first updated arrived on CRAN yesterday.

RcppSimdJson wraps the fantastic simdjson library by Daniel Lemire which truly impressive. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed.

This release syncs the simdjson headers with upstream, and polishes the build a little by conditioning on actually having a C++17 compiler rather than just suggesting it. The NEWS entry follows.

Changes in version 0.0.2 (2020-02-21)

  • Sychronized with upstream (Dirk in #4 and #5).

  • The R side of validateJSON now globs the file argument, expanding symbols like ~ appropriately.

  • C++ code in validateJSON now conditional on C++17 allowing (incomplete) compilation on lesser systems.

  • New helper function returning value of __cplusplus macro, used in package startup to warn if insufficient compiler used.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 13 Feb 2020

RcppSimdJson 0.0.1 now on CRAN!

A fun weekend-morning project, namely wrapping the outstanding simdjson library by Daniel Lemire (with contributions by Geoff Langdale, John Keiser and many others) into something callable from R via a new package RcppSimdJson lead to a first tweet on January 20, a reference to the brand new github repo, and CRAN upload a few days later—and then two weeks of nothingness.

Well, a little more than nothing as Daniel is an excellent “upstream” to work with who promptly incorporated two changes that arose from preparing the CRAN upload. So we did that. But CRAN being as busy and swamped as they are we needed to wait. The ten days one is warned about. And then some more. So yesterday I did a cheeky bit of “bartering” as Kurt wanted a favour with an updated digest version so I hinted that some reciprocity would be appreciated. And lo and behold he admitted RcppSimdJson to CRAN. So there it is now!

We have some upstream changes already in git, but I will wait a few days to let a week pass before uploading the now synced upstream code. Anybody who wants it sooner knows where to get it on GitHub.

simdjson is a gem. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in persing gigabytes of JSON parsed per second which is quite mindboggling. I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk).

The NEWS entry (from a since-added NEWS file) for the initial RcppSimdJson upload follows.

Changes in version 0.0.1 (2020-01-24)

  • Initial CRAN upload of first version

  • Comment-out use of stdout (now updated upstream)

  • Deactivate use computed GOTOs for compiler compliance and CRAN Policy via #define

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 09 Feb 2020

RcppArmadillo 0.9.850.1.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 685 other packages on CRAN.

A new upstream release 9.850.1 of Armadillo was just released. And as some will undoubtedly notice, Conrad opted for an increment of 50 rather 100. We wrapped this up as version 0.9.850.1.0, having prepared a full (github-only) tarball and the release candidate 9.850.rc1 a few days ago. Both the release candidate and the release got the full reverse depends treatment, and no issues were found.

Changes in the new release below.

Changes in RcppArmadillo version 0.9.850.1.0 (2020-02-09)

  • Upgraded to Armadillo release 9.850.1 (Pyrocumulus Wrath)

    • faster handling of compound expressions by diagmat()

    • expanded .save() and .load() to handle CSV files with headers via csv_name(filename,header) specification

    • added log_normpdf()

    • added .is_zero()

    • added quantile()

  • The sparse matrix test using scipy, if available, is now simplified thanks to recently added reticulate conversions.

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 24 Jan 2020

RcppArmadillo 0.9.800.4.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 680 other packages on CRAN.

A second small Armadillo bugfix upstream update 9.800.4 came out yesterday for the 9.800.* series, following a similar bugfix release 9.800.3 in December. This time just one file was changed (see below).

Changes in RcppArmadillo version 0.9.800.4.0 (2020-20-24)

  • Upgraded to Armadillo release 9.800.4 (Horizon Scraper)

    • fixes for incorrect type promotion in normpdf()

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 16 Jan 2020

RcppRedis 0.1.10: Switch to tinytest

Another minor release of RcppRedis just arrived on CRAN, following a fairly long break since the last release in October 2018.

RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has carried production loads for several years now.

This release switches to the fabulous tinytest package, allowing for very flexible testing during development and deployment—three cheers for easily testing installed packages too.

Changes in version 0.1.10 (2020-01-16)

  • The package now uses tinytest for unit tests (Dirk in #41).

Courtesy of CRANberries, there is also a diffstat report for this release. More information is on the RcppRedis page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 09 Dec 2019

RcppClassic 0.9.12

A maintenance release 0.9.12 of the RcppClassic package arrived earlier today on CRAN. This package provides a maintained version of the otherwise deprecated initial Rcpp API which no new projects should use as the normal Rcpp API is so much better.

Changes are all internal. Testing is now done via tinytest, vignettes are now pre-built and at the request of CRAN we no longer strip the resulting library. No other changes were made.

CRANberries also reports the changes relative to the previous release from July of last year.

Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 07 Dec 2019

RcppArmadillo 0.9.800.3.0

armadillo image

A small Armadillo bugfix upstream update 9.800.3 came out a few days ago. The changes, summarised by Conrad in email to me (and for once not yet on the arma site are fixes for matrix row iterators, better detection of non-hermitian matrices by eig_sym(), inv_sympd(), chol(), expmat_sym() and miscellaneous minor fixes. It also contains a bug fix by Christian Gunning to his sample() implementation.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 679 other packages on CRAN.

Changes in RcppArmadillo version 0.9.800.3.0 (2019-12-04)

  • Upgraded to Armadillo release 9.800.3 (Horizon Scraper)

  • The sample function passes the prob vector as const allowing subsequent calls (Christian Gunning in #276 fixing #275)

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 16 Nov 2019

RcppEigen 0.3.3.7.0

A new minor release 0.3.3.7.0 of RcppEigen arrived on CRAN today (and just went to Debian too) bringing support for Eigen 3.3.7 to R.

This release comes almost a year after the previous minor release 0.3.3.5.0. Besides the upgrade to the new upstream version, it brings a few accumulated polishes to the some helper and setup functions, and switches to the very nice tinytest package for unit tests; see below for the full list. As before, we carry a few required changes to Eigen in a diff. And as we said before at the previous two releases:

One additional and recent change was the accomodation of a recent CRAN Policy change to not allow gcc or clang to mess with diagnostic messages. A word of caution: this may make your compilation of packages using RcppEigen very noisy so consider adding -Wno-ignored-attributes to the compiler flags added in your ~/.R/Makevars.

The complete NEWS file entry follows.

Changes in RcppEigen version 0.3.3.7.0 (2019-11-16)

  • Fixed skeleton package creation listing RcppEigen under Imports (James Balamuta in #68 addressing #16).

  • Small RNG use update to first example in skeleton package used by package creation helper (Dirk addressing #69).

  • Update vignette example to use RcppEigen:::eigen_version() (Dirk addressing #71).

  • Correct one RcppEigen.package.skeleton() corner case (Dirk in #77 fixing #75).

  • Correct one usage case with pkgKitten (Dirk in #78).

  • The package now uses tinytest for unit tests (Dirk in #81).

  • Upgraded to Eigen 3.3.7 (Dirk in #82 fixing #80).

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 12 Nov 2019

RcppAnnoy 0.0.14

annoy image

A new minor release of RcppAnnoy is now on CRAN, following the previous 0.0.13 release in September.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release once again allows compilation on older compilers. The 0.0.13 release in September brought very efficient 512-bit AVX instruction to accelerate computations. However, this could not be compiled on older machines so we caught up once more with upstream to update to conditional code which will fall back to either 128-bit AVX or no AVX, ensuring buildability “everywhere”.

Detailed changes follow below.

Changes in version 0.0.14 (2019-11-11)

  • RcppAnnoy again synchronized with upstream to ensure builds with older compilers without AVX512 instructions (Dirk #53).

  • The cleanup script only uses /bin/sh.

Courtesy of CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 09 Nov 2019

Rcpp 1.0.3: More Spit and Polish

rcpp logo

The third maintenance release 1.0.3 of Rcpp, following up on the 10th anniversary and the 1.0.0. release both pretty much exactly one year ago, arrived on CRAN yesterday. This deserves a special shoutout to Uwe Ligges who was even more proactive and helpful than usual. Rcpp is a somewhat complex package with many reverse dependencies, and both the initial check tickles one (grandfathered) NOTE, and the reverse dependencies typically invoke a few false positives too. And in both cases did he move the process along before I even got around to replying to the auto-generated emails. So just a few hours passed between my upload, and the Thanks, on its way to CRAN email—truly excellent work of the CRAN team. Windows and macOS binaries are presumably being built now. The corresponding Debian package was also uploaded as a source package, and binaries have since been built.

Just like for Rcpp 1.0.1 and Rcpp 1.0.2, we have a four month gap between releases which seems appropriate given both the changes still being made (see below) and the relative stability of Rcpp. It still takes work to release this as we run multiple extensive sets of reverse dependency checks so maybe one day we will switch to six month cycle. For now, four months seem like a good pace.

Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1832 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 190 in BioConductor. And per the (partial) logs of CRAN downloads, we are currently running at 1.1 millions downloads per month.

This release features a number of different pull requests by five different contributors as detailed below.

Changes in Rcpp version 1.0.3 (2019-11-08)

  • Changes in Rcpp API:

    • Compilation can be sped up by skipping Modules headers via a toggle RCPP_NO_MODULES (Kevin in #995 for #993).

    • Compilation can be sped up by toggling RCPP_NO_RTTI which implies RCPP_NO_MODULES (Dirk in #998 fixing #997).

    • XPtr tags are now preserved in as<> (Stephen Wade in #1003 fixing #986, plus Dirk in #1012).

    • A few more temporary allocations are now protected from garbage collection (Romain Francois in #1010, and Dirk in #1011).

  • Changes in Rcpp Modules:

    • Improved initialization via explicit Rcpp:: prefix (Riccardo Porreca in #980).
  • Changes in Rcpp Deployment:

    • A unit test for Rcpp Class exposure was updated to not fail under r-devel (Dirk in #1008 fixing #1006).
  • Changes in Rcpp Documentation:

    • The Rcpp-modules vignette received a major review and edit (Riccardo Porreca in #982).

    • Minor whitespace alignments and edits were made in three vignettes following the new pinp release (Dirk).

    • New badges for DOI and CRAN and BioConductor reverse dependencies have been added to README.md (Dirk).

    • Vignettes are now included pre-made (Dirk in #1005 addressing #1004)).

    • The Rcpp FAQ has two new entries on 'no modules / no rtti' and exceptions across shared libraries (Dirk in #1009).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2255 previous questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 20 Oct 2019

RcppGSL 0.3.7: Fixes and updates

A new release 0.3.7 of RcppGSL is now on CRAN. The RcppGSL package provides an interface from R to the GNU GSL using the Rcpp package.

Stephen Wade noticed that we were not actually freeing memory from the GSL vectors and matrices as we set out to do. And he is quite right: a dormant bug, present since the 0.3.0 release, has now been squashed. I had one boolean wrong, and this has now been corrected. I also took the opportunity to switch the vignette to prebuilt mode: Now a pre-made pdf is just included in a Sweave document, which makes the build more robust to tooling changes around the vignette processing. Lastly, the package was converted to the excellent tinytest unit test framework. Detailed changes below.

Changes in version 0.3.7 (2019-10-20)

  • A logic error was corrected in the wrapper class, vector and matrix memory is now properly free()'ed (Dirk in #22 fixing #20).

  • The introductory vignettes is now premade (Dirk in #23), and was updated lightly in its bibliography handling.

  • The unit tests are now run by tinytest (Dirk in #24).

Courtesy of CRANberries, a summary of changes to the most recent release is also available.

More information is on the RcppGSL page. Questions, comments etc should go to the issue tickets at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 09 Oct 2019

RcppArmadillo 0.9.800.1.0

armadillo image

Another month, another Armadillo upstream release! Hence a new RcppArmadillo release arrived on CRAN earlier today, and was just shipped to Debian as well. It brings a faster solve() method and other goodies. We also switched to the (awesome) tinytest unit test frameowrk, and Min Kim made the configure.ac script more portable for the benefit of NetBSD and other non-bash users; see below for more details. One again we ran two full sets of reverse-depends checks, no issues were found, and the packages was auto-admitted similarly at CRAN after less than two hours despite there being 665 reverse depends. Impressive stuff, so a big Thank You! as always to the CRAN team.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 665 other packages on CRAN.

Changes in RcppArmadillo version 0.9.800.1.0 (2019-10-09)

  • Upgraded to Armadillo release 9.800 (Horizon Scraper)

    • faster solve() in default operation; iterative refinement is no longer applied by default; use solve_opts::refine to explicitly enable refinement

    • faster expmat()

    • faster handling of triangular matrices by rcond()

    • added .front() and .back()

    • added .is_trimatu() and .is_trimatl()

    • added .is_diagmat()

  • The package now uses tinytest for unit tests (Dirk in #269).

  • The configure.ac script is now more careful about shell portability (Min Kim in #270).

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 23 Sep 2019

RcppAnnoy 0.0.13

annoy image

A new release of RcppAnnoy is now on CRAN.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release brings several updates. First and foremost, the upstream Annoy C++ code was updated from version 1.12 to 1.16 bringing both speedier code thanks to AVX512 instruction (where available) and new functionality. Which we expose in two new functions of which buildOnDisk() may be of interest for some using the file-back indices. We also corrected a minor wart in which a demo file was saved (via example()) to a user directory; we now use tempfile() as one should, and contributed two small Windows build changes back to Annoy.

Detailed changes follow below.

Changes in version 0.0.13 (2019-09-23)

  • In example(), the saved and loaded filename is now obtained via tempfile() to not touch user directories per CRAN Policy (Dirk).

  • RcppAnnoy was again synchronized with Annoy upstream leading to enhanced performance and more features (Dirk #48).

  • Minor changes made (and send as PRs upstream) to adapt both annoylib.h and mman.h changes (Dirk).

  • A spurious command was removed from one vignette (Peter Hickey in #49).

  • Two new user-facing functions onDiskBuild() and unbuild() were added (Dirk in #50).

  • Minor tweaks were made to two tinytest-using test files (Dirk).

Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 02 Sep 2019

RcppArmadillo 0.9.700.2.0

armadillo image

A new RcppArmadillo release based on a new Armadillo upstream release arrived on CRAN, and will get to Debian shortly. It brings continued improvements for sparse matrices and a few other things; see below for more details. I also appear to have skipped blogging about the preceding 0.9.600.4.0 release (which was actually extra-rigorous with an unprecedented number of reverse-depends runs) so I included its changes (with very nice sparse matrix improvements) as well.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 656 other packages on CRAN.

Changes in RcppArmadillo version 0.9.700.2.0 (2019-09-01)

  • Upgraded to Armadillo release 9.700.2 (Gangster Democracy)

    • faster handling of cubes by vectorise()

    • faster faster handling of sparse matrices by nonzeros()

    • faster row-wise index_min() and index_max()

    • expanded join_rows() and join_cols() to handle joining up to 4 matrices

    • expanded .save() and .load() to allow storing sparse matrices in CSV format

    • added randperm() to generate a vector with random permutation of a sequence of integers

  • Expanded the list of known good gcc and clang versions in configure.ac

Changes in RcppArmadillo version 0.9.600.4.0 (2019-07-14)

  • Upgraded to Armadillo release 9.600.4 (Napa Invasion)

    • faster handling of sparse submatrices

    • faster handling of sparse diagonal views

    • faster handling of sparse matrices by symmatu() and symmatl()

    • faster handling of sparse matrices by join_cols()

    • expanded clamp() to handle sparse matrices

    • added .clean() to replace elements below a threshold with zeros

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 24 Aug 2019

RcppExamples 0.1.9

A new version of the RcppExamples package is now on CRAN.

The RcppExamples package provides a handful of short examples detailing by concrete working examples how to set up basic R data structures in C++. It also provides a simple example for packaging with Rcpp.

This releases brings a number of small fixes, including two from contributed pull requests (extra thanks for those!), and updates the package in a few spots. The NEWS extract follows:

Changes in RcppExamples version 0.1.9 (2019-08-24)

  • Extended DateExample to use more new Rcpp features

  • Do not print DataFrame result twice (Xikun Han in #3)

  • Missing parenthesis added in man page (Chris Muir in #5)

  • Rewrote StringVectorExample slightly to not run afould the -Wnoexcept-type warning for C++17-related name mangling changes

  • Updated NAMESPACE and RcppExports.cpp to add registration

  • Removed the no-longer-needed #define for new Datetime vectors

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 21 Aug 2019

Rcpp now used by 1750 CRAN packages

1751 Rcpp packages

Since this morning, Rcpp stands at just over 1750 reverse-dependencies on CRAN. The graph on the left depicts the growth of Rcpp usage (as measured by Depends, Imports and LinkingTo, but excluding Suggests) over time.

Rcpp was first released in November 2008. It probably cleared 50 packages around three years later in December 2011, 100 packages in January 2013, 200 packages in April 2014, and 300 packages in November 2014. It passed 400 packages in June 2015 (when I tweeted about it), 500 packages in late October 2015, 600 packages in March 2016, 700 packages last July 2016, 800 packages last October 2016, 900 packages early January 2017,
1000 packages in April 2017, 1250 packages in November 2017, and 1500 packages in November 2018. The chart extends to the very beginning via manually compiled data from CRANberries and checked with crandb. The next part uses manually saved entries. The core (and by far largest) part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of packages using Rcpp is availble too.

Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent July of 2015, seven percent just before Christmas 2015, eight percent last summer, nine percent mid-December 2016, cracked ten percent in the summer of 2017 and eleven percent in 2018. We are currently at 11.83 percent: a little over one in nine packages. There is more detail in the chart: how CRAN seems to be pushing back more and removing more aggressively (which my CRANberries tracks but not in as much detail as it could), how the growth of Rcpp seems to be slowing somewhat outright and even more so as a proportion of CRAN – just like one would expect a growth curve to.

1753 Rcpp packages

1750+ user packages is pretty mind-boggling. We can use the progression of CRAN itself compiled by Henrik in a series of posts and emails to the main development mailing list. Not that long ago CRAN itself did not have 1500 packages, and here we are at almost 14810 with Rcpp at 11.84% and still growing (though maybe more slowly). Amazeballs.

The Rcpp team continues to aim for keeping Rcpp as performant and reliable as it has been. A really big shoutout and Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 19 Aug 2019

RcppQuantuccia 0.0.3

A maintenance release of RcppQuantuccia arrived on CRAN earlier today.

RcppQuantuccia brings the Quantuccia header-only subset / variant of QuantLib to R. At the current stage, it mostly offers date and calendaring functions.

This release was triggered by some work CRAN is doing on updating C++ standards for code in the repository. Notably, under C++11 some constructs such ptr_fun, bind1st, bind2nd, … are now deprecated, and CRAN prefers the code base to not issue such warnings (as e.g. now seen under clang++-9). So we updated the corresponding code in a good dozen or so places to the (more current and compliant) code from QuantLib itself.

We also took this opportunity to significantly reduce the footprint of the sources and the installed shared library of RcppQuantuccia. One (unexported) feature was pricing models via Brownian Bridges based on quasi-random Sobol sequences. But the main source file for these sequences comes in at several megabytes in sizes, and allocates a large number of constants. So in this version the file is excluded, making the current build of RcppQuantuccia lighter in size and more suitable for the (simpler, popular and trusted) calendar functions. We also added a new holiday to the US calendar.

The complete list changes follows.

Changes in version 0.0.3 (2019-08-19)

  • Updated Travis CI test file (#8)).

  • Updated US holiday calendar data with G H Bush funeral date (#9).

  • Updated C++ use to not trigger warnings [CRAN request] (#9).

  • Comment-out pragmas to suppress warnings [CRAN Policy] (#9).

  • Change build to exclude Sobol sequence reducing file size for source and shared library, at the cost of excluding market models (#10).

Courtesy of CRANberries, there is also a diffstat report relative to the previous release. More information is on the RcppQuantuccia page. Issues and bugreports should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 03 Aug 2019

RcppCCTZ 0.2.6

A shiny new release 0.2.6 of RcppCCTZ is now at CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do—using copies in their packages which remains less than ideal.

This version updates to CCTZ release 2.3 from April, plus changes accrued since then. It also switches to tinytest which, among other benefits, permits continued testing of the installed package.

Changes in version 0.2.6 (2019-08-03)

  • Synchronized with upstream CCTZ release 2.3 plus commits accrued since then (Dirk in #30).

  • The package now uses tinytest for unit tests (Dirk in #31).

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 25 Jul 2019

Rcpp 1.0.2: Small Polish

The second maintenance release of Rcpp, following up on the 10th anniversary and the 1.0.0. release, was prepared last Saturday and released to both the Rcpp drat repo and CRAN. Following all the manual inspection (including a false positive result from reverse dependencies), it has finally arrived on CRAN earlier today. The corresponding Debian package was also uploaded, and binaries have since been built.

Just like for Rcpp 1.0.1, we have a four month gap between releases which seems appropriate given both the changes still being made (see below) and the relative stability of Rcpp. It still takes work to release this as we run multiple extensive sets of reverse dependency checks so maybe one day we will switch to six month cycle.

Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 1713 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 176 in BioConductor. Per the (partial) logs of CRAN downloads, we have had over one million downloads a month following the previous release.

This release features a number of different pull requests by four different contributors as detailed below.

Changes in Rcpp version 1.0.2 (2019-07-20)

  • Changes in Rcpp API:

    • Files in src/ are now consistentely lowercase (Dirk in #956).

    • The Rcpp 'API Version' is now accessible via getRcppVersion() (Dirk in #963).

  • Changes in Rcpp Attributes:

    • The second END wrapper macro also gets UNPROTECT and a variable reference suppressing compiler warnings (Dirk in #953 fixing #951).

    • Default function arguments are parsed correctly (Pierrick Roger in #977 fixing #975)

  • Changes in Rcpp Sugar:

    • Added decreasing parameter to sort_unique() (James Balamuta in #958 addressing #950).
  • Changes in Rcpp Deployment:

    • Travis CI unit tests are now always running irrespective of the package version (Dirk in #954).
  • Changes in Rcpp Documentation:

    • The Rcpp-modules vignette now covers the RCPP_EXPOSED_* macros, and the Rcpp-extending vignette references it (Ralf Stubner in #959 fixing #952)

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 25 Jun 2019

RcppTOML 0.1.6: Tinytest support and more robustification

A new RcppTOML release is now on CRAN. RcppTOML brings TOML to R.

TOML is a file format that is most suitable for configurations, as it is meant to be edited by humans but read by computers. It emphasizes strong readability for humans while at the same time supporting strong typing as well as immediate and clear error reports. On small typos you get parse errors, rather than silently corrupted garbage. Much preferable to any and all of XML, JSON or YAML – though sadly these may be too ubiquitous now. TOML has been making inroads with projects such as the Hugo static blog compiler, or the Cargo system of Crates (aka “packages”) for the Rust language.

Václav Hausenblas sent a number of excellent and very focused PRs helping with some input format corner cases, as well as with one test. We added support for the wonderful new tinytest package. The detailed list of changes in this incremental version is below.

Changes in version 0.1.6 (2019-06-25)

  • Propagate the escape switch to calls of getTable() and getArray() (Václav Hausenblas in #32 completing #26). Hausenblas in #36 completing #26)

  • The README.md file now mentions TOML v0.5.0 support (Watal Iwasaki in #35 addressing #33).

  • Encodings in arrays is to UTF-8 for character (Václav Hausenblas in #36 completing #28)

  • The package now use tinytest (Dirk in #38 fixing #37, also Václav in #39).

Courtesy of CRANberries, there is a diffstat report for this release.

More information is on the RcppTOML page page. Issues and bugreports should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 12 Jun 2019

RcppArmadillo 0.9.500.2.0

armadillo image

A new RcppArmadillo release based on a new Armadillo upstream release arrived on CRAN, and will get to Debian shortly. It brings a few upstream changes, including extened interfaces to LAPACK following the recent gcc/gfortran issue. See below for more details.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 610 other packages on CRAN.

Changes in RcppArmadillo version 0.9.500.2.0 (2019-06-11)

  • Upgraded to Armadillo release 9.500.2 (Riot Compact)

    • Expanded solve() with solve_opts::likely_sympd to indicate that the given matrix is likely positive definite

    • more robust automatic detection of positive definite matrices by solve() and inv()

    • faster handling of sparse submatrices

    • expanded eigs_sym() to print a warning if the given matrix is not symmetric

    • extended LAPACK function prototypes to follow Fortran passing conventions for so-called "hidden arguments", in order to address GCC Bug 90329; to use previous LAPACK function prototypes without the "hidden arguments", #define ARMA_DONT_USE_FORTRAN_HIDDEN_ARGS before #include <armadillo>

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 12 May 2019

RcppAnnoy 0.0.12

A new release of RcppAnnoy is now on CRAN.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release brings several updates: Seed settings follow up on changes in the previous release 0.0.11, this is also documented in the vignette thanks to James Melville; more documentation was added thanks to Adam Spannbauer, unit tests now use the brandnew tinytest package, and vignette building was decoupled from package building. All these changes in this version are summarized with appropriate links below:

Changes in version 0.0.12 (2019-05-12)

  • Allow setting of seed (Dirk in #41 fixing #40).

  • Document setSeed (James Melville in #42 documenting #41).

  • Added documentation (Adam Spannbauer in #44 closing #43).

  • Switched unit testing to the new tinytest package (Dirk in #45).

  • The vignette is now pre-made in included as-is in Sweave document reducing the number of suggested packages.

Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 11 May 2019

RcppArmadillo 0.9.400.3.0

armadillo image

The recent 0.9.400.2.0 release of RcppArmadillo required a bug fix release. Conrad follow up on Armadillo 9.400.2 with 9.400.3 – which we packaged (and tested extensively as usual). It is now on CRAN and will get to Debian shortly.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 597 other packages on CRAN.

A brief discussion of possibly issues under 0.9.400.2.0 is at this GitHub issue ticket. The list of changes in 0.9.400.3.0 is below:

Changes in RcppArmadillo version 0.9.400.3.0 (2019-05-09)

  • Upgraded to Armadillo release 9.400.3 (Surrogate Miscreant)

    • check for symmetric / hermitian matrices (used by decomposition functions) has been made more robust

    • linspace() and logspace() now honour requests for generation of vectors with zero elements

    • fix for vectorisation / flattening of complex sparse matrices

The previous changes in 0.9.400.2.0 were:

Changes in RcppArmadillo version 0.9.400.2.0 (2019-04-28)

  • Upgraded to Armadillo release 9.400.2 (Surrogate Miscreant)

    • faster cov() and cor()

    • added .as_col() and .as_row()

    • expanded .shed_rows() / .shed_cols() / .shed_slices() to remove rows/columns/slices specified in a vector

    • expanded vectorise() to handle sparse matrices

    • expanded element-wise versions of max() and min() to handle sparse matrices

    • optimised handling of sparse matrix expressions: sparse % (sparse +- scalar) and sparse / (sparse +- scalar)

    • expanded eig_sym(), chol(), expmat_sym(), logmat_sympd(), sqrtmat_sympd(), inv_sympd() to print a warning if the given matrix is not symmetric

    • more consistent detection of vector expressions

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 29 Apr 2019

RcppArmadillo 0.9.400.2.0

armadillo image

A new RcppArmadillo release based on the very recent Armadillo upstream release arrived on CRAN earlier today, and will get to Debian shortly.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 587 other packages on CRAN.

The (upstream-only again this time) changes are listed below:

  • Upgraded to Armadillo release 9.400.2 (Surrogate Miscreant)

    • faster cov() and cor()

    • added .as_col() and .as_row()

    • expanded .shed_rows() / .shed_cols() / .shed_slices() to remove rows/columns/slices specified in a vector

    • expanded vectorise() to handle sparse matrices

    • expanded element-wise versions of max() and min() to handle sparse matrices

    • optimised handling of sparse matrix expressions: sparse % (sparse +- scalar) and sparse / (sparse +- scalar)

    • expanded eig_sym(), chol(), expmat_sym(), logmat_sympd(), sqrtmat_sympd(), inv_sympd() to print a warning if the given matrix is not symmetric

    • more consistent detection of vector expressions

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 22 Mar 2019

RcppArmadillo 0.9.300.2.0

armadillo image

A new RcppArmadillo release based on a new Armadillo upstream release arrived on CRAN and Debian today.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 583 other packages on CRAN.

The (upstream-only this time) changes are listed below:

Changes in RcppArmadillo version 0.9.300.2.0 (2019-03-21)

  • Upgraded to Armadillo release 9.300.2 (Fomo Spiral)

    • Faster handling of compound complex matrix expressions by trace()

    • More efficient handling of element access for inplace modifications in sparse matrices

    • Added .is_sympd() to check whether a matrix is symmetric/hermitian positive definite

    • Added interp2() for 2D data interpolation

    • Added expm1() and log1p()

    • Expanded .is_sorted() with options "strictascend" and "strictdescend"

    • Expanded eig_gen() to optionally perform balancing prior to decomposition

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 17 Mar 2019

Rcpp 1.0.1: Updates

Following up on the 10th anniversary and the 1.0.0. release, we excited to share the news of the first update release 1.0.1 of Rcpp. package turned ten on Monday—and we used to opportunity to mark the current version as 1.0.0! It arrived at CRAN overnight, Windows binaries have already been built and I will follow up shortly with the Debian binary.

We had four years of regular bi-monthly release leading up to 1.0.0, and having now taken four months since the big 1.0.0 one. Maybe three (or even just two) releases a year will establish itself a natural cadence. Time will tell.

Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 1598 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 152 in BioConductor release 3.8. Per the (partial) logs of CRAN downloads, we currently average 921,000 downloads a month.

This release feature a number of different pull requests detailed below.

Changes in Rcpp version 1.0.1 (2019-03-17)

  • Changes in Rcpp API:

    • Subsetting is no longer limited by an integer range (William Nolan in #920 fixing #919).

    • Error messages from subsetting are now more informative (Qiang and Dirk).

    • Shelter increases count only on non-null objects (Dirk in #940 as suggested by Stepan Sindelar in #935).

    • AttributeProxy::set() and a few related setters get Shield<> to ensure rchk is happy (Romain in #947 fixing #946).

  • Changes in Rcpp Attributes:

    • A new plugin was added for C++20 (Dirk in #927)

    • Fixed an issue where 'stale' symbols could become registered in RcppExports.cpp, leading to linker errors and other related issues (Kevin in #939 fixing #733 and #934).

    • The wrapper macro gets an UNPROTECT to ensure rchk is happy (Romain in #949) fixing #948).

  • Changes in Rcpp Documentation:

    • Three small corrections were added in the 'Rcpp Quickref' vignette (Zhuoer Dong in #933 fixing #932).

    • The Rcpp-modules vignette now has documentation for .factory (Ralf Stubner in #938 fixing #937).

  • Changes in Rcpp Deployment:

    • Travis CI again reports to CodeCov.io (Dirk and Ralf Stubner in #942 fixing #941).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 08 Mar 2019

RcppArmadillo 0.9.200.7.1

armadillo image

A minor RcppArmadillo bugfix release arrived on CRAN today. This version 0.9.200.7.1 has two local changes. R 3.6.0 will bring a change in sample() (to correct a subtle bug for large samples) meaning many tests will fail, so in one unit test file we reset the generator to the old behaviour to ensure we match the (old) test expectation. We also backported a prompt upstream fix for an issue with drawing Wishart-distributed random numbers via Armadillo which was uncovered this week. I also just uploaded the Debian version.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 559 other packages on CRAN.

Changed are listed below:

Changes in RcppArmadillo version 0.9.200.7.1 (2019-03-08)

  • Explicit setting of RNGversion("3.5.0") in one unit test to accomodate the change in sample() in R 3.6.0

  • Back-ported a fix to the Wishart RNG from upstream (Dirk in #248 fixing #247)

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 25 Feb 2019

RcppStreams 0.1.3: Keeping CRAN happy

Streamulus

Not unlike the Rblpapi release on Thursday and the RVowpalWabbit release on Friday (both of which dealt with the upcoming staged install), we now have another CRAN-requested maintenance release. This time it is RcppStreams which got onto CRAN as of early this morning. RcppStreams brings the excellent Streamulus C++ template library for event stream processing to R.

Streamulus, written by Irit Katriel, uses very clever template meta-programming (via Boost Fusion) to implement an embedded domain-specific event language created specifically for event stream processing.

This release provides suppresses issue reported by the UBSAN detector set up at Oxford; I simply no longer run the examples that triggered it as the errors came from very deep down inside Boost Proto and Fusion. As a more positive side effect, I updated the Rocker R-Devel SAN/UBSAN Clang image and corresponding Docker container. So if you need SAN/UBSAN detection, that container may become your friend.

The NEWS file entries follows below:

Changes in version 0.1.3 (2019-02-24)

  • No longer run examples as they upset the UBSAN checks at CRAN

Courtesy of CRANberries, there is also a copy of the DESCRIPTION file for this initial release. More detailed information is on the RcppStreams page page and of course on the Streamulus page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 17 Jan 2019

RcppArmadillo 0.9.200.7.0

armadillo image

A new RcppArmadillo bugfix release arrived at CRAN today. The version 0.9.200.7.0 is another minor bugfix release, and based on the new Armadillo bugfix release 9.200.7 from earlier this week. I also just uploaded the Debian version, and Uwe’s systems have already create the CRAN Windows binary.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 559 other packages on CRAN.

This release just brings minor upstream bug fixes, see below for details (and we also include the updated entry for the November bugfix release).

Changes in RcppArmadillo version 0.9.200.7.0 (2019-01-17)

  • Upgraded to Armadillo release 9.200.7 (Carpe Noctem)

  • Fixes in 9.200.7 compared to 9.200.5:

    • handling complex compound expressions by trace()

    • handling .rows() and .cols() by the Cube class

Changes in RcppArmadillo version 0.9.200.5.0 (2018-11-09)

  • Upgraded to Armadillo release 9.200.5 (Carpe Noctem)

  • Changes in this release

    • linking issue when using fixed size matrices and vectors

    • faster handling of common cases by princomp()

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 06 Jan 2019

RcppStreams 0.1.2

Streamulus

A maintenance release of RcppStreams arrived on CRAN this afternoon. RcppStreams brings the excellent Streamulus C++ template library for event stream processing to R.

Streamulus, written by Irit Katriel, uses very clever template meta-programming (via Boost Fusion) to implement an embedded domain-specific event language created specifically for event stream processing.

This release provides a minor update, motivated by the upcoming BH release 1.69 for which we made a pre-release – and RcppStreams is one of three packages identified by that pre-release as needed minor accomodations for the new Boost release. We also updated packaging to current CRAN standards, but made no other changes.

The NEWS file entries follows below:

Changes in version 0.1.2 (2019-01-05)

  • Added symbol registration

  • Converted a few files from CRLF to CR [CRAN checks]

  • Added a few final newlines [CRAN checks]

  • Disabled one optional output to enable BH 1.69 builds

Courtesy of CRANberries, there is also a copy of the DESCRIPTION file for this initial release. More detailed information is on the RcppStreams page page and of course on the Streamulus page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 28 Nov 2018

RcppArmadillo 0.9.200.5.0

armadillo image

A new RcppArmadillo release arrived at CRAN overnight. The version 0.9.200.5.0 is a minor upgrade and based on the new Armadillo bugfix release 9.200.5 from yesterday. I also just uploaded the Debian version.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 539 other packages on CRAN.

This release just brings one upstream bug fix, see below for details.

Changes in RcppArmadillo version 0.9.200.5.0 (2018-11-09)

  • Upgraded to Armadillo release 9.200.5 (Carpe Noctem)

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 24 Nov 2018

RcppEigen 0.3.3.5.0

Another minor release 0.3.3.5.0 of RcppEigen arrived on CRAN today (and just went to Debian too) bringing support for Eigen 3.3.5 to R.

As we now carry our small set of patches to Eigen as diff in our repo, it was fairly straightforward to bring these few changes to the new upstream version. I added one trivial fix of changing a return value to void as this is also already in the upstream repo. Other than that, we were fortunate to get two nice and focussed PRs since the last release. Ralf allowed us to use larger index values by using R_xlen_t, and Michael corrected use of RcppArmadillo in a benchmarking example script.

Next, it bears repeating what we said in February when we release 0.3.3.4.0:

One additional and recent change was the accomodation of a recent CRAN Policy change to not allow gcc or clang to mess with diagnostic messages. A word of caution: this may make your compilation of packages uses RcppEigen very noisy so consider adding -Wno-ignored-attributes to the compiler flags added in your ~/.R/Makevars.

It’s still super-noise, but hey, CRAN made us do it …

The complete NEWS file entry follows.

Changes in RcppEigen version 0.3.3.5.0 (2018-11-24)

  • Updated to version 3.3.5 of Eigen (Dirk in #65)

  • Long vectors are now supported via R_xlen_t (Ralf Stubner in #55 fixing #54).

  • The benchmarking example was updated in its use of RcppArmadillo (Michael Weylandt in #56).

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 18 Nov 2018

RcppMsgPack 0.2.3

Another maintenance release of RcppMsgPack got onto CRAN today. Two new helper functions were added and not unlike the previous 0.2.2 release in, some additional changes are internal and should allow compilation on all CRAN systems.

MessagePack itself is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it is faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves. RcppMsgPack brings both the C++ headers of MessagePack as well as clever code (in both R and C++) Travers wrote to access MsgPack-encoded objects directly from R.

Changes in version 0.2.3 (2018-11-18)

  • New functions msgpack_read and msgpack_write for efficient direct access to MessagePackage content from files (#13).

  • Several internal code polishes to smooth compilation (#14 and #15).

Courtesy of CRANberries, there is also a diffstat report for this release.

More information is on the RcppRedis page. Issues and bugreports should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 16 Nov 2018

RcppGetconf 0.0.3

A second and minor update for the RcppGetconf package for reading system configuration — not unlike getconf from the libc library — is now on CRAN.

Changes are minor. We avoid an error on a long-dead operating system cherished in one particular corner of the CRAN world. In doing so some files were updated so that dynamically loaded routines are now registered too.

The short list of changes in this release follows:

Changes in inline version 0.0.3 (2018-11-16)

  • Examples no longer run on Solaris where they appear to fail.

Courtesy of CRANberries, there is a diffstat report. More about the package is at the local RcppGetconf page and the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 15 Nov 2018

Rcpp now used by 1500 CRAN packages

1500 Rcpp packages

Right now Rcpp stands at 1500 reverse-dependencies on CRAN. The graph is on the left depicts the growth of Rcpp usage (as measured by Depends, Imports and LinkingTo, but excluding Suggests) over time. What an amazing few days this has been as we also just marked the tenth anniversary and the big one dot oh release.

Rcpp cleared 300 packages in November 2014. It passed 400 packages in June 2015 (when I only tweeted about it), 500 packages in late October 2015, 600 packages in March 2016, 700 packages last July 2016, 800 packages last October 2016, 900 packages early January 2017,
1000 packages in April 2017, and 1250 packages in November 2018. The chart extends to the very beginning via manually compiled data from CRANberries and checked with crandb. The next part uses manually saved entries. The core (and by far largest) part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of packages using Rcpp is kept on this page.

Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent July of 2015, seven percent just before Christmas 2015, eight percent last summer, nine percent mid-December 2016, cracked ten percent in the summer of 2017 and eleven percent this year. We are currently at 11.199 percent or just over one in nine packages. There is more detail in the chart: how CRAN seems to be pushing back more and removing more aggressively (which my CRANberries tracks but not in as much detail as it could), how the growth of Rcpp seems to be slowing somewhat outright and even more so as a proportion of CRAN – just like every growth curve should, eventually. But we leave all that for another time.

1500 Rcpp packages

1500 user packages is pretty mind-boggling. We can use the progression of CRAN itself compiled by Henrik in a series of posts and emails to the main development mailing list. Not that long ago CRAN itself did not have 1500 packages, and here we are at almost 13400 with Rcpp at 11.2% and still growing (albeit slightly more slowly). Amazeballs.

This puts a whole lot of responsibility on us in the Rcpp team as we continue to keep Rcpp as performant and reliable as it has been.

And with that, and as always, a very big Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 10 Nov 2018

RcppArmadillo 0.9.200.4.0

armadillo image

A new RcppArmadillo release, now at 0.9.200.4.0, based on the new Armadillo release 9.200.4 from earlier this week, is now on CRAN, and should get to Debian very soon.

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 532 (or 31 more since just the last release!) other packages on CRAN.

This release once again brings a number of improvements, see below for details.

Changes in RcppArmadillo version 0.9.200.4.0 (2018-11-09)

  • Upgraded to Armadillo release 9.200.4 (Carpe Noctem)

    • faster handling of symmetric positive definite matrices by rcond()

    • faster transpose of matrices with size ≥ 512x512

    • faster handling of compound sparse matrix expressions by accu(), diagmat(), trace()

    • faster handling of sparse matrices by join_rows()

    • expanded sign() to handle scalar arguments

    • expanded operators (*, %, +, ) to handle sparse matrices with differing element types (eg. multiplication of complex matrix by real matrix)

    • expanded conv_to() to allow conversion between sparse matrices with differing element types

    • expanded solve() to optionally allow keeping solutions of systems singular to working precision

    • workaround for gcc and clang bug in C++17 mode

  • Commented-out sparse matrix test consistently failing on the fedora-clang machine CRAN, and only there. No fix without access.

  • The 'Unit test' vignette is no longer included.

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 07 Nov 2018

Rcpp 1.0.0: The Tenth Birthday Release

As mentioned here two days ago, the Rcpp package turned ten on Monday—and we used to opportunity to mark the current version as 1.0.0! Thanks to everybody who liked and retweeted our tweet about this. And of course, once more a really big Thank You! to everybody who helped along this journey: Rcpp Core team, contributors, bug reporters, workshop and tutorial attendees and last but not least all those users – we did well. So let’s enjoy and celebrate this moment.

As indicated in Monday’s blog post, we had also planned to upload this version to CRAN, and this 1.0.0 release arrived on CRAN after the customary inspection and is now available. I will build the Debian package in a moment, it will find its way to Ubuntu and of the CRAN-mirrored backport that Michael looks after so well.

While this release is of course marked as 1.0.0 signifying the feature and release stability we have had for some time, it also marks another regular release at the now-common bi-monthly schedule following nineteen releases since July 2016 in the 0.12.* series as well as another five in the preceding 0.11.* series.

Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 1493 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with another 150 in the (very recent) BioConductor release 3.8. Per the (partial) logs of CRAN downloads, we were reaching more than 900,000 downloads a month of late.

Once again, we have a number of nice pull requests from the usual gang of contributors in there, see below for details.

Changes in Rcpp version 1.0.0 (2018-11-05)

  • Happy tenth birthday to Rcpp, and hello release 1.0 !

  • Changes in Rcpp API:

    • The empty destructor for the Date class was removed to please g++-9 (prerelease) and -Wdeprecated-copy (Dirk).

    • The constructor for NumericMatrix(not_init(n,k)) was corrected (Romain in #904, Dirk in #905, and also Romain in #908 fixing #907).

    • Rcpp::String no longer silently drops embedded NUL bytes in strings but throws new Rcpp exception embedded_nul_in_string. (Kevin in #917 fixing #916).

  • Changes in Rcpp Deployment:

    • The Dockerfile for Continuous Integration sets the required test flag (for release versions) inside the container (Dirk).

    • Correct the R CMD check call to skip vignettes (Dirk).

  • Changes in Rcpp Attributes:

    • A new [[Rcpp::init]] attribute allows function registration for running on package initialization (JJ in #903).

    • Sort the files scanned for attributes in the C locale for stable output across systems (JJ in #912).

  • Changes in Rcpp Documentation:

    • The 'Rcpp Extending' vignette was corrected and refers to EXPOSED rather than EXPORTED (Ralf Stubner in #910).

    • The 'Unit test' vignette is no longer included (Dirk in #914).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 05 Nov 2018

Happy 10th Bday, Rcpp – and welcome release 1.0 !!

Ten years ago today I wrote the NEWS.Rd entry in this screenshot for the very first Rcpp_release:

First Rcpp release
First Rcpp release

So Happy Tenth Birthday, Rcpp !! It has been quite a ride. Nearly 1500 packages on CRAN, or about one in nine (!!), rely on Rcpp to marshall data between R and C++, and to extend R with performant C++ code.

Rcpp would not exist without Dominick Samperi who recognised early just how well C++ template classes could fit the task of seamlessly interchanging data between the two systems, and contributed the first versions to my RQuantLib package. After a few versions he left this and R around 2006. About two years later, as I needed this for what I was working on, I rejuvenated things with the initial “second wave” release whose 10th birthday we celebrate today.

In late 2009 Romain François joined, intially contacted because of the needs of RProtoBuf for some Java tooling. (As an aside, there is a neat story of the very first steps of RProtoBuf here.) And from late 2009 to some time in 2013 or so Romain pushed Rcpp incredibly hard, far and well. We all still benefit from the work he did, and probably will for some time.

These days Rcpp is driven by a small and dedicated core team: JJ Allaire, Kevin Ushey, Qiang Kou, Nathan Russell, Doug Bates, John Chambers and myself – plus countless contributors and even more users via GitHub and the mailing list. Thanks to all for making something excellent together!

And because ten is written as one-oh, I felt the time was right to go to the big one oh release and just committed this:

Release 1.0
Release 1.0

I will probably wrap this up as a release in a day or two and sent it to CRAN where it may remain for a few days of testing and checking. If you want it sooner, try the GitHub repo or the Rcpp drat repo where I will push the release once completed.

Again, a very big cheers and heartfelt Thank You! to everybody who helped on this ten year journey of making Rcpp what it is today.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 01 Nov 2018

RcppAnnoy 0.0.11

A new release of RcppAnnoy is now on CRAN.

RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release updates to a new upstream version (including a new distance measure), and includes a spiffy new vignette by Aaron Lun describing how to use who Annoy from C++ as he does in his new BioConductor package BiocNeighbours.

All changes in this version are summarized below:

Changes in version 0.0.11 (2018-10-30)

  • Synchronized with Annoy upstream (#26, #30, #36).

  • Added new Hamming distance measure functionality; should be considered experimental as the functionality depends on integer values.

  • Travis CI use was updated to the R 3.5 PPA (#28)

  • New vignette about Annoy use from C++ via Rcpp (Aaron Lun in #29 addressing #19; also #32, #33)

  • The vignette was rewritten using pinp (#34, #35).

Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link