Fri, 19 Dec 2014

Rocker is now the official R image for Docker

big deal

Something happened a little while ago which we did not have time to commensurate properly. Our Rocker image for R is now the official R image for Docker itself. So getting R (via Docker) is now as simple as saying docker pull r-base.

This particular container is essentially just the standard r-base Debian package for R (which is one of a few I maintain there) plus a mininal set of extras. This r-base forms the basis of our other containers as e.g. the rather popular r-studio container wrapping the excellent RStudio Server.

A lot of work went into this. Carl and I also got a tremendous amount of help from the good folks at Docker. Details are as always at the Rocker repo at GitHub.

Docker itself continues to make great strides, and it has been great fun help to help along. With this post I achieved another goal: blog about Docker with an image not containing shipping containers. Just kidding.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rocker | permanent link

Sat, 13 Dec 2014

rfoaas 0.0.4.20141212

A new version of rfoaas is now on CRAN. The rfoaas package provides an interface for R to the most excellent FOAAS service -- which provides a modern, scalable and RESTful web service for the frequent need to tell someone to eff off.

The FOAAS backend gets updated in spurts, and yesterday a few pull requests were integrated, including one from yours truly. So with that it was time for an update to rfoaas. As the version number upstream did not change (bad, bad, practice) I appended the date the version number.

CRANberries also provides a diff to the previous release. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rfoaas | permanent link

Thu, 11 Dec 2014

RProtoBuf 0.4.2

A new release 0.4.2 of RProtoBuf is now on CRAN. RProtoBuf provides R bindings for the Google Protocol Buffers ("Protobuf") data encoding library used and released by Google, and deployed as a language and operating-system agnostic protocol by numerous projects.

Murray and Jeroen did almost all of the heavy lifting. Many changes were triggered by two helpful referee reports, and we are slowly getting to the point where we will resubmit a much improved paper. Full details are below.

Changes in RProtoBuf version 0.4.2 (2014-12-10)

  • Address changes suggested by anonymous reviewers for our Journal of Statistical Software submission.

  • Make Descriptor and EnumDescriptor objects subsettable with "[[".

  • Add length() method for Descriptor objects.

  • Add names() method for Message, Descriptor, and EnumDescriptor objects.

  • Clarify order of returned list for descriptor objects in as.list documentation.

  • Correct the definition of as.list for EnumDescriptors to return a proper list instead of a named vector.

  • Update the default print methods to use cat() with fill=TRUE instead of show() to eliminate the confusing [1] since the classes in RProtoBuf are not vectorized.

  • Add support for serializing function, language, and environment objects by falling back to R's native serialization with serialize_pb and unserialize_pb to make it easy to serialize into a Protocol Buffer all of the more than 100 datasets which come with R.

  • Use normalizePath instead of creating a temporary file with file.create when getting absolute path names.

  • Add unit tests for all of the above.

CRANberries also provides a diff to the previous release. RProtoBuf page which has a draft package vignette, a a 'quick' overview vignette, and a unit test summary vignette. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rprotobuf | permanent link

Wed, 10 Dec 2014

digest 0.6.6 (and 0.6.5)

A new release 0.6.6 of the digest package is now on CRAN and in Debian.

This release brings the xxHash non-cryptographic hash function by Yann Collet, thanks to several pull requests by Jim Hester. After the upload of version 0.6.5 we uncovered another lovely non-standardness of Windoze: you cannot format unsigned long long via printf() format strings. Great. Luckily Jim found a quick (and portable) fix via the inttypes.h header, and that went into the 0.6.6 release.

The release also contains an earlier extension for hmac() to also cover crc32 hashes, kindly provided by Suchen Jin.

I also made a number of small internal changes such as

  • switching (compiled) function registration to package load via a the useDynLib() declaration in NAMESPACE,
  • (finally!!) formating code to proper four-space indentation,
  • adding some documentation around Jim's pull request, and
  • adding a few GPL copyright headers.

CRANberries provides the usual summary of changes to the previous version.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/digest | permanent link

RcppRedis 0.1.3

A very minor bugfix release of RcppRedis is now on CRAN. The zcount function now returns the correct type.

Changes in version 0.1.3 (2014-12-10)

  • Bug fix setting correct return type of zcount

Courtesy of CRANberries, there is also a diffstat report for the most recent release. More information is on the RcppRedis page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 09 Dec 2014

Wilco!!

Wilco at The Riviera

With a bit of luck due to a collegue having a spare ticket, I managed to make it to an awesome Wilco show at The Riviera in Uptown.

This concert was part of a set a six shows. Tweedy and the band were fast, and loose, and wonderful, and totally beloved by the home crowd. An truly outstanding show, and a great evening.

Also: I should get out more often. Last blog entry about Wilco was from 2005. Ouch.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/music/rock | permanent link

RcppAnnoy 0.0.4

A few weeks ago, RcppAnnoy had its initial release 0.0.2 and subsequent update in release 0.0.3. The latter brought Windows support, thanks to a neat pull request by Qiang Kou.

RcppAnnoy wraps the small, fast, and lightweight C++ template header library Annoy written by Erik Bernhardsson for use at Spotify. RcppAnnoy uses Rcpp Modules to offer the exact same functionality as the Python module wrapped around Annoy.

In the 0.0.3 release, I overlooked one thing: that with builds on Windows, we would also get builds against what CRAN calls R-oldrel: the previous release, which cannot turn on C++11 via the simple CXX_STD = CXX11 declaration in src/Makevars (and which we need because use of Boost brings in long long which R can only cope with under C++11 ...).

So this new release 0.0.4 does nothing more than add a constraint in a Depends: R (>= 3.1.0) to avoid builds not being able to turn on C++11.

Courtesy of CRANberries, there is also a diffstat report for this release. More detailed information is on the RcppAnnoy page page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 02 Dec 2014

RQuantLib 0.4.0

A new major release of RQuantLib is now on CRAN and getting to Debian.

All C++ source files have been rewritten to take advantage of newer Rcpp features. Several Fixed Income functions have been added, or refreshed, thanks to contributions by Michele Salvadore. Calendar support was greatly expanded thanks to a contribution by Danilo Dias da Silva. All key changes are listed in detail below.

Changes in RQuantLib version 0.4.0 (2014-12-01)

  • Changes in RQuantLib code:

    • All function interfaces have been rewritten using Rcpp Attributes. No SEXP remain in the function signatures. This make the code shorter, more readable and more easily extensible.

    • The header files have been reorganized so that plugin use is possible. An impl.h files is imported once for each compilation unit: for RQuantLib from the file src/dates.cpp directory, from a sourced file via a #define set by the plugin wrapper.

    • as<>() and wrap() converters have added for QuantLib Date types.

    • Plugin support has been added, allowing more ad-hoc use via Rcpp Attributes.

    • Several Fixed Income functions have been added, and/or rewritten to better match the QuantLib signatures; this was done mostly by Michele Salvadore.

    • Several Date and Calendar functions have been added.

    • Calendar support has been greatly expanded thanks to Danilo Dias da Silva.

    • Exported curve objects are now more parsimonious and advance entries in the table object roughly one business month at a time.

    • The DiscountCurve and Bond curve construction has been fixed via a corrected evaluation date and omitted the two-year swap rate, as suggested by Luigi Ballabio.

    • The NAMESPACE file has a tighter rule for export of *.default functions, as suggested by Bill Dunlap

    • Builds now use OpenMP where available.

    • The package now depends on QuantLib 1.4.0 or later.

  • Changes in RQuantLib tests:

    • New unit tests for dates have been added.

    • C++ code for the unit tests has also been converted to Rcpp Attributes use; a helper function unitTestSetup() has been added.

    • Continuous Integration via Travis is now enabled from the GitHub repo.

  • Changes in RQuantLib documentation:

    • This NEWS file has been added. Better late than never, as they say.

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list off the R-Forge page. Issue tickets can be filed at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rquantlib | permanent link

Sat, 29 Nov 2014

RcppArmadillo 0.4.550.1.0

A week ago, Conrad provided another minor release 4.550.0 of Armadillo which has since received one minor correction in 4.550.1.0. As before, I had created a GitHub-only pre-release of his pre-release which was tested against the almost one hundred CRAN dependents of our RcppArmadillo package. This passed fine as usual, and results are as always in the rcpp-logs repository.

Processing and acceptance at the CRAN took a little longer as around the same time a fresh failure in unit tests had become apparent on an as-of-yet unannounced new architecture (!!) also tested at CRAN. The R-devel release has since gotten a new capabilities() test for long double, and we now only run this test (for our rmultinom()) if the test asserts that the given R build has this capability. Phew, so with all that the new version in now on CRAN; Windows binaries have been built and I also uploaded new Debian binaries.

Changes are summarized below; our end also includes added support for conversion of Field types takes to short pull request by Romain.

Changes in RcppArmadillo version 0.4.550.1.0 (2014-11-26)

  • Upgraded to Armadillo release Version 4.550.1 ("Singapore Sling Deluxe")

    • added matrix exponential function: expmat()

    • faster .log_p() and .avg_log_p() functions in the gmm_diag class when compiling with OpenMP enabled

    • faster handling of in-place addition/subtraction of expressions with an outer product

    • applied correction to gmm_diag relative to the 4.550 release

  • The Armadillo Field type is now converted in as<> conversions

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 28 Nov 2014

CRAN Task Views for Finance and HPC now (also) on GitHub

The CRAN Task View system is a fine project which Achim Zeileis initiated almost a decade ago. It is described in a short R Journal article in Volume 5, Number 1. I have been editor / maintainer of the Finance task view essentially since the very beginning of these CRAN Task Views, and added the High-Performance Computing one in the fall of 2008. Many, many people have helped by sending suggestions or even patches; email continues to be the main venue for the changes.

The maintainers of the Web Technologies task view were, at least as far as I know, the first to make the jump to maintaining the task view on GitHub. Karthik and I briefly talked about this when he was in town a few weeks ago for our joint Software Carpentry workshop at Northwestern.

So the topic had been on my mind, but it was only today that I realized that the near-limitless amount of awesome that is pandoc can probably help with maintenance. The task view code by Achim neatly converts the very regular, very XML, very boring original format into somewhat-CRAN-website-specific html. Pandoc, being as versatile as it is, can then make (GitHub-flavoured) markdown out of this, and with a minimal amount of sed magic, we get what we need.

And hence we now have these two new repos:

Contributions are now most welcome by pull request. You can run the included converter scripts, it differs between both repos only by one constant for the task view / file name. As an illustration, the one for Finance is below.

#!/usr/bin/r
## if you do not have /usr/bin/r from littler, just use Rscript

ctv <- "Finance"

ctvfile  <- paste0(ctv, ".ctv")
htmlfile <- paste0(ctv, ".html")
mdfile   <- "README.md"

## load packages
suppressMessages(library(XML))          # called by ctv
suppressMessages(library(ctv))

r <- getOption("repos")                 # set CRAN mirror
r["CRAN"] <- "http://cran.rstudio.com"
options(repos=r)

check_ctv_packages(ctvfile)             # run the check

## create html file from ctv file
ctv2html(read.ctv(ctvfile), htmlfile)

### these look atrocious, but are pretty straight forward. read them one by one
###  - start from the htmlfile
cmd <- paste0("cat ", htmlfile,
###  - in lines of the form  ^<a href="Word">Word.html</a>
###  - capture the 'Word' and insert it into a larger URL containing an absolute reference to task view 'Word'
  " | sed -e 's|^<a href=\"\\([a-zA-Z]*\\)\\.html|<a href=\"http://cran.rstudio.com/web/views/\\1.html\"|' | ",
###  - call pandoc, specifying html as input and github-flavoured markdown as output
              "pandoc -s -r html -w markdown_github | ",
###  - deal with the header by removing extra ||, replacing |** with ** and **| with **:              
              "sed -e's/||//g' -e's/|\\*\\*/\\*\\*/g' -e's/\\*\\*|/\\*\\* /g' -e's/|$/  /g' ",
###  - make the implicit URL to packages explicit
              "-e's|../packages/|http://cran.rstudio.com/web/packages/|g' ",
###  - write out mdfile
              "> ", mdfile)

system(cmd)                             # run the conversion

unlink(htmlfile)                        # remove temporary html file

cat("Done.\n")

I am quite pleased with this setup---so a quick thanks towards the maintainers of the Web Technologies task view; of course to Achim for creating CRAN Task Views in the first place, and maintaining them all those years; as always to John MacFarlance for the magic that is pandoc; and last but not least of course to anybody who has contributed to the CRAN Task Views.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/snippets | permanent link

Tue, 25 Nov 2014

Rcpp now used by 300 CRAN packages

max-heap image

This morning, Rcpp reached another round milestone: 300 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations). The graph is on the left depicts the growth of Rcpp usage over time. There are 41 more on BioConductor (which is not included in the chart).

The first and less detailed part uses manually save entries, the second half of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of user package is kept on this page.

Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We may well hit five per-cent before the end of the year.

300 is a pretty humbling and staggering number. Also interesting that we we cleared 200 only at the end of April, and 250 in early August.

So from everybody behind Rcpp, a heartfelt Thank You! to all the users and of course other contributors.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 24 Nov 2014

YATORP -- Yet Another Tutorial on R Packaging

What the world needs right now is yet another tutorial on R packages and their creation. Luckily, this last Friday and Saturday, I had the opportunity to present in a workshop organized by Frank DiTraglia at Penn's shiny new Warren Center, and held at Wharton.

Given the Warren Center's focus, the workshop centered around Big Data and Open Science with R. Yihui Xie and myself alternated on delivering four units on an Introduction to R, Writing R packages, Dynamic Documents with R, and HPC with Rcpp and RcppArmadillo.

So I had to come up with a plan for teaching creating R packages -- and decided to do it from the very bottom up, clearly introducing the underlying R CMD ... commands and only then switching to taking advantage of an environment such as the RStudio IDE.

The resulting slides are now available on my presentations page. The code examples are in a repo subdirectory on GitHub as well. While both were designed to support the parallel live instruction offered in the workshop, I would be interested in feedback (preferably via email) about how useful the slides are by themselves.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/computers/R | permanent link

Tue, 18 Nov 2014

R / Finance 2015 Call for Papers

Earlier today, Josh send the text below to the R-SIG-Finance list, and I updated the R/Finance website, including its Call for Papers page, accordingly.

We are once again very excited about our conference, thrilled about the four confirmed keynotes, and hope that many R / Finance users will not only join us in Chicago in May 2015 -- but also submit an exciting proposal.

So read on below, and see you in Chicago in May!

Call for Papers:

R/Finance 2015: Applied Finance with R
May 29 and 30, 2015
University of Illinois at Chicago, IL, USA

The seventh annual R/Finance conference for applied finance using R will be held on May 29 and 30, 2015 in Chicago, IL, USA at the University of Illinois at Chicago. The conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading.

Over the past six years, R/Finance has included attendees from around the world. It has featured presentations from prominent academics and practitioners, and we anticipate another exciting line-up for 2015. This year will include invited keynote presentations by Emanuel Derman, Louis Marascio, Alexander McNeil, and Rishi Narang.

We invite you to submit complete papers in pdf format for consideration. We will also consider one-page abstracts (in txt or pdf format) although more complete papers are preferred. We welcome submissions for both full talks and abbreviated "lightning talks." Both academic and practitioner proposals related to R are encouraged.

All slides will be made publicly available at conference time. Presenters are strongly encouraged to provide working R code to accompany the slides. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages.

The conference will award two (or more) $1000 prizes for best papers. A submission must be a full paper to be eligible for a best paper award. Extended abstracts, even if a full paper is provided by conference time, are not eligible for a best paper award. Financial assistance for travel and accommodation may be available to presenters, however requests must be made at the time of submission. Assistance will be granted at the discretion of the conference committee.

Please make your submission online at this link. The submission deadline is January 31, 2015. Submitters will be notified via email by February 28, 2015 of acceptance, presentation length, and financial assistance (if requested).

Additional details will be announced via the R/Finance conference website as they become available. Information on previous years' presenters and their presentations are also at the conference website.

For the program committee:

Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson, Dale Rosenthal,
Jeffrey Ryan, Joshua Ulrich

/computers/R | permanent link

RcppAnnoy 0.0.3

Hours after the initial blog post announcing the first release of the new package RcppAnnoy, Qiang Kou sent us a very nice pull request adding mmap support in Windows.

So a new release with Windows support is on now CRAN, and Windows binaries should be available by this evening as usual.

To recap, RcppAnnoy wraps the small, fast, and lightweight C++ template header library Annoy written by Erik Bernhardsson for use at Spotify. RcppAnnoy uses Rcpp Modules to offer the exact same functionality as the Python module wrapped around Annoy.

Courtesy of CRANberries, there is also a diffstat report for this release. More detailed information is on the RcppAnnoy page page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 16 Nov 2014

Introducing RcppAnnoy

A new package RcppAnnoy is now on CRAN.

It wraps the small, fast, and lightweight C++ template header library Annoy written by Erik Bernhardsson for use at Spotify.

While Annoy is setup for use by Python, RcppAnnoy offers the exact same functionality from R via Rcpp.

A new page for RcppAnnoy provides some more detail, example code and further links. See a recent blog post by Erik for a performance comparison of different approximate nearest neighbours libraries for Python.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 10 Nov 2014

BH release 1.54.0-5

A new release of BH, our package providing Boost headers for use by R is now on CRAN. This release was triggered via a request by Ben Goodrich as noted below who asked for Boost Circular Buffer which RStan uses.

No other changes were made.

Changes in version 1.54.0-5 (2014-11-09)

  • Added Boost Circular Buffer requested by Ben Goodrich for RStan

Courtesy of CRANberries, there is also a diffstat report for the most recent release

Comments and suggestions are welcome via the mailing list or the issue tracker at the GitHubGitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/bh | permanent link

Thu, 06 Nov 2014

RcppRedis 0.1.2

A new release of RcppRedis is now on CRAN. It contains additional commands for hashes and sets, all contributed by John Laing and Whit Armstrong.

Changes in version 0.1.2 (2014-11-06)

  • New commands execv, hset, hget, sadd, srem, and smembers contributed by John Laing and Whit Armstrong over several pull requests.

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppRedis page page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 05 Nov 2014

A Software Carpentry workshop at Northwestern

On Friday October 31, 2014, and Saturday November 1, 2014, around thirty-five graduate students and faculty members attended a Software Carpentry workshop. Attendees came primarily from the Economics department and the Kellogg School of Management, which was also the host and sponsor providing an excellent venue with the Allen Center on the (main) Evanston campus of Northwestern University.

The focus of the two-day workshop was an introduction, and practical initiation, to working effectively at the shell, getting introduced and familiar with the git revision control system, as well as a thorough introduction to working and programming in R---from the basics all the way to advanced graphing as well as creating reproducible research documents.

The idea of the workshop had come out of discussion during our R/Finance 2014 conference. Bob McDonald of Northwestern, one of this year's keynote speakers, and I were discussing various topic related to open source and good programming practice --- as well as the lack of a thorough introduction to either for graduate students and researcher. And once I mentioned and explained Software Carpentry, Bob was rather intrigued. And just a few months later we were hosting a workshop (along with outstanding support from Jackie Milhans from Research Computing at Northwestern).

We were extremely fortunate in that Karthik Ram and Ramnath Vaidyanathan were able to come to Chicago and act as lead instructors, giving me an opportunity to get my feet wet. The workshop began with a session on shell and automation, which was followed by three session focusing on R: a core introduction, a session focused on function, and to end the day, a session on the split-apply-combine approach to data transformation and analysis.

The second day started with two concentrated session on git and the git workflow. In the afternoon, one session on visualization with R as well as a capstone-alike session on reproducible research rounded out the second day.

Things that worked

The experience of the instructors showed, as the material was presented and an effective manner. The selection of topics, as well as the pace were seen by most students to be appropriate and just right. Karthik and Ramnath both did an outstanding job.

No students experienced any real difficulties installing software, or using the supplied information. Participants were roughly split between Windows and OS X laptops, and had generally no problem with bash, git, or R via RStudio.

The overall Software Carpentry setup, the lesson layout, the focus on hands-on exercises along with instruction, the use of the electronic noteboard provided by etherpad and, of course, the tried-and-tested material worked very well.

Things that could have worked better

Even more breaks for exercises could have been added. Students had difficulty staying on pace in some of the exercise: once someone fell behind even for a small error (maybe a typo) it was sometimes hard to catch up. That is a general problem for hands-on classes. I feel I could have done better with the scope of my two session.

Even more cohesion among the topics could have been achieved via a single continually used example dataset and analysis.

Acknowledgments

Robert McDonald from Kellogg, and Jackie Milhans from Research Computing IT, were superb hosts and organizers. Their help in preparing for the workshop was tremendous, and the pick of venue was excellent, and allowed for a stress-free two days of classes. We could not have done this without Karthik and Ramnath, so a very big Thank You to both of them. Last but not least the Software Carpentry 'head office' was always ready to help Bob, Jackie and myself during the earlier planning stage, so another big Thank You! to Greg and Arliss.

/computers/misc | permanent link

RPushbullet 0.1.1

A minor bugfix release 0.1.1 of the RPushbullet package (interfacing the neat Pushbullet service) landed on CRAN yesterday morning.

It cleans up a small issue related to the ability to transfer files between devices via the Pushbullet service where the ability to select a (non-default) target device has now been restored.

With that, allow me to borrow one excellent use case of RPushbullet from the blog of Karl Broman: how to use RPushbullet for error notifications from R:

options(error = function() {
    library(RPushbullet)
    pbPost("note", "Error", geterrmessage())
})

This is very clever: should an error occur, you get immediate notification in browser or on your phone. Left as an exercise for the reader is to combine this with the equally excellent rfoaas package (github|cran) to get appropriately colourful error messages...

More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rpushbullet | permanent link

Sun, 02 Nov 2014

RcppArmadillo 0.4.500.0

A few days ago, Conrad provided another minor release of Armadillo. Once again, I had created a GitHub-only pre-release of his pre-release which was tested against (then) all ninety (!!) CRAN dependents of our RcppArmadillo package, providing a further test for Conrad's code and uploaded RcppArmadillo 0.4.500.0 to CRAN and Debian once his release was finalized.

The few changes from his end are summarized below; our end also includes an update / extension to the sample() method provided by Christian Gunning --- and used to great effect in the excellent Rcpp Gallery post by Jonathan Olmsted.

Changes in RcppArmadillo version 0.4.500.0 (2014-10-30)

  • Upgraded to Armadillo release Version 4.500 ("Singapore Sling")

    • faster handling of complex vectors by norm()

    • expanded chol() to optionally specify output matrix as upper or lower triangular

    • better handling of non-finite values when saving matrices as text files

  • The sample functionality has been extended to provide the Walker Alias method (including new unit tests) via a pull request by Christian Gunning

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 23 Oct 2014

Introducing Rocker: Docker for R

You only know two things about Docker. First, it uses Linux
containers. Second, the Internet won't shut up about it.

-- attributed to Solomon Hykes, Docker CEO

So what is Docker?

Docker is a relatively new open source application and service, which is seeing interest across a number of areas. It uses recent Linux kernel features (containers, namespaces) to shield processes. While its use (superficially) resembles that of virtual machines, it is much more lightweight as it operates at the level of a single process (rather than an emulation of an entire OS layer). This also allows it to start almost instantly, require very little resources and hence permits an order of magnitude more deployments per host than a virtual machine.

Docker offers a standard interface to creation, distribution and deployment. The shipping container analogy is apt: just how shipping containers (via their standard size and "interface") allow global trade to prosper, Docker is aiming for nothing less for deployment. A Dockerfile provides a concise, extensible, and executable description of the computational environment. Docker software then builds a Docker image from the Dockerfile. Docker images are analogous to virtual machine images, but smaller and built in discrete, extensible and reuseable layers. Images can be distributed and run on any machine that has Docker software installed---including Windows, OS X and of course Linux. Running instances are called Docker containers. A single machine can run hundreds of such containers, including multiple containers running the same image.

There are many good tutorials and introductory materials on Docker on the web. The official online tutorial is a good place to start; this post can not go into more detail in order to remain short and introductory.

So what is Rocker?

rocker logo

At its core, Rocker is a project for running R using Docker containers. We provide a collection of Dockerfiles and pre-built Docker images that can be used and extended for many purposes.

Rocker is the the name of our GitHub repository contained with the Rocker-Org GitHub organization.

Rocker is also the name the account under which the automated builds at Docker provide containers ready for download.

Current Rocker Status

Core Rocker Containers

The Rocker project develops the following containers in the core Rocker repository

  • r-base provides a base R container to build from
  • r-devel provides the basic R container, as well as a complete R-devel build based on current SVN sources of R
  • rstudio provides the base R container as well an RStudio Server instance

We have settled on these three core images after earlier work in repositories such as docker-debian-r and docker-ubuntu-r.

Rocker Use Case Containers

Within the Rocker-org organization on GitHub, we are also working on

  • Hadleyverse which extends the rstudio container with a number of Hadley packages
  • rOpenSci which extends hadleyverse with a number of rOpenSci packages
  • r-devel-san provides an R-devel build for "Sanitizer" run-time diagnostics via a properly instrumented version of R-devel via a recent compiler build
  • rocker-versioned aims to provided containers with 'versioned' previous R releases and matching packages

Other repositories will probably be added as new needs and opportunities are identified.

Deprecation

The Rocker effort supersedes and replaces earlier work by Dirk (in the docker-debian-r and docker-ubuntu-r GitHub repositories) and Carl. Please use the Rocker GitHub repo and Rocker Containers from Docker.com going forward.

Next Steps

We intend to follow-up with more posts detailing usage of both the source Dockerfiles and binary containers on different platforms.

Rocker containers are fully functional. We invite you to take them for a spin. Bug reports, comments, and suggestions are welcome; we suggest you use the GitHub issue tracker.

Acknowledgments

We are very appreciative of all comments received by early adopters and testers. We also would like to thank RStudio for allowing us the redistribution of their RStudio Server binary.

Published concurrently at rOpenSci blog and Dirk's blog.

Authors

Dirk Eddelbuettel and Carl Boettiger

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rocker | permanent link

Sun, 19 Oct 2014

littler 0.2.1

max-heap image

A new maintenance release of littler is available now.

The main change are a few updates and extensions to the examples provided along with littler. Several of those continue to make use of the wonderful docopt package by Edwin de Jonge. Carl Boettiger and I are making good use of these littler examples, particularly to install directly from CRAN or GitHub, in our Rocker builds of R for Docker (about which we should have a bit more to blog soon too).

Full details for the littler release are provided as usual at the ChangeLog page.

The code is available via the GitHub repo, from tarballs off my littler page and the local directory here. A fresh package has gone to the incoming queue at Debian; Michael Rutter will probably have new Ubuntu binaries at CRAN in a few days too.

Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/littler | permanent link

Sun, 12 Oct 2014

Seinfeld streak at GitHub

Early last year, I referred to a Seinfeld Streak in a blog post referring to almost two months of updates to the Rcpp Gallery. This is sometimes called Jerry Seinfeld's secret to productivity: Just keep at it. Don't break the streak.

I now have different streak:

github activity october 2013 to october 2014

Now we'll see how far this one will go.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/computers/misc | permanent link

Fri, 10 Oct 2014

RPushbullet 0.1.0 with a lot more awesome

A new release 0.1.0 of the RPushbullet package (interfacing the neat Pushbullet service) landed on CRAN today.

It brings a number of goodies relative to the first release 0.0.2 of a few months ago:

  • pushing of files is now supported thanks to a nice pull request bu Mike Birdgeneau
  • a default device can be designated in the ~/.rpushbullet.json file or options
  • initialization has been rewritten to use recpients which can be indices, device names or, if missing entirely, the (new) default device
  • alternatively, email is supported as another recipient option in which case the Pushbullet service will send an email to the give address
  • pbGetDevices() now returns a proper S3 object with corresponding print() and summary() methods
  • the documentation regarding package initialization, and setting of key, devices, etc has been expanded
  • more examples has been added to the documentation
  • various minor cleanups, fixes, corrections throughout

There is a whole boat load of more wickedness in the Pushbullet API so if anybody feels compelled to add it, fire off pull requests at GitHub.

More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rpushbullet | permanent link

Mon, 29 Sep 2014

Rcpp 0.11.3

A new release 0.11.3 of Rcpp is now on the CRAN network for GNU R, and an updated Debian package has been uploaded too.

Rcpp has become the most popular way of enhancing GNU R with C++ code. As of today, 273 packages on CRAN depend on Rcpp for making analyses go faster and further.

This release brings a fairly large number of continued enhancements, fixes and polishing to Rcpp. These were provided by a total of seven different contributors---which is a new record as well.

See below for a detailed list of changes extracted from the NEWS file, but some highlights included in this release are

  • Several API cleanups, polishes and a pre-announced code removal
  • New InternalFunction interface, and new Timer functionality.
  • More robust functionality of Rcpp Attributes as well as a new dryRun option.
  • The Rcpp FAQ was updated, as was the main Description: in the DESCRIPTION file.
  • Rcpp.package.skeleton() can now deploy functionality from pkgKitten to create Rcpp packages that purr.

One sore point, however, is that we missed that packages using Rcpp Modules appear to require a rebuild. We are sorry for the inconvenience; this has highlighted a shortcoming in our fairly robust and extensive tests. While we test our packages against all known CRAN dependents, such tests check for the ability to compile and run freshly and not whether previously built packages still run. We intend to augment our testing in this direction to avoid a repeat occurrence of such a misfeature.

Changes in Rcpp version 0.11.3 (2014-09-27)

  • Changes in Rcpp API:

    • The deprecation of RCPP_FUNCTION_* which was announced with release 0.10.5 last year is proceeding as planned, and the file macros/preprocessor_generated.h has been removed.

    • Timer no longer records time between steps, but times from the origin. It also gains a get_timers(int) methods that creates a vector of Timer that have the same origin. This is modelled on the Rcpp11 implementation and is more useful for situations where we use timers in several threads. Timer also gains a constructor taking a nanotime_t to use as its origin, and a origin method. This can be useful for situations where the number of threads is not known in advance but we still want to track what goes on in each thread.

    • A cast to bool was removed in the vector proxy code as inconsistent behaviour between clang and g++ compilations was noticed.

    • A missing update(SEXP) method was added thanks to pull request by Omar Andres Zapata Mesa.

    • A proxy for DimNames was added.

    • A no_init option was added for Matrices and Vectors.

    • The InternalFunction class was updated to work with std::function (provided a suitable C++11 compiler is available) via a pull request by Christian Authmann.

    • A new_env() function was added to Environment.h

    • The return value of range eraser for Vectors was fixed in a pull request by Yixuan Qiu.

  • Changes in Rcpp Sugar:

    • In ifelse(), the returned NA type was corrected for operator[].

  • Changes in Rcpp Attributes:

    • Include LinkingTo in DESCRIPTION fields scanned to confirm that C++ dependencies are referenced by package.

    • Add dryRun parameter to sourceCpp.

    • Corrected issue with relative path and R chunk use for sourceCpp.

  • Changes in Rcpp Documentation:

    • The Rcpp-FAQ vignette was updated with respect to OS X issues.

    • A new entry in the Rcpp-FAQ clarifies the use of licenses.

    • Vignettes build results no longer copied to /tmp to please CRAN.

    • The Description in DESCRIPTION has been shortened.

  • Changes in Rcpp support functions:

    • The Rcpp.package.skeleton() function will now use pkgKitten package, if available, to create a package which passes R CMD check without warnings. A new Suggests: has been added for pkgKitten.

    • The modules=TRUE case for Rcpp.package.skeleton() has been improved and now runs without complaints from R CMD check as well.

  • Changes in Rcpp unit test functions:

    • Functions from the RUnit package are now prefixed with RUnit::

    • The testRcppModule and testRcppClass sample packages now pass R CMD check --as-cran cleanly with NOTES or WARNINGS

Thanks to CRANberries, you can also look at a diff to the previous release As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 25 Sep 2014

R and Docker

r and docker talk picture by @mediafly

Earlier this evening I gave a short talk about R and Docker at the September Meetup of the Docker Chicago group.

Thanks to Karl Grzeszczak for setting the meeting, and for providing a pretty thorough intro talk regarding CoreOS and Docker.

My slides are now up on my presentations page.

/computers/R | permanent link

Mon, 22 Sep 2014

RcppArmadillo 0.4.450.1.0

Continuing with his standard pace of approximately one new version per month, Conrad released a new minor release of Armadillo a few days ago. As before, I had created a GitHub-only pre-release which was tested against all eighty-seven (!!) CRAN dependents of our RcppArmadillo package and then uploaded RcppArmadillo 0.4.450.0 to CRAN.

The CRAN maintainers pointed out that under the R-development release, a NOTE was issued concerning the C-library's rand() call. This is a pretty new NOTE, but it means using the (sometimes poor quality) rand() generator is now a no-no. Now, Armadillo being as robustly engineered as it is offers a new random number generator based on C++11 as well as a fallback generator for those unfortunate enough to live with an older C++98 compiler. (I would like to note here that I find Conrad's continued support for both C++11, offering very useful modern language idioms, as well as the fallback code for continued deployment and usage by those constrained in their choice of compilers rather exemplary --- because contrary to what some people may claim, it is not a matter of one or the other. C++ always was, and continues to be, a multi-paradigm language which can be supported easily by several standard. But I digress...)

In any event, one cannot argue with CRAN about their prescription of a C++98 compiler. So Conrad and I discussed this over email, and came up with a scheme where a user-package (such as RcppArmadillo) can provide an alternate generator which Armadillo then deploys. I implemented a first solution which was then altered / reflected by Conrad in a revised version 4.450.1 of Armadillo. I packaged, and now uploaded, that version as RcppArmadillo 0.4.450.1.0 to both CRAN and into Debian.

Besides the RNG change already discussed, this release brings a few smaller changes from the Armadillo side. These are detailed below in the extract from the NEWS file. On the RcppArmadillo side, we now have support for pkgKitten which is both very exciting and likely the topic of another blog post with an example of creating an RcppArmadillo package that purrs. In the process, I overhauled and polished how new packages are created by RcppArmadillo.package.skeleton(). An upcoming blog post may provide an example.

Changes in RcppArmadillo version 0.4.450.1.0 (2014-09-21)

  • Upgraded to Armadillo release Version 4.450.1 (Spring Hill Fort)

    • faster handling of matrix transposes within compound expressions

    • expanded symmatu()/symmatl() to optionally disable taking the complex conjugate of elements

    • expanded sort_index() to handle complex vectors

    • expanded the gmm_diag class with functions to generate random samples

  • A new random-number implementation for Armadillo uses the RNG from R as a fallback (when C++11 is not selected so the C++11-based RNG is unavailable) which avoids using the older C++98-based std::rand

  • The RcppArmadillo.package.skeleton() function was updated to only set an "Imports:" for Rcpp, but not RcppArmadillo which (as a template library) needs only LinkingTo:

  • The RcppArmadillo.package.skeleton() function will now prefer pkgKitten::kitten() over package.skeleton() in order to create a working package which passes R CMD check.

  • The pkgKitten package is now a Suggests:

  • A manual page was added to provide documentation for the functions provided by the skeleton package.

  • A small update was made to the package manual page.

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 11 Sep 2014

pkgKitten 0.1.2: Still creating R Packages that purr

A brown bag release 0.1.2 of pkgKitten is now on CRAN, following yesterday's 0.1.1 upload

Next time I'll try to remember that when I have parameters name and path, it won't work so well to call them as path and name ...

Changes in version 0.1.2 (2014-09-11)

  • Brown-bag fix of calling the new helper function with then correct order of arguments.

More details about the package are at the pkgKitten webpage and the pkgKitten GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/pkgkitten | permanent link

Wed, 10 Sep 2014

pkgKitten 0.1.1: Still creating R Packages that purr

A maintenance release 0.1.1 of pkgKitten is now on CRAN.

It has only one small change: the function playWithPerPackageHelpPage() was factored out of the main function kitten() as I happened to be needing something just like playWithPerPackageHelpPage() to make packages created by the Rcpp function Rcpp.package.skeleton() a little nicer.

We also added a NEWS.Rd file which restates major release features. As it is so short, we include it in its entirety.

Changes in version 0.1.1 (2014-09-09)

  • New (exported) function playWithPerPackageHelpPage() which lets other packages create a non-complaint-generating package help page

Changes in version 0.1.0 (2014-06-13)

  • Initial public version and CRAN upload

More details about the package are at the pkgKitten webpage and the pkgKitten GitHub repo.

Courtesy of CRANberries, there is also a diffstat report for this release

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/pkgkitten | permanent link

Tue, 02 Sep 2014

littler is faster at doing nothing!

With yesterday's announcement of littler 0.2.0, I kept thinking about a few not-so-frequently-asked but recurring questions about littler. And an obvious one if of course the relationship to Rscript.

As we have pointed out before, littler preceded Rscript. Now, with Rscript being present in every R installation, it is of course by now more widely known.

But there is one important aspect which I stressed once or twice in the past and which bears repeating. Due to the lean and mean way in which littler is designed and set-up, it actually starts a lot faster than either Rscript or R. How, you ask? Well we actually query the environement at build time and hardcode a number of settings which R and Rscript re-acquire and learn each time they start. Which is more flexible. But slower.

So consider the following, really simple example. In it, we create a simple worker function f which launches the given (and simplest possible) R command of just quitting 250 times (by launching a shell command which loops). We then let littler, Rscript and R (in its just execute this expression mode) run this, and time it via one of the common benchmarking packages.

R> library(rbenchmark)
R> f <- function(cmd) { system(paste0("bash -c \"for i in \\$(seq 1 250); do ", cmd, " -e 'q()'; done\"")); }
R> res <- benchmark(f("r"), f("Rscript"), f("R --slave"), replications=5, order="relative")
R> res
            test replications elapsed relative user.self sys.self user.child sys.child
1         f("r")            5  32.256    1.000     0.001    0.004     26.538     5.706
2   f("Rscript")            5 180.154    5.585     0.001    0.004    152.751    28.522
3 f("R --slave")            5 302.714    9.385     0.001    0.004    270.569    33.098
R> 

So there: littler does "nothing" about five times as fast as Rscript, and about nine times as fast as R. Not that this matters greatly -- but when you design something for repeated (unsupervised) execution, say in a cronjob, it might as well be lightweight.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/littler | permanent link

Mon, 01 Sep 2014

littler 0.2.0

We are happy to announce a new release of littler.

A few minor things have changes since the last release: max-heap image
  • A few new examples were added or updated, including use of the fabulous new docopt package by Edwin de Jonge which makes command-line parsing a breeze.
  • Other new examples show simple calls to help with sweave, knitr, roxygen2, Rcpp's attribute compilation, and more.
  • We also wrote an entirely new webpage with usage example.
  • A new option -d | --datastdin was added which will read stdin into a data.frame variable X.
  • The repository has been move to this GitHub repo.
  • With that, the build process was updated both throughout but also to reflect the current git commit at time of build.

Full details are provided at the ChangeLog

The code is available via the GitHub repo, from tarballs off my littler page and the local directory here. A fresh package will got to Debian's incoming queue shortly as well.

Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/littler | permanent link

Fri, 29 Aug 2014

BH release 1.54.0-4

Another small new release of our BH package providing Boost headers for use by R is now on CRAN. This one brings a one-file change: the file any.hpp comprising the Boost.Any library --- as requested by a fellow package maintainer needing it for a pending upload to CRAN.

No other changes were made.

Changes in version 1.54.0-4 (2014-08-29)

  • Added Boost Any requested by Greg Jeffries for his nabo package

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/bh | permanent link

Thu, 21 Aug 2014

RcppEigen 0.3.2.2.0

A new upstream release of the Eigen C++ template library for linear algebra was released a few days ago. And Yixuan Qiu did some really nice work rolling this into a new RcppEigen released and then sent me a nice pull requent. The new version is now on CRAN, and I will prepare a Debian in a moment too.

Upstream changes for Eigen are summarized in their changelog. On the RcppEigen side, Yixuan also rolled in some more changes on as<>() and wrap() converters as noted below in the NEWS entry.

Changes in RcppEigen version 0.3.2.2.0 (2014-08-19)

  • Updated to version 3.2.2 of Eigen

  • Rcpp::as() now supports the conversion from R vector to “row array”, i.e., Eigen::Array<T, 1, Eigen::Dynamic>

  • Rcpp::as() now supports the conversion from dgRMatrix (row oriented sparse matrices, defined in Matrix package) to Eigen::MappedSparseMatrix<T, Eigen::RowMajor>

  • Conversion from R matrix to Eigen::MatrixXd and Eigen::ArrayXXd using Rcpp::as() no longer gives compilation errors

Courtesy of CRANberries, there are diffstat reports for the most recent release.

Questions, comments etc about RcppEigen should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 19 Aug 2014

RcppArmadillo 0.4.400.0

After two pre-releases in the last few days, Conrad finalised a new Armadillo version 4.400 today. I had kept up with the pre-releases, tested twice against all eighty (!!) CRAN dependents of RcppArmadillo and have hence uploaded RcppArmadillo 0.4.400.0 to CRAN and into Debian.

This release brings a number of new upstream features which are detailed below. As included is s bugfix for sparse matrix creation at the RcppArmadillo end which was found by the ASAN tests at CRAN --- which are similar to the sanitizers tests I recently blogged. I was able to develop and test the fix in the very docker r-devel-san images I had written about which was nice. Special thanks also to Ryan Curtin for help with the fix.

Changes in RcppArmadillo version 0.4.400.0 (2014-08-19)

  • Upgraded to Armadillo release Version 4.400 (Winter Shark Alley)

    • added gmm_diag class for statistical modelling using Gaussian Mixture Models; includes multi-threaded implementation of k-means and Expectation-Maximisation for parameter estimation

    • added clamp() for clamping values to be between lower and upper limits

    • expanded batch insertion constructors for sparse matrices to add values at repeated locations

    • faster handling of subvectors by dot()

    • faster handling of aliasing by submatrix views

  • Corrected a bug (found by the g++ Address Sanitizer) in sparse matrix initialization where space for a sentinel was allocated, but the sentinel was not set; with extra thanks to Ryan Curtin for help

  • Added a few unit tests for sparse matrices

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 07 Aug 2014

Rcpp now used by 250 CRAN packages

Rcpp growth

Rcpp reached a nice round milestone yesterday: 250 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations).

The graph is on the left depicts the growth of Rcpp over time. Or at least since I started to write down some usage numbers: first informally, then via a script.

Also displayed is the relative proportion of CRAN packages using it. Rcpp cleared the four per-cent hurdle just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk.

250 is a pretty impressive, and rather humbling, number.

From everybody behind Rcpp, I would like to say a heartfelt Thank You! to all the users and of course contributors.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 04 Aug 2014

BH release 1.54.0-3

A new release of our BH package providing Boost headers for use by R is now on the CRAN mirrors. max-heap image This release is the third based on Boost 1.54.0.

At the request of the maintainer of the recent added RcppMLPACK package, it adds the Boost.Heap library. Boost.Heap implements priority queues which extend beyond the corresponding (and somewhat simpler) class in the STL. Key features of the Boost.Heap priority queues are mutability, iterators, ability to merge, stable sort, and comparison.

No other changes were made.

Changes in version 1.54.0-3 (2014-08-03)

  • Added Boost Heap library which will be needed by the next version of RcppMLPACK

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/bh | permanent link

Sun, 03 Aug 2014

Introducing sanitizers 0.1.0

A new package sanitizers is now on CRAN. It provides test cases for Address Sanitizers, and Undefined Behaviour Sanitizers. These are two recent features of both g++ and clang++, and described in the Checking Memory Access section of the Writing R Extension manual.

I set up a new web page for the sanitizers package which illustrates their use case via pre-built Docker images, similar to what I presented at the end of my useR! 2014 keynote a few weeks ago.

So instead of repeating this over here, I invite you to read the detailed discussion on the sanitizers page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/sanitizers | permanent link

Wed, 16 Jul 2014

Introducing RcppParallel: Getting R and C++ to work (some more) in parallel

A common theme over the last few decades was that we could afford to simply sit back and let computer (hardware) engineers take care of increases in computing speed thanks to Moore's law. That same line of thought now frequently points out that we are getting closer and closer to the physical limits of what Moore's law can do for us.

So the new best hope is (and has been) parallel processing. Even our smartphones have multiple cores, and most if not all retail PCs now possess two, four or more cores. Real computers, aka somewhat decent servers, can be had with 24, 32 or more cores as well, and all that is before we even consider GPU coprocessors or other upcoming changes.

And sometimes our tasks are embarassingly simple as is the case with many data-parallel jobs: we can use higher-level operations such as those offered by the base R package parallel to spawn multiple processing tasks and gather the results. I covered all this in some detail in previous talks on High Performance Computing with R (and you can also consult the Task View on High Performance Computing with R which I edit).

But sometimes we can't use data-parallel approaches. Hence we have to redo our algorithms. Which is really hard. R itself has been relying on the (fairly mature) OpenMP standard for some of its operations. Luke Tierney's (awesome) keynote in May at our (sixth) R/Finance conference mentioned some of the issues related to OpenMP. One which matters is that OpenMP works really well on Linux, and either not so well (Windows) or not at all (OS X, due the usual issue with the gcc/clang switch enforced by Applem but the good news is that the OpenMP toolchain is expected to make it to OS X is some more performant form "soon"). R is still expected to make wider use of OpenMP in future versions.

Another tool which has been around for a few years, and which can be considered to be equally mature is the Intel Threaded Building Blocks library, or TBB. JJ recently started to wrap this up for use by R. The first approach resulted in a (now superseded, see below) package TBB. But hardware and OS issues bite once again, as the Intel TBB is not really building that well for the Windows toolchain used by R (and based on MinGW).

(And yes, there are two more options. But Boost Threads requires linking which precludes easy use as e.g. via our BH package. And C++11 with its threads library (based on Boost Threads) is not yet as widely available as R and Rcpp which means that it is not a real deployment option yet.)

Now, JJ, being as awesome as he is, went back to the drawing board and integrated a second threading toolkit: TinyThread++, a small header-only library without further dependencies. Not as feature-rich as Intel Threaded Building Blocks, but at least available everywhere. So a new package RcppParallel, so far only on GitHub, wraps around both TinyThread++ and Intel Threaded Building Blocks and offers a consistent interface available on all platforms used by R.

Better still, JJ also authored several pieces demonstrating this new package for the Rcpp Gallery:

All four are interesting and demonstrate different aspects of parallel computing via RcppParallel. But the last article is key. Based on a question by Jim Bullard, and then written with Jim, it shows how a particular matrix distance metric (which is missing from R) can be implemented in a serial manner in both R, and also via Rcpp. The key implementation, however, uses both Rcpp and RcppParallel and thereby achieves a truly impressive speed gain as the gains from using compiled code (via Rcpp) and from using a parallel algorithm (via RcppParallel) are multiplicative! Between JJ's and my four-core machines the gain was between 200 and 300 fold---which is rather considerable. For kicks, I also used a much bigger machine at work which came in at an even larger speed gain (but gains become clearly sublinear as the number of cores increases; there are however some tuning parameters).

So these are exciting times. I am sure there will be lots more to come. For now, head over to the RcppParallel package and start playing. Further contributions to the Rcpp Gallery are not only welcome but strongly encouraged.

/code/snippets | permanent link

Sat, 12 Jul 2014

RcppArmadillo 0.4.320.0

While I was out at the (immensely impressive and equally enjoyable) useR! 2014 conference at UCLA, Conrad provided a bug-fix release 4.320 of Armadillo, the nifty templated C++ library for linear algebra. I quickly rolled that into RcppArmadillo release 0.4.320.0 which has been on CRAN and in Debian for a good week now.

This release fixes some minor things with sparse and dense Eigen solvers (as well as one RNG issue probably of lesser interest to R users deploying the RNGs from R) as shown in the NEWS entry below.

Changes in RcppArmadillo version 0.4.320.0 (2014-07-03)

  • Upgraded to Armadillo release Version 4.320 (Daintree Tea Raider)

    • expanded eigs_sym() and eigs_gen() to use an optional tolerance parameter

    • expanded eig_sym() to automatically fall back to standard decomposition method if divide-and-conquer fails

    • automatic installer enables use of C++11 random number generator when using gcc 4.8.3+ in C++11 mode

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Fri, 13 Jun 2014

Introducing pkgKitten: Creating R Packages that purr

A new package pkgKitten is now on CRAN in its infant version 0.1.0.

It does only one thing: create R packages that do not suck. Or at least not as much as those created by the (R default) function package.skeleton() which creates files which the very standard, and very recommended, R CMD check then complains about. That is silly. Because each time R CMD check complains, something bad happens to a kitten somewhere.

So pkgKitten offers a wrapper function kitten() which creates neat little packages which do not upset R CMD check.

A complete little demo is on the pkgKitten webpage which also has pointers to the pkgKitten GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/pkgkitten | permanent link

Mon, 09 Jun 2014

Introducing RcppRedis

RcppRedis, another new package of mine, appeared on CRAN a few weeks ago just in time for our annual R/Finance conference.

And as of today, we also have Windows binaries thanks to generous help from John Buonagurio who help building the required 32- and 64-bit libraries for Windows, and Uwe Ligges who now installed them for the CRAN and win-builder machiness. Binaries for release 0.1.1 should be available within a day.

To describe the package, I also just set up a page about RcppRedis so rather than repeating everything here I invite you to follow the link.

The ever-so-brief NEWS entry for the two first uploads follows.

Changes in version 0.1.1 (2014-06-09)

  • Now with Windows support thanks to the installation of builds of the hiredis library (created by John Buonagurio) at CRAN / win-builder (thanks to Uwe Ligges)

  • Added support for new command zcount

Changes in version 0.1.0 (2014-05-10)

  • Initial CRAN upload

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sun, 08 Jun 2014

Rcpp 0.11.2

A new minor release 0.11.2 of Rcpp is now on the CRAN network for GNU R, and binaries for Debian have also been uploaded.

The release smoothes a few edges on both the Rcpp side itself, as well as on the interaction between Rcpp and R which since release 3.1.0 offers a few new features related to C++11.

Kevin added a couple of neat extensions related to vectors, a new ListOf templated list class, as well as a new option to warn on implicit casts. We decided not to make this option the default as it may be too common in some packages. JJ took care of a few buglets related to the wonderful Rcpp Attributes. See the NEWS file section below for details, or the ChangeLog file in the package and on the Rcpp Changelog page. Note that the diffstat reported by CRANberries is very large as Kevin also committed a whitespace cleanup which touched almost all files.

As before, we tested this release by building against all CRAN packages which depend upon Rcpp. In fact we did three such runs leading up to the release. Only one package was blacklisted (as I currently don't have CUDA set-up), two had what may be internal errors or tests which were too restrictive, sixteen suffered from missing packages or RGL devices --- but the remaining 202 packages all built and tested cleanly. Detailed results of those tests (and the scripts for it) are in the rcpp-logs repo GitHub.

There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page.

Changes in Rcpp version 0.11.2 (2014-06-06

  • Changes in Rcpp API:

    • Implicit conversions, e.g. between NumericVector and IntegerVector, will now give warnings if you use #define RCPP_WARN_ON_COERCE before including the Rcpp headers.

    • Templated List containers, ListOf<T>, have been introduced. When subsetting such containers, the return is assumed to be of type T, allowing code such as ListOf<NumericVector> x; NumericVector y = x[0] + x[1] + x[2].

    • In a number of instances, returned results are protected and/or cast more carefully.

  • Changes in Rcpp Attributes

    • Trailing line comments are now stripped by the attributes parser. This allows the parser to handle C++ source files containing comments inline with function arguments.

    • The USE_CXX1X environment variable is now defined by the cpp11 plugin when R >= 3.1. Two additional plugins have been added for use with C++0x (eg when using g++ 4.6.* as on Windows) as well as C++1y for compilers beginning to support the next revision of the standard; additional fallback is provided for Windows.

    • compileAttributes() now also considers Imports: which may suppress a warning when running Rcpp.package.skeleton().

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Wed, 04 Jun 2014

RcppArmadillo 0.4.300.8.0

A new minor / bug fix release 4.300.8 of Armadillo, the templated C++ library for linear algebra, was tagged by Conrad in his SVN repo a few days ago, following earlier snapshots in the 4.300.* series.

We had prepared two earlier releases for GitHub but not CRAN in order to accomodate the CRAN maintainer's desire of "a release every one to two months" expressed in the CRAN Repo Policy. However, two actual bugs in the interaction between MinGW and C++11 were reported on the rcpp-devel mailing list, and this releases addresses these. Hence time for new release 0.4.300.8.0 of RcppArmadillo which is now on CRAN and in Debian.

This release brings a few upstream changes detailed below such as nice new upstream changes such as more robust norm() (and related) functions, and fixes related to matrix and cube interactions. From our end, we added a better detection of Windows via both _WIN32 and WIN32 (as the former apparently gets undefined by MinGW in C++11 mode). We also added the ability to turn on C++11 support from R (possible since R 3.1.0) yet also turn it off for Armadillo. This is needed as the prescribed compiler on Windows is g++ 4.6.2 -- which offers a subset of C++11 which is good enough for a number of things from the C++11 standard, but not advanced enough for everything which Armadillo uses when C++11 support is turned on. As Armadillo continues to offer a choice of C++ standards, we can use the ability to deploy C++11 only outside of its internals. It is worth repeating that this issue should only affect Windows users wishing to use C++11; other platforms are fine as they generally have more modern compilers.

Changes in RcppArmadillo version 0.4.300.8.0 (2014-05-31)

  • Upgraded to Armadillo release Version 4.300.8 (Medieval Cornea Scraper)

    • More robust norm-related functions

    • Fixes between interactions between cube and vector types.

  • Adds a #define ARMA_DONT_USE_CXX11 to provide an option to turn C++11 off for Armadillo (but client packages may still use it)

  • More robust Windows detection by using _WIN32 as well as WIN32 as the latter gets diabled by MinGW with C++11

  • On Windows, C++11 can be turned off as the Armadillo code base uses more features of C++11 than g++ 4.6.2 version in Rtools implements

Changes in RcppArmadillo version 0.4.300.5.0 (2014-05-19)

  • Upgraded to Armadillo release Version 4.300.5 (Medieval Cornea Scraper)

    • Handle possible underflows and overflows in norm(), normalise(), norm_dot()

    • Fix for handling of null vectors by norm_dot()

Changes in RcppArmadillo version 0.4.300.2.0 (2014-05-13)

  • Upgraded to Armadillo release Version 4.300.2 (Medieval Cornea Scraper)

    • faster find()

Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 03 Jun 2014

Introducing RPushbullet

A new package of mine appeared on CRAN today: RPushbullet.

A few days ago, I had tweeted about a first version available via its GitHub repo. The package is now on CRAN which will make it easier to install for many. You still need an API key (free, after registration at Pushbullet) and one or more target devices to message to: an modern smartphone (Android, iPhone, ...) or a Chrome browser provided you also installed the corresponding app.

I also set up a short web page on my server with some more information and an example. So pushbullet away...

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rpushbullet | permanent link

Sat, 31 May 2014

RcppGSL 0.2.1 and 0.2.2

A few days ago, version 0.2.1 of RcppGSL---our interface package between R and the GNU GSL using our Rcpp package for seamless R and C++ integration---appeared on CRAN making it the first release in some time.

And it turned that this version tickled an obscure and long-dormant bug under clang which was found on OS X Mavericks---which Kevin Ushey consequently squashed. So this is now reflected in version 0.2.2 which just arrived on CRAN.

Besides the bugfix, a few things were modernized to reflect capabilities of Rcpp 0.11.0 and later. The releases also contain a few changes that had accumulated since the previous release in 2012 such as an additional example using B-splines, and use of updated vignette build options provided by R.

The NEWS file entries follows below:

Changes in version 0.2.2 (2014-05-31)

  • A subtle bug (tickled only by clang on some OS versions) in vector and matrix view initialization was corrected by Kevin Ushey

Changes in version 0.2.1 (2014-05-26)

  • Added new example based on B-splines example in GSL manual illustrating simple GSL use via Rcpp attributes

  • Vignette compilation has been reverted to using highlight since version 0.4.2 or greater can be used as a vignette engine (with R 3.0.* or later).

  • Vignette compilation is now being done by R CMD build as R 3.0.0 supports different vignette engines, so the vignette build process has been simplified. A convenience helper script has also been added for command-line builds.

  • Unit tests now use sourceCpp() instead of cxxfunction() from the inline package

  • The DESCRIPTION file now uses Suggests: Rcpp (instead of Depends: Rcpp) to permit building of the vignette

  • The package now takes advantage of the simplified build process available with Rcpp (>= 0.11.0)

  • Similar updates to the build process were made for the example package included with RcppGSL

And courtesy of CRANberries, a summaries of changes to release 0.2.0 and this week's release 0.2.1 are available.

More information is on the RcppGSL page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 12 May 2014

Updated list of upcoming Rcpp talks and workshops

This is an updated version of an earlier post in March.

A number of talks about Rcpp are scheduled over the next couple of months:

  • On Wednesday May 14 I will be speaking at the Chicago R Users Group meeting charmingly titled Warm Up for R/Finance, along with fellow R/Finance organizer Josh Ulrich, fellow R/Finance regular Kent Hosxey as well as Cliff Clive. I will talk about Redis and my rather recent nascent RcppRedis package.
  • Our sixth international conference R/Finance will take place May 16 and 17, 2014, at UIC in Chicago. See the current agenda for details of the program---which once again will be preceded by one-hour workshops. And as in the past, I will offer a one-hour gentle introduction to Rcpp. This workshop requires registration and a fee, and registraion will close shortly. I will also present a short lightning talk about the RcppRedis package as part of the conference proper.
  • A few days later, on May 20, 2014, I will present a one-hour webinar which is part of a series organized by Ray DiGiacomo and the Orange County R Users Group. See this page for details and registration information; the talk itself will be free of charge.
  • Later in the month, on May 30, 2014, I will give a talk / guest lecture about Rcpp and RcppArmadillo and over in Economics at the University of Chicago. I reckon that this event is not open to anybody not part of U of C.
  • Another three-hour tutorial is scheduled for the tutorial day preceding useR! 2014 at UCLA in early July. This tutorial will also require registration as part of the annual useR! conference. I am very honoured to have been asked to give one of the invited keynotes at useR! 2014. The talk may (or may not) cover at least some aspects of Rcpp. As my talk proposal about the recent RcppZiggurat package for faster Normal RNGs was accepted, I will also give a regular conference talk.

So the month of June remains pretty open. If someone wants me to talk somewhere else, preferably on a Saturday, feel free to get in touch.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 06 May 2014

RcppEigen 0.3.2.1.2

A bug-fix release 0.3.2.1.2 is of RcppEigen is now on CRAN and in Debian. It contains a short upstream patch by Gaek Guennebaud for the UBSAN warnings that were flagged at CRAN (from a rather harmless cpu-id comparison).

The NEWS entry follows.

Changes in RcppEigen version 0.3.2.1.2 (2014-05-05)

  • Applied upstream patch to not trigger g++ UBSAN warning on the cpu id comparison, with thanks to Gael Guennebaud for the patch

Courtesy of CRANberries, there are diffstat reports for the most recent release.

Questions, comments etc about RcppEigen should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 05 May 2014

RcppArmadillo 0.4.300.0

A new release 4.300 of Armadillo, the templated C++ library for linear algebra, was released by Conrad a few days ago.

As usual, this was rolled up in a new RcppArmadillo release 0.4.300.0 which is now on CRAN and in Debian.

This release brings nice new upstream changes such as faster find() as well as corresponding functions to find elements which are either finite or non-finite, based on a suggestion we made. This may need an RcppArmadillo specific extension as R extends the IEEE logic to supported NA, NaN and Inf on floating point variables whereas standard C++ (and hence Armadillo) only supports the latter two. We hope to work on this shortly.

Kevin also put in some code to make sure unsigned int vectors and matrices are converted correctly, and I updated the short configure script checking for complete LAPACK support to account for the R 3.0.3 or later case.

Changes in RcppArmadillo version 0.4.300.0 (2014-05-04)

  • Upgraded to Armadillo release Version 4.300 (Medieval Cornea Scraper)

    • faster find()

    • added find_finite() and find_nonfinite() for finding indices of finite and non-finite elements

    • expressions X=inv(A)*B*C and X=A.i()*B*C are automatically converted to X=solve(A,B*C)

  • Corrected conversion to unsigned int vectors and matrices

  • Configure script now checks for R version 3.0.3 or newer to enable complex divide-and-conquer SVD in case of R-supplied LAPACK

Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 03 May 2014

RcppSMC 0.1.3 and 0.1.4

The very useful Valgrind tool had flagged an actual error in the package which the CRAN maintainers asked us to address. This was followed by a minor brown-bad oversight of a missing delete, also tagged by Valgrind. Both are pretty ancient bugs which we probably should have found aeons ago. Releases 0.1.3 and 0.1.4 made it to CRAN yesterday in short succession.

To recap, RcppSMC combines the SMCTC template classes for Sequential Monte Carlo and Particle Filters (Johansen, 2009, JSS) with the Rcpp package for R/C++ Integration (Eddelbuettel and Francois, 2011, JSS) and thereby allows for easier and more direct access from R to the computational core of the Sequential Monte Carlo algorithm.

The two NEWS entries are below:

Changes in RcppSMC version 0.1.4 (2014-05-02)

  • Added missing delete operator to destructor in sampler

Changes in RcppSMC version 0.1.3 (2014-05-01)

  • Bugfix in Sampler.iterate() for memory overrun detected by valgrind

Courtesy of CRANberries, there are also diffstat reports for 0.1.3. and 0.1.4. As always, more detailed information is on the RcppSMC page,

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Mon, 14 Apr 2014

BH release 1.54.0-2

Yesterday's release of RcppBDT 0.2.3 lead to an odd build error. If one used at the same time a 32-bit OS, a compiler as recent as g++ 4.7 and the Boost 1.54.0 headers (directly or via the BH package) then the file lexical_cast.hpp barked and failed to compile for lack of an 128-bit integer (which is not a surprise on a 32-bit OS).

After looking at this for a bit, and looking at some related bug report, I came up with a simple fix (which I mentioned in an update to the RcppBDT 0.2.3 release post). Sleeping over it, and comparing to the Boost 1.55 file, showed that the hunch was right, and I have since made a new release 1.54.0-2 of the BH package which contains the fix.

Changes in version 1.54.0-2 (2014-04-14)

  • Bug fix to lexical_cast.hpp which now uses the test for INT128 which the rest of Boost uses, consistent with Boost 1.55 too.

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/bh | permanent link

Sun, 13 Apr 2014

RcppBDT 0.2.3

A new release of the RcppBDT package is now on CRAN.

Several new modules were added; the package can now work on dates, date durations, "ptime" (aka posix time), and timezones. Most interesting may be the fact that ptime is configured to use 96 bits. This allows a precise representation of dates and times down to nanoseconds, and permits date and time calculations at this level.

The complete NEWS entry is below:

Changes in version 0.2.3 (2014-04-13)

  • New module 'bdtDt' replacing the old 'bdtDate' module in a more transparent style using a local class which is wrapped, just like the three other new classes do

  • New module 'bdtTd' providing date durations which can be added to dates.

  • New module 'bdtTz' providing time zone information such as offset to UTC, amount of DST, abbreviated and full timezone names.

  • New module 'bdtDu' using 'posix_time::duration' for time durations types

  • New module 'bdtPt' using 'posix_time::ptime' for posix time, down to nanosecond granularity (where hardware and OS permit it)

  • Now selects C++11 compilation by setting CXX_STD = CXX11 in src/Makevars* and hence depend on R 3.1.0 or later – this gives gives us long long needed for the nano-second high-resolution time calculations across all builds and platforms.

Courtesy of CRANberries, there is also a diffstat report for the lastest release. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

Update: I just learned the hard way that the combination of 32-bit OS, g++ at version 4.7 or newer and a Boost version of 1.53 or 1.54 does not work with this new upload. Some Googling suggests that this ought to be fixed in Boost 1.54; seemingly it isn't as our trusted BH package with Boost headers provides that very version 1.54. However, the Googling also suggested a quick two-line fix which I just committed in the Github repo. A new BH package with the fix may follow in a few days.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Thu, 10 Apr 2014

RcppCNPy 0.2.3

R 3.1.0 came out today. Among the (impressive and long as usual) list of changes is the added ability to specify CXX_STD = CXX11 in order to get C++11 (or the best available subset on older compilers). This brings a number of changes and opportunities which are frankly too numerous to be discussed in this short post. But it also permits us, at long last, to use long long integer types.

For RcppCNPy, this means that we can finally cover NumPy integer data (along with the double precision we had from the start) on all platforms. Python encodes these as an int64, and that type was unavailable (at least in 32-bit OSs) until we got long long made available to us by R. So today I made the change to depend on R 3.1.0, and select C++11 which allowed us to free the code from a number if #ifdef tests. This all worked out swimmingly and the new package has already been rebuilt for Windows.

I also updated the vignette, and refreshed its look and feel. Full changes are listed below.

Changes in version 0.2.3 (2014-04-10)

  • src/Makevars now sets CXX_STD = CXX11 which also provides the long long type on all platforms, so integer file support is no longer conditional.

  • Consequently, code conditional on RCPP_HAS_LONG_LONG_TYPES has been simplified and is no longer conditional.

  • The package now depends on R 3.1.0 or later to allow this.

  • The vignette has been updated and refreshed to reflect this.

CRANberries also provides a diffstat report for the latest release. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 08 Apr 2014

BH release 1.54.0-1

A new release of the BH package is now on CRAN and its mirrors. BH provides (a sizeable subset of) the Boost library for C++, particularly (large) parts delivered as pure template headers not requiring linking. See the BH page for more details.

This release provides our first update relative to the Boost tarballs we started with. It moves us from 1.51.0 (which was getting a little long in the tooth) to 1.54.0. This is just about the first time ever that I didn't package something straight from the current release (now 1.55.0). My aim was to to balance the oh, shiny, new aspect with some stability. Comments welcome--maybe I will go to the bleeding edge next time.

As before, the CRAN is created by running bcp over a number of selected components of Boost. If you'd like to see additional ones include, please do get in touch too. Before uploading, I also tested against all of these sixteen CRAN dependents I could quickly test on my server given the installed dependencies there.

The complete list changes follows below.

Changes in version 1.54.0-1 (2014-04-07)

  • Upgraded to Boost 1.54.0

  • Adjust build script local/script/CreateBoost.sh accordingly

  • Renamed generation_runge_kutta_cash_karp54_classic.hpp to generation_runge_kutta_cash_karp54_cl.hpp to remain within 100-character limit for tar

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/bh | permanent link

Mon, 07 Apr 2014

RcppArmadillo 0.4.200.0

Conrad released a new upstream release 4.200 for Armadillo, his templated C++ library for linear algebra, earlier today.

As usual, this was rolled up in a new RcppArmadillo release 0.4.200.0; I had actually made two pre-releases leading up his 4.200 release and was hence not expecting any surprises. Prior to uploading to CRAN, I also tested against all 61 reverse dependencies of the package. Last but not least, I also updated the corresponding Debian package.

Changes in RcppArmadillo version 0.4.200.0 (2014-04-07)

  • Upgraded to Armadillo release Version 4.200 (Flintlock Swoop)

    • faster transpose of sparse matrices

    • more efficient handling of aliasing during matrix multiplication

    • faster inverse of matrices marked as diagonal

Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Sat, 29 Mar 2014

R / Finance 2014 Open for Registration

The annoucement below just went to the R-SIG-Finance list. More information is as usual at the R / Finance page:

Now open for registrations:

R / Finance 2014: Applied Finance with R
May 16 and 17, 2014
Chicago, IL, USA

The registration for R/Finance 2014 -- which will take place May 16 and 17 in Chicago -- is now open!

Building on the success of the previous conferences in 2009, 2010, 2011, 2012 and 2013, we expect around 300 attendees from around the world. R users from industry, academia, and government will joining 30+ presenters covering all areas of finance with R.

We are very excited about the four keynotes by Bill Cleveland, Alexios Ghalanos, Bob McDonald and Luke Tierney. The main agenda (currently) includes sixteen full presentations and twenty-one shorter "lightning talks". We are also excited to offer four optional pre-conference seminars on Friday morning.

To celebrate the sixth year of the conference in style, the dinner will be returning to The Terrace of the Trump Hotel. Overlooking the Chicago River and skyline, it is a perfect venue to continue conversations while dining and drinking.

More details of the agenda are available at:
http://www.RinFinance.com/agenda/
Registration information is available at
http://www.RinFinance.com/register/
and can also be directly accessed by going to
http://www.regonline.com/RFinance2014
We would to thank our 2014 Sponsors for the continued support enabling us to host such an exciting conference:

International Center for Futures and Derivatives at UIC

Revolution Analytics
MS-Computational Finance at University of Washington

OneMarketData
RStudio

On behalf of the committee and sponsors, we look forward to seeing you in Chicago!
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

See you in Chicago in May!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/computers/R | permanent link

Thu, 27 Mar 2014

Upcoming Rcpp talks and workshops

A number of talks about Rcpp are scheduled over the next couple of months:

  • Our sixth internation conference R/Finance, will take place May 16 and 17, 2014, at UIC in Chicago. See the current draft agenda for details of the program---which once again will be preceded by one-hour workshops. And as in the past, I will offer a one-hour gentle introduction to Rcpp. This workshop requires registration and a fee, and registraion details should be available at the conference website in the next few days.
  • A few days later, on May 20, 2014, I will present a one-hour webinar which is part of a series organized by Ray DiGiacomo and the Orange County R Users Group. See this page for details and registration information; the talk itself will be free of charge.
  • Later in the month, on May 30, 2014, I will give a talk / guest lecture about Rcpp and RcppArmadillo and over in Economics at the University of Chicago. I reckon that this event is not open to anybody not part of U of C.
  • Another three-hour tutorial is scheduled for the tutorial day preceding useR! 2014 at UCLA in early July. This tutorial will also require registration as part of the annual useR! conference.
  • Last but not least I am very honoured to have been asked to give one of the invited keynotes at useR! 2014. The talk may (or may not) cover at least some aspects of Rcpp.

So the month of June is still pretty open. If someone wants me to talk somewhere else, preferably on a Saturday, feel free to get in touch.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 25 Mar 2014

RProtoBuf 0.4.1

A new bug-fix release release 0.4.1 of RProtoBuf, is now on CRAN. RProtoBuf provides GNU R bindings for the Google Protocol Buffers ("Protobuf") data encoding library used and released by Google, and deployed as a language and operating-system agnostic protocol by numerous projects.

Murray once more shouldered most of the actual burden and fixed a number of issues detailed below.

Changes in RProtoBuf version 0.4.1 (2014-03-25)

  • Document and add a test for the deprecated group functionality.

  • Add a CITATION file pointing to our arXiv.org preprint.

  • Fix a bug in the show method for EnumDescriptor types.

  • Import all top-level enums from imported .proto files.

  • Removed duplicate enum value type from the unit tests that caused problems with the most recent libprotobuf-2.5. (without option allow_alias).

CRANberries also provides a diff to the previous release. More information is at the RProtoBuf page which has a draft package vignette, a 'quick' overview vignette and a unit test summary vignette. Questions, comments etc should go to the rprotobuf mailing list off the RProtoBuf page at R-Forge.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rprotobuf | permanent link

Fri, 14 Mar 2014

Rcpp 0.11.1

A new minor release 0.11.1 of Rcpp is now on the CRAN network for GNU R; binaries for Debian have also been uploaded.

The release fixes a number of bugs that have come up since the 0.11.0 release in January, but also brings some extensions. See the NEWS file section below for details, or the ChangeLog file in the package and on the Rcpp Changelog page

Once again, we tested this release by building against all CRAN packages which depend upon Rcpp. In short, three packages are blacklisted from tests, and three came up with something we noted --- but the remaining 177 packages all build and test cleanly. Detailed results of those tests (and the scripts for it) are on GitHub.

There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page.

Changes in Rcpp version 0.11.1 (2014-03-13)

  • Changes in Rcpp API:

    • Preserve backwards compatibility with Rcpp 0.10.* by allowing RObject extraction from vectors (or lists) of Rcpp objects

    • Add missing default constructor to Reference class that was omitted in the header-only rewrite

    • Fixes for NA and NaN handling of the IndexHash class, as well as the vector .sort() method. These fixes ensure that sugar functions depending on IndexHash (i.e. unique(), sort_unique(), match()) will now properly handle NA and NaN values for numeric vectors.

    • DataFrame::nrows now more accurately mimics R's internal behavior (checks the row.names attribute)

    • Numerous changes to permit compilation on the Solaris OS

    • Rcpp vectors gain a subsetting method – it is now possible to subset an Rcpp vector using CharacterVectors (subset a by name), LogicalVectors (logical subsetting), and IntegerVectors (0-based index subsetting). Such subsetting will also work with Rcpp sugar expressions, enabling expressions such as x[ x > 0].

    • Comma initialization (e.g. CharacterVector x = "a", "b", "c";, has been disabled, as it causes problems with the behavior of the = operator with Rcpp::Lists. Users who want to re-enable this functionality can use #define RCPP_COMMA_INITIALIZATION, but be aware of the above caveat. The more verbose CharacterVector x = CharacterVector::create("a", "b", "c") is preferred.

  • Changes in Rcpp Attributes

    • Fix issue preventing packages with Rcpp::interfaces attribute from compiling.

    • Fix behavior with attributes parsing of ::create for default arguments, and also allow constructors of a given size (e.g. NumericVector v = NumericVector(10)) gives a default value of numeric(10) at the R level). Also make NAs preserve type when exported to R (e.g. NA_STRING as a default argument maps to NA_character_ at the R level)

  • Changes in Rcpp modules

    • Corrected the un_pointer implementation for object

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rcpp | permanent link

Tue, 11 Mar 2014

QuantLib 1.4 packages also available for Ubuntu and Windoze

QuantLib release 1.4 came out a couple of days ago. Sticking with standard practice, I immediately updated the Debian packages I have been maintaining for over a dozen years now, and corresponding binaries are available for Debian flavors unstable ("sid") and testing ("jessie"). I also updated my RQuantLib package accordingly (though it needed no changes -- yay to stable APIs).

However, I also run Ubuntu on a few machines at work and home, and proceeded to create local packages for both 32-bit ("i386") and 64-bit ("amd64") variants of the current release 13.10 ("saucy"). As these may be of use to others, I am now making these available here from my box so that folks can run a current QuantLib version on those machines.

Lastly, Brian Ripley was kind enough to also fire up his cross-compilation setup on his Linux box to create similar Windows binaries, These are e.g. used on CRAN and by the win-builder service to compile the RQuantLib CRAN Windows builds of RQuantLib. That corresponding archive file can be obtained as Quantlib14.zip see this short page for details. My thanks to Brian for helping with this.

I hope this may be useful to the few people wishing to compile and/or extend RQuantLib and QuantLib.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/snippets | permanent link

Sun, 09 Mar 2014

RQuantLib 0.3.12

A new bug-fix / maintenance release RQuantLib 0.3.12 is now on CRAN and in Debian.

While adding the plugin feature, I was too optmistic about finding the quantlib-config script. This is now tested more carefully. A few things related to building with Rcpp were updated as well now that Rcpp 0.11.0 is out.

Thanks to CRANberries, there is also a diff to the previous release 0.3.11. Full changelog details, examples and more details about this package are at my RQuantLib page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

/code/rquantlib | permanent link