Dirk Eddelbuettel Thinking inside the box
 
Fri, 12 Mar 2010

Rcpp 0.7.9
Version 0.7.8 of Rcpp, released just a few days ago contained a nasty bug or two which we noticed when trying to built the initial release of RcppArmadillo on 64-bit platforms.

So a quick bug-fix release 0.7.9 is now in Debian and should be on CRAN shortly.

The full NEWS entry for this release follows:

0.7.9   2010-03-12

    o	Another small improvement to Windows build flags

    o	bugfix on 64 bit platforms. The traits classes (wrap_type_traits, etc)
	used size_t when they needed to actually use unsigned int

    o	fixed pre gcc 4.3 compatibility. The trait class that was used to 
	identify if a type is convertible to another had too many false positives
	on pre gcc 4.3 (no tr1 or c++0x features). fixed by implementing the 
	section 2.7 of "Modern C++ Design" book. 

As always, even fuller details are in Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Update: First version number corrected to 0.7.8.

/code/rcpp | permanent link

Thu, 11 Mar 2010

RcppArmadillo 0.1.0
Besides the new RcppExamples, another new package RcppArmadillo got spun out of Rcpp with the recent release 0.7.8 of Rcpp.

Romain and I already had an example of a simple but fast linear model fit using the (very clever) Armadillo C++ library by Conrad Sanderson. In fact, I had used this as a motivational example of why Rcpp rocks in a recent talk to the ACM chapter at U of Chicago which, thanks to David Smith at REvo, got some further exposure.

Now this example is more refined as further glue got added. Given that both Armadillo and Rcpp make use of C++ templates, the actual amount of code in RcppArmadillo is not that large: just over 200 lines in a header file, and a little less for some testing accessor and example functions in a source file. And this makes for some really nice example code: the 'fast regression' example becomes this (where I simply removed two blocks with conditional on the Armadillo version):

#include <RcppArmadillo.h>

extern "C" SEXP fastLm(SEXP ys, SEXP Xs) {

    Rcpp::NumericVector yr(ys);			// creates Rcpp vector from SEXP
    Rcpp::NumericMatrix Xr(Xs);			// creates Rcpp matrix from SEXP
    int n = Xr.nrow(), k = Xr.ncol();

    arma::mat X(Xr.begin(), n, k, false);   	// reuses memory and avoids extra copy
    arma::colvec y(yr.begin(), yr.size(), false);

    arma::colvec coef = solve(X, y);            // fit model y ~ X
    arma::colvec resid = y - X*coef; 		// residuals
    double sig2 = arma::as_scalar( trans(resid)*resid/(n-k) );
    						// std.error of estimate
    arma::colvec stderrest = sqrt( sig2 * diagvec( arma::inv(arma::trans(X)*X)) );

    Rcpp::Pairlist res(Rcpp::Named( "coefficients", coef),
                       Rcpp::Named( "stderr", stderrest));
    return res;
}

No extra copies! Armadillo instantiates directly from the underlying R objects for the vector and matrix, solves the regression equations, computes the standard error of the estimates and returns the two vectors. Leaving us to write about eleven lines of code. Moreover, as Armadillo is well designed and uses template meta-programming to avoid extra copies (see these lecture notes for details), it is about as efficient as it can be (and will use Atlas or other BLAS where available).

And, this is just one example. Rcpp should be suitable for other C++ libraries, and provides an easy to use seamless interface between C++ and R.

However, we should note that (at about the last minute) we found out about some unit test failures in OS X as well as some issues in a Debian chroot -- cran2deb ran into some build issues on i386 and amd64 in the testing chroot even this 'it all works' swimmingly on our Debian, Ubuntu and Fedora build environments. A follow-up with fixes for either Rcpp and/or RcppArmadillo appears likely.

Update: The build issues seems to be with 64-bit systems and everything appears cool in 32-bit.

/code/rcpp | permanent link

Wed, 10 Mar 2010

RcppExamples 0.1.0
Version 0.1.0 of RcppExamples, a simple demo package for Rcpp should appear on CRAN some time tomorrow.

As mentioned in the post about release 0.7.8 of Rcpp, Romain and I carved this out of Rcpp itself to provide a cleaner separation of code that implements our R / C++ interfaces (which remain in Rcpp) and code that illustrates how to use it --- which is now in RcppExamples. This also provides an easier template for people wanting to use Rcpp in their packages as it will be easier to wrap one's head around the much smaller RcppExamples package.

A simple example (using the newer API) may illustrate this:

#include <Rcpp.h>

RcppExport SEXP newRcppVectorExample(SEXP vector) {

    Rcpp::NumericVector orig(vector);			// keep a copy (as the classic version does)
    Rcpp::NumericVector vec(orig.size());		// create a target vector of the same size

    // we could query size via
    //   int n = vec.size();
    // and loop over the vector, but using the STL is so much nicer
    // so we use a STL transform() algorithm on each element
    std::transform(orig.begin(), orig.end(), vec.begin(), sqrt);

    Rcpp::Pairlist res(Rcpp::Named( "result", vec),
                       Rcpp::Named( "original", orig));

    return res;
}

With essentially five lines of code, we provide a function that takes any numeric vector and returns both the original vector and a tranformed version---here by applying a square root operation. Even the looping along the vector is implicit thanks to the generic programming idioms of the Standard Template Library.

Nicer still, even on misuse, exceptions get caught cleanly and we get returned to the R prompt without any explicit coding on the part of the user:

R> library(RcppExamples)
Loading required package: Rcpp
R> print(RcppVectorExample( 1:5, "new" )) # select new API
$result
[1] 1.000 1.414 1.732 2.000 2.236

$original
[1] 1 2 3 4 5

R> RcppVectorExample( c("foo", "bar"), "new" )
Error in RcppVectorExample(c("foo", "bar"), "new") :
  not compatible with INTSXP
R>

There is also analogous code for the older API in the package, but it is about three times as long, has to loop over the vector and needs to set up the execption handling explicitly.

As of right now, RcppExamples does not document every class but it should already provide a fairly decent start for using Rcpp. And many more actual usage examples are ... in the over two-hundred unit tests in Rcpp.

Update: Now actually showing new rather than classic API.

/code/rcpp | permanent link

Tue, 09 Mar 2010

Rcpp 0.7.8
Version 0.7.8 of the Rcpp R / C++ interface classes is now on CRAN and in Debian. As of right now. Debian has already built packages for eight more architectures; and CRAN has built the Windows binary. Oh, and cran2deb had Debian packages for 'testing' before I was done with the blog entry.

This is a minor feature release based on a over three weeks of changes that are summarised below in the extract from the NEWS file. Some noteworthy highlights are

  • something that isn't there: we have split most of the example code and their manual pages off into a new package RcppExamples which can now be released given that 0.7.8 is out
  • another new package RcppArmadillo will also be forthcoming shortly: it shows how to use Rcpp with Conrad Sanderson's excellent Armadillo C++ library for linear algebra; this required some internal code changes to seamlessly pass data from R via Rcpp to Armadillo and back;
  • there is a new example fastLm using Armadillo for faster (than lm() or lm.fit()) linear model fits
  • yet more internal improvements to the class hierarchy as detailed below; more support for STL iterators and algorithms;
  • more build fixes; paths with spaces in the name should now be tolerated
  • and last but not least a new introduction / overview vignette based on a just-submitted paper on Rcpp.

The full NEWS entry for this release follows:

0.7.8   2010-03-09

    o	All vector classes are now generated from the same template class
    	Rcpp::Vector where RTYPE is one of LGLSXP, RAWSXP, STRSXP,
    	INTSXP, REALSXP, CPLXSXP, VECSXP and EXPRSXP. typedef are still 
    	available : IntegerVector, ... All vector classes gain methods 
    	inspired from the std::vector template : push_back, push_front, 
    	erase, insert
    	
    o	New template class Rcpp::Matrix deriving from 
    	Rcpp::Vector. These classes have the same functionality
    	as Vector but have a different set of constructors which checks
    	that the input SEXP is a matrix. Matrix<> however does/can not
    	guarantee that the object will allways be a matrix. typedef 
    	are defined for convenience: Matrix is IntegerMatrix, etc...
    	
    o	New class Rcpp::Row that represents a row of a matrix
    	of the same type. Row contains a reference to the underlying 
    	Vector and exposes a nested iterator type that allows use of 
    	STL algorithms on each element of a matrix row. The Vector class
    	gains a row(int) method that returns a Row instance. Usage 
    	examples are available in the runit.Row.R unit test file
    	
    o	New class Rcpp::Column that represents a column of a 
    	matrix. (similar to Rcpp::Row). Usage examples are 
    	available in the runit.Column.R unit test file

    o	The Rcpp::as template function has been reworked to be more 
    	generic. It now handles more STL containers, such as deque and 
    	list, and the genericity can be used to implement as for more
    	types. The package RcppArmadillo has examples of this

    o   new template class Rcpp::fixed_call that can be used in STL algorithms
	such as std::generate.

    o	RcppExample et al have been moved to a new package RcppExamples;
        src/Makevars and src/Makevars.win simplified accordingly

    o	New class Rcpp::StringTransformer and helper function 
    	Rcpp::make_string_transformer that can be used to create a function
    	that transforms a string character by character. For example
    	Rcpp::make_string_transformer(tolower) transforms each character
    	using tolower. The RcppExamples package has an example of this.
        
    o	Improved src/Makevars.win thanks to Brian Ripley

    o	New examples for 'fast lm' using compiled code: 
        - using GNU GSL and a C interface
        - using Armadillo (http://arma.sf.net) and a C++ interface
        Armadillo is seen as faster for lack of extra copying

    o	A new package RcppArmadillo (to be released shortly) now serves 
        as a concrete example on how to extend Rcpp to work with a modern 
	C++ library such as the heavily-templated Armadillo library

    o	Added a new vignette 'Rcpp-introduction' based on a just-submitted 
        overview article on Rcpp

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Update: Two links corrected.

/code/rcpp | permanent link

Sun, 14 Feb 2010

Rcpp 0.7.7
Version 0.7.7, a shiny new bug fix release of Rcpp, our set of R / C++ interface classes, just arrived on CRAN and in Debian. The Language class had a real bug leading to this new release just two days after 0.7.6.

0.7.7	2010-02-14

    o	new template classes Rcpp::unary_call and Rcpp::binary_call
    	that facilitates using R language calls together 
    	with STL algorithms.
    	
    o	fixed a bug in Language constructors taking a string as their
    	first argument. The created call was wrong.

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link

Sat, 13 Feb 2010

Rcpp 0.7.6
The new 0.7.6 release of Rcpp, our set of R / C++ interface classes, is now at CRAN and Debian. This comes just a few days after 0.7.5 as we had made a mistake in Makefile.win which is now fixed. A few other things sneaked in while were at it, see the snippet from the NEWS file below or look at Romain's blog where he highlights named-based indexing in vectors and the addition of iterator as well as begin() and end() members that now allow the use of STL algorithms on our R objects which is nifty.

The changes are summarised below in the NEWS file snippet, more details are in the ChangeLog as well.

0.7.6   2010-02-12

    o   SEXP_Vector (and ExpressionVector and GenericVector, a.k.a List) now
	have methods push_front, push_back and insert that are templated

    o   SEXP_Vector now has int- and range-valued erase() members

    o   Environment class has a default constructor (for RInside)

    o   SEXP_Vector_Base factored out of SEXP_Vector (Effect. C++ #44)

    o   SEXP_Vector_Base::iterator added as well as begin() and end()
        so that STL algorithms can be applied to Rcpp objects

    o   CharacterVector gains a random access iterator, begin() and end() to
	support STL algorithmsl; iterator dereferences to a StringProxy

    o   Restore Windows build; successfully tested on 32 and 64 bit;

    o   Small fixes to inst/skeleton files for bootstrapping a package

    o   RObject::asFoo deprecated in favour of Rcpp::as

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link

Tue, 09 Feb 2010

Rcpp 0.7.5
A new release of our Rcpp R / C++ interface classes is now out, the version number is 0.7.5. It comes on the heels of the release 0.7.4 and keeps with our semi-frantic schedule of releases every ten or so days going. The package is now on CRAN and Debian, and mirrors start to get the new versions. As before, my local page provides more details and Romain's blog is always worth watching too.

The changes are summarised below in the NEWS file snippet, more details are in the ChangeLog as well.

0.7.5	2010-02-08

    o 	wrap has been much improved. wrappable types now are :
    	- primitive types : int, double, Rbyte, Rcomplex, float, bool
    	- std::string
    	- STL containers which have iterators over wrappable types:
    	  (e.g. std::vector, std::deque, std::list, etc ...). 
    	- STL maps keyed by std::string, e.g std::map
    	- classes that have implicit conversion to SEXP
    	- classes for which the wrap template if fully or partly specialized
    	This allows composition, so for example this class is wrappable: 
    	std::vector< std::map > (if T is wrappable)
    	
    o 	The range based version of wrap is now exposed at the Rcpp::
    	level with the following interface : 
    	Rcpp::wrap( InputIterator first, InputIterator last )
    	This is dispatched internally to the most appropriate implementation
    	using traits

    o	a new namespace Rcpp::traits has been added to host the various
    	type traits used by wrap

    o 	The doxygen documentation now shows the examples

    o 	A new file inst/THANKS acknowledges the kind help we got from others

    o	The RcppSexp has been removed from the library.
    
    o 	The methods RObject::asFoo are deprecated and will be removed
    	in the next version. The alternative is to use as.

    o	The method RObject::slot can now be used to get or set the 
    	associated slot. This is one more example of the proxy pattern
    	
    o	Rcpp::VectorBase gains a names() method that allows getting/setting
    	the names of a vector. This is yet another example of the 
    	proxy pattern.
    	
    o	Rcpp::DottedPair gains templated operator<< and operator>> that 
    	allow wrap and push_back or wrap and push_front of an object
    	
    o	Rcpp::DottedPair, Rcpp::Language, Rcpp::Pairlist are less
    	dependent on C++0x features. They gain constructors with up
    	to 5 templated arguments. 5 was choosed arbitrarily and might 
    	be updated upon request.
    	
    o	function calls by the Rcpp::Function class is less dependent
    	on C++0x. It is now possible to call a function with up to 
    	5 templated arguments (candidate for implicit wrap)
    	
    o	added support for 64-bit Windows (thanks to Brian Ripley and Uwe Ligges)

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link

Sun, 31 Jan 2010

Rcpp 0.7.4
Yesterday, and about nine days after release 0.7.3 of Rcpp (a set of R / C++ interface classes), Romain and I released version 0.7.4. It has been uploaded to CRAN and Debian, and mirrors should already have new versions. As before, my local page is also available for downloads and some more details.

The release once again combines a number of necessary fixes with numerous new features:

  • Building on OS X did not support multi-arch, and we are grateful for Simon who once again came to the rescue. Things should be fine now. The big take-away is that under no circumstances, include either a file configure or src/Makefile if you want multi-arch builds for free. As Rcpp is effectively a library to be used by other packages, this mattered.
  • We added a file NEWS from which I include the relevant section below.
  • Much more code re-organisation and enhancement making passage of various C++ types even easier -- see the NEWS entry below.
  • More unit tests, now including ones for the 'old Rcpp API'.
Post-release, I also reworked the doxygen setup slightly so that all examples are now browseable, and the whole documentation is now searchable as well.

Lastly, we had a remaining Windows build issue. Also, Brian Ripley and Uwe Ligges kindly sent us a small patch supporting the new Windows 64-bit builds using the new MinGW 64-bit compiler for Windows -- so release 0.7.5 may follow in due course.

The NEWS file entry for release 0.7.4 is as follows:

0.7.4	2010-01-30

    o	matrix matrix-like indexing using operator() for all vector 
    	types : IntegerVector, NumericVector, RawVector, CharacterVector
    	LogicalVector, GenericVector and ExpressionVector. 

    o	new class Rcpp::Dimension to support creation of vectors with 
    	dimensions. All vector classes gain a constructor taking a 
    	Dimension reference.

    o	an intermediate template class "SimpleVector" has been added. All
    	simple vector classes are now generated from the SimpleVector 
    	template : IntegerVector, NumericVector, RawVector, CharacterVector
    	LogicalVector.

    o	an intermediate template class "SEXP_Vector" has been added to 
    	generate GenericVector and ExpressionVector.

    o	the clone template function was introduced to explicitely
    	clone an RObject by duplicating the SEXP it encapsulates.

    o	even smarter wrap programming using traits and template
        meta-programming using a private header to be include only
        RcppCommon.h

    o 	the as template is now smarter. The template now attempts to 
    	build an object of the requested template parameter T by using the
    	constructor for the type taking a SEXP. This allows third party code
    	to create a class Foo with a constructor Foo(SEXP) to have 
    	as for free.

    o	wrap becomes a template. For an object of type T, wrap uses
    	implicit conversion to SEXP to first convert the object to a SEXP
    	and then uses the wrap(SEXP) function. This allows third party 
    	code creating a class Bar with an operator SEXP() to have 
    	wrap for free.

    o	all specializations of wrap :  wrap, wrap< vector >
    	use coercion to deal with missing values (NA) appropriately.

    o	configure has been withdrawn. C++0x features can now be activated
    	by setting the RCPP_CXX0X environment variable to "yes".

    o	new template r_cast to facilitate conversion of one SEXP
    	type to another. This is mostly intended for internal use and 
    	is used on all vector classes

    o	Environment now takes advantage of the augmented smartness
    	of as and wrap templates. If as makes sense, one can 
    	directly extract a Foo from the environment. If wrap makes
    	sense then one can insert a Bar directly into the environment. 
    	Foo foo = env["x"] ;  /* as is used */
	Bar bar ;
	env["y"] = bar ;      /* wrap is used */    	

    o	Environment::assign becomes a template and also uses wrap to 
    	create a suitable SEXP

    o	Many more unit tests for the new features; also added unit tests
        for older API

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link