Rcpp 0.7.9
Version 0.7.8 of Rcpp, released just a few days ago contained a nasty bug or two which we noticed when trying to built the initial release of RcppArmadillo on 64-bit platforms.

So a quick bug-fix release 0.7.9 is now in Debian and should be on CRAN shortly.

The full NEWS entry for this release follows:

0.7.9   2010-03-12

    o	Another small improvement to Windows build flags

    o	bugfix on 64 bit platforms. The traits classes (wrap_type_traits, etc)
	used size_t when they needed to actually use unsigned int

    o	fixed pre gcc 4.3 compatibility. The trait class that was used to 
	identify if a type is convertible to another had too many false positives
	on pre gcc 4.3 (no tr1 or c++0x features). fixed by implementing the 
	section 2.7 of "Modern C++ Design" book. 

As always, even fuller details are in Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Update: First version number corrected to 0.7.8.

/code/rcpp | permanent link

RcppArmadillo 0.1.0
Besides the new RcppExamples, another new package RcppArmadillo got spun out of Rcpp with the recent release 0.7.8 of Rcpp.

Romain and I already had an example of a simple but fast linear model fit using the (very clever) Armadillo C++ library by Conrad Sanderson. In fact, I had used this as a motivational example of why Rcpp rocks in a recent talk to the ACM chapter at U of Chicago which, thanks to David Smith at REvo, got some further exposure.

Now this example is more refined as further glue got added. Given that both Armadillo and Rcpp make use of C++ templates, the actual amount of code in RcppArmadillo is not that large: just over 200 lines in a header file, and a little less for some testing accessor and example functions in a source file. And this makes for some really nice example code: the 'fast regression' example becomes this (where I simply removed two blocks with conditional on the Armadillo version):

#include <RcppArmadillo.h>

extern "C" SEXP fastLm(SEXP ys, SEXP Xs) {

    Rcpp::NumericVector yr(ys);			// creates Rcpp vector from SEXP
    Rcpp::NumericMatrix Xr(Xs);			// creates Rcpp matrix from SEXP
    int n = Xr.nrow(), k = Xr.ncol();

    arma::mat X(Xr.begin(), n, k, false);   	// reuses memory and avoids extra copy
    arma::colvec y(yr.begin(), yr.size(), false);

    arma::colvec coef = solve(X, y);            // fit model y ~ X
    arma::colvec resid = y - X*coef; 		// residuals
    double sig2 = arma::as_scalar( trans(resid)*resid/(n-k) );
    						// std.error of estimate
    arma::colvec stderrest = sqrt( sig2 * diagvec( arma::inv(arma::trans(X)*X)) );

    Rcpp::Pairlist res(Rcpp::Named( "coefficients", coef),
                       Rcpp::Named( "stderr", stderrest));
    return res;
}

No extra copies! Armadillo instantiates directly from the underlying R objects for the vector and matrix, solves the regression equations, computes the standard error of the estimates and returns the two vectors. Leaving us to write about eleven lines of code. Moreover, as Armadillo is well designed and uses template meta-programming to avoid extra copies (see these lecture notes for details), it is about as efficient as it can be (and will use Atlas or other BLAS where available).

And, this is just one example. Rcpp should be suitable for other C++ libraries, and provides an easy to use seamless interface between C++ and R.

However, we should note that (at about the last minute) we found out about some unit test failures in OS X as well as some issues in a Debian chroot -- cran2deb ran into some build issues on i386 and amd64 in the testing chroot even this 'it all works' swimmingly on our Debian, Ubuntu and Fedora build environments. A follow-up with fixes for either Rcpp and/or RcppArmadillo appears likely.

Update: The build issues seems to be with 64-bit systems and everything appears cool in 32-bit.

/code/rcpp | permanent link

RcppExamples 0.1.0
Version 0.1.0 of RcppExamples, a simple demo package for Rcpp should appear on CRAN some time tomorrow.

As mentioned in the post about release 0.7.8 of Rcpp, Romain and I carved this out of Rcpp itself to provide a cleaner separation of code that implements our R / C++ interfaces (which remain in Rcpp) and code that illustrates how to use it --- which is now in RcppExamples. This also provides an easier template for people wanting to use Rcpp in their packages as it will be easier to wrap one's head around the much smaller RcppExamples package.

A simple example (using the newer API) may illustrate this:

#include <Rcpp.h>

RcppExport SEXP newRcppVectorExample(SEXP vector) {

    Rcpp::NumericVector orig(vector);			// keep a copy (as the classic version does)
    Rcpp::NumericVector vec(orig.size());		// create a target vector of the same size

    // we could query size via
    //   int n = vec.size();
    // and loop over the vector, but using the STL is so much nicer
    // so we use a STL transform() algorithm on each element
    std::transform(orig.begin(), orig.end(), vec.begin(), sqrt);

    Rcpp::Pairlist res(Rcpp::Named( "result", vec),
                       Rcpp::Named( "original", orig));

    return res;
}

With essentially five lines of code, we provide a function that takes any numeric vector and returns both the original vector and a tranformed version---here by applying a square root operation. Even the looping along the vector is implicit thanks to the generic programming idioms of the Standard Template Library.

Nicer still, even on misuse, exceptions get caught cleanly and we get returned to the R prompt without any explicit coding on the part of the user:

R> library(RcppExamples)
Loading required package: Rcpp
R> print(RcppVectorExample( 1:5, "new" )) # select new API
$result
[1] 1.000 1.414 1.732 2.000 2.236

$original
[1] 1 2 3 4 5

R> RcppVectorExample( c("foo", "bar"), "new" )
Error in RcppVectorExample(c("foo", "bar"), "new") :
  not compatible with INTSXP
R>

There is also analogous code for the older API in the package, but it is about three times as long, has to loop over the vector and needs to set up the execption handling explicitly.

As of right now, RcppExamples does not document every class but it should already provide a fairly decent start for using Rcpp. And many more actual usage examples are ... in the over two-hundred unit tests in Rcpp.

Update: Now actually showing new rather than classic API.

/code/rcpp | permanent link

Rcpp 0.7.8
Version 0.7.8 of the Rcpp R / C++ interface classes is now on CRAN and in Debian. As of right now. Debian has already built packages for eight more architectures; and CRAN has built the Windows binary. Oh, and cran2deb had Debian packages for 'testing' before I was done with the blog entry.

This is a minor feature release based on a over three weeks of changes that are summarised below in the extract from the NEWS file. Some noteworthy highlights are

• something that isn't there: we have split most of the example code and their manual pages off into a new package RcppExamples which can now be released given that 0.7.8 is out
• another new package RcppArmadillo will also be forthcoming shortly: it shows how to use Rcpp with Conrad Sanderson's excellent Armadillo C++ library for linear algebra; this required some internal code changes to seamlessly pass data from R via Rcpp to Armadillo and back;
• there is a new example fastLm using Armadillo for faster (than lm() or lm.fit()) linear model fits
• yet more internal improvements to the class hierarchy as detailed below; more support for STL iterators and algorithms;
• more build fixes; paths with spaces in the name should now be tolerated
• and last but not least a new introduction / overview vignette based on a just-submitted paper on Rcpp.

The full NEWS entry for this release follows:

0.7.8   2010-03-09

    o	All vector classes are now generated from the same template class
    	Rcpp::Vector where RTYPE is one of LGLSXP, RAWSXP, STRSXP,
    	INTSXP, REALSXP, CPLXSXP, VECSXP and EXPRSXP. typedef are still 
    	available : IntegerVector, ... All vector classes gain methods 
    	inspired from the std::vector template : push_back, push_front, 
    	erase, insert
    	
    o	New template class Rcpp::Matrix deriving from 
    	Rcpp::Vector. These classes have the same functionality
    	as Vector but have a different set of constructors which checks
    	that the input SEXP is a matrix. Matrix<> however does/can not
    	guarantee that the object will allways be a matrix. typedef 
    	are defined for convenience: Matrix is IntegerMatrix, etc...
    	
    o	New class Rcpp::Row that represents a row of a matrix
    	of the same type. Row contains a reference to the underlying 
    	Vector and exposes a nested iterator type that allows use of 
    	STL algorithms on each element of a matrix row. The Vector class
    	gains a row(int) method that returns a Row instance. Usage 
    	examples are available in the runit.Row.R unit test file
    	
    o	New class Rcpp::Column that represents a column of a 
    	matrix. (similar to Rcpp::Row). Usage examples are 
    	available in the runit.Column.R unit test file

    o	The Rcpp::as template function has been reworked to be more 
    	generic. It now handles more STL containers, such as deque and 
    	list, and the genericity can be used to implement as for more
    	types. The package RcppArmadillo has examples of this

    o   new template class Rcpp::fixed_call that can be used in STL algorithms
	such as std::generate.

    o	RcppExample et al have been moved to a new package RcppExamples;
        src/Makevars and src/Makevars.win simplified accordingly

    o	New class Rcpp::StringTransformer and helper function 
    	Rcpp::make_string_transformer that can be used to create a function
    	that transforms a string character by character. For example
    	Rcpp::make_string_transformer(tolower) transforms each character
    	using tolower. The RcppExamples package has an example of this.
        
    o	Improved src/Makevars.win thanks to Brian Ripley

    o	New examples for 'fast lm' using compiled code: 
        - using GNU GSL and a C interface
        - using Armadillo (http://arma.sf.net) and a C++ interface
        Armadillo is seen as faster for lack of extra copying

    o	A new package RcppArmadillo (to be released shortly) now serves 
        as a concrete example on how to extend Rcpp to work with a modern 
	C++ library such as the heavily-templated Armadillo library

    o	Added a new vignette 'Rcpp-introduction' based on a just-submitted 
        overview article on Rcpp

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Update: Two links corrected.

/code/rcpp | permanent link

R and Sudoku solvers: Plus ca change...
Christian Robert blogged about a particularly heavy-handed solution to last Sunday's Sudoku puzzle in Le Monde. That had my symapthy as I like evolutionary computing methods, and his chart is rather pretty. From there, this spread on to the REvolutions blogs where David Smith riffed on it, and showed the acual puzzle. That didn't stop things as Christian blogged once more about it, this time welcoming his post-doc Robin Ryder who posts a heavy analysis on all this that is a little much for me at this time of day.

But what everybody seems to be forgetting is that R has had a Sudoku solver for years, thanks to the sudoku package by David Brahm and Greg Snow which was first posted four years ago. What comes around, goes around.

With that, and about one minute of Emacs editing to get the Le Monde puzzle into the required ascii-art form, all we need to do is this:

R> library(sudoku)
R> s <- readSudoku("/tmp/sudoku.txt")
R> s
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
 [1,]    8    0    0    0    0    1    2    0    0
 [2,]    0    7    5    0    0    0    0    0    0
 [3,]    0    0    0    0    5    0    0    6    4
 [4,]    0    0    7    0    0    0    0    0    6
 [5,]    9    0    0    7    0    0    0    0    0
 [6,]    5    2    0    0    0    9    0    4    7
 [7,]    2    3    1    0    0    0    0    0    0
 [8,]    0    0    6    0    2    0    1    0    9
 [9,]    0    0    0    0    0    0    0    0    0
R> system.time(solveSudoku(s))
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
 [1,]    8    4    9    6    7    1    2    5    3
 [2,]    6    7    5    2    4    3    9    1    8
 [3,]    3    1    2    9    5    8    7    6    4
 [4,]    1    8    7    4    3    2    5    9    6
 [5,]    9    6    4    7    8    5    3    2    1
 [6,]    5    2    3    1    6    9    8    4    7
 [7,]    2    3    1    8    9    4    6    7    5
 [8,]    4    5    6    3    2    7    1    8    9
 [9,]    7    9    8    5    1    6    4    3    2
   user  system elapsed
  5.288   0.004   5.951
R>

Just in case we needed another illustration that it is hard to navigate the riches and wonders that is CRAN...

/computers/R | permanent link

U of C ACM talk
Fellow GSoC mentor and local ACM masterminder Borja Sotomayor had invited me a few months ago to give a talk at the ACM chapter at the University of Chicago. Today was the day, and the slides from the 50-minutes talk on R and extending R with Rcpp are now on my presentations page.

/computers/R | permanent link

Rcpp 0.7.7
Version 0.7.7, a shiny new bug fix release of Rcpp, our set of R / C++ interface classes, just arrived on CRAN and in Debian. The Language class had a real bug leading to this new release just two days after 0.7.6.

0.7.7	2010-02-14

    o	new template classes Rcpp::unary_call and Rcpp::binary_call
    	that facilitates using R language calls together 
    	with STL algorithms.
    	
    o	fixed a bug in Language constructors taking a string as their
    	first argument. The created call was wrong.

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link

Rcpp 0.7.6
The new 0.7.6 release of Rcpp, our set of R / C++ interface classes, is now at CRAN and Debian. This comes just a few days after 0.7.5 as we had made a mistake in Makefile.win which is now fixed. A few other things sneaked in while were at it, see the snippet from the NEWS file below or look at Romain's blog where he highlights named-based indexing in vectors and the addition of iterator as well as begin() and end() members that now allow the use of STL algorithms on our R objects which is nifty.

The changes are summarised below in the NEWS file snippet, more details are in the ChangeLog as well.

0.7.6   2010-02-12

    o   SEXP_Vector (and ExpressionVector and GenericVector, a.k.a List) now
	have methods push_front, push_back and insert that are templated

    o   SEXP_Vector now has int- and range-valued erase() members

    o   Environment class has a default constructor (for RInside)

    o   SEXP_Vector_Base factored out of SEXP_Vector (Effect. C++ #44)

    o   SEXP_Vector_Base::iterator added as well as begin() and end()
        so that STL algorithms can be applied to Rcpp objects

    o   CharacterVector gains a random access iterator, begin() and end() to
	support STL algorithmsl; iterator dereferences to a StringProxy

    o   Restore Windows build; successfully tested on 32 and 64 bit;

    o   Small fixes to inst/skeleton files for bootstrapping a package

    o   RObject::asFoo deprecated in favour of Rcpp::as

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link

Rcpp 0.7.5
A new release of our Rcpp R / C++ interface classes is now out, the version number is 0.7.5. It comes on the heels of the release 0.7.4 and keeps with our semi-frantic schedule of releases every ten or so days going. The package is now on CRAN and Debian, and mirrors start to get the new versions. As before, my local page provides more details and Romain's blog is always worth watching too.

The changes are summarised below in the NEWS file snippet, more details are in the ChangeLog as well.

0.7.5	2010-02-08

    o 	wrap has been much improved. wrappable types now are :
    	- primitive types : int, double, Rbyte, Rcomplex, float, bool
    	- std::string
    	- STL containers which have iterators over wrappable types:
    	  (e.g. std::vector, std::deque, std::list, etc ...). 
    	- STL maps keyed by std::string, e.g std::map
    	- classes that have implicit conversion to SEXP
    	- classes for which the wrap template if fully or partly specialized
    	This allows composition, so for example this class is wrappable: 
    	std::vector< std::map > (if T is wrappable)
    	
    o 	The range based version of wrap is now exposed at the Rcpp::
    	level with the following interface : 
    	Rcpp::wrap( InputIterator first, InputIterator last )
    	This is dispatched internally to the most appropriate implementation
    	using traits

    o	a new namespace Rcpp::traits has been added to host the various
    	type traits used by wrap

    o 	The doxygen documentation now shows the examples

    o 	A new file inst/THANKS acknowledges the kind help we got from others

    o	The RcppSexp has been removed from the library.
    
    o 	The methods RObject::asFoo are deprecated and will be removed
    	in the next version. The alternative is to use as.

    o	The method RObject::slot can now be used to get or set the 
    	associated slot. This is one more example of the proxy pattern
    	
    o	Rcpp::VectorBase gains a names() method that allows getting/setting
    	the names of a vector. This is yet another example of the 
    	proxy pattern.
    	
    o	Rcpp::DottedPair gains templated operator<< and operator>> that 
    	allow wrap and push_back or wrap and push_front of an object
    	
    o	Rcpp::DottedPair, Rcpp::Language, Rcpp::Pairlist are less
    	dependent on C++0x features. They gain constructors with up
    	to 5 templated arguments. 5 was choosed arbitrarily and might 
    	be updated upon request.
    	
    o	function calls by the Rcpp::Function class is less dependent
    	on C++0x. It is now possible to call a function with up to 
    	5 templated arguments (candidate for implicit wrap)
    	
    o	added support for 64-bit Windows (thanks to Brian Ripley and Uwe Ligges)

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link

R / Finance 2010 Open for Registration
The annoucement below went out to R-SIG-Finance earlier today. For information is as usual the the R / Finance 2010 page:

Now open for registrations:

R / Finance 2010: Applied Finance with R
April 16 and 17, 2010
Chicago, IL, USA

The second annual R / Finance conference for applied finance using R, the premier free software system for statistical computation and graphics, will be held this spring in Chicago, IL, USA on Friday April 16 and Saturday April 17.

Building on the success of the inaugural R / Finance 2009 event, this two-day conference will cover topics as diverse as portfolio theory, time-series analysis, as well as advanced risk tools, high-performance computing, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management and trading.

Invited keynote presentations by Bernhard Pfaff, Ralph Vince, Mark Wildi and Achim Zeileis are complemented by over twenty talks (both full-length and 'lightning') selected from the submissions. Four optional tutorials are also offered on Friday April 16.

R / Finance 2010 is organized by a local group of R package authors and community contributors, and hosted by the International Center for Futures and Derivatives (ICFD) at the University of Illinois at Chicago.

Conference registration is now open. Special advanced registration pricing is available, as well as discounted pricing for academic and student registrations.

More details and registration information can be found at the website at
http://www.RinFinance.com

For the program committee:

Gib Bassett, Peter Carl, Dirk Eddelbuettel, John Miller,
Brian Peterson, Dale Rosenthal, Jeffrey Ryan

See you in Chicago in April!

/computers/R | permanent link

RProtoBuf 0.1-0
Romain uploaded our first release of RProtoBuf to CRAN yesterday. RProtoBuf provides bindings for GNU R to the Google Protobuf implementation. Google Protobuf is (and I quote) a way of encoding structured data in an efficient yet extensible format that is used for almost all internal RPC protocols and file formats at Google.

RProtoBuf had a funny start. I had blogged about the 12 hour passage from proof of concept to R-Forge project following the ORD session hackfest in October. What happened next was as good. Romain emailed within hours of the blog post and reminded me of a similar project that is part of Saptarshi Guha's RHIPE R/Hadoop implementation. So the three of us--Romain, Saptarshi and I---started emailing and before long it becomes clear that Romain is both rather intrigued by this (whereas Saptarshi has slightly different needs for the inner workings of his Hadoop bindings) and was able to devote some time to it. So the code kept growing and growing at a fairly rapid clip. Til that stopped as we switched to working feverishly on Rcpp to both support the needs of this project, and to implement ideas we had while working on this. That now lead to the point where Rcpp is maturing in terms of features, so we will probably have time come back to more work on RProtoBuf to take advantage of the nice templated autoconversions we now have in Rcpp. Oddly enough, the initial blog post seemed to anticipate changes in Rcpp.

Anyway -- RProtoBuf is finally here and it already does a fair amount of magic based of code reflection using the proto files. The Google documentation has a simple example of a 'person' entry in an 'addressbook' which, when translated to R, goes like this:

R> library( RProtoBuf )                      ## load the package
R> readProtoFiles( "addressbook.proto" )     ## acquire protobuf information
R> bob <- new( tutorial.Person,              ## create new object
+   email = "bob@example.com",
+   name = "Bob",
+   id = 123 )
R> writeLines( bob$toString() )              ## serialize to stdout
name: "Bob"
id: 123
email: "bob@example.com"

R> bob$email                                 ## access and/or override
[1] "bob@example.com"
R> bob$id <- 5
R> bob$id
[1] 5

R> serialize( bob, "person.pb" )             ## serialize to compact binary format

There is more information at the RProtoBuf page, and we already have a draft package vignette, a 'quick' overview vignette and a unit test summary vignette.

More changes should be forthcoming as Romain and I find time to code them up. Feedback is as always welcome.

/code/rprotobuf | permanent link

Rcpp 0.7.4
Yesterday, and about nine days after release 0.7.3 of Rcpp (a set of R / C++ interface classes), Romain and I released version 0.7.4. It has been uploaded to CRAN and Debian, and mirrors should already have new versions. As before, my local page is also available for downloads and some more details.

The release once again combines a number of necessary fixes with numerous new features:

• Building on OS X did not support multi-arch, and we are grateful for Simon who once again came to the rescue. Things should be fine now. The big take-away is that under no circumstances, include either a file configure or src/Makefile if you want multi-arch builds for free. As Rcpp is effectively a library to be used by other packages, this mattered.
• We added a file NEWS from which I include the relevant section below.
• Much more code re-organisation and enhancement making passage of various C++ types even easier -- see the NEWS entry below.
• More unit tests, now including ones for the 'old Rcpp API'.

Lastly, we had a remaining Windows build issue. Also, Brian Ripley and Uwe Ligges kindly sent us a small patch supporting the new Windows 64-bit builds using the new MinGW 64-bit compiler for Windows -- so release 0.7.5 may follow in due course.

The NEWS file entry for release 0.7.4 is as follows:

0.7.4	2010-01-30

    o	matrix matrix-like indexing using operator() for all vector 
    	types : IntegerVector, NumericVector, RawVector, CharacterVector
    	LogicalVector, GenericVector and ExpressionVector. 

    o	new class Rcpp::Dimension to support creation of vectors with 
    	dimensions. All vector classes gain a constructor taking a 
    	Dimension reference.

    o	an intermediate template class "SimpleVector" has been added. All
    	simple vector classes are now generated from the SimpleVector 
    	template : IntegerVector, NumericVector, RawVector, CharacterVector
    	LogicalVector.

    o	an intermediate template class "SEXP_Vector" has been added to 
    	generate GenericVector and ExpressionVector.

    o	the clone template function was introduced to explicitely
    	clone an RObject by duplicating the SEXP it encapsulates.

    o	even smarter wrap programming using traits and template
        meta-programming using a private header to be include only
        RcppCommon.h

    o 	the as template is now smarter. The template now attempts to 
    	build an object of the requested template parameter T by using the
    	constructor for the type taking a SEXP. This allows third party code
    	to create a class Foo with a constructor Foo(SEXP) to have 
    	as for free.

    o	wrap becomes a template. For an object of type T, wrap uses
    	implicit conversion to SEXP to first convert the object to a SEXP
    	and then uses the wrap(SEXP) function. This allows third party 
    	code creating a class Bar with an operator SEXP() to have 
    	wrap for free.

    o	all specializations of wrap :  wrap, wrap< vector >
    	use coercion to deal with missing values (NA) appropriately.

    o	configure has been withdrawn. C++0x features can now be activated
    	by setting the RCPP_CXX0X environment variable to "yes".

    o	new template r_cast to facilitate conversion of one SEXP
    	type to another. This is mostly intended for internal use and 
    	is used on all vector classes

    o	Environment now takes advantage of the augmented smartness
    	of as and wrap templates. If as makes sense, one can 
    	directly extract a Foo from the environment. If wrap makes
    	sense then one can insert a Bar directly into the environment. 
    	Foo foo = env["x"] ;  /* as is used */
	Bar bar ;
	env["y"] = bar ;      /* wrap is used */    	

    o	Environment::assign becomes a template and also uses wrap to 
    	create a suitable SEXP

    o	Many more unit tests for the new features; also added unit tests
        for older API

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/code/rcpp | permanent link

Rcpp 0.7.3
A quick nine days after release 0.7.2 of Rcpp, our R / C++ interface classes, Romain and I are happy to roll out a new version 0.7.3. It has been uploaded to CRAN and Debian, and mirrors should have the new versions shortly. As before, my local page is also available for downloads and some more details.

This release combines a number of under-the-hood fixes and enhancements with one bug fix:

• The Rcpp:::LdFlags() helper function to dynamically provide linker options for packages using Rcpp now defaults to static linking on OS X as well. For installation from source dynamic linking always worked, but not for binary installation (as e.g. from CRAN). As on the other platforms, this default can be overridden. Thanks to the phylobase team for patient help in tracking this down.
• Accessing various types via [] should now be faster due to some enhancements in the internal representations.
• configure now has a command-line option (as well as an environment variable) to select support for the draft of the upcoming C++0x standard.
• A new function Rcpp.package.skeleton(), modelled after package.skeleton() in R itself, helps to set up a new package with support for using Rcpp.
• A number of other minor tweaks and improvements...

As always, full details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

/computers/linux/debian/packages | permanent link

RQuantLib 0.3.2 released
A new version of RQuantLib (a package combining the quantitative analytics of QuantLib with the R statistical computing environment and language) is now out at CRAN and in Debian (where it depends on the 1.0.0 beta of QuantLib that is currently in the NEW queue with its new library version). This RQuantLib release works with either the current release 0.9.9 as well as with the just-released first beta of QuantLib 1.0.0.

This versions brings a few cleanups due to minor Rcpp changes (in essence: we now define the macro R_NO_REMAP before including R's headers and this separate non-namespaced functions like error() or length() out into prefixed-versions Rf_error() and Rf_length() which is a good thing).

It also adds a number of calendaring and holiday utilities that Khanh just added: tests for weekend, holiday, endOfMonth as well dayCount, date advancement and year fraction functions commonly used in fixed income.

Full changelog details, examples and more details about this package are at my RQuantLib page.

/computers/linux/debian/packages | permanent link

Rcpp 0.7.2
Not even two weeks after the Rcpp 0.7.1 release, Romain and I have a new one to present: Rcpp 0.7.2. It has been uploaded to CRAN and Debian, and the respective package management systems should carry them around in the next few hours. As always, the local page is also available for download too.

A lot of the momentum for the new API is continuing, thanks in large part to Romain. A number of new classes have been added, and existing ones have been enhanced. There are more unit tests than ever, and more documentation. We have better build support (with g++ version detection so that we can add some C++0x support where available) and a new examples sub-directory.

We did take one toy away, though. The Doxygen-generated docs were getting so big that we decided to keep them out of the source tarball. (And arguably, they are also too volatile.) We still have the browseable html docs as well as the pdf version (now at over 300 pages!). And we added zip archives of the docs in html, latex, and man format for download.

As always, full details are in the ChangeLog on the Rcpp page. Questions, comments etc: bring them to the rcpp-devel mailing list off the R-Forge page

/computers/linux/debian/packages | permanent link

Review of 'Computational Statistics: An Introduction to R' in JSS
Somehow missed during the the end-of-year switchover was the fact that my review of Guenther Sawitzki's Computational Statistics: An Introduction to R (CRC / Chapman \& Hall, 2009) is now up on the Journal of Statistical Software website.

/computers/R | permanent link

RInside release 0.2.1
The shiny new 0.2.1 release of RInside, a set of convenience classes to facilitate embedding of R inside of C++ applications, just went out to CRAN; sources are also at my RInside page

This is a maintenance release building on the recent 0.2.0 release which added Windows support (provided you use the Rtools toolchain for Windows). In this release, we changed the startup initialization so that interactive() comes out FALSE (just as we had done for littler just yesterday) and with that no longer call Rf_KillAllDevices() from the destructor as we may not have had devices in the first place. A few minor things were tweaked around the code organisation and build process, see the ChangeLog for details.

The new release should hit CRAN mirrors tomorrow, and is (as always) available from my machine too.

/computers/linux/debian/packages | permanent link

littler 0.1.3
A new littler release (now at 0.1.3) just went out of the door this evening.

littler provides r (pronounced littler), a shebang / scripting / quick eval / pipelining front-end to the the R language and system.

This version adds a few minor behind-the-scenes improvements:

• interactive() now evaluates to false as you'd expect in a non-interactive scripting front-end. To restore the previous behaviour, new switches -i or --interactive have been added.
• Some of the 'cleanup' functionality described in Section 8.1.2 on 'Setting R callbacks' from the R Extension manual have been adopted.
• Example scripts install.r and update.r received an update based on lessons learned from the R 2.10.0 roll-out and package rebuilding.
• A few build issues were improved, a minor manual page formatting bug was fixed.

As usual, our code is available via our svn archive or from tarballs off my littler page and the local directory here. A fresh package is in Debian's incoming queue and will hit mirrors shortly.

/computers/linux/debian/packages | permanent link

Rcpp 0.7.1
Two weeks after the Rcpp 0.7.0 release, Romain and I are happy to announce release 0.7.1 of Rcpp. It is currently in the incoming section of CRAN and has been accepted into Debian. Mirrors will catch up over the next few days, in the meantime the local page is available for download too.

A lot has changed under the hood since 0.7.0, and this is the first release that really reflects many of Romain's additions. Some of the changes are

• A new base class Rcpp::RObject that replaces RcppSexp (which is still provided for compatibility); it provides basic R object handling and other new classes derive from it.
• Rcpp::RObject has real simple wrappers for object creation and a SEXP operator for transfer back to R that make simple interfaces even easier.
• New classes Rcpp::Evaluator and Rcpp::Environment for expression evaluation and R environment access, respectively.
• A new class Rcpp::XPtr for external pointer access and management.
• Enhanced exception handling: exception can be trapped at the R even outside of try/catch blocks, see Romain's blog post for more.
• Namespace support with the addition of a Rcpp namespace; we will be incremental in phasing this in keeping compatibility with the old interface
• Unit test for most all of the above via use of the RUnit package, and several new examples.
• Inline support has been removed and replaced with a Depends: on inline (>= 0.3.4) as our patch is now part of the current inline package as mentioned <here.

/computers/linux/debian/packages | permanent link

inline 0.3.4 released
Oleg has updated the inline package to version 0.3.4 which is now on CRAN. It is includes my patch for both Rcpp support as well as extended header / library options for PKG_CPPFLAGS, PKG_CXXFLAGS, and PKG_LIBS which I had mentioned recently here and here. There is also a brand-new and shiny R-Forge project for it.

A new release of Rcpp will probably follow in due course as we can now depend on this new inline version; Romain has also put nice new code in around exceptions and we may add some more on external pointers. Good times ...

/computers/linux/debian/packages | permanent link

Beancounter minor bug fix release 0.8.9
Another small update to beancounter and just like the previous release due to changes in Perl's Date::Manip. It seems that Date_Calc now really wants a reference to store error codes, so we supply one. I also changed the FX code to use OandA ad there are often issues with Yahoo's FX data. The version 0.8.9 is now in Debian, at CPAN and on my beancounter page here.

/computers/linux/debian/packages | permanent link

RInside release 0.2.0
RInside, a set of convenience classes to facilitate embedding of R inside of C++ applications, was updated to release 0.2.0 over the weekend. Sources as well as binary packages are at CRAN; sources are also at my RInside page

The biggest news is that we now support builds on Windows -- if and only if you use the R toolchain -- more on that below. Since I had first released RInside people had emailed about support on Windows. However, Richard Holbrey deserves special mention as he actually sat down and tried to build it. Over a few email exchanges we made decent progress, but stalled. More recently, I sat down and had another go and lo and behold, it works. This required a few #ifdef statements here and there, as well as a function setenv() which Windows does not have -- Richard kindly helped here and borrowed most of the code from R itself.

Now, to make this plain: you need gcc (as this is how R itself is built). This gcc compiler is the native compiler on Linux and OS X. But on Windows this requires installing the Rtools compiled by Duncan Murdoch as detailed in the The Windows Toolset appendix to the R Installation manual. Do not, I repeat, do not report an error if you fail to build this with another compiler as this is not supported.

That said, if you have the proper tools, things are peachy. Install the package, change into the examples directory installed with it and say make (or on Windows: make -f Makefile.win). That's it -- now run the examples (which on Windows will most likely require setting the environment variable R_HOME). Now you can run simple examples like the one from the previous blog post or any of the other ones.

Happy embedding!

/computers/linux/debian/packages | permanent link

Rcpp and inline example
Following up on yesterday's Rcpp 0.7.0 release, here is a quick example of why the new feastures can be so useful.

The following R code defines a character variable gslrng. This variable contains a short C++ code segment, which is then transformed by the function cfunction into a function of two arguments assigned to funx:

    ## now use Rcpp to pass down a parameter for the seed, and a vector size
    gslrng <- '
       int seed = RcppSexp(s).asInt();
       int len = RcppSexp(n).asInt();

       gsl_rng *r;
       gsl_rng_env_setup();
       std::vector<double> v(len);

       r = gsl_rng_alloc (gsl_rng_default);

       gsl_rng_set (r, (unsigned long) seed);
       for (int i=0; i<len; i++) {
           v[i] = gsl_rng_get (r);
       }
       gsl_rng_free(r);

       return RcppSexp(v).asSexp();
    '

    ## turn into a function that R can call
    ## compileargs redundant on Debian/Ubuntu as gsl headers are found anyway
    funx <- cfunction(signature(s="numeric", n="numeric"),
                      gslrng,
                      includes="#include <gsl/gsl_rng.h>",
                      Rcpp=TRUE,
                      cppargs="-I/usr/include",
                      libargs="-lgsl -lgslcblas")
    print(funx(0, 5))

The signature argument to cfunction defines two variables s and n -- which the C++ function then reads in from R and converts to two integers seed and len. seed is used to initialize the random-number generator, and len draws are then taken and stored in the STL vector v which returned at the end.

As the R level, we now have a function of two arguments returning a vector of RNG draws of the given lenth and using the given seed.

Also note how we tell cfunction to add the GSL include line, specify that we want to compile and link against Rcpp and provide -I and -L arguments to compile and link with the GSL. (The include statement is not needed as the compiler would have found them in /usr/include anyway, but it shows how to set this if needed.)

Finally, we simply call our freshly compiled, linked and loaded C++ function with arguments zero for the seed and five for the length, and print the results vector returned to R from C++.

/computers/linux/debian/packages | permanent link

Rcpp 0.7.0
A new release of Rcpp awaits inclusion on CRAN and has also been sent as a .deb to Debian. It will hit mirrors over the next few days, in the meantime you can get it here as well.

This release has a couple new features :

• inline support: the cfunction from Oleg Sklyar's excellent inline package has been imported and adapted. This means simple C++ programs can be defined in an R character vector and passed to cfunction which will create a complete file that it then compiles, links and loads --- giving you access to compiled C++ code right from the R prompt without having to worry about compiler flags, linker options, ... Better still, we extended this to not only support Rcpp but any external library via addtional header / linker arguments that will be passed to R via the PKG_CPPFLAGS, PKG_CXXFLAGS and PKG_LIBS environment variables.
• this even works on Windoze (if you have the Rtools installed as detailed in the Windows Toolset appendix to the R Installation manual) in exactly the same way. And no folks, that still does NOT mean you can use Visual Whatever -- R really requires MinGW as the links in this parapgraph document very plainly. But if and when you have the tools, R's remarkable consistency across operating systems allows you to use Rcpp and inline in pretty much the same way.
• A handful of new examples for the inline support have been added.
• A new type RcppSexp for simple int, double or std::string scalars as well as vectors; this is particularly useful for the inline support.
• This also completes the source code reorginsation: every class now has its own header and implementation file
• Last but not least, the package has been relicensed from LGPL-2.1 (or later) to GPL 2 (or later).

/computers/linux/debian/packages | permanent link

RQuantLib 0.3.1 released
A new version of RQuantLib is now out at CRAN and in Debian. It updates the package to the 0.9.9 release of QuantLib and is mostly a maintenance release with a few minor fixes along the way.

Full changelog details, examples and more details about this package are at my RQuantLib page.

/computers/linux/debian/packages | permanent link

digest 0.4.2
digest version 0.4.2 is now on CRAN. This new version features a fix by Henrik Bengtsson who also joined the R-Forge-hosted digest project.

/computers/linux/debian/packages | permanent link

Updated slides for 'Introduction to HPC with R' (now with correct URLs)
This is an updated version of yesterday's post with corrected URLs -- by copy-and-pasting I had still referenced the previous slides from UseR! 2009 in Rennes instead of last Friday's slides from the ISM presentation in Tokyo. The presentations page had the correct URLs, and this has been corrected below for this re-post. My apologies!

As mentioned yesterday, I spent a few days last week in Japan as I had an opportunity to present the Introduction to High-Performance Computing with R tutorial at the Institute for Statistical Mathematics in Tachikawa near Tokyo thanks to an invitation by Junji Nakano.

An updated version of the presentations slides (with a few typos corrected) is now available as is a 2-up handout version. Compared to previous versions, and reflecting the fact that this was the 'all-day variant' of almost five hours of lectures, the following changes were made:

• the 'parallel computing' section was expanded further with discussion of the recent R packages multicore, iterators, foreach, doNWS, doSNOW, doMPI;
• a first discussion of GPU computing using the gputools package was added;
• the section on 'out of memory computing' using ff, bigmemory and biglm (including an example borrowed from Jay Emerson) reappeared in this longer version;
• minor fixes and polishing throughout.

Comments and suggestions are, as always, appreciated.

/computers/R | permanent link

Back from Tokyo
Just got back from Tokyo a few hours ago: I had an opportunity to give my 'Introduction to High-Performance Computing with R' tutorial / lectures (for which previous slides can be found here). This was an all-day talk at the Institute for Statistical Mathematics at their new site in Tachikawa in the greater Tokyo area, thanks to an invitation by Junji Nakano.

Lisa and I turned this into a brief one-week trip to Kyoto and Tokyo, and we had a truly wonderful time on what was our first visit to Japan. I should blog some more about it, but now I will give in to the jet lag and catch up on some sleep...

/misc | permanent link

Rcpp 0.6.8
As memtioned when I announced release 0.6.7 of Rcpp two days ago, a new version 0.6.8 was due because of my oversight of also adapting Makevars.win (used on that other OS) to the new file layout.

So 0.6.8 went onto CRAN and into Debian earlier in the day. Beside the aforementioned fix, I also split off a more class headers and implementations into their own files but changed no actual functionality.

As always, more details are at the Rcpp page.

/computers/linux/debian/packages | permanent link

R / Finance 2010 Call for Papers
Jeff sent the following while I had connectivity issues and I hadn't gotten around to posting it here.

So without further ado, and given the success of our initial R / Finance 2009 conference about R in Finance, here is the call for papers for next spring:

Call for Papers:

R/Finance 2010: Applied Finance with R
April 16 and 17, 2010
Chicago, IL, USA

The second annual R/Finance conference for applied finance using R will be held this spring in Chicago, IL, USA on April 16 and 17, 2010. The two-day conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management and trading.

One-page abstracts or complete papers (in txt or pdf format) are invited for consideration. Academic and practitioner research proposals related to R are encouraged. We will accept submissions for full talks, abbreviated "lightning talks", and a limited number of pre-conference tutorial sessions. Please indicate with your submission if you would be willing to produce a formal paper (10-15 pages) for a peer-reviewed conference proceedings publication.

Presenters are strongly encouraged to provide working R code to accompany the presentation/paper. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages.

Please send submissions to: committee at RinFinance.com

The submission deadline is December 31st, 2009.

Submissions will be evaluated and submitters notified via email on a rolling basis. Determination of whether a presentation will be a long presentation or a lightning talk will be made once the full list of presenters is known.

R/Finance 2009 included keynote presentations by Patrick Burns, Robert Grossman, David Kane, Roger Koenker, David Ruppert, Diethelm Wuertz, and Eric Zivot. Attendees included practitioners, academics, and government officials. We anticipate another exciting line-up for 2010 and will announce details at the conference website http://www.RinFinance.com as they become available.

For the program committee:

Gib Bassett, Peter Carl, Dirk Eddelbuettel, John Miller,
Brian Peterson, Dale Rosenthal, Jeffrey Ryan

See you in Chicago in April!

/computers/R | permanent link

Rcpp 0.6.7
Version 0.6.7 of the Rcpp package---a set of C++ classes making it easier to glue C/C++ code to R---is now on CRAN and Debian. This is a fairly incremental release that contains just a few small changes that arose since the previous release in August.

One change is that a new type RcppList was added which allows us to build R 'list' types in C++. In particular, this makes it possible to have data structures of different types and dimensions. We had not previously accomodated this as our data structures where more regularised (i.e. think matrices or vectors of various types). The need for this came out of the work with the recently started RProtoBuf package which interfaces the Google ProtoBuf library. The incredible Romain François has joined the project and added a number of nice tricks; this will need another blogpost sooner rather than later. In the meantime, check out the RProtoBuf pages at R-Forge.

Another small change was the addition of a self-contained example of a function callback from C++ into R which wasn't all that well documented before; this was in response to a user request.

Finally, I started RcppList as a first step in reorganising the code a little better and split the class header and body off into separate files. This required changes to the infrastrucure files Makevars, cleanup, ... and I promptly forgot the Makevars.win Windoze variant which will require a 0.6.8 release real soon.

/computers/linux/debian/packages | permanent link

Adventures with Comcast: Part ohnoesnotanotherone in an ongoing series
Regular readers of this blog (yes, both of you!) may remember the computer/broadband/ directory that this post appears in as the collection of my Comcastic (yeah right) experiences with my ISP.

But I think this week may top everything. I'll just try to jot down some notes before I forget all the gory details:

• On Tuesday, I edited one of the internal nvram configuration variables of my trusted wrt54g router in order to add the older daughter's shiny new iPod Touch to the set of 'permitted' MAC addresses. This router, running a custom Linux variant called OpenWRT had essentially not been upgraded since I first installed it, and still required a quick reboots after updating of configuration values. However, that worked fairly flawelessly for 4 1/2 years. Until Tuesday.
• Upon reboot, I got what appeared to be an invalid network setup from the cable modem. IP and Gateway assigned , but no DNS and no ability to ping anywhere. Crap. So I fiddled with this all evening, including a service call to Comcast but to no avail. When a laptop was directly plugged into the cable modem, it got correct settings albeit on a completely different subnet. So for the next day, we left one machine directly plugged so that my wife could at least telecommute.
• Somehow it became apparent that waiting insanely long for the router to remain powered-down -- and we're talking five minutes or longer -- helped. So by now we were suspecting the cable modem. I use a standard Motorola SB5101 I once had to buy in a rush because of the Comcastic ones who all of a sudden changed their minimum requirements which meant they would no longer connect to my existing modem. Anyway. So on Wednesday I called Motorola and had a decent service call with them but as I was at work I couldn't follow up with part numbers etc pp. At least I learned that I seem to have two months of the two-year warranty left...
• So by Wednesday evening I decided to fall back to the really cheap and old Speedstream router I had used before the Linksys wrt54g. That worked, albeit sloooooowly. Wired and wireless ethernet, direct assignment from the cable modem. All well. But did I mention it was sloooow though?
• Thursday evening was skipped as I was at the Chicago R meeting we organize to complement our R / Finance conferences in the spring.
• Given that the cable modem worked with the old Speedstream and with directly-connected machines, I decided to finally go for a long overdue update of the wrt54g software. So that happened on Friday, i.e. yesterday. And similar to my previous wrt54g notes, I needed to flash the new software with the tftp protocol and a helper script on a laptop connected to the router. All this took a while as I needed to remember to also send to a ping flood to the router to be able to catch the tftp request, needed to test which of the atftp and tftp binaries worked reliably, and whether the router prefers .bin images over .trx images when using the tftp protocol. But lo and behold this worked, and I configured a shiny new Kamikaze aka 8.09.1 version of OpenWRT. This even connected to the cable modem once I helped with DNS entries. OpenWRT generally rocks, and this new release is a lot nice than the more bare-bones version I used to run.
• Unfortunately, I had picked the bcrm47xx variant -- the 2.6.* kernel version of the OpenWRT Project's software for my WRT device. And guess what, that one does not include wireless support due to issues with Broadcom drivers and the kernel. Grrr. So once I had that confirmed this morning, I quickly switched to the bcrm-2.4 variant of the same 8.09.1 release. At least now I can flash from within using the mtd command from the commandline.
• But once up and running with the bcrm-2.4 release, I ran into the same issue we have had with the Motorola cable modem and Comcast behind them. Each time I connect with the wrt54g, I end up on a specific subnet, without DNS and with no ability to connect. The Speedstream still worked. So what to do? Well, MAC Cloning to the rescue. Now the Linksys wrt54g pretends to be the Speedstream, and all, at last, is well again.
• So after four days of intermittent service, which means that my few web pages, blog, and goodies like CRANberries were invisible, I now have better router software. That could have come a little easier, and I still don't quite know why Comcast decides to no longer service the wrt54g under the MAC address it presented itself with for 4 1/2 years. I have paid thousands of dollars over that time to get broadband access. But this, I don't quite call service. To top it all off, guess who cold-called to sell VOIP service while I wrote this up? Oh, it's Comcastic ...

/computers/broadband | permanent link

From ORD Sessions to R-Forge in 12 hours with RProtoBuf
Yesterday, via in invitation from fellow Chicago-area Google Summer of Code mentor Borja Sotomayor, I attended the Second ORD Sessions. These are happening at the HQ of Inventable where a couple of technologists and Open Source geeks from the Chicagoland area get together and riff on code for a few hours after work over some pizza and beer.

Sounded good, and I needed an excuse to try to mix the awesome Protocol Buffers with my favourite data tool, R. What are Protocol Buffers? To quote from the Google overview page referenced above:

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.

Protocol buffers are now Google's lingua franca for data – at time of writing, there are 48,162 different message types defined in the Google code tree across 12,183 .proto files. They're used both in RPC systems and for persistent storage of data in a variety of storage systems.

So three hours later, I had an implementation of the 'addressbook reader' C++ example wrapped in a tiny yet complete R package that passed R CMD check. And one lingua franca for data has met another.

So before going to bed, I quickly registered a new project at R-Forge, everybody's favourite R hosting site, and thanks to the tireless Stefan Theussl (and some favourable timezone differences) the project was approved and the stanza available by the time I got up. So I quickly filled the SVN repo and, presto, we had the RProtoBuf project at R-Forge within 12 hours of the ORD Sessions hackfest. I will try to follow up on RProtoBuf in a couple of days, this may lead to some changes in my Rcpp R / C++ interface package as well.

/computers/misc | permanent link

RPostgreSQL 0.1-6
A quick bug fix released of RPostgreSQL was uploaded to CRAN earlier today; it updates last week's 0.1-5 release. An editing error left one of the helper functions broken. I also updated a few of the test scripts to also allow for optional environment variable for the port at which the PostgreSQL server is listening. This is handy when you have parallel installations of different PostgreSQL versions which Debian makes really easy.

As always, all the details (including the full ChangeLog) are at the local RPostgreSQL page, the RPostgreSQL package page at CRAN and of course the Google Code repository for RPostgreSQL.

/computers/linux/debian/packages | permanent link

Dianne Reeves: Strings Attached at the CSO
Another trio concert at CSO, but very different from the most recent classic piano trio. Tonight was once again a chance to see Dianne Reeves (wikipedia) but this time accompanied simply by two guitarists: Russell Malone and Romero Lubambo (wikipedia).

Given that Dianne Reeves (who we had seen in just a few month earlier in our neighbourgood) has plenty of stage presence, the format made for a more intimate concert yet with plenty of groove at times too. The three had been touring in Europe for 25 shows, and it was a really nice performance and a great way to end the week. Recommend if you can catch them somewere.

/music/jazz/live | permanent link

RPostgreSQL 0.1-5
A new version of RPostgreSQL just went out to CRAN and should propagate via the mirrors over the next few days.

This is a maintenance and bug fix release that addresses four of the seven issue at the Google Code page. Two more are really enhancement requests for which we encourage patches as we are unlikely to have to write them, and the last issue I have been unable to replicate.

This release has benefitted greatly from the generous help of Neil Tiffin who now looks after all things OS X and keeps a good eye on the Google Code site (where I had, ahem. overlooked the existence of these open issue tickets for a few months), Joe Conway who we can count on for hard-core PostgreSQL issues as well as Jeff Horner for general R, DBI and database smarts.

Some more details (including the full ChangeLog) are at the local RPostgreSQL page, the RPostgreSQL package page at CRAN and of course the Google Code repository for RPostgreSQL.

/computers/linux/debian/packages | permanent link

Zut alors!
Christian: of course you will be running the Chicago 2010 marathon as your next fall marathon. Not only are Paris and Chicago jumelée (i.e. sister cities), but the Dollar is so conveniently falling vis-à-vis the Euro that you'd get all this done for about a Euro and a half by the time the race happens. Moreover, 2010 is going to be Debian's US year anyway. I crammed my two European races into two four day trips which I won't exactly recommend, but you do not need ten days either. Lastly, I cannot promise you many interesting cheeses but there are at least a few pretty tasty beers to sample. À bientôt à Chicago!

/sports/running | permanent link

Chicago Marathon 2009
Earlier today the 33rd Chicago Marathon was held. I will organise my comments by pointing out three epic fails.

First, of course, was the Chicago weather. After two successive marathon in excessive heat --- the 2007 race I completed, poorly, with its thousands of runners forced to abandon when the race was cancelled due to excessive heat, and the 2008 version which I skipped as I ran Berlin that year just weeks before Chicago, this year had forecasts of temperatures in the thirties and possibly snow the night before. Well, the weather cleared up -- but with those clear skies we still got a severe weather alert for the area due to frost! So at the (now earlier by 30 minutes) race start, it was very nippy and in the higher 30s, improving steadily under sunny skies. Overall, a little chilly and hence not exactly ideal, but not too bad in the grand scheme of things. A little windy coming up Michigan Avenue. But hey, Wanjiru still finished with a course record though still well off the world record. So maybe not a fail after all.

The second fail, though, was my GPS which I had just blogged about yesterday (albeit indirectly). I had had my issues with the previous (much simpler and older) Forerunner 205 which lost satellite tracking when running downtown amidst the skyscrapers. I had high hopes that this newer model would do better. But no, not only did it loose track within the first few miles, it even managed to outdo the older model by turning its GPS tracking off in the later stages. Now that's a fail!

The third and final fail, unfortunately, concerns my run. Training had gone well enough to let me hope for another decent race. However, a latent cold during the last two weeks had left me somewhat afraid I might not do well. Things felt better yesterday, and I ended up running a decent first eighteen miles to the waterstop the rest of my household was working. And thereafter: well let's just say that the wheels came off. I ended up with a 3:25:40 (or around 7:51 min/mile and 4:53 min/km) which is not terrible but also not one of my better races.

I updated the two race data geekery charts shown here before to illustrate the fail. As I am running out of shades of blue for Chicago, I show the 2009 race in purple. A decent, flat chat indicating a reasonably steady pace throughout most of the race ... followed by one of the very worst finished. And as discussed above, I cannot really blame the weather either.

I also updated my 'performance by race type through time' chart which suggests that I may be getting slower for marathons (if we allow for a non-linear effect). Eek.

/sports/running | permanent link

R for system administration and scripting
On several occassions, R had suggested itself as a language for systems scripting. By this I mean random little adminstrative task such as (re-)moving or maybe renaming files or directories and the like.

One of such cases just happened a few minutes ago. The aforementioned Garmin Forerunner 405 can cooperate quite nicely with Linux using the gant reader for the ant wireless communication protocol between the usb hardware dongle and the Garmin 405. (Sources for gant are both this file and this git archive.) I had meant to blog about this tool and the resulting files one of these days anyway, but today I just want to mention that the default filenames created by the program were somewhat horrid such as 20.09.2009 101112.TCX to denote the 20th of September of this year at 10:11h and 12 seconds. As we all know, filenames with spaces are bad for the environment as well as plain annoying. So I had made the simple change in the C sources to switch to a saner format such as 20090920-101112.TCX (and I see that the git archive now contains a similar fix). But that still left me with some 80+ files with the dreaded names.

There are of course many ways to skin this cat and to rename the files in bulk. However, I found the following four lines to be fairly succinct

#!/usr/bin/r
files <- dir(".", pattern=".*\\.TCX$")
res <- lapply(files, function(f) {
    pt <- strptime(f, "%d.%m.%Y %H%M%S.TCX")  # parsed time
    ft <- strftime(pt, "%Y%m%d-%H%M%S.TCX")   # formatted time
    file.rename(f, ft)
})

• the access to one of the three (soon four) regexp engines, here as a simple patterns argument to dir()
• the functional programming nature of the beast: files is a vector of filenames, and lapply() unrolls the vector one-by-one calling the anonymous function and passing the current element off as f
• computing on times is particularly easy as we get strptime and strftime as any self- and POSIX-respecting language should
• similarly, we get access to file system-level operations natively avoiding all quoting issues that make files with spaces such fun in the first place.
• the littler scripting frontend providing /usr/bin/r rules.

Lastly, I do not mean to imply that Python or Perl or Ruby or (insert favourite tool here) cannot do it equally well. I simply meant to say that programmatically creating new filenames is definitely easier in R than it would have been in shell. And as an added bonus, we even get fully parsed time objects that I could have tested for. But then tests and documentation never get written on a Saturday.

/computers/R | permanent link

RInside release 0.1.1, and a fresh example
Last week's 0.1.0 release of RInside, and the first to have been published on CRAN, still had some issues with builds and use on OS X. Thanks to testing and fixes by Jan de Leeuw, Jeff Horner and particularly Simon Urbanked, things are said to be better now with the new release 0.1.1 which went onto CRAN yesterday. So no new features, but fixes to the main Makefile as well as the Makefile for the examples directory, some minor fixes and editing for the examples. I also added a file THANKS to show some appreciation for the various patches and fixes I have been receiving -- they are appreciated!

However, today I committed a new example to SVN archive at R-Forge. It is based on this thread on r-devel. Abhijit Bera tries to do this in C, but to me his questions provide rather clear motivation for showing how much simpler things can be via C++ and the Rcpp classes along with RInside. Using a small example, the task was to pass a weight vector to a portfolio solver from the Rmetrics package fPortfolio and to then access the computed solution. The original poster struggled with access from C to the S4 classes used by fPortfolio and could not set the weights. But when using RInside, we simply pass a C++ vector of weights down to R, solve the problem and pass a solution vector back using the handy evaluation of R expressions:

// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
//
// Another simple example inspired by an r-devel mail by Abhijit Bera
//
// Copyright (C) 2009 Dirk Eddelbuettel and GPL'ed

#include "RInside.h"                    // for the embedded R via RInside
#include "Rcpp.h"                       // for the R / Cpp interface used for transfer
#include <iomanip>

int main(int argc, char *argv[]) {

    try {
        RInside R(argc, argv);          // create an embedded R instance
        SEXP ans;

        std::string txt = "suppressMessages(library(fPortfolio))";
        if (R.parseEvalQ(txt))          // load library, no return value
            throw std::runtime_error("R cannot evaluate '" + txt + "'");

        txt = "lppData <- 100 * LPP2005.RET[, 1:6]; "
	  "ewSpec <- portfolioSpec(); "
	  "nAssets <- ncol(lppData); ";
        if (R.parseEval(txt, ans))      // prepare problem
            throw std::runtime_error("R cannot evaluate '" + txt + "'");

	const double dvec[6] = { 0.1, 0.1, 0.1, 0.1, 0.3, 0.3 }; // choose any weights you want
	const std::vector<double> w(dvec, &dvec[6]);

	R.assign( w, "weightsvec");	// assign STL vector to R's 'weightsvec' variable

	txt = "setWeights(ewSpec) <- weightsvec";
        if (R.parseEvalQ(txt))		// evaluate assignment
            throw std::runtime_error("R cannot evaluate '" + txt + "'");

	txt = "ewPortfolio <- feasiblePortfolio(data = lppData, spec = ewSpec, constraints = \"LongOnly\"); "
	  "print(ewPortfolio); "
	  "vec <- getCovRiskBudgets(ewPortfolio@portfolio)";
        if (R.parseEval(txt, ans))      // assign covRiskBudget weights to ans
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	RcppVector<double> V(ans);      // convert SEXP variable to an RcppMatrix

	R.parseEval("names(vec)", ans);	// assign columns names to ans
	RcppStringVector names(ans);

	for (int i=0; i<names.size(); i++) {
	  std::cout << std::setw(16) << names(i) << "\t"
		    << std::setw(11) << V(i) << "\n";
        }

    } catch(std::exception& ex) {
        std::cerr << "Exception caught: " << ex.what() << std::endl;
    } catch(...) {
        std::cerr << "Unknown exception caught" << std::endl;
    }

    exit(0);
}

/computers/linux/debian/packages | permanent link

digest 0.4.0 and 0.4.1
After almost two years, I just made a small update to the digest package for R by adding sha-256 as another supported hashing function. While at this, a few minor edits where made here or there. Version 0.4.0 also included a file src/Makefile.win but as that actually prevents a successful build on the Windows platform I promptly removed it for a (more-or-less brown-bag) release 0.4.1. Thanks to Uwe Ligges and his automated win-builder service that permits easy unattended build tests on that platform.

The new package will appear shortly on its CRAN page; the R-Forge page has access to the Subversion repo etc and otherwise there's always my local my local digest page.

Oh, and 0.3.1 was never blogged about as it was just a maintenance release that added a single cast and an edit to the DESCRIPTION file.

/computers/linux/debian/packages | permanent link

RInside release 0.1.0 -- and now on CRAN
Version 0.1.0 of RInside, my C++ wrapper classes which facilitate embedding R into your own C++ application, has been released and is now an official CRAN package.

This release improves on the build process and should work on any sane Unix-alike operating system. A few more examples were added or extended. Details and the full ChangeLog are on my RInside page, and there is now a RInside page on CRAN as well.

/computers/linux/debian/packages | permanent link

Chicago Half Marathon 2009
Today it was once again time for the Chicago Half Marathon (which I have now been running in 2003, 2004, 2005, 2006, 2007 and 2008). Conditions were much much better than last year's very heavy rainfall---we were once again treated to a sunny and clear Chicago sky. It was however a little on the humid side and got somewhat warm; a few friends I talked to suffered on the second half of the race and I also slowed down.

As for the race itself, I didn't run it quite as evenly as I had hoped, faded a little around mile ten and eleven and came in at 1:34:54, or just under 7:15 min/mile. Not a bad time---I have done better (e.g. last year) but also worse. All in all still one my very favourite races, and this year with a once-again improved course that featured even more running along the lake. I will probably be back next year.

/sports/running | permanent link

RQuantLib 0.3.0 Windows build snag
Yesterday's upload of RQuantLib 0.3.0 contained one minor oversight: I had failed to update src/Makefile.win to the new and enlarged set of source files. The proper fix (of using wildcards and implicit rules) was simple, and Uwe Ligges kindly rebuilt RQuantLib on the Windows builder for CRAN using an updated Makefile so that binaries for Windows will be available. This fix will also be in the next upload. Should you need it in the meantime, here is a link to the new version directly from SVN.

/computers/linux/debian/packages | permanent link

RQuantLib 0.3.0 released
Earlier this evening, I rolled up a new version of RQuantLib. It has been pushed to CRAN and Debian, and source and binary version should appear on the respective mirror networks in due course.

This version, the first in a new '0.3.*' release series, contains all the work that Khanh Nguyen did during his Google Summer of Code participation which I had mentioned a while back and for which I acted as mentor.

And Khanh did great. We now have a lot more Fixed Income functionality; I include the full ChangeLog entry below. I also added some simple calendaring support and the odd fix here or there while mentoring. All of this had been available while technically 'in progress' thanks to support provided by the R-Forge project hosting where SVN checkouts as well as nightly binaries are provided. A few more details are on the RQuantLib page and I hope to add a few more examples.

As promised, the ChangeLog entry:

2009-09-05  Dirk Eddelbuettel  

	* DESCRIPTION: Release 0.3.0 reflecting all the excellent
	  Google Summer of Code 2009 work by Khanh Nguyen as well
	  as other small enhancements

	[ Changes by Khanh Nguyen below ]

	* R/bond.R: Added pricing functionality for various new instrument
	* R/discount.R: Idem
	* src/bonds.cpp: Idem
	* src/discount.cpp: Idem
	* src/utils.cpp: Idem

	* man/Bond.Rd: Added documentaiont for new functions
	* man/CallableBond.Rd: Idem
	* man/ConvertibleFixedCouponBond.Rd: Idem
	* man/ConvertibleFloatingCouponBond.Rd: Idem
	* man/ConvertibleZeroCouponBond.Rd: Idem
	* man/Enum.Rd: Idem
	* man/FittedBondCurve.Rd: Idem
	* man/FixedRateBond.Rd: Idem
	* man/FixedRateBondCurve.Rd: Idem
	* man/FixedRateBondPriceByYield.Rd: Idem
	* man/FixedRateBondYield.Rd: Idem
	* man/FloatingRateBond.Rd: Idem
	* man/ZeroCouponBond.Rd: Idem
	* man/ZeroPriceByYield.Rd: Idem
	* man/ZeroYield.Rd: Idem

	* rests/RQuantLib.R: Added tests for new functions
	* rests/RQuantLib.Rout.save: Added tests ouput for new functions

	[ Changes by Dirk Eddelbuettel below ]

	* man/BondUtilities.Rd: Added documentation for new function

	* R/calendars.R: Add support to access QuantLib calendars from R
	* src/calendars.cpp Idem
	* man/Calendars.Rd: Idem

	* src/bonds.cpp: Small C++ fixes to suppres g++ warnings
	* INDEX: Updated via 'R CMD build --force'
	* inst/QuantLib-License.txt: Updated to version from QL 0.9.7

/computers/linux/debian/packages | permanent link

random 0.2.1
A maintenance fix was applied to random: following the changes in release 0.2.0 a few months ago that reflected changes at the random.org site, we needed to make one more change in randomSequence. Thanks for Lynn Amon for spotting this.

The new package is available here and should appear on CRAN shortly.

/computers/linux/debian/packages | permanent link

State of the Art in Parallel Computing with R: Now published
Our survey paper on the current state of the art in parallel computing with R, previously mentioned here as a technical report is now out as Vol 31, Issue 1 of the all-electronic Journal of Statistical Software.

/computers/R | permanent link

Rcpp 0.6.6
Version 0.6.6 of Rcpp [ a set of C++ classes making it easier to glue C/C++ code to R ] went off to CRAN and Debian earlier as another small maintenance release.

While the diffstat output that will appear in cranberries may look impressive, it stems chiefly from updating the Doxygen documentation of the C++ classes. Only one small feature was added: one can now test via exists() whether a named parameter exists in an instance of the RcppParams class.

/computers/linux/debian/packages | permanent link

New RInside release
I just rolled up a new release of RInside, my C++ wrapper classes which facilitate embedding R into your own C++ application.

This releases owes a big Thank you! to Miguel Lechón who not only noticed errant behaviour and occassional segfaults with overly long commands sent to the embedded R, but even traced it to an oversight of mine in a simple memory buffer class and provided the one-line fix to reset a pointer! Much appreciated, especially as I got his mail two days before I talked about Rcpp and RInside at UseR! 2009 . The other changes are mostly cleanups, additions of two more test examples (to replicate the bug report) and some minor additions of a few new assign() functions to pass data between C++ and the embedded R session.

/computers/linux/debian/packages | permanent link

cran2deb: Would you like 1700+ new Debian / R packages ?
As I mentioned in my quick write-up of UseR 2009, one of my talks was about cran2deb: a system to turn (essentially) all CRAN packages into directly apt-get-able binary packages.

This is essentially a '2.0' version of earlier work with Steffen Moeller and David Vernazobres which we had presented in 2007. Then, the approach was top-down and monolithic which started to show its limits. This time, the idea was to borrow the successful bottom-up approach of my CRANberries feed.

The bulk of the work was done by Charles Blundell as part of his Google Summer of Code 2008 project which I had suggested and mentored. After that project had concluded, we both felt we should continue with it and bring it to 'production'. The CRAN hosts provided us with a (virtual Xen) machine to build on, and we are now ready to more publically announce the availability of the repositories for i386 and amd64:

  deb http://debian.cran.r-project.org/cran2deb/debian-i386 testing/

  deb http://debian.cran.r-project.org/cran2deb/debian-amd64 testing/

A few more details are provided in our presentation slides. We look forward to hearing from folks using; the r-sig-debian list may be a good venue for this.

/computers/R | permanent link

useR 2009 in Rennes: Recap and slides
I spent most of last week in Rennes, the capital of Brittany in France, as it was time for UseR! 2009, the annual R conference. Francois Husson, Aline Legrand and others at the Agrocampus Ouest had put together a really well-run conference, and it was a pleasure to reconnect with so many people. It was also pretty nice to walk around town and see bits and pieces of the Tombees de la nuit festival which happened at the same time.

As last year (and again at the BoC in December), I presented a three-hour tutorial on high-performance computing with R. This covers profiling, vectorisation, interfacing compiled code, debugging, parallel computing, as well as scripting and automation. Slides, and a 2-up version, are now on my presentations page.

I also gave two regular conference presentations. The first was on my Rcpp and RInside packages which facilitate interfacing R and C++. The second talk, based on joint work with Charles Blundell, describes our cran2deb system for creating Debian packages of essentially all CRAN packages. I will try to follow up on this with another post. Slides from these talks are also on my presentations page.

/computers/R | permanent link

R 2.9.1, CRANberries outage, and missing Java support
Just a short note that version 2.9.1 of R was released yesterday. And a corresponding Debian release went out as usual on the same day. One sour note: as the Java toolchain is currently broken, I had to disable compile-time support for Java. Just run R CMD javareconf once installed if you need it.

Speaking of broken, I had neither noticed that this R version now returns an additional field (for the repository) in the per-package metadata via available.packages(), nor that this change had broken my oh-so-useful and increasingly popular CRANberrries html and rss summaries of CRAN changes. So with the usual beta and rc releases or R 2.9.1 in Debian starting a week prior, CRANberries had been silent for six days from Friday the 21st to last Thursday. I rectified it once I noticed, and changed the code to no longer fall on its nose at that spot. Sorry for the few days without service.

/computers/R | permanent link

Ubuntu Developer Summit in Barcelona
Due to some things falling into place, I had an opportunity to attend the first two days of last week's Ubuntu Developer Summit in beautiful Barcelona. Somehow, I had never managed to attend a Debian conference either, so it was good to meet a few of the old Debian hands now moving Ubuntu along, as well as a few of the Ubuntu folks. I also gave a short presentation on R in Debian / Ubuntu and the plans for the upcoming Ubuntu release. More on that another time.

All told, a well-organised conference in a nice setting -- two stone throws from the legendary Camp Nou. Unfortunately, I had to leave by Wednesday so I missed what was undoubtedly quite a scene in Barcelona following Barca's dismantling of Man U in this year's Champions League final.

/computers/misc | permanent link

JPM Chase Corporate Challenge 2009
The 28th annual JP Morgan Chase Corporate Challenge race took place a couple of days ago May 21. Participation was down from the record of 23,000 runners set last year at around 17,125. With splendid weather, it is always a nice way to start the Memorial day weekend.

We fielded a small but spirited team of nine runners. I finished with a decent (hand-stopped) time of 22 minutes and 27.93 seconds for the 3.5 miles -- or a 6:25 min/mile pace. That is among the fasters times but not quite the fastest compared to the other six times I have run this.

Most importantly, everybody seems to have had a blast. And we did set a record for longer post-race party which sets a nice precedent for 2010.

/sports/running | permanent link

Temporary Debian mail outage
It would appear that debian.org rejected mail for maybe up to twelve hours from late yesterday afternoon (Central timezone) to some time shortly after I got up this morning. Things appear to be back to normal, so a big Thanks to the mail admins.

If you happened to have sent me mail to my debian.org address during that time period, you may have gotten a hard reject ('550 Administrative prohibition') as did a test mail of mine. In this case mail may not be respooled, so please do send it again.

My alternate address, formed by my first name followed by the family name and the commercial top-level domain, remained functional as a fallback.

/computers/misc | permanent link

Brad Mehldau at the CSO
Just got home from a wonderful concert by Brad Mehldau at the CSO. This was long overdue as I kept reading about Mehldau. And even though he performs quite regularly around here, I had never seen him. Big mistake.

The first set was performed as a (strictly acoustic) trio with Larry Grenadier on bass and Jeff Ballard on drums. After several compositions by Mehldau and a brazilian samba piece, the first set closed with a rendition of 'Holland' from Sufjan Stevens' album Michigan which was truly beautiful. The second set had Mehldau performing solo, again with several compositions of his own as well as one from Neil Young's classic 'The Needle and the Damage Done' leading two two pieces from the Sound of Music including an amazing, yet really different 'My favourite things' that just hushed a piece of the central melody along with a strond rhythmic element. Lovely. And then to cap it all off, four encores.

Highly recommended.

/music/jazz/live | permanent link

GSoC 2009 Chicago area meeting
Thanks to the effort of the tireless Borja Sotomayor for the local ACM chpater as well as the good folks at Google in Chicago, we had a local kickoff meeting for this year's Google Summer of Code at Google's Chicago office. A few accepted GSoC students, mentors, and a few Google engineers gave short presentations to a bunch of ACM-affiliated students from U of C, Northwestern, DePaul, IIT, UIC, ... I gave my first ever 'lightning talk' --- on R and the GSoC --- and after chatting a bit more rushed home to catch the end of the amazing triple-overtime win of the Bulls over the Celtics. Go Bulls for the improbable game seven in Boston!

/computers/misc | permanent link

Slides from most recent R and HPC tutorial
A little earlier I put the slides from my Introduction to High-Performance Computing with R tutorial at the R / Finance conference last week onto my talks / presentation page. Other tutorials, talks and keynotes are also being posted on the conference program page.

This tutorial was a shorter format of just an hour which did not allow for any parallel computing with R. However, parallel computing with R via MPI, snow, nws, ... is covered in the slides from December's workshop at the BoC.

/computers/R | permanent link

Google Summer of Code 2009: R / Quantlib
With everything that has been going on of late, I have yet to mention that Khanh Nguyen, a Ph.D. student in Computer Science at U Mass / Boston, will be working with me on RQuantLib as part of the Google Summer of Code program this year.

We had twenty-two applications to review for the R project, including three for the RQuantLib topic I had proposed. Khanh's application was clearly among the best, and I look forward to helping him do cool stuff over the summer. He already posted two short emails on the r-sig-finance and the quantlib-user lists soliciting suggestions and comments. So if you have comments regarding R and QuantLib, please get in touch with him or me!

/computers/misc | permanent link

Review of 'Analysis of Integrated and Cointegrated Time Series with R (2nd ed)' in JSS
A few weeks ago I wrote up a short review of Bernhard Pfaff's nice (but somewhat dry) Analysis of Integrated and Cointegrated Time Series with R (2nd ed) on unit root and cointegration modeling with R. This is now online at the Journal of Statistical Software.

/computers/R | permanent link