R Project and Google Summer of Code: Wrapping up
As this year's admin, I wrote up the following summary which has now been posted at the R site in the appropriate slot. My thanks to this year's students, fellow mentors and everybody else who helped to make it happen.

Projects 2010

As in 2008 and 2009, the R Project has again participated in the Google Summer of Code during 2010.

Based on ideas collected and disussed on the R Wiki, the projects and students listed below (and sorted alphabetically by student) were selected for participation and have been sponsored by Google during the summer 2010.

The finished projects are available via the R / GSoC 2010 repository at Google Code, and in several cases also via their individual repos (see below). Informal updates and final summaries on the work was also provided via the GSoC 2010 R group blog.

rdx - Automatic Differentiation in R

Proposal: radx is a package to compute derivatives (of any order) of native R code for multivariate functions with vector outputs, f:R^m -> R^n, through Automatic Differentiation (AD). Numerical evaluation of derivatives has widespread uses in many fields. rdx will implement two modes for the computation of derivatives, the Forward and Reverse modes of AD, combining which we can efficiently compute Jacobians and Hessians. Higher order derivatives will be evaluated through Univariate Taylor Propagation.

Delivered: Two packages radx: forward automatic differentiation in R and tada: templated automatic differentiation in C++ were created; see this blog post for details.

A GUI for Graphics using ggplot and Deducer

Proposal: R puts the latest statistical techniques at one's fingertips through thousands of add-on packages available on the CRAN download servers. The price for all of this power is complexity. Deducer is a cross-platform cross-console graphical user interface built on top of R designed to reduce this complexity. This project proposes to extend the scope of Deducer by creating an innovative yet intuitive system for generating statistical graphics based on the ggplot2 package.

Delivered: All of the major features have been implemented, and are outlined in the video links in this blog post.

rgeos - an R wrapper for GEOS

Proposal: At present there does not exist a robust geometry engine available to R, the tools that are available tend to be limited in scope and do not easily integrate with existing spatial analysis tools. GEOS is a powerful open source geometry engine written in C++ that implements spatial functions and operators from the OpenGIS Simple Features for SQL specification. rgeos will make these tools available within R and will integrate with existing spatial data tools through the sp package.

Delivered: The rgeos project on R-Forge; see the final update blog post.

Social Relations Analyses in R

Proposal: Social Relations Analyses (SRAs; Kenny, 1994) are a hot topic both in personality and in social psychology. While more and more research groups adopt the methodology, software solutions are lacking far behind - the main software for calculating SRAs are two DOS programs from 1995, which have a lot of restrictions. My GSOC project will extend the functionality of these existing programs and bring the power of SRAs into the R Environment for Statistical Computing as a state-of-the-art package.

Delivered: The TripleR package is now on CRAN and hosted on RForge.Net; see this blog post for updates.

NoSQL Interface for R

Proposal: So-called NoSQL databases are becoming increasingly popular. They generally provide very efficient lookup of key/value pairs. I'll provide several implementation of NoSQL interface for R. Beyond a sample interface package, I'll try to support generic interface similar to what the DBI package does for SQL backends

Status: An initial prototype is available via RTokyoCabinet on Github. No updates were made since June; no communication occurred with anybody related to the GSoC project since June and the project earned a fail.

Last modified: Wed Sep 22 19:39:43 CDT 2010

/computers/misc | permanent link

R Project and Google Summer of Code: Welcome to our students!
A few hours ago, I sent the following to both the R development list and the informal R / GSoC list:

Date: Mon, 26 Apr 2010 15:27:29 -0500
To: R Development List 
CC: gsoc-r 
Subject: R and the Google Summer of Code 2010 -- Please welcome our new students!
From: Dirk Eddelbuettel 

Earlier today Google finalised student / mentor pairings and allocations for
the Google Summer of Code 2010 (GSoC 2010).  The R Project is happy to
announce that the following students have been accepted:

   Colin Rundel, "rgeos - an R wrapper for GEOS", mentored by Roger Bivand of
      the Norges Handelshoyskole, Norway

   Ian Fellows, "A GUI for Graphics using ggplot2 and Deducer", mentored by
      Hadley Wickham of Rice University, USA

   Chidambaram Annamalai, "rdx - Automatic Differentiation in R", mentored by
      John Nash of University of Ottawa, Canada

   Yasuhisa Yoshida, "NoSQL interface for R", mentored by Dirk Eddelbuettel,
      Chicago, USA

   Felix Schoenbrodt, "Social Relations Analyses in R", mentored by Stefan
      Schmukle, Universitaet Muenster, Germany

   Details about all proposals are on the R Wiki page for the GSoC 2010 at
   http://rwiki.sciviews.org/doku.php?id=developers:projects:gsoc2010

The R Project is honoured to have received its highest number of student
allocations yet, and looks forward to an exciting Summer of Code.  Please
join me in welcoming our new students.

At this time, I would also like to thank all the other students who have
applied for working with R in this Summer of Code. With a limited number of
available slots, not all proposals can be accepted -- but I hope that those
not lucky enough to have been granted a slot will continue to work with R and
towards making contributions within the R world. 

I would also like to express my thanks to all other mentors who provided for
a record number of proposals.  Without mentors and their project ideas we
would not have a Summer of Code -- so hopefully we will see you again next
year. 

Regards,  

Dirk (acting as R/GSoC 2010 admin)

/computers/misc | permanent link

R Project selected for the Google Summer of Code 2010
Earlier today, Google announced the list of accepted mentor organizations for the Google Summer of Code 2010 (GSoC 2010). And we are happy to report that the R Project is once again a participating organization (and now for the third straight year) joining a rather august group of open source projects from around the globe.

An R Wiki page had been created and serves as the central point of reference for the R Project and the GSoC 2010. It contains a list of project ideas, currently counting eleven and spanning everything from research-oriented topics (such as spatial statistics or automatic differentiation) to R community-support (regarding CRAN statistics and the CRANtastic site) to extensions (NoSQL, RPy2 data interfaces, Rserve browser integration) and more. I also just created a mailing list gsoc-r@googlegroups.com where prospective students and mentors can exchange ideas and discuss. As for other details, the Google Summer of Code 2010 site has most of the answers, and we will try to keep R-related information on the aforementioned R Wiki page.

/computers/misc | permanent link

From ORD Sessions to R-Forge in 12 hours with RProtoBuf
Yesterday, via in invitation from fellow Chicago-area Google Summer of Code mentor Borja Sotomayor, I attended the Second ORD Sessions. These are happening at the HQ of Inventable where a couple of technologists and Open Source geeks from the Chicagoland area get together and riff on code for a few hours after work over some pizza and beer.

Sounded good, and I needed an excuse to try to mix the awesome Protocol Buffers with my favourite data tool, R. What are Protocol Buffers? To quote from the Google overview page referenced above:

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.

Protocol buffers are now Google's lingua franca for data – at time of writing, there are 48,162 different message types defined in the Google code tree across 12,183 .proto files. They're used both in RPC systems and for persistent storage of data in a variety of storage systems.

So three hours later, I had an implementation of the 'addressbook reader' C++ example wrapped in a tiny yet complete R package that passed R CMD check. And one lingua franca for data has met another.

So before going to bed, I quickly registered a new project at R-Forge, everybody's favourite R hosting site, and thanks to the tireless Stefan Theussl (and some favourable timezone differences) the project was approved and the stanza available by the time I got up. So I quickly filled the SVN repo and, presto, we had the RProtoBuf project at R-Forge within 12 hours of the ORD Sessions hackfest. I will try to follow up on RProtoBuf in a couple of days, this may lead to some changes in my Rcpp R / C++ interface package as well.

/computers/misc | permanent link

Ubuntu Developer Summit in Barcelona
Due to some things falling into place, I had an opportunity to attend the first two days of last week's Ubuntu Developer Summit in beautiful Barcelona. Somehow, I had never managed to attend a Debian conference either, so it was good to meet a few of the old Debian hands now moving Ubuntu along, as well as a few of the Ubuntu folks. I also gave a short presentation on R in Debian / Ubuntu and the plans for the upcoming Ubuntu release. More on that another time.

All told, a well-organised conference in a nice setting -- two stone throws from the legendary Camp Nou. Unfortunately, I had to leave by Wednesday so I missed what was undoubtedly quite a scene in Barcelona following Barca's dismantling of Man U in this year's Champions League final.

/computers/misc | permanent link

Temporary Debian mail outage
It would appear that debian.org rejected mail for maybe up to twelve hours from late yesterday afternoon (Central timezone) to some time shortly after I got up this morning. Things appear to be back to normal, so a big Thanks to the mail admins.

If you happened to have sent me mail to my debian.org address during that time period, you may have gotten a hard reject ('550 Administrative prohibition') as did a test mail of mine. In this case mail may not be respooled, so please do send it again.

My alternate address, formed by my first name followed by the family name and the commercial top-level domain, remained functional as a fallback.

/computers/misc | permanent link

GSoC 2009 Chicago area meeting
Thanks to the effort of the tireless Borja Sotomayor for the local ACM chpater as well as the good folks at Google in Chicago, we had a local kickoff meeting for this year's Google Summer of Code at Google's Chicago office. A few accepted GSoC students, mentors, and a few Google engineers gave short presentations to a bunch of ACM-affiliated students from U of C, Northwestern, DePaul, IIT, UIC, ... I gave my first ever 'lightning talk' --- on R and the GSoC --- and after chatting a bit more rushed home to catch the end of the amazing triple-overtime win of the Bulls over the Celtics. Go Bulls for the improbable game seven in Boston!

/computers/misc | permanent link

Google Summer of Code 2009: R / Quantlib
With everything that has been going on of late, I have yet to mention that Khanh Nguyen, a Ph.D. student in Computer Science at U Mass / Boston, will be working with me on RQuantLib as part of the Google Summer of Code program this year.

We had twenty-two applications to review for the R project, including three for the RQuantLib topic I had proposed. Khanh's application was clearly among the best, and I look forward to helping him do cool stuff over the summer. He already posted two short emails on the r-sig-finance and the quantlib-user lists soliciting suggestions and comments. So if you have comments regarding R and QuantLib, please get in touch with him or me!

/computers/misc | permanent link

Google Summer of Code 2009
Word is out that there will be a 2009 edition of the Google Summer of Code. I have some follow-up ideas based on last year's mentoring for both Debian and R, but there will be a better time and place to discuss possible project ideas.

/computers/misc | permanent link

Google Summer of Code 2008 Mentors Summit
Spent last weekend in Mountain View where Google had invited a number of mentors for the Summer of Code project that Google once again graciously sponsored. A rather impressive list of projects sent up to two people each, giving a probably unparalled sample of major Open Source projects.

I had a blast. Chris, Leslie and the rest of the Google's Open Source Programs Office facilitated a really nice unconference that spawned a few really nice sessions, and they took very good care of us. And just about everybody met a number of folks in person that were previously known only via email or irc. As the saying goes: nothing like the bandwidth of a face-to-face meeting...

Last but not least I should issue a health warning. Sharing a room with the fearless Debian DPL is not for the faint of heart: His snooring is truly world-class.

/computers/misc | permanent link

RPostgreSQL 0.1.0
As part of the Google Summer of Code program for 2008 (which I mentioned here and here), Sameer and I are happy to announce that RPostgreSQL is now on the CRAN mirror network for R. RPostgreSQL provides a (DBI-compliant) interface between R and the Postgresql database system. I also just sent this short announcement to the r-packages list.

/computers/misc | permanent link

Google Summer of Code 2008 projects assigned
As mentioned a few weeks ago, I had submitted two entries as possible topics for this year's version of the Google Summer of Code. And lo and behold, the response was very, very good and both applications were slotted!

For the topic 'create a PostgreSQL package for R that uses the standard DBI interface', a number of interested students contacted me, and a total of three applications were submitted. And while the R Foundation was only able to allocate four topics among a number of really good applications, Sameer Prayaga was our pick for this topic. It would be nice to fill this gap among the existing database connection methods for R, and I feel that Sameer can pull this off.

For the second topic of 'create a cran2deb tool for converting CRAN sources into Debian package' which I had submitted within Debian, Charles Blundell wrote an excellent application. In a way, this topic is a '2.0' version of our previous attempts of a 'top-down' set of tools in the pkg-bioc project on Alioth. This time, we will try something smaller, maybe more modular and lighter and see how far we get there if we try it 'bottom-up'.

And as we are currently in the community bonding phase, say Hi to Sameer or Charles when you come across them these days.

Lastly, I'd like to thank everybody who submitted an entry at Debian or R, or who contacted me about one of the topics I posted. The respone was very humbling, many of you were imminently qualified and seemingly very motivated -- but even Google's pockets can only pay for a finite number of projects. Sorry if yours did not get picked.

/computers/misc | permanent link

Google Summer of Code 2008 projects are up
For the past few years, Google has been running their Summer of Code events where Google offers $5000 for college students who are asked to code for a summer for the benefit of various Open Source efforts.

And just like in 2006 and 2007, I put up proposals and offered to act as a mentor. The first one is up at both Debian and R: an opportunity to help with the ongoing efforts of 'turning more CRAN package into Debian packages'. The second one is only at the R page: a proposal to fill the missing link of DBI database interface modules with a matching one for PostgreSQL. More details for either idea are at the respective pages. Anybody interested should ping me by email.

/computers/misc | permanent link

Dead disks, and lvm woes
As posted earlier on the rather recently announced CRANberries feed (that is generated and hosted on my box) covering CRAN package updates, my main server bit the dust. The hard disk no longer wants to talk to any of its lvm partitions, leaving me without /usr, /var/, /var/local/, /home, /srv. Fortunately, most of this was backed-up but now I have to go through reconfiguring the replacement machine I already set up. Not fun.

If anybody has tips on recovering the lvm partitions, I'm all ears. /etc/lvm/ seems fine if that helps as a starting point.

/computers/misc | permanent link

Patch to build palm-db-tools_0.3.6 with recent g++ versions
A freshly downloaded version of palm-db-tools (using the most recent version 0.3.6 from June 2003) will not compile with g++ 4.0.3 and its pickier interpretation of C++.

A simple patch which removes trailing semicolons after class definitions, and adds virtual destructors for a few classes, is below. The mailing list archives show one other patch for a different (yet probably relevant :) problem, and its from November 2003.

By the way, would be nice if someone added the package to Debian. The Makefile's make clean target isn't quite right, and make install isn't exactly correct either. Plus, I seem to be building this by hand every few years, and it's so much easier to just apt-get. And no, my plate is already too full.

diff -ru palm-db-tools-0.3.6.orig/libpalm/Block.h palm-db-tools-0.3.6/libpalm/Block.h
--- palm-db-tools-0.3.6.orig/libpalm/Block.h	2003-06-19 18:37:47.000000000 -0500
+++ palm-db-tools-0.3.6/libpalm/Block.h	2006-07-29 15:04:19.000000000 -0500
@@ -176,7 +176,7 @@
 	size_type m_size;
     };
 
-};
+}
 
 bool operator == (const PalmLib::Block& lhs, const PalmLib::Block& rhs);
 
diff -ru palm-db-tools-0.3.6.orig/libpalm/File.h palm-db-tools-0.3.6/libpalm/File.h
--- palm-db-tools-0.3.6.orig/libpalm/File.h	2003-06-19 18:37:47.000000000 -0500
+++ palm-db-tools-0.3.6/libpalm/File.h	2006-07-29 15:04:42.000000000 -0500
@@ -89,6 +89,6 @@
 	uid_map_t m_uid_map;
     };
 
-};
+}
 
 #endif
diff -ru palm-db-tools-0.3.6.orig/libsupport/infofile.h palm-db-tools-0.3.6/libsupport/infofile.h
--- palm-db-tools-0.3.6.orig/libsupport/infofile.h	2003-06-19 18:37:47.000000000 -0500
+++ palm-db-tools-0.3.6/libsupport/infofile.h	2006-07-29 15:07:08.000000000 -0500
@@ -33,6 +33,7 @@
         {
         public:
             virtual void parse(int linenum, std::vector< std::string> array) = 0;
+	  virtual ~Parser() {};
         };
         class ConfigParser: public Parser
         {
@@ -40,6 +41,7 @@
                 ConfigParser(DataFile::CSVConfig& state)
                     : m_Config(state)
                     {}
+	        virtual ~ConfigParser() {};
                 virtual void parse(int linenum, std::vector< std::string> array);
             private:
                 DataFile::CSVConfig& m_Config;
@@ -50,6 +52,7 @@
                 DatabaseParser(PalmLib::FlatFile::Database& db)
                     : m_DB(db)
                     {}
+	        virtual ~DatabaseParser() {};
                 virtual void parse(int linenum, std::vector< std::string> array);
             private:
                 PalmLib::FlatFile::Database& m_DB;
@@ -60,6 +63,7 @@
                 TypeParser(std::string& p_Type)
                     : m_Type(p_Type)
                     {}
+	        virtual ~TypeParser() {};
                 virtual void parse(int linenum, std::vector< std::string> array);
             private:
                 std::string& m_Type;

/computers/misc | permanent link

Undeleting from ext3
Kudos to Paul Wise who suggested late last night to try magicrescue to recover an OpenOffice file with meeting notes I had mistakenly deleted from the laptop before I even got to editing these very notes, or backing them up.

And it worked -- despite the dire warnings from the ext3 FAQ about the near impossibility of undeleting from ext3 partitions. But a few things worked in my favour here: OpenOffice files are zip files 'under the hood', and magicrescue knows how to deal with certain file formats via its collection of 'recipes' (which one can extend), and zip is among the shipped recipes. Also, I still had the 'ls -l' session in one konsole tab so I knew the exact file size I was looking for. And I wasn't in a hurry. So this morning, after magicrescue had faithfully restored 4479 files in over four hours, its companion magicsort then filtered these into a handful of directories, one of which was 'OpenDocument Text' -- which contained a few files, including two versions of the desired file as well as copies of the version from the previous meeting. Nice. And OpenOffice gladly opened the file. Lastly, it cetainly helped that I happened to have a lurking debian-devel irc session open where --- where Paul not only caught my remark, but almost immediately suggested to try magicrescue. So here goes a well-earned Thank You to Paul!

/computers/misc | permanent link

Mail broken
Comcast, in their infinite wisdom, decided to RBL block (at least one of) Debian's mailhost(s), master.debian.org. As essentially all my mail is forwarded from Debian to Comcast, mail traffic was affected.

At first, I noticed dropped posts from mailing list subscriptions relative to the list web archives. I posted the suggested noticed to the Comcast RBL admins, so far to no avail other than a to yield a more complete stop. The mail forwarding has been disabled for now, and I am dealing with this more manually.

If you tried to contact me in the last few days and have not gotten a response, please resend your mail. I should now be in a position to receive it. Sorry for any inconveniences.

Offers of better mail hosting would be gladly accepted, of course.

/computers/misc | permanent link

Smokes, that ain't pretty
From the headers of a mail I just got:

Received: from master.debian.org ([146.82.138.7])
   by sccrmxc11.comcast.net (sccrmxc11) with ESMTP
   id <20051105044942s1100j9crge>; Sat, 5 Nov 2005 04:49:42 +0000
X-Originating-IP: [146.82.138.7]
Received: from wproxy.gmail.com [64.233.184.204]
   by master.debian.org with esmtp (Exim 3.35 1 (Debian))
   id 1E7EfR-0004zq-00; Mon, 22 Aug 2005 10:56:58 -0500
Received: by wproxy.gmail.com with SMTP id i34so340646wra
   for <edd©debian.org>; Mon, 22 Aug 2005 08:56:47 -0700 (PDT)


So if you, anonymous reader, happened to have emailed me lately and are still awaiting a reply ... please consider the possibility that I may not have received your mail yet.

I should add that the underlying problem has been rectified at Debian's end as one of our hard-working admins stated yesterday.

/computers/misc | permanent link

Quite right
The widely-read and rather influential Walt Mossberg was reviewing Windows security in today's column and concluded

Bottom line: If you use Windows, you're asking for trouble.

As they say, truer words have never been spoken... Other than recommending OS X in no uncertain terms, he listed a host of required add-ons to make that one dominant OS cope with its own lack of security.

/computers/misc | permanent link

Compiling afio under Cywgin
Here is a simple patch against the current afio-2.5 sources, derived mostly from an older patch from Japan against 2.4.7. This allows compilation of afio under Cygwin by simply calling make, no further config needed.

I sent it upstream to Koen, maybe it'll show up in a future versiom.

 diff -ru afio-2.5.orig/afio.c afio-2.5/afio.c ---
afio-2.5.orig/afio.c 2003-12-20 16:16:13.000000000 -0600 +++ afio-2.5/afio.c
2004-09-13 17:12:50.548515800 -0500 @@ -184,7 +184,11 @@
 #include 
 #include 
 #include 
+#ifdef __CYGWIN32__
+#include 
+#else
 #include 
+#endif
 #include 
 #include 
 #include 
Only in afio-2.5: afio.exe
diff -ru afio-2.5.orig/afio.h afio-2.5/afio.h
--- afio-2.5.orig/afio.h	2003-12-20 07:59:42.000000000 -0600
+++ afio-2.5/afio.h	2004-09-13 15:17:51.700744200 -0500
@@ -477,7 +477,7 @@
 #ifndef MKDIR
 int rmdir (char *);
 #endif
-#if !defined (linux) && !defined(__FreeBSD__) && !defined(sun)
+#if !defined (linux) && !defined(__FreeBSD__) && !defined(sun) && !defined(__CYGWIN32__)
 VOIDFN (*signal ())();
 #endif
      int fswrite (int, char*, uint);
diff -ru afio-2.5.orig/compfile.c afio-2.5/compfile.c
--- afio-2.5.orig/compfile.c	2003-06-24 16:32:20.000000000 -0500
+++ afio-2.5/compfile.c	2004-09-13 15:03:44.532576200 -0500
@@ -210,7 +210,7 @@
  * version;
  */
 
-#if ( defined(sun) && defined(__svr4__) )
+#if ( defined(sun) && defined(__svr4__) ) || defined(__CYGWIN32__)
 #include 
 #else
 #include 

/computers/misc | permanent link

That was fast
Ordered a new box to replace our dual-amd, with which we continue to have random locks every couple of days, on Wednesday evening just before one of these Dell special deals expired -- it's their entry level server which has a nice Unofficial Dell PowerEdge 400SC FAQ.

Shipment was supposedly to be in three days from now ... yet the box arrived today! It boots Knoppix as well as my Quantian just fine. A pIV 2.8 GHz and a mobo with sata support, gigabit lan, graphics, sounds, whathaveyou all integrated along with a puny little disk and a laughable amount of Ram (order for a gb from crucial is on its way), it all came to just over $500 of which $100 should come back via a mail-in rebate.

/computers/misc | permanent link

ATX power supplies suck
Grr. Power went out on Thursday, and one of our computers didn't come back on. Took two days to figure that the stupid connector from the power switch to the motherboard was loose so that the power supply was able to operate. Is it just me, or does anybody else think that AT power supplied were simpler? Power on whne pressed, and off otherwise. Clean. Simple.

/computers/misc | permanent link

Compiling a2ps under Cywgin and adding R / S support
For those of use with fingers and brains hardwired to Unix, Cygwin is a godsend. One package I was always missing from Cygwin is a2ps -- which is really nice for pretty-printing just about anything, which would include source code. Thanks to a rather useful support 'style sheet' for the S language (i.e. GNU R and its commercial sibbling S-Plus), you can typeset S code as well. The patch is once again part of the Debian package and can also be found on CRAN.

I simply grabbed the most recent Debian tarball and diff.gz, unpacked the tarball and applied the patch. You then need to run the appropriate debian/patches patch from the a2ps sources directory as per

sh debian/patches/10_s_support.dpatch -patch

After that, it's just a matter of adding a two-line patch in /usr/include/string.h to comment out two lines

#ifndef __CYGWIN__
_PTR	 _EXFUN(memccpy,(_PTR, const _PTR, int, size_t));
_PTR	 _EXFUN(mempcpy,(_PTR, const _PTR, size_t));
#endif

configure --medium=letter; make; make install

/computers/misc | permanent link