Over the last few months, I have been working casually on a new package to integrate MessagePack with R. What is MessagePack, you ask? To quote its website, "It's like JSON, but fast and small."
Or in more extended terms:
MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.
Now, serialization formats are a dime a dozen: JSON, BSON, Protocol Buffers, CaptnProto, Flatbuffer. The list goes on and on. So why another? In a nutshell: "software ecosystems".
I happen to like working with Redis, and within the world of Redis, MessagePack is a first-class citizen supported by things close to the core like the embedded Lua interpreter, as well as fancy external add-ons such as the Redis Desktop Manager GUI. So nothing overly fundamentalist here, but a fairly pragmatic choice based on what happens to fit my needs. Plus, having worked on and off with Protocol Buffers for close to a decade, the chance of working with something not requiring a friggin' schema compiler seemed appealing for a chance.
So far, we have been encoding a bunch of data streams at work via MessagePack into Redis (and of course back). It works really well---header-only C++11 libraries for the win. I'll provide an updated RcppRedis which uses this (if present) in due course.
For now and the foreseeable future, this RcppMsgPack package will live only on the ghrr drat repository. To make RcppMsgPack work, I currently have to include the MessagePack 1.4.0 headers. A matching package for this version of the headers is in Debian but so far only in experimental. Once this hits the mainline repository I can depend on it, and upload a (lighter, smaller) RcppMsgPack to CRAN.
Until then, please just do
## install drat if not present
if (!require(drat)) install.packages("drat")
## use drat to select ghrr repo
drat::addRepo("ghrr")
## install RcppMsgPack
install.packages("RcppMsgPack")More details, issue tickets etc are at the GitHub repository.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.