Using 3rd party header files with Rcpp

Question

I have a header file called coolStuff.h that contains a function awesomeSauce(arg1) that I would like to use in my cpp source file.

Directory Structure:

• RworkingDirectory
  • sourceCpp
    • theCppFile.cpp
  • cppHeaders
    • coolStuff.h

The Code:

#include <Rcpp.h>
#include <cppHeaders/coolStuff.h>
using namespace Rcpp;

// [[Rcpp::export]]
double someFunctionCpp(double someInput){

 double someOutput = awesomeSauce(someInput);

return someOutput;
}

I get the error:

 theCppFile.cpp:2:31: error: cppHeaders/coolStuff.h: No such file or directory

I have moved the file and directory all over the place and can't seem to get this to work. I see examples all over the place of using 3rd party headers that say just do this:

#include <boost/array.hpp>

(Thats from Hadley/devtools)

https://github.com/hadley/devtools/wiki/Rcpp

So what gives? I have been searching all morning and can't find an answer to what seems to me like a simple thing.

UPDATE 01.11.12

Ok now that I have figured out how to build packages that use Rcpp in Rstudio let me rephrase the question. I have a stand alone header file coolStuff.h that contains a function I want to use in my cpp code.

1) Where should I place coolStuff.h in the package directory structure so the function it contains can be used by theCppFile.cpp?

2) How do I call coolStuff.h in the cpp files? Thanks again for your help. I learned a lot from the last conversation.

Note: I read the vignette "Writing a package that uses Rcpp" and it does not explain how to do this.

The Answer:

Ok let me summarize the answer to my question since it is scattered across this page. If I get a detail wrong feel free to edit this or let me know and I will edit it:

So you found a .h or .cpp file that contains a function or some other bit of code you want to use in a .cpp file you are writing to use with Rcpp.

Lets keep calling this found code coolStuff.h and call the function you want to use awesomeSauce(). Lets call the file you are writing theCppFile.cpp.

(I should note here that the code in .h files and in .cpp files is all C++ code and the difference between them is for the C++ programer to keep things organized in the proper way. I will leave a discussion of the difference out here, but a simple search here on SO will lead you to discussion of the difference. For you the R programer needing to use a bit o' code you found, there is no real difference.)

IN SHORT: You can use a file like coolStuff.h provided it calls no other libraries, by either cut-and-pasteing into theCppFile.cpp, or if you create a package you can place the file in the \src directory with the theCppFile.cpp file and use #include "coolStuff.h" at the top of the file you are writing. The latter is more flexible and allows you to use functions in coolStuff.h in other .cpp files.

DETAILS:

1) coolStuff.h must not call other libraries. So that means it cannot have any include statements at the top. If it does, what I detail below probably will not work, and the use of found code that calls other libraries is beyond the scope of this answer.

2) If you want to compile the file with sourceCpp() you need to cut and paste coolStuff.h into theCppFile.cpp. I am told there are exceptions, but sourceCpp() is designed to compile one .cpp file, so thats the best route to take.

(NOTE: I make no guarantees that a simple cut and paste will work out of the box. You may have to rename variables, or more likely switch the data types being used to be consistent with those you are using in theCppFile.cpp. But so far, cut-and-paste has worked with minimal fuss for me with 6 different simple .h files)

3) If you only need to use code from coolStuff.h in theCppFile.cpp and nowhere else, then you should cut and paste it into theCppFile.cpp.

(Again I make no guarantees see the note above about cut-and-paste)

4) If you want to use code contained in coolStuff.h in theCppFile.cpp AND other .cpp files, you need to look into building a package. This is not hard, but can be a bit tricky, because the information out there about building packages with Rcpp ranges from the exhaustive thorough documentation you want with any R package (but that is above your head as a newbie), and the newbie sensitive introductions (that may leave out a detail you happen to need).

Here is what I suggest:

A) First get a version of theCppFile.cpp with the code from coolStuff.h cut-and-paste into theCppFile.cpp that compiles with sourceCpp() and works as you expect it to. This is not a must, but if you are new to Rcpp OR packages, it is nice to make sure your code works in this simple situation before you move to the more complicated case below.

B) Now build your package using Rcpp.package.skeleton() or use the Build functionality in RStudio (HIGHLY recommended). You can find details about using Rcpp.package.skeleton() in hadley/devtools or Rcpp Attributes Vignette. The full documentation for writing packages with Rcpp is in the Writing a package that uses Rcpp, however this one assumes you know your way around C++ fairly well, and does not use the new "Attributes" way of doing Rcpp.

Don't forget to "Build & Reload" if using RStudio or compileAttributes() if you are not in RStudio.

C) Now you should see in your \R directory a file called RcppExports.R. Open it and check it out. In RcppExports.R you should see the R wrapper functions for all the .cpp files you have in your \src directory. Pretty sweet.

D) Try out the R function that corresponds to the function you wrote in theCppFile.cpp. Does it work? If so move on.

E) With your package built you can move coolStuff.h into the src folder with theCppFile.cpp.

F) Now you can remove the cut-and-paste code from theCppFile.cpp and at the top of theCppFile.cpp (and any other .cpp file you want to use code from coolStuff.h) put #include "coolStuff.h" just after #include <Rcpp.h>. Note that there are no brackets around ranker.h, rather there are "". This is a C++ convention when including local files provided by the user rather than a library file like Rcpp or STL etc...

G) Now you have to rebuild the package. In RStudio this is just "Build & Reload" in the Build menu. If you are not using RStudio you should run compileAttributes()

H) Now try the R function again just as you did in step D), hopefully it works.

Solution 1

The problem is that sourceCpp is expressly designed to build only a single standalone source file. If you want sourceCpp to have dependencies then they need to either be:

1. In the system include directories (i.e. /usr/local/lib or /usr/lib); or

2. In an R package which you list in an Rcpp::depends attribute

As Dirk said, if you want to build more than one source file then you should consider using an R package rather than sourceCpp.

Note that if you are working on a package and perform a sourceCpp on a file within the src directory of the package it will build it as if it's in the package (i.e. you can include files from the src directory or inst/include directory).

Solution 2

I was able to link any library (MPFR in this case) by setting two environment variables before calling sourceCpp:

Sys.setenv("PKG_CXXFLAGS"="-I/usr/include")
Sys.setenv("PKG_LIBS"="-L/usr/lib/x86_64-linux-gnu/ -lm -lmpc -lgmp -lmpfr")

The first variable contains the path of the library headers. The second one includes the path of the library binary and its file name. In this case other dependent libraries are also required. For more details check g++ compilation and link flags. This information can usually be obtained using pkg-config:

pkg-config --cflags --libs mylib

For a better understanding, I recommend using sourceCpp with verbose output in order to print the g++ compilation and linking commands:

sourceCpp("mysource.cpp", verbose=TRUE, rebuild=TRUE)

Solution 3

I was able to link a boost library using the following global command in R before calling sourceCpp

Sys.setenv("PKG_CXXFLAGS"="-I \path-to-boost\")

Basically mirroring this post but with a different compiler option: http://gallery.rcpp.org/articles/first-steps-with-C++11/

Solution 4

Couple of things:

1. "Third party header libraries" as in your subject makes no sense.

2. Third-party headers can work via templated code where headers are all you need, ie there is only an include step and the compiler resolves things.

3. Once you need libraries and actual linking of object code, you may not be able the powerful and useful sourceCpp unless you gave it meta-information via plugins (or env. vars).

4. So in that case, write a package.

Easy and simple things are just that with Rcpp and the new attributes, or the older inline and cxxfunction. More for complex use --- and external libraries is more complex you need to consult the documentation. We added several vignettes to Rcpp for that.

Solution 5

This worked for me in Windows:

Sys.setenv("PKG_CXXFLAGS"='-I"C:/boost/boost_1_66_0"')

Edit: Actually you don't need this if you use Boost Headers (thanks to Ralf Stubner):

// [[Rcpp::depends(BH)]]

View more solutions

Comments

[JasonD]

#include "../cppHeaders/coolStuff.h"

[politicalEconomist]

No dice: I tried: #include "../cppHeaders/coolStuff.h" #include <"../cppHeaders/coolStuff.h"> #include <../cppHeaders/coolStuff.h> Still: : No such file or directory

[JasonD]

Well if your structure is as described, it ought to work. Is there a typo in the name or the include path? Is your file-system case-sensitive and you got the case wrong?

[politicalEconomist]

JasonD, I think that is true in vanilla C++ world. But with Rcpp it does not work. Rather I should say it depends on how one is compiling the C++ file. If you are using sourceCpp() to compile the file and create the R function, then a simple #include path/filename.h will not work. However, it looks like if you use the Rcpp.package.skeleton() or the equivalent "package creation wizard" in Rstudio then a simple #include path/filename.h will work under some conditions. Once I finally get all of this sorted I will write a summary of "calling other code in your cpp files using Rcpp".

[jjallaire]

Within a package you need to provide a Makevars that adds directories to the compilation, e.g. PKG_CPPFLAGS += -I../inst/include/

[Mullefa]

Great answer - thanks for the detail. On a related I found Hadley's reference on writing packages to strike a good balance betweebn detail and readability. In particular, it discusses using Rcpp in packages in a very friendly way.

[James Hirschorn]

@jjallaire This was the most helpful comment on this whole page. Perhaps add it to your answer?

[politicalEconomist]

Thanks for the explanation about the <> and "" However this still does not work. I even put a copy of coolStuff in the same folder with theCppFile.cpp AND a copy in the Rworking directory. Then I tried: #include "coolStuff.h" and still get the error.

[politicalEconomist]

I fixed the title. I think I am in case (2) and not case (3), but I am not sure. It is just one .h file with 3 functions in it. How does one know if they can just #include it and go or if they must build a package? Is this documented somewhere? I am building a package, however I am just prototyping everything now and I don't want to have to rebuild my package every time I debug one small cpp file.

[politicalEconomist]

Ok I am starting to get there. How can I look at a .h file and determine if it is the simple case Dirk refers to below or if it needs to build it's own source file and therefore I cannot simply #include it? Given the vast numbers of "ready to go" C++ files out there it seems like it would be a common occurrence for someone to run across one that does something they need and then want to use the file. I know my vocabulary is not quite right but I hope you can still get what i am saying. I'm a long time R guy, trying to convert my very slow package into Rcpp.

[politicalEconomist]

Of course after spending a whole day trying to figure out why my path for my file was wrong I could have just written the function I found myself. But hopefully this will help me in the future and help others.

[politicalEconomist]

I was able to use the header file by simply pasting all of it's contents directly into my cpp file. Then sourceCpp compiled it all without a hitch. This seems live a very "unmodular" way to program, because I will likely want to use the same functions in other cpp files. This will needlessly create repeated code all over the place. Also, I will want to use some of my own cpp code in other cpp files that I write. This will make changing things very time consuming.

[politicalEconomist]

There is really no way to pull in cpp code from other files at compile time like we do with source() in R at runtime? In any event it would be nice if the error produced when you try to sourceCpp a file that contains an #inclide "filename.h" was not so misleading. When an error says theCppFile.cpp:2:31: error: filename.h: No such file or directory the last thing I assume is that the compiler (or is it the sourceCpp() wrapper) doesn't like the fact that there is an #include of a file that just so happens to not be in /usr/local/lib or /usr/lib.

[jjallaire]

The underlying philosophy is that within R there is a system for building multiple source files that depend on each-other (and other libraries) and that's the package system. The idea behind sourceCpp wasn't to create an alternate parallel system to packages, but rather have an easy way to write a single function or source file without having to build a package. I realize you are looking for a middle ground and perhaps that's the right thing to do but we want to take those steps carefully.

[politicalEconomist]

Makes sense @jjallaire. Thanks for having the patience to explain it. looks like I need to explore how to incrementally create packages. Up until now I have protyped interactively with scripts then packaged at the end.

[Dirk Eddelbuettel]

@user1922182: Package may look intimidating at first, they are not. Look at package.skeleton() and Rcpp.package.skeleton(), or start from simple existing packages. Packages have a huge upside, use them. Lots of helpful questions answered here to so do search SO as well.

[politicalEconomist]

Ok now that I have figured out how to build packages that use Rcpp in Rstudio let me rephrase the question. I have a stand alone header file coolStuff.h that contains a function I want to use in my cpp code. 1) Where should I place coolStuff.h in the package directory structure so the function it contains can be used by theCppFile.cpp? 2) How do I call coolStuff.h in the cpp files? Thanks again for your help. I learned a lot from the last conversation. I read the vignette "Writing a package that uses Rcpp" and it does not explain how to do this.

[politicalEconomist]

I've edited the original post to include a summary of the answer to my question. Thanks to everyone for the help and patience.

[Dirk Eddelbuettel]

No you were not linking, you were including header files which happen to be usable without linking.

[ivan-k]

This answer is the most general of all here. Thanks for the details!