C++ lambda with captures as a function pointer

Question

I was playing with C++ lambdas and their implicit conversion to function pointers. My starting example was using them as callback for the ftw function. This works as expected.

#include <ftw.h>
#include <iostream>

using namespace std;

int main()
{
    auto callback = [](const char *fpath, const struct stat *sb,
        int typeflag) -> int {
        cout << fpath << endl;
        return 0;
    };

    int ret = ftw("/etc", callback, 1);

    return ret;
}

After modifying it to use captures:

int main()
{

    vector<string> entries;

    auto callback = [&](const char *fpath, const struct stat *sb,
        int typeflag) -> int {
        entries.push_back(fpath);
        return 0;
    };

    int ret = ftw("/etc", callback, 1);

    for (auto entry : entries ) {
        cout << entry << endl;
    }

    return ret;
}

I got the compiler error:

error: cannot convert ‘main()::<lambda(const char*, const stat*, int)>’ to ‘__ftw_func_t {aka int (*)(const char*, const stat*, int)}’ for argument ‘2’ to ‘int ftw(const char*, __ftw_func_t, int)’

After some reading. I learned that lambdas using captures can't be implicitly converted to function pointers.

Is there a workaround for this? Does the fact that they can't be "implicitly" converted mean s that they can "explicitly" converted? (I tried casting, without success). What would be a clean way to modify the working example so that I could append the entries to some object using lambdas?.

Solution 1

Since capturing lambdas need to preserve a state, there isn't really a simple "workaround", since they are not just ordinary functions. The point about a function pointer is that it points to a single, global function, and this information has no room for a state.

The closest workaround (that essentially discards the statefulness) is to provide some type of global variable which is accessed from your lambda/function. For example, you could make a traditional functor object and give it a static member function which refers to some unique (global/static) instance.

But that's sort of defeating the entire purpose of capturing lambdas.

Solution 2

I just ran into this problem.

The code compiles fine without lambda captures, but there is a type conversion error with lambda capture.

Solution with C++11 is to use std::function (edit: another solution that doesn't require modifying the function signature is shown after this example). You can also use boost::function (which actually runs significantly faster). Example code - changed so that it would compile, compiled with gcc 4.7.1:

#include <iostream>
#include <vector>
#include <functional>

using namespace std;

int ftw(const char *fpath, std::function<int (const char *path)> callback) {
  return callback(fpath);
}

int main()
{
  vector<string> entries;

  std::function<int (const char *fpath)> callback = [&](const char *fpath) -> int {
    entries.push_back(fpath);
    return 0;
  };

  int ret = ftw("/etc", callback);

  for (auto entry : entries ) {
    cout << entry << endl;
  }

  return ret;
}

Edit: I had to revisit this when I ran into legacy code where I couldn't modify the original function signature, but still needed to use lambdas. A solution that doesn't require modifying the function signature of the original function is below:

#include <iostream>
#include <vector>
#include <functional>

using namespace std;

// Original ftw function taking raw function pointer that cannot be modified
int ftw(const char *fpath, int(*callback)(const char *path)) {
  return callback(fpath);
}

static std::function<int(const char*path)> ftw_callback_function;

static int ftw_callback_helper(const char *path) {
  return ftw_callback_function(path);
}

// ftw overload accepting lambda function
static int ftw(const char *fpath, std::function<int(const char *path)> callback) {
  ftw_callback_function = callback;
  return ftw(fpath, ftw_callback_helper);
}

int main() {
  vector<string> entries;

  std::function<int (const char *fpath)> callback = [&](const char *fpath) -> int {
    entries.push_back(fpath);
    return 0;
  };
  int ret = ftw("/etc", callback);

  for (auto entry : entries ) {
    cout << entry << endl;
  }

  return ret;
}

Solution 3

ORIGINAL

Lambda functions are very convenient and reduce a code. In my case I needed lambdas for parallel programming. But it requires capturing and function pointers. My solution is here. But be careful with scope of variables which you captured.

template<typename Tret, typename T>
Tret lambda_ptr_exec(T* v) {
    return (Tret) (*v)();
}

template<typename Tret = void, typename Tfp = Tret(*)(void*), typename T>
Tfp lambda_ptr(T& v) {
    return (Tfp) lambda_ptr_exec<Tret, T>;
}

Example

int a = 100;
auto b = [&]() { a += 1;};
void (*fp)(void*) = lambda_ptr(b);
fp(&b);

Example with a return value

int a = 100;
auto b = [&]() {return a;};
int (*fp)(void*) = lambda_ptr<int>(b);
fp(&b);

UPDATE

Improved version

It was a while since first post about C++ lambda with captures as a function pointer was posted. As It was usable for me and other people I made some improvement.

Standard function C pointer api uses void fn(void* data) convention. By default this convention is used and lambda should be declared with a void* argument.

Improved implementation

struct Lambda {
    template<typename Tret, typename T>
    static Tret lambda_ptr_exec(void* data) {
        return (Tret) (*(T*)fn<T>())(data);
    }

    template<typename Tret = void, typename Tfp = Tret(*)(void*), typename T>
    static Tfp ptr(T& t) {
        fn<T>(&t);
        return (Tfp) lambda_ptr_exec<Tret, T>;
    }

    template<typename T>
    static void* fn(void* new_fn = nullptr) {
        static void* fn;
        if (new_fn != nullptr)
            fn = new_fn;
        return fn;
    }
};

Exapmle

int a = 100;
auto b = [&](void*) {return ++a;};

Converting lambda with captures to a C pointer

void (*f1)(void*) = Lambda::ptr(b);
f1(nullptr);
printf("%d\n", a);  // 101 

Can be used this way as well

auto f2 = Lambda::ptr(b);
f2(nullptr);
printf("%d\n", a); // 102

In case return value should be used

int (*f3)(void*) = Lambda::ptr<int>(b);
printf("%d\n", f3(nullptr)); // 103

And in case data is used

auto b2 = [&](void* data) {return *(int*)(data) + a;};
int (*f4)(void*) = Lambda::ptr<int>(b2);
int data = 5;
printf("%d\n", f4(&data)); // 108

Solution 4

Using locally global (static) method it can be done as followed

template <class F>
auto cify_no_args(F&& f) {
  static F fn = std::forward<F>(f);
  return [] {
    return fn();
  };
}

Suppose we have

void some_c_func(void (*callback)());

So the usage will be

some_c_func(cify_no_args([&] {
  // code
}));

This works because each lambda has an unique signature so making it static is not a problem. Following is a generic wrapper with variadic number of arguments and any return type using the same method.

template <class F>
struct lambda_traits : lambda_traits<decltype(&F::operator())>
{ };

template <typename F, typename R, typename... Args>
struct lambda_traits<R(F::*)(Args...)> : lambda_traits<R(F::*)(Args...) const>
{ };

template <class F, class R, class... Args>
struct lambda_traits<R(F::*)(Args...) const> {
    using pointer = typename std::add_pointer<R(Args...)>::type;

    static pointer cify(F&& f) {
        static F fn = std::forward<F>(f);
        return [](Args... args) {
            return fn(std::forward<Args>(args)...);
        };
    }
};

template <class F>
inline typename lambda_traits<F>::pointer cify(F&& f) {
    return lambda_traits<F>::cify(std::forward<F>(f));
}

And similar usage

void some_c_func(int (*callback)(some_struct*, float));

some_c_func(cify([&](some_struct* s, float f) {
    // making use of "s" and "f"
    return 0;
}));

Solution 5

Hehe - quite an old question, but still...

#include <iostream>
#include <vector>
#include <functional>

using namespace std;

// We dont try to outsmart the compiler...
template<typename T>
int ftw(const char *fpath, T callback) {
  return callback(fpath);
}

int main()
{
  vector<string> entries;

  // ... now the @ftw can accept lambda
  int ret = ftw("/etc", [&](const char *fpath) -> int {
    entries.push_back(fpath);
    return 0;
  });

  // ... and function object too 
  struct _ {
    static int lambda(vector<string>& entries, const char* fpath) {
      entries.push_back(fpath);
      return 0;
    }
  };
  ret = ftw("/tmp", bind(_::lambda, ref(entries), placeholders::_1));

  for (auto entry : entries ) {
    cout << entry << endl;
  }

  return ret;
}

View more solutions

Comments

[Ramon Zarazua B.]

What compiler are you using? is it VS10?

[duncan]

gcc version 4.6.1 20110801 [gcc-4_6-branch revision 177033] (SUSE Linux)

[Alexandre C.]

Usually, the C way of passing state to callbacks is done via an extra argument to the callback (usually of type void *). If the library you are using allows for this extra argument, you will find a workaround. Otherwise, you have no way to achieve cleanly what you want to do.

[duncan]

Yes. I realize the api of ftw.h and nftw.h is flawed. I will try fts.h

[duncan]

Great! /usr/include/fts.h:41:3: error: #error "<fts.h> cannot be used with -D_FILE_OFFSET_BITS==64"

[danijar]

@RamonZarazua. My compiler is Visual Studio 11.

[danijar]

@AlexandreC. It allows that. But I have issues to provide a clean API that only expects the lambda. I believe at least lambdas with capture clause must store a reference to their state.

[danijar]

The answers given here are correct but can't solve my actual problem. Therefore I formulated another, hopefully clearer, question. stackoverflow.com/questions/15748961/…

[fiorentinoing]

please, remember that a capturing lambda has a lifetime. so many solutions here proposed are prone to error (i.e. segfault at runtime) if misused.

[user7610]

Does this answer your question? Passing capturing lambda as function pointer

[Raymond Chen]

A cleaner solution is to wrap the lambda inside an adapter, assuming that the function pointer has a context parameter.

[Kerrek SB]

@RaymondChen: Well, if you're at liberty to define how the function is to be used, then yes, that's an option. Though in that case it'd be even easier to just make the parameter an argument of the lambda itself!

[Gregory Pakosz]

No this should not be the accepted answer. The point is not changing ftw to take std::function instead of a function pointer...

[Kjell Hedström]

@KerrekSB put the global variables in a namespace and mark them as thread_local, that's the ftw approach I chose for solving something similar.

[Dexter]

"a function pointer points to a single, global function, and this information has no room for a state." -> How the hell can languages like Java accomplish this then? Well, of course, because that single, global function is created at runtime and embeds the state (or rather the reference to it) in its own code. That is the whole point - there should not be a single, global function but multiple global functions - one for each time lambda is used in runtime. Is there really NOTHING in C++ that does that? (I thought std::function is made exactly for that single purpose)

[Kerrek SB]

@Dexter: errr.. the short answer is no, the long answer involves operator overloading. Regardless, my point stands. Java is a different language that is not the same as C++; Java doesn't have pointers (or overloadable call operators) and the comparison doesn't work well.

[Codoscope]

This is definitely the most convenient solution I have seen to convert a lambda to a C-style function pointer. The function taking it as an argument will just need an extra parameter representing its state, often named "void *user" in C libraries, so that it can pass it to the function pointer when calling it.

[prideout]

The second solution proposed within this answer addresses the concern from @gregory-pakosz by preserving the original signature, but it's still not great because it introduces global state. If ftw had a void* userdata argument, then I'd prefer the answer from @evgeny-karpov.

[Jay West]

@prideout agreed - I don't like the global state, either. Unfortunately, assuming ftw's signature cannot be modified and given that it does not have void* userdata, state has to be stored somewhere. I ran into this problem using a 3rd party library. This will work fine as long as the library doesn't capture the callback and use it later, in which case the global variable simply acts like an extra parameter on the call stack. If ftw's signature can be modified, then I'd prefer using std::function instead of void* userdata.

[Ivan Sanz Carasa]

be aware that this will copy the closure (when getting the ptr) + args (when calling). Otherwise, it is a elegant solution

[Ivan Sanz Carasa]

header-only helper library: gist.github.com/isc30/fab67e5956fe8f2097bed84ebc42c1e8

[Vladimir Talybin]

@IvanSanz-Carasa Thanks for pointing out. Closure types are not CopyAssignable, but functors are. So you right, it is better to use perfect forwarding here. For args on other hand we can't do much as plain C does not support universal references, but at least we can forward values back to our lambda. This may save an extra copy. Has edited code.

[fiorentinoing]

this is an extremely complicated and useful solution, @Gregory I should tell you "it works".

[Vladimir Talybin]

@RiaD Yes, because lambda a static instance here you will need to capture by reference instead, e.g. instead of = use &i in your for-loop.

[Günther the Beautiful]

The link is no longer valid

[tjwrona1992]

I've been looking for a way to do this for a LONG time... This solution still goes way over my head but this is amazing.

[tjwrona1992]

One specific thing that is confusing me is the use of F::*, what does this mean exactly?

[Vladimir Talybin]

@tjwrona1992 It is a pointer to a non-static member of class F (F::operator()(...) in this case), read more here en.cppreference.com/w/cpp/language/pointer

[Vladimir Nikishkin]

The web archive has the link web.archive.org/web/20161012142308/http://meh.schizofreni.co‌​/…

[Vladimir Nikishkin]

Which standard is required for this answer to work? clang 12 is complaining with "error: missing 'typename' prior to dependent type name 'lambda_traits<F>::pointer'"

[Vladimir Talybin]

@VladimirNikishkin You are correct, there is a typo in return type. Fixed.

[hlovdal]

This solution works well in Arduino as well, where only a subset of C++ is available.