Create a shared-memory vector of strings

Question

I am trying to create a class managing a shared-memory vector of (std)strings.

typedef boost::interprocess::allocator<std::string, boost::interprocess::managed_shared_memory::segment_manager> shmem_allocator;
typedef boost::interprocess::vector<std::string, shmem_allocator> shmem_vector;

shmem_mgr::shmem_mgr() :
    shmem_(create_only, SHMEM_KEY, SHMEM_SIZE),
    allocator_(shmem_.get_segment_manager())
{
    mutex_  = shmem_.find_or_construct<interprocess_mutex>(SHMEM_MUTEX)();
    condition_ = shmem_.find_or_construct<interprocess_condition>(SHMEM_CONDITION)();
    //buffer_ is of type shmem_vector
    buffer_  = shmem_.construct<shmem_vector>(SHMEM_BUFFER_KEY)(allocator_);
}

void shmem_mgr::run() {
    running_ = true;

    while(running_) {
        scoped_lock<interprocess_mutex> lock ( *mutex_ );

        int size = buffer_->size();

        log_.debug() << size << " queued request(s) found" << std::endl; //LINE 27
        for(int i=0; i<size; i++) {
            log_.debug() << buffer_->at(i); // at() crashes my app
        }

        buffer_->clear(); //so does clear()
        condition_->wait (lock);
    }
}

The client successfully adds a string to the vector (it also succeeds to read that string from the buffer for debug), the manager (code above) receives the signal (condtion variable), writes that there is a string in the vector (line 27), but when it tries to get that string via at() the application crashes.

---

Edit: I've figured out, that the use of std::string is not possible, there is a string container in boost ipc just for this case. This doesn't change the fact that I need a vector of (boost/std) strings...

---

Q: How can I pass strings across shared memory? I need to store them in some buffer (capable of storing >1 at a time) in shmem, and then fetch in second process - that's the requirement. The input is always std::string and so is the output, but the internal representation in shmem may be different.

Solution 1

From the docs.

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/interprocess/allocators/allocator.hpp>

int main ()
{
   using namespace boost::interprocess;
   //Typedefs
   typedef allocator<char, managed_shared_memory::segment_manager>
      CharAllocator;
   typedef basic_string<char, std::char_traits<char>, CharAllocator>
      MyShmString;
   typedef allocator<MyShmString, managed_shared_memory::segment_manager>
      StringAllocator;
   typedef vector<MyShmString, StringAllocator>
      MyShmStringVector;

   //Open shared memory
   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   managed_shared_memory shm(create_only, "MySharedMemory", 10000);

   //Create allocators
   CharAllocator     charallocator  (shm.get_segment_manager());
   StringAllocator   stringallocator(shm.get_segment_manager());

   //This string is in only in this process (the pointer pointing to the
   //buffer that will hold the text is not in shared memory).
   //But the buffer that will hold "this is my text" is allocated from
   //shared memory
   MyShmString mystring(charallocator);
   mystring = "this is my text";

   //This vector is only in this process (the pointer pointing to the
   //buffer that will hold the MyShmString-s is not in shared memory).
   //But the buffer that will hold 10 MyShmString-s is allocated from
   //shared memory using StringAllocator. Since strings use a shared
   //memory allocator (CharAllocator) the 10 buffers that hold
   //"this is my text" text are also in shared memory.
   MyShmStringVector myvector(stringallocator);
   myvector.insert(myvector.begin(), 10, mystring);

   //This vector is fully constructed in shared memory. All pointers
   //buffers are constructed in the same shared memory segment
   //This vector can be safely accessed from other processes.
   MyShmStringVector *myshmvector =
      shm.construct<MyShmStringVector>("myshmvector")(stringallocator);
   myshmvector->insert(myshmvector->begin(), 10, mystring);

   //Destroy vector. This will free all strings that the vector contains
   shm.destroy_ptr(myshmvector);
   return 0;
}

Solution 2

you need a custom allocator for your sharable stl classes. you need a self-based pointer (ACE & boost have these) defined in the allocator. On opposite sides the (CONTIGUOUS) shared memory typically resides at different addresses. You need a shared memory allocation subsystem(heap manager) too (that the allocator allocates from) - all non-trivial low level code, but most definitely doable and once you have it, it is usable everywhere. If you do all that, you need only pass the displacement (from beginning of the (CONTIGUOUS!!) heap area) of the non-flat structure around.

You can create queues and everything else you might want - all provided that the "pointers" in the objects are self-based and that the dis-contiguous pieces in your non- flat pieces come from one large contiguous piece.

You can't use std::string because, unless you control the allocation, the memory in standard string has NOTHING to do with your shared memory - same for any other stl structure

It is also imperative (as usual) to resolve/agree ownership issues

Solution 3

You can use boost::interprocess::managed_shared_memory. The following program passes a boost::interprocess::string between 2 processes. Works fine on my machine (Ubuntu Linux). You can use managed_shared_memory to pass vectors or objects. boost::interprocess::string has a c_str() method.

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <cstring>
#include <cstdlib>
#include <string>
#include <iostream>

int main(int argc, char *argv[])
{
  using namespace boost::interprocess;
  typedef boost::interprocess::allocator<char, boost::interprocess::managed_shared_memory::segment_manager> CharAllocator;
  typedef boost::interprocess::basic_string<char, std::char_traits<char>, CharAllocator> string;
  if(argc == 1){  //Parent process

      boost::interprocess::shared_memory_object::remove("MySharedMemory");

      //Create a shared memory object.
      managed_shared_memory shm (create_only, "MySharedMemory", 1024);

      string *s = shm.find_or_construct<string>("String")("Hello!", shm.get_segment_manager());
      std::cout << *s << std::endl;

      //Launch child process
      std::string s1(argv[0]); s1 += " child ";
      if(0 != std::system(s1.c_str()))
         return 1;
  }
  else{
      //Open already created shared memory object.
      managed_shared_memory shm (open_only, "MySharedMemory");
      std::pair<string *,std::size_t> ret = shm.find<string>("String");
      std::cout << *(ret.first) << std::endl;
  }
  return 0;
}

Comments

[imreal]

Where is "i" being declared? Dont you mean to use it?

[emesx]

Ok. How about a boost::interproces::vector of boost::interprocess::string or any other vector of some strings? I need to transfer some strings across 2 processes, what do I do

[emesx]

Almost perfect. Just describe how to convert MyShmString to a std::string.

[Paul Groke]

std::string s = std::string(myShmString.begin(), myShmString.end());