Share variables/memory between all PHP processes

Question

Is it possible to share variables and arrays between all PHP processes without duplicating them?

Using memcached, I think PHP duplicates the used memory:
$array = $memcache->get('array');
$array will contain a copy from memcached.

So my idea is, there could be a static variable that was already defined, and shared between all processes.

Solution 1

By default its simply not possible. Every solution will always copy the content into the current scope, because if not, there is no way to access it.

I dont know, what exactly want to do, but maybe you can do that "outside", for example as a gearman job, and then just catch the results of the process, instead of the whole array.

You can also think about splitting the "big" array into slices and then always retrieve the part you currently need from an apc or memcached.

Solution 2

Using Shmop:

Shmop is an easy to use set of functions that allows PHP to read, write, create and delete Unix shared memory segments.

from: http://www.php.net/manual/en/intro.shmop.php

No external libraries are needed to build this extension.

The shared Memory Functions

• shmop_close — Close
• shared memory block
• shmop_delete — Delete shared memory block
• shmop_open — Create or open shared memory block
• shmop_read — Read data from shared memory block
• shmop_size — Get size of shared memory block
• shmop_write — Write data into shared memory block

Basic usage

// Create 100 byte shared memory block with system id of 0xff3
$shm_id = shmop_open(0xff3, "c", 0644, 100);
if (!$shm_id) {
    echo "Couldn't create shared memory segment\n";
}

// Get shared memory block's size
$shm_size = shmop_size($shm_id);
echo "SHM Block Size: " . $shm_size . " has been created.\n";

// Lets write a test string into shared memory
$shm_bytes_written = shmop_write($shm_id, "my shared memory block", 0);
if ($shm_bytes_written != strlen("my shared memory block")) {
    echo "Couldn't write the entire length of data\n";
}

// Now lets read the string back
$my_string = shmop_read($shm_id, 0, $shm_size);
if (!$my_string) {
    echo "Couldn't read from shared memory block\n";
}
echo "The data inside shared memory was: " . $my_string . "\n";

//Now lets delete the block and close the shared memory segment
if (!shmop_delete($shm_id)) {
    echo "Couldn't mark shared memory block for deletion.";
}
shmop_close($shm_id);

Solution 3

One way to share memory between PHP processes is to install a PHP-bytecode cache like APC. APC is primarily used for storing the bytecode into an OS managed shared-memory segment, but it also has an API for sharing anything you want between processes (like a local version of memcache).

<?php
   $foobar = array('foo', 'bar');
   apc_store('foobar', $foobar);
?>

Then elsewhere:

<?php
    $foobar = apc_fetch('foobar');
    var_dump($foobar);
?>

The big problem with sharing-memory is that it becomes very easy for two processes to step on each other's foot. So shared memory is best for things that don't change too much, like big global arrays.

Solution 4

PHP has magic methods:

• __get($property) let us implement the access of a $property on an object
• __set($property, $value) let us implement the assignation of a $property on an object

PHP can serialize variables:

• serialize($variable) returns a string representation of the variable
• unserialize($string) returns back a variable from a string

PHP can handle files, with concurrent-access management:

• fopen($file, 'c+') opens a file with advisory lock options enabled (allow you to use flock)
• flock($descriptor, LOCK_SH) takes a shared lock (for reading)
• flock($descriptor, LOCK_EX) takes an exclusive lock (for writting)

So, the easiest way to share an object between apps is to create a class that implements and use all those stuffs to save and restore instantly all its data into a file.

A simple implementation of that class could be :

class Synchro
{

   private $_file;

   public function __construct($file)
   {
       $this->_file = $file;
   }

   public function __get($property)
   {
       // File does not exist
       if (!is_file($this->_file))
       {
           return null;
       }

       // Check if file is readable
       if ((is_file($this->_file)) && (!is_readable($this->_file)))
       {
           throw new Exception(sprintf("File '%s' is not readable.", $this->_file));
       }

       // Open file with advisory lock option enabled for reading and writting
       if (($fd = fopen($this->_file, 'c+')) === false)
       {
           throw new Exception(sprintf("Can't open '%s' file.", $this->_file));
       }

       // Request a lock for reading (hangs until lock is granted successfully)
       if (flock($fd, LOCK_SH) === false)
       {
           throw new Exception(sprintf("Can't lock '%s' file for reading.", $this->_file));
       }

       // A hand-made file_get_contents
       $contents = '';
       while (($read = fread($fd, 32 * 1024)) !== '')
       {
           $contents .= $read;
       }

       // Release shared lock and close file
       flock($fd, LOCK_UN);
       fclose($fd);

       // Restore shared data object and return requested property
       $object = json_decode($contents);
       if (property_exists($object, $property))
       {
           return $object->{$property};
       }

       return null;
   }

   public function __set($property, $value)
   {
       // Check if directory is writable if file does not exist
       if ((!is_file($this->_file)) && (!is_writable(dirname($this->_file))))
       {
           throw new Exception(sprintf("Directory '%s' does not exist or is not writable.", dirname($this->_file)));
       }

       // Check if file is writable if it exists
       if ((is_file($this->_file)) && (!is_writable($this->_file)))
       {
           throw new Exception(sprintf("File '%s' is not writable.", $this->_file));
       }

       // Open file with advisory lock option enabled for reading and writting
       if (($fd = fopen($this->_file, 'c+')) === false)
       {
           throw new Exception(sprintf("Can't open '%s' file.", $this->_file));
       }

       // Request a lock for writting (hangs until lock is granted successfully)
       if (flock($fd, LOCK_EX) === false)
       {
           throw new Exception(sprintf("Can't lock '%s' file for writing.", $this->_file));
       }

       // A hand-made file_get_contents
       $contents = '';
       while (($read = fread($fd, 32 * 1024)) !== '')
       {
           $contents .= $read;
       }

       // Restore shared data object and set value for desired property
       if (empty($contents))
       {
           $object = new stdClass();
       }
       else
       {
           $object = json_decode($contents);
       }
       $object->{$property} = $value;

       // Go back at the beginning of file
       rewind($fd);

       // Truncate file
       ftruncate($fd, strlen($contents));

       // Save shared data object to the file
       fwrite($fd, json_encode($object));

       // Release exclusive lock and close file
       flock($fd, LOCK_UN);
       fclose($fd);

       return $value;
   }

}

Now, you can use this class like stdClass, but with a file path when constructing.

$obj = new Synchro("/tmp/test.sync"); 
$obj->hello = 'world';

// ... and in another process...
echo $obj->hello;

This example is of course very simple, it takes care about concurrent access to a file but not to a variable, in a better implementation you'll use a mutex-like lock.

I just pushed this class (after completing it) on github, you can find it here.

View more solutions

Comments

[Nuno]

That seems the same as Memcached. Thank you for your answer, anyway :)

[Pacerier]

@NunoPeralta, What about Shmop? See below.

[Pacerier]

You have misunderstood the question.

[Pang]

To use shmop you will need to compile PHP with the --enable-shmop parameter in your configure line.

[bhelm]

This is wrong, apc_store etc. cannot share memory between processes. Each process allocates its own memory segment. you can i.e. not share memory between php-fpm and php-cli for that reason (while sharing between different php-fpm requests works).

[Meloman]

like it... using file is maybe the simpliest way, and more secure way, because no attempt to server's memory. I think it's quicker than asking a database too.

[Pablo Pazos]

this is no different than using a database, the idea is to share variables in memory, not on disk.

[Viktor Joras]

There is no other way in PHP than using disk files to share data between non pre forked processes. APC/APCu and Memcached work only within the same master process like FPM for example. If the processes are distinct then adios amigos, so the this is the only possible answer.

[ttvd94]

If it's not possible, then what's Shmop?

[KingCrunch]

@ttvd94 One part of the question was "without duplication", which is not possible shmop neither. The moment you call shmop_read the return value is actually a copy of the actual value in shared memory. You cannot read by reference from shmop. The benefit of shmop is the read-and-retrieve performance, which is in many cases negligible compared to a database.

[Meloman]

I just wanted to re-use this solution on a project and - I don't know why - when I put a value true on a var, the __get function doesn't find my var. I found }} at end of tmp file instead of }. So, I added fwrite($fd, str_replace('}}','}',json_encode($object))); but it is a workaround I don't like.