Write binary data to file, literally

Question

I have an array of integers

Array
(
    [0] => Array
        (
            [0] => 1531412763
            [1] => 1439959339
            [2] => 76
            [3] => 122
            [4] => 200
            [5] => 4550
            [6] => 444
        )
...

And so on, I suppose if I look at it as if it were a database - the elements of the outermost array are the rows and the elements of the inner arrays are the columns.

I want to save that information into a file, so that I will be able to retrieve it later but I want to save it as binary data to save space. Basically if I write the first integer from the example 1531412763 to a file it will take up 10 bytes but if I could save it as a signed integer it would take up 4 bytes.

I have looked at a number of other answers which all suggest using fwrite which I can't understand how to use in such manner?

Solution

For writing binary data to a file, you can use the functions pack() and unpack(). Pack will produce a binary string. As the result is a string, you can concatenate the ints into a single string. Then write this string as a line to your file.

This way you can easily read with file() which will put the file into an array of lines. Then just unpack() each line, and you have your original array back.

Something like this :

$arr = array(
    array ( 1531412763, 1439959339 ),
    array ( 123, 456, 789 ),
);

$file_w = fopen('binint', 'w+');

// Creating file content : concatenation of binary strings 
$bin_str = '';
foreach ($arr as $inner_array_of_int) {
    foreach ($inner_array_of_int as $num) {
        // Use of i format (integer). If you want to change format
        // according to the value of $num, you will have to save the
        // format too.
        $bin_str .= pack('i', $num);
    }

    $bin_str .= "\n";
}

fwrite($file_w, $bin_str);
fclose($file_w);


// Now read and test. $lines_read will contain an array like the original.
$lines_read = [];
// We use file function to read the file as an array of lines.
$file_r = file('binint');

// Unpack all lines
foreach ($file_r as $line) {
    // Format is i* because we may have more than 1 int in the line
    // If you changed format while packing, you will have to unpack with the
    // corresponding same format
    $lines_read[] = unpack('i*', $line);
}

var_dump($lines_read);

Comments

[Zimmi]

pack ?

[Andrei]

If you really need to save space, why not compress the data too? Might as well at this point.

[php_nub_qq]

@Zimmi yeah that's exactly what I needed, but do I need to call pack on each individual value or is there an easier way?

[Zimmi]

You cannot pack($array) as it is. But you can give multiple arguments to the pack function and use repeaters (*) after the format. Some thing like pack('i*', $int_1, $int_2,....) for packing multiple integers. You will have to handle your array according to the format you want.

[php_nub_qq]

@Zimmi So basically how I have to do it is pack('LLSSSQ', $row[0], ..., $row[6]) then write that as a single line in the file and when reading I need to use unpack with the same format which will give me the array back. That's perfect, you can put it as an answer for someone else in my situation.

[jww]

Possible duplicate of Write binary file in PHP

[php_nub_qq]

And if each row contains the exact same number of elements you don't even need new lines, you just need to calculate the row length when converted to binary and then fread($handle, $length).

[Zimmi]

Absolutely! And optimize the format, as you suggested in your last comment in the question.

[php_nub_qq]

Using this method, instead of storing plain text I managed to save quite some space. From 2.72GB down to 400MB, that's a 6.8 times reduction!