Sharing data array between two applications in Delphi

Question

I want to share array data between two applications. In my mind, first program create the array and the second program can read the array from already allocated memory area. The array is not a dynamic array.

I found a way to share pointer using OpenFileMapping and MapViewOfFile. I have no luck to implement array sharing and I think i don't want to use IPC method yet.

Is it possible to plan a scheme like this (sharing array)? My purpose is to minimize memory usage and reading data quickly.

Solution 1

Scratched my head thinking of what a short-but-complete example of sharing memory between two applications might be. The only option is a console application, GUI applications require a minimum of 3 files (DPR + PAS + DFM). So I cooked up a small example where one integers array is shared using a memory mapped file (backed by the page file so I don't need to have a phisical file on disk for this to work). The console application responds to 3 commands:

• EXIT
• SET NUM VALUE Changes the value at index NUM in the array to VALUE
• DUMP NUM displays the value in the array at index NUM
• DUMP ALL displays the whole array

Of course, the command processing code takes up about 80% of the whole application. To test this compile the following console application, find the executable and start it twice. Go to the first window and enter:

SET 1 100
SET 2 50

Go to the second console and enter this:

DUMP 1
DUMP 2
DUMP 3
SET 1 150

Go to the first console and enter this:

DUMP 1

There you have it, you've just witnessed sharing memory between two applications.

program Project2;

{$APPTYPE CONSOLE}

uses
  SysUtils, Windows, Classes;

type
  TSharedArray = array[0..10] of Integer;
  PSharedArray = ^TSharedArray;

var
  hFileMapping: THandle; // Mapping handle obtained using CreateFileMapping
  SharedArray: PSharedArray; // Pointer to the shared array
  cmd, s: string;
  num, value, i: Integer;
  L_CMD: TStringList;

function ReadNextCommand: string;
begin
  WriteLn('Please enter command (one of EXIT, SET NUM VALUE, DUMP NUM, DUMP ALL)');
  WriteLn;
  ReadLn(Result);
end;

begin
  try
    hFileMapping := CreateFileMapping(0, nil, PAGE_READWRITE, 0, SizeOf(TSharedArray), '{C616DDE6-23E2-425C-B871-9E0DA54D96DF}');
    if hFileMapping = 0 then
      RaiseLastOSError
    else
      try
        SharedArray := MapViewOfFile(hFileMapping, FILE_MAP_READ or FILE_MAP_WRITE, 0, 0, SizeOf(TSharedArray));
        if SharedArray = nil then
          RaiseLastOSError
        else
          try
            WriteLn('Connected to the shared view of the file.');

            cmd := ReadNextCommand;
            while UpperCase(cmd) <> 'EXIT' do
            begin
              L_CMD := TStringList.Create;
              try
                L_CMD.DelimitedText := cmd;
                for i:=0 to L_CMD.Count-1 do
                  L_CMD[i] := UpperCase(L_CMD[i]);

                if (L_CMD.Count = 2) and (L_CMD[0] = 'DUMP') and TryStrToInt(L_CMD[1], num) then
                  WriteLn('SharedArray[', num, ']=', SharedArray^[num])
                else if (L_CMD.Count = 2) and (L_CMD[0] = 'DUMP') and (L_CMD[1] = 'ALL') then
                  begin
                    for i:= Low(SharedArray^) to High(SharedArray^) do
                      WriteLn('SharedArray[', i, ']=', SharedArray^[i]);
                  end
                else if (L_CMD.Count = 3) and (L_CMD[0] = 'SET') and TryStrToInt(L_CMD[1], num) and TryStrToInt(L_CMD[2], value) then
                  begin
                    SharedArray^[num] := Value;
                    WriteLn('SharedArray[', num, ']=', SharedArray^[num]);
                  end
                else
                   WriteLn('Error processing command: ' + cmd);

              finally L_CMD.Free;
              end;

              // Requst next command
              cmd := ReadNextCommand;
            end;


          finally UnmapViewOfFile(SharedArray);
          end;
      finally CloseHandle(hFileMapping);
      end;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

Solution 2

A Named File Mapping would be the easiest solution, here is some short example code. In this sample there is a main program that writes some data and reader(s) that only read from it.

Main:

type
  TSharedData = record
    Handle: THandle;
  end;
  PSharedData = ^TSharedData;

const
  BUF_SIZE = 256;
var
  SharedData: PSharedData;
  hFileMapping: THandle;  // Don't forget to close when you're done

function CreateNamedFileMapping(const Name: String): THandle;
begin
  Result := CreateFileMapping(INVALID_HANDLE_VALUE, nil, PAGE_READWRITE, 0,
    BUF_SIZE, PChar(Name));

  Win32Check(Result > 0);

  SharedData := MapViewOfFile(Result, FILE_MAP_ALL_ACCESS, 0, 0, BUF_SIZE);

  Win32Check(Assigned(SharedData));
end;

procedure TForm1.Button1Click(Sender: TObject);
begin
  hFileMapping := CreateNamedFileMapping('MySharedMemory');
  Win32Check(hFileMapping > 0);
  SharedData^.Handle := CreateHiddenWindow;
end;

reader:

var
  hMapFile: THandle;   // Don't forget to close

function GetSharedData: PSharedData;
begin
  hMapFile := OpenFileMapping(FILE_MAP_ALL_ACCESS, False, 'MySharedMemory');
  Win32Check(hMapFile > 0);

  Result := MapViewOfFile(hMapFile, FILE_MAP_ALL_ACCESS, 0, 0, BUF_SIZE);

  Win32Check(Assigned(Result));
end;

Comments

[Cosmin Prund]

What have you tried and how it failed? Because that's precisely how you do it, both applications map a view of the same file so they're both working with the same memory. What's in your array?

[user]

@Cosmin, Why I can't show all items using looping? It shows many 0. `for j := low(SharedArray^) to high(SharedArray^) do WriteLn(SharedArray^[j]);'. I need to match a specific data with the array. I just tried looping, I hope the shared arary can work with binary search.

[Cosmin Prund]

The shared block of memory is nothing magic, it's just a shared block of memory. You can implement whatever algorithm you want using it. It shows lots of zeros because the block of memory is zero-initialized.

[user]

Cosmin, I tried to show all items but I see ^only^ lots of zeros.

[Cosmin Prund]

You are leaking handles: the handle returned by CreateFileMapping is only assigned to a local variable in TForm1.Button1Click so there's no way to close it. This is not required: ZeroMemory(SharedData, BUF_SIZE); - File mappings backed by the page file are (thankfully) zero-initialized. It would be a security threat if that memory was not zero-initialized!

[Cosmin Prund]

Did you compile my program? Did you issue the "SET 1 2" command? You'll only get ZEROes unless you write something else yourself.

[Cosmin Prund]

An other possible issue: You need to keep at least one instance of the program running to maintain the data in memory. The moment the last instance of the shared memory is closed, all data is lost. If you start the application again, it gets a fresh zero-filled buffer. If want to be able to stop the program you need to use memory mapped files backed by real files on disk.

[user]

I did compile your program and I did fill the array. The instances of both program are still running. First I ran the both of program then I issued the "SET 1 2" command in first program. Second program can access the array by using "DUMP" but I can not show all items. I am trying OpenFileMapping instead of CreateFileMapping in second program.

[user]

Same result using OpenFileMapping.

[Cosmin Prund]

I don't understand what you're doing. I did not provide a command for dumping the whole array, so I assume you're doing something else (not using my program). After issuing set 1 2, issuing dump 1 should display the value 2. It should not matter if you do dump 1 from the same instance or from the other instance, the result should be the same.

[Remko]

@Cosmin Prund: thanks for the hint about zero init (I didn't know that). Regarding the handle, yes you are right (and this is for the reader as well).

[user]

I've modified the second program to add "DIR" command. else if Copy(cmd, 1, 3) = 'DIR' then begin for j := low(SharedArray^) to high(SharedArray^) do WriteLn(SharedArray^[j]); end;. All I got is lots of zeros.

[user]

Okay I can do a trick. set 1 3 3 is the size of array then I dump the whole array with for j := 2 to (SharedArray^[1])+1 do WriteLn(SharedArray^[j]); . I can do this because in my case I know the size of my array.

[Cosmin Prund]

I've edited my answer to streamline command processing and provide a DUMP ALL command to show the whole array. I reduced the size of the array to [0..10] to make DUMP ALL manageable (this is only a sample after all). I have no idea why your way doesn't work because I can't see your code, I can only see mine. And mine works. If possible copy-paste your modified code to a pastebin and I'll take a look at it.

[user]

I don't know if I am missing something. Your code works! Mine not pastebin.com/00m7WVqj

[Cosmin Prund]

@user, your code also works. Try this SET 1024 1024 followed by DIR. Since you kept the [0..1024] array, the last printed line will show 1024. (I actually copy-pasted, compiled and tested your code)

[user]

Yes thanks. But why 'High(SharedArray^)' is 1024? A memory block? Why I try SET 1025 1025 then DIR, there is nothing in SharedArray^[1025]? This is confusing me.

[user]

I mean the looping only until 1024

[Cosmin Prund]

Because SharedArray is of type array [0..1024] of Integer. There's no SharedArray[1025] and you'd get a range check error if you compiled with {$R+}. You can obviously define TSharedArray to be whatever you need it to be.

[user]

Ok. Actually I still have one question, I hope you give a direction. I need to use dynamic array in my program. So my array declaration is TSharedArray = array of Integer; and I don't forget to put SetLength(SharedArray^, 10); .It works for first program. I try to put and not to put SetLength(SharedArray^, 10); for second program but I still can't access the data from first program. pastebin.com/mfD3tak2

[Cosmin Prund]

Dynamic arrays are a totally different issue: They are heap-allocated, where the variable is you only have an pointer. The allocated memory is not allocated in the shared buffer, and even if it were, the allocated buffer isn't seen by all applications at the same address (so pointers established by first app aren't valid for the second app). You can work around the issue by designing your own data structure that doesn't use absolute pointers but relative pointers (indexes). Or simply allocate a large-enough array to start with.

[Warren P]

What is your CreateHiddenWindow doing?