Delphi 2010: How to save a whole record to a file?

Question

I have defined a record which has lots of fields with different types (integer, real , string, ... plus dynamic arrays in terms of "array of ..."). I want to save it as a whole to a file and then be able to load it back to my program. I don't want to go through saving each field's value individually. The file type (binary or ascii or ...) is not important as long Delphi could read it back to a record.

Do you have any suggestions?

Solution 1

You can load and save the memory of a record directly to and from a stream, as long as you don't use dynamic arrays. So if you use strings, you need to make them fixed:

type TTestRecord = record 
  FMyString : string[20]; 
end; 

var 
  rTestRecord: TTestRecord;
  strm : TMemoryStream; 

strm.Write(rTestRecord, Sizeof(TTestRecord) );

You can even load or save an array of record at once!

type TRecordArray = array of TTestRecord;

var ra : TRecordArray; 

strm.Write(ra[0], SizeOf(TTestRecord) * Length(ra));

In case you want to write dynamic content:

iCount   := Length(aArray);
strm.Write(iCount, Sizeof(iCount) );      //first write our length
strm.Write(aArray[0], SizeOf * iCount);   //then write content

After that, you can read it back:

strm.Read(iCount, Sizeof(iCount) );       //first read the length
SetLength(aArray, iCount);                //then alloc mem
strm.Read(aArray[0], SizeOf * iCount);    //then read content

Solution 2

As promised here it is: https://github.com/KrystianBigaj/kblib

When you defined for example record as:

TTestRecord = record
  I: Integer;
  D: Double;
  U: UnicodeString;
  W: WideString;
  A: AnsiString;
  Options: TKBDynamicOptions;

  IA: array[0..2] of Integer;

  AI: TIntegerDynArray;
  AD: TDoubleDynArray;
  AU: array of UnicodeString;
  AW: TWideStringDynArray;
  AA: array of AnsiString;

  R: array of TTestRecord; // record contain dynamic array of itself (D2009+)
end;

You can save whole dynamic record to stream (as binary data) by :

TKBDynamic.WriteTo(lStream, lTestRecord, TypeInfo(TTestRecord));

To load it back:

TKBDynamic.ReadFrom(lStream, lTestRecord, TypeInfo(TTestRecord));

It not need to be a record, you can do same for any dynamic type like:

TKBDynamic.WriteTo(lStream, lStr, TypeInfo(UnicodeString));
TKBDynamic.WriteTo(lStream, lInts, TypeInfo(TIntegerDynArray));
TKBDynamic.WriteTo(lStream, lArrayOfTestRecord, TypeInfo(TArrayOfTestRecord)); // TArrayOfTestRecord = array of TTestRecord;

Tested on Delphi 2006/2009/XE. License: MPL 1.1/GPL 2.0/LGPL 3.0 See readme for information.

Solution 3

In addition to the answers that indicate how you do this, please also be aware of these:

1. You must be aware that writing records out to a file will be Delphi version specific (usually: specific to a series of Delphi versions that share the same memory layout for the underlying data types).

2. You can only do that if your record does not contain fields of a managed type. Which means that fields cannot be of these managed types: strings, dynamic arrays, variants, and reference types (like pointers, procedural types, method references, interfaces or classes) and file types, or types that contain those manages types. Which basically limits to to these unmanaged types:

   • A: Simple types (including bytes, integers, floats, enumerations, chars and such)
   • B: Short strings
   • C: Sets
   • D: Static arrays of A, B, C, D and E
   • E: Records of A, B, C, D and E

In stead of writing out records to/from a file, it might be better to go with class instances and convert them to/from JSON, and them write the JSON string equivalent to a file and read it back in.

You can use this unit to do the JSON conversion for you (should work with Delphi 2010 and up; works for sure with Delphi XE and up) from this location this location.

unit BaseObject;

interface

uses DBXJSON, DBXJSONReflect;

type
  TBaseObject = class
  public
    { public declarations }
    class function ObjectToJSON<T : class>(myObject: T): TJSONValue;
    class function JSONToObject<T : class>(json: TJSONValue): T;
  end;

implementation

{ TBaseObject }

class function TBaseObject.JSONToObject<T>(json: TJSONValue): T;
var
  unm: TJSONUnMarshal;
begin
  if json is TJSONNull then
    exit(nil);
  unm := TJSONUnMarshal.Create;
  try
    exit(T(unm.Unmarshal(json)))
  finally
    unm.Free;
  end;

end;

class function TBaseObject.ObjectToJSON<T>(myObject: T): TJSONValue;
var
  m: TJSONMarshal;
begin

  if Assigned(myObject) then
  begin
    m := TJSONMarshal.Create(TJSONConverter.Create);
    try
      exit(m.Marshal(myObject));
    finally
      m.Free;
    end;
  end
  else
    exit(TJSONNull.Create);

end;

end.

I hope this helps you getting an overview of things.

--jeroen

Solution 4

Another option which works very well for records (Delphi 2010+) is to use the SuperObject library. For example:

type
  TData = record
    str: string;
    int: Integer;
    bool: Boolean;
    flt: Double;
  end;
var
  ctx: TSuperRttiContext;
  data: TData;
  obj: ISuperObject;
  sValue : string;
begin
  ctx := TSuperRttiContext.Create;
  try
    sValue := '{str: "foo", int: 123, bool: true, flt: 1.23}';
    data := ctx.AsType<TData>(SO(sValue));
    obj := ctx.AsJson<TData>(data);
    sValue := Obj.AsJson;
  finally
    ctx.Free;
  end;
end;

I also tested this briefly with a simple TArray<Integer> dynamic array and it did not have a problem storing and loading the array elements.

Solution 5

You could also define an object instead of a record, so you can use RTTI to save your object to XML or whatever. If you have D2010 or XE, you can use DeHL to serialize it: Delphi 2010 DeHL Serialization XML and custom attribute : how it work?

But if you "google" you can find other libs with RTTI and serialization (with D2007 etc)

View more solutions

Comments

[Mahm00d]

Good solution, but sadly I have dynamic arrays, too. What can I do? Isn't there any way that I could save the "current" state of the record to the stream?

[Mahm00d]

As I mentioned above: I have dynamic arrays in my record. That doesn't work with "file of". Any other solutions?

[Krystian Bigaj]

There is enough RTTI info for records/dyn.arrays/strings to save ANY dynamic record/array/strings/etc. with a one function. Tested in D2009. Records that contains for example dynamic array of dynamic record ... is not a problem as well. Analyze System.pas:_FinalizeArray

[Krystian Bigaj]

"... solution doesn't exist for Delphi ..." but you can make yourself one, and not only for D2010, but for any Delphi (win-native) version (didn't tried .NET). Read my comments from this 'topic'

[Cosmin Prund]

@kibab, it looks like you're trying to sell something, but you forgot to provide the link to your 1k-wander; And when you quote me, would you mind quoting the whole phrase, not just the 5 words that suite your purpose? The whole phrase is: ` Even those a Embarcadero-provided solution doesn't exist for Delphi, one can be implemented using the extended RTTI available in Delphi 2010 `. And did you noticed I posted a link to DeHL, right? And you do actually know that not everything gets RTTI, not even with Delphi 2010? Example: There's no RTTI for the elements in an enumeration type.

[Mahm00d]

+1 for what seems like a nice solution (although as a noob, it is a little hard to understand!). But I have 3 questions: 1) How do you make the sValue from a big existing record to pass it to "SO()"? 2) What is the object here to be stored to the file? The sValue or the obj? 3) How do you load it back to the record? Thanks again for your answer.

[skamradt]

The sValue is a representation of the record as a JSON packet. You could easily save this single string to a text file. In the above example the two lines "Obj := ctx.AsJson<TData>(data)" and "sValue := Obj.AsJson" are what perform the magic translation from record to string. Store the sValue. The previous line "data := ctx.AsType<tData>(so(sValue));" is what parses this and populates the record Data with the proper values.

[Arnaud Bouchez]

+1 This works very well. Code could be made compatible with upcoming 64 bit compiler, by using NativeUInt instead of cardinal for pointer arithmetic.

[Mahm00d]

+1, It seems like a good solution. I will definitely try it out. Thanks a lot for mentioning it.

[Mahm00d]

Is it part of SQLite or can it be added as an independent unit to the project? Can it Save/Load to the file?

[Arnaud Bouchez]

@Flom It's used by our SQLite-using ORM, but it's not part of it. It can be used to save/load to a file, even after compression if necessary. The SynCommons.pas unit is self contained (in fact, it does link to Synopse.inc and SynLZ.pas) and do not require SQLite or any other part of our libraries. And you've some nice other low-level features in this unit, like UTF-8 encoding, JSON serialization, and enhanced logging - with stack trace and exception tracing. Enjoy the Open Source!

[Krystian Bigaj]

@Arnaud Bouchez - x64 compiler support added (see readme for details about binary data compatibility between x86 and x64)

[TLama]

Nice! Thanks also for project maintenance.

[Server Overflow]

The record must be packed to be saved this way!

[Ingo]

Perfect -> uses uKBDynamic;