String to Byte [delphi]

Question

I need to store my data into memory. My type data of my data is string. I want to minimize the memory usage. I guess I have to change string into byte. Am I right? If I convert string to byte, that means I have to convert string to TMemoryStream?

Solution 1

If you really want to convert it then this code will get it done

var
  BinarySize: Integer;
  InputString: string;
  StringAsBytes: array of Byte;
begin
  BinarySize := (Length(InputString) + 1) * SizeOf(Char);
  SetLength(StringAsBytes, BinarySize);
  Move(InputString[1], StringAsBytes[0], BinarySize);

But as already stated this will not save you memory. The ammount of it used will be practically the same. You will gain nothing from this alone. If you are having to many strings take a different approach. Like something from this list of choices:

1. Use a dictionary and only store each same string once
2. Only hold a portion of all strings in memory. Some sort of cache. Have others on hard drive and use streams to load them
3. If you have very large string consider compressing them.
4. If you are reading from file and you target is binary data, skip the string in the middle. Read the source directly into a byte buffer.

It is hard to give further help without knowing more about the problem.

EDIT:

If you really want a minimum memory footprint and you can live with a little lower speed (but still very fast) you can use Suffix Trie or B-Tree or event a simple Binary Tree. They can work directly from hard drive and can be very fast for searching. If you then cache a subset of the data to RAM, you get the optimal solution memory vs. speed wise.

Anyway given the ammount of data you claim to have it seems no memory optimization is needed at all. 22MB of RAM is hardly an issue and not worth optimizing.

Solution 2

Are you certain this is an optimization that is needed?

2000 lines that are 10 characters long is only 20000 characters.

In most environments, that's tiny. Most machines have considerably more RAM than that. Most disks are considerably larger than that. And, usually, sending and receiving that much information is trivial over the web.

Perhaps your situation is unique. Maybe you have large number of 20000 character data sets, or very slow web access over which to transmit this date, etc. But, I'd encourage you to consider whether you aren't perhaps trying to optimize something that even if you are very successful in implementing, won't significantly change your application's performance in the real world.