Delphi 2009, Indy 10, TIdTCPServer.OnExecute, how to grab all the bytes in the InputBuffer

Question

I am messing around with the Indy 10 supplied with Delphi 2009 and am having trouble with getting all the data from the IOHandler when OnExecute fires...

procedure TFormMain.IdTCPServerExecute(AContext: TIdContext);
var
  RxBufStr: UTF8String;
  RxBufSize: Integer;
begin

  if AContext.Connection.IOHandler.Readable then
  begin
    RxBufSize := AContext.Connection.IOHandler.InputBuffer.Size;
    if RxBufSize > 0 then
    begin
      SetLength(RxBufStr, RxBufSize);
      AContext.Connection.IOHandler.ReadBytes(TBytes(RxBufStr), RxBufSize, False);
    end;
  end;

end;

AContext.Connection.IOHandler.InputBuffer.Size doesn't seem reliable and often returns 0, but on the next run throug the OnExecute it'll pick up the right number of bytes, but that is too late.

Essentially I want to be able to just grab all the data, stuff it into a UTF8String (not a Unicode string) and then parse for a special marker. So I have no headers and messages are variable length. It seems the Indy 10 IOHandlers are not setup for this or I am just using it wrong.

It would be nice to do something like pass a buffer of a certain size, fill it as much as possible and return the number of bytes actually filled and then keep going if there are more.

As an aside what's the status of TIdSchedulerOfFiber, this looks very interesting, does it work? Is anyone using it? I notice that it isn't in the standard install of Delphi 2009 though.

Update: I found Msg := AContext.Connection.IOHandler.ReadLn(#0, enUTF8); which works but I still would like to know the answer to the above question, is it because it is based on blocking IO? Which makes even more keen on this TIdSchedulerOfFiber.

Solution 1

You should not be using Readable() like that. Try the following instead:

procedure TFormMain.IdTCPServerExecute(AContext: TIdContext);
var
  RxBuf: TIdBytes;
begin
  RxBuf := nil;
  with AContext.Connection.IOHandler do
  begin
    CheckForDataOnSource(10);
    if not InputBufferIsEmpty then
    begin
      InputBuffer.ExtractToBytes(RxBuf);
      // process RxBuf as needed...
    end;
  end;
end;

Alternatively:

procedure TFormMain.IdTCPServerExecute(AContext: TIdContext);
var
  RxBufStr: String; // not UTF8String
begin
  with AContext.Connection.IOHandler do
  begin
    CheckForDataOnSource(10);
    if not InputBufferIsEmpty then
    begin
      RxBufStr := InputBuffer.Extract(-1, enUtf8);

      // Alternatively to above, you can set the
      // InputBuffer.Encoding property to enUtf8
      // beforehand, and then call TIdBuffer.Extract()
      // without any parameters.
      //
      // Or, set the IOHandler.DefStringEncoding
      // property to enUtf8 beforehand, and then
      // call TIdIOHandler.InputBufferAsString()

      // process RxBufStr as needed...
    end;
  end;
end;

As for TIdSchedulerOfFiber - the SuperCore package is effectively dead at this time. It has not been worked on in a very long time, and is not up-to-date with the latest Indy 10 architecture. We may try to resurrect it at a later date, but it is not in our plans for the near future.

Solution 2

procedure TFormMain.IdTCPServerExecute(AContext: TIdContext); 
var
  RxBufStr: UTF8String;
  RxBufSize: Integer;
begin    
  if AContext.Connection.IOHandler.Readable then
  begin     
    AContext.Connection.IOHandler.ReadBytes(TBytes(RxBufStr),-1, False);
  end;
end; 

Comments

[Bruce]

It seems with the code presented above that it runs the OnExecute loop all the time, I assume that is by design and normally a user would want a blocking Readln; or similar? Also it's a pitty TIdSchedulerOfFiber isn't being worked on, was there a show stopper with regards development of it?

[Bruce]

Oh I see CheckForDataOnSource(10); has a timeout :-)

[Bruce]

I have found that using similar code to above that the AContext.Connection.IOHandler.CheckForDataOnSource(10) returns false when I can see in the debugger that the InputBuffer as data, is this a bug or is there some other setting I should use?

[Remy Lebeau]

Yes, the OnExecute event is automatically looped by TIdTCPServer, by design. The timeout parameter of CheckForDataOnSource() helps prevent the calling thread from eating up all of the CPU cycles during idle periods when no data is available yet.

[Remy Lebeau]

CheckForDataOnSource() does not look to see if the InputBuffer already has data in it. It goes right to the socket. If it is returning False, then the socket has no pending data in it.

[Remy Lebeau]

The best way to use CheckForDataOnSource() is to call it only when the InputBuffer is empty, not on every loop iterator unconditionally.

[Martijn Pieters]

Rather than only post a block of code, please explain why this code solves the problem posed. Without an explanation, this is not an answer.

[Ben]

Just a random question, but do you need SetLength (RxBuf, 0) to free if after the Execute Event?

[Remy Lebeau]

@BenjaminWeiss: RxBuf is a compiler-managed dynamic array, it will be freed automatically when it goes out of scope.