Protocol buffers in C# projects using protobuf-net - best practices for code generation

Question

I'm trying to use protobuf in a C# project, using protobuf-net, and am wondering what is the best way to organise this into a Visual Studio project structure.

When manually using the protogen tool to generate code into C#, life seems easy but it doesn't feel right.

I'd like the .proto file to be considered to be the primary source-code file, generating C# files as a by-product, but before the C# compiler gets involved.

The options seem to be:

1. Custom tool for proto tools (although I can't see where to start there)
2. Pre-build step (calling protogen or a batch-file which does that)

I have struggled with 2) above as it keeps giving me "The system cannot find the file specified" unless I use absolute paths (and I don't like forcing projects to be explicitly located).

Is there a convention (yet) for this?

---

Edit: Based upon @jon's comments, I retried the pre-build step method and used this (protogen's location hardcoded for now), using Google's address-book example:

c:\bin\protobuf\protogen "-i:$(ProjectDir)AddressBook.proto" 
       "-o:$(ProjectDir)AddressBook.cs" -t:c:\bin\protobuf\csharp.xslt

---

Edit2: Taking @jon's recommendation to minimise build-time by not processing the .proto files if they haven't changed, I've knocked together a basic tool to check for me (this could probably be expanded to a full Custom-Build tool):

using System;
using System.Diagnostics;
using System.IO;

namespace PreBuildChecker
{
    public class Checker
    {
        static int Main(string[] args)
        {
            try
            {
                Check(args);
                return 0;
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                return 1;
            }
        }

        public static void Check(string[] args)
        {
            if (args.Length < 3)
            {
                throw new ArgumentException(
                    "Command line must be supplied with source, target and command-line [plus options]");
            }

            string source = args[0];
            string target = args[1];
            string executable = args[2];
            string arguments = args.Length > 3 ? GetCommandLine(args) : null;

            FileInfo targetFileInfo = new FileInfo(target);
            FileInfo sourceFileInfo = new FileInfo(source);
            if (!sourceFileInfo.Exists) 
            {
                throw new ArgumentException(string.Format(
                    "Source file {0} not found", source));
            }

            if (!targetFileInfo.Exists || 
                sourceFileInfo.LastWriteTimeUtc > targetFileInfo.LastAccessTimeUtc)
            {
                Process process = new Process();
                process.StartInfo.FileName = executable;
                process.StartInfo.Arguments = arguments;
                process.StartInfo.ErrorDialog = true;

                Console.WriteLine(string.Format(
                     "Source newer than target, launching tool: {0} {1}",
                     executable,
                     arguments));
                process.Start();
            }
        }

        private static string GetCommandLine(string[] args)
        {
            string[] arguments = new string[args.Length - 3];
            Array.Copy(args, 3, arguments, 0, arguments.Length);
            return String.Join(" ", arguments);
        }
    }
}

My pre-build command is now (all on one line):

$(SolutionDir)PreBuildChecker\$(OutDir)PreBuildChecker 
    $(ProjectDir)AddressBook.proto 
    $(ProjectDir)AddressBook.cs 
    c:\bin\protobuf\protogen 
      "-i:$(ProjectDir)AddressBook.proto" 
      "-o:$(ProjectDir)AddressBook.cs" 
      -t:c:\bin\protobuf\csharp.xslt

Solution 1

As an extension of Shaun's code, I am pleased to announce that protobuf-net now has Visual Studio integration by way of a Custom Tool. The msi installer is available from the project page. More complete information here: protobuf-net; now with added Orcas.

Solution 2

Calling a pre-build step but using project variables (e.g. $(ProjectPath)) to create absolute filenames without having them actually in your solution would seem a reasonable bet to me.

One thing you might want to consider, based on my past experience of code generators: you might want to write a wrapper for protogen which generates code to a different location, then checks whether the newly generated code is the same as the old code, and doesn't overwrite it if so. That way Visual Studio will realise nothing's changed and not force that project to be rebuilt - this has cut build times dramatically for me in the past.

Alternatively, you could keep an md5 hash of the .proto file the last time protogen was executed, and only execute protogen if the .proto file has changed - even less to do on each build!

Thanks for raising this as a question though - it clearly suggests I should work out a way to make this an easy pre-build step for my own port.

Solution 3

Add this to the relevant project file.

Advantage, incremental build.

Disadvantage, you need to edit manually when adding files.

<ItemGroup>
    <Proto Include="Person.proto" />
    <Compile Include="Person.cs">
        <DependentUpon>Person.proto</DependentUpon>
    </Compile>
</ItemGroup>
<PropertyGroup>
    <CompileDependsOn>ProtobufGenerate;$(CompileDependsOn)</CompileDependsOn>
</PropertyGroup>
<Target Name="ProtobufGenerate" Inputs="@(Proto)" Outputs="@(Proto->'$(ProjectDir)%(Filename).cs')">
    <ItemGroup>
        <_protoc Include="..\packages\Google.Protobuf.*\tools\protoc.exe" />
    </ItemGroup>
    <Error Condition="!Exists(@(_protoc))" Text="Could not find protoc.exe" />
    <Exec Command="&quot;@(_protoc)&quot; &quot;--csharp_out=$(ProjectDir.TrimEnd('\'))&quot; @(Proto->'%(Identity)',' ')" WorkingDirectory="$(ProjectDir)" />
</Target>

Solution 4

Add the following pre-build event to your project settings to generate the C# file only when the .proto file has changed. Just replace YourFile with the name of base name of your .proto file.

cd $(ProjectDir) && powershell -Command if (!(Test-Path YourFile.proto.cs) -or (Get-Item YourFile.proto).LastWriteTimeUtc -gt (Get-Item YourFile.proto.cs).LastWriteTimeUtc) { PathToProtoGen\protogen -i:YourFile.proto -o:YourFile.proto.cs }

This works in any recent version of Visual Studio, unlike the protobuf-net Custom-Build tool, which does not support Visual Studio 2012 or Visual Studio 2013, according to issues 338 and 413.

Solution 5

well, that gave me an idea (something about reinventing the wheel)...

• create simple Makefile.mak, something like

.SUFFIXES : .cs .proto

.proto.cs:
    protogen\protogen.exe -i:$? -o:$@ -t:protogen\csharp.xlst

(obviously, don't forget to replace paths to protogen and csharp.xlst). IMPORTANT - protogen\protogen.exe command starting from TAB character, not 8 spaces

• If you don't want to specify files needed to be build all the time, you might use something like

.SUFFIXES : .cs .proto

all: mycs1.cs myotherfile.cs

.proto.cs:
    protogen\protogen.exe -i:$? -o:$@ -t:protogen\csharp.xlst

• in pre-build step to add

cd $(ProjectDir) && "$(DevEnvDir)..\..\vc\bin\nmake" /NOLOGO -c -f Makefile.mak mycs1.cs myotherfile.cs

or, if you have nmake in your path, one can use

cd $(ProjectDir) && nmake /NOLOGO -c -f Makefile.mak mycs1.cs myotherfile.cs

View more solutions