Determining outputs of a ProjectReference in MSBuild without triggering redundant rebuilds

Question

As part of a solution containing many projects, I have a project that references (via a <ProjectReference> three other projects in the solution, plus some others). In the AfterBuild, I need to copy the outputs of 3 specific dependent projects to another location.

Via various SO answers, etc. the way I settled on to accomplish that was:

    <MSBuild 
        Projects="@(ProjectReference)" 
        Targets="Build" 
        BuildInParallel="true" 
        Condition="'%(Name)'=='ProjectA' OR '%(Name)'=='ProjectB' OR '%(Name)'=='ProjectC'">
        <Output TaskParameter="TargetOutputs" ItemName="DependentAssemblies" />
    </MSBuild>
    <Copy SourceFiles="@(DependentAssemblies)" DestinationFolder="XX" SkipUnchangedFiles="true" />

However, I ran into problems with this. The <MSBuild step's IncrementalClean task ends up deleting a number of the outputs of ProjectC. When running this under VS2008, a build.force file being deposited in the obj/Debug folder of ProjectC which then triggers ProjectC getting rebuilt if I do a Build on the entire solution if the project containing this AfterBuild target, whereas if one excludes this project from the build, it [correctly] doesn't trigger a rebuild of ProjectC (and critically a rebuild of all dependents of ProjectC). This may be VS-specific trickery in this case which would not occur in the context of a TeamBuild or other commandline MSBuild invocation (but the most common usage will be via VS so I need to resolve this either way)

The dependent projects (and the rest of the solution in general) have all been created interactively with VS, and hence the ProjectRefences contain relative paths etc. I've seen mention of this being likely to causing issues - but without a full explanation of why, or when it'll be fixed or how to work around it. In other words, I'm not really interested in e.g. converting the ProjectReference paths to absolute paths by hand-editing the .csproj.

While it's entirely possible I'm doing something stupid and someone will immediately point out what it is (which would be great), be assured I've spent lots of time poring over /v:diag outputs etc. (although I havent tried to build a repro from the ground up - this is in the context of a relatively complex overall build)

Solution 1

As noted in my comment, calling GetTargetPath on the referenced project only returns the Primary output assembly of that project. To get all the referenced copy-local assemblies of the referenced project it's a bit messier.

Add the following to each project that you are referencing that you want to get the CopyLocals of:

    <Target
    Name="ComputeCopyLocalAssemblies"
    DependsOnTargets="ResolveProjectReferences;ResolveAssemblyReferences"
    Returns="@(ReferenceCopyLocalPaths)" /> 

My particular situation is that I needed to recreate the Pipeline folder structure for System.AddIn in the bin folder of my top level Host project. This is kinda messy and I was not happy with the MSDN suggested solutions of mucking with the OutputPath - as that breaks on our build server and prevents creating the folder structure in a different project (eg a SystemTest)

So along with adding the above target (using a .targets import), I added the following to a .targets file imported by each "host" that needs the pipeline folder created:

<Target
    Name="ComputePipelineAssemblies"
    BeforeTargets="_CopyFilesMarkedCopyLocal"
    Outputs="%(ProjectReference.Identity)">

    <ItemGroup>
        <_PrimaryAssembly Remove="@(_PrimaryAssembly)" />
        <_DependentAssemblies Remove="@(_DependentAssemblies)" />
    </ItemGroup>

    <!--The Primary Output of the Pipeline project-->
    <MSBuild Projects="%(ProjectReference.Identity)"
             Targets="GetTargetPath"
             Properties="Configuration=$(Configuration)"
             Condition=" '%(ProjectReference.PipelineFolder)' != '' ">
        <Output TaskParameter="TargetOutputs"
                ItemName="_PrimaryAssembly" />
    </MSBuild>

    <!--Output of any Referenced Projects-->
    <MSBuild Projects="%(ProjectReference.Identity)"
             Targets="ComputeCopyLocalAssemblies"
             Properties="Configuration=$(Configuration)"
             Condition=" '%(ProjectReference.PipelineFolder)' != '' ">
        <Output TaskParameter="TargetOutputs"
                ItemName="_DependentAssemblies" />
    </MSBuild>

    <ItemGroup>
        <ReferenceCopyLocalPaths Include="@(_PrimaryAssembly)"
                                 Condition=" '%(ProjectReference.PipelineFolder)' != '' ">
            <DestinationSubDirectory>%(ProjectReference.PipelineFolder)</DestinationSubDirectory>
        </ReferenceCopyLocalPaths>
        <ReferenceCopyLocalPaths Include="@(_DependentAssemblies)"
                                 Condition=" '%(ProjectReference.PipelineFolder)' != '' ">
            <DestinationSubDirectory>%(ProjectReference.PipelineFolder)</DestinationSubDirectory>
        </ReferenceCopyLocalPaths>
    </ItemGroup>
</Target>

I also needed to add the required PipelineFolder meta data to the actual project references. For example:

    <ProjectReference Include="..\Dogs.Pipeline.AddInSideAdapter\Dogs.Pipeline.AddInSideAdapter.csproj">
        <Project>{FFCD0BFC-5A7B-4E13-9E1B-8D01E86975EA}</Project>
        <Name>Dogs.Pipeline.AddInSideAdapter</Name>
        <Private>False</Private>
        <PipelineFolder>Pipeline\AddInSideAdapter\</PipelineFolder>
    </ProjectReference>

Solution 2

Your original solution should work simply by changing

Targets="Build"

to

Targets="GetTargetPath"

The GetTargetPath target simply returns the TargetPath property and doesn't require building.

Solution 3

You may protect your files in ProjectC if you call a target like this first:

  <Target Name="ProtectFiles">
    <ReadLinesFromFile File="obj\ProjectC.csproj.FileListAbsolute.txt">
        <Output TaskParameter="Lines" ItemName="_FileList"/>
    </ReadLinesFromFile>
    <CreateItem Include="@(_DllFileList)" Exclude="File1.sample; File2.sample">
        <Output TaskParameter="Include" ItemName="_FileListWitoutProtectedFiles"/>
    </CreateItem>      
      <WriteLinesToFile 
        File="obj\ProjectC.csproj.FileListAbsolute.txt"
        Lines="@(_FileListWitoutProtectedFiles)"
        Overwrite="true"/>
  </Target>

Solution 4

My current workaround is based on this SO question, i.e, I have:

    <ItemGroup>
        <DependentAssemblies Include="
            ..\ProjectA\bin\$(Configuration)\ProjectA.dll;
            ..\ProjectB\bin\$(Configuration)\ProjectB.dll;
            ..\ProjectC\bin\$(Configuration)\ProjectC.dll">
        </DependentAssemblies>
    </ItemGroup>

This however will break under TeamBuild (where all the outputs end up in one directory), and also if the names of any of the outputs of the dependent projects change.

EDIT: Also looking for any comments on whether there's a cleaner answer for how to make the hardcoding slightly cleaner than:

    <PropertyGroup>
        <_TeamBuildingToSingleOutDir Condition="'$(TeamBuildOutDir)'!='' AND '$(CustomizableOutDir)'!='true'">true</_TeamBuildingToSingleOutDir>
    </PropertyGroup>

and:

    <ItemGroup>
        <DependentAssemblies 
            Condition="'$(_TeamBuildingToSingleOutDir)'!='true'"
            Include="
                ..\ProjectA\bin\$(Configuration)\ProjectA.dll;
                ..\ProjectB\bin\$(Configuration)\ProjectB.dll;
                ..\ProjectC\bin\$(Configuration)\ProjectC.dll">
        </DependentAssemblies>
        <DependentAssemblies 
            Condition="'$(_TeamBuildingToSingleOutDir)'=='true'"
            Include="
                $(OutDir)\ProjectA.dll;
                $(OutDir)\ProjectB.dll;
                $(OutDir)\ProjectC.dll">
        </DependentAssemblies>
    </ItemGroup>

View more solutions