Getting PartCover to work

Question

I want to try PartCover for code coverage. I'm running Visual Studio 2008 Professional with MSTest. The Professional Edition does not include the Team Testing tools, like Code Coverage.

So, I'm trying PartCover, but I can't get it to work. In the PartCover.Browser I've selected the MSTest executable, I've pointed the working arguments to my tests.dll, and I've tried pointing my Working Directory to the TestResults folder, but I get an error:

"Report is empty. Check settings and run target again."

I don't know what to try next.

Edit

It turns out I had two problems. First, I wasn't putting my Rules right. Second, I had spaces in my working arguments. The spaces were giving an error, but not showing up anywhere.

Solution 1

Yep, I had this problem too. Check out the format for the Rules field.

In the browser add something like:

+[MyNamespace.MyAssemblyName]*

Where the assembly name you specify is the name of the assembly containing the types you want coverage for. Start off with:

+[*]*

and partcover will happily give you coverage metrics for the unit test project, any libraries you reference and on and on.

From the command line you specify the same pattern in the --include argument: --include=[MyNamespace.MyAssembly]*

You can also exclude contained namespaces or types or restrict which types from within the namespace you get coverage data for in the report. The format for the rules is a subset of regular expression syntax according to the manual (consisting of asterix as a wildcard and characters that make up assembly and class names, so pretty limited but enough to get the data you want). Check out the section on rules in the manual. If you don't have the manual, download it from sourceforge.

Solution 2

I had to go through a number of steps to finally get PartCover working when calling it from a NAnt script. I collected everything I had to do here for others' convenience; note that some of this was already answered by others but I spent tons of time putting it all together.

First, as is answered elsewhere here, if your OS is 64-bit, you'll need to run [most recent Windows SDK]\bin\CorFlags.exe [PartCover install dir]\PartCover.exe /32BIT+ /Force

This is a one-time step, after PartCover install. It will change the executable, and warn you that the assembly will need to be re-signed, but I did not do that and it (eventually) worked fine. Note that even though it looks like CorFlags didn't do what you asked and warned you about signing, it did change the .exe, it just does not point that out explicitly.

Next, again if your OS is 64-bit, and you use NUnit (or another test exe) with PartCover, you will need to invoke a version explicitly compiled for x86. In NUnit's case, that would be nunit-console-x86.exe. Calling nunit-console.exe would just hang indefinitely for me after doing the work, and not return to a prompt.

Next, as is also answered elsewhere here, PartCover 2.3, a dev build, was failing silently even after running CorFlags on it. However, 2.2 worked.

Next, when PartCover.exe is invoked, the syntax for arguments is -- arg-name ... and NOT --=arg-name (i.e. dash dash space arg name, not dash dash equals arg name); the PartCover docs seem to go both ways but equal sign just did not work for me.

After the above, PartCover was finally working from the command line. I used a settings file (you can use the PartCover browser UI app to save a settings file, which you can then use from the command line), so that the only args I specified were the settings file full path, and the output report file name full path.

This still wasn't working when invoked from a NAnt script, so I finally realized that the arg values had to be quoted... and to use the HTML encoded tokens for quotes. Thus...

NAnt excerpt:

<property name="PartCoverExePath" value="c:\Program Files (x86)\PartCover .NET 2\PartCover.exe" />
<property name="PartCoverWorkPath" value="c:\Projects\MyProject\trunk\CI\" />
<property name="PartCoverSettingsFileName" value="PartCover.Settings.xml" />
<property name="PartCoverReportFileName" value="PartCover.Report.xml" />

<target name="MyTarget">
<exec program="${PartCoverExePath}">
<arg value="--settings &quot;${PartCoverWorkPath}${PartCoverSettingsFileName}&quot;" />
<arg value="--output &quot;${PartCoverWorkPath}${PartCoverReportFileName}&quot;" />
</exec>
</target>

And the PartCover settings file:

<PartCoverSettings>
<Target>C:\CI\Binaries\NUnit2.5.2\bin\net-2.0\nunit-console-x86.exe</Target>
<TargetWorkDir>c:\Projects\MyProject\trunk\MyProject.Test\bin\Debug</TargetWorkDir>
<TargetArgs>MyProject.Test.dll</TargetArgs>
<Rule>+[*]*</Rule>
<Rule>-[log4net*]*</Rule>
<Rule>-[nunit*]*</Rule>
<Rule>-[MyProject.Test*]*</Rule>
</PartCoverSettings>

Phew! Hopefully this will save someone else the headaches I had.

Solution 3

I had the same problem with the PartCover reports. So I have been trying to make it work right and I just discovered that the problem was the two XSLT files that come with the PartCover distribution.

I fixed these files and now everything is working fine for me:

report by assembly

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxml="urn:schemas-microsoft-com:xslt">
  <xsl:output method="html" indent="yes"/>
    <xsl:template match="/">

    <xsl:variable name="cov0style" select="'background:#E79090;text-align:right;'"/>
    <xsl:variable name="cov20style" select="'background:#D79797;text-align:right;'"/>
    <xsl:variable name="cov40style" select="'background:#D7A0A0;text-align:right;'"/>
    <xsl:variable name="cov60style" select="'background:#C7A7A7;text-align:right;'"/>
    <xsl:variable name="cov80style" select="'background:#C0B0B0;text-align:right;'"/>
    <xsl:variable name="cov100style" select="'background:#D7D7D7;text-align:right;'"/>

<table style="border-collapse: collapse;">
  <tr style="font-weight:bold; background:whitesmoke;">
    <td colspan="2">Coverage by assembly</td>
  </tr>

  <xsl:variable name="asms" select="/PartCoverReport/Assembly"/>
  <xsl:for-each select="$asms">
    <xsl:variable name="current-asm-node" select="."/>
    <tr>

      <xsl:element name="td">
        <xsl:attribute name="style">background:ghostwhite; padding: 5px  30px 5px  5px;</xsl:attribute>
        <xsl:value-of select="$current-asm-node/@name"/>
      </xsl:element>

      <xsl:variable name="codeSize" select="sum(/PartCoverReport/Type[@asmref=$current-asm-node/@id]/Method/pt/@len)+0"/>
      <xsl:variable name="coveredCodeSize" select="sum(/PartCoverReport/Type[@asmref=$current-asm-node/@id]/Method/pt[@visit>0]/@len)+0"/>

      <xsl:element name="td">
        <xsl:if test="$codeSize=0">
          <xsl:attribute name="style">
            <xsl:value-of select="$cov0style"/>
          </xsl:attribute>
          0%
        </xsl:if>
        <xsl:if test="$codeSize &gt; 0">
          <xsl:variable name="coverage" select="ceiling(100 * $coveredCodeSize div $codeSize)"/>
          <xsl:if test="$coverage &gt;=  0 and $coverage &lt; 20">
            <xsl:attribute name="style">
              <xsl:value-of select="$cov20style"/>
            </xsl:attribute>
          </xsl:if>
          <xsl:if test="$coverage &gt;= 20 and $coverage &lt; 40">
            <xsl:attribute name="style">
              <xsl:value-of select="$cov40style"/>
            </xsl:attribute>
          </xsl:if>
          <xsl:if test="$coverage &gt;= 40 and $coverage &lt; 60">
            <xsl:attribute name="style">
              <xsl:value-of select="$cov60style"/>
            </xsl:attribute>
          </xsl:if>
          <xsl:if test="$coverage &gt;= 60 and $coverage &lt; 80">
            <xsl:attribute name="style">
              <xsl:value-of select="$cov80style"/>
            </xsl:attribute>
          </xsl:if>
          <xsl:if test="$coverage &gt;= 80">
            <xsl:attribute name="style">
              <xsl:value-of select="$cov100style"/>
            </xsl:attribute>
          </xsl:if>
          <xsl:value-of select="$coverage"/>%
        </xsl:if>
      </xsl:element>
    </tr>
  </xsl:for-each>
</table>

report by class

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxml="urn:schemas-microsoft-com:xslt">
<xsl:output method="html" indent="no"/>

<xsl:template match="/">

<xsl:variable name="cov0style" select="'background:#FF4040;text-align:right;'"/>
<xsl:variable name="cov20style" select="'background:#F06060;text-align:right;'"/>
<xsl:variable name="cov40style" select="'background:#E78080;text-align:right;'"/>
<xsl:variable name="cov60style" select="'background:#E0A0A0;text-align:right;'"/>
<xsl:variable name="cov80style" select="'background:#D7B0B0;text-align:right;'"/>
<xsl:variable name="cov100style" select="'background:#E0E0E0;text-align:right;'"/>

<table style="border-collapse: collapse;">
    <tr style="font-weight:bold; background:whitesmoke;"><td colspan="2">Coverage by class</td></tr>

    <xsl:for-each select="/PartCoverReport/Type">
        <tr>

            <xsl:element name="td">
                <xsl:attribute name="style">background:ghostwhite; padding: 5px  30px 5px  5px;</xsl:attribute>
                <xsl:value-of select="@name"/>
            </xsl:element>

            <xsl:variable name="codeSize" select="sum(./Method/pt/@len)+0"/>
            <xsl:variable name="coveredCodeSize" select="sum(./Method/pt[@visit>0]/@len)+0"/>

            <xsl:element name="td">
                <xsl:if test="$codeSize=0">
                    <xsl:attribute name="style"><xsl:value-of select="$cov0style"/></xsl:attribute>
                    0%
                </xsl:if>

                <xsl:if test="$codeSize &gt; 0">
                    <xsl:variable name="coverage" select="ceiling(100 * $coveredCodeSize div $codeSize)"/>

                    <xsl:if test="$coverage &gt;=  0 and $coverage &lt; 20"><xsl:attribute name="style"><xsl:value-of select="$cov20style"/></xsl:attribute></xsl:if>
                    <xsl:if test="$coverage &gt;= 20 and $coverage &lt; 40"><xsl:attribute name="style"><xsl:value-of select="$cov40style"/></xsl:attribute></xsl:if>
                    <xsl:if test="$coverage &gt;= 40 and $coverage &lt; 60"><xsl:attribute name="style"><xsl:value-of select="$cov60style"/></xsl:attribute></xsl:if>
                    <xsl:if test="$coverage &gt;= 60 and $coverage &lt; 80"><xsl:attribute name="style"><xsl:value-of select="$cov80style"/></xsl:attribute></xsl:if>
                    <xsl:if test="$coverage &gt;= 80"><xsl:attribute name="style"><xsl:value-of select="$cov100style"/></xsl:attribute></xsl:if>
                    <xsl:value-of select="$coverage"/>%
                </xsl:if>

            </xsl:element>
        </tr>
    </xsl:for-each>
</table>    
</xsl:template>
</xsl:stylesheet>

I hope you find this useful. Also, any feedback about this files is welcomed, so we can provide the commutiy with correct files. See this related question

Solution 4

@pelazm - Thanks for some excellent guidance.

Two minor things to add to your solution:

(a) If you don't want to an external PartCover.settings.xml

<!-- Runs unit tests through PartCover to calculate unit test covereage-->
<!-- Use %2a instead of * and %3f instead of ? to prevent expansion -->
<!-- %40 = @  %25 = % %24 = $ -->
<Target Name="RunTests">
  <ItemGroup>
     <pc4_settings Include="--target &quot;$(NUnitEXE)&quot;"/>
     <pc4_settings Include="--target-work-dir &quot;$(RootDirectory)\src&quot;"/>
     <pc4_settings Include="--include [%2a]%2a"/>
     <pc4_settings Include="--exclude [nunit%2a]%2a"/>
     <pc4_settings Include="--exclude [log4net%2a]%2a"/>
     <pc4_settings Include="--exclude [MetadataProcessor.Tests%2a]%2a"/>
   </ItemGroup>

   <CreateItem Include="$(RootDirectory)\src\**\bin\$(Configuration)\*.Tests.dll">
     <Output TaskParameter="Include" ItemName="TestAssemblies" />
   </CreateItem>

   <Exec Command="&quot;$(PartCover4Directory)\PartCover.exe&quot; --register    @(pc4_settings,' ') --target-args &quot;%(TestAssemblies.Identity) $(NUnitArgs) /xml:%(TestAssemblies.Identity).NUnitResults.xml&quot; --output $(BuildDirectory)\PartCover-results.xml"
  ContinueOnError="true"
  WorkingDirectory="$(BuildDirectory)">
     <Output TaskParameter="ExitCode" ItemName="ExitCodes"/>
   </Exec>

   <XslTransformation XslInputPath="$(RootDirectory)\tools\partcover4\xslt\PartCoverFullReport.xslt"
                  XmlInputPaths="$(BuildDirectory)\PartCover-results.xml"
                  OutputPaths="$(BuildDirectory)\PartCover-results-PartCoverFullReport.html" />

   <Error Text="Test error occurred" Condition="'%(ExitCodes.Identity)'>0"/>
 </Target>

(b) Gáspár Nagy's HTML report is pretty good - http://gasparnagy.blogspot.com/2010/09/detailed-report-for-partcover-in.html

View more solutions

Comments

[Lucas B]

Look here for a great walk-through, though it is for NUnit not MSTest: csharpcity.com/using-partcover-and-nunit-for-code-coverage

[Damien]

PartCover is far too cumbersome to set up. It may be free but it's a giant headache compared nCover - particularly in Testdriven .NET.

[JBRWilkinson]

I'd dispute that - PartCover.exe --target YourProgram.exe is pretty easy to do.

[RichardOD]

@Damien- hardly a major headache. I just found Hamish answer to the bit I got stuck on and 10 seconds later it is working with NUnit.

[Nate Cook3]

+1! I was going batty trying to set it up via the command line. Using the gui and exporting the settings was a brilliant idea! Thanks!

[Chris Nicola]

Thanks for these, though the percentages don't match what I get in the browser so I don't think they work quite right.

[Shaun Wilde]

Please note that the latest version of partcover that has .NET4 support is now hosted here github.com/sawilde/partcover.net4