Best practices with Nuget: Debug or Release?

Question

Currently, I package the release builds with Nuget for the official builds to nuget.org, but I package the debug builds with Nuget for the symbol source pushes to symbolsource.org.

EDIT: (Jon Skeet, with some bias from Noda Time development)

NuGet now supports pushing to both NuGet gallery and symbolsource.org (or similar servers), as documented. Unfortunately, there are two contradictory requirements here:

• When just using a library without any need for debugging, you really want a release build. That's what release builds are for, after all.
• When debugging into a library for diagnostic purposes, you really want a debug build with all the appropriate optimizations disabled. That's what debug builds are for, after all.

That would be fine, but NuGet doesn't (as far as I can tell) allow both the release and debug builds to be published in a useful way, in the same package.

So, the choices are:

• Distribute the debug builds to everyone (as shown in the example in the docs) and live with any size and performance hits.
• Distribute the release builds to everyone and live with a slightly impaired debug experience.
• Go for a really complicated distribution policy, potentially providing separate release and debug packages.

The first two really boil down to the effect of the differences between debug and release builds... although it's worth noting that there's also a big difference between wanting to step into the code of a library because you want to check some behaviour, and wanting to debug the code of a library because you believe you've found a bug. In the second case, it's probably better to get the code of the library as a Visual Studio solution and debug that way, so I'm not paying too much heed to that situation.

My temptation is to just keep with the release builds, with the expectation that relatively few people will need to debug, and the ones who do won't be impacted much by the optimizations in the release build. (The JIT compiler does most of the optimizing anyway.)

So, are there other options we hadn't considered? Are there other considerations which tip the balance? Is pushing NuGet packages to SymbolSource sufficiently new that "best practice" really hasn't been established?

Solution 1

It's been 8 years since OP's post (with previous answer still top-voted below), so I'll give it a crack with what's used nowadays.

There are 2 ways of "stepping into" a NuGet package:

1. Distribution of PDBs

.symbols.nupkg symbol packages are considered as legacy and have been replaced with .snupkg packages that get published to Symbol Server. It’s supported by most vendors with Azure DevOps being the biggest outsider where the feature request is still "under review" (thank you @alv for the link).

Here are the official instructions. Simply add these two lines to the csproj file:

<PropertyGroup>
  <IncludeSymbols>true</IncludeSymbols>
  <SymbolPackageFormat>snupkg</SymbolPackageFormat>
</PropertyGroup>

On the consumer side, make sure that your IDE is configured for the NuGet.org (or Azure, etc.) Symbol Server to allow stepping into package code when debugging.

2. Source Link.Linking the actual code

In some cases, the PDBs may hide some specifics of the source code due to MSIL/JIT optimisation. So there is a way of ”Stepping Into” the actual source of your NuGet while debugging. It’s called Source Link from the .NET Foundation – ”a language- and source-control agnostic system for providing source debugging experiences for binaries“.

It’s supported by Visual Studio 15.3+ and all the major vendors (also supports private repos).

It’s trivial to setup (official docs) – just add a development dependency to the project file (depends on the type of your repo) along with some flags:

<PropertyGroup>
    <PublishRepositoryUrl>true</PublishRepositoryUrl>
    <EmbedUntrackedSources>true</EmbedUntrackedSources>
</PropertyGroup>
<ItemGroup>
  <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0" PrivateAssets="All" />
</ItemGroup>

Check out more on this subject in "5 steps to better NuGet package".

Solution 2

Speaking for SymbolSource, I believe that the best practice is to:

1. Push release binary+content packages to nuget.org only (or any other production feed)
2. Push debug binary+content packages to a development feed:
   • on-premise
   • on myget.org
   • on nuget.org as pre-release packages.
3. Push both release and debug binary+symbols packages to symbolsource.org or any other symbol store.

While we're at it, it is a common misconception that release and debug builds in .NET really differ much, but I am assuming that the differentiation is here because of various code that might or might not be included in either build, like Debug.Asserts.

That said, it is really worth pushing both configurations to SymbolSource, because you just never know when you're going to need to debug production code. Remotely in production to make it harder. You're going to need the help you can get from your tooling when that happens. Which I obviously do not wish upon anyone.

There is still a matter to consider regarding versioning: is it correct to have 2 different packages (build in debug and in release) sharing 1 version number? SymbolSource would accept that, because it extracts packages and stores binaries in separate build mode branches, IF ONLY NuGet allowed to tag packages accordingly. There is no way at present to determine if a package is debug or release-mode.

Solution 3

I completely agree with your conclusion. NuGet packages with RELEASE and SymbolSource with debug. It seems pretty rare to step directly into packages and the occasional debug misstep with optimizations enabled might be acceptable.

If there were really a problem, I think the ideal solution would be to have NuGet support it. For example, imagine if when debugging, it could replace the release DLL with the one included in the SymbolSource package.

Ideally, what would happen then is that nuget pack SomePackage -Symbols against a release version would create a release nuget package, but a debug symbols package. And the VS plugin would be updated to be smart enough to see the association and pull in the Debug assemblies when running in a debugger and load those instead. Kind of crazy, but would be interesting.

However, I just don't see enough people complaining about this that it'd be worth it at the moment.

NuGet team accepts pull requests. :)

Solution 4

The example in the Creating and Publishing A Symbol Package references files in the Debug directories as sources for the dll and pdb files.

Specifying Symbol Package Contents

A symbol package can be built by conventions, from a folder structured in the way described in the previous section, or its contents can be specified using the files section. If you wanted to build the example package described previously, you could put this into your nuspec file:

<files>
  <file src="Full\bin\Debug\*.dll" target="lib\net40" /> 
  <file src="Full\bin\Debug\*.pdb" target="lib\net40" /> 
  <file src="Silverlight\bin\Debug\*.dll" target="lib\sl40" /> 
  <file src="Silverlight\bin\Debug\*.pdb" target="lib\sl40" /> 
  <file src="**\*.cs" target="src" />
</files>

Since the purpose of publishing the symbols is to allow others to step through your code when debugging, it seems most prudent to publish a version of the code intended for debugging without optimizations that might affect the code step through.

View more solutions

Comments

[Jon Skeet]

I was about to ask the same question - although currently I'm pushing the Release configuration to symbolsource too, given that I'm just using nuget pack ... -Symbol and pushing the generated packages...

[gzak]

I feel like I should submit this Q/A session to the folks behind NuGet and see if they can weigh in on it.

[CShark]

Bake logging into your package and then publish only the release build. You can allow logger injection which will provide the consumer to set up logging as per his/her preferences.

[Jon Skeet]

The "publishing two builds to NuGet" part is the the tricky bit. I've always published the Release build of Noda Time, on the grounds that I had to choose between the two. (I've got a nuspec file, rather than doing it just from the C# project.) You mention debugging production code as if that's significantly more difficult with only a release build - despite the previous "they don't differ much" part. I'm aware of the NOPs in debug builds, and obviously Debug.Assert etc - but could you expand (in your answer) on the implications when debugging? How bad is it using a Release build?

[Jon Skeet]

Yes, that's what the example does - but I'd rather not end up with the ability to debug overriding the desire for most developers to just run the release version. The problem is that NuGet doesn't have a way of publishing both release and debug builds, as far as I can tell. (It would also be a slight pain for me to publish both, as it means creating yet another set of configurations, for signed debug builds...)

[TripleEmcoder]

What I meant was just that you want to have symbols available for all the binaries in use: if there are both debug and release builds in the wild, you'll want to provide symbols for both. They don't need to differ much in terms of code, but they will differ in checksums, which makes loading symbols impossible without some hacking.

[Jon Skeet]

Right. My possible solution is to only ever release the Release binaries, but I'm worried about how much that's going to hurt in terms of debugging. For example, does the lack of NOPs mean you can't add breakpoints?

[tvanfosson]

Here we get into a little subjectivity. Typically, my choice for "best practice" would be that which the author recommends, either explicitly or implicitly, because I presume they have more insight (or oversight, in the case of assumptions that end up in examples) than I. Speaking personally, I think the symbols should match the version I'm using. I typically wouldn't use a package that I thought I would be likely to have to debug, unless the source were public and I could compile from scratch if needed, so the lack of a packaged, debug build isn't particularly important.

[TripleEmcoder]

The problem is more with JIT optimizations than build-time optimizations. So if you push release-mode packages and recommend using COMPLUS_ZapDisable=1 when debugging is needed, that would be good enough for me. Still NuGet should allow debug and release together in some way.

[Remus Rusanu]

I had no idea symbolssource.org exists. As an itch I had many a times a need to scratch, I welcome it.

[Jon Skeet]

@TripleEmcoder: Thanks. I'll stick with release build only for the moment, and log a feature request around NuGet (aside from anything else, the documentation should give recommendations around this :)

[Jon Skeet]

In my case if anyone needs to debug Noda Time because they believe there's a bug in Noda Time, they'd be better off downloading the source (which they can do, of course). However, it's still useful to be able to step into the Noda Time source code just to see what's going on. Basically I think there are (at least) two different scenarios for debugging, with different solutions...

[Jon Skeet]

@TripleEmcoder: Looking at blogs.msdn.com/b/sburke/archive/2008/01/29/…, that suggests COMPLUS_ZapDisable is more about not using ngen'd code, which wouldn't be the case for us anyway - so is it actually relevant in this case? I'd expect that if you're launching in a debugger, it will already have the appropriate JIT settings.

[gzak]

One thing I like to think about is .NET itself: I think it's safe to say Microsoft publishes release builds and not debug builds of .NET. A release build implies a certain level of quality and stability, so the idea is that you should rarely be needing to step into the code to diagnose things.

[gzak]

However, that's also a rather "closed-source" mentality. I've found that in the open source world, it's more expected to be able to step into the code of "stable releases" of various libraries not because things are broken, but because sometimes the best documentation is to just see what the library is doing.

[gzak]

Maybe the entire distinction between a release and debug build is what's wrong here? In other words, maybe rather than having NuGet support both, Microsoft should consider removing the distinction and just publish one build of a library (and let the version number + documentation specify what's stable and what's not).

[Jon Skeet]

I'm not sure I understand your second sentence - I suspect the OP (before I edited) was doing manual pushes to SymbolSource for the debug builds. Do you envisage significant issues if the release version's PDB ends up in SymbolSource instead of the debug version? Or is that what you were advocating, and I just misunderstood?

[gzak]

"However, I just don't see enough people complaining about this that it'd be worth it at the moment." Perhaps they're not complaining, but if you asked a sample of users what they thought of this, I bet you'd find that most people would admit a bit of confusion at this particular step (what to publish to NuGet.org and SymbolSource.org). If you asked what they ended up picking, you'd probably find that there's no single agreed-upon practice, everyone just does their own thing.

[Alex]

I want to complain about this, where do I sign up?

[christ.s]

For Nuget and releasing both configuration of your package. Just create a .tt (T4 Template) file that'll be processed in a pre/post-build step and generate your NuSpec file. You can then feed it to your build system. To help you generate the proper NuSpec file, you might want to pass some argument to the .tt processing like your configuration and changelist number. Enjoy.

[felickz]

Once you have internal NuGets and you start deploying them to a symbol server ... you are inevitably going to want to debug them.... and even though you can step through the code you will end up with "Cannot obtain value of the local variable or argument ... it has been optimzed away". As Phil mentions... IF nuget had a way to load debug dlls in debug mode and release dlls in release mode (from nuget packages), this would be the ultimate. Until then we are stuck with Nuget package switcher

[htm11h]

I concur, this would be a nice product enhancement.

[alv]

snupkg are not supported by Azure DevOps, see developercommunity.visualstudio.com/idea/657354/…

[Pieterjan]

The question was: Debug or Release?