Assets file project.assets.json not found when running a build on Azure Devops

Question

I have a build pipeline configured for a Service Fabric solution on Azure DevOps like this:

Everything was fine until a few days ago when the build started failing on a particular build agent (private), with the following error (for a few projects):

C:\Program Files\dotnet\sdk\2.1.200\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(327,5): Error : Assets file 'F:\Agent03\w\84\s\src\MyProject.Sam.Tiles.Domain\obj\project.assets.json' not found. Run a NuGet package restore to generate this file.

The failing task is the Build solution $(PathToSolution) one.

The weird thing is that the build fails when running on some agents but with others the build is fine.

Some details:

• Use NuGet 4.x task started using NuGet v4.9.1 very recently, I think. I tried using v4.8.1 with no luck;
• Most of the projects use the PackageReference format, but the .sfproj project uses the packages.config file
• I tried using the dotnet restore task but there is an error when trying to restore the packages for the .sfproj project:

  `Error : Unable to find the '....\packages\Microsoft.VisualStudio.Azure.Fabric.MSBuild.1.6.7\build\Microsoft.VisualStudio.Azure.Fabric.Application.props' file. Please restore the 'Microsoft.VisualStudio.Azure.Fabric.MSBuild' Nuget package

Any idea on what might be causing this issue?

Solution 1

Some of the projects use the PackageReference format but the .sfproj project uses the packages.config file.

I still don't understand why the build started failing, but I was able to find a workaround. Given that PackageReference is not yet supported in Service Fabric projects, my workaround was to use both restore tasks as follows:

Solution 2

Trevor's comment on 2/20 gave me the clue. You likely don't have the complete set of projects referenced by the solution. (ProjectReferences may go to other projects, which are not in the solution).

Here is why this crazy workaround (run dotnet.exe and nuget.exe restore tasks) worked:

dotnet restore will walk project references by default to ensure they are restored also. --no-dependencies switch can turn that off.

nuget.exe restore has the opposite default, because we didn't want to break old users. -recursive can turn this on.

The right solution is to make your solution contain all the projects.

-Rob Relyea NuGet Client Team, Engineering Manager

Solution 3

My problem turned out to be a solution that didn't include all the necessary projects.

I have a master solution file that includes all my projects, and a number of smaller solution files with only some of the projects. The master solution built fine in Azure DevOps, but the partial solution failed.

I realized that the missing project.assets.json file belonged to a project that needed to be included in this failing solution.

Comments

[Daniel Mann]

Do you have an appropriate .tfignore/.gitignore (depending on which you're using for source control) that excludes the bin, obj, and packages folders? Can you confirm that these folders are not in source control?

[Rui Jarimba]

There is a .gitignore file in the root folder. I believe it is the default one for Visual Studio with some minro changes. Yes there are rules for the bin, obj and **/packages/* (nuget)

[Daniel Mann]

Did someone override the .gitignore and accidentally commit them to source control?

[Rui Jarimba]

@DanielMann you mean the nuget packages? No, packages are not under source control.

[Oleg Karasik]

Does CI use clean copy of sources? Have you tried to checkout clean copy on your local box and restore + build solution?

[Rui Jarimba]

@OlegKarasik yes I'm cleaning the sources - All build directories is selected in the Clean options dropdown. This build is failing only when using some particular agents, and it started only a few days ago.

[Oleg Karasik]

Because this is a private build agent do you have remote access to it? Can you check that all directories are in fact gets deleted (or delete them manually)? Similar error happen to my localbox a few times and it always was related to undeleted things.

[Rui Jarimba]

@OlegKarasik yes I do have remote access to the VM of the private agent. I can give it a try, even though I'm not sure which folders to delete? Also, I'm not completely sure it would make a difference because the agent seems to create a new folder for each build (e.g. F:\Agent01\w\499\s)

[zivkan]

you could try adding a build step to do msbuild /t:restore in addition to the nuget restore. msbuild /t:restore won't work for packages.config projects. I think nuget.exe should be fine for "classic" csproj projects using PackageReference, but I'm really not sure about SDK projects. Although I thought building SDK projects automatically restores them, so I'm surprised it doesn't work automatically.

[zivkan]

By any chance, does your repo have more than one solution, and MyProject.Sam.Tiles.Domain is not in one of them, but another project in the solution has a project reference to MyProject.Sam.Tiles.Domain?

[Rui Jarimba]

@Ziv I can confirm there are 2 solutions in that repository. I'm not sure about the references, I need to ask that to the development team that is responsible for that repository

[Rui Jarimba]

@Ziv running both restore tasks (standard and dotnet restore) will do the trick, but the thing I'm trying to understand is why this issue only started a few days ago, and why the build doesn't break using some agents?

[zivkan]

You could try adding nuget.exe -version and dotnet --info to your build logs, to check if there are different versions or different sdks installed on different agents. You could also try searching/opening an issue on NuGet/Home where someone smarter than me can give better insights.

[Rui Jarimba]

@Ziv I can confirm there are different projects being referenced in both solutions.

[zivkan]

I know that nuget needs every project in the dependency graph to be restored. I believe building a solution in Visual Studio will not build or restore projects not in the solution, even if a project in the solution has a project reference to it, but I don't know if msbuild has the same limitation. I suspect yes, because Visual Studio heavily uses MSBuild under the covers. If this is the case you're running into, it's a known issue/by design. It doesn't answer why there's a change in behaviour though.

[Anand Gaurav]

NuGet restore should be sufficient as it works for both PackageReference and packages.config. If not, there might be an issue. Would appreciate if you could file an issue with some repro at github.com/NuGet/Home/issues