Strange intermittent SQL connection error, fixes on reboot, comes back after 3-5 days (ASP.NET)

Question

For some reason every 3-5 days our web app loses the ability to open a connection to the db with the following error, the strange thing is that all we have to do is reboot the container (it is a VPS) and it is restored to normal functionality. Then a few days later or so it happens again. Has anyone ever had such a problem? I have noticed a lot of ANONYMOUS LOGONs in the security log in the middle of the night from our AD server which is strange, and also some from an IP in Amsterdam. I am not sure how to tell what exactly they mean or if it is related or not.

Server Error in '/ntsb' Application.
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)

Source Error:

Line 11:        
Line 12:        
Line 13:        dbConnection.Open()
Line 14:        
Line 15:        


Source File: C:\Inetpub\wwwroot\includes\connection.ascx    Line: 13

Stack Trace:

[SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)]
   System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection) +248
   System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj) +245
   System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, SqlConnection owningObject) +475
   System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnection owningObject) +260
   System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart) +2445449
   System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance) +2445144
   System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance) +354
   System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection) +703
   System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options) +54
   System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject) +2414696
   System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject) +92
   System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject) +1657
   System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) +84
   System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) +1645687
   System.Data.SqlClient.SqlConnection.Open() +258
   ASP.includes_connection_ascx.getConnection() in C:\Inetpub\wwwroot\includes\connection.ascx:13
   ASP.default_aspx.Page_Load(Object sender, EventArgs e) in C:\Inetpub\wwwroot\Default.aspx:16
   System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr fp, Object o, Object t, EventArgs e) +25
   System.Web.Util.CalliEventHandlerDelegateProxy.Callback(Object sender, EventArgs e) +42
   System.Web.UI.Control.OnLoad(EventArgs e) +132
   System.Web.UI.Control.LoadRecursive() +66
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +2428


Version Information: Microsoft .NET Framework Version:2.0.50727.3053; ASP.NET Version:2.0.50727.3053 

Solution 1

The number of allowed connections to sql server is a fixed resource. It sounds like you have code somewhere that isn't closing it's connection correctly, and after a period you can't open any more new ones.

Solution 2

We recently faced a similar issue in our production environment and after many rounds of "debugdiag"ging and analyzing the dumps we came to the conclusion that a largely fragmented large object heap was causing this. Ref http://msdn.microsoft.com/en-us/magazine/cc534993.aspx. In short, your application might have run short of memory and didn't have enough contiguos space left to open connections. You can use VMMap http://technet.microsoft.com/en-us/sysinternals/dd535533 for the purpose of identifying available free space. Are you recycling your AppPools regularly? AppPool recycle tears down the application and frees up any help memory hence, resolves any related issues.

Comments

[RobS]

I've seen that happen more than a few times

[MetaGuru]

OK I tried perfmon and it opened but there were no logs available.

[MetaGuru]

That is interesting, I will check on such a thing. However we had this web app running on a previous server and never had this problem, it was only after we migrated to this new virtual server that the problem started. So I kind of think it's not that...

[Sam]

Do you have access to SQL Management Studio? msdn.microsoft.com/en-us/library/ms187929.aspx I'm not familiar what capabilities you have in a vps situation. Perfmon - no logs available? You mean no counters to pick from?

[Sam]

It could be that though, perhaps you have less resources.