IIS ASP.Net Website - ManagedPipelineHandler Error ( An operation was attempted on a nonexistent network connection)

Question

I am experiencing a strange issue. I have an ASP.Net web application that uses Forms Authentication. If I open the log-in page and there is a couple of minutes of inactivity, the log-in attempt times out or IE / FireFox display "Cannot display this page". If there is no pause or lack of activity, there is no issue and the website run as expected.

I enabled tracing in IIS. The area that appears to be hanging is NOTIFY_MODULE_START Module Name:ManagedPipelineHandler. After 132141 ms, I get the following error The I/O operation has been aborted because of either a thread exit or an application request. (0x800703e3) - ErrorCode 2147943395.

Error Code 2147943629

Error Code An operation was attempted on a nonexistent network connection. (0x800704cd)

Screen Shot: http://www.plasmawebservices.com/images/errorScreenShot.png

Server is Windows 2008 SP2. IIS 7

Has anyone come across an error like this?

<asp:Content ID="BodyContent" runat="server" ContentPlaceHolderID="MainContent">
<telerik:RadFormDecorator ID="RadFormDecorator" runat="server" DecoratedControls="All" />
<p>
    Please enter your username and password.
</p>
<asp:Login ID="LoginUser" runat="server" EnableViewState="false" RenderOuterTable="false">
    <LayoutTemplate>
        <asp:Panel runat="server" ID="LoginPanel" DefaultButton="LoginButton">
            <p>
                <span class="failure-notification">
                    <asp:Literal ID="FailureText" runat="server"></asp:Literal>
                </span>
            </p>
            <asp:ValidationSummary ID="LoginUserValidationSummary" runat="server" CssClass="failure-notification"
                ValidationGroup="LoginUserValidationGroup" />
            <div class="account-info">
                <fieldset class="login">
                    <legend>Account Information</legend>
                    <p>
                        <asp:Label ID="UserNameLabel" runat="server" AssociatedControlID="UserName">Username:</asp:Label>
                        <asp:TextBox ID="UserName" runat="server" CssClass="text-entry"></asp:TextBox>
                        <asp:RequiredFieldValidator ID="UserNameRequired" runat="server" ControlToValidate="UserName"
                            CssClass="failure-notification" ErrorMessage="User Name is required." ToolTip="User Name is required."
                            ValidationGroup="LoginUserValidationGroup">*</asp:RequiredFieldValidator>
                    </p>
                    <p>
                        <asp:Label ID="PasswordLabel" runat="server" AssociatedControlID="Password">Password:</asp:Label>
                        <asp:TextBox ID="Password" runat="server" CssClass="password-entry" TextMode="Password"></asp:TextBox>
                        <asp:RequiredFieldValidator ID="PasswordRequired" runat="server" ControlToValidate="Password"
                            CssClass="failure-notification" ErrorMessage="Password is required." ToolTip="Password is required."
                            ValidationGroup="LoginUserValidationGroup">*</asp:RequiredFieldValidator>
                    </p>
                    <%--<p>
                        <asp:CheckBox ID="RememberMe" runat="server"/>
                    <asp:Label ID="RememberMeLabel" runat="server" AssociatedControlID="RememberMe" CssClass="inline">Keep me logged in</asp:Label>
                    </p>--%>
                </fieldset>
                <p class="margin-top-10">
                    <asp:Button ID="LoginButton" runat="server" CommandName="Login" Text="Log In" ValidationGroup="LoginUserValidationGroup" />
                </p>
            </div>
        </asp:Panel>
    </LayoutTemplate>
</asp:Login>

Thanks, Brennan

Solution 1

Solution

The client had AVG Security workstation components installed on their servers. The workstation components caused communication issues with IIS. By disabling, Online Shield and Resident Shield the symptoms are no longer present.

Case Notes:

After two minutes of inactivity on any page on the web application, the page locks. You will see an Ajax loading panel ( aka spinning icon) for 120 seconds on your next action and then nothing. The application has locked.

To find out the root cause as to why we are getting the error "An operation was attempted on a nonexistent network connection. (0x800704cd)" when we browse any web page after an idle time of minutes. This issue can occur due to multiple reasons and for the purpose of this ticket we will concentrate on only one root cause If the issue is caused by a Microsoft component, we will work diligently and efficiently to resolve this issue. If the cause of the issue is identified to be a third party or custom component, we will be happy to discuss our findings in detail and provide possible methods of resolution/workaround. It will ultimately be up to the third party vendor to resolve these types of issues.

For troubleshooting, the default ASP.Net website was created and deployed as a new website to IIS. The results were the same as the problematic website.

IIS HTTP Errors:

start -> run -> LogFiles

2011-09-20 21:25:25 xx.xxx.xxx.xxx 36482 192.168.10.32 80 HTTP/1.1 POST /orders/mail/mailorderentry - - Timer_HeaderWait –

2011-09-20 21:22:29 xx.xxx.xxx.xxx 44399 192.168.10.32 80 HTTP/1.1 POST /orders/mail/mailorderentry - - Client_Reset –

At first, the issue apperared to be with Telrik. While loading up the Telerik.Web.UI.RadCompression on the REQUEST_ACQUIRE_STATE event, the following error was logged via IIS tracing:

Telerik.Web.UI.RadCompression 132187 ms

1. -GENERAL_READ_ENTITY_END BytesReceived 0 ErrorCode 2147943395 ErrorCode The I/O operation has been aborted because of either a thread exit or an application request. (0x800703e3) Informational

2. -GENERAL_FLUSH_RESPONSE_END BytesSent 0 ErrorCode 2147943629 ErrorCode An operation was attempted on a nonexistent network connection. 0x800704cd)

The RadCompression module was removed from the web application web.config file; however, the problem persisted.

Proposed Solution:

For now disable, AVG Online Shield and Resident Shield on the servers.

From AVG:

The following AVG components are designed only for a use on workstations:

• AVG Firewall
• AVG Online Shield
• AVG Personal E-mail Scanner
• AVG LinkScanner

The above components were not tested under a system load. If installed on a server operating system (such as ISA, proxy, mail server, terminal server, etc.) the components might cause issues with server communication. To avoid these issues it is not recommended to install the mentioned components on a server operating system. Note: This does not apply in case that you use the computer with some server operating system installed (e.g. Windows Server 2000, 2003 etc.) as a standard computer without the server features enabled.

More information about the AVG installation on specific servers can be found in these FAQs:

• FAQ 2129 - Terminal Server
• FAQ 2176 - E-mail Server

Solution 2

I know this is an old post but just in case someone runs into the same issue...

We were receiving the same errors when our load balancer appliance was trying to check https for one of our sites. Trying to hit the link via a pc browser worked fine.

• error while reading HTTP response

Failed Request Tracing showed these 2 errors

• BEGIN_REQUEST - The I/O operation has been aborted because of either a thread exit or an application request. (0x800703e3)
• An operation was attempted on a nonexistent network connection. (0x800704cd)

The fix for us was in IIS under "SSL Settings". "Accept" was currently selected for "Client Certificates." Changing to "Ignore" fixed the issue.

Comments

[Brennan Mann]

I did increase the executionTimeout property; however, in my custom membership provider the ValidateUser method is never executed. This method hits a WCF service to validate the user. Due to the inactivity, it's hanging somewhere in ASP.Net pipeline. Any ideas on other troubleshooting methods? - Screen Shot Thanks

[Bernie White]

@Brennan Are you able to post your web.config? Specifically do you have any custom defined modules or handlers? Also can I confirm that you are operating in IIS 7.0 Integrated.

[Brennan Mann]

We did have both custom modules and handlers. We can removed them for troubleshooting with no luck. We were running Windows 2008 SP 2.

[Brennan Mann]

The client did not have the E-mail Scanner components installed on their server. What a nightmare this caused. As AVG was only installed in the production environment and we had the "it must be an issue with our web application" blinders on :). Thanks for the feedback. I hope others find this information helpful.

[krisdyson]

Thank you for this. I just spent 10 hours trying to work out why an HTTP POST didn't work from an external server to my local machine (for testing purposes).