Fixing WebRequest’s desire to throw exceptions instead of returning status

September 2nd, 2011 Categories: CoApp, Development, Uncategorized

My pappy always used to tell me “There are three kinds of men: The ones that learn by reading. The few who learn by observation. The rest of ‘em have to pee on the electric fence.” … somedays, I’m surprised how often I get to pee on the damn fence.

So, here I am writing some code to download files from remote servers, and I wandered across one of the dumbest designs in the .NET framework.

When using WebRequest (really, the child class HttpWebRequest ) write some code to make an HTTP request of the faraway server (I’m showing the non-async way to keep it simple):

var webRequest = (HttpWebRequest) WebRequest.Create("http://microsoft.com/foo");
webRequest.AllowAutoRedirect = true;
webRequest.Method = WebRequestMethods.Http.Get;

var webResponse = webRequest.GetResponse() as HttpWebResponse;

if( webResponse != null ) {
    Console.WriteLine("Status: {0}", webResponse.StatusCode );

    // ... finish reading from the webResponse....
}

That’s fine and dandy. Except one thing; if the request returns 404 NOT FOUND, GetResponse throws an Exception!?!

Now, I can understand how someone might think that not finding the requested resource is exceptional behavior, but it’s not a condition that I can check for before making the request itself. It’s certainly not in the same class of responses as timing out, or not having network connectivity.

So, through no fault of my own, the code throws an exception without even letting me check to see the result. Which is at best, rude and at worst, the lousiest way to handle that which is not exceptional behavior.

Eric Lippert calls these “Vexing Exceptions”

This only gets worse when using the Task Parallel Library in .NET 4.0 :

var webRequest = (HttpWebRequest) WebRequest.Create("http://microsoft.com/foo");
webRequest.AllowAutoRedirect = true;
webRequest.Method = WebRequestMethods.Http.Get;

var task = Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse, webRequest.BetterEndGetResponse , this)
    .ContinueWith(asyncResult => {
        var webResponse = asyncResult.Result as HttpWebResponse;
        Console.WriteLine("Status: {0}", webResponse.StatusCode );
        // ... finish reading from the webResponse....
    } );

So now, rather than checking the response, I have to check the asyncResult.IsFaulted property, grab the asyncResult.Exception property (which is actually an AggregateException), call aggregateException.Flatten() and iterate thru the collection to find the WebException, and ask that object for the WebResponse and from there, get the damn StatusCode… just so I can find out WHY the damn call failed!

I searched all the way thru the source code in the .NET framework, and I didn’t see any workaround… then it hit me. All we have to do is make the call not throw a bleedin’ exception when it actually created a WebResponse.

A quick little static class with a couple extension methods fixes the problem nicely (taken from the CoApp project for where I wrote it):

//-----------------------------------------------------------------------
//
//     Copyright (c) 2011 Garrett Serack. All rights reserved.
//
//
//     The software is licensed under the Apache 2.0 License (the "License")
//     You may not use the software except in compliance with the License.
//
//-----------------------------------------------------------------------

namespace CoApp.Toolkit.Extensions {
    using System;
    using System.Net;

    public static class WebRequestExtensions {
        public static WebResponse BetterEndGetResponse(this WebRequest request, IAsyncResult asyncResult) {
            try {
                return request.EndGetResponse(asyncResult);
            }
            catch (WebException wex) {
                if( wex.Response != null ) {
                    return wex.Response;
                }
                throw;
            }
        }

        public static WebResponse BetterGetResponse(this WebRequest request) {
            try {
                return request.GetResponse();
            }
            catch (WebException wex) {
                if( wex.Response != null ) {
                    return wex.Response;
                }
                throw;
            }
        }
    }
}

Now, just call webRequest.BetterGetResponse() or webRequest.BetterEndGetResponse() for the async version. Presto, it returns the WebResponse object instead of throwing (and will still throw for truly exceptional reasons.)

If you want to use it with the TPL’s FromAsync methods, prefix your webRequest.BetterEndGetResponse with the a cast to (Func<IAsyncResult, WebResponse>) before it so that the compiler can curry the extension method into the appropriate delegate for you:

var task = Task.Factory.FromAsync<WebResponse>(
    webRequest.BeginGetResponse, (Func<IAsyncResult, WebResponse>)webRequest.BetterEndGetResponse , this)
    .ContinueWith(asyncResult => {
        // ... yada yada yada....
    } );

Of course, this does mean that you MUST check the webResponse.StatusCode, and that you probably still need want an exception handler, but if you’re trying to handle truly exceptional cases differently than simple response conditions, I think you’ll like what you see here.

Tags: ,
  • http://fearthecowboy.com Garrett Serack

    Just switched to Disqus for comments. Testing now.

  • Srinath Janakiraman

    nice post dude…

  • David Sutton

    In my experience so far today (Framework 4, VS 2010) it’s returning 404 errors even on good URLs.I can use the debug Immediate window to verify the URL, copy it and paste it into my browser.In the browser the URL comes up just fine.
    The webRequest is somehow trashing the URL:

    The code:
    ohrRequest = (HttpWebRequest)WebRequest.Create(sURL);   //<- this is the URL I can type into the browser and get it.    ohrRequest.AllowAutoRedirect = _allowAutoRedirect;    ohrRequest.Method = "GET";        if(wProxy != null) ohrRequest.Proxy = wProxy;    ohrRequest.Headers["Cookie"] = getCookieHeader();    HttpWebResponse ohrResponse = (HttpWebResponse)ohrRequest.GetResponse();    return handleResponse(ohrResponse);

    As you pointed out it throws an error on the GetResponse() line.CLASSIC MICROSOFT – STUPID

  • Michael Lutz

    Thanks! Simple yet very effective!