Added Functionality - LoadUri

Topics: Developer Forum, User Forum
Mar 22, 2011 at 2:39 PM

So I was trying to use the HtmlAgilityPack to load as web resource and generate a DOM tree and found that a piece of functionality that I was wanting to use, the ability to call Load() with a web uri, was missing.  I have added the functionality myself and wanted to contribute my solution back to the project. 

In HtmlDocument.cs

Added dependancy:

using System.Net;

Added Functionality:

 /// <summary>
 /// Loads an HTML document from a uri passed as a string.
 /// </summary>
 /// <param name="uri">The complete url to the web resource to be loaded.</param>
public void LoadUri(string uri)
{
if (uri == null)
throw new ArgumentNullException("uri");

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Timeout = 30*60*1000;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream());
Load(sr);
sr.Close();
}

Please feel free to add/use this code as needed. I have found it useful.

-HTH


John Bouma
May 30, 2013 at 6:36 PM
Edited May 30, 2013 at 6:36 PM
Hi,

Thanks for the great add-on-method, exactly what I was looking for...

I also needed to pass in credentials, so I'm posting the changes I did here as well...
        public void LoadUri(string uri)
        {
            LoadUri(uri, System.Net.CredentialCache.DefaultCredentials);
        }
        public void LoadUri(string uri, string userName, string password, string domain)
        {
            LoadUri(uri, new NetworkCredential(userName, password, domain));
        }
        public void LoadUri(string uri, ICredentials credentials)
        {
            if (uri == null)
                throw new ArgumentNullException("uri");
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
            request.Credentials = credentials;
            request.Timeout = 1 * 60 * 1000;
            
            using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
                using (StreamReader sr = new StreamReader(response.GetResponseStream()))
                {
                    Load(sr);
                }
        }