This project has moved. For the latest updates, please go here.

Count size of loaded content, or preferably transferred size

Aug 20, 2014 at 2:40 PM

First of all, thanks for a great piece of software! Neat and powerful!

I use it for a quite specialized web spider that parse selected info from a site of mine. My use of HTML Agility Pack is rather basic I think.

I'd like to keep track of how many bytes my program reads from the web. Cannot find any obvious solution for this, so I might as well ask.

Currently, I load pages more or less by using snippets like this (error handling excluded for sake of readability):
var webPage = new HtmlWeb();
HtmlDocument htmlDoc = webPage.Load(uriBillboardSearch);                    

// Some parsing...                
I was thinking of using HtmlDocument.Save() and count bytes but it seems a waste of work and it would only count what I have read, not what I have transferred from the web to my client.

I am aware of that this might not be an HTML Agility Pack thing at all, but if anyone have some ideas I would be grateful!


Aug 20, 2014 at 3:12 PM
Edited Aug 20, 2014 at 3:12 PM
Of course I came up with one possible solution just after I posted this. Using System.Net namespace, I can download the web page as a string before passing it on to HTML Agility Pack:
var wc = new WebClient();
var htmlText = wc.DownloadString(uriBillboardSearch);
var htmlDoc = new HtmlDocument();
htmlText.Length would hold the size of downloaded content, right?

In other words, instead of HtmlWeb.Load() use WebClient.DownloadString combined with HtmlDocument.LoadHtml().
Is there any better way or a way? Any other thoughts?