This project has moved and is read-only. For the latest updates, please go here.

Sending Browser String?

Topics: Developer Forum, User Forum
May 13, 2008 at 10:06 AM
Hi there.
First of all: awesome tool!

But now I'm a bit stuck: I try to spider a website. I send some Params (faking a form-Search) and want to read the results. When I use the WinForm-WebBrowser everything works fine, but when reading the site via HtmlWeb.Load() the HTML only contains something like "your Browser does not support our technology".

So: Can I fake an IE or something like that whit HtmlAgilityPack?

Thx in advance
Nov 24, 2008 at 6:24 AM
Add the following to your HtmlWeb.cs (patched to codebase, line numbers in []'s):

[43] private string _userAgent;
[323]req.UserAgent = _userAgent;
[777]public string UserAgent{ get { return _userAgent; }  set { _userAgent = value; } }

Then use as follows:

HtmlWeb htweb = new HtmlWeb();
htweb.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: Gecko/2008092417";
HtmlDocument htdoc = htweb.Load( "" );