htmlagilitypack Work Item Rss Feed

Created Issue: end tag tag in javascript string code is missed read [25290]

zivr — Tue, 10 Nov 2009 09:15:09 GMT

Many people create a javascript code that write more script code such as the following example.
The Html Parsing recognize the '</script>' string as an end tag while it's part of the string value.
This result is an incorrect parsing.

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<script >
var code = '<sc'+'ript>';
code += 'alert("this is a demo, I can put any code here");'
code += '</script>';
document.write(code);
alert('this code is part of the script');
</script>
<title>test</title>
</head>
<b>normal text</b>
</html>

Commented Issue: OptionOutputOriginalCase is ignored for attributes when not outputting XML [25278]

Amtiskaw — Mon, 09 Nov 2009 19:45:58 GMT

I've been using Html Agility Pack to automatically modify some HTML forms and insert ASP.NET tags into them. I ran into a problem whereby I could specify attributes with names such as "AssociatedControlID", but they would always be output as "associatedcontrolid", even when OptionOutputOriginalCase was set to true. I had a look in the source for the library and the problem appears to be in HtmlNode.WriteAttribute. The check for OptionOutputOriginalCase = true is only carried out if OptionOutputAsXml is also true. I think this option should be checked in either case, and if it is true, the original attribute name used. Unfortunately, I can't switch on XML output, as this causes problems with ASP.NET tag names like "asp:TextBox", so I'm using a modified version of the library to get around the problem for now.

Created Issue: OptionOutputUpperCase is ignored for attributes when not outputting XML [25278]

Amtiskaw — Mon, 09 Nov 2009 19:41:46 GMT

I've been using Html Agility Pack to automatically modify some HTML forms and insert ASP.NET tags into them. I ran into a problem whereby I could specify attributes with names such as "AssociatedControlID", but they would always be output as "associatedcontrolid", even when OptionOutputUpperCase was set to true. I had a look in the source for the library and the problem appears to be in HtmlNode.WriteAttribute. The check for OptionOutputUpperCase = true is only carried out if OptionOutputAsXml is also true. I think this attribute should be checked in either case, and if it is true, the original attribute name used. Unfortunately, I can't switch out the XML output, as this causes problems with ASP.NET tag names like "asp:TextBox", so I'm using a modified version of the library to get around the problem for now.

Commented Issue: UTF8 is not a supported encoding [25273]

JudahGabriel — Mon, 09 Nov 2009 16:13:32 GMT

We've deployed our application to several users, and we just received this error coming from the HTML Agility Pack:

Message: 'utf8' is not a supported encoding name.

Parameter name: name

Stacktrace:
at System.Globalization.EncodingTable.internalGetCodePageFromName(String name)
at System.Globalization.EncodingTable.GetCodePageFromName(String name)
at HtmlAgilityPack.HtmlDocument.ReadDocumentEncoding(HtmlNode node)
at HtmlAgilityPack.HtmlDocument.PushNodeEnd(Int32 index, Boolean close)
at HtmlAgilityPack.HtmlDocument.Parse()
at HtmlAgilityPack.HtmlDocument.Load(TextReader reader)
at HtmlAgilityPack.HtmlDocument.LoadHtml(String html)

According to this MS Connect case (http://bit.ly/3tcHmN), the bug is due to the malformed "utf8" encoding, when it should be "utf-8" (notice the hyphen). Can HTML Agility Pack work around this issue by checking for this condition?

Created Issue: UTF8 is not a supported encoding [25273]

JudahGabriel — Mon, 09 Nov 2009 15:23:05 GMT

We've deployed our application to several users, and we just received this error coming from the HTML Agility Pack:

Message: 'utf8' is not a supported encoding name.

Parameter name: name

Stacktrace:
at System.Globalization.EncodingTable.internalGetCodePageFromName(String name)
at System.Globalization.EncodingTable.GetCodePageFromName(String name)
at HtmlAgilityPack.HtmlDocument.ReadDocumentEncoding(HtmlNode node)
at HtmlAgilityPack.HtmlDocument.PushNodeEnd(Int32 index, Boolean close)
at HtmlAgilityPack.HtmlDocument.Parse()
at HtmlAgilityPack.HtmlDocument.Load(TextReader reader)
at HtmlAgilityPack.HtmlDocument.LoadHtml(String html)

According to this MS Connect case: http://bit.ly/3tcHmN the bug is due to the malformed "utf8" encoding, when it should be "utf-8" (notice the hyphen). Can HTML Agility Pack work around this issue by checking for this condition?

Created Issue: Document.WriteTo() striping tags [25231]

kowboy33 — Thu, 05 Nov 2009 21:58:39 GMT

When i load the string into the document all of the html is there.

HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(output.ToString());

The string is html, a dropdown element with options
...Any html page info(headers, html tags and such)...
<ul><li>
<select>
<option selected="selected" value=""></option>
<option value="10 Radial Arm Saw">10 Radial Arm Saw - </option>
<option value="18V Packing">18V Packing - </option>
</select>
</li></ul>
...(any closing tags needed for page)...
Tried to get this as close as possible without giving to much unessacary code. There is some onchnage events in the sleect and a name and id attribute.

The main thing i am doing is trying to strip the UL's out of the page code. And it works, but when i go to writeout the string using

doc.DocumentNode.WriteTo()

I get this below. Notice all the closing option tags were stripped off.

...Any html page info(headers, html tags and such)...
<ul><li>
<select>
<option selected="selected" value="">
<option value="10 Radial Arm Saw">10 Radial Arm Saw -
<option value="18V Packing">18V Packing -
</select>
</li></ul>
...(any closing tags needed for page)...

I tried fooling with option setting and sometimes get this(below), it add the "/" to the openning option tag

...Any html page info(headers, html tags and such)...
<ul><li>
<select>
<option selected="selected" value=""/></option>
<option value="10 Radial Arm Saw"/>10 Radial Arm Saw - </option>
<option value="18V Packing"/>18V Packing - </option>
</select>
</li></ul>
...(any closing tags needed for page)...

Anyone have any suggestiosn or ideas to what is going on?
Not performing any type of filtering or such yet, just printing out the extra string in the doc.DocumentNode. This is wher I am tracing all my problems back too.

This also leads to slowness problems further along in the code, when i go to parse out the UL tag, and i believe it is straing on the malformed "option tags".

thanks

Created Issue: Bug in HtmlDocument.Save(TextWriter writer) [25104]

BasKnowmax — Tue, 27 Oct 2009 10:09:09 GMT

A line writer.Flush() is missing in the above mentioned method. This could cause loss of contents when saving. The overloaded method Save(XmlWriter writer) already has this line.

This bug (and solution) where mentioned before in the discussions here:
http://htmlagilitypack.codeplex.com/Thread/View.aspx?ThreadId=1260

Created Issue: Character Encoding fix [24995]

DraconPern — Tue, 20 Oct 2009 10:41:45 GMT

Line 418 of HtmlWeb.cs has these incorrect lines

if ((resp.ContentEncoding != null) && (resp.ContentEncoding.Length>0))
{
respenc = System.Text.Encoding.GetEncoding(resp.ContentEncoding);
}

ContentEncoding is not the same as CharacterEncoding!
The correct code should be

if ((resp.CharacterSet != null) && (resp.CharacterSet.Length > 0))
{
respenc = System.Text.Encoding.GetEncoding(resp.CharacterSet);
}

Created Issue: can't load diacritics [24925]

mcibelli — Thu, 15 Oct 2009 07:19:10 GMT

is there a way to load an HTMLDocument without loosing diatrics?

Created Issue: &#x????; (HEX html entities) not being decoded properly [24812]

defza — Tue, 06 Oct 2009 12:36:58 GMT

If I have the following characters in an html document:




it becomes
&##x000A;

(with two hashes)

Commented Issue: Option End Tags Dropped [21249]

defza — Tue, 06 Oct 2009 12:27:38 GMT

To replicate, create an HtmlDocument and LoadHtml with the following string:

<select id="foo_Bar" name="foo.Bar"><option selected="selected" value="1">One</option><option value="2">Two</option></select>

This does some unexpected things. First, it gives two parser errors, EndTagNotRequired. Second, the SELECT node has 4 children - two for the OPTION tags and two more for the text. That is, the text is not seen as the InnerText of the OPTION. Last, the OuterHtml is like this:

<select id="foo_Bar" name="foo.Bar"><option selected="selected" value="1">One<option value="2">Two</select>

I'm pretty confident that having and end tag for an OPTION element is considered well formed. It should not result in errors. Also, the text should be InnterText of the OPTION. I have tried setting these as follows, to no avail:

doc.OptionAutoCloseOnEnd = false;
doc.OptionCheckSyntax = false;
doc.OptionFixNestedTags = false;

I have found this can be corrected by removing line 92 of HtmlNode.cs:

ElementsFlags.Add("option", HtmlElementFlag.Empty);
Comments: ** Comment from web user: defza **

But there's a problem when you have an empty option:

Closed Feature: Patch for supporting Proxys [8055]

DarthObiwan — Sat, 03 Oct 2009 17:45:18 GMT

I have patched HtmlWeb.cs to support proxies without changing the existing interfaces and adding new ones. Please integrate this into the base code.
Comments:

Commented Feature: Patch for supporting Proxys [8055]

DarthObiwan — Sat, 03 Oct 2009 17:45:07 GMT

I have patched HtmlWeb.cs to support proxies without changing the existing interfaces and adding new ones. Please integrate this into the base code.
Comments: ** Comment from web user: DarthObiwan **

I have integrated this into the current code base and it will be included in the next release

Closed Issue: Memory Leak for HtmlNode [14297]

DarthObiwan — Sat, 03 Oct 2009 16:34:48 GMT

HtmlNode.WriteTo() and HtmlNode.WriteContentTo() causes a memory leak not closing the StringStream opened.

public string WriteTo()
{
StringWriter sw = new StringWriter();
WriteTo(sw);
sw.Flush();
return sw.ToString();
}

Should be
public string WriteTo()
{
StringWriter sw = new StringWriter();
WriteTo(sw);
sw.Flush();
string s = sw.ToString();
sw.Close();
return s;
}

Or better yet, using {} block
Comments:

Closed Issue: SelectNodes from ChildNodes [13102]

DarthObiwan — Sat, 03 Oct 2009 14:53:09 GMT

It would be nice if I could use SelectNodes() from a child node and have it only select those type of nodes from the child nodes rather then from the entire tree.

A good example, would be that I need to find all links from a specific table. I search for the Table I want, Now all the links I want are child nodes of the table node.

Right now, if I do TableNode.SelectNodes("//a"), it gets ALL links in the document.

I have coded a work around, but it would be a very useful feature that would save a lot of extra code down the road.
Comments:

There is now an alternative, use Descendants(string name) on the node itself. This will return an IEnumerable<HtmlNode> that is yeilded to be inline with Linq to Objects.

Commented Issue: HtmlNode.XPath Property [12095]

DarthObiwan — Sat, 03 Oct 2009 05:44:41 GMT

It would be nice to see an XPath property of the HTML Node class that return's a node's absolute xpath.
Comments: ** Comment from web user: DarthObiwan **

I have implemented a property that returns a valid Xpath to that particular node. Beta 1 contains the property but the path created had the #document element on it that was causing the path to not work. The next release will have this fixed

Commented Issue: Parsing problem [23074]

DarthObiwan — Sat, 03 Oct 2009 05:02:39 GMT

Hi, found parsing problem with following html code:

<html id="thisisTest">
<head>
<title>TT</title>
</head>
<body id="test">
<form>
<b>test</b>
</form>
<br />
This is plain text
<span>
<b>BOLD</b>
whee
</span>wheeagain
</body>
</html>

All going well for HTML, HEAD, TITLE, BODY tag, they all appear as proper child tree (e.g. HEAD is child of HTML, TITLE is child of HEAD etc.) but then you get to BODY tag, all elements in the BODY tag pear as it children no matter in what tag they are located in. Good example is FORM tag.
Comments: ** Comment from web user: DarthObiwan **

In the HtmlNode class there is a list of elements and what the parser should do with those elements.
Comment out Following line and you will get the behavior your are expecting.
ElementsFlags.Add("form", HtmlElementFlag.CanOverlap | HtmlElementFlag.Empty);

This is where the power and the drawbacks HAP has when parsing Html. This line helps it deal with malformed code but unfortunately also ends up parsing it wrong when the code is correct. I'm thinking of making an xml format to define these better to make it easier to tweak. Also the list shouldn't be in html node, it should be it's own separate strongly typed collection.

Commented Issue: Memory Leak for HtmlNode [14297]

DarthObiwan — Sat, 03 Oct 2009 04:40:16 GMT

Yep, the Using block would be the better way. Copying to the s string would just cause more memory to be used.
I'm incorporating a using block in the next release

Commented Issue: HtmlNode.XPath Property [12095]

DarthObiwan — Thu, 17 Sep 2009 04:57:31 GMT

It would be nice to see an XPath property of the HTML Node class that return's a node's absolute xpath.
Comments: ** Comment from web user: DarthObiwan **

I've added preliminary support for this on HtmlNode and HtmlAttribute . The properties get their relative xpath and add that on to their parent xpath.

Closed Issue: Whitespace in Content-Encoding HTTP header [4911]

DarthObiwan — Thu, 17 Sep 2009 00:13:19 GMT

I have been using HtmlAgilityPack for a work project, and at one point we encountered a page that had whitespace in the value of the Content-Encoding: HTTP header line (like "iso-8859-1 "). Attached is a small bugfix to trim the text so that this isn't a problem for others.

Thanks,
# Chris
Comments: