Have Geek Brain

HTML Parsing - Get the freshest HTML tags

When I'm parsing HTML, I only want to get the innermost tags for the entire document. My intention is to semantically parse data from an HTML document.

So if I have html like this

<html>
     <table>
           <tr><td>X</td></tr>
           <tr><td>Y</td></tr>
     </table>
</html>

I only want <td>X</td>

and <td>Y</td>

. Is this possible using Beautiful Soup or lxml?

+1


source to share


4 answers


In .NET I used HtmlAgilityPack to make all html syntax easy. It loads the DOM and you can select by nodes, in your case select nodes with no children. Perhaps it helps.



+3


source


Once you've made sure your document is well-formed (for example, by parsing it using lxml), you can use XPath to query for all nodes that have no additional children.



// * [count (*) = 0]
+2


source


This is one of the few situations where you can actually use a regex to parse an HTML string.

\<(\w+)[^>]*>[^\<]*\</\1\s*>
0


source


If you can use or manipulate the DOM (i.e. in the browser), you can work with the parentNode attribute for all tags and recursively count the total and keep the largest.

In javascript pseudocode (tested on FireFox):

var allElements = document.getElementsByTagName("*");
var maxElementReference, maxParentNodeCount = 0;
var i;

for (i = 0; i < allElements.length; i++) {

    var count = recursiveCountParentNodeOn(allElements[i]);

    if (maxParentNodeCount < count) {
        maxElementReference = allElements[i];
        maxParentNodeCount = count;
    }
}
0


source






More articles:


All Articles