Have Geek Brain

Tidy breaks ties with non-latin characters

I am using Tidy java library to sanitize html code. Some code contains links with Russian letters. for instance

<a href="http://example.com/">link with Russian letters</a>

I understand that "Russian" needs to be escaped, but I am getting this html from users. And my job is to convert it to XHTML.

I think I am neatly trying to avoid non-latin letters, but as a result I get

<a href="http://example.com/%420%443%441%441%43A%438%439">link with Russian letters</a>

This is not a consequence. Correct version

<a href="http://example.com/%D0%A0%D1%83%D1%81%D1%81%D0%BA%D0%B8%D0%B9">link with Russian letters</a>

Java code

private static Tidy getTidy() {
    if (null == tidy) {
      tidy = new Tidy();
      tidy.setQuiet(true);
      tidy.setShowErrors(0);
      tidy.setShowWarnings(false);
      tidy.setXHTML(true);
      tidy.setOutputEncoding("UTF-8");
    }
    return tidy;
}

public static String sanitizeHtml(String html, URI pageUri) {
    boolean escapeMedia = false;
    String ret = "";
    try {
      Document doc = getTidy().parseDOM(new StringReader("<body>" + html + "</body>"), null);

      // here I make some processing

      // string output
      ByteArrayOutputStream out = new ByteArrayOutputStream();
      Node node = doc.getElementsByTagName("body").item(0);
      getTidy().pprint(node, out);
      ret = out.toString().trim();
    }
    catch (Exception e) {
      ret = html;
      e.printStackTrace();
    }

    return ret;
}
+2


a source to share


1 answer


This is hardcoded behavior and is probably a bug. They use UTF-16 to remove non-ASCII characters from URLs when they should be using UTF-8. Cm org/w3c/tidy/AttrCheckImpl.java

.



+1


a source






More articles:


All Articles