XML UTF-8 encoding check
I have an XML structure like this, some Student elements contain invalid UTF-8 byte sequences that could cause XML parsing to fail for the entire XML document.
What I want to do is filter out the Student element which contains UTF-8 byte sequences and keep valid byte sequences. Any advice or examples on how to do this in .Net (C # preferred)?
BTW: Invalid Byte Sequences I mean => http://en.wikipedia.org/wiki/UTF-8#Invalid_byte_sequences
<?xml version="1.0" encoding="utf-8"?>
<AllStudents>
<Student>
Mike
</Student>
<Student>
(Invalid name here)
</Student>
</AllStudents>
thanks in advance george
a source to share
This is quite difficult to do. You won't get an XML parser to parse a document with invalid characters in it, so I think you boiled down to a few options:
- Find out why the coding is wrong. A common problem is marking a document as UTF-8 (or missing an encoding declaration) when the document is actually written in Latin-1.
- Remove the bad partitions manually.
- Try to find a .NET tag soup parser that will continue parsing after an error.
- Reject an invalid XML document.
a source to share
I don't know C #, so I'm afraid I can't give you the code to do this, but the basic idea is to read the entire file as a utf-8 text file using DecoderFallback to replace invalid sequences with question marks or the unicode character 0xFFFD. Then write the file as utf-8 text file and parse it.
Basically, you are decoupling the "wipe out bad utf-8" operation from the "parsing XML file" operation.
You can probably even skip overwriting the file before running the XML parser to read into the fixed data; there must be some way to write the file to a byte stream in memory and parse that byte stream as XML. (Again, sorry I don't know C #)
a source to share