Hunch Kod

What I did:

1. Open a non-utf8-encoded file (in my case, Shift-JIS)
2. The encoding detection algorithm of Kod will fallback to using UTF-8 encoding (which is wrong).
3. Attempt to fix the file by adding an "encoding: shift-JIS" comment line at the top.
4. Save the file. At this point, Kod will save the file with the xattr of "utf-8".
5. Reopen the file, Kod will first look at the xattr and resolve the encoding to "utf-8". Instead of "Shift-JIS"

Note that step 3 and 4 could be done with TextEdit (since it doesn't write the file's xattr) and reopening it in Kod will show the correct encoding... though this doesn't quite "solve the problem".

What I expected to happen: The behavior is pretty much up to discussion. Though here are the suggestions I offer:

1. Change the order of detection: honor the explicit "charset/encoding" line in the file over the xattr of the file. The rationale behind this is that it's easier to modify the file's contents than manually editing the xattr.
2. If encoding detection utterly fails, instead of defaulting to UTF-8 or "NSISOLatin1StringEncoding", just ask the user what encoding to use.
3. In the event the encoding detection utterly fails, don't write the xattr when saving.
4. In the file open dialog, provide an option to selection which encoding to use (default to "auto-detect")
5. The status bar could be more useful by showing the current encoding of the file plus the file type.
6. Provide a menu for the user to select which encoding to use. Upon inspecting the code however, this is impossible without reloading the file (the NSData is released after assigning the converted text to the view), which is not a good solution because:
   • it discards the changes in the current buffer and asking the user if he wants to save first is kinda funky (what if the file is over HTTP?)
   • it will throw away the undo history
   • if the file is over the network, it will take some time to load again (I don't think Kod caches files over networks)

I'm willing to work on this but I'll need some inputs from the devs on what is the best course of action. I don't think implementing all of the above items will be good (6, for instance is already a no-go).

A related question to this is "can the user select the encoding to use when saving?" because right now, the encoding when saving will always be UTF-8 unless the file opening already has a previous marked encoding (xattr, file markers, etc.).