For any reader not familiar with Unicode, reading SysconsUnicodeProject/UnicodeBasics is suggested.
This project aims to enhance Unicode support on FreeBSD operating system by implementing support for UTF-8 in the Syscons console driver. This is not about full support for Unicode-based applications (because has no impact on correctness of sorting text strings, for example). Currently system console driver deals with 8-bit character sets only.
| Description | Assigned to | Status |
|---|---|---|
| Make terminal emulator UTF-8 aware | Ed | Committed (01/01/2009) |
| Implement unicode-to-8-bit mapper for existing fonts | Alexander | Implemented (29/01/2009) |
| UTF-8 aware input layer, keymaps. | Ed | Semi-done |
| UTF-8 aware font renderer | Ed | TBD |
| Unicode fonts, not limited to 256 chars | Alexander | TBD, not scheduled |
The old terminal renderer used by Syscons is not suitable to use with UTF-8. Right now we use a terminal type called cons25. The disadvantages of cons25:
0x7f. This means it is impossible to use UTF-8, because if someone runs a non-UTF-8 application or sets LC_CTYPE to, say, C, applications like dialog(1) would generate illegal output.cons25 and UTF-8 is not something that exists right now and is proven to work. xterm and UTF-8 is known to Just Work ™.cons25 also has some other problems, not related to UTF-8. It is not supported by many operating systems and devices and lacks a lot of optimizations (scrolling regions).
This is why we should migrate to xterm at the same time. This adds a little more complexity to the terminal emulator (old VT100 character sets, keypad application mode, etc), but has its advantages.
As of 01/01/2009, the old Syscons emulator has been replaced by libteken. libteken is still configured to be a cons25 emulator, but already supports a lot more escape sequences. This means you should already be able to SSH to your Cisco/HP switch without problems.
If you want to try the new xterm + UTF-8 support, recompile your kernel with TEKEN_XTERM and TEKEN_UTF8 in your kernel configuration file. All non-ASCII characters will still be displayed as a yellow/red question mark. Be sure to replace cons25 with xterm in /etc/ttys!
Right now the input layer only has 8-bit keymaps. This should be changed to Unicode. XXX: Extend.
Because Syscons (and the hardware!) is restricted to 8-bit fontmaps, we should add a Unicode-to-8-bit fontmap conversion. An advantage is that we should be able to display exactly 256 different characters now, instead of 240-250 (not sure). We could even map multiple Unicode characters to the same graphical character (German Eszett and Beta?).
We could consider writing only a very small amount of fonts and some kind of file format, describing which characters should be included in the 8-bit map. This means we can generate a lot of fontmaps. I guess it's a little more tricky to add support for full width characters, because these should consume two graphical characters. Maybe we can save some space by just rendering full-width ASCII characters as regular characters, followed by a space?
This is still very premature. WIP. TBD.
In order to make integration into the operating system as easy as possible, we should at least make UTF-8 support a compile-time switch. Users/developers may still want to migrate their key/fontmaps to the new format; POLA.
To verify that the project has reached its goals and met all requirements performing the following is necessary: