This is an old revision of the document!
Improve Unicode support in FreeBSD
- Student: DmitrySelyutin (ghostmansd@)
- Mentor: PedroGiffuni (pfg@)
Project description
There are lot of functions in libc that are ethier unimplemented or buggy or may be significantly improved. The most known functions that need an implementation are strcoll and strcoll_l (being the reason why some projects use external libraries like PostgreSQL uses ICU). Functions that may need reimplementation are for example wc/mb and uchar.h-related functions; their support can be significantly improved to be at least as fast as they are in glibc or ICU library.
Approach to solving the problem
Implement string collation algorithm as discussed in UTS#10. Fix various bugs in libc Unicode support.
Deliverables
Implement strcoll and strcoll_l functions, improve wc/mb functions as well as C11 and POSIX standart functions. One of the goals of this project is to let some projects like PostgreSQL to be built without ICU as dependency (thus strcoll and strcoll_l are the absolutely necessary functions). However, I personally treat this project as a possibility to provide better Unicode support everywhere where we can.
Milestones
- 26 March – 15 May: investigate strcoll implementation that was written by Konrad Jankowski; if it works, it must be modified according current library, since it predates xlocale.h; if it is unusable, we have to implement it from scratch, so during this period I propose mainly reading documentation and existing sources.
- 15 May – 27 June: actual strcoll and strcoll_l implementation that is correct following Unicode standart. It must work for at least UTF-8 encoding; probably it would be better to make strcoll convert byte sequence to UCS-4 sequence and only then perform actual comparison. We must also decide if we want to support diacritics, ligatures, etc.; since I think that we must follow the latest Unicode standart we have to support it. The main goal for this part is to allow PostgreSQL to be built without ICU dependency.
- 27 June – 22 August: fix broken or buggy Unicode functions; improve perfomance everywhere where possible.