Differences

This shows you the differences between two versions of the page.

--- summerofcode2014:unicode [2014/05/19 20:19] – DmitrySelyutin
+++ summerofcode2014:unicode [2015/06/05 16:48] (current) – Add Code Review link PedroGiffuni
@@ Line 3: / Line 3: @@
   * Mentor: [[/PedroGiffuni|PedroGiffuni]] (pfg@)
-===== Project description =====
+===== Description =====
-There are lot of functions that are ethier unimplemented or buggy or may be significantly improved. The most known functions that need an implementation are strcoll and strcoll_l (being the reason why some projects use external libraries like %%PostgreSQL%% uses ICU). Functions that may need reimplementation are for example wc/mb and uchar.h-related functions; their support can be significantly improved to be at least as fast as they are in glibc or ICU library.
+There are lot of functions in libc that are ethier unimplemented or buggy or may be significantly improved. The most known functions that need an implementation are strcoll and strcoll_l (being the reason why some projects use external libraries like %%PostgreSQL%% uses ICU). Functions that may need reimplementation are for example wc/mb and uchar.h-related functions; their support can be significantly improved to be at least as fast as they are in glibc or ICU library.
-===== Approach to solving the problem =====
+===== Approach =====
 Implement string collation algorithm as discussed in UTS#10.
 Fix various bugs in libc Unicode support.
@@ Line 13: / Line 13: @@
 Implement strcoll and strcoll_l functions, improve wc/mb functions as well as C11 and POSIX standart functions. One of the goals of this project is to let some projects like %%PostgreSQL%% to be built without ICU as dependency (thus strcoll and strcoll_l are the absolutely necessary functions). However, I personally treat this project as a possibility to provide better Unicode support everywhere where we can.
-===== Milestones =====
+===== Checkpoints =====
-  * 26 March – 15 May: investigate strcoll implementation that was written by Konrad Jankowski; if it works, it must be modified according current library, since it predates xlocale.h; if it is unusable, we have to implement it from scratch, so during this period I propose mainly reading documentation and existing sources.
+Functions for normalization and canonicalization are implemented in source files under lib/libc/unicode directory.
-  * 15 May – 27 June: actual strcoll and strcoll_l implementation that is correct following Unicode standart. It must work for at least UTF-8 encoding; probably it would be better to make strcoll convert byte sequence to UCS-4 sequence and only then perform actual comparison. We must also decide if we want to support diacritics, ligatures, etc.; since I think that we must follow the latest Unicode standart we have to support it. The main goal for this part is to allow %%PostgreSQL%% to be built without ICU dependency.
+New collation database interface was provided in order to extend collation support as well as its Python bindings.
-  * 27 June – 22 August: fix broken or buggy Unicode functions; improve perfomance everywhere where possible.
+Right now source code is placed under lib/libcolldb directory, though it's likely to be moved to some other place.
-===== Test Plan =====
 ===== The Code =====
+https://socsvn.freebsd.org/socsvn/soc2014/ghostmansd
+https://reviews.freebsd.org/D2736
 ===== Useful links =====
 http://www.unicode.org/reports/tr10
-http://www.unicode.org/reports
+http://www.unicode.org/unicode/reports/tr15
+http://userguide.icu-project.org/collation/api
+http://source.icu-project.org/repos/icu/icuhtml/trunk/design/collation/ICU_collation_design.htm
 https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt