Re: [OSS] Re: To Do: Localization of PHP [MySQL, PostgreSQL, etc]

Dear Stefan,
Many thanks for the recent flood of useful information on OSS. I have read them
all, but I felt that policy issues are better left to bureaucrats to sort them
out !

The localisation issue is not a simple task to accomplish if we also aim at
supporting Unicode or UTF-8. The reasons are explained below.

Stefan Probst wrote:
>
> sorting is a very common function in mySQL. I would select something from a
> table and sort it, before displaying the result on a webpage.
> (SELECT * FROM table SORT BY field)
> A very simple example: a list with names, e.g. a payroll. Right now, mySQL
> cannot sort by name in that example, because it cannot handle the
> Vietnamese characters correctly.
>
RDBMS performs the sorting tasks using a table called collating sequence (ie.
the sequential order of characters in the character set). One must define the
collating sequence in advance. In this case we have more complication due to
Unicode implementation. It is probably easier if we use single byte
characterset, like TCVN-ABC. There are facilities in most RDBMS to help define
the new collating sequence. We have here an issue using Unicode or UTF-8. In
order to use Unicode, the RDBMS must accept 16-bit characterset or locale. In
other words, it must support 16-bit implementation. Professional RDBMS's like
Oracle, IBM Informix, DB2 do have 16-bit implementation. We have used MySQL and
PostgreSQL in a number of real life systems but we have never tried to install
them in 16-bit locale. I am not sure that they support 16-bit. If one tries to
use UTF-8 (a multi-byte characterset), one will not be able to define the
collating sequence in 8-bit width. This will require extensive programming in
the kernel to recognise and rank multi-byte characters.
Sorry to tell everyone a very difficult issue, but we have had extensive
experience in performing Vietnamese localisation for Oracle and IBM Informix
RDBMS in Vietnam since 1996.
There is still a wide gap in performance between
FLOSS RDBMS and professional ones ! Please do not expect that MySQL and
PostgreSQL perform like professional RDBMS. If one considers other issues like
backup, recovery, replication, distributed, logical and physical logging,
migration, parallel processing, EJB, etc ... One will appreciate their
limitations.
MySQL and PostgreSQL are only suitable for small and non-mission critical
departmental systems. They are also much slower than professional RDBMS. I am
not trying to advertise for professional RDBMS, but one should know the
limitations in FLOSS RDBMS when exploiting them.

Furthermore, one needs to perform localisation for all string related functions
like Date, Time, Currency, String length, etc ... and associated utilities like
SQL Editor, etc .. to accept and process Unicode. It is a huge project.

There is no doubt that "Moving to Unicode" is the right way eventually, but one
should study and evaluate its consequences, options and migration techniques.
This is a technical and economic decision, not just a pure political sensation !
I have not seen a single document that studied and offers solutions for: moving
a massive amount of code to support Unicode, moving FoxPro DB, Access, Oracle,
Informix to Unicode. And there is no study of its possible migration costs and
funding arrangements.
Is there anyone interested in discussing Unicode Migration issues and techniques
?

> Of course, you can get around with a LOT of extra coding, i.e. do all your
> sorting in own SLOW routines, instead of using the built-in functionality,
> but this means, that things are getting more slow, need more resources, and
> localizations are not that easy anymore. Try to enter Vietnamese user names
> in phpNuke and then sort them ....

Yes, this is a poor man implementation option. And it will give lots of
performance problems.
>
> Stefan
>

Best regards
Viet Tran

To facilitate co-ordination regarding the introduction of OSS SW in Vietnam

Subscribe to OSS:

Subscribe | Unsubscribe