len0rd/rockbox
1
0
Fork 1
mirror of https://github.com/Rockbox/rockbox.git synced 2026-04-11 16:37:45 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

utf8proc: Add utf8proc for normalizing UTF-8 strings

Browse source 
Due to the unicode data tables, this is a pretty large library,
adding ~340K to the binary size (ie a 50% increase on some targets)

Note that nothing in-tree actually _uses_ utf8proc yet.  The plan:

 1) In the metadata parser, fully normalize+compose all strings.
 2) When looking up font glyphs, if we do not find an exact
    match for the codepoint, try to decompose and render the result.
 3) Normalize all filenames as we read them, ie before we store them
    in lists or compare filenames/pathnames.
 4) Use utf8proc for casefolding-aware sorting (on native devices)

Change-Id: I2252cee3e7891d12ff37867231aa2c20353f857b
This commit is contained in:
Solomon Peachy 2024-10-13 08:49:31 -04:00
parent ad45b2910a
commit 4209170cb7
10 changed files with 19319 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/CREDITS
View file

@ -801,3 +801,4 @@ The xrick team
The Terminus Font team
Attila Tarpai (The RAINBOW image decoder lib author)
The WenQangYi team
The utf8proc team

68
docs/LICENSES
View file

@ -165,3 +165,71 @@ From: https://git.alsa-project.org/?p=alsa-utils.git
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
*************************************************************************
In: lib/utf8proc
From: https://github.com/JuliaStrings/utf8proc.git
*************************************************************************
# utf8proc license ##
**utf8proc** is a software package originally developed
by Jan Behrens and the rest of the Public Software Group, who
deserve nearly all of the credit for this library, that is now maintained by the Julia-language developers. Like the original utf8proc,
whose copyright and license statements are reproduced below, all new
work on the utf8proc library is licensed under the [MIT "expat"
license](http://opensource.org/licenses/MIT):
*Copyright © 2014-2021 by Steven G. Johnson, Jiahao Chen, Tony Kelman, Jonas Fonseca, and other contributors listed in the git history.*
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
## Unicode data license ##
This software contains data (`utf8proc_data.c`) derived from processing
the Unicode data files. The following license applies to that data:
**COPYRIGHT AND PERMISSION NOTICE**
*Copyright (c) 1991-2007 Unicode, Inc. All rights reserved. Distributed
under the Terms of Use in http://www.unicode.org/copyright.html.*
Permission is hereby granted, free of charge, to any person obtaining a
copy of the Unicode data files and any associated documentation (the "Data
Files") or Unicode software and any associated documentation (the
"Software") to deal in the Data Files or Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, and/or sell copies of the Data Files or Software, and
to permit persons to whom the Data Files or Software are furnished to do
so, provided that (a) the above copyright notice(s) and this permission
notice appear with all copies of the Data Files or Software, (b) both the
above copyright notice(s) and this permission notice appear in associated
documentation, and (c) there is clear notice in each modified Data File or
in the Software as well as in the documentation associated with the Data
File(s) or Software that the data or software has been modified.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS
INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR
CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF
USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THE DATA FILES OR SOFTWARE.
Except as contained in this notice, the name of a copyright holder shall
not be used in advertising or otherwise to promote the sale, use or other
dealings in these Data Files or Software without prior written
authorization of the copyright holder.
Unicode and the Unicode logo are trademarks of Unicode, Inc., and may be
registered in some jurisdictions. All other trademarks and registered
trademarks mentioned herein are the property of their respective owners.