We used 16-bit variables to store the 'character code' everywhere but
this won't let us represent anything beyond U+FFFF.
This patch changes those variables to a custom type that can be 32 or 16
bits depending on the build, and adjusts numerous internal APIs and
datastructures to match. This includes:
* utf8decode() and friends
* font manipulation, caching, rendering, and generation
* on-screen keyboard
* FAT filesystem (parsing and generating utf16 LFNs)
* WIN32 simulator platform code
Note that this patch doesn't _enable_ >16bit unicode support; a followup
patch will turn that on for appropriate targets.
Appears to work on:
* hosted linux, native, linux simulator in both 16/32-bit modes.
Needs testing on:
* windows and macos simulator (16bit+32bit)
Change-Id: Iba111b27d2433019b6bff937cf1ebd2c4353a0e8
We used 16-bit variables to store the 'character code' everywhere but
this won't let us represent anything beyond U+FFFF.
This patch changes those variables to a custom type that can be 32 or 16
bits depending on the build, and adjusts numerous internal APIs and
datastructures to match. This includes:
* utf8decode() and friends
* on-screen keyboard
* font manipulation, caching, rendering, and generation
* VFAT code parses and generates utf16 dirents
* WIN32 simulator reads and writes utf16 filenames
Note that this patch doesn't _enable_ >16bit unicode support; a followup
patch will turn that on for appropriate targets.
Known bugs:
* Native players in 32-bit unicode mode generate mangled filename
entries if they include UTF16 surrogate codepoints. Root cause
is unclear, and may reside in core dircache code.
Needs testing on:
* windows simulator (16bit+32bit)
Change-Id: I193a00fe2a11a4181ddc82df2d71be52bf00b6e6
Making splash -> splashf means its arguments are now checked
at compile time, but the "format" is nearly
always one of our virtual pointers instead of a string/format
literal.
Our gcc494 (and upcoming gcc950) toolchains handle this fine,
but simulator builds with gcc7 complain about this, so suppress
the warning for those environments.
(GCC 12, GCC 14, and GCC15 all seem to be okay with this too)
Change-Id: Ifaf061f14e2552db73a7515f61950ad83116e8b5
characters less than the first diacritic in the symbol table (0x300)
return false after checking the MRU table
we gain some performance by eliding the function call all together if less than first diacritic
Change-Id: I02c14e350eb168eca808523affad443cd43888b4
Tested on MacOS Sequoia (Apple Silicon) with the
latest Xcode command line tools, gcc 14
(Homebrew GCC 14.2.0_1) and sdl2 (Homebrew 2.30.9)
Make sure 'gcc' (and 'gcc-ar') is in your PATH
ahead of the Xcode-provided "gcc"(clang). E.g.
by setting up symlinks in /usr/local/bin that
point to gcc-14 and gcc-ar-14.
Notes:
- The appropriate bmp from uisimulator/bitmaps
has to be manually copied to your build folder
and renamed to UI256.bmp, if you want the sim
background to be displayed
Change-Id: I559f33d2165065f913f30c016b85906af380fb81
This reverts commit dcec6828a3.
This did not solve the reported problem, and it turns out that the original
code did the right thing with respect to the device's logical sector size.
Change-Id: I5bed2520ba8f5ca01df0ddd71588d3ed69c1d77c
The file I/O code operates in terms of the FILESYSTEM sector size, which
may be much larger than the underlying device's logical sector size. So
the disk cache buffers need to be the the larger of:
* Max allowed filesystem sector size (MAX_VIRT_SECTOR_SIZE) (if defined)
* Max allowed logical sector size (MAX_VARIABLE_LOG_SECTOR) (if defined)
* Fixed logical size (SECTOR_SIZE)
Change-Id: Ica0ec36d113406f24c12c696317a8d3520e7852c
This isn't strictly needed for FAT32, but the core file cache code
needs to be able to reference >32bit sector addresses.
Change-Id: I57838f1228c1d45f6a8c4755c5d1f9063c13b3dd
When enabled this allows 512n and 4Kn drives to be used with a single build.
(So far all ATA SSDs use 512 byte logical sectors)
Change-Id: I902d2318ca8abb581699c0bca68d6e3ec227d064
* Create new 'sector_t' type alias:
* uint64_t for all targets with HAVE_LBA48 or HAVE_SDUC
* unsigned long for the everything else
* Alter all storage APIs to use sector_t instead of 'unsigned long'
* Alter Volume/Partition/storage info structures to use sector_t
* Disk cache converted to sector_t
* ATA Core:
* convert to using sector_t for sector addresses and drive sizes
* Always fill out upper 16 bits of LBA48 addresses
* IDENTIFY INFO is fixed at 512 bytes, not SECTOR_SIZE
* USB mass storage:
* convert to using sector_t for sector addesses and drive sizes
* Implement READ_16/WRITE_16 for LBA48 addresses
* Convert FAT code to use sector_t for all sector references
* output_dyn_value() now accepts int64_t instead of 'int'
* Corrected "rockbox info" to work for (MULTIVOLUME & !MULTIDRIVE)
* Better reporting of disk and (logical+physical) sector sizes in debug info
* Detect SDUC cards and report on storage debug_info screen
To-do: SDUC
* Refactor SD core to remove duplicate code in every driver
* Card probe and init state machine
* Implement core SDUC support
* SD2.0 needs to be 2.0+ (fixed for jz47xx and x1000)
* Host and Card ID (ACMD41)
* 32-bit addressing for all read/write/erase operations (CMD22)
* ADD SDUC to target device drivers, defining HAVE_SDUC as appropriate
Change-Id: Ib0138781a0081664d11511037685503df1b93608
v1 passes the drive and partition number of the boot volume
instead of using the volume number. The volume number isn't
reliable because the same filesystem might get a different
volume number once the firmware is loaded, which will cause
the firmware to use the wrong root volume and fail to locate
the correct .rockbox directory.
Using drive and partition numbers avoids this issue because
drive numbering is fixed and determined by the target.
Change-Id: I7e68b892d9424a1f686197a6122e139b438e5f7e
Instead of verifying the CRC before every access of the boot data,
verify the CRC once at startup and set a flag to indicate the boot
data is valid.
Also add a framework to support multiple boot protocol versions.
Firmware declares the maximum supported protocol version using a
version byte in the boot data header. The bootloader chooses the
highest version supported by it and the firmware when deciding
what boot protocol to use.
Change-Id: I810194625dc0833f026d2a23b8d64ed467fa6aca
When the sd card is mounted into the root namespace the database
picks up files through both paths
previously we hid the mounted drive but this causes issues with users
databases when the drive letter changes
Adds a way to keep track of volumes mounted in the root namespace
Hides the enumerated volume in root
Database:
we can just parse the root directory ('/') and get to any mounted
volume but we can also enumerate a volume in the root directory
when this occurs it leads to multiple entries since the files can
be reached through multiple paths ex, /Foo could also be /SD1/Foo
Instead we will attempt to rewrite the root with any non-hidden volumes
failing that just leave the paths alone
Change-Id: I7bdba8cfaf63902d2a3852d28484bcf8ca317ebd
These are more efficient than separate pin/unpin calls because
pin count increment and decrement can be done cheaply when the
data pointer is known.
Secondly, pinned access can be made safe against preemption by
hardware interrupts or other CPU cores; buflib_get_data() can't.
This makes it more useful under different threading models and
for SMP targets; both of which are not particularly relevant to
Rockbox now, but might be in the future.
Change-Id: I09284251b83bbbc59ef88a494c8fda26a7f7ef26
This is intended for improving the effectiveness of tools like
ASAN when debugging memory errors in the sim. It's not meant to
be a serious allocator for hosted targets.
Enable it by changing the buflib backend in config.h.
Change-Id: I0cf23cefa47ee35dede7b49e0e5b72dac60e8d3e
To minimize code duplication between buflib backends move the
public part of the API to buflib.h. Also rewrote documentation
for the whole API.
Change-Id: I4d7ed6d02084d7130cb41511e63c25ec45b51703
Gate buflib_get_data() checking, debug printing, and buflib
integrity checks behind individual defines in buflib.h, and
turn them all off by default. If needed, they can be turned
on manually when compiling.
The buflib debug menu is only available if debug printing is
enabled, so after this commit it will no longer be included
in normal builds -- it isn't very useful to end users.
Change-Id: Iab25b7852bc7c5592ce04c9c45762046a87d5bc3
The CRC is a fairly useless safety check because we already
have specific checks to validate the metadata, and CRCs are
only verified before calling the move callback. Removing the
check should not significantly reduce buflib's robustness.
Change-Id: Ica99bd92fc514819b4fd9f359b4272e581020f75
chunk_alloc allows arrays (or any data) to be allocated in smaller chunks
you have to save the indices..
variable data will have variable indices you need to
store these as [chunk_alloc] doesn't keep track
if you have a fixed size for each
alloc you can do indice / sizeof(data)
or index * sizeof(data) to convert
Lots of debug stuff still in and it needs optimization
User provides chunk_size and max_chunks
max_chunks * struct chunk will be allocated at start
with (1) chunk_size allocation initially
alloc_chunk() with size = 0 shrinks the last allocation to the size of the data used
add OOM checks on buflib_alloc -- oops
move bytes available to the header -- less memory per chunk & better alignment
keep track of the current in use chunk index -- should speed things up a bit
Now allows:
realloc chunk header
larger allocations than chunk size
reallocs smaller than existing will shrink the current array
rather than alloc a new and copy data
Comments welcome :)
Change-Id: I8ed170eef73da95da19430a80b32e5debf0c8276
HAVE_MULTIVOLUME is a subset of HAVE_MULTIDRIVE
in normal circumstances they are interchangable
but when the internal drive is not found the assumption falls apart
at the moment most of the bootloader are less selective about what drives/volumes
they will mount therefore the volume might not match between what the bootloader
returns for multiboot and what the firmware sees
updating bootloaders will fix this but it should probably be made more robust
at the same time
Change-Id: I93acd4a539894f093211b74b030df3b2c6a0aa11
replace applicable calls to strlcpy with calls to strmemccpy
which null terminates on truncation
in theory the strmemccpy calls should be slightly faster since they
don't traverse the rest of the source string on truncation
but I seriously doubt there is too much of that going on in the code base
Change-Id: Ia0251514e36a6242bbf3f03c5e0df123aba60ed2
splits on spaces also considers \r\n\f\v\t as mandatory breaks
I'm still working on the strptokspn function
my goal is to use it directly rather than storing the matched char
and modifying the source string with \0 in order to tokenize the output
--Done
Change-Id: I7f378b5b9c4df8f10899b9a55a98950afb3931dc
An allocation is pinned by calling buflib_pin() to up its pin count.
The pin count is like a reference count: when above 0, buflib won't
move the allocation and won't call its move callbacks. This makes it
safe to hold the pointer returned by buflib_get_data() across yields
or allocations.
Note that pinned allocations can still shrink because there are some
use cases where this would be valid, if buffer users coordinate with
the shrink callback.
Change-Id: I0d0c2a8ac7d891d3ad6b3d0eb80c5b5a1b4b9a9d
Using a length 1 char array to point to the name buffer triggers
a -Warray-bounds warning from GCC when fortified strcpy is used.
This type of construct isn't safe in general -- if the compiler
makes assumptions based on the array bound it can create subtle
bugs when accessing the array out of bounds.
Instead, add a function get_block_name() which returns a pointer
to the name field by casting. This suppresses the warning and it
should be a bit more portable.
Change-Id: I25d4f46f799022ad0ec23bef0218f7595cc741ea
These don't have any users and there is already another way to
print blocks (which is actually used by the debug menu).
Change-Id: Ic6a4f874c6499c42bc046e8af3e4aaddc9e68276
Define common functions for loading 16/32/64-bit unsigned integers
with big, little, or host endianness, and distinguishing unaligned
and aligned cases.
Unaligned loads are supported generically by default, but this can
be overridden with a more efficient implementation on architectures
which support unaligned loads natively.
Change-Id: I3d826ec1a7646777876366eeece2cbccab60c1fb
