len0rd/rockbox
1
0
Fork 1
mirror of https://github.com/Rockbox/rockbox.git synced 2026-10-10 08:03:04 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

flac: decode residuals that use all 32 bits

Browse source 
The folded Rice value was unfolded with a signed shift, which is wrong
once it reaches 2^31, and the unary length limit was (INT_MAX >> k) + 2,
about half of what a 32-bit residual can need. Streams with very large
residuals (FLAC decoder testbench file 63) were misparsed, overran the
frame and lost sync at the next one. Unfold as unsigned and derive the
limit from UINT_MAX, clamped to INT_MAX. The fast path is unchanged.

The existing 0x80000000 error check now also works as intended, since
the Golomb reader's error value maps to it. The FLAC spec forbids a
residual of -2^31.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Change-Id: I0bdce9db69e1b30565ecc23126923ba401b7deca
This commit is contained in:
Michael Giacomelli 2026-09-19 16:16:45 -04:00 • committed by Solomon Peachy
parent deaef503bb
commit 330236911b
2 changed files with 6 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

3
lib/rbcodec/codecs/libffmpegFLAC/decoder.c
View file

@ -165,7 +165,8 @@ static int decode_residuals(FLACContext *s, int32_t *decoded, int pred_order)
for (; i < samples; i++)
*decoded++ = tmp ? get_sbits(&gb, tmp) : 0;
} else {
int real_limit = tmp ? (INT_MAX >> tmp) + 2 : INT_MAX;
unsigned lim = tmp ? (UINT_MAX >> tmp) + 2 : INT_MAX;
int real_limit = lim > INT_MAX ? INT_MAX : lim;
for (; i < samples; i++) {
int v = get_sr_golomb_flac(&gb, tmp, real_limit, 0);
if ((unsigned) v == 0x80000000){

7
lib/rbcodec/codecs/libffmpegFLAC/golomb.h
View file

@ -85,7 +85,7 @@ static inline int get_ur_golomb_jpegls(GetBitContext *gb, int k, int limit,
buf = 0;
}
buf += ((int32_t)i << k);
buf += ((unsigned int)i << k);
} else if (i == limit - 1) {
buf = SHOW_UBITS(re, gb, esc_len);
LAST_SKIP_BITS(re, gb, esc_len);
@ -103,8 +103,9 @@ static inline int get_ur_golomb_jpegls(GetBitContext *gb, int k, int limit,
* read signed golomb rice code (flac).
*/
static inline int get_sr_golomb_flac(GetBitContext *gb, int k, int limit, int esc_len){
int v= get_ur_golomb_jpegls(gb, k, limit, esc_len);
return (v>>1) ^ -(v&1);
/* The folded value can use all 32 bits, so unfold it as unsigned. */
unsigned int v= get_ur_golomb_jpegls(gb, k, limit, esc_len);
return (int)((v>>1) ^ -(v&1));
}
/**