devs/foxbox
1
0
Fork 0
forked from len0rd/rockbox
Code Pull requests Activity

libmad: Use 32bit unsigned for requantize table.

Browse source 
Implicit promotion of integer literals to unsigned long introduced a subtle bug
on 64-bit systems due to weird sign extensions (leads to audible glitches in a
few files). The table is originally designed for unsigned 32bit integers, and
it works with those so use them. As a consequence the lookup table size is
halved as well.

Change-Id: I35d878d6df03300387f0e403e0f3c3bdc73eea00
This commit is contained in:
Thomas Martitz 2014-04-15 23:49:07 +02:00
parent bd10245e89
commit 466441dc14
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
lib/rbcodec/codecs/libmad/layer3.c
View file

@ -339,7 +339,7 @@ unsigned char const pretab[22] = {
* format rq_table: bit31-27=exponent bit26-0=mantissa * format rq_table: bit31-27=exponent bit26-0=mantissa
*/ */
static static
unsigned long const rq_table[8207] = { uint32_t const rq_table[8207] = {
# include "rq_table.dat" # include "rq_table.dat"
}; };
@ -885,7 +885,7 @@ mad_fixed_t III_requantize(unsigned int value, signed int exp)
{ {
mad_fixed_t requantized; mad_fixed_t requantized;
signed int frac; signed int frac;
unsigned long power; uint32_t power;
frac = exp % 4; /* assumes sign(frac) == sign(exp) */ frac = exp % 4; /* assumes sign(frac) == sign(exp) */
exp /= 4; exp /= 4;