Skip to content

Reject symbol IDs and VarInts that overflow int in the binary reader - #1173

Open
radhika1314 wants to merge 1 commit into
amazon-ion:masterfrom
radhika1314:binary-reader-int-overflow
Open

radhika1314 wants to merge 1 commit into
amazon-ion:masterfrom
radhika1314:binary-reader-int-overflow

Conversation

@radhika1314

Copy link
Copy Markdown

Issue #, if available: n/a

Description of changes:

The Ion 1.0 binary reader decodes VarUInt and VarInt sub-fields into a Java int without checking that they fit, and narrows field name and symbol value IDs from long with a plain cast. A 5-byte VarUInt of 2^32 + 4 wraps to 4, so an out-of-range symbol ID used as a field name, annotation or symbol value resolves to $4 (name) instead of failing the way fieldNameSymbolIDUnmapped.10n and annotationSymbolIDUnmapped.10n in ion-tests require. The same wrap turns a timestamp year of 2^32 + 2000 into 2000T, and a decimal exponent of 2^31 + 1 into -(2^31 - 1), so 1d2147483649 reads back as 1d-2147483647. Two Ion implementations can end up disagreeing about the contents of the same document.

Reproducer for the field name case (the others are covered by the new tests):

e0 01 00 ea d7 10 00 00 00 84 21 01

Before: {name: 1}. After: IonException: Found a symbol ID that was too large to fit in an int.

The checks live in the sub-field readers so that every caller (field names, annotations, symbol values, timestamp fields, decimal exponents) is covered in one place: readVarUInt_1_0 bounds its accumulator, readVarInt_1_0 accumulates in a long with a 2^31 magnitude bound and the decimal scale and timestamp offset are range checked before narrowing, and IonCursorBinary.symbolIdToInt rejects symbol IDs outside the int range. The VarInt bound still admits an exponent of exactly 2^31, so a BigDecimal with scale Integer.MIN_VALUE remains readable.

One side effect worth a look: IonRawBinaryWriter.writeDecimalValue computes the exponent as -value.scale() in int arithmetic, so a scale of Integer.MIN_VALUE is written as exponent -2^31 and only round tripped because the reader overflowed the same way. That encoding now fails to read back. I kept this change on the read side, but can fold the writer fix in here if you would rather have both together.

Regression tests were added to IonReaderContinuableCoreBinaryTest and IonReaderContinuableTopLevelBinaryTest; ./gradlew build and minifyTest21 pass locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant