One decode attempt of 9 bytes of bad CBOR data can exhaust memory (UnmarshalCbor in dots_common/messages/message.go)

[x448]

@lieunguyen-tma go-dots is using a CBOR library that could exhaust memory in 1 decode attempt of 9-10 bytes of malformed data.

Relevant Code

go-dots/dots_common/messages/message.go

import (
...
	"github.com/ugorji/go/codec"
...
)
...
func UnmarshalCbor(pdu *libcoap.Pdu, typ reflect.Type) (interface{}, error) {
    ...
    m := reflect.New(typ).Interface()
	d := codec.NewDecoderBytes(pdu.Data, dots_common.NewCborHandle())
    err := d.Decode(m)
    ...

Error (fatal error: out of memory)

For info about CBOR and security, see Section 8 of RFC 7049 (Security Considerations).

For more comparisons, see fxamacker/cbor.

How to Reproduce Problem

To reproduce the problem, attempt to decode 9-10 bytes of malformed CBOR data described in Section 8 of RFC 7049 using nttdots/go-dots function:
func UnmarshalCbor(pdu *libcoap.Pdu, typ reflect.Type) (interface{}, error)

Examples of CBOR data that can exhaust memory can be found on GitHub since Sep 2019 (possibly a lot earlier if you look beyond Go projects).

Background

RFC 7049 was published in 2013 with Section 8 warning of malformed CBOR data being used to exhaust system resources.

In Sep 2019, oasislabs/oasis-core discovered tiny malformed CBOR data can exhaust memory and traced the problem to the same CBOR library (ugorji/go) being used by nttdots/go-dots. They fixed the problem by switching to a more secure CBOR library.

In Feb 2020, smartcontractkit/chainlink had a CBOR security issue involving ugorji/go which was fixed by a GitHub PR titled "Switch to more secure CBOR library".

Decoding 9 bytes of bad CBOR data shouldn't exhaust memory.

[x448]

@lieunguyen-tma updated with more info about vulnerability in nttdots/go-dots.