Reject NaN before exporting evaluation results

RFC 8259, published in December 2017, excludes NaN and infinity from JSON numbers. Python's JSON encoder can nevertheless emit those tokens with its default settings. A floating-point result can therefore survive your evaluation code and fail at the next system's parser.

Reject NaN before exporting evaluation results. My rule is to make strict serialisation a separate acceptance gate after scoring, even when the object already passed ordinary Python type checks. “It is a float” and “another service can consume it as JSON” are different promises.

Why Python can emit non-JSON numbers

The Python JSON documentation describes this behaviour explicitly. The default encoder accepts non-finite floating-point values, and its decoder accepts the corresponding spellings too. A round trip through that same library can therefore miss an interoperability defect.

The JSON number grammar supplies the boundary: these values are not permitted. This is a format constraint, separate from the question of what an undefined score means to your evaluation.

An empty slice, an overflow or an upstream numeric operation can leave you with a non-finite result. The producer should decide how to represent that state instead of letting a permissive encoder make the decision accidentally.

Reject NaN at the export boundary

This small Python helper rejects non-finite values, including ones nested inside the result object. It does not coerce them to zero or hide the failure.

import json

def strict_result(value):
    return json.dumps(value, allow_nan=False)

assert json.loads(strict_result({"score": 0.75, "count": 4})) == {
    "score": 0.75, "count": 4
}
for bad in [float("nan"), float("inf"), -float("inf")]:
    try:
        strict_result({"score": bad})
    except ValueError:
        pass
    else:
        raise AssertionError("non-finite result accepted")

The finite fixture has two fields so the check exercises an actual result object. The three rejected values cover NaN and both infinity signs. Run the snippet in your supported interpreter as part of the export tests, not only in a notebook where you inspect the printed object.

The synthetic export check

The helper and fixtures were executed with Python 3.14.6. The finite object round-tripped unchanged. Each of the three non-finite inputs raised ValueError. A separate control confirmed that the default encoder emitted the NaN spelling.

Those are local format checks. They do not measure an evaluation system's accuracy, validate a production endpoint or establish that a finite score is meaningful. A perfectly serialisable number can still use the wrong denominator.

Undefined scores need a domain meaning

For a hypothetical retrieval evaluation, I would keep the metric value, eligible-case count and reason for an unavailable result together. If a slice has no eligible cases, an explicit unavailable state is more informative than either zero or a silently emitted NaN.

Do not simply replace every non-finite number with null in a generic cleanup pass. That may be the right wire representation, but only if the schema and downstream calculation agree on its meaning. Otherwise, the chart may quietly drop the case and make the aggregate look better.

There are further boundaries. Strict encoding does not reject every undesirable object shape, enforce numeric ranges or detect duplicate fields in incoming text. It also does not stop an incoming parser from accepting non-standard constants; that is a separate decoder setting and test.

What's in it for you

• Add NaN and both infinity signs to result-export fixtures.
• Give undefined metrics an explicit domain meaning before serialising them.

A score is ready to leave Python only when its meaning and its wire format both survive the handoff.