This issue started in December 2023, and…[Read more]

Reading a jjwt issue and opening a new PR

Introduction

Hello. Today I brought along issue #877 (Option to enable strict duplicate detection), which was opened in December 2023. This post is a record of how jjwt and Jackson behave, and of how I ended up opening a PR.

jjwt and jackson

If you do backend work, jjwt and Jackson are probably both familiar. Spring Boot uses Jackson as its default JSON library, which makes it more so, and jjwt is literally java-jwt, so even without knowing the library you can tell it has something to do with JWT.

Still, this story happens on the boundary between the two, so let me briefly cover what each of them does.

First, JWT. It is a format that lets a server identify a logged-in user from a single token instead of looking up a session every time. It looks like three pieces separated by two dots: a header, a payload and a signature. The header and the payload are just JSON encoded in Base64, and the signature is the header and payload locked together by a hash algorithm. Tamper with the contents and verification catches it. Put the other way round, the contents themselves are ordinary JSON that anyone can open.

jwtk/jjwt is the library that builds and verifies those JWTs in Java. With over ten thousand stars it is the de facto standard in the Java world. It creates tokens, checks signatures, tells you whether a token has expired, and hands you the claims.

Jackson is the library that turns JSON into Java objects and objects back into JSON. If you use Spring Boot you already use it every day. When you put @RequestBody on a controller, Jackson is what turns the request body into a DTO.

The reason both show up here is that jjwt does not read JSON itself. It leaves parsing the token payload to an external library and focuses on the JWT format. Jackson is what most often gets plugged into that slot.

Back to the issue, then. Its request boils down to this. If the token payload contains the same name twice, let me reject it.

An issue… or is it?

It was opened on 4 December 2023 by a user named laurids, titled Option to enable strict duplicate detection. In other words, a request to let strict duplicate detection be switched on.

By default Jackson overwrites a duplicated name with whatever value came later.

{
  "k": "A",
  "k": "B"
}

→  {k=B}

Wondering whether some other rule such as ordering was at play, I fed the values in ascending and then descending order. Sorting had nothing to do with it; the winner was simply decided by the order written in the document.

fed in ascending order
{
  "k": "aaa",
  "k": "zzz"
}
→  {k=zzz}

fed in descending order
{
  "k": "zzz",
  "k": "aaa"
}
→  {k=aaa}

So what problem can this cause? Let me put real claims where k was.

{
  "sub":  "alice",
  "sub":  "attacker",
  "role": "user",
  "role": "admin"
}

→  sub=attacker   role=admin

The subject goes from alice to attacker, the role from user to admin, and Jackson allows it. Hence the request to make duplicate detection possible.

Yet Jackson already has that ability, through the JsonParser.Feature.STRICT_DUPLICATE_DETECTION option. Turn it on and it throws instead of overwriting.

JsonParseException: Duplicate field 'sub'

laurids, who opened the issue, mentions that option too.

This can be done by enabling JsonParser.Feature.STRICT_DUPLICATE_DETECTION on the underlying ObjectMapper. The problem is that there is no way to access the objectMapper in JacksonDeserializer.

You enable it on the underlying ObjectMapper, but there is no way to reach the objectMapper inside JacksonDeserializer. In short, a request to make an existing switch reachable.

Two replies follow. First bdemers answers: why not just build your own ObjectMapper and pass it in?

You can create a new instance of JacksonDeserializer, and customize the ObjectMapper

That constructor does already exist. Here are the three lines he linked to in his comment, opened at 0.12.3, the release current at the time.

// extensions/jackson/…/io/JacksonDeserializer.java · 0.12.3 · L91-93
public JacksonDeserializer(ObjectMapper objectMapper) {
    this(objectMapper, (Class<T>) Object.class);
}

Build an ObjectMapper with your settings, pass it here, and duplicate detection comes along with it. Three hours later the maintainer lhazlewood raises a far heavier objection.

It is unclear to me if we should change the default behavior of implicitly-created ObjectMapper instances since the RFCs explicitly allow this behavior. Assuming signature verification or AEAD decryption are successful before accessing the payload, what vulnerabilities are you referring to?

He is not sure the default behavior of implicitly created ObjectMapper instances should change, because the RFCs explicitly allow it. And assuming signature verification succeeds before the payload is read, what vulnerabilities are we even talking about?

It is a short exchange, but several things are left to check. What exactly do the RFCs say, such that "explicitly allow" comes up? Why was "just build your own ObjectMapper" not enough? And is it really dangerous even though signature verification comes first? Let me take them one at a time.

A closer look at the RFCs

The JSON standard itself only recommends that names be unique, but the JWT RFCs raise the bar. RFC 7515 §4 and RFC 7519 §4 repeat the same sentence.

The Header Parameter names within the JOSE Header¹ MUST be unique; JWS² parsers MUST either reject JWSs with duplicate Header Parameter names or use a JSON parser that returns only the lexically last duplicate member name.

There are two rules in one sentence. The producer must keep names unique, and the consumer must either reject or take the last value. So a token containing duplicates already violates the spec, yet for the parser reading it both options are legitimate.

Jackson's behavior — the later value wins — is what the RFC calls lexically last. It means last in the order written in the document, not in dictionary order, and the RFC cites a section of ECMAScript 5.1 precisely to remove that ambiguity.

And RFC 7515 §10.12 states outright that having two options is itself the danger.

Ambiguous and potentially exploitable situations could arise if the JSON parser used does not enforce the uniqueness of member names or returns an unpredictable value for duplicate member names.

A vulnerability born of ambiguity

The maintainer lhazlewood adds this. In DefaultJwtParser signature verification happens before claim parsing, so if a duplicate key reached the parser the token was already signed with a trusted key and an attacker could not have forged it.

The next day, however, laurids takes the threat in a different direction.

In general, I would think duplicate properties in a JWT is a sign of a buggy or compromised authorization server. … If multiple parsers are used in a stack including JJWT, and some parsers are wrongly implemented to use the first value instead of the last, that could lead to vulnerabilities. However, if JJWT rejects it, this type of vulnerability is not possible.

In short, the threat is not forging a token but reading the same token differently at each layer. A valid token and a healthy issuer are two different things. As evidence he attached Bishop Fox's analysis of JSON interoperability vulnerabilities and a Black Hat US-23 talk on attacks against JWTs.

In the same comment he also restates what he wanted.

Yes, I did notice the constructor taking an ObjectMapper. However, I wanted to use the MappedTypeDeserializer as well, since it was conveniently implemented there:) Maybe I was not clear enough about that part. So my request was for an option to configure the ObjectMapper, specifically when using the JacksonDeserializer(Map<String, Class<?>> claimTypeMap) constructor.

He had seen the constructor taking an ObjectMapper, but he wanted to use MappedTypeDeserializer as well. Put plainly, the request was to be able to configure the ObjectMapper while using the JacksonDeserializer(Map<String, Class<?>> claimTypeMap) constructor.

The ambiguity, removed

Two days later lhazlewood changes position.

A more targeted solution … would be to just default to enabling STRICT_DUPLICATE_DETECTION. I agree this would likely be a better default, and, since the RFC allows it, application developers can override/unset that feature if they desire.

The fix landed in commit 86e0655 on 17 January 2024, released as 0.12.4. Instead of building the requested API, it turned duplicate detection on in the default ObjectMapper.

static ObjectMapper newObjectMapper() {
    return new ObjectMapper()
            .registerModule(MODULE)
            .configure(JsonParser.Feature.STRICT_DUPLICATE_DETECTION, true) // issues/877
            ...
}

The requested constructor did go into the same commit. It was just locked.

//TODO: Make this public on a minor release
private JacksonDeserializer(ObjectMapper objectMapper, Map<String, Class<?>> claimTypeMap) {

That is semver. 0.12.x is a patch release, so growing the public API would break the contract; rather than open the requested API right away, they chose to flip the default.

A closer look at jjwt

The vulnerability born of ambiguity is gone, but a question remains. What does the claimTypeMap that laurids wanted actually do? To answer that we first have to look at how jjwt reads JSON.

jjwt cannot know in advance which claims a token will carry. The issuer decides that at runtime. So it reads with Object.class.

{
  "sub": "alice",
  "user": {
    "first": "Jill",
    "last":  "Coder"
  }
}

readValue(json, Object.class)
→ {sub=alice, user={first=Jill, last=Coder}}      ← user is just a Map too

The trouble is that this is inconvenient. To use user as a User object you have to pull values out of the map and assemble them by hand, and since the declared type is Object a cast appears at every level.

Map claims = readValue(json, Object.class);
Map um = (Map) claims.get("user");
User u = new User();
u.first = (String) um.get("first");
u.last  = (String) um.get("last");

claimTypeMap solves that inconvenience. Pass a name paired with a class, such as ["user": User.class], and as jjwt reads down the JSON it checks whether the name it is reading is on that list, and if so builds an object out of just that value.

// jjwt · JacksonDeserializer.MappedTypeDeserializer
public Object deserialize(JsonParser parser, DeserializationContext context) throws IOException {
    String name = parser.currentName();                        // ← the name of the current claim
    if (claimTypeMap != null && name != null && claimTypeMap.containsKey(name)) {
        Class<?> type = claimTypeMap.get(name);                // ← look the class up by name
        return parser.readValueAsTree().traverse(parser.getCodec()).readValueAs(type);
    }
    return super.deserialize(parser, context);                 // otherwise a Map, as usual
}

What actually happens in the middle line shows the nature of this feature. When a name matches, it buffers that value's subtree in memory, puts a fresh parser on top of that tree and reads it again to build the object. The type is only decided mid-read, so the stream cannot be rewound — hence that one fragment is read a second time.

This is also where it becomes clear that the decision rests on the claim name, not on the fields inside. Even with identical inner fields, a name that is not on the list simply yields a Map.

This convenience is what laurids could not give up. He wanted to keep the code that no longer assembles values out of a map by hand, and block duplicate keys at the same time.

Runtime and compile time

Which raises a question, doesn't it? In Spring Boot, using the very same Jackson, adding @RequestBody is enough for it to read in one go, like readValue(json, Dto.class).

So why does jjwt pin everything to Object.class instead of reading into a type the user defines from the start?

The difference is not the library but when the type is known.

@PostMapping("/api/orders")
public OrderResponse create(@RequestBody OrderRequest request) {
    // ↑ the type is pinned right here

A controller writes the type it will receive into the method signature. The contract is fixed at compile time, so Jackson fills the DTO as it reads the stream. jjwt, on the other hand, cannot know in advance which claims a token will carry, because the issuer decides that at runtime. So it reads with Object.class, and the result is not a POJO but a nested Map.

Spring behaves the same way when no type is stated.

@RequestBody OrderRequest request       →   OrderRequest
@RequestBody Map<String, Object> body   →   LinkedHashMap

The same mapper, the same bytes, different results. What separates them is not the library but whether a type was given.

You might think jjwt could offer an option to read into a type matched to the issuer, but there is a reason it cannot. Consider these two tokens.

A · the issuer included an expiry
{
  "sub": "alice",
  "exp": 1786000000
}

B · the issuer did not
{
  "sub": "alice"
}

Receive both into a DTO that does not declare exp and the results become identical. Two tokens with different contents turn into the same thing. Receive them as a Map and containsKey("exp") is true for one and false for the other, so they stay distinguishable.

In a Map, a missing exp can only mean the issuer did not send it. In a DTO, a missing exp could mean the issuer did not send it or the consumer did not declare the field, and you cannot tell which.

Behavior that was never unified

Everything so far happened on top of Jackson. But Jackson is not the only JSON library jjwt uses. The user picks one of jjwt-jackson, jjwt-gson or jjwt-orgjson, and that choice decides how JSON is handled.

So I ran the same payload through all three. The duplicated claims we saw earlier.

{
  "sub":  "alice",
  "sub":  "attacker",
  "role": "user",
  "role": "admin"
}

Jackson is covered by the issue we just followed, since from 0.12.4 jjwt turns STRICT_DUPLICATE_DETECTION on in the default ObjectMapper.

org.json blocks it too, even though jjwt never touched it. The library itself throws when it meets a duplicate key.

org.json 20250517
JSONException: Duplicate key "sub" at 21 [character 22 line 1]

Gson, however, just lets it through.

gson 2.13.2 · jjwt default settings (LONG_OR_DOUBLE · disableHtmlEscaping)
{sub=attacker, role=admin}

So I decided to contribute here. There is nothing new to argue for — it is a spot an already declared standard simply has not reached yet.

A closer look at gson

My first plan was to swap Gson's adapter for Object with my own. There was a problem, though: Gson forbids replacing the adapter for Object.

① intercept an ordinary type (Person)  → works
② intercept the Map type               → works
③ intercept the Object type            → IllegalArgumentException
④ work around it with a factory        → no exception, but never even called

The reason is that Object is the last resort Gson looks to when it does not know a type. It is not used only when reading with Object.class; every value of unknown type inside a map or an array goes to it as well. In other words, the blast radius of the change is far too wide.

private static boolean hasNonOverridableAdapter(Type type) {
    return type == Object.class;
}

So I went one layer deeper and ended up changing the JsonReader. The idea came from Jackson: there, STRICT_DUPLICATE_DETECTION is a feature of JsonParser, and ObjectMapper merely exposes the entry point.

private static final class DuplicateNameRejectingJsonReader extends JsonReader {
    private final Deque<Set<String>> names = new ArrayDeque<>();

    @Override public void beginObject() throws IOException { super.beginObject(); names.push(new HashSet<String>()); }
    @Override public void endObject()   throws IOException { super.endObject();   names.pop(); }

    @Override public String nextName() throws IOException {
        String name = super.nextName();
        Set<String> seen = this.names.peek();
        if (seen != null && !seen.add(name)) {
            throw new JsonParseException("Duplicate JSON member name '" + name + "' at " + getPath());
        }
        return name;
    }
}

Because each depth keeps its own set, identical names in sibling objects pass while duplicates inside a nested object or an array element are rejected.

Check the fire that looks out

Every existing test passed. Passing, however, does not mean safe. A test only confirms the cases someone wrote down in advance.

So I built a set of inputs by hand and fed them to the old code and the new code side by side. Only the inputs containing duplicates should differ and everything else should match — but one more turned up.

{
  "sub": "alice"
}trailing

before rejected     after: passes

To see why, we first have to note that Reader and JsonReader live on different layers.

What jjwt hands to an extension module is a java.io.Reader. It is the JDK's standard stream that feeds characters in order, and it knows nothing about JSON. From jjwt's point of view, since it cannot know whether Jackson, Gson or org.json will be plugged in, it cannot put a library-specific type in the contract.

// jjwt-api
public interface Deserializer<T> {
    T deserialize(Reader reader);        // java.io.Reader
}

Gson takes that Reader and builds its own JsonReader from it, internally.

But to add a duplicate check I have to hook into nextName(), and for that I have to construct the JsonReader myself and pass it in. At that moment the method being called changes.

// before — the argument is a Reader
gson.fromJson(reader, returnType);
  → fromJson(Reader, Class)      Gson builds the reader and, once done, checks whether anything is left

// after — the argument is a JsonReader
gson.fromJson(new DuplicateNameRejectingJsonReader(reader), returnType);
  → fromJson(JsonReader, Type)   the reader came from someone else, so it does not check

There is a reason the latter does not check. A single stream can carry several documents one after another, so having read one document you cannot declare that to be the end. Checking the bytecode confirmed it.

fromJson(Reader, ...)      assertFullConsumption called 1 time
fromJson(JsonReader, ...)  assertFullConsumption called 0 times

A change meant to block duplicates ended up loosening validation elsewhere. No test covered such input in the first place, so a green build could not have told me. I fixed it by performing the same check myself after parsing, and added two more tests.

Work like this feels like the most important part. When adding a new feature, what matters is whether it stays perfectly compatible with the existing one and whether the consistency of the behavior is preserved.

I also checked whether it adds any performance overhead. The design grows by one HashSet per object, so that cost had to be measured rather than assumed. I used JMH, and the code and raw result files live in a separate repository.

I built jjwt with the PR applied and ran the before and after side by side, and parsing one signed token grew by 136ns. Signature verification and Base64 decoding take most of the end-to-end time, so that came to 1.8% of the whole, with memory allocation up by 2.3%, and since it is such a small share of the total it does not seem like something to worry about.

Closing

PR #1073 (Align the Gson extension with Jackson's duplicate member name rejection) has been submitted and, as of writing, is still waiting for review.

Looking back, the funny part is that the code I actually wrote comes to about thirty lines. Most of the time went into reading one issue from its beginning, taking RFC sentences apart, and counting how far an already made decision had been applied.

It reminds me once again that we live in an age where code is cheap and verification is expensive.

Thank you for reading.