Skip to content

[BUG] Web Service Listener fails every SOAP request on Java 19+ (Unable to create SAAJ meta-factory) #509

Open

Description

@tonygermano

Describe the bug

On Java 19 and later, every SOAP POST to a Web Service Listener fails with a 500 and logs:

java.lang.Error: javax.xml.soap.SOAPException: Unable to create SAAJ meta-factory: Provider com.sun.xml.internal.messaging.saaj.soap.SAAJMetaFactoryImpl not found
	...
	at com.mirth.connect.connectors.ws.LoggingSOAPHandler.handleMessage(LoggingSOAPHandler.java:63)

Java 17 and 18 are not affected. WSDL requests (GET /services/<name>?wsdl) keep returning 200 because they never touch SAAJ, so a working WSDL doesn't mean the listener is healthy.

The same bug was reported against BridgeLink in Innovar-Healthcare/BridgeLink#205, where dsvanstedt found the root cause and workaround in this comment. Our code and jars match, and their analysis holds for this repository.

Root cause

The problem is the thread context classloader (TCCL), not a missing jar.

  • WebServiceReceiver.onStart() runs the listener on a com.sun.net.httpserver.HttpServer with a fixed thread pool (WebServiceReceiver.java:166-168). The pool's threads are created from the server's dispatcher thread, which ServerImpl starts without inheriting thread-locals, and they inherit its TCCL.
  • Through JDK 18, such a thread still copied its parent's TCCL, which is the launcher classloader that can see server-lib. From JDK 19, it gets the system classloader instead.
  • The SAAJ API does all of its provider lookups through the TCCL. When they all miss, it falls back to a hard-coded class from the JDK's internal SAAJ, which was removed in Java 11. That's the class named in the error.

To Reproduce

  1. Run the engine on Java 19 or later.
  2. Deploy a channel with a Web Service Listener source using the default service.
  3. POST any SOAP request to http://<host>:8081/services/<service name>.

Expected behavior

The listener accepts the message, as it does on Java 17.

Actual behavior

Every POST returns 500 with the error above.

Environment

  • OS: any (reproduced on Linux)
  • Java: 19, 20 and 21 affected; 17 and 18 work
  • Version: 4.7.0 (current main)

Workaround(s)

Run on Java 17, or add the -classpath/a line for your version from this comment to conf/custom.vmoptions (or to the VMOPTIONS environment variable for Docker).

The -Xbootclasspath/a workaround that was originally posted here isn't enough. Valid requests work with it, but every request that should get a SOAP fault gets an empty 500 instead. The comment explains why.

Additional context

Claude checked this with a standalone harness that loads server-lib/javax the way MirthLauncher does and mirrors WebServiceReceiver.onStart() and LoggingSOAPHandler, on JDK 17, 18, 19 and 21:

  • On 17 and 18, plain SOAP, MTOM and SwA POSTs return 200. On 19 and 21 they all return 500 with the error above.
  • With the workaround, all three return 200 on every version tested.

It hasn't been checked on a full engine deployment yet.

Proposed fix

In onStart(), capture the TCCL of the thread that starts the server, and have the executor's threads use it explicitly, for example through a custom ThreadFactory. This restores the pre-JDK 19 behavior.

Using the channel's application classloader instead would be a behavior change, not a fix. Custom web service classes don't need it at request time, because they're loaded through it and JAX-WS builds its model during publish(). With child-first resources, it could also pick up a second copy of SAAJ.


🤖 Drafted with Claude Code

Activity

  1. tonygermano commented on Oct 24, 2026

    @tonygermano
    MemberAuthor

    Updated workaround

    The -Xbootclasspath/a workaround in the description isn't enough. Valid requests work with it, but every request that should get a SOAP fault gets an empty 500 instead. Use this instead, as a single line in conf/custom.vmoptions.

    4.6.0 and 4.7.0:

    -classpath/a server-lib/javax/jaxws/javax.xml.soap-api-1.4.0.jar:server-lib/javax/jaxws/ext/saaj-impl-1.0.jar:server-lib/javax/jaxws/ext/stax-ex-1.8.jar:server-lib/javax/jaxws/ext/mimepull-1.9.7.jar:server-lib/javax/jakarta.activation-1.2.1.jar:server-lib/javax/jaxb/jaxb-api-2.4.0-b180725.0427.jar:server-lib/javax/jaxb/jaxb-runtime-2.4.0-b180725.0644.jar:server-lib/javax/jaxb/ext/istack-commons-runtime-3.0.6.jar:server-lib/javax/jaxb/ext/txw2-2.4.0-b180725.0644.jar
    

    4.5.2:

    -classpath/a server-lib/javax/jaxws/javax.xml.soap-api-1.4.0.jar:server-lib/javax/jaxws/ext/saaj-impl-1.0.jar:server-lib/javax/jaxws/ext/stax-ex-1.8.jar:server-lib/javax/jaxws/ext/mimepull-1.9.7.jar:server-lib/javax/javax.activation-1.2.0.jar:server-lib/javax/javax.activation-api-1.2.0.jar:server-lib/javax/jaxb/jaxb-api-2.4.0-b180725.0427.jar:server-lib/javax/jaxb/jaxb-runtime-2.4.0-b180725.0644.jar:server-lib/javax/jaxb/ext/istack-commons-runtime-3.0.6.jar:server-lib/javax/jaxb/ext/txw2-2.4.0-b180725.0644.jar
    
    • On Windows, replace each : between the paths with ;.
    • In Docker, pass the same line as the VMOPTIONS environment variable. The official 4.5.2 images run Java 17, so they aren't affected.
    • This puts SAAJ, JAXB and activation on the system classpath for the whole engine, not just the Web Service Listener. Remove the line once you're on a release with Fix Web Service Listener SOAP requests failing on JDK 19+ #508. Running on Java 17 also avoids the bug, with no workaround needed.
    Why the original workaround doesn't work, and how this was tested

    The original workaround was:

    -Xbootclasspath/a:server-lib/javax/jaxws/javax.xml.soap-api-1.4.0.jar:server-lib/javax/jaxws/ext/saaj-impl-1.0.jar:server-lib/javax/jakarta.activation-1.2.1.jar
    

    It has two problems:

    • JAXB finds its implementation through the thread context classloader too. When a request needs a SOAP fault, SOAPFaultBuilder creates a JAXB context on the worker thread. That fails with Implementation of JAXB-API has not been found, the class can't be initialized, and every fault after that is an empty 500.
    • The JVM doesn't verify classes on the boot classpath, which hides missing dependencies. saaj-impl references stax-ex and mimepull, which stay in server-lib, where the boot classloader can't see them. The workaround only works because those code paths didn't run. With -XX:+UnlockDiagnosticVMOptions -XX:+BytecodeVerificationLocal, every request fails with NoClassDefFoundError: org/jvnet/staxex/....

    Classes on the system classpath are verified, so a jar missing from the list fails immediately instead of on a rare code path. The lists above include every dependency of the jars in them, checked with jdeps. They differ between versions because 4.5.2's javax.activation-1.2.0.jar contains only the com.sun.activation classes, and the javax.activation API is in javax.activation-api-1.2.0.jar. In 4.6.0 and 4.7.0, jakarta.activation-1.2.1.jar contains both.

    Claude checked each workaround using the install4j launcher from the 4.5.2, 4.6.0 and 4.7.0 Linux release tarballs on Temurin 21.0.13, with Zulu 17.0.17 as the baseline, by deploying a Web Service Listener and sending requests to it:

    Valid request Unknown operation mustUnderstand header
    Java 17, no workaround 200 SOAP fault SOAP fault
    Java 21, no workaround 500 empty 500 empty 500
    Java 21, original workaround 200 empty 500 empty 500
    Java 21, updated workaround 200 SOAP fault SOAP fault

    All three releases gave the same results. With the updated workaround, MTOM and SOAP-with-Attachments requests also return 200 on all three, as on Java 17. A Docker image built from current main (Temurin 21.0.9) with the workaround passed in VMOPTIONS gave the same results for the valid and unknown-operation requests.


    🤖 Drafted with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions