Skip to content

Payara 7.2026.8 upgrade plus multiple library updates - #12498

Merged
pdurbin merged 34 commits into
IQSS:developfrom
QualitativeDataRepository:payara7.2026.6plus
Aug 28, 2026
Merged

Payara 7.2026.8 upgrade plus multiple library updates#12498
pdurbin merged 34 commits into
IQSS:developfrom
QualitativeDataRepository:payara7.2026.6plus

Conversation

@qqmyers

@qqmyers qqmyers commented Jun 30, 2026

Copy link
Copy Markdown
Member

What this PR does / why we need it: This PR updates multiple libraries used in Dataverse along with updating to Payara 7.2026.8.

Which issue(s) this PR closes:

Special notes for your reviewer: There are a couple fixes for tests to pass because the updated libraries pull in a new XML processor. Using TransformerFactory.newDefaultInstance(); keeps the current one. Alternately we could use the new processor (TransformerFactory.newInstance()) and address the changes (it seems to be hardcoded at 3 spaces indent for pretty printing and I haven't checked for other differences.)
The PR also includes a couple test changes to allow them to pass on Windows - they were hardcoded to a specific line separator or the existence of a /tmp dir.

Lastly, there's an odd comment about jhove 1.34 in the PR - when I initially tried updating to it, there was a commons-compress conflict as some of the jhove modules pulled in an old version. I eventually solved that (by pulling in commons-compress first), but eventually decided to revert to the current jhove. (jhove 1.34 doesn't yet have all the modules packaged for maven and the 1.30 version separated the modules into separate maven dependencies. 1.20.1 is old, but for QDR I decided to stop working on it for now.)

Suggestions on how to test this: Regression testing - nothing should be changed with these updates. (QDR has been testing as well).

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?: release notes would need the normal payara version update instructions.

Additional documentation:

@coveralls

coveralls commented Jun 30, 2026

Copy link
Copy Markdown

Coverage Status

Coverage is 25.024%QualitativeDataRepository:payara7.2026.6plus into IQSS:develop. No base build found for IQSS:develop.

@pdurbin pdurbin added this to the 6.12 milestone Jul 8, 2026
@Ozoubia

Ozoubia commented Jul 10, 2026

Copy link
Copy Markdown

Hi @qqmyers,
following the topic on Zulip (#containers > Upgrading payara only), I have a question regarding the upgrade. We built the base image from the plain v6.11 tag with only -Dpayara.version=7.2026.6, without any of the code/library changes from this PR. It deploys cleanly on our test instance and basic smoke tests (login, upload, search, metadata exports) pass.
The Payara version was flagged by our IT security, so we cannot wait for the next Dataverse release.
Do you know of anyone running Payara 7.2026.6 in production? And are any of the changes here required at runtime? We're planning to run this combination in production until 6.12 ships.
Thank you in advance :)

@pdurbin

pdurbin commented Jul 10, 2026

Copy link
Copy Markdown
Member

-Dpayara.version=7.2026.6

For a bit more context, this comes from https://guides.dataverse.org/en/6.11/container/base-image.html#build-instructions

Screenshot 2026-07-10 at 6 40 55 AM

@qqmyers

qqmyers commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

The code changes here were due to other library updates, so I suspect that you'd be fine just deploying the newer Payara. You could update the payara bom version and rebuild the Dataverse war, but I'd usually expect compatibility across minor version updates. QDR often uses Payara versions newer than the recommended and have only encountered issues (which then get fixed in PRs we submit) when Payara or one of the libraries they use has had a major change.

@pdurbin

pdurbin commented Jul 10, 2026

Copy link
Copy Markdown
Member

I just checked with @qqmyers and now I understand better that this PR could have only included an update for Payara. He just happened to include some unrelated library updates plans to roll out.

@Ozoubia

Ozoubia commented Jul 13, 2026

Copy link
Copy Markdown

that's good to know, thank you for the help :)

@qqmyers qqmyers mentioned this pull request Aug 7, 2026
@cmbz cmbz moved this to Sprint Ready in IQSS Dataverse Project Aug 12, 2026
@qqmyers qqmyers self-assigned this Aug 12, 2026
@cmbz cmbz moved this to This Sprint 🏃‍♀️ 🏃 in IQSS Dataverse Project Aug 12, 2026
@cmbz cmbz added the FY27 Sprint 4 FY27 Sprint 4 (2026-08-12 - 2026-08-26) label Aug 12, 2026
(cherry picked from commit 2e69a61)
# Conflicts:
#	doc/sphinx-guides/source/api/changelog.rst
#	modules/dataverse-parent/pom.xml
#	pom.xml
@qqmyers
qqmyers marked this pull request as ready for review August 12, 2026 18:54
@pdurbin pdurbin changed the title Payara7.2026.6 Plus Multiple Library updates Payara 7.2026.7 upgrade plus multiple library updates Aug 12, 2026
@pdurbin

pdurbin commented Aug 14, 2026

Copy link
Copy Markdown
Member

@qqmyers to get the API and JSF tests passing, can you please merge from develop or rebase? I just merged this pull request from @srmanda-cs:

@pdurbin pdurbin added the Status: Merge Conflicts Merge conflicts must be resolved. label Aug 19, 2026
@qqmyers
qqmyers force-pushed the payara7.2026.6plus branch from a168349 to 2f980d5 Compare August 25, 2026 21:13
@qqmyers

qqmyers commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

I was incorrect - there is a new payara-bom for 7.2028.8 and it has a change to microprofile that caused a compile error in the recently added DataverseOenApiFilter (in #12439). I was able to fix that, but the build is now failing at on the smallrye plugin and I haven't been able to fix that. One issue may be that the newest payara-bom is not yet on maven central - only the payara repo. Our compile step was getting that but we did not have the payara repo in the pluginRepositories section of the pom. However, adding that, and trying to force the smallrye plugin to use the new microprofile api haven't fixed the issue:
[ERROR] Failed to execute goal io.smallrye:smallrye-open-api-maven-plugin:3.10.0:generate-schema (default) on project dataverse: Execution default of goal io.smallrye:smallrye-open-api-maven-plugin:3.10.0:generate-schema failed: An API incompatibility was encountered while executing io.smallrye:smallrye-open-api-maven-plugin:3.10.0:generate-schema: java.lang.NoSuchMethodError: 'void org.eclipse.microprofile.openapi.models.media.Schema.setExclusiveMaximum(java.lang.Boolean)'

Strangely, smallrye appears to run ok when I trigger it from my IDE.

Any ideas? @beepsoft or anyone else?

@poikilotherm poikilotherm self-assigned this Aug 26, 2026
@pdurbin

pdurbin commented Aug 26, 2026

Copy link
Copy Markdown
Member

We'll switch to testing/merging .7 if we can get .8 working by Tuesday.

@cmbz cmbz added the FY27 Sprint 5 FY27 Sprint 5 (2026-08-26 - 2026-09-09) label Aug 26, 2026
@qqmyers

qqmyers commented Aug 26, 2026

Copy link
Copy Markdown
Member Author

Looks like updating smallrye to the latest works, so this is ready to go again. (I just updated the smallrye-config version and remove a test adding the mp config dependency in the plugin. If those cause issues, I'll revert.)

@qqmyers

qqmyers commented Aug 26, 2026

Copy link
Copy Markdown
Member Author

FWIW: SearchIT failed at line 1444 in a prior run because the call to sleepForIndexing had a timeout of 10 seconds and that wasn't long enough. The method being used in UtilIT is called from 60 places, upping the limit would up it for many uses. (Not a bad thing - sleepForIndexing calls once per second and stops as soon as indexing happens so a longer timeout only slows things when needed. In any case - some other PR.

@pdurbin pdurbin moved this from In Review 🔎 to Ready for QA ⏩ in IQSS Dataverse Project Aug 27, 2026
@pdurbin

pdurbin commented Aug 27, 2026

Copy link
Copy Markdown
Member

@qqmyers fantastic! Tests are passing! I just re-approved it!

@pdurbin pdurbin moved this from Ready for QA ⏩ to QA ✅ in IQSS Dataverse Project Aug 27, 2026
@pdurbin pdurbin self-assigned this Aug 27, 2026

<!-- Major system components and dependencies -->
<payara.version>7.2026.2</payara.version>
<payara.version>7.2026.8</payara.version>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran mvn -Pct -f modules/container-base install as of c64b662 to build the base image and got this error:

[INFO] DOCKER> --------------------
[INFO] DOCKER> ERROR: failed to build: failed to solve: process "/bin/bash -euo pipefail -c     # Set admin password\n    echo \"AS_ADMIN_PASSWORD=\" > /tmp/password-change-file.txt\n    echo \"AS_ADMIN_NEWPASSWORD=${PAYARA_ADMIN_PASSWORD}\" >> /tmp/password-change-file.txt\n    asadmin --user=${PAYARA_ADMIN_USER} --passwordfile=/tmp/password-change-file.txt change-admin-password --domain_name=${DOMAIN_NAME}\n\n    # Prepare shorthand\n    PASSWORD_FILE=$(mktemp)\n    echo \"AS_ADMIN_PASSWORD=${PAYARA_ADMIN_PASSWORD}\" >> ${PASSWORD_FILE}\n    ASADMIN=\"${PAYARA_DIR}/bin/asadmin --user=${PAYARA_ADMIN_USER} --passwordfile=${PASSWORD_FILE}\"\n\n    # Start domain for configuration\n    ${ASADMIN} start-domain ${DOMAIN_NAME}\n    # Allow access to admin with password only\n    ${ASADMIN} enable-secure-admin\n\n    ### CONTAINER USAGE ENABLEMENT\n    # List & delete memory settings from domain\n    for MEMORY_JVM_OPTION in $(${ASADMIN} list-jvm-options | grep \"Xm[sx]\\|Xss\\|NewRatio\");\n       do\n         ${ASADMIN} delete-jvm-options $(echo $MEMORY_JVM_OPTION | sed -e 's/:/\\\\:/g');\n       done\n    # Tweak memory settings for containers\n    ${ASADMIN} create-jvm-options \"-XX\\:+UseContainerSupport\"\n    ${ASADMIN} create-jvm-options \"-XX\\:MaxRAMPercentage=\\${ENV=MEM_MAX_RAM_PERCENTAGE}\"\n    ${ASADMIN} create-jvm-options \"-Xss\\${ENV=MEM_XSS}\"\n    ${ASADMIN} create-jvm-options \"-XX\\:MinHeapFreeRatio=\\${ENV=MEM_MIN_HEAP_FREE_RATIO}\"\n    ${ASADMIN} create-jvm-options \"-XX\\:MaxHeapFreeRatio=\\${ENV=MEM_MAX_HEAP_FREE_RATIO}\"\n    ${ASADMIN} create-jvm-options \"-XX\\:HeapDumpPath=\\${ENV=DUMPS_DIR}\"\n    # Set logging to console only for containers\n    ${ASADMIN} set-log-attributes com.sun.enterprise.server.logging.GFFileHandler.logtoFile=false\n\n    ### PRODUCTION READINESS\n    ${ASADMIN} create-jvm-options '-XX\\:+UseG1GC'\n    ${ASADMIN} create-jvm-options '-XX\\:+UseStringDeduplication'\n    ${ASADMIN} create-jvm-options '-XX\\:+DisableExplicitGC'\n    ${ASADMIN} create-jvm-options '-XX\\:MaxGCPauseMillis=${ENV=MEM_MAX_GC_PAUSE_MILLIS}'\n    ${ASADMIN} create-jvm-options '-XX\\:MetaspaceSize=${ENV=MEM_METASPACE_SIZE}'\n    ${ASADMIN} create-jvm-options '-XX\\:MaxMetaspaceSize=${ENV=MEM_MAX_METASPACE_SIZE}'\n    ${ASADMIN} create-jvm-options '-XX\\:+IgnoreUnrecognizedVMOptions'\n    # Disable autodeploy and hot reload\n    ${ASADMIN} set configs.config.server-config.admin-service.das-config.dynamic-reload-enabled=\"false\"\n    ${ASADMIN} set configs.config.server-config.admin-service.das-config.autodeploy-enabled=\"false\"\n    # Enlarge thread pools\n    ${ASADMIN} set server-config.thread-pools.thread-pool.http-thread-pool.max-thread-pool-size=\"50\"\n    ${ASADMIN} set server-config.thread-pools.thread-pool.http-thread-pool.max-queue-size=\"\"\n    ${ASADMIN} set default-config.thread-pools.thread-pool.thread-pool-1.max-thread-pool-size=\"250\"\n    # Enable file caching\n    ${ASADMIN} set server-config.network-config.protocols.protocol.http-listener-1.http.file-cache.enabled=\"true\"\n    ${ASADMIN} set server-config.network-config.protocols.protocol.http-listener-2.http.file-cache.enabled=\"true\"\n    ${ASADMIN} set default-config.network-config.protocols.protocol.http-listener-1.http.file-cache.enabled=\"true\"\n    ${ASADMIN} set default-config.network-config.protocols.protocol.http-listener-2.http.file-cache.enabled=\"true\"\n    # Set SameSite cookie value: https://docs.payara.fish/community/docs/6.2024.6/Technical%20Documentation/Payara%20Server%20Documentation/General%20Administration/Administering%20HTTP%20Connectivity.html\n    # The following dynamic version is what we want, modeled off \"${MPCONFIG=dataverse.http.timeout:900}\"\n    # but it's not working so it's commented out. Instead, we hard code the value to \"Lax\". This means you have\n    # to build your own base image if you'd like to change it.\n    #${ASADMIN} set server-config.network-config.protocols.protocol.http-listener-1.http.cookie-same-site-value=${MPCONFIG=dataverse.cookie-same-site-value:Lax}'\n    ${ASADMIN} set server-config.network-config.protocols.protocol.http-listener-1.http.cookie-same-site-value=\"Lax\"\n    ${ASADMIN} set server-config.network-config.protocols.protocol.http-listener-1.http.cookie-same-site-enabled=\"true\"\n    # Disable the HTTPS listener (we are always fronting our appservers with a reverse proxy handling SSL)\n    ${ASADMIN} set configs.config.server-config.network-config.network-listeners.network-listener.http-listener-2.enabled=\"false\"\n    # Enlarge and tune EJB pools (cannot do this for server-config as set does not create new entries)\n    ${ASADMIN} set default-config.ejb-container.pool-resize-quantity=\"2\"\n    ${ASADMIN} set default-config.ejb-container.max-pool-size=\"128\"\n    ${ASADMIN} set default-config.ejb-container.steady-pool-size=\"10\"\n    # Misc settings\n    ${ASADMIN} create-system-properties fish.payara.classloading.delegate=\"false\"\n    ${ASADMIN} create-system-properties jersey.config.client.readTimeout=\"300000\"\n    ${ASADMIN} create-system-properties jersey.config.client.connectTimeout=\"300000\"\n\n    ### DATAVERSE APPLICATION SPECIFICS\n    # Configure the MicroProfile directory config source to point to /secrets\n    ${ASADMIN} set-config-dir --directory=\"${SECRETS_DIR}\"\n    # Password alias store = 105, default = 100 - lets sort between those to enable overriding from all of the others\n    # except alias config source and microprofile-config.properties\n    ${ASADMIN} set-config-ordinal --ordinal=104 --source=secrets\n    # Make request timeouts configurable via MPCONFIG (default to 900 secs = 15 min)\n    ${ASADMIN} set 'server-config.network-config.protocols.protocol.http-listener-1.http.request-timeout-seconds=${MPCONFIG=dataverse.http.timeout:900}'\n    # TODO: what of the below 3 items can be deleted for container usage?\n    ${ASADMIN} create-network-listener --protocol=http-listener-1 --listenerport=8009 --jkenabled=true jk-connector\n    ${ASADMIN} set server-config.network-config.protocols.protocol.http-listener-1.http.comet-support-enabled=true\n    ${ASADMIN} create-system-properties javax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl\n    # Always disable phoning home\n    ${ASADMIN} disable-phone-home\n\n    ### CLEANUP\n    # Stop domain\n    ${ASADMIN} stop-domain \"${DOMAIN_NAME}\"\n    # Disable JSP servlet dynamic reloads\n    sed -i 's#<servlet-class>org.glassfish.wasp.servlet.JspServlet</servlet-class>#<servlet-class>org.glassfish.wasp.servlet.JspServlet</servlet-class>\\n    <init-param>\\n      <param-name>development</param-name>\\n      <param-value>false</param-value>\\n    </init-param>\\n    <init-param>\\n      <param-name>genStrAsCharArray</param-name>\\n      <param-value>true</param-value>\\n    </init-param>#' \"${DOMAIN_DIR}/config/default-web.xml\"\n    # Cleanup old CA certificates to avoid unnecessary log clutter during startup\n    ${SCRIPT_DIR}/removeExpiredCaCerts.sh\n    # Delete generated files\n    rm -rf \\\n        \"$PASSWORD_FILE\" \\\n        \"/tmp/password-change-file.txt\" \\\n        \"${PAYARA_DIR}/glassfish/domains/${DOMAIN_NAME}/osgi-cache\" \\\n        \"${PAYARA_DIR}/glassfish/domains/${DOMAIN_NAME}/logs\"\n" did not complete successfully: exit code: 1
[ERROR] DOCKER> Error status (1) when building
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10:49 min
[INFO] Finished at: 2026-08-27T14:50:06-04:00
[INFO] ------------------------------------------------------------------------

Not sure what's going on. 🤔

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind, I tried again after running mvn -Pct -f modules/container-base clean and it worked!

@pdurbin

pdurbin commented Aug 27, 2026

Copy link
Copy Markdown
Member

The Payara upgrade went fine on my dev1 server (I pushed a couple updates to the upgrade steps in 3a75937) based on what we wrote at https://github.com/IQSS/dataverse/releases/tag/v6.10

It also seems to work fine in Docker.

My finger is hovering over the merge button! 😄

@pdurbin
pdurbin merged commit 82e84a1 into IQSS:develop Aug 28, 2026
18 checks passed
@github-project-automation github-project-automation Bot moved this from QA ✅ to Merged 🚀 in IQSS Dataverse Project Aug 28, 2026
@pdurbin pdurbin removed their assignment Aug 28, 2026
@jp-tosca jp-tosca moved this from Merged 🚀 to Done 🧹 in IQSS Dataverse Project Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

FY27 Sprint 4 FY27 Sprint 4 (2026-08-12 - 2026-08-26) FY27 Sprint 5 FY27 Sprint 5 (2026-08-26 - 2026-09-09)

Projects

Status: Done 🧹

Development

Successfully merging this pull request may close these issues.

7 participants