Add certificate renewal and regeneration how-to

[miharp]

Adds a "Renewing and regenerating certificates" how-to to the openvox-server collection, prompted by a community Slack thread about an expired primary server certificate at the 5-year mark.

The page covers:

• Diagnosis: checking host cert vs CA cert expiry with openssl, and the ca_ttl naming trap. The puppet.conf setting (default 5y) governs certs the CA signs; the CA certificate's own 15-year lifetime is a fallback in puppetserver ca setup that was previously documented nowhere.
• Primary server cert regeneration (server is the CA): offline reissue with puppetserver ca generate --ca-client, with the subject-alt-names caveat and OpenVoxDB copy refresh.
• Agent/compiler cert regeneration (CA elsewhere): puppetserver ca clean, puppet ssl clean, puppet ssl bootstrap, puppetserver ca sign.
• Automatic renewal: allow-auto-renewal and auto-renewal-cert-ttl in ca.conf (off by default; the packaged config ships 60d, the built-in default is 90d), the agent's hostcert_renewal_interval (30d), and on-demand renewal with puppet ssl renew_cert / --if-expiring-in, including its silent no-op (exit 0) when the CA has renewal disabled.
• Extending an expired CA without reissuing agent certs: the supported puppetlabs/ca_extend Bolt plan first (matching how help.puppet.com handles this scenario), with the manual openssl re-signing procedure as the by-hand equivalent, adapted with attribution from bastelfreak's walkthrough.

Also adds an auto-renewal settings section to the ca.conf page, the nav entry, and a tip in the getting-started guide pointing readers at automatic renewal right after the step that issues their first 5-year agent certificates.

Validated with markdownlint and a local jekyll build; the page renders at /openvox-server/8.x/certificate_renewal.html and is cross-linked from config_file_ca.html.

Container verification: every command was exercised on Ubuntu 24.04 Docker containers running openvox-server 8.15.2, openvox-agent 8.28.1, and openvoxdb: CA setup, offline generate --ca-client regeneration, the full agent clean/bootstrap/sign cycle (including --dns_alt_names with allow-subject-alt-names, and the non-blocking --waitforcert 0 variant), auto-renewal issuing a 60-day cert, agent CA re-download after deleting localcacert, and the manual CA extension on a backdated expired CA (leaf verification fails before, passes after). Findings from that run are reflected in the page:

• Certs created by the puppetserver ca CLI (setup, generate --ca-client) get the 15-year fallback, so only certs signed by the running CA default to 5 years.
• generate --ca-client does not carry over subject alt names, and setup-created certs include DNS:puppet by default; the page says to inspect the backed-up cert (puppet config print dns_alt_names is empty in the default case) and covers the --force fallback.
• The packaged ca.conf ships auto-renewal-cert-ttl: "60d"; 90d is only the built-in default when the setting is absent.
• OpenVoxDB refresh uses puppetdb ssl-setup -f instead of manual copies.
• The ca.conf page also documents the ca-ttl override.

Container verification transcript (excerpted console output)

Two Ubuntu 24.04 containers on a shared Docker network: voxserver.test (openvox-server 8.15.2, CA) and voxagent.test (openvox-agent 8.28.1). Output is verbatim except for stripped ANSI codes and trimmed bootstrap retry loops.

CA setup and diagnosis commands

# dpkg -l | grep openvox
openvox-agent   8.28.1-1+ubuntu24.04
openvox-server  8.15.2-1+ubuntu24.04
# puppetserver ca setup
Generation succeeded. Find your files in /etc/puppetlabs/puppetserver/ca
# openssl x509 -enddate -noout -in /etc/puppetlabs/puppetserver/ca/ca_crt.pem
notAfter=Aug  7 10:33:14 2041 GMT
# openssl storeutl -noout -text /etc/puppetlabs/puppetserver/ca/ca_crt.pem | grep -E "Subject:|Not After"
            Not After : Aug  7 10:33:14 2041 GMT
        Subject: CN=Puppet CA: voxserver.test
            Not After : Aug  7 10:33:13 2041 GMT
        Subject: CN=Puppet Root CA: 3c2a14ce78189f

Primary server regeneration (server stopped) — also shows the 15-year CLI fallback and the dropped DNS:puppet alt name:

# CERTNAME="$(puppet config print certname)"
# rm /etc/puppetlabs/puppet/ssl/certs/"$CERTNAME".pem \
     /etc/puppetlabs/puppet/ssl/private_keys/"$CERTNAME".pem \
     /etc/puppetlabs/puppet/ssl/public_keys/"$CERTNAME".pem \
     /etc/puppetlabs/puppetserver/ca/signed/"$CERTNAME".pem
# puppetserver ca generate --certname "$CERTNAME" --ca-client
Successfully saved certificate for voxserver.test to /etc/puppetlabs/puppet/ssl/certs/voxserver.test.pem
Successfully saved certificate for voxserver.test to /etc/puppetlabs/puppetserver/ca/signed/voxserver.test.pem
Successfully saved private key for voxserver.test to /etc/puppetlabs/puppet/ssl/private_keys/voxserver.test.pem
Successfully saved public key for voxserver.test to /etc/puppetlabs/puppet/ssl/public_keys/voxserver.test.pem
# openssl x509 -enddate -noout -in "$(puppet config print hostcert)"
notAfter=Aug  7 10:41:16 2041 GMT    <-- 15-year fallback, not 5
# openssl x509 -text -noout -in "$(puppet config print hostcert)" | grep -A1 "Alternative Name"
            X509v3 Subject Alternative Name:
                DNS:voxserver.test   <-- setup-time cert had DNS:puppet, DNS:voxserver.test
# puppet config print dns_alt_names
                                     <-- empty, which is why the page says to inspect the backup

Agent regeneration cycle (5-year cert from the running CA):

server# puppetserver ca clean --certname voxagent.test
Certificate for voxagent.test has been revoked
Cleaned files related to voxagent.test
agent# puppet ssl clean
Notice: Removed private key /etc/puppetlabs/puppet/ssl/private_keys/voxagent.test.pem
Notice: Removed certificate /etc/puppetlabs/puppet/ssl/certs/voxagent.test.pem
agent# puppet ssl bootstrap
Info: Creating a new SSL certificate request for voxagent.test
Info: Certificate for voxagent.test has not been signed yet
Couldn't fetch certificate from CA server; you might still need to sign this agent's certificate (voxagent.test).
Info: Will try again in 5 seconds.
server# puppetserver ca sign --certname voxagent.test
Successfully signed the following certificate requests:
  voxagent.test
agent# openssl x509 -enddate -noout -in "$(puppet config print hostcert)"
notAfter=Aug 10 10:43:05 2031 GMT    <-- 5 years

Non-blocking variant (--waitforcert 0):

agent# puppet ssl bootstrap --waitforcert 0
Info: Certificate for voxagent.test has not been signed yet
Couldn't fetch certificate from CA server; you might still need to sign this agent's certificate (voxagent.test).
Error: Exiting now because the waitforcert setting is set to 0.
(exit 1)
server# puppetserver ca sign --certname voxagent.test
agent# puppet ssl bootstrap --waitforcert 0
Info: Downloaded certificate for voxagent.test from https://voxserver.test:8140/puppet-ca/v1
Notice: Completed SSL initialization
(exit 0)

Auto-renewal and subject alt names (packaged ca.conf ships 60d):

# grep -E "allow-auto-renewal|allow-subject-alt-names|auto-renewal-cert-ttl" /etc/puppetlabs/puppetserver/conf.d/ca.conf
    allow-subject-alt-names: true
    allow-auto-renewal: true
    auto-renewal-cert-ttl: "60d"
agent# puppet ssl bootstrap --dns_alt_names voxagent.test,agent-alias.test   # then signed on the CA
agent# openssl x509 -enddate -noout -in "$(puppet config print hostcert)"
notAfter=Oct 10 10:44:19 2026 GMT    <-- 60 days, not ca_ttl
agent# openssl x509 -text -noout -in "$(puppet config print hostcert)" | grep -A1 "Alternative Name"
            X509v3 Subject Alternative Name:
                DNS:agent-alias.test, DNS:voxagent.test

On-demand renewal (puppet ssl renew_cert), including the silent no-op:

# with allow-auto-renewal: false on the CA
agent# puppet ssl renew_cert; echo exit=$?
exit=0                               <-- no output, certificate unchanged
# with allow-auto-renewal: true
agent# puppet ssl renew_cert
Notice: Downloaded certificate 'voxagent.test' with fingerprint (SHA256) 1F:32:44:41:87:81:B5:22:...
agent# openssl x509 -enddate -noout -in "$(puppet config print hostcert)"
notAfter=Oct 10 11:12:20 2026 GMT    <-- 5-year cert replaced by 60-day cert
agent# puppet ssl renew_cert --if-expiring-in 30d; echo exit=$?
exit=0                               <-- fresh 60d cert, renewal skipped

Agent CA re-download after deleting localcacert:

agent# rm "$(puppet config print localcacert)"
agent# puppet agent -t --noop
Notice: Applied catalog in 0.01 seconds
agent# openssl x509 -subject -noout -in "$(puppet config print localcacert)"
subject=CN = Puppet CA: voxserver.test

Manual CA extension on a deliberately expired CA (backdated with faketime; leaf cert signed by it is still in its own validity window):

# openssl x509 -enddate -noout -in ca_crt.pem
notAfter=Dec 31 00:00:00 2024 GMT    <-- CA expired
# openssl verify -CAfile ca_crt.pem leaf.pem
error 10 at 1 depth lookup: certificate has expired
error leaf.pem: verification failed
# openssl rsa -noout -modulus -in ca_key.pem | openssl md5
MD5(stdin)= 1ea01589782b9f937e8f11b7a45a2f50
# openssl x509 -noout -modulus -in ca_crt.pem | openssl md5
MD5(stdin)= 1ea01589782b9f937e8f11b7a45a2f50
# openssl x509 -x509toreq -in ca_crt.pem -signkey ca_key.pem -out ca_csr.pem
# openssl x509 -req -days 5475 -in ca_csr.pem -signkey ca_key.pem -out ca_crt.pem \
    -extfile extension.cnf -extensions CA_extensions
# openssl x509 -enddate -noout -in ca_crt.pem
notAfter=Aug  7 10:45:58 2041 GMT
# openssl verify -CAfile ca_crt.pem leaf.pem
leaf.pem: OK                         <-- old leaf verifies against the extended CA

OpenVoxDB refresh:

# puppetdb ssl-setup -f
Setting ssl-port in /etc/puppetlabs/puppetdb/conf.d/jetty.ini already correct.
Setting ssl-key in /etc/puppetlabs/puppetdb/conf.d/jetty.ini already correct.
Setting ssl-cert in /etc/puppetlabs/puppetdb/conf.d/jetty.ini already correct.
Setting ssl-ca-cert in /etc/puppetlabs/puppetdb/conf.d/jetty.ini already correct.
# openssl x509 -enddate -noout -in /etc/puppetlabs/puppetdb/ssl/public.pem
notAfter=Aug  7 10:41:16 2041 GMT    <-- matches the regenerated host cert

Closes #446

Drafted with AI assistance (Claude Code); reviewed and verified by me.