feat: add optional clearing of ActiveRecord connections after each job

[rafael-pissardo]

Summary

This pull request introduces an optional mechanism to release Active Record database connections at the end of every job execution processed by Solid Queue workers.

• Adds a new configuration flag: SolidQueue.clear_connections_after_job (default: false).
• When enabled, the worker pool executes ActiveRecord::Base.clear_active_connections! in the ensure block of each thread after a job completes.
• Includes unit tests for both enabled and disabled scenarios.

Motivation

Under normal operation, each worker thread keeps its Active Record connection open for the lifetime of the process.
In environments with:

• Many worker threads (e.g., 10–20),
• Intermittent or bursty workloads where threads remain idle for extended periods,

these unused connections can account for a significant share of RSS:

• Roughly 1.5 – 3 MB of memory per connection (Ruby objects, native buffers, prepared-statement caches, etc.).
• For 10 threads per worker that can translate to 15–30 MB of resident memory per process during idle periods.

By explicitly clearing idle connections, memory that would otherwise remain tied up in driver buffers and Ruby objects can be reclaimed by the garbage collector and, on modern Ruby/glibc, returned to the OS.

Expected Impact

• Memory footprint per worker should drop by ~2 MB × number_of_threads while threads are idle.
• On a system with 4 workers × 10 threads the total saving can reach 60–120 MB during low-traffic windows.
• No behavioural changes when the flag is left at its default (false).

Trade-offs & Considerations

• Enabling the flag increases checkout/checkin churn on the Active Record connection pool, which can introduce minor overhead for high-throughput workloads.
• Prepared statement caches and any session-level settings are reset after each job.
• Therefore the flag is opt-in so that each deployment can decide based on its memory vs throughput priorities.

How to Enable

# config/initializers/solid_queue.rb
SolidQueue.clear_connections_after_job = true

Testing

• Added PoolConnectionClearTest to assert that ActiveRecord::Base.clear_active_connections! is invoked when the flag is enabled.
• Added PoolConnectionClearToggleTest to ensure it is not invoked when the flag is disabled.
• All existing test suites pass.

[p-schlickmann]

Thanks @rafael-pissardo!

I know some of our tests are flaky, but this one seems to be a real failure.

Please take a look.

[rafael-pissardo]

@p-schlickmann thanks for your comment.

Specs fixed. 👍

[rafael-pissardo]

@rosa WDYT?

[rosa]

Hey @rafael-pissardo, thanks for working on this. I haven't had time to review Solid Queue PRs in a while, or work on some features myself, but I'm hoping I'll have a block of a few days dedicated just to this in a couple of weeks. I'll look into it then 🤞🏻

[rosa]

@rafael-pissardo I have a question: is it possible to do this with an after_perform callback in Active Job?

[rafael-pissardo]

@rosa Good question!

An app could clear connections via Active Job callbacks, but it wouldn't achieve the same effect here for two reasons:

1. Timing: after_perform runs inside ActiveJob::Base.execute, which happens before Solid Queue finishes its own bookkeeping (finished! / failed_with / unblock_next_job). Those steps check out AR connections again, so when the thread goes idle it would still hold them — which is what we're trying to avoid.

2. Failures: after_perform doesn't run when the job raises. Connection cleanup needs an ensure so it always happens.

That's why the clear sits in the Pool's ensure after ClaimedExecution#perform completes (that's the point where the thread is truly done with the job and about to go idle).

Happy to adjust the approach if you'd prefer something different!