[WIP] Add opt-in Usage Reporter (call-home) for the CloudStack project - #13985
[WIP] Add opt-in Usage Reporter (call-home) for the CloudStack project#13985wido wants to merge 4 commits into
Conversation
The Management Server periodically sends an anonymous usage report to an API endpoint of the CloudStack project. This is opt-in and disabled by default (usage.report.interval = 0); only aggregated statistics are sent, nothing that can directly identify an environment. Reports are only sent over HTTPS. Includes the server-side collector, a Python Flask/WSGI application which validates incoming reports and stores them as JSON files on the local filesystem, in a directory per environment with the receive timestamp as filename. Submissions are rate limited and bounded per environment and reports with unexpected structure are rejected.
|
This is a really nice idea and will help us understand better the CloudStack users, their stack and environment specifics. IMO the information shall be limited to PMCs only, as it opens a bit of security gap for proprietary vendors to target the CloudStack users. But we can create Qly reports and publish them as blog posts on the ACS website. I was hoping for years to get such information, so really nice job! |
…enses
Pin down the JSON that the management server POSTs to usage.report.uri so
any change to the wire format has to be a deliberate one.
UsageReporterTest mocks the seven DAOs the report is built from and drives
the real report builders through the real AtomicGsonAdapter, comparing the
result against a checked-in fixture, usage-report-expected.json. Keys are
sorted on both sides before comparing: AtomicLongMap is backed by a
ConcurrentHashMap and the report itself by a HashMap, so key order on the
wire is not deterministic and must not be part of the contract. Sorting
also makes a mismatch print a readable diff.
The tests document three things that are not obvious from reading the code:
- provisioning_type keys are lowercase ("thin"/"fat"). Storage.Provisio-
ningType overrides toString() and the adapter keys on String.valueOf(),
so the payload does not carry the enum constant names that Gson would
emit by default.
- boolean counters reach the wire as the string keys "true" and "false",
used by ha_enabled, dynamically_scalable, compute_only and
use_local_storage.
- a report from an empty install still carries all eight sections with
empty counter objects, and avg_disk_size falls back to 0 rather than
dividing by zero.
AtomicGsonAdapterTest covers the adapter on its own: null and empty maps,
counts as numbers, boolean and enum keys, and that read() consumes a null.
Also add the ASF license headers that apache-rat flags on reporter/README.md
and reporter/requirements.txt.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #13985 +/- ##
============================================
+ Coverage 19.74% 19.81% +0.06%
- Complexity 19960 20031 +71
============================================
Files 6371 6374 +3
Lines 575784 576211 +427
Branches 70478 70536 +58
============================================
+ Hits 113665 114148 +483
+ Misses 449765 449624 -141
- Partials 12354 12439 +85
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
It would be definitely useful but I'm not sure how many users would want to share it. It would be useful to some control on toggling configs on the first login screen. |
DaanHoogland
left a comment
DaanHoogland
left a comment
There was a problem hiding this comment.
code looks generally good, but prove is in the eating of the pudding.
|
I think if we do not disclose the information to the general public we should abandon this project. It is definitely not for PMC only, or even restricted to any apache circle. Its use is for any contributor from anywhere. |
…n identity util Address the first points of the architectural review on the usage reporter. The reporting destination is no longer a Global Setting. The point of telemetry is to give the Apache CloudStack project authoritative project-wide statistics, so "telemetry enabled" has to mean the data reaches the project rather than wherever an operator points it. The endpoint is now the constant https://call-home.cloudstack.org/report, with DNS providing whatever indirection the receiving infrastructure needs. Rename usage.report.interval to telemetry.interval. The usage.* prefix collides with the Usage Server / usage records subsystem and reads like an accounting setting. Zero still means disabled. The installation identity stays derived rather than generated: the version the database was created with plus the moment it was created already identifies an installation, is shared by every Management Server on that database, and survives restarts and upgrades without anything being stored. Move the derivation out of UsageReporter into InstallationIdentity in the utils module, fed by a new VersionDao.getInitialVersion(). That removes the raw JDBC from UsageReporter and makes both halves unit testable. Two changes to the hash follow from taking the timestamp as a Date instead of whatever string the JDBC driver returned: it is formatted as UTC, so Management Servers in different timezones sharing a database derive the same identity, and the two inputs are separated so different pairs cannot concatenate into the same string. The derived value therefore differs from the previous implementation, which is a one-time change for anyone already running this branch and has no effect otherwise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Agree @DaanHoogland 100%. Once you put your business/comercial hat on, you will support PMC-only idea @DaanHoogland 😉 In the meantime, it's for the whole Project, IMO. Sorry @IvetPM |
There you are wrong there @andrijapanicsb. I made OSS my business and I know the conditions it requires for that to work. 🤑 |
|
That was a joke @DaanHoogland - and that why I value you so much! |
|
Thanks @wido, I think this is going in a much better direction. I like the change to I don't want to repeat my previous very long review here :) but I also don't want it to look like the remaining points are no longer important. Since this is still WIP, I think there are a few important things we should keep in mind before this gets anywhere close to merge-ready:
There are still some other points from the previous review: only one MS should probably send the report in an HA setup, doing the aggregation in SQL instead of loading all objects into Java, active/stale installation tracking, actual volume statistics vs configured disk offerings, collector storage, and making the opt-in/report preview visible to operators. I don't think all of those need to be solved in the same commit though. They can be handled step by step while this is still WIP. Overall, definitely +1 from me on continuing with this and thx again for such a great initiative! |
Address the next round of the architectural review on the telemetry feature. Rename the reporter/ directory to telemetry/ to match what the feature is called since the telemetry.* settings rename. The report now carries a schema_version and the collector validates every payload against the exact schema belonging to that version, on top of the existing generic sanity limits: all sections and counters must be present, no other keys are accepted anywhere and every value must have the expected type. An arbitrary well-formed JSON object no longer makes it to disk, and any change to the report structure requires incrementing the version and teaching the collector the new schema. Rows of removed Instances are kept in the database, so both current and lifetime statistics can be reported, and they mean different things. The instances section is now split in two: "current" covers the non-removed rows in any state, Destroyed Instances that have not been expunged yet included, while "lifetime" also counts the removed rows and therefore describes every Instance which ever existed in the cloud, as total/removed counts plus hypervisor and type counters. Document in the README that the installation ID is not guaranteed globally unique but a collision is negligible (two databases created with the same version in the same second), that a cloned or restored database intentionally reports as the same installation, and that the collector cannot verify a report was genuinely produced by a Management Server: the source is open, so the statistics are best-effort by nature, with validation, rate limiting and the per-ID storage bound limiting abuse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Thanks for the review! Addressed in the latest push:
The directory has also been renamed from reporter/ to telemetry/ to match the feature name. |
5 participants
Description
This is a WIP / RFC to gather feedback before it is finished. Not intended to be merged as-is.
This proposes an opt-in Usage Reporter ("call-home") for CloudStack. The goal is to give the project insight into how CloudStack is actually deployed in the wild: which hypervisors, storage types, network offerings and versions are in use, and how large environments typically are. Today we simply do not know, which makes it hard to decide what to prioritise, what to deprecate and what to test.
Two parts are included:
UsageReporter): periodically collects aggregated counters and POSTs them as JSON to an endpoint of the CloudStack project over HTTPS.reporter/): a small Python Flask/WSGI application that validates incoming reports and stores them as JSON files on disk, one directory per environment, with the receive timestamp as filename. Submissions are rate limited and bounded per environment, and reports with an unexpected structure are rejected.Opt-in and privacy
usage.report.interval = 0. An operator has to explicitly set an interval (7 days recommended) and restart the Management Server.usage.report.uridefaults tohttps://reporting.cloudstack.org/reportand can be pointed elsewhere. Only HTTPS is accepted; plain HTTP is refused.versiontable, so reports from the same environment can be correlated over time without identifying it.Open points for discussion
reporter/collector may be better off in a separate repository rather than in the main tree.Types of changes
How Has This Been Tested?
Manually against a local Management Server with
usage.report.intervalset to a low value and the collector running locally behind HTTPS. Verified that nothing is sent with the default configuration.