Conversation
The metadata readers used `hasattr(ds, 'asstr')` to decide whether a dataset holds strings, but every h5py Dataset has that method regardless of dtype -- it only raises when called on a numeric one. The TypeError was swallowed by the surrounding except, so numeric arrays fell through to the stringifying fallback: voxel_spacing, grid_origin, grid_dimensions_cells and intensity_range all came back as reprs like '[6 6 6]' instead of [6, 6, 6]. Downstream volume code has defensive fallbacks, so this surfaced as volumes silently rendering with default spacing/origin rather than as an error. Use h5py.check_string_dtype(ds.dtype) at all three sites instead.
…ce volume Ports rsMap3D's core capability: gridding every frame of one or more completed scans onto a single interpolated 3D volume via xrayutilities' Gridder3D. DashPVA already converted angles to Q per frame but could only show a bounded live point cloud, never a merged, interpolated volume. Launch with `DashPVA rsmgrid` or the Post Analysis launcher tile. Output uses the existing HDF5 volume format, so results open in the Workbench 3D viewer. Engine (utils/rsm_gridder.py): - Two passes are required, not an optimization. Gridder3D latches fixed_range on its first call when KeepData is set, after which points outside the first batch's range are silently dropped -- so global bounds must be known and set via dataRange(fixed=True) before any gridding. - Masked pixels are excluded from the arrays, never zeroed. Gridder3D.data is a per-bin mean, so a zeroed-but-present pixel would count as a real 'intensity = 0' measurement and bias its voxel low. - Optional I0 normalization from entry/data/metadata/ca/<name>, matching rsMap3D's monitor division. Without it, scans taken at different exposure or attenuation do not match in intensity where they overlap. - Energy/UB differences warn rather than block: each file's own UB is applied, so the merge already lands in a common crystal-fixed HKL frame, and HKL is geometrically energy-independent. rsMap3D does not block either. - Batch size is derived from a byte budget rather than a fixed frame count, which would need ~6 GB for the coordinate arrays on a 2048^2 detector. - grid_origin is shifted half a voxel: Gridder3D's axes are bin centers but the PyVista viewer treats the origin as a cell corner. - Axes are labelled H/K/L, not Q: passing UB to Ang2Q.area returns hkl. utils/rsm_converter.py gains build_file_geometry()/q_for_frames() so QConversion/HXRD/init_area are built once per file instead of once per frame, using xrayutilities' batched angle arguments. create_rsm() and get_sample_and_detector_circles() are unchanged for existing callers. The mask is stored in the viewer's display orientation and that state is never persisted, so the dialog asks rather than guessing. Known gaps vs rsMap3D: no flat-field correction, no user-specified grid crop, no detector tilt (a pre-existing rsm_converter limitation).
Collaborator
Author
|
@Osayi-ANL Please let me know if you have started working on this. Open to hearing your thoughts on this since you have extensively worked on 3D HKL work. This is a new feature that beamlines would find useful. |
Collaborator
Author
|
@Osayi-ANL when you have a chance, please work on this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the one thing rsMap3D (https://github.com/AdvancedPhotonSource/rsMap3D) can do that DashPVA could not: gridding every frame of one or more completed scans onto a single interpolated 3D reciprocal-space volume (
xu.Gridder3D). We already converted angles to Q per frame, but could only show a bounded live point cloud — never a merged volume.Launch with
DashPVA rsmgridor the Post Analysis launcher tile. Output uses the existing HDF5 volume format, so results open in Workbench's 3D viewer with no new viewer code.Design notes worth reviewing
Gridder3Dlatchesfixed_rangeon its first call whenKeepDatais set, after which points outside the first batch's range are silently dropped. Global bounds must be known and set viadataRange(fixed=True)before any gridding.Gridder3D.datais a per-bin mean, so a zeroed-but-present pixel would count as a realintensity = 0measurement and bias its voxel low.Ang2Q.areareturns hkl, not Q in Å⁻¹.entry/data/metadata/ca/<name>, matching rsMap3D's monitor division. Without it, scans at different exposure/attenuation won't match in intensity where they overlap.grid_originis shifted half a voxel:Gridder3D's axes are bin centers, but the PyVista viewer treats the origin as a cell corner.Refactor
utils/rsm_converter.pygainsbuild_file_geometry()/q_for_frames()soQConversion/HXRD/init_areaare built once per file instead of once per frame, using xrayutilities' batched angle arguments. Purely additive — no existingdefremoved,create_rsm()andget_sample_and_detector_circles()keep their signatures and behavior (a test asserts bit-identical Q against the old path).area_det_viewer.pyandhpc_rsm_consumer.pyhave their own inline setups and are untouched.Separate commit
687e943fixes a pre-existing bug found while testing: the HDF5 metadata readers usedhasattr(ds, 'asstr')to detect string datasets, but every Dataset has that method — it only raises on numeric ones, and the error was swallowed.voxel_spacing/grid_origin/grid_dimensions_cells/intensity_rangewere coming back as reprs like'[6 6 6]'. Reviewable/cherry-pickable on its own.Testing
227 tests pass (18 new). Grid dims in tests are deliberately asymmetric — with
nx==ny==nza transpose bug still produces a correctly-shaped array. Coverage: batched-vs-per-frame Q equivalence, hot-pixel→voxel round trip through save/load, mask exclusion vs zeroing, mask transpose, I0 normalization, cell-corner origin, and a guard on thefixed_rangelatch we depend on.Disjoint from #127 — zero file overlap.
Not yet validated on real data
cch1/pw1vs row/col) against a known Bragg peak — pre-existing inrsm_converter, but now load-bearing. A swap would produce a transposed volume that still looks plausible.metadata/ca/key is I0 at the beamline.Known gaps vs rsMap3D
No flat-field correction, no user-specified grid crop, no detector tilt (pre-existing
rsm_converterlimitation).