AMR¶
English | Japanese
OpenRheo refines the lattice in a block forest: the domain is cut into blocks that all hold the same number of cells, and a block is refined by replacing it with 2 × 2 × 2 blocks of half the size. Neighbouring blocks differ by at most one level (2:1 balance), and a fine level takes two time steps per coarse step. This page says how the forest is built, how the levels exchange data, where it refines, what has been measured and what is still open. The design records behind it are listed at the end.

The forest¶
The tree is kept on the host (OctForest in forest_topology.hpp: refine,
coarsen, 2:1 balance, Morton order); the cells live on the GPU in one pooled
array (ForestField in forest_field.hpp). The block edge is set per driver:
4 cells in the D3Q19 forest drivers (forest_ahmed_body.cpp), 8 by default in
the D3Q27 Ahmed driver (KNB), 16 by default in the multi-GPU channel driver.
The leaves are ordered along a Morton curve and cut into equal-weight pieces
for MPI (forest_partition.hpp, a block weighs 2^level).
The older dense AMR (block_forest.hpp: a coarse grid with fine wall-band and
interior-box patches) is still in the tree and still tested; it is where the
static wall-band AMR was first measured (below).
How the levels exchange data¶
This was the hard part. Three couplings were built in July 2026, and the
driver forest_ahmed_body.cpp still offers all three (coupling argument):
| coupling | how | result |
|---|---|---|
| 0, partition | a coarse leaf is replaced by its fine children; the levels meet at the interface by copy, explosion (trilinear) and coalescence (2³ average), with a time lag | the Ahmed body diverged at max_level >= 1 (NaN around step 200–600) after twelve rounds of fixes; uniform max_level = 0 was stable (2026-07-13) |
| 1, overlap ("shadow blocks") | the coarse level runs everywhere and the fine level overlays it; the fine interior is restricted to the coarse level | stable, but not mass-conservative by construction: the far-field mean of abs(ρ − 1) rose from 0.00037 (no refinement) to 0.00095 at three levels (2026-07-13). With a two-layer buffer and regularized (RR) collision it became the recommended setting for deep levels, but the force at levels 3–4 stayed wrong (2026-07-14) |
| 2, partition2 | a partition again, with conservative explosion: the fine populations are rescaled to the density of the covering coarse cell | all four conservation and stability gates passed; the mass drift fell about 100-fold (figure). The D3Q27 path always uses it, with the two-layer buffer (2026-07-14) |

The default of forest_ahmed_body.cpp was coupling = 0, the coupling its own
comment says diverges at max_level >= 1; since 2026-11-02 it is 2, as on the
D3Q27 path.
Where it refines¶
- Static: a band around the surface, nested boxes (
OFBOX), a box around the wake, a surface-following shell (SURFBAND) or a level map read from a CSV file (LEVEL_MAP);forest_mesh_spec.hppmakes nested boxes from a bounding box. - Sensors: the non-equilibrium part of the populations, the vorticity, and
the τ-pass sensor (
forest_taupass), which estimates the local error from two levels (Richardson) and has a surface channel for the wall. - Dynamic:
forest_regrid/forest_regrid_latrebuild the forest everyREGRID_EVERYsteps; a buffer and a dwell time keep blocks from flickering, new fine blocks are filled with a slope-limited (minmod) conservative prolongation, and a protection floor keeps moving interfaces away from the body so a regrid never lands in the force. The free-surface solver refines along the interface and keeps the fill mass across a regrid.
The wake of the dynamic Ahmed case (forest-ahmed-dynamic) shows the count of fine blocks following the flow:

Solid interior. BAND_SOLID=<n> keeps only n cells of solid behind the
surface at the finest level and leaves deeper solid blocks coarse. On the Ahmed
body (nh16, four levels) it cut the solid cells by 43 % (6,967,754 → 3,985,754)
and the GPU memory by 1.6 GB, with the same drag over 400 steps; it is off by
default.
Measured¶
| what | result | record |
|---|---|---|
| Lid-driven cavity Re = 1000, MRT, wall-band AMR | centreline error against the fine grid 0.0407, uniform coarse 0.0523 (22 % better), 2026-07-10 | native-amr-backend.md |
| Same, CPU (x86, GCC) and GPU (GH200, nvc++) | bit-identical | native-amr-backend.md |
| partition2, two-level periodic test | mass drift 4.9e-6 / 5.3e-6 against 5.75e-4 | forest-partition2-design.md |
| MPI, uniform 3D test, 1 and 2 GH200 | bit-identical to the serial run; weak scaling 1577 → 2660 MLUPS (~84 %), 2026-07-17 | mpi-amr-design.md |
| Solid-interior coarsening, Ahmed nh16 | solid cells −43 %, memory −1.6 GB, same drag over 400 steps | solid-interior-coarsening.md |
The operator matters as much as the coupling: TRT diverges with AMR, so MRT is the AMR standard; at deep levels the regularized collision RR worked best.
Still open¶
- The force on a body at levels 3–4 (deep AMR) is not trusted yet.
- Multi-GPU runs beyond two GPUs have not been measured.
BAND_SOLIDis off by default; the plan was to turn it on.- The gates of the sensor design (rank correlation with the error, the drop in error per budget) have no recorded results.
Design records¶
Frozen; they are not updated and are not translated.
- native-amr-backend.md — the native backend and the first AMR (2026-07-09 to 07-11, Japanese)
- forest-ahmed-design-space-study.md — the forest Ahmed log and the divergence at
max_level >= 1 - forest-overlap-coupling-design.md, forest-conservative-coupling-design.md, forest-coupling-rewrite-design.md — the overlap coupling
- amr-coupling-retrospective.md, amr-interface-noise-soa.md — the retrospective and the literature on interface noise
- forest-partition2-design.md — the conservative partition coupling
- dynamic-amr-design.md, dynamic-amr-design-review.md — dynamic AMR
- amr-sensor-brief.md, amr-sensor-design.md — the sensors
- mpi-amr-design.md — MPI
- solid-interior-coarsening.md — solid-interior coarsening