DPM Validation Workflow -- Comparing Simulation to Experimental PM2.5 Data
#+CATEGORY: cfd
#+TAGS: DPM validation experimental PM2.5 concentration comparison histogram uncertainty grid-independence
#+MATURITY: growing
DPM Validation Workflow :: Comparing Simulation to PM2.5 Experimental Data
You have experimental PM2.5 data for the shisha simulation. This page provides a systematic validation workflow: what to compare, how to extract simulation quantities at the same locations as the measurements, how to quantify uncertainty, and how to perform a grid independence study specifically for DPM.
Validation Strategy Overview
Three levels of validation
| Level | What | Data required | Effort |
| ------- | ------ | --------------- | -------- |
| 1. Qualitative | Do smoke patterns look physical? | Visual observation (flow visualization) | Low |
| 2. Quantitative (point) | Concentration at specific locations | PM2.5 sensor at known positions | Medium |
| 3. Quantitative (field) | Full concentration/distribution map | Spatially resolved measurement (e.g., scanning sensor) | High |
For your shisha case
If your experimental data is PM2.5 concentration at discrete points (e.g., breathing zone, outlet), use Level 2. If you have spatially resolved data (e.g., from a scanning optical particle counter), use Level 3.
Extracting Simulation Data for Comparison
Point monitoring at specific locations
For each experimental sensor location (x_i, y_i, z_i):
1. Create a point surface at each location:
- Surface > Point > enter
x, y, zcoordinates - Name it to match the experimental sensor ID
2. Extract DPM concentration:
- Report > DPM Concentration at specified surfaces
- Select the point surfaces
- Output: mass concentration [kg/m^3] at each point
3. For number concentration:
- Report > DPM Histogram at point surfaces
- Compute number concentration: =c_N = c_{mass} / (\rho_p \pi d_p^3 / 6)=
Area-averaged data
If experimental data is from a filter or sampler that integrates over an area:
1. Create a surface matching the sampler geometry (circle, rectangle) 2. Report > Surface Integrals > Area-Weighted Average > DPM Concentration 3. Compare to the experimental area-integrated measurement
Time-resolved comparison (transient only)
If experimental data includes time series:
1. Set up monitor points at sensor locations (see cfd_dpm-transient-tracking) 2. Run transient simulation 3. Extract time series of concentration at monitors 4. Compare to experimental time series using:
- Time-aligned overlay plot
- Cross-correlation function (identifies time lag)
- Root-mean-square error (RMSE)
Validation Metrics
For point comparisons
| Metric | Formula | Good if | ||
| -------- | --------- | --------- | ||
| Relative error | =\frac{ | c_{sim} - c_{exp} | }{c_{exp}}= | < 20% |
| RMSE | \sqrt{\frac{1}{N}\sum(c_{sim,i} - c_{exp,i})^2} | < 15% of mean | ||
| Normalized RMSE (NRMSE) | RMSE / \bar{c}_{exp} | < 0.2 | ||
Correlation coefficient R^2 | Pearson r | > 0.85 | ||
| Factor of 2 (FAC2) | Fraction within 0.5 \leq c_{sim}/c_{exp} \leq 2 | > 70% |
FAC2 is the most robust metric
FAC2 (Fraction of predictions within a factor of 2 of observations) is widely used in atmospheric dispersion model evaluation. A FAC2 > 0.7 (70% of predictions within factor of 2) is considered acceptable for environmental concentration predictions.
Reporting template
Grid Independence Study for DPM
DPM results can be grid-dependent even when the flow field is not. The concentration field depends on cell volume (because concentration = parcel mass / cell volume), so refinement changes the local concentration values.
Three-mesh protocol
| Mesh | Cells | \Delta x_{min} [mm] |
| ------ | ------- | --------------------- |
| Coarse | 500k | 3.0 |
| Medium | 2M | 1.5 |
| Fine | 5M | 0.75 |
Procedure
1. Generate three meshes (coarse, medium, fine) differing by factor \approx 2 in \Delta x 2. For each mesh: a. Solve continuous phase to convergence b. Track 100,000 parcels (same injection for all meshes) c. Extract DPM concentration at validation points 3. Compare the concentration at each point across meshes
Grid Convergence Index (GCI)
where:
- =F_s = 1.25= (safety factor for 3 grids)
- =r = \Delta x_{medium} / \Delta x_{fine} \approx 2=
- =p = 2= (formal order for 2nd-order scheme)
Acceptance criterion
If GCI < 5% at all validation points, the medium mesh is sufficient. If GCI > 10%, further refinement is needed.
DPM-specific grid independence notes
| Observation | Cause | Fix |
| ------------- | ------- | ----- |
| Concentration varies but total escaped mass is constant | Cell-volume effect on concentration | Use mass-flow rate (not concentration) for grid check |
| Number of trapped particles changes with mesh | Near-wall cell size controls deposition | Refine near-wall region; keep y+ < 1 |
| Stochastic tracking gives different results per mesh | DRW trajectory is stochastic | Compare ensemble-averaged results (many tries); check convergence |
Uncertainty Quantification
Sources of uncertainty
| Source | Typical magnitude | How to reduce |
| -------- | ------------------- | -------------- |
| Mesh discretization | 5--20% | Grid convergence study |
| Turbulence model | 10--30% | Try k-omega SST vs. RSM |
| Particle size distribution | 20--50% on number concentration | Measure PSD experimentally |
| Particle density | 5--15% | Measure or estimate from literature |
| Injection mass flow rate | 10--20% | Calibrate experimentally |
| DRW statistical convergence | 5--15% | Increase number of tries |
| Experimental measurement error | 5--15% | Report measurement uncertainty |
Combined uncertainty
For a typical shisha DPM validation: U_{total} \approx 30--50\% (dominated by particle size distribution uncertainty).
Sensitivity Analysis
One-at-a-time (OAT) sensitivity
Vary one parameter at a time by \pm 20% and observe the change in predicted concentration:
| Parameter | Nominal | -20% | +20% | Sensitivity |
| ----------- | --------- | ------- | ------- | ------------- |
d_g (geometric mean diameter) | 1.0 um | c = 52 | c = 38 | High |
\rho_p | 1200 kg/m^3 | c = 41 | c = 44 | Medium |
\dot{m}_{inj} | 3 mg/min | c = 34 | c = 51 | High |
U_{inlet} | 0.3 m/s | c = 48 | c = 40 | Low |
| Number of tries (DRW) | 20 | c = 43 \pm 5 | c = 42 \pm 3 | Low (convergence) |
The parameters with highest sensitivity should be measured or constrained most precisely.
Procedure
Validation Checklist
Before declaring the simulation validated:
- [ ] Continuous phase validated independently (velocity profile matches PIV or hot-wire)
- [ ] DPM grid independence confirmed (GCI < 5% at validation points)
- [ ] Parcel convergence confirmed (100k+ parcels; concentration stable with increasing N)
- [ ] Experimental data quality documented (measurement uncertainty stated)
- [ ] Comparison at matching conditions (same boundary conditions, same geometry)
- [ ] Validation metrics computed (RMSE, NRMSE, FAC2, R^2)
- [ ] Sensitivity analysis performed on uncertain parameters
- [ ] Uncertainty budget assembled (combined uncertainty reported)
- [ ] Discrepancies are explained (not just "the model is wrong")