Counting Through Occlusion: Framework for Open World Amodal Counting

Abstract

Object counting has achieved remarkable success on visible instances, yet state-of-the-art (SOTA) methods fail under occlusion. This failure stems from a fundamental architectural limitation where backbone networks encode occluding surfaces rather than target objects, thereby corrupting the feature representations required for accurate enumeration. To address this, we present CountOCC, an amodal counting framework that explicitly reconstructs occluded object features through hierarchical multimodal guidance. Rather than accepting degraded encodings, we synthesize complete representations by integrating spatial context from visible fragments with semantic priors from text and visual embeddings, generating features at occluded locations across multiple pyramid levels. We further introduce a visual equivalence objective that enforces consistency in attention space, ensuring that both occluded and unoccluded views of the same scene produce spatially aligned gradient-based attention maps. Together, these complementary mechanisms preserve discriminative properties essential for accurate counting under occlusion. For rigorous evaluation, we establish occlusion-augmented versions of FSC-147 and CARPK (FSC-147-OCC and CARPK-OCC). CountOCC achieves SOTA performance on FSC-147-OCC with 26.72% and 20.80% MAE reduction over prior baselines under occlusion in validation and test, respectively. CountOCC also demonstrates exceptional generalization by setting new SOTA results on CARPK-OCC with 49.89% MAE reduction and on CAPTURe-Real with 28.79% MAE reduction, validating robust amodal counting.

Counters only count what they see

Motivation

Open-world counters such as CountGD count any category from a text prompt or a few exemplar boxes. When objects are partly hidden, they fail in a predictable way. The backbone encodes the occluding surface instead of the objects behind it, so the features needed to count those objects are simply missing. Asking a model to count harder does not help, because the evidence is already corrupted.

CountOCC does not accept the degraded features. It reconstructs them, using what is visible around the occluder together with what the text and exemplars say the object should look like. It then checks that the occluded view attends to the same places as a clean view of the scene.

Method

Reconstruct, then align

1. Feature Reconstruction Module (FRM)

At each of three Swin Transformer pyramid levels (256, 512 and 1024 channels), visible tokens are kept and occluded positions are replaced by a learnable mask embedding. These queries self-attend, cross-attend to the visible tokens for spatial context, and then cross-attend to the fused text–exemplar embedding for semantics.

A frozen teacher sees the clean image and provides target features at the occluded positions. Reconstruction is supervised with a Charbonnier, cosine and \(\ell_2\) loss, summed over levels.

2. Visual Equivalence (VisEQ)

FRM fixes the features. VisEQ fixes where the model looks. It computes language-conditioned Grad-CAM maps across pyramid levels for a teacher on the clean image and a student on the occluded one, and aligns them.

A region-of-interest consistency loss \(\mathcal{L}_{\mathrm{cst}}\) rewards high, low-variance activation wherever either network is confident, which prevents the trivial solution of both maps going flat.

3. New occlusion benchmarks

FSC-147-OCC and CARPK-OCC add controlled occlusion to the FSC-147 open-world counting benchmark (147 categories) and the CARPK car-counting dataset, with separate ground truth for visible and occluded objects.

Results

MAE / RMSE · lower is better

FSC-147-OCC

Where the gain comes from

Splitting the error into visible and occluded regions shows CountOCC keeps visible-region accuracy about the same while cutting occluded-region error by roughly a factor of three. The model is reasoning about what is hidden, not just exploiting the occluder's appearance.

Generalization

On the original, unoccluded FSC-147, CountOCC stays competitive with a test MAE of 7.02, second only to CountGD (5.74), so robustness to occlusion does not come at the expense of normal counting.

What matters

Ablations on FSC-147-OCC

• Reconstruct at every scale. FRM at one level helps MAE but not large errors. All three levels cut validation MAE by 28.5%.
• VisEQ tames the worst cases. Adding attention alignment more than halves test RMSE (91.45 → 38.68), meaning far fewer large miscounts.

Limitations

Citation

@article{arib2025countocc,
  title   = {Counting Through Occlusion: Framework for Open World Amodal Counting},
  author  = {Arib, Safaeid Hossain and Akter, Rabeya and Chowdhury, Abdul Monaf and
             Sourov, Md Jubair Ahmed and Hasan, Md Mehedi},
  journal = {arXiv preprint arXiv:2511.12702},
  year    = {2025}
}