A³Bench annotations cover Aggressor, Victim, Bystander, Aggressive Action (17 categories + 1 "none" control), and Environment using appearance-based clothing descriptions rather than biometric identity. Questions progress across three compositional tiers:
Tests whether a model can recover an isolated scene attribute before relational binding: Primary Action (1,802), Role ID (2,233), Aggressor ID (1,519), and Victim Recognition (1,479).
Tests whether models can jointly bind two scene components correctly: Action+Aggressors (2,254), Action+Victims (2,211), and Aggressor+Victim (1,203).
Requires recovering complete who-did-what-to-whom interaction directionality: Aggressor+Action+Victim (1,212) and Sequence Verification (1,091).
Zero-shot evaluation of 15 MLLM configurations under a unified multiple-choice VQA protocol. Most state-of-the-art models score below 50% overall accuracy, with steep degradation on Detailed questions.
| Model | Type / Prompting | Basic (%) ↑ | Compound (%) ↑ | Detailed (%) ↑ | Overall (%) ↑ | Category Avg (%) ↑ |
|---|---|---|---|---|---|---|
| Qwen3-VL-8B-Instruct | Open-Weight (8B) | 51.07 | 46.65 | 35.19 | 46.96 | 44.30 |
| InternVL2.5-8B | Open-Weight (8B) | 52.61 | 45.01 | 26.75 | 45.77 | 41.46 |
| InternVL3.5-8B DoT | Dream-of-Thoughts Prompt | 49.64 | 40.31 | 40.90 | 44.77 | 43.62 |
| InternVL3.5-8B CoT | Chain-of-Thought Prompt | 51.53 | 38.92 | 33.13 | 43.94 | 41.19 |
| Ovis2.5-9B | Open-Weight (9B) | 50.62 | 37.99 | 33.09 | 43.16 | 40.57 |
| gemma-4-26B-A4B-it | Open-Weight MoE (26B-A4B) | 49.07 | 39.65 | 30.86 | 42.72 | 39.86 |
| InternVL3-9B | Open-Weight (9B) | 50.05 | 39.06 | 28.22 | 42.55 | 39.11 |
| GPT 5.1 | Closed-Source API | 45.63 | 39.17 | 33.32 | 41.30 | 39.37 |
| InternVL3.5-8B | Open-Weight (8B) | 48.34 | 37.03 | 28.75 | 41.06 | 38.04 |
| InternVideo2.5-8B | Open-Weight (8B) | 48.07 | 39.34 | 21.51 | 40.70 | 36.31 |
| Qwen2.5-VL-72B-Instruct | Open-Weight (72B Quant.) | 45.73 | 39.47 | 26.91 | 40.53 | 37.37 |
| Qwen3-VL-8B-Thinking | Reasoning Variant (8B) | 44.59 | 36.41 | 30.77 | 39.37 | 37.26 |
| VideoLLaMA3-7B | Open-Weight (7B) | 49.65 | 30.86 | 21.41 | 38.22 | 33.97 |
| Qwen2.5-VL-7B-Instruct | Open-Weight (7B) | 41.87 | 37.15 | 23.73 | 37.30 | 34.25 |
| LLaVA-Video-7B-Qwen2 | Open-Weight (7B) | 19.58 | 19.34 | 13.11 | 18.50 | 17.34 |
| Model | Primary Act. | Role ID | Agg. ID | Victim Rec. | Act.+Agg. | Act.+Vic. | Agg.+Vic. | Agg.+Act.+Vic. | Seq. Verif. | Overall |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-VL-8B-Instruct | 46.47 | 55.42 | 53.06 | 48.04 | 49.27 | 43.89 | 46.80 | 34.60 | 35.84 | 46.96 |
| InternVL2.5-8B | 46.00 | 56.02 | 55.89 | 52.13 | 42.15 | 45.59 | 49.29 | 25.90 | 27.59 | 45.77 |
| InternVL3.5-8B DoT | 37.85 | 51.86 | 57.08 | 53.01 | 40.15 | 36.64 | 47.38 | 39.77 | 42.16 | 44.77 |
| InternVL3.5-8B CoT | 37.74 | 58.53 | 58.20 | 50.91 | 38.60 | 36.23 | 44.47 | 32.51 | 33.82 | 43.94 |
| Ovis2.5-9B | 39.84 | 54.81 | 55.63 | 52.27 | 35.63 | 34.83 | 48.21 | 33.50 | 32.63 | 43.16 |
| gemma-4-26B-A4B-it | 48.92 | 48.81 | 53.10 | 45.50 | 37.92 | 36.80 | 48.13 | 30.00 | 31.81 | 42.72 |
| InternVL3-9B | 40.34 | 55.71 | 53.19 | 50.10 | 39.93 | 35.28 | 44.39 | 30.94 | 25.21 | 42.55 |
| GPT 5.1 | 44.06 | 51.66 | 45.59 | 38.50 | 39.77 | 37.87 | 40.43 | 33.75 | 32.84 | 41.30 |
| InternVL3.5-8B | 36.13 | 53.61 | 53.85 | 49.63 | 35.98 | 32.43 | 47.46 | 28.80 | 28.69 | 41.06 |
| InternVideo2.5-8B | 43.70 | 54.68 | 49.67 | 41.77 | 37.39 | 38.98 | 43.64 | 21.32 | 21.72 | 40.70 |
| Qwen2.5-VL-72B-Instruct | 39.00 | 50.54 | 50.59 | 41.67 | 39.22 | 35.10 | 48.67 | 26.53 | 27.34 | 40.53 |
| Qwen3-VL-8B-Thinking | 33.70 | 51.64 | 51.19 | 40.42 | 35.44 | 32.19 | 45.97 | 30.41 | 31.16 | 39.37 |
| VideoLLaMA3-7B | 43.34 | 53.87 | 53.98 | 46.52 | 22.63 | 35.05 | 38.57 | 20.96 | 21.91 | 38.22 |
| Qwen2.5-VL-7B-Instruct | 41.48 | 42.90 | 43.94 | 38.66 | 36.77 | 34.18 | 43.31 | 24.96 | 22.36 | 37.30 |
| LLaVA-Video-7B-Qwen2 | 19.37 | 23.11 | 20.67 | 13.39 | 16.19 | 15.51 | 32.25 | 13.20 | 13.02 | 18.50 |
| Benchmark | Action Classes (Cls) | # Videos | Explicit Roles | Person Descriptions | Multi-Person Assoc. | Diagnostic VQA Pairs |
|---|---|---|---|---|---|---|
| DVD (2025) | 2 | 500 | ✗ | ✗ | ✗ | ✗ |
| Movies Fight (2011) | 2 | 200 | ✗ | ✗ | ✗ | ✗ |
| RLVS (2025) | 2 | 2,000 | ✗ | ✗ | ✗ | ✗ |
| RWF-2000 (2021) | 2 | 2,000 | ✗ | ✗ | ✗ | ✗ |
| VID (2024) | 2 | 3,020 | ✗ | ✗ | ✗ | ✗ |
| Violent Flows (2012) | 2 | 246 | ✗ | ✗ | ✗ | ✗ |
| CABAD (2025) | 6 | 900 | ✗ | ✗ | ✗ | ✗ |
| CRxK (2025) | 13 | 3,484 | ✗ | ✗ | ✗ | ✗ |
| IARD (2025) | 4 | 802 | ✗ | ✗ | ✗ | ✗ |
| UCF-Crime (2018) | 14 | 1,900 | ✗ | ✗ | ✗ | ✗ |
| XD-Violence (2020) | 7 | 4,754 | ✗ | ✗ | ✗ | ✗ |
| A³Bench (Ours) | 18 | 2,683 | ✓ | ✓ | ✓ | ✓ (15,004) |
| Model | Primary Action (%) ↑ | Action + Location (%) ↑ | Delta (Δ) |
|---|---|---|---|
| gemma-4-26B-A4B-it | 48.92 | 31.90 | -17.02 |
| Qwen3-VL-8B-Instruct | 46.47 | 46.74 | +0.27 |
| InternVL2.5-8B | 46.00 | 38.22 | -7.78 |
| GPT 5.1 | 44.06 | 40.91 | -3.15 |
| InternVideo2.5-8B | 43.70 | 33.53 | -10.17 |
| VideoLLaMA3-7B | 43.34 | 35.90 | -7.44 |
| Qwen2.5-VL-7B-Instruct | 41.48 | 33.02 | -8.46 |
| InternVL3-9B | 40.34 | 35.90 | -4.44 |
| Ovis2.5-9B | 39.84 | 37.70 | -2.14 |
| Qwen2.5-VL-72B-Instruct | 39.00 | 35.71 | -3.29 |
| InternVL3.5-8B DoT | 37.85 | 40.70 | +2.85 |
| InternVL3.5-8B CoT | 37.74 | 38.30 | +0.56 |
| InternVL3.5-8B | 36.13 | 38.99 | +2.86 |
| Qwen3-VL-8B-Thinking | 33.70 | 31.39 | -2.31 |
| LLaVA-Video-7B-Qwen2 | 19.37 | 17.91 | -1.46 |
Why do MLLMs fail on aggressive interactions? Our analysis reveals monotonic degradation with compositional complexity and a systematic Aggressor Bias where victims (37%) and bystanders (31%) are misclassified as aggressors.
Accuracy (%) across Basic (46.54% avg), Compound (37.76% avg), and Detailed (28.51% avg) tiers.
Predicted role distribution (%) for ground-truth Victims, Aggressors, and Bystanders averaged across models.
To mitigate aggressor bias and improve role-action binding without additional training, we formulate aggressive-scene understanding as a causal Directed Acyclic Graph (DAG) prepended to each query.
Directed edges enforce step-by-step participant grounding before action classification:
| Model | Baseline Overall (%) | Role-Graph Basic (%) | Role-Graph Compound (%) | Role-Graph Detailed (%) | Role-Graph Overall (%) ↑ | Gain (Δ) |
|---|---|---|---|---|---|---|
| InternVL2.5-8B | 45.77 | 54.54 | 46.35 | 33.06 | 48.15 | +2.38% |
| Ovis2.5-9B | 43.16 | 55.08 | 46.94 | 43.83 | 50.28 | +7.12% |
| Qwen3-VL-8B-Instruct | 46.96 | 54.81 | 48.65 | 42.88 | 50.65 | +3.69% |
Your primary task is to act as a causal reasoning engine for understanding aggressive scenes in videos. You will observe the persons present and use a causal graph to identify each person's role and the nature of the aggressive interaction. ### Component Definitions - P (Persons): All individuals visible in the video. Each person must be assigned exactly one role. - I (Interaction): Whether a given person is engaged in a direct physical interaction with another person (YES / NO). - D (Direction): For persons where I = YES, the direction of the aggressive interaction (INITIATING / RECEIVING). - R (Role): * Aggressor: a person where I = YES and D = INITIATING. * Victim: a person where I = YES and D = RECEIVING. * Bystander: a person where I = NO, not physically involved. - A (Aggressive Action): The specific type of aggressive behavior being performed, inferred jointly from the Aggressor's motions and the Victim's reactions. ### Directed Edges of the Causal Graph (DAG) P -> I I(YES) -> D I(NO) -> R(Bystander) D(INITIATING) -> R(Aggressor) D(RECEIVING) -> R(Victim) R(Aggressor) + R(Victim) -> A
If you find A³Bench useful in your research, please consider citing our NeurIPS 2026 paper:
@inproceedings{a3bench2026neurips,
title = {A\textsuperscript{3}Bench: A Benchmark for Compositional Reasoning over Aggressive Interactions in Videos},
author = {Anonymous Authors},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2026}
}