Human experts excel at fine-grained visual discrimination by leveraging domain knowledge to refine perceptual features, a capability that remains underdeveloped in current Multimodal Large Language Models (MLLMs). Despite possessing vast expert-level knowledge, MLLMs struggle to integrate reasoning into visual perception, often generating direct responses without deeper analysis.
To bridge this gap, we introduce knowledge-intensive visual grounding (KVG), a novel visual grounding task that requires both finegrained perception and domain-specific knowledge integration. To address the challenges of KVG, we propose DeepPerception, an MLLM enhanced with cognitive visual perception capabilities. Our approach consists of (1) an automated data synthesis pipeline that generates high-quality, knowledge-aligned training samples, and (2) a two-stage training framework combining supervised fine-tuning for cognitive reasoning scaffolding and reinforcement learning to optimize perception-cognition synergy. To benchmark performance, we introduce KVG-Bench, a comprehensive dataset spanning 10 domains with 1.3K manually curated test cases.
Experimental results demonstrate that DeepPerception significantly outperforms direct fine-tuning, achieving +8.08% accuracy improvements on KVG-Bench and exhibiting +4.60% superior cross-domain generalization over baseline approaches. Our findings highlight the importance of integrating cognitive processes into MLLMs for human-like visual perception and open new directions for multimodal reasoning research.

The primary objective of the first stage training is to endow models with visual perception-oriented cognitive capabilities through synthesized CoT training data. This process fosters an initial cognitive-perceptual synergy by explicitly modeling reasoning trajectories in the KVG task while establishing fundamental knowledge association patterns for subsequent optimization. Specifically, we leveraged the powerful Qwen2-VL-72B model to generate CoT reasoning data. As illustrated in Fig. 4, we generated CoT rationales by inputting images, ground-truth annotations (labels and bounding boxes), and CoT reasoning prompts into the model. This process synthesizes reasoning chains that combine the model's domain knowledge with human-like cognitive processes (e.g., iterative hypothesis verification through joint domain knowledge and visual feature analysis). We then performed SFT on the Qwen2-VL-7B model using these data, yielding the stage-1 model with foundational cognitive visual perception capabilities. This model demonstrates preliminary abilities to integrate domain knowledge and generate stepwise reasoning chains for visual grounding, which serves as a stable foundation for the second-stage reinforcement learning training. This approach bridges the gap between raw perception and expert-level cognition, enabling models to emulate human-like visual perception through knowledge-visual co-verification.

Following the initial CoT-SFT training, we conducted reinforcement learning using a separate subset of training data to further enhance the model's perception capabilities, building upon its acquired cognitive visual perception foundation. As illustrated in Fig. 3, we adopted GRPO following Guo et al, which optimizes the policy model by sampling a group of outputs to a question and calculating a group-relative advantage instead of a critic model. To adapt GRPO for visual grounding tasks, we designed a rule-based reward system and performed data filtering on the training data using the stage-1 model.

Training Objective. Let $q$ denote the question and ${o_1, . . . , o_G}$ denote the sampled outputs from the old policy model πθold , then the optimization objective of policy model $\pi_{\theta}$ can be formally defined as:

$\mathcal{J}_{G R P O}(\theta)=\mathbb{E}\left[q \sim P(Q),\left\{o_i\right\}_{i=1}^G \sim \pi_{\theta_{o l d}}(O \mid q)\right]\frac{1}{G} \sum_{i=1}^G \left(\frac{\pi_\theta\left(o_i \mid q\right)}{\pi_{\theta_{o l d}}\left(o_i \mid q\right)} A_i - \beta \mathbb{D}_{K L}\left(\pi_\theta| | \pi_{r e f}\right)\right),$

$\mathbb{D}_{K L}\left(\pi_\theta| | \pi_{r e f}\right)=\frac{\pi_{r e f}\left(o_i \mid q\right)}{\pi_\theta\left(o_i \mid q\right)}-\log \frac{\pi_{r e f}\left(o_i \mid q\right)}{\pi_\theta\left(o_i \mid q\right)}-1$

where $\varepsilon$ and $\beta$ are hyper-parameters and $A_i$ is the advantage computed based on a group of rewards $\{r_1, \ldots, r_G\}$ corresponding to the outputs of each group:

$A_i=\frac{r_i-mean(\{r_1, \ldots, r_G\})}{std(\{r_1, \ldots, r_G\})}$

Reward Modeling. We adopted a rule-based reward system specifically tailored for visual grounding tasks, comprising two types of rewards: Intersection over Union (IoU) reward and Format reward. The IoU reward evaluates the spatial alignment between predicted bounding boxes and ground-truth annotations. Given the ground-truth bounding box $B$ and the predicted bounding box $\tilde{B}$, the IoU reward is formally defined as:

$R_{\mathrm{IoU}}= \begin{cases} \operatorname{IoU}(B, \tilde{B}), & \text { if } \operatorname{IoU}(B, \tilde{B}) \geq \tau \\ 0, & \text { otherwise } \end{cases}$

As demonstrated in Tab. 1, our DeepPerception model achieves significant performance improvements by integrating cognitive visual perception capabilities, validating the hypothesis that human-inspired cognition-perception synergy enhances visual perception. On in-domain categories, DeepPerception attains 63.13% average accuracy, surpassing all 7B-scale baselines (e.g., +10.5% over SFT) and outperforming 70B-scale models such as InternVL2-Llama3-76B (59.22%). For out-of-domain generalization, it achieves 60.85% accuracy—comparable to 70B-scale models—while demonstrating near-human performance in categories where the model possesses richer knowledge, such as Dog (85.60% vs. human evaluators’78.89%).
These results confirm that DeepPerception’s success stems from its ability to emulate human-like cognitive processes: iterative knowledge-guided reasoning refines perceptual hypotheses (e.g., verifying anatomical features stepwise), while reinforcement learning aligns these hypotheses with precise visual outputs. The consistent superiority over non-cognitive baselines, particularly in fine-grained categories like Car (86.54% vs. human 85.56%), proves that structured knowledge integration—not mere visual memorization —drives performance gains. By bridging knowledge reasoning with perception, DeepPerception establishes new state-of-the-art results on KVG-Bench, demonstrating that cognitive mechanisms central to biological vision can be effectively operationalized in multimodal AI systems.

To validate the generalizability of cognitive visual perception, we conducted experiments on FGVR datasets. Unlike the two-stage training paradigm used for KVG, only the reinforcement learning phase was applied here, as base models already exhibit strong baseline performance on FGVR. As shown in Tab. 2, our DeepPerception model achieves state-of-the-art results across all categories, demonstrating universal performance gains: DeepPerception achieves state-of-the-art results across all FGVR categories (83.21% average accuracy), outperforming Qwen2-VL-7B by +3.64%. This evidence strongly supports that cognitive visual perception, characterized by iterative knowledge-visual alignment, provides universal benefits for fine-grained perception, mirroring human experts’ ability to combine sensory input with conceptual understanding.

To comprehensively assess the model’s general capabilities across diverse multimodal scenarios, we conducted evaluations on established multimodal benchmarks. As shown in Tab. 3, DeepPerception maintains performance comparable to the base model, demonstrating preserved general capabilities without degradation.

How knowledge influences perception. We systematically evaluated the base model’s knowledge by probing entity-specific discriminative features and validating factual alignment against Wikipedia references using Qwen2.5-14B. Entities were then classified into four groups based on knowledge possession (Known/Unknown) and training data presence (in-domain/out-of-domain). As evidenced in Fig. 5, DeepPerception achieves significantly greater performance gains on known entities versus unknown entities regardless of domain boundaries, confirming that its improvements originate from knowledge-driven cognitive processing rather than superficial perceptual enhancements, thereby validating its capacity to harness domain knowledge for visual perception. This aligns with human experts’ reliance on domain expertise to resolve ambiguity, confirming our framework’s success in emulating cognitive visual perception.

The roles of CoT-SFT and GRPO. As shown in Fig. 6, we quantified the impact of our two-stage training paradigm by measuring the KL Divergence between stage-2 models (trained with CoT-SFT or GRPO) and the stage-1 model, separately evaluating the CoT and answer components. For CoT-SFT, the significantly higher KL Divergence in the CoT components versus answer segments indicates that CoT-SFT primarily equips the model with perception-oriented reasoning capabilities. In contrast, GRPO exhibited greater divergence in the answer components than in the CoT, demonstrating its focus on refining perceptual accuracy grounded in the reasoning process. By combining the two stages, the framework achieves synergistic enhancement of cognitive visual perception, effectively bridging knowledge-driven reasoning and sensory processing.

We discuss two empirical findings of GRPO, with detailed results provided in the appendix: (1) Unlike DeepSeek-R1-Zero’s progressive increase in response length during training, GRPO training on both KVG and FGVR data exhibits stable completion lengths fluctuating within specific ranges, suggesting bounded reasoning complexity in visual perception tasks; (2) We observed instances where incorrect cot rationales led to correct answers, indicating that the presence of cognitive processes, rather than their length or even factual accuracy, is the primary determinant of performance improvement.