Paper Library
A personal library of papers I have read.
3D Perception & BEV
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers
BEVFormer turns a calibrated multi-camera rig into a persistent BEV query space, using geometry-aware spatial attention and recurrent temporal fusion instead of explicit depth-based lifting.
MonoScene: Monocular 3D Semantic Scene Completion
MonoScene shows that dense 3D geometry and semantics can be inferred from one RGB image by connecting a 2D encoder to a 3D completion network through line-of-sight feature projection and context-aware supervision.
TPVFormer: Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction
TPVFormer extends BEV into three orthogonal feature planes, preserving height-aware 3D structure at near-quadratic rather than cubic representation cost.
OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction
OccFormer makes dense voxel reasoning tractable by decomposing 3D attention into local horizontal structure and a pooled global scene path, then adapts mask-classification decoding to sparse, imbalanced occupancy labels.
OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
OccWorld treats semantic occupancy as the state of a generative driving world model, tokenizing past 3D scenes and autoregressively predicting both future occupancy and ego motion.
Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
Where2comm converts collaborative perception from full-map broadcasting into spatially selective exchange: agents transmit only confidence-selected regions and request complementary evidence over multiple rounds.
Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles
CoHFF extends camera-based collaboration from boxes and BEV maps to dense semantic occupancy by fusing occupancy and semantic task features locally while exchanging compressed orthogonal plane features across vehicles.
Token Merging: Your ViT But Faster (ToMe)
The canonical training-free token merging paper: it treats redundancy as information compression, not token deletion.
ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformers
A dense-prediction token merging paper whose key lesson is that token similarity is depth-dependent: shallow layers support local merging, while deeper layers support global semantic merging.
Accelerating Transformers with Spectrum-Preserving Token Merging (PiToMe)
PiToMe adds an importance-aware gate before similarity matching: redundant clusters may be compressed, while isolated and distinctive tokens are protected.
Agglomerative Token Clustering (ATC)
ATC reframes token merging as bottom-up hierarchical clustering, producing coherent groups instead of committing to one-shot bipartite pairs.
Token Fusion: Bridging the Gap between Token Pruning and Token Merging (ToFu)
ToFu argues that pruning and merging are appropriate in different feature regimes, then introduces MLERP to reduce the feature-norm shift caused by plain averaging.
Training-Free Acceleration of ViTs with Delayed Spatial Merging (DSM)
DSM shows that merging should be delayed until attention becomes convergent, then widened from local windows to global matching as representations grow more semantic.
ToSA: Token Merging with Spatial Awareness
ToSA fuses semantic similarity with explicit 3D spatial similarity from depth, giving early ViT layers a geometric signal that visual features alone do not yet provide.
MPM: Mutual Pair Merging for Efficient Vision Transformers
MPM accepts a merge only when two tokens choose each other as nearest neighbors, then records a composable map that reconstructs dense spatial features before an unchanged segmentation decoder.
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
CacheBlend makes reusable document KV caches practical for RAG by selectively repairing the tokens whose cached states become invalid in a new context.
Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation
Cache-Craft treats RAG prefill as a cache-management problem and repairs reusable chunk states according to how much their new context changes attention.
Towards Tight Bounds for Streaming Attention
This paper gives nearly tight memory bounds for approximating softmax attention in a stream and clarifies when sublinear KV-cache representations are theoretically possible.
Gradient Intrinsic Dimensionality Alignment: Narrowing the Gap Between Low-Rank Adaptation and Full Fine-Tuning
RaLoRA argues that LoRA fails when its fixed rank is misaligned with the layer-specific intrinsic dimension of full-fine-tuning gradients.
E²LoRA: Efficient and Effective Low-Rank Adaptation with Entropy-Guided Adaptive Sharing
E²LoRA reduces adapter redundancy by sharing parameters only across locally similar layers and assigning rank according to layer information content.
FedLEASE: Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
FedLEASE learns how many LoRA experts a heterogeneous federation needs, which clients should train each expert, and how many experts each client should use.
Don't Reinvent the Wheel, Just Realign the Spokes: Resource-Efficient Federated Fine-Tuning via Rank-Wise Expert Assembly
SMARTFED replaces federated LoRA training from scratch with fine-grained reuse of existing LoRA rank components.
Heterogeneous Customizable Personalized Federated Fine-Tuning Approach for Large Language Models
Het-CPFLoRA lets clients choose different personalization capacity while preserving a shared generalized LoRA subspace.
Federated Sketching LoRA: A Flexible Framework for Heterogeneous Collaborative Fine-Tuning of LLMs
FSLoRA keeps a high-rank global adapter while allowing each client to update only a resource-compatible sketched submatrix.
Efficient Agents: Building Effective Agents While Reducing Cost
Effective agent design is a joint optimization problem across model choice, reasoning policy, tool orchestration, and prompt context rather than a single-model accuracy contest.
ParaVT: Parallel Visual Tool Calling for Efficient Multi-Agent Visual Reasoning
ParaVT replaces serial visual-tool execution with coordinated parallel calls, making tool diversity useful without paying the full latency of a long sequential chain.
Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning
Two agents can reach the same answer while differing sharply in wasted calls, redundant verification, tool choice, and format stability; accuracy alone cannot reveal that difference.
Context Compression for LLM Agents: A Survey of Methods, Failure Modes, and Evaluation
Agent context compression should be understood as a controlled Select-Compress-Store-Recover pipeline whose failures can occur before, during, or after compression.
ContextBudget: Efficient and Effective Context Allocation for LLM Agents
ContextBudget treats context length as a sequential resource-allocation decision, spending tokens where they increase task success instead of enforcing one static window for every step.
HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems
An agent adapts best when its external execution structure and internal reasoning policy evolve as a compatible pair rather than as independent components.
FederatedSkill: Federated Learning for Agentic Skill Evolution
FederatedSkill communicates semantic skill diffs instead of raw trajectories and uses server-side capability memories to evolve a different skill library for each client.
Autogenesis: A Self-Evolving Agent Protocol
Autogenesis makes prompts, agents, tools, environments, and memory explicit versioned resources, then governs their evolution through auditable propose-verify-commit operations.
Agent Skill Evaluation and Evolution: Frameworks and Benchmarks
Agent skills are reusable procedural packages whose quality must be evaluated across creation, retrieval, execution, evolution, and safety rather than by downstream success alone.
Automating Agent Hijacking via Structural Template Injection
Phantom attacks the serialized role structure of agent context rather than relying only on persuasive malicious text, exposing a boundary failure between trusted instructions and retrieved data.
GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs
MoE safety behavior can concentrate in a small routed subset of experts and neurons, creating an architecture-specific failure mode that aggregate alignment evaluation can hide.
The Cold-Start Safety Gap in LLM Agents
Tool-calling agents are least safe at the beginning of a session and become safer after ordinary preceding tasks, showing that safety depends on conversational state rather than only the current request.
PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails
Image safety is a relation between visual evidence and the active policy, so a useful guardrail must correctly flip its decision when the policy changes while the image stays fixed.
Rate-Distortion Optimized Communication for Collaborative Perception
RDComm gives collaborative perception an information-theoretic objective: encode task-relevant features compactly and transmit only complementary messages.
UniRefiner: Teaching Pre-trained ViTs to Self-Dispose Dross via Contrastive Register
UniRefiner diagnoses spurious visual tokens and turns learned register supervision into a plug-in token filter for frozen pretrained ViTs.
CoST: Efficient Collaborative Perception From Unified Spatiotemporal Perspective
CoST unifies temporal memory and multi-agent communication in one feature-retrieval mechanism instead of building two separate fusion stacks.
Categorical Depth Distribution Network for Monocular 3D Object Detection
CaDDN keeps monocular depth uncertainty as a categorical distribution and uses it to lift image features into a metric 3D volume.
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Depth Anything shows that data scale, pseudo-label quality, and semantic preservation can build a robust zero-shot depth prior without a complicated depth-specific architecture.
Depth Anything V2
Depth Anything V2 uses precise synthetic supervision for a giant teacher and diverse real images for student distillation, separating geometric accuracy from visual coverage.
Depth Anything 3: Recovering the Visual Space from Any Views
Depth Anything 3 reduces multi-view geometry to a single Transformer and a unified depth-ray target that jointly represents scene structure and camera motion.
Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
Marigold-DC treats sparse measurements as test-time guidance for a pretrained generative depth prior, enabling zero-shot completion without task-specific training.
VGGT: Visual Geometry Grounded Transformer
VGGT predicts cameras, depth, point maps, and tracks for many images in one feed-forward pass, making a Transformer itself the geometry engine.
MCOP: Multi-UAV Collaborative Occupancy Prediction
MCOP moves collaborative occupancy from road vehicles to UAV swarms and jointly compresses messages, requests useful regions, and decodes a shared aerial scene.
Cross-view Transformers for Real-time Map-view Semantic Segmentation
CVT learns a direct attention bridge from camera feature maps to a canonical map-view grid and shows that explicit depth is not required for useful real-time BEV segmentation.
PETR: Position Embedding Transformation for Multi-View 3D Object Detection
PETR injects camera-aware 3D coordinates directly into image features so object queries can reason in 3D without first building a dense BEV grid.
ODG: Occupancy Prediction Using Dual Gaussians
ODG assigns different Gaussian query families to static structure and moving objects, then uses hierarchical decoding and rendering supervision for efficient dense occupancy.
AutoOcc: Automatic Open-Ended Semantic Occupancy Annotation via Vision-Language Guided Gaussian Splatting
AutoOcc turns vision-language attention and differentiable Gaussian reconstruction into an automatic open-ended 3D occupancy annotation pipeline.
COME: Adding Scene-Centric Forecasting Control to Occupancy World Model
COME disentangles scene evolution from ego-view change and injects a scene-centric forecast as ControlNet guidance into a generative occupancy world model.
COTR: Compact Occupancy Transformer for Vision-based 3D Occupancy Prediction
COTR replaces a full-resolution 3D occupancy tensor with compact triplane features and semantic groups, reserving dense decoding for the final prediction.
GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splatting
GaussianOcc learns occupancy from images alone by representing the scene with predicted Gaussians and supervising them through cross-view semantic and depth rendering.
Monocular Occupancy Prediction for Scalable Indoor Scenes
ISO injects a pretrained depth prior into a depth-distribution lifting module to scale monocular semantic scene completion across larger indoor volumes.
OPUS: Occupancy Prediction Using a Sparse Set
OPUS predicts a dense occupancy scene from a sparse set of point queries that iteratively sample image features and refine their positions and semantics.
OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexity
OccRWKV replaces quadratic global attention with linear RWKV-style mixing across semantic, BEV, and point branches for efficient LiDAR occupancy.
OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries
OctreeOcc spends fine-resolution queries only near complex occupied regions and keeps empty or simple space coarse.
3D Occupancy Prediction with Low-Resolution Queries via Prototype-aware View Transformation
This ProtoOcc preserves fine image structure under low-resolution 3D queries by mapping 2D feature hierarchies into learned voxel prototypes before view transformation.
QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Prediction
QuadricFormer represents a scene with a small set of volumetric superquadrics whose shape parameters naturally model both occupied interiors and object geometry.
Semantic Causality-Aware Vision-Based 3D Occupancy Prediction
CausalOcc couples the 2D backbone, 3D lifting, and occupancy decoder through semantic causality so downstream errors update the features that caused them.
Fully Sparse 3D Occupancy Prediction
SparseOcc keeps the entire camera-to-occupancy pipeline sparse, from 3D reconstruction through semantic and instance mask prediction.
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving
SurroundOcc establishes a direct multi-camera dense-voxel baseline by lifting multi-scale image features into 3D and progressively refining occupancy.
UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving
UniOcc unifies current occupancy prediction and future occupancy forecasting across real and simulated datasets, with voxel flow and ground-truth-free quality metrics.
VGMOcc: Sparse Gaussian Occupancy Prediction with Visual Geometry Models
VGMOcc converts VGM surface points into volumetric Gaussian occupancy by sampling inward along camera rays and incrementally fusing predictions across frames.
ProtoOcc: Accurate, Efficient 3D Occupancy Prediction Using Dual Branch Encoder-Prototype Query Decoder
ProtoOcc combines fine voxel detail and broad BEV context, then decodes occupancy from a small set of scene-aware semantic prototypes.
Discovering Visual Patterns in Art Collections with Spatially-Consistent Feature Learning
The paper mines hard positive correspondences from neighborhood consistency and uses them to adapt features across artistic style changes without labels.
Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception
SCOPE calibrates both temporal and cross-agent domain gaps before fusion, then weights evidence according to confidence and task importance.
OccuFly: A 3D Vision Benchmark for Semantic Scene Completion from the Aerial Perspective
OccuFly establishes real-world aerial semantic scene completion with RGB, metric depth, and occupancy labels across seasons, altitudes, and scene types.
V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Prediction
V2XPnP couples multi-agent temporal fusion with current 3D detection and future trajectory prediction in one shared V2X scene state.
Vision-Only Gaussian Splatting for Collaborative Semantic Occupancy Prediction
VOGS-CP communicates sparse scene Gaussians rather than dense voxels or flat BEV features, preserving three-dimensional structure for collaborative occupancy.
Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding
Video-3D LLM aligns video patch tokens with global 3D coordinates so an LLM can answer questions, caption regions, and ground objects in metric space.
NeRF²: Neural Radio-Frequency Radiance Fields
NeRF² adapts coordinate-based neural fields and differentiable rendering from light to radio propagation, reconstructing RF signals throughout a complex environment.
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
NeRF represents a scene as a continuous coordinate MLP and trains it through differentiable volume rendering from posed images.
Bridging 3D Anomaly Localization and Repair via High-Quality Continuous Geometric Representation
PASDF aligns a test point cloud to a canonical pose and learns a continuous signed-distance representation that supports both anomaly localization and geometric repair.
CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridization
CoSDH selects messages by matching what the sender can supply with what the receiver needs, then combines intermediate and late fusion for robustness.
Task-Oriented Communication for Vehicle-to-Infrastructure Cooperative Perception
TOCOM-V2I uses infrastructure-side spatial relevance and hierarchical entropy coding to send compact features that directly maximize vehicle detection quality.
Vehicle-to-Everything Cooperative Perception for Autonomous Driving
This survey organizes V2X cooperative perception from sensing and preparation through communication, alignment, fusion, and downstream output, while exposing deployment gaps.
V2X-PC: Vehicle-to-Everything Collaborative Perception via Point Cluster
V2X-PC replaces dense BEV messages with point clusters that preserve local geometry, semantics, and explicit object structure over long collaboration ranges.
Who2com: Collaborative Perception via Learnable Handshake Communication
Who2com separates cheap partner discovery from expensive data exchange through a request-match-connect handshake learned for perception utility.
TopNet: Transformer-Efficient Occupancy Prediction Network for Octree-Structured Point Cloud Geometry Compression
TopNet predicts octree-node occupancy with a CNN-Transformer hybrid that combines local structure, adaptive-window context, channel gating, and adjacent-node priors.
Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Acceleration
SDTM observes that DiT redundancy changes over denoising time and layer depth, then prunes weak structure first and merges weak detail later with adaptive thresholds.
Attention Is All You Need
The Transformer replaces recurrence with parallel multi-head attention, positional encoding, residual pathways, and feed-forward blocks in an encoder-decoder architecture.