πŸ“„ Notable* Recent AI/ML arXiv Papers

Last updated just now...

πŸ“„ UR-VC: Unsupervised Robotic Value Correction for Time-Derived Progress Proxies
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12892v1
πŸ‘₯ Authors: Lirui Zhao, Modi Shi, Li Chen, Qi Liu (possible past Tencent (China) affiliation), Ping Luo (possible past Shanghai Artificial Intelligence Laboratory affiliation), Hongyang Li
Abstract

Modern robot learning systems increasingly rely on dense progress or value signals to evaluate intermediate states, guide policy learning, and detect task completion, making the quality of these signals critical. Since such dense labels are rarely available at scale, normalized time within a demonstration is often used as a scalable substitute: later frames are treated as higher progress. However, this time-derived label is only a noisy proxy for physical task progress. In contact-rich manipulat...

πŸ“„ Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12659v1
πŸ‘₯ Authors: Zebin Yang, Qi Wang (possible past Tsinghua University affiliation), Yunhe Wang, Xiurui Guo, Bo Yu (possible past Baidu (China) affiliation), Shaoshan Liu, Jiafeng Xu, Hao Dong, Meng Li (possible past Meta (United States) affiliation)
Abstract

Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks. However, deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, which leads to substantial inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misa...

πŸ“„ Evidence-Grounded AI for Musculoskeletal Care
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12527v1
πŸ‘₯ Authors: Wenjie Li, Yujie Zhang, Fanrui Zhang, Haoran Sun, Renhao Yang, Junjun He, Weiran Huang, Yuanfeng Ji, Chenrun Wang, Kailing Wang, Hongcheng Gao, Kaipeng Zhang (possible past Tencent (China) affiliation), Hanyu Wang, Angela Lin Wang, Xingqi He, Yilin Huang, Shiyi Yao, Lilong Wang, Yankai Jiang, Yirong Chen, Chenglong Ma, Jiyao Liu, Ming Hu, Gen Li (possible past University Of Edinburgh affiliation), Yidong Xu, Chengyu Zhuang, Jiawei Liu, Yin Zhang, Lequan Yu, Lu Chen, Yinpeng Dong (possible past Tsinghua University affiliation), Lei Liu, Carlos Gutierrez Sanroman, Yu Qiao (possible past Shanghai Artificial Intelligence Laboratory affiliation), Weijie Ma, Xiaosong Wang (possible past Nvidia (United States) affiliation), Lei Wang (possible past Baidu (China) affiliation)
Abstract

Musculoskeletal diseases are among the leading causes of disability worldwide and create the greatest global need for rehabilitation. Because recovery, remodelling and degeneration often unfold over months to years, musculoskeletal care requires longitudinal management that repeatedly integrates evolving patient evidence, external medical knowledge and stage-specific functional goals. In routine practice, this evidence is fragmented across visits, departments and hospital systems, limiting indiv...

πŸ“„ The Computational Basis of Confidence in Large Language Models
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12447v1
πŸ‘₯ Authors: Dharshan Kumaran (possible past Google (United States) affiliation), Viorica Patraucean, Maks Ovsanikov, Petar VeličkoviΔ‡ (possible past University Of Cambridge affiliation), Nathaniel Daw
Abstract

Reliable confidence -- the probability that a model's own answer is correct -- is essential for the trustworthy deployment of language models. Existing work has largely evaluated confidence by how well it predicts correctness and whether it is calibrated, leaving open a more fundamental question: what does the confidence signal itself represent? Answer logits may reflect a latent decision variable sufficient to compute normative confidence, or instead a heuristic preference signal that combines ...

πŸ“„ ARDepth: Auto-regressive Monocular Depth Estimation with Progressive Visual Conditioning
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12433v1
πŸ‘₯ Authors: Zijie Wang, Wei Zhang (possible past Tsinghua University affiliation), Weiming Zhang, Xiao Tan (possible past Baidu (China) affiliation), Weikai Chen, Xiaoxu Li, Guanbin Li
Abstract

Diffusion models have recently become the dominant paradigm for monocular depth estimation (MDE). However, they implicitly assume that depth can be recovered as a globally smooth field through iterative denoising, which does not explicitly reflect the piecewise and scale-dependent organization of scene geometry. In practice, geometric structure emerges progressively across spatial scales, where coarse layout, surfaces, and boundaries are constructed in a hierarchical manner. Motivated by this ob...

πŸ“„ Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12406v1
πŸ‘₯ Authors: Huihao Jing, Wenbin Hu, Shaojin Chen, Haochen Shi, Sirui Zhang, Hanyu Yang, Changxuan Fan, Zhongwei Xie, Hongyu Luo, Wun Yu Chan, Wei Fan (possible past Tencent (China) affiliation), Haoran Li, Yangqiu Song (possible past Tsinghua University affiliation)
Abstract

The capability of LLM agents to function as the ``brain'' of a system fundamentally expands the scope of analysis beyond a standalone model. Consequently, safety is no longer only about input--output content alignment. It also concerns system behavior and real-world execution outcomes. However, the current literature is fragmented across attack types, applications, and benchmarks. This makes it hard to explain why failures such as prompt injection, tool misuse, and memory poisoning often share t...

πŸ“„ IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12375v1
πŸ‘₯ Authors: Jinjian Wu, Jiaqi Tang, Wei Wei (possible past Google (United States) affiliation), Yingying Yan, Jianmin Chen (possible past Google (United States) affiliation), Botong Geng, Lei Zhang, Qifeng Chen
Abstract

Image Quality Assessment (IQA) in open-world environments remains challenging due to limited generalization and interpretability. Recent approaches based on multimodal large language models (MLLMs) introduce textual reasoning for quality prediction, yet their judgments rely heavily on semantically biased internal representations, making them insensitive to low-level perceptual degradations. We propose IQA-T1, a tool-based visual evidence reasoning framework that augments MLLM reasoning with expl...

πŸ“„ Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.12112v1
πŸ‘₯ Authors: Jing Liu (possible past Baidu (China) affiliation), Chenxuanyin Zou, Jiayang Ren, Gaoyun Fang, Chengfang Li, Yan Wang (possible past Tencent (China) affiliation), Zhenchao Ma, Bo Hu
Abstract

Federated fine-tuning of Multimodal Large Language Models (MLLMs) across distributed networks enables privacy-sensitive adaptation to evolving data streams, yet a fundamental obstacle prevents robust deployment in dynamic environments: catastrophic forgetting, wherein sequential task updates erase previously acquired knowledge across visual, linguistic, and cross-modal representations. Addressing this challenge is especially critical for autonomous networked AI operating in safety-sensitive doma...

πŸ“„ PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMs
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.12111v1
πŸ‘₯ Authors: Jing Liu (possible past Baidu (China) affiliation), Kun Yang, Yan Wang (possible past Tencent (China) affiliation), Dingkang Yang, Xiaoshuai Hao, Wei Zhang (possible past Tsinghua University affiliation), Yang Liu (possible past Tsinghua University affiliation), Wei Zhou
Abstract

Agentic AI systems are reshaping communications and networking by deploying autonomous intelligent agents capable of collaborative learning while maintaining data privacy at network edges. Within distributed network environments, Multimodal Large Language Models (MLLMs) serve as cognitive engines for edge devices, yet federated fine-tuning faces substantial challenges in balancing global knowledge aggregation with local adaptation under heterogeneous network conditions. Conventional federated pr...

πŸ“„ Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11875v2
πŸ‘₯ Authors: Tiberiu Musat, Tiago Pimentel (possible past Eth Zurich affiliation), Nicolas Zucchet, Thomas Hofmann (possible past Google (United States) affiliation)
Abstract

We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models. While previous works on Transformer learning dynamics have so far been mostly tied to specific tasks, we study a generalized class of inductive tasks that unifies several synthetic tasks known in the literature, including in-context n-grams and multi-hop reasoning. In this class, we theoretically prove that the training dynamics of attention models can be confined to a hig...

πŸ“„ Evidence-Backed Video Question Answering
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11862v1
πŸ‘₯ Authors: Shijie Wang, Honglu Zhou, Ziyang Wang, Ran Xu, Caiming Xiong (possible past Salesforce (United States) affiliation), Silvio Savarese (possible past Stanford University affiliation), Chen Sun (possible past Google (United States) affiliation), Juan Carlos Niebles (possible past Stanford University affiliation)
Abstract

Current Video Large Language Models (Video LLMs) excel in question answering (QA) but largely operate as black boxes, providing textual answers without verifiable visual grounding. Existing explainability efforts rely on textual rationales or sparse bounding boxes, which struggle to capture complex video dynamics such as occlusions and non-rigid deformations. We propose Evidence-Backed Video Question Answering (E-VQA), a novel task requiring models to jointly output a semantic answer and precise...

πŸ“„ From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11689v1
πŸ‘₯ Authors: Yuanzhi Liang (possible past Baidu (China) affiliation), Xufeng Zhan, Haibin Huang, Chi Zhang (possible past Peking University affiliation), Xuelong Li (possible past Tencent (China) affiliation)
Abstract

Artificial general intelligence ultimately requires agents that can reason and act in the physical world. Action models, vision-language-action policies, and world models have advanced this goal, while World Action Models (WAMs) are particularly promising because they connect candidate interventions with predicted consequences. However, progress remains fragmented: models use incompatible action spaces and prediction targets, datasets and tasks follow different conventions, and runtime systems e...

πŸ“„ Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11643v1
πŸ‘₯ Authors: Xinghang Li, Jun Guo, Qiwei Li (possible past Tsinghua University affiliation), Long Qian, Hang Lai, Yueze Wang, Hongyu Yan, Jiahang Cao, Xi Chen (possible past University Of California, Berkeley affiliation), Jingen Qu, Jiaxi Song, Nan Sun, Hanye Zhao, Futeng Liu, Wanli Peng, Heyun Wang, Yunhong Wang, Caoyu Xia, Jack Zhao, Diyun Xiang, Hangjun Ye, Heng Qu, Huaping Liu (possible past Tsinghua University affiliation), Jason Li (possible past Nvidia (United States) affiliation)
Abstract

Recent foundation image and video generation models offer strong generalization and controllability, but their direct application to embodied scenarios is limited by requirements for multi-view consistency, geometric coherence, and robot embodiment constraints. Existing methods typically adapt foundation models with limited robot data, often sacrificing visual knowledge acquired during large-scale pre-training. We present Xiaomi-Robotics-U0, a 38-billion-parameter multimodal autoregressive model...

πŸ“„ Lesioned Multimodal Language Models Reproduce Aphasic Picture-Naming Patterns
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11621v1
πŸ‘₯ Authors: Yong Yang (possible past Tencent (China) affiliation), Xiang Guan, Sophie Arheix-Parras, Saeed Ahmadi, Roger Newman-Norlund, Leonardo Bonilha, Christopher Rorden, Julius Fridriksson, Rutvik H. Desai (possible past Google (United States) affiliation), Srihari Nelakuditi
Abstract

Aphasia following stroke commonly produces systematic naming errors with characteristic profiles, but whether general-purpose language models not designed for clinical simulation can reproduce these patterns remains untested. We investigated (1) whether lesions or controlled perturbations to a multimodal language model can reproduce different types of errors in picture naming, and (2) whether the framework can reproduce the complete error profile of individual persons with aphasia (PWAs). Using ...

πŸ“„ Technical Report on the CVPR 2026@AdvML Workshop Challenge
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11560v1
πŸ‘₯ Authors: Tianyuan Zhang, Zonglei Jing, Jiangfan Liu, Ligong Zhang, Ke Ma, Chengzhi Sun, Xiaohai Xu, Zhirui Zhang (possible past Tencent (China) affiliation), Qianqian Xu, Qingming Huang, Hanyu Fang, Junhua Liu, Zheng Wang, Xiaoliang Liu, Yuanbo Li, Shuai Gui, Bin Wang, Menghe Zheng, Jing Nie, Hanyang Meng, Zeyang Zhang, Xiang Zhang, Yongxuan Zhu, Rui Ding, Hainan Li, Yongkang Zhang, Zhilei Zhu, Xianglong Kong, Jin Hu, Zonghao Ying, Yisong Xiao, Lei Chen, Haotong Qin, Jiakai Wang, Aishan Liu, Ruikai Li, Julia Karbing, Yinpeng Dong (possible past Tsinghua University affiliation), Zhenfei Yin, Shao Jing, Xia Hu, Jingyi Xu, Juntao Dai, Xinyun Chen (possible past University Of California, Berkeley affiliation), Vishal M. Patel, Xianglong Liu, Dawn Song (possible past University Of California, Berkeley affiliation), Alan Yuille (possible past Google (United States) affiliation), Philip H. S. Torr (possible past University Of Oxford affiliation), Dacheng Tao
Abstract

Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning. This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs. Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs. Participants generate adversarial images an...

πŸ“„ LightMem-Ego: Your AI Memory for Everyday Life
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11487v1
πŸ‘₯ Authors: Yijun Chen, Boyi Xiao, Yixian Zhao, Haoting Xia, Buqiang Xu, Jizhan Fang, Yanya Li, Yaqi Zheng, Xuehai Wang, Zirui Xue, Liuxin Zhang, Hui Li (possible past Baidu (China) affiliation), Ningyu Zhang (possible past Tencent (China) affiliation)
Abstract

Personal AI assistants on mobile and wearable devices continuously perceive users' daily lives through visual and audio streams. However, answering queries about past experiences requires lightweight multimodal memory that can continuously accumulate, organize, and retrieve long-term experiences, which remains challenging. To address this challenge, we present LightMem-Ego, a lightweight streaming multimodal memory system for everyday-life assistance. The system continuously captures egocentric ...

πŸ“„ Agentic Routing: The Harness-Native Data Flywheel
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11399v1
πŸ‘₯ Authors: Xinchen Liu, Hang Zhou (possible past Baidu (China) affiliation), Yingjie Zong, Yuchuan Tian, Liuyang Song, Shuo Zhang (possible past National University Of Defense Technology affiliation), Yulong Li, Wei He (possible past Baidu (China) affiliation), Mengyu Zheng, Runke Liu, Siyang Cheng, Xiang Kuang, Hailin Hu, Kai Han, Yunhe Wang
Abstract

Large language model agents are increasingly executed not by a single model call, but by an execution harness that manages observation, context, control, action, state, and verification. At the same time, frontier and open models are becoming structurally specialized: a model that is strong at code editing, long-context recovery, tool use, mathematical reasoning, or low-latency response may not dominate on the other axes. This makes model selection inside an agent a core systems problem rather t...

πŸ“„ OpsMem: Dual-Memory Reasoning with Cross-Memory Resonance for Failure Diagnosis
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11357v1
πŸ‘₯ Authors: Yongqian Sun (possible past Tsinghua University affiliation), Rongchen Gao, Yu Luo, Wenwei Gu, Shenglin Zhang, Qingyi Guo, Qiuai Fu, Yaoliang Wu, Dan Pei (possible past Tsinghua University affiliation)
Abstract

Failure diagnosis in modern software systems requires iterative evidence acquisition and hypothesis reasoning guided by operational experience. Existing LLM-based methods improve diagnosis through agentic reasoning or knowledge augmentation, but they often lack a mechanism to coordinate the evolving diagnostic state with operational experience during iterative diagnosis. We propose OpsMem, a dual-memory framework that maintains a short-term memory for the current diagnostic state and a long-term...

πŸ“„ Verifier-Guided Twelve-Tone Composition: A Generate-Verify-Repair Harness for Symbolic Music Generation
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11334v1
πŸ‘₯ Authors: Congren Dai, Danni Zhao, Enyang Liu, Michael Ching Yam, Zhancheng Guo, Siyi Gu, Wentao Yang (possible past Google (United States) affiliation), Bo Dai, Xiaobing Li, Maosong Sun (possible past Tsinghua University affiliation)
Abstract

Large language models can produce superficially legal twelve-tone scores that collapse into degenerate textures. We introduce a neuro-symbolic harness that wraps a language-model proposer in a generate-verify-repair-trace loop with symbolic verification. The complete pipeline improves event-local consistency without claiming whole-piece legality. Across 40 controlled tasks and four paired models, audited delivery yield rises from 13.3% under raw generation to 48.1% with the harness, which explic...

πŸ“„ SinAE: A Single-Architecture Flow-Matching Autoencoder for Cross-Domain Atomic Systems
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12380v1
πŸ‘₯ Authors: Yuxuan Ren, Fan Yang (possible past Tencent (China) affiliation), Jianhua Yao (possible past Tencent (China) affiliation), Yatao Bian
Abstract

Small molecules, crystals, and proteins all reduce to atoms in 3D space, yet their generative pipelines remain fragmented across domains, each with its Small molecules, crystals, and proteins all reduce to atoms in 3D space, yet their generative pipelines remain fragmented across domains, each with its own graph, equivariant, or frame-based architecture. Cross-domain training would mitigate per-domain data scarcity, but direct generation in 3D coordinate space cannot easily handle the heterogene...

πŸ“„ SlimPer: Make Personalization Model Slim and Smart
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12281v1
πŸ‘₯ Authors: Siqi Wang, Xianjie Chen, Shaofeng Deng, Albert Chen, Romil Shah, Jiawei Huang, Zhaoqin Wang, Zhang Zhang, Yiqun Liu, Meilei Jiang, Anish Dubey, Moyan Mei, Tongxin Wang, Nathan Berrebbi, Misael Manjarres, Armand Sauzay, Shardul Kothapalli, Aryaman Vinchhi, Kevin Johnstone, Juheon Lee, Gufan Yin, Ziheng Huang, Justin Lin, Mert Terzihan, Yilin Qi, Cynthia Yang, Colin Peppler, Qi Ding, Ruohan Sun, Ge Song, Litao Deng, Parichay Kapoor, Matt Ma, Huihui Cheng, Jiyuan Zhang (possible past Tencent (China) affiliation), Yanli Zhao, Yiping Han, Fangqiu Han, Ning Yao, Arun Singh, Jordan Edwards, Zhengyu Su, Abhishek Kumar (possible past Google (United States) affiliation), Guangdeng Liao, Ankit Asthana
Abstract

Transformer-style architectures are increasingly adopted for industrial recommendation systems, yet they inherit a design premise misaligned with the task: generative models rely on per-token autoregressive prediction, which justifies maintaining large intermediate tensors that scale with sequence length. In contrast, recommendation systems produce a single set of relevance scores for each pair without token-level supervision. Leveraging this observation, we propose SlimPer, which r...

πŸ“„ Speculate with Memory: Lossless Acceleration for LLM Agents
πŸ—“οΈ Published: 7/14/2026
πŸ”— http://arxiv.org/abs/2607.12236v1
πŸ‘₯ Authors: Yu Li (possible past Tencent (China) affiliation), Qinyuan Ye, Prafulla Kumar Choubey, Jiaxin Zhang, Chien-Sheng Wu (possible past Salesforce (United States) affiliation)
Abstract

Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre-launch the next step while the environment is idle. However, existing speculators are stateless and discard all information between tasks, preventing prediction quality from improving with experience. We equip the speculator with three online memory systems that learn from past agent trajectories: a contrastive transition table tracking action-sequence statistics, an episodic memory retrieving conte...

πŸ“„ Paradoxes of Game Theoretic Equilibria and Price of Anarchy
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11752v1
πŸ‘₯ Authors: Georgios Piliouras, Ian Gemp, Siqi Liu (possible past University Of Oxford affiliation), Luke Marris (possible past Deepmind (United Kingdom) affiliation)
Abstract

For decades, static solution concepts (Nash, Correlated, and Coarse Correlated Equilibria) and the Price of Anarchy (PoA) have formed the bedrock of algorithmic game theory, with no-regret learning proving fast convergence to such game-theoretic equilibria. We show that reducing multi-agent learning to static equilibrium and black-box regret analysis obscures underlying dynamic disequilibrium and game theoretic bounds. First, interior Nash equilibria lack $C^1$ vector field information, meanin...

πŸ“„ DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11510v1
πŸ‘₯ Authors: Yikang Chen, Zhengkang Guan, Haoyuan Qian, Peng Cui (possible past Tsinghua University affiliation), Yi Yang (possible past Baidu (China) affiliation), Kun Kuang
Abstract

Causal discovery from observational tabular data remains fundamentally challenging, primarily due to the heterogeneity of underlying causal mechanisms and the high-dimensional combinatorial search space of Directed Acyclic Graphs (DAGs). In this paper, we propose \textbf{DAG-FM}, a novel foundation model architecture that amortizes causal discovery. Unlike direct matrix prediction, DAG-FM decomposes the causal discovery process into two auto-regressive stages using two specialized Transformer-ba...

πŸ“„ FastTPS: An Optimized Method for LLM Token Phase for AI accelerators
πŸ—“οΈ Published: 7/13/2026
πŸ”— http://arxiv.org/abs/2607.11211v1
πŸ‘₯ Authors: Wenzong Yang, Danyang Zhang, Kun Cao, Tejus Siddagangaiah, Rajeev Patwari, Zhanxing Pu, Siyin Kong, Zijiang Yang, Hao Zhu (possible past Tsinghua University affiliation), Varun Sharma (possible past Google (United States) affiliation), Yue Gao (possible past Tsinghua University affiliation), Tianping Li, Fan Yang (possible past Tencent (China) affiliation), Jicheng Chen, Yushan Chen, Fennian Zhao, Aaron Ng, Elliott Delaye, Ashish Sirasao, Sudip Nag
Abstract

The popularity of large language models (LLMs) escalates an ongoing demand for effective inference. However, due to the sequential processing of tokens during the token phase in decoder-only LLMs inference, the inherent low parallelism leads to reduced throughput and suboptimal utilization of the computing units on artificial intelligence (AI) accelerators, particularly when handling long-sequence inputs that impose significant memory overhead. Recently, many reported methods have been developed...

*Notable papers are those with at least two authors from a "big" AI/ML lab.