multimodal — AI Digest

4 июн NVIDIA Releases Cosmos 3: Open Omnimodal World Foundation Model for Physical AI NVIDIA research
28 июл Moonshot Kimi K3 Open-Weight Release: 2.8T Parameter MoE Model on Hugging Face Moonshot AI models-llm
26 авг Tencent releases WeMM-Embedding multimodal embedding models (2B/4B/9B) — new MMEB-v2/v3 SOTA Tencent models-llm
30 апр GLM-5V-Turbo: a natively multimodal foundation model for agents Z.ai research
13 мая Thinking Machines Lab Unveils TML-Interaction-Small: 276B MoE Real-Time Multimodal Model Thinking Machines Lab models-llm
13 мая SenseNova-U1: Open-Source Unified Multimodal Understanding and Generation via NEO-unify SenseTime research
15 мая MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Images Technion research
19 мая Google I/O 2026: Gemini 4, Jules V2, Firebase Studio GA, Android XR, and Aluminium OS Google DeepMind models-llm
20 мая Google Introduces Gemini Omni: Any-to-Any Video Generation in Consumer Products Google DeepMind video
2 июн Alibaba Launches Qwen3.7-Plus: Multimodal Agent with Vision, Reasoning, and Autonomous Execution Alibaba / Qwen models-llm
2 июн MiniMax Releases M3: Open-Weight Frontier Model with 1M-Token Context and MSA Architecture MiniMax models-llm
4 июн Google DeepMind Releases Gemma 4 12B: Encoder-Free Multimodal Model That Runs on a 16 GB Laptop Google DeepMind models-llm
10 июн MiniMax M3 Open Weights Released: 1M Context, MoE, Frontier Coding MiniMax models-llm
24 июн ByteDance Launches Doubao-Seed-2.1-Pro at Volcano Engine FORCE Conference ByteDance models-llm
26 июн ByteDance Launches Doubao-Seed-2.1 Pro Flagship LLM at FORCE Conference ByteDance / Doubao models-llm
2 июл ByteDance Seedance 2.5 Launches Publicly: 30-Second Native 4K Video ByteDance video
2 июл Orca: BAAI's General World Foundation Model Trained on 125K Hours of Video BAAI research
16 июл Thinking Machines Lab Releases Inkling: 975B Open-Weight Multimodal MoE Thinking Machines Lab models-llm
24 июл Black Forest Labs launches FLUX 3, a multimodal model generating video with synced audio Black Forest Labs video
19 авг AI-Generated Video Attacks on Real-World Crisis Events: RA-Bench shows detectors fail on crisis-media forgery research
19 авг S²VOPD: self-supervised visual on-policy distillation lifts Qwen3.5-4B from 70.7% to 77.4% on six fine-grained perception benchmarks UC San Diego research
25 авг Alibaba launches Wan 3.0 video model with 30-second single-pass generation and native audio Alibaba (Tongyi Lab) video
28 авг Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds research
11 сен DeepSeek V4.1 Flash goes GA with open MIT weights and V4 Pro retirement DeepSeek models-llm
16 сен Google launches Gemini 3.8 Live and 3.8 Live Extended Thinking speech models Google DeepMind models-llm
23 окт WeVisOne-VL: Unified Visual Perception from Objects to Pixels Tencent WeChat Vision research
2 мая Eywa: heterogeneous collaboration framework between LLM agents and scientific foundation models University of Illinois at Urbana-Champaign research
3 мая MiniCPM-o 4.5: Real-Time Full-Duplex Omni-Modal AI on Edge Devices OpenBMB / Tsinghua University research
5 мая AI2 Open-Sources MolmoAct2: Robotics VLA That Claims to Beat GPT-5 on Embodied Reasoning AI2 research
5 мая UniVidX: One Diffusion Backbone for RGB, Intrinsic Maps, and RGBA Video Generation research
9 мая ByteDance Launches Doubao-Seed-2.0-lite: First Omni-Modal Model in Seed Series ByteDance models-llm
12 мая Qwen-Image-2.0: Unified Image Generation and Editing at 2K Resolution, Top-1 on AI Arena Alibaba research
13 мая Google DeepMind Unveils Magic Pointer: AI-Aware Mouse Cursor for Chrome and Googlebook Google DeepMind tools
20 мая Lance: 3B Unified Multimodal Model for Understanding, Generation, and Editing (314 HF upvotes) ByteDance Research research
23 июн World Action Models: A Survey National University of Singapore research
24 июн Mistral Releases OCR 4 with Bounding Boxes, Block Classification, and 170-Language Support Mistral tools
25 июн Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence research
26 июн Yandex Releases Major Alice AI Update: Cross-Session Memory, Personalization, and Live Accessibility Mode Yandex models-llm
26 июн Qwen-AgentWorld: Language World Models for General Agents at 35B and 397B Scale Qwen Team, Alibaba research
26 июн ByteDance Announces Seedream 5.0 Pro: Image Generation with Built-In Online Search and Deep Reasoning ByteDance image
28 июн ViQ: Text-Aligned Visual Quantized Representations at Any Resolution (ECCV 2026) Tencent Hunyuan research
2 июл ByteDance Releases Seed2.0 Model Card for Frontier Model ByteDance research
6 июл ByteDance Seedance 2.5 Goes Public via Dreamina and Jimeng ByteDance video
7 июл GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation GigaAI research
9 июл LingBot-VLA 2.0: Bridging the Gap Between Foundation VLA Models and Real-World Deployment LingBot Team research
9 июл AlayaWorld: Long-Horizon and Playable Video World Generation (Open-Source) AlayaWorld Team research
12 июл Vidu S1: A Real-Time Interactive Video Generation Model ShengShu Technology research
13 июл Video Generation Models are General-Purpose Vision Learners Google DeepMind research
20 июл Alibaba previews Qwen3.8-Max, a 2.4-trillion-parameter multimodal flagship Alibaba / Qwen models-llm
31 июл Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model Microsoft research
16 авг Intern-S2-Preview: 397B scientific agentic foundation model from Shanghai AI Lab Shanghai AI Lab research
17 авг Alaya-EVOKE: From Linear-Scaling Supervision to Endless World Zhejiang University research
24 авг DeepSeek releases experimental multimodal V4-Flash-Vision-Exp DeepSeek models-llm
26 авг EchoWM: Open and Enterable Omnimodal World Models JD.com research
26 авг TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming Tencent research
27 авг Alibaba ships Wan 3.0 with 30-second clips and document-to-video Alibaba video
18 сен Alibaba releases Qwen 3.8 Omni Flash as a cheap audio-visual default Alibaba (Qwen) models-llm
25 сен Gemini 3.8 Live adds near real-time video with Live Avatar Google DeepMind models-llm
19 мая LongLive-2.0: NVFP4 Parallel Infrastructure for Long Video Generation (NVIDIA, 1,220 HF upvotes) NVIDIA research
11 июн Kwai Keye-VL-2.0: Open-Source 30B MoE Multimodal Model with 256K Context for Long Video Kwai research
14 июн Moonshot AI Releases Kimi K2.7-Code: 1T-Parameter Open-Weight Coding Model with Vision Moonshot AI models-llm
17 июн JoyAI-VL-Interaction: Open-Source 8B Real-Time VLM with Autonomous Turn-Taking JD.com research
8 июл InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization InternRobotics research
8 июл Multiplayer Interactive World Models with Representation Autoencoders research
11 июл Meta Releases Muse Spark 1.1 and Launches Meta Model API Meta models-llm
5 авг SwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks ByteDance research
1 сен DeepSeek open-sources V4-Flash-Vision-Exp, its first multimodal V4 model, under MIT DeepSeek models-llm
11 сен Show-Harness: just a VLM agent can play robots Show Lab, NUS research
12 сен SenseNova-U1.5: native unified visual intelligence without encoders or VAEs SenseTime research
17 окт OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding research
7 мая RLDX-1: Multi-Stream Action Transformer Achieves 86.8% on ALLEX Humanoid Tasks RLWRLD research
9 мая OpenSearch-VL: Open Recipe for Training Frontier Multimodal Search Agents Tencent Hunyuan research
16 мая SANA-WM: Minute-Scale 720p World Modeling on a Single GPU NVIDIA research
16 мая MemLens: Benchmark for Multimodal Long-Term Memory in Vision-Language Models NVIDIA research
19 мая CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence (178 HF upvotes) Peking University / Shanghai Artificial Intelligence Laboratory research
19 мая PhysBrain 1.0: Human Egocentric Video as Robot Training Data for VLA Models (133 HF upvotes) DeepCybo research
19 мая MMSkills: Reusable Multimodal Skills for General Visual Agents (105 HF upvotes) Shanghai Jiao Tong University research
6 июн Audio Interaction Model: Unified Streaming Framework Combining Offline and Real-Time Audio Instruction Following research
11 июн Z-Reward: Score Distributions Instead of Scalar Rewards for Image Generation RLHF Alibaba research
14 июн vLLM Adds Day-0 Support for MiniMax M3 Open Weights with 1M-Context Sparse Attention MiniMax tools
14 июн InterleaveThinker: RL Planner+Critic Pipeline for Interleaved Text-and-Image Generation CUHK Multimedia Lab research
29 июн SingGuard: Runtime Policy-Adaptive Multimodal LLM Guardrail with 56K-Example Benchmark inclusionAI research
2 июл Google Releases Gemini Omni Flash for Video Generation via API Google DeepMind video
2 июл GitHub Copilot Vision Generally Available for All Plan Tiers GitHub tools
3 июл PerceptionRubrics: Atomic Rubric Evaluation Reveals 8% Perception Gap Between Open and Closed Models research
14 июл ABot-AgentOS: General Robotic Agent OS with Lifelong Multi-modal Memory Alibaba research
16 июл Boogu-Image-0.1: Strong Open-Source 10B Diffusion Transformer for Text-to-Image and Editing Boogu Project research
17 июл VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding MCG-NJU research
21 июл RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources Microsoft Research research
22 июл TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs MCG-NJU (Nanjing University) research
14 авг Intern-S2-Preview: Scientific Agentic Foundation Model Shanghai AI Lab research
23 авг 4DAnyone: Create Anyone in 4D from a Casual Monocular Video Ant Research research
26 авг llama.cpp v0.3.0 — first tagged stable release of the b10621 nightly; dots3-note multimodal, GLM-4.5-Air MTP, DeepSeek 4 tensor-split, ggml v0.22.0 ggml-org tools
27 авг Zhipu releases GLM-5.3-Flash, first natively multimodal GLM-5 model Zhipu AI / Z.ai models-llm
10 сен Gander: end-to-end omni interaction agent with Cerebellum-Brain architecture Tencent Hunyuan research
21 сен Can MiniMax-H3 Reason About the Physical World? research
23 сен Intern-S2-397B: Shanghai AI Lab's strongest open scientific multimodal model Shanghai AI Laboratory (InternLM) models-llm
2 мая CoPD: co-evolving policy distillation for unified multi-capability models research
5 мая Odysseus: Training VLMs for 100+ Turn Interactive Decision-Making via RL Princeton University research
13 мая World Action Models: First Systematic Survey of Embodied Foundation Models Unifying World Modeling and Action OpenMOSS research
20 мая Google Project Genie World Model Now Simulates Real Places Using Street View Google DeepMind research
12 июн InterleaveThinker: RL Framework for Agentic Text-and-Image Interleaved Generation research
12 июн Astra: RL-Trained VLM Queries World Simulator for Spatial Reasoning research
25 июн Wan-Streamer v0.1: End-to-End Real-Time Interactive Foundation Model Under 550ms Latency Wan-AI research
28 июн Qwen-Image-Agent: Agentic Context Building to Bridge the Prompt Underspecification Gap in T2I Qwen (Alibaba) research
4 июл WorldDirector: Controllable World Simulator with Persistent Dynamic Object Memory research
6 июл Elon Musk Declares Grok Imagine Development Complete xAI tools
13 июл Scalable Visual Pretraining for Language Intelligence research
18 авг VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End? Tencent research
24 авг ForgeWM: Progressive Causal Training for Few-Step Action-Conditioned Video World Models research
25 авг ReWorld: An Interactive World Model with Long-Horizon Memory TongyiLab (Alibaba) research
25 авг InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter Alibaba (Tongyi) research
17 сен Wuhan AI Lab open-sources ZDTaichu5.0-9B, a spatial-reasoning VLM under 10B parameters Wuhan Artificial Intelligence Research Institute (Taichu) models-llm
29 апр DeepSeek launches image recognition mode in a gray-scale test DeepSeek models-llm
12 мая Google's Gemini 'Omni' Video Model Surfaces in Early Demos Ahead of I/O 2026 Google DeepMind video
9 сен DeepSeek opens a two-day beta of V4.1 Flash, a new-architecture natively multimodal model DeepSeek models-llm
16 мая llama.cpp b9161/b9169: Codex CLI Compatibility and Qwen3A Multimodal Support ggml-org tools
8 июн VideoKR: 315K-Example Training Corpus for Knowledge- and Reasoning-Intensive Video Understanding Yale University research
9 июн Echo-Memory: Controlled Study of Memory Mechanisms in Action-Conditioned Video World Models Microsoft Research research
10 июн SCAIL-2: End-to-End Character Animation via In-Context Conditioning Tsinghua University research
19 июн llama.cpp b9716 Builds: InternVL Multimodal Batching, CUDA col2im, and Nginx SSE Fix tools
19 июн StylisticBias: 15 Visual Attributes Account for 80% of Social Bias in Multimodal LLMs research
19 июн Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agent Loops research
22 июн S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence in VLMs Nanyang Technological University research
4 июл VRRL: Visually Grounded Self-Reflection for Vision-Language Models via RL UT Austin / Cornell research
11 июл UniClawBench: Benchmark for Proactive AI Agents on Real-World Tasks University of Hong Kong research
14 июл LightMem-Ego: Lightweight Streaming Multimodal Memory for Smartphones and AI Glasses ZJUNLP research
23 июл Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing Microsoft research
29 июл Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model Microsoft research
4 авг Scaling Properties of Text Conditioning in Visual Generation ByteDance Seed research
7 авг Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes Meta AI research
14 авг OmniScientist: An Omni-Modal Omni-Discipline AI Scientist research
20 авг Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL UC San Diego (Yunhao Yang, Nuno Vasconcelos, Yijiang Li et al.) research
1 сен DreamX-Creator: native joint audio-video generation at 2K resolution AMAP-ML (Alibaba) research
7 сен Last Translation Benchmark: human-authored tasks that still break frontier translation models research
13 сен SpatialBlock: Teaching LVLMs Spatial Reasoning with Synthetic Block-Stacking KAIST AI research
28 сен InternLM ships Intern-Decision, an open-weights multimodal decision model family InternLM (Shanghai AI Lab) models-llm
28 апр Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond HKUST/NUS/Oxford/NTU research
16 июл MWS AI Releases Cotype Pro 3 (27B) and Light 3 (9B) for Enterprise AI Agents MWS AI models-llm
16 авг Rutube launches proprietary multimodal ML model for video categorization under IAB taxonomy industry
16 июл Yandex Extends ISO/IEC 42001 AI Safety Certification to Full Alice AI Model Family Yandex industry