心智字幕：從人類大腦活動中演化出心理內容的描述性文本

Mind captioning: Evolving descriptive text of mental content from human brain activity

——《科學進展》第11卷第45期，2025年11月7日——

Vol 11, Issue 45, 7 Nov 2025

【摘要】神經科學的核心挑戰之一是解碼大腦活動，以揭示包含多個組成部分及其相互作用的心理內容。儘管在從人腦活動中解碼語言相關信息方面取得了進展，但生成與結構化視覺語義相關的複雜心理內容的全面描述仍然具有挑戰性。我們提出了一種方法，該方法通過深度語言模型計算的語義特徵生成反映大腦表徵的描述性文本。我們構建了線性解碼模型，將視頻誘發的大腦活動轉化為相應字幕的語義特徵，並通過詞語替換和插值將候選描述的特徵與大腦解碼的特徵對齊，從而優化了這些描述。即使不依賴規範語言網絡，該過程也能生成結構良好的描述，準確捕捉觀看的內容。該方法還可以推廣到將回憶的內容口頭表達出來，作為心理表徵和文本之間的解釋性接口，同時展示了基於非語言思維的腦-文本交流的潛力，這可能為語言表達障礙（例如失語症）患者提供一種替代的溝通途徑。 

[Abstract] A central challenge in neuroscience is decoding brain activity to uncover mental content comprising multiple components and their interactions. Despite progress in decoding language-related information from human brain activity, generating comprehensive descriptions of complex mental content associated with structured visual semantics remains challenging. We present a method that generates descriptive text mirroring brain representations via semantic features computed by a deep language model. Constructing linear decoding models to translate brain activity induced by videos into semantic features of corresponding captions, we optimized candidate descriptions by aligning their features with brain-decoded features through word replacement and interpolation. This process yielded well-structured descriptions that accurately capture viewed content, even without relying on the canonical language network. The method also generalized to verbalize recalled content, functioning as an interpretive interface between mental representations and text and simultaneously demonstrating the potential for nonverbal thought–based brain-to-text communication, which could provide an alternative communication pathway for individuals with language expression difficulties, such as aphasia. 

論文原文：Tomoyasu Horikawa (2025). Mind captioning: Evolving descriptive text of mental content from human brain activity. Science Advances, Volume 11, Issue 45, 7 Nov 2025

https://doi.org/10.1126/sciadv.adw1464

（需要英文原文的朋友，請聯繫微信：millerdeng95或iacmsp）