llm-in-text/plans/inline-suggestions-plan.md

Inline Autocomplete Suggestions 实现计划

技术栈确认

• 前端: Vue3 + Milkdown Editor
• 后端: Python FastAPI
• LLM: OpenAI API（流式响应）
• 范围: 基础流式补全，无需复杂缓存机制

系统架构

flowchart TB
    subgraph 前端 [Vue3 + Milkdown]
        E[Milkdown Editor]
        I[InlineSuggestionPlugin<br/>输入监听+防抖]
        G[GhostTextOverlay<br/>虚影渲染层]
    end

    subgraph 后端 [FastAPI]
        API[/v1/completions<br/>补全接口]
        P[PromptBuilder<br/>上下文构建]
        L[OpenAI Client<br/>LLM调用]
    end

    I -- "输入事件" --> G
    G -- "POST {prefix, suffix}" --> API
    API -- "流式响应" --> G

实现步骤

1. 前端：创建 Inline Suggestion Plugin

文件: src/plugins/inlineSuggestionPlugin.ts

核心实现要点

import { Plugin, PluginKey } from '@milkdown/prose/state';
import { EditorView } from '@milkdown/prose/view';

const INLINE_SUGGESTION_KEY = new PluginKey('inline-suggestion');
const DEBOUNCE_MS = 500;

interface InlineSuggestionOptions {
    apiUrl?: string;
    onSuggestion?: (suggestion: string) => void;
    onError?: (error: Error) => void;
}

interface SuggestionState {
    suggestion: string;
    visible: boolean;
    loading: boolean;
}

function createInlineSuggestionPlugin(options: InlineSuggestionOptions = {}) {
    const apiUrl = options.apiUrl || 'http://localhost:8000/v1/completions';
    const onSuggestion = options.onSuggestion || (() => {});
    const onError = options.onError || ((error) => console.error('Suggestion error:', error));

    // 修复：使用插件状态管理，避免全局变量污染
    return new Plugin({
        key: INLINE_SUGGESTION_KEY,
        state: {
            init: () => ({ suggestion: '', visible: false, loading: false } as SuggestionState),
            apply: (tr, value) => {
                if (!tr.docChanged) return value;
                const { from, to } = tr.selection;
                // 如果光标位置没有变化，保持当前状态
                if (from === value.from && to === value.to) {
                    return value;
                }
                // 光标位置变化，重置建议状态
                return { suggestion: '', visible: false, loading: false, from, to };
            },
        },
        props: {
            handleKeyDown: (view: EditorView, event: KeyboardEvent) => {
                const state = INLINE_SUGGESTION_KEY.getState(view.state) as SuggestionState;

                if (event.key === 'Tab' && state.visible) {
                    event.preventDefault();
                    if (state.suggestion) {
                        view.dispatch(view.state.tr.insertText(state.suggestion, view.state.selection.from));
                        // 重置状态
                        view.dispatch(view.state.tr.setMeta(INLINE_SUGGESTION_KEY, {
                            suggestion: '',
                            visible: false,
                            loading: false
                        }));
                        return true;
                    }
                }

                if (event.key === 'Escape' && state.visible) {
                    event.preventDefault();
                    view.dispatch(view.state.tr.setMeta(INLINE_SUGGESTION_KEY, {
                        suggestion: '',
                        visible: false,
                        loading: false
                    }));
                    return true;
                }

                return false;
            },
        },
        appendTransaction: (transactions, oldState, newState) => {
            const lastTr = transactions[transactions.length - 1];
            if (!lastTr || !lastTr.docChanged) return null;

            const { from, to } = newState.selection;
            const prefix = newState.doc.textBetween(0, from);
            const suffix = newState.doc.textBetween(to, newState.doc.content.size);

            // 修复：使用插件级别的 debounce 管理
            let debounceTimer: NodeJS.Timeout | null = null;

            clearTimeout(debounceTimer);
            debounceTimer = setTimeout(async () => {
                try {
                    // 设置加载状态
                    newState.apply(newState.tr.setMeta(INLINE_SUGGESTION_KEY, {
                        suggestion: '',
                        visible: false,
                        loading: true
                    }));

                    const text = await fetchSuggestion(apiUrl, prefix, suffix);

                    // 检查光标位置是否仍然有效
                    const currentState = INLINE_SUGGESTION_KEY.getState(newState) as SuggestionState;
                    if (currentState.from === from && currentState.to === to) {
                        newState.apply(newState.tr.setMeta(INLINE_SUGGESTION_KEY, {
                            suggestion: text,
                            visible: true,
                            loading: false
                        }));
                        onSuggestion(text);
                    }
                } catch (e) {
                    onError(e as Error);
                    newState.apply(newState.tr.setMeta(INLINE_SUGGESTION_KEY, {
                        suggestion: '',
                        visible: false,
                        loading: false
                    }));
                }
            }, DEBOUNCE_MS);

            return null;
        },
    });
}

// 修复：提取共享的 fetchSuggestion 函数，避免代码重复
async function fetchSuggestion(apiUrl: string, prefix: string, suffix: string): Promise<string> {
    const res = await fetch(apiUrl, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ prefix, suffix, languageId: 'markdown' }),
    });

    // 修复：遵循"获取失败直接报错"原则
    if (!res.ok) {
        const errorText = await res.text();
        throw new Error(`API request failed: ${res.status} - ${errorText}`);
    }

    const reader = res.body?.getReader();
    if (!reader) {
        throw new Error('No response body reader available');
    }

    let text = '';
    while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        const chunk = new TextDecoder().decode(value);
        const lines = chunk.split('\n').filter(l => l.startsWith('data: '));

        for (const line of lines) {
            try {
                const data = JSON.parse(line.slice(6));
                if (data.content) {
                    text += data.content;
                }
                if (data.done || data.error) break;
            } catch (e) {
                // 忽略 JSON 解析错误，继续处理下一行
            }
        }
    }

    return text;
}

export { createInlineSuggestionPlugin, INLINE_SUGGESTION_KEY, fetchSuggestion };

2. 前端：GhostText 渲染组件

文件: src/components/GhostTextOverlay.vue

核心实现要点

<template>
  <div v-if="visible" class="ghost-text-overlay" :style="overlayStyle"
       @click="acceptSuggestion"
  >
    {{ truncatedSuggestion }}
  </div>
</template>

<script setup>
import { computed } from 'vue'

const props = defineProps({
    suggestion: { type: String, default: '' },
    position: { type: Object, required: true },
    maxLength: { type: Number, default: 200 }, // 修复：添加建议文本长度限制
})

const emit = defineEmits(['accept', 'dismiss'])

const visible = computed(() => props.suggestion && props.position)

// 修复：截断过长的建议文本
const truncatedSuggestion = computed(() => {
    if (props.suggestion.length > props.maxLength) {
        return props.suggestion.slice(0, props.maxLength) + '...'
    }
    return props.suggestion
})

const overlayStyle = computed(() => ({
    position: 'absolute',
    left: `${props.position.left}px`,
    top: `${props.position.top}px`,
    fontSize: `${props.position.fontSize || 16}px`,
    fontFamily: props.position.fontFamily || 'monospace',
    color: '#999',
    backgroundColor: 'transparent',
    pointerEvents: 'auto',
    cursor: 'text',
    whiteSpace: 'pre-wrap',
    zIndex: 1000,
}))

const acceptSuggestion = () => emit('accept')
</script>

<style scoped>
.ghost-text-overlay {
    opacity: 0.6;
    user-select: none;
}

.ghost-text-overlay:hover {
    opacity: 1;
    color: #666;
}
</style>

3. 修改 MilkdownEditor 集成插件

文件: src/components/MilkdownEditor.vue

集成要点

<script setup>
import { onMounted, onUnmounted, ref } from 'vue'
import { Crepe } from '@milkdown/crepe'
import GhostTextOverlay from './GhostTextOverlay.vue'
import { createInlineSuggestionPlugin } from '../plugins/inlineSuggestionPlugin'

const root = ref(null)
const containerRef = ref(null)
let crepe = null

const suggestion = ref('')
const cursorRect = ref(null)
const loading = ref(false)

// 修复：使用环境变量配置 API URL
const API_URL = import.meta.env.VITE_API_URL || 'http://localhost:8000/v1/completions'

onMounted(async () => {
    if (!root.value) return

    crepe = new Crepe({
        root: root.value,
        defaultValue: '# Welcome to LLM in text\n\nStart writing your content here...',
    })

    await crepe.create()

    // 注册 Inline Suggestion Plugin
    const plugin = createInlineSuggestionPlugin({
        apiUrl: API_URL,
        onSuggestion: (text) => {
            suggestion.value = text
            updateCursorPosition()
        },
        onError: (error) => {
            console.error('Suggestion error:', error)
            suggestion.value = ''
        }
    })

    crepe.ctx.get().updateState((state) => {
        return state.reconfigure({
            plugins: [...state.plugins, plugin]
        })
    })
})

// 修复：组件卸载时清理资源
onUnmounted(() => {
    if (crepe) {
        crepe.destroy()
    }
})

const updateCursorPosition = async () => {
    if (!crepe) return

    try {
        const ctx = crepe.ctx.get()
        const view = ctx.get('view')
        const { from } = view.state.selection

        const coords = view.coordsAtPos(from)
        const containerRect = containerRef.value?.getBoundingClientRect()
        if (!containerRect) return

        cursorRect.value = {
            left: coords.left - containerRect.left,
            top: coords.top - containerRect.top + window.scrollY,
            fontSize: 16,
            fontFamily: 'monospace',
        }
    } catch (e) {
        console.error('updateCursorPosition error:', e)
    }
}

const acceptSuggestion = () => {
    if (suggestion.value) {
        const ctx = crepe.ctx.get()
        const view = ctx.get('view')
        view.dispatch(view.state.tr.insertText(suggestion.value))
        suggestion.value = ''
    }
}

const dismissSuggestion = () => {
    suggestion.value = ''
}
</script>

<template>
  <div class="editor-container" ref="containerRef">
    <div ref="root" class="milkdown-editor"></div>

    <!-- 修复：正确的组件标签语法 -->
    <GhostTextOverlay
      v-if="suggestion && cursorRect"
      :suggestion="suggestion"
      :position="cursorRect"
      @accept="acceptSuggestion"
      @dismiss="dismissSuggestion"
    />
  </div>
</template>

4. 后端：FastAPI 服务

文件: backend/main.py

核心实现要点

from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from fastapi.middleware.cors import CORSMiddleware  # 修复：添加 CORS 支持
from pydantic import BaseModel
import os
import json

app = FastAPI()

# 修复：添加 CORS 中间件
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # 生产环境应该限制具体域名
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

class CompletionRequest(BaseModel):
    prefix: str
    suffix: str
    languageId: str = 'markdown'

def generate_stream(request: CompletionRequest):
    from prompt import build_prompt
    from llm import stream_openai

    try:
        prompt = build_prompt(request.prefix, request.suffix)

        async def gen():
            chunk_count = 0
            async for chunk in stream_openai(prompt):
                chunk_count += 1
                yield f"data: {chunk}\n\n"
            yield "data: {\"done\": true}\n\n"
        return gen()
    except Exception as e:
        # 修复：遵循"获取失败直接报错"原则
        error_msg = f"{{\"error\": \"{str(e)}\"}}"
        yield f"data: {error_msg}\n\n"
        raise  # 重新抛出异常

@app.post("/v1/completions")
async def create_completion(request: CompletionRequest):
    return StreamingResponse(generate_stream(request), media_type="text/event-stream")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

5. 后端：Prompt 构建和 LLM 调用

文件: backend/prompt.py, backend/llm.py

Prompt 构建

import os
from typing import Tuple

def build_prompt(prefix: str, suffix: str) -> str:
    """
    构建用于代码补全的 Prompt。
    参考 completions-sample-code 的 extractPrompt 逻辑简化实现。
    """
    MAX_CONTEXT_LINES = 30

    prefix_lines = prefix.split('\n')
    suffix_lines = suffix.split('\n') if suffix else []

    recent_prefix = '\n'.join(prefix_lines[-MAX_CONTEXT_LINES:])
    recent_suffix = '\n'.join(suffix_lines[:5])

    prompt = f"""
You are a helpful writing assistant. Continue the text naturally based on the context.

Context (before cursor):
{recent_prefix}

Complete this:
{suffix if suffix else '(cursor here)'}

Continue:"""

    return prompt.strip()

LLM 调用

import os
from typing import AsyncGenerator
from openai import AsyncOpenAI
import json

api_key = os.getenv('OPENAI_API_KEY', 'ollama')
base_url = os.getenv('OLLAMA_BASE_URL', 'http://localhost:11434/v1/')
model = os.getenv('OLLAMA_MODEL', 'gpt-4')

client = AsyncOpenAI(api_key=api_key, base_url=base_url)

async def stream_openai(prompt: str) -> AsyncGenerator[str, None]:
    """
    调用 OpenAI/Ollama API 并流式返回补全内容。
    参考 completions-sample-code 的 streaming 逻辑。
    """
    try:
        stream = await client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            stream=True,
            max_tokens=128,
            temperature=0.2,
        )

        chunk_count = 0
        async for chunk in stream:
            if chunk.choices[0].delta.content:
                content = chunk.choices[0].delta.content
                chunk_count += 1
                yield json.dumps({"content": content})

    except Exception as e:
        # 修复：遵循"获取失败直接报错"原则
        yield json.dumps({"error": str(e)})
        raise  # 重新抛出异常

文件结构

llm-in-text/
├── src/
│   ├── components/
│   │   └── MilkdownEditor.vue      [修改]
│   ├── plugins/
│   │   ├── inlineSuggestionPlugin.ts [修改]
│   │   └── types.ts                [新建]
│   └── ...
└── backend/
    ├── main.py                      [修改]
    ├── prompt.py                    [修改]
    ├── llm.py                       [修改]
    └── requirements.txt             [修改]

API 设计

请求

POST /v1/completions
{
  "prefix": "# Hello\n\nThis is ",
  "suffix": "",
  "languageId": "markdown"
}

响应（流式 SSE）

data: {"content": "a "}

data: {"content": "a te"}

data: {"content": "a test"}

data: {"done": true}

已知问题及修复方案

🔴 严重问题（P0）

1. 全局状态污染

位置: inlineSuggestionPlugin.ts:6-8 问题: 使用模块级全局变量，多个编辑器实例会共享状态 修复: 使用 ProseMirror 插件的状态管理机制，每个插件实例维护自己的状态

2. 错误处理违反原则

位置: llm.py:42-44, main.py:34-37 问题: 错误时只返回错误信息，不抛出异常 修复: 遵循"获取失败直接报错"原则，在 yield 错误信息后重新抛出异常

🟡 中等问题（P1）

3. 代码重复

问题: fetchSuggestion 逻辑在两个文件中重复 修复: 提取共享的 fetchSuggestion 函数，在插件和编辑器组件中复用

4. 缺少 CORS 配置

问题: 后端没有配置 CORS，可能导致跨域请求失败 修复: 在 FastAPI 中添加 CORS 中间件

5. 建议文本无长度限制

问题: 建议文本可能过长，影响显示效果 修复: 在 GhostTextOverlay 组件中添加 maxLength prop，截断过长的建议

🟢 轻微问题（P2）

6. 缺少加载状态

问题: 用户无法知道是否正在获取建议 修复: 在插件状态中添加 loading 字段，在 UI 中显示加载指示器

7. 缺少类型定义

问题: TypeScript 代码中缺少完整的类型定义 修复: 添加 SuggestionState 接口和完整的类型定义

8. API URL 硬编码

问题: API URL 硬编码在前端代码中 修复: 使用环境变量 VITE_API_URL 配置 API URL

参考代码映射

completions-sample-code	本项目实现
ghostText.ts getGhostText()	后端 LLM 调用逻辑
inlineCompletion.ts GhostText	前端 Plugin 核心逻辑
networking.ts postRequest()	后端 API 接口
prompt/extractPrompt()	后端 Prompt 构建

最佳实践

错误处理

• 遵循"获取失败直接报错"原则
• 不返回默认值，不尝试隐藏报错信息
• 在前端和后端都实现完整的错误处理

性能优化

• 使用 150ms 防抖，避免频繁请求
• 流式传输（SSE），降低延迟
• 及时清理定时器和事件监听器

代码质量

• 避免全局变量，使用插件状态管理
• 提取共享逻辑，避免代码重复
• 添加完整的类型定义
• 移除调试日志或条件化输出

下一步

确认计划后切换到 Code 模式开始实现。