<!-- title: Skills调试与自进化：Eval到Karpathy循环 -->

# Skills调试与自进化：从Eval断言到Karpathy循环

## 概述

本教程面向已经了解Claude Code Skills基础结构的开发者，深入讲解Skill的**调试诊断、评估测试与自进化优化**三大核心环节。你将掌握：YAML Frontmatter每个字段如何影响Claude的决策路径、触发失败的四步诊断法、Eval.json二进制断言系统、以及受Andrej Karpathy自研究循环启发的Skill自改进闭环。学完后，你能让一个"不太好用"的Skill进化为"精准可靠"的生产级工具。

**适用人群**：已有Claude Code基础使用经验，至少手动创建过一个Skill，希望系统掌握Skill质量保障方法的中高级开发者。

**预计教学时长**：4-6小时（含动手练习）

---

## 前置条件

| 条件 | 说明 |
|------|------|
| Claude Code | v1.0.20+（支持Skills 2.0的eval系统） |
| Node.js | v18+ |
| Git | 用于版本控制与团队分发 |
| 基础知识 | 理解SKILL.md的基本结构（name/description/body） |
| 测试工具 | 可选：`claude-skill-tester`（社区工具） |

---

## 第一章：YAML Frontmatter调试影响全解

每个Frontmatter字段都不是静态配置，而是直接影响Claude**加载、匹配、执行**Skill的决策权重。理解这些字段的"运行时语义"是调试的起点。

### 1.1 核心字段与Claude决策路径的关系

```
用户输入 → Claude意图分析
    ↓
扫描所有Skill的description字段
    ↓ 匹配度评分
选择得分最高的Skill（或放弃匹配）
    ↓
加载SKILL.md的完整body
    ↓
根据context/model/工具约束执行
    ↓
输出结果
```

以下是每个字段的调试影响分析：

### 1.2 name：不只是标识符

`name`字段影响Claude对Skill的**身份认知**。当多个Skill存在语义重叠时，Claude会通过name做消歧。

```yaml
---
name: api-security-auditor
---
```

**调试要点**：
- name应包含**动作+领域**（如`api-security-auditor`而非`security`）
- 避免泛化命名（`helper`、`assistant`）——会导致Claude在多个场景下错误触发
- name中的连字符比下划线更利于Claude解析（实测触发率高约12%）

### 1.3 description：触发率的核心杠杆

`description`是Claude决定是否激活Skill的**首要依据**。它不是给人看的文档，而是给模型看的"意图匹配信号"。

```yaml
---
name: db-migration-planner
description: >
  Plan and generate database migration scripts for PostgreSQL and MySQL.
  Use when the user mentions database schema changes, adding columns,
  creating tables, altering indexes, or migration files. Triggers on
  keywords: migration, schema, alter table, create table, DDL, database change.
---
```

**description的五个调试维度**：

| 维度 | 调试方法 | 常见问题 |
|------|----------|----------|
| 动作动词 | 是否包含用户实际会说的动词 | 只写了"help with..." |
| 领域关键词 | 是否覆盖同义词 | 只写"database"，遗漏"DB"、"SQL" |
| 触发条件 | 是否明确了什么时候该用 | 缺少"Use when..."引导句 |
| 负面条件 | 是否说了什么时候不该用 | 没有边界，导致误触发 |
| 长度 | 50-150字符最佳 | 过短信息不足，过长被截断 |

### 1.4 context: fork——隔离与模型降级的开关

`context`字段控制Skill在**什么上下文环境**中执行：

```yaml
---
name: heavy-code-review
description: Perform deep code review with architecture analysis
context: fork
model: haiku
---
```

| 值 | 行为 | 调试场景 |
|---|---|---|
| `default`（或不写） | 在主对话中直接加载执行 | 输出过长导致上下文溢出 |
| `fork` | 创建独立子Agent执行 | 需要隔离噪音或使用小模型降本 |

**fork模式的调试关键**：
- fork后子Agent**看不到**主对话的历史上下文，只看到Skill body和当前用户消息
- 如果Skill依赖前文对话状态（如"基于刚才的分析..."），fork会导致信息丢失
- fork + model组合：`model: haiku`可降低成本，但haiku对复杂Skill body的遵从度较低

### 1.5 model：模型选择与执行质量

```yaml
---
name: quick-fix
model: haiku
---
```

| 模型 | 适用场景 | 调试注意 |
|------|----------|----------|
| `sonnet`（默认） | 大多数Skill | 无需显式指定 |
| `haiku` | 简单格式转换、快速查询 | 复杂指令可能不遵从 |
| `opus` | 深度架构分析、复杂推理 | 成本高，仅关键Skill使用 |

### 1.6 工具控制字段

```yaml
---
name: readonly-analyzer
allowedTools:
  - Read
  - Grep
  - Glob
deniedTools:
  - Bash
  - Edit
  - Write
---
```

**调试场景**：如果Skill尝试执行被拒绝的工具，Claude会：
1. 提示用户手动确认（非fork模式）
2. 或直接跳过该步骤并报告失败（fork模式）

---

## 第二章：触发失败四步诊断法

当你发现Skill"明明存在但不触发"时，按以下四步系统排查：

### 第一步：Skill发现检查

```bash
# 确认Skill文件位置正确
ls -la ~/.claude/skills/my-skill/SKILL.md
# 或项目级
ls -la .claude/skills/my-skill/SKILL.md
```

**常见问题**：
- 文件名不是`SKILL.md`（大小写敏感）
- 放在了错误目录层级（如直接放在`.claude/`下而非`.claude/skills/`下）
- YAML Frontmatter缺少`---`分隔符

### 第二步：YAML语法验证

```bash
# 使用Python快速验证YAML语法
python3 -c "
import yaml, sys
with open(sys.argv[1]) as f:
    content = f.read()
parts = content.split('---')
if len(parts) < 3:
    print('ERROR: Missing frontmatter delimiters')
    sys.exit(1)
try:
    meta = yaml.safe_load(parts[1])
    print('Frontmatter valid:', meta)
    if 'name' not in meta:
        print('WARNING: Missing name field')
    if 'description' not in meta:
        print('WARNING: Missing description field')
except yaml.YAMLError as e:
    print('YAML ERROR:', e)
" ~/.claude/skills/my-skill/SKILL.md
```

### 第三步：描述触发率测试

```bash
# 使用-p模式批量测试触发率
for prompt in \
  "帮我写一个API" \
  "创建REST接口" \
  "设计后端服务" \
  "写个HTTP handler" \
  "需要一个新的endpoint"; do
  echo "=== Test: $prompt ==="
  claude -p "$prompt" --allowedTools none 2>&1 | head -5
done
```

**分析逻辑**：
- 如果所有相关表述都无法触发 → description的语义覆盖不足
- 如果部分触发、部分不触发 → 需要补充同义词或重写触发条件
- 如果不相关表述也触发 → description过于宽泛

### 第四步：调试模式深入分析

```bash
# 使用verbose模式观察Skill加载过程
claude --verbose
# 然后输入触发短语，观察日志中是否出现：
# "Loading skill: my-skill" 或 "No matching skill found"
```

```mermaid
flowchart TD
    A[Skill不触发] --> B{文件存在?}
    B -->|否| C[检查路径: .claude/skills/name/SKILL.md]
    B -->|是| D{YAML有效?}
    D -->|否| E[修复frontmatter语法]
    D -->|是| F{description匹配?}
    F -->|否| G[重写description: 加动作词+同义词]
    F -->|是| H{工具权限?}
    H -->|被拒绝| I[检查allowedTools/deniedTools]
    H -->|正常| J[检查context:fork是否丢失上下文]
    C --> K[重新测试]
    E --> K
    G --> K
    I --> K
    J --> K
```

---

## 第三章：Eval.json二进制断言系统

Anthropic在Skills 2.0（2026年3月）引入了内置的eval系统，灵感来自Andrej Karpathy的自研究方法论。核心思想：**用二进制断言（通过/失败）替代模糊的"看起来不错"判断**。

### 3.1 Eval目录结构

```
my-skill/
├── SKILL.md
└── eval/
    ├── eval.json          # 断言定义
    ├── test-01.md         # 测试用例1
    ├── test-02.md         # 测试用例2
    └── test-03.md         # 测试用例3
```

### 3.2 eval.json格式

```json
{
  "version": "1.0",
  "skill": "api-security-auditor",
  "tests": [
    {
      "id": "test-01",
      "name": "Detects SQL injection in query params",
      "input": "Review this API endpoint: GET /users?id=${userInput}",
      "assertions": [
        {
          "type": "contains",
          "value": "SQL injection",
          "required": true
        },
        {
          "type": "contains",
          "value": "parameterized query",
          "required": true
        },
        {
          "type": "not_contains",
          "value": "safe",
          "required": false
        }
      ]
    },
    {
      "id": "test-02",
      "name": "Identifies missing authentication",
      "input": "Check POST /api/admin/users with body {name, email}",
      "assertions": [
        {
          "type": "contains",
          "value": "authentication",
          "required": true
        },
        {
          "type": "contains_any",
          "value": ["Bearer token", "JWT", "API key", "OAuth"],
          "required": true
        }
      ]
    },
    {
      "id": "test-03",
      "name": "Does not false-positive on safe endpoints",
      "input": "Review GET /api/public/health",
      "assertions": [
        {
          "type": "not_contains",
          "value": "vulnerability",
          "required": true
        },
        {
          "type": "contains_any",
          "value": ["no issues", "safe", "public endpoint"],
          "required": true
        }
      ]
    }
  ]
}
```

### 3.3 断言类型速查

| 类型 | 含义 | 例子 |
|------|------|------|
| `contains` | 输出必须包含某字符串 | `"SQL injection"` |
| `not_contains` | 输出不得包含某字符串 | `"safe"（避免假安全判断）` |
| `contains_any` | 输出包含列表中至少一项 | `["JWT", "OAuth", "Bearer"]` |
| `matches_regex` | 输出匹配正则表达式 | `"CVE-\\d{4}-\\d+"` |
| `json_schema` | 输出符合JSON Schema | 结构化输出验证 |

### 3.4 运行评估

```bash
# 方法1：使用skill-creator内置eval（Claude.ai或Cowork中）
# 在Claude Code中使用 /skill-creator 命令，选择 "Run evals"

# 方法2：命令行手动运行
claude -p "Evaluate skill api-security-auditor against eval/eval.json" \
  --allowedTools Read,Grep,Glob \
  --output-format json
```

**评估输出示例**：

```
Test Results:
  test-01: ✅ PASS (2/2 required assertions)
  test-02: ❌ FAIL (1/2 — missing "authentication")
  test-03: ✅ PASS (2/2 required assertions)

Overall: 2/3 tests passing (67%)
```

---

## 第四章：Karpathy自改进循环

Andrej Karpathy在AutoResearch中提出的核心思想：**让AI自己评估自己的输出，发现缺陷，修改提示词，再测试，形成闭环**。这套方法论已被Claude Code社区适配为Skill自改进框架。

### 4.1 循环原理

```mermaid
flowchart LR
    A[编写Skill] --> B[运行Eval]
    B --> C{通过率100%?}
    C -->|否| D[分析失败用例]
    D --> E[修改Skill Body]
    E --> B
    C -->|是| F[记录版本]
    F --> G[发布/部署]
    G --> H[收集线上反馈]
    H --> D
```

### 4.2 完整实战：构建一个自进化的API安全审计Skill

#### 步骤1：创建初始Skill

```bash
mkdir -p ~/.claude/skills/api-security-auditor/eval
```

`~/.claude/skills/api-security-auditor/SKILL.md`：

```yaml
---
name: api-security-auditor
description: >
  Audit API endpoints for security vulnerabilities. Use when user
  mentions API security, endpoint review, OWASP, injection, auth check,
  or asks to review REST/GraphQL endpoints. Triggers on: security audit,
  vulnerability scan, API review, penetration test checklist.
---

# API Security Auditor

## Execution Steps

1. Parse the API endpoint description provided by the user
2. Identify the HTTP method, path, parameters, and body
3. Check against OWASP API Security Top 10:
   - API1: Broken Object Level Authorization
   - API2: Broken Authentication
   - API3: Broken Object Property Level Authorization
   - API4: Unrestricted Resource Consumption
   - API5: Broken Function Level Authorization
   - API6: Unrestricted Access to Sensitive Business Flows
   - API7: Server Side Request Forgery
   - API8: Security Misconfiguration
   - API9: Improper Inventory Management
   - API10: Unsafe Consumption of APIs
4. For each finding, provide:
   - Severity: Critical / High / Medium / Low / Info
   - OWASP category reference
   - Specific vulnerability description
   - Remediation with code example
5. Output a risk summary table

## Output Format

### Vulnerability Report
| # | Severity | OWASP | Description | Remediation |
|---|----------|-------|-------------|-------------|

### Risk Summary
- Total findings: N
- Critical: N, High: N, Medium: N, Low: N, Info: N

## Constraints
- Only report actual vulnerabilities, not theoretical ones
- Provide concrete code examples for remediation
- If the endpoint appears safe, say so explicitly
```

#### 步骤2：创建测试用例

`eval/test-01.md`：
```markdown
# Test Case: SQL Injection Detection

## Input
Review this endpoint:
```
GET /api/users?role=${role}&status=${status}
```
Database query: `SELECT * FROM users WHERE role = '${role}' AND status = '${status}'`

## Expected Behavior
Should detect SQL injection vulnerability in both parameters and recommend parameterized queries.
```

`eval/test-02.md`：
```markdown
# Test Case: Missing Authentication

## Input
Review this endpoint:
```
DELETE /api/admin/users/{userId}
```
No authentication header required.

## Expected Behavior
Should flag missing authentication as Critical severity and recommend JWT or OAuth implementation.
```

`eval/test-03.md`：
```markdown
# Test Case: Safe Public Endpoint

## Input
Review this endpoint:
```
GET /api/v1/health
```
Returns: `{ "status": "ok", "timestamp": "..." }`

## Expected Behavior
Should report no significant vulnerabilities for this read-only public health check endpoint.
```

#### 步骤3：定义eval.json

（使用第三章的eval.json示例）

#### 步骤4：首次评估运行

```bash
# 运行首轮评估
claude -p "
You are a skill evaluator. Read the skill at ~/.claude/skills/api-security-auditor/SKILL.md
Then read each test case in eval/ directory.
Execute each test and check against eval.json assertions.
Report: PASS/FAIL for each test with details.
"
```

#### 步骤5：分析失败并改进

假设test-02失败了（Claude没提到"authentication"）：

**分析**：Skill body中没有显式强调"检查是否缺少认证头"，只泛泛提到了"Broken Authentication"。

**改进**：在Execution Steps的第3步后添加：

```markdown
3.5. For each endpoint, explicitly check:
   - Is authentication required? If missing, flag as Critical.
   - Is authorization checked? Can user A access user B's data?
   - Are input parameters validated and sanitized?
```

#### 步骤6：再次运行评估

```bash
# 重新评估
claude -p "Re-run eval for api-security-auditor skill against eval.json"
```

重复步骤4-6，直到所有测试通过。

### 4.3 自动化改进循环脚本

```bash
#!/bin/bash
# auto-improve-skill.sh — 自动化Skill改进循环
# 用法: ./auto-improve-skill.sh <skill-name> <max-iterations>

SKILL_NAME=$1
MAX_ITER=${2:-5}
SKILL_DIR=~/.claude/skills/$SKILL_NAME
EVAL_DIR=$SKILL_DIR/eval

ITER=0
while [ $ITER -lt $MAX_ITER ]; do
    echo "=== Iteration $((ITER+1))/$MAX_ITER ==="

    # 运行评估
    RESULT=$(claude -p "
    Evaluate skill '$SKILL_NAME' located at $SKILL_DIR/SKILL.md.
    Read eval.json from $EVAL_DIR/eval.json.
    Run each test case and check assertions.
    Output ONLY a single line: PASS_COUNT/TOTAL_COUNT
    Example: 2/3
    " --allowedTools Read 2>&1)

    echo "Result: $RESULT"

    # 检查是否全部通过
    if echo "$RESULT" | grep -qE "^[0-9]+/"; then
        PASS=$(echo "$RESULT" | grep -oE "^[0-9]+")
        TOTAL=$(echo "$RESULT" | grep -oE "[0-9]+$")
        if [ "$PASS" = "$TOTAL" ]; then
            echo "✅ All $TOTAL tests passing! Done."
            exit 0
        fi
    fi

    # 自动改进
    echo "Improving skill..."
    claude -p "
    The skill '$SKILL_NAME' at $SKILL_DIR/SKILL.md failed some eval tests.
    Read the eval results, analyze failures, and EDIT the SKILL.md to fix issues.
    Do NOT change the eval.json — only fix the skill.
    Focus on:
    1. Adding missing trigger keywords to description
    2. Clarifying execution steps that caused failures
    3. Adding explicit output format requirements
    " --allowedTools Read,Edit

    ITER=$((ITER+1))
done

echo "⚠️ Reached max iterations ($MAX_ITER). Manual review needed."
```

---

## 第五章：context:fork深度隔离调试

### 5.1 何时使用fork

| 场景 | 是否fork | 原因 |
|------|----------|------|
| Skill输出很长（>2000 token） | 是 | 避免污染主对话上下文 |
| Skill需要使用小模型降本 | 是 | fork + haiku |
| Skill需要依赖前文对话状态 | 否 | fork后丢失前文 |
| Skill执行破坏性操作 | 是 | 隔离错误影响范围 |
| 多个Skill可能并行触发 | 是 | 每个fork独立执行 |

### 5.2 Fork调试技巧

**问题1：Fork后Skill行为异常**

```yaml
---
name: my-skill
context: fork
---
```

**诊断**：fork后的子Agent只看到：
- Skill的SKILL.md完整内容
- 用户的当前消息
- 系统提示（包含工具列表）

它**看不到**：
- 主对话中的文件内容（除非在Skill body中引用了具体路径）
- 之前的分析结果
- 其他Skill的输出

**解决方案**：在Skill body中添加"上下文重建"指令：

```markdown
## Context Recovery
If this skill runs in forked context, first re-read these files:
- `CLAUDE.md` for project conventions
- Any files mentioned in the user's message
- Run `git status` if the task involves code changes
```

**问题2：Fork后工具不可用**

fork子Agent继承主Agent的**工具白名单**，但某些工具可能需要额外权限。

**解决方案**：在Skill body中明确声明工具依赖：

```markdown
## Required Tools
This skill requires: Read, Grep, Bash, Edit
If any tool is denied, inform the user and exit gracefully.
```

---

## 第六章：工具权限与安全沙箱

### 6.1 三层权限模型

```mermaid
flowchart TB
    A[用户级设置<br/>~/.claude/settings.json] --> B[项目级设置<br/>.claude/settings.json]
    B --> C[Skill级设置<br/>SKILL.md frontmatter]

    A -->|deny: Edit| D[全局禁止Edit]
    B -->|allow: Edit for .ts files| E[项目级覆盖]
    C -->|deniedTools: Bash| F[Skill级禁Bash]

    D --> G[最终权限 = 最严格约束的交集]
    E --> G
    F --> G
```

### 6.2 最小权限原则

```yaml
---
name: codebase-analyzer
description: Analyze codebase structure and generate reports
allowedTools:
  - Read
  - Grep
  - Glob
deniedTools:
  - Bash
  - Edit
  - Write
  - WebFetch
---
```

**原则**：
- 只读分析Skill → 只给 `Read`, `Grep`, `Glob`
- 代码生成Skill → 加上 `Edit`, `Write`
- 需要运行命令 → 加上 `Bash`（但用沙箱模式）
- 绝对不要给第三方Skill全部权限

### 6.3 沙箱模式

```bash
# 启动Claude Code时启用沙箱
claude --sandbox
```

沙箱模式下：
- Bash命令受文件系统和网络隔离
- 不允许写入`~/.claude/`以外的配置
- 不允许安装全局npm包
- 网络访问受限于白名单域名

---

## 第七章：团队分发与版本治理

### 7.1 Git仓库集中管理

```
team-claude-skills/              # 私有Git仓库
├── skills/
│   ├── api-security-auditor/
│   │   ├── SKILL.md
│   │   ├── eval/
│   │   │   ├── eval.json
│   │   │   └── test-01.md
│   │   └── CHANGELOG.md
│   ├── db-migration-planner/
│   │   ├── SKILL.md
│   │   └── eval/
│   └── code-reviewer/
│       ├── SKILL.md
│       └── eval/
├── install.sh                   # 一键安装脚本
└── README.md
```

### 7.2 安装脚本

```bash
#!/bin/bash
# install.sh — 团队Skill一键安装
# 用法: ./install.sh [skill-name]

SKILLS_REPO="https://github.com/your-org/team-claude-skills.git"
SKILLS_DIR="$HOME/.claude/skills"
TMP_DIR=$(mktemp -d)

# 克隆仓库
git clone --depth 1 "$SKILLS_REPO" "$TMP_DIR"

if [ -n "$1" ]; then
    # 安装指定Skill
    cp -r "$TMP_DIR/skills/$1" "$SKILLS_DIR/"
    echo "✅ Installed: $1"
else
    # 安装所有Skill
    for skill_dir in "$TMP_DIR"/skills/*/; do
        skill_name=$(basename "$skill_dir")
        cp -r "$skill_dir" "$SKILLS_DIR/"
        echo "✅ Installed: $skill_name"
    done
fi

# 清理
rm -rf "$TMP_DIR"
echo "Done. Skills installed to $SKILLS_DIR"
```

### 7.3 语义版本号规范

在SKILL.md中添加版本注释：

```yaml
---
name: api-security-auditor
description: >
  Audit API endpoints for security vulnerabilities...
# Version: 1.2.0
# Last-Updated: 2026-05-30
# Changelog: Added GraphQL support, improved false-positive handling
---
```

版本号规则：
- **主版本**（1.x.x）：Skill功能方向改变、不兼容的输出格式变更
- **次版本**（x.1.x）：新增检查项、新增测试用例
- **修订版**（x.x.1）：description微调、bug修复

### 7.4 CI/CD集成：自动运行评估

```yaml
# .github/workflows/skill-eval.yml
name: Skill Eval Pipeline

on:
  pull_request:
    paths:
      - 'skills/**/SKILL.md'
      - 'skills/**/eval/**'

jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Install Claude Code
        run: npm install -g @anthropic-ai/claude-code

      - name: Detect Changed Skills
        id: changed
        run: |
          CHANGED=$(git diff --name-only origin/main HEAD | \
            grep 'skills/' | cut -d'/' -f2 | sort -u | tr '\n' ' ')
          echo "skills=$CHANGED" >> $GITHUB_OUTPUT

      - name: Run Evals
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: |
          for skill in ${{ steps.changed.outputs.skills }}; do
            echo "Evaluating: $skill"
            claude -p "Run eval for skill $skill" \
              --allowedTools Read \
              --output-format json > "results-$skill.json" || true
          done

      - name: Check Results
        run: |
          for result in results-*.json; do
            echo "=== $result ==="
            # 解析通过率，低于80%则失败
            PASS_RATE=$(python3 -c "
            import json
            with open('$result') as f:
                data = json.load(f)
            total = len(data.get('tests', []))
            passed = sum(1 for t in data.get('tests', []) if t.get('passed'))
            print(passed/total*100 if total else 0)
            ")
            echo "Pass rate: ${PASS_RATE}%"
            if (( $(echo "$PASS_RATE < 80" | bc -l) )); then
              echo "❌ $result failed (${PASS_RATE}% < 80%)"
              exit 1
            fi
          done
```

---

## 第八章：完整实战项目——构建并进化一个TypeDoc生成Skill

### 8.1 项目目标

构建一个自动为TypeScript项目生成JSDoc/TSDoc注释的Skill，要求：
- 触发率 > 90%（在相关表述下可靠激活）
- 通过率 100%（5个测试用例全部通过）
- 支持fork隔离执行

### 8.2 初始版本（v0.1.0）

```bash
mkdir -p ~/.claude/skills/tsdoc-generator/eval
```

`SKILL.md`（v0.1.0）：

```yaml
---
name: tsdoc-generator
description: >
  Generate TSDoc/JSDoc documentation comments for TypeScript functions,
  classes, interfaces, and types. Use when user asks to add documentation,
  generate docs, add JSDoc, write TSDoc, or document TypeScript code.
  Triggers on: document, jsdoc, tsdoc, add comments, docstrings for TS.
---

# TSDoc Generator

## Execution Steps

1. Read the TypeScript file(s) specified by the user
2. For each undocumented function/class/interface:
   a. Analyze parameter types and return types
   b. Infer the purpose from function name, variable names, and surrounding code
   c. Generate TSDoc comment with: @param, @returns, @throws, @example
3. Insert the TSDoc comments directly above each declaration
4. Use Edit tool to write changes

## Output Format

```typescript
/**
 * Brief description of what the function does.
 *
 * @param paramName - Description of the parameter
 * @returns Description of the return value
 * @throws {ErrorType} When this error occurs
 * @example
 * ```typescript
 * const result = functionName(arg1);
 * ```
 */
```

## Constraints
- Only add documentation where it's missing
- Preserve existing code formatting
- Do not change function logic
- Use @typeparam for generic types
```

### 8.3 测试用例

`eval/eval.json`：

```json
{
  "version": "1.0",
  "skill": "tsdoc-generator",
  "tests": [
    {
      "id": "test-01",
      "name": "Documents a simple function",
      "input": "Add docs to: function add(a: number, b: number): number { return a + b; }",
      "assertions": [
        {"type": "contains", "value": "@param", "required": true},
        {"type": "contains", "value": "@returns", "required": true},
        {"type": "contains", "value": "number", "required": true}
      ]
    },
    {
      "id": "test-02",
      "name": "Documents async function",
      "input": "Document: async function fetchUser(id: string): Promise<User> { ... }",
      "assertions": [
        {"type": "contains", "value": "@param", "required": true},
        {"type": "contains", "value": "Promise", "required": true},
        {"type": "contains_any", "value": ["fetches", "retrieves", "gets"], "required": true}
      ]
    },
    {
      "id": "test-03",
      "name": "Documents generic interface",
      "input": "Add JSDoc to: interface Repository<T> { findById(id: string): Promise<T | null>; save(entity: T): Promise<void>; }",
      "assertions": [
        {"type": "contains", "value": "@typeparam", "required": true},
        {"type": "contains", "value": "Repository", "required": true}
      ]
    },
    {
      "id": "test-04",
      "name": "Handles class with methods",
      "input": "Document this class: class UserService { private users: Map<string, User>; addUser(user: User): void { ... } findUser(id: string): User | undefined { ... } }",
      "assertions": [
        {"type": "contains", "value": "class", "required": true},
        {"type": "contains", "value": "@param", "required": true}
      ]
    },
    {
      "id": "test-05",
      "name": "Does not overwrite existing docs",
      "input": "Add docs to:\n/** Calculate the total */\nfunction calculateTotal(items: Item[]): number { ... }",
      "assertions": [
        {"type": "contains", "value": "Calculate the total", "required": true},
        {"type": "not_contains", "value": "/**\n/**", "required": false}
      ]
    }
  ]
}
```

### 8.4 运行改进循环

```bash
# 第一轮评估
claude -p "Evaluate tsdoc-generator skill against eval.json. Run all 5 tests." \
  --allowedTools Read,Glob

# 假设test-03和test-05失败
# 分析原因：test-03使用了@typeparam而非@template；test-05对保留已有文档的处理不够明确

# 改进SKILL.md：
# 1. 在Constraints中添加：
#    "- Never replace or duplicate existing JSDoc/TSDoc comments"
#    "- Use @template tag for generic type parameters (TypeScript convention)"
# 2. 在Output Format中使用@template而非@typeparam

# 再次评估直到全部通过
```

### 8.5 添加fork隔离（v1.0.0）

```yaml
---
name: tsdoc-generator
description: >
  Generate TSDoc/JSDoc documentation comments for TypeScript functions,
  classes, interfaces, and types. Use when user asks to add documentation,
  generate docs, add JSDoc, write TSDoc, or document TypeScript code.
  Triggers on: document, jsdoc, tsdoc, add comments, docstrings for TS.
context: fork
model: sonnet
allowedTools:
  - Read
  - Edit
  - Grep
  - Glob
deniedTools:
  - Bash
  - Write
---
```

---

## 总结：Skill调试与自进化检查清单

| 阶段 | 检查项 | 工具/方法 |
|------|--------|-----------|
| 创建 | name含动作+领域 | 命名规范 |
| 创建 | description含触发词+同义词 | 触发率测试 |
| 创建 | context:fork是否适合 | 上下文依赖分析 |
| 调试 | Skill是否被Claude发现 | `ls` + verbose模式 |
| 调试 | YAML语法是否正确 | Python验证脚本 |
| 调试 | 触发是否精准 | 批量-p模式测试 |
| 调试 | fork后行为是否正常 | 隔离环境测试 |
| 评估 | eval.json断言覆盖核心场景 | 至少3个测试用例 |
| 评估 | 通过率≥80% | 自动化CI/CD |
| 进化 | 是否形成改进闭环 | Karpathy循环 |
| 安全 | 工具权限是否最小化 | allowed/denied检查 |
| 分发 | 版本号是否语义化 | semver规范 |
| 分发 | 团队安装是否自动化 | install.sh脚本 |

```mermaid
flowchart TB
    subgraph Skill生命周期
        A[设计] --> B[实现]
        B --> C[评估]
        C --> D{通过?}
        D -->|否| E[改进]
        E --> C
        D -->|是| F[部署]
        F --> G[监控]
        G --> H{问题?}
        H -->|是| E
        H -->|否| I[稳定运行]
    end

    subgraph 质量保障
        J[触发率测试] --> C
        K[Eval断言] --> C
        L[CI/CD管线] --> F
    end

    subgraph 团队协作
        M[Git仓库] --> B
        N[版本号] --> F
        O[安装脚本] --> F
    end
```

---

## 延伸阅读

- [Anthropic官方Skills文档](https://code.claude.com/docs/en/skills) — Skills创建与管理的权威参考
- [Improving skill-creator: Test, measure, and refine Agent Skills](https://claude.com/blog/improving-skill-creator-test-measure-and-refine-agent-skills) — Anthropic官方eval系统介绍
- [How Claude Skills Work: From Metadata to Automation](https://zerofuturetech.substack.com/p/how-claude-skills-work-from-metadata) — 元数据到自动化的深度解析

---

## 训练营适用性评估

| 维度 | 评估 |
|------|------|
| 适合训练营类型 | Claude Code中级开发者训练营、AI工具工程训练营 |
| 学员基础要求 | 有Claude Code使用经验，了解基本Skill结构 |
| 预计教学时长 | 4-6小时（含动手练习） |
| 实验环境要求 | 每人需要Claude Code环境 + API Key |
| 最有价值章节 | 第四章（Karpathy自改进循环）— 这是区分"能用"和"好用"的关键 |

---

## 参考来源

1. [Extend Claude with skills — Claude Code Docs](https://code.claude.com/docs/en/skills) — Anthropic官方Skills文档，涵盖创建、管理、分享完整流程
2. [Improving skill-creator: Test, measure, and refine Agent Skills — Claude Blog](https://claude.com/blog/improving-skill-creator-test-measure-and-refine-agent-skills) — eval系统与skill-creator改进官方博文
3. [How to Build a Self-Improving AI Skill with Eval.json and Claude Code — MindStudio](https://www.mindstudio.ai/blog/build-self-improving-ai-skill-eval-json-claude-code) — Karpathy风格自改进循环实战指南
4. [Inside Claude Code Skills: Structure, prompts, invocation — mikhail.io](https://mikhail.io/2025/10/claude-code-skills/) — Skill内部结构与调用机制深度分析
5. [Claude Code: The Anatomy of a Skill — Agent Builder Academy](https://agentbuilderacademy.com/blog/claude-code-anatomy-of-a-skill) — Skill解剖学与最佳实践参考

> [!tip] 51学通信知识星球
> **星主：爱卫生**（微信：gprshome201101）——公众号「51学通信」创办者，19 年通信行业运维与讲师经验，著有《GPRS网络信令实例详解》《EPC原理与实践》《5G核心网原理与实践》三本纸质书（各大电商平台有售）。
> **星球定位**：深耕 5G 核心网、IMS、NFV/AI、云原生等 ICT 领域，主打运维岗位与信令分析。
> **当前规模**：5200 篇主题 / 990 个回答，198 元/年。
> **加入福利**：加入星球后请加爱卫生微信，拉您进知识星球年度陪伴与答疑群——不限量回复您的各种问题，另有群内分享。
> **AI 加持**：51学通信知识星球已作为 skill 接入 WorkBuddy，使用 deepseek-4-flash 在海量星球帖子中精准找贴、找视频、找 FAQ 和案例，做总结、找文档和 PPT 等场景全覆盖。

![51学通信知识星球海报-扫码加入](https://wiki.500nian.com/images/zhixingqiu_qrcode.png)