大整理: 仓库重组 + 新词表 + 新README

- 清理仓库: 删除所有废弃的generate_*脚本和旧版数据文件
- 目录重组:
  - data/       → 训练数据
  - scripts/    → 数据生成脚本
  - templates/  → Web UI
- 重命名:
  - train_v2.py  → train.py
  - vocab_v3.json → vocab.json (BPE, 6.8k tokens)
- 更新 train.py 指向新词表路径
- 重写 README.md 完整文档
- 更新 .gitignore
This commit is contained in:
shuiyu1123 committed 2026-06-18 11:17:30 +00:00
1 parent 6e0ced4772
commit 231af72de9
26 files changed
+124 -126

No files matched your search

+21 -40
View File
@@ -2,54 +2,26 @@
__pycache__/
*.pyc
*.log
# 模型文件 — 只追踪发布版本
*.pth
!/llm_model_v2_resume_epoch_7.pth
/tmp/
# 中间数据(旧版临时文件)
mega_interim_*.json
essay_interim_*.json
checkpoints_v2/
# 模型权重 — 只追踪发布版本
*.pth
!/llm_model_v2_resume_epoch_7.pth
# 中间输出目录(生成脚本的输出)
high_temp_outputs/
diverse_answers/
fresh_qa_outputs/
multiturn_outputs/
checkpoints_v2/
old_ckpts/
backup_old_arch/
# 旧版数据(不再使用)
xiaoli_mega_*.json
xiaoli_multiturn_memory*.json
xiaoli_offline_distill*.json
xiaoli_personality*.json
xiaoli_safety*.json
xiaoli_sassy*.json
xiaoli_short_cot*.json
xiaoli_cot*.json
xiaoli_distill*.json
xiaoli_diverse*.json
xiaoli_drowning*.json
xiaoli_english*.json
xiaoli_final*.json
xiaoli_girl*.json
xiaoli_knowledge*.json
xiaoli_math*.json
xiaoli_merged_data.json
xiaoli_body*.json
xiaoli_child*.json
xiaoli_cute*.json
xiaoli_complete*.json
xiaoli_data_optimized*.json
xiaoli_riddle*.json
xiaoli_final*.json
# 测试输出
test_*.json
test_*.py
/var/folders
# 旧版配置文件
# 旧版废弃数据
xiaoli_*.json
mega_interim_*.json
essay_interim_*.json
distill_*.json
cot_*.json
memory_long*.json
multiturn_batch_*.json
@@ -60,7 +32,16 @@ essay_questions*.json
cot_candidates*.json
model_test_results*.json
online_distill_report*.json
# 测试/临时文件
test_*.json
test_*.py
/var/folders
# 日志
batch_runner*.log
mega_*.log
mimo_*.log
knowledge_*.log
knowledge_*.log
train_*.log
web_server*.log