-
小狸LLM v1.3-lite-flash
shuiyuPyLab released this
2026-06-19 10:35:10 +08:00 | 25 commits to master since this release模型配置:
- vocab_size: 47725 (CharTokenizer.build_vocab从977K数据生成)
- d_model: 320, n_heads: 8, n_layers: 6, d_ff: 1536
- 参数量: 41.85M
- 训练: batch=224×accum=2(等效448) + AMP + compile
- GPU: NVIDIA L40
- Epoch 1: train_loss=2.6979, val_loss=2.0374
新特性:
- 用build_vocabulary从数据重新生成词表(不再手写)
- torch.compile加速训练约2x
- 977786条QA数据全量训练
- 字符+BPE混合分词(中文按字符+英文按空格)
Downloads