armenian-llm-hw1

Summary

A complete implementation of a decoder-only GPT model trained from scratch on Armenian text. The project includes corpus preparation (ArmWeb and Wikipedia), custom Byte Pair Encoding (BPE) tokenizer training, pretraining, continued pretraining on a mixed dataset, and comprehensive evaluation. The final model has 97.2M parameters and was trained on an H100 GPU. The repository is well-structured with modules for data, tokenization, modeling, training, inference, evaluation, and reporting, accompanied by detailed notebooks and documentation.