~/bookmarks

William's Bookmark Library

/**/

GitHub - Saivineeth147/lora-speedrun: Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

github.comSaved July 20, 20265 min

Preview of GitHub - Saivineeth147/lora-speedrun: Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

Saivineeth147 · via GitHub

Summary

A GitHub repository benchmarking LoRA fine-tuning speed: reach ≥57% GSM8K accuracy on Qwen2.5-1.5B on a single NVIDIA L40S, tracked on a public wall-clock leaderboard.

Submissions use modded-nanogpt with frozen base weights and a ≤30M-parameter PEFT adapter; records require three fresh-seed verification runs in a network-blocked Modal sandbox. The current record is 6 minutes 05 seconds via sequence packing and completion-only loss masking over 2 epochs.

Topics

LoRA Fine-Tuning Parameter-Efficient Fine-Tuning GSM8K Benchmark LLM Fine-Tuning Qwen2.5

View on GitHub All Bookmarks

William's Bookmark Library

/**/

GitHub - Saivineeth147/lora-speedrun: Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

github.comSaved July 20, 20265 min

Saivineeth147 · via GitHub

Summary

A GitHub repository benchmarking LoRA fine-tuning speed: reach ≥57% GSM8K accuracy on Qwen2.5-1.5B on a single NVIDIA L40S, tracked on a public wall-clock leaderboard.

Submissions use modded-nanogpt with frozen base weights and a ≤30M-parameter PEFT adapter; records require three fresh-seed verification runs in a network-blocked Modal sandbox. The current record is 6 minutes 05 seconds via sequence packing and completion-only loss masking over 2 epochs.

Topics

LoRA Fine-Tuning Parameter-Efficient Fine-Tuning GSM8K Benchmark LLM Fine-Tuning Qwen2.5

View on GitHub All Bookmarks

~/bookmarks

GitHub - Saivineeth147/lora-speedrun: Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

Summary

Topics

Discover Similar Content

Overview - Together AI docs

RedHatAI/gemma-4-31B-it-NVFP4 · Hugging Face

GitHub - MakazhanAlpamys/Soup: Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

Discover Similar Content

GitHub - Saivineeth147/lora-speedrun: Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

Summary

Topics

Discover Similar Content

Overview - Together AI docs

RedHatAI/gemma-4-31B-it-NVFP4 · Hugging Face

GitHub - MakazhanAlpamys/Soup: Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.