DiscoLM 120b

DiscoLM 120b is DiscoResearch's 120-billion-parameter, instruction-tuned LLM, built on the Alpindale Goliath merge and scoring 73.198 on the HF Leaderboard.

DiscoLM 120b is DiscoResearch's experimental 120B large language model (LLM), trained by Björn Plüster. The architecture is based on Alpindale's Goliath 120b: a strategic merge of Llama2-70b models. We then applied extensive instruction finetuning, incorporating high-quality open-source datasets like SlimOrca-Dedup and OpenHermes. This process yielded strong results: the model achieved an average score of 73.198 on the Hugging Face Leaderboard tasks, positioning it as a top performer in the >70B parameter class upon its alpha release.

https://huggingface.co/DiscoResearch/DiscoLM-120b