Petals: Run LLMs at home, BitTorrent-style
Petals allows users to run large language models at home using a BitTorrent-style network to share model parts. It supports models like Llama 3.1, Mixtral, Falcon, and BLOOM, enabling fine-tuning and single-batch inference speeds of up to 6 tokens/sec for Llama 2 and 4 tokens/sec for Falcon.
petals.dev
1 min
8h ago
Petals: Run LLMs at home, BitTorrent-style
Petals allows users to run large language models at home using a BitTorrent-style network to share model parts. It supports models like Llama 3.1, Mixtral, Falcon, and BLOOM, enabling fine-tuning and single-batch inference speeds of up to 6 tokens/sec for Llama 2 and 4 tokens/sec for Falcon.
petals.dev
1 min
8h ago
Petals: Run LLMs at home, BitTorrent-style
Petals allows users to run large language models at home using a BitTorrent-style network to share model parts. It supports models like Llama 3.1, Mixtral, Falcon, and BLOOM, enabling fine-tuning and single-batch inference speeds of up to 6 tokens/sec for Llama 2 and 4 tokens/sec for Falcon.
petals.dev
1 min
8h ago
No more articles to load