Run dolphin-2.2-yi-34b on IoT Devices (Also works as a Private OpenAI API Server)

Home › Games, Other Content › Run dolphin-2.2-yi-34b on IoT Devices (Also works as a Private OpenAI API Server)

Channel: Second State • Sourced: Second State • Published 8 months ago

Run dolphin-2.2-yi-34b on IoT Devices (Also works as a Private OpenAI API Server)

The Yi-34B model is an impressive bilingual LLM with a large context length. It is then fine-tuned with the Dolphin dataset to “learn” reasoning and uncensor its biases. The result is an excellent LLM that can be further tuned for personality and alignment.

Using WasmEdge, I am running dolphin-2.2-yi-34b on a Jetson IoT device at 6 tps. Try it on your device and get uncensored LLM answers https://secondstate.io/articles/dolphin-2.2-yi-34b/