Our Morpheus is unique with its technical infrastructure and training process.
21 Sep 2023, 18:34
Our Morpheus is unique with its technical infrastructure and training process! 💎
"In actual training, Morpheus uses two different mask identifiers ([MYTH] and [gMYTH]), respectively for short and long text generation. In addition, it adopts recent techniques such as Rotary Position Embedding (RoPE), DeepNorm layer normalization, and Gaussian Error GLU (GeGLU). All these designs and techniques contribute to the stable training and high-precision performance of Morpheus's large-scale language model. Specifically, in the current public beta version of Morpheus, the model has 55 layers of Transformer, a hidden layer dimension of 7,200, a maximum sequence length of 2,048, and a bilingual tokenizer based on icetk with 100,000 identifiers."
🔹Detailed information about Morpheus' training process and technical features is in our Chief AI Scientist Steve Deng's article.