Stability AI has launched StableLM, an open-source language model that can generate text and code and will power a range of downstream applications. The Alpha version of the model is available in 3 billion and 7 billion parameters, with 15 billion to 65 billion parameter models to follow. StableLM is trained on a new experimental dataset built on The Pile, but three times larger with 1.5 trillion tokens of content. The richness of this dataset gives StableLM surprisingly high performance in conversational and coding tasks, despite its small size of 3 to 7 billion parameters. StableLM is designed to be transparent, accessible, and supportive. The open-source models can be adapted for commercial or research purposes, and researchers can look under the hood to verify performance, work on interpretability techniques, identify potential risks, and help develop safeguards. StableLM is part of Stability AI’s commitment to AI technology that is transparent, accessible, and supportive. The economic benefits of AI are shared by a broad community of users and developers. StableLM is now available in the company’s GitHub repository, and a full technical report will be published in the near future. The company is also growing its team and looking for experienced LLMs passionate about democratizing access to this technology.
