According to the introduction, the model is built from scratch with a brand-new architecture, and its pre-training corpus includes a wealth of programming content, optimized after training on datasets ...