AI NEWS.ccstarcoder2-3b
1. Model Summary 2. Use 3. Limitations 4. Training 5. License 6. Citation StarCoder2-3B model is a 3B parameter model trained on 17 programming languages from The Stack v2, with opt-out requests excluded. The model uses Grouped Query Attention, a context window of 16,384 tokens with a sliding window attention of 4,096 tokens, and was trained using the Fill-in-the-Middle objective on 3+ trillion tokens. - **Project Website:** bigcode-project.org - **Paper:** Link - **Point of Contact:** contact@bigcode-project.org - **Languages:** 17 Programming languages
1. Model Summary 2. Use 3. Limitations 4. Training 5. License 6. Citation StarCoder2-3B model is a 3B parameter model trained on 17 programming languages from The Stack v2, with opt-out requests excluded. The model uses Grouped Query Attention, a context window of 16,384 tokens with a sliding window attention of 4,096 tokens, and was trained using the Fill-in-the-Middle objective on 3+ trillion tokens. - **Project Website:** bigcode-project.org - **Paper:** Link - **Point of Contact:** contact@bigcode-project.org - **Languages:** 17 Programming languages