Meta releases powerful ‘Code Llama 70B’, an open-source coding tool to rival GitHub Copilot

Code Llama is derived from Llama 2 and is fine-tuned for generating computer code.

Code Llama model is available for free to researchers and developers (Image credit: Meta)

Meta’s most advanced and free-to-use large language model, Code Llama, is now available for download for researchers and developers. This generative AI model boasts 70 billion parameters and was originally announced in August 2023.

Today we’re releasing Code Llama 70B: a new, more performant version of our LLM for code generation — available under the same license as previous Code Llama models.

Download the models ➡️ https://t.co/fa7Su5XWDC
• CodeLlama-70B
• CodeLlama-70B-Python
• CodeLlama-70B-Instruct pic.twitter.com/iZc8fapYEZ

— AI at Meta (@AIatMeta) January 29, 2024

“We’re open-sourcing a new and improved Code Llama, including a larger 70B parameter model. Writing and editing code has emerged as one of the most important uses of AI models today. The ability to code has also proven to be essential for AI models to process information in other domains more rigorously and logically. I’m proud of the progress here and look forward to including these advances in Llama 3 and future models as well,” said Meta CEO Mark Zuckerberg in a Facebook post.

Also read | Llama 2 from Meta signals worry for OpenAI’s ChatGPT: What is it & how to access

Meta states that Code Llama is the most advanced and best-performing model in the Llama family, available for free in three versions: CodeLlama – 70B, the foundational code model, CodeLlama – 70B – Python, 70B specialized for Python, and Code Llama – 70B – Instruct 70B, which a version fine-tuned with human instruction and self-instruction code synthesis.

Code Llama is built on top of Llama and is capable of generating code. According to the company, the model has scored 67.8 on HumanEval, a generative AI benchmark, while the GPT-4 Turbo, a much bigger model, has scored 81.7. Meta also claims that Code Llama is tuned for code generation, and the best part is that it is an open-source model.

The latest generative AI model from Meta can write code in C++, Java, PHP, and Python. The model has been trained using 500 billion tokens of code and related data, and Meta has also expanded the context window, which can now accept inputs with up to 100,000 tokens or around 75,000 words.

Also read | Meta launches LLaMA model, a research tool more potent than OpenAI’s GPT-3

It is worth noting that Code Llama is based on Llama, a 175-billion-parameter general-purpose AI model. Users can access Code Llama through platforms like Hugging Face or PyTorch, and the company has also confirmed that some of these advancements will be included in the upcoming Llama 3 and future models by Meta.