Devin: Introducing the World’s First Ever AI Software Engineer
Devin AI, ‘the world’s first fully autonomous software engineer,’ likely marks the next disruption that artificial intelligence (AI) leaves on the world: Who needs a skilled coder anymore?
That’s the question we pose to various experts across the AI and software engineering fields, as we are not convinced that human coding is going anywhere soon.
When Cognition, the team behind Devin AI, unveiled videos of its early-access master coder last week — writing code from prompts, bug-fixing on the fly, even handling paid-for Upwork tasks — there have been breathless exclamations that this is the end of coding as we know it.
Benchmark results
The company said that Devin AI is on the SWE-bench coding benchmark, the company which is a dataset that comprises 2,294 software engineering problems that are extracted from authentic GitHub issues and their corresponding pull request.
You might wonder how Devin compares to AI chatbots like ChatGPT and Claude. In SWE-bench, a benchmark comprising thousands of software engineering problems sourced from GitHub issues, Devin outperformed every competitor, including Claude 2, LLAMA, and GPT 4. Moreover, unlike with other AI chatbots, Cognition Labs claims that they only evaluated Devin on a 25% subset of the dataset without any assistance.
When will Devin be available to the public?
While Cognition Labs has not disclosed specific details regarding the rollout, the company is currently accepting applications from businesses interested in implementing the chatbot.
Conclusion
Devin AI is a huge stride forward in the Generative AI realm, revolutionizing the software development field by automating coding tasks and complex problems. With models like GPT-4, Claude 3, and now Devin out, the future seems hopeful in Generative AI; they are not here to replace us but to assist us.