AI chatbots run a software development company: This is what happened – The Indian Express
Can AI run a tech start-up all by itself? Even as Artificial Intelligence technologies continue to bask in all the limelight, newer use cases are emerging. And, with each new use case, the conversations surrounding AI are getting more and more amplified. AI has been touching various aspects of work, now it seems AI can operate a software company in a cost-efficient manner on its own. Researchers documented how they created “ChatDev” a virtual software development company that worked on a task created by a human “client” in a pre-print paper available on the arXiv.
The team of researchers from Brown University and other institutions conducted an experiment to see if AI bots powered by GPT -3.5 could accomplish a software development process without any training. As part of the study, researchers developed a hypothetical software development company known as ChatDev, based on the waterfall model. The waterfall model is essentially a chronological approach to developing software. ChatDev was fragmented into four stages – designing, testing, coding and documenting.
AI bots were assigned different roles with prompts that described their tasks and roles, communication protocols, constraints, and termination criteria. After the bots were given their roles, the researchers later designated them to their respective stages. To simplify, the CEO and CTO of ChatDev were deployed in the designing stage, while the art designer and the programmer were allocated to the coding stage.
“In this study, we have presented ChatDev, a chat-based end-to-end software development framework that leverages LLMs to facilitate effective communication and collaboration among multiple roles involved in the software development process. By decomposing the development process into sequential atomic subtasks through the use of the chat chain, ChatDev enables granular focus and promotes desired outputs for each subtask,” the researchers wrote.
According to the research paper, during each stage, all the AI bots interacted with each other with minimal to no human input on the specific parts of the software development process. These processes ranged from deciding on a programming language to checking bugs in the code. The researchers conducted the experiment on various software scenarios and also applied a series of scenarios, to gauge how much time it took the bots to complete each type of software and the cost.
The group of researchers also tasked ChatDev to create a basic Gomoku game (a strategic board game). According to the paper, 86.66 per cent of the generated software systems were executed ‘flawlessly’.
The latest study throws light on how generative AI technologies such as ChatGPT have the ability to perform certain specific tasks. The study has, however, been not perfect as researchers also identified a host of issues such as biases in the language models and errors. However, the team is hopeful that the study may benefit budding programmers and engineers.
“The overarching objective is to achieve even greater efficiency in software production by improving various characteristics, such as reducing the length of chat chains or optimizing subtask-solving logic and strategies, ultimately leading to more streamlined and effective software production processes. We hope the potential of the proposed natural-language-to-software framework can illuminate fresh possibilities for integrating LLMs into software development and mark the dawn of a new frontier in the field of natural language processing, software engineering, and collective intelligence,” read the paper.
