ChatGPT and the Future of Software Development Process
Summary
The promise of ChatGPT and the underlying AI technologies as an optimization tool for the software development process is great. In this Technology Overview we gave an engineer 24 hours to conduct a spike on how ChatGPT could assist with code related searches, debugging, template creation, test data creation, refactoring existing code, writing documentation, and coding education.
We expect that ChatGPT or equivalent technologies will be quickly integrated into common development environments such as Visual Studio Code and Github, in a way that is optimized for development team members.
Technology experts should encourage their teams to learn how to ask questions of ChatGPT. As the technology is integrated into common tools and workflows, we feel the ability to converse and ask the right questions the right way will be an asset.
Optimized Searching with ChatGPT
Google is our best friend in software engineering, but what if we could Google faster? What if we could find the exact command or documentation to support our use case? What if we could do this without sifting through multiple pages of links and instead just be provided with an answer? Enter ChatGPT, which promises to do the leg work by providing consolidated natural language (NLG) results in seconds. Our early research shows that the correct answer is not guaranteed with ChatGPT although it is generally close. Often a search result from ChatGPT can be improved by rephrasing the question and by the end user learning how to interact with the software to prompt a desired result.
Let’s look at how ChatGPT thinks a React application should be built and deployed:
The answer is a general overview of steps to deploy and can be used as building blocks to begin deployment process. In a follow-up question, we could ask how to deploy using AWS Amplify and ChatGPT would provide the step-by-step instructions as well. ChatGPT is conversational therefore it will remember previous discussion topics. In this case the end user doesn’t need to specify the deployment of a “react application” again as the AI will reference previous questions in the chat to determine the context of the discussion.
Debugging with ChatGPT
Moving onto debugging, when provided an error message the AI can provide a detailed explanation of the potential issue beyond the error message itself. When provided a sample piece of code where the error exists in addition to the error message, the chance of finding a solution is greater. We can also detect vulnerabilities in code such as an SQL injection with this line: sql = "SELECT id FROM users WHERE username='" + user + "' AND password='" + pass + "'".
Here is ChatGPT’s solution:
In this response, we have an updated piece of code to use and an explanation of the logic. Next, we can ask for some SQL injection payloads to test against a potential attack.
Creating Template Code with ChatGPT
ChatGPT can create simple templates in many languages. It can build template code for a simple front-end application, a server, or tests for both user interfaces and API services. With a template on hand, a developer could then prompt the AI to build off the template to add specific functionality, whether it is an automatic slider with navigation buttons or an endpoint that accepts and returns data. Our experience was that the generated code did not immediately work and required manual adjustments. Like reviewing stack overflow or other developer resources, a developer may find a piece of code with an explanation that closely suits their use case but requires some refactoring to meet the exact use case.
To set up an Express Server in Node.js, we could ask ChatGPT for a template:
Refactoring Code with ChatGPT
The ability for ChatGPT to refactor and optimize existing code may be one of its most powerful features and is one that we could see being quickly integrated into popular development environments. Ask ChatGPT to improve a piece of code and it will return an improved result if possible. The technology will modify the code for efficiency and may implement best practices. For example, when asked the following question: “Add slider navigation buttons to this code: ”
ChatGPT provided the following result:
The buttons were added and the logic within setInterval() was abstracted to handleNextSlide() to reduce code. Note that sometimes larger chunks of code produced by ChatGPT can get cut off or look incomplete, to solve for this type in “continue” and the remaining code will be displayed. Ask for styles to go with the code to get fully functional styles as well.
We achieved the best results in our experiments when we iterated over the same piece of code with multiple small questions/requests. Granted we are still working to understand how to ask more complex questions in an optimum way for the technology.
Creating Test Data with ChatGPT
In many scenarios, software engineers require test data to build out required functionality. Even though it is a relatively simple process to create test data, it can be time-consuming and lack complexity in comparison to other aspects of the software being built. With ChatGPT, test data could be generated on the fly according to the specifications provided. This would allow software engineers to focus on bigger problems that AI cannot solve.
Let’s create some simple user data:
Notice that the required customer order data was not requested in the search yet ChatGPT and the underlying AI algorithms made an assumption and created order data anyways. With experimentation, the request could be further and further refined to add or remove customer order data to meet test data needs.
Code Documentation with ChatGPT
Another necessary task is to write documentation on code bases. While this can be considered easy in comparison to building software, it still needs to be done and can be augmented via AI. Given a chunk of code, a detailed overview of the software could be generated. Definitions of props, functions, and dependencies are all described in a cohesive document that is easy to understand. For example, when asked these questions along with a code snippet: How to run, build, and version release? What is this endpoint? What is this component? What is this reducer? What is this middleware? We found that all results accurately described the purpose of the software and how it worked.
Code Mentoring
ChatGPT typically provides a detailed description along with source code results. For new software engineers and developers learning a new area, ChatGPT represents a 24/7 mentor. Through natural language generation (NLG), users can ask questions and receive answers in digestible text with step by step walk through explanations in many cases. Learning independently involves a lot of hurdles and with the benefit of ChatGPT we believe those hurdles can be overcome more quickly.
When it comes to utilizing ChatGPT to its full potential, the ability of the end user to write pseudo-code well and ask questions in an optimal way for ChatGPT is critical. Providing context and often the code to steer results in the desired direction is as much art as it is science. Software engineers must start now to understand the nuances of interacting with AI.
