Ethical considerations in AI-driven software development

Artificial intelligence has dominated the headlines in 2023. The universal release of the large language model ChatGPT v3 took the world by storm, sending a shock wave through nearly every industry. Thanks to OpenAI’s free platform, ChatGPT has surpassed 100 million users with 1.6 billion website visits per month.  

‍

AI-driven software development is one of the most exciting capabilities of the latest LLM technologies. While implementing AI into business workflows can save countless human hours, the ethics of generated code is among the most concerning issues of modern society. 

‍

The importance of ethics in AI-driven software development

As our lives become increasingly digitalized, ethical software development is more urgent. Whether it's data collection for advertising or algorithms used to predict recidivism rates for prisoners, the software has real-world consequences. When we include artificial intelligence in the development loop, we must be extremely cautious to ensure we don’t have disastrous, unintended outcomes.  

‍

The alignment problem

AI allows us to utilize massive data sets to build algorithms. While this can improve software applications, there’s a disconnect between the capacity of the human brain and the capabilities of a large language model. In simple terms, we can give an AI a task to get a seemingly favorable result, but we do not know where it came from. Experts in the field, like Brain Christian, call this issue “the alignment problem.”

‍

Understanding the consequences of AI

In some cases, such as allowing artificial intelligence to detect nuclear weapons deployment and request to launch nukes in response, the stakes couldn’t be higher. In 1983, lieutenant colonel in the Soviet Air Defense Forces, Stanislav Petrov may have single-handedly saved the world. He was in a bunker in Moscow when the Oko early-warning satellite system detected the launch of multiple intercontinental ballistic missiles from the United States. Thankfully, Petrov used his judgement and waited to inform his superiors. The computer-generated alert ended up being caused by sunlight reflecting off high-altitude clouds.

‍

While instances like Petrov’s help understand the potential threat of unmonitored AI, using Codex to produce basic code like a sticky header has far less drastic consequences. So, keep in mind the impact of your project when considering AI ethics.  

‍

How does AI know how to code?

Another important aspect of AI ethics is understanding how machine learning algorithms work. Here’s a brief overview of how ChatGPT learned to code:  

‍

LLMs like ChatGPT analyze code examples to generate new code. It uses natural language processing to understand user questions and create code snippets with solutions. Thanks to training models from a massive collection of books, articles, and web pages, ChatGPT can produce code in several computer languages, including Python, Java, and C#.

‍

Ethics in AI software development

AI learns based on thousands, millions, or billions of examples and then delivers an answer to the best of its capability. So, whether it's pulling from W3 Schools, Harvard CS courses, or real examples, the data comes from a human’s work. As you know, humans make mistakes and have biases, prejudices, and flaws. As a result, we must be conscious of how machine learning algorithms are trained, limiting the “alignment problem” as much as possible.

‍

Bias in training data

The outcomes of an AI application are limited to its training models and the bias included in the data. This is known as machine learning bias, algorithm bias, or AI bias. The event occurs when the algorithm produces results that are influenced by systemic prejudices due to:

‍

• How the data was collected
• Availability of data
• How the outputs interpreted
• Assumptions made during development
• Unconscious human biases reflected in the applications

‍

Possibly the most famous example is the racist mistake made by Google Photos in 2015. The app labeled a black couple as gorillas and was noticed by a New York-based software developer who was also featured in the image. While Google and its developers were dragged online as racists, the issue is much more complicated. The data sets use images in the public domain, causing two significant problems: black people are underrepresented in the models, and camera technology at the time was still optimized for lighter skin tones.  

‍

Reusing code and plagiarism in AI software development

Another ethical concern is plagiarism. This issue is more of a concern for AI-generated text sourced from copyrighted material (Open AI is being sued at the time of writing by a group of 17 authors, including John Grisham, Jodi Picoult, and George R.R. Martin) but also concerns software development.

‍

Reusing code is an intrinsic part of computer programming. We all learn to code using examples, and utilizing templates, themes, and boilerplates is standard practice. The problem with AI-generated code is that you don’t know where it comes from. For example, if the sticky header is sourced from a paywalled WordPress theme, this would be considered intellectual property theft and unethical.

‍

AI taking jobs from humans

Entry-level jobs in multiple sectors are at risk of being shifted to AI or automation, including software development. A McKinsey study reports that between 2016 and 2030, 15% or 400 million people will be displaced by AI. In May of 2023 alone, AI eliminated 4,000 jobs, according to CBS News. The threat of AI displacement is real, but how software developers will be affected is still uncertain.

‍

LLMs can produce parts or entire applications that usually require a team of programmers. As a result, we will undoubtedly experience a shift in how software is developed. However, optimists believe AI will empower the industry, making the work less monotonous by completing tedious tasks in seconds. Contrarily, AI pessimists believe that the world will need far fewer software developers in the near future because of advancements in machine learning.

‍

Security and privacy concerns in AI

‍

Using AI-generated code also presents concerns related to cybersecurity and data privacy. According to a study by Stanford affiliates, engineers who use code-generating systems are more likely to create security issues. Relying on an AI to build parts of an application outside a developer’s expertise causes significant risks. If the developer doesn’t understand the code, it will likely contain vulnerabilities.

‍

Equally concerning is what is known as model poisoning attacks. This is when the training data has been compromised to produce code that is vulnerable or ready to easily execute cyber-attacks like malware.

‍

Ethical AI software development practices

Human oversight is the most important aspect of utilizing AI for software development. The example of the Oko early-warning satellite system is the most apparent representation of the risks and benefits of keeping humans in the loop. Relying too much on AI systems to make decisions is problematic, regardless of the application. Moving forward, we must understand algorithm bias and implement safeguards allowing humans to control outcomes. AI is an incredibly exciting tool with incredible potential, but there are serious ethical considerations with real-world impact.

‍

‍