Mitigating AI Risks in Software Development: A Critical Imperative

Black Duck

By Javvad Malike, Lead CISO Advisor

---

AI code generators like Copilot and Claude Code are trained on patterns observed in open source projects and publicly available code. While they can significantly accelerate development, they often prioritize functional code over security and license compliance.

A recent Cornell University study revealed that AI code generators introduce vulnerabilities in one-third of the code they produce, and when attempts are made to fix these issues, new vulnerabilities are introduced 42% of the time.

The risks are twofold:

1. Security Vulnerabilities: AI tools can flood pipelines with potentially vulnerable code, leading to massive backlogs for AppSec review. Two major issues that AI tools can introduce at scale are improper input validation and OS command injections.

2. License Risks: AI coding assistants are typically trained on open source projects, which carry specific licensing obligations. If developers use AI-generated code without understanding these licensing terms, they risk unintentionally "open sourcing" their proprietary code, devaluing intellectual property, and exposing their organization to legal implications.

Taking Control: Three Steps to Reduce AI Risk

The problem isn't AI; it's how your developers are using it. To prevent potentially devastating consequences, you need to take a proactive approach. Here are three critical steps to mitigate AI risks:

1. Automate Security Checks
   Automating security scans is essential for timely, consistent, and repeatable results. Implement CI/CD pipelines with automated scanning to detect new code in real-time, add it to test queues, and run application security testing scans. This ensures early detection of issues when they're easiest and least costly to fix. Prioritize issues for remediation based on risk-tolerance policies and automate guidance for developers on what to fix and how.

2. Implement Snippet Scanning
   Snippet scanning can quickly identify potential software license conflicts. Integrate snippet scanning tools into your dependency management systems or initiate analysis with every code commit. This maintains an up-to-date inventory of third-party components and helps avoid license compliance issues.

3. Adopt AI in Phases
   Instead of a big-bang approach, adopt AI gradually. Restrict AI access to certain teams based on their readiness and project criticality. Ensure these teams have robust mitigation controls, including established protocols, code review processes, and testing frameworks. Use pilot projects to evaluate AI's impact and establish a feedback loop to refine your AI strategy.

Securing Your AI-Generated Code

To effectively mitigate risks, implement the following security controls:

• Training and Education: Educate developers on potential risks associated with AI-generated code.
• Security Tools and Practices: Implement security best practices and tools discussed in this article.
• Code Review Policies: Ensure all code is reviewed by experienced developers with security and compliance expertise.
• Testing Frameworks: Validate AI-generated code with comprehensive testing, including unit tests, integration tests, and security tests.
• Incident Response Plans: Develop plans for handling breaches and issues related to AI-generated code.
• Regular Audits: Conduct regular audits of AI-generated code and development processes.

By taking these steps, you can harness the power of AI in software development while minimizing its risks.

---

www.blackduck.com