Cyber security professionals must embrace a narrow window of opportunity to develop safeguards around AI-enhanced software generation – popularly known as vibe coding – or risk losing control of the narrative and exposing organisations to cyber attacks and other disruptions, National Cyber Security Centre (NCSC) chief executive Richard Horne has said.
In a keynote speech delivered at the annual RSAC Conference in San Francisco today, Horne called on the security community to work together to develop safeguards around vibe coding, highlighting how modern-day society faces ongoing and fundamental issues with technology thanks to exploitable vulnerabilities.
However, Horne also argued that while it was true insecure software produced without human eyes on the code could propagate vulnerabilities far and wide, well-trained AI tooling could yet create software that is secure-by-design, which would be transformative for cyber security outcomes throughout its lifecycle.
“The attractions of vibe coding are clear. Disrupting the status quo of manually produced software that is consistently vulnerable is a huge opportunity, but not without risk of its own,” he said.
“The AI tools we use to develop code must be designed and trained from the outset so that they do not introduce or propagate unintended vulnerabilities.”
Horne said cyber pros also have a responsibility to ensure that the future in which vibe-coding and other AI code-generation tools are widely adopted proves to be a “net positive”.
New paradigm
In a thought leadership blog published alongside Horne’s speech today, senior NCSC technical leadership argued that while vibe-coding poses an “intolerable risk” for many organisations as things stand, the trend offers “glimpses of a new paradigm”.
Indeed, wrote the agency’s architecture CTO, AI-backed coding could ultimately prove to be as much a technological revolution as software-as-a-service (SaaS) – pioneered at the turn of the century by the likes of Salesforce – proved to be.
While careful not to state that organisations will suddenly use AI to whip up a replacement for their CRM tools or other platforms, the NCSC said there are now clear indications that the cost versus effort curve for ‘bespoke enough’ software is shifting and as such, more and more organisations will soon begin to make different choices when it comes to software.
Given the many security concerns around SaaS – such as appropriate authentication and access controls, misconfigurations, and third-party risks – which have never really been fully addressed to the satisfaction of all, this therefore raises the question of what technology, guardrails, platforms and assurances does the security community need to have in place to ensure that the vibe-coded future is safer than the status quo.
Things to consider
Some of the safeguards that security leaders need to start to advocate for are obvious, said the NCSC. For example, AI models must be schooled in security-by-design, humans need to have confidence in the provenance of the model and trust that it hasn’t been badly-developed, and thought needs to be given to how AI can be used to review both human- and AI-generated code.
But there are also more nuanced questions, such as how to use deterministic architectures to limit what code can do should it prove malicious, compromised or unsafe, what platforms need to be designed to host AI-generated services that implement the needed controls to protect data and users, and how AI might be used to ensure the security hygiene of software through practices such as documentation, test cases, fuzzing, or updating threat models.
The NCSC noted the possibility of a future where AI code is more restricted and locked down than even the most secure on-premise or SaaS products ever were.
Ironically, it concluded, this may at long last address the unsolved security issues that still dog SaaS and that have prevented the last, most cyber-conscious hold-outs from going all in on the cloud.