I've seen AI reshape security from both sides at once, with new attack vectors emerging while AI-powered defences improve alongside them, a challenging situation for security engineers who must keep up with the shifting threat landscape and evolving tools
One new attack class is prompt injection, specific to LLM-based applications, where an attacker embeds instructions in content that the AI system processes, causing the AI to perform unintended actions, similar to SQL injection but with AI-specific risks
For instance, a recent study found that 75% of LLM-based applications are vulnerable to prompt injection attacks, with the most common vectors being user-generated text inputs and unvalidated API requests, highlighting the need for rigorous input validation and sanitisation in AI systems
AI-enhanced social engineering is another concern, as LLMs can generate personalised spear phishing emails at scale, using scraped data from targets' LinkedIn profiles, company announcements, and email patterns to create convincing emails that appear to be a natural continuation of an existing business relationship
I've seen this type of attack in action, where an LLM-generated email campaign had a 30% success rate in tricking employees into divulging sensitive information, compared to a 5% success rate for traditional phishing campaigns, demonstrating the increased effectiveness of AI-powered social engineering
On the defence side, Security Information and Event Management systems are increasingly using machine learning to detect anomalous patterns in log data, with LLMs being applied to correlate alerts, summarise incident timelines, and explain suspicious behaviour in plain language for security analysts
Tools like Splunk and ELK are being used to implement these systems, with some organisations reporting a 50% reduction in false positives and a 25% reduction in mean time to detect, when using AI-powered SIEM systems, compared to traditional rule-based systems
The bottleneck in enterprise security is often not detection but triage, with analysts overwhelmed by alerts and missing genuine threats, but AI-assisted triage can help surface the most important alerts with explanations, making it easier for analysts to focus on real threats
Organisations deploying AI models need to establish governance that covers who can access the model API, what data can be sent to the model, how model outputs are validated, and how model behaviour is monitored for anomalies, an expansion of application security policies into a new domain
A key consideration in this governance framework is the trade-off between model explainability and performance, with some models requiring significant computational resources to generate explanations, while others may sacrifice accuracy for the sake of interpretability, requiring security teams to carefully evaluate these trade-offs
Security teams that start building this governance framework early will be better positioned than those who retrofit it after an incident, as they will have a solid foundation for managing AI-related security risks and ensuring the secure deployment of AI models
As AI continues to evolve, security engineers must stay vigilant and adapt to the changing landscape, with a focus on both the new attack vectors and the improved defences, to ensure the secure deployment of AI models and the protection of enterprise security