As expected, OpenAI is coming out strong against Anthropic’s security releases
“Defenders can bring secure code review, threat modeling, patch validation, dependency risk analysis, detection, and remediation guidance into the everyday development loop so software becomes more resilient from the start.”
As expected…
“Defenders can bring secure code review, threat modeling, patch validation, dependency risk analysis, detection, and remediation guidance into the everyday development loop”
Over the last few weeks, I've been building out an open source security and compliance tool for AWS and Azure. The initial output looked **pretty decent**, but as I put it to the test against real-world cloud environments, a number of **key gaps** emerged.
I decided to go deeper into Claude Code's working and ask it out how we could have avoided or reduced these gaps. It's response was super interesting and probably not surprising for others on this subreddit. But definitely enlightening for me.
I then asked it to document all these gaps into a markdown, which reference we then added into Claude.md to make sure we avoided them into the future. Some of the key lessons were:
This is besides extensive use of plan, ultraplan, brainstorm and other modes that I found very insightful, but they didn't fix the basic coding hallucination and quality issues I've enumerated above.
What are your guardrails to ensure you build trustworthy and reliable software?
Over the last few weeks, I've been building out an open source security and compliance tool for AWS and Azure. The initial output looked **pretty decent**, but as I put it to the test against real-world cloud environments, a number of **key gaps** emerged.
I decided to go deeper into Claude Code's working and ask it out how we could have avoided or reduced these gaps. It's response was super interesting and probably not surprising for others on this subreddit. But definitely enlightening for me.
I then asked it to document all these gaps into a markdown, which reference we then added into Claude.md to make sure we avoided them into the future. Some of the key lessons were:
This is besides extensive use of plan, ultraplan, brainstorm and other modes that I found very insightful, but they didn't fix the basic coding hallucination and quality issues I've enumerated above.
What are your guardrails to ensure you build trustworthy and reliable software?
One issue that we all have is the flood of information on any given topic. Much of which is AI slop.
I’ve been on an AI Security learning journey these past 18 months and had been maintaining a list of good resources.
Curated them here - Project Basecamp
https://kkmookhey.github.io/basecamp-ai-sec/
Hope you’ll find them useful as well! Do suggest any I might’ve missed.
If you have been following the “Trivy -> Checkmarx -> Dependabot -> Who else” saga, here are the top 10 things to secure your dev environment:
Pin GitHub actions to SHA keys, not version tags
If you aren’t sure you’ve been compromised or not, rotate all your creds anyway - Github keys, API keys, DB credentials, LLM keys, etc.
Use short-lived credentials via OIDC, not long-lasting cloud keys
Protect publisher and maintainer accounts with MFA - even investing in hardware keys if you can afford it
Scope every token to the minimum access it needs - be it a PyPi or npm token or a cloud account. Probably do an end-to-end access review immediately
Add dependency cooldowns - don’t auto-install a newer version of a package the day it is released
Audit OAuth grants in Google Workspace, Microsoft Entra (the Vercel hack was partly because of this)
Have a supply chain incident response playbook
Run SCA to check and fix all known vulnerable or malicious package dependencies
I’d love to say implement egress filtering, but in fast moving dev environments that may not always be possible.
Anything you’d add or change?