Skip to content

improve aws-detective skill structure + eval score#36

Open
fernandezbaptiste wants to merge 1 commit intomukul975:mainfrom
fernandezbaptiste:improve/aws-detective-skill-quality
Open

improve aws-detective skill structure + eval score#36
fernandezbaptiste wants to merge 1 commit intomukul975:mainfrom
fernandezbaptiste:improve/aws-detective-skill-quality

Conversation

@fernandezbaptiste
Copy link
Copy Markdown

@fernandezbaptiste fernandezbaptiste commented Apr 3, 2026

hey @mukul975, thanks for building this cybersecurity skills collection. really like the MITRE ATT&CK mapping approach across 750+ structured skills. Kudos on passing 4k stars! I've just starred it.

ran your aws-detective skill through agent evals and spotted a few quick wins that took it from ~72% to ~90% performance:

  • expanded description with trigger terms like AWS Detective, security investigation, GuardDuty findings, VPC flow logs so agents reliably match user requests

  • restructured workflow with clear verification checkpoints + error recovery for common API failures

  • tightened verbose prose + consolidated tables to improve conciseness

these were easy changes to bring the skill in line with what performs well against Anthropic's best practices. honest disclosure, I work at tessl.io where we build tooling around this. not a pitch, just fixes that were straightforward to make!

you've got 754 skills, if you want to do it yourself, spin up Claude Code and run tessl skill review. alternatively, let me know if you'd like an automatic review in your repo via GitHub Actions. it doesn't require signup, and this means you and your contributors get an instant quality signal before you have to review yourself.

@mukul975
Copy link
Copy Markdown
Owner

Hi @fernandezbaptiste Thanks for this — genuinely useful feedback. Before we add the GitHub Action, two questions: does the reviewer understand domain-specific context (e.g., MITRE ATLAS technique IDs, D3FEND mappings)? And is the quality rubric publicly documented so contributors know what it's optimising for?

@mukul975 mukul975 self-requested a review April 11, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants