Skip to content

Feature request: SecOps review update for copilot security instructions #5

@scottqueen-bixal

Description

@scottqueen-bixal

Summary

Update security.instructions.md based on SecOps feedback.

  • ensure updates keep with concision and low token utilization

Additional context

A few recommendations based on your clarification and the file as an "input" artifact to the agent:

  • Fail-safe: make sure to incorporate a fail-safe, safe defaults behaviors and mechanisms for the agent. I.e.: when a developer needs to call a specific token to perform an action, do not assume the developer has the permissions, instead request for validation. Also, if there's uncertainty on whether the data is sensitive, this will help default to treating it as sensitive regardless.
  • Adversarial usage: Assume adversarial usage for the instructions to the agent. The guidance here is always based on "good intentions" or us expecting the agent to be benign. If we turn it around and think of this looking from an adversarial perspective — how can I manipulate the prompt to get me something I want? This "thought" exercise can help tighten up the guidance in cases where AI / Copilot might misbehave or it's being manipulated. Which, brings me to the next point.
  • Strict phrasing: for items we clearly want to avoid, instead of using the word "avoid" I'd recommend using a more direct and strict approach — machines are machines, haha. We're better off telling them straight yes or no. For example, for "Avoid logging sensitive data (passwords, tokens, PII)", I'd instead use the following phrasing "Do not log sensitive data (passwords, tokens, PII)".
  • Testing and validation: may be good to add some guidance and propose testing and validating the instructions are effective. You can instruct the agent when a case is found, to test it or simulate misuse.
  • Versioning: if you're keeping this as a living document that may potentially evolve, perhaps add version that could be tied to model updates and/or Bixal policy changes.

Metadata

Metadata

Labels

enhancementNew feature or request

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions