Talk to UsResponsible AI resourceMenuHomeModel policyDecisionsResearch & approachGet support
Leadership guide/Operations & assurance
Can the boundaries hold when the system behaves unexpectedly?
Download the guide↓Editable working records→
Protect the service around the model: accounts, information stores, interfaces, connected tools and logs. Match the testing and controls to what the service can actually reach and do. Apply the institution's existing security standards and establish the requirements below before release.
Responsible roles: Information security and IT with the service and data owners.
Leave with: A verified control record for the approved environment.
Document information flows, storage, retrieval sources and recipients. Enforce access separately for each user and service identity. Restrict retrieval and connected actions to the approved task. Recheck permissions when roles or integrations change.
Use institution-managed service identities and secret storage. Keep passwords, tokens and keys out of prompts, shared documents and source repositories. Assign credential owners, rotation and revocation arrangements. Authenticate interfaces, enforce authorization, validate tool inputs and set rate and spending limits.
Retrieved pages, email, documents and tool results may contain instructions intended to redirect the system. Test whether those instructions can change its task, obtain protected information or trigger unauthorized actions. Keep critical permissions in system controls and authorized approval steps.
Use controlled test records to attempt cross-user access, unauthorized retrieval, leakage through outputs and inappropriate storage in logs. Verify deletion and retention settings, access revocation and the behavior of connected services. Do not use live sensitive records merely to test a boundary.
Monitor unusual access, attempted privilege changes, repeated tool calls, unexpected recipients and spending or usage spikes. Name who receives alerts and can disconnect the service. Protect logs with limited access and a justified retention period; avoid unnecessary prompt or record content.
These requirements supplement the existing control checks. Apply them to the actual deployment and document which controls are provided by the institution and which by the supplier. Required protections cannot be assumed from a tool approval.
For services handling protected information or acting within institutional systems, require multifactor authentication for human access, managed task-limited service identities and timely removal of access. Encrypt protected information in transit and at rest. Control credential and key storage, rotation and revocation. Apply institutional vulnerability-remediation requirements, protected logging and tested recovery or fallback arrangements. Record evidence and any permissible alternative under the established exception process.
Treat outputs as untrusted input. Validate tool arguments, encode displayed content for its context and use parameterized database operations. Execute generated code or commands only within an approved, constrained process with appropriate isolation. Test attempts to turn an answer into unauthorized code, a database operation or a connected action. Human approval does not replace application security controls.
Restrict who and what can change source documents, retrieval indexes, training data and model components. Record provenance and versions; review changes for unauthorized or malicious content. Preserve each user's access limits during retrieval. Test removal of a compromised source and restoration of an approved version, including indexes and caches. Verify removal from retrieval, and reassess affected models and downstream uses before restoring service.
Record the control, responsible owner, applicable institutional standard, test or supplier evidence, reviewer and result. Where a required control cannot be demonstrated, resolve the finding or restrict the use before release.
Illustrative campus example
A document in an advising knowledge base instructs an assistant to export student information to an external address. The test succeeds only if the system preserves access and action limits and the incident can be detected and contained. A polite refusal in one demonstration is insufficient evidence.
Use the editable companion to capture the decision in your institution’s own systems. These prompts are also available in the printable guide.
List identities, information sources, recipients, tools and actions permitted.
Record the institutional baseline, authentication, access removal, encryption, credentials, keys, remediation, logging and recovery controls, with owners.
Link evidence for access boundaries, safe outputs, controlled knowledge-source changes and recovery. Record failures, reviewer and corrections.
Identify mandatory blockers, the reviewer, alert recipients and the authority to disconnect.
Model policy: Section 5 · Section 9 · Section 11.
HumanSkills implementation recommendations informed by the sources below. These voluntary resources are not legal requirements or a certification. Apply current institutional requirements and specialist review to the actual use.