AIF-C01 · Question #107
A company is testing the security of a foundation model (FM). During testing, the company wants to get around the safety features and make harmful content. Which security technique is this an…
The correct answer is D. Jailbreak. The scenario describes intentionally bypassing a foundation model's safety features to generate harmful content, which is known as a jailbreak.
Question
A company is testing the security of a foundation model (FM). During testing, the company wants to get around the safety features and make harmful content. Which security technique is this an example of?
Options
- AFuzzing training data to find vulnerabilities
- BDenial of service (DoS)
- CPenetration testing with authorization
- DJailbreak
How the community answered
(30 responses)- A3% (1)
- C3% (1)
- D93% (28)
Why each option
The scenario describes intentionally bypassing a foundation model's safety features to generate harmful content, which is known as a jailbreak.
Fuzzing training data involves feeding invalid or unexpected inputs to find vulnerabilities in the *training process* or model's robustness, not necessarily bypassing safety features during inference to generate harmful content.
Denial of service (DoS) attacks aim to make a service unavailable to legitimate users, which is not the goal described in the scenario of generating harmful content.
Penetration testing with authorization is a broad security testing category, but 'jailbreak' specifically describes the technique used against generative AI models to bypass safety features.
Jailbreaking refers to the specific technique of manipulating a large language model or foundation model to bypass its intended safety mechanisms and produce outputs that it was designed to refuse, often involving harmful or restricted content. This is a common security testing method for adversarial attacks on AI systems.
Concept tested: AI model adversarial attacks, jailbreaking
Topics
Community Discussion
No community discussion yet for this question.