Full Report
A flaw in the Google Cloud Vertex AI SDK for Python let an attacker with no access to a victim's project hijack the victim's machine learning model upload and run code inside Google's serving infrastructure. Palo Alto Networks Unit 42, which found and reported the bug through Google's bug bounty program, calls the technique "Pickle in the Middle" and said it saw no exploitation in the wild.
Analysis Summary
# Vulnerability: "Pickle in the Middle" RCE in Google Cloud Vertex AI SDK
## CVE Details
- **CVE ID:** CVE-2024-37310
- **CVSS Score:** 8.5 (High)
- **CWE:** CWE-502 (Deserialization of Untrusted Data)
## Affected Systems
- **Products:** Google Cloud Vertex AI SDK for Python (`google-cloud-aiplatform`)
- **Versions:** All versions prior to v1.53.0
- **Configurations:** Systems utilizing the SDK to upload or deploy machine learning models using formats that rely on Python's `pickle` module (e.g., scikit-learn, XGBoost, or PyTorch).
## Vulnerability Description
The vulnerability stems from the way the Vertex AI SDK handled the packaging and uploading of machine learning models. When a user initiated a model upload, the SDK would locally serialize the model—often using the inherently insecure `pickle` format—and then transfer it to a Google-managed Cloud Storage bucket.
The flaw allowed an attacker to intercept the upload process (Man-in-the-Middle style within the cloud infrastructure context) and replace the legitimate model file with a malicious pickle payload. Because the Vertex AI backend infrastructure would subsequently deserialize this file to serve the model, the attacker could achieve Remote Code Execution (RCE) within Google’s internal serving infrastructure.
## Exploitation
- **Status:** PoC available (developed by Unit 42 researchers); No evidence of exploitation in the wild.
- **Complexity:** High (Requires intercepting/manipulating cloud-level model upload workflows).
- **Attack Vector:** Network / Cloud Infrastructure.
## Impact
- **Confidentiality:** High (Potential access to Google's internal serving environment).
- **Integrity:** High (Ability to modify models and execute arbitrary code).
- **Availability:** High (Potential to disrupt model serving services).
## Remediation
### Patches
- **Google Cloud AI Platform (Python):** Upgrade to version **1.53.0** or higher. This version implements more secure model handling and addresses the insecure serialization path.
### Workarounds
- **Manual Signing:** Implement manual integrity checks (checksums/hashes) for model artifacts before upload.
- **Service Account Hardening:** Ensure the Principle of Least Privilege is applied to the service accounts running the SDK to limit the scope of a potential compromise.
## Detection
- **Indicators of Compromise:** Unusual outbound network connections from model-serving containers; unexpected file system modifications within the Vertex AI environment.
- **Detection Methods:**
- Use static analysis tools to scan Python code for insecure usage of `pickle.load()`.
- Monitor Cloud Logging for unauthorized or anomalous `storage.objects.create` events in staging buckets used by Vertex AI.
## References
- **Unit 42 Research:** hxxps[://]unit42[.]paloaltonetworks[.]com/pickle-in-the-middle-google-cloud-vertex-ai/
- **GitHub Repository:** hxxps[://]github[.]com/googleapis/python-aiplatform/releases/tag/v1.53.0
- **Google Security Advisory:** hxxps[://]cloud[.]google[.]com/security/bulletins/physics-cloud-ai-sdk-vulnerability