Full Report
In the cloud, logs are often the only way to get real-time visibility into what's happening, making them critical to any cloud detection and response program.
Analysis Summary
# Best Practices: Cloud Logging for Detection and Response
## Overview
These practices address the critical need for robust cloud logging to support Security Operations Centers (SOCs) in threat detection and response. The focus is on categorizing, prioritizing, and correctly configuring log collection across cloud control planes, identity, data, network, and compute resources to maximize security coverage against modern cloud attacks.
## Key Recommendations
### Immediate Actions
1. **Prioritize Control Plane Logs:** Immediately ensure that management/control plane logs are enabled and collected across all cloud accounts. These logs offer the broadest visibility across the cyber kill chain (initial access to impact).
2. **Verify Default Retention:** Identify the default log retention settings for critical logs (e.g., AWS CloudTrail, Azure Activity Logs, GCP Audit Logs) and immediately implement mechanisms to extend retention past default periods where inadequate (e.g., Azure default of 30 days).
3. **Establish Centralized Ingestion:** Begin the process of streaming or shipping logs from all managed cloud environments into a single, centralized solution for unified security monitoring and analysis.
### Short-term Improvements (1-3 months)
1. **Implement Organizational Trails (AWS):** Configure AWS **Organization Trails** to automatically create and manage CloudTrail across all existing and newly provisioned accounts, eliminating gaps in coverage.
2. **Integrate Workspace/SaaS Logs (GCP/Other):** Stream Google Workspace audit logs (including Admin, Single Sign-On, Suspicious Activity) into the main GCP logging stream for unified consumption.
3. **Enable Error Log Monitoring:** Specifically review configurations to ensure that logs which omit error reports (e.g., some Google Workspace non-Login categories) are supplemented or adjusted, especially for identity resource modifications.
### Long-term Strategy (3+ months)
1. **Map Logs to MITRE ATT&CK:** Utilize the established log categories (Identity, Data, Network, Compute, Control Plane) and map collected logs to the MITRE ATT&CK framework to assess and systematically close coverage gaps against the organizational threat model.
2. **Implement Continuous Validation:** Deploy solutions or configure automated checks to continuously evaluate cloud logging coverage, ensuring new resources and services are immediately included in the logging scope.
3. **Integrate Detection Layers:** Plan for the integration of log-based detection mechanisms with runtime detectors to create a comprehensive, multi-layered threat detection and response capability.
4. **Optimize Cost vs. Value:** Systematically review log collection configurations, leveraging free tiers where possible (e.g., AWS free copy of management events) and ensuring that high-cost logs provide demonstrable, critical security value.
## Implementation Guidance
### For Small Organizations
- **Leverage Free Tiers:** Utilize the free storage options provided by cloud vendors for initial management event logs (e.g., the first AWS CloudTrail copy to S3).
- **Focus on Control Plane:** For budget constraints, focus the initial, comprehensive logging effort *only* on the Control Plane/Management activity logs, as these provide the highest return on security visibility for the least operational overhead.
- **Centralize via Standard Tools:** Use basic native features (like Azure Log Analytics or standard S3 replication) for initial centralization rather than complex third-party solutions.
### For Medium Organizations
- **Establish Organizational Trails/Policies:** Enforce logging centrally using organizational units or policy definitions to automate log deployment on new accounts/subscriptions/projects.
- **Implement Cross-Cloud Aggregation:** Select and deploy a dedicated Security Information and Event Management (SIEM) or log management platform capable of ingesting and normalizing logs from multiple cloud providers (AWS, Azure, GCP).
### For Large Enterprises
- **Formalize the Framework:** Officially adopt the categorization framework (Identity, Data, Network, Compute, Control Plane) as the governance standard for all future cloud compliance and log procurement requirements.
- **Implement Dedicated Log Teams:** Assign specific engineering resources to maintenance, cost optimization, and regular alignment checks between logging configurations and evolving threat intelligence.
- **Automate Coverage Testing:** Integrate continuous security posture management (CSPM) tools that can automatically check for disabled logging on newly deployed resources or services within defined organizational boundaries.
## Configuration Examples
| Cloud Provider | Log Type Priority | Configuration Best Practice |
| :--- | :--- | :--- |
| **AWS** | Management Events | Use **Organization Trail** to enforce logging across all member accounts automatically. |
| **AWS** | Cost Optimization | Leverage the free tier by delivering one copy of ongoing management events to a designated S3 bucket via the initial trail setup. |
| **Azure** | Activity Logs | Configure **Diagnostic Settings** to ensure Subscription Activity Logs are sent to a Log Analytics Workspace or Storage Account to extend default 90-day retention. |
| **Azure** | Identity | Ensure **Entra ID Sign-in and Audit Logs** are configured for collection, as they contain crucial location data for compromise detection. |
| **GCP** | Workspace Integration | Configure **Streaming** of Google Workspace logs directly into the central GCP Audit Logs sink for free, unified consumption. |
## Compliance Alignment
The recommended logging enhancements directly support adherence to several major security frameworks focused on visibility and monitoring:
* **NIST CSF:** Essential for the **Detect** function (e.g., continuous monitoring) and **Respond** function (e.g., analysis and investigation).
* **ISO 27001/27017:** Supports Annex A controls related to monitoring, logging, and change management via control plane visibility.
* **CIS Benchmarks:** Directly addresses configuration hardening required for effective logging and auditing across major cloud providers.
## Common Pitfalls to Avoid
- **Assuming Default Logs Suffice:** Relying only on default log retention periods (often too short) or default log configurations (often missing critical data events).
- **Inconsistent Cross-Account Coverage:** Failing to implement organizational-level mechanisms (like AWS Organization Trails), leading to visibility gaps when new accounts are spun up.
- **Ignoring Non-Identity Errors:** Overlooking log sources that specifically omit error reporting unless specifically configured (e.g., non-Login category logs in Google Workspace).
- **Data Silos:** Allowing logs from different cloud providers or SaaS applications to remain siloed, preventing holistic security investigations.
## Resources
- **Cloud Provider Documentation:** Refer to specific documentation for enabling Organization Trails (AWS), Diagnostic Settings (Azure), and Configuring Sinks (GCP).
- **MITRE ATT&CK Framework:** Use for mapping log coverage against adversary techniques.
- **Cyber Kill Chain:** Use as a conceptual model for prioritizing logs that cover Initial Access and Discovery phases (Control Plane logs excel here).