Full Report
This guide is part of a collaboration between Bellingcat and Evident on detecting AI-generated products. You can watch Evident’s video here. Sipping coffee from a mug carved from mineral rock, its surface glimmering with amethyst, rose quartz and other crystals, sounds almost too magical to be real. And unfortunately, as some shoppers discovered, it was. […] The post Don’t Get Scammed! Tips For Spotting AI-Generated Fake Products Online appeared first on bellingcat.
Analysis Summary
# Best Practices: Detecting AI-Generated Product Listings and Content
## Overview
These practices focus on helping consumers, investigators, and platform administrators identify fraudulent or misleading product listings and content that have been created or heavily augmented using Artificial Intelligence (AI) image and text generation tools. The goal is to reduce successful scams involving unrealistic or non-existent products.
## Key Recommendations
### Immediate Actions
1. **Scrutinize Image Quality for Digital Defects (The "Sheen"):** Immediately check product images for a noticeable "sheen" or unnatural smoothness characteristic of AI generation.
2. **Inspect for Inconsistent or Broken Lines:** Look closely at product edges, surfaces, and features for broken, inconsistent, or non-aligning lines (e.g., on crystal structures or complex patterns).
3. **Identify Digital Smudges/Fading:** search for areas in the image that appear blurry, smeared, or inexplicably fade out of existence, which suggests digital painting artifacts rather than physical material flaws.
4. **Verify Practical Realism:** Question how the product would function physically (e.g., glowing elements requiring unseen batteries or power sources) if the listing omits necessary operational details.
### Short-term Improvements (1-3 Months)
1. **Mandate Multiple, Consistent Angles:** Flag listings that only feature a single photograph of the item. Authentic listings should provide several angles of the same physical object.
2. **Test Image Consistency Across Angles:** If multiple photos exist, verify that the product details (color, shape, material) are identical across all views. Inconsistencies suggest an AI generating variations of a concept rather than documenting one item.
3. **Conduct Reverse Image Searches on Profile Pictures:** Systematically perform reverse image searches on seller profile pictures. If results point primarily toward stock image sites or known AI image generation platforms (like Midjourney examples), treat the seller as highly suspect.
4. **Investigate Seller Identity Uniqueness:** For digital products like e-books or patterns, search for the author/seller's name, biography, and associated sites. A complete lack of external presence, social media, or verifiable publications is a strong red flag.
### Long-term Strategy (3+ Months)
1. **Develop Internal AI Detection Checklists:** Formalize these visual inspection techniques into a standardized quality assurance checklist for vetting new product uploads, especially from unverified third-party sellers.
2. **Implement Automated Metadata Analysis:** Develop or procure tools capable of scanning image metadata for anomalies or generative tags, though this requires sophisticated tooling.
3. **Establish Authoritative Verification Process:** For high-risk categories (e.g., artisan goods, personalized items), require sellers to submit **authenticity proof**, such as a time-stamped video of the actual item being handled or manufactured.
4. **Platform-Level Disclosures (If Applicable):** Advocate for or implement requirements where sellers must affirmatively disclose content generated by AI, or where AI-generated content is flagged automatically.
## Implementation Guidance
### For Small Organizations
- **Focus on Manual Vetting:** Implement the "Immediate Actions" checklist rigorously for all new listings, focusing heavily on visual inspection and simple reverse image searches on high-value or suspiciously perfect items.
- **Limit Seller Profiles:** Start with a small, vetted pool of trusted vendors until automated auditing tools become feasible.
### For Medium Organizations
- **Create Standard Operating Procedures (SOPs):** Document the short-term improvements into formal SOPs for your content moderation team.
- **Utilize Basic Reverse Search Tools:** Integrate accessible reverse image search functionality into the standard review dashboard.
- **Cross-Reference Seller Data:** Begin automated lookups against known stock image databases or lists of AI-generated portraits for seller profile verification.
### For Large Enterprises
- **Deploy Advanced Image Forensics:** Invest in specialized software that analyzes subtle pixel artifacts, noise patterns, and frequency domain statistics indicative of GAN/Diffusion models.
- **Automated Inconsistency Detection:** Implement machine learning pipelines to compare images within a single listing set for geometric and textural inconsistencies (Variation Consistency Check).
- **Integrate Identity Verification:** Establish robust Know Your Customer (KYC) processes, especially for sellers of digital goods, linking profiles to verifiable real-world identities where appropriate to prevent the use of fake author personas.
## Configuration Examples
*Since the context describes investigative techniques rather than specific software configurations, this section focuses on how to configure a *search utility* for verification:*
**Reverse Image Search Configuration (Conceptual):**
If using an API or web utility for reverse searching seller profile photos, configure the query to prioritize results from:
1. Known AI image generation platforms (e.g., Midjourney result URLs).
2. Stock photo aggregation sites flagged for containing synthetic imagery.
3. **Avoid:** Setting the tolerance too high initially, as legitimate sellers using stock photos might be erroneously flagged.
## Compliance Alignment
While this topic directly addresses fraud prevention rather than traditional IT compliance, the underlying principles align with:
- **NIST SP 800-218 (Secure Software Development Framework - SSDf):** Promoting integrity and authenticity throughout the supply chain.
- **Consumer Protection Laws:** Ensuring transparency regarding the nature and origin of goods being sold.
- **ISO/IEC 27001 (Information Security Management):** Protecting customer information and transactions against fraudulent representation.
## Common Pitfalls to Avoid
- **Assuming High Resolution Means Authenticity:** AI images can be rendered in very high resolution; focus on internal image coherence, not just pixel density.
- **Ignoring Contextual Red Flags:** Don't dismiss poor seller identification just because the product image itself looks visually appealing—the two elements of fraud development (the fake visual and the fake identity) should both be checked.
- **Over-relying on Single Evidence:** Do not reject a listing based on one minor digital smudge alone; look for a **pattern** of multiple red flags (e.g., poor lensing *and* inconsistent angles *and* fake author profile).
## Resources
- **Bellingcat Investigative Techniques:** Guides on critical thinking and basic OSINT investigation tools used for source verification. (Focus on methodology.)
- **AI Image Detection Tooling:** Explore tools that analyze image noise and artifacts specific to common generative models (search for academic papers on "GAN fingerprinting").
- **Evident Media Resources:** Utilize video guides related to visual analysis of synthetic media.