AI Image Detector Benchmark
A Comparative Evaluation of Consumer AI Image Detection Tools
- Published:
- November 2026
- Conducted by:
- aimagedetector.ai
- Status:
- Initial evaluation, ongoing
Abstract
This report evaluates aimagedetector.ai against three publicly available AI-generated image detection tools — Sightengine, AI or Not, and isitai.com — on a fixed sample of 11 images (5 AI-generated, 6 real and unedited). A fourth tool, Illuminarty, was unable to process any image in the sample due to file-size constraints and is reported separately. aimagedetector.ai classified all 11 images correctly. Results, per-image data, and the complete testing methodology are reported below for independent verification.
1. Summary of Findings
11 / 11
aimagedetector.ai
11 / 11
Sightengine
9 / 11
AI or Not
7 / 7
isitai.com (4 untestable)
Scores reflect correct AI-vs-real classifications only; see §4 for per-tool detail, including generator-attribution accuracy and upload-limit constraints that prevented full-sample testing for two tools.
2. Dataset
The sample consists of 11 images: 5 AI-generated (primarily Midjourney output) and 6 unedited photographs sourced from Unsplash contributors. This is an initial, limited-sample evaluation rather than a comprehensive benchmark; sample size and generator diversity are planned to expand in subsequent revisions of this report.
AIAI-generated
98.7%
AIAI-generated
99.9%
AIAI-generated
91.5%
AIAI-generated
99.98%
AIAI-generated
99.98%
RealReal photo
99.97% not AI
RealReal photo
99.97% not AI
RealReal photo
99.97% not AI
RealReal photo
98.8% not AI
RealReal photo
99.9% not AI
RealReal photo
99.8% not AI
Verdicts and scores shown above were produced by aimagedetector.ai's production detection pipeline, invoked directly rather than through the web interface — equivalent to the result a live scan on the public site would return.
3. Methodology
Each competing tool was evaluated manually through its own publicly available, free web interface, with no account-gated features enabled. No tool was given preferential treatment in file preparation; the same unmodified image files were submitted to every tool in the sample.
aimagedetector.ai was evaluated by invoking its production detection pipeline directly on each file, bypassing the web UI but using the identical classification logic a live scan performs.
This methodology prioritizes real-world applicability over laboratory conditions: images were used at their original, unedited file sizes rather than pre-compressed to fit any individual tool's constraints, since a detector that cannot process ordinary photography has limited practical utility.
4. Comparative Results
Findings are reported as observed, including instances where a competing tool performed well.
Sightengine
11 / 11 correctTested on 11 / 11 images
- •Every real photograph scored 0–3% likelihood of AI generation; every AI-generated image scored 97–99%.
- •No ambiguous or incorrect classifications were observed across the full sample.
AI or Not
Mixed — see findingsTested on 11 / 11 images
- •Classified one unedited real photograph as "Likely Deepfake" (66%), despite no face manipulation being present.
- •Classified one AI-generated image as "Likely Real" (99% human) — a false negative.
- •Generator attribution (the specific model named for each AI-generated image) did not match the true source model in any instance where the AI/real classification itself was correct.
isitai.com
7 / 7 correct (4 untestable)Tested on 7 / 11 images
- •Correctly classified both testable real photographs as "Likely human-made."
- •Correctly classified all 5 AI-generated images as "Likely AI-generated."
Constraint encountered: A 5MB upload limit rejected 4 of the 6 real photographs in the sample; unedited camera and phone photography routinely exceeds this threshold. The unregistered free tier is additionally capped at 3 checks per session.
Illuminarty
Not testableTested on 0 / 11 images
Constraint encountered: A 3MB upload limit rejected every real photograph in the sample (3.6–8.5MB each), making the tool unusable for unedited, real-world photography of any kind.
5. Exclusions & Limitations
Illuminarty rejected every real photograph in the sample due to a 3MB upload limit, and is reported in §4 as untestable rather than scored.
Sample size. This evaluation is limited to 11 images and should be read as an initial, reproducible spot-check rather than a statistically comprehensive benchmark. Findings may not generalize to all generator types, compression levels, or editing techniques.
Temporal validity. Results reflect each vendor's publicly available tool as of November 2026 and are subject to change as vendors update their underlying models.
6. Conclusion
Within the scope of this sample, aimagedetector.ai correctly classified all 11 images. Sightengine achieved an equivalent result on the same sample. AI or Not produced both a false positive and a false negative, and isitai.com was unable to process the majority of real photographs due to file-size restrictions. This report will be revised as the evaluation sample is expanded.