Finding passports in public buckets with computer vision

What it does Octopii is a PII scanner that ingests images, PDFs, and text files from local disks, Amazon S3 buckets, or Apache open directory listings. It uses Tesseract OCR and a battery of image-cleaning filters to extract text from documents, then classifies the results against a keyword database to identify government IDs, contact details, and addresses. The tool also counts faces in images using an OpenCV Haar cascade and reports the tally alongside any extracted identifiers.

The interesting bit The heavy lifting isn’t just the OCR; it’s the preprocessing pipeline. Before Tesseract touches a file, Octopii auto-rotates, grayscales, thresholds, and deskews the image up to three times to improve OCR accuracy on rotated or skewed documents. Because these steps strip color data, face detection runs on the original image first—a sensible ordering that keeps the face-counting honest while maximizing text extraction.

Key highlights

• Scans beyond the local filesystem into S3 XML listings and Apache open directories
• Classifies documents via Gestalt pattern matching against a definitions.json keyword list, not just blind regex
• Extracts geolocation data and contact info using NLP and regex on top of OCR output
• Reports face counts per document alongside extracted identifiers, emails, and phone numbers
• Appends findings sequentially to a plain-text output file in real time

Caveats

• It depends on external system-level OCR and NLP tools, so it is not a standalone binary
• Image support is limited to JPG and PNG; PDFs are converted to images before scanning
• The authors explicitly restrict usage to research and education, disclaiming liability for malicious use

Verdict Security researchers auditing cloud storage and open web directories will find Octopii a useful automated first pass, but developers looking for a drop-in SaaS API or a general-purpose document parser should look elsewhere.