OSINT Passive Recon: Map a Domain's Public Footprint

OSINT (open-source intelligence) is intelligence drawn from sources open to the public: websites, DNS records, certificate logs, internet archives, social media, and public documents. The point isn't secret data but the ability to combine public pieces into a complete picture that can be verified.

For developers, OSINT has two uses. Attackers use it as the first step (reconnaissance) before touching a target, so we need to know what they see. And in a pentest or bug bounty, OSINT is how you map the attack surface without sending a single packet to the target's servers. This post is the companion to OPSEC for developers: that one closes leaks, this one reads what has already leaked.

Passive and active

Recon comes in two kinds:

Kind	Examples	Touches the target's servers?
Passive	CT logs, public DNS, archives, search engines, device search engine data	no
Active	port scans, directory brute forcing, fuzzing	yes

Active recon, such as a port scan with Nmap (covered in the Auto Nmap Scanner project), sends traffic straight to the target. This post covers passive recon only. Every example below uses your-domain.com: practice these techniques on your own domain.

Subdomains from Certificate Transparency logs

Every TLS certificate from a public CA is recorded in Certificate Transparency logs. crt.sh serves the results as JSON, so a list of subdomains takes a single command:

Subdomains from crt.sh

curl -s 'https://crt.sh/?q=%25.your-domain.com&output=json' \
  | jq -r '.[].name_value' | tr 'A-Z' 'a-z' | sort -u

The result often surfaces names that are never linked from anywhere: staging., vpn., jira., or internal project names. One caveat: certificates stay in the logs long after a subdomain is gone, so this list is a set of candidates, not of live hosts.

DNS and domain registration data

Public DNS records say a lot about infrastructure:

The most informative DNS records

dig +short NS your-domain.com     # DNS provider
dig +short MX your-domain.com     # email provider
dig +short TXT your-domain.com    # SPF, service verification
dig +short TXT _dmarc.your-domain.com

TXT records are the most interesting. SPF (v=spf1 include:…) lists the services allowed to send email on the domain's behalf, such as the email provider, a CRM, or a newsletter service. Verification tokens like google-site-verification= or MS= reveal services the organization uses.

Registration data now comes from RDAP, the successor to WHOIS, which returns JSON. rdap.org forwards the request to the right registry:

Registration dates and registrar via RDAP

curl -sL https://rdap.org/domain/your-domain.com \
  | jq '{events: [.events[] | {(.eventAction): .eventDate}],
         registrar: [.entities[] | select(.roles | index("registrar")) | .vcardArray[1][] | select(.[0] == "fn") | .[3]]}'

Much of the owner data is now redacted for privacy, but registration and expiry dates and the registrar remain visible. The RDAP JSON structure can vary a little between registries, so if the jq filter above fails, look at the raw output first.

Archives: pages that were deleted

The Wayback Machine keeps copies of pages over time, and its CDX API lists every URL it has archived:

Unique URLs ever archived

curl -s 'https://web.archive.org/cdx/search/cdx?url=your-domain.com/*&fl=original&collapse=urlkey' \
  | sort -u | head -50

The list often includes old API endpoints, configuration files, PDF documents, or admin panels that are no longer linked but may still be reachable. For site owners, it's a reminder that deleting a page doesn't delete its archive.

Search engines and device search engines

Search operators narrow results down to what matters:

Example search operators

site:your-domain.com -www               pages on subdomains other than www
site:your-domain.com filetype:pdf       indexed documents
site:your-domain.com inurl:admin        URLs containing "admin"
"your-domain.com" site:github.com       public code that mentions the domain

GitHub deserves special attention: configuration files, code samples, or logs that mention an organization's domain sometimes carry internal endpoints along, or even credentials.

Device search engines like Shodan scan the internet regularly and store the service banners they find. Searching their index sends nothing to the target:

Shodan filters

hostname:your-domain.com
ssl.cert.subject.cn:your-domain.com

The results show open ports, software versions, and certificates as Shodan saw them on its last scan. That data can be stale, so treat it as a lead, not as current fact.

Verification: the heart of OSINT

Collecting data is easy; the hard part is making sure it's true. The investigative group Bellingcat, for example, traced the Buk missile launcher linked to the downing of flight MH17 in 2014 by matching social media photos and videos against satellite imagery and buildings on the ground, one by one. The same principles apply to technical recon:

Cross-check sources. A subdomain from crt.sh only matters if it also has a live DNS record.
Mind the time. Archives, Shodan data, and certificate logs are all snapshots of the past.
Write everything down. Record the source, when you retrieved it, and the command you ran, so findings can be repeated and verified by others.

A findings log entry

[2026-10-29 14:05 UTC+7] crt.sh: staging.your-domain.com (certificate from 2024-03)
  dig A staging.your-domain.com -> no answer (subdomain not live)

From the owner's side

Run every step above against your own domain, then ask of each finding: does this need to be public? A staging subdomain can be put behind authentication, old verification tokens can be removed from DNS, and documents that don't need to be indexed can be taken down. How to close leaks like these is covered in more depth in OPSEC for developers.

If you find anything sensitive about this site with the techniques above, please report it via security.txt.