theHarvester OSINT Tutorial: Passive Recon from the Command Line

Photo by Anete Lusina on Pexels

Before a penetration tester touches a target system, they spend time building a picture of what’s exposed — what email addresses are out there, what subdomains exist, what IP ranges the organization uses, who the employees are. This phase, called passive reconnaissance, produces intelligence that shapes everything that comes after: phishing simulations, attack surface mapping, initial access vectors.

theHarvester is one of the standard tools for this phase. It’s been around since 2009, it’s maintained by Christian Martorella (laramies on GitHub), and it ships by default in Kali Linux and ParrotOS. The core value is simple: aggregate data from multiple public sources, normalized into a single report, without generating a single packet toward the target’s infrastructure.

What is theHarvester?

The distinction matters both legally and operationally. Passive recon is generally considered low-risk and is standard in external penetration tests from day one. Active scanning typically requires explicit scope authorization. theHarvester sits firmly in the passive column for most of its sources.

Installation

Kali Linux / ParrotOS (already installed)

$ apt install

sudo apt update && sudo apt install theharvester -y
theHarvester --version

From GitHub (latest version)

$ git clone

git clone https://github.com/laramies/theHarvester
cd theHarvester
pip3 install -r requirements/base.txt
python3 theHarvester.py --help

Basic Usage and Flags

The minimum viable theHarvester command needs two things: a target domain (-d) and at least one data source (-b).

$ minimum command

theHarvester -d example.com -b google -l 300

Full flag reference:

Flag	Value	Purpose
-d	domain.com	Target domain (required)
-b	google, bing, all…	Data source(s) — comma-separated for multiple
-l	integer	Max results per source (default 500)
-f	filename	Save output to file (.json, .html, .xml auto-generated)
-n	—	DNS lookup on discovered hosts (resolve IPs)
-c	—	DNS brute force on subdomains
-e	DNS IP	Use a specific DNS server for lookups
-S	integer	Start at result number (for pagination)
-v	—	Verify discovered hosts via DNS lookup
-r	—	Perform reverse DNS lookup on discovered IPs

$ common invocations

# Multiple free sources
theHarvester -d example.com -b google,bing,yahoo,netcraft,dnsdumpster -l 300

# With output file (creates example_recon.html and example_recon.json)
theHarvester -d example.com -b google,bing -l 500 -f example_recon

# With DNS resolution of discovered hosts
theHarvester -d example.com -b google -l 500 -n

# All sources (requires API keys for many; slow)
theHarvester -d example.com -b all -l 50

Data Sources Breakdown

Not all sources are equal. Some return emails; others return subdomains; some require API keys; some hit rate limits quickly. Understanding what each source gives you informs which to prioritize for a given objective.

Source	What it returns	Requires API?	Notes
google	Subdomains, emails, URLs	Free	Most useful for subdomain discovery via dorks; rate limits quickly
bing	Subdomains, emails, URLs	Free	Complementary to Google; often returns different results
baidu	Subdomains, emails	Free	Useful for orgs with presence in China; different index
yahoo	Subdomains, emails	Free	Declining index but still returns unique results occasionally
dnsdumpster	Subdomains, DNS records, MX	Free	Excellent for DNS enumeration; often finds subdomain IPs directly
netcraft	Subdomains, hosting info	Free	Good for discovering hosting providers and CDN configurations
shodan	Open ports, banners, IPs, tech stack	API key	High value — reveals internet-exposed services; free tier available
censys	Certificates, IPs, open ports	API key	Certificate transparency gives excellent subdomain discovery
hunter	Email addresses, employee names	API key	Best source for professional email enumeration; 25 free req/month
github-code	Emails, tokens, secrets in code	API key	Can surface leaked credentials, API keys, internal domains
linkedin	Employee names, titles	Rate limited	Hits LinkedIn’s scraping defenses quickly; use with caution
virustotal	Subdomains, passive DNS	API key	VirusTotal’s passive DNS database is a solid subdomain source

API Keys Setup

Sources that require API keys read credentials from a YAML file that theHarvester looks for at ~/.config/theHarvester/api-keys.yaml (or in the installation directory). The file format is straightforward:

api-keys.yaml

apikeys:
  shodan:
    key: YOUR_SHODAN_API_KEY_HERE
  censys:
    id: YOUR_CENSYS_API_ID_HERE
    secret: YOUR_CENSYS_API_SECRET_HERE
  hunter:
    key: YOUR_HUNTER_API_KEY_HERE
  github:
    key: YOUR_GITHUB_PERSONAL_ACCESS_TOKEN_HERE
  virustotal:
    key: YOUR_VIRUSTOTAL_API_KEY_HERE

Verify a specific source is working with API credentials:

$ test a specific source

theHarvester -d example.com -b shodan -l 50
# Should return hosts, open ports, and banner information rather than an API error

Practical Recon Workflow

The most efficient way to use theHarvester is in stages. Each stage builds on the previous, and the output feeds downstream tools.

$ stage 1 — free sources

theHarvester -d target.com -b google,bing,yahoo,baidu,netcraft,dnsdumpster -l 300 -f stage1_results

$ stage 2 — authenticated sources

theHarvester -d target.com -b shodan,censys,virustotal,hunter -l 100 -f stage2_results

$ stage 3 — resolve and feed

# DNS resolve discovered hosts
theHarvester -d target.com -b google,bing -l 500 -n -f final_with_dns

# Extract unique subdomains for gobuster/ffuf
cat final_with_dns.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
for host in data.get('hosts', []):
    print(host.split(':')[0])
" | sort -u > subdomains.txt

# Feed IPs to nmap for port scan
nmap -iL ips.txt -sV -T4 -p 80,443,8080,8443,22,21,25,445

Reading and Using the Output

When saved with -f filename, theHarvester generates both an HTML report and a JSON file. The JSON is the more useful format for piping to downstream tools.

$ JSON output structure

{
  "emails": [
    "[email protected]",
    "[email protected]"
  ],
  "hosts": [
    "api.target.com: 203.0.113.45",
    "dev.target.com: 198.51.100.5"
  ],
  "ips": [
    "203.0.113.0/24"
  ],
  "interesting_urls": []
}

Emails feed into: phishing simulation target lists, username discovery (first.last@domain suggests Active Directory naming convention), validation via Hunter.io, credential stuffing reference (not for unauthorized use — for checking exposure in breach databases).

Subdomains/Hosts feed into: nmap port scans, gobuster/ffuf virtual-host enumeration, subdomain brute force expansions (using tools like subfinder, amass), Shodan lookups per IP.

IPs feed into: masscan for rapid port discovery across the discovered range, nmap service detection on specific hosts.

Defensive Context

Everything theHarvester collects is already public. The question for defenders is: what does that public picture look like, and are there things in it that shouldn’t be there?

Email enumeration: Exposed email addresses accelerate social engineering. Knowing that [email protected] exists, and that the organization uses first.last as a naming convention, makes spear-phishing substantially easier to execute. Email obfuscation on public-facing web pages has limited value at this point — search engines have already indexed them. The more durable defense is security awareness training and MFA on email accounts.

Subdomain exposure: dev.company.com and staging.company.com in DNS records or certificate transparency logs represent internal environments that were never intended to be public-facing. Certificate transparency in particular is a permanent record — once a certificate is issued for a subdomain, it appears in CT logs forever. Running theHarvester against your own organization surfaces this exposure before an attacker does.

Shodan/Censys data: If Shodan has indexed services on your IP range, those services have been externally scanned and their banners collected. This isn’t an attack — it’s observation. But the data is used by attackers for targeting. Regularly auditing your Shodan exposure (via the Shodan web interface or API) is a worthwhile practice — filter by your ASN or IP range and review what’s visible.

GitHub code search: Developer email addresses appearing in commit history, API keys hardcoded in repositories, internal domain names referenced in config files — all of this is discoverable through theHarvester’s github-code source. A GitHub secret scanning policy and pre-commit hooks that detect credential patterns address this at the source.

Frequently Asked Questions