brave-search

CLI for extracting URLs from Brave Search API results

brave-search is a modern, Linux-first command-line tool maintained by a member of Haltman.io. It queries the Brave Search API and extracts URLs only, producing clean, pipeline-friendly output for cybersecurity workflows.

The tool is designed for OSINT and automation scenarios where analysts need reliable URL discovery with:

  • Explicit pagination control

  • Rate limiting and retries

  • Proxy and TLS controls (HTTP / SOCKS5)

  • Deterministic, deduplicated output

Typical use cases:

  • OSINT URL collection

  • Recon pipelines (seed URLs for crawlers/scanners)

  • Brand and domain footprint discovery

  • Automated search-driven enrichment

Help menu

Installation

Requirements

  • Go 1.22+

  • Valid Brave Search API key

Build from source (recommended)

git clone https://github.com/haltman-io/brave-search.git
cd brave-search
go build -o brave-search ./cmd/brave-search

Run:

./brave-search -h

Install into $GOBIN

go install github.com/haltman-io/brave-search/cmd/brave-search@latest

Verify:

brave-search -h

If the binary is not found, ensure $GOBIN (or $(go env GOPATH)/bin) is in your PATH.

Configuration

API key configuration (required)

An API key is required to query the Brave Search API.

Option 1 — CLI flag (highest priority)

brave-search -ak "YOUR_API_KEY" -sq "site:example.com"

Option 2 — YAML config file (recommended)

On first run, the tool ensures a config file exists next to the executable:

.brave-search.yaml

Initial structure:

api_keys: []

Add one or more keys:

api_keys:
  - "KEY_1"
  - "KEY_2"

Behavior:

  • If multiple keys exist, the tool rotates keys per request

  • If no key is available, execution aborts with an error

Usage

brave-search -sq "site:thc.org" --debug --all

This performs a Brave web search with debug and auto-scroll.

Debug run

Multiple queries

Comma-separated:

brave-search -sq "site:example.com,site:example.org"

Repeatable flags:

brave-search -sq "site:example.com" -sq "site:example.org"

From file:

brave-search -sqf queries.txt

From stdin (explicit):

cat queries.txt | brave-search --stdin

Pagination

Manual page control:

brave-search -sq "site:example.com" --count 20 --page 0

Auto-pagination (recommended):

brave-search -sq "site:example.com" --all

Pagination run

Output to file (sorted & deduplicated)

brave-search -sq "site:example.com" --all -o urls.txt

Behavior:

  • Results are sorted and deduplicated

  • One URL per line

  • File is overwritten if it exists

Proxy and TLS usage

HTTP proxy + TLS bypass (common with Burp/ZAP):

brave-search -sq "site:example.com" --proxy http://127.0.0.1:8080 -k

SOCKS5 proxy:

brave-search -sq "site:example.com" --proxy socks5://127.0.0.1:9050

Proxy run

Possible Problems / Important Notes

Rate limits

  • Default tool limit: 5 requests/second

  • Brave API tiers may enforce stricter limits (e.g., 1 rps)

If you see 429 RATE_LIMITED:

brave-search -sq "site:example.com" --rate-limit 1

You can also increase retry wait time:

brave-search -sq "site:example.com" --retry-wait-time 5s

Retries and failures

  • Default retries: 3

  • If retries are exhausted, the tool exits with a fatal error

This is intentional to avoid silent data loss.

STDIN behavior

  • Reading from stdin is disabled by default

  • You must explicitly enable it with --stdin

This prevents accidental blocking or unintended pipeline reads.

TLS bypass warning

brave-search -k

Disables TLS verification. Use only in controlled lab environments where MITM visibility is expected.

External References

Updated on
sub-alter
Get started