OSINT for market insight: reading a competitor's moves from its public footprint

In OSINT for developers, open sources were used to map a domain's infrastructure. The same techniques can answer business questions: what product a competitor is preparing, when they raised prices, which markets they're targeting, and what their customers complain about. That is OSINT-based market intelligence, or competitive intelligence.

The principle doesn't change: no secret data, only public pieces combined and verified. Every company leaves traces as it operates, such as job postings, pricing-page changes, mandatory filings to regulators, and customer reviews. A single trace rarely means much, but several traces pointing the same way can become an insight.

Start with a question

As with technical recon, market research without a question just produces a pile of screenshots. Frame questions that lead to a clear decision, for example:

• Will competitor A move into the small-business segment in the next six months?
• How has the main players' pricing structure changed over the past year?
• Which features do users of a competitor's app complain about most?

A good question decides which sources you open and when the research is done.

Signals and their sources

Pricing-page history

The pricing page is one of the most honest sources on a product's strategy. The Wayback Machine's CDX API can list every capture of that page, and the collapse=digest parameter drops captures with identical content, leaving only the points where the page changed:

curl -s 'https://web.archive.org/cdx/search/cdx?url=competitor.com/pricing&fl=timestamp,digest&filter=statuscode:200&collapse=digest&from=2025'

Two captures can be compared directly. The id_ suffix after the timestamp fetches the original HTML without the Wayback toolbar:

fetch() {
  curl -s "https://web.archive.org/web/${1}id_/https://competitor.com/pricing" \
    | sed 's/<[^>]*>/ /g' | tr -s ' \n' '\n' | grep -v '^$'
}
diff <(fetch 20250115000000) <(fetch 20251020000000)

Wayback serves the capture closest to the requested timestamp. Pricing pages rendered with JavaScript are often archived as an empty shell, so if the output is blank, open the capture in a browser.

Job postings as a roadmap

Job postings reveal a company's plans months before the product is announced. A "Head of Expansion, Vietnam" posting is more candid than any press release. Many tech companies use an applicant tracking system that publishes their openings as JSON:

# Greenhouse: the board name usually appears in the careers page URL
curl -s 'https://boards-api.greenhouse.io/v1/boards/BOARD-NAME/jobs' \
  | jq -r '.jobs[] | [.title, .location.name, .updated_at] | @tsv'
 
# Lever
curl -s 'https://api.lever.co/v0/postings/COMPANY-NAME?mode=json' \
  | jq -r '.[] | [.text, .categories.location, .categories.team] | @tsv'

Save the results every week. One snapshot says little; the changes say a lot: a data team that suddenly grows, the first opening in a new city, or a new technology in the requirements. For LinkedIn and similar job boards, take notes by hand. Don't run scrapers that break their terms of service.

The same goes for the Certificate Transparency logs covered in the previous post: a paylater. or merchant. subdomain appearing before any official announcement is worth noting, even though it may also be an experiment that never ships.

Financial filings and regulatory documents

Public companies must disclose financial statements and annual reports, usually through their stock exchange's disclosure portal. For companies that file with the SEC in the United States, EDGAR offers a JSON API. The SEC asks every client to send a User-Agent with a name and contact:

curl -s -H 'User-Agent: Your Name contact@your-domain.com' \
  'https://data.sec.gov/submissions/CIK0000320193.json' \
  | jq -r '.filings.recent | [.form, .filingDate, .primaryDocument] | transpose | .[:10][] | @tsv'

The most useful sections are often the most boring: risk factors, business segments, and use of proceeds. That's where the company itself writes down what worries it and where the money is going. For market size and macro data, a national statistics office is a more defensible starting point than a number on a pitch deck.

App reviews and search trends

User reviews are a competitor's free customer research. The App Store has a legacy RSS feed that still returns recent reviews as JSON. It isn't officially documented, so it can change at any time:

curl -s 'https://itunes.apple.com/us/rss/customerreviews/id=APP-ID/sortby=mostrecent/json' \
  | jq -r '.feed.entry[]? | [.["im:rating"].label, .title.label] | @tsv'

Don't read reviews one by one. Group them by theme (price, performance, customer support, specific features), then count which themes grow month to month. A theme that spikes after an app update usually means more than the average rating.

Google Trends fills in the demand side. Compare brand or category names for a region, then export the results to CSV. Keep in mind that the numbers are relative (100 = the peak within the chosen range), not search counts. A low-volume keyword can look like it's surging because of a handful of searches.

From data to insight

Data only becomes insight once it's analyzed. Three habits help:

• Cross-check sources. One "Country Manager Thailand" posting is a hint. Add a pricing page in baht in the archive and a th. subdomain in the certificate logs, and the hint becomes a finding.
• State your confidence. Write whether a conclusion is high, medium, or low confidence, and name the sources behind it. Readers of the report deserve to know what's fact and what's inference.
• Answer "so what". A good insight ends with an implication for a decision, not just a description of what the competitor did.

[2026-11-03] Competitor A is likely entering Thailand (confidence: medium)
  - Greenhouse: "Country Manager, Thailand" opening (updated 2026-10-21)
  - Wayback: /th/pricing first archived 2026-09-30
  - crt.sh: certificate for th.competitor-a.com issued 2026-09-12
  Implication: revisit our localization priorities for Q1.

Ethical and legal limits

Market intelligence is not industrial espionage, and the line has to be clear:

• Public sources only. Don't log in to other people's systems, don't use leaked credentials, and don't buy internal documents.
• No impersonation. Calling a competitor's employees while posing as a prospect or researcher is pretexting, a social engineering technique covered in social engineering research. Don't do it.
• Respect terms of service and server load. Read robots.txt, pause between requests, and use official APIs where they exist.
• Handle personal data carefully. Reviews and employee profiles contain personal data protected by laws such as the GDPR or Indonesia's Personal Data Protection Law (Law No. 27 of 2022). Analyze in aggregate and don't profile individuals.
• Insider information. When the research involves a listed company, material non-public information is no longer OSINT, and trading on it can break securities law.

If you're unsure whether a method is fair, ask whether you'd be comfortable with the competitor knowing how you got the information. If the answer is no, don't do it.