Dedicated site parsers
How Search1API returns cleaner, more structured output for selected mainstream sites, and which URL patterns are verified for each.
POST /crawl works on any public URL through a generic
extraction chain. For a small set of mainstream sites, Search1API also maintains
dedicated handling that returns cleaner, more structured output than generic
extraction — article-like Markdown plus site-specific metadata.
How dedicated handling works
Dedicated handling takes two forms:
- Dedicated HTML parsers read the rendered page structure directly and map site-specific elements (author, publish date, body, counts) into a consistent result shape.
- Dedicated feed or API paths read a site's own structured data source instead of the HTML page — for example, Reddit URLs are served through Reddit's public Atom feeds, which return post bodies and comment threads without an account or OAuth app.
When a dedicated path cannot serve a request, enableFallback lets the request
fall back to browser-rendered extraction. The response still succeeds, but its
content may include surrounding page chrome rather than the cleaner dedicated
output.
Support is verified per URL pattern
Coverage is measured by URL shape, not by domain. A landing page for a site lists only the URL patterns verified with real requests — a site may support posts and listings while other sections remain unsupported or login-walled. Pages that require authentication return an honest error instead of fabricated content.
Sites with dedicated handling
| Site | Mechanism | Verified URL patterns | Details |
|---|---|---|---|
| Dedicated HTML parser | Company pages, job search and listings, posts, Pulse articles, public profiles | LinkedIn Scraper API | |
| Public Atom feeds | Posts with comment threads, subreddit listings including sorted views, user pages | Reddit Scraper API | |
| X (Twitter) | Public embed/syndication JSON | Tweets, quote tweets, and long-form Articles | X Scraper API |
Each linked page documents the verified URL patterns, a real request example, and real response output for that site.