Blocked pages and read options
Load public pages, understand a thin result, and decide what to do when a site blocks Bowmark.
Reading this with an agent?
curl -s https://bowmark.ai/docs/blocked-pages.md. The whole site as one file: https://bowmark.ai/llms-full.txt. Index of every page: https://bowmark.ai/llms.txt.Control how read.page loads a page
await bowmark.read.page(url, { strategy: "auto", format: "markdown", maxChars: 200000 })| Strategy | What it does | Use when |
|---|---|---|
"auto" (default) | Fetch first, then a full page load only for a wall, interstitial, or empty shell | You do not know what the page needs |
"fetch" | A fast read only | The page is static or you accept an incomplete answer |
"browser" | Always load and wait for page JavaScript | You know the page is rendered client-side |
Browser loading is slower (often 30–55 seconds on a heavy page) and its rendering time is
billed. format is "markdown" by default; choose "text", "cleanHtml", or "html"
when your parser needs those shapes. content is capped at 200,000 characters by default;
raise maxChars when truncated is true and the data you need is below the cut.
read.page chooses and retries its available routes automatically. There is no caller-facing
egress option; if every route is refused, the result explains that in wall and warnings.
What read.page returns
Check more than content before trusting it — ok and warnings distinguish a real page from
a wall, login page, empty shell, or truncated result.
| Field | What it tells you |
|---|---|
ok | false when the read failed or returned a wall, login page, or empty shell |
title | The page title or first heading, or null when neither exists |
content | The page in the requested format, empty when nothing could be read |
format / chars | The content format and its character count |
truncated | true when content was cut at maxChars |
warnings | Why the result may be thin: a wall, shell, redirect, failed page request, or budget |
url / requestedUrl | Where the read ended versus the URL requested |
status / error | The HTTP status and why the read failed, when it failed |
needsLogin | true when the returned page is a sign-in form |
escalationReason | Why the page needed a fuller load than this call provided |
wall | The wall vendor and whether it was cleared, or null |
json | Parsed JSON when the URL served JSON |
headers | Response headers when requested with { headers: true } |
PDF URLs return extracted text with a page-count warning. A rendered table may append its
inline rows to content; a table loaded by a separate request reports that it is incomplete.
When truncated is true, raise maxChars above the page length named in warnings.
When a site blocks Bowmark
read.page first tries to clear common CAPTCHA and bot-wall shapes itself. If it returns
ok: false with wall: { cleared: false }, Bowmark has already tried its available routes;
stop retrying the same read and treat the wall as the site's refusal. A failed or walled read is
still billed for what it used: browser time if it rendered, and $0.0001 per static-residential
request. See Pricing.
browser_agent uses a different browser and network, so it can be worth one billed attempt
when the answer is worth the 1–3 minute wait and per-run charge. On a public page, your own
cron or scheduled script can start and poll it unattended. Bowmark does not schedule it for
or monitor it for you. A login or CAPTCHA that needs a person needs that person on every run; a
high-volume or live, seconds-long lookup needs a typed provider instead. Its answer can vary from
run to run, so do not treat one successful run as a permanent guarantee.
If that browser-agent attempt also returns blocked:, Bowmark cannot read that site today.
Use another public source, the manufacturer's site, or a provider returned by get_library.
Read the result before trusting it
ok: false, warnings, wall, needsLogin, escalationReason, and truncated explain
whether content is the page you asked for. A login page is not your data; read.page cannot
pause for a person. Use Scripting for the surrounding capability examples.