Policy
Crawler policy
Last updated · Version 2026-09-29.2
What we import
Everybarn may show public listings from other sites so buyers can compare. Imported listings are labeled, link to their source, and never carry the verified badge. We read public listing pages on other sites, marketplaces and classifieds sites included, wherever the site's robots.txt allows those pages. Storage is fact-only: the animal's details (species, breed, sex, age, colour, size, registry, stated health checks and guarantees, ready date, parents as listed), the asking price, town and state, the title and date, the photo count, the breeder's business name as the listing shows it, and the page's address. We never copy description text (any summary is written by us from the facts) and never copy, download or store photos: where robots.txt allows the image, we may show the main photo by linking to it on the site's own server, like a link preview. We never collect a person's name, username or contact detail from a marketplace. An imported listing gets its own page here, in the same layout as a listing on Everybarn, with the original site named as its source and a link to the original. It is not a listing on Everybarn: it carries no fee, no seller account, and no messaging, and it counts least among the comparable sales and listings behind Fair Value. Photo files and descriptions are copied to Everybarn only from their owners: a seller importing their own listing, after confirming the content is theirs to post, or a partner sending its listings under a written licence.
Breeder directory
Breeder pages show a business name, its city and state, and the public records it appears in (a breeder's own website, USDA APHIS and state licensing lists), each with its source and date. We never publish a person's name or any contact details. A page stays labelled Unclaimed until its business claims it, by email or website, never by phone. Anyone can ask us to remove a page: it is hidden at once and deleted within 24 hours, and it does not come back.
How our crawler behaves
It identifies itself, respects robots.txt, and limits its request rate. It never logs in, and never collects contact details or private data. Its user agent links to this page and names us (EverybarnBot). It reads robots.txt first, keeps it for at most 24 hours, and checks every page against it just before asking for it. It asks one site for one page at a time, at least 3 seconds apart or longer when Crawl-delay asks, reads sitemaps first, and asks only for pages that changed. We never crawl behind logins: no cookies, passwords or credentials are ever sent, and a page that asks for a sign-in is skipped. It never gets around a site's defences. A refusal, a bot challenge or CAPTCHA, or a sign-in redirect stops it on that site at once, and the site stays blocked until a person has looked at why. A rate limit (too many requests) stops it at once too: it waits at least 24 hours, longer each time it happens again (up to 7 days), and then resumes at half the rate it was using. It visits only sites on our own list, each with its own kill switch. A listing that disappears from its site leaves search at once and is deleted after 14 days. It does not run until this policy is signed off.
Opting out
Block our user agent in robots.txt, or contact us and we will remove your listings within 24 hours. Every imported listing's page also has a button that removes just that listing: hidden at once, deleted within 24 hours, never shown again. Every opt-out, by domain or URL prefix, is a hard delete: the entries are physically removed, not hidden, and the domain is disabled for future crawling. A cease-and-desist disables the domain the same day it arrives. A whole site can opt out with the form below: enter its domain and an email address at that domain, and follow the link we send. Nothing changes until it is followed. Other requests are handled by email, with no phone step.
Claiming a listing
If an imported listing is yours, you can claim it, verify your identity, and manage it directly. Import it from its link and claim the imported copy in the same step: it retires when your listing goes live here, and anyone who asked to hear from you is told. Until then it never receives the Verified mark or Papers records: those belong only to native listings from identity-verified sellers.
Opt a whole site out
Enter your site’s domain and an email address at that domain. We email that address a link; nothing changes until it is followed. Then our crawler stops visiting the site, and everything we indexed from it is deleted within 24 hours.