Reachific Add website window with crawl mode options and URL field

How do I add my website to a knowledge base with the web crawler?

November 03, 2026

Quick answer: Open your knowledge base in AI Agents > Knowledge Base, click Web crawler, then Add website. Pick a mode: Exact URL, All URLs with the path or All URLs in this domain. Enter your website address and click Train this page.

Before you start

Step 1: Open the web crawler

  1. Go to AI Agents > Knowledge Base and click the pencil on your knowledge base.
  2. Click the Web crawler tab, or the Web crawler quick button on an empty knowledge base.
  3. Click Add website.

Step 2: Choose what to crawl

  1. Open Select mode (1):
    • Exact URL reads one page only.
    • All URLs with the path reads every page under a section, such as yoursite.com/services.
    • All URLs in this domain reads your whole site.
  2. Enter the address in Enter URL (2).
  3. Click Train this page (3).

Add website window with Select mode list, Enter URL and Train this page numbered

Step 3: Check the results

The crawled pages show under URLs on the Web crawler tab. Use Test retrieval to ask a question and confirm the agent finds the right answer.

FAQ

What if my website changes?

Train the pages again after big updates so your agents use the latest information.

Should I crawl my whole site?

Only if every page is useful to customers. Crawl service, pricing, area and FAQ pages, and skip pages like old blog posts that might confuse answers.

Want us to handle it? Our team can choose the right pages, train your agents and keep them current. Book a free strategy call and get a free quote.

Related articles


Still stuck? Submit a ticket and our team will help. Need faster answers? See our support plans.

blog author avatar

Sayo Tadashi

Reachific is the all in one platform that helps businesses get found, follow up and get paid, with AI built in.

Back to Blog