Skip to main content
homie can extract the content of your websites and turn it into knowledge the assistant answers from. Open your knowledge store, go to Sources → Websites, and select Add to open the Add website dialog.

​
Choose how to add pages

The dialog offers three modes:
Enter a domain URL (for example www.example.com). homie follows the links and discovers all reachable pages, then shows a preview before importing. Select Crawl website to start discovery.Best when you want most of a site and don’t maintain a sitemap.

​
Review and import

For Crawl Website and Sitemap, homie first discovers the pages and opens a preview so you decide exactly what to import. A capacity summary at the top shows: Below it, a tree of discovered URLs is grouped into expandable folders and loads lazily — use Load more URLs on large sites. If discovery was only partial (for example a sub-sitemap failed), you’ll see Preview incomplete. You don’t tick pages individually here. Instead you shape the set with filters — edit them in the Filter panel, or hover a folder or page row in the tree and select the trash icon (Exclude folder on folders, Add exclude rule on pages) to add the matching rules to Exclude paths. The new rules appear immediately at the top; the counts and the tree update once you select Refresh preview. When the set looks right, select Create to import (or Back to discard). homie imports every page that matches, up to your importable limit, and starts processing.

​
Filters

Open Filter to narrow what gets imported:
  • Include product pages — off by default. Leave it off if you already provide your catalog via Product data or a connected Shopify store, otherwise product pages would be captured twice.
  • Include category pages — off by default. Enable to include category or collection pages.
  • Include paths / Exclude paths — add rules with a match type: Starts with, Ends with, Contains, Exact, or Wildcard (*). After changing filters, select Refresh preview to update the counts.
Product and category detection is based on URL patterns, not page content — so review the preview before importing.

​
Manage your website sources

Each website you add appears as a source you can expand:
  • A type badge — Crawled, Sitemap, or Single — with the page count and when it was last crawled or scraped.
  • For crawl and sitemap sources, the active filter pills and a gear icon to edit settings.
  • Expand a source to see its URL tree (folders and individual pages), each with a status.
Use the search box (Search URLs…) to find pages, the status filter (All / Error / Processing), and Select all for bulk actions. Long lists load more as you scroll.

​
Status

Statuses update live while homie reads your pages. Source status Page status

​
Keep website content current

Website content changes over time, so homie keeps it fresh in two ways:
  • Automatically — homie re-reads your website sources on the store’s update interval (manual, daily, weekly, or monthly). Set it under Settings → Website updates (see Introduction).
  • Manually — select one or more sources and choose Reprocess → All to re-discover and re-read everything, or Reprocess → Errors only to retry just the failed pages. You can also retry a single failed page from its status badge.

​
Edit a source’s settings

On a crawl or sitemap source, select the gear icon to adjust its filters and toggle Automatically include new URLs (on by default — future re-reads add newly discovered pages; turn it off to only refresh pages homie already knows). Saving applies the settings and immediately re-reads the source.
After a major site change — new policy pages, updated shipping terms, or a redesign — reprocess the source so the assistant never quotes outdated information.

​
Inspect or remove a page

Select a page to open URL details — its Domain, Total characters, document ID, last update, any last error, and a read-only preview of the extracted Page content. To remove pages from a crawl or sitemap source, use Exclude permanently on a URL (or Exclude folder permanently on a folder). The exclusion buttons are also available while you search or filter the tree. To remove several pages at once, tick the checkboxes on URL and folder rows — selecting a folder covers everything beneath it. A selection bar appears below the tree with Exclude permanently and Clear selection. One confirmation dialog lists every rule that will be created, and redundant selections (for example a page inside a selected folder) are collapsed automatically. Excluded pages are skipped on future re-reads. Single-page sources offer Delete URL instead.

​
Page quota

The Page quota card shows how much website content you can store:
  • Total — your plan’s page limit (or Unlimited).
  • Used — pages used across your whole organization (all knowledge stores combined).
  • In this store — pages stored here.
  • Available — how many you have left.
When the quota is reached, Importable drops to 0 and new imports are blocked until you remove pages or upgrade your plan.