> ## Documentation Index
> Fetch the complete documentation index at: https://docs.yourhomie.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Websites

> Pull content from your own websites into a knowledge store — crawl a site, read a sitemap, or add a single page — then keep it current.

homie can extract the content of your websites and turn it into knowledge the assistant answers from. Open your [knowledge store](/en/user-guide/knowledge/knowledge-stores), go to **Sources → Websites**, and select **Add** to open the **Add website** dialog.

## Choose how to add pages

The dialog offers three modes:

<Tabs>
  <Tab title="Crawl Website">
    Enter a domain URL (for example `www.example.com`). homie follows the links and **discovers all reachable pages**, then shows a preview before importing. Select **Crawl website** to start discovery.

    Best when you want most of a site and don't maintain a sitemap.
  </Tab>

  <Tab title="Sitemap">
    Enter your sitemap URL (for example `www.example.com/sitemap.xml`). homie reads it — including nested sub-sitemaps — to list the pages, then shows the same preview. Select **Read sitemap**.

    Best when you maintain a sitemap and want an accurate, complete list.
  </Tab>

  <Tab title="Single page">
    Enter one exact URL (for example `www.example.com/contact`) and select **Add page**. The page is read directly — no discovery, no preview.

    Best for adding one specific page.
  </Tab>
</Tabs>

## Review and import

For **Crawl Website** and **Sitemap**, homie first discovers the pages and opens a preview so you decide exactly what to import.

A capacity summary at the top shows:

| Number         | Meaning                                                                                              |
| -------------- | ---------------------------------------------------------------------------------------------------- |
| **Total**      | Pages remaining after your filters.                                                                  |
| **Known**      | Pages already in this store.                                                                         |
| **New**        | Total minus Known.                                                                                   |
| **Importable** | New pages, capped by your plan's remaining page quota — shown inline as `(unlimited)` or `(max. N)`. |

Below it, a **tree of discovered URLs** is grouped into expandable folders and loads lazily — use **Load more URLs** on large sites. If discovery was only partial (for example a sub-sitemap failed), you'll see **Preview incomplete**.

You don't tick pages individually here. Instead you shape the set with **filters**, then select **Create** to import (or **Back** to discard). homie imports every page that matches, up to your importable limit, and starts processing.

### Filters

Open **Filter** to narrow what gets imported:

* **Include product pages** — off by default. Leave it off if you already provide your catalog via [Product data](/en/user-guide/knowledge/product-data) or a connected Shopify store, otherwise product pages would be captured twice.
* **Include category pages** — off by default. Enable to include category or collection pages.
* **Include paths** / **Exclude paths** — add rules with a match type: **Starts with**, **Ends with**, **Contains**, **Exact**, or **Wildcard (\*)**. After changing filters, select **Refresh preview** to update the counts.

<Note>
  Product and category detection is based on URL patterns, not page content — so review the preview before importing.
</Note>

## Manage your website sources

Each website you add appears as a source you can expand:

* A **type** badge — **Crawled**, **Sitemap**, or **Single** — with the page count and when it was last crawled or scraped.
* For crawl and sitemap sources, the active **filter** pills and a **gear** icon to edit settings.
* Expand a source to see its **URL tree** (folders and individual pages), each with a status.

Use the **search** box (**Search URLs…**) to find pages, the **status filter** (**All** / **Error** / **Processing**), and **Select all** for bulk actions. Long lists load more as you scroll.

## Status

Statuses update live while homie reads your pages.

**Source status**

| Status         | Meaning                                               |
| -------------- | ----------------------------------------------------- |
| **In queue**   | Waiting to be processed.                              |
| **Processing** | Pages are being read; a progress percentage is shown. |
| **Error**      | Something went wrong — check the failed pages.        |

**Page status**

| Status                   | Meaning                                 |
| ------------------------ | --------------------------------------- |
| **Training in progress** | The page is queued or being read.       |
| **Status is updating**   | The page status is refreshing.          |
| **Trained**              | The page is available to the assistant. |
| **Error**                | The page failed — select it to retry.   |

## Keep website content current

Website content changes over time, so homie keeps it fresh in two ways:

* **Automatically** — homie re-reads your website sources on the store's update interval (manual, daily, weekly, or monthly). Set it under **Settings → Website updates** (see [Introduction](/en/user-guide/knowledge/knowledge-stores)).
* **Manually** — select one or more sources and choose **Reprocess → All** to re-discover and re-read everything, or **Reprocess → Errors only** to retry just the failed pages. You can also retry a single failed page from its status badge.

### Edit a source's settings

On a crawl or sitemap source, select the **gear** icon to adjust its filters and toggle **Automatically include new URLs** (on by default — future re-reads add newly discovered pages; turn it off to only refresh pages homie already knows). Saving applies the settings and immediately re-reads the source.

<Tip>
  After a major site change — new policy pages, updated shipping terms, or a redesign — reprocess the source so the assistant never quotes outdated information.
</Tip>

## Inspect or remove a page

Select a page to open **URL details** — its **Domain**, **Total characters**, document ID, last update, any last error, and a read-only preview of the extracted **Page content**.

To remove pages from a crawl or sitemap source, use **Exclude permanently** on a URL (or **Exclude folder permanently** on a folder). Excluded pages are skipped on future re-reads. Single-page sources offer **Delete URL** instead.

## Page quota

The **Page quota** card shows how much website content you can store:

* **Total** — your plan's page limit (or **Unlimited**).
* **Used** — pages used across your whole organization (all knowledge stores combined).
* **In this store** — pages stored here.
* **Available** — how many you have left.

When the quota is reached, **Importable** drops to 0 and new imports are blocked until you remove pages or upgrade your plan.