> For the complete documentation index, see [llms.txt](https://docs.blockbrain.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.blockbrain.ai/for-users/all-about-knowledge-management/smart-knowledge-bases.md).
# Smart Knowledge Bases
{% hint style="info" %}
**Standard and Smart.** When you create a knowledge base you choose its type: **Standard** (the original, reliable and feature complete) or **Smart** (the newer generation with smarter document processing, first class folders, richer retrieval, and knowledge graph understanding). This guide focuses on **Smart** knowledge bases and calls out where Smart does more than Standard. Smart is a new version that lives alongside Standard for now (you may see it marked **Beta** in the product), and you can mix both. See [#standard-and-smart-knowledge-bases](#standard-and-smart-knowledge-bases "mention").
{% endhint %}
#### What is smarter about a Smart knowledge base
A Smart knowledge base reads your documents more thoroughly and retrieves from them more precisely than a Standard one:
* It reads **scanned pages and photos** with OCR, and describes **images, charts, and diagrams** so they become searchable.
* It understands **tables** as rows and columns, not just as a block of text (where enabled for your organisation).
* It writes a short **summary and tags** for every passage, and a summary for every document and folder.
* It builds a lightweight **knowledge graph** of the people, companies, and things your documents mention, so answers can follow those connections.
* Folders are **first class**: they have their own summaries and can be used to focus a search.
* Retrieval combines meaning based and keyword matching, and citations can show you the **original PDF page with the answer highlighted**.
#### Advanced settings (Smart only)
A Smart knowledge base gives you fine control over how documents are processed and retrieved, under **Smart KB advanced settings**. The defaults are tuned to work well out of the box, so you can safely skip this section and change it later. When you do want to tune it, the most useful options are:
* **Semantic chunking.** On by default. Blockbrain splits documents on natural meaning shifts rather than at a fixed length, so each passage stays coherent. Chunk size and overlap are adjustable.
* **Generate AI tags and summaries.** On by default. Blockbrain writes tags and per document summaries during processing, which improves retrieval for broad questions.
* **Table processing.** How tables in your documents are read: **Off**, **Standard** (rows and columns), **Enhanced** (learns a repeating column layout), or **Maximum** (reads complex tables in digital PDFs from the page image). Higher tiers fall back safely, so results are never worse. Available where enabled for your organisation.
* **System models.** Which models handle enrichment, vision (image captioning), OCR, and reranking. Each offers a **System default**, and you only override with a reason, because it affects processing cost and quality.
* **Build knowledge graph.** Off by default. Turn it on to extract and merge the entities in your documents into a knowledge graph that powers entity aware retrieval and the graph view. It adds processing cost per document, and turning it on reprocesses existing documents.
* **Folder scoped retrieval.** Uses each folder's summary and description to steer retrieval toward the right folder. Recommended for well organized knowledge bases.
Settings that change how documents are read (chunking, models, table processing) apply to documents added afterwards. To apply them to existing documents, reindex those documents (see Reprocessing documents).
### Uploading documents
#### Supported file types
Same as the [Standard Database.](https://docs.blockbrain.ai/for-users/all-about-knowledge-management#supported-file-types)
### How documents are processed
When you upload a file, Blockbrain runs it through an automatic processing pipeline in the background. Each step is durable: if something goes wrong (for example a temporary hiccup with a model), the step is retried on its own without redoing the work that already succeeded. Here is what happens, in plain terms.
* **Reading the file.** Blockbrain detects the file type and pulls out its content as text, tables, and images, keeping track of which page each piece came from.
* **Splitting into passages.** The text is split into overlapping passages sized for search, preferring natural topic boundaries so each passage reads coherently.
* **Reading scanned pages and images.** Pages with little or no selectable text are read with OCR. Images, charts, and diagrams are described by a vision model, and that description becomes searchable text. The image itself is stored safely and only referenced from the knowledge base.
* **Understanding tables.** Where table processing is enabled for your organisation, tables are read as rows and columns so retrieval can match the actual cell values. The most capable tier also reads tables in ordinary digital PDFs such as invoices and receipts. If a table cannot be read cleanly, it safely falls back to plain text, so results are never worse.
* **Removing repetition.** Repeated boilerplate (headers, footers, disclaimers, page numbers) is collapsed so it does not crowd out real answers.
* **Writing summaries and tags.** Blockbrain writes a short summary and a set of tags for each passage, and (optionally) a small knowledge graph of the relationships it finds. If you set a tag vocabulary, tags are mapped to your standard words.
* **Making it searchable.** Each passage is turned into a vector that captures its meaning, and everything is saved together so a document becomes searchable as a whole.
* **Summarizing documents and folders.** Finally, Blockbrain writes a summary for the whole document and rolls folder summaries up from the documents and subfolders they contain.
#### How long it takes
Processing time depends on the file size, the number of images, and whether scanned pages need **OCR**. A typical 10 to 50 page document is ready in under a minute. Large PDFs with many scanned pages can take a few minutes. While a document is queued, its row shows a rough estimated time (for example "3 minutes remaining", or "Calculating…" when there is not enough history yet), and a banner appears if the system is unusually busy. Treat the estimate as a guide, not a guarantee.
#### Reprocessing documents
**Reindexing** re-runs the whole pipeline on content that is already in the knowledge base, using the knowledge base's **current** settings. Reach for it when you have changed the settings and want existing documents to adopt them, when an underlying model has improved, or when a document ended up in a bad state and you want a clean run.
You can **Reindex** a single document, a folder (and everything under it), or the whole knowledge base. Documents that are already processing are left alone, and you are told how many were queued. Reindexing replaces a document's content in place, so retrieval keeps returning the current version right up until the new one is ready, with no gap where the document is unavailable.
When you change a setting that only affects how passages are stored rather than how they are read, Blockbrain reprocesses the existing documents with the new configuration and reuses the already extracted text where it can, so it is much quicker and cheaper than a full reindex. It confirms this with an **Apply new settings** prompt before it runs.
### Table processing
Tables are one of the hardest things to search well, so a Smart knowledge base can structure them into rows and columns rather than treating a table as one block of text. You choose how much effort goes into this, per knowledge base, in the Smart settings under **Table processing**.
{% embed url="" %}
There are four tiers, and each one does everything the tier before it does and adds more:
* **Off.** Tables are searched as plain text. No table work and no added cost.
* **Standard** (the default). Tables are structured into rows and columns by fast built-in parsing. This adds no extra AI cost.
* **Enhanced.** Everything Standard does, plus it learns a reusable column layout for each repeating table shape. It spends a small amount of AI once per layout and then reuses it, so cost stays low even across hundreds of similar tables (for example many invoices in the same format).
* **Maximum.** Everything Enhanced does, plus it reads each table with a vision model from an image of the table. This recovers the hardest tables (dense grids, image-like invoices) but it does one vision read per table, so it is the most expensive and slowest tier, and the cost grows with the number of tables.
#### What runs at each tier
This chart follows one table in an ordinary digital PDF (an invoice or receipt) and shows what each tier does with it. Reading left to right, each tier only ever adds a step, so a higher tier can only add structure, never take it away.
| Processing step | Off | Standard (default) | Enhanced | Maximum |
| -------------------------------------------- | ------------------- | ------------------------------------------------------ | ----------------------------------- | --------------------------------------------- |
| Read the page text | Yes | Yes | Yes | Yes |
| Locate the table on the page | No | Yes | Yes | Yes |
| Structure it into rows and columns | No | Yes (clean grids) | Yes | Yes |
| Learn a repeating column layout and reuse it | No | No | Yes (once per layout) | Yes |
| Read the table with a vision model | No | No | No | Yes (every table) |
| **What the table becomes for search** | Plain text, no rows | Structured rows (plain text if the grid is too sparse) | Structured rows with tidied columns | Structured rows, with every table vision read |
| **Extra AI work** | None | None (built-in parsing) | A little, once per layout | One vision read per table (highest) |
**Never worse.** The tiers only ever add structure. If a higher tier cannot read a table cleanly, it falls back to the tier below, and ultimately to plain text, so a harder setting never gives a worse result than a simpler one.
**Which files this applies to.** The steps above describe ordinary digital PDFs, which carry no built-in table grid. Spreadsheets, Word, and HTML tables already arrive as a grid, so they are structured from Standard upward without the locate step. Tables on scanned pages and in uploaded images are read by OCR first, then structured the same way.
**Choosing a tier.** Standard is a good default. Move up to Enhanced when your documents repeat the same table layout, and to Maximum only when the lower tiers miss tables you need. A tier change applies to documents added afterwards, so reindex existing documents to restructure their tables. Table processing is available where enabled for your organisation.
### How your documents get searched
The real power of a **Smart knowledge base** is how it is searched when an AI assistant answers a question. You do not have to configure any of this. When a knowledge base is attached to a conversation, data room, or bot, an assistant retrieves the most relevant passages and grounds its answer in them.
Behind the scenes, retrieval can combine several techniques:
| Technique | What it does |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Meaning based** | Finds passages that mean the same as the question, even when they use different words. This is the default. |
| **Keyword** | Matches the exact terms in the question. Useful for product codes, names, and jargon. |
| **Hybrid** | Blends meaning based and keyword matching so results strong on either surface. |
| **Entity (knowledge graph)** | Follows a named person, company, or product through the knowledge graph, and resolves different names for the same thing, so "Apple" also finds "Apple Inc." Available when **Build knowledge graph** is on. |
| **Folder** | Uses folder summaries to find the most relevant folder for a topic. |
By default, retrieval finds passages by meaning and then re-orders them with a reranking model for extra precision, and near duplicate passages are collapsed so the assistant never sees the same content twice. Several optional refinements can be turned on per knowledge base to push quality further, such as hybrid keyword matching, folder scoped retrieval, knowledge graph aware ranking, and a recency first pass.
### Knowledge graph and entity search
While it processes your documents, a Smart knowledge base can also pull out **facts** and build a **knowledge graph** from them. A fact is a simple statement it finds in the text, made of a subject, a relationship, and an object. Gather enough of these across a document set and they form a graph of how the people, companies, products, and places in your documents relate to each other.
{% embed url="" %}
#### Turning it on
Building the graph is optional. In the knowledge base's Smart settings, turn on **Build knowledge graph**. It is off by default because it does extra AI work on every document, so it adds some processing cost, and turning it on reprocesses your existing documents so their facts are added to the graph. New documents then join the graph automatically as they are processed.
When the graph is built, the knowledge base also merges the different ways the same thing is written into a single entity. "Apple", "Apple Inc.", and "the company" become one node, matched by meaning rather than spelling, and each entity is given a type (person, organization, location, and so on).
#### What it gives you
* **Entity search.** Ask for everything about a named person, company, or product, and the knowledge base returns the passages where that entity appears in a fact, ranked by how much is known about it. Because names are merged, a search for "Apple" also surfaces passages that only mention "Apple Inc." An optional expand mode pulls in passages about directly connected entities too.
* **Sharper ranking.** When an answer is being put together, passages whose facts mention the same entities as the question are nudged up the ranking, so the most on-topic passages win.
Both of these need the graph to exist (**Build knowledge graph**) and to be switched into retrieval (**Use knowledge-graph triples in retrieval**, also in the Smart settings, off by default). The retrieval switch applies at query time, so once the graph is built you can turn it on or off without reprocessing.
#### Exploring the graph
Open a knowledge base and select **Knowledge graph view** to see the graph itself. Each node is an entity and each line is a relationship, labelled with the fact that connects them (for example "is an early customer of", "was founded by", "made follow-on investments in"). You can:
* Switch **Layout** between **Hierarchy** and **Cluster**.
* Use **Show** to cap how many entities are drawn (the header notes, for example, "Showing 50 of 149 entities") so a large graph stays readable.
* **Click a node to see its connections**, and zoom or fit the view with the controls in the corner.
Each document also has its own **Triples** tab, listing the facts extracted from that single document, with a filter box to search the entities and facts.
### Tracking processing status
Each document in the browser shows where it is: queued, processing, embedding, done, or failed. A document that finished successfully is searchable. A document can also come out **partial**, which means most of it is searchable but one part (for example an image that could not be read even after retries) was dropped. Failed and partial documents carry an error message explaining what happened.
| State | Meaning |
| ---------------------- | ------------------------------------------------ |
| In queue | Uploaded and waiting to start. |
| Processing / Embedding | Being read and made searchable. |
| Done | Finished and fully searchable. |
| Partial | Searchable, but one part could not be processed. |
| Failed | Could not complete after automatic retries. |
### Standard and Smart knowledge bases
Blockbrain offers two types of knowledge base side by side. Both are fully supported, and you do not need to do anything about the Standard knowledge bases you already have.
| Capability | Standard | Smart |
| ---------------------------------------------------- | ------------------ | ------------------- |
| Smart passage boundaries (semantic chunking) | No | Yes |
| Reading scanned pages (OCR) | No | Yes |
| Describing images, charts, and diagrams | No | Yes |
| Understanding tables as rows and columns | No | Yes (where enabled) |
| Knowledge graph and entity aware retrieval | No | Yes |
| First class folders with summaries | No | Yes |
| Folder rename or move without breaking attachments | Breaks attachments | Stays stable |
| Document and folder summaries | No | Yes |
| Per knowledge base processing and retrieval settings | Limited | Full |
| Original PDF page with the answer highlighted | No | Yes |
### FAQs
Can I mix Standard and Smart knowledge bases?
Yes. You can attach both types to the same conversation, data room, or bot. Blockbrain searches both and merges the results before returning them to you or the AI, so you never have to think about which type a result came from. If one side is briefly unavailable, the other still answers.
Can I move a Standard knowledge base to Smart?
If you own a Standard knowledge base, a **Migrate to Smart** action is available (where migration is enabled for your organisation). Starting a migration:
* **Creates a separate Smart copy** named "\[knowledge base name] - Smart". The original stays exactly as it is and keeps working, so you end up with both.
* **Reprocesses your documents** through the full Smart pipeline, so they gain Smart quality. This takes time and has a processing cost, which a confirmation dialog explains before you start.
* **Lets you choose what to bring over**, with **Migrate connections** (move the conversations, data rooms, and bots that use it onto the copy) and **Keep the same people** (share the copy with the same users). Both are optional. An **Advanced Smart settings** section lets you set Smart only options at the same time.
{% embed url="" %}
The Smart copy only appears in your list once it is fully processed and ready to use, so you never open a half migrated copy. A few things do not carry over (for example group based shares and some Standard only settings), and they are reported so you can reconcile them.
Which type should I use for new knowledge bases?
Use **Smart** for everything new. It offers noticeably better retrieval quality, especially for large or image heavy collections. Standard remains available for your existing data for now.
### Quick reference
**Add documents and make them searchable**
1. Create a knowledge base (**New Database Source**, type **Smart**).
2. Upload files (**Upload Files** or drag and drop).
3. Wait for documents to finish processing (the row shows when each is done).
4. Attach the knowledge base to a conversation, data room, or bot.
**Organize a large document set**
1. Create top level folders, or upload with relative paths such as `legal/contracts/nda.pdf` so folders are created for you.
2. Add folder descriptions so folders are findable by concept.
3. When attaching to a bot or data room, narrow the scope to the relevant folder.
**Give a teammate access**
1. Open the knowledge base and select **Share**.
2. Under **Add people**, enter their email and pick a role (Viewer, Contributor, or Content Manager).
3. They can now see the knowledge base under **Shared with Me** and use it in their own conversations.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.blockbrain.ai/for-users/all-about-knowledge-management/smart-knowledge-bases.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.