Smart Knowledge Bases

The newer generation with smarter document processing, first class folders, richer retrieval, and knowledge graph understanding.

Standard and Smart. When you create a knowledge base you choose its type: Standard (the original, reliable and feature complete) or Smart (the newer generation with smarter document processing, first class folders, richer retrieval, and knowledge graph understanding). This guide focuses on Smart knowledge bases and calls out where Smart does more than Standard. Smart is a new version that lives alongside Standard for now (you may see it marked Beta in the product), and you can mix both. See Standard and Smart knowledge bases.

What is smarter about a Smart knowledge base

A Smart knowledge base reads your documents more thoroughly and retrieves from them more precisely than a Standard one:

• It reads scanned pages and photos with OCR, and describes images, charts, and diagrams so they become searchable.

• It understands tables as rows and columns, not just as a block of text (where enabled for your organisation).

• It writes a short summary and tags for every passage, and a summary for every document and folder.

• It builds a lightweight knowledge graph of the people, companies, and things your documents mention, so answers can follow those connections.

• Folders are first class: they have their own summaries and can be used to focus a search.

• Retrieval combines meaning based and keyword matching, and citations can show you the original PDF page with the answer highlighted.

Advanced settings (Smart only)

A Smart knowledge base gives you fine control over how documents are processed and retrieved, under Smart KB advanced settings. The defaults are tuned to work well out of the box, so you can safely skip this section and change it later. When you do want to tune it, the most useful options are:

• Semantic chunking. On by default. Blockbrain splits documents on natural meaning shifts rather than at a fixed length, so each passage stays coherent. Chunk size and overlap are adjustable.

• Generate AI tags and summaries. On by default. Blockbrain writes tags and per document summaries during processing, which improves retrieval for broad questions.

• Table processing. How tables in your documents are read: Off, Standard (rows and columns), Enhanced (learns a repeating column layout), or Maximum (reads complex tables in digital PDFs from the page image). Higher tiers fall back safely, so results are never worse. Available where enabled for your organisation.

• System models. Which models handle enrichment, vision (image captioning), OCR, and reranking. Each offers a System default, and you only override with a reason, because it affects processing cost and quality.

• Build knowledge graph. Off by default. Turn it on to extract and merge the entities in your documents into a knowledge graph that powers entity aware retrieval and the graph view. It adds processing cost per document, and turning it on reprocesses existing documents.

• Folder scoped retrieval. Uses each folder's summary and description to steer retrieval toward the right folder. Recommended for well organized knowledge bases.

Settings that change how documents are read (chunking, models, table processing) apply to documents added afterwards. To apply them to existing documents, reindex those documents (see Reprocessing documents).

Uploading documents

Supported file types

Same as the Standard Database.

How documents are processed

When you upload a file, Blockbrain runs it through an automatic processing pipeline in the background. Each step is durable: if something goes wrong (for example a temporary hiccup with a model), the step is retried on its own without redoing the work that already succeeded. Here is what happens, in plain terms.

• Reading the file. Blockbrain detects the file type and pulls out its content as text, tables, and images, keeping track of which page each piece came from.

• Splitting into passages. The text is split into overlapping passages sized for search, preferring natural topic boundaries so each passage reads coherently.

• Reading scanned pages and images. Pages with little or no selectable text are read with OCR. Images, charts, and diagrams are described by a vision model, and that description becomes searchable text. The image itself is stored safely and only referenced from the knowledge base.

• Understanding tables. Where table processing is enabled for your organisation, tables are read as rows and columns so retrieval can match the actual cell values. The most capable tier also reads tables in ordinary digital PDFs such as invoices and receipts. If a table cannot be read cleanly, it safely falls back to plain text, so results are never worse.

• Removing repetition. Repeated boilerplate (headers, footers, disclaimers, page numbers) is collapsed so it does not crowd out real answers.

• Writing summaries and tags. Blockbrain writes a short summary and a set of tags for each passage, and (optionally) a small knowledge graph of the relationships it finds. If you set a tag vocabulary, tags are mapped to your standard words.

• Making it searchable. Each passage is turned into a vector that captures its meaning, and everything is saved together so a document becomes searchable as a whole.

• Summarizing documents and folders. Finally, Blockbrain writes a summary for the whole document and rolls folder summaries up from the documents and subfolders they contain.

How long it takes

Processing time depends on the file size, the number of images, and whether scanned pages need OCR. A typical 10 to 50 page document is ready in under a minute. Large PDFs with many scanned pages can take a few minutes. While a document is queued, its row shows a rough estimated time (for example "3 minutes remaining", or "Calculating…" when there is not enough history yet), and a banner appears if the system is unusually busy. Treat the estimate as a guide, not a guarantee.

Reprocessing documents

Reindexing re-runs the whole pipeline on content that is already in the knowledge base, using the knowledge base's current settings. Reach for it when you have changed the settings and want existing documents to adopt them, when an underlying model has improved, or when a document ended up in a bad state and you want a clean run.

You can Reindex a single document, a folder (and everything under it), or the whole knowledge base. Documents that are already processing are left alone, and you are told how many were queued. Reindexing replaces a document's content in place, so retrieval keeps returning the current version right up until the new one is ready, with no gap where the document is unavailable.

When you change a setting that only affects how passages are stored rather than how they are read, Blockbrain reprocesses the existing documents with the new configuration and reuses the already extracted text where it can, so it is much quicker and cheaper than a full reindex. It confirms this with an Apply new settings prompt before it runs.

Table processing

Tables are one of the hardest things to search well, so a Smart knowledge base can structure them into rows and columns rather than treating a table as one block of text. You choose how much effort goes into this, per knowledge base, in the Smart settings under Table processing.

There are four tiers, and each one does everything the tier before it does and adds more:

• Off. Tables are searched as plain text. No table work and no added cost.

• Standard (the default). Tables are structured into rows and columns by fast built-in parsing. This adds no extra AI cost.

• Enhanced. Everything Standard does, plus it learns a reusable column layout for each repeating table shape. It spends a small amount of AI once per layout and then reuses it, so cost stays low even across hundreds of similar tables (for example many invoices in the same format).

• Maximum. Everything Enhanced does, plus it reads each table with a vision model from an image of the table. This recovers the hardest tables (dense grids, image-like invoices) but it does one vision read per table, so it is the most expensive and slowest tier, and the cost grows with the number of tables.

What runs at each tier

This chart follows one table in an ordinary digital PDF (an invoice or receipt) and shows what each tier does with it. Reading left to right, each tier only ever adds a step, so a higher tier can only add structure, never take it away.

Processing step	Off	Standard (default)	Enhanced	Maximum
Read the page text	Yes	Yes	Yes	Yes
Locate the table on the page	No	Yes	Yes	Yes
Structure it into rows and columns	No	Yes (clean grids)	Yes	Yes
Learn a repeating column layout and reuse it	No	No	Yes (once per layout)	Yes
Read the table with a vision model	No	No	No	Yes (every table)
What the table becomes for search	Plain text, no rows	Structured rows (plain text if the grid is too sparse)	Structured rows with tidied columns	Structured rows, with every table vision read
Extra AI work	None	None (built-in parsing)	A little, once per layout	One vision read per table (highest)

Never worse. The tiers only ever add structure. If a higher tier cannot read a table cleanly, it falls back to the tier below, and ultimately to plain text, so a harder setting never gives a worse result than a simpler one.

Which files this applies to. The steps above describe ordinary digital PDFs, which carry no built-in table grid. Spreadsheets, Word, and HTML tables already arrive as a grid, so they are structured from Standard upward without the locate step. Tables on scanned pages and in uploaded images are read by OCR first, then structured the same way.

Choosing a tier. Standard is a good default. Move up to Enhanced when your documents repeat the same table layout, and to Maximum only when the lower tiers miss tables you need. A tier change applies to documents added afterwards, so reindex existing documents to restructure their tables. Table processing is available where enabled for your organisation.

How your documents get searched

The real power of a Smart knowledge base is how it is searched when an AI assistant answers a question. You do not have to configure any of this. When a knowledge base is attached to a conversation, data room, or bot, an assistant retrieves the most relevant passages and grounds its answer in them.

Behind the scenes, retrieval can combine several techniques:

Technique	What it does
Meaning based	Finds passages that mean the same as the question, even when they use different words. This is the default.
Keyword	Matches the exact terms in the question. Useful for product codes, names, and jargon.
Hybrid	Blends meaning based and keyword matching so results strong on either surface.
Entity (knowledge graph)	Follows a named person, company, or product through the knowledge graph, and resolves different names for the same thing, so "Apple" also finds "Apple Inc." Available when Build knowledge graph is on.
Folder	Uses folder summaries to find the most relevant folder for a topic.

By default, retrieval finds passages by meaning and then re-orders them with a reranking model for extra precision, and near duplicate passages are collapsed so the assistant never sees the same content twice. Several optional refinements can be turned on per knowledge base to push quality further, such as hybrid keyword matching, folder scoped retrieval, knowledge graph aware ranking, and a recency first pass.

Knowledge graph and entity search

While it processes your documents, a Smart knowledge base can also pull out facts and build a knowledge graph from them. A fact is a simple statement it finds in the text, made of a subject, a relationship, and an object. Gather enough of these across a document set and they form a graph of how the people, companies, products, and places in your documents relate to each other.

Turning it on

Building the graph is optional. In the knowledge base's Smart settings, turn on Build knowledge graph. It is off by default because it does extra AI work on every document, so it adds some processing cost, and turning it on reprocesses your existing documents so their facts are added to the graph. New documents then join the graph automatically as they are processed.

When the graph is built, the knowledge base also merges the different ways the same thing is written into a single entity. "Apple", "Apple Inc.", and "the company" become one node, matched by meaning rather than spelling, and each entity is given a type (person, organization, location, and so on).

What it gives you

• Entity search. Ask for everything about a named person, company, or product, and the knowledge base returns the passages where that entity appears in a fact, ranked by how much is known about it. Because names are merged, a search for "Apple" also surfaces passages that only mention "Apple Inc." An optional expand mode pulls in passages about directly connected entities too.

• Sharper ranking. When an answer is being put together, passages whose facts mention the same entities as the question are nudged up the ranking, so the most on-topic passages win.

Both of these need the graph to exist (Build knowledge graph) and to be switched into retrieval (Use knowledge-graph triples in retrieval, also in the Smart settings, off by default). The retrieval switch applies at query time, so once the graph is built you can turn it on or off without reprocessing.

Exploring the graph

Open a knowledge base and select Knowledge graph view to see the graph itself. Each node is an entity and each line is a relationship, labelled with the fact that connects them (for example "is an early customer of", "was founded by", "made follow-on investments in"). You can:

• Switch Layout between Hierarchy and Cluster.

• Use Show to cap how many entities are drawn (the header notes, for example, "Showing 50 of 149 entities") so a large graph stays readable.

• Click a node to see its connections, and zoom or fit the view with the controls in the corner.

Each document also has its own Triples tab, listing the facts extracted from that single document, with a filter box to search the entities and facts.

Tracking processing status

Each document in the browser shows where it is: queued, processing, embedding, done, or failed. A document that finished successfully is searchable. A document can also come out partial, which means most of it is searchable but one part (for example an image that could not be read even after retries) was dropped. Failed and partial documents carry an error message explaining what happened.

State	Meaning
In queue	Uploaded and waiting to start.
Processing / Embedding	Being read and made searchable.
Done	Finished and fully searchable.
Partial	Searchable, but one part could not be processed.
Failed	Could not complete after automatic retries.

Standard and Smart knowledge bases

Blockbrain offers two types of knowledge base side by side. Both are fully supported, and you do not need to do anything about the Standard knowledge bases you already have.

Capability	Standard	Smart
Smart passage boundaries (semantic chunking)	No	Yes
Reading scanned pages (OCR)	No	Yes
Describing images, charts, and diagrams	No	Yes
Understanding tables as rows and columns	No	Yes (where enabled)
Knowledge graph and entity aware retrieval	No	Yes
First class folders with summaries	No	Yes
Folder rename or move without breaking attachments	Breaks attachments	Stays stable
Document and folder summaries	No	Yes
Per knowledge base processing and retrieval settings	Limited	Full
Original PDF page with the answer highlighted	No	Yes

FAQs

Can I mix Standard and Smart knowledge bases?

Yes. You can attach both types to the same conversation, data room, or bot. Blockbrain searches both and merges the results before returning them to you or the AI, so you never have to think about which type a result came from. If one side is briefly unavailable, the other still answers.

Can I move a Standard knowledge base to Smart?

If you own a Standard knowledge base, a Migrate to Smart action is available (where migration is enabled for your organisation). Starting a migration:

• Creates a separate Smart copy named "[knowledge base name] - Smart". The original stays exactly as it is and keeps working, so you end up with both.

• Reprocesses your documents through the full Smart pipeline, so they gain Smart quality. This takes time and has a processing cost, which a confirmation dialog explains before you start.

• Lets you choose what to bring over, with Migrate connections (move the conversations, data rooms, and bots that use it onto the copy) and Keep the same people (share the copy with the same users). Both are optional. An Advanced Smart settings section lets you set Smart only options at the same time.

The Smart copy only appears in your list once it is fully processed and ready to use, so you never open a half migrated copy. A few things do not carry over (for example group based shares and some Standard only settings), and they are reported so you can reconcile them.

Which type should I use for new knowledge bases?

Use Smart for everything new. It offers noticeably better retrieval quality, especially for large or image heavy collections. Standard remains available for your existing data for now.

Quick reference

Add documents and make them searchable

1. Create a knowledge base (New Database Source, type Smart).

2. Upload files (Upload Files or drag and drop).

3. Wait for documents to finish processing (the row shows when each is done).

4. Attach the knowledge base to a conversation, data room, or bot.

Organize a large document set

1. Create top level folders, or upload with relative paths such as legal/contracts/nda.pdf so folders are created for you.

2. Add folder descriptions so folders are findable by concept.

3. When attaching to a bot or data room, narrow the scope to the relevant folder.

Give a teammate access

1. Open the knowledge base and select Share.

2. Under Add people, enter their email and pick a role (Viewer, Contributor, or Content Manager).

3. They can now see the knowledge base under Shared with Me and use it in their own conversations.