↓Skip to main content

Using Insightful Search

If you’re here you should already have Insightful Search and all it’s pre-requisites up and running. If not, click here to learn how to set it up.

Table of Contents
#

In RocketChat
#

Go to the channel you created for Insightful Search and type #> help

That will show you a list of commands and how to write them. It’s kinda clunky and annoying to write, but it works.

Request Structure
#

Here I’ll go over what the #> help command says.

First are the commands. COLLECTION_NAME should be replaced. This allows for multiple separate collections so if you want to use this for multiple people or have multiple sets of documents, they don’t mix:

  • #> reset/COLLECTION_NAME - Deletes the collection specified and re-creates it.
  • #> ingest/COLLECTION_NAME - Adds the documents of the specified collection. Usually used after #> reset/COLLECTION_NAME.
  • #> help - Displays info on how to structure your request (like what I’m explaining here)

Next we have the requests. The structure of the request should look similar to the following:

  • !> COLLECTION_NAME // DATABASE_QUERY @OPERATOR:MONTH:DAY:YEAR@ // NUMBER_OF_RESULTS // AI_PROMPT

Some of that is optional. I’ll break it down for you:

  • !> and #> are “trigger words” you should have set up in RocketChat when creating the integration.
  • COLLECTION_NAME is pretty self explanatory, just tell it which collection you want to search.
  • // is used to separate each component of the request. This makes it easier for the code to understand what each part of the request is.
  • DATABASE_QUERY is what you want to “ask” the Database. The Database is “Semantically Searchable”, so you aren’t matching keywords, you should type in the general idea or theme of what you want.
    • Example: If you type personal growth in this spot, it won’t necessarily find a document that contains the words personal growth. It will look for similar meaning in your documents, so if you describe something in your document that could be considered personal growth it will likely find that.
  • @OPERATOR:MONTH:DAY:YEAR@ is the first optional parameter. This can be used to only return documents from before, after, or during a specific date. More on that below.
  • NUMBER_OF_RESULTS is also optional, but the default is 10. This determines how many results the database should return. If your GPU has less than 16GB of VRAM, you should consider lowering this to somewhere between 5-7. If your GPU has 16GB of VRAM (like mine), you should be able to increase to 15 without issues. If you do increase it, make sure you created a Custom Ollama Model with a larger context window.
  • AI_PROMPT is also optional. If you include this, it will send this AI_PROMPT and the results from the DATABASE_QUERY to the AI Model you set up in Node-RED. The AI Model will respond in RocketChat, but it will take a couple of minutes, so don’t panic when it doesn’t show up right away.
    • The AI_PROMPT comes before the database results when sent to the AI Model. So phrase your AI_PROMPT accordingly.
    • Example: Can you help me understand how much I’ve grown as a person based on the following excerpts from my journal? + DATABASE RESULTS

Full Example Request
#

The following request is getting 10 results from the journal collection related to personal growth from after December 1st 2025 then sending those results and the following prompt to the AI Model:

  • !> journal // personal growth @>:12:1:2025@ // 10 // Can you help me understand how much I’ve grown as a person based on the following excerpts from my journal?

Supported File Types
#

Insightful Search only supports text files (.txt) and Markdown Files (.md)

Prepare Documents For The Date Filter
#

In order for the date filter to work, you must do at least one of the following:

  • Name your documents with the following format: Weekday_Month_Day_Year
    • Examples: Monday_February_23_2026 or Tuesday_January_1_2026
    • You can include st, nd, rd, and th when writing the Day, but it’s not required.
      • Examples: Monday_February_23rd_2026 or Tuesday_January_1st_2026
    • The file extension does not matter here.
  • Add an extra line with the following format: Date:Month_Number/Day/Year
    • Examples: Date:2/23/2026 or Date:1/1/2026
    • Do not abbreviate or shorten the year
    • You can include st, nd, rd, and th when writing the Day, but it’s not required.
      • Date:2/23rd/2026 or Date:1/1st/2026

Using The Date Filter
#

To actually use the Date Filter in your request it must be surrounded with @s and must go after the first // and before the second //. Both of the following are correct and will only return a maximum of 10 documents after January 1st 2026:

  • !> journal // personal growth @>:1:1:2026@ // 10
  • !> journal // @>:1:1:2026@ personal growth // 10

Date Filter Format: @OPERATOR:MONTH:DAY:YEAR@

Each component in the Date Filter should be separated by a :. Those components are:

  • OPERATOR:
    • > will only look for documents from after the given date.
    • < will only look for documents from before the given date
    • = will only look for documents matching the given date.
      • Note: When using this operator, you can replace the day, month, or both with an x. If you replace just the day with an x, you will only get results for the specific month of the specified year. If you replace the day and month with an x, you will only get results from the specified year.
  • MONTH: Just the number that represents the month you want, 1-12.
  • DAY: The day of the month, 1-31. Try not to use invalid dates, like 2/31/2025. I haven’t had a chance to test it, and I don’t know if it breaks anything.
  • YEAR: The full year, not an abbreviation. 2026 and 1990 are valid. 26 and 90 are not valid.