Your Highlight app should expose a suggestions prompt, a Handlebars template that will offer the user suggestions about what your app can accomplish in Highlight. This Handlebars template should include a system and user message which will be used to generate a JSON array of suggestions.

​
Implementation

Expose an HTTP GET endpoint through your web app that will return raw text. This text should be a Handlebars template that uses our custom {{#system}} and {{#user}} Handlebars helpers. These helpers define which messages are system messages and which are user messages.

Generally, you should use the system message for instructions and the user message for context. When executed through an LLM, your template should always return a JSON array of suggestions. Highlight developers have found using this language towards the end of your prompt will yield the best results:

Format the tasks in a JSON array like this: [task1,task2,task3,task4,task5].

Tell Highlight where your endpoint can be located by adding the endpoint to the Developers Portal.

​
Testing

You’ll want to extensively test your suggestions prompt to ensure it’s working as expected. Follow our Developing Locally guide for more information, instead of using the URL specified through the Developers Portal, Highlight will listen for your suggestions prompt at http://localhost:3000/suggestions.hbs.

​
Example

For example, your app may offer a list of suggestions like:

  • For a code debugger app: Help me debug this problem
  • For a dating app: Generate dating advice based on your open conversation
  • For a travel advisor: Show me places I may want to visit

These suggestions will be shown through the Highlight floating window UI. When the user “activates” a suggestion by pressing it, they will be taken to your app. The suggestion line item will be sent through an event. See the App Runtime Guide for further information.

Here’s what your suggestion template could look like:

{{! suggestions.hbs }}
{{#system}}
  List the 5 most insightful and impressive tasks that an LLM like
  {{currentApp}}
  could complete for someone with the below screen activity. Tasks should be
  completely achievable within an LLM, based on the information provided. Write
  each task as a concise sentence, maximum 15 words long, without periods at the
  end. Tasks should lean into the unique strengths of
  {{currentApp}}
  versus other LLMs. Format the tasks in a JSON array like this:
  [task1,task2,task3,task4,task5].
{{/system}}

{{#user}}
  # SCREEN ACTIVITY:
  {{#if application.focusedWindow.selectedText}}
    The text that is selected right now:
    {{application.focusedWindow.selectedText}}
  {{/if}}
  {{#if application.focusedWindow.title}}
    The name of the window that is open:
    {{application.focusedWindow.title}}
  {{/if}}
  {{#if application.focusedWindow.domain}}
    The name of the domain on the browser window that is open:
    {{application.focusedWindow.domain}}
  {{/if}}
  {{#if environment.ocrScreenContents}}
    The contents of what is being looked at right now:
    {{environment.ocrScreenContents}}
  {{/if}}
  {{#if application.focusedWindow.rawContents}}
    The contents of everything else on the screen:
    {{application.focusedWindow.rawContents}}
  {{/if}}
  {{#if audioRecent}}
    What I have said and heard recently:
    {{audioRecent}}
  {{/if}}
  {{#if environment.clipboardText}}
    The text I have copied on my clipboard currently:
    {{environment.clipboardText}}
  {{/if}}
  {{#if factsAboutMe}}
    Other facts about the person:
    {{factsAboutMe}}
  {{/if}}
{{/user}}

The Handlebars prompt file will be injected with the HighlightContext type defined in the App Runtime package.

​
Production Example

Our demo app hosts a suggestions prompt at https://highlight-demo-app.vercel.app/suggestions.hbs.

Developing LocallyAuthentication