Checkable Forced Authority by instruction/category
Fleur Nouwens
Great to hear your perspective, Dion van Communicatie! I have a few more questions for you:
- What specific scenarios or use cases do you envision where forcing authority on an instruction would be necessary?
- How frequently do you anticipate needing to use the 'Force Authority' feature in your operations?
- Are there any potential risks or downsides you foresee with implementing a 'Force Authority' option for instructions?
D
Dion van Communicatie
Hi Fleur Nouwens,
Thank you for your response and your thoughtful questions — I really appreciate that.
- Our website contains hundreds of content-rich pages that we deliberately want to scrape, given the valuable information they hold. However, some of these pages include specific phrases that, when taken out of context, can lead to undesirable chatbot responses. In such cases, we apply crystal-clear instructions as a workaround. Still, since both the scraper and the instructions carry equal authority, the chatbot may occasionally default to generating its own (undesired) response rather than following the instruction.
- Each week, we create new instructions and retire outdated ones, based on real conversations that reveal problematic responses — which, by the way, makes Watermelon a fantastic feedback tool in itself. This is feasible because we've allocated dedicated capacity for instruction management. The website is also actively maintained, but due to its scale, the update cycle is structured differently. Moreover, it's not always a matter of keeping information up to date. You mentioned scenarios: there are situations in which you want almost all content on a page to be used by the chatbot — but not all of it. This happens more frequently in our organization than you might expect, which is why we optimize our instructions on a weekly basis. Without Forced Authority, though, we’re at the mercy of the chatbot’s cherry picking.
- Naturally, we understand that both the webpage and the associated instructions need to remain current. Active content and instruction management are essential for that, and we've ensured both are in place. A potential improvement could be to show a warning message when enabling Force Authority, such as: "Please note: when you enable Force Authority, this instruction will take precedence over the existing website content. Even if the webpage is later updated, the instruction remains authoritative. Only use Forced Authority if you have the capacity to regularly review, update, and optimize your instructions."
D
Dion van Communicatie
Normally, Domain Knowledge is the central brain and instructions will only be additional. That is why Domain Knowledge has a higher authority.
Often, instructions are followed less strictly when there is overlap with the web crawler or document scraper because they have the same level of authority.
But in some cases, you definitely want to force a specific instruction to be strictly followed, regardless of the fact that it's “just” an instruction.
In that case, the “Force Authority” checkbox is welcome. When checked, the instruction's authority is “forced”.
This is a user-friendly solution for continuing to distinguish instructions and Domain knowledge while providing specific authority in certain cases.