Configuring Semantic filters

AutoMod Intent Designer is an AI-based moderation tool that can detect and action incoming messages in real time, based on their meaning and intent. Intent Designer does not require any software development, data science or machine learning skills to use. Instead, you provide Intent Designer with a list of example phrases corresponding to the meaning and intent you want to detect, and it will detect messages with similar meaning and intent to those example phrases, even across languages. You can change the phrases at any point through the moderation dashboard, and the changes will be reflected almost immediately.

Intent Designer allows Trust & Safety specialists, Content Moderators, Community Managers, and other moderation professionals to implement their specific, fine-grained content moderation policies in an automated manner, without having to resort to either costly and time consuming training of machine learning models, cumbersome Regular Expressions or broad word filters.

Intent Designer is designed to moderate based on the intent, rather than pattern matching or exact word patches, and makes it simple for moderators to autonomously update and action changes in their organisation’s moderation policies and react to emerging challenges as soon as they appear.

Navigate to moderation.getstream.io . If you do not have access to this dashboard, speak with your Customer Success Representative at Stream, your Sales Engineer or Account Manager.

If your organisation has multiple applications, make sure you have selected the application you want to test Intent Designer on. You can do this from the Application dropdown at the top of the page:

From here, navigate to ‘Semantic Filters’

You should arrive on the following page:

Since there are no filters set up, lets add a new one. To do so, we need to add a new phrase list:

Which takes us to the following interface:

A Phrase List is a group of phrases that serve as examples of the intent and meaning that you want to detect in incoming messages.

Phrases in a Phrase List should be at least 5 words long, and no longer than 120 words. Intent Designer works best when the phrases provided have clear meaning and intent, and are legible sentences with meaning. Short phrases and single words will still result in detections, but generally lack enough information to convey specific meaning and intent.

✅ Examples of Good Seed Phrases:

Darth Vader is Luke Skywalker’s Father I am a freight provider. Contact me if you are interested in my services. Hello dear if you are interested in buying the item I can do a discount for you Hi everyone feel free to connect with me on LinkedIn via my profile Does anyone have the answer to the third question? we have clients looking for those vehicles, do you want help selling them?

❌ Examples of bad Seed Phrases:

Yes Darn Bieber vip guests classified ads marketplace

In this example, we’ll set up a seed phrase to detect a specific set of spoilers across a wide range of classical media.

The seed phrases we have used in this example are:

1. Darth Vader is Luke's father

2. Romeo and Juliet die at the end

3. The Titanic sinks

4. The ring is destroyed

5. Batman is Bruce Wayne

6. Snape kills Dumbledore

A name for your phrase list will be automatically generated. It is advised that you provide a name of your own. You should pick a name that reflects the behaviour and/or intent that a phrase list is designed to detect. This will make managing multiple phrase lists easier. In our example, we will name the phrase list ‘Spoiling Classical Media’

You can have up to 3 Phrase Lists running on the same channel type.

At this point, you can ‘submit’ the phrase list and it will be saved, but it won’t be active. To make a phrase list active, you need to assign it to a channel type.

Once a phrase list is assigned to a channel type, it will begin filtering all incoming messages sent in channels of that channel type.

To assign the phrase list to a channel type, select from the channel types in the dropdown:

You can assign a phrase list to as many channel types as you want.

In the above example, we have assigned the ‘Spoiling Classical Media’ phrase list to the commerce channel type.

Press submit to activate the phrase list. This will save any changes made, and activate the phrase list on the selected channel types. Once submitted, it should appear on the Semantic Filters page:

Now we can test messages against the phrase list.

Open the app in which the phrase list is active on your device or browser.

In the below example, we are using a demonstration application which is available at this link. The Spoiling Classical Media phrase list is active in the ‘Semantic Filters: Spoilers’ channel

To test out the filtering, we will send a few messages.

The first message is an identical match to one of the seed phrases in the ‘Spoiling Classical Media Phrase List:

Darth Vader is Luke's father

Which is caught by the filter. Lets try something a little different, but with similar meaning and intent:

the Jedi Knight Skywalker was sired by the sith lord vader

This is also caught.

To demonstrate Intent Designer’s cross-lingual functionality, let’s send the following message in Arabic:

لوك إبن دارث فايدر

Which translates to ‘Luke is Darth Vader’s son’. The implied meaning is the same as ‘Darth Vader is Luke’s Father’, but the explicit meaning is not:

Again, this is captured.

When a message is captured by Intent Designer or by any of the AutoMod Engines, the default behaviour is it is ‘bounced’ back to the user the first time it is sent, then flagged. in practice, this means the message is errored out, and not sent to other users the first time the user tries to send it, and in subsequent attempts is sent and flagged. This error can be hooked into quite easily, and allows for much customisation and for various flows to be implemented with very little code on the client side.

You can set a different ‘action’ for Intent Designer at the channel type level. For each channel type you can select an action that will happen whenever a message that matches a phrase in a phrase list that is active on that channel type is sent:

1. Block: Blocks the message

2. Flag: Sends the message and flags it for review

3. Bounce: Bounces the message every time it is sent

4. Bounce, then Flag: Bounces the message the first time a user sends it in a channel, then flags it (default flow)

5. Bounce, then Block: Bounces the message the first time a user sends it in a channel, then blocks it.

In the above demonstration app, we have expanded upon the default functionality provided in the Stream SDKs when it comes to error handling, to give the user the option to edit the message or send it anyway.

Other customisation examples that you can implement include:

• Provide ‘behavioural nudges’ to users, with a prompt or modal appearing that educates the user on the rules of the platform, and encourages (or warns them) to stick to them. Real time behavioural nudges can reduce moderation workloads by up to 40%!

• Hiding messages that are flagged by Intent Designer from other users, until they are reviewed by a moderator.

• Informing other users in the channel that the message that has been sent might break the rules, and asking them to report it if it does.

So far, what we’ve done is:

1. Set up a phrase list

2. Enable it on a channel type

3. Test some messages in a channel of that channel type

But we haven’t seen how messages caught by AutoMod in general, and Intent Designer in particular are flagged for review by a moderator.

This guide so far demonstrates the first part of the ‘bounce flow’ that is the default flow for messages moderated by AutoMod.

By default, messages that are caught by an AutoMod engine, including Intent Designer, are handled according to the following logic:

1. Has this user sent this exact message in this channel in the last 5 minutes?

2. No, they haven’t: If not, then check it against the applicable AutoMod engines. If it is caught by an applicable AutoMod engine, then bounce the message, and do not send it to other users. If the message is not caught by an applicable AutoMod engine, then send it through.

3. Yes, they have: If so, then has it been caught by an applicable AutoMod engine? If it has, then send the message through and flag it. If not, then send the message through and do not flag it.

In the above examples, we sent each message only once. To demonstrate what happens when a message is flagged by Intent Designer, we will send one of the messages twice in a row.

Going back to the previous example, try sending a message that is caught by the AutoMod filter twice in a row. The first time it should error out, but the second time (if sent within 5 minutes of the first attempt), it should go through:

This message now shows up in the moderation dashboard, in the ‘Messages’ section:

From here, you can view more details about the message, see in in the context of the channel it was sent in, and take action against the user who sent it, such as temporary or permanent bans.

1. single words phrases can give unexpected behaviour

2. phrases with unrelated meanings in them confuse the model (e.g. This is a pretty sweater. I don’t like that bag)

3. negations are not well handled (e.g. This is a pretty sweater is similar to this is an ugly sweater)

4. Intent Designer, like Data, has issues understanding the difference between hate and love, good and bad, and other relatively subjective measures that fleshy humans are fond of. Thus, Intent Designer does not see any difference between statements such as ‘I love Justin Bieber’ and ‘I hate Justin Bieber’, or ‘Transformers movies have outstanding character development’ and ‘Transformers movies have terrible character development’. </aside>

One of our B2B marketplace customers had issues with shipping providers on their platform. So we used the following seedphrase:

We are a shipping provider. Contact us to buy our services.

Which captured messages like this:

Greetings. This is Alessandro from Huge LLC. I am a broker and I would like to offer our services. We have contact carriers and we can move your shipments with cheaper rates. If you are intersted in working with us then this is my cell phone number. xxxxxxxx

and:

Good morning,My name is Cara and I am a freight broker with over 10 years of experience moving LTL and FTL shipments in the United States and Canada.I am reaching out today to learn how I can provide your company or at least your division with shipping services. My success thus far has been largely due to my customer service, relationship with reliable carriers and transparent communication. I aim for 100% satisfaction on each load.I would love an opportunity on your next shipment. Are you able to make shipping decisions?Thank you,