The results from the supplied Einstein (Metamind) Sentiment model will only be as good as the input data that Salesforce used to create it. If your text varies significantly from the training data you are unlikely to get the expected result. You can't refine this model with more examples.
You could create a new language dataset with the type "text-sentiment". Then you would be free to train it as required based on actual examples.
Einstein Sentiment analysis is currently based on a model that:
"was trained on data aggregated from multiple sources. Mostly short snippets of text and comparable to what you might find in a public forum." Source.
What they mean here is that the
CommunitySentiment model is something that has been pre-created for this task. They collected many examples of text (thousands, hundreds of thousands, millions... ?) and then manually categorized them as being either positive, negative, or neutral. They then used this input data to build the model such that when text like the training data came in a similar response came out. This is known as supervised learning.
Those probabilities are telling you how similar the input text is to the categories that were applied to the training text.
It's probably easier to understand this process with something like Einstein Vision API. Yes, it is imaged rather than text based, but the underlying principle is the same. You train a model on some input data (the dataset). Then use that model to make predictions. The closer the input is to the training data the better the results you'll get. Conversely, when you put something into the model that it hasn't been trained on then you are likely to get a poor result.
The challenge with the Einstein Sentiment Analysis is that you don't get to modify the model with further training data. It's a fixed thing provided by Salesforce. This is great in that you don't need to build up a massive training data set, but it does limit your ability to refine it with actual examples from your specific data.
Here are some slides from my Mad Catter talk that hopefully clarify things:
- You start of with some labels. In the sentiment analysis case that is positive, negative, and neutral. You then collect many example inputs and manually apply the labels to them.
- Now you feed those labels and examples into your AI structured learning tool to produce a model.
- That model only knows about the labels you trained it on. Feed it anything else, and it won't know what to make of it.
- Often internally these models are made up of a number of layers that are then connected by weighted edges. The strength of those edges is altered by the training process so the model produces results as close as possible to the examples.
- When you do give that model a new input its the process through those layers and edges that produces the probabilities. Give if something similar to the training data and you will get a high probability for the corresponding label.
As an aside, it appears that the Sentiment Model in Social Studio can be "tuned".