0

I am trying to test out the path optimizer in JB. There are 7 records in my DE for testing purpose (though I know it's very small number). In my setting, I set 60% for targeting ( around 4 records) and 40% for holdback (around 3 records). Of which the 60% targeting, 50% for path A (2 records) and 50% for path B (2 records). However, the actual split done is path A (1 record) and path B (4 records). Does anyone has any clue of the disproportionate of 50%-50% split for path optimizer?

path optimizer

2 Answers 2

2

The answer lies in your question itself i.e., (though I know it's very small number)

The short answer/reason is:

If you are not seeing the expected distribution, increase your sample size.

More details here --> https://help.salesforce.com/s/articleView?id=000333919&type=1

Within the Marketing Cloud May Release, Journey Builder released a new feature called the Path Optimizer that can also be considered for split testing in the Marketing Cloud. Note, the same logic applies to Path Optimizer. If you are not seeing the expected distribution, increase your sample size.

0

So, @VishalKumarCV is correct in the solution that your sample size is small, but the reason for this is one that is not always intuitive. So when a split is made, the percentages you see are not a guarantee, think of it more as a coin toss or a dice roll.

If you used a 6 sided dice, you would have around a 16.6% chance of getting a 1. So if you roll this die 100 times, does this guarantee you will get 16 or 17 of number 1? No it does not. Percentage is just an estimate of possibility of getting that number, the same is true with the split.

The split percentages you give are based on random assignment to see. For this example, let's assume the Split activity uses NTile() from SQL. NTile() will randomly assign a number from 1 to the max value you set in the property. So for instance if you have a 60/40 split, you would want to use an NTile(10) in a subquery and then use a CASE statement to assign to the right bucket, so for Path 1 (60%) you would want to do something like myField IN (1,2,3,4,5,6) and then myField IN (7,8,9,10) for the Path 2 (40%).

Now, the number generated is random, so it could output the number 9 50 times out of your 100 records, greatly skewing the percentage split numbers. This is why at a higher volume, that margin of error gets greatly reduced and becomes much more accurate. For example, say you only have 10 records and you push them through the above split - if you get even one that falls outside of the expected split, you are looking at a 10% shift in your split percentage. (70/30 or 50/50 etc)

1
  • 1
    Thanks Gregory, so much for explaining and sharing the detailed explanation :) Commented Oct 19, 2021 at 13:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .