0

I'm running into an issue that I don't think is a SSJS timeout issue. The default timeout is 30 minutes, and I end up getting "Bad Request" after about 4 minutes.

Basically, I have some code that is iterating through 988 records. The code is performing some math/trig operations to calculate the differences between two locations given the latitude and longitude. When I get the "Bad Request" message, it's about 4 minutes into the code running and it completes around 700 of the 988 records. Has anyone ever ran into something like this before? Should the code run to its max timeout of 30 minutes IF there are no issues with the data being processed?

EDIT

What's even MORE weird is that when the "Bad Request" message happens after about 4 minutes, I'll rerun the code to pick up where it left off, and it will finish up completely fine which tells me that there really is no issue with the data. I have exceptions being output to the browser as you can see, but the only thing that outputs is "Bad Request".

Here's the code:

    <script type="text/javascript" runat="server">
  Platform.Load("core","1.1.1");
  var debug = true;

  
  try {

    var subscribersDE = DataExtension.Init("Subscribers");
    var locationsDE = DataExtension.Init("GroceryStores");
    
    var subscribersFilter = {
            LeftOperand: {
              Property: "location1Name",
              SimpleOperator: "isNull"
            },
            LogicalOperator: "OR",
            RightOperand: {
              Property: "location1Name",
              SimpleOperator: "equals",
              Value: " "
            }
          }

    // filteredSubscribers is the potential subset of subscribers to operate on
    // if a record already has "closest location" data, we don't want to operate
    // on them.
    var filteredSubscribers = subscribersDE.Rows.Retrieve(subscribersFilter);
    
Write("filteredSubscribers count = " + filteredSubscribers.length + "</br>");
// currently times out after 300 or so subscriber count below only for CloudPages
// which is why I set it to iterate to 100
    
    for (var i = 0; i < filteredSubscribers.length; i++)
    {
      // iterate through subscribers that do not have location1Name value
      var sub = filteredSubscribers[i];
      var subKey = sub["SubscriberKey"];
      var subLat = sub["Latitude"];
      var subLong = sub["Longitude"];
      var temp = [];
      
Write("subscriber iteration # " + i + "</br>");      
      
      var locationsFilter = {
        LeftOperand: {
          Property: "Latitude",
          SimpleOperator: "greaterThan",
          Value: subLat-0.5
        },
        LogicalOperator: "AND",
        RightOperand: {
          Property: "Latitude",
          SimpleOperator: "lessThan",
          Value: subLat+0.5
        }
      }

      // retrieve function only returns max 2500 records, so we need to limit to less than that
      var locationsDERows = locationsDE.Rows.Retrieve(locationsFilter);
      
//Write(Stringify(locationsDERows));
      
Write("locationsDERows Count = " + locationsDERows.length + "</br>");
      
      for (var j = 0; j < locationsDERows.length; j++)
      {
        var loc = locationsDERows[j];
        var locLat = loc["Latitude"];
        var locLong = loc["Longitude"];

        var R = 6371000; // radius of Earth in meters
        var φ1 = subLat * Math.PI/180; // φ, λ in radians
        var φ2 = locLat * Math.PI/180;
        var Δφ = (locLat-subLat) * Math.PI/180;
        var Δλ = (locLong-subLong) * Math.PI/180;

        var a = Math.sin(Δφ/2) * Math.sin(Δφ/2) +
                  Math.cos(φ1) * Math.cos(φ2) *
                  Math.sin(Δλ/2) * Math.sin(Δλ/2);
        var c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a));

        var d = R * c; // in meters
        var distanceInMiles = d * 0.000621371;  
        
        var obj = {};
        
        obj['distance'] = distanceInMiles;
        obj['storeName'] = loc['StoreName'];
        obj['address'] = loc['Address'];
        obj['city'] = loc['City'];
        obj['state'] = loc['State'];
        obj['zip'] = loc['Zip'];
        
        temp.push(obj); 
      }

      temp.sort(function (a, b) {
        return a.distance - b.distance;
      });

  
      
      subscribersDE.Rows.Update({
        'location1Name':temp[0].storeName + " " + temp[0].address + " " + temp[0].city + ", " + temp[0].state + " " + temp[0].zip,
        'location1Distance':temp[0].distance,
        'location2Name':temp[1].storeName + " " + temp[1].address + " " + temp[1].city + ", " + temp[1].state + " " + temp[1].zip,
        'location2Distance':temp[1].distance,
        'location3Name':temp[2].storeName + " " + temp[2].address + " " + temp[2].city + ", " + temp[2].state + " " + temp[2].zip,
        'location3Distance':temp[2].distance
      },['SubscriberKey'],[subKey]);
    }
    
  }
  catch(e) {
    var errorMsg = Stringify(e)
    Variable.SetValue("@err",errorMsg);
    Write(errorMsg);
  }
</script>
  • 2
    Are you running this in a CloudPage? If so, they time out after two minutes. In any case, SSJS isn't probably the most efficient way of doing this as all the maths functions you need are available to SQL Query Activities and scale far better than something like an SSJS activity. Adam did a nice article here: sprignaturemoves.com/geolocation-in-sfmc – Macca Jul 24 at 4:41
  • ...and if you cannot use sql, then make use of a batching logic. In all your records, use a field "processed" and set it to "false": Make your script take 200 records WHERE processed == false (or whatever number worked within the timeout window; you could even put this number in a DE and look it up for later potential changes), process them, then set their flag "processed = true". Then call the same page again. That way it iterates through without ever "taking on too much at once". – Jonas Lamberty Jul 24 at 7:10
  • As you all know, I am a huge fan of SFMC SSJS, but I have to agree that SQL would be the better language for large scale data manipulation. Although there may be ways to increase performance on your script, but in comparison to risk and processing speed, using SSJS is the inferior option. Also, as stated above, the timeout on a webpage is usually dependent on the browser, but is around 2 minutes - so you should only really use it for debugging - large scale needs to be inside of a script activity. – Gortonington Jul 24 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.