Converting Set to List creates duplicates of my Sobjects

Question

I am adding manipulated Accounts to a Set multiple times and update it. It fails due to a Duplicate id in list exception. What should be impossible, because the values are coming from a set. After some research I think it is tightly coupled to Can I use set instead of List to resolve the duplicate id issue

But I feel like the hashes should be updated since I am working on the same account reference meaning the account I add is the exact same that it already contains. The fact that the set has a size of 1 underlines this, converting it to a list I get a size of 2. What is definitely a bug and can't be explained with hashes.

Account a = new Account(Name='a');
insert a;

Set<Account> accountSet = new Set<Account>();
accountSet.add(a);
a.BillingCity = 'Foo'; // this makes it fail later
accountSet.add(a);

update new List<Account>(accountSet);

Line: 9, Column: 1 System.ListException: Duplicate id in list: 0015400000DuAR7AAN

So I started debugging it. The weirdest thing about it is, that if I debug the set once it gets recalculated and cleaned up internally and it doesn't fail anymore.

System.debug(accountSet.size()); // 1
// System.debug(accountSet); // Comment in and it stops failing

List<Account> accountList = new List<Account>(accountSet);

System.debug(accountList.size());  // 2 
System.debug(accountList); // two equivalent accounts

Converting the list to a map to get it's unique values resulted in this exception:

System.ListException: Row with duplicate Id at index: 1

Summary

If I would not do any dml I would never recognize that there is a duplicate value in my list, what might lead to hardly traceable bugs.

It seems to be only related to SObjects (custom and standard) since I was not able to reproduce it with a class.

I just wanted to share this with the community hoping to save you a couple hours of debugging and hope we can manage a fix soon. I guess the simplest workaround so far is using a Map as Adrian described here.

Answer 1

Map and Set are internally based on HashMap and HashSet, respectively. In these classes, there are "buckets" created for each hashCode, and within each bucket, you have a list of values that each have the same hashCode but where equals return false.

For performance reasons, these buckets are only calculated when calling add or addAll. Altering the hashCode of an object in the Map key or in a Set will cause that object to be "not found" until the internal state is reset. You can do this with System.debug, although you should not rely on that behavior being reliable.

This isn't so much a bug as it is an implementation detail. Modifying the key or a value in a set such that the hashCode changes is not supported. Attempting to do so is undefined behavior, and usually will cause issues. This is very specifically called out in the documentation:

---

If the object in your map keys or set elements changes after being added to the collection, it won’t be found anymore because of changed field values.

---

Keep in mind the following when implementing the equals method. Assuming x, y, and z are non-null instances of your class, the equals method must be:

Reflexive: x.equals(x)

Symmetric: x.equals(y) should return true if and only if y.equals(x) returns true

Transitive: if x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) should return true

Consistent: multiple invocations of x.equals(y) consistently return true or consistently return false

For any non-null reference value x, x.equals(null) should return false

---

Keep in mind the following when implementing the hashCode method.

If the hashCode method is invoked on the same object more than once during execution of an Apex request, it must return the same value.

If two objects are equal, based on the equals method, hashCode must return the same value.

If two objects are unequal, based on the result of the equals method, it is not required that hashCode return distinct values.

---

As you can see, the bits on hashCode are very specific: each object must return the same hashCode each time, two objects that return true for equals must also return the same hashCode.

Modifying an Sobject key is a specific violation of the rules, and the behavior is undefined/not supported.

---

Edit: Additional note: There is a related bug where if you change your Apex debug log levels high enough, the map/set will work correctly, but when you turn it back down to normal debug/none levels, it will stop working. This bug does make it more challenging to troubleshoot the problem, since our first instinct is to bump up the debug log levels.

Here's a functional example for you:

public class DemoValue {
    public Integer code = 0;
    public Boolean equals(Object o) {
        return this.code == ((DemoValue)o).code;
    }
    public Integer hashCode() {
        return code;
    }
}

---

With the following execute anonymous script:

Set<DemoValue> demos = new Set<DemoValue>();
DemoValue d1 = new DemoValue(), d2 = new DemoValue();
demos.add(d1);
d1.code = 5;
d2.code = 5;
demos.add(d2);
System.debug(demos.size());

If you use Apex debug level "DEBUG", you'll see "2" as the output, but if you use "FINER" or higher, you'll see "1" as the output. You'll want to read the background information on why this happens over on this question.