Adding/removing records in a child relationship list

Question

I am passing a list of SObjects with child relationships to a Lightning Web Component. If I query and pass my query results back to my LWC component directly, this works.

In Apex:

return [
    SELECT Id, (SELECT Id, LastName FROM Contacts)
    FROM Account
];

In LWC, I get a nice, nested JavaScript object:

[
  {
    Id: "0012D00000PDWKAQA5", 
    Contacts: {
      Id: '0032D00000Ni9yZQAR',
      LastName: 'Test' 
    } 
  },
  ...
]

Now I have a requirement that involves adding and removing child records before returning the response to my LWC component. But I'm noticing some weird behavior when I try to manipulate the child relationship, which I had assumed was just a List type. When I use add(), nothing is added. When I use remove(), nothing is removed. And yet, no exception is thrown.

For example, you can run this in Execute Anonymous:

Account acc = [
    SELECT Id, (SELECT Id, LastName FROM Contacts)
    FROM Account
    WHERE Id IN (SELECT AccountId FROM Contact)
    LIMIT 1
];
Integer contactNum = acc.Contacts.size();
acc.Contacts.add(new Contact(LastName = 'Test'));
// THIS FAILS
System.assertEquals(contactNum + 1, acc.Contacts.size());

...or...

...
acc.Contacts.remove(0);
// THIS FAILS TOO
System.assertEquals(contactNum - 1, acc.Contacts.size());

I tried working around this by setting acc.Contacts directly, e.g. acc.Contacts = new List<Contact>();. This returns an exception: "Field is not writeable: Account.Contacts"

Any help?