(A+B)2=A2+AB+BA+B2
Since matrix multiplication is not commutative, AB=BA in general, so the cross terms cannot be combined.
Given (A+B)2=A2+B2, this means:
AB+BA=O
AB=[12−1−1][ab1−1]
=[a−b2a−b23]
BA=[ab1−1][12−1−1]
=[a+2b−2−a−1−b+1]
AB+BA=[2a−b+22a−21−a4−b]=[0000]
From entry (1,2):
1−a=0
a=1
From entry (2,2):
4−b=0
b=4
From entry (2,1):
2(1)−2=0 ✓
From entry (1,1):
2(1)−4+2=0 ✓
a=1,b=4