Given system of equations.
2x+y−z=5
2x−5y+λz=μ
x+2y−5z=7
For infinite solution Δ=0=Δ1=Δ2=Δ3
⇒Δ=∣2211−52−1λ−5∣=0
⇒51−3λ=0
⇒λ=17
⇒Δ3=∣5μ72211−52∣=0
⇒−3(μ+13)=0
⇒μ=−13
Now (λ+μ)2+(λ−μ)2
=(17+13)2+(17−13)2
=900+16
=916
Hence this is the correct option.