Let X=[ … ],Y=α l+β X+γ X^2 and Z=α ^2I-α β X+(β ^2-α γ )X^2,α ,β ,γ ∈ mathbbR. If Y^-1=[ … ], then (α -β +γ )^2 is equal to .

Given, $X=[\begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{matrix}]$X=[000​100​010​], so ${X}^{2}=[\begin{matrix}0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0\end{matrix}]$X2=[000​000​100​]

Now, finding $Y=\alpha l+\beta X+\gamma {X}^{2}$Y=αl+βX+γX2 &$Z={\alpha }^{2}I-\alpha \beta X+({\beta }^{2}-\alpha \gamma ){X}^{2}$Z=α2I−αβX+(β2−αγ)X2 by putting the value of X&{X}^{2} we get,

Y=[\begin{matrix}\alpha & \beta & \gamma \\ 0 & \alpha & \beta \\ 0 & 0 & \alpha \end{matrix}]&Z=[\begin{matrix}{\alpha }^{2} & -\alpha \beta & {\beta }^{2}-\alpha \gamma \\ 0 & {\alpha }^{2} & -\alpha \beta \\ 0 & 0 & {\alpha }^{2}\end{matrix}]

We know that $Y\cdot {Y}^{-1}=I$Y⋅Y−1=I

$$\Rightarrow [\begin{matrix}\alpha & \beta & \gamma \\ 0 & \alpha & \beta \\ 0 & 0 & \alpha \end{matrix}][\begin{matrix}\frac{1}{5} & \frac{-2}{5} & \frac{1}{5} \\ 0 & \frac{1}{5} & \frac{-2}{5} \\ 0 & 0 & \frac{1}{5}\end{matrix}]=[\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{matrix}]$$⇒[α00​βα0​γβα​][51​00​5−2​51​0​51​5−2​51​​]=[100​010​001​]

$$\Rightarrow [\begin{matrix}\frac{\alpha }{5} & \frac{-2\alpha }{5}+\frac{\beta }{5} & \frac{\alpha }{5}-\frac{2\beta }{5}+\frac{\gamma }{5} \\ 0 & \frac{\alpha }{5} & \frac{-2\alpha }{5}+\frac{\beta }{5} \\ 0 & 0 & \frac{\alpha }{5}\end{matrix}]=[\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{matrix}]$$⇒[5α​00​5−2α​+5β​5α​0​5α​−52β​+5γ​5−2α​+5β​5α​​]=[100​010​001​]

On comparing L.H.S and R.H.S we get,

$$\Rightarrow \frac{\alpha }{5}=1\Rightarrow \alpha =5$$⇒5α​=1⇒α=5

$$\Rightarrow -\frac{2}{5}\alpha +\frac{\beta }{5}=0\Rightarrow \beta =10$$⇒−52​α+5β​=0⇒β=10

$$\Rightarrow \frac{\alpha }{5}-\frac{2\beta }{5}+\frac{\gamma }{5}=0\Rightarrow \gamma =15$$⇒5α​−52β​+5γ​=0⇒γ=15

So, ${(\alpha -\beta +\gamma )}^{2}={(5-10+15)}^{2}=100$(α−β+γ)2=(5−10+15)2=100