For k∈ mathbbR, let the solutions of the equation cos( sin^-1(x cot( tan^-1( cos( sin^-1x)))))=k,0<|x|< 1 √2 be α and β, where the inverse…

Given,

$$\mathrm{cos}({\mathrm{sin}}^{-1}(x\mathrm{cot}({\mathrm{tan}}^{-1}(\mathrm{cos}({\mathrm{sin}}^{-1}x)))))=k$$cos(sin−1(xcot(tan−1(cos(sin−1x)))))=k

Now simplifying $\mathrm{cos}({\mathrm{sin}}^{-1}x)=\mathrm{cos}({\mathrm{cos}}^{-1}\sqrt{1-{x}^{2}})=\sqrt{1-{x}^{2}}$cos(sin−1x)=cos(cos−11−x2​)=1−x2​

So, $\mathrm{cos}({\mathrm{sin}}^{-1}(x\mathrm{cot}({\mathrm{tan}}^{-1}(\mathrm{cos}({\mathrm{sin}}^{-1}x)))))=k$cos(sin−1(xcot(tan−1(cos(sin−1x)))))=k

becomes $\mathrm{cos}({\mathrm{sin}}^{-1}(x\mathrm{cot}({\mathrm{tan}}^{-1}\sqrt{1-{x}^{2}})))=k$cos(sin−1(xcot(tan−11−x2​)))=k

And now solving $\mathrm{cot}({\mathrm{tan}}^{-1}\sqrt{1-{x}^{2}})={\mathrm{cotcot}}^{-1}(\sqrt{\frac{1}{\sqrt{1-{x}^{2}}}})=\frac{1}{\sqrt{1-{x}^{2}}}$cot(tan−11−x2​)=cotcot−1(1−x2​1​​)=1−x2​1​

So, $\mathrm{cos}({\mathrm{sin}}^{-1}(x\mathrm{cot}({\mathrm{tan}}^{-1}\sqrt{1-{x}^{2}})))=k$cos(sin−1(xcot(tan−11−x2​)))=k becomes

$$\mathrm{cos}({\mathrm{sin}}^{-1}(\frac{x}{\sqrt{1-{x}^{2}}}))=k$$cos(sin−1(1−x2​x​))=k

Now solving $\mathrm{cos}({\mathrm{sin}}^{-1}(\frac{x}{\sqrt{1-{x}^{2}}}))=\frac{\sqrt{1-2{x}^{2}}}{\sqrt{1-{x}^{2}}}$cos(sin−1(1−x2​x​))=1−x2​1−2x2​​

So, $\frac{\sqrt{1-2{x}^{2}}}{\sqrt{1-{x}^{2}}}=k$1−x2​1−2x2​​=k

$$\Rightarrow 1-2{x}^{2}={k}^{2}(1-{x}^{2})$$⇒1−2x2=k2(1−x2)

$$\Rightarrow ({k}^{2}-2){x}^{2}={k}^{2}-1$$⇒(k2−2)x2=k2−1

$$\Rightarrow {x}^{2}=\frac{{k}^{2}-1}{{k}^{2}-2}$$⇒x2=k2−2k2−1​

So, roots are $\alpha =\sqrt{\frac{{k}^{2}-1}{{k}^{2}-2}}\Rightarrow {\alpha }^{2}=\frac{{k}^{2}-1}{{k}^{2}-2}$α=k2−2k2−1​​⇒α2=k2−2k2−1​

And $\beta =-\sqrt{\frac{{k}^{2}-1}{{k}^{2}-2}}\Rightarrow {\beta }^{2}=\frac{{k}^{2}-1}{{k}^{2}-2}$β=−k2−2k2−1​​⇒β2=k2−2k2−1​

Now finding $\frac{1}{{\alpha }^{2}}+\frac{1}{{\beta }^{2}}=2(\frac{{k}^{2}-2}{{k}^{2}-1})$α21​+β21​=2(k2−1k2−2​) and $\frac{\alpha }{\beta }=-1$βα​=−1

So, sum of roots of ${x}^{2}-bx-5=0$x2−bx−5=0 will be $=\frac{1}{{\alpha }^{2}}+\frac{1}{{\beta }^{2}}+\frac{\alpha }{\beta }=b$=α21​+β21​+βα​=b

$$\Rightarrow \frac{2({k}^{2}-2)}{{k}^{2}-1}-1=b\cdots (1)$$⇒k2−12(k2−2)​−1=b⋯(1)

Product of roots of ${x}^{2}-bx-5=0$x2−bx−5=0 will be $=(\frac{1}{{\alpha }^{2}}+\frac{1}{{\beta }^{2}})\frac{\alpha }{\beta }=-5$=(α21​+β21​)βα​=−5

$$\Rightarrow \frac{2({k}^{2}-2)}{{k}^{2}-1}(-1)=-5$$⇒k2−12(k2−2)​(−1)=−5

$$\Rightarrow 2{k}^{2}-4=5{k}^{2}-5$$⇒2k2−4=5k2−5

$$\Rightarrow 3{k}^{2}=1\Rightarrow {k}^{2}=\frac{1}{3}\cdots \mathrm{Put}\mathrm{in}(1)$$⇒3k2=1⇒k2=31​⋯Putin(1)

$$\Rightarrow b=\frac{2({k}^{2}-2)}{{k}^{2}-1}-1=5-1=4$$⇒b=k2−12(k2−2)​−1=5−1=4

$$\frac{b}{{k}^{2}}=\frac{4}{\frac{1}{3}}=12$$k2b​=31​4​=12