If the coefficient of x^7 in (ax- 1 bx^2)^13 and the coefficient of x^-5 in (ax+ 1 bx^2)^13 are equal, then a^4 b^4 is equal to:

Given,

The coefficient of ${x}^{7}$x7 in the expansion of ${(a{x}^{2}-\frac{1}{bx})}^{13}$(ax2−bx1​)13 is equal to the coefficient of ${x}^{-5}$x−5 in ${(ax+\frac{1}{b{x}^{2}})}^{13}$(ax+bx21​)13.

We know that, the general term ${T}_{r+1}$Tr+1​ in the expansion ${(a+b)}^{n}$(a+b)n is

$${T}_{r+1}=Crn{a}^{n-r}{b}^{r}$$Tr+1​=Crnan−rbr

Applying to ${(ax-\frac{1}{b{x}^{2}})}^{13}$(ax−bx21​)13, we get

$${T}_{r+1}=Cr13{(ax)}^{13-r}{(-\frac{1}{b{x}^{2}})}^{r}$$Tr+1​=Cr13(ax)13−r(−bx21​)r

$$\Rightarrow {T}_{r+1}={(-1)}^{r}\times Cr13{(a)}^{13-r}{(x)}^{13-3r}{(b)}^{-r}$$⇒Tr+1​=(−1)r×Cr13(a)13−r(x)13−3r(b)−r

Therefore, $13-3r=7\Rightarrow r=2$13−3r=7⇒r=2 for coefficient of ${x}^{7}$x7.

Thus,

$${T}_{3}=C213(\frac{{a}^{11}}{{b}^{2}})$$T3​=C213(b2a11​)

Similarly, applying to ${(ax+\frac{1}{b{x}^{2}})}^{13}$(ax+bx21​)13, we get

$${T}_{r+1}=Cr13{(ax)}^{13-r}{(\frac{1}{b{x}^{2}})}^{r}$$Tr+1​=Cr13(ax)13−r(bx21​)r

$$\Rightarrow {T}_{r+1}=Cr13{(a)}^{13-r}{(x)}^{13-3r}{(b)}^{-r}$$⇒Tr+1​=Cr13(a)13−r(x)13−3r(b)−r

Therefore, $13-3r=-5$13−3r=−5 for coefficient of ${x}^{-5}$x−5

$$\Rightarrow r=6$$⇒r=6

So,

$${T}_{7}=C613{(a)}^{7}{(b)}^{-6}$$T7​=C613(a)7(b)−6

Hence, applying the given condition we get

$$C213(\frac{{a}^{11}}{{b}^{2}})=C613{(a)}^{7}{(b)}^{-6}$$C213(b2a11​)=C613(a)7(b)−6

$$\Rightarrow {a}^{4}{b}^{4}=\frac{C613}{C213}$$⇒a4b4=C213C613​

$$\Rightarrow {a}^{4}{b}^{4}=\frac{13!}{7!\cdot 6!}\times \frac{2!\cdot 11!}{13!}$$⇒a4b4=7!⋅6!13!​×13!2!⋅11!​

$$\Rightarrow {a}^{4}{b}^{4}=\frac{11\times 10\times 9\times 8}{6\times 5\times 4\times 3}$$⇒a4b4=6×5×4×311×10×9×8​

$$\Rightarrow {a}^{4}{b}^{4}=22$$⇒a4b4=22