Given,
The coefficient of x7 in the expansion of (ax2−bx1)13 is equal to the coefficient of x−5 in (ax+bx21)13.
We know that, the general term Tr+1 in the expansion (a+b)n is
Tr+1=Crnan−rbr
Applying to (ax−bx21)13, we get
Tr+1=Cr13(ax)13−r(−bx21)r
⇒Tr+1=(−1)r×Cr13(a)13−r(x)13−3r(b)−r
Therefore, 13−3r=7⇒r=2 for coefficient of x7.
Thus,
T3=C213(b2a11)
Similarly, applying to (ax+bx21)13, we get
Tr+1=Cr13(ax)13−r(bx21)r
⇒Tr+1=Cr13(a)13−r(x)13−3r(b)−r
Therefore, 13−3r=−5 for coefficient of x−5
⇒r=6
So,
T7=C613(a)7(b)−6
Hence, applying the given condition we get
C213(b2a11)=C613(a)7(b)−6
⇒a4b4=C213C613
⇒a4b4=7!⋅6!13!×13!2!⋅11!
⇒a4b4=6×5×4×311×10×9×8
⇒a4b4=22