Given,
r×b=c×b
⇒r×b−c×b=0
(r−c)×b=0
Hence, we can say that r−c is parallel to vector b,
Now let , r−c=λb
⇒r=c+λb......(1)
∵r⋅a=0⇒c⋅a+λ(b⋅a)=0
⇒λ=b⋅a−c⋅a=5−8
Now putting the value of λ in equation (1) we get,
r=c−58b=(5i^−3j^+3k^)−58(i^−j^+2k^)
⇒5r=17i^−7j^−k^
⇒25∣r∣2=(17)2+(−7)2+(−1)2
⇒25∣r∣2=289+49+1=339