Binomial Theorem

As we know that, coefficient of x11 in the expansion of (1+x)11 is ‌11C11.
Now, we have to find coefficient of x11 in the expansion of (1+αx+βx2)(1+x)11. When we choosing 1 from (1+αx+βx2) then we need x11 from (1+x)11.
So, coefficient =‌11C11=1
Similarly, we choosing αx from (1+αx+βx2) then we need x10 from (1+x)11= Coefficient =α×‌11C10=11α and when we choosing βx2 from (1+αx+βx2) then we need x9 from (1+x)= Coefficient =β×‌11C9=55⋅β
∴ Coefficient of x11 in the expansion of (1+αx+βx2)
(1+x)11=1+11α+55β=144‌ ( given ) ‌
Just, in similar way, we have to find coefficient of x10 such that
11+55α+165β=396
On solving Eqs. (i) and (ii), we get;
α=−2‌ and ‌β=3
∴α2+β2=(−2)2+(3)2=13