Let $y=\left({\cot }^{-1}x\right)\left({\cot }^{-1}\right(-x\left)\right)$ and range of $y\in (0,\frac{a{\pi }^2}{b}]$, then the value of $a+b$ is

The correct option is C $5$
Given : $y=\left({\cot }^{-1}x\right)\left({\cot }^{-1}\right(-x\left)\right)$
$\Rightarrow y={\cot }^{-1}x[\pi -{\cot }^{-1}(x\left)\right]$
As ${\cot }^{-1}\left(x\right)>0,(\pi -{\cot }^{-1}(x\left)\right)>0$

Using A.M. $\ge$ G.M., we get
$\frac{{\cot }^{-1}x+(\pi -{\cot }^{-1}(x\left)\right)}{2}\ge \sqrt{\left({\cot }^{-1}x\right)[\pi -{\cot }^{-1}(x\left)\right]}\Rightarrow 0<\sqrt{{\cot }^{-1}\left(x\right)(\pi -{\cot }^{-1}(x\left)\right)}\le \frac{\pi }{2}\Rightarrow 0<y\le \frac{{\pi }^2}{4}\Rightarrow a=1,b=4\therefore a+b=5$

Alternate solution
$y=\left({\cot }^{-1}x\right)\left({\cot }^{-1}\right(-x\left)\right)$
$\Rightarrow y=\left({\cot }^{-1}x\right)(\pi -{\cot }^{-1}(x\left)\right)$
Let $t={\cot }^{-1}x$
$\Rightarrow y=-t^2+\pi t$
$\because t\in (0,\pi )$

$f\left(0\right)=0$
$f\left(\pi \right)=0$
and
$-\frac{D}{4a}=-\frac{{\pi }^2-0}{-4}=\frac{{\pi }^2}{4}$
Hence, $y\in (0,\frac{{\pi }^2}{4}]$
$\Rightarrow a=1,b=4$
$\therefore a+b=1+4=5$