Trial-by-trial variability is a ubiquitous property of neuronal activity in vivo which shapes the stimulus response. Computational models have revealed how local network structure and feedforward inputs shape the trial-by-trial variability. However, the role of input statistics and different interneuron subtypes in this process is less understood. To address this, we investigate the dynamics of stimulus response in a cortical microcircuit model with one excitatory and three inhibitory interneuron populations (PV, SST, VIP). Our findings demonstrate that the balance of inputs to different neuron populations and input covariances are the primary determinants of output trial-by-trial variability. The effect of input covariances is contingent on the input balances. In general, the network exhibits smaller output trial-by-trial variability in a PV-dominated regime than in an SST-dominated regime. Importantly, our work reveals mechanisms by which output trial-by-trial variability can be controlled in a context, state, and task-dependent manner.