We present a geometric setting for the differential Galois theory of $G$-invariant connections with parameters. As an application of some classical results on differential algebraic groups and Lie algebra bundles, we see that the Galois group of a connection with parameters with simple structural group $G$ is determined by its isomonodromic deformations. This allows us to compute the Galois groups with parameters of the general Fuchsian special linear system and of Gauss hypergeometric equation.