Studies of the spin, parity and tensor couplings of the Higgs boson in the $H\rightarrow ZZ^*\rightarrow 4\ell$, $H\rightarrow WW^* \rightarrow e\nu\mu\nu$ and $H\rightarrow \gamma \gamma$ decay processes at the LHC are presented. The investigations are based on $25$ fb$^{-1}$ of $pp$ collision data collected by the ATLAS experiment at $\sqrt{ s} = 7$ TeV and $\sqrt{ s }= 8$ TeV. The Standard Model (SM) Higgs boson hypothesis, corresponding to the quantum numbers $J ^P=0^+$, is tested against several alternative spin scenarios, including non-SM spin-0 and spin-2 models with universal and non-universal couplings to fermions and vector bosons. All tested alternative models are excluded in favour of the SM Higgs boson hypothesis at more than 99.9% confidence level. Using the $H\rightarrow ZZ^*\rightarrow 4\ell$ and $H\rightarrow WW^* \rightarrow e\nu\mu\nu$ decays, the tensor structure of the $HVV$ interaction in the spin-0 hypothesis is also investigated. The observed distributions of variables sensitive to the non-SM tensor couplings are compatible with the SM predictions and constraints on the non-SM couplings are derived.