We describe a three axis vector magnet system for cryogenic scanning probe microscopy measurements. We discuss the magnet support system and the power supply, consisting of a compact three way 100 A current source. We obtain tilted magnetic fields in all directions with maximum value of 5T along z-axis and of 1.2T for XY-plane magnetic fields. We describe a scanning tunneling microscopy-spectroscopy (STM-STS) set-up, operating in a dilution refrigerator, which includes a new high voltage ultralow noise piezodrive electronics and discuss the noise level due to vibrations. STM images and STS maps show atomic resolution and the tilted vortex lattice at 150 mK in the superconductor β-Bi2Pd. We observe a strongly elongated hexagonal lattice, which corresponds to the projection of the tilted hexagonal vortex lattice on the surface. We also discuss Magnetic Force Microscopy images in a variable temperature insert.