USB port and bench power supply ground difference

So I recently coughed up the $250 for a semi decent bench power supply, the thing that got me though is when powering a circuit from this unit and connecting the BusPirate via USB is that the ground levels aren't equal. Measuring the difference between the ground levels with a multimeter gives me a +-0.4V AC signal.

I ended up tying the ground levels together to prevent the BusPirate from putting out a voltage greater than 3.3V (relative to the power supply). This works but gives me a fair tingle when plugging out the USB connector and touching the ground sleeve as well as the circuit.

I have two questions:

1. Can AC signals this low cause any damage?


2. What precautions should I be taking to prevent damage to myself and components?

---

Here's some additional info:

1. The PSU ground is not connected to mains ground (It does have a mains ground but I'm not using it).


2. When touching the scope ground to the PSU ground (not mains ground) I can see a tiny little spark.


3. The PSU is powering a circuit on a breadboard, I'm using this ground as my one reference. The other reference is from the ground of a USB cable powering the BusPirate (A small piece of test equipment I'm using for USB <-> RS232 3.3V)



4. With the scope ground connected to the PSU ground (not mains ground) and the scope probe on the USB ground I get a 50Hz 3V p-p signal.


5. The USB cable is attached to my laptop which only has two leads. The resistance between the laptop ground at the laptop side and the plug mains ground pin is 800Ohms.