Haha, I actually experienced this with my mouse mats before I upgraded to a higher-end Razer eXactMat X (reviewed
here).
The only thing I've found that works is to either use an anti-cling/anti-static cloth and wipe down the surface (which should keep it clean for a while, Dirt gaming rags work for this); you'll probably have to remove the dust surrounding the mat with some sort of swiffer/dusting device (I, sadly, am not well-versed in cleaning utilities). Once you get the dust out of the immediate area, the rags should keep it clean for a while.
I'm guessing the fabric on yours is what's attracting dust -- most of the ones I've used have been either cheapo $20 ones that are anti-static and rubbery or the one expensive one I have, which is a hard plastic exactmat x. Hopefully this helps somehow