It may partly depend on what sort of velvet it is, e.g. cotton velvet, viscose velvet, wool velvet, mohair velvet. Do you have any info about that? For example, cotton velvet can be gently cleaned with the right soap and water and a soft brush (I've had a pair of velvet boots for about 7 years, and I think they're cotton velvet based on the feel of it, and they still look incredibly good despite all the rain and mud and subsequent cleaning I've given them!), but viscose velvet on the other hand can react really badly to water and the fibres clump together and almost seem like they fuse to each other and create ugly patches.
However, I have seen plenty of bag spas do wonders with velvet bags (I've seen some before and after photos are that are spectacular), so if you're in any doubt, maybe look into that as an option? Good bag spas usually have the expertise to clean most types of fabric.