Normally you can only do this if you are a visitor to the UK from outside the European Union. The system in Scotland is the same as in other parts of the UK: as an overseas visitor, you can claim VAT back on anything you buy in any shop that participates in the VAT Retail Export Scheme. This is a voluntary scheme and not all stores operate it, so you have to look for a Tax Free Shopping sign outside the shop before you buy anything.

Some famous Edinburgh names such as the Woollen Mill have this service, but generally you need to check. When you have bought something in a participating store, you can ask for a Tax Free Shopping form, which you can fill out (you will need to show your passport) and keep to show to HM customs when you leave Britain.

You can get more information here.