The following infographic details eleven locations you can visit which were used in the Harry Potter movies. (The 12th location is a movie studio, but the sets no longer exist, so ... )
All of the locations are in the UK, and most are in London, so with a little legwork a fan should be able to visit all of them in a single week.
P.S. To refresh your memory of the events in the Harry potter series, here's an infographic (the Summharry) which tells the story of all seven novels.