Or you could simply buy a vinyl fence which looks nice on the both sides. From what I know, the "ugly side" has to be facing your garden. It's kind of backwards to have it facing your neighbor.

I was always under the impression that if you put the fence up you have to have the "ugly" side facing you. As someone else suggested, you could always have the ones that slide between 2 concrete posts.

There is no law to say you have to have the rough side of the fence facing you, after all you have paid for it and erected iot so you have the right to have the good side facing you. if your neighbours don`t like it they can grow shrubs or erect their own fence close to yours. the general rule is if the fence is facing the road you should have the good side facing out.

I have always understood that you are supposed to have the 'back' of the panels towards you but I don't know if it's enforceable. I've always thought it unfair since you are paying for the fence!

Unfortunately your side has to have the fence posts.  Otherwise the fence couldn't be put up unless your neighbour allows you to go into their garden.  The only other alternative would be to have the ones that slide in to concrete posts.  When you look at garden fences the person who has the fence post to their garden owns that fence. Sorry