/diy/ - Do It Yourself

What is the deal with building permits?
I grew up in a fixer-upper helping my Dad do plumbing, wiring, siding, remodeling a bathroom, lots of stuff and we never filed for a permit. Once my Dad had the roof replaced and the roofers filed a permit but that's it.
I'm looking for a cheap house to buy and looking at the building permit schedule to get an idea how much a fixer-upper would cost me. I see permit fees for stuff I've done without a permit for my Dad, including a fence. Really? A fence?
Does anybody here (contractor or DIY) ever had to deal with applying for building permits and dealing with inspectors? What is it like?