For after the rash appears: If you are in an area where there are tan oak trees, boil some of the shredded bark until the water is a dark tan. Apply the cooled water to the rash several times a day. This was the remedy my grandparents used on me way back in the 40's. I'm not sure it actually worked all that well, but it was all we had, and seemed to be better than nothing. The tannins that leach into the water tend to "dry up" the rash, relieve the itching, as well as tanning your own hide somewhat. Most oaks contain tannins in both bark and leaves, but the so-called "tan-oak" has the most, so is what we used to use. I used to come home to Nevada from summers spent in the wilds of Mendocino County with my grandparents looking like a shriveled up brown forest elf with bark rot. I loved the look, and all of the attention it got from my school chums, but my mother, who liked to dress me up like Shirley Temple for school, suffered a lot.
A better, but less natural and more modern solution is to go to a drug store and get some OTC hydrocortisone cream and use this. However, you aren't supposed to use this on your face for some reason I've never quite fathomed. The anti-itch creams containing Benadryl and Benadryl pills also help with the itching, but don't do much for the rash itself right away.
Some people are more susceptible (allergic) to poison oak than others. I have always been one of the "others," but both my grandfather and my husband could wade around in it with impunity, while I would get a total body rash just handling their clothes doing their laundry afterwards, or even sitting in a chair where they'd been sitting in their contaminated clothes.
There was a wives' at the time that the Indians used to eat the leaves of poison oak to impart immunity. DON'T TRY THIS. It also goes in one end and even worse, out the other, before doing so, causing severe rashes on both ends. Whether or not it imparts some immunity (as severe over-exposure to many allergens may do if it doesn't kill you first,) is up for debate.