I have recently read Origin of Species and Why Evolution is True. The sections on Geographic Distribution were very interesting; only those creatures that could, possibly, survive a short ocean voyage end up on oceanic islands.

Both books mentioned that a mating pair, or at the most extreme, a pregnant female could get stranded on the island and start a life there.

But I'm wondering about genetic diversity. Wouldn't cases like that lead to inbreeding and all the problems that causes? I thought I remembered reading somewhere that for optimal genetic diversity, there should be at least 64 breeding pairs, and I think as few as 8 were allowable, though I'm not sure about those numbers.

The situation you describe, where a population is started by only a few individuals, is possible. The reduction of genetic diversity that comes with this extreme genetic bottleneck is called the founder effect. The number of founding individuals necessary for establishing a stable population is heavily dependant on what the new environment is like, eg is there competition from organisms already there? Competition would in general require more diversity to stay viable, whereas if they find an empty niche with plenty of food a population would not be under so much selective pressure. Is the founding organism well suited to the new environment, or will some significant adaptation need to occur for the population to stabilise? Inevitably some adaptation occurs following migration to a new environment, and the extent of this is limited by the existing diversity. Lower diversity means lower chances of successful adaptation and survival.

On the other hand, I would think that very few of these hypothetical island colonisation occur as single, isolated events. If one imagines that, say, a lizard clinging to some driftwood reaches an island simply by the passage of ocean currents, it is reasonable to suspect that another lizard might do the same on the same currents at another time. So, while you are correct in thinking about a theoretical minimum diversity, establishing a population doesn't require that all of the breeding pairs reach the island all at the same time. Instead, successive introductions over a period of years works to keep the gene pool diverse.