Sometime our intuitions about the world are very wrong. As in, they just suck. Those mountains are way further away than they look. That optical illusion your friend just showed you isn’t at all how you thought it was. And, particularly relevant, your gut feel about how investing actually works is all wrong.
“Compound interest is the eighth wonder of the world. He who understands it, earns it … he who doesn’t … pays it.”Potentially apocryphal quote from Albert Einstein about the counter-intuitive reality of compound interest.
Researchers on the topic claim our intuitions are bad unless three conditions are met:
- There is regularity in the topic (intuition is irrelevant when dealing with randomness).
- You get a lot of practice.
- You get immediate feedback from this practice.
If these conditions are met then there is something to learn and you can learn it with effective practice, training your intuition to be more accurate.
In many areas these conditions aren’t met. As a result our intuitions are bad (even if we don’t know it). This is famously true in the finance world, where patterns often aren’t sufficiently regular. As a result, many people manage to convince themselves they have a good feel for something, only to be surprised in a big way by later events.
For parts of my career I’ve worked to fix this. I would build mathematical models to analyze those pieces of the world that do have more regular patterns. The world might not give you practice with instant feedback in these areas, but a model of it can simulate this. With repeated use it can teach your intuition to be better.
I particularly focused on trying to provide my users with the kind of feedback that built up their intuition over time. The point wasn’t just the model result but also training the intuition of the users to have a deeper understanding of how something worked; particularly when it seemed counter-intuitive.
This can be dangerous of course; if the model isn’t a good representation of reality it will build up your confidence without improving your judgment, leaving you worse off than you started. It’s a challenge in this work to track when a model is helpful and when it might be making things worse. With care, however, the results can be deeply useful.
An example of bad intuition
Recently a friend of mine was trying to get pregnant. Every month she’d write to say she was not pregnant, yet again, and every month she’d be somewhat devastated; convinced that this reflected badly on her chances at getting pregnant. After a while it struck me that this was a great example of intuitions being misleading. Finding out you’re not pregnant this month might be disappointing, but it’s definitely not evidence that you won’t get pregnant.
As a result I worked up a simple explanation of how the statistics on this worked and sent it her way. I’m not sure I’ve ever gotten a more adamant “thank you” in response in my entire life. Accordingly I’m sharing this here in hopes other women might find comfort in a better understanding.
Each month you have a window of around six days prior to ovulation in which you can get pregnant. (For statistics on this and just a generally excellent, data-based discussion of pregnancy see Emily Oster’s fantastic book Expecting Better. A great intro to the book is available here) What people often don’t realize is that women at their peak fertility ages of 20-24 only have around at 25% chance of getting pregnant each month (some sources will claim this is as high as 30%, others put it at 20%). This means that the most fertile woman has around the same monthly pregnancy chance as the chance of getting the same result in a coin toss three times in a row.
Click to expand if you want a discussion of the math on this comparison.
To be fair, I’m cheating a bit in this comparison. Getting the same result three times in a row in a coin flip is actually two results; it’s either getting three heads or three tails. The results of either of these is the same (0.5*0.5*0.5)=0.125 or 12.5%. Add these results together for the chance at getting either of them, resulting in a chance of 0.25 or 25%.
I’m cheating to make a point here though; most people would be surprised to learn that these chances are the same, and this article is all about seeing where our intuitions are wrong!
Isn’t that shocking? Before studying this I’d have assumed the chances were much higher than that. You might wonder then; if that’s so how do more fertile women seem to get pregnant so easily?
The answer is that they’re not trying for one month; they try EVERY month. And while each month might have only a 25% chance, chances go up with repeated attempts (for you soccer fans; this is the old “enough shots on goal” principle in action). Here’s a table showing how these chances accumulate over time:
Notice something here? That 25% chance each month turns into a cumulative 97% chance of getting pregnant within the first year! By the second year it’s so high that it is nearly 100%! (it’s actually 99.9% but the above numbers are rounded).
This is why many discussions of fertility statistics discuss pregnancy rates by the year instead of by the month and doctors will often tell you to try for a year before going in for testing. Even the most fertile woman has a less than a 50% chance of conceiving in fewer than 3 months. In this view, taking longer to conceive is actually just…normal.
How normal? In the above, even after 10 months 6% of women (more than 1 in 20) at peak fertility still haven’t conceived! Think about that; statistically 1 in 20 women at peak fertility will take longer than 10 months to conceive even when nothing is wrong! Just blind damn luck.
How does this change with age?
Many women I know are very concerned about how this shifts with age. Can you still conceive at 30? 35? 40? Let’s explore how these numbers shift over time.
The core difference with age is that the 25% chance per month starts to go down. Sources vary in exact estimates but by age 30 the chance is generally estimated at somewhere around 20% per month. Here are the numbers for that:
Not much change here, is there? In some cases it might take a bit longer, but not much.
How about even older?
By age 35 chances are often cited as being somewhere around 15%. At that point the numbers look like this:
Here we can see that it may indeed take longer to conceive, but the odds are still very high that a woman at this age would conceive inside a year, and extremely high inside two years.
After age 35
After age 35 chances start to shift in ways that have more of an effect. Some sources indicate ages 35-39 have a monthly rate ranging between 8% and 15%. Using the worst case of 8% the numbers look like this:
Note: I’m using the most pessimistic estimate here, and even with that number there’s a greater than 50% chance of conceiving after 9 months, and the 2 years rates are still very high. It’s definitely harder to conceive in this age range, but it’s also still likely if you give it time.
And after 40?
After 40 is where the real difference becomes visible. Monthly rates drop to a 5% chance between ages 40-42:
Here for the first time we see that chances of conceiving inside one year are worse than a coin flip. It should of course be noted that chances inside two years are still quite good, but it’s definitely harder. For ages 43 and older the chances drop even lower, landing somewhere in the 1-2% range per month.
What does this mean to me?
For any woman under the age of 40 who’s trying to conceive, there is one big message here:
In the nitty gritty of the moment and the fact that each month is, well, a month, it can be easy to take every single no really damn hard. This makes no sense though. In the case of my friend, she would get deeply disheartened each month and worry that this meant she couldn’t conceive at all; this makes no sense! (incidentally, she felt very relieved on seeing these numbers, and the very next month she was pregnant). As the math shows, it’s the cumulative success that matters, not the individual monthly one.
Of course, if you’re like many of my friends, nothing I say is going to let you relax completely. Some of you will just stress about this no matter what, and some of the researching can be fun, so by all means research what days you should be having sex, look up all the tricks and any crazy advice you want. It certainly makes sense to maximize your chance each month; have at it! But when the test comes up a no that seventh or eighth time; don’t let it stress you out too much. Carry a coin around with you and flip it a few times or roll a dice and remember that this is a game that’s more about time than anything else. Be disappointed for a moment that your time hasn’t yet come, and then move on.
What about other risks or actual problems?
To be complete here I must also note that there are other factors not accounted for in this discussion. Things like miscarriage rates and birth defect chances are not included here, and there are of course also cases where people genuinely do have problems conceiving.
The purpose however is to note that in most cases it makes more sense to try for a while and relax before stressing about it. If you’re really worried you can always consult your doctor earlier, but don’t be surprised if you get this same recommendation; when you’ve looked at the math you can see that it makes sense after all.
For those who wish a more complete picture, which includes things like miscarriages and birth defects, I’d again recommend Emily Oster’s Expecting Better. I will note however that many of the other concerns seem to follow the same general pattern shown above; which is to say that up to age 35 things generally work without issue. Between ages 35 and 39 they’re still okay but become harder or take longer, and from the age of 40 and on things are much more difficult. As one example of that, here’s one table on down syndrome risk from the Oster book mentioned above:
Again, it’s worth emphasizing that if you’re 40 or younger you have a far greater chance of a car accident next year (double the chance) than a child with down syndrome. This doesn’t even take into account the various tests you can do to detect it during pregnancy (again; read the book, it’s great!)
Also worth noting from the numbers above: if you buy lottery tickets you should do so for the chance to dream for a day; not with any expectation of winning.
How can I calculate this myself?
For those who want to explore deeper, the excel sheet with these calculations is available for download here. I created this sheet entirely myself so you can activate editing and anything else necessary for it to automatically recalculate. If you use Microsoft Excel just download the xlsx file. Those of you without Microsoft Excel should also be able to open it via the free Google Sheets, LibreOffice, or any other software that can read an excel file, but in that case the xls-format of the file can sometimes be a bit easier to use.
The Math (optional)
The above example discusses the nature of the probabilities involved here, and their interplay with time/repeats. For purposes of general understanding one needn’t actually understand more than this, and the Excel sheets provided should be sufficient for readers to play with the numbers and build up an intuition without understanding the math.
For anyone interested in the more technical details though, a brief discussion of the math might be of interest. This is available in the expandable section below; written with non-technical readers in mind:
How these probabilities work
A simple example of the math here is calculating the odds of a coin flip. If you flip a coin you have a 50% chance (0.50) of getting heads, and a 50% chance (0.50) of getting tails.
What’s the chance of getting heads twice in a row? Well it turns out the way to do this is to calculate the probability of not getting heads and multiply it by itself, once for each flip:\[0.5*0.5=0.25\]
Here we see a 50% chance of no heads on flip 1 times a 50% change of no heads on flip 2 gives a 25% chance of not getting heads on either flip.
We can of course extend this logic out. If we flip three times we just add on another chance of not getting a heads again:\[0.5*0.5 *0.5=0.125\]
Another way of representing this is \[0.5^n\] where n is the number of coin flips.
And of course, if we at any point want to calculate the inverse probability; i.e. the chance of getting heads at least once, we subtract the probability of not getting heads from 100%. To calculate this for four flips:
\[0.5^4=0.0625\] gives a 6.25% chance of never getting heads, and \[1-0.0625=.9375\] shows a corresponding 93.75% chance of getting a heads at least once in four flips.
The math with the pregnancy case above is the same. The rates are different, so the numbers change, but the formulas are the same; just change the numbers and replace getting heads with getting pregnant.
It’s also worth noting that this is a particularly easy case as events don’t impact one another (mathematically this means they’re “independent”; i.e. not getting pregnant last month doesn’t change your chances of getting pregnant this month) and because there are only two possibilities (pregnant or not pregnant, heads or tails). This makes the math a bit cleaner.