I have read that the androgen receptor gene in humans, located on the X-chromosome, determines how sensitive follicles of body hair are to androgens, but does it also determine the age when these follicles start to get sensitive.

I'm not an expert but this sort of thing tends to be quite complicated and regulated by a number of genes and mechanisms so while the androgen receptor, it's distribution and sensitivity can have a role it won't be the only thing that has an effect on age of hair growth stimulation.