You should consider the fact that he hasn’t spoken French for years. (allowances)

__________________________________________________________________________________________

You should make allowances for the fact that he hasn't spoken French for years.

   You should make allowances for his not speaking French for years.