The relationship between serum total testosterone and prostate cancer (PCa) risk is controversial. The hypothesis that faster age-related reduction in testosterone is linked with increased PCa risk remains untested. We conducted this study at a tertiary-level hospital in southeast of the US, and derived data from the Medical Registry Database of individuals that were diagnosed of any prostate-related disease from 2001-2015. Cases were those diagnosed of PCa and had one or more measurements of testosterone prior to PCa diagnosis. Controls were those without PCa and had one or more testosterone measurements. Multivariable logistic regression models for PCa risk of absolute levels (one-time measure and 5-year average) and annual change in testosterone were respectively constructed. Among a total of 1559 patients, 217 were PCa cases, and neither one-time measure nor 5-year average of testosterone was found to be significantly associated with PCa risk. Among the 379 patients with two or more testosterone measurements, 27 were PCa cases. For every 10 ng/dL increment in annual reduction of testosterone, the risk of PCa would increase by 14% [adjusted odds ratio, 1.14; 95% confidence interval (CI), 1.03-1.25]. Compared to patients with a relatively stable testosterone, patients with an annual testosterone reduction of more than 30 ng/dL had 5.03 [95% CI: 1.53, 16.55] fold increase in PCa risk. This implies a faster age-related reduction in, but not absolute level of serum total testosterone as a risk factor for PCa. Further longitudinal studies are needed to confirm this finding. This article is protected by copyright. All rights reserved.