Cellular senescence restrains the expansion of neoplastic cells through several layers of regulation. We report that the histone H3-specific demethylase KDM4 is expressed as human stromal cells undergo senescence. In clinical oncology, upregulated KDM4 and diminished H3K9/H3K36 methylation correlate with poorer survival of patients with prostate cancer after chemotherapy. Global chromatin accessibility mapping via assay for transposase-accessible chromatin with high-throughput sequencing, and expression profiling through RNA sequencing, reveal global changes of chromatin openness and spatiotemporal reprogramming of the transcriptomic landscape, which underlie the senescence-associated secretory phenotype (SASP). Selective targeting of KDM4 dampens the SASP of senescent stromal cells, promotes cancer cell apoptosis in the treatment-damaged tumor microenvironment and prolongs survival of experimental animals. Our study supports dynamic changes of H3K9/H3K36 methylation during senescence, identifies an unusually permissive chromatin state and unmasks KDM4 as a key SASP modulator. KDM4 targeting presents a new therapeutic avenue to manipulate cellular senescence and limit its contribution to age-related pathologies, including cancer.