Computer-aided diagnosis (CAD) of pituitary microadenoma (PM) can assist doctors in decision-making, leading to improved lesion detection rates and diagnostic accuracy. However, the performance of existing CAD methods for PM detection has been hindered by the difficulty in obtaining high-quality segmentation results. This is primarily due to the small size of PM lesions and the relatively low resolution of Magnetic Resonance Imaging (MRI) images. To address these challenges, this paper proposes a new medical image detection and segmentation model based on spatio-temporal information. The proposed model aims to addresses the disease classification of PM by designing a network module based on multi-scale feature fusion. This module ensures comprehensive extraction of target semantic information while retaining clear spatial information, achieving classification from dynamic contrast-enhanced MRI(DCE-MRI) to identify positive PM samples. For the lesion segmentation of PM, after ROI Align alignment, the model further adds a semantic segmentation module named Dual-path Semantic Segmentation Module (DSSM) behind the mask head and classification head. This module captures more precise spatio-temporal semantic information, reducing accuracy loss and achieving pituitary segmentation. Finally, leveraging the results of pituitary detection, a feature pyramid network (FPN) layer is redesigned named Reuse Underlying Information Module (RUIM) to reuse low-level information, enhancing the detection capability for PM and thus achieving precise object detection and segmentation. The proposed model achieves an accuracy of 97.10% for PM, mAP of 50.24%, which is superior to multiple representative deep models for medical data. The code is available at https://github.com/BUCT-IUSRC/Research__PM-CAD.