Pedestrian detection has large relevance to the understanding of static and moving scenes of video sequences. The increasing demand for safety and security of people has resulted in more research on intelligent visual surveillance in a wide range of applications, such as moving human detection. With the great success of deep learning methods, researchers decided to switch from traditional methods based hand-crafted feature extractors to recent deep learning-based techniques in order to detect and track people. In this work, the topic of person detection with a Top-view moving fisheye camera is addressed. Although the fisheye camera is a useful tool for video monitoring, most of object detection techniques, with (or without) deep learning, concern classical perspective cameras. However, due to the distortions of fisheye images, we are expected to have higher requirements and challenges on the pedestrian detection using this device. In this paper, we propose an end-to-end learning people detection method based on YOLOv3 detector that detects people using oriented bounding boxes. The proposed model customizes the traditional YOLOv3 for the detection of oriented bounding boxes, by regressing the angle of each bounding box using a periodic loss function. With rotation bounding box prediction, our approach is efficient, reaching 98,1% of true detection. The proposed method is evaluated on a new available dataset where rotated bounding boxes represent annotations from several fisheye videos.