The abundance of tracking sensors in recent years has led to the generation of high-frequency and high-volume streams of data, including vessels, vehicles' tracking data, smartwatches, cameras, and earth observation sensors. However, there are cases where the trajectory of a moving object has gaps, errors, or is unavailable. However, a vast pool of tracking data is available but remains unexplored or underutilized and has the potential to reveal important information. The MUlti-Sensor Inferred Trajectories (MUSIT) project aims at exploring and fusing data from all heterogeneous sources to provide detailed information about a moving object’s whereabouts and behavior, reduce gaps, and produce a refined and inferred trajectory with minimal errors. The fusion of multi-sensor data is required to fill in the trajectory gaps of moving objects and attach useful semantics to the trajectory and its components. AI algorithms and spatio-temporal methodologies that can fuse information and infer the “missing knowledge” are crucial to the implementation of MUSIT. Furthermore, different representation models from multiple domains within the ICT sector will also be explored. Datasets will be made available in cases where it was previously thought impossible, and infer knowledge thus improving the overall surveillance. Therefore, the MUSIT project will tackle the aforementioned issues in a process that can be categorized into three parts: i) data collection and creation, ii) exploitation and utilization of cross-domain representation models within the ICT sector for trajectories, and iii) analysis and processing of outcomes to produce information-rich results related to vessel monitoring and urban mobility.