Abstract: Smart city situational awareness has recently emerged as a hot topic in research societies, industries, and governments because of its potential to integrate cutting-edge information technology and solve urgent challenges that modern cities face. For example, in the latest five-year plan, the Chinese government has highlighted the demand to empower smart city management with new technologies such as big data and Internet of Things, for which situational awareness is normally the crucial first step. While traditional static surveillance data on cities have been available for decades, this review reports a type of relatively new yet highly important urban data source, i.e., the big mobile data collected by devices with various levels of mobility representing the movement and distribution of public and private agents in the city. We especially focus on smart city situational awareness enabled by synthesizing the localization of hundreds of thousands of mobile software Apps using the Global Positioning System (GPS). This technique enjoys advantages such as a large penetration rate (~50% urban population covered), uniform spatiotemporal coverage, and high localization precision. We first discuss the pragmatic requirements for smart city situational awareness and the challenges faced. Then we introduce two suites of empowering technologies that help fulfill the requirements of (1) cybersecurity insurance for smart cities and (2) spatiotemporal modeling and visualization for situational awareness, both via big mobile data. The main contributions of this review lie in the description of a comprehensive technological framework for smart city situational awareness and the demonstration of its feasibility via real-world applications.

移动大数据赋能的智慧城市态势感知

[1]Abdelnabi S, Krombholz K, Fritz M, 2020. VisualPhishNet: zero-day phishing website detection by visual similarity. Proc ACM SIGSAC Conf on Computer and Communications Security, p.1681-1698.

[2]Albanese M, Jajodia S, Venkatesan S, et al., 2019. Adaptive cyber defenses for botnet detection and mitigation. In: Jajodia S, Cybenko G, Liu P, et al (Eds.), Adversarial and Uncertain Reasoning for Adaptive Cyber Defense: Control- and Game-Theoretic Approaches to Cyber Security. Springer, Cham, p.156-205.

[3]Andrienko G, Andrienko N, Bosch H, et al., 2013. Thematic patterns in georeferenced tweets through space-time visual analytics. Comput Sci Eng, 15(3):72-82.

[4]Andrienko G, Andrienko N, Fuchs G, et al., 2017. Revealing patterns and trends of mass mobility through spatial and temporal abstraction of origin-destination movement data. IEEE Trans Vis Comput Graph, 23(9):2120-2136.

[5]Andrienko N, Andrienko G, 2013. Visual analytics of movement: an overview of methods, tools and procedures. Inform Visual, 12(1):3-24.

[6]Bak P, Mansmann F, Janetzko H, et al., 2009. Spatiotemporal analysis of sensor logs using growth ring maps. IEEE Trans Vis Comput Graph, 15(6):913-920.

[7]Borland D, Ii RMT, 2007. Rainbow color map (still) considered harmful. IEEE Comput Graph Appl, 27(2):14-17.

[8]Buschmann C, Pfisterer D, Fischer S, et al., 2005. SpyGlass: a wireless sensor network visualizer. ACM SIGBED Rev, 2(1):1-6.

[9]Calabrese F, Pereira FC, di Lorenzo G, et al., 2010. The geography of taste: analyzing cell-phone mobility and social events. Proc 8^th Int Conf on Pervasive Computing, p.22-37.

[10]Caldarelli G, Arcaute E, Barthelemy M, et al., 2023. The role of complexity for digital twins of cities. Nat Comput Sci, 3(5):374-381.

[11]Chan PK, Mahoney MV, 2005. Modeling multiple time series for anomaly detection. Proc 5^th IEEE Int Conf on Data Mining, p.90-97.

[12]Chen DQ, Manning C, 2014. A fast and accurate dependency parser using neural networks. Proc Conf on Empirical Methods in Natural Language Processing, p.740-750.

[13]Chen Q, 2014. Progress and application practices of smart city construction abroad. Shanghai Inform, (10):81-83 (in Chinese).

[14]Chen W, Zhang TY, Zhu HY, et al., 2021. Perspectives on cross-domain visual analysis of cyber-physical-social big data. Front Inform Technol Electron Eng, 22(12):1559-1564.

[15]Cheng SJ, Chen C, Pan SL, et al., 2022. Citywide package deliveries via crowdshipping: minimizing the efforts from crowdsourcers. Front Comput Sci, 16(5):165327.

[16]Cottineau C, Vanhoof M, 2019. Mobile phone indicators and their relation to the socioeconomic organisation of cities. ISPRS Int J Geo-Inform, 8(1):19.

[17]Fan YJ, Hou SF, Zhang YM, et al., 2018. Gotcha-Sly malware!: scorpion a metagraph2vec based malware detection system. Proc 24^th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining, p.253-262.

[18]Franke U, Brynielsson J, 2014. Cyber situational awareness— a systematic review of the literature. Comput Secur, 46:18-31.

[19]Fu XY, Zhang JN, Meng ZQ, et al., 2020. MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding. Proc Web Conf, p.2331-2341.

[20]González MC, Hidalgo CA, Barabási AL, 2008. Understanding individual human mobility patterns. Nature, 453(7196):779-782.

[21]Gu YK, 2023. Digital twins: making cities smarter. People’s Daily, May 17, (07) (in Chinese).

[22]Guo RZ, Lin HH, He B, et al., 2020. GIS framework for smart cities. Geom Inform SciWuhan Univ, 45(12):1829-1835.

[23]Hassani K, Khasahmadi AH, 2020. Contrastive multi-view representation learning on graphs. Proc 37^th Int Conf on Machine Learning, p.4116-4126.

[24]Hazel GG, 2000. Multivariate Gaussian MRF for multispectral scene segmentation and anomaly detection. IEEE Trans Geosci Remote Sens, 38(3):1199-1211.

[25]Herr D, Kurzhals K, Ertl T, 2020. Visual analysis for spatiotemporal event correlation in manufacturing. Proc 53^rd Hawaii Int Conf on System Sciences, p.1-10.

[26]Iliadis LA, Kaifas T, 2021. Darknet traffic classification using machine learning techniques. Proc 10^th Int Conf on Modern Circuits and Systems Technologies, p.1-4.

[27]Jiang LY, Jayatilaka A, Nasim M, et al., 2022. Systematic literature review on cyber situational awareness visualizations. IEEE Access, 10:57525-57554.

[28]Jiang S, Fiore GA, Yang YX, et al., 2013. A review of urban computing for mobile phone traces: current methods, challenges and opportunities. Proc 2^nd ACM SIGKDD Int Workshop on Urban Computing, Article 2.

[29]Kamal M, Rashid I, Iqbal W, et al., 2023. Privacy and security federated reference architecture for Internet of Things. Front Inform Technol Electron Eng, 24(4):481-508.

[30]Kilincer IF, Ertam F, Sengur A, 2021. Machine learning methods for cyber security intrusion detection: datasets and comparative study. Comput Netw, 188:107840.

[31]Lampe OD, Hauser H, 2011. Interactive visualization of streaming data with kernel density estimation. Proc IEEE Pacific Visualization Symp, p.171-178.

[32]Lan JH, Liu XD, Li B, et al., 2022a. DarknetSec: a novel self-attentive deep learning method for darknet traffic classification and application identification. Comput Secur, 116:102663.

[33]Lan JH, Liu XD, Li B, et al., 2022b. MEMBER: a multi-task learning model with hybrid deep features for network intrusion detection. Comput Secur, 123:102919.

[34]Li QM, Han ZC, Wu XM, 2018. Deeper insights into graph convolutional networks for semi-supervised learning. Proc 32^nd AAAI Conf on Artificial Intelligence, p.3538-3545.

[35]Liao CC, Li XM, Hong WY, et al., 2023. Multi-dimensional measurement of network structure of Guangdong-Hong Kong-Macao Greater Bay Area from the perspective of traffic flows space. Geogr Res, 42(2):550-562.

[36]Liao RZ, Chen LP, 2022. An evolutionary note on smart city development in China. Front Inform Technol Electron Eng, 23(6):966-974.

[37]Liu C, Li B, Zhao J, et al., 2021. MG-DVD: a real-time framework for malware variant detection based on dynamic heterogeneous graph learning. Proc 30^th Int Joint Conf on Artificial Intelligence, p.1512-1519.

[38]Liu C, Li B, Zhao J, et al., 2022. FewM-HGCL: few-shot malware variants detection via heterogeneous graph contrastive learning. IEEE Trans Depend Secur Comput, early access.

[39]Liu YX, Li Z, Pan SR, et al., 2021. Anomaly detection on attributed networks via contrastive self-supervised learning. IEEE Trans Neur Netw Learn Syst, 33(6):2378-2392.

[40]Miao X, Liu KB, He Y, et al., 2011. Agnostic diagnosis: discovering silent failures in wireless sensor networks. Proc IEEE INFOCOM, p.1548-1556.

[41]Miao X, Liu K, He Y, et al., 2013. Agnostic diagnosis: discovering silent failures in wireless sensor networks. IEEE Trans Wirel Commun, 12(12):6067-6075.

[42]Miranda F, Doraiswamy H, Lage M, et al., 2017. Urban pulse: capturing the rhythm of cities. IEEE Trans Vis Comput Graph, 23(1):791-800.

[43]Mittelstädt S, Jäckle D, Stoffel F, et al., 2015. ColorCAT: guided design of colormaps for combined analysis tasks. Eurographics Conf on Visualization, p.115-119.

[44]National Bureau of Statistics, 2022. Statistical Bulletin of the National Economy and Social Development of the People’s Republic of China in 2022. National Bureau of Statistics, People’s Republic of China (in Chinese).

[45]Neshenko N, Nader C, Bou-Harb E, et al., 2020. A survey of methods supporting cyber situational awareness in the context of smart cities. J Big Data, 7(1):92.

[46]Ni K, Ramanathan N, Chehade MNH, et al., 2009. Sensor network data fault types. ACM Trans Sens Netw, 5(3):25.

[47]Noble CC, Cook DJ, 2003. Graph-based anomaly detection. Proc 9^th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.631-636.

[48]Palmisano SJ, 2008. A Smarter Planet: the Next Leadership Agenda. Report of IBM.

[49]Park S, Gondal I, Kamruzzaman J, et al., 2019. Oneshot malware outbreak detection using spatio-temporal isomorphic dynamic features. Proc 18^th IEEE Int Conf on Trust, Security and Privacy in Computing and Communications/13^th IEEE Int Conf on Big Data Science and Engineering, p.751-756.

[50]Phithakkitnukoon S, Horanont T, di Lorenzo G, et al., 2010. Activity-aware map: identifying human daily activity pattern using mobile phone data. Proc 1^st Int Workshop on Human Behavior Understanding, p.14-25.

[51]Qamar S, Anwar Z, Rahman MA, et al., 2017. Data-driven analytics for cyber-threat intelligence and information sharing. Comput Secur, 67:35-58.

[52]Qi YN, Fang CR, Liu HY, et al., 2021. A survey of cloud network fault diagnostic systems and tools. Front Inform Technol Electron Eng, 22(8):1031-1045.

[53]Ratti C, Frenchman D, Pulselli RM, et al., 2006. Mobile landscapes: using location data from cell phones for urban analysis. Environ Plan B Plan Des, 33(5):727-748.

[54]Schreck T, Bernard J, Tekusová T, et al., 2008. Visual cluster analysis of trajectory data with interactive Kohonen maps. Proc IEEE Symp on Visual Analytics Science and Technology, p.3-10.

[55]Shi L, Liao Q, He Y, et al., 2011. SAVE: sensor anomaly visualization engine. Proc IEEE Conf on Visual Analytics Science and Technology, p.201-210.

[56]Shi L, Huang CC, Liu MJ, et al., 2021a. UrbanMotion: visual analysis of metropolitan-scale sparse trajectories. IEEE Trans Vis Comput Graph, 27(10):3881-3899.

[57]Shi L, Guo ZC, Jiang T, et al., 2021b. Visual analysis of steady-state human mobility in cities. Human-Centr Comput Inform Sci, 11:31.

[58]Shi L, Luo YK, Ma S, et al., 2023. Mobility inference on long-tailed sparse trajectory. ACM Trans Intell Syst Technol, 14(1), Article 18.

[59]Singh S, Sharma PK, Moon SY, et al., 2019. A comprehensive study on APT attacks and countermeasures for future networks and communications: challenges and solutions. J Supercomput, 75(8):4543-4574.

[60]Skopik F, Settanni G, Fiedler R, 2016. A problem shared is a problem halved: a survey on the dimensions of collective cyber defense through security information sharing. Comput Secur, 60:154-176.

[61]Sun YJ, Han JW, Yan XF, et al., 2011. PathSim: meta path-based top-K similarity search in heterogeneous information networks. Proc VLDB Endow, 4(11):992-1003.

[62]Tang SJ, Wu B, Zhu Q, 2016. Combined adjustment of multiresolution satellite imagery for improved geo-positioning accuracy. ISPRS J Photogr Remote Sens, 114:125-136.

[63]Tang SJ, Li Y, Yuan ZL, et al., 2019. A vertex-to-edge weighted closed-form method for dense RGB-D indoor SLAM. IEEE Access, 7:32019-32029.

[64]Tang SJ, Li XW, Zheng XW, et al., 2022. BIM generation from 3D point clouds by combining 3D deep learning and improved morphological approach. Autom Constr, 141:104422.

[65]Thom D, Bosch H, Koch S, et al., 2012. Spatiotemporal anomaly detection through visual analysis of geolocated Twitter messages. Proc IEEE Pacific Visualization Symp, p.41-48.

[66]Tominski C, Fuchs G, Schumann H, 2008. Task-driven color coding. Proc 12^th Int Conf on Information Visualisation, p.373-380.

[67]Tounsi W, Rais H, 2018. A survey on technical threat intelligence in the age of sophisticated cyber attacks. Comput Secur, 72:212-233.

[68]UN Department of Economic and Social Affairs (UN DESA), 2019. World Urbanization Prospects: the 2018 Revision.

[69]Vongkusolkit J, Huang QY, 2021. Situational awareness extraction: a comprehensive review of social media data classification during natural hazards. Ann GIS, 27(1):5-28.

[70]von Landesberger T, Brodkorb F, Roskosch P, et al., 2016. MobilityGraphs: visual analysis of mass mobility dynamics via spatio-temporal graphs and clustering. IEEE Trans Vis Comput Graph, 22(1):11-20.

[71]Wang JY, Liu C, Fu XC, et al., 2019. A three-phase approach to differentially private crucial patterns mining over data streams. Comput Secur, 82:30-48.

[72]Wang Q, Hassan WU, Li D, et al., 2020. You are what you do: hunting stealthy malware via data provenance analysis. Proc Network and Distributed Systems Security Symp.

[73]Wang X, Ji HY, Shi C, et al., 2019. Heterogeneous graph attention network. Proc World WideWeb Conf, p.2022-2032.

[74]Wang XB, Mei J, Cui SG, et al., 2023. Realizing 6G: the operational goals, enabling technologies of future networks, and value-oriented intelligent multi-dimensional multiple access. IEEE Netw, 37(1):10-17.

[75]Wang Z, 2021. China’s first large-scale city operational digital signaling system goes online. China News, June 10 (in Chinese).

[76]Wei W, Zhu XR, Wang Y, 2022. Novel robust simultaneous localization and mapping for long-term autonomous robots. Front Inform Technol Electron Eng, 23(2):234-245.

[77]Willems N, van de Wetering H, van Wijk JJ, 2009. Visualization of vessel movements. Comput Graph Forum, 28(3):959-966.

[78]Woodward J, Ruiz J, 2023. Analytic review of using augmented reality for situational awareness. IEEE Trans Vis Comput Graph, 29(4):2166-2183.

[79]Xiao Y, Zheng KH, Lonapalawong S, et al., 2022. EcoVis: visual analysis of industrial-level spatio-temporal correlations in electricity consumption. Front Comput Sci, 16(2):162604.

[80]Xiong CL, Li ZY, Chen Y, et al., 2022. Generic, efficient, and effective deobfuscation and semantic-aware attack detection for PowerShell scripts. Front Inform Technol Electron Eng, 23(3):361-381.

[81]Xu LF, Liang Y, Duan ZS, et al., 2020. Route-based dynamics modeling and tracking with application to air traffic surveillance. IEEE Trans Intell Transp Syst, 21(1):209-221.

[82]Yan H, Xu H, 2022. Advancing smart city construction in Barcelona, Spain. People’s Daily, Dec. 1, (17) (in Chinese).

[83]Yan J, Shi L, Tao J, et al., 2020. Visual analysis of collective anomalies using faceted high-order correlation graphs. IEEE Trans Vis Comput Graph, 26(7):2517-2534.

[84]Yang Z, Liu X, Li T, et al., 2022. A systematic literature review of methods and datasets for anomaly-based network intrusion detection. Comput Secur, 116:102675.

[85]Yao JY, Fan XN, Cao N, 2019. Survey of network security situational awareness. Proc 11^th Int Symp on Cyberspace Safety and Security, p.34-44.

[86]Yao YY, Wu WM, Zhang GF, et al., 2022. Power diagram based algorithm for the facility location and capacity acquisition problem with dense demand. Front Comput Sci, 16(6):166709.

[87]Yuan B, Chen DJ, Xu DM, et al., 2019. Conceptual model of real-time IoT systems. Front Inform Technol Electron Eng, 20(11):1457-1464.

[88]Yuan J, Zheng Y, Xie X, et al., 2013. T-Drive: enhancing driving directions with taxi drivers’ intelligence. IEEE Trans Knowl Data Eng, 25(1):220-232.

[89]Yuan ZL, Li Y, Tang SJ, et al., 2021. A survey on indoor 3D modeling and applications via RGB-D devices. Front Inform Technol Electron Eng, 22(6):815-826.

[90]Zhang RJ, 2019. Optimization Study of Urban Bus Network Based on Shared Bicycle Trajectories. MS Thesis, Xi’an University of Technology, Xi’an, China (in Chinese).

[91]Zhang XH, Zhang Y, Zhong M, et al., 2020. Enhancing state-of-the-art classifiers with API semantics to detect evolved android malware. Proc ACM SIGSAC Conf on Computer and Communications Security, p.757-770.

[92]Zhao H, Yao QM, Li JD, et al., 2017. Meta-graph based recommendation fusion over heterogeneous information networks. Proc 23^rd ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.635-644.

[93]Zhao J, Cao N, Wen Z, et al., 2014. #FluxFlow: visual analysis of anomalous information spreading on social media. IEEE Trans Vis Comput Graph, 20(12):1773-1782.

[94]Zhao J, Yan QB, Liu XD, et al., 2020a. Cyber threat intelligence modeling based on heterogeneous graph convolutional network. Proc 23^rd Int Symp on Research in Attacks, Intrusions and Defenses, p.241-256.

[95]Zhao J, Liu XD, Yan QB, et al., 2020b. Multi-attributed heterogeneous graph convolutional network for bot detection. Inform Sci, 537:380-393.

[96]Zheng Y, Capra L, Wolfson O, et al., 2014. Urban computing: concepts, methodologies, and applications. ACM Trans Intell Syst Technol, 5(3):38.

[97]Zhou ZG, Meng LH, Tang C, et al., 2019. Visual abstraction of large scale geospatial origin-destination movement data. IEEE Trans Vis Comput Graph, 25(1):43-53.