EVALUATION CRITERIA OF SMART CITY MOBILITY SYSTEM USING MCDM METHOD

. While many cities around the world qualify themselves as “smart cities”, there is no comprehensive way to evaluate to what extent they are “smart”. This article proposes a framework for comparison of the level of “smartness” of the urban mobility systems. The most relevant indicators that have the greatest impact on smart mobility systems were selected in the course of literature review. The impact of indicators on smart mobility systems is variable. Evaluating smart mobility systems, different authors distinguish between different indicators, which usually do not duplicate. The paper categorizes the indicators of the smart mobility system into five groups, called “factors”: motor travel and congestion reduction measures; pollution reduction measures; travel safety and accident reduction measures; traffic management tools and services; smart infrastructure measures. A number of indicators are attributed to each of the listed groups. A Multiple Criteria Decision-Making method, namely, the Analytic


Introduction
Rapid urban population growth has a negative impact on urban mobility systems. It has brought the need to address the problems of improving transport services and increasing demand for public transport. However, increase in transport supply is often accompanied by undesirable results. Cities are facing challenges such as traffic jams, rising property prices, environmental pollution, overpopulation and reliance on private vehicles (Farooq, Xie, Stoilova, & Ahmad, 2019). While the world population is increasing, it is declining in the Baltic States. Nevertheless, the number of cars is growing and cities are facing the same problems. Thus, the right strategy to effectively manage transport services is a major concern for cities around the world. Intelligent mobility system or transport is a critical sub-system of the city. The transport sector consumes a lot of energy and is one of the biggest polluters in cities (Miloševi´c, Miloševi´c, Stevi´c, & Stanojevi´c, 2019).
The term smart or intelligent mobility appeared at the beginning of the nineties in order to point out at a city with a mobility system more and more dependent on technology and innovation (Papa & Lauwers, 2015). Some cities that have been more proactive in developing and installing those systems call themselves "smart cities".
Despite various initiatives to promote smart mobility systems in urban areas, little is known about how these systems work in cities, and even less about how to be a flagship city with smart mobility systems (Debnath, Chin, Haque, & Yuen, 2014). Intelligent and sustainable mobility system limits emissions and waste within the planet's ability to absorb them, uses renewable resources at or below their rates of generation, and uses non-renewable resources at or below the rates of development of renewable substitutes, while minimising the impact on the use of land and generation of noise (Litman, 2008). Smart mobility is related to urban planning, which focuses on collective modes of transportation through extensive use of information and communication technologies (Dudzevičiūtė, Šimelytė, & Liučvaitienė, 2017). Therefore, it is necessary to formulate a comprehensive concept of the smart mobility system and to select the indicators of smartness that characterise it. It can be argued that the lack of appropriate harmonised concepts and indicators may be the reason for the lack of comprehensive comparative research on smart mobility systems in cities. Most of the current practices for assessing mobility are reflected in the analysis of traffic (movement of vehicles) or mobility (movement of people and goods). They tend to favour road transport over other forms of accessibility, including alternative modes of transport, mobility management and infrastructure.
By adopting a hierarchical model for evaluation of smart mobility, we consider all forms of accessibility, mobility management, technologies, services, systems, infrastructures and measures that improve the quality of life. The aim of this article is to create a hierarchical model for evaluation of smart mobility and to calculate the weights of the selected criteria according to the chosen calculation methodology.
The literature reports on many smart mobility system indicators, evaluation criteria and sub-criteria, it is may help in selecting the most relevant and significant: travel planning and online payment, variable information and message signs, centralised traffic light management systems, bicycle sharing, car sharing, modern parking solutions, and more.
This paper proposes 23 smart mobility system evaluation indicators, which are divided into 5 groups. The hierarchical model of the evaluation system is developed and the significance of the evaluation indicators is determined. This model allows for comparison of the level of "smartness" of the urban mobility systems. This evaluation model is sufficiently versatile, so there are no special requirements for cities that will be compared in terms of smart mobility system. It is recommended that the cities to be compared belong to the same size class: large cities (over 500 000 inhabitants) and medium (between 100 000 and 500 000 inhabitants) (Giffinger, Fertner, Karmar, & Meijers, 2007), but this is not a prerequisite.
The paper is organised as follows: First, we select the factors and indicators based on the literature review. Then, we present the methodology used for calculation and evaluation. In Section 3, we present the results of the experiment conducted with a group of experts, followed by conclusions.

Literature review
The concept of the smart city appeared in the second half of the 1980s. According to Google Scholar, the first uses of the term "smart city" can be traced back to the period between 1986 and 1990; it was used in the literature on urban development innovation and in the context of Japan's Technopolis program.
The concept of a smart city is far from being limited to technology for cities. Several different concepts, definitions and meanings of the notion of "Smart City" are given in Table 1. Giffinger et al. (2007) identified four components of a smart city: industry, education, governance, and technical infrastructure. This list was expanded within a project of the Regional Science Center at Vienna University of Technology (Albino, Berardi, & Dangelico, 2015). After examining the models of the smart city, six aspects of the smart city were singled out: smart management, smart economy, smart mobility, smart environment, smart people, and smart living. Each of them has its own specific structure, which forms an integral holistic vision of a smart city.  Hall et al. (2000) Smart mobility is one aspect of a smart city. Orlowski & Romanowska (2019) state that smart mobility refers to the area of a smart city representing mobility, or, broadly defined, the components which comprise not only the traditionally understood transportation of people and goods, but also dissemination of information by digital means. The main function of smart mobility is to connect all city's resourcespeople, goods, and information. Battarra Gargiulo, Tremiterra, & Zucaro (2018) emphasize that smart mobility and its use of ICT could play a potential role in sustainable development of transport systems only if the highest quality and quantity of information is converted into sustainable behaviour by the citizens. So, it can be argued that smart mobility must serve people, not technology.

Review of indicators of smart city mobility system
Scientists do a lot of research about smart city mobility systems, but little work has been done to differentiate the smart mobility system evaluation indicators. In addition, different researchers distinguish different indicators. 30 scientific papers with one or another The paper develops a multicriteria analysis approach for evaluating the performance of different cities based on the smartness of their transportation systems classification of indicators have been analysed in order to select the indicators that comprehensively characterise the smart mobility system and its infrastructure. In these papers, more than 100 different indicators were used to describe the smartness of the urban transport system. When searching for scientific articles on smart mobility, we found that most articles were published by the Italian research teams. The indicators and factors presented in the articles are different and their number is also different. An example given in Table 2.

Review of MCDM method in field of transport
Many multi-criteria analysis methods such as AHP, TOPSIS, DEMATEL, ELEKTRE, PROMETHEE and many more can be used to evaluate a smart mobility system. The same problem is often solved using identical indicators and their values demonstrate different results. Thus, it is important to consider which of these methods are most appropriate for solving specific tasks. In the scientific articles, researchers (Broniewicz & Ogrodnik, 2020;Deluka-Tibljaš, Karleuša, & Dragičević, 2013;Erdogan & Kaya, 2019;Mardani, Zavadskas, Khalifah, Jusoh, & MD Nor, 2016) examine the popularity of MCDM methods within the projects associated with transport infrastructure. Based on the literature, it can be stated that the AHP method is one of the most popular, it is used to determine the importance coefficients of smart mobility system indicators (Broniewicz & Ogrodnik, 2020;Boselli et al., 2015;Castillo & Pitfield, 2010;Deluka-Tibljaš et al., 2013, Erdogan & Kaya, 2019Farooq et al., 2019;Podvezko, Sivilevicius, & Podviezko, 2014). The AHP is a powerful tool of multicriteria decision-making developed by Saaty (1980). The AHP is used for solving complex decision-making problems in different areas (e.g. civil engineering, transport, social and economic development, project selection, and materials science) (Podvezko et al., 2014). A number of articles related to the calculation of weights and alternatives for ITS or smart city criteria have been reviewed. Methods used are presented in Table 3.
AHP is one of the most popular MCDM methods for calculating weights and alternative criteria for a smart city, its mobility system, and infrastructure. Therefore, we have chosen the AHP method for further calculations.

Methodology
Weights for multicriteria evaluation criteria are based on expert judgement, i.e., expert evaluations provide the basis for determining subjective criteria weights. The opinions of individual experts are often contradictory. That is to say, the importance and priority ranking of individual expert criteria evaluation will vary.
It is important to use expert methods to determine the significance of indicators, otherwise, it is difficult to assess the reliability and alternatives of the study. Evaluations depend on the qualifications of the experts, specifics of the job, interest in obtaining certain evaluation results, work experience, etc. Criteria weights as aggregated averages of expert opinions can be used in a multicriteria evaluation provided that there is no contradiction between expert assessments, more precisely, opinions are proven to be statistically consistent (Podvezko, & Podviezko, 2014).

Gauging the level of concordance of expert opinions
The compatibility of evaluations can be determined using Kendall concordance coefficient, which is calculated by Eq. (1) (Sivilevičius, 2011): where S -sum of the squares of the total deviation from the evaluation results; p -the number of experts; m -the number of criteria. The sum of squares of the deviation from the overall mean is calculated using Eq. (2): where R j -the sum of the criteria rank; R -the overall average. The concordance coefficient W is a calculated variable that may take a random value. Therefore, significance of the coefficient must be calculated. The significance of the concordance coefficient is determined by Eq. (3): The consistency of the expert evaluation is determined by calculating the minimum value of the concordance coefficient using Eq. (4): If we get that W >W crit , it confirms that the expert opinions are harmonised.

Determination of subjective significance of indicators using APH method
AHP (Analytic Hierarchy Process) method, proposed by Saaty (1980), is one of the most well-known and most frequently applied in the world. This method is used in the majority of cases evaluating various transport projects involving organisational, technological, environmental and infrastructure solutions. AHP is particularly useful in terms of its ability to decompose a complex problem into its components and ease of use (Turcksin et al., 2011). The AHP method allows comparisons of individual indicators (criteria) in pairs, giving them importance in relation to each other. It elicits subjective comparisons from the decision-maker, and then synthesises these judgements into ratio-scale weights (Castillo & Pitfield, 2010).
The AHP method is convenient because it is easier for the experts to compare two criteria at a time, rather than all at once. It allows using qualitative assessment criteria to provide a quantitative assessment. The method is based on a pairwise comparison matrix P. Experts compare all evaluated criteria R i and R j (i, j = 1, ..., m), m -number of criteria. Saaty (2008) suggested using a five point (1-3-5-7-9) or sometimes nine-point scale (1-2-3-4-5-6-7-8-9), often used in practice (Table 4).
Elements of matrix P are numbers from p ij = 1, where the meanings of the two compared criteria are equal to p ij = 9, when criterion R i is incomparably more important than criterion R j . The elements p ij of matrix P can be regarded as ratios of weights R i ir R j : The matrix is reciprocal p p ij ji 1 . In order to determine the weights, the problem of the real values of matrix P and the real vectors w = (w 1 , w 2 , ..., w m ) T is solved.
The degree of compatibility of individual assessments of each expert is determined by consistency index C.I. and consistency ratio C.R. The Compatibility Index is defined (Saaty, 1980) as the ratio expressed by the formula: The better the matrix compatibility, the lower the compatibility index value is. Ideally C.I. = 0.
The ratio of specific matrix computed compatibility index C.I. and the average random index R.I. is called the compatibility ratio; it evaluates the degree of compatibility of the matrix: The matrix is considered compatible when compatibility ratio C.R. is less than 0.1 (Saaty, 1980).

Determination of subjective significance of indicators using ranking method
Using the expert ranking method, it is necessary to reform the ranking. The purpose of rearrangement is to assign weights in descending order of the rank. Doing so would give the top rank the highest value. The most accurate result is the linear transformation of estimates. In this case, the values of criteria weights can be calculated using the formula (Sivilevičius, 2011): where R j -the average (normative) of the criteria rank.

Evaluation criteria of the smart mobility system
In order to create the hierarchical evaluation system model, 5 factors and 23 indicators were selected on the basis of literature review. In the context of this paper, these factors are groups of indicators describing different measures that could make a mobility system smart. All 5 groups are closely related to each other, as is the whole mobility system.

Selection of indicators for the smart city mobility system
5 factors and 23 indicators have been selected using the exclusion method (Figure 1) rental services that substitute for private automobile ownership; A4. Carpooling. Also called ridesharing, the purpose of the system is to bring together the users, who travel in the same direction with the same vehicle; A5. Park & Ride system. Park and ride facilities are parking lots with public transport connections that allow commuters and other people heading to city centres to leave their vehicles and transfer to a bus, rail system, or carpool for the remainder of the journey; B1. Use of alternative fuels and renewable energy sources in the public transport. The alternative to fossil fuels: electricity, liquefied gas, methane, hydrogen, biodiesel, fuel cells; B2. Environmentally friendly private vehicles. These include electric, hydrogen vehicles and others that use alternative fuels rather than fossil fuels. B3. Modern parking solutions. These are the parking lots with a routing system and/or reservation of parking space; B4. Emissions Testing and Mitigation. The system uses advanced sensors to monitor areas of polluted air and control access to such areas; C1. Smart pedestrian and bicycle crossings. Built-in hardware detects pedestrians and activates roadside light panels and vertical panels on either side of the road, sensor-driven cycling signal management, creating green waves. C2. Smart speed reduction. Sensors of the automatic speed reduction system detect when the number of cars exceeds the set limits on the streets and automatically reduce the speed limit; C3. Traffic monitoring systems. The travel safety system helps create a safe environment for passengers, drivers and support staff.
The sensors monitor the environment in the vehicles, stations, parking lots, basic transit infrastructure; C4. Automated control of safe travel for commercial vehicles. This is a system that gives drivers alerts on their driving characteristics, vehicle condition and road condition. The system monitors the performance of components such as tyres and brakes and warns of impending failure. A start-up compatible alcoblock can be installed; C5. Autonomous vehicles. The equipment installed in the cars tracks the environment, has databases of the road and surrounding areas that are constantly scanned. The system provides information to the driver and helps drive safely. Autonomous driving is also possible in some sections; D1. Traffic lights management system. Controlled and coordinated or adaptive control traffic lights, automatically controlled in response to traffic conditions or upon request of pedestrians; D2. Vehicular communication systems.
Vehicle-to-everything (V2X) system components monitor the environment. The system incorporates vehicle-to-vehicle (V2V) and vehicleto-infrastructure (V2I) technologies, which detect and prevent potential obstacles and alert other road users. 5G internet connectivity provides autonomous car development opportunities; D3. Information, travel planning and online payment. Electronic ticketing on-board, interactive information on routes, timetables and waiting times, travel planner for optimal routes, dedicated website for travel tickets; D4. Variable information and message signs. Automatic road signs (variable message), SMS traffic alerts, electronic payment of parking, dedicated info applications for mobile devices, real-time public transport information; E1. Special traffic lanes. Only vehicles meeting the lane marking requirements (buses, high-occupancy, electric cars, taxis) can drive in these lanes. Smart special lanes must be equipped with sensors to detect offenders -the ANPR system is used; E2. Restricted traffic zones. Smart restricted traffic zones capture vehicles entering the area. If the system does not find a specific vehicle on the list of vehicles allowed to enter the area, a fine will be imposed;

E3. Electric vehicles charging stations, hydrogen stations.
These include electric vehicles charging and hydrogen filling stations, and other smart electric vehicles charging applications; E4. Smart street surfacing. These are the surfaces of the street specially equipped: illuminated, heated, de-icing, charging electric vehicles, etc.; E5. Smart street lighting. Smart street lighting controls work with climatic conditions, time of the day, traffic volume, and more. The main advantage of this system is the centralised control of all luminaries connected to the system, remote lighting-level change, information on luminary malfunctions, pavement, traffic conditions. The selected indicators are grouped into five groups (factors) according to their influence on the mobility system. The model for evaluation of the smart city mobility system and its infrastructure was created (Figure 1).
The factors and indicators shown in Figure 1 are used as criteria and sub-criteria in the calculation of their weights by AHP and ranking methods. After calculating the weights of the factors and indicators using both methods, the final weights of the indicators that can be used for comparison of the smartness levels of city mobility systems are calculated.

Calculation of subjective significance of the indicators
Based on the structure of the presented hierarchy model, five survey questionnaires were prepared for the assessment of individual factor indicators and the sixth questionnaire -for factor assessment. Questionnaires were forwarded to the experts for completion in two ways: ranking the factors/indicators and completing the paired comparison tables drawn up by the AHP method. The experts were selected strictly according to their competence in the fields of transportation and civil engineering. These are the heads of the following institutions: ME "Susisiekimo paslaugos" for transport services in Vilnius City (3), Lithuanian Road Administration under the Ministry of Transport and Communications (1), the Ministry of Transport and Communications of the Republic of Lithuania (1), VGTU Road Research Institute staff with a scientific degree (3), researchers of the Department of Roads, Faculty of Environmental Engineering, VGTU (5) and researchers of the VGTU Faculty of Transport Engineering (2). The total of 15 experts were interviewed one by one or in exceptional cases in small groups, as the AHP method is quite complicated and requires an explanation by an interviewer.
Factor and indicator weights, compatibility of expert opinions and other necessary calculations were performed with the Microsoft Office Excel package using the above-presented formulas.
W crit was calculated first with significance levels α = 0.5 and v = 5 -1 = 4 or v = 4 -1 = 3, depending on the number of factors and the number of indicators in the group.
The Kendall concordance coefficients for W factors and indicators were calculated after receiving 10 completed questionnaires. The Kendall concordance coefficients, calculated for the factors (A, B, C, D, E) and the factor "Motor travel and congestion reduction measures" (A1, A2, A3, A4, A5), W < W crit = 0.15813, which means that expert assessments are   (Table 5), therefore, it can be stated that the expert estimates are not contradictory and opinion averages can be used in multicriteria evaluation. Expert opinions were the most consistent when evaluating the indicators of pollution abatement measures (W = 0.65156) and traffic management tools and services (W = 0.41511). Most inconsistent expert opinions were on motor travel and congestion reduction measures (W = 0.16000). This indicates that some of the indicators for this factor collected a similar number of expert-ranked estimates.
The AHP matrices were immediately checked for compatibility, and when the matrix was found to be misaligned, the errors were explained to the expert and the matrix data were corrected. Otherwise, many matrices would have to be declared ineligible for calculations. Alternatively, if an error was found, an expert was asked to refill the matrices.
After obtaining criteria rank tables and pairs of comparison matrices filled in by the expert, calculations were performed: true vectors w i of each of the six matrices of each expert were calculated, the rankings were determined, the maximum true values of λ max were calculated, compatibility indices C.I. and compatibility relations C.R. were determined. All six matrices completed by fifteen experts (90 in total) were harmonised, C.R. < 0.1. Therefore, it can be stated that these experts were consistent in completing the pairing comparison matrices and did not provide contradictory estimates. The following is an example of a completed pairwise comparison matrix with calculations performed by a random expert filled in by a random pairwise comparison matrix and the results of its calculations (Table 6).
After calculating all 90 matrices and ranking them according to their vector w i it was found that the ranks calculated in all matrices correspond to the ranks given by the experts in the survey questionnaires. The criteria weights w i, summarised and calculated from the evaluations provided by all the experts, are calculated as arithmetic averages of the AHP weights.

Results
The averages and weightings of all experts' weighting factors calculated by the AHP method were calculated according to formula (9) are presented in Table 7. The calculated weighting sequence (ranks) may not always be the same using AHP and ranking methods. The rankings for some factors such as Travel Safety and Accident Reduction measures and Smart Infrastructure measures did not match.

Analysis of Result Comparison
Rank weights calculated by AHP and ranking method are different, so the results are shown in the diagram (Figure 2). Criteria weighting done by both methods indicates that the AHP method is "more sensitive" than the expert ranking method. The

AHP-weighted maximum difference between indicators of Pollutant Abatement Measures, Environmentally Friendly Vehicles and Modern
Parking Solutions measures is by 15 percentage points higher than that calculated by the ranking method.

Summarised indicator weight
We will use averages obtained by both methods for further research. Final indicator priorities are calculated according to Saaty (2008) methodology. "We need to multiply each ranking by the priority of its criterion or subcriterion and add the resulting weighs for each alternative to get its final priority. We call this part of the process, synthesis" (Saaty, 2008). Rank weights by factors and indicators are shown in Figure 3.
According   To conclude, more focus is made on the measures aimed to improve the functionality of the transport system than on individual smart technologies. The assessment of factors and indicators may be subjective due to the nature of the expert's work.

Analysis of the sensitivity of criteria values
To solve multi-criteria problems using quantitative multiobjective decision-making methods, deterministic calculations are performed without estimating randomness. Decision makers apply existing models that are strictly mathematically based, regardless of possible inaccuracies in the original data. The resulting solution then does not adequately reflect the real situation and may raise doubts (Simanavičienė & Ustinovičius, 2011). Results of application of any MCDM method depend on criteria weights to a great extent. Sometimes, the final selection may change when there is a change in the weight coefficients of the criteria (Biswas et al., 2020). In order to ensure that the results obtained are valid and applicable in the real world, it is necessary to perform sensitivity analysis and check the stability of the final results. Changes in the weight values of the criteria are analysed to determine how their significance affects the results. Dynamic sensitivity analysis shows that the change in the priorities of one criterion influences the change in the priorities of other criteria and the priorities of the alternatives within the observed criterion (Stanković et al., 2019). Sensitivity analysis was performed by comparing the impact of the change in the criterion with the highest weight on the final results. The sensitivity analysis of the weights of the indicators belonging to   (Figure 4). The dominant value of the Park and Ride System indicator (A5) with the highest weight is 0.8 higher compared to the next weight-based Bike Sharing indicator (A2). The Carpooling (A4) indicator, which has the lowest weight, ranges from 0.45. This means that a small change in the weight value of the Park and Ride System (10%) will not affect the changes in the significance of the indicators of this factor.
The sensitivity analysis of the weights of the indicators belonging to the factor Pollution Abatement Measures is presented in Figure 5.
The indicators Environmentally Friendly Vehicles (B2) and Use of Alternative Fuels and Renewable Energy Sources in Public Transport (B1) have a very small difference in weights, therefore the dominance of the indicators changes when the value of indicator B2 decreases by at least 5%. Other indicators of this factor dominate from 0.4, so a small change in indicator B2 will not affect their importance. The sensitivity analysis of the weights of the indicators belonging to the factor Travel Safety and Accident Reduction Measures is presented in Figure 6.
The indicators Smart Speed Reduction (C2) and Smart Pedestrian and Bicycle Crossings (C1) have a very slight difference in weights, so when the value of indicator C2 decreases by at least 1%, the dominance of the indicators changes. Other indicators of this factor dominate from 0.85, therefore, a small change in indicator B2 (10%) will not affect their importance.
The sensitivity analysis of the weights of the indicators belonging to the factor Traffic Management Tools and Services is presented in Figure 7.
The dominant value of the factor Traffic Light Management System (D1) is up to 0.93. This means that a small change in the value of the indicator Vehicular Communication Systems (D2) (10%) will not change the significance and dominance of the indicator.
The sensitivity analysis of the weights of the indicators belonging to the factor Smart Infrastructure Measures is presented in Figure 8.
The predominant value of the factor Special Traffic Lanes (for public or special transport) (E1) is up to 0.81. This means that a small change in the value of the indicator Restricted Traffic Zones (E2) (10%) will not change the significance and dominance of the indicator.

Conclusions
This article has presented results of application of comprehensive multicriteria analysis method for comparing smart mobility systems of the cities based on the intelligence of their transport systems considering all evaluation criteria.
Most current mobility assessment practises involve analysis of traffic (movement of vehicles) or mobility (movement of people and goods). They tend to favour road transport over other forms of accessibility, including alternative modes of transport, mobility management and infrastructure.
The results of expert evaluations of the factors and indicators can be applied in practice, provided that there is a sufficient degree of harmonisation among expert opinions. In the expert survey, the concurrence of expert opinions on all factors and indicators (questionnaires) was achieved by interviewing 15 experts.
The next step in this research is to develop an evaluation system and collect indicator data. Later it will be possible to compare mobility systems in the selected cities.