Keynote

Abstract

Major key players in the software arena no longer view Open Source Software as simply a research subject or a type of software developed by groups of enthusiastic amateurs. Ethical and political reasons also motivate the use of Open Source Software. As a result, Open Source Software components are now at the core of many software systems that are widely used by large numbers of users. However, the quality of Open Source Software, notably its trustworthiness, is sometimes considered as an issue. Open Source Software provides both opportunities and challenges that may be relevant for quality assessment by software developers, companies, and final users. The talk describes the approach used in the QualiPSo Project, a EU-funded initiative that specifically addresses the trustworthiness of Open Source Software. The talk shows how the factors influencing trustworthiness have been identified, ranked, and assessed. In addition to the data and the results that have been obtained so far, the talk will also point out a few issues that Open Source Software development communities should address to increase the opportunities and reduce the challenges related to Open Source Software quality assessment.

» keynote slides

Short Biography of Sandro Morasca

Sandro Morasca is a Professor of Computer Science at the Università degli Studi dell'Insubria in Como, Italy. In the past, he was an Associate Professor and Assistant Professor at the Politecnico di Milano in Milano, Italy. He was a Faculty Research Assistant and later a Visiting Scientist at the Department of Computer Science of the University of Maryland at College Park. Sandro Morasca has been actively carrying out research in the Software Engineering field in Empirical Software Engineering, Specification of Concurrent and Real-time Software Systems, Software Verification, and Open Source Software, and has published about 20 journal papers and 50 conference papers. Sandro Morasca has been involved in a number of national and international projects. He is currently the Leader of the activity related to the trustworthiness of Open Source Software products in the QualiPSo project, financed by the European Union. Sandro Morasca has served on the PC of a number of international software engineering conferences and the Steering Committee of the international conference "ESEM" and he serves on the editorial board of "Empirical Software Engineering: An International Journal," published by Springer-Verlag.

List of accepted papers

It is hard for security practitioners and decision-makers to know what level of protection they are getting from their investments in security, especially when they have invested in a number of technologies and processes which interact and combine together. It is even harder to estimate how well these investments can be expected to protect their organizations in the future as security policies, regulations and the threat environment are constantly changing. In this paper we propose that for measuring the effectiveness of security processes in large organizations, a greater emphasis needs to be put on process-based metrics, in contrast to the more commonly used symptomatic lagging indicators. We show how these process-based metrics can be combined with executable, predictive models, based on a sound mathematical foundation, to both assess organizations' security processes under current conditions and predict how well they are likely to perform in potential future scenarios, which may include changes in working practices, policies or threat levels, or new investments in security.

We present two case studies, in the areas of vulnerability threat management, and identity and access management, as significant examples to illustrate how this modeling and simulation-based approach can be used to provide a rich picture of how well existing security processes are protecting the organization and to answer "what-if" questions, such as exploring the effects of a change in security policy or an investment in new security technology. Our approach enables the organization to apply the metrics that are most relevant to its business, and provide a comprehensive view that shows the benefits and losses to the different stakeholders.

» slides

Security principles like least privilege and attack surface reduction play an important role in security engineering, especially in the early phases such as architectural design. However, the side effects of strategies meant to introduce a principle in an architecture have not been studied so far. Therefore it is hard if not impossible to make grounded trade-off decisions when it comes to the interplay among security principles and between security and other qualities. This paper tackles this problem from a quantitative perspective by presenting the results of three experiments in the context of three case studies.

» slides

The growing number of software security vulnerabilities is an ever-increasing challenge for organizations. As security managers in the industry have to operate within limited budgets they also have to prioritize their vulnerability responses. The Common Vulnerability Scoring System (CVSS) aids in such prioritization by introducing a metric for the severity of vulnerabilities. In its most prominent application as the severity metric in the U.S. National Vulnerability Database (NVD), CVSS scores lack all information pertaining potential exploit victims' context. Research has however shown that the severity of vulnerabilities varies greatly among different organizational contexts. Therefore NVD and CVSS are of limited use for vulnerability prioritization in practice. Security managers could address this limitation by adding the missing context information themselves to improve the quality of their CVSS-based vulnerability prioritization. It is unclear for them however, whether the potential improvements are worth the additional effort.

We present a method that enables practitioners to estimate these improvements. Our method is of particular use to practitioners who do not have the resources to gather large amounts of empirical data, because it allows them to simulate the improvement potential using only publicly available data in the NVD and distribution models from the literature. We applied the method on a sample set of 720 vulnerability announcements from the NVD and found that adding context information significantly improved the prioritization and selection of vulnerability response process. Our findings contribute to the discourse on returns on security investment, measurement of security processes and quantitative security management.

» slides

The Common Vulnerability Scoring System (CVSS) is a specification that is used to measure and score the relative severity of software flaw vulnerabilities, primarily for use in prioritizing patching efforts. CVSS version 2, which was finalized in 2007, was designed to address several deficiencies discovered during analysis and use of the original version of CVSS. This paper analyzes CVSS version 2 to determine how effectively it addresses the deficiencies in the original version and to identify any major deficiencies the new version may have. This analysis is based primarily on an experiment that applied both CVSS version 1 and version 2 scoring to a large set of recent vulnerabilities. The analysis also involved examination of the theoretical characteristics of version 1 and version 2 scores.

» slides

This paper quantitatively analyzes sectoral security interdependency based on an Input-Output analysis. Related prior studies including Inoperability Input-Output Model, IIM, show sectoral impacts of security incident from the viewpoint of interdependency. However, there are mainly two limitations. Firstly, they do not count each sector's feature in terms of information and communication technology, ICT, and information security, IS. Secondly, they focus on a sectoral damage of IS incident but not a level of security interdependency itself. The author proposes practical methodology to measure sectoral information security interdependency by introducing forward linkage and backward linkage analyses into IIM. The methodology counts each sector's dependency of ICT and its level of IS measurement. Furthermore, the paper applies the methodology to recent statistical data of Japanese sectors and shows implications of sectoral security interdependency.

This paper presents the quantitative characterization of vulnerability life cycle and of exploit creation by probability distributions. This work aims at helping the production of quantitative measures of information system security considering system environment. In this paper, we focus on two environmental factors: 1) the vulnerability life cycle and 2) the attacker behaviour. We look for the probability distributions and their parameters that could model quantatively these environmental factor events. Thus, to obtain precise measures, it is needed to characterize these events using real data. For that purpose, we first selected an appropriate vulnerability database by comparing the existing and available ones. We choose the Open Source Vulnerability DataBase. After having brought back the data we need, we evaluate quantitatively the model parameters related to the vulnerability life cycle and the attacker behaviour. In doing so, we look for specificities of vulnerability categories to define the parameterization of our quantitative security evaluation modelling more precisely.

» slides

In an empirical study of fourteen widely used open source PHP web applications, we found that the vulnerability density of the aggregate code base decreased from 8.88 vulnerabilities/KLOC to 3.30 from Summer 2006 to Summer 2008. Individual web applications varied widely, with vulnerability densities ranging from 0 to 121.4 at the beginning of the study. While the total number of security problems decreased, vulnerability density increased in eight of the fourteen applications over the analysis period. We developed a security resources indicator metric, which we found to be significantly correlated (rho = 0.67) with change in vulnerability density over time. Traditional software metrics, such as code size, cyclomatic complexity, nesting complexity, and churn, had much smaller correlations (rho = 0.31 at best) with vulnerability density. Vulnerability density was measured using the Fortify Source Code Analyzer static analysis tool.

» slides