• <tr id="yyy80"></tr>
  • <sup id="yyy80"></sup>
  • <tfoot id="yyy80"><noscript id="yyy80"></noscript></tfoot>
  • 99热精品在线国产_美女午夜性视频免费_国产精品国产高清国产av_av欧美777_自拍偷自拍亚洲精品老妇_亚洲熟女精品中文字幕_www日本黄色视频网_国产精品野战在线观看 ?

    Empirical Research on the Application of a Structure-Based Software Reliability Model

    2021-06-18 03:36:20JieZhangYangLuKeShiandChongXu
    IEEE/CAA Journal of Automatica Sinica 2021年6期

    Jie Zhang, Yang Lu, Ke Shi, and Chong Xu

    Abstract—Reliability engineering implemented early in the development process has a significant impact on improving software quality. It can assist in the design of architecture and guide later testing, which is beyond the scope of traditional reliability analysis methods. Structural reliability models work for this, but most of them remain tested in only simulation case studies due to lack of actual data. Here we use software metrics for reliability modeling which are collected from source codes of post versions. Through the proposed strategy, redundant metric elements are filtered out and the rest are aggregated to represent the module reliability. We further propose a framework to automatically apply the module value and calculate overall reliability by introducing formal methods. The experimental results from an actual project show that reliability analysis at the design and development stage can be close to the validity of analysis at the test stage through reasonable application of metric data. The study also demonstrates that the proposed methods have good applicability.

    I. INTRODUCTION

    THE essence of software reliability engineering lies in improving software quality, and its role covers all development stages. Most software reliability studies use failure data from the test stage, which focuses on predicting the relatively stable interval of the reliability growth curve and providing the basis for adjustment of test workload and software release strategy. Recent research suggests that reliability evaluation in the early stages has important implications for avoiding possible revision costs in the middle and later stages of development, especially for reliabilitysensitive and safety-critical software systems [1]-[4].Generally, in order to optimize software design, appropriate methods can be introduced as early as possible to evaluate the overall reliability of different architectures. Structural reliability modeling works for this, which takes a component as a basic granularity and considers interaction modes and structural styles of component compositions. Typical models include semi-Markov process (SMP) [5], discrete time Markov chain (DTMC) [6], and continuous-time Markov chain (CTMC) [7]. Compared with the traditional reliability models such as Goel-Okumoto (G-O) [8], they belong to a Markovian model which is based on a stochastic modeling method.

    Markovian models will be state-explosion-prone when used in large-scale and complex software. As such, state-free models are also used to evaluate software system recently.Zhenget al. [9] present an analytical approach based on Pareto distribution for performance estimation of unreliable infrastructure-as-a-service (IaaS) clouds. Liet al. [10] employ an autoregressive moving average model in time series for quality-of-service (QoS) prediction of real composite services.The above method considers the dynamic change in continuous runtime, but does not focus on the static analysis of component structure. Xiaet al. [11] use a stochastic-Petrinet to calculate the process-normal-completion probability as the reliability estimate since Petri-nets are highly capable of describing complex component-based software. However, this study still needs experimental data and is not suitable for software reliability evaluation at early design stage. In contrast, the structural model represented by DTMC is more suitable for a reliability assessment at this stage, because it usually does not require runtime data.

    Stochastic modeling is limited in practical applications of structure-based software reliability analysis. Take the DTMC model as an example. Its two key parameters required for modeling, component reliability and control transfer probability among components, are given by simulated cases rather than from actual projects [12]. Consensus of previous research is that a single component can be regarded as a black-box, and the component parameters tend to be stable in continuous reuse and iteration. But reliability cannot be the inherent property of software components because of the difference in requirements and external environments between actual projects. Moreover, the state explosion caused by component-level modeling is also a difficult problem in the application of large-scale complex software. Recently, more attention was paid to service/cloud-based software reliability.Xiaet al. [13] present a stochastic model based on a Poisson arrival process for quality evaluation of IaaS clouds. This work considers expected request completion time, rejection probability, and system overhead rate as key metrics, and can be used to help design and optimize cloud systems. Luoet al.[14] propose a scheme to build the ensemble of matrixfactorization based QoS estimators and achieve an AdaBoostbased aggregation result. This work presents high estimation accuracy at low time cost. Furthermore, the authors incorporate second-order solvers into a latent-factor based QoS predictor which can achieve higher prediction accuracy for industrial applications [15]. The above studies come from actual QoS data and emphasize the role of different measure factors in software performance analysis and prediction. When conducting a structural reliability analysis of a software project, an important revelation is whether the actual software metrics data can be effectively used. In order to avoid state explosion, the trade-off of module granularity should be based on actual project scale and difficulty of obtaining metrics.

    Recent reliability empirical studies most use traditional software-reliability-growth-models (SRGMs) which focus on failure data from the test stage. Luan and Huang [16] present a Pareto distribution of faults in large-scale open source projects for better prediction fitting curve. Sukhwaniet al. [17] apply SRGMs to NASA’s SpaceFlight software to analysis of relevant experience information in the software development process and version management. Aversano and Tortorella[18] propose a reliability evaluation framework for free/open source projects. It was applied to evaluate quality of an open source ERP (enterprise resource planning) software based on a project bug report. Hondaet al. [19] discuss the effect of two types of measurement units — hourly time and calendar time — in reliability prediction of industrial software systems.Tamura and Yamada [20] present a hierarchical Bayesian model based on fault detection rate around a series of open source solutions and address the impact of conflict behavior of components when dealing with system reliability. This work considers structural information at the component-level, but only takes into account the noise caused by these conflicts in the early stage of the fault detection curve due to the missing discussion of the structural analysis.

    The above works cannot be employed in the early stage of development since they require failure data obtained from software tests. Failure data can be attributed to the scope of software metrics. Besides, another type of software metrics has been applied to reliability analysis and defect prediction of actual projects. Complexity measurement data, which has low collection cost, can also be utilized in cognitive modeling[21]. Shibataet al. [22] combine a cumulative discrete-rate risk model with time-related measurement data, and prove that the new model is equivalent to a generalized fault detection process whose goodness of fit and predictive performance are better than the popular NHPP SRGMs. Kushwaha and Misra[23] consider the importance of the cognitive measure of complexity and use it in a more reliable software development process. Chu and Xu [24] present a general functional relationship between complexity metrics and software failure rate, which can be used to predict reliability on exponential SRGMs. Bharathi and Selvarani [25] calculate the reliability influence factors separately for several object-oriented design metrics, and finally, deduce the reliability formula of the class-level granularity by merging these factors through an aggregation strategy. This work is a beneficial trial for evaluating software reliability during the design phase.D’Ambroset al. [26] compare the performance of several software defect prediction methods and explain the factors of threat validity in practical applications. This work is generally based on static source code metrics and dynamic evolution metrics. Zhanget al. [27] group existing defect prediction models based on software metrics into four categories.Through verification of a large number of open source projects, they describe how to aggregate these metrics in order to achieve a significant effect on predictive performance and recommend the simple and efficient aggregation scheme of summation.

    However, measurement and SRGM-based empirical studies have several limitations: 1) they cannot calculate the reliability in the early design stage of software system; 2) they are not structure-based methods, and cannot assist in architecture design and optimization. Markovian models such as DTMC can work for this, but they usually have great difficulty in practical application due to lack of real data. We have seen the combination of software metrics with traditional reliability models and detect-prediction methods [22], [24],[25]. We aim to incorporate software metrics into early modeling for software reliability in order to effectively quantify system performance. Our empirical study focuses on the following two research questions:

    RQ1:From the perspective of early reliability analysis, what kind of structure granularity is appropriate and how does one use software metrics?

    RQ2:How does one evaluate overall system reliability in practical engineering?

    The remainder of this paper is organized as follows. Section II introduces the reliability model DTMC and the formal method we used for model construction and calculation. Section III describes the experimental methods of this paper, including object selection, metric data processing and aggregation. The results of this study are presented in Section IV. We evaluate the performance of our approach against other models in Section V, and discuss the threats to validity of our work in Section VI. Conclusions are drawn and future work is described in Section VII.

    II. RELATED WORK

    In this section, we introduce the typical early reliability model first, and then present the method we utilized to apply the model.

    A. Structure-Based Reliability Model

    In the early stage of software development, the traditional reliability model is not available due to lack of failure data. A class of structure-based models work for this [1]-[3]. The most popular one is the DTMC (discrete time Markov chain)model [6]. Here is an example of the ESA’s (European space agency) control system software [28]. It contains four main components (N1toN4), and the control transfer between components as shown in Fig. 1.

    Correspondingly, we have a one-step stochastic transfer matrix as follows:

    Fig. 1. The ESA software architecture.

    The entryQi,jrepresents the transition probability from stateitojin the Markov process. Here,Qi,j=R1P1,2, reflects the probability that the system execution state is transferred fromN1toN2in one step, which equals to the product of reliabilityR1and the transfer probabilityP1,2.R1expresses the probability of executingN1successfully. Each componentNimay fail with the probability 1-Ri, and each component failure causes system failure. This is a failure independent hypothesis for this model.

    The powerQnof the matrixQis defined as ann-step stochastic transfer matrix whose entryQi,jnreflects the transition probability from stateitojinnsteps. The Neumann series of matrixQis

    whereIis the identity matrix. LetN1be the starting module andN4the ending module. The entryS1,4in matrixSdescribes the transition probability fromN1toN4through all possible steps. Then the system reliability can be expressed as the probability of successfully reachingN4and successfully executingN4, which can be calculated by

    This model emphasizes the influence of structural changes on overall reliability from the perspective of control flow.Subsequent improvements have been proposed on this basis.

    B. Algebraic Method

    The DTMC model is built around the control transfer relationship between components. However, the reality is that designers seldom use it to represent the system architecture,and module developers only generate control flow graphs at the method-level. For this we have provided an easy-to-use method in our previous studies [29]. In general, we use an algebraic paradigm instead of graphical expression for higher abstraction. The transfer relationshipN1→N2in Fig. 1 is expressed asN1⊕N2, where the operator ⊕ denotes that the basic control transfers between components are motivated[30]. So Fig. 1 can be expressed as a set as follows:

    This solves the problem that where the system structure is too complicated for graphic expressions. Especially, when components are designed as a coupling substructure, such as parallel, it is naturally solved by adding an operator in algebra.We can even express the nesting of structures by bracketing them.

    We propose a parser which left-to-right scan and rightmost derivate (LR) each expression in the collection as above. The following functions are used in the parser algorithm:

    where C is a collection of components, S is a collection of state nodes. This series off? for different algebraic expressions are used to generate system state nodes which contains reliability attributes, and mark them on the transition matrixQ. The elements in the matrix are constantly updated in the scan. When the scan is complete, the matrix can be directly applied to (2).

    This formal method can be easily instrumentalized to facilitate engineers. In the simulation study, it performed well.We summarize the above method as a process framework, as shown in Fig. 2.

    Fig. 2. A framework for automatically applying DTMC.

    III. EXPERIMENTAL DESIGN

    Properties of the research object and its experimental data are explained in detail. Some methods are applied to reasonably preprocess and aggregate metric data. And the improved framework is used to calculate the overall reliability.

    A. Research Object

    We select the open source project jEdit [31] as the research object. jEdit is a mature programmer’s text editor with hundreds of person-years of development behind it. It is written in Java and runs on any operating system with Java support. This project is in proper scale, and it is representative for development technology.

    The current version is set as jEdit 4.3. It is because that the metric data of jEdit in the PROMISE repository [32] contains version 3.2.1 to version 4.3. Although version 4.3 is not the last stable version, it does not affect the universal property of this study since there is no fundamental change in higher versions. We consider ourselves as developers and designers,so we can get the necessary information of structure from thedesign document or source code. This information includes packages, files, and classes, as listed in Table I.

    TABLE I THE STRUCTURAL INFORMATION OF JEDIT 4.3

    Here we define the granularity of structural module analysis at the package-level, and the corresponding level can be found in other language environments. It needs to be based on the previous version when we are seeking detailed module information at the design stage. It usually works because of the limited changes in modules between versions. As the coding continues, we can continuously adjust the changes to the files and classes of the modules. The final structure information of version 4.3 is as described in Table I, and the metrics are based on this.

    B. Metric Data Processing

    The PROMISE library includes nearly thirty metrics for jEdit. We use three main categories— traditional metrics,object-oriented (OO) metrics, and process metrics — to describe the metric data. The data are summarized at the method-level, class-level and file-level respectively [33], and the developer can easily collect them via the specified tools.At the method-level, the number of lines of code (LOC) and the cyclomatic complexity (CC) [34] are still suitable for code analysis inside a class, which existed before object-oriented programming appeared. The CK set [35] has a wide range of applications, but there are also metrics that emphasize perspectives such as encapsulation, coupling, etc. [36]. The eight metrics recommended by Moseret al. [37] have typical process characteristics, and are further improved in the MJ set [38].

    Simple data sampling is not used in this paper. For early reliability analysis, the initial metric data needs to be cleaned to highlight the structural characteristics of the metric elements. We use the following method:

    1) The process metric elements (from Moser and MJ set) do not apply to static analysis for the target version. This work only uses traditional metrics and OO metrics.

    2) The remaining 20 metrics are divided into five categories: complexity, coupling, cohesion, inheritance and size. We use Spearman’s coefficient [39] to measure correlation in 5 subsets. The data is not required to follow any particular distribution in the state of Spearman correlation,and it ranges from -1 to +1 where a larger absolute value indicates the stronger correlation. Table II shows the calculation results (e.g., the coupling metrics).

    Absolute values of more than 0.5 have relevance and values of more than 0.8 have high correlation. Observe that there are redundant metrics that can be removed to further simplify calculations. According to the CFS algorithm of attribute selection [40], one metric with relatively high relevance is

    TABLE II CORRELATION OF THE COUPLING METRICS

    selected as a main feature in each category. The maximum cyclomatic-complexity (MAX_CC), coupling between object classes (CBO), cohesion among methods (CAM), depth of inheritance tree (DIT), and response for a class (RFC) are finally selected elements as representative for the five categories separately. It ensures that subsequent calculations which can characterize 5 different characteristics instead of examining correlations across all metrics. Table III lists the data we actually used in this research when deal with the browser module in Table I.

    TABLE III THE METRIC DATA USED IN THIS RESEARCH

    C. Aggregation Scheme

    The metric data are only collected at the class-level. It is necessary to propose a descriptive scheme to aggregate the final value of software system reliability. We suggest a twostep frame and the details are presented as follows.

    Step 1:To establish a functional relationship between metrics and degree of reliability in order to calculate module reliability value.

    We define the degree of reliability influence, which is similar to the definition in [25]. LetRImirepresent influence of the metric scalarmi. It is calculated by

    wheredpcindicates the number of defect prone classes atmi,andmcindicates the number of classes contained in one module. Notice thatRIis only calculated in one module, since the coding style of each module may not be consistent. The range of scalarmiincludes all samples of the specified metric element. Table IV listsmi,dpc, andRImiof 5 metric elements in the module browser, whenmc= 10.

    It is observed from Table IV thatdpcis counted for eachmiper elements, and eachmivalue is a sample of the metric space. The counting method ofdpcis summarized as: If the class, which is associated with the specific value ofmiat one metric element, a) is submitted with a bug in the previous version,dpc←dpc+ 1; b) or is newly developed andmiexceeds threshold,dpc←dpc+ 1.

    The thresholds of RFC(222), CBO(24), and DIT(6) refer to the NASA standard [41]. The threshold of CC(15) is derived from [34], and the threshold of CAM(0.50) comes from [38].

    In general, lower metric value means higher reliability. But it cannot be concluded that there is a linear correlation between metric value and reliability influence. We use polynomial regression to estimate the relationship between the two, because of changes of software reliability mostly in the form of curves. As an example, the functional estimation equations of the browser module are

    wherea,b,c,d,eare the independent variables in each fitting function. According to the average value of metrics listed in Table IV, the five degree of reliability influence in the browser module can be calculated as:RIRFC(89.4) = 90.941,RIMAX_CC(13.3) = 91.330,RICAM(0.335) = 96.414,RICBO(22.0) = 92.744,RIDIT(4.5) = 78.600.

    Aggregation strategy can significantly alter correlations between software metrics and the defect count. Zhanget al.[27] indicate that the summation strategy can often achieve the best performance when constructing models predict defect rank or count. Although the perspectives of defect prediction and reliability evaluation are different, they are effectively the same in terms of the use of metrics. The difference is that the former focuses on the situation after the current version, and the latter focuses on the current developing version.Therefore, in this paper, we use the summation strategy to calculate the reliability degree of the individual module according to the reliability influence of each factor. It is calculated as

    whereriis weight of theith metric element. Since the five metrics used in this paper are pre-screened to represent a logical category, they are considered equally important. Letri= 1/n= 1/5, where reliability of the browser module is 90.006(%) after the weighted summation.

    Step 2:To establish the structural reliability model according toRmodule.

    In the software design and development phase, reliability engineering often needs to be carried out by architects andcoders. The module developer calculatesRmoduleby counting the related metrics. One can gather this information of all modules to aggregate into system reliability.

    TABLE IV THE RI STATISTICS IN THE BROWSER MODULE

    Here we use the formal method presented in Section II-B to establish the DTMC reliability model quickly and accurately.The method is based on algebraic expressions, which can accurately express the control transfer between modules instead of using graphical representation. In addition to the value ofRmodule, module developers are required to submit related expressions based on their understanding of the business. For the browser module (N1), the developers need to submit:

    Note that the expressionN1⊕N21replaces the directed arc of graph to describe the control transfer relationship betweenN1andN21. It indicates that further processing of the file system will go to the utility module (N21).

    A module developer can separately submit expressions that confirm design intent, which formally starts with the developing module and links all possible next modules in the control flow. In some cases, architects can also submit or modify expressions based on overall understanding. As a description of the structure, algebraic expressions are precise and unambiguous. They are lightweight and easy to use for engineers. One can do this at the same time as designing and coding, without any distractions in describing the whole system.

    Eventually, an expression set can be collected which contains two key parameters required for the DTMC model —RiandPij.Ricomes from the submitted information which is usually attached to a concrete expression. Transfer probabilityPij, which is indicated byNi⊕Nj, is equally divided by all possible transfers from moduleNi. The LR parser presented in Section II-B can calculatePijautomatically during scanning.

    Finally, as an improvement to Fig. 2, the new framework is proposed as Fig. 3.

    Fig. 3. The improved framework for empirical research.

    IV. RESULTS

    This section provides our experimental results. We focus on findings from actual application of methods in Section III, and answer the research questions presented in Section I.

    RQ1:How to determine the appropriate structure granularity in order to apply the corresponding metrics to reliability analysis?

    In general, software design has the characteristics of modularization, which is advantageous to the maintenance of projects and the deployment of resources. Early reliability analysis can only start with the structure of the software system. The structural characteristics are represented by the relationships between modules. This requires selecting the appropriate module granularity. In this paper, we suggest that Java OO projects should be analyzed at the package-level. The corresponding granularity of other types of OO projects can be found in the directory structure and design documents.

    Table I lists the module division of jEdit. Some modules of version 4.3 do not exist in previous versions, and some have changed in later versions. However, analysis at the packagelevel can cover all relevant metrics collected at the method- or class-level. Different packages show significant differences in metrics due to the diversity of functionality and developer. We randomly choose eight out of 23 modules, and present boxplots of them under the five selected metrics. As shown in Fig. 4, the distributions of metric data are obviously different,and most of them are different from the entire distribution. It shows that the coding style of each module is different, i.e.,the quality of each module in the same software is also different. Furthermore, the life cycle and application scope of many components (packages in java projects) goes beyond the project itself. E.g., “util” and “xml”, which are independent of jEdit’s main business, came from or could be used for other projects. Therefore, we think that for each different module,the reliability influence from its metric data should be analyzed separately. And then the reliability data of all modules should be integrated with a specific structural analysis method.

    Fig. 4. Boxplots of 5 metric values resourced from eight modules and whole.

    RQ2:How to evaluate the reliability of a software?

    We calculate the reliability value of a single module based on (6) and (8). Table V lists the reliability degree for all modules in 4 versions of the jEdit project. Note that version 4.3 is the last version in PROMISE repository, and it is also the one set for the current review. The first version 3.2.1 is discarded because we need to trace back a version for countingdpcin (6).

    Fig. 5 presents the boxplots of the numerical results in Table V.It shows that statistically, the overall distribution gradually increases as the version changes. It can be seen from the diagram that most module reliability values increase as the version changes. This is in line with the expectation that the module quality will improve as more bugs are fixed. From the perspective of structural analysis, we still want to estimate a specific value to reflect the overall reliability of software.

    With version 4.3 as the goal, module developers should present the relationship between this module and other modules in addition toRmodule, which can be described conveniently by algebraic expressions. Then, the expression set of {N12⊕N3,N12⊕N6,N12⊕N19, …} can be collected.ExpressionN23⊕Cshould be added into the collection in order to facilitate model calculation, which means the requirement of business termination is only initiated by the core moduleN23.Cis the ending node and its reliability value is 100%. Thestarting module isN12, which means that the message occurs when the business is coming.

    TABLE V THE RMODULE VALUES IN DIFFERENT VERSIONS OF JEDIT

    Fig. 5. Boxplot of reliability degree in all modules among four versions.

    We also set up an expression set for three other versions,and use the Fig. 3 framework to automate the overall software reliability calculation. Fig. 6 shows the results. To illustrate the effectiveness of the proposed method, two traditional growth models: the classical G-O model [8] and Huang’s testeffort model [42] are used as comparisons. In this work, we follow the common practice in empirical research of SRGMs[16]-[19], where the failure data is extracted from the bug reports. This is under basis of the following assumptions: 1)Confirmed bugs cause failures; 2) Each failure is independent;3) Fixing bugs will not introduce new bugs. According to the reports obtained from the official website of jEdit [31],version 4.0 has 226 closed bugs within 215 days since the previous release. The G-O model use a statistical process based on this failure intensity, and the prediction result at the release time of jEdit 4.0 is 88.736%. Version 4.1 has 167 closed bugs within 407 days, and the prediction result is 89.413%. The data for versions 4.2 and 4.3 show that there are 106 bugs within 557 days and 12 bugs within 1848 days. The prediction results are 92.185% and 96.587%,respectively.Furthermore, as suggested in [42], the cumulative interval is used here as the test-effort which is available from the detailed entry of each bug in the report. The prediction results of Huang model across the four versions are 90.156%, 91.142%,92.868% and 96.224%, respectively.

    Fig. 6. Reliability trends on four versions of jEdit.

    V. DISCUSSION

    A. Characteristics of Early Reliability Modeling

    Unlike SRGMs, early reliability modeling only relies on structural analysis of software system. As mentioned in the first section, the modeling parameters come from a single software module and the relationship between modules.Therefore, it is meaningless to compare the predictive performance with a class of models based on failure data.Most reliability-prediction models, such as SRGMs, are utilized to optimize the release strategy to achieve a balance between quality and test costs. But the significance of the early model is to optimize the structure and guide the test.

    We show the trend of changes in the results of the proposed method in Fig. 6, and also use two representative models to illustrate the correctness. It should be explained that the calculation of each version of this method can be performed at the design stage. And based on the framework in Section IIIC, the calculation process is automatically complemented by tools. This means that the reliability evaluation can be carried out at any time with structural design changes. It helps decrease the difficulty of applying the structural reliability model into practice.

    As shown in Fig. 6, the numerical values evaluated by the proposed method (87.267%, 87.184%, 89.381%, 94.223%)are lower than by the traditional models. This is because calculations based on metric data can magnify the impact of defect-prone classes. For the calculation of defect tendency,the method of this paper tends to be conservative.Correspondingly, the SRGMs which are based only on test data assume that the reliability curve increases with bugs fixing. Their evaluations are relatively optimistic in general.

    B. Sensitivity Analysis

    The influence of software local structure on reliability can be analyzed through sensitivity. This effect can only be manifested in the computation of the structural reliability model. As shown here jEdit 4.0 is upgraded to 4.1. The result,which differs from the traditional models’, appears to be slightly reduced. By partially fixing pre-existing defects, new versions can also have reduced reliability with the introduction of new modules and features (such asN7in jEdit 4.1).

    We use the criticality to analyze sensitivity. It is computed by the following formula:

    where ΔRiis the reliability increment of moduleNi, ΔRsysis the overall reliability increase corresponding to this increment.In fact, when the ΔRiis very small, we useCiinstead of the partial derivative, i.e., the criticality of moduleNiis an approximate value of the module sensitivity. However, in terms of computational complexity, computing criticality is much better than computing sensitivity. The sensitivity(partial derivative) onRican only be solved based on cofactor expansion, which leads to a geometric increase in computational time and space when the total number of modules grows. In contrast, (9) can be calculated automatically by the proposed framework in Fig. 3.

    For jEdit 4.3, we set a small negative increment of 0.005 for all module reliability. Fig. 7 shows theCicorresponding to each change of ΔRi. It can be seen that the reliability sensitivity of core moduleN23and message moduleN12are significantly higher than others. This conforms to the expectation of actual control flow at design stage. And this means that later testing on such modules will help significantly improve the overall reliability of software.However, some modules (such asN18) are not considered to be the important nodes in structure because they only interact with very few modules. These may not be the goals that reliability engineering needs to focus on.

    VI. THREATS TO VALIDITY

    1)Project Selection:In this work, the open source project jEdit is selected. In addition to using metrics, we need to analyze the software structure and apply a complex evaluation framework. We only discussed one project because of space constraints.

    Fig. 7. Analysis of reliability sensitivity of modules in jEdit 4.3.

    The threat to our treatment mainly arises from applicability of our method on other OO projects. Metric data can be obtained from public repositories, but also can be directly calculated from the source code. Structural information can be analyzed from design documents and source codes. We believe that the applicability of the proposed method is not a problem. Furthermore, it is recommended that non-java projects should be tested using our approach and the results may be slightly different.

    2) Metrics and Techniques Selection:These metrics and its classification we used are popular and commonly investigated in defect prediction literature. With the same kind of metrics,correlation analysis technique is used to eliminate redundant impacts. A study on more correlation analysis techniques is required. In addition, the DTMC model used in this paper is the most important structural analysis model at present. The proposed framework uses formal techniques to show the application of this model. Formal techniques have been fully elaborated and verified in our previous studies. We will provide an open-source implementation of analytical algorithms online. Replication studies using different early reliability models, such as CTMC, to utilize this framework and may prove fruitful.

    VII. CONCLUSION

    Although there are many practical difficulties in reliability analysis and evaluation of software projects in early development, the calculation results from the reliability model can provide reference for the optimization of software architecture design. By analyzing the software structure, we find the sensitive modules that affect the overall reliability,which provides a practical basis for subsequent reliability distribution and stress testing. Theoretical research of structure-based software reliability models has been studied extensively, but the biggest obstacle to its further development is determining how to apply it to practical projects.

    Our main work is to provide a complete solution for early reliability evaluation. We first give a method for processing metric data and aggregating it into module reliability. Then we propose a process framework, which automatically calculates the overall reliability value based on module parameters and expression forms. Using a series of different methods, our research shows the implementation of a structural reliability model in practical projects. Evaluation of an OO project shows that we can get an approximating result vs. traditional models which are based on software failure data. It provides new ideas and methods for the empirical research of reliability.

    The work is original and operable in the selection and aggregation of software metric data and the practice of structural analysis model. The next steps of research will be continued in two areas:

    1) Consider establishing a functional relationship between metrics and reliability on more metric elements. The fitting method of the function is not limited to one but also, selects the appropriate method according to different modules and developers, such as Stepwise Regression and Ridge Regression.

    2) Defect prediction Technology continues to show new results. In addition to the traditional methods, a method using random neural networks (RNN) has recently been proposed,and the results are good. This prediction model is established on the time axis and describes the evolution of each software module. We will consider the use of better-performing solutions in the prediction of defect-prone classes to improve the reliability evaluation of modules.

    国产白丝娇喘喷水9色精品| 亚洲色图综合在线观看| 亚洲精品成人av观看孕妇| 熟妇人妻不卡中文字幕| 国产毛片在线视频| 黄片无遮挡物在线观看| 国产亚洲最大av| 最后的刺客免费高清国语| 国产精品蜜桃在线观看| 一本一本久久a久久精品综合妖精 国产伦在线观看视频一区 | 国产午夜精品一二区理论片| 亚洲国产精品一区二区三区在线| 欧美精品国产亚洲| 亚洲欧美中文字幕日韩二区| 久久国产精品男人的天堂亚洲 | 国产精品嫩草影院av在线观看| 伊人久久国产一区二区| 国产熟女欧美一区二区| 免费不卡的大黄色大毛片视频在线观看| 少妇猛男粗大的猛烈进出视频| 蜜桃国产av成人99| 女性生殖器流出的白浆| 精品一区二区三区四区五区乱码 | 国产午夜精品一二区理论片| 91午夜精品亚洲一区二区三区| 我的女老师完整版在线观看| 久久国内精品自在自线图片| 亚洲国产看品久久| 国产永久视频网站| 一级毛片电影观看| 国产 精品1| 日本欧美视频一区| 天堂中文最新版在线下载| 国产成人91sexporn| 黄色毛片三级朝国网站| 亚洲精品456在线播放app| 免费看不卡的av| 精品人妻偷拍中文字幕| 国产高清国产精品国产三级| 国产无遮挡羞羞视频在线观看| 国产精品人妻久久久久久| 伊人亚洲综合成人网| 亚洲综合色网址| 又黄又粗又硬又大视频| 亚洲伊人色综图| 女人久久www免费人成看片| 亚洲天堂av无毛| 黑人欧美特级aaaaaa片| 人妻系列 视频| 国产精品国产av在线观看| 伊人亚洲综合成人网| 国产精品三级大全| 侵犯人妻中文字幕一二三四区| 在线观看免费视频网站a站| 在线看a的网站| 国产成人精品婷婷| 成年人午夜在线观看视频| 在线观看一区二区三区激情| 久久久久久久久久久久大奶| 国产日韩一区二区三区精品不卡| 美女视频免费永久观看网站| 90打野战视频偷拍视频| 少妇的丰满在线观看| 最近手机中文字幕大全| 亚洲欧美色中文字幕在线| 你懂的网址亚洲精品在线观看| 美女国产视频在线观看| 亚洲精品日本国产第一区| 男人添女人高潮全过程视频| 1024视频免费在线观看| 观看美女的网站| 亚洲av综合色区一区| 亚洲性久久影院| 日本与韩国留学比较| 两个人看的免费小视频| 纯流量卡能插随身wifi吗| 在线天堂中文资源库| 欧美人与性动交α欧美精品济南到 | 男女边摸边吃奶| 最新的欧美精品一区二区| 超色免费av| 国产一区二区在线观看日韩| 国产av国产精品国产| 欧美 日韩 精品 国产| 人妻一区二区av| 成人手机av| 国产成人av激情在线播放| 免费看av在线观看网站| 久久久久久久久久久免费av| 亚洲精品色激情综合| 热99国产精品久久久久久7| av在线老鸭窝| 国产av精品麻豆| 我的女老师完整版在线观看| 免费日韩欧美在线观看| 国产不卡av网站在线观看| 视频在线观看一区二区三区| 亚洲,一卡二卡三卡| 18禁观看日本| 欧美xxⅹ黑人| 制服人妻中文乱码| 天天躁夜夜躁狠狠躁躁| 制服丝袜香蕉在线| 久久人人爽av亚洲精品天堂| 久久精品国产亚洲av天美| 久久久久久久久久人人人人人人| 精品久久国产蜜桃| 国产成人欧美| 波野结衣二区三区在线| 五月玫瑰六月丁香| 天堂俺去俺来也www色官网| 久久久久久久久久成人| 99热网站在线观看| 日韩 亚洲 欧美在线| 午夜视频国产福利| 日韩中字成人| 99九九在线精品视频| 丝袜喷水一区| 国产极品天堂在线| 99久久综合免费| 涩涩av久久男人的天堂| 汤姆久久久久久久影院中文字幕| 在线观看免费高清a一片| 日日啪夜夜爽| 久久久亚洲精品成人影院| 制服人妻中文乱码| 晚上一个人看的免费电影| 国产白丝娇喘喷水9色精品| 成人国产av品久久久| 97在线视频观看| 青春草国产在线视频| 免费看光身美女| 成人国产麻豆网| 一级毛片我不卡| 夜夜爽夜夜爽视频| 日本av免费视频播放| 国产精品久久久久久av不卡| tube8黄色片| 久久免费观看电影| 亚洲欧美日韩卡通动漫| 久久婷婷青草| 亚洲欧洲国产日韩| 丝袜脚勾引网站| 18+在线观看网站| 亚洲熟女精品中文字幕| 国产精品久久久久成人av| 久久这里有精品视频免费| 2022亚洲国产成人精品| 国产免费视频播放在线视频| 亚洲丝袜综合中文字幕| 99香蕉大伊视频| av国产久精品久网站免费入址| 国产精品麻豆人妻色哟哟久久| 母亲3免费完整高清在线观看 | 国产免费一区二区三区四区乱码| 久热这里只有精品99| av播播在线观看一区| 大片免费播放器 马上看| 国产色婷婷99| 日韩不卡一区二区三区视频在线| 黄色视频在线播放观看不卡| 国产免费现黄频在线看| 国产高清国产精品国产三级| 熟妇人妻不卡中文字幕| 2022亚洲国产成人精品| 高清欧美精品videossex| 人妻 亚洲 视频| 91aial.com中文字幕在线观看| 男男h啪啪无遮挡| 亚洲第一av免费看| 日本午夜av视频| 美国免费a级毛片| 亚洲人成网站在线观看播放| 99久久中文字幕三级久久日本| 人妻少妇偷人精品九色| 成年人免费黄色播放视频| 我的女老师完整版在线观看| 亚洲精品456在线播放app| 在线天堂最新版资源| 高清av免费在线| 欧美丝袜亚洲另类| 国产熟女欧美一区二区| 看免费av毛片| 午夜视频国产福利| 少妇的丰满在线观看| 韩国高清视频一区二区三区| 免费观看在线日韩| 久久热在线av| 国产日韩欧美在线精品| 日本av免费视频播放| 亚洲综合精品二区| 99国产精品免费福利视频| 菩萨蛮人人尽说江南好唐韦庄| 亚洲国产色片| 国产精品一区二区在线观看99| 久久久国产一区二区| 欧美3d第一页| 纵有疾风起免费观看全集完整版| 99九九在线精品视频| 国产成人午夜福利电影在线观看| 黑人高潮一二区| 亚洲人与动物交配视频| 免费观看在线日韩| xxxhd国产人妻xxx| 纯流量卡能插随身wifi吗| 99精国产麻豆久久婷婷| 中文字幕最新亚洲高清| a级毛色黄片| 国产欧美亚洲国产| 伦理电影免费视频| 国产色爽女视频免费观看| 亚洲美女黄色视频免费看| 最后的刺客免费高清国语| 国产精品不卡视频一区二区| xxx大片免费视频| 欧美3d第一页| 日韩av在线免费看完整版不卡| 99久久综合免费| 18+在线观看网站| 色婷婷av一区二区三区视频| 免费看av在线观看网站| 1024视频免费在线观看| 国产老妇伦熟女老妇高清| 黄色一级大片看看| 久久综合国产亚洲精品| 巨乳人妻的诱惑在线观看| 免费高清在线观看视频在线观看| 亚洲国产色片| 中文字幕另类日韩欧美亚洲嫩草| 青青草视频在线视频观看| 久久久a久久爽久久v久久| 欧美精品一区二区免费开放| 黄色视频在线播放观看不卡| 久久久久久久久久人人人人人人| 亚洲色图综合在线观看| 午夜福利,免费看| 成人国语在线视频| 国产欧美日韩一区二区三区在线| 久久免费观看电影| 热99国产精品久久久久久7| 国产欧美亚洲国产| 免费大片18禁| kizo精华| 亚洲成国产人片在线观看| 欧美3d第一页| 丝袜人妻中文字幕| 亚洲精品日本国产第一区| 女性生殖器流出的白浆| 久久久久久久精品精品| 日韩一区二区视频免费看| 久久精品国产自在天天线| 51国产日韩欧美| 插逼视频在线观看| 人人澡人人妻人| 少妇精品久久久久久久| 一区二区av电影网| 在线亚洲精品国产二区图片欧美| 久久99精品国语久久久| 亚洲国产精品成人久久小说| 大陆偷拍与自拍| 黑人欧美特级aaaaaa片| 免费人成在线观看视频色| 国产国语露脸激情在线看| 麻豆乱淫一区二区| 欧美精品人与动牲交sv欧美| 老司机影院成人| 五月开心婷婷网| av一本久久久久| 国产亚洲精品第一综合不卡 | tube8黄色片| 精品国产乱码久久久久久小说| 亚洲五月色婷婷综合| 永久网站在线| 性色avwww在线观看| 日韩,欧美,国产一区二区三区| 九色成人免费人妻av| 国产深夜福利视频在线观看| 国产欧美日韩综合在线一区二区| 久久精品久久久久久久性| 亚洲精品aⅴ在线观看| 91精品伊人久久大香线蕉| 国产伦理片在线播放av一区| 欧美国产精品一级二级三级| 成年女人在线观看亚洲视频| 久久久久视频综合| 极品少妇高潮喷水抽搐| 人人妻人人澡人人看| 欧美成人午夜精品| 久久免费观看电影| 水蜜桃什么品种好| 日本黄大片高清| av国产久精品久网站免费入址| 国产探花极品一区二区| 亚洲国产精品一区三区| 宅男免费午夜| 制服人妻中文乱码| 97超碰精品成人国产| xxx大片免费视频| 高清在线视频一区二区三区| 精品久久久久久电影网| 久久久久久久国产电影| 欧美bdsm另类| 国产精品久久久久久久久免| 日本色播在线视频| 丝袜在线中文字幕| 中文欧美无线码| 最近的中文字幕免费完整| 中文字幕人妻丝袜制服| 高清欧美精品videossex| 久久精品人人爽人人爽视色| 男男h啪啪无遮挡| 精品人妻熟女毛片av久久网站| 精品久久国产蜜桃| 久久精品aⅴ一区二区三区四区 | 黄色怎么调成土黄色| 精品人妻在线不人妻| 99久久中文字幕三级久久日本| 久久人人97超碰香蕉20202| 久久99蜜桃精品久久| 午夜激情av网站| av.在线天堂| 9191精品国产免费久久| 亚洲成色77777| 国精品久久久久久国模美| 欧美激情国产日韩精品一区| 久久人人爽av亚洲精品天堂| 亚洲,欧美,日韩| 在线观看国产h片| 赤兔流量卡办理| 国产激情久久老熟女| 免费黄频网站在线观看国产| 亚洲精品乱久久久久久| 在线观看人妻少妇| 女人精品久久久久毛片| av在线老鸭窝| 啦啦啦在线观看免费高清www| 十八禁高潮呻吟视频| 免费在线观看黄色视频的| 国产成人欧美| 99热国产这里只有精品6| 啦啦啦啦在线视频资源| 哪个播放器可以免费观看大片| 国产日韩欧美视频二区| 国产精品偷伦视频观看了| 欧美精品亚洲一区二区| 午夜日本视频在线| 建设人人有责人人尽责人人享有的| 久久久久久久久久久久大奶| 一二三四在线观看免费中文在 | 高清欧美精品videossex| 色网站视频免费| 91精品伊人久久大香线蕉| 国产成人精品福利久久| 99久久综合免费| 久久久国产精品麻豆| 高清不卡的av网站| 亚洲成色77777| 亚洲高清免费不卡视频| 熟女电影av网| 最近手机中文字幕大全| 男女免费视频国产| 国产片特级美女逼逼视频| 亚洲第一区二区三区不卡| 欧美人与性动交α欧美软件 | 99热6这里只有精品| av网站免费在线观看视频| 国产精品熟女久久久久浪| 欧美xxⅹ黑人| 午夜激情av网站| 久久免费观看电影| 黄网站色视频无遮挡免费观看| 亚洲精品视频女| 欧美xxxx性猛交bbbb| 成人毛片a级毛片在线播放| 久久久久精品性色| 日韩一本色道免费dvd| 黄色视频在线播放观看不卡| 国产精品国产三级专区第一集| 秋霞在线观看毛片| 一区二区三区精品91| 麻豆乱淫一区二区| 波多野结衣一区麻豆| 欧美成人精品欧美一级黄| 男人操女人黄网站| 三上悠亚av全集在线观看| 99热国产这里只有精品6| 国产爽快片一区二区三区| 亚洲欧美成人精品一区二区| 韩国av在线不卡| 色5月婷婷丁香| 777米奇影视久久| 少妇人妻精品综合一区二区| 国产在线视频一区二区| 最近最新中文字幕免费大全7| 99热这里只有是精品在线观看| 色网站视频免费| 午夜免费鲁丝| 夜夜爽夜夜爽视频| 国产国语露脸激情在线看| 97精品久久久久久久久久精品| 亚洲人成77777在线视频| 精品第一国产精品| 欧美人与性动交α欧美软件 | 国产成人91sexporn| 亚洲国产成人一精品久久久| 午夜日本视频在线| 国产精品一区www在线观看| 国产av精品麻豆| 制服人妻中文乱码| 久久狼人影院| 亚洲av男天堂| 美女中出高潮动态图| 欧美另类一区| 一本一本久久a久久精品综合妖精 国产伦在线观看视频一区 | 久久精品久久精品一区二区三区| 精品酒店卫生间| 99九九在线精品视频| 日本-黄色视频高清免费观看| 国产片特级美女逼逼视频| a级毛片在线看网站| 人妻系列 视频| 如日韩欧美国产精品一区二区三区| 久久久久久久大尺度免费视频| 日本wwww免费看| 极品少妇高潮喷水抽搐| 少妇的逼好多水| 蜜桃在线观看..| 国产精品女同一区二区软件| 成人亚洲精品一区在线观看| 国产精品熟女久久久久浪| a级毛片在线看网站| 激情视频va一区二区三区| 波多野结衣一区麻豆| 免费黄网站久久成人精品| 色吧在线观看| 国产av一区二区精品久久| 亚洲情色 制服丝袜| 少妇高潮的动态图| 国产在视频线精品| 一级,二级,三级黄色视频| 欧美丝袜亚洲另类| 亚洲国产av影院在线观看| 国产日韩欧美在线精品| 青春草亚洲视频在线观看| 91精品三级在线观看| 毛片一级片免费看久久久久| 亚洲av日韩在线播放| 欧美精品亚洲一区二区| 日本色播在线视频| 国产亚洲av片在线观看秒播厂| 大码成人一级视频| 另类亚洲欧美激情| 久久99热这里只频精品6学生| 日韩熟女老妇一区二区性免费视频| 久久精品国产综合久久久 | 免费看av在线观看网站| 日本vs欧美在线观看视频| 精品一区二区三卡| 黄片播放在线免费| 欧美日韩综合久久久久久| 色婷婷久久久亚洲欧美| 中文欧美无线码| 在现免费观看毛片| 汤姆久久久久久久影院中文字幕| 18禁动态无遮挡网站| 成人国产av品久久久| 妹子高潮喷水视频| 日本vs欧美在线观看视频| 精品视频人人做人人爽| 校园人妻丝袜中文字幕| 全区人妻精品视频| 国产乱来视频区| 日韩不卡一区二区三区视频在线| 亚洲三级黄色毛片| 日韩成人av中文字幕在线观看| 成人亚洲欧美一区二区av| 丝袜脚勾引网站| 男人操女人黄网站| 亚洲国产毛片av蜜桃av| 两个人看的免费小视频| 亚洲成人av在线免费| 精品一区二区三区视频在线| 亚洲天堂av无毛| 久久久a久久爽久久v久久| 亚洲精品国产色婷婷电影| 免费看光身美女| 夜夜骑夜夜射夜夜干| freevideosex欧美| 免费黄网站久久成人精品| 在线观看三级黄色| 精品少妇黑人巨大在线播放| 午夜老司机福利剧场| 91午夜精品亚洲一区二区三区| 国产欧美日韩一区二区三区在线| 亚洲综合色网址| 亚洲成av片中文字幕在线观看 | a 毛片基地| 亚洲色图 男人天堂 中文字幕 | 又大又黄又爽视频免费| 久久国产精品男人的天堂亚洲 | 国产av精品麻豆| 欧美xxxx性猛交bbbb| 国产极品粉嫩免费观看在线| 成年女人在线观看亚洲视频| 一区二区三区四区激情视频| 日本-黄色视频高清免费观看| 三级国产精品片| 亚洲综合色惰| 少妇被粗大的猛进出69影院 | 亚洲国产精品999| av卡一久久| 国产一区二区三区综合在线观看 | 丝袜喷水一区| 又大又黄又爽视频免费| 热re99久久精品国产66热6| 欧美精品人与动牲交sv欧美| 国产片特级美女逼逼视频| 免费高清在线观看视频在线观看| 亚洲少妇的诱惑av| 黑人猛操日本美女一级片| 久久鲁丝午夜福利片| 欧美国产精品va在线观看不卡| 国产极品天堂在线| 涩涩av久久男人的天堂| 国产日韩一区二区三区精品不卡| 人体艺术视频欧美日本| 国产精品一二三区在线看| 国产一区二区在线观看av| 精品一品国产午夜福利视频| 午夜福利视频精品| 男女边摸边吃奶| 韩国精品一区二区三区 | 精品酒店卫生间| 下体分泌物呈黄色| 日韩中字成人| 一区二区日韩欧美中文字幕 | 久久精品熟女亚洲av麻豆精品| 成人国语在线视频| av在线老鸭窝| 亚洲综合色网址| 久久国产精品男人的天堂亚洲 | 精品视频人人做人人爽| 中文字幕av电影在线播放| 亚洲av日韩在线播放| 欧美 日韩 精品 国产| 一边亲一边摸免费视频| 国产白丝娇喘喷水9色精品| 亚洲人与动物交配视频| 少妇人妻久久综合中文| 国产伦理片在线播放av一区| av不卡在线播放| 如日韩欧美国产精品一区二区三区| 国产日韩欧美在线精品| 亚洲 欧美一区二区三区| 国产女主播在线喷水免费视频网站| 三上悠亚av全集在线观看| 国产亚洲午夜精品一区二区久久| 五月开心婷婷网| 久久精品夜色国产| 菩萨蛮人人尽说江南好唐韦庄| 18禁国产床啪视频网站| 欧美人与性动交α欧美软件 | 一级爰片在线观看| 国精品久久久久久国模美| 精品人妻偷拍中文字幕| 97在线人人人人妻| 人人澡人人妻人| 高清黄色对白视频在线免费看| 三上悠亚av全集在线观看| 一级毛片电影观看| 国产熟女欧美一区二区| 蜜桃国产av成人99| 国产激情久久老熟女| 免费av不卡在线播放| 天堂俺去俺来也www色官网| 国产精品人妻久久久影院| 免费看光身美女| 欧美精品亚洲一区二区| 日日撸夜夜添| 十八禁网站网址无遮挡| 国产精品99久久99久久久不卡 | 看非洲黑人一级黄片| 欧美变态另类bdsm刘玥| 最近手机中文字幕大全| 99香蕉大伊视频| 亚洲人成网站在线观看播放| 夜夜骑夜夜射夜夜干| 五月伊人婷婷丁香| 国产午夜精品一二区理论片| 中文字幕精品免费在线观看视频 | 国产色婷婷99| 精品少妇内射三级| 最新的欧美精品一区二区| 一级爰片在线观看| 91精品国产国语对白视频| 欧美亚洲 丝袜 人妻 在线| 久久国产精品男人的天堂亚洲 | 国产永久视频网站| 精品亚洲成a人片在线观看| 人人澡人人妻人| 男的添女的下面高潮视频| 97在线人人人人妻| 久久99热这里只频精品6学生| 日韩av不卡免费在线播放| 老女人水多毛片| 免费观看无遮挡的男女| 蜜臀久久99精品久久宅男| 国产精品久久久av美女十八| 亚洲经典国产精华液单| 国产精品一二三区在线看| 高清av免费在线| 最近中文字幕2019免费版| 欧美日韩亚洲高清精品| 夫妻午夜视频| 亚洲第一区二区三区不卡|