Advanced Softwrae Engineering Methodologies (ASEM) - Module A - 2024-2025

Reviews

Audits

Software analysis

Testing

Security management

Safety management

Risk management

Important part of the more genereal activities of software quality assurance

Techniques

Methods

Tools

Define software quality for the project

Define policies and processesto achieve quality

Asses quality and find bugs/defects

Improve quality

Notations and algorithms

First-order logic

Dataflow

Deductive verifications

Principled ways of applying techniques

Design by contract

Security by design

Invariant methods

B method

Implenting and supporting the application of techniques and methods

Model checkers

Theorem provers

Establish that software behaves according to some properties

Method m always terminates

If the input is positive, the program returns its inverse

The program crashes with input 3

There are no loops in method m

If code is indented using tabs

Every class has 3 subclasses

There are 3 lines of comments per class

Correctness

Correctness is relative to the specification

Correctness is that implementation and specification are consistent

Specification may be implicit or explicit

Type correctness

Termination

No overflows

No memory leaks

No race conditions

Restriction on input

Guarantees on output

Effects on the state

Non-functional properties: timeliness, memory usage, etc.

Human action that produces an incorrect result

any programming mistake

manifestation of an error

the incorrect value is computed

a variable is not initialized

an imperfection or deficiency in a program

this function should always return a positive value, but returns a negative value in this case

Event were program execution cannot continue

uncaught exception

divide by zero

Absence of bugs or defects

fault

failure

defect

error

mistake

mistake

error

Apple goto fail

Heartbleed

Missing sanitization

It is not cost effective

Infeasible on large systems

Tests may be incorrect

Tests may be incomplete

Tests may be inconsistent

Bugs may only be reproducable under certain conditions

Does this program deadlock?

Will this program thwo a null pointer exception?

Will this program crash on any input?

Are variables initialized before they are read?

Will the value of variable x affect the value of variable y?

Will this piece of code ever be executed?

Analysis of behavor of a program without executing it?

Compiler optimization

Program correctness

Automated debugging

Security analysis

Over-appoximation

Safety properties

Integer variables do not overflow

There are no type errors

There are no null-pointer dereferencing

There are no race conditions

Infer by Facebook

Uber

Synopsys

IBM Rational AppScan

AbsInt

Cppcheck

Analysis of how data flows through a program

Represented as a directed graph

Can create a Control Flow Graph (DFG)

Statements

Predicates

Control flow

Reduced part of the program, where irrelevant statements are removed

Extracted program, that only contains statements that potentially influence the execution of a given statement

S is a valid program

whenever P halts for a given input, S also halts for that input and computes the same values for the variables in V whenever the statement n is executed

May remove non-termination

Slice may no longer contain infinite loops

Slice terminates, while original program may not

May not preserve program semantics

Slicing focusing on reachability

Slice is statements that (in-)directly affect slicing criterion

Nodes are identical to Control Flow Graph (CFG)

Edges are data and control dependence

One statement directly control the exectuion of another

In structured program equivalence to "indentation level"

Examples of this, is statements in a while loop, that are control dependent on the while predicate

Known from optimizing compilers

For slicing only "flow dependance" is relevant

$x\underset{dd}{\to }y$, means node x computes a value that may be used at node y in some feasible execution

there exists a variable v which is defined at x and used at y

In the Control flow graph (CFG), there is a path from x to y, where v is not redefined and can still affect y (killed)

What statements can influence the slicing criterion?

Finding all statements that may affect the slicing criterion

A node v in a the dependance graph

Backward slice of v

x is a node in the program dependance graph (PDG)

There exist a path from x to v (x →∗ v)

The path may be multiple steps (→∗)

generating a slice of an entire program, where the slice crosses the boundaries of procedure calls

Slicing that does not take into account the context of the slicing criterion

Treats every call to a procedure as if it was the same

May lead to inclusion of spurious (imprecise, not needed) nodes

First phase: Do not descend into called methods, instead traverse summary edges

Second phase: Go into omitted methods and repeat the process in the first phase

Debugging

Fault localization

Security analysis

Information flow control

Program understanding

Aid the test case generation

Deals with locating the faulty operation, that causes a bug

Bug-observed sites often do not reflect the real fault location

Print statements

Break points

Assertions

Manual

Time consuming

Metric-Based

Program dependence-based

Artificial intelligence-based

Statistics-based

Mutation-based

Each program statement outputs a suspiciousness score

Program elements executed in failing test-cases are likely to be faulty

Tarantula

Ochiai

Op2

Barinel

Star

Suspiciousness = $\frac{failed\left(s\right)/totalfailed}{failed\left(s\right)/totalfailed+passed\left(s\right)/totalpassed}$

Suspiciousness = $\frac{failed\left(s\right)/totalfailed}{failed\left(s\right)/totalfailed+passed\left(s\right)/totalpassed}$

Suspiciousness = $\frac{failed\left(s\right)}{\sqrt{totalfailed.\left(failed\right(s)+passed(s)}}$

Suspiciousness = $1-\frac{passed\left(S\right)}{passed\left(s\right)+failed\left(s\right)}$

Suspiciousness = $\frac{failed\left(s\right)*}{passed\left(s\right)+(totalfailed -failed(s\left)\right)}$

Relies (and biased) on the quality of the test suite

May contain too many suspicious statements

Extract semantic information from the bug/crash site

Find program locations that may affect the bug/crash site

Derive some matrices to prioritize these locations

Slicing based on a particular execution of the program

No

It may miss effect of unexecuted code

It is important to extend slicing to include relevent elements, using potential dependence

LLVM

Soot/Wala

LLM-based FL Technique

Explainable FL technique

Must have the whole software repo available

Automatically retrieves information for relevant areas

Performance is closely matched with raw LLM performance

Uses a two stage process

Uses multiple runs, to increase accuracy

Generate the root cause explanation

Predict the culprit methods