To study the problem of assessing the quality of
flight crew operations when using the simulator, the following objectives are
solved in the work:
– the definition of formalization elements that
would allow the implementation of evaluation logic;
– development of automated assessment methods;
– development of automated assessment system
software components-prototypes with the purpose of working capacity experimental
approbation of the proposed automated assessment method.
The problem of fully automatic assessment of the
crew’s activities at the moment cannot be solved, because there are no
exhaustive formal criteria and strict methodology for crew activities in the
industry. A key determinant of the assessment system is an expert instructor,
who forms integral assessments,
based on his experience. Therefore, the main objectives of the work are limited
to the automated generation of generalized data (partial estimates), which
simplify the process of forming integral final evaluations done by the
The developed methodology includes the following set
of basic sequential actions of the system: receiving and recording of flight
data, the formation of certain particular assessments of pilot actions tied to
flight phases, the recording and visualization of the pilot’s actions
assessment results. Recording of the pilot’s actions assessment results will
not be covered in this work. In general, the developed methodology does not
depend on the type of aircraft, although to formalize particular estimates for
a particular aircraft, it is necessary to define the evaluation elements, since
they depend on aircraft design and operation characteristics.
The recorded flight data from the simulators will
serve as the input data for the assessment, that will include both instrument
readings and information from aircraft systems. According to paragraph 4 of
section 1, this data will be sufficient to determine the main crew control
activities. The remaining data, which can not be formalized and subsequently
automatically identified, will need to be identified and assessed by the
instructor. It should be noted that, perhaps, this technique can be used to
assess the actions of the crew in real flight on the basis of flight parameters
from onboard flight data recorder. In this case, it will be possible to assess
the actions of the crew after the actual flight.
According to paragraph 3 section 1, aircraft flight
is divided into phases, and each phase will have different assessment criteria.
For example, the normal speed of the aircraft at the cruise phase of flight and
landing phase is significantly different. Also, based on the aircraft flight
operation manuals, at some phases of the flight, some aircraft systems must be
turned off, for example, on an L-410 aircraft it is necessary to turn off the
central electronic limiter (CEBO) during landing and take-off to ensure that
the engines power will reach the maximum operating mode. This objective will be
solved by method at the stage of recognition of the flight phase.
assessment of the crew actions is done by the instructor through comparing the
actions of the aircraft crew members in each section of the flight and in
emergency situations with criteria that are described in the flight manuals and
the general international rules to check if crew sustains proper operations and
permissible flight parameters.
To solve the problem of determining the formalization of
evaluation elements, the situational calculus and event calculus are used in
the work. Situational calculus allows to form sequences of transitions of
flight situations, based on a certain initial situation and transitions to
subsequent situations depending on the actions of pilots. This makes it
possible to identify the classified situations in which the crew may be in
flight, and which are selected in terms of the impact on the assessment of the
The situation calculus is a logic formalism
designed for representing and reasoning about dynamical domains.
situation calculus represents changing scenarios as a set of first-order
logic formulae. The basic elements of the calculus are:
The actions that can be performed in
The fluents that describe
the state of the world
A domain is
formalized by a number of formulae, namely:
Action precondition axioms, one for
Successor state axioms, one for each
Axioms describing the world in
The foundational axioms of the
The main elements of the situation
calculus are the actions, fluents and the situations. A number of objects are
also typically involved in the description of the world. The situation calculus
is based on a sorted domain with three sorts: actions, situations, and objects,
where the objects include everything that is not an action or a situation.
Variables of each sort can be used. While actions, situations, and objects are
elements of the domain, the fluents are modeled as either predicates or
functions. The actions form a sort of the domain. Variables of sort action can
be used. Actions can be quantified. In the situation calculus, a dynamic world
is modeled as progressing through a series of situations as a result of various
actions being performed within the world. A situation represents a history of
action occurrences. Statements whose truth value may change are
modeled by relational fluents, predicates which take a situation as their
final argument. Also possible are functional fluents, functions which take
a situation as their final argument and return a situation-dependent value.
Fluents may be thought of as “properties of the world”‘.
Situation calculus language Lsitcalc is a second order language with
equality. It has three disjoint sorts: action
for actions, situation for
situations, and a catch-all sort object
for everything else depending on the domain of application. Apart from the
standard alphabet of logical symbols there is ?, ¬ and ?, with the usual definitions of a
full set of connectives and quantifiers —
Lsitcalc has the
infinitely many individual variable symbols of each sort. We shall use s and a, with subscripts and superscripts, for variables of sort situation and action, respectively. We normally use lower case roman letters
other than a; s, with subscripts and superscripts for variables of sort object. In addition, because Lsitcalc is second order, its alphabet includes countably infinitely
many predicate variables of all arities.
function symbols of sort situation:
1. A constant symbol S0,
denoting the initial situation.
2. A binary function symbol do
: action × situation ® situation. The
intended interpretation is that do(a, s) denotes the successor situation
resulting from performing action a in
binary predicate symbol ?: situation × situation, defining an ordering relation
on situations. The intended interpretation of situations is as action
histories, in which case s ? s’ means that s is a proper subhistory of s’.
binary predicate symbol Poss : action × situation. The intended interpretation of Poss(a, s) is that it is
possible to perform the action a in
each n ? 0, countably infinitely many predicate symbols with arity n, and sorts (action ? object)n. These
are used to denote situation independent relations.
each n ? 0, countably infinitely many function symbols of sort (action ? object)n ® object. These are used to denote
situation independent functions.
each n ? 0, a finite or countably infinite number of function symbols of sort (action ? object)n
® action. These are called action
functions, and are used to denote actions. In most applications, there will be
just finitely many action functions, but we allow the possibility of an
infinite number of them.
each n ? 0, a finite or countably infinite number of predicate symbols with
arity n + 1, and sorts (action ? object)n × situation.
These predicate symbols are called relational fluents. In most applications,
there will be just finitely many relational fluents, but we do not preclude the
possibility of an infinite number of them. These are used to denote situation
dependent relations. Notice that relational fluents take just one argument of
sort situation and this is always its last argument.
each n ? 0, a finite or countably infinite number of function symbols of sort (action ? object)n × situation ® action ? object. These function symbols are called functional fluents. In
most applications, there will be just finitely many functional fluents, but we
do not preclude the possibility of an infinite number of them. These are used
to denote situation dependent functions. Notice that functional fluents take
just one argument of sort situation and this is always its last argument.
that only two functions symbols of Lsitcalc
and do —
are permitted to take values in sort situation.
Thus, the situational calculus allows us to identify the
current situation on the basis of the previous situation and the pilot’s
actions. For example, during take-off, when the aircraft speeds up on runway,
the pilot uses brakes, which causes the rejected takeoff:
do(useBrakes, takeoffRun) ®
In this case useBrakes is the action of the pilot, namely the use of brakes, takeoffRun – the current situation,
namely the takeoff and rejectedTakeoff
– the next situation, namely the rejected takeoff:
of situation calculus
Situational calculus fully justifies itself if there is a
single agent performing instant, discrete actions and if an action occurs
before or after another within a situation.
There is no way of expressing that an action occurs at a particular time, or
that two or more actions occur concurrently. Moreover, the definition of the
state of the external world in the situational calculus is closed on the agent
who performs the actions, and therefore external influences or effects created
not by the agent are not taken into account.
The above limitations make it
impossible to determine such situations as, for example, during takeoff when
the airplane speeds up on runway, the pilot briefly uses brakes, the airplane
does not stop and continues to take off. Similarly, if in this case the pilot,
in parallel with the use of brakes, switches on the reverse, it will be
impossible to determine the parallel use of these systems.
To solve the problem of identifying the pilot’s
actions leading to a situation change, let’s try to apply an event calculus
that takes into account the time intervals of actions.
The event calculus is a formalism for reasoning
about action and change. Like the situation calculus, the event calculus has
actions, which are called events, and time-varying properties or fluents. In
the situation calculus, performing an action in a situation gives rise to a
successor situation. Situation calculus actions are hypothetical, and time is
tree-like. In the event calculus, there is a single time line on which actual
A narrative is a possibly incomplete specification
of a set of actual event occurrences, i.e. a time structure which is
independent of any action occurrences is established or assumed, and then
statements about when various actions occur within this structure are
incorporated in the description of the domain under consideration. The event
calculus is narrative-based, unlike the standard situation calculus in which an
exact sequence of hypothetical actions is represented.
Like the situation calculus, the event calculus
supports context-sensitive effects of events, indirect effects, action
preconditions, and the commonsense law of inertia. Certain phenomena are
addressed more naturally in the event calculus, including concurrent events,
continuous time, continuous change, events with duration, nondeterministic
effects, partially ordered events, and triggered events.
Informally, the basic idea of the Event Calculus is
to state that fluents (time-varying properties of the world) are true at
particular time-points if they have been initiated by an action occurrence at
some earlier time-point, and not terminated by another action occurrence in the
meantime. Similarly, a fluent is false at a particular time-point if it has
been previously terminated and not initiated in the meantime. Domain dependent
axioms are provided to describe which actions initiate and terminate which
fluents under various circumstances, and to state which actions occur when. In
the context of the Event Calculus, individual action occurrences are often
referred to as “events”, so that “actions” are “event types”.
Event Calculus given here is written in a sorted predicate calculus with
equality, with a sort A for actions
(variables a, a1, a2,…),
a sort F for fluents (variables f, f1, f2, …), a
sort T for timepoints (here either
real numbers or integers, variables t, t1,
t2, …) and a sort X
for domain objects (variables x, x1,
x2, …). To describe a very basic calculus we need five
predicates (other than equality); Happens
F ×T ,
A× F× T ,
A× F× T and
< ? T×T. Happens(A, T) indicates that action A occurs at time T , HoldsAt(F, T) means that fluent F is true at time T, and Initiates(A, F, T) (respectively Terminates (A, F, T)) expresses that if A occurs at T it will initiate (respectively terminate) the fluent F. "<" is the standard order relation for time. It is also defined auxiliary predicates Clipped ? T×F×T and Declipped ? T×F×T in terms of Happens , Initiates , Terminates, and <. Clipped(T1, F, T2) (respectively Declipped(T1, F, T2)) means "the fluent F is terminated (respectively initiated) between times T1 and T2". The corresponding definitional axioms are: We can now axiomatise the two principles stated in the introduction to this section. Fluents which have been initiated by an occurrence of an action continue to hold until an occurrence of an action which terminates them and fluents which have been terminated by an occurrence of an action continue not to hold until an occurrence of an action which initiates them: The four axioms above capture the behavior of fluents once initiated or terminated by an action. So fluents can change their truth values only via the occurrence of initiating and terminating actions. Let's describe the previous example with the rejected take-off using the event calculus. It is to be reminded that during take-off, when the aircraft speeds up on runway, the pilot uses brakes and reverse, which causes the rejected takeoff. Due to the peculiarities of event calculus, namely the ability to capture the parallel events that are not associated with an agent, we can define the event when the aircraft speed reaches the minimum required for rejected takeoff. This refinement will allow us to verify that the aircraft did not continue takeoff. Moreover, event calculus can also determine other parallel ways of stopping the aircraft, in this case it is the usage of the reverse. In total we will have the following events: · brakesOn - the pilot uses brakes; · brakesOff - the pilot stops using the brakes; · reversOn - the pilot activates the reverse; · reversOff - the pilot turns off the reverse; · rejectedSpeed - speed that indicates rejected takeoff. The main fluents for this example are: · brakesUsed - the brakes are active; · reversUsed - the reverse is turned on; · takeoffRun - the aircraft runs over the runway; · rejectedTakeoff - rejected takeoff. The conditions for the initiation and termination of the fluents are to be described: The initial conditions in the example will be as follows: The following sequence of events will be: In the interval t = 3, the events "brakesOn" and "reversOn" occur, which in turn initiate the "brakesUsed" and "reversUsed" fluent respectively. In the interval t = 6, the event "rejectTakeOffSpeed" occurs, which first terminates the fluent "takeoffRun", and then initiates the "rejectedTakeOff" fluent. For clarity, let's build a table of the fluents states relation in time and add a time interval t = 7 to verify the conservation of fluents states. Time Fluents 0 1 2 3 4 5 6 7 takeoffRun True True True True True True False False rejectedTakeoff False False False False False False True True brakesUsed False False False True True True True True reversUsed False False False True True True True True As can be seen from the table, the event calculus coped with the task and made it possible to determine the rejected takeoff, and also to record parallel events and events that are not related to the agent. 2.2.2. Limitations of event calculus The phases of flight have a strict sequence of fluents, which characterizes each phase. This feature requires from used calculus an ability to construct the chains of these events. The event calculus allows you to set the conditions HoldsAt (f, tn-m) during the initiation with the help of which the fluent state can be checked in the past in time. This feature together with the data sampling makes it possible to construct the necessary fluent chains. But with further analysis of the flight phases, there were revealed limitations in the application of event calculus for their determination. So it was discovered that in different phases the events are repeated and it becomes necessary to build long chains of events to make sure that the phase is determined accurately. Although the event calculus allows you to build chains of events, they are actually limited by the depth of the constructions. Due to the fact that in the event calculus fluents do not store information about their initiation and termination time, it becomes impossible to determine the sequence of events in the chain deeper than the interval tn-1. It follows that in each initiation it is possible to accurately determine the state in a running before time point. To solve this problem, it is proposed to apply some principles of situation calculus together with event calculus. So, to determine the phases, it is proposed to build a candidate chain to determine the phase. This chain is a strict sequence of events. In this case, each next event must begin before or during the previous event's termination. The candidate chain starts with the initial event, the start time of which is considered to be the beginning of the phase. After successful determination of the whole sequence of events, this chain is considered to be correct and the phase is been determined by the time of initiation of the initial event, and the termination time of the last event is considered to be the end of the phase. If the sequence was broken during the determination of the candidate chain, then this chain is considered not to be suitable for phase determination. Thus, the application of these two calculus makes it possible to develop logic for evaluating the actions of the crew, depending on the flight situations that are based on the primary flight data supplied by the flight simulator recording system for the flight parameters. 2.3. Formal description of assessment system 2.3.1. Data Evaluation is based on data. The data is represented by "Packets" - sets of "parameters" that are received in a certain time interval. From the interval to the interval the number of "parameters" does not change. The time interval between the "packages" should be the same and sufficient to perform an objective assessment based on the data of these packets. "Parameter" - consists of a unique name and value. Values can be represented in any form, but in this work values are used as a floating-point number. 2.3.2. Recognition Based on the above proposed calculus, you can define the following elements that will be used in recognition: • Events - are based on the same element of the event calculus. • States - are based on the fluents of the event calculus; • Phases - based on the combination of situation and event calculus; "Events" and "States" are based on "Conditions". "Condition" consists of the name of the "parameter", the condition sign and the condition number. During "Events" and "States" initiation, the execution of the "Conditions" specified for them is being checked. This happens as follows: the "parameter" specified in the "condition" is being selected by name from the "package" and the condition sign specified in the "condition" is checked with respect to the value of the selected "parameter" and the number specified in the "condition". If the condition sight operations of all "conditions" are fulfilled, then it is considered that the "Event" or "State" is happening in this package. 2.3.3. Assessment The main type of crew actions assessment is monitoring compliance with normal flight criteria that are established in the aircraft flight manual. In simulator training monitoring is performed by the instructor with the help of the instruments of his instructor's workplace or by direct flight crew action observation in the simulator's cockpit. To assess the actions of the crew, it is necessary to ensure the possibility of registering exceedance of certain parameters within certain phases of the flight. To do this, you can use events and states by specifying the appropriate parameters and normal values in them. In this case, after recognizing all events, states and phases, the presence in all phases of states and events with parameters exiting normal values is being checked. This process will be provided by the "Detector". The "Detector" shall specify: • Phase to which the "Detector" will be applied; • A set of events and conditions associated with exiting normal parameters. "Detector" registers all cases of presence of the specified event and states in the established phase, recording the points of their beginning and end. Based on the received data, the crew's actions will be assessed.