framework.oracles

Note

Note we provide 11 oracles in the release, but only used 4 of them for the paper. Hence we will only be providing documentation for the used ones and a quick tutorial on Defining Your Own Oracle. Please refer to our main paper listed in Publication for a formal definition of the 4 oracles used.

framework.oracles.init

class framework.oracles.RecordAnalyzer(record_path: str)

Record analyzer, used for analyzing cyber record files

Parameters: record_path (str) – filename of the record

analyze() → List[Tuple]

Analyze the record file and retrieve results from each of the oracles

Returns: list of violations
Return type: List[Tuple]

get_results() → List[Tuple]

Retrieve results from each of the oracles

Returns: list of violations
Return type: List[Tuple]

register_oracles(): Register oracles to be used

framework.oracles.OracleInterface

class framework.oracles.OracleInterface.OracleInterface

Interface that defines how a oracle should be implemented

abstract get_interested_topics() → List[str]

Returns a list of topics this oracle is interested in. The oracle manager will call on_new_message for this oracle when a message from its interested topic is parsed

Returns: list of topics
Return type: List[str]

abstract get_result() → List

Returns a list of violations from this oracle

Returns: list of violations
Return type: List

abstract on_new_message(topic: str, message, t): Called when a message from a interested topic is parsed, i.e., this function should define what the oracle should do for messages it is interested in

framework.oracles.OracleManager

class framework.oracles.OracleManager.OracleManager

Helper class to organize all oracles

get_results() → List[Tuple]

Retrieve the violations analyzed from each of the oracles

Returns: all violations detected
Return type: List[Tuple]

on_new_message(topic, message, t)

Calls on_new_message for each of the oracle interested in a specific topic

Parameters

topic (str) – the topic of the message
message (any) – actual message data
t (float) – timestamp of the message

register_oracle(oracle: framework.oracles.OracleInterface.OracleInterface)

Register a specific oracle

Parameters: oracle (OracleInterface) – oracle to be registered

framework.oracles.ViolationTracker

class framework.oracles.ViolationTracker.ViolationTracker

Helper class to track violations detected during scenario generation

add_violation(gname, sname, record_file, mt, st, data, force=True) → bool

Adds a violation to the tracker

Parameters

gname (str) – generation name
sname (str) – scenario name
record_file (str) – name of the record file
mt (str) – main type of the violation
st (str) – sub type of the violation
data (any) – any underlying data to distinguish violation
force (bool) – forcing add even if it is a duplicate

Returns

True if added, False otherwise

Return type

bool

clear(): Clears all tracked violations

static get_instance() → framework.oracles.ViolationTracker.ViolationTracker

Gets the singleton instance

Returns: instance
Return type: ViolationTracker

print(): Helper function to print tracked violations to terminal

save_to_file(): Save the tracked violations to a csv file in f'{RECORDS_DIR}/summary.csv'

framework.oracles.impl.CollisionOracle

class framework.oracles.impl.CollisionOracle.CollisionOracle

The Collision Oracle is responsibe for determining whether a collision occurred between the ADC instance and another road traffic participants.

get_interested_topics(): The collision oracle is interested in Localization messages and Perception messages.

get_result() → List[Tuple]

Returns violations detected by this oracle, the obstacle’s ID is included in the output to distinguish between pedestrian and vehicle

Returns: all violations detected by this oracle
Return type: List[Tuple]

is_adc_completely_stopped() → bool

Helper function to check if the ADS instance is completely stopper or not.

Returns: True if completely stopped
Return type: bool

on_new_message(topic: str, message, t)

Upon receiving a new planning/perception message, the oracle analyzes the position of the ADS instance and position of all obstacles to determine whether a collision occurred, i.e., the distance between 2 objects is 0

Parameters

topic (str) – topic of the message
message (any) – either Planning or Localization message
t (float) – the timestamp

framework.oracles.impl.ModuleOracle

class framework.oracles.impl.ModuleOracle.ModuleOracle

The Module Oracle is responsible for determining whether the routing module failed. One root cause of this violation is the HD Map missing predecessor/successor information so that the routing module failed to find a valid route. We have fixed the issue with borregas_ave by updating the HD Map. (More details in Apollo Issue #14529)

get_interested_topics(): The module oracle is interested in Routing messages only

get_result() → List[Tuple]: Returns violations detected by this oracle

on_new_message(topic: str, message, t)

Upon receiving 1 routing response, the scenario passes this oracle

Parameters

topic (str) – topic of the message
message (any) – either Planning or Localization message
t (float) – the timestamp

framework.oracles.impl.StopSignOracle

class framework.oracles.impl.StopSignOracle.StopSignOracle

The Stop Sign Oracle is responsible for checking if the ADS instance made a complete stop before crossing a stop line controlled by a stop sign. After the oracle detects the ADS instance crossing a stop line, it looks backward to check if the ADS stopped for stop sign by looking at its main planning decision and its speed trace.

check_if_adc_intersecting_any_stop_lines() → str

Check if the AV is intersecting any stop line

Returns: the ID of the stop line intersecting with, or empty string
Return type: str

get_interested_topics(): The stop sign oracle is interested in Planning messages and Localization messages. Localization messages are used to check if the AV is crossing a stop line, and check if the AV’s speed reached 0.0 m/s. Planning messages are used to check if the AV’s main decision was indeed for the stop sign, not stopping for other reasons (e.g., obstacle)

get_result() → List[Tuple]: Returns a list of violations from this oracle

is_planning_main_decision_to_stop_at_stop_sign(planning_message: modules.planning.proto.planning_pb2.ADCTrajectory, stop_sign_id: str) → bool

Given a planning message, check if its main decision is to stop for stop sign

Returns: True if includes a stop sign stop decision, False otherwise
Return type: bool

on_new_message(topic: str, message, t)

Upon receiving a planning/localization message, this oracle may not perform any analysis if the AV is not crossing any stop line. Once the oracle detects it is crossing a stop line, it looks at messages cached and analyze whether the AV stopped for the stop sign before crossing the stop line.

Parameters

topic (str) – topic of the message
message (any) – either Planning or Localization message
t (float) – the timestamp

parse_stop_sign_stop_line_string_on_map(map_parser: hdmap.MapParser.MapParser) → None

Parse and store stop line for every stop sign on the map

Parameters: map_parser (MapParser) – the MapParser singleton instance

prune_old_messages() → None: Remove messages that are too old, i.e., only keep the last 200 messages for analyzing

reset_all_oracle_states() → None: Resets all oracle states

static was_adc_completely_stopped(past_localization) → bool

Given a localization message, check if the AV completely stopped

Returns: True if completely stopped, False otherwise
Return type: bool

framework.oracles.impl.TrafficSignalOracle

class framework.oracles.impl.TrafficSignalOracle.TrafficSignalOracle

The Traffic Signal Oracle is responsible for checking if the ADS instance violates a red light signal. When the AV is facing a red light signal and the AV is already intersecting the stop line, it should not have positive speed because it should have stopped before the stop line. If the AV stopped, also check if its main decision is to stop for the traffic signal.

check_if_adc_intersecting_any_stop_lines() → str

Check if the AV is intersecting any stop line

Returns: the ID of the signal controlling that stop line, or empty string
Return type: str

get_interested_topics(): The traffic signal oracle is interested in Localization, Traffic Light, and Planning messages. Localization messages are used for checking AV’s speed and whether it is crossing a stop line, Traffic Light messages are used for checking if the AV is facing a red signal, and Planning messages are used for checking if the AV made a stop decision for the signal.

get_result() → List[Tuple]: Returns a list of violations from this oracle

is_adc_completely_stopped() → bool

Check if the AV is currently stopped

Returns: True if stopped, False otherwise
Return type: bool

is_planning_main_decision_to_stop_at_traffic_signal(planning_message: modules.planning.proto.planning_pb2.ADCTrajectory, traffic_signal_id: str) → bool

Check if the AV’s main decision is stop for the traffic signal with the specified ID

Returns: True if a stop decision for the specified signal ID is made, False otherwise
Return type: bool

on_new_message(topic: str, message, t)

Upon receiving a new message, the oracle saves the message to analyze later if the AV is not crossing a stop line. Once it is crossing a stop line, it checks if it has a positive speed. To further gurantee the correctness of this oracle, we also attempt to check whether the AV made a complete stop for the signal before crossing the stop line.

Parameters

topic (str) – topic of the message
message (any) – either Planning or Localization message
t (float) – the timestamp

parse_traffic_signal_stop_line_string_on_map(map_parser: hdmap.MapParser.MapParser) → None: Parse and store stop line for every traffic signal on the map

framework.oracles

framework.oracles.__init__

framework.oracles.OracleInterface

framework.oracles.OracleManager

framework.oracles.ViolationTracker

framework.oracles.impl.CollisionOracle

framework.oracles.impl.ModuleOracle

framework.oracles.impl.StopSignOracle

framework.oracles.impl.TrafficSignalOracle

framework.oracles.init