tango-simlib: Easily generate TANGO device simulators¶
Introduction¶
tango-simlib
is a library that aids the data-driven development of TANGO device
simulators. It aims to make it easy to develop basic simulators while making it
possible to implement more complex simulators. In addition to the simulated
device interface, a separate TANGO simulation-control interface is generated,
allowing the simulator to be manipulated via a back-channel to simulate
e.g. failure conditions on the simulated device interface.
Using only the basic TANGO interface description captured via a POGO generated XMI file, a basic simulator with randomly varying attributes and no-op command handlers can be generated with no further coding. Attribute simulation parameters and simple command behaviour can be specified using a Simulator Description Datafile (SimDD). The format of this file is currently a working proposal and subject to change. A more formal format specification is being worked on.
Note that tango-simlib
does not generate simulator code. Rather, the
simulator’s behaviour is driven by the description data at run-time using Python’s
dynamic programming features. If the description files (XMI or SimDD) are
modified, the simulator device server only needs to be restarted for the changes
to take effect.
Rationale¶
During the development of the control and monitoring (CAM) systems for the KAT-7 (KAT-7-wiki) and MeerKAT (MeerKAT-wiki) at SARAO it was found that having control-interface simulators available for all hardware and subsystems that CAM needs to control and monitor is an incredibly valuable resource. In early CAM development it:
- Makes it possible to start developing the CAM system before hardware or vendor-provided hardware simulators are available.
- Allows gaps in interfaces to be identified early on in the development process.
As development progresses, the full, actual, MeerKAT/KAT-7 CAM is run against the simulated devices, allowing CAM functionality to be tested without the need of real telescope hardware. The simulators also expose a back-channel that can be used to modify the behaviour of the simulators during tests e.g. to simulate error conditions. This is exploited by CAM developers in their own development environments for day to day development tasks and also allows daily automated functional integration tests to be run. It was also found that initially having even a very simple simulator available is quite valuable, and that for many devices the simple simulator is always sufficient.
The KAT-7 and MeerKAT telescopes use the Karoo Array Telescope Control Protocol (KATCP) for inter-device, subsystem and component communications. In light of that a library was developed that makes it very easy to code a basic simulator, providing no-op command (KATCP request) handlers and randomly varying attribute (KATCP sensor) values along with the back-channel interface for “free”.
The planned SKA telescope project that the SARAO team is participating in has decided to standardise on the TANGO control systems framework. This library is an attempt to bring the same simulation approach used for the KAT-7 and MeerKAT telescope to the TANGO world.
Basic Usage¶
Installation¶
Note that installation requires the TANGO binary prerequisites to be
installed. If you cannot install the PyTango package you will not be able to
install tango-simlib
.
Installation from source, working directory where source is checked out
$ pip install .
This package is available on PYPI, allowing
$ pip install tango-simlib
Device Simulators¶
Generating Simulators¶
The basic way of generating a TANGO device simulator using this library is to make use of the TANGO simulator generator module. Give it a path to the description files (XMI or SimDD or both).
$ tango-simlib-generator --sim-data-file Weather.xmi\
--dserver-name weather-DS\
--directory .
This will generate a python executable file in your current working directory named weather-DS
.
In order to run this generated device simulator code, you can execute the tango-launcher
script,
a helper script which will register the TANGO device server, setup any required device properties and
in turn start up the device server process, all in one go.
$ tango-simlib-launcher --name mkat_sim/weather/1 --class Weather\
--name mkat_simcontrol/weather/1\
--class WeatherSimControl\
--server-command ./weather-DS --port 0\
--server-instance tango-launched\
--put-device-property mkat_simcontrol/weather/1:model_key:mkat_sim/weather/1
The simulator limits the rate of calls to the internal model update
method. The default is
0.99 seconds. This can be overridden via the min_update_period
property on the main device.
For example, we can reduce it to 0.5 seconds by adding the last argument below.
$ tango-simlib-launcher --name mkat_sim/weather/1 --class Weather\
--name mkat_simcontrol/weather/1\
--class WeatherSimControl\
--server-command ./weather-DS --port 0\
--server-instance tango-launched\
--put-device-property mkat_simcontrol/weather/1:model_key:mkat_sim/weather/1\
--put-device-property mkat_sim/weather/1:min_update_period:0.5
Ready-made Simulators¶
Weather simulators¶
A code snippet of starting the Weather
simulator generated from the Weather.xmi
file
with a SimControl
instance using the tango_launcher.py
script.
Change directory to tango_simlib/examples/
$ tango-simlib-launcher --name mkat_sim/weather/1 --class Weather\
--name mkat_simcontrol/weather/1\
--class WeatherSimControl\
--server-command weather1.py --port 0\
--server-instance tango-launched\
--put-device-property mkat_simcontrol/weather/1:model_key:mkat_sim/weather/1
An example of starting the Weather
simulator generated from the Weather_SimDD.json
file with a SimControl
instance using the tango_launcher.py
script.
$ tango-simlib-launcher --name mkat_sim/weather/2 --class Weather\
--name mkat_simcontrol/weather/2\
--class WeatherSimControl\
--server-command weather2.py --port 0\
--server-instance tango-launched\
--put-device-property mkat_simcontrol/weather/2:model_key:mkat_sim/weather/2
MeerKAT Video Display System simulator¶
An example of starting the VDS simulator generated from both the MkatVds.xmi
and
the MkatVds_SimDD.json
files with a SimControl
instance using the tango_launcher.py
script.
$ tango-simlib-launcher --name mkat_sim/vds/1 --class MkatVds\
--name mkat_simcontrol/vds/1\
--class MkatVdsSimControl\
--server-command mkat_vds.py --port 0\
--server-instance tango-launched\
--put-device-property mkat_simcontrol/vds/1:model_key:mkat_sim/vds/1
Once the tango-simlib-tango-launcher
script has been executed, the TANGO server will be created in the TANGO database. The TANGO device server will be registered along with its properties and the server process will be started. This will start the server instance which has the two classes Weather
and WeatherSimControl
registered under it, respectively, which in turn will start the devices from each of the TANGO classes.
Screenshots of Interfaces¶
This is what you would have in the TANGO DB once the device server has been registered
In this instance, we have the simulated device in an alarm state after executing the SetOffRainStorm command on the test device interface, or what we call the simulator controller.
Translating a Tango device specification or a running Tango device to YAML¶
After installing tango_simlib, the tango-yaml
script will be available to use
$ tango-yaml -h
usage: tango_yaml [-h] {xmi,fandango,tango_device,validate} ...
This program translates various file formats that describe Tango devices to
YAML. Or validates the conformance of a device against a specification.
positional arguments:
{xmi,fandango,tango_device,validate}
sub command help
xmi Build YAML from a XMI file
fandango Build YAML from a fandango file
tango_device Build YAML from a running Tango device
validate Check conformance of a Tango device against a
specification in YAML format
optional arguments:
-h, --help show this help message and exit
XMI¶
$ tango-yaml xmi -h
usage: tango_yaml xmi [-h] xmi_file
positional arguments:
xmi_file Path to the XMI file
optional arguments:
-h, --help show this help message and exit
Example
$ tango-yaml xmi ./tango_simlib/tests/config_files/Weather.xmi
- class: Weather
meta:
attributes:
- name: integer2
data_format: SCALAR
data_type: DevULong
delta_t: ''
delta_val: ''
description: ''
display_unit: ''
format: ''
inherited: 'false'
label: integer2
max_alarm: ''
max_dim_x: 1
max_dim_y: 0
max_value: ''
max_warning: ''
min_alarm: ''
min_value: ''
min_warning: ''
period: '1000'
standard_unit: ''
unit: ''
writable: READ
...
commands:
- name: State
doc_in: none
doc_out: Device state
dtype_in: DevVoid
dtype_out: DevState
inherited: 'true'
...
properties:
- name: sim_xmi_description_file
Fandango¶
$ tango-yaml fandango -h
usage: tango_yaml fandango [-h] fandango_file
positional arguments:
fandango_file Path to the fandango file
optional arguments:
-h, --help show this help message and exit
Example
$ tango-yaml fandango ./tango_simlib/tests/config_files/database2.fgo
- class: DataBase
meta:
attributes:
- name: Status
data_format: SCALAR
data_type: DevString
description: ''
display_unit: No display unit
format: '%s'
label: Status
max_alarm: Not specified
max_dim_x: 1
max_dim_y: 0
max_value: Not specified
min_alarm: Not specified
min_value: Not specified
standard_unit: No standard unit
unit: ''
writable: READ
...
commands:
- name: DbGetExportdDeviceListForClass
doc_in: Class name
doc_out: Device exported list
dtype_in: DevString
dtype_out: DevVarStringArray
...
properties: []
Tango device¶
$ tango-yaml tango_device -h
usage: tango_yaml tango_device [-h] tango_device_name
positional arguments:
tango_device_name Tango device name in the domain/family/member format or
the FQDN
tango://<TANGO_HOST>:<TANGO_PORT>/domain/family/member
optional arguments:
-h, --help show this help message and exit
Example
$ tango-yaml tango_device ska_mid/tm_subarray_node/1
- class: SubarrayNode
meta:
attributes:
- name: buildState
data_format: SCALAR
data_type: DevString
description: Build state of this device
disp_level: OPERATOR
display_unit: No display unit
format: '%s'
label: buildState
max_alarm: Not specified
max_dim_x: 1
max_dim_y: 0
max_value: Not specified
min_alarm: Not specified
min_value: Not specified
standard_unit: No standard unit
unit: ''
writable: READ
writable_attr_name: None
...
commands:
- name: Abort
disp_level: OPERATOR
doc_in: Uninitialised
doc_out: Uninitialised
dtype_in: DevVoid
dtype_out: DevVoid
...
properties:
- name: CspSubarrayFQDN
- name: CspSubarrayLNFQDN
- name: DishLeafNodePrefix
- name: LoggingLevelDefault
- name: LoggingTargetsDefault
...
Validation¶
A Tango device’s conformance to an interface specification can be checked. By default check that the device provides a superset of the specification. When the optional -bidirectional flag is specified, the check is stricter - items on the device interface and not in the specification are also reported.
$ tango-yaml validate -h
usage: tango_yaml validate [-h] (--url URL | --path PATH) [--bidirectional]
tango_device_name
positional arguments:
tango_device_name Tango device name in the domain/family/member format or
the FQDN
tango://<TANGO_HOST>:<TANGO_PORT>/domain/family/member
optional arguments:
-h, --help show this help message and exit
--url URL The URL to a YAML specification file
--path PATH The file path to a YAML specification file
--bidirectional When bidirectional is included, any details on the device
that is not in the spec is also listed.
Example
$ tango-yaml validate --path ./DishMaster.yaml mid_d0001/elt/master
Command differs, [SetDSStandbyFPModeTask] specified but missing in device
Command [GetVersionInfo] differs:
doc_out:
specification: Uninitialised
device: Version strings
Command [GetVersionInfo] differs:
dtype_out:
specification: DevString
device: DevVarStringArray
Command [Scan] differs:
doc_in:
specification: [timestamp]
device: The timestamp indicates the time, in UTC
Attribute differs, [loggingLevelElement] specified but missing in device
Attribute [adminMode] differs:
description:
specification: No description
device: The admin mode reported for this device.
Attribute [controlMode] differs:
description:
specification: No description
device: The control mode of the device. REMOTE, LOCAL
Attribute [versionId] differs:
description:
specification: LMC version id (from git tag)
device: Version Id of this device
Property [LoggerInitPollPeriod] differs, specified but missing in device