The Integrated Test and Operations System (ITOS)
ITOS, the Integrated Test and Operations System, is a suite of software for control of spacecraft and spacecraft components during development, test, and on-orbit operations. It is a low-cost, portable, highly configurable system that runs under a variety of UNIX operating systems, including SolarisTM, FreeBSDTM, and Linux on workstation or PC hardware. ITOS currently is supporting more than a dozen missions including: SAMPEX, TRACE, FAST, SWAS, WIRE, Triana, Spartan 251, Spartan 401, Spartan 402, SWIFT, HESSI, and ULDB. ITOS is being used for all phases of mission life from board and box level development through spacecraft integration and test and on-orbit operations.
The diagram below depicts a typical ground data system configuration for spacecraft integration and test or on-orbit operations.
Figure 1 - Typical ITOS Configuration for Spacecraft Integration or On-Orbit Operations
The ITOS configuration consists of a cluster[1]of workstations interconnected over a local area network. Each workstation runs the complete ITOS software, with one designated as the primary operator console. This console receives telemetry data from the spacecraft interface over an IP ethernet connection, and sends commands to the spacecraft interface over an IP ethernet connection. The primary console feeds the telemetry data it receives from the spacecraft interface to all other ITOS workstations. Each ITOS workstation unpacks the telemetry packets and performs data processing tasks such as limit checking, engineering unit conversions, and configuration monitoring. Spacecraft commands typically are sent from the primary console only.
In addition, the primary ITOS console can distribute telemetry data to other systems attached to the local area network; for example, the Science Processing Facility, Command Management System, Flight Dynamics System, and Mission Planning System. The telemetry data is provided over an IP ethernet connection in a variety of formats, and can be transferred by frame, packet, or individual data value. A protocol is provided which allows external systems to request the type of data desired
Typically, ITOS installations in mission operations or I&T are set up on private networks. One ITOS computer will, however, be connected to the internet as a server for ITOSs Java-language, web-based telemetry and event displays. In our I&T configurations, this server typically is configured with dual ethernet interfaces and sits on both the private and public networks, serving as a gateway between the two (but never as a router). In operations, the ITOS web interface usually resides outside of a firewall, and the systems inside the firewall initiate a connection across the firewall to forward telemetry to the public system.
ITOS Features
ITOS scales from inexpensive PCs running free UNIX operating systems such as FreeBSD and Linux to high-end UNIX workstations from companies such as Sun and Hewlett-Packard. Furthermore, compute-intensive operations such as real-time, high-rate telemetry processing and distribution and real-time telemetry plotting can be distributed among workstations to achieve higher performance. These features allow missions to minimize hardware expenses necessary to achieve required data rates.
ITOS is not customized from mission to mission. Instead, mission customization is through a database, which provides a given missions telemetry and command specifications, and through a small set of configuration files. The database contains descriptions of how spacecraft commands are entered mnemonically and translated into a binary command, and how telemetry packets are to be unpacked into a set of named variables. The database includes specifications for telemetry limit checking and for engineering unit and discrete text conversions. Telemetry displays, plots, sequential prints, configuration monitors, and spacecraft command and table loads are defined in simple text files stored on the computers file system. ITOS includes a tool for creating command and table loads.
ITOS can receive telemetry from spacecraft interfaces and ground stations based on a configuration file that describes the telemetry and command encapsulation used by each interface. Due to the systems modular design, adding support for new interfaces can be accomplished with minimal effort.
ITOS can accept telemetry from multiple sources simultaneously. It can, for example, process spacecraft telemetry and ground station status blocks or telemetry from a dynamic simulator. It also can control and monitor external devices such as receivers, bit synchronizers, and simulators using IEEE-488, RS-232, or network connections.
ITOS provides a World Wide Web interface to spacecraft telemetry and system events. The same telemetry and event displays which are available to engineers sitting at an ITOS workstation are available to users through any Java-capable web browser. This allows engineers to monitor tests or flight operations and assist in resolving anomalies from remote locations, including their offices and homes.
ITOS Architecture
ITOS may be functionally divided into five subsystems: telemetry, command, control, database, and events. (See Figure 2.) The telemetry subsystem contains programs and functions for ingesting, unpacking, displaying, and checking spacecraft telemetry. The command subsystem contains programs and functions for generating and outputting spacecraft commands. The control subsystem provides functions for controlling and monitoring the spacecraft and ground system elements. The database subsystem consists of programs and functions for creating and accessing the ITOS operational database (ODB).
Figure 2 Major ITOS Software Components
The telemetry subsystem carries telemetry data from the spacecraft interface to the user by way of the operational database. Spacecraft frames are de-encapsulated, packets (for CCSDS telemetry) are extracted from the frames, and individual data values are extracted from the packets and put in the database. Data values flow from the database to display pages, x-y plots, strip charts, sequential prints, and configuration monitors. Incoming frames and extracted packets also may be displayed in a hex-dump format or be passed to processes external to ITOS.
The command subsystem carries telecommands from the STOL interpreter to the spacecraft interface. The user enters commands mnemonically in STOL. CCSDS commands are converted to packets, which are put in transfer frames, optionally converted to command link transmission units, encapsulated, and sent to the electrical GSE or ground station. The command subsystem contains an implementation of the CCSDS COP-1 (Command Operations Procedure) Frame Operations Procedure (FOP) for verifying that the spacecraft has received the commands. It also includes programs for creating spacecraft command and configuration table loads, uplinking those loads, capturing table and memory dumps, and comparing loads and dumps.
The control subsystem consists of the STOL interpreter and the configuration monitor. STOL is a scripting language that provides full control of the spacecraft, ITOS, and other ground system elements, including IEEE-488 and serial devices. STOL features standard programming logic constructs such as IF-THEN-ELSE and DO-WHILE along with specialized directives for controlling the ITOS. Perhaps the most powerful features of STOL are its ability to perform network communication, file and device I/O, and UNIX shell commands. These features, combined with configuration monitors, provide a base for autonomous spacecraft operation. Configuration monitors monitor telemetry and perform specified actions when specified conditions are met. For example, a configuration monitor can start capturing data whenever it detects that an instrument is turned on. Configuration monitors also can be used to calculate derived data points, for example, power from voltage and current.
The database is the heart of the ITOS. It contains the descriptions that enable the telemetry subsystem to unpack spacecraft telemetry values from the telemetry stream and link those values to names, which can be referenced in STOL or ITOS displays, along with limits and conversions for each value. It also contains the descriptions that enable the command subsystem to construct binary spacecraft commands from mnemonic commands entered through STOL. ITOS provides access to the database through a set of library functions and through a network-based data server.
All messages generated by ITOS programs and by the spacecraft are handled by the event subsystem which logs them to disk, displays them, and forwards them. Operators may open any number of event displays, and control what message types are displayed in each. At user request, the subsystem also can display event messages from other ITOS workstations. ITOS also can be set up to forward event messages to external programs which may be monitoring system events.
Because of its modular design, most components of the ITOS software may be replaced or augmented. If a user desires a deluxe display system with features not currently offered by the standard ITOS display subsystem, a commercial off-the-shelf component can be integrated without affecting the other subsystems. The currently offered and newly integrated display capabilities could coexist. (This actually is how our effort to include Java technology is being carried out.) The ITOS development team has even done work toward offering TCL as an alternative to our STOL interpreter.
ITOS clusters have consisted of anywhere from 1 to 22 workstations, and larger configurations are possible. During integration and test or launch and early orbit operations, each spacecraft subsystem lead engineer typically has a dedicated ITOS workstation. For normal on-orbit operations, the number of workstations is scaled down to two or three. In development lab environments, an ITOS cluster usually consists of a single workstation and a spacecraft interface.
