Computation and Imagery--Roberto Aranibar

As of October 17th:           

            Computation and Imagery addresses all design issues regarding tasks or systems which require data processing.  The most obvious of these include instrumentation and on- and off-board displays for vehicle and crew health monitoring and diagnostics, logging of pertinent data throughout a rover mission, and a video system.  Communications will, of course, require a great deal of data processing.  Due to its complexity, this is being handled as a separate design entity.  Many of the decisions in computation and imagery will be made according to the design of the communications system, as data transmission rates and quality will be strongly influenced by the limits and capabilities of the communication system.

All of the rover’s computational aspects will be addressed as much as possible, but due to the project circumstances (e.g. time frame, number of members working on the project), the system within this area that will be addressed in the most detail over the next two semesters is its video system.  The TSGC Design Challenge criteria for the video system is that it be controllable remotely and able to transmit scenes to a base.  Through further investigation of what an effective video system for planetary exploration must be capable of doing, goals including and exceeding these requirements have been set. 

The overall goal of the video system design is to ensure capabilities for providing real time, color video of mission footage to a base station (and earth, if possible) at a flicker-free rate.  To accomplish this, hardware requirements and methods of camera control, configuration, reliability, and redundancy must be determined to meet the goals mentioned and satisfy the requirements imposed by these goals.

To provide adequate depth and image resolution when necessary, a stereo camera configuration is required.  However, a mono camera configuration should be supported for redundancy purposes; that is, if one of the cameras fails.  Adjustable resolution should be available to lower the level of data processing when possible, but still satisfy high resolution demands if necessary.  The placement of the cameras should be at the top center of the vehicle, and a camera mount capable of pan, tilt, and elevation motions will be necessary so that images at any location around the rover can be acquired upon demand.  To control the cameras’ direction and focus, manual and automatic control methods should be available.  The manual method would allow the cameras’ direction and focal distance to be controlled by an astronaut on-board or remotely from a base station.  In contrast, the automatic method would be a programmed method of control in which the cameras automatically keep an astronaut who has gone off-board for scientific sampling in center frame and in focus. 

            Several other important design requirements for the video system still remain under investigation.  These include exact specifications required of the equipment that will be used, such as the field of view required by the cameras, illumination needs (in case of dark conditions), and maximum allowable power consumption and data transmission rate, which also specifies the minimum video data compression ratio required and therefore the processing capabilities of the hardware that will be used.  These specifications are still being investigated and are anticipated to be determined in the near future.

Developing design concepts for the rover’s video system has required a great deal of research on video systems of past and present planetary exploration vehicles (and general purpose video systems), and is the result of much discussion among the team and its collaborators over what the imagery requirements of the rover should be.  The current plan in designing the video system is to determine what range of resolution is required from the video cameras, at what rate image frames must be acquired, and what the maximum allowable power consumption and weight of the video equipment must be.  When this is determined, cameras and frame grabbers can be selected to meet these specifications.  The rate of data being passed from the cameras to the compression hardware can then be determined, and a minimum data compression ratio which the compression hardware must accomplish can be established according to the limits of the communications system. Compression hardware can then be selected according to these requirements.  A camera mount / control platform must also be used to control the direction of the stereo camera configuration. 

The approach taken in developing a design for the video system has been to start with the most general functional block diagram possible, and then to design within the blocks as the project progresses.  This method is sometimes referred to as top-down design.  The most general functional diagram of the video system is shown below in Figure 1.

Video Cameras

Frame Grabbers

Video Data Compression Hardware

Communication System

Onboard Display

Figure 1: Functional Diagram of the Video System

            As mentioned earlier, a camera mount / control platform must be used in the design to control the direction of the stereo camera configuration.  One such platform is the Metrica Biclops Pan/Tilt/Vergence Camera Mount (PTVM).  The PTVM is a three-axis motion control platform for aiming stereo cameras.  Its axes are under closed-loop computer control, with motion commanded through a standard RS232 port.  The PTVM will be incorporated into the design of the rover’s video system and used in the second semester prototyping of the system. 

As of November 14th:

Computation & Imagery

I. Design Requirements

As mentioned earlier, computation and imagery tasks are areas or systems that require data processing, with the exception of the communications subsystem, which is being handled as a separate design entity.  The main goal of the computation and imagery area is to provide a means for the following:

  1. Instrumentation with user interfaces if necessary: measurement devices including those for
    1. crew health monitoring

                                                               i.      astronaut health

    1. vehicle and vehicle systems status and diagnostics

                                                               i.      speed, direction, attitude, altitude, distance traveled

                                                             ii.      vehicle and vehicle systems temperatures

                                                            iii.      proper operation of all systems

    1. ambient conditions

                                                               i.      temperature, pressure

  1. Onboard Display Monitor also with a user interface to select data displayed: must be able to display all in (1) as well as video signal being transmitted
  2. Data logging: analogous to black boxes on airplanes
    1. Record keeping of all pertinent data in (1) throughout a rover exploration mission
  3. Video system

Due to the circumstances of this project (time frame, number of people working on the project), the only one of these four areas that will be addressed in detail is the video system.  Therefore, the requirements and restraints specific to the video system must be examined.  Below is an overview of these requirements and restraints.

  1. Provide real-time, color video images that capture appropriate mission footage and transmit these scenes to a base station (and earth, if permitted by the communications subsystem)
  2. Support of scientific imagery operations
  3. Operation within the capabilities of the communications subsystem
  4. Operation within the limits of power available for computation and imagery purposes
  5. Maximize the area around the rover at which images can be acquired
  6. Controllable remotely
  7. Reliable
  8. Efficient in mass, power consumption, and cost

The purpose of the first requirement is to ensure that quality imagery of important mission footage is available to overseers of the mission, and to fulfill one of the design criteria specified for this project by the TSGC design challenge, which states that the video system must be capable of transmitting scenes to a base.  Providing quality mission footage is the primary goal of the video system.  The second requirement is established to allow for scientific uses of the video system.  This is important since the main purpose of most planetary exploration missions is to collect scientific information.  The third and fourth requirements are for ensuring that the video system operates properly in conjunction with the rest of the rover and its components.  The maximum allowable data transmission rate specified by the communications link is 7.5 Mb/s, where 6 Mb/s are allotted to video transmission.  The purpose of the fifth requirement is to ensure that the video system and its placement on the rover be specified in a manner that allows the camera(s) to capture images at as many locations around the rover as possible.  Requirement 7 serves the purpose of making sure that all possibilities of failure are considered and accounted for when designing.  All equipment must be able to operate properly under the harsh conditions of Mars’ surface including low temperature, dust interference, and vibration and shock due to rough terrain.  Finally, the last requirement, which is necessary for all components and subsystems of the rover, sets a goal for maximizing efficiency by minimizing cost, mass, and power consumption of the video system. 

 

II. Final Design

            A top-down design approach is taken to begin developing a design for the video system.  That is, the most general functional block diagram that accounts for the tasks that the system must carry out to fulfill the data transmission requirements is created.  The block diagram is then broken down further into more detail as the design progresses.  Such a diagram is shown in Figure 1. 

      Video Camera

      Frame Grabber

      Video Data Compression Hardware

Communication System

Onboard Display

    Light

 

Figure 1: General block diagram of the data path taken in the video system

 

Figure 1 shows the data path that must be taken to satisfy the criteria mentioned in the requirements and restraints section. 

            Since the primary goal of the video system is to provide general, yet top quality, mission footage, a mono camera configuration is the primary camera configuration used.  However, since most planetary exploration missions are geared towards collecting scientific information, the use of a stereo camera configuration is also supported.  This configuration provides a means for measuring depth in images for scientific purposes.

            As mentioned in the requirements and restraints section, equipment specifications will have to be selected so that the video system operates within the capabilities of the communications subsystem and the limits of power available for computation and imagery purposes.  Keep in mind that the power required by other computation and imagery components must also be estimated and accounted for.  In order to make appropriate design decisions to satisfy these requirements while providing the best quality video possible, Figure 1 must be broken down into further detail, as shown in Figure 2.  


 

 

 

 

 

Figure 2: Detailed block diagram of the data path taken in the video system

 

            From Figure 2 it can be seen that light passes through a lens and is converted into electrical signals by an array of CCDs (Charge-Coupled Devices).  Consider one of these electrical signals.  If using an analog output camera, the electrical signal passes through the camera circuitry (which is not of interest right now), and is then output as an analog signal.  An ADC converter in a frame grabber then takes this analog signal and quantizes it into a digital signal represented by some number of bits.  Finally, video data compression hardware takes signals and uses algorithms to compress the data by taking advantage of the limits of the human visual system.  Thus the amount of data needed to be transmitted is reduced by some compression ratio. 

            From Figure 2 and the discussion in the preceding paragraph, it can be seen that the design variables that determine the video data transmission rate are the following:

  1. number of cameras used
  2. image resolution (or size of the CCD array)
  3. frame sampling frequency
  4. pixel resolution (digital bit representation)
  5. compression ratio produced by the video data compression hardware

In order to provide the best video quality while staying within the limits of the 6 Mb/s communications link available for video data transmission, the following values are used for each of the above parameters:  Either one or two cameras are used at a time to support the mono or stereo configurations mentioned earlier.  The number of cameras being used is a user-controlled option.  The image resolution is 1024 x 1024 pixels.  The frame frequency is also a user-controlled option that can be selected to be between 15 and 30 Hz in increments of 5 Hz.  Faster sampling frequencies are required when the images collected by the camera are changing at a fast rate.  This could occur when the vehicle is in motion or when the camera’s direction is quickly being changed.  In contrast, a lower sampling frequency could be used to conserve space on the communications link when the camera and vehicle sit still and the characteristics of the image being input are changing at a slow rate.  An 12-bit digital representation is created by the ADC converter, allowing for 4096 color representations.  Lastly, the MPEG compression method is used to compress the data by at least 150:1(insert footnote for CMU reference).  Therefore, the only two parameters which are varied are the number of cameras used and the frame sampling frequency.  The rest of the parameters are held constant.  The result is a video data transmission rate ranging from approximately 1.3 to 5.0 Mb/s, which satisfies the 6 Mb/s limit allotted to video data transmission, and leaves a sufficient 2.5 Mb/s communications link available for transmitting audio data.  

            Recall that one of the requirements of the video system is that it maximizes the area around the rover at which images can be acquired.  To achieve this goal, the cameras incorporate lens systems with focusing capabilities and are placed in a location on the rover that has the fewest visual obstructions, and on a platform that easily allows for gimballing movements.  Figure Z shows the location of the video system on the rover.

(Figure Z, very rough idea of the rover shape and video system shape mounted at top center)

The cameras are mounted on a removable camera mast and platform slightly more towards the front of the top center of the rover.  An estimated 20° x 20° square field of view from each of the two cameras and 30 cm horizontal separation is required to acquire adequate views of the surroundings.  The camera platform allows ±180° azimuthal rotation, ±90° range of direction from the vertical, and pan, tilt, rotation, and elevation movements.  A camera platform with such capabilities is the Metrica Biclops Pan/Tilt/Vergence Camera Mount (PTVM).  The PTVM is a three-axis motion control platform for aiming stereo cameras.  Its axes are under closed-loop computer control, with motion commanded through a standard RS232 port.  The camera mast and platform is modeled after the PTVM, which will be used in the second semester prototyping of the system.

            Another issue of concern is the method of controlling the camera platform’s movement.  Two modes of control are used; that is, a manual and automatic control mode.  The manual mode enables a rover crew member or base station operator at a remote location to control the cameras’ movement.  In contrast, an automatic mode of control is implemented through the use of algorithms which work to keep an astronaut in center frame.  This mode of control can be accomplished by using image processing techniques or by tracking a specific color of an astronaut’s space suit, and is useful for keeping watch on an astronaut that leaves the rover for purposes such as collecting geological samples.  

            One of the most important requirements of the video system is that of reliability.  As mentioned earlier, the video system must be able to operate properly under the harsh conditions of Mars’ surface including low temperature, dust interference, and any shock and vibration imposed by rough terrain.  Furthermore, backup methods of operation must be devised. 

            The fact that the video system is composed of two cameras accounts for the backup method of operation.  If one of the cameras fails, another camera is still operational and the worst consequence is that stereo imaging is not supported. 

To account for the low temperatures that the video system may be exposed to, the bulk of the system’s electronics are kept in a thermally controlled electronics box, which is where the majority of the vehicles electronics are stored.  However, this does not account for the cameras’ direct exposure to the atmosphere.  To compensate for this, the cameras are enclosed in a box of material that makes physical contact with the cameras, through which heat taken from the thermally controlled electronics box can flow and be exchanged with the cameras to maintain appropriate operating temperatures.  Shown below in Figure A is a drawing of the video system, where the box just described can be noticed.  This box will be referred to as the camera protection box. 

(Insert Figure A, CAD drawing of video system)        

            When the video system power is shut down, the front of the camera protection box is closed.  This serves useful in protecting the cameras during severe dust storms.  Methods of preventing dust interference during milder dust storms, when video system is needed to be used, are still under investigation. 

Other considerations for maintaining good reliability are shock and vibration dampening, proper illumination conditions, and lens fogging prevention methods.  Shock and vibration dampening rely on the vehicle’s suspension system, which is designed to prevent any damage or difficulties to all of the vehicle’s electronic systems.  Methods of illumination and preventing foggy lenses are still under investigation. 

Finally, the last issues of concern are those of cost, mass, and power efficiency.  The summary of maximum estimates of the mass and power budget for the video system as well as the major computation and imagery components are given in Table 1.  Cost estimates are unknown at this point.

 

Table 1: Maximum estimates of computation and imagery mass and power

System

Component

Quantity

Total Mass (kg)

Avg Power (W)

Instrumentation

-

-

6.0

15

Onboard display

Monitor

1

9.6

90

Data Logging

Black Box

1

2.4

10

Video

Cameras

2

2.4

15

 

Mast / Platform

1

2.4

10

 

Compression Boards

2

1.2

20

 

Frame Grabbers

2

0.6

20

 

Illumination equipment

-

3.6

10

 

Protective box

1

6.0

10

 

 

 

 

 

 

 

Total Estimates

34.2

200