Mars Science Laboratory: the technical reasons behind its delay
by Adrian Brown
|After the success of MER, JPL engineers dreamed big.|
The MER success story makes it difficult to follow up. The next planned rover to the Martian surface was to have been launched in 2009. MSL was designed to be twice the size and five to six times the weight of a MER, requiring new entry, descent, and landing (EDL) technology. MER had pushed the limits of parachute landing, and MSL would be the first to try a revolutionary lifting body and “sky crane” that would maneuver the envisioned 900-kilogram rover to within feet of the surface, and drop it ever so lightly to the ground.
After the success of MER, JPL engineers dreamed big. Talking to planetary scientists, they heard concerns about the relatively small areas of Mars where the MER rovers could land. MER was restricted to within 15 degrees of the equator because it needed solar power to survive, and required moderate temperatures for its stainless steel actuator motors. Scientists wanted more of the planet to be available: some of the most geologically interesting landing sites were outside the equatorial landing envelope (for example, possible delta deposits had been spotted by orbital imagery nearly 30 degrees south of the equator). JPL engineers looked into what they could do to get MSL into these challenging landing sites.
One of the first restrictions they sought to lift was that surrounding the MER lubrication system. Actuators on MER used a “wet lubrication” system to enable motors to move joints, instruments, and other components. This lubrication system was suspect in the very cold temperatures further from the Martian equator. MSL was to get a brand new type of lubrication—a dry, titanium-based actuator system—in order for it to operate in really low latitudes, up to 35 degrees south of the equator. In 2006, the JPL engineers got to work on testing out their ideas.
By 2007–08, as the launch date drew closer, things were not working out with the new actuators. Required braking and torques were not being achieved with the new dry lubricants. A decision to change course had to be made, because work was being held up on other components that relied on the actuators.
A total of 51 actuators and 54 stand-alone motors are required for MSL, including both engineering models and the flight models that will launch with the rover2. These are spread throughout the rover. JPL outsourced the work to a highly capable company, Aeroflex of Long Island2, which has historically produced actuators for space missions very successfully.
In 2008, a decision was finally made to change back to the old “wet type” stainless steel actuators, and the order was passed to Aeroflex. But the actuator redesign set back actuator development by about nine months2. As it turned out, this change came too late. Despite their best efforts, including working double and triple shifts, Aeroflex was not able to deal with such a large order of such complex components while maintaining schedule and quality2. By October 2008, this was becoming apparent to Doug McCuistion, the head of the Mars Exploration Program (MEP) and NASA HQ, who managed the purse strings for the MSL budget.
It is normal practice at JPL to build engineering models of components and test them extensively before starting work on the flight model. JPL procedure is for all components to be assembled in the Assembly Test and Launch Operations (ATLO) facility for integration testing, before they are taken apart and individually run though thermal vacuum and vibration testing3.
|In the end, it was a small set of engineering tests for brake release and insufficient torque provided by the latest batch of actuators—a totally fixable problem—that broke the camel’s back.|
Because the actuators had been delayed, JPL had been forced to divert from normal procedures and build the engineering model and flight model of the MSL robotic arm at the same time in order to keep to the 2009 launch schedule4. This is not optimal, because lessons learned on the engineering model would not necessarily by easily transferable to the flight model.
By November 2008, the MSL spacecraft, including the cruise stage, the descent stage, heat shield and the rover, had almost been completed. Engineers working around the clock had almost kept the MSL project on target, though some corners had been cut.
In the end, it was a small set of engineering tests for brake release and insufficient torque provided by the latest batch of actuators—a totally fixable problem—that broke the camel’s back: there was no more room for delays in the schedule, and at NASA HQ Doug McCuistion and his boss Ed Weiler knew they would have to pull the trigger on the MSL launch delay process.
As you’d expect with a unique engineering project of this magnitude, there were other smaller engineering issues that had caused problems for the engineers at JPL. They included significant issues with the avionics software.
MSL was built from the ground up to have a fully redundant Command and Data Handling System (CDHS), unlike the MER rovers which had non-redundant (“single string”) systems5. This would make MSL more bulletproof when it was on the surface: if a node of the CDHS failed, engineers on the ground could reroute to the backup system. However, this redundancy came at a complexity cost, and given MSL was already far more complex and capable rover than MER, the effort required to develop a redundant CDHS was taxing JPL software engineers to the limit.
Prior to the launch delay, JPL had already accepted that most of the operations software would be developed and tested after launch and sent to the rover while it was still cruising to Mars. Now that the launch delay has been announced, a less frenetic design and test path can be followed.
In addition to the redundant CDHS, a centralized motor control system for rover mechanisms and EDL thrusters (as opposed to software running on each motor) had been designed, and a power system capable of handling the radioisotope generator (RTG) and the batteries and solar arrays used for power during the cruise stage had to be developed. In addition, field programmable gate arrays (FPGA) on the MSL project had snowballed into a major issue. FPGAs are flexible electronics chips that can be reprogrammed on the fly. However this turned into a nightmare as engineers loaded more and more capability onto each tiny chip. FPGAs typically have 600–700 pins, and reprogramming and testing these FPGAs that were spread throughout the rover was proving challenging given the overall stress the project was under.
By December 4th, the avionics package was starting to come together: approximately 20 engineering model boxes and all flight model boxes except the centralized rover motor controller had been delivered to integration testing at the ATLO building5. By a herculean effort, the MSL avionics system seemed to be coming together for the launch date, but with less testing than JPL engineers (and NASA HQ) would like.
JPL engineers studying the reentry problem for MSL realized that with almost a metric ton of mass entering the Mars atmosphere (more than three times Viking and almost 80% of Apollo mass requirements), they were not going to be able to rely on the thermal ablative system used by MER. The MSL rover’s large size required a heat shell 15 feet [4.6 meters] in diameter (MER’s heat shell was half that size at 8.5 feet [2.6 meters] and even the Apollo re-entry shells were only 13 feet [4.0 meters] across). The increased heat loads called for a different, more resistive ablative material. Fortunately, just such a material had recently flown and been tested on the Stardust comet sample return mission, called Phenolic Impregnated Carbon Ablator (PICA). The large MSL shield would require tiles (the small Stardust heat shield was built in one piece) of PICA material. As the shuttle experience has shown, tiles introduce more risk into a thermally ablative system, so extra care was being taken during assembly of the heat shield at Lockheed Martin in Denver. On December 4th, the heat shield was on track for a February completion6 and April delivery to JPL.
Since the announcement of the launch delay, NASA has been calculating new EDL profiles. If MSL is forced to come in “hotter” than required by the 2009 launch profile then the PICA shield may need to be remanufactured.
Because MSL (in yet another first for Mars missions) is a lifting body that uses the resistance of the atmosphere to “float” to its landing site (unlike the Viking, Pathfinder, and the MERs, which were ballistic) a hotter entry into the atmosphere would actually be beneficial for landing site selection: somewhat paradoxically, MSL would have an increased range of elevations over which it could land safely on the surface.7
MSL has been billed as the most capable scientific mission every sent to another planet. Almost every instrument is a unique, cutting-edge item never used before on a space mission. Sample Analysis at Mars (SAM) is the heart of the sample analysis suite: it takes small amounts of rock or soil collected by the rover arm and runs it through a gas chromatograph, a quadrupole mass spectrometer, and a tunable laser spectrometer to completely determine the chemical (and especially organic) content of the sample. For those following the Phoenix instrument last year, NASA has recently found out how challenging this sort of work is. SAM is the instrument the Phoenix team would have loved to have onboard, but that’s the difference between a Scout and a Flagship mission.
CHEMCAM is a laser used to burn into rocks and detect plasma lines emitted by excited electrons and thus tell the chemistry of rocks at a distance. CHEMIN is an ingeniously miniaturized x-ray diffractometer that will determine the mineralogy of rocks and soil collected by the rover arm. Then there are an array of cameras: a descent imager (MARDI), a hand lens (MAHLI), and panoramic camera (MASTCAM). There is a Russian-built neutron detector (DAN), and a Spanish-built atmospheric monitoring package (REMS). For future Martian astronauts, there is a radiation detector designed to monitor the Martian radiation environment (RAD). An old standby with a large heritage from Pathfinder and MER, the Alpha Proton X-Ray Spectrometer (APXS), is also mounted on the MSL robot arm.
Because SAM and CHEMIN require soil and rock samples to be fed to them, a completely new rover arm and sampling attachment had to be built for MSL, called Sample Acquisition/Sample Processing and Handling (SA/SPaH). Such a complicated instrument must have a complicated acronym. The SA/SPaH consists of the 1.9-meter Robot Arm (RA) and a turret structure upon which five instruments are mounted. The first two are APXS and MAHLI, the other three are the Powder Acquisition Drill System (PADS), Dust Removal Tool (DRT), and the Collection and Handling for Interior Martian Rock Analysis (CHIMRA). The PADS will drill up to five centimeters into rocks, while the CHIMRA is a scoop that sorts rocks and soil, gets the sample back to the rover body, and tips it into sample ports. The Phoenix experience here, with blocked entrance ports and unexpected soil cohesiveness, is also instructive and has raised concerns at NASA HQ. JPL engineers have redesigned the SA/SPaH sampling system several times from the ground up since 2006 as new problems and requirements have come to light. The continuing moving baseline for SA/SPaH is highlighted by the decision late in 2008 to take off a sample cache that was to be attached to the front of the rover for the storage of important samples.
|MSL has been billed as the most capable scientific mission every sent to another planet. Almost every instrument is a unique, cutting-edge item never used before on a space mission.|
When you consider that JPL were forced to build the engineering model and flight model of the robotic arm at the same time and due to the delay in actuators being delivered, the sample handling system became a big headache. A failure on the Martian surface would have been catastrophic. The launch delay will allow more cautious approach to assemble and test of the critical robotic arm subsystem. JPL now plan a full end-to-end test of the drill and sample handling system to begin in 2010, after the actuators are finally delivered and engineering models built and tested.4
When the delay was announced, MARDI, MAHLI, DAN, and APXS had already been delivered to JPL.8 A redesign of the MASTCAM zoom system meant that system had experienced some time on the critical path, but it was essentially complete and ready to ship.8 The three cutting-edge instruments, CHEMCAM, CHEMIN, and SAM, were finished (CHEMIN at JPL) or in the final stages of environmental testing (SAM at Goddard and ChemCam at Los Alamos National Laboratory).8 The REMS wind sensor, built in Spain, had experienced serious technical problems and it was planned that the wind sensors would undergo change out and replacement at Cape Canaveral. This last-minute change out can now be done in a more relaxed fashion.
As they look forward to 2011, NASA is trying to find a way to fit MSL into the Cape Canaveral launch manifest. MSL will launch on an Atlas 5, managed by the United Launch Alliance (ULA). ULA currently requires 60–90 days between Atlas launches, which drives the launch calendar. But competing with MSL for a place on an Atlas 5 is the Juno mission to Jupiter.9 Optimal launch windows to Jupiter come up less frequently than launch windows to Mars, meaning Juno must get higher priority, so the actual launch window for MSL has yet to be announced.
3. See for example http://cassini-huygens.jpl.nasa.gov/cassini/Spacecraft/ATLO.shtml
5. ibid., page 12.
6. ibid., page 9.
7. Richard Cooke addressing the Planetary Sciences Subcommittee of the NASA Advisory Council Science Committee on 9 Jan 2009.
9. ibid., page 24.