NOAA’s National Data Buoy Center Data Assembly Center
Evolving to Support the U.S. Integrated Ocean Observing System
Richard L. Crout, PhD and William H. Burnett, PhD
NOAA NWS NDBC, Stennis Space Center, MS 39529
Overview
REAL-TIME marine weather and ocean observations – why are they important? If we use the military as an analogy – they require real-time intelligence information to provide situational awareness before going into hostile territory. They same could be said for anyone who ventures into the open waters – except instead of intelligence information, they should have the same situational awareness of the environment. Real-time marine weather and ocean observations provide the boater, ship captain, fisher, oil and gas operator, deep-sea diver, and the military with situational awareness of the marine environment. “How high are the waves?” “Which way are the waves moving?” “Will the wind change direction soon?” “How long will I survive in the cold water?”
Marine observations are also critical to the forecaster and weather/ocean numerical prediction systems since these observations provide accurate initial conditions for the forecasts. Inaccurate initial conditions may produce a large error in the hurricane’s 72 hour forecasted position, or a marine forecaster may miss the location of a cold-core ocean eddy. Inaccurate wave forecasts may result in significant damage to unsuspecting vessels. Therefore, any real-time, marine weather and/ or ocean observation that is taken should be carefully quality controlled and evaluated before being used by the boater, forecaster or the public.
Background
The National Oceanic and Atmospheric Administration’s (NOAA) National Data Buoy Center (NDBC) began operations in 1967 as the National Data Buoy Development Program of the United States Coast Guard. In 1970, the NOAA was formed and the NDBC was moved under that agency. The first buoys deployed by NDBC were large 12-m discus hulls, constructed of steel, and deployed in deep water off the U.S. East Coast and in the Gulf of Mexico. By 1979, NDBC maintained 26 stations, 16 in the Pacific, seven in the Atlantic and three in the Gulf of Mexico. NDBC deployed eight more stations in the Great Lakes during the 1980s. These real-time observations were released through the National Weather Service’s (NWS) Telecommunications Gateway (NWSTG) in Silver Spring, Maryland, USA and sent to the World Meteorological Organization (WMO) Global Telecommunication System (GTS) where other governments and ships would receive the observations.
In response to the growing number of stations, the NDBC established a Data Quality Assurance Group that quality controlled data from the NDBC coastal weather buoys and Coastal Marine Automated Network (CMAN) stations following transmission, on a daily basis. During the 1990s, automated quality control was established for meteorological data from the approximately 100 stations maintained by NDBC. Automated quality control consisted of checking for parity errors in the data message, a range check for obviously spurious points, a time continuity check, and a check for zero wave heights. In addition, the NDBC Quality Assurance Group reevaluated the data collected and ensured that monthly archive data sets contained only accurate data. The data were made available to the NOAA’s National Environmental Satellite, Data, and Information Service (NESDIS) for archival at the National Climatic Data Center (NCDC) and the National Oceanographic Data Center (NODC).
DAC Functions
NDBC’s marine observation network now consists of 105 moored, coastal buoys located in the Bering Sea, North Pacific Ocean, Gulf of Mexico, Great Lakes, Caribbean Sea, and Atlantic Ocean that collect timely weather and oceanographic information. NDBC also operates 56 CMAN located around the coastal United States, the Caribbean Sea and the Atlantic Ocean. In 2005, NDBC assumed responsibility for the “Crown Jewel” of global climate observing, the equatorial Pacific Tropical Atmosphere Ocean (TAO) array of 55 moorings and is partnering with the NOAA/Pacific Marine Environmental Laboratory (PMEL) on the Pilot Research Array in the Tropical Atlantic (PIRATA). In February 2008, NDBC completed the 39 buoy Deep-ocean Assessment and Reporting of Tsunamis (DART®) system, that provides deep-ocean water level data from around the Pacific Ocean basin, Caribbean Sea, Gulf of Mexico and western Atlantic Ocean in support tsunami warnings. Finally, NDBC is supporting the development of the latest marine observing technology being deployed around the coastal United States – a network of High Frequency (HF) radar systems, that collect an enormous amount of ocean surface current (and potentially surface wave) observations in the near shore region.
Figure 1. Once a buoy is placed on station, NDBC personnel usually have to wait a few years before visiting the station. Therefore, all platforms and instruments must be in excellent condition before deployment – and the data carefully monitored during station operations.
In July 2005, NDBC began a 24- hour, 7-day per week Data Assembly Center (DAC) in response to these increasing data loads and the need to support buoy recovery and deployment operations in the far reaches of the Pacific and Indian Oceans. DAC personnel support operations by forecasting weather and sea-state prior to and during these field missions. These five day forecasts assign a “Go,” “No Go,” or “Marginal” status for each deployment or service operation to be used for guidance only. In addition to weather and sea-state predictions, the DAC also supports NDBC buoy operations and maintenance by analyzing data acquired during integration, test and deployment activities to ensure proper operation of all systems. All platforms and instruments must be in excellent condition before deployment and the data carefully monitored during station operations. Failing anemometers, inaccurate barometer readings, low power levels, and irregular wave spectrums all contribute to poor marine weather/ocean observations, and emphasize the need for trained eyes to stop the data before they are released to the public.
Multi-DAC Functions
Besides monitoring the health and well-being of the NDBC’s legacy moored-buoy and CMAN platforms, the DAC is monitoring the newly transitioned platforms – TAO, DART® and HF Radar. The NOAA TAO array in the equatorial Pacific provides real-time and delayed-mode data from moored ocean buoys to improve detection, understanding and prediction of El Nińo and La Nińa events. In 2005, NDBC and the NOAA/PMEL initiated the transition of the TAO IT and quality control system PMEL to NDBC. TAO’s real-time processing subsystem starts with the processing of Automated Distribution Service messages from Service Argos. An automated real-time quality control check is performed for gross error checking and then the TAO database is updated with the corrected data (http://tao.ndbc.noaa.gov).
Figure 2. NDBC’s weekly platform count. NDBC’s weekly platform count from December 28, 2004 through present. The green line (crosses) is the total number of platforms. The dark blue line (diamonds) is the NDBC platforms. The light blue line (x) is the National Ocean Service (NOS) platforms. The orange line (triangles) is the IOOS partner’s platforms. The red line (circles) is the Tropical Ocean Atmosphere (TAO) platforms. The dark purple line (stars) is the Oil and Gas platforms in the Gulf of Mexico. The light purple line is the Deep-ocean Assessment and Reporting of Tsunami (DART) buoys.(Click to enlarge)
The TAO Real-time Data Monitoring Subsystem supports daily, weekly, and monthly Quality Assurance/Quality Control (QA/QC) activities by providing on-demand data checking functionality to the DAC. In addition to the automated gross error checking, the real-time data monitoring subsystem provides on-demand reports for once-daily, thorough examination of all current buoy data and detailed review of the real-time data. The reports cover daily QC, platform transmissions, deployment, present positions, Argos positions, latitude/longitude time series, data plots, etc.
To ensure early detection of tsunamis and to acquire data critical to real-time forecasts, NOAA has placed DART®stations at sites in regions with a history of generating destructive tsunamis. NOAA completed the original 6-buoy operational array in 2001 and expanded to a full network of 39 stations in March 2008. The DAC monitors the various real-time transmissions of DART® messages depending on the operating mode of the bottom pressure recorder.
Transmission of real-time water level heights occurs when the tsunami detection algorithm is triggered by a seismic event, when interrogated by the NOAA Tsunami Warning Centers (TWCs) or NDBC, or at pre-scheduled intervals. The bottom unit transmits the messages to the surface buoy via underwater acoustic systems. The surface buoy is equipped with duplicate and independent communications systems to transmit data to Iridium’s satellite constellation and then on to the Iridium Gateway in Arizona, U.S., where the data is routed to the NDBC server. Then NDBC’s real-time processing systems attach NOAA header information and message identifiers and send the data to the NWSTG in Silver Spring, Maryland, for internet distribution via the GTS. NDBC is also responsible for the storage and archival of the DART observations (http://www.ndbc.noaa.gov/dart.shtml).
Various academic, meteorological, and oceanographic institutions and contributing partners of the Integrated Ocean Observing System (IOOS) have implemented High Frequency Radar (HF Radar) sites in their local coastal regions around the continental United States (http://hfradar.ndbc.noaa.gov). These radar sites provide coastal-ocean surface current and wave information offshore out to 300 km. The NOAA has undertaken a project in association with the Scripps Institution of Oceanography (SIO) and the NDBC to combine the data measured from these IOOS partner radar sites onto a national grid and display the surface currents graphically in the form of vectors on the NDBC website. This is a demonstration of the HF Radar display capability currently available. The HF Radar vector data has not been quality controlled by NDBC. The user can access the NOAA HF Radar site at the NDBC website and select a region of interest. Once selected, the site will graphically display the surface current vectors (speed and direction) within the user selected region. This region may be magnified or reduced. The site will also provide a user-option of displaying the data from marine weather stations (buoys and fixed) that are ingested by the NDBC. The data may be viewed from current time back to 24 hours. Data older than 24 hours will be accessible in the future (initial website will only provide most recent 24 hours of data).
DAC Infrastructure
The NDBC DAC is a well-adapted data center capable of supporting the future needs of NDBC as well as those of the evolving U.S. Integrated Ocean Observing System (IOOS). The NDBC DAC is a fully meshed, redundant, enterprise class network capable of delivering for the needs of a 24x7x365 operation. A high-performance platform on which 10 Gigabit Ethernet technology is supported, the NDBC data center network delivers multigigabit bandwidth to all resources while maintaining a 100% uptime status. The computing core of the NDBC network is designed around high-end enterprise class systems. The minimum standards include Dual-Core 1.8Ghz systems, 4GB RAM, redundant Gigabit network interface cards, and redundant power. These ensure that the computing core delivers highavailability and total efficiency.
With network security at the forefront of Information Technology (IT), NDBC has introduced new technology into the network ensuring security is the cornerstone of the data center. Firewalls deliver multi-gigabit wire-speed security at the perimeter of the network protecting resources from compromise attempts. Intrusion detection systems provide wire-speed in-line inspection of all network traffic. This provides an additional layer of security that coupled with firewalls ensures the NDBC network is welldefended and is capable of staying online even in the event of compromise attempts.
Evolving Data Assembly Center
In 2001, NDBC began receiving, processing, quality controlling and disseminating observations from 10 stations deployed by the Gulf of Maine Ocean Observing System. Today, as the Data Assembly Center for IOOS, NDBC provides a similar service for over 40 partners, including the eleven IOOS Regional Associations, and releases approved observations from 284 stations as well as 60 oil and gas platforms in the Gulf of Mexico (http://www.ndbc.noaa.gov/maps/ ADCP_WestGulf.shtml).
A growing service for NDBC is the quality control of these IOOS partner data before the observations are received by boaters, forecasters or the public. Partners include other NOAA and non-NOAA data providers and most importantly, established and nascent coastal ocean observatories. Here again, the DAC provides a 24x7 service for full-spectrum quality assurance and quality control of IOOS real-time observations. DAC personnel report station failures to data providers within three hours of determination, detect real-time data anomalies via manual and automated quality control methods, and assign WMO identifiers to enable the release of partner data in real-time to the GTS. Additionally, the DAC maintains platform metadata in a data base for proper processing and archival at the NOAA data centers and reports communication outages that impact data quality or the ability to deliver observations, and contacts providers to stop/correct the degraded data.
Cooperative Activities
Throughout its history, NDBC has worked closely with the World Meteorological Organization/ Intergovernmental Oceanographic Commission (WMO/IOC) in supporting international cooperation of moored and drifting buoy programs. When the WMO/IOC Data Buoy Cooperation Panel (DBCP) formed in 1984, NDBC brought schematics, documents, and standards to the table to help other nations deploy platforms and quality control their observations. Presently, NDBC chairs the DBCP Drifter Evaluation Panel; serves as the U.S. National Representative to the DBCP and Joint Tariff Agreement meetings; chairs a new DBCP Task Team on Quality Management; and is a member of the North Pacific Data Buoy Advisory Panel and the TAO Implementation Panel. In 2001 NDBC assumed management and oversight of the U.S. Voluntary Observing Ship program, which oversees 16 Port Meteorological Officers and releases observations from approximately 1000 participating vessels. NDBC also participates in the Ship Observing Team meetings and provides direct financial and deployment support for drifters and the ARGO float array. Finally, NDBC’s DAC is responsible for the publication of the Mariners Weather Log.
Understanding climate variability requires the development, maintenance and evaluation of a sustained global climate observing system. NDBC operates the Observing System Monitoring Center (OSMC), which is being funded by the NOAA Office of Climate Observations (OCO) branch, to provide a tool that will assist managers and scientists with monitoring the performance of the global in-situ ocean observing system, identifying problems in real-time, and evaluating the adequacy of the observations in support of ocean/climate state estimation, forecasting and research (http://www.osmc.noaa.gov).
Future DAC
As seen in Figure 2, the number of platforms that the DAC must monitor continues to increase. Each platform collects at least one type of observation – and each observation may have a number of different variables (wind direction, wave height, air temperature, etc.). An enormous increase in observations will require an associated increase in the DAC’s monitoring capabilities. NDBC’s future DAC will be characterized by an infrastructure that is stable and robust. The DAC visualization tool suite and modular framework will bring the DAC into the future by utilizing existing modules and requiring less human intervention during the final stages prior to data distribution. The key is using a modular, higher order algorithm model. The future DAC will have a framework that will allow for improvements by providing for the constant addition of new and better algorithms to the observation modules. Also, with the realignment of the DAC staff, experts will spend less time looking at good observations and more time in development of even higher order algorithms.
Finally, the DAC visualization tool suite will be visual in every sense of the word, providing color contrasts, auto-notification, and other visual indicators of suspect data. Robust servers and efficient communications pipes will transmit data effortlessly and routinely. The DAC will be a visual showroom of screens displaying observation sources and the status of those sources. Contingency sites will be in place providing for load balancing and full scale operations in case of a catastrophic event. Personnel will be aligned to the new DAC enterprise to maximize efficiency while minimizing costs. The NDBC DAC will continue to grow to keep pace with the Data Management and Communications (DMAC) functionality of the IOOS.
Figure 3. The operations watch floor of the Integrated Ocean Observing System (IOOS) Data Assembly Center at the National Data Buoy Center (NDBC). This watch floor is maintained by NDBC personnel on a 24 hour a day, 7 day a week, 365 days a year basis. The editor of the Mariners Weather Log can be seen in the chair to the left of the picture.
Conclusion
The statement, “Quality Goes In, Before the Data Goes Out,” is posted outside the NDBC IOOS DAC. At the DAC, real-time marine weather, climate and ocean observations are continuously received, processed, quality controlled and disseminated to the world. NDBC’s DAC monitors the condition of the ocean tsunami observation network. Since the DAC’s ultimate goal is to maximize the release of quality observations to the world and minimize the release of degraded observations, a 24x7 watch will continue to oversee the entire end-to-end data system until enhanced automated tools are available.