A free software license for the BUFR software H. Paulitsch / TU-Graz At the first OPERA-III meeting the need for a license for the BUFR software has been discussed in WG2 and TU-Graz has been asked to prepare such a license. There are a lot of different free software licenses available (see the list), with the GNU General Public License. OPERA Operational Programme for the Exchange of Weather Radar Information FMBUFR Encoding and Decoding Software Version For BUFR Software.

1. Free Decoding Software Download
2. Mpeg-2 Decoding Software
3. Bufr Decoding Software Programming

NCEP Home > NCO Home > Systems Integration Branch > Decoders > BUFRLIB > BUFRLIB Table of Contents > Decoding NCEP PREPBUFR Files
Printer Friendly Version

Example Program: Decoding NCEP PREPBUFR files

In this example, we will demonstrate how to decode(i.e. read) data from an NCEP PREPBUFRfile. These files are special BUFR files containing the entire set of data(including quality control!)that is input into the analysis step of a particular NCEP model run, and theirformat is defined by the tables fileprepobs_prep.bufrtable.

The sample program below assumes that the PREPBUFR file to be read has alreadybeen FORTRAN-blocked, viathe NCEP cwordsh utility orotherwise, and that it is thusdirectly readable as a result of using the FORM='UNFORMATTED' attribute withinthe relevant FORTRAN'OPEN' statement. However, aside from that, it will beworthwhile to take the time now to discuss certain other aspects of the PREPBUFRformat in greater detail before actually delving into the sample program itself,since such details will go a long way towards explaining and clarifying theactual program design.

To begin with, a PREPBUFR file does not always contain,within each single data subset, the data for an entire report!Instead, for reports which containmass (i.e. temperature, moisture, etc.) as well aswind (i.e. directionand speed, U and V component, etc.) data values, such data values are storedwithin two separate but adjacent (within the overall file) data subsets, whereeach related subset, quite obviously,contains the same report time, location, stationidentification, etc. information as the other, but where the 'mass' subsetcontains the pressures and/or height levels at which 'mass' data valuesoccur, while the corresponding 'wind' subset contains the levels at which'wind' data values occur. While it is true that this may, in some cases,cause the same pressure and/or height level to appear inboth subsets, this separation is nonethelessmaintained for historical reasons peculiar to NCEP.At any rate, the below program will actually merge all of the datafrom both subsets into a single,unified report in such cases, so that the final decoded output is clearerand more intuitive.

In addition, and owing to the rich amount of quality control (QC) informationthatit contains, an NCEP PREPBUFR file contains anevents stack for each of thebasic data values such as pressure, temperature, wind, etc. in the file.This 'events stack' contains the history of QC activitiesassociated with that particular data value, including codes describing whichNCEP QC program modified that particular data value (and why?). The result isa stack describing the entire QC history of that particular data value, fromthe original reported value (at the bottom of the stack) to the 'latest andgreatest' QC'ed value (at the top of the stack) that was used in therespective NCEP model analysis. This 'events stack' mechanism is implementedwithin the BUFRLIB software via the use of subroutineUFBEVN (and associated '[]'notation surrounding the affected mnemonics within theprepobs_prep.bufrtable file),and it can perhaps be best pictured mentallyin the context of our usual 2-dimensional r8arr array,containing data values for pressure, temperature, wind, etc. at some number of levels,but where eachdata value itself now has an 'events stack' behind it, so that theoverall result is now a 3-dimensional 'box' of data values, so to speak!

Of further note, since an NCEP PREPBUFR file contains the entire set of datavalues that is input into a particular NCEP model analysis, it thereforestands to reason that there would be many different types of data subsetsrepresenting many different types of reports (e.g. rawinsonde, METAR, ship,satellite, etc.) within such a file, and this is indeed reflected by thelarge variety of Table A mnemonics that are listed in the tables fileprepobs_prep.bufrtable.However, the method by which a PREPBUFR file isconstructed guarantees that all BUFR messages of a particular type (i.e.Table A mnemonic) will occur adjacent to one another within the file, and thisfeature, in tandem with the aforementioned realization that corresponding 'mass'and 'wind' subsets for a particular report are always adjacent to one anotheras well within the overall context of the file, is conveniently exploitedwithin the below sample program:

Unlike in some of our other examples, the above program can be directly compiledand run 'as is', without any additional code supplied by the user. Theprogram consists of a main routine and subroutine, where each of theseexplicitly INCLUDEs the file 'readpb.prm',which itself contains an importantCOMMON block as well as some other useful global definitions. The main routinebegins by opening a unique output file in which to store decoded data valuesfrom each type of BUFR message that will be read. Then, the PREPBUFR fileitself, which presumably exists as filename 'prepbufr.in' on the local system,is assigned logical unit #11 and FORM='UNFORMATTED' since, again, we areassuming that the file has already been FORTRAN-blocked for reading. Next, wemake the required call to OPENBF, where the astute readermay have alreadynoticed that '11' was passed in as both the first andthird call argument,meaning that the necessary BUFR tables information is embedded within the firstfew BUFR messages of the file itself! Recall that this is the case for allBUFR files (such as PREPBUFR) that are written out directly by the BUFRLIBsoftware; thus, when subsequently reading such files, we do not need togo to the trouble of assigning an additional logicalunit to an ASCII-text version of the same BUFR tables file(in this case, prepobs_prep.bufrtable)for use by OPENBF.

Continuing on within the main program, a looping mechanism is now set up toread each 'complete' report, one at a time, from the PREPBUFR file.By 'complete', here we mean all of the basic data(e.g. pressure,temperature, moisture, wind, etc.) for a particular observing siteat a particular time and location, even when such data is stored withinthe file across separate 'mass' and 'wind' data subsets! This is accomplishedvia the included subroutine above, which uses function IREADNS to movethrough the PREPBUFR file one data subset at a time, and which automaticallymerges data from corresponding 'mass' and 'wind' subsets by taking advantage ofthe fact that such corresponding subsets are guaranteed to be adjacent to eachother within the overall context of the file. The decoded data from eachso-called 'complete' report is then written out to the appropriate output filein an easy-to-read format, with each level of the report listed separately andcontaining the entire 'events stack' for every data value at that level.

Before leaving this discussion, we should point out that, for users who areinterested in working further with the NCEP PREPBUFR format, there is additionaldocumentation available, including explanations of the meanings of all of thedifferent QC program codes and reason codes used within PREPBUFR files. See theNCEP/EMC PREPBUFR Processing Information Pagefor all of the latest details.

The Binary Universal Form for the Representation of meteorological data, or BUFR, is a data format maintained by the WMO. It is used mainly for encoding meteorological observations of all sorts, including those from synoptical stations, radiosondes and satellites. Because BUFR is encoded in binary, it can be stored, transmitted and processed efficiently. It is table driven, so that it can encode data in a flexible fashion. This makes it easy to use BUFR to describe new types of observations or describe old types of observations in new ways.

A short overview of the structure of BUFR is presented below. For more in-depth technical documentation of BUFR, see the WMO's Manual on Codes.

BUFR is designed to be processed using machines, not directly by humans. Therefore, specialized software is required in order to work with it. Some available software packages at the end of this text.

Free Decoding Software Download

Structure of a BUFR message

Data encoded in BUFR is called a message. A message consists of one or more subsets of related meteorological data. The structure of a BUFR message is described below, followed by a short example of how it could be read. Please note that this is intended to be a summary, not technical documentation. If you need technical, in-depth information about the BUFR format, see the WMO's Manual on Codes.

• Section 0: Indicator section
  Contains the letters 'BUFR', the length of the message and the BUFR edition number.
• Section 1: Identification section
  Contains information about the type of data contained in the BUFR.
• Section 2: Optional section
  Contains local data that is not regulated by the WMO.
• Section 3: Data description section
  Describes what variables the BUFR contains in the data section and where they are to be found. It also states how many times this description is repeated in the form of subsections. Elements in section 3 can be meteorological variables that were observed and other metadata, such as the instrument used to make the observation or data quality. They can also signify known sequences of elements that occur commonly; it's possible to encode a sequence that always contains height, latitude and longitude, for example, rather than describing these elements individually. This saves space. Other elements in section 3 can be tell a processing program e.g. that multiple elements follow, that elements will follow in great precision than usual, etc.
• Section 4: Data section
  This section contains the actual data described in section 3.
• Section 5: End section
  This section contains the letters '7777', signifying that the message has come to an end. It can be used together with the length of the message, encoded in section 0, to check that the BUFR was transferred properly.

It may seem strange to format data like this. Why does it make sense to describe the data in data description section and then write the data in another section? Other formats write data descriptions and data together. An extremely simple XML snippet, for example, could look like this:

Mpeg-2 Decoding Software

In this example, the metadata is stored next to the data it's referring to. The data is also quite verbose - 202 characters long.

Bufr Decoding Software Programming

In BUFR, the same piece of message could be organized using the following logical structure expressed in XML (this is a representation of the message's logical content for simplicity's sake; in binary it would be different):

This is a lot shorter than the previous representation of the same data (125 characters). When interpreting the above message, the decoding software would first look up what the elements described in section 3 in a table stored elsewhere in order to find out the units and length of the data belonging to the element in section 4. Again, this is an example of BUFR's logical structure - in the real BUFR tables the lengths and data types are expressed in bits, not characters:

element	length (characters)	unit
temperature	3	K
humidity	2	%
wind speed	4	m/s
wind direction	2	°

The data in section 4 (2968510.372) would then be partitioned according to the number of the characters and interpreted as instances of the elements in section 3:

temperature (K)	humidity (%)	wind speed (m/s)	wind direction (°)
296	85	10.3	72

As you can see, the BUFR way of storing the data is much more compact, but not as easy to read for a human. In reality, the BUFR could be much more complex than in this example, so that the advantages of using it compared to traditional codes would be more pronounced.

Allowing software to find out how the data in the message is structured before having to process the data itself means that the software can create a strategy for decoding the data that is especially efficient. It also makes it possible to separate the data decoding from interpretation, so that one part could be delegated to another program. In general, with good software, storing data like this can make processing more efficient and allow it to take up less space.

Software for working with BUFR

There is no definitive software library for working with BUFR. Most open software that can process BUFR is written for UNIX-like systems. Currently there is no well-known software for working with BUFR that includes a graphical interface, so some programming knowledge is required.

DWD owns software that can process BUFR extremely efficiently. It is also able to show it graphically. Unfortunately, because DWD is a government agency, it is currently not clear whether we are allowed to give access to our software. For this reason we cannot give out our software for encoding or decoding BUFR. However, we can provide a list of software packages from other providers that can process BUFR:

BUFRDC	A Fortran library by ECMWF to encode and decode BUFR. It will be replaced in the future by ecCodes.
ecCodes	A beta C library by ECMWF to encode and decode meteorological codes in general, including BUFR. It includes command line tools that are completely text-based, but do not require programming to use. It is currently under active development.
BUFRLIB	A Fortran library by NCEP for encoding and decoding BUFR.
libECBUFR	A C library by Environment Canada for encoding and decoding BUFR.
wmobufr	A Java library by UCAR for decoding BUFR and translating it to XML.
wreport	A C++ library by Arpae for encoding and decoding BUFR.
pybufr-ecmwf	A Python interface for BUFRDC with an object oriented / Pythonic interface.
pytroll.python-bufr	A Python interface for BUFRDC by the PyTroll project.