Q: How can AIS data help me understand vessel traffic?
A: The Automatic Identification System (AIS) reports vessel characteristics over time. For mariners it can show location and status of vessels in real time. When analyzed over long time periods, AIS data can show trends and distribution of vessel traffic density, compliance with route and speed recommendations, and relative use patterns by vessel type, as well as function as a tool to model ocean noise and help determine potential location-based conflicts or alternatives when planning the construction of offshore structures and conducting dredge operations.
Q: Who uses AIS?
A: AIS is required on ships of 300 gross tons or more, with passenger ships and tankers having a lower threshold of 150 or more gross tons. In addition to the weight requirements, vessel length comes into play; as of early 2016, all self-propelled vessels greater than or equal to 65 feet in length and towing vessels greater than 26 feet in length are required to use AIS. Vessels owned, leased, or operated by the military or other U.S. government entities are exempt from having to use AIS. For a more complete description of these requirements, please see Title 33, Code of Federal Regulations.
Q. Are AIS data available everywhere?
A: The AIS data published by MarineCadastre.gov originate from the U.S. Coast Guard’s National AIS (NAIS), which is composed of approximately 200 land-based receiving stations located near important navigation routes in the conterminous U.S., Alaska, Hawaii, and parts of the Caribbean. NAIS data are generally not available for the Arctic, waters beyond 40 or 50 miles of the coast, or foreign waters. MarineCadastre.gov partitions the data by UTM zone to streamline downloading, and these zones are indexed on a UTM zone index map. Data may not be available for all zones and dates.
Q: Why doesn’t MarineCadastre.gov have AIS data for every year?
A: Every year MarineCadastre.gov prioritizes its data development activities (including AIS) according to available funding and needs expressed by its user community. We cannot guarantee AIS data every year without a defined user need and the resources to do the work. AIS data will likely only be processed for every other year with the assumption that funding levels remain stable.
Information on specific user needs can help the MarineCadastre.gov staff prioritize AIS in the data development queue. Please let us know how you are using the data, and why having yearly data is important to your application.
Q: What does MMSI stand for, and what is it used for?
A: MMSI is an acronym for “Maritime Mobile Service Identity” and is a 9-digit sequence used to identify a vessel. The first three digits catalog the country of origin. U.S. flagged vessels are assigned a code of 338, 366, 367, 368, or 369. The remaining six digits can each range from 0 to 9 and catalogue the individual vessel in combination with the country of origin code. For the 2010 and 2011 data distributed by MarineCadastre.gov, the MMSIs have been encrypted to obscure the last six digits at the request of the Coast Guard. The country code of the original MMSI remains unchanged. The encrypted MMSIs still provide a unique ID for each vessel and allow the broadcast points to be linked to the additional records in the vessel and voyage tables.
Q: Should I use the point data, make track lines, or make density maps?
A: How the data are processed largely depends on their intended use. The MarineCadastre.gov point data can be used as is. Point records contain the richest content and have values such as Speed over Ground and Course over Ground. These are not retained when the points are converted to track lines. Points are also an excellent choice if animation is necessary to show the interaction of vessels over time. However, the high volume of point data can become cumbersome to manage and requires additional filtering if the ultimate goal is to make heat maps. Track lines offer a very efficient storage model, make for an intuitive display, and can simplify spatial analysis. They also provide the added benefit of normalizing the effect of inconsistent point counts as a result of varied vessel speed. Generally, track lines are the easiest and most accurate geometry type to use when building density or heat maps. Generating track lines can present a large up-front time commitment.
Heat or density maps are used to distill large volumes of vessel traffic into a single concise picture. All unique values like Vessel Type, Length, and Drought are ignored in the process and cannot be linked to the final density measurement—which represents a total vessel count per unit area. Density maps are relatively easy and quick to generate, and are highly effective at visualizing traffic volume.
Q: How can I speed up my processing time?
A: AIS data sets can contain millions of records and result in very long processing times. A few approaches can be used to reduce processing times, such as
- Limiting the geographic scope of analysis by using Clip or Erase functions
- Limiting the time period of analysis
- Converting the points immediately to tracks before applying any other filtering techniques
- Eliminating the relationship classes and any unnecessary field values
- Enabling the 64-bit background geoprocessing option in ArcGIS—especially with Track Builder
- Compact or Export the file geodatabase after any large edits to free up unused space
Q: How can I output my records to Excel or MatLab?
A: The easiest approach is to save the data in an ASCII-delimited text format.
- Load the feature class into ArcMap, open the table view, select the drop-down arrow of the upper left tool, and select Export. Select the folder to see the Saving Data Export window, and then select the drop-down arrow on the Save As Type menu entry
- Use the arcpy.ExportXYv_stats function
- Use a combination of the arcPy.SearchCursor and the Python function CSVFile
- Use ArcPy functions to write NumPy arrays as an input to MatLab tools
- Use the OGR/GDAL library to translate the FileGeodatabase records into your desired format
- SAFE Software has also been around for a long time and is used by many people for translating formats: safe.com
Q: Are the relationship classes necessary? Can I make my own?
A: The relationship classes maintain the logical link between the broadcast point features and the vessel and voyage records for processing when using the AIS Data Handler. The relationship classes are not required otherwise, but can be useful for automatically selecting the vessel or voyage records associated with a particular point feature. Alternative relationship classes can be constructed that may suit a particular workflow. See the ArcGIS Help topic on relationship classes at resources.arcgis.com/en/help/main/10.1/index.html#//004t00000006000000. The section entitled “An overview of creating and maintaining relationship classes” is a good place to start. In some workflows, using the relationship class can result in longer processing times.
Q: What are the best density settings to use?
A: Appropriate density settings depend on the properties of the input data—such as the distribution and volume of records. There are two methods to calculate density: 1) the point/line or 2) the kernel. Point/line density is calculated from a neighborhood around each output cell. The number of points and lines in that neighborhood is divided by the area of the neighborhood, giving a magnitude per unit area.
Kernel density is calculated by fitting a smooth, curved surface over each input point or line. The surface value is highest directly over the point or line and tapers to zero at the edge of the search radius. The density of each output cell is calculated by adding the values of all the kernel surfaces that overlap the cell center and dividing by the search area. Kernel density is the most popular method for creating vessel traffic heat maps.
The analysis radius defines the search radius for the density function. A good starting point is to calculate the dimensions of the area of interest and divide the shorter of the width or height by 30. Increasing the radius does not change the calculated density values significantly; while more features will be included in the density calculation, the search area will also be larger. The inclusion of more points will lead to a more generalized output raster.
The population field can be used to assign a count or quantity to each input feature. For point and line density, this value scales the number of times the feature is counted when calculating density. For kernel density, the population field represents the total value spread over the surface representing each feature. When calculating basic vessel densities, it is not necessary to use a population field.
The output cell size defines the resolution of the output raster grid. A good starting point for this setting is to calculate the dimensions of the area of interest and divide the shorter of the width or height by 250.
Q: How can I use data from multiple UTM zones or multiple months?
A: MarineCadastre.gov AIS data are divided into file geodatabase by month and Universal Transverse Mercator (UTM) zone. If the area of interest spans multiple zones, use the ArcGIS Merge command to paste the component zones together. To speed up this process, delete the relationship classes first and then run the Merge once each for the broadcast points, the vessel tables, and the voyage tables. As necessary, rebuild the relationship classes using the new feature class names.
Q: What is the difference between the AIS Data Handler and the Track Builder?
A: The AIS Data Handler performs many operations on AIS data, including removal of duplicates and orphans, filtering, aggregation, analysis, reporting, and creation of output products such as density maps and track lines. Most of the tools depend on the database structure of the MarineCadastre.gov AIS file geodatabases. The Data Handler is an ArcGIS 10.0 application, has a custom graphical user interface, and is written in C# and ArcObjects. As of mid-2015 the Data Handler tool is no longer supported; however we continue to make it available as reference material for developers only.
The Track Builder is used exclusively for creating track lines. It is compatible with ArcGIS 10.1 or later. Users comfortable with Python can alter the version test in the script to make it compatible with ArcGIS 10.4. While the AIS Data Handler also builds track lines, there are several key differences with data input requirements and the track-building methods. The AIS Data Handler requires the MarineCadastre.gov AIS file geodatabase format, while the Track Builder can use any point feature class with a unique identifier field and a date-time field. The differences in method are based on how track lines are segmented for a single vessel. The Track Builder tool will split tracks based on a maximum time or distance between points, or using a user-defined time interval—such as every 24 hours. This function works particularly well to remove outliers.
The AIS Data Handler will create tracks using all points for a given MMSI, based on a user-defined time interval, or using the concept of purposeful motion. Because track building is such an important component of AIS data processing, the Track Builder was designed to optimize this step. The Track Builder is an ArcGIS 10.X ArcPy script that can be customized to fit into various workflows and has a single user interface panel. Please see the Track Builder tutorial for more information.
Q: In the 2014 data, I will occasionally find a month where the record count is obviously too low, what is happening here?
A: Over the course of 2014, the U.S. Coast Guard began making changes to its method of identifying and recording the raw data. Documentation of these changes is not yet available. The MarineCadastre.gov team made several efforts to analyze the raw input data in order to correctly read and transform the records. This process was unable to find all changes to the input data; thus, omissions do exist in the 2014 products released by MarineCadastre.gov.
Q: Can I get the raw AIS data that the MarineCadastre.gov team uses to generate its products?
A: MarineCadastre.gov does not distribute the raw AIS data it receives from the U.S. Coast Guard. State and federal agencies, and contractors directly working for those agencies, may request AIS data in various formats from the U.S. Coast Guard at navcen.uscg.gov/?pageName=dataRequest&dataRequest=aisHistoricalRequestForm.
Q: Where can I find out more details about the data elements and their units recorded in an AIS record?
A: From this U.S. Coast Guard NAVCEN at navcen.uscg.gov/?pageName=AISMessages, and from this undocumented source at catb.org/gpsd/AIVDM.html.
Q: Do you provide access to real-time AIS records?
A: No. The MarineCadastre.gov project generally acquires raw data from the Coast Guard for the previous calendar year. These data are converted into derived products in the File Geodatabase format.
Q: How do I cite AIS data from MarineCadastre.gov?
A: “AIS data made available from MarineCadastre.gov, Bureau of Ocean Energy Management, and National Oceanic and Atmospheric Administration (YEAR), marinecadastre.gov”