Coping with the deluge of 'Big Data': The challenge of exploiting satellite earth observation data in the new era of High Performance Data Matthew B J Purss, Adam Lewis, Alexander Ip, Lesley Wyborn ABSTRACT Australia's Earth Observation Program has acquired and archived satellite data for the Australian Government since the establishment of the Australian Landsat Station in 1979. Data have been acquired from many sensors and platforms including ERS, EnviSAT, MODIS, ASTER, SPOT and ALOS, although the bulk of the continuous observations are from the Landsat instruments. The Landsat mission is the longest continuous environmental monitoring experiment in history; producing a global archive of earth observations spanning over 41 years. Geoscience Australia maintains an archive of Landsat data for Australia and produces products and information to support the delivery of government policy objectives. Future Earth observation missions promise an exponential increase in the volumes of open data from Earth observing satellites. For the Australian region the NASA/USGS Landsat-8 satellite is now contributing up to 50 GB of data per day and ESA's Sentinel-2 constellation (due for launch in early 2014) will provide close to 500 GB of data per day to Australia's existing archive of earth observation data. With just these two new data sources the Australian Satellite Earth Observation archive is expected to grow to around 1 PB by the end of 2014. Extracting information from satellite data is a long-standing challenge made more difficult by increased data volumes. Recognising this issue, the Australian Government funded the 'Unlocking the Landsat Archive' (ULA) consortium project from 2010 to 2013 to process Australia's Landsat archive to fully calibrated sensor and scene independent data products for the period from 1998 to 2012 and to investigate methods of arranging this archive so that it can be exploited to produce value added information products. The data outputs from the ULA project, currently totalling close to 400 TB, have become a fundamental component of Australia's eResearch infrastructure. The data are hosted on the National Computational Infrastructure (NCI) and are openly available under a Creative Commons licence. A key challenge for data custodians in this era of High Performance Data (HPD) is how to store and organise very large datasets that extend from terra-scale to peta-scale in a way that will facilitate data interoperability. Building on the foundation laid by the ULA project, Geoscience Australia is developing advanced data cube technologies that will enable the Australian Government to cope with the anticipated flow of new Earth observation data from future platforms. By collocating the data with the high performance computing capability of the NCI, and taking a standards approach to provide access to this data as a service, there is an opportunity to truly unlock the potential of Earth observation data to address questions in ways that previously were not possible.

Australian Petroleum Blocks (2006) - Aligned with the historical Australian Maritime Boundary Dataset (AMB2006) V2.0 The digital realisation of Australia's Graticular Framework as set out under the Offshore Petroleum and Greenhouse Gas Storage Act 2006. Web Service Capabilities WMS,WFS,WCS. The dataset was created internally by Geoscience Australia using the framework described in Section 33 of the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGS Act) and previous Acts. The international, scheduled areas and coastal waters used in this dataset are those found in the current Australian Maritime Boundary Dataset (AMB2006). The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be know to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. Existing production licences AC/P17, AC/RL4, and AC/RL5 are included in the dataset. The blocks are attributed with fields containing information on: Block ID Parent 1 Million Mapsheet Offshore Area Epoch of the boundaries used to cut the data - AMB2006 Datum Origin of the mapsheet in AGD66 The position of all vertices in the block The number of vertices in the block The area of the block in acres The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the OPGGS Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB2014 is a GIS data product, replacing AMBIS 2006, providing access to the data for Australia's maritime zones. Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 is a digital representation of the scheduled and offshore areas defined under this Act. In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: - Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 comprise the scheduled and offshore areas defined under the Act. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 8 S to 70 S and the longitude limits 39 E to 174 E. This dataset includes the following disclaimers: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Offshore Petroleum and Greenhouse Storage Act 2006. In the event of an inconsistency between AMB data and the areas under the legislation, the latter prevails.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB2014 is a GIS data product, replacing AMBIS 2006, providing access to the data for Australia's maritime zones. Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 is a digital representation of the scheduled and offshore areas defined under this Act. In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: - Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 comprise the scheduled and offshore areas defined under the Act. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 8 S to 70 S and the longitude limits 39 E to 174 E. This dataset includes the following disclaimers: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Offshore Petroleum and Greenhouse Storage Act 2006. In the event of an inconsistency between AMB data and the areas under the legislation, the latter prevails.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB2020 is a GIS data product, replacing AMB2014a, providing access to the data for Australia's maritime zones. Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2020 is a digital representation of the scheduled and offshore areas defined under this Act. In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. The AMB2020 version has been updated to reflect the 2018 Timor Sea Treaty, including the Bayu-Undan Offshore Area. Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 comprise the scheduled and offshore areas defined under the Act. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: August 2019. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 8 S to 70 S and the longitude limits 39 E to 174 E. This dataset includes the following disclaimers: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Offshore Petroleum and Greenhouse Storage Act 2006. In the event of an inconsistency between AMB data and the areas under the legislation, the latter prevails.

AusSeabed (ASB) conducted a point in time assessment of the service (The Service) to identify what, if any, improvements could be made to the usability, accessibility, presentation and collection of marine specific information and data between April and August 2020. The assessment was an opportunity to apply an independent critical eye of the services AusSeabed provides and gain valuable insights about what working well and what’s not working well. This will help the AusSeabed team identify improvements and form a backlog of activities and enhancements. The following document provides and overview of the assessment process, the assessment and summarises the insights gained from the assessment.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB is a GIS data product, replacing Australian Maritime Boundaries 2014 (AMB2014), providing access to the data for Australia's maritime zones. AMB is a digital representation of Australia's limits and boundaries as established under the Seas and Submerged Lands Act 1973 and associated proclamations. The data also include a representation of the limits by which the scheduled and offshore areas of each of the States and of the Northern Territory are determined under the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGSA). In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: · Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 AMB data comprise the territorial straight baselines and the outer limits of each zone, together with maritime boundaries determined by treaties between Australia and neighbouring countries. All data coordinates are supplied in the GDA94 datum. The data reflects the location of the coastline within the constraints of available source material at the time of production. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 7 S to 70 S and the longitude limits 40 E to 175 E. This dataset includes the following disclaimer: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Seas and Submerged Lands Act 1973. The data also includes a representation of the limits by which the waters adjacent to each of the Australian States and of the Northern Territory are determined under the Coastal Waters (State Powers) Act 1980, Coastal Waters (Northern Territory Powers) Act 1980 and the Offshore Petroleum and Greenhouse Gas Storage Act 2006. In the event of an inconsistency between AMB data and the baselines and limits under the legislation, the latter prevails.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB is a GIS data product, replacing Australian Maritime Boundaries 2014 (AMB2014), providing access to the data for Australia's maritime zones. AMB is a digital representation of Australia's limits and boundaries as established under the Seas and Submerged Lands Act 1973 and associated proclamations. The data also include a representation of the limits by which the scheduled and offshore areas of each of the States and of the Northern Territory are determined under the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGSA). In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: · Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 AMB data comprise the territorial straight baselines and the outer limits of each zone, together with maritime boundaries determined by treaties between Australia and neighbouring countries. All data coordinates are supplied in the GDA94 datum. The data reflects the location of the coastline within the constraints of available source material at the time of production. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 7 S to 70 S and the longitude limits 40 E to 175 E. This dataset includes the following disclaimer: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Seas and Submerged Lands Act 1973. The data also includes a representation of the limits by which the waters adjacent to each of the Australian States and of the Northern Territory are determined under the Coastal Waters (State Powers) Act 1980, Coastal Waters (Northern Territory Powers) Act 1980 and the Offshore Petroleum and Greenhouse Gas Storage Act 2006. In the event of an inconsistency between AMB data and the baselines and limits under the legislation, the latter prevails.

Australian Mineral Blocks (2014) - Aligned with the current Australian Maritime Boundary Dataset (AMB2014). ESRI Geodatabase. The dataset was created by Geoscience Australia using the framework described in Section 17 of the Offshore Minerals Act 1994. The international, scheduled areas and coastal waters used in this dataset are those found in the current Australian Maritime Boundary Dataset 2014 (AMB2014). The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be known to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. The blocks are attributed with fields containing information on: Block ID Parent 1 Million Mapsheet Offshore Area Epoch of the boundaries used to cut the data AMB2014 Datum Origin of the mapsheet in AGD66 The position of all vertices in the block The number of vertices in the block The area of the block in acres The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the Offshore Minerals Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference.

Australian Petroleum Blocks (2001) - Aligned with the historical Australian Maritime Boundary Dataset (AMB2001) The digital realisation of Australia's Graticular Framework as set out under the Offshore Petroleum and Greenhouse Gas Storage Act 2006. ESRI Geodatabase. The dataset was created internally by Geoscience Australia using the framework described in Section 33 of the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGS Act) and previous Acts. The international, scheduled areas and coastal waters used in this dataset are those found in the historic Australian Maritime Boundary Dataset (AMB2001). The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be know to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. Existing production licences AC/P17, AC/RL4, and AC/RL5 are included in the dataset. The blocks are attributed with fields containing information on: Block ID Parent 1 Million Mapsheet Offshore Area Epoch of the boundaries used to cut the data - AMB2001 Datum Origin of the mapsheet in AGD66 The position of all vertices in the block The number of vertices in the block The area of the block in acres The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the OPGGS Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference.