The New world of 'Big Data' Analytics and High Performance Data: A Paradigm shift in the way we interact with very large Earth Observation datasets Purss B J Matthew, Lewis Adam, Ip Alexander, Evans Ben ABSTRACT The next decade promises an exponential increase in volumes of open data from Earth observing satellites. The ESA Sentinels, the Japan Meteorological Agency's Himawari 8/9 geostationary satellites, various NASA missions, and of course the many EO satellites planned from China, will produce petabyte scale datasets of national and global significance. It is vital that we develop new ways of managing, accessing and using this 'big-data' from satellites, to produce value added information within realistic timeframes. A paradigm shift is required away from traditional 'scene based (and labour intensive) approaches with data storage and delivery for processing at local sites, to emerging High Performance Data (HPD) models where the data are organised and co-located with High Performance Computational (HPC) infrastructures in a way that enables users to bring themselves, their algorithms and the HPC processing power to the data. Automated workflows, that allow the entire archive of data to be rapidly reprocessed from raw data to fully calibrated products, are a crucial requirement for the effective stewardship of these datasets. New concepts such as arranging and viewing data as 'data objects' which underpin the delivery of 'information as a service' are also integral to realising the transition into HPD analytics. As Australia's national remote sensing and geoscience agency, Geoscience Australia faces a pressing need to solve the problems of 'big-data', in particular around the 25-year archive of calibrated Landsat data. The challenge is to ensure standardised information can be extracted from the entire archive and applied to nationally significant problems in hazards, water management, land management, resource development and the environment. Ultimately, these uses justify government investment in these unique systems. A key challenge was how best to organise the archive of calibrated Landsat data (estimated to grow to almost 1 PB by the end of 2014) in a way that supports HPD applications yet with the ability to trace each observation (pixel) back to its original satellite acquisition. The approach taken was to develop a multi-dimensional array (a data cube) underpinned by the partitioning the data into tiles, without any temporal aggregation. This allows for flexible spatio-temporal queries of the archive whilst minimising the need to perform geospatial processing just to locate the pixels of interest. Equally important is the development and implementation of international data interoperability standards (such as OGC web services and ISO metadata standards) that will provide advanced access for users to interact with and query the data cube without needing to download any data or to go through specialised data portals. This new approach will vastly improve access to, and the impact of, Australia's Landsat archive holdings.

The AusSeabed Marine Data Portal provides openly accessible seabed data to users. This report details the results and recommendations following a community survey on the portals functionality and usability. The report informs future development of the AusSeabed Marine Data Portal to ensure it meets end-user needs.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB2020 is a GIS data product, replacing AMB2014a, providing access to the data for Australia's maritime zones. Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2020 is a digital representation of the scheduled and offshore areas defined under this Act. In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. The AMB2020 version has been updated to reflect the 2018 Timor Sea Treaty, including the Bayu-Undan Offshore Area. Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 comprise the scheduled and offshore areas defined under the Act. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: August 2019. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 8 S to 70 S and the longitude limits 39 E to 174 E. This dataset includes the following disclaimers: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Offshore Petroleum and Greenhouse Storage Act 2006. In the event of an inconsistency between AMB data and the areas under the legislation, the latter prevails.

The dataset was created by Geoscience Australia using the framework described in Section 33 of the Offshore Minerals Act 1994. The international, scheduled areas and coastal waters used in this dataset are those found in the historical Australian Maritime Boundary Dataset 2006 (AMB2006). The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be known to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. The blocks are attributed with fields containing information on: -Block ID -Parent 1 Million Mapsheet -Offshore Area -Epoch of the boundaries used to cut the data AMB2006 -Datum -Origin of the mapsheet in AGD66 -The position of all vertices in the block -The number of vertices in the block -The area of the block in acres -The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the Offshore Minerals Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference. This dataset is published with the permission of the CEO, Geoscience Australia

The dataset was created by Geoscience Australia using the framework described in Section 33 of the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGS Act) and previous Acts. The international, scheduled areas and coastal waters used in this dataset are those found in the current Australian Maritime Boundary Dataset 2014 (AMB2014a). This version contains a number of technical amendments made on the basis of user feedback. These include: - Removal of lease AC/P17 - Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be known to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. Existing production licences AC/RL4, and AC/RL5 are included in the dataset. The blocks are attributed with fields containing information on: - Block ID - Parent 1 Million Mapsheet - Offshore Area - Epoch of the boundaries used to cut the data AMB2014 - Datum - Origin of the mapsheet in AGD66 - The position of all vertices in the block - The number of vertices in the block - The area of the block in acres - The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the OPGGS Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB2014 is a GIS data product, replacing AMBIS 2006, providing access to the data for Australia's maritime zones. Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 is a digital representation of the scheduled and offshore areas defined under this Act. In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: - Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 Offshore Petroleum and Greenhouse Storage Act 2006 - Australian Maritime Boundaries Dataset 2014 comprise the scheduled and offshore areas defined under the Act. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 8 S to 70 S and the longitude limits 39 E to 174 E. This dataset includes the following disclaimers: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Offshore Petroleum and Greenhouse Storage Act 2006. In the event of an inconsistency between AMB data and the areas under the legislation, the latter prevails.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB is a GIS data product, replacing Australian Maritime Boundaries 2014 (AMB2014), providing access to the data for Australia's maritime zones. AMB is a digital representation of Australia's limits and boundaries as established under the Seas and Submerged Lands Act 1973 and associated proclamations. The data also include a representation of the limits by which the scheduled and offshore areas of each of the States and of the Northern Territory are determined under the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGSA). In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: · Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 AMB data comprise the territorial straight baselines and the outer limits of each zone, together with maritime boundaries determined by treaties between Australia and neighbouring countries. All data coordinates are supplied in the GDA94 datum. The data reflects the location of the coastline within the constraints of available source material at the time of production. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 7 S to 70 S and the longitude limits 40 E to 175 E. This dataset includes the following disclaimer: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Seas and Submerged Lands Act 1973. The data also includes a representation of the limits by which the waters adjacent to each of the Australian States and of the Northern Territory are determined under the Coastal Waters (State Powers) Act 1980, Coastal Waters (Northern Territory Powers) Act 1980 and the Offshore Petroleum and Greenhouse Gas Storage Act 2006. In the event of an inconsistency between AMB data and the baselines and limits under the legislation, the latter prevails.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB is a GIS data product, replacing Australian Maritime Boundaries 2014 (AMB2014), providing access to the data for Australia's maritime zones. AMB is a digital representation of Australia's limits and boundaries as established under the Seas and Submerged Lands Act 1973 and associated proclamations. The data also include a representation of the limits by which the scheduled and offshore areas of each of the States and of the Northern Territory are determined under the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGSA). In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: · Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 AMB data comprise the territorial straight baselines and the outer limits of each zone, together with maritime boundaries determined by treaties between Australia and neighbouring countries. All data coordinates are supplied in the GDA94 datum. The data reflects the location of the coastline within the constraints of available source material at the time of production. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 7 S to 70 S and the longitude limits 40 E to 175 E. This dataset includes the following disclaimer: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Seas and Submerged Lands Act 1973. The data also includes a representation of the limits by which the waters adjacent to each of the Australian States and of the Northern Territory are determined under the Coastal Waters (State Powers) Act 1980, Coastal Waters (Northern Territory Powers) Act 1980 and the Offshore Petroleum and Greenhouse Gas Storage Act 2006. In the event of an inconsistency between AMB data and the baselines and limits under the legislation, the latter prevails.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB is a GIS data product, replacing Australian Maritime Boundaries 2014 (AMB2014), providing access to the data for Australia's maritime zones. AMB is a digital representation of Australia's limits and boundaries as established under the Seas and Submerged Lands Act 1973 and associated proclamations. The data also include a representation of the limits by which the scheduled and offshore areas of each of the States and of the Northern Territory are determined under the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGSA). In the event of an inconsistency between AMB data and the limits under any legislation, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: · Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 AMB data comprise the territorial straight baselines and the outer limits of each zone, together with maritime boundaries determined by treaties between Australia and neighbouring countries. All data coordinates are supplied in the GDA94 datum. The data reflects the location of the coastline within the constraints of available source material at the time of production. AMB Data can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the AMB data is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 7 S to 70 S and the longitude limits 40 E to 175 E. This dataset includes the following disclaimer: AMB data is a digital representation of the territorial sea baseline and of the outer limits of Australia's maritime zones. The baseline and zones are established under the Seas and Submerged Lands Act 1973. The data also includes a representation of the limits by which the waters adjacent to each of the Australian States and of the Northern Territory are determined under the Coastal Waters (State Powers) Act 1980, Coastal Waters (Northern Territory Powers) Act 1980 and the Offshore Petroleum and Greenhouse Gas Storage Act 2006. In the event of an inconsistency between AMB data and the baselines and limits under the legislation, the latter prevails.

The dataset was created by Geoscience Australia using the framework described in Section 33 of the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGS Act) and previous Acts. The international, scheduled areas and coastal waters used in this dataset are those found in the historical Australian Maritime Boundary Dataset 2006 (AMB2006). This version contains a number of technical amendments made on the basis of user feedback. These include: - Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be known to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. Existing production licences AC/P17, AC/RL4, and AC/RL5 are included in the dataset. The blocks are attributed with fields containing information on: - Parent 1 Million Mapsheet - Offshore Area - Epoch of the boundaries used to cut the data AMB2006 - Datum - Origin of the mapsheet in AGD66 - The position of all vertices in the block - The number of vertices in the block - The area of the block in acres - The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the OPGGS Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference.