Coping with the deluge of 'Big Data': The challenge of exploiting satellite earth observation data in the new era of High Performance Data Matthew B J Purss, Adam Lewis, Alexander Ip, Lesley Wyborn ABSTRACT Australia's Earth Observation Program has acquired and archived satellite data for the Australian Government since the establishment of the Australian Landsat Station in 1979. Data have been acquired from many sensors and platforms including ERS, EnviSAT, MODIS, ASTER, SPOT and ALOS, although the bulk of the continuous observations are from the Landsat instruments. The Landsat mission is the longest continuous environmental monitoring experiment in history; producing a global archive of earth observations spanning over 41 years. Geoscience Australia maintains an archive of Landsat data for Australia and produces products and information to support the delivery of government policy objectives. Future Earth observation missions promise an exponential increase in the volumes of open data from Earth observing satellites. For the Australian region the NASA/USGS Landsat-8 satellite is now contributing up to 50 GB of data per day and ESA's Sentinel-2 constellation (due for launch in early 2014) will provide close to 500 GB of data per day to Australia's existing archive of earth observation data. With just these two new data sources the Australian Satellite Earth Observation archive is expected to grow to around 1 PB by the end of 2014. Extracting information from satellite data is a long-standing challenge made more difficult by increased data volumes. Recognising this issue, the Australian Government funded the 'Unlocking the Landsat Archive' (ULA) consortium project from 2010 to 2013 to process Australia's Landsat archive to fully calibrated sensor and scene independent data products for the period from 1998 to 2012 and to investigate methods of arranging this archive so that it can be exploited to produce value added information products. The data outputs from the ULA project, currently totalling close to 400 TB, have become a fundamental component of Australia's eResearch infrastructure. The data are hosted on the National Computational Infrastructure (NCI) and are openly available under a Creative Commons licence. A key challenge for data custodians in this era of High Performance Data (HPD) is how to store and organise very large datasets that extend from terra-scale to peta-scale in a way that will facilitate data interoperability. Building on the foundation laid by the ULA project, Geoscience Australia is developing advanced data cube technologies that will enable the Australian Government to cope with the anticipated flow of new Earth observation data from future platforms. By collocating the data with the high performance computing capability of the NCI, and taking a standards approach to provide access to this data as a service, there is an opportunity to truly unlock the potential of Earth observation data to address questions in ways that previously were not possible.

PLEASE NOTE: These data have been updated. See Related Links for new data. Geodatabase of the Commonwealth Seas and Submerged Lands Act 1973 - An Act relating to Sovereignty in respect of certain Waters of the Sea and in respect of the Airspace over, and the Sea bed and Subsoil beneath, those Waters and to Sovereign Rights in respect of the Continental Shelf and the Exclusive Economic Zone and to certain rights of control in respect of the Contiguous Zone.

This paper presents the development of a new settlement model to predict the long-term settlement of municipal solid wastes (MSWs). The total settlement of the MSWs results from the direct loss of solids due to decomposition and compression due to stress variation induced by loss of solids, flow of gas and liquid and mechanical creep. The geotechnical properties of MSWs are considered as functions of degrees of biodegradation (DOB). To validate the model, two settlement profilers (2.5 years) and three geodetic monitoring networks (2 years) were installed at Mugga Lane Landfill, ACT, Australia to monitor the settlement of an MSW lift and three closed landfill cells, respectively. The settlement rates of the landfill cells with the ages of 7, 8, 14, and 8, 9 and 15 years are 0.53, 0.35, 0.06 and 0.46, 0.36, 0.05 mm/day, respectively. In addition, 5 large scale and 3 small scale bioreactors were set up in the field and laboratory to address the long-term physical, mechanical and biochemical behaviours of the MSWs under different stress levels. The predicted settlement is compared to the test results from the bioreactors, in-situ monitoring data and the settlement predicted using an existing model, which has been improved by coupling the geotechnical properties of MSWs with DOBs. The predicted settlement using the proposed model well fits the test results and monitoring data. The settlement strain is predicted to be 28.2 % during the filling stage of the landfill and 5.9 % in 5 years after the closure. <b>Citation:</b> Yuekai Xie, Jianfeng Xue, Alistair Deane, Numerical modelling of settlement of municipal solid waste in landfills coupled with effects of biodegradation, <o>Waste Management</i>, Volume 163, 2023, Pages 108-121, ISSN 0956-053X, https://doi.org/10.1016/j.wasman.2023.03.025.

Geoscience Australia has the primary role in the delineation of Australia's domestic and international maritime limits and boundaries. An output of this activity is the development of the Australian Maritime Boundaries (AMB) data. AMB is a GIS data product, replacing AMBIS 2006, providing access to the data for Australia's maritime zones. Treaties - Australian Maritime Boundaries Dataset 2014 is a digital representation of Australia's international treaty boundaries. In the event of an inconsistency between AMB data and the limits under any treaty, the latter prevails. The data have been published by Geoscience Australia in consultation with other relevant Commonwealth Government agencies including the Attorney-General's Department, the Department of Foreign Affairs and Trade and the Australian Hydrographic Office. This version contains a number of technical amendments made on the basis of user feedback. These include: - Harmonisation of maritime boundaries, scheduled area and graticular block datasets in block SC51 The Treaties - Australian Maritime Boundaries Dataset 2014a comprises the maritime boundaries determined by treaties between Australia and neighbouring countries. All data coordinates are supplied in the GDA94 datum. Treaties - Australian Maritime Boundaries Dataset 2014a can also be viewed on-line via AMSIS (http://www.ga.gov.au/marine/jurisdiction/amsis.html). Product Specifications Coverage: Australia including all external territories Currency: May 2014. Coordinates: Geographical Datum: GDA94 (functionally equivalent to WGS84). Format: ESRI Geodatabase and REST web service - free online. The area of coverage of the Treaties - Australian Maritime Boundaries Dataset 2014 is for the whole of the Australian marine jurisdiction which includes waters adjacent to the mainland, offshore islands and External Territories. The geographical extent is approximately between the latitude limits of 8 S to 70 S and the longitude limits 39 E to 174 E. This dataset includes the following disclaimers: AMB data is a digital representation of the international maritime boundary treaties of Australia. In the event of an inconsistency between AMB data and the limits under the various treaties, the latter prevails.

The New world of 'Big Data' Analytics and High Performance Data: A Paradigm shift in the way we interact with very large Earth Observation datasets Purss B J Matthew, Lewis Adam, Ip Alexander, Evans Ben ABSTRACT The next decade promises an exponential increase in volumes of open data from Earth observing satellites. The ESA Sentinels, the Japan Meteorological Agency's Himawari 8/9 geostationary satellites, various NASA missions, and of course the many EO satellites planned from China, will produce petabyte scale datasets of national and global significance. It is vital that we develop new ways of managing, accessing and using this 'big-data' from satellites, to produce value added information within realistic timeframes. A paradigm shift is required away from traditional 'scene based (and labour intensive) approaches with data storage and delivery for processing at local sites, to emerging High Performance Data (HPD) models where the data are organised and co-located with High Performance Computational (HPC) infrastructures in a way that enables users to bring themselves, their algorithms and the HPC processing power to the data. Automated workflows, that allow the entire archive of data to be rapidly reprocessed from raw data to fully calibrated products, are a crucial requirement for the effective stewardship of these datasets. New concepts such as arranging and viewing data as 'data objects' which underpin the delivery of 'information as a service' are also integral to realising the transition into HPD analytics. As Australia's national remote sensing and geoscience agency, Geoscience Australia faces a pressing need to solve the problems of 'big-data', in particular around the 25-year archive of calibrated Landsat data. The challenge is to ensure standardised information can be extracted from the entire archive and applied to nationally significant problems in hazards, water management, land management, resource development and the environment. Ultimately, these uses justify government investment in these unique systems. A key challenge was how best to organise the archive of calibrated Landsat data (estimated to grow to almost 1 PB by the end of 2014) in a way that supports HPD applications yet with the ability to trace each observation (pixel) back to its original satellite acquisition. The approach taken was to develop a multi-dimensional array (a data cube) underpinned by the partitioning the data into tiles, without any temporal aggregation. This allows for flexible spatio-temporal queries of the archive whilst minimising the need to perform geospatial processing just to locate the pixels of interest. Equally important is the development and implementation of international data interoperability standards (such as OGC web services and ISO metadata standards) that will provide advanced access for users to interact with and query the data cube without needing to download any data or to go through specialised data portals. This new approach will vastly improve access to, and the impact of, Australia's Landsat archive holdings.

Web services of the Commonwealth Seas and Submerged Lands Act 1973 - An Act relating to Sovereignty in respect of certain Waters of the Sea and in respect of the Airspace over, and the Sea bed and Subsoil beneath, those Waters and to Sovereign Rights in respect of the Continental Shelf and the Exclusive Economic Zone and to certain rights of control in respect of the Contiguous Zone.

Australian Petroleum Blocks (2006) - Aligned with the historical Australian Maritime Boundary Dataset (AMB2006) V2.0 The digital realisation of Australia's Graticular Framework as set out under the Offshore Petroleum and Greenhouse Gas Storage Act 2006. Web Service Capabilities WMS,WFS,WCS. The dataset was created internally by Geoscience Australia using the framework described in Section 33 of the Offshore Petroleum and Greenhouse Gas Storage Act 2006 (OPGGS Act) and previous Acts. The international, scheduled areas and coastal waters used in this dataset are those found in the current Australian Maritime Boundary Dataset (AMB2006). The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be know to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. Existing production licences AC/P17, AC/RL4, and AC/RL5 are included in the dataset. The blocks are attributed with fields containing information on: Block ID Parent 1 Million Mapsheet Offshore Area Epoch of the boundaries used to cut the data - AMB2006 Datum Origin of the mapsheet in AGD66 The position of all vertices in the block The number of vertices in the block The area of the block in acres The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the OPGGS Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference.

AusSeabed (ASB) conducted a point in time assessment of the service (The Service) to identify what, if any, improvements could be made to the usability, accessibility, presentation and collection of marine specific information and data between April and August 2020. The assessment was an opportunity to apply an independent critical eye of the services AusSeabed provides and gain valuable insights about what working well and what’s not working well. This will help the AusSeabed team identify improvements and form a backlog of activities and enhancements. The following document provides and overview of the assessment process, the assessment and summarises the insights gained from the assessment.

Australian Mineral Blocks (2014) - Aligned with the current Australian Maritime Boundary Dataset (AMB2014). ESRI Geodatabase. The dataset was created by Geoscience Australia using the framework described in Section 17 of the Offshore Minerals Act 1994. The international, scheduled areas and coastal waters used in this dataset are those found in the current Australian Maritime Boundary Dataset 2014 (AMB2014). The dataset is comprised of both polygons and points created to very high precision, accurate to within millimetres. The blocks have been cut by Australia's international boundaries, the scheduled areas and the coastal waters. Each block is assigned a polygon, including partial blocks. All blocks are titled with their block ID, and a list of vertices that make up the blocks. Each vertex of the dataset is also replicated as a discrete point in the points dataset. The design of the dataset allows for the exact location of every vertex to be known to millimetre precision. The corner coordinates of blocks are now defined to a high precision, and can be found by querying the appropriate point. The blocks are attributed with fields containing information on: Block ID Parent 1 Million Mapsheet Offshore Area Epoch of the boundaries used to cut the data AMB2014 Datum Origin of the mapsheet in AGD66 The position of all vertices in the block The number of vertices in the block The area of the block in acres The area of the block in hectares The calculation used to find the area of the blocks is estimated to be precise to better than 1%. This is considered to be sufficient as under the permit and licensing arrangements in the Offshore Minerals Act, the area of a block has no relevance. Therefore the area figure is provided solely for reference.

Web Services of the Commonwealth Offshore Petroleum and Greenhouse Gas Storage Act 2006 - An Act about petroleum exploration and recovery, and the injection and storage of greenhouse gas substances, in offshore areas, and for other purposes.