Enabling Big Geoscience Data Analytics with a Cloud-Based, MapReduce-Enabled and Service-Oriented Workflow Framework

PLOS ONE, Published 05 March 5 2015

By Zhenlong Li, Chaowei Yang , Baoxuan Jin, Manzhu Yu, Kai Liu, Min Sun, and Matthew Zhan

“Geoscience observations and model simulations are generating vast amounts of multi-dimensional data. Effectively analyzing these data are essential for geoscience studies. However, the tasks are challenging for geoscientists because processing the massive amount of data is both computing and data intensive in that data analytics requires complex procedures and multiple tools. To tackle these challenges, a scientific workflow framework is proposed for big geoscience data analytics. In this framework techniques are proposed by leveraging cloud computing, MapReduce, and Service Oriented Architecture (SOA).

 Framework architecture.

Framework architecture.

“Specifically, HBase is adopted for storing and managing big geoscience data across distributed computers. MapReduce-based algorithm framework is developed to support parallel processing of geoscience data. And service-oriented workflow architecture is built for supporting on-demand complex data analytics in the cloud environment. A proof-of-concept prototype tests the performance of the framework. Results show that this innovative framework significantly improves the efficiency of big geoscience data analytics by reducing the data processing time as well as simplifying data analytical procedures for geoscientists.”

Geographic Information System in a Multi-Criteria Tool for Mariculture Site Selection

Coastal Management

Coastal Management, published online 13 February 2015

By Joana Micael, Ana C. Costa, Paula Aguiar, António Medeiros, and Helena Calado

“The continuous growth in fish consumption and related activities is stressing the fishing industry worldwide. To counteract this, mariculture might represent an opportunity for consumers, industry and marine resource sustainability, as long as careful site selection is taken into consideration. The current study was developed to assess potential sites for the implementation of marine fish-cage industries on the Azores Archipelago (North Atlantic), through the application of a multi-criteria approach based on geographic information. Descriptors that may have either direct or indirect influences on the development of mariculture activity in the Azores were discriminated into factors and constraints and grouped into environmental, socioeconomic, and administrative categories.

Overall suitability map for offshore mariculture implementation in the waters around S˜ao Miguel.

Overall suitability map for offshore mariculture implementation in the waters around Sao

“Factors were weighted and data integrated using geographic information system (GIS) methods. Suitability maps were generated and a total area of 17.7 km2 was identified as suitable for mariculture in Sao Miguel Island, segmented into different option levels. This multiple criteria approach provides the information necessary for stakeholders to realize the effects of each descriptor in possible implementation sites for mariculture. This will be a useful tool to improve environmental planning, management and decision-making for mariculture activities.”

Introducing the New Esri Science Kit

Also known as the Science Organization Site License, the new Esri Science Kit is designed to deliver easy access to the full suite of GIS technology to researchers.

As entities of universities or as standalone organizations, researchers at a small non-commercial, science organizations or research institutes are unique in that they not only conduct research, but may also coordinate diverse sectors (government agencies, NGOs, small businesses, etc.), build consensus among experts, and pay special attention to GIS project implementation, tool development, and technology transfer. These science organizations are typically much smaller than a university (e.g., 50 to 500 employees).

In terms of licensing of Esri technology, these science organizations may fall through the cracks (i.e., they may not be affiliated with a university, not truly a government agency, not truly a conservation or humanitarian non-profit, and if part of a university, may be located off-campus and thus have difficulties getting resources or recognition from the broader campus university site license). The science kit, a US domestic offering, designed to meet their unique needs.

Researchers at such organizations are typically equivalent to faculty members at a college or university, but not engaged in nearly as much teaching–or perhaps no teaching at all. Their focus is more along the lines of data collection, analysis, interpretation, and publishing of research results in the areas of geography, ocean science, hydrology, ecology, forestry, climate science, geology/geophysics, agricultural science, conservation biology, geographic information science, computer science, sustainability science and/or geodesign. These researchers may be skilled in GIS and computer programming, or rely on the support of GIS analysts or information systems managers at their organization.

A Foundation for Scientific Research

The Esri Science Kit is designed to deliver easy access to the full suite of GIS technology to researchers.  ArcGIS Online is a key offering within the kit so that it may be used as an organizational platform for research, as the infrastructure to perform that research, and as a digital science information laboratory.


The Esri Science Kit is aimed at fostering science collaboration, visualization, analytics, modeling, communication, and in building a complete science data management system. Therefore, if a research grant is forthcoming to the organization, this is the magic ingredient to use.

Load your own data and go,” with Esri basemaps, story maps, ArcGIS Collector, sharing/collaboration space with an ArcGIS Online for Organizations account. The science kit includes the ArcGIS Open Data app as part of the organizational account as another option for easy and open sharing of data (e.g., drag and drop maps, tables, shapefiles into the app to share on the web, open for developers, open for science communicators).

ArcGIS is a comprehensive geospatial platform for science, supporting research, supporting collaboration, supporting spatial analysis, visualization, open data, and science communication at multiple SCALES:

  • At the scale of the individual researcher with basemaps, content management, apps (e.g., field collection, open data, viewing, cloud based spatial analysis, GeoEvent Processor for Server, raster processing in the cloud);
  • At the scale of workgroups in a lab, collaboration and data sharing with partners, plug ins;
  • At the scale of supporting inter-organization collaboration from research institute to research institute, research institute to federal agency (e.g., NOAA/NASA); research institute to national or global initiative such as NSF EarthCube or GEOSS.
  • At the scale of broader citizen engagement, in communicating its science to the general public (e.g., story maps and open web map publishing).

What’s Included

updated science kit

  • Unlimited use of products as defined in the “What’s Included” section
  • 2 User Conference Passes
  • 2 GIS Specialists appointed by Research Institute for Technical Support contacts
  • 2,500 Virtual Campus Credits
  • Esri Press has authorized the gift of two science-related Esri Press Books to Esri Science Organization Site License customers. The choice of the books will be up to Esri Chief Scientist Dawn Wright and the customer.

[Note: The Esri Science Organization Site License (“Esri Science Kit”) does not include installation or implementation services and is for US domestic organizations only. International research organizations already have an International Research Institute Program available through the Esri distributor for a particular country.]


Technical Support Services: Technical support services are included and will be provided in accordance with the then-current Esri technical support policy, provided Licensee remains current on its payment of the Annual Fee. Licensee’s GIS Specialist(s) will be the only individual(s) authorized to contact Esri for technical support.

Upgrades: During the general commercial release of the Products, Esri, at its sole discretion, will deliver to Licensee one (1) upgrade package for each of the Programs included in the Site License or make updates available for downloading from Esri’s website, provided the Annual Fee has been paid and the Agreement is current.

For a Quote, Please Contact:

Dawn Wright, Esri Chief Scientist, dwright@esri.com.

[Updated 25 March 2015]

Spatially Prioritizing Seafloor Mapping for Coastal and Marine Planning

Coastal ManagementCoastal Management, published online 13 February 2015

By Tim Battista and Kevin O’Brien

“Coastal and marine areas provide vital services to support the economic, cultural, recreational, and ecological needs of human communities, but sustaining these benefits necessitates a balance between growing and often competing uses and activities. Minimizing coastal zone conflict and reducing human-induced impacts to ecological resources requires access to consistent spatial information on the distribution and condition of marine resources. Seafloor mapping provides a detailed and reliable spatial template on the structure of the seafloor that has become a core data need for many resource management strategies. The absence of detailed maps of the seafloor hinders the effectiveness of priority setting in marine policy, regulatory processes, and marine stewardship.

Spatial prioritization results from Hot Spot Analysis.

Spatial prioritization results from Hot Spot Analysis.

“For large management areas, the relatively high cost of seafloor mapping and limited management budgets requires careful spatial prioritization. In order to address this problem, a consensus based approach, aided by decision-support tools, and participatory geographic information systems (GIS), was implemented in Long Island Sound to spatially prioritize locations, define additional data collection efforts needed, and identify products needed to inform decision-making. The methodology developed has utility for other states and regions in need of spatially prioritizing activities for coastal planning, and organizations charged with providing geospatial services to communities with broad informational needs.”

Book Provides Progress Report on Creating Spatial Data Infrastructures in Europe

Building European Spatial Data Infrastructures promotes a general understanding of SDI concepts and provides examples of practical applications.

Building European Spatial Data Infrastructures promotes a general understanding of SDI concepts and provides examples of practical applications.

The third edition of Building European Spatial Data Infrastructures presents an update on the efforts of the European Union (EU) to create a system for spatial data sharing among the 28 EU member nations. Building a spatial data infrastructure (SDI)—a common framework or language for sharing geographic information—will make providing geographic information and developing geographic information system (GIS)-based applications easier within the EU.

This nontechnical book covers several topics of interest to government agencies, businesses, and other organizations that want to improve the ability to share environmental, public health, or other types of spatial data throughout the EU. An SDI also creates a more transparent government and improves services to citizens.

Early chapters focus on the many uses of GIS technology, how GIS works, and why SDIs are needed. Later chapters are devoted to how member nations are complying with the plan to create a European SDI, called the Infrastructure for Spatial Information in the European Community (INSPIRE) by the deadline of 2020. Authors Ian Masser and Joep Crompvoets also describe what milestones have been reached and where more work needs to be done.

“GIS can help the EU better deal with many growth-related and other challenges,” says Esri president Jack Dangermond. “However, to collaborate successfully, it’s imperative to implement an SDI with rules for exchanging data and services across boundaries.”

Masser and Crompvoets are two of the world’s leading experts on SDIs. Masser is an emeritus professor at the University of Sheffield, United Kingdom, and was founder chairman of the Association of Geographic Information Laboratories for Europe (AGILE). He currently is a visiting professor at the Centre for Advanced Spatial Analysis at University College London. Crompvoets is an associate professor at KU Leuven Public Governance Institute in Belgium and secretary-general of EuroSDR, a European spatial data research network that promotes applied research between national mapping agencies and research institutes and universities.

Building European Spatial Data Infrastructuresis available in print (ISBN: 9781589483835, 100 pages, US$34.99) or as an e-book (ISBN: 9781589484061, US$34.99). The book is available at online retailers worldwide, at esri.com/esripress, or by calling 1-800-447-9778. Outside the United States, visit esri.com/esripressorders for complete ordering options, or visit esri.com/distributors to contact your local Esri distributor. Interested retailers can contact Esri Press book distributor Ingram Publisher Services.

[Source: Esri press release]

OGC Seeks Public Comment on Candidate 3D Portrayal Service Standard

OGC_newThe membership of the Open Geospatial Consortium (OGC®) seeks public comment on the candidate OGC 3D Portrayal Service (3DPS) Standard.

In the last decade, progress in airborne and mobile laser scanning and photogrammetric methods has brought an explosion of terrain data and detailed 3D models of the built environment. The 3d models are often maintained in proprietary environments.
The Open Geospatial Consortium (OGC) and the Web3D Consortium have both been working to address the need for interoperability, as well as the content challenges of volume, access speed, and diversity of devices. The Web3D Consortium has focused on open standards for real-time 3D visualization, including streaming, and their members developed a Geospatial Component extension for X3D. The OGC has focused on developing a service interface to provide interoperable access to 3D geospatial data servers. In 2012, a group of OGC members, building on work done in both organizations, completed the 3D Portrayal Interoperability Experiment (3DPIE) to develop and evaluate best practices for 3D portrayal services.

Based on the results of the 3DPIE, an OGC 3D Portrayal Service Standards Working group (3D Portrayal Service SWG) was chartered to progress two different OGC proposals to the state of one integrated, adopted OGC standard. The current draft candidate 3D Portrayal Service Standard, a unified web service for 3D portrayal, is intended to make it easy for applications to present, explore, and analyze complex 3D geospatial data from diverse sources.

The candidate OGC 3D Portrayal Service Standard is designed to support both client and server side rendering. For client-side rendering, the client requests a 3D model from the server. The server extracts the requested model from the 3D geodata server and generates a 3D scene graph including geometry and textures. Depending on the server’s capabilities, data formats such as X3D, KML and COLLADA can be used to retrieve the scene graph. The rendering of the scene is done on the client side. In a web client, X3DOM and/or XML3D can be used to integrate the scene into an immersive HTML5 experience. However, there are no fixed format requirements, opening the service for other technologies such as JSON-based glTF. For server side rendering, the client passes the requested content and view parameters to the server. The server then generates layered image depictions of the 3D environment for display on the client. In either scenario, the client’s user can query and navigate through the 3D content.

The documents for the candidate OGC 3D Portrayal Service Standard are available for review and comment at http://www.opengeospatial.org/standards/requests/130. Comments are due by 27 February, 2015. The candidate standard is hosted on GitHub, making it is easy to follow the latest changes and the rationales behind those changes.

[Source: OGC press release]

New Book, “Python Scripting for ArcGIS,” Now Available

Python Scripting for ArcGIS is a guide for experienced users of ArcGIS Desktop to get started with Python scripting without needing previous programming experience.

Python Scripting for ArcGIS is a guide for experienced users of ArcGIS Desktop to get started with Python scripting without needing previous programming experience.

Python Scripting for ArcGISis a guide to help experienced users of ArcGIS for Desktop get started with Python scripting. The book teaches users how to write Python code that works with spatial data to automate geoprocessing tasks in ArcGIS. Experience with other scripting or programming languages is helpful but not required.

Key topics in this book include Python language fundamentals, exploring and manipulating spatial data, working with geometries and rasters, map scripting, debugging and error handling, creating functions and classes, and creating and sharing script tools. Python Scripting for ArcGIS contains 14 chapters with corresponding online data and exercises available on the Esri Press book resource page at esripress.esri.com/bookresources.

Author Paul A. Zandbergen is an associate professor of geography at the University of New Mexico in Albuquerque where he teaches classes in GIS and spatial analysis. His areas of expertise include GIS applications in criminology, economics, health, and ecology, as well as spatial and statistical analysis techniques using GIS.

Python Scripting for ArcGIS is available at online retailers worldwide, at esri.com/esripress, or by calling 1-800-447-9778. Outside the United States, visit esri.com/esripressorders for complete ordering options, or visit esri.com/distributors to contact your local Esri distributor. (Print ISBN: 978-1-58948-371-2, 358 pages, US$79.99) (E-book ISBN: 978-1-58948-362-0, 358 pages, US$79.99).

A mobile-optimized edition is available from the Esri Books app (ISBN: 978-1-58948-402-3, US$59.99).