Integrating Scientific Data with GIS: NetCDF and HDF Data in ArcGIS

ArcGIS for Desktop software provides tools to help you access, read, and manage data formatted in Network Common Data Form (netCDF) and hierarchical data format (HDF). These file formats have been designed to support creating, accessing, and sharing scientific data.

Major organizations, such as the National Oceanic and Atmospheric Administration (NOAA), National Aeronautics and Space Administration (NASA), and US Geological Survey (USGS), deliver scientific data in netCDF and HDF. Used extensively by the atmospheric, oceanographic, and remote-sensing communities, these formats are built to store data such as temperature, wind speed, and wave height or satellite imagery representing land surface temperature or evapotranspiration.

A GIS user quickly accesses historical precipitation data in netCDF format to create a weather map and uses the Time Slider tool to see changes in weather patterns over time.

A GIS user quickly accesses historical precipitation data in netCDF format to create a weather map and uses the Time Slider tool to see changes in weather patterns over time.

ArcGIS supports netCDF in the following ways:

  • Read netCDF files natively and access data without converting or importing it.
  • Represent netCDF data as a raster layer, feature layer, and table view.
  • Use netCDF data in spatial and statistical analysis workflows.
  • View a two-dimensional slice of three- or fourdimensional data in the map window.
  • Animate a time-aware netCDF file, use the Time Slider tool, and record and save video clips of the animation.
The ArcGIS Multidimension toolbox provides tools for working with netCDF files.

The ArcGIS Multidimension toolbox
provides tools for working with
netCDF files.

ArcGIS supports HDF in the following ways:

  • Read HDF4 and HDF5 files as raster data, natively, gaining immediate access to the data without the need for converting or importing data.
  • Select which subdatasets to use from a multidataset HDF file interactively in ArcMap™ or using a geoprocessing tool.
  • Manage huge collections of HDF data using a mosaic dataset.
  • Use HDF data in spatial analysis and statistical analysis workflows.
Moderate-resolution imaging spectroradiometer (MODIS) evapotranspiration data in HDF format can be brought directly into the ArcGIS mapping environment.

Moderate-resolution imaging spectroradiometer (MODIS) evapotranspiration data in HDF format can be brought directly into the ArcGIS mapping environment.

An Evaluation of Internet Versus Paper-based Methods for Public Participation Geographic Information Systems (PPGIS)

Transactions in GISTransactions in GIS, Volume 16, Issue 1, February 2012

Amy Pocewicz, Max Nielsen-Pincus, Greg Brown and Russ Schnitzer

“Public participation geographic information systems (PPGIS) are an increasingly important tool for collecting spatial information about the social attributes of place. The availability of Internet-based options for implementing PPGIS presents new opportunities for increased efficiency and new modes of access. Here we used a mixed-mode approach to evaluate paper versus Internet mapping methods for the same PPGIS survey in Wyoming. We compared participant characteristics, mapping participation, and the spatial distribution of mapped attributes between participants who responded to the paper versus Internet option.

Locations of Internet versus paper map points for a subset of survey attributes.

Locations of Internet versus paper map points for a subset of survey attributes: recreation, open space, residential development, and wind development. Internet points are displayed transparently over top of paper points to better illustrate overlap between the two point types

“The response rate for those who completed the paper version of the survey was nearly 2.5 times the response rate of the Internet version. Paper participants also mapped significantly more places than did Internet participants (43 vs. 18). Internet participants tended to be younger, more likely to have a college degree, and had lived in the region for less time than paper participants. For all but one attribute there was no difference in the spatial distribution of places mapped between Internet and paper methods. Using a paper-based PPGIS survey resulted in a higher response rate, reduced participant bias, and greater mapping participation. However, survey mode did not influence the spatial distribution of the PPGIS data.”

Segmentation of Shadowed Buildings in Dense Urban Areas from Aerial Photographs

Remote Sensing, 2012, 4(4), 911-933; published online 29 March 2012

Junichi Susaki

“Segmentation of buildings in urban areas, especially dense urban areas, by using remotely sensed images is highly desirable. However, segmentation results obtained by using existing algorithms are unsatisfactory because of the unclear boundaries between buildings and the shadows cast by neighboring buildings. In this paper, an algorithm is proposed that successfully segments buildings from aerial photographs, including shadowed buildings in dense urban areas. To handle roofs having rough textures, digital numbers (DNs) are quantized into several quantum values. Quantization using several interval widths is applied during segmentation, and for each quantization, areas with homogeneous values are labeled in an image.

Edge completion using filters: (a) non-completed edges, (b) segmentation result using non-completed edges, (c) completed edges, and (d) segmentation result using completed edges. All results were generated with Δdi = 40.

Edge completion using filters: (a) non-completed edges, (b) segmentation result using non-completed edges, (c) completed edges, and (d) segmentation result using completed edges. All results were generated with Δdi = 40.

“Edges determined from the homogeneous areas obtained at each quantization are then merged, and frequently observed edges are extracted. By using a “rectangular index”, regions whose shapes are close to being rectangular are thus selected as buildings. Experimental results show that the proposed algorithm generates more practical segmentation results than an existing algorithm does. Therefore, the main factors in successful segmentation of shadowed roofs are (1) combination of different quantization results, (2) selection of buildings according to the rectangular index, and (3) edge completion by the inclusion of non-edge pixels that have a high probability of being edges. By utilizing these factors, the proposed algorithm optimizes the spatial filtering scale with respect to the size of building roofs in a locality. The proposed algorithm is considered to be useful for conducting building segmentation for various purposes.”

Present at CalGIS 2013

URISAThe four California URISA Chapters, along with the California Geographic Information Association (CGIA), are pleased to present the 19th Annual CalGIS Conference taking place April 15-17, 2013 in Long Beach. The deadline for abstract submissions (and also for registration at the super-early discounted rate) is November 1.

The CalGIS 2013 Conference Committee welcomes the submission of individual papers, complete sessions, luncheon presentations, panels, posters and lightning talks within these overall program tracks/themes (note that all abstracts received will be reviewed and considered for the conference program regardless of the list below):

  • Geospatial Technologies – This track will focus on GIS technology and innovations. Topics in this track could include: software, web mapping, programming, automation, databases, tools, customization and similar technologies.
  • Ports, Transportation and Logistics – This track encompasses presentations with a nod to topics of local (and statewide) interest.  The Port of Long Beach is a major player in the area as well as highway, rail and air hubs. Security and risk assessment topics are also welcome.
  • Education/Certification/Training/Mentoring – This track includes presentations focused on GIS education and skills development.  Topics could include GIS education in K-12 and Higher Education environments as well as professional education and certification (e.g. GISCI – GISP, Esri Certification, ASPRS Certification as examples of professional development). Other topics may include job skills (DOL Competency Model) and industry/agency needs and training for both students and continuing professionals.
  • Water Resources/Marine – Consider this track for presentations focused on statewide and local water issues in both freshwater and marine environments. With Southern California being both a major importer of water as well as having a major coastal/marine connection, this track should have particular significance and relevance for attendees.
  • Government/ Non-Profit – This track will focus on examples and applications of the use of GIS in Government and non-profit sectors.  Sub-tracks focusing on public safety, public health, community planning, etc. will be of particular interest.

Melisa Caric Lee, GISP, President of Compass Rose GIS and 2013 Conference Chair, noted, “CalGIS always offers an exciting mix of technical workshops, presentations, and networking opportunities. GIS professionals at all levels are invited to share ideas and solve problems together. CalGIS is the place to create life-long partnerships and friendships alike! Please join us at the 19th Annual California GIS Conference in beautiful Long Beach, CA. We hope to see you there!”

For information about CalGIS, early registration opportunities and presentation proposals, visit www.calgis.org. The conference will feature training courses and workshops, breakout sessions, keynote addresses, exhibits and networking events. Specific program details will be available in early 2013.

Conference Sponsors:

[Source: URISA press release]

Utility Upgrades Billing System with GIS

Esri logoHoly Cross Energy Saves Money, Improves Customer Service with Street and Security Light Project

Holy Cross Energy, a member-owned electric cooperative utility that serves more than 55,000 consumers in western Colorado, recently used Esri technology for a street and security light project that has helped update and correct its billing system.

“We have saved $442 per month for one of the towns we serve, as they were paying for devices that no longer existed and were paying for higher wattage bulbs than they should have been,” said Holy Cross Energy meter supervisor Tonya Warmenhoven. “An association in our service area, on the other hand, had been getting free street lighting and the address marker power for at least 10 years. They are now billed $600 per month for 178 new devices we have added.”

The utility’s security and streetlight project involved digitally mapping security lights and streetlights within its service area using GPS, aerial photographs, and a custom web mapping application powered by Esri’s ArcGIS technology.

Holy Cross Energy can now provide correct information to its consumers and its billing department. Service and billing personnel now have precise data about devices in the field. In the past, field crews would have had to call the billing department to get information about street and security lights. Now they have access to that data via laptops in the line trucks.

Read more about how Holy Cross Energy implemented its street and security light project.

[Source: Esri press release]

Space-Time Analyses for Forecasting and Understanding Future Incident Occurrence: A Case Study from Yosemite National Park Search and Rescue

GIScience 2012

Paul Doherty, Qinghua Guo, Wenkai Li, Otto Alvarez, and Jared Doke

“The application of GISystems to solve real-world problems continues to expand from reactive, where we simply document and visualize where and when a phenomenon happened, to proactive, where we are able to reliably forecast event locations based on what we have learned from previous events. During this process we often test GIScience theories and techniques, leading to new scientific discovery. This is especially true in the field of spatial epidemiology, which merges spatial analysis with studies from public health (Ostfeld et al. 2005, Robertson et al. 2010). The objectives of such studies are to collect information about spatially varying factors that may contribute to the occurrence of disease, illness or injury.

Yosemite National Park with the overall PBL probability for search and rescue occurrence based on 2001 – 2009 incident data.

Yosemite National Park with the overall PBL probability for search and rescue occurrence based on 2001 – 2009 incident data. Red indicates areas with the highest likelihood for incident occurrence based on conditional probability. Trailheads (access points) are shown as white dots.

“Within spatial epidemiology, datasets often consist of incident coordinates or other locality descriptions that need to be georeferenced. Furthermore, most data describe locations where illness or injuries have previously occurred (presence) but not where they have not occurred (absence). Therefore, analyses have often been limited to descriptive analyses (density or “heat maps”) and spatial statistics (hot spot or Getis Ord G* maps; Getis and Ord 1992) because traditional modeling approaches requires presence and absence data to derive relationships from underlying factors (Hirzel et al. 2002). This limitation is known as the geographic one-class data issue (GOCD; Guo et al. 2011) and requires a specialized approach to generate probability maps.

“Here we study a real-world problem: wilderness or wildland search and rescue (WiSAR) incident prevention in Yosemite National Park using a novel GIScience technique. WiSAR is the process of locating, accessing, stabilizing, and transporting people in remote environments (Worsing 1993). Therefore, our objectives are to describe our methodology, present our results, and discuss the preliminary implications of our findings for WiSAR incident prevention, spatial epidemiology, and GIScience. To do so we used a novel machine learning approach based on GOCD (incident occurrence) to forecast areas of probable occurrence in the future.”

Computational and Methodological Aspects of Terrestrial Surface Analysis based on Point Clouds

Computers & GeosciencesComputers & Geosciences, Volume 42, May 2012, Pages 64-70

Igor Rychkov, James Brasington, and Damià Vericat

“Highlights

  • We present a software toolkit for processing terrestrial point clouds.
  • The toolkit can be applied to TLS surveys of gravel river beds.
  • Improved DEM differencing is one outcome.
  • Estimating surface roughness and grain size distribution is possible now with point-based, statistical metrics.
  • Other applications and extensions are enabled by the library being freely available and open source.
    Keywords

“Processing of high-resolution terrestrial laser scanning (TLS) point clouds presents methodological and computational challenges before a geomorphological analysis can be carried out. We present a software library that effectively deals with billions of points and implements a simple methodology to study the surface profile and roughness. Adequate performance and scalability were achieved through the use of 64-bit memory mapped files, regular 2D grid sorting, and parallel processing. The plethora of the spatial scales found in a TLS dataset were grouped into the “ground” model at the grid scale and per cell, sub-grid surface roughness. We used centroid-thinning to build a piecewise linear ground model, and studied “detrended” standard deviation of relative elevations as a measure of surface roughness. Two applications to the point clouds from gravel river bed surveys are described. Linking empirically the standard deviation to the grain size allowed us to retrieve morphological and sedimentological models of channel topology evolution and movement of the gravel with richer quantitative results and deeper insights than the previous survey techniques.”

ArcNLET: A GIS-based Software to Simulate Groundwater Nitrate Load from Septic Systems to Surface Water Bodies

Computers & GeosciencesComputers & Geosciences, Published online 15 October 2012

J. Fernando Rios, Ming Ye, Liying Wang, Paul Z. Lee, Hal Davis, and Rick Hicks

“Highlights

  • We present an ArcGIS-based software for estimating nitrate loads from point sources.
  • Based on simplified models of groundwater flow and nitrate transport/fate.
  • It is easy to use and has low data requirements.
  • The software provides screening -level estimates of nitrate concentration and load.

“Onsite wastewater treatment systems (OWTS), or septic systems, can be a significant source of nitrates in groundwater and surface water. The adverse effects that nitrates have on human and environmental health have given rise to the need to estimate the actual or potential level of nitrate contamination. With the goal of reducing data collection and preparation costs, and decreasing the time required to produce an estimate compared to complex nitrate modeling tools, we developed the ArcGIS-based Nitrate Load Estimation Toolkit (ArcNLET) software. Leveraging the power of geographic information systems (GIS), ArcNLET is an easy-to-use software capable of simulating nitrate transport in groundwater and estimating long-term nitrate loads from groundwater to surface water bodies. Data requirements are reduced by using simplified models of groundwater flow and nitrate transport which consider nitrate attenuation mechanisms (subsurface dispersion and denitrification) as well as spatial variability in the hydraulic parameters and septic tank distribution. ArcNLET provides a spatial distribution of nitrate plumes from multiple septic systems and a load estimate to water bodies.

“ArcNLET’s conceptual model is divided into three sub-models: a groundwater flow model, a nitrate transport and fate model, and a load estimation model which are implemented as an extension to ArcGIS. The groundwater flow model uses a map of topography in order to generate a steady-state approximation of the water table. In a validation study, this approximation was found to correlate well with a water table produced by a calibrated numerical model although it was found that the degree to which the water table resembles the topography can vary greatly across the modeling domain. The transport model uses a semi-analytical solution to estimate the distribution of nitrate within groundwater, which is then used to estimate a nitrate load using a mass balance argument. The estimates given by ArcNLET are suitable for a screening-level analysis.”

Spatial Analysis of Livestock Production Patterns in Ethiopia

International Food Policy Research Institute International Food Policy Research Institute (IFPRI) and Ethiopia Strategy Support Program II (ESSP II) Working Paper 44, October 2012

“The livestock sector is a large contributor to the Ethiopian economy as well as a mainstay in the livelihoods of many Ethiopians. It comprised 11 percent of national GDP and 24 percent of agricultural GDP between the years of 1995/96 and 2005/06 (NBE 2005/06). Livestock production and markets vary substantially across space in Ethiopia due to a variety of reasons including topographical variations, market access, water availability, and population characteristics.

Estimated sheep and goat population per household and woreda, 2007/08

Estimated sheep and goat population per household and woreda, 2007/08

“This study links smallholder livestock population data from the Agricultural Census (2001/02) and data from the annual CSA Agricultural Sample Survey (2005 to 2008) with Geographic Information Systems (GIS) data in order to assess livestock population, market access, and grazing land. We utilize existing studies of travel time (Schmidt and Kedir 2009) to calculate shares of livestock (cattle, sheep and goat) populations within defined travel time thresholds of major markets. In addition, we attempt to provide greater insight of changes in available grazing land given increasing human and livestock population pressure.”

Measuring Historical Coastal Change using GIS and the Change Polygon Approach

Transactions in GISTransactions in GIS, Volume 16, Issue 1, February 2012

Martyn J. Smith and Robert G. Cromley

“This study compares two automated approaches, the transect-from-baseline technique and a new change polygon method, for quantifying historical coastal change over time. The study shows that the transect-from-baseline technique is complicated by choice of a proper baseline as well as generating transects that intersect with each other rather than with the nearest shoreline. The change polygon method captures the full spatial difference between the positions of the two shorelines and average coastal change is the defined as the ratio of the net area divided by the shoreline length.

A change polygon denoting positive and negative areas of change

A change polygon denoting positive and negative areas of change (a red area represents coastline loss due to erosion and a green area represents land gain due to deposition)

“Although then change polygon method is sensitive to the definition and measurement of shoreline length, the results are more invariant to parameter changes than the transect-from-baseline method, suggesting that the change polygon technique may be a more robust coastal change method.”