OpenTopography Blog

Information and discussion related to high-resolution LiDAR topography for the Earth sciences

---

Category: OpenTopography Updates

---

Short Course: Imaging and Analyzing Southern California’s Active Faults with Lidar

Posted on Wed, September 14, 2011 by Chris Crosby in Education • OpenTopography Updates • Workshops

Mike Oskin (UC Davis), Ramon Arrowsmith (ASU), and I will be teaching a lidar short course October 24 and 25, 2011 at University of California, Davis.  The course, Imaging and Analyzing Southern California’s Active Faults with High-Resolution Lidar Topography, will focus on lidar technology, data processing and analysis techniques. We will emphasize fault trace and geomorphic mapping applications, integration with other geospatial data, and data visualization and analysis approaches.  The course will be held at KeckCAVES at UC Davis and will combine lectures and hands-on use of several different software packages.

The course is supported by the Southern California Earthquake Center, UC Davis KeckCAVES, and OpenTopography.

More information on the course and a link to the course application are available via the short course page.  The application deadline is September 21st, 2011.

---

Tahoe Standard DEM Data

Posted on Wed, June 08, 2011 by Chris Crosby in Data • OpenTopography Updates

The final piece of the Lake Tahoe Lidar dataset - standard digital elevation model (DEM) and intensity rasters - are now available for download from the OpenTopography standard DEM page.  These products, produced by Watershed Sciences, the vendor who performed the Tahoe data collection, consist of three separate data layers all at 0.5 meter resolution in the ERDAS Imagine (.IMG) format:

1. Highest hit DEM
2. Hydro enforced bare earth DEM
3. 8-bit intensity raster

We’ve packaged the data based on the USGS quarter quadrangle (3.75 minute) naming conventions used by Watershed Sciences (Tile index file in shapefile format).  Thus, each quarter quad .zip file contains the three grid data products noted above.  For example:

39120A22.zip corresponds to USGS quarter quad 39120A22 and contains the following three grid products stored in sub-directories:
- hh_39120A22 (highest hit)
- he_39120A22 (hydro enforced bare earth)
- int_39120A22 (intensity grids)

There are 55 quarter quads worth of data in the Lake Tahoe dataset, for a total of ~17 GB of data (zipped).  The OpenTopography standard DEM download interface uses a Java applet to automate the download.  These are large data files so patience and bandwidth are required. 

Examples images derived from the products contained in 39120A22.zip (the quarter quad that corresponds to the Homewood Mountain Ski Area):

Hillshade of highest hit DEM

Hillshade of hydro-enforced bare earth DEM

Intensity raster

---

Tahoe Lidar Imagery in Google Earth

Posted on Tue, May 03, 2011 by Chris Crosby in Data • Google Earth • OpenTopography Updates

As we announced last week, lidar data for the whole Lake Tahoe basin are now available via OpenTopography.  Over the past week this dataset has seen a quite a bit of traffic, with over 160 jobs run (10+ billion points processed) by more than 50 unique users.  However, our experience indicates that a large number of people just want to look at these data, and processing point cloud data to DEMs is clearly not the most efficient way to go about this.  So, as I’ve done in the past for many of the larger datasets OpenTopography hosts, I ran the whole Tahoe Basin dataset through a routine to generate lidar derived imagery (hillshades and “slopeshade” images) that can be viewed in Google Earth.  The resulting KMZ file can now be downloaded via our Lidar Derived Imagery in Google Earth page. 

The file provides access to four layers of imagery, all at half meter pixel resolution:  45 and 315 degree sun angle hillshades of the hydro enforced bare earth grid, a slopeshade of the hydro bare earth grid, and a 315 degree sun angle hillshade of the highest hit (vegetation, buildings etc) surface.  See images below for an illustration of the four imagery layers.

Once you download and open the KMZ file in Google Earth, the imagery is streamed from OpenTopography servers at San Diego Supercomputer Center to the Google Earth client for viewing.  This is a large dataset (~14 GB of imagery) so initial display of the imagery can be sluggish, especially if your internet connection is not great.  As you browse the data Google Earth fills its cache, and browsing speeds should pick up.

This set of imagery took somewhere in the neighborhood of 96 hours of time on my workstation to generate.  But it is an excellent method for reducing an otherwise massive dataset down to something that is relatively easy for anyone with a computer, a network connection and Google Earth to access:

• Original Tahoe lidar point cloud dataset = 325 GB of LAS data (12.4 billion points)
• Tahoe Basin half meter DEMs generated from point cloud data (bare earth + highest hit) = 64 GB
• Four layers of imagery (pngs) tiled and pyramided for display in Google Earth = 14 GB

So, through this approach a 325 GB dataset is reduced to something small enough to be delivered dynamically across the network to users with a widely available, free, and familiar and intuitive client.  It is important to note that the Google Earth imagery layers are not meant to be a substitute for going back to the actual elevation data to perform your scientific analysis; but for initial synoptic browsing, site selection, and education and outreach applications it is hard to beat the Google Earth approach.

Check out the images below, download the Tahoe Lidar Imagery KMZ file, and enjoy these amazing data.

For more information about the Lake Tahoe Basin Lidar Dataset, please see the initial OpenTopography news item HERE

---

OpenTopography Overview Video

Posted on Mon, June 21, 2010 by Chris Crosby in OpenTopography Updates • Video

The OpenTopography ASU Capstone team - a group of senior undergraduate (who have now graduated - congrats!) School of Computing and Informatics students at Arizona State supervised by OpenTopography Co-I Ramon Arrowsmith - have released another nice video about OpenTopography.  This video provides an introduction to the OpenTopography Facility:

---

New Point Cloud Processing and Custom DEM Video Tutorial

Posted on Mon, May 03, 2010 by Chris Crosby in Education • OpenTopography Updates

Thanks to the ASU Capstone team - a group of senior undergraduate School of Computing and Informatics students at Arizona State supervised by OpenTopography Co-I Ramon Arrowsmith - we now have a very nice video tutorial on how to use OpenTopography to download and process LiDAR point cloud data to digital elevation models:

This is the first of what we hope will be a series of instructional videos that will appear on the OpenTopography Tutorials page and the OpenTopography YouTube channel.  Stay tuned for more soon.

---

Cyber-GIS Opportunities for High-Resolution Topography Data Access, Processing, and Analysis

Posted on Mon, February 22, 2010 by Chris Crosby in Meetings • OpenTopography Updates

Earlier this month, I had the privilege of participating in the National Science Foundation TeraGrid Workshop on Cyber-GIS in Washington, DC.  The workshop was sponsored by the National Science Foundation (NSF) TeraGrid Science Gateway program and the Office of Cyberinfrastructure with the goal of “underpin fundamental issues of Cyber-GIS for enhancing cyberinfrastructure while advancing the next-generation GIS with synergistic high-performance, distributed, and collaborative capabilities.”

Each participant in the workshop was required to submit a position paper that highlighted an issue or opportunity in Cyber-GIS.  My paper, “Cyber-GIS Opportunities for High-Resolution Topography Data Access, Processing, and Analysis”, highlights activities OpenTopography is currently engaged in, and also points to opportunities and challenges we are pursuing.  You can download a PDF of the position paper, or read it below.

Cyber-GIS Opportunities for High-Resolution Topography Data Access, Processing, and Analysis

Christopher Crosby
San Diego Supercomputer Center, University of California, San Diego, CA

High-resolution topography data acquired with lidar (light detection and ranging) technology are revolutionizing the way we study the geomorphic, biologic and anthropogenic processes acting along the Earth’s surface (e.g. Carter et al., 2007).  These data, acquired from either an airborne platform or a tripod-mounted scanner, are emerging as a fundamental tool for research on a variety of topics ranging from earthquake hazards to urban modeling.  Lidar topography data are powerful because they represent processes and features at a scale not previously possible yet essential for their appropriate representation.  These data sets also have significant implications for earth science education and outreach because when visualized, they provide an accurate digital representation of landforms, natural hazards and processes, and the built environment. 

However, along with the potential of lidar topography comes an increase in the volume and complexity of data that must be efficiently managed, archived, distributed, processed and integrated in order for them to be of use to the community.  A single lidar data acquisition may generate terabytes of data in the form of point clouds, digital elevation models (DEMs), and derivative products.  This massive volume of data is often difficult to manage and poses significant distribution challenges when trying to allow access to the data for a large scientific user community. Furthermore, the data sets can be technically challenging to work with and may require specific software and computing resources that are not readily available to many users.

Projects such as the National Science Foundation-funded OpenTopography Facility (http://www.opentopography.org) (e.g. Crosby et al., 2009) are successfully leveraging emerging cyberinfrastructure technologies such as portal-based data access, service oriented architectures, high-performance parallel database systems (Nandigam et al., 2010), and optimized processing algorithms to improve internet-based access to these massive geospatial data sets.  The OpenTopography system provides free and on-demand access to tens of billions of lidar point cloud measurements as well as processing tools that permit users to generate custom digital elevation models on-the-fly.  OpenTopography’s growing user community of several thousand scientists, educators, students, government agency staff, and private sector users illustrate that cyberinfrastructure-based geospatial data access systems can have a significant impact by democratizing access to these massive data sets.

OpenTopography’s success is an illustration of the potential opportunities that exist through the application of cyberinfrastructure resources to geospatial data management and processing. However, the OpenTopography effort has only just scratched the surface of how routine data management and processing tasks could be enhanced with access to cloud or grid-based resources.  As any regular user of high-resolution topography appreciates, many of the existing geographic information system (GIS) algorithms currently available for processing, analysis, and visualization point cloud and DEM data fail, or perform very slowly, when applied to lidar data.  Taking a Cyber-GIS approach to lidar topography processing and analysis would allow users to carry out computationally intensive LiDAR data processing without having appropriate hardware locally.  Resources such as Hadoop (http://hadoop.apache.org/)-based processing in the cloud, the TeraGrid (http://www.teragrid.org/), or Condor pools (http://www.cs.wisc.edu/condor/) could allow users to “outsource” their geospatial data processing to computing resources better equipped to handling significant data volumes.

However, to effectively utilize high-performance grid or cloud resources will require that the user community develop a new “toolkit” of algorithms and tools that are optimized to perform in these environments.  This new toolkit should exist in the open source domain and consist of libraries that allow users to construct customized processing workflows that run in a distributed environment. Examples of necessary algorithms include those for high-performance gridding of lidar point cloud data (e.g. Kim et al., 2006), algorithms for hydrologic processing of DEMs (e.g. Wallis et al., 2009) including calculations of slope, slope-aspect, stream profiles, catchment areas, and topographic roughness and curvature, geomorphic change detection analysis, feature extraction (including vegetation classification and structural analysis, and building footprint extraction), as well as tools for the processing and analysis of full waveform lidar data.

REFERENCES:
Carter, W. E., R. Shrestha and K.C. Slatton, 2007, Geodetic Laser Scanning, Physics Today, Vol. 60, Number 12, pp 41-47.

Crosby, C.J., Nandigam, V., Arrowsmith, R., Baru, C., 2009, Enhancing Access to High-Resolution Lidar Topography – From Point Clouds To Google Earth, Geological Society of America Abstracts with Programs, Vol. 41, No. 7, p. 384

Kim, H., Arrowsmith, J R., Crosby, C.J., Jaeger-Frank, E., Nandigam, V., Memon, A., Conner, J., Badden, S.B., Baru, C., An Efficient Implementation of a Local Binning Algorithm for Digital Elevation Model Generation of LiDAR/ALSM Dataset, Eos Trans. AGU, 87(52), Fall Meet. Suppl., Abstract G53C-0921, 2006.

Nandigam, V., Baru, C., Crosby, C.J., Database Design for High-Resolution LIDAR Topography Data in preparation, 2010 International Conference on Scientific and Statistical Database Management

Wallis, C., Watson, D., Tarboton, D., Wallace, R., 2009, Parallel Flow-Direction and Contributing Area Calculation for Hydrology Analysis in Digital Elevation Models, Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications, PDPTA 2009, Las Vegas, Nevada, USA

---

Pardon Our Dust

Posted on Fri, November 20, 2009 by Chris Crosby in OpenTopography Updates

OpenTopography is taking advantage of the expected lower than normal visitor traffic over the upcoming holiday week to make some site updates.  We will be rolling out a new logo, site skin, and modified navigation among other things.  We apologize in advance if you visit over the next week or so and things seem a little off - rest assured that the site should be back to normal by the first week in December.  Here’s a sneak peek at what the site will look like when we are done:

Also in the coming weeks, OpenTopography plans to release quite a bit more data and to roll out some new features.  Stay tuned…

---

An Update on Southern California LiDAR Coverage of Active Faults

Posted on Tue, September 15, 2009 by Chris Crosby in Data • OpenTopography Updates

OpenTopography has released yet another round of GeoEarthScope LiDAR data for active faults in southern California.  The most recent release includes the Elsinore fault, the Burro Flats segment of San Andreas fault, and the Crater Mountain portion of the Owens Valley, adding to the existing SoCal fault coverage that includes the Garlock, the San Andreas and San Jacinto and a number of faults in the Eastern California Shear Zone in the Mojave.  At the Southern California Earthquake Center Annual Meeting in Palm Springs this week, I presented a poster that provides an update on the status of LiDAR coverage for active faults in southern California available via OpenTopography.  The image below comes from my poster and nicely summarizes the southern California LiDAR data currently available via OpenTopography as well as what will be available in the near future.

As you can see, if the area is shown in yellow, those data were collected by GeoEarthScope and are currently available via OpenTopography.  Likewise, areas outlined in orange are also available via OpenTopography, but these data were collected by campaigns other than GeoEarthScope (e.g. the B4 and the ECSZ projects). Finally, areas shown in green are GeoEarthScope data that have not been delivered to OpenTopography for distribution to the community but that we expect to arrive in the next few months.  As always, you can use the data overview page in OpenTopography to see what data is available in the system.

---

OpenTopography Hardware

Posted on Fri, March 06, 2009 by Chris Crosby in OpenTopography Updates

Today the GEON team was in the SDSC machine room moving GEON and OpenTopography machines to new racks as part of a reorganization of the machine room.  Since I don’t visit the machine room very often, I took some photos of the hardware that makes OpenTopo work.  This photo shows the rack that holds the OpenTopo Portal (Gridsphere) server as well as the LIDAR database cluster (8 front-end nodes plus a four node disk array (20 TB total).  In a different rack is the 3 node compute cluster for LIDAR data processing (DEM generation) and the visualization machine that generates the browse images (jpgs) and Google Earth KMZ files for each job.  Other hardware involved in making OpenTopography work is the SDSC “webfarm” server that runs the OpenTopography.org website and the tile DEM server that is currently running at ASU.

---

Page 1 of 1 pages

RSS