1. Gigapixel resolution imaging fornear-remote sensing andphenomics7.3 billion pixel image of the National Arboretum, Canberra, ACT, AustraliaAssembled from 900, 18MP images. (http://gigapan.com/gigapans/120215)Magpies (1.3km) ANU resaerch forest (500m)Dr. Tim Brown, Borevitz Lab, Australia National University

2. ParadigmsHow do we do science? Use limited data to build models of how the world works The Old: data limited Sit under a tree and record what you see Sample what you can afford to for a limited period oftime Usually temp/hg being recorded; if youre lucky theressatellite data Very low res version of reality The future: software limited (and paradigm limited) Long time series High spatial and temporal resolution Flux towers, mesh networks, gigapixel cameras, UAVs,3D models of ecosystems 3. We are building the tools to enableNextGen ecology Total ecosystem awareness What hardware do we need to do this? What novel data streams exist that we canuse? The hardware questions will be solved, but 4. Imagine if we could watch every plant in everyresearch sites from our desks? Map out population genetics, biotic/abiotic data on thelandscape Light sequence every plant in the landscape (~$10/plant) Slide back in time and watch any interaction for as long asthere have been sensors Students start new research projects beginning with all thedata previously collected at a site View time-series data in situ, on site with a tablet Tons of opportunities hereThe technology is (almost) hereWe need to dream big! 5. A lot is going at every scale from the individual plant to awhole ecosystem. Depending on what questions you want to answer someof this detail is may really matter See the timelapse here:www.youtube.com/watch?v=ymYQCrmDN8Y&hd=1 6. What technologies enable this new wayof monitoring ecosystems? Mesh networks Internet Need standards Gigapixel imaging UAVs Low cost land/UAV-based LIDAR Smartphone science Automated processing & QC/QANeed better software at all levels 7. Collaboration betweenBorevitz Lab (U. Chicago, now ANU) and TimeScience (my company)The Challenge: Build a solar powered, weatherproof gigapixel camera that can recorddaily phenology from every plant in a field area.Gigavision: gigapixel timelapse cameraFrom Gigapan to gigapixel timelapse 8. (Single 15MP image)Area: ~7haArea: ~1m2The Gigapan andGigavision systemsallow you to capturehundreds orthousands ofzoomed-in images ina panorama.Images are thenStitched into aseamless panorama.The super-highresolution of the finalpanorama lets youmonitor hugelandscape areas ingreat detail.Gigapixel Imaging How it works 9. ~1.5 billion pixels / panorama (could be more) Avg. resolution of ~1 pixel / cm over 7 hectares (~600 million times the pixel resolution of MODIS) Open-source - Built with off-the-shelf components Gigapixel imaging isnt that hard. Really the hard part is Software (again) Cellular (3G) or 802.11g wireless access (160MP thumbnails) Automated capture up to 1 image / hr Solar powered (500 plants for ~$60/plantGigavision Camera SpecificationsFor full specs, see Brown et al. 2012(Google: gigavision chapter) 10. Gigavision Camera Location:Big Blowout EastIndiana Dunes State Park, IN, USA 11. Camera Field of View (FOV) 12. Camera Field of View (FOV)Actual camera view 13. Dataset statistics Time period recorded: Oct 2009 Oct 2011 2 N. Hemisphere growing seasons (April Oct) 1 - 4 panoramas / day (~154 15MP images/panorama) Initial image dataset: ~184,000 individual jpg images Processed data = ~70 million 200x200px image tiles 6TB of space 417 usable noon panoramas 14. Gigavision camera coverage, 2009 - 2011Growing seasonsColors denote a successful panorama captured at the indicated time of day.Note missing data at key points in the spring (N. Hemisphere spring).When each season is a data point it is important to have support staffavailable at point when the hardware must not fail. 15. Image Visualization and Data CollectionBrowse the timelapse online: http://bit.ly/GVDemo2012-1Data Collection demo: http://bit.ly/GVDemoMovie2012 16. Data Collection demo movie:http://bit.ly/GVDemoMovie2012 17. 513 individual plants identified 8 prominent species (non grasses) Species: Hoary Puccoon = 344 Unidentified (yet) = 52 Cottonwood = 47 Black Oak = 36 Sand Cherry = 18 Juniper = 9 Wormwood = 3 Pitchers Thistle (Endangered) = 2 Marsh Marigold = 2Indiana Dunes Gigavision Camera Data Summary 18. Gigapan Low Cost Gigapixel ImagingNon-timelapse Gigapixel robotic camera heads byGigaPan) cost $300-$900 and work with any camera Great for: Repeat photography and monitoring Site Documentation and detection Visualization; grant proposals; impressingfunders! More examples of gigapans here:http://gigapan.com/profiles/TimeScience Camera hardware: http://www.gigapan.com/ 19. Alta Ski Area Bark Beetle ProjectCollaboration w/ Maura Olivos at the Alta Environmental CenterGoal: Improve early detection of bark beetle outbreaks within AltaSki Areas (Alta, UT, USA) management areaSolution Augment traditional aerial survey data with annual gigapixelphoto surveys to enhance detection success. Gigapixel imaging provides a new survey tool that allowsAlta staff to examine the health of almost every tree in a810ha area via a web interface. 20. Path data collected with EveryTrail smartphone app (http://www.everytrail.com/ )Initial survey path for potential panorama locationsView the panoramas from the initial survey: http://gigapan.org/galleries/6787/gigapans 21. GreeleyApproximate coverage of the four Gigapan survey images.[Note that an additional image taken from the Greeley area (green arrow) would cover most of the missing areaon the east side of Alta. This area is currently not at high risk for beetle outbreaks and so was not surveyed.] 22. (1) Collins Weather(2) Baldy Shoulder(3) Road Shot(4) GrizzlyView all survey panoramas online here: http://gigapan.org/galleries/5582/gigapans 23. (1) Collins Weather(2) Baldy Shoulder(3) Road Shot(4) GrizzlyExample of beetle-killed tree detected in image. 24. Project summary statisticsSite Name Approximate Area(Hectares)Image Resolution(Gigapixels)Average Pixel Resolution(Pixels per square inch)Collins Weather Station 282 4.07 0.93Baldy Shoulder 312 8.84 1.83Road Shot 85 3.26 2.46Grizzly Gulch 76 3.65 3.1TOTALS 810 ha Total: 23.5 billion pxAvg: 4.7 billion pxAvg: ~2 px/in2(Area Estimates: http://www.earthpoint.us/Shapes.aspx)Online Data collection portalhttp://bit.ly/Rvn4p3 25. Australian time-series monitoring Canberra, Telstra Tower National ArboretumGigapixel imaging lets us monitor seasonal change for huge areas 26. 20 gigapixel image of Canberra, Australia from the Black Mountain Telstra TowerZoom in to the National Arboretum 27. MidsummerZoom in to the Each forest at the Arboretum 28. Low cost sequencing lets us genotype every individual tree and identify genetic loci thatcorrelate with observed phenotypic differences between trees.We can do this for all trees at the arboretum within view of the camera.Fall Color change shows differing rates of fall senescence in treesLate fall 29. Gigavision cameras with mesh sensor networksNatl Arboretum & ANU Coastal Campus (2013 LIEF proposal) 30. UAVs Increasingly cheap and easy to use DIY version is ~$600 with full GPS andautopilot Commercial solutions are $10K - $100K Commercial software (e.g. Pix4D) $10-15K 100mm resolution DEMs + image layers Some pay per use and cloud based solutionsare emerging Regulatory framework is a challenge 31. Test aerial survey using a 40MPNokia cell phone and a low costquadcopter. Images stitched in Autopano Gigasoftware yield a 1 billion pixel image. Autopilot enables easy repeatsurveys. See full panorama here: http://gigapan.com/gigapans/127099 32. SmartphonesImage: 40 consumer gadgets have converged into one device.: http://www.wired.com/magazine/2013/04/convergence/ 33. We shouldnt underestimate the utility ofsmartphones There are currently more then 1 billionsmartphones in use globally Phone tech: 4G networks have the same bandwidth as MODIS More processing power than most satellites Can sense natively: Position (w/in 3-4m), Temp, Light, magenetic field, orientation, proximity,sound levels Have internet; OS, can ftp images, data, etc Can take up 40mp images (Nokia 808) 34. Citizen Science and Crowdsourced ScienceThere are many sources of data coming onlinethat can provide useful time-series informationabout the world.Example: Geo-tagged photos Facebook: 300 million images uploaded / day w/240 billion total This is up from 83 million 2 years ago Most (?) are from mobile and geolocated US National Parks 280 million visitors / yr How many of these people visit the same spot, take the samegeolocated picture with their smartphone and then put it online? How do we begin to access and use data sets like this? 35. Map of pixels of how much of the worldhas been captured by geo-referencedimage on flickr (not including google streetview of course). Eric Fischer (http://www.flickr.com/photos/walkingsf/ ) 36. Geotagged Chicago Eric Fischer (http://www.flickr.com/photos/walkingsf/ ) 37. PhotosynthMicrosoft and U. Washington PhotoTourism and Photosynth projects http://photosynth.net/ (free online tools for object mapping with images) http://grail.cs.washington.edu/projects/rome/ http://phototour.cs.washington.edu/findingpaths/Photosynth Project Building 3D maps of the world from online images 38. For more informationPrimary project contact (all projects)Tim Brown ANU, Time-Science (tim@time-science.com)Ph: +1 801-554-9296Skype: TimeScienceAdditional project participantsCamera Systems and Data VisualizationChristopher Zimmermannwww.time-science.comwww.time-science.comGigavision Camera ProjectJustin BorevitzNina Noah, Whitney PannetonUniversity of ChicagoData: http://gigavision.orgPurchase: http://gigavision.netAlta Bark Beetle ProjectTim BrownMaura OlivosAlta Ski Area / Alta Environmental Centerhttp://www.altaence.com/