GIS and big data advanced analytics using Hadoop

GIS and big data advanced analytics using Hadoop

What is Big Data?
Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. But it’s not the amount of data that’s important. It’s what organizations do with the data that matters. Big data can be analyzed for insights that lead to better decisions and strategic business moves.
More info.





 Esri Has Wrote an Article Here showing ways to 

Building a Big Spatial Data Application

ArcGIS and Big Data

ArcGIS helps you filter and organize big data so it can be analyzed and visualized from a geographic perspective. It brings together not only massive volumes of data but also data of disparate types.

Building a Big Spatial Data Application

GIS Tools for Hadoop is an open-source toolkit intended for big spatial data analytics. The toolkit allows users to leverage the Hadoop Framework to do spatial analysis on spatial data.
read More

Empowering GIS with Big Data

Big Data first made mainstream headlines in 2011. When traditional intelligence had failed to trace Osama bin Laden, it was Big Data analysis that pulled disparate spatio-temporal data in real-time to pinpoint his location. This information was used in conjunction with satellite imagery and next generation drone data to support intelligence operations.

The success of geospatial application in any project depends upon the selection, collection, sorting and end-usage of data. As the amount of remote sensing and other geographic data keeps getting bigger every day, the traditional GIS systems are often insufficient for meaningful interpretation.

KALEV LEETARU’S BIN LADEN MAPPING – WAR 2.0

Mapping and analysis become further complicated with the explosion of disruptive technologies like the cloud, embedded sensors, mobile and social media. Pioneers like the Esri have thus extended the capabilities of traditional GIS across such technologies, to emerge in a more powerful format. Esri ArcGIS extensions produce more granular insights of trends, patterns and relationships in a spatial context.

ADVANCES IN REMOTE SENSING DATA PRODUCTS (1970S TO PRESENT)

GIS and Big Data – Two Parts of a Whole

A map displaying the shopping patterns and demographic profile in a neighborhood is useful for analyzing the customer base and driving marketing strategies. However, with Big Data technology, a 360 degree view of the customer with a spatial component can be obtained. Predictive models can be built with data from social media conversations, mobile phones and browsing behavior. Customer demand is predicted and integrated with supply chain management for a smooth delivery mechanism along shortest path. This drives better customer relationship and efficient use of resources.

As Big Data technology is able to process large quantities of data in the shortest curve, it has emerged as a new frontier for geospatial analysis. For instance, NASA has made inroads into climate research integrating Big Data (past and current global climate data) with geographies. This has enabled a synergized approach to climate modeling, analysis and resilience building.

STRUCTURED AND UNSTRUCTURED DATA AS EQUAL PARTNERS IN SPATIAL ANALYSIS.

A map story of disease spread helps analyze patterns and relationships. However, can you imagine how powerful the same would be with analysis of vast data streamed from social media, multiple devices, international statistics, spatial logs and more – to predict the precise location of the next outbreak and prepare for the same?

How GIS Works with Big Data

GIS tools search, sift and sieve data from multiple and disparate databases to organize it for better workflows and spatial analysis. They run operations that aggregate terabytes and more of spatial information, run analysis, and visualize results as maps. All this occurs in real-time, with multiple data streaming into the existing GIS for better understanding of spatial trends and relationships

How Big Data Helps Maximize Spatial Analysis for Predictive Modeling  

Big Data technology helps to enhance operational efficiencies and add value for business edge, sustainable practices and policy making.

• Taps huge datasets for policy measures – GIS tools for Big Data processing facilitate deep insights and predictive modeling for policy making in health care, crime detection, disaster response and more.
• Supports spatial analysis of unstructured data in real-time – Maps integrate unstructured data (e-mails, blogs, social media content, in-store sensor data, meteorological data, driving times, etc.) in real-time. This is useful for location analysis in retail, finance and insurance.
• Integrates multiple data layers for a complete picture – Huge amount of data is pulled from different formats, devices or systems and given a geographic context for a complete picture or analysis.
• Empowers Business Intelligence (BI) approach to businesses –   The convergence of Big Data and mapping leads to deeper insights, profitability, lesser time-to-market, improved customer engagement and better ROI.
• Enables spatiotemporal queries over big geospatial data – Case-by-case query processing and data mining of huge spatiotemporal data is possible for different projects.

Application Areas

The Big Data approach to GIS allows analysis and decision making from huge datasets, by using algorithms, query processing and spatiotemporal data mining. In simple words, this means extracting information from maximum possible sources using established procedures and computational techniques.

Areas where geospatial technology has applied Big Data for enhanced analysis:

• Climate modeling and analysis
• Location analytics
• Retail and E-commerce
• Intelligence gathering
• Terrorist financing
• Aviation industry
• Disease surveillance
• Disaster response
• Political campaigns and elections
• Banking
• Insurance and Fraud analysis

Products, Extensions and Mashups

Esri offers an open source spatial framework for Big Spatial Data analysis in Hadoop, where both developers and existing ArcGIS users can take advantage of the various tools. Additionally, ESRI offers solutions that leverage the power of Big Data platforms like Microsoft, SAP and IBM, for enterprise GIS. The ArcGIS GeoEvent Extension for Server  also incorporates information streams within the existing GIS data and IT infrastructure, in real-time.

MapLarge is a fully featured mapping platform for Big Data visualization, analytics and simultaneous publishing. MapCite administers location attributes within Big Data, through advanced predictive and decision modelng and real-time analytics.

The GeoXray from TerraGo Technologies uses location intelligence to “filter news feeds, blog posts, social media, maps and imagery and more by place, time and topic.”

Case studies

Various application scenarios facilitate business opportunities and research studies for frontline decisions:

Tracking Ebola with Big Data: GIS and IT firms are using various Big Data mashup technologies to help combat the Ebola outbreak (related: Mapping the Outbreak of Diseases)

THE IBM EBOLA TRACKER.

IBM’s Ebola heat map uses Big Data technology in combination with a GPS app to track and fight Ebola.

AN INTERACTIVE STORY MAP FROM ESRI.

Esri is applying technological know-how to map, monitor and predict the Ebola outbreaks from 1976 till date. The HealthMap tool used open source applications, news aggregators and curated content from Government and social media to identify the Ebola outbreak even before it was officially declared by the World Health Organization (WHO).

Using Big Data in GIS for observing earth’s climate – The NASA Langley Research Center is using Big Data in GIS for a better understanding, planning and response to climate change.

Government becomes more proactive and responsive through Big Data analysis – The city of Glendale used Big Data and ArcGIS web mapping functionality to assess and manage street signs across the city.

References and Resources

Bedard Y., 2014, Beyond GIS: Spatial On-Line Analytical Processing and Big Data, Univ. of Maine [Online]

Chen Y, Suel T., Markowetz A., 2006, Efficient Query Processing in Geographic Web Search Engines [Online]

Video – Big Data in ArcGIS

Infographics – Becoming competitive with Big Data

Article – Mapping the Future with Big Data

Esri Resource – Big Data in GIS

Empowering GIS with Big Data

PDF Book academia.edu

0 00000000000

-- ADD YOUR CONTENT HERE --

Share this post