Integrating with business systems – where to start

Esri’s goal with the WebGIS (see this blog on “What is Web GIS”)concept is to break down silos and make spatial data and processing services accessible to anyone that wishes to use them. This concept falls right into the hands of many organisations, who wish to integrate their systems, allowing processes and decision making to be based on authoritative and accurate data. Should an organisation not reap as much benefit out of a legacy system as possible, the risk of losing the backing of management could inevitably lead to budget cuts or even closure of departments. It is therefore of integral importance that any system provides an organisation with a business case and functionality that is critical to the day-to-day running of the organisation.

For this purpose, one of the core functionalities that all systems must provide their clients is the capability of integrating with other systems. The value of accessing data and services from other systems, through a single platform, is immense. The intention being that processes are streamlined; product quality and performance are improved and ultimately, decision making is based on a holistic view off all relevant authoritative dataset from the key stakeholders in your organisation.

Integration?

But what does it mean to integrate and how does one go about integrating one system with another?

Quite simply put, integration means, accessing information and\or services of one (or many) system(s), on a different platform. We want to do this for 3 main reasons:

  1. Geocentric Application: As the “geocentric” term alludes to, this is a GIS orientated pattern, where the organisation’s business data is accessed via the geospatial platform, like the municipal cadastre, that is updated daily by diligent GIS staff at your local municipal offices. The content consists predominantly of spatial data and geoprocessing tools to enrich the business data and is utilised by staff that have a background in GIS.
  2. Geo-Enable Application: This pattern leans heavily towards the business’ system hosting the data, while the GIS platform’s role is to provide functionality and capabilities to the business system, that would not otherwise be offered. Be it address geocoding of data, map visualisation window for location verification during a process or field mobility capabilities, the geo-enablement pattern enriches the business system with useful capabilities that fall outside of the its core strengths.
  3. Composite Applications: Should an organisation have multiple systems and none of them are considered as the central hosting framework, then the composite application is considered as the business pattern to pursue. In this case, web services and capabilities from the various systems are integrated to provide superior functionality. 
    Capture
    Integration patterns for business systems

    What about web services?

    Having got the complicated concepts out of the way, we can go ahead and look at how to go about integrating with the ArcGIS platform.

    First and foremost, you don’t have to be a developer to integrate your ArcGIS Enterprise with another system. Esri has made the integration process very easy by allowing users to add web service URL as items, to their ArcGIS . As in “https://service1.arcgis.com/<item’s id>/arcgis/rest/services/<feature name>/FeatureServer”. Be it ArcGIS Desktop, ArcGIS Online, ArcGIS Enterprise or a wide variety of ArcGIS apps, adding web services is a simple procedure.

mini-frame
Integration Example – ArcGIS Online Add Item

Should your organisation wish to integrate their system with certain features of your GIS platform, you can simply provide them with the desired feature service’s URL to the system’s administrator. They will then gain access to the data or GIS capabilities you have provided them with, for their own use.

Complexity comes in the form of the data’s format. In some cases, it is necessary to make alterations to the format of the incoming service, in order for a system to consume the incoming service correctly. In cases like this, some additional configuration will be required.

uncompatibe
Integration fails due to compatibility issues

In this regard, Esri has you covered, with the Data Interoperability for ArcGIS extension.

It’s interoperable, silly!

This extension allows users to build ETL (Extract, Transform and Load) tools, that can convert almost any datatype into a format that the destination system can consume. The interoperability extension can be a once-off ETL task that is run from the ArcGIS desktop software, or an automated ETL task (check out this blog, which covers ETL automation) can be set to run, which will translate the incoming service stream into a compatible format, in real-time.

ETL
ETL tool converting data from its source, into compatible format

Even though integration is a daunting process, that can easily become complex and messy process, Esri provides simple tools and literature to aid any GIS professional to start the process and do the job right the first time.

So, if have not integrated your GIS implementation with other business systems, then now is the time to get going!

 

P.S.: Here are some helpful resources to start off with.

–              Architecting the ArcGIS Platform: Best Practices

–              wiki.gis.com: Data integration

–              All you need to know for ArcGIS web services

–             An informative blog post on the Data Interoperability for ArcGIS extension

OSM – OpenStreetMap

OSM – a valuable source of free geographic information

OSM (OpenStreetMap) is free data that is compiled by a community of mappers from all over the world.  The mappers contribute and maintain road data, trails, points of interest and much more.  OSM emphasises local knowledge and contributors use GPS devices, aerial imagery and field maps to verify the data and ensure that it is up to date.

OSM is open data and free for you to use for any purpose as long as you credit OSM and the contributors.  If you change or build on the data in certain ways you may also distribute the result ONLY under the same license.  For more information on the latter refer to the Copyright and License page.

Esri South Africa has processed the OSM data to a file geodatabase for all clients that are current on maintenance, free of charge.  Additionally a Community Basemap for South Africa was created using the Esri Inc. template. This is also available free of charge as part of the Portal for ArcGIS offering to clients that are current on maintenance.  Clients who are interested in the OSM Community Basemap can contact Esri South Africa.  The screen print below shows a clip of the OSM Community Basemap for South Africa.OSM

 

Finding and downloading OSM data for your country or area can be tricky.  If your maintenance is not up to date and you wish to download the raw OSM data, the steps below will help you achieve this.  The OSM data is available can be downloaded from the OSM website https://www.openstreetmap.org/#map=5/-17.099/51.021 by clicking on the Export Button and selecting one of the options listed.

export

 

 

Click on the Planet OSM link which will take you to a webpage with some more options.

planetosm

To download the OSM data in shapefile format click on the BBBike.org link.  This takes you to http://download.bbbike.org/osm/.  Click on “Select your own region” as shown below.

BBBIke

From this page you are able to select and download the data.  Specify in which format you want to download the data, select Shapefile (Esri) from the dropdown menu.  Enter your email address and complete the Province, Town, or Country name of the area that you want to download and click on the “search” option.

Format

After clicking on the “or search” option you can select the area form the list provided or you can zoom in to the area that you want to download and click “here” to create a bounding box.  To add extra points to the bounding box polygon click on the add points icon.  You can also resize and reposition the bounding box.

Bounding Box

As an example I have selected South Africa and included extra points on the bounding box to make a better selection.

Once you are satisfied with the area that you selected you can click on the “extract” button.

extract

A pop up message will confirm that the area that you selected is acceptable and will contain information on the size of area, coordinates and format.

Popup

You will receive a confirmation email via the email address that you supplied.  Click on the download link to download the zip file.  The zip file will contain a “shape” folder containing the shapefiles.  You can now convert the shapefiles to feature classes for optimal storage in ArcGIS.

If you require more information you can visit the OpenStreetmaps website https://www.openstreetmap.org/#map=5/-17.099/51.021 or OpenStreetMap Facebook page.  https://www.facebook.com/OpenStreetMap?fref=ts