Esri has fully embraced Python for ArcGIS and sees Python as the language that fulfills the needs of the user community. Previous versions of ArcGIS Desktop (ArcMap and ArcGIS Pro up to version 1.2) used Python 2. Since the ArcGIS Pro 1.3 release, ArcGIS Desktop has made use of Python 3, bringing with it some changes to the tools.
ArcPy is a Python site package that, when used with Python, provides a useful and productive way to perform geographic data analysis, data conversion, data management, and map automation. By default ArcPy is installed as part of ArcGIS Desktop.
The ArcGIS API is another Python library focused on Web GIS. It provides powerful tools that can be used for vector and raster analysis, geocoding, map making as well as managing an ArcGIS Enterprise system.
In a lot of cases, one might need to use both of these packages in the same script in order to automate a specific workflow.
The online help documentation has a lot of tips, tricks and other bits of information to help you get started, including methods of installing the API. The easiest way to install the ArcGIS API is through the ArcGIS Pro Python Package Manager, however, this needs internet connectivity in order to download the package from the internet. So, what happens when I am in a disconnected environment?
According to the online help, the API can still be installed to the Anaconda instance of Python if you have downloaded the package beforehand. The challenge here is that now one can use the ArcGIS API, but only through the Anaconda Python installation. However, the Anaconda version of Python does not have the ArcPy package installed.
In the same way, I can use the Python instance installed with ArcGIS Pro in order to utilise the ArcPy capabilities, but I will be unable to access the ArcGIS API functionality.
A quick workaround
The default install location for ArcGIS Pro Python is installed at:
Placing an address on a map either to find or place or to provide business context is becoming vitally important in our society. Location matters.
Most commonly address information that is stored in a database is not something that is regularly maintained. Often information is captured in free text fields which results in data irregularities and inconsistencies.
The purpose of this article is to provide some insight into how to better manage an address dataset which would potentially be batch geocoded and how to optimise the capturing of these address datasets for geocoding in ArcGIS Online.
There are a number of variables at play which can affect the final outcome of a geocoding exercise (the most pivotal being the quality and accuracy of the reference data you are matching against) and it is never as simple as receiving an address dataset and geocoding it, often times clients want quantifiable measures of accuracy for the geocoded dataset and the GIS personnel working on the project are often expected to clean and normalise addresses in order to improve match rates.
Here are a few helpful tips which will help ensure accurate geocodes when using the World geocoder in ArcGIS Online.
Use single-line addresses
Geocoding single-line addresses is both faster and often more accurate than feeding the address records to the geocoder field by field. This is for a number of reasons, the most obvious being that often the incorrect information is captured in the wrong field.
An address should look like an address
The ArcGIS Online geocoder uses a form of programmatic pattern matching. If an address does not match the patterns in the locator, your geocodes suffer.
Best practice is to ensure your addresses look as follows:
Suburbs in South Africa remain loosely defined and differ from dataset to dataset. The inclusion of extensions creates an additional host of problems and often suburb names change, or an individual may say their street falls in a neighbouring suburb for various reasons. You are more likely to get an accurate geocode using a city alone instead of using a suburb which does not match the suburb in the reference data you’re matching against.
Never trust a postal code
Many people do not even know their postal code and it does more harm than good by including an incorrect postal code in an address for geocoding in ArcGIS Online as the address will be scored down. What makes things even more confusing is the fact that a particular street may have a ‘box’ code and ‘street’ code which differ and both may not be accurately represented in the reference data being matched against. If you are going to include postal codes in your addresses to geocode, please ensure they all have four digits, otherwise ArcGIS Online will not recognise the postal code for what it is.
Preparing addresses for batch geocoding can be quite tedious, so we have created a toolbox to get you started with automating the process!
Clicking the image above will download an archive containing a toolbox with a simple Python script that uses a lookup table of freely available data from Statistics South Africa and the South African Post Office to attempt to normalise and clean address datasets prior to geocoding particularly for ArcGIS Online. You can use it in the same way you would use any other tool in ArcMap. Applying the 80/20 principal we have attempted to use the minimal amount of code in order to clean and normalise the majority of addresses, however each dataset is going to have its own nuances so it will be up to you modify the script in order to optimise it for each of your use cases.
If you’ve never used Python, don’t despair, the tool already does most of the heavy lifting for you and there is still much to be gained by adding text replacements and additional street types to the portions of the code indicated below. Simply navigate to the toolbox in an ArcCatalog window, right click on the script and select “Edit…” to be able to incorporate the additional records as and when required. If you would like to add additional functionality, some Python scripting knowledge will be advantageous.
Ultimately, the expectations for any geocoding exercise need to be realistically aligned with the quality of input address data. We must be aware that many datasets in South Africa still have a long way to go and with the dynamic nature of road networks there will always be gaps in the reference data used for geocoding, even in ArcGIS Online. It is up to us as the GIS users to ensure that we prepare our data correctly prior to geocoding in order to achieve the favourable results we seek.