Modelling the holiday-based redistribution of South Africans in December

MAP SERIES

Over the coming months, I’m planning on developing a map series to showcase often overlooked aspects of Cartography and GIS. The idea is to explore topical subject matter to create insightful and (hopefully) beautiful maps.

This is the first in the series.

OLYMPUS DIGITAL CAMERA

Every December hundreds of thousands of South African holiday-makers push pause on their lives and scatter across the country; making time to explore, relax and unwind.

I got to wondering if there would be a simple way of modelling this behaviour. Surely there must be some universal underlying factors that could be used to help explain where people go in December? I also knew I wanted to represent my data in a non-traditional way.

For the sake of simplicity, I limited my sights on South Africans moving within South Africa for the holiday season and eventually settled on four broad factors to consider:

  • F1 [-] Distribution of population during the rest of the year
  • F2 [+] Accessibility (using major roads as a proxy)
  • F3 [+] Distribution of holiday accommodation
  • F4 [+] Distribution of National Parks

There are obviously many more factors at play however these four seemed to interact spatially in a dynamic enough way across the country that I was happy to move forward with my investigation.

The density per factor was calculated per municipality, normalised across the country and combined into an equation that attempts to model the interaction between these factors as a linear function.

equation

In the formula, population density acts as a push factor – people will be moving away from areas of high population density towards areas with low population density. The availability of accommodation, how accessible the area is and the distribution of national parks all act as pull factors.

The amount that each factor contributes towards the final index is controlled with weights and the global difference within each variable is exaggerated by squaring it’s normalised value to highlight the most favourable areas more clearly.

The final index can be used to rank order each municipality based on the likelihood that it will be visited in December by people who do not live in that region.

These values were then used to generate the following cartogram:

dec_mapseries_cartogram_screenshot

  • You can explore the map right down to the municipal level
  • The shades of blue represent the percentage change in surface area relative to the region’s usual size. This is affected by the rank as well as the relative difference in the ranks surrounding the area.
  • National parks are included as well as major cities as you zoom in for context
  • The top 20 sites are highlighted with the concentrically banded points
  • Clicking anywhere on the map will return the overall rank for that region

Cartograms have been around since the 1800s. They provide us with a new perspective to our world by taking a thematic variable and typically substituting it for the area of the land that it represents.

The creation of cartograms comes with several challenges as regions must be scaled and still fit together. A recent (2004) and popular method of generating contiguous cartograms is the Gastner-Newman Method. This method is faster, conceptually simpler to understand and produces easily readable cartograms. The algorithm guarantees topology and general shape preservation (albeit with some distortion). This method allows its users to choose their own balance between good density equalization and low distortion of map regions, making it flexible for a wide variety of applications.

Now I need YOUR help.

Taking this one step further, I’ve configured a crowd sourcing web application which will allow users to post about their holiday destinations in a collaborative manner.

You will be able to access this from anywhere on any device and see information contributed by all users of the application. My hope with this is that this information will further support the outcome of the formula and cartogram produced in this exercise.

destinationwhere

Please share far and wide and happy holidays!

Insights for ArcGIS – A new spatial Business Intelligence (BI) tool in ArcGIS 10.5

ArcGIS 10.5 is on its way, with the planned release scheduled for mid-December 2016. One of the big changes coming is a new product being unveiled called Insights for ArcGIS.

So, what exactly is Insights for ArcGIS? I would like to look at it as a map centric BI tool. It allows you to perform analytics which helps you uncover secrets about your data. The data in this app is displayed on “cards”. On each card a user can display a map, chart or table with data.

I have played around with the app at pre–release stage and thought it would be nice to give you a preview before the app is released officially. I summarised some important things to know about insights as well as of my favourite things about the app.

The important things about Insights for ArcGIS:

  • It is only available with ArcGIS Enterprise 10.5.
  • It is a premium extension to Portal for ArcGIS with an additional cost.
  • It is a web application and can be accessed through a web browser.
  • You need to have an identity (Username and password) to use it.

Now for some of my favourite features about the app

1. Location is key

In most BI systems, the map card is there just to visualise data. With Insights, you have similar “mapping & visualisation” power as you would in your ArcGIS Online map viewer. This means you can create heatmaps, change symbology, set transparency etc.

In the example below I have added a card that has carjacking data collected at police stations in Gauteng over 10 years. You will note that smart mapping options included in Insights.

picture1

Within the card, I could change several things. The attribute field I choose to style my data by, the symbol type as well as the symbol style. With points data, you can also easily create a heatmap.

2. Multiple data sources

The real power of Insights for ArcGIS is that it allows you to pull data from multiple data sources into one dashboard view.

Currently supported data sources are:

  • Web maps and feature services from your organisation portal.
  • The Esri living atlas
  • Databases (SAP HANA, MS SQL, Teradata …tbc)
  • Excel spreadsheets

Insights is only available on Portal for ArcGIS. One of the limitations for now is that you cannot pull data from your ArcGIS Online organisational account into a card. This may well be on the future development path of the product. We will have to wait and see.

3. Document, share and re-run workflows

This is one of my favourite features on this app. Insights gives you the ability to document and share your analysis workflow with other users.

Remember model builder from ArcGIS Desktop? Insights has a similar tool. The only difference is that the workflow gets created for you. Below are 4 cards with different visualisations, analysing crime in September 2016 around the Johannesburg area.

As you create your visualisations, all your steps are being recorded in the background. You can switch to workflow view to see your workflows. The workflows can be shared and re-run. There is an update button that gives you the option to update the model. Here you can replace data and click update. When you switch to Page view this will then update the graphs and chart on your cards.

picture2

 4. Questions that guide your spatial analysis workflows

As I was doing my analysis, I noticed a button at the bottom of my active card. It’s called an action button, circled in red in the image below this tool makes spatial analysis easy. The tool asks a geographical question, and uses geoprocessing tools to then answer the questions. This puts geoprocessing tools in easy to understand everyday language.

Picture3.png5. Easy to use

What I love the most about Insights is the ease of use. I love the fact that tools and functionality are contextual. It’s very modern and uncluttered and has this drag and drop functionality that makes all analysis easy because it suggests tools as you pick data.

In conclusion, we live in an era where timely business information is critical to success. For a lot of our clients, ArcGIS is the system of record and business critical to their operations. Insights for ArcGIS offers a configurable BI tool specifically adapted to combine Esri’s spatial analysis platform with other record systems in your operation.

How does one get access to Insights for ArcGIS? You will need to have ArcGIS Enterprise licensed and installed on your premises. For more information regarding licensing and prices do not hesitate to contact your account manager.

 

 

Mathematically Verifying South African ID Numbers with Survey123

This blog post describes how South African ID numbers can be verified mathematically in Survey123. South African ID numbers have the following format:

{YYMMDD}{G}{SSS}{C}{A}{Z}

YYMMDD : Date of birth.
G  : Gender. 0-4 Female; 5-9 Male.
SSS  : Sequence No. for DOB/G combination.
C  : Citizenship. 0 SA; 1 Other.
A  : Usually 8, or 9
Z  : Control digit

The most challenging part of verifying the ID number is the control digit which is calculated by using the Luhn algorithm – this will be the focus of this blog post.

The best way to tackle complex mathematical functions in Survey123 is to break it up into separate mathematical calculations and using calculated fields:

The check digit is the last digit of the SA ID number so it can be retrieved with the following function: substr(${idnr}, string-length(${idnr}) – 1, string-length(${idnr})) where ${idnr} refers to the captured ID number.

Once you have an understanding of the substr() function the rest of the calculations used to verify the ID number is pretty much straight forward.

The survey’s XLSForm file can be found here (and can be freely used): SA Id number

  1. Copy the file to your downloads folder
  2. Open Survey123 Connect
  3. Create a New Survey and base it on an existing file
  4. Choose the Excel file that you have downloaded
  5. Your survey will be generated

Notes:

  1. The SA ID Number does not indicate if a user was born in 19yy or 20yy so both options are catered for – with a logic test to see if the birth date is in the future (age not greater than zero)
  2. Race is no longer indicated in the SA ID Number