Survey123 with a single attribute for repeating generic questions (instead of multiple attributes); and an auto-incremented list!

Example Requirement:

RiskAssessmentExcel

You are being tasked with creating a Disaster Risk Assessment form (as above) which requires the field worker to capture the risk factor associated with 14 Hazard Items:

  1. Avalanche
  2. Animal disease outbreak
  3. Drought
  4. Earthquake
  5. Epidemic
  6. Flood
  7. Hurricane
  8. Landslide
  9. Pandemic
  10. Tornado
  11. Tsunami
  12. Volcanic eruption
  13. Wildfire
  14. Winter storm

Each of these Hazard Items require the field worker to assess the risk factors of the hazard as follows:

  1. Affected Area (Very small=1, Small=2, Medium=3, Large=4, Very large=5)
  2. Probability (0% chance=1, 30% chance=2, 60% chance=3, 80% chance=4, 100% chance=5)
  3. Frequency (> 20 years = 1, 1 to 20 years = 2, Annually = 3, Monthly = 4, Weekly = 5)
  4. Predictability (100% Predictable = 1, Fairly Accurate to Predict = 2, 50/50 predictable = 3, Slightly Predictable = 4, Cannot Predict = 5)
  5. Magnitude (Low = 1, Low to Medium = 2, Medium = 3, Medium  to High = 4, High = 5)

A hazard score is calculated with the above risk factors as follow:

Score = Affected Area*0.5 + Probability + Predictability + Magnitude*1.5 + Frequency


Survey / Feature Class Design:

Having worked with Survey123 before we might be tempted to jump into it and simply create 14 questions, one for each Hazard Item listed in the requirement. The number of attributes quickly escalates however, since we also need to capture the associated Risk Factor for each Hazard Item – which results in a LOT of attributes! That doesn’t sound like a good design anymore, does it?

If we first designed this Feature Class in ArcGIS Pro we would probably design it simpler and lighter; perhaps a design like the one below, where each hazard item is not a separate attribute but a generic field which stores the Hazard Item descriptions:

FCRisk.JPG

That looks better, but how do we marry these two designs and also keep in mind the “limitations” of Survey123, seeing that Survey123 only allows one answer per “question”? An alternative would be to use a repeat section in the survey. A repeat section would give us the ability to add each Hazard Item as a (related) record whilst saving the Risk Factors in attributes.

Good, a Repeat Section it is… but how do we make it easy for the hapless field worker to capture 14 attributes and then still expect him/her to remember which hazards have been captured and which ones haven’t? It would be downright cruel to have the user choose the hazard item from a drop-down, like this (mistakes will happen!):

ice_video_20171213-115850(1)

Let us improve on this design.

It would be great if the field worker didn’t have to remember which hazard is next in the sequence, so if we can add an auto-incremented hazard list to the design it would be great! It would also be nice if the field worker could see some kind of progress as the survey is completed – for instance Hazard 5 of 14 captured.

Luckily Survey123 allows us to do all of these things. Look at the final design in action:

ice_video_20171213-160453

This solution comprises:

  • An auto-incremented Hazard Item (leaving no room for mistakes!)
  • An incremented counter showing the capturing progress (for example 1 of 14)
  • A clean database design that looks like this:SchemaS123
  • A happy field worker!

So how did we achieve this design?

In the repeat section of the survey, add a counter variable (count1) that we use for the increment:

calculate count1 count1 1
calculate counter Counter once(count(${count1}))

The count() function does a count of the number of repeats by simply counting all the count1 values. The once() function ensures that the count calculation only happens once, otherwise we will have an issue when the user traverses backwards through the list and the count() is recalculated.

Now, for the magic bit…

Create a CSV file with all the Hazard Items and their indexes in it. The file will look like this:

CSVFile

Now all we need is to hook up our incremental index (the ${Counter}) with the Hazard Item. This is done with the pulldata() function as follow, where HazardEntryLookup is our CSV file:

pulldata('HazardEntryLookup', 'hazard', 'hazardindex', string(${counter}))

Now, all that is left is to enable the 1 of 14, 2 of 14, etc functionality. This is done by specifying that the Repeat count should not exceed 14 (our number of hazard items). So setting the repeat_count option for the Repeat sorts that out nicely!

As always, here is the complete set of files used for this demonstration, feel free to use an adapt as necessary:

Source files

For my upcoming Blog I am going to show how to make an interactive progress / risk visualization bar in Survey123! Here is a sneak peek of the end result:

ice_video_20171214-172437

Calculating distances between locations (geopoints) in Survey123 with the Haversine Formula

Consider the scenario where a person is applying for a liquor licence and the law stipulates: New liquor premises must be located at least five hundred meters (500m) away from schools, places of worship; recreation facilities, rehabilitation or retreat centers, residential areas and public institutions.

For this requirement Survey123 can be a valuable tool and in this blog post I will show you how you can enable this functionality in a Survey123 form:

Step 1: Create a new survey with Survey123 Connect

Step 2: Create a geopoint field in the main survey which will be used to capture the location of the new licence premises

Step 3: Create a repeated section to capture the occurrences of nearby schools, places of worship etc. Each with a premise type and a geopoint

Your survey design should look similar to this:

type name label
geopoint new_licence_location New Licence Location
begin repeat nearby_places Nearby Permises
select_one category place_type Type of Premises
geopoint nearby_location Nearby Location
end repeat

Step 4: Now, split both the coordinates sets into their x,y pairs and convert them to radians:

calculate gps_lat gps lat pulldata(‘@geopoint’, ${new_licence_location}, ‘y’) * pi() div 180
calculate gps_long gps long pulldata(“@geopoint”, ${new_licence_location}, “x”) * pi() div 180

Do the same for the nearby_location geopoint field.
Step 5: Use the X, Y pairs to perform the distance calculation (in meters) with the Haversine formula:

acos(sin(${gps_lat_end})*sin(${gps_lat}) + cos(${gps_lat_end})*cos(${gps_lat})*cos(${gps_long}-${gps_long_end}) ) * 6371000

After having made some cosmetic enhancements and adding the 500m stipulation your survey should look like this:

DistanceCalcScreen

Feel free to use and adapt as you require. Here is a link to the Survey123 Excel design file: Licence Application

Mathematically Verifying South African ID Numbers with Survey123

This blog post describes how South African ID numbers can be verified mathematically in Survey123. South African ID numbers have the following format:

{YYMMDD}{G}{SSS}{C}{A}{Z}

YYMMDD : Date of birth.
G  : Gender. 0-4 Female; 5-9 Male.
SSS  : Sequence No. for DOB/G combination.
C  : Citizenship. 0 SA; 1 Other.
A  : Usually 8, or 9
Z  : Control digit

The most challenging part of verifying the ID number is the control digit which is calculated by using the Luhn algorithm – this will be the focus of this blog post.

The best way to tackle complex mathematical functions in Survey123 is to break it up into separate mathematical calculations and using calculated fields:

The check digit is the last digit of the SA ID number so it can be retrieved with the following function: substr(${idnr}, string-length(${idnr}) – 1, string-length(${idnr})) where ${idnr} refers to the captured ID number.

Once you have an understanding of the substr() function the rest of the calculations used to verify the ID number is pretty much straight forward.

The survey’s XLSForm file can be found here (and can be freely used): SA Id number

  1. Copy the file to your downloads folder
  2. Open Survey123 Connect
  3. Create a New Survey and base it on an existing file
  4. Choose the Excel file that you have downloaded
  5. Your survey will be generated

Notes:

  1. The SA ID Number does not indicate if a user was born in 19yy or 20yy so both options are catered for – with a logic test to see if the birth date is in the future (age not greater than zero)
  2. Race is no longer indicated in the SA ID Number

Three Steps to Asset Management with ArcGIS Online and Collector

Three quick steps to do Asset Management with Collector and ArcGIS Online:

Step 1 (ArcMap) – Authoring your feature classes and tables:

What you need: ArcMap and an ArcGIS Online Organizational Account

  • Create a database in ArcMap
  • Create a (point) Feature Class to store your Assets
  • Create a Table to store your Assets’ Condition Assessments
  • Create a Table to store Maintenance Tasks associated with your Assets
  • Link them all together with Relationship Classes to create a 1:M link between Assets and Assessments and Assets and Maintenance Tasks

ArcMap Asset Management FCs

  • Add the Asset Feature Class and the two related tables to a map document and publish it as a service to your ArcGIS Online account

Step 2 (ArcGIS Online) – Authoring your Web Map for use in Collector:

What you need: An ArcGIS Online Organizational Account

  • Create a new Map in AGOL and add your newly created Asset Management service to it
  • Add a basemap of your choice to your Map
  • Save your Map

Step 3 (Collector) – Deploying your Asset Management application:

What you need: An iPhone or Android phone with the Collector for ArcGIS Application installed

  • Start Collector and log in with your AGOL credentials
  • Open the Map created in Step 2
  • Start performing your Asset Management tasks!

Collector 1

For some additional images relating to this post – please see here: Asset Management Images

Homework for the avid reader:

  • Add a Tracking Layer for Field Workers to the AGOL Web Map
  • Install ArcGIS Operations Dashboard
  • Create a Dashboard that monitors real-time field activities such as Field Worker positions, Assets that require maintenance etc

OPS Dashboard