SCHOOL EARLY WARNING SYSTEM

The North-West Department of Education and Sport Development is one of the largest departments having staff complement of 30000 and serving 800000 learners annually.

The department have a need to respond to the needs of schools on the ground. This has resulted in a need for a department to require a solution that will follow a systematic approach of reporting and recording incidence and emergencies occurring at schools. The solution should enable real time incident recording and must have the ability to send alerts to the department’s officials, thus enabling the department to promptly respond to the needs of schools. This has triggered a need for the optimal implementation of the ArcGIS Platform to support these development initiatives.

The North-West Department of Education and Sport Development is a huge administration with diverse products and service offering especially to the schools. The biggest challenge that is faced by the department is the turnaround time on issues raised by any sector at any given time. The current method of disseminating information is primitive and non-effective because it’s still relying the most on paper methods.

The implementation of the platform should assist the operations within the department, but also establish standards that will ensure interoperability and integration of data, systems and other GIS resources.

Each school would require access to the solution through the principal or management of the school. The people assigned would be responsible to communicate incidents through the solution, this should give the department the best chance to respond with effective reaction times.

Incidents
Spatial view of Schools with Incidents

The solution identified for these specific requirements was School Early Warning System. The Esri platform forms and integral part of the development of this system. The platform provides a spatial data infrastructure within the department. The ArcGIS Platform is an integrated web-GIS system that provides a wide range of capabilities for storing, managing, analysing, visualising and sharing geospatial data for all users within the organisation. The Platform provides the framework of data and technology needed to address a wide range of location-related business processes. Through change management processes, the department utilised ArcGIS Platform to introduce a method of reporting incidents at schools, backed up by photographic evidence.

survey123
Integrated mobile component of the School Early Warning System

The implementation of the School Early Warning System through ArcGIS Platform, support the Programmes of the Department’s Strategic Plan Document, the Annual Performance Plan and Medium-Term Framework Plan which are in a form of Five Year Plan and are identified as indicators within the following Strategic Objectives:

  • Ensure effective governance processes
  • Improved functionality and performance of schools
  • Enhanced accessibility of special schools
  • Accessible quality Grade R Education
  • Infrastructure management in schools and
  • Examination services managed

                                                                          

As part of the solution, the incidents will be loaded and locked in a centralised server via mobile device (Survey 123) wherein a unique identifier will be allocated to the incident for tracking purposes.

The Department Official will acknowledge receipt of the incident to update its status to received or in progress, in an Operations Dashboard.

The school will have access to track the status of the incident logged via a Dashboard

dashboard
Dashboard to give an overall indication of incident status

By understanding the main objectives of the North-West Department of Education and Sport Development to support the various schools and functions, the optimal implementation of the ArcGIS Platform serves as the right solution to meet the challenges faced by Department.

Projection, Georeferencing and Spatial Adjustment CAD

Projection

A map projection is a method for taking the curved surface of the Earth (3D) and displaying it on a flat surface (2D). A projected coordinate system is always based on a geographic coordinate system. The below table shows a few key differences between a GCS and PCS.

GCS (Geographic Coordinate System) PCS (Projected Coordinate System)
3D Spheroid Surface 2D Flat Surface
Latitude and Longitude XY Locations
Datum Map Projection

Map projections are designed for specific purposes. Conformal projections preserve the shape of the features, Equal area projections preserve the area of the feature displayed, Equidistant projections preserve the distance between certain features on a map while the Azimuthal projection maintains the directions of all points on the map.

Common Errors Due to Incorrectly set Coordinate Systems

 

Quick Easy Steps for Projecting

  1. Enquire with the source owner of the data, research or decide which coordinate system the data should be assigned. If the original coordinate system of the data cannot be sourced, it will be your responsibility to assign the correct coordinate system to the data.
  2. Before deciding which coordinate system the data should be assigned; visually assess the data or the layer properties of the data, to check whether the data is originally in GCS (Geographic Coordinate System) or PCS (Projected Coordinate System). Add the data in the Map Window in ArcMap.
    extent
    Take note of the Extent that has large numbers 361332,327979dd. This indicates that this is a Projected Coordinate System.

    decimal degrees extent
    Notice the Extent has small numbers -26,333041?? starting with to decimals. This indicates that they are Decimal Degree measurement which is a Geographic Coordinate System
  3. Define the coordinate of your data by using the Define Projection tool (This tool is for datasets that have an unknown or undefined coordinate system defined)define projection
  4. After defining the coordinate system of the data, check against a Basemap if it is located at the correct place.add basemap
  5. Project the data using the Projection Tool (This tool is to change the dataset from one coordinate system to another)project tool

 

Georeferencing and Spatial Adjustment

 

Georeferencing

 

Provides a correct, real-world spatial reference to Raster or CAD datasets; which is either missing a real-world spatial reference or has an unknown spatial reference.

Georeferencing is the process of aligning geographic data to a reference dataset in a known coordinate system. This method helps to associate a physical map, raster or CAD document with a spatial location. When you Georeferencing a dataset, you define where the data is located using map coordinates. Georeferencing uses Control Points, that associates the data with a specific location on earth; which allow the georeferenced dataset to be viewed, queried and analysed with other geographic data.

During Georeferencing it is important to use correct Reference Datasets; raster or vector feature classes can be used as reference data only if the data has the correct spatial reference. Identify distinctive locations that are visible in both datasets, these will be used as Control Points. The control points link the original dataset that is being georeferenced to the reference data –  the first control point is plotted on the original dataset (data that needs to be aligned with the reference data) the second control point is then plotted on the corresponding location on the reference data. Corresponding Links are established from the control points, which will be used to align the original dataset with the reference data.

Spatial Adjustment

Like Georeferencing; Spatial Adjustment aligns the original dataset to a reference data, based on links between corresponding control points. The major difference between the two methods is the original datasets and the usage of the method; Georeferencing is used to re-create a missing or unknown spatial reference for Raster or CAD data while Spatial Adjustment is used to correct the alignment of editable vector data.

Data in GIS usually comes from different sources, which means the user is required to perform additional work to integrate and use the data together. Spatial adjustment is used to correct; inconsistencies between data sources, correct geometric distortions and align features together. There are a variety of adjustment methods that can be used to adjust all editable data sources. Another interesting task in spatial adjustment is the ability to transfer attributes from one feature to another.

There are three methods for performing spatial adjustment: transformation, edgematching and rubber sheeting.  The edgematching method is typically used for connecting the end points of features with each other, rubber sheeting is best used for aligning minor geometric adjustments; this method stretches, shrinks and reorients features to match the reference data and the transformation method is like the transformation method used in georeferencing; it will shift, scale, rotate and skew the data if necessary.

 

The table below shows the major differences between the two methods above.

Georeferencing Spatial Adjustment
The process of aligning data with missing or unknown spatial reference to reference data in a known coordinate system Editing functionality for aligning data with a spatially accurate reference dataset
Transformation Method Transformation Method

Edgematching Method

Rubber Sheeting Method

 

Works Out of the Edit session Works in the Edit Session
CAD, Raster Imagery, Aerial Imagery Feature Class or Shapefile (editable vector data)

Common Errors Due to Inconsistency in Data

georef_cad
Error showing an un-georeferenced CAD Layer
inconsitency
Errors showing Features Classes (vector data) that do not align to the Reference Data

 

Quick Easy Steps for Correctly Aligning CAD/DWG Data

  1. Assign GCS (Geographic Coordinate System) to the CAD/DWG file (WGS_1984) in ArcCatalog, if it has no spatial coordinate system / unknown coordinate system.unknown coordinates
  2. Load a Basemap for reference purposes, it is advisable to choose the South Africa Cadastre Basemap available on ArcGIS Online Basemap especially if you are using CAD/DWG Files.basemap
  3. Georeference the CAD/DWG file, using the reference data (data in the correct geographic location that has the correct Spatial Reference i.e. farm portions).georef
  4. When Georeferencing; it is vital to have two distinctive locations in both the CAD/DWG File and the Reference Data, these two distinctive locations will be used as Control Points.georef_control_points
  5. Export the georeferenced CAD Feature to a Feature Class place it in a file geodatabase once georeferenced, first create a new file geodatabase if necessary. Working with Feature Classes is recommended especially if you must spatially adjust the data after Georeferencing.
  6. Adjust the feature class by using Spatial Adjustment (use the Affine method)spatialadjustment
  7. Use the Define Projection tool to define a coordinate for the feature class that perfectly aligns with the Reference data.
  8. Project, it accordingly and assess the resultsresults

Limitations of Georeferencing CAD datasets

Georeferencing a CAD dataset is limited to one- and two-point transformations using the similarity transformation method:

  • one-point transformation comprises one link and moves the dataset
  • two-point transformation comprises two links and moves, rotates, and scales the dataset uniformly

Both methods preserve the shape and angles of the CAD dataset, however, the aspect ratio (the ratio of the width to the height of an image or screen) of the CAD drawing is distorted

Spatial Adjustment of CAD datasets

The spatial adjustment method maintains the aspect ratio of the CAD drawing and prevents skewing to the x- and y- axes. However, it should be noted there will be an inherent deformation of the aspect ratio, from the georeferencing step.

 

 

 

 

Contributor: Busisiwe Ngobe

 

Understanding Survey Diagrams

Surveying is all about measuring distances, angles and positions on or near the surface of the earth. Surveyors use mathematical techniques to analyse field data. Survey measurement relies on understanding two basic scientific, accuracy and reliability.

Types

  • Plane surveying: Earth surface is considered a 2D plane with x-y dimensions.
  • Geodetic surveying: Earth surface is considered spherical (ellipsoid) 3 dimensional

Classes

  1. Preliminary survey (data gathering): is the gathering of data (distances, position and angles) to locate physical features (rivers, roads and other structures) so that data can be plotted to scale on a map or plan, also include the difference in elevation so that contour could be plotted.
  2. Layout survey: Marking on the ground (using sticks, iron bar or concrete monuments) the features shown on a design plan features: – Property lines (subdivision survey). – Engineering work (construction survey). – Z-dimensions are given for x-y directions.
  3. Control survey: used to reference preliminary and layout surveys.
  • Horizontal control: arbitrary line tied to the property line or HWY centre or coordinated control stations.
  • Vertical control: Benchmarks: points whose elevation above sea level is defined accurately.

Different methods of surveying

  1. Topographic survey: preliminary surveys used to tie earth surface features.
  2. Hydrographic survey: preliminary surveys tie underwater feature to surface control line
  3. Route surveys: preliminary, layout and control surveys that range over a narrow but long strip of land (highways, railroads, electricity transmission lines and channels).
  4. Aerial survey: preliminary and final surveys to convert an aerial photograph into scale map using photogrammetric techniques.
  5. Construction survey: layout of engineering work.
  6. Built survey: preliminary surveys tie in features that just have been constructed
  7. Property surveys: preliminary, layout and control surveys determine boundary locations.

Unit of measurement

There are two main measuring systems:

English system and Metric system (SI units).

  • Angles are measured by: Degrees, minutes and seconds.
  • 1 revolution = 360 degrees, 1 degree = 60 minutes and 1 minutes = 60 seconds

http://faculty.kfupm.edu.sa/CE/hawahab/WEBPAGE/CE260/NOTES/1CE260%20CH%20%201.pdf

Types of survey diagram

  • Servitude diagrams (powerlines, pipelines or municipal services) for registering servitudes over an existing property.
  • Lease diagrams for registering long leases over portions of properties.
  • Consolidation diagrams when it is required to consolidate several individual properties into one, taking out certificates of consolidated title.
  • Mineral diagrams to register mineral rights separately from the land rights.
  • Mining title diagrams for registering the right to extract minerals from the land.

http://csg.dla.gov.za/diagram.htm

Coordinate systems

SA_Coordinate

Angles of direction

Directions are measured clockwise which is the opposite for GIS data (counter clockwise).

Azimuths

  • Azimuths are horizontal angles measured clockwise from any reference meridian.
  • In a plane surveying, azimuths are generally measured from north/south starting at 0⁰.
  • Azimuths are used advantageously in the boundary, topographic, control and other kinds of survey, as well as in computations.

 

In the south azimuth system, the angles are measured clockwise from south 0⁰.

south azimuth system

 

ArcGIS by default, accept angular measurements of the polar direction measuring system

Polar angles are measured counter-clockwise from the positive x-axis, east 0⁰.

c_clockwise

Bearing

  • Assumes that total station is set up at points A, B, C, D, E, F and G; bearing read on lines AB, BC, CD, DE, EF and FG.
  • AB, BC, CD, DE, EF and FG are forward bearings and the inverse will give backwards bearings
  • Both the forward and backwards bearing should have the same numerical values.

One can change the direction measuring system and angular units the editing tools use on the Units tab of the Editing Options dialog box.

http://moodle.najah.edu/pluginfile.php/47169/mod_resource/content/0/Angles_Azimuths_Bearings.pdf

http://desktop.arcgis.com/en/arcmap/10.3/manage-data/editing-fundamentals/about-direction-measuring-systems-and-units.htm

 

 

 

 

Contributor: Lutho Mbeki

Improving underground mine safety

Pillar safety analysis on the map:

Thinking about Lily Mine as reported by Mail & Guardian on February 15,2016, that the central pillar of ore, called a crown pillar, collapsed subsequently leading to the lamp room which was at the entrance swallowed up by the sinkhole burying three mine employees. This made me to think of the underground solutions we have implemented using 1ArcGIS for Mining to visually analyse on the map the safety factor of pillars individually and accumulatively, and be able to overlay on that single map – mine progress, surface infrastructure and all other related mine assets.

Many mine incidents recorded in the past have always been linked with pillar failure. Geotechnical engineers in an effort to predict and minimise such incidents; use proven, yet complex mathematical calculations to determine the safety factor of pillars in the mine. Most of these calculations are time consuming to replicate and difficult to display to relate them with other possible surrounding risks as well as the cumulative effect with surrounding pillars.

Applying Geography using ArcGIS has proven that it can remove technology barriers between different mine technical systems and save time to get to the answer required to making a decision. Using the application in a coal mine to calculate and display the pillar safety factor in historical workings and active areas proved to be highly beneficial. A complex geotechnical method was incorporated into ArcGIS platform with variable parameters taken into consideration including pillar shape, distance between pillars, size, material and other surface irregularities of the hanging wall and foot-wall.  The pillar safety factors were easily displayed and shared to key stakeholders and accessible using any device such as iPad, enabling decision makers to make decisions wherever they were. It became much easier to replicate the model for other mines with flexibility to change parameters. The time to load new data entries and to verified them on accuracy drop from taking months into seconds.

Safety Factor

This did not only improve the method used and data quality. It also flagged risk areas that were never considered before in the underground workings and on surface by enabling visibility of other assets that were at risk of sinkholes should a pillar collapse.

1ArcGIS for Mining is a mining focused solution leveraging ArcGIS Platform technology

 

 

Migrating Python Scripts to ArcGIS Pro

What’s new?

MainImage

With the migration towards 64-bit processing in ArcGIS Pro, some big changes have come to the Python environment as well.

  1. Python in ArcGIS Pro has been upgraded to version 3.4. All other ArcGIS products are still using version 2.7. Both versions of Python are being developed in parallel and share much the same functionality.
  2. Changes have been made to the functionality within the arcpy site package. This includes the dropping of some functionality and the augmentation of others, e.g. arcpy.mapping has been replaced with arcpy.mp in ArcGIS Pro to support ArcGIS Pro’s mapping workflows. For a detailed overview of changes consult the following page.

Assessing the situation

ArcGIS Pro comes with a geoprocessing tool called Analyze Tools for Pro (Data Management Tools > General). This uses the Python utility 2to3 to identify issues when migrating a script and even goes so far as to identify functionality that has not been migrated to ArcGIS Pro. Running this tool will generate an output for you that will state which lines have errors and suggest appropriate changes which you can manually go through and assess. Often the required changes are small and can be make quickly without automation.

Converting your scripts

Sometimes a script is simply too big to go through manually. Thankfully Python 3 comes with a tool to help automate the conversion process.

NOTE: The following steps make changes to the input script. We recommend making a copy of the script being converted and appending 34 to the end of it and making changes to this version of your script so that you leave the original intact. E.g. script.py -> script34.py. If you choose not to do this, don’t worry, the script creates a copy of the file in the directory with a .bak extension to ensure the original script is preserved.

  1. Run Command Prompt as an Administrator
  2. Type in the following:

PythonInPro34

where C:\script34.py is the path to the script you want to convert

  1. Once done, the script should have most of the changes done required to make a script functional using Python 3 in ArcGIS Pro. We say most because some functionality within Python could potentially have moved, been renamed or replaced which would require manual intervention on your behalf where 2to3 utility could not make the required changes.

Writing scripts to work in both Python 2 and Python 3

The following tips will help greatly in ensuring a Python script will work in both ArcMap and ArcGIS Pro as long as the tools referenced in the script are available in the ArcGIS Pro version of arcpy. By making these practices a habit when scripting in the Python 2 environment, you will greatly ease the transition into the world of Python 3.

  • Tip 1:

Adding the following line to the top of your script will import some of the new rules enforced in Python 3 to your Python 2 script:

PythonInPro34_2

print_function

The print statement has been replaced with the print function. This function is also available in Python 2 and by using it you ensure your scripts will work in both environments.

PythonInPro34_4.png

 division

Python 3 handles division of integers in a more natural line of thinking. This is one of my favourite new things as it makes things far less confusing for people just starting out with Python.

Python 2: 3/2 = 1
Python 3: 3/2 = 1.5

If you find you need to use the old truncating division, you can simply use ‘//’ instead of ‘/’.

 absolute_import

The behavior in Python 3 means that by default top level imports are honoured. Lower level imports need to be explicitly stated. This relates to complex scripts referencing other scripts and for the most part will not affect the majority of users. For more on the implications of this click here.

unicode_literals

String literals are unicode on Python 3 and making them unicode on Python 2 leads to more consistency of your string types across the two runtimes. This can make it easier to understand and debug your code!

Basically “Some string” in Python 3 is now equivalent to u”Some string” in Python 2.

If you want to use 8-bit strings like the default in Python 2, simply place a ‘b’ in front of it and you’re good to go.

  • Tip 2:

Import known modules with changes in the following fashion to ensure that the required functionality will be available within your script:

PythonInPro34_3

Cheat sheet to changes in Python 3

  • Adding the following line at the top of your script will enforce encoding within your script in Python 3 as it’s parsed to utf-8: PythonInPro34_5

You no longer have to cast to string in Python 3 – anything within quotation marks will explicitly be treated as an encoded string of the document’s encoding type!

  • Exceptions are no longer iterable, you are required to use the exception attribute args to print messages:

PythonInPro34_6

  • int and long types have been merged. Before, one could simply write 5L and the 5 would be a long integer, now this will give you a syntax error. If you explicitly need to set a long integer the following approach is required:

PythonInPro34_7

One of the foundations of the ArcGIS Platform is the concept of extensibility – the ability to allow users to extend the functionality of the software beyond it’s out-of-the box processing capabilities to suit the required workflow. The Python scripting language lends itself very effectively to this end. Using some of the tips outlined in this post you’ll be well on your way towards producing adaptable Python scripts that speak to the needs of users within multiple environments.

Happy scripting!

Esri AUC 2015 – the highlights package!

blog auc

The very first Esri Africa User Conference has shut its doors. And what an incredible experience it was for us all! It’s not every day in Africa that you can walk around with over 900 people that actually know about GIS 🙂 I think the best way to sum up the success of the conference is to look at how our colleagues from Esri Inc were taken aback by the enthusiasm, participation and general level of interest in the technology and the science of GIS. Well done to all those who participated and for all the hard work that the organisers put into the event. I have asked my colleagues to help me put together our Top 7 observations from the conference.

1. Education

This was so popular that we had to turn people away from the dedicated Education track due to lack of space! A number of people I spoke to singled out Michael Goodchild’s keynote about GIS as a science as one of the most interesting talks of the week – what an honour it was to see the ‘informal chat’ he had with Mr Dangermond and Michael Gould of Esri. And there was also significant interest at the Education stand in the Expo centre that was promoting the Funda Lula initiative as well the new upcoming Diploma in Geographical Information Science and Technology (GISc & T).

2. App speed dating

Having been an extremely popular item at the User Conference in San Diego this year, Esri South Africa decided to show of our own talents with a uniquely African twist. All for a serious purpose too – understanding the (very wide) range of apps that run on the ArcGIS Platform and what they are geared to do. The video will certainly be played out in our own offices a few times in the years to come! Well done to the ‘apps’ for putting on a great show.

3. Sandbox and 3D printer

People are always interested in toys… While the software demo stands were busy throughout the Expo, it was the educational sandbox and 3D printer that were drawing the most unexpected crowds. If you missed it, the Sandbox is an interactive teaching tool that dynamically projects a contour map onto a box full of sand as you move it around. I saw some new and interesting types of topography being sculpted in there all week! The 3D printer was used for many fun things, but also showing how to turn an abstract map of something like a volcano, into a real, tangible object that you can interact with… Very exciting what we will be doing with this technology in the next few years.

4. Imagery

A sometimes overlooked strength of the ArcGIS Platform is working with imagery. Lots of imagery. Some exciting topics that were covered by Peter Bekker in the Plenary that looked at how you can quickly (!) and easily access petabytes of Landsat data from your basic web browser. You can dynamically adjust the data to show change over time or calculate an NDVI. For more information, have a look at Esri Imagery Services.

5. Free the data

A common area of discussion we had with a number of the delegates was our old friend “data”. Something that came up a few times is how can they engage the public to provide more data – that is, allow the public or clients become active participants in the processes of government and private sector service providers. This is commonly referred to as crowd-sourcing and there are a number of ways of enabling this through the rich apps provided by the ArcGIS Platform (e.g. Survey1-2-3, Collector, etc.). But remember that the tools are only the final piece of the puzzle – first you need to design how this will fit in with your current business processes, who will be the owner of the data, how will it be managed, etc.

The flip side of this coin, is that organisations holding data want to be able to share it more effectively – both with each other – and back to the public. This certainly means that the ArcGIS Open Data initiative is something that will be growing rapidly on Africa in the coming years. For more info, go to ArcGIS Open Data.

6. People!

The final item on our list was… You! I have never seen a closing session quite so full before – this showed to me that everyone (despite their best efforts at the party on Thursday) was keen to squeeze every last drop they could out of the conference. All the technical workshops and training sessions were full; all the paper presentations I went to were very well attended. It was also great to have a good representation from our friends and colleagues in other parts of Africa – I enjoyed some talks about natural resource management in central Africa – and even managed to introduce two experts that are working on the same topic in the same region!

7. Doing it all over again?

Finally, Mr. Dangermond did a pretty good job in helping us decide whether we would be doing this again. Based on some pretty enthusiastic responses from the crowd at the closing plenary, it looks very likely that we will do this all again in 2 years’ time – but nothing is confirmed just yet! The crowd ‘noise-o-meter’ was a little more difficult to read when Jack asked where it should be hosted… but seemed like Cape Town edged it to me? Or was it Durban?

Also, for those of you who would like to review any of the presentations or watch videos of the plenary sessions, they will be made available soon – check back here for details!

– Richard