As part of the future posts on our website we thought it would be interesting to share some technical components of projects that the Epoch team works on. Our first technical post by Mike Buller focuses on network tracing on cross platform GIS integration work that we do.
We are commonly called upon to implement EpochSync, allowing Smallworld data to be synchronized on an ongoing basis into an ESRI format, most often enterprise geodatabase (SDE). This creates a world of possibilities for a business to open up their Smallworld data in an easy manner throughout the enterprise. However, sometimes clients also want Smallworld-specific functionality to come along with their Smallworld data. One recent challenge we faced was synchronizing data from a Smallworld Electric Office database and replicating the EO tracing functionality using ESRI’s geometric network functionality.
First, a little bit of background about the Smallworld EO data model and tracing. Both are conceptually different than those used by ESRI. In Smallworld, tables can have multiple geometry fields, whereas in ESRI, a given feature class can have only 1 (or, in the case of a simple table, none). When a geometry field is added to a Smallworld table, it can be specified to be part of a manifold. This is the conceptual equivalent of an ESRI geometric network, as Smallworld geometry that is set on a field that is part of a manifold can connect to other geometry records that are part of the same manifold. However, just like an ESRI geometric network can be configured so that not all feature classes participating in the network connect to each other, so can Smallworld geometry that is part of a manifold be configured. In Smallworld, geometry that participates in a manifold (points and chains, which are equivalent to polylines) is not actually the item on which the tracing occurs. Underlying points and chains are nodes and links which are automatically created and maintained as points and chains are created, edited, and deleted. These nodes and links are the actual records holding the network connectivity information and against which tracing occurs. This is different than ESRI, as all polyline (i.e. edge) connectivity in a geometric network requires 1 intervening point (i.e. junction) feature. When an ESRI geometric network is rebuilt, any edges that are coincident to a degree that would cause them to connect, but that do not have a junction feature at that connectivity point, will have a default geometric network junction feature created at that point.
With the data model and conceptual differences in mind, consider the actual methodology by which tracing occurs in the two different systems. In Smallworld EO, all tracing is done via the proprietary Magik programming language. At its most basic level, the API simply traverses node-links-node-links until all breakpoints have been reached. However, a very high degree of customization is possible via the tracing API, as a method (or function, if you will) can be called on every record involved at each node and link to determine how the tracing should proceed. This is highly leveraged in EO, where a person running the electric trace tool can specify how the trace should run based on any combination of: electric phases; voltage levels; upstream/downstream/any direction; mounting (overhead or underground); future or current network state; circuit ID; bounding box; maximum allowable distance; and device normal status. It is relatively easy to modify the trace behavior as it is all governed by code. This is markedly different from ESRI’s geometric networks, as there are really only 2 ways to govern the trace behavior once the geometric network is defined…the Enabled field, and the bitgate field. In other words, in order to modify trace behavior in an ESRI geometric network, one must modify the data model; specifically, the field used to store bitgate values and the manner in which those values are derived.
To put it simply, all of the tracing flexibility in Smallworld EO needed to be able to be accomplished via the use of a single bitgate field in the ESRI data model.
We started by working with our client to define exactly which types of EO tracing functions were required in the ESRI geometric network. This went a long way to firming up how the bitgate fields were to be derived because, as it turned out, we only needed to support:
- Course-grained voltage differences…specifically, medium/low (i.e. distribution and delivery voltage levels) versus high (i.e. transmission voltage levels).
- Phases…A, B, C.
- Direction…upstream, downstream, or any.
- Device normal status…this was not to be provided as a user option in the ESRI geometric network. The trace was to stop at normally-open devices.
Feature Dataset and Geometric Network Feature Classes
Once we had the exact requirements and a copy of the data synced across from Smallworld EO, we had to define exactly what feature classes were needed for geometric network participation. We started by creating a feature dataset called Electric (feature datasets being needed, as geometric networks are created for feature dataset and the feature classes that participate in the geometric network must be part of that feature dataset). The feature classes we then added to that feature dataset are:
All the junctions are defined as simple, and all the edges are defined as complex, save for the isolating equipment connector. Everything connects to everything else.
In addition to the above, 3 supplementary feature classes were also created for the network. These are:
- EO_NETWORK_HYPERNODE_LINK: an edge class that represents a link between ‘worlds’, a Smallworld concept.
- EO_ISO_EQPT_INST_PIN2: a junction class that represents the far end of an isolating equipment installation connector.
- EO_MV_CIRCUIT_TRACE_SRC: a junction feature that represents the start of a EO circuit.
There are 2 more features of Smallworld that require description here in order to understand how the ESRI EO tracing was accomplished. Those are ‘worlds’ and ‘hypernodes’.
When geometry is created in Smallworld, it is created in a ‘world’. This represents an explicit geographic holder for that geometry. Most geometry is created in the main world (the world that we all understand when we look at a map), but other ‘worlds’ can be created to hold geometry in Smallworld…for instance, a world that represents the inside of a building such that geometry can be drafted in this world to show electric schematics.
Smallworld tracing can actually traverse worlds, as the trace does not need a ‘link’ in order to travel node-to-node. A Smallworld table, commonly called a hypernode table, provides 2 point geometry fields, one for one world, the other for a different world. When a trace is performed in Smallworld and the trace reaches one of these points, the trace ‘jumps’ to the other point owned by the hypernode record to continue tracing in the other world.
ESRI has no concept of ‘worlds’…all geometry in a dataset is in the same world. Therefore, EpochSync re-projects all internal world geometry into an appropriate bounding box in the main world when syncing from Smallworld to an ESRI database. However, in order to allow for tracing to proceed a real geometry is required to link the hypernode points. Hence, the creation of a feature class called EO_NETWORK_HYPERNODE_LINK that is simply a straight line between the 2 hypernode points in the ESRI database.
In Smallworld EO, any feature can be defined as the ‘beginning’ of a circuit, whether point or polyline. This concept did not meet the needs of the ESRI geometric network, and so we created a point feature class called EO_MV_CIRCUIT_TRACE_SRC to represent the starting point of any medium/low voltage network.
In Smallworld EO, devices that can be used to interrupt network flow (e.g. fuses, switches), are modeled as Isolating Equipment Installations. These features are polylines, not points (there are very good reasons for this that are not part of the scope of this discussion). However, in ESRI, we want the trace to traverse the polyline but stop at the far end if the device is opened. This cannot be accomplished using only the edge feature class. Therefore, we added a junction feature class called EO_ISO_EQPT_INST_PIN2 that represents the far end of the connector and is used as the stop point for open devices.
Bitgates for Modeling Phasing and Voltage Level
In order to allow users to specify phasing and voltage levels for the tracing, we needed to use bitgates. To start with, a field called TRACE_BITGATE was added to all feature classes participating in the electric geometric network. 6 bits were used to model this functionality. They were used as-follows:
|Meaning||Does not participate||C phase||B phase||A phase||High Voltage||Medium/Low Voltage|
With the chart above we can construct the values required to represent each combination of the above tracing parameters for a given feature. For instance, if the feature is medium or low voltage and has phases A and B, then the bit values are 001101, which translates to a bitgate value of 13. We can do the same for any combination of phase and voltage values. Note that if other tracing parameters were required (e.g. overhead versus underground), we would simply add as many bits as required to model those other parameters.
One of the client requirements was to be able to specify, for any given trace, upstream, downstream, or any direction. However, this was not possible to model using sources and sinks in the geometric network itself. This is simply because of the nature of electric networks as represented in maps.
Any given line or point in an electric network actually represents up to 4 different geometric network paths…one per phase, including neutral. Therefore, sources and sinks buy us nothing in an electric geometric network in terms of defining direction because the direction could be different per phase, and also because large parts of the network end up with indeterminate direction due to divergence and subsequent convergence of polylines.
Therefore, direction determination is actually a function of the tracing client itself. The EO data model, on the linear feature classes, has a field called reverse_flow_ind. This is a true/false field that specifies, for any given record, whether a trace from the circuit source flows from the start to the end of the polyline or in the opposite direction.
This field was leveraged by the ESRI trace client (itself a different topic altogether) to determine the direction in which the client wished the trace to proceed and place barrier flags on the features that are immediately in the opposite direction, thus simulating trace directionality.
Tying It All Together – Medium/Low Voltage Network
Finally, to maintain the electric geometric network, a Python script was created to perform all the data maintenance and rebuild the network. This script runs after an EpochSync run (i.e. right after the data has been re-synchronized, as is done on a periodic basis, usually daily). The script does the following:
- Initially sets the Enabled flag on all edge features to Yes.
- Sets the Enabled flag on all Isolating Equipment connectors to Yes, except for Circuit Breakers, which are set to No. This is done via direct SQL.
- This is a vagary of the client data model. The circuit start point is always a Circuit Breaker, and we do not wish the trace to pass through the breaker, so we disable them, causing the trace to stop at circuit breakers.
- Sets the TRACE_BITGATE field on everything to 32, which means ‘does not participate’. This is done via direct SQL.
- Sets the TRACE_BITGATE field to 29 for connector point installations, power transformer installations, protective equipment installations, and regulating equipment installations that are MV or LV. This means that they will be able to be traced on all phases for MV/LV traces. This is done via direct SQL.
- Examines the value of the phasing field on energy consumer, energy source, and light and sets the TRACE_BITGATE field to a value that properly models the phases on which these features can be traced. This is done via direct SQL.
- Constructs and runs SQL to update the TRACE_BITGATE field on connector segment installation, cable segment installation, isolating equipment installation, and wire segment installation. These are complex SQL calls as the determination as to whether or not any of these installations participate in a given phase is determined by the presence and attributes of the related ‘existing phase’ records that are not abandoned and are normally energized.
- For every hypernode and transformer record that has 2 points that are not coincident, creates a ‘hypernode connector’ line. This is done via arcpy.
- Places pin 2 features at the end of all isolating equipment connector lines, except for circuit breakers, for which the circuit source feature is placed at the beginning of the connector line. This is done via arcpy.
- Create and rebuild the network.
At this point, we’ve got an ESRI geometric network that traces a set of features synced from Smallworld EO in the same manner as can be done in the Smallworld EO desktop client.