Quality Attributes Series, Chapter 2: Integrability in a city digital twin as a system of systems

Quality Attributes Series. A city digital twin is not one system. It is a system of systems: domain twins, shared platforms, field devices and third-party services, each owned by someone different and each changing on its own schedule. This series looks at the quality attributes that decide whether such a thing works in practice, one attribute per post, and at the architectural mechanisms that achieve each one. Chapter 1 was about maintainability. Chapter 2 is about integrability.

A city digital twin is never finished being integrated. The water utility connects its own twin. A tender awards an air-quality sensor network to a new vendor. The transit operator moves to a new GTFS release. A re-tendered tunnel concession arrives with a controller that has operating modes the previous one never had.

Chapter 1 asked how far a change travels. This chapter asks how far apart two things are before they can work together. That is integrability: the cost and technical risk of future integration tasks, anticipated or not.

Table of contents

The integration problem

The abstract form of the problem is a system S and a set of components {Ci} that have to be integrated into it. S may be a platform into which the {Ci} are plugged, or an existing system that already contains {C1 … Cn} and has to absorb {Cn+1 … Cm}.

Figure 1. The integration problem. System S exposes a set of sockets, the integrability mechanisms built into its architecture. Each component Ci fits one of them directly, fits after adaptation, or fits none, in which case S has to change.

Integration is a task: both sides exist, both can be changed, or something can be inserted between them. Integrability is a property of the architecture of S, and it is predictive: it estimates the cost and risk of integrating a Ci that does not exist yet, using only the current architecture of S and whatever is known about Ci.

In a city twin the asymmetry is total. S is the platform you own: backbone, context broker, ontology, registry, the twins your organisation built. Every Ci belongs to someone else, and most of them arrive through procurement. Two consequences follow:

• Changes land on S. When an integration requires changes, the architect controls S and rarely Ci. The architecture has to be designed expecting to absorb them.
• S becomes S′. Each integration changes S, and previous integrability analyses may no longer hold. A city twin accumulates integrations for decades, so the trajectory of integration cost matters more than any single task.

Size and distance

Integration difficulty is a function of two quantities over the interfaces between {Ci} and S:

• Size: the number of potential dependencies between {Ci} and S.
• Distance: the difficulty of resolving the differences at each dependency.

Notionally, integration cost ≈ Σ f(size(x, y) × distance(x, y)) over every pair of integration points (x, y) in S and {Ci}. The formula is not a validated metric. It states what has to be inventoried (size) and what has to be estimated (distance).

Size is an inventory. Distance is where the analysis lives, and it has five dimensions. In a city twin:

Two technical points matter in a system of systems.

Interfaces are not APIs. An interface includes the semantics of the interaction, the interpretation of the data and the shared assumptions about resources and timing. None of those appear in an API description. In a city they live in the heads of a contractor at a utility, which makes implicit interfaces the dominant source of integration risk.

Service orientation only reduces syntactic dependency. Services, microservices and publish-subscribe decouple identity and call structure. If two “decoupled” twins share detailed knowledge of each other’s data, modes or timing, they are tightly coupled in the dimensions that cost money, and no bus will change that.

Integrability scenarios

A quality attribute scenario has six parts: source, stimulus, artifact, environment, response and response measure. The general scenario for integrability:

Four concrete scenarios for a city twin:

1. New component. A new air-quality sensor network, from a vendor chosen by tender, is integrated into the environmental twin’s ingestion pipeline within one month and one person-month. All readings are processed and all diagnostic messages ignored.
2. New version. The transit operator moves to a new GTFS Realtime release. The mobility twin re-integrates it within one person-day, all integration tests pass, no backbone resource limit is violated.
3. Existing components, new need. A heat-wave protocol needs a joint view from the energy, water and health twins. It is operational within two days of elapsed time, with less than one person-day of code changes and no change to the existing response measures of those twins.
4. New version with new states or modes. A re-tendered tunnel concession replaces its controller with one that has modes the previous one lacked. It is integrated into the mobility and emergency twins within six person-months.

Response measures are expressed as cost and risk, not as size and distance. Size and distance are the analyst’s instruments for estimating them. The fourth scenario is deliberately expensive: six person-months says the old controller was not decoupled in its behavioral semantics. Scenarios like it belong in the set because they test what the architecture did not anticipate.

Tactics

Integrability tactics reduce the size of the interface between {Ci} and S or the distance at each dependency. They fall into three families.

Figure 2. Integrability tactics grouped in three families, each with its reading in a city digital twin. The first four were covered in Chapter 1 as maintainability tactics.

Limit dependencies. Encapsulate, use an intermediary, restrict communication paths and abstract common services keep their meaning from Chapter 1: an API per twin, the context broker and event backbone, no twin-to-twin access, shared identity, geolocation, time series and ontology. What changes is what they buy. Encapsulation reduces syntactic and semantic distance by hiding exactly what differs between vendors. An intermediary reduces whichever distances it actually handles: a bus removes identity, a translator removes syntax and units, neither removes timing assumptions. Restricting paths and abstracting services reduce size. The fifth tactic, adhere to standards, predetermines part of the syntax and semantics of every future dependency: NGSI-LD, GTFS, CityGML, SensorThings, SAREF, IEC CIM. Its effect is bounded by what the standard defines and by how likely future suppliers are to conform. GTFS fixes syntax and data semantics and says nothing about timing. NGSI-LD fixes a data model and an API, not what the city means by “road segment”.

Adapt. Discover service (static) is a catalogue describing every twin, feed and service: location, API, and ideally semantics, which most catalogues omit. Discover service (dynamic) is the twin registry: binding at runtime, and the expectation that a component relies only on published descriptions. Neither reduces distance on its own; both reduce size and make distance estimable before committing. Tailor interface adds or removes capability around an interface without touching the API or the implementation: translation, buffering, validation, hiding operations from untrusted consumers. In a city this is the ingestion pipeline and the intercepting filters around each twin API; it resolves syntactic and data semantic distance at integration time. Configure behavior makes a component configurable at build, initialisation or runtime so that it matches a wider range of peers, for instance two versions of a standard selected by configuration. Plug-ins are the usual form.

Coordinate. Orchestrate invokes services from a control mechanism so that they stay unaware of each other: the workflow engine behind the heat-wave protocol calls the energy, water and health twins without any of them knowing the others. It centralises behavioral and temporal dependencies at the orchestrator. Manage resources makes components request computing resources from a manager instead of assuming them: backbone quotas, admission control for models, explicit ownership of each control loop. It is the only tactic that addresses resource distance.

Table 1. Integrability tactics against size and the five dimensions of distance. A plus sign means the tactic positively addresses the challenge; an asterisk means it may address it positively or negatively depending on its realisation; a blank means no consistent effect.

The matrix has two readings for a city platform. Most tactics reduce size; few reduce distance. A backbone, a registry and strict network policy reduce the number of ways a component can touch the platform, and still leave a six-month integration when a controller arrives with new modes. Distance is reduced by encapsulation, by tailored interfaces, and conditionally by intermediaries, standards and configuration. Temporal and resource distance are nearly unaddressed. One tactic covers resources; temporal distance appears only as asterisks. In a system with a dozen clocks and shared infrastructure, that is where analysis effort should go.

Patterns

Patterns are built from tactics, and the analysis of a pattern is the analysis of which tactics survived its implementation.

Three technical observations:

• A context broker is the Broker pattern. It bundles five tactics in one component, so a new source integrates once, with the broker, instead of once per consumer. The cost is latency, which matters for a ventilation loop and not for a monthly report.
• Publish-subscribe reduces size, not distance. A subscription is a cheap integration, but the bus does not prescribe event semantics, ordering or rate. That requires an event catalogue with owners, versions and timing contracts. Without it the backbone offers minimal syntactic distance and no control over the other four.
• Adapters are applied late, by design. They resolve whatever distance remains once the real Ci is known. The architecture’s job is to guarantee a place for the adapter and to make it the only place that changes.

A pattern can be present in name only. Publishers that broadcast messages addressed to one recipient, which replies only to the sender, are doing call-return over a bus. A broker bypassed “for performance” is a decoration in front of point-to-point connections. The question is never whether there is a bus; it is which distances it reduces.

Analysing a city twin for integrability

What can be analysed depends on the artifacts available: a list of chosen mechanisms, a documented architecture, or a running system with a commit history.

Tactics-based questionnaire. One question per tactic, put to the architecture team: supported or not, where, at what risk, with what rationale. Thirty to ninety minutes per attribute. In a city, run it once per organisation that owns a constituent system. The disagreements between answers are the semantic distances.

Design-decision checklist. Seven categories: allocation of responsibilities, coordination model, data model, resource management, mapping among architectural elements, binding time, choice of technology. Two carry most weight here. The data model, because the shared ontology is the only place where data semantic distance can be controlled for every future Ci at once. And choice of technology, because an enterprise bus makes connections easy to change and may introduce vendor lock-in.

Coupling metrics. Structural coupling counts connections from a component to the rest of the system. Historical coupling mines the revision history for co-commits. Dynamic coupling instruments the running system. System-wide measures such as Propagation Cost and Decoupling Level work on architectural representations, before any code exists. In a city twin, historical coupling is the one to watch: two twins repeatedly committed together with no structural relation between them have an undocumented semantic or temporal dependency, a modularity violation, and those correlate with defect rate and cost of change.

Playbook, applied

Seven steps: collect artifacts; identify the mechanisms used to satisfy the requirement; locate them in the architecture; identify derived decisions and special cases; assess satisfaction; assess impact on other attributes; assess cost against benefit.

Apply it to scenario 1. The rationale says the ingestion pipeline is pipe-and-filter with its topology read from a configuration file at initialisation. Both are integrability mechanisms, so the quick check passes. The derived decisions are where the mechanism can be undone:

• Binding time. Topology at initialisation is good. If editing the configuration triggers a rebuild, or filters are statically linked, the new vendor’s filter means a rebuild anyway.
• Allocation of responsibilities. If every filter is instantiated and initialised through one common interface, the configurator knows nothing about any specific filter and does not change. If it special-cases existing vendors, it does.
• Data model. If the configuration schema cannot express the new sensor’s initialisation, the schema changes and so does every reader. An untyped file with no published schema means the vendor discovers the format from code.
• Special cases. The first and last pipeline elements touch interfaces outside the pattern. A vendor filter that also does protocol translation mixes processing and interoperation. Pipes that transform data couple filters semantically to pipes.

Step six checks latency margin for one more filter, prescribed logging and fault handling, and the maintainability cost of new wrappers. Step seven asks whether a configurable topology is warranted at all for a pipeline with two filters.

Trade-offs

Latency returns from Chapter 1 with the same answer: measure against the use case and keep a deliberate short path for the control loops that need one. Two trade-offs are specific to integrability:

• Standards against flexibility. A mandated standard lowers distance for every conforming component and raises it for every legacy system that will never conform. Which standards, in which domains, at which versions, is a procurement decision with architectural consequences.
• Binding choices against testability. Discovery, configuration and orchestration defer decisions to runtime and multiply the configurations the platform can be in. Past a point, the dependencies among binding choices are themselves unknown and change is impeded. Configuration as code, versioned and tested, is the only answer that scales.

Governance

Mechanisms are a precondition, not a guarantee. For integrability in a city, governance is concrete because most {Ci} arrive through tenders:

• The catalogue is a contract: no component joins without a published description of interface, data semantics, timing, resource needs and owner.
• Standards go in the tender with versions and conformance evidence; “supports open standards” integrates nothing.
• Implicit interfaces get written down: units, identifiers, ordering, rates, who initiates, who owns the device.
• The event catalogue and the ontology have named owners who decide what things mean and when they change.
• After each major integration, S is S′: re-run the questionnaire and re-check the scenarios.

In summary

A city twin spends its life integrating things built by someone else. Integrability is the predicted cost and risk of those integrations, estimated from the size of the interface between S and {Ci} and the distance at each dependency, in five dimensions. Syntax is cheap; semantics, time and resources are where the cost is.

Three families of tactics, limit dependencies, adapt, coordinate, reduce size or distance, and four common patterns bundle them. Most of what a city platform invests in reduces size. Distance is reduced by encapsulation, standards that are actually followed, tailored interfaces and adapters at integration time, and a shared ontology that someone owns.

Write the scenarios, including the expensive ones. Break patterns into tactics and check which distances they reduce. Mine the commit history for the coupling nobody documented. Treat the tender as part of the architecture.

Next in the Quality Attributes Series: further attributes for the city twin as a system of systems, each with its own mechanisms and its own trade-offs against the ones already covered.

References

• Kazman, R., Bianco, P., Ivers, J. and Klein, J. (2020). Integrability. Technical Report CMU/SEI-2020-TR-001, Software Engineering Institute, Carnegie Mellon University. Figure 1 in this post follows its Figure 1, Abstract Representation of the Integration Problem.
• Bass, L., Clements, P. and Kazman, R. (2012). Software Architecture in Practice, 3rd edition. Addison-Wesley.
• Garlan, D., Allen, R. and Ockerbloom, J. (1995). Architectural Mismatch or Why It’s Hard to Build Systems Out of Existing Parts. 17th International Conference on Software Engineering.
• MacCormack, A., Rusnak, J. and Baldwin, C. Y. (2006). Exploring the Structure of Complex Software Designs. Management Science, 52(7).
• Mo, R., Cai, Y., Kazman, R., Xiao, L. and Feng, Q. (2016). Decoupling Level: A New Metric for Architectural Maintenance Complexity. Proceedings of the International Conference on Software Engineering.