Meta-Technicalities: Retaining Values in Formal Systems
When the rules themselves can be exploited, meta-rules provide the coordination needed to restore alignment to shared values.
Formal Systems Create Coordination
We live in a mesh of customs, norms, and formal laws nudging our behaviour. Some of these systems are simple and intuitive, such as the customary respect for others’ property and refrain from violence. Others like the patchwork of taxes buried in the price of everything we buy are formalised, complex, and require experts to interpret.
Formal systems enable coordination across geographical and cultural boundaries, as well as across considerable time. Local customs vary wildly, while a rich formal system provides a common framework for planning and action. However, this coordination comes at a cost: Expensive judiciary systems and highly paid legal specialists who could apply their intellect elsewhere.
Abusing Technicalities
Formal systems come at the expense of exploitable technicalities. A superior understanding of complex and contrived rules can be used to unfairly exploit others with legitimacy. Much of the modern finance industry operates on this basis, finding subtle means buried deep in a mesh of legal and technical complexity to shift money from the less sophisticated. Expending so much talent in zero-sum conflict is an immense cost to society.1
More concerning is where rule-makers themselves exploit technicalities. This can be done in such a way to trap the subjects of some system into a variant they never intended but was legitimately established. If a parliament votes to eliminate elections, even in a flagrant violation of the intent of a formal system and the expectation of its subjects, this may be entirely legitimate. Despite public and even military dissent, resistance becomes difficult as the very means of societal coordination through the legal system are compromised.
Meta-Technicalities: Rules Above Rules
Meta-technicality: A recognized rule or principle that exists outside the normal amendment processes of a formal system, providing a coordination mechanism when that system is compromised.
Meta-technicalities allow a coordinated response against a legitimate but unwanted change, like a catalyst for organisation. By remaining rarely used but widely recognised, they can avoid becoming a routine feature of a formal system’s operation and retain their superseding authority.2
Historically, there are rare but powerful examples of the robustness these features can provide. In 1944, King Michael I of Romania used his ceremonial but formally legitimate power to dismiss the country’s fascist government. Upon this planned dismissal, armed resistance swept in to provide the violence needed for this deposition to take place. The coup succeeded not because of the king’s practical power, but because his formal authority allowed coordination of the pre-existing resistance.3
The most famous examples of meta-technicalities are likely the US constitution and European monarchies. These institutions are well-known and respected. If the laws of a country diverged too far from their expected purpose and the values of their subjects, these institutions could serve as a rallying point for coordinated upheaval.
Corporate boards often have emergency provisions to override normal decision-making processes during crises. Constitutional courts can strike down laws which violate fundamental rights or expectations, even if enacted legitimately within the formal system of a country. A well-respected and widely-understood meta-technicality serves as a powerful force driving an institution to remain aligned with the values and expectations of its subjects.4
Applications to Institution Design
Organisations which seek to guard against value drift, especially in their senior ranks, should embed meta-technicalities into their structure. These should be known throughout the organisation and frequently referred to as key values which individuals are expected to defend. By referring to fundamental values or governance principles, they can be given the best chance of surviving great institutional stress.
Depending on the nature of the organisation and the stresses it may come under, there are a few meta-technicalities which could be incorporated into the structure and relied upon for continued coordination.
Ethics committees with veto power: A safety committee with veto power over key decisions such as military partnerships and major product launches.
Employee Democracy: A supermajority voting system allowing technical staff to halt development of concerning systems, or wind up a company altogether.
Constitutional Values: Legally binding commitments to human welfare, transparency requirements, or prohibitions against certain business lines such as weapons development.
Leadership Approval: Democratic approval by technical staff of senior leadership, ensuring governance which stays aligned to those who understand a technology’s capabilities.
There is a balance to strike when intentionally introducing a meta-technicality. Create an institution with real power, and a takeover risk is created when the organsation may be subverted even when functioning properly. Avoid giving any power, and that institution will likely be ignored. By appealing to existing legal frameworks and well-recognised values, the problem of deriving legitimacy from scratch can be avoided.5
This is, of course, about AI development and the institutions responsible for it. Leading AI companies have promised to act responsibly in the case that they successfully develop powerful AI. However, they have not provided clear means to enforce this behaviour in the laws their companies follow or the internal regulation they coordinate themselves by. If their staff saw a serious risk in a technology or product being developed and wanted to halt its deployment, it is unclear how they would coordinate to do so.6
These institutions remain vulnerable to capture by actors who may exploit the technicalities upholding governance while violating the institution’s mission. Deliberately instituting a meta-technicality, even one never intended to be used, helps improve organisational robustness against this disruption. As advanced AI development accelerates, these safeguards will become increasingly important in ensuring that technical innovation remains aligned to humanity’s best interests.
Note that I, the author, work in high frequency trading. My view of the finance industry is from more combative and exploitative perspective than might be fair for the whole industry.
If such a power is too frequently invoked, it becomes a regular part of the day to day operations and loses the separate nature needed to function as a meta-technicality. For example, take the US presidential veto. Originally intended to be a last-ditch check on the legislature, it quickly became a routine part of the governance process. A veto now would be far less effective at coordinating resistance against a corrupt congress than if it had remained on the books but out of regular use.
King Michael’s coup was a violent coup. The existing government did not simply step down at the King’s formal request. Rather, his statement that they should leave served as a sign to dissidents that now was the time for a semi-coordinated response.
These external controls themselves present a takeover risk, where power is seized despite the underlying institution functioning correctly. However, if established appropriately these institutions would have next to no power in their own right. If the King of England attempted to dismiss parliament at a normal time he would simply be ignored despite his formal power. It is only when there is considerable dissent that he acts as a coordinator of last resort, rather than a power broker in his own right.
It is hard to imagine the combination of universally recognised formal power with complete practical irrelevance seen in modern constitutional monarchies forming quickly. To add such a feature to your institution, relying on existing legal frameworks of company charters and non-executive boards will help providing legitimacy to new processes without adding unnecessary legal and operational risk.
Well argued piece.
I like the argument that one of the oldest governing systems - monarchy - could be a model for checks on what humanity does in the future.
Keep writing.