{"id":25068,"date":"2017-05-31T00:00:00","date_gmt":"2024-09-13T08:48:13","guid":{"rendered":"https:\/\/wordpress-583806-4798031.cloudwaysapps.com\/auditability-and-vv-in-the-era-of-machine-learning-are-worth-a-close-review\/"},"modified":"2024-11-15T16:22:12","modified_gmt":"2024-11-15T15:22:12","slug":"auditability-and-vv-in-the-era-of-machine-learning-are-worth-a-close-review","status":"publish","type":"post","link":"https:\/\/informator.se\/en\/auditability-and-vv-in-the-era-of-machine-learning-are-worth-a-close-review\/","title":{"rendered":"Auditability and V&V in the era of Machine Learning are worth a close review\u2026"},"content":{"rendered":"\n
\n
\u00a0<\/div>\n
Developers, more often than architects, tend to get frustrated by declarative programming, because it boosts expressive power at the cost of less testability. Yet both the boost and the drop are a mild breeze compared to self-modifying and sub-symbolic Machine Learning (ML). <\/span><\/span><\/b><\/div>\n
\u00a0<\/div>\n
Reviews by humans<\/span><\/span><\/b><\/div>\n
You can hardly get away without ML in today\u2019s AI. It would be like a mainstream relational DB without SQL, or like a rule engine without declarative rules. And, you\u2019ll run into auditability issues in all three, but they tend to be an order of magnitude bigger in \u201csub-symbolic\u201d ML. It\u2019s much easier to check whether the learning works <\/i>accurately (a black-box test of a \u201cblack organism\u201d ), than to see why or how<\/i>. A clear-box view relates to the generated logic almost as loosely as an MRI-scan relates to what a patient is thinking.<\/span><\/span><\/div>\n
\u00a0<\/div>\n
Japanese industry robots started to assemble robots back in 2001. Today, robotics vendors offer complete lights-out <\/a>manufacturing systems (in other sectors referred to as \u201cno man in the loop\u201d earlier). So, needless to say, auditability and r<\/span>eviews by humans are a key architectural issue. Be discrete precision manufacturing, avionics, automotive, power generation, medical equipment, or any other application where approval criteria (ISO etc.) are strict.<\/span><\/span><\/div>\n
\u00a0<\/div>\n
Decision-tree induction from data<\/span><\/span><\/b><\/div>\n
\u00a0When the work of professor DonaldMichie<\/a> (a distinguished codebreaker ahead of D-day) resulted in the first ML algorithms for rule mining and decision-tree induction from data, those were able to show the learnt-in logic as diagrams or rules, and even to generate conditional constructs in a mainstream programming language. This became particularly useful in the elicitation and processing of tacit knowledge (the implicit \u201cknowwhy\u201d underlying the explicit knowhow of a human).<\/span><\/span><\/div>\n
\u00a0<\/span><\/span><\/div>\n
Many IT roles are quite familiar with declarative rules, although in narrower contexts: In DB schemas (or UML models), the rules are typically about referential integrity <\/i>along the lines of \u201con Delete cascade\u201d in SQL DDL (see also<\/i> the exclusive-or, {xor} , in the UML diagram<\/i>). In design, rules are often invariants and preconditions\/postconditions. In modeling, a UML state diagram defines visually the rules that govern event handling and state transitions. Etc.<\/span><\/span><\/div>\n
\u00a0<\/span><\/span><\/div>\n
\u00a0<\/span><\/span><\/div>\n
<\/a><\/div>\n
\u00a0<\/span><\/span><\/div>\n
\u00a0<\/div>\n
<\/div>\n
A role model, UML pun intended, with an either-or <\/b>referential integrity.<\/span><\/span><\/div>\n
From Informator course AvanceradModellering, T2716<\/a> <\/b>(structure-models chapter).\u00a0 <\/span><\/span><\/div>\n
\u00a0<\/div>\n
\u00a0<\/div>\n
Case Bases<\/span><\/span><\/b><\/div>\n
The technology that followed in inductive ML (after decision-tree induction), Case Based Reasoning (CBR), was instance-based and used lazy generalization which expressed the induced commonalities (between cases in the learning-data set) only later, at test time. Although less explicit than decision trees or rules, this was still fairly comprehensible<\/span> for humans to review.<\/span><\/span><\/div>\n
\u00a0<\/div>\n
Sub-symbolic \u201crepresentation\u201d of knowledge<\/span><\/span><\/b><\/div>\n
However, the commercial breakthrough of self-modifying technologies, mainly neural networks and evolutionary algorithms, made audits both hot and tricky. In a neural network, the logic learnt during training is sub-symbolic, \u201cpacked\u201d into weight values on connections and artificial neurons. Roughly speaking, a clear-box view relates to the generated logic almost as loosely as an EEG record relates to what a patient is thinking.<\/span><\/span><\/div>\n
\u00a0<\/div>\n
Therefore, many researchers argue that audit-related abilities of a subsymbolic-ML system, like reasoning about itself and meaningfully visualizing\/animating\/explaining its logic or its conclusions, improve greatly when its ML is combined with other (likely symbolic) AI techniques. If so, this is one of the rare cases in architecture where complexity and testability actually enhance each other rather than restrict.<\/span><\/span><\/div>\n
\u00a0<\/span><\/span>\u00a0 \u00a0<\/span>\u00a0<\/span><\/div>\n
Summing up<\/span><\/b><\/span><\/div>\n
Architecture is not exempt from ML\u2019s impact on AI, IT, and business processes. Although auditability was brought to the table already when robots started to assemble other robots, it is crucial today in all kinds of ML systems in a vast variety of applications and industry sectors.<\/span><\/span><\/div>\n
\u00a0<\/div>\n<\/div><\/div>\n<\/span><\/span>\r\n

Milan Kratochvil\u00a0<\/span><\/span><\/h4>\r\n
Trainer at Informator, senior modeling and architecture consultant at <\/span>Kiseldalen.com<\/a>, Advanced UML2 Professional (OCUP<\/a> cert level 3\/3). M<\/span>ain author: UML Extra Light (Cambridge University Press) and Growing Modular (Springer)<\/span>.<\/span><\/span><\/div>\n
<\/a><\/div>\n
Milan\u00a0and Informator collaborate since 1996 on architecture, modelling, UML, requirements, analysis and design. In the next couple of months, you can meet\u00a0him at Architecture (\u00a0T1101<\/span><\/a>\u00a0, T1430<\/span><\/a><\/span>, in English or Swedish<\/span>) or Modeling courses ( T2715<\/span><\/a>,\u00a0T2716<\/span><\/a>\u00a0, mostly in Swedish).<\/span><\/div><\/span><\/span>","protected":false},"excerpt":{"rendered":"

Developers, more often than architects, tend to get frustrated by declarative programming, because it boosts expressive power at the cost of less testability.<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[571,570,922],"tags":[591,628,594,627],"class_list":["post-25068","post","type-post","status-publish","format-standard","hentry","category-ai","category-blogg","category-blogi","tag-ai","tag-architecture","tag-arkitektur","tag-robotics"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t