AI
Edge AI: Architects, Keep Edging! Even after the Skiing Season
The current slalom between insecurity, logistics, and volatile energy/material/component supplies, is adding momentum to resource-efficient computing and green AI. Moreover, a recent video presentation by Amazon points out that up to 90% of the infrastructure costs for developing and running ML apps is inference.
Train it sometimes, run it a lot of times
In plain speak, for each training or retraining time of an ML app, we’ve a lot of runtimes.
(Figure source: aws.amazon.com/ec2 )
Edge AI comes in useful in a growing variety of devices ranging from wearables in healthcare and fitness to high-velocity trains or Industry 4.0.
Edge computing in general contributes to green IT by improved quality parameters, not least real-time performance, availability (less dependencies on a single point of failure), and data ownership & privacy, yet at a lower resource cost (from 2022 onward, a brief presentation of three basic Edge architecture patterns is given in our Agile Architecture course). AI, including ML, fits into most Edge computing architectures. Some ML architectures, but far from all, are a flavor of one Edge pattern. For example Federated Learning, well known in enterprise architecture and even more in today’s AI, is a kin of the Edge-Preprocessing pattern with intelligent data reduction.
“Tiny ML” from small data: from WWII Operational Analysis to ML for Industry 4.0 One of my ten architect hints for Green AI in my last blog contrasted highly relevant small clean data to big data containing noise or gaps. Big data doesn’t help when the data that would tell you “the correct story of a scenario” is missing. For example, survival bias due to some warped “sifting off” in the real world or in data ingestion. In WWII, statisticians at Columbia University examined bullet holes on aircraft after missions and recommended adding armor to the areas that showed the least damage. Although counterintuitive at first glance, it turned out correct in practice. Here’s why:
The damaged portions show parts where planes sustain some damage and still come home alive. Planes hit in other parts usually did not return… (figure: en.m.wikipedia.org/wiki/Survivorship_bias)
Millions of data wouldn’t compensate for the missing data about a handful of flaws. Andrew Ng of Landing AI and founder of Google Brain told ZDnet in January: ”Those techniques don’t really work when you have only 50 images (…) Rather than Big Data, we’ve had to focus on good data.” A sample of dozens rather than millions can be workable; Landing AI has been able to develop useful industrial models for clients with a “relative handful” of data samples. ”The tools we have been innovating at Landing AI are: you only have 50 images, so how do you label it to drive the best possible performance out of on only 50 images,” arguing for a greater focus on what data points are most important, and making the model fit that.
A Wave of New Edge HW and ML-Ops tool suites.
Ng’s point also inspires vendors of edge-AI chips, platforms, and tools. To “include both designing/tuning andthe rest of theentire ML-Ops pipeline in your Cost/Benefit estimates, in both monetary terms and green terms” (number 2 in my abovementioned ten green architectural hints) favors ML from small relevant data on small built-in HW resources in Edge devices: C/B ratio, rapid RT deployment and RT response to new risks, less network load, local ownership/tenancy of local data. Unsurprisingly, big tech firms such as MS, Amazon, Google, or Invidia combine their edge-AI hardware with user-friendly ML-Ops tools for edge. Along with that, a growing number of new ones offer resource-saving AI-optimized chips (EsperantoTech, Sima.ai, Perceive, Blaize, Deep Vision, Hailo, FlexLogix, AIStorm, among others).
So, push AI to the Edge for a number of reasons, not least for rapid RT response to new situations and risks. When I saw ski- and SnB-star Ester Ledecká’s epic 100km/h rescue during her Olympic fourth-gold attempt, I remembered the discussion ten years ago between epi-geneticists and national-level trainers.
Likely, after thousands of repeats, a learnt exercise or movement pattern is performed with flow and subconscious rapid “lightning” automatism – as were it an inherited reflex. This minimizes “signal traffic” to and from the brain. Edge ML is pretty similar to that view of the body. Some enterprise architects and leaders probably remember playwright and president Václav Havel (brother of cognitive AI researcher Ivan M. Havel), the protagonist of the Czech Charter 77 civil-rights movement that led to an end of a long Russian occupation. The Havels encouraged societies and organizations to evolve into organisms whose parts are empowered, communicate, take autonomous action, and immediately respond to new needs locally. Intelligent-edge sensors along a robotic assembly line of Industry 4.0 follow a similar pattern; as this headline from CIO.com said a week ago: 2022 is the Year of the Edge.
Trainer at Informator, senior modeling and architecture consultant at Kiseldalen’s, main author: UML Extra Light (Cambridge University Press) and Growing Modular (Springer). Advanced UML2 Professional (OCUP cert level 3/3).
Milan and Informator collaborate since 1996 on architecture, AI, rules, modeling, UML, requirements, and design. You can meet him this year at these courses in English or Swedish (remote participation is offered and recommended) :
AI, Architecture, and Machine Learning
Agile Architechture Fundamentals
Avancerad objektmodellering med UML
(on demand: Modular Product Line Architecture )
Trainer at Informator, senior modeling and architecture consultant at Kiseldalen’s, main author: UML Extra Light (Cambridge University Press) and Growing Modular (Springer). Advanced UML2 Professional (OCUP cert level 3/3).
Milan and Informator collaborate since 1996 on architecture, rules, AI, modeling, UML, requirements, and design. You can meet him this year at public courses, in English or Swedish (remote participation is offered, encouraged, and currently highly recommended by epidemiology experts).
