{"id":25492,"date":"2022-03-22T00:00:00","date_gmt":"2024-09-13T08:48:52","guid":{"rendered":"https:\/\/wordpress-583806-4798031.cloudwaysapps.com\/edge-ai-architects-keep-edging-even-after-the-skiing-season\/"},"modified":"2024-11-15T16:21:21","modified_gmt":"2024-11-15T15:21:21","slug":"edge-ai-architects-keep-edging-even-after-the-skiing-season","status":"publish","type":"post","link":"https:\/\/informator.se\/en\/edge-ai-architects-keep-edging-even-after-the-skiing-season\/","title":{"rendered":"Edge AI: Architects, Keep Edging! Even after the Skiing Season"},"content":{"rendered":"\n

The current slalom between insecurity, logistics, and volatile energy\/material\/component supplies, is adding momentum to resource-efficient computing and green AI. Moreover, a recent video presentation<\/a> by Amazon<\/a> points out that up to 90<\/strong>% of the infrastructure costs for developing and running ML apps is inference.<\/em><\/p>\n\n\n\n

Train it sometimes, run it a lot of times<\/strong><\/p>\n\n\n\n

In plain speak, for each training or retraining time of an ML app, we\u2019ve a lot<\/em> of runtimes.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

(Figure source:  aws.amazon.com\/ec2 )<\/em><\/p>\n\n\n\n

Edge AI comes in useful in a growing variety of devices ranging from wearables in healthcare and fitness to high-velocity trains or Industry 4.0.<\/p>\n\n\n\n

Edge<\/em> computing in general contributes to green IT by improved quality<\/em> parameters, not least real-time performance, availability (less dependencies on a single point of failure), and data ownership & privacy, yet at a lower resource<\/em> cost (from 2022 onward, a brief presentation of three basic Edge architecture patterns<\/em> is given in our Agile Architecture<\/a> course). AI, including ML, fits into most Edge computing architectures. Some ML architectures, but far from all, are a flavor of one Edge pattern. For example Federated Learning<\/a>, well known in enterprise architecture and even more in today\u2019s AI, is a kin of the Edge-Preprocessing<\/em> pattern with intelligent data reduction.<\/p>\n\n\n\n

\u201cTiny ML\u201d from small data: from WWII Operational Analysis to ML for Industry 4.0<\/strong> One of my ten architect hints for Green AI in my last blog<\/a> contrasted highly relevant small clean data<\/em> to big data containing noise or gaps. Big data doesn\u2019t help when the data that would tell you \u201cthe correct story of a scenario\u201d is missing<\/em>. For example, survival bias<\/a> due to some warped \u201csifting off\u201d in the real world or in data ingestion. In WWII, statisticians at Columbia University examined bullet holes on aircraft after missions and recommended adding armor to the areas that showed the least<\/em> damage. Although counterintuitive at first glance, it turned out correct <\/em>in practice. Here\u2019s why:  <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The damaged<\/em> portions show parts where planes sustain<\/em> some damage and still come home<\/em> alive<\/em>. Planes hit in other parts usually did not <\/em>return\u2026  (figure: en.m.wikipedia.org\/wiki\/Survivorship_bias)<\/em><\/p>\n\n\n\n

Millions of data wouldn\u2019t compensate for the missing data about a handful of flaws. Andrew Ng<\/a> of Landing AI and founder of Google Brain told ZDnet in January: “Those techniques don’t really work when you have only 50 images (\u2026) Rather than Big Data, we’ve had to focus on good data.” A sample of dozens rather than millions can be workable; Landing AI has been able to develop useful industrial models for clients with a \u201crelative handful\u201d of data samples. “The tools we have been innovating at Landing AI are: you only have 50 images, so how do you label it to drive the best possible performance<\/em> out of on only 50<\/em> images,” arguing for a greater focus on what <\/em>data points are most important<\/em>, and making the model fit that.<\/p>\n\n\n\n

A Wave of New Edge HW and ML-Ops tool suites.<\/strong><\/p>\n\n\n\n

Ng\u2019s point also inspires vendors of edge-AI chips, platforms, and tools. To \u201cinclude both<\/em> designing\/tuning and<\/em>the rest of theentire ML-Ops pipeline<\/em> in your Cost\/Benefit estimates, in both monetary terms and green terms\u201d (number 2 in my abovementioned ten green architectural hints) favors ML from small relevant data on small built-in HW resources in Edge devices: C\/B ratio, rapid RT deployment and RT response to new risks, less network load, local ownership\/tenancy of local data. Unsurprisingly, big tech firms such as MS<\/a>, Amazon<\/a>, Google<\/a>, or Invidia<\/a> combine their edge-AI hardware with user-friendly ML-Ops tools for edge. Along with that, a growing number of new ones offer resource-saving AI-optimized chips (EsperantoTech<\/a>, Sima.ai<\/a>, Perceive<\/a>, Blaize<\/a>, Deep Vision<\/a>, Hailo<\/a>, FlexLogix,<\/a> AIStorm<\/a>, among others).<\/p>\n\n\n\n

So, push AI to the Edge for a number of reasons, not least for rapid RT response<\/em> to new situations and risks. When I saw ski- and SnB-star Ester Ledeck\u00e1\u2019s epic 100km\/h rescue<\/a> during her Olympic fourth-gold attempt, I remembered the discussion ten years ago between epi-geneticists and national-level trainers.<\/p>\n\n\n\n

Likely, after thousands of repeats, a learnt<\/em> exercise or movement pattern is performed with flow and subconscious rapid \u201clightning\u201d automatism – as were it an inherited<\/em> reflex. This minimizes \u201csignal traffic\u201d to and from the brain. Edge ML is pretty similar to that view of the body. Some enterprise architects and leaders probably remember playwright and president V\u00e1clav Havel (brother of cognitive AI<\/a> researcher Ivan M. Havel<\/a>), the protagonist of the Czech Charter 77 civil-rights movement that led to an end of a long Russian occupation. The Havels encouraged societies and organizations to evolve into organisms<\/em> whose parts are empowered, communicate, take autonomous action, and immediately respond to new needs locally. Intelligent-edge sensors along a robotic assembly line of Industry 4.0 follow a similar pattern; as this headline from CIO.com<\/em> said a week ago: 2022 is the Year of the Edge<\/em>.<\/strong><\/p>\n\n\n\n

by <\/em>Milan Kratochvil<\/a><\/u><\/em><\/em><\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/a> Trainer at Informator, senior modeling and architecture consultant at Kiseldalen\u2019s, main author: UML Extra Light (Cambridge University Press) and Growing Modular (Springer). Advanced UML2 Professional (OCUP<\/a> cert level 3\/3).<\/em><\/p>\n\n\n\n

Milan and Informator collaborate since 1996 on architecture, AI, rules, modeling, UML, requirements, and design. You can meet him this year at these courses <\/em>in English or Swedish (remote<\/strong> participation is offered and recommended) <\/em>:<\/em><\/p>\n\n\n\n

AI, Architecture, and Machine Learning<\/a><\/em><\/em><\/p>\n\n\n\n

Agile Architechture Fundamentals<\/a><\/em><\/p>\n\n\n\n

Agile Modeling with UML<\/a><\/em><\/p>\n\n\n\n

Avancerad objektmodellering med UML<\/a><\/em><\/p>\n\n\n\n

(on demand: Modular Product Line Architecture<\/em> )<\/p>\n<\/span><\/span>\r\n

\u00a0<\/span><\/span><\/h4>\r\n<\/span><\/span>","protected":false},"excerpt":{"rendered":"

The current slalom between insecurity, logistics, and volatile energy\/material\/component supplies, is adding momentum to resource-efficient computing and green AI. Moreover, a recent video presentation by Amazon points out that up to 90% of the infrastructure costs for developing and running ML apps is inference. Train it sometimes, run it a lot of times In plain […]<\/p>\n","protected":false},"author":6,"featured_media":26691,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[571,570,922],"tags":[671,591,594,670],"class_list":["post-25492","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-blogg","category-blogi","tag-agile-architecture","tag-ai","tag-arkitektur","tag-edge-ai"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t