Ruminating on ONNX format

Open Neural Network Exchange (ONNX) is an open standard format for representing machine learning models. While there are proprietary formats such as pickle (for Python) and MOJO (for H20 AI), there was a need to drive interoperability. 

ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers. ONNX also provides a definition of an extensible computation graph model and each computation dataflow graph is structured as a list of nodes that form an acyclic graph. Nodes have one or more inputs and one or more outputs. Each node is a call to an operator. The entire source code of the standard is available here - https://github.com/onnx/

Thus ONNX enables an open ecosystem for interoperable AI models. The ONNX Model Zoo is a collection of pre-trained, state-of-the-art models in the ONNX format that can be easily reused in a plethora of AI frameworks. All popular AI frameworks such as TensorFlow, CoreML, Caffe2, PyTorch, Keras, etc. support ONNX. 

There are also opensource tools that enable us to convert existing models into ONNX format - https://github.com/onnx/onnxmltools

ML Data Preparation - Impute missing values

 In any AI project, data plumbing/engineering takes up 60-80% of the effort. Before training a model on the input dataset, we have to cleanse and standarize the dataset.

One common challenge is around missing values (features) in the dataset. We can either ignore these records or try to "impute" the value. The word "impute" means to assign a value to something by inference. 

Since most AI models cannot work with blanks or NaN values (for numerical features), it is important to impute values of missing features in a recordset.  

Given below are some of the techniques used to impute the value of missing features in the dataset. 

• Numerical features: We can use the MEAN value of the feature or the MEDIAN value. 
• Categorical features: For categorical features, we can use the most frequent value in the training dataset or we can use a constant like 'NA' or 'Not Available'. 

Computer Vision Use Cases

viso.ai has published a series of interesting posts, where they list down a number of potential usecases of computer vision (CV) across various industries. 

Posting a few links below: 

https://viso.ai/applications/computer-vision-in-manufacturing/

https://viso.ai/applications/computer-vision-in-retail/

https://viso.ai/applications/computer-vision-in-healthcare/

Jotting down a few usecases, where we see traction in the industry: 

• Quality inspection: CV can be used to inspect the final OEM product for quality defects - e.g. cracks on the surface, missing component in a PCB, dents on a metal surface, etc. Google has launched a specialized service for this called as Visual Inspection AI. 
• Process monitoring: CV can ensure that proper guidelines are being followed during the manufacturing process - e.g. In meat plants, are knives steralized after each cut?, Are healthcare workers wearing helmets and hand gloves? Do workers keep the tools back in the correct location? You can also measure the time spent in specific factory floor areas to implement 'Lean Manufacturing'. 
• Worker safety: Is a worker immobile for quite some time (potential accident?), Are they too many workers in a dangerous factory area? Are workers wearing proper protective gear? During Covid times, monitor the social distancing between workers and ensure compliance. 
• Inventory management: Using drones (fitted with HD cameras) or even a simple smartphone, CV can be used to count the inventory of items in a warehouse. They can also be used for detecting unused space in the warehouse and automate many tasks in the supply chain.
• Predictive maintenance: Deep learning has been used to find cracks in industrial components such as spherical tanks and pressure vessels.
• Patient monitoring during surgeries: Just have a look at the cool stuff done by Gauss Surgical (https://www.gausssurgical.com/) With AI-enabled, real-time blood loss monitoring, Triton identifies four times more hemorrhages.
• Machine-assisted Diagnosis: CV can help radiologists in more accurate diagnosis of ailments and detect patterns and minor changes far better than humans. Siemens Healthineers is doing a lot of interesting AI work to assist radiologists. More info can be found here -- https://www.siemens-healthineers.com/digital-health-solutions/artificial-intelligence-in-healthcare
• Home-based patient monitoring: Detect human fall scenarios in old patients - at home or in elderly care centers. 
• Retail heat maps: Test new merchandising strategies and experiment with various layouts. 
• Customer behavior analysis: Track how much time a user has spent looking at a display item. What is the conversion rate?
• Loss prevention: Detect potential theft scenarios - e.g. suspicious behavior. 
• Cashierless retail store: Amazon Go !
• Virtual mirrors: Equivalent of SnapChat filters where you can virtually try the new fashion. 
• Crowd managment: Detect unsual crowds in public spaces and proactively take action. 

A huge number of interesting CV case studies are also available on roboflow's site here - https://blog.roboflow.com/tag/case-studies/

Ruminating on Stochastic vs Deterministic Models

The below article is an excellent read on the difference between stochastic forecasting and deterministic forecasting. 

https://blog.ev.uk/stochastic-vs-deterministic-models-understand-the-pros-and-cons

Snippets from the article:

"A Deterministic Model allows you to calculate a future event exactly, without the involvement of randomness. If something is deterministic, you have all of the data necessary to predict (determine) the outcome with certainty. 

Most financial planners will be accustomed to using some form of cash flow modelling tool powered by a deterministic model to project future investment returns. Typically, this is due to their simplicity. But this is fundamentally flawed when making financial planning decisions because they are unable to consider ongoing variables that will affect the plan over time.

Stochastic models possess some inherent randomness - the same set of parameter values and initial conditions will lead to an ensemble of different outputs. A stochastic model will not produce one determined outcome, but a range of possible outcomes, this is particularly useful when helping a customer plan for their future.

By running thousands of calculations, using many different estimates of future economic conditions, stochastic models predict a range of possible future investment results showing the potential upside and downsides of each."

Ruminating on Memory Mapped Files

 Today, both Java and .NET have full support for memory mapped files. Memory mapped files are not only very fast for read/write operations, but they also serve another important function --- they can be shared between different processes. This enables us to use memory mapped files as shared data store for IPC. 

The following excellent articles by Microsoft and MathWorks explains how to use memory mapped files. 

https://docs.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files

https://www.mathworks.com/help/matlab/import_export/overview-of-memory-mapping.html

Snippets from the above articles:

A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory. 

Memory-mapped files can be shared across multiple processes. Processes can map to the same memory-mapped file by using a common name that is assigned by the process that created the file.

Accessing files via memory map is faster than using I/O functions such as fread and fwrite. Data is read and written using the virtual memory capabilities that are built in to the operating system rather than having to allocate, copy into, and then deallocate data buffers owned by the process. 

Memory-mapping works best with binary files, and in the following scenarios:

• For large files that you want to access randomly one or more times
• For small files that you want to read into memory once and access frequently
• For data that you want to share between applications
• When you want to work with data in a file as if it were an array

Due to limits set by the operating system, the maximum amount of data you can map with a single instance of a memory map is 2 gigabytes on 32-bit systems, and 256 terabytes on 64-bit systems.

Ruminating on TTS and ASR (STT)

 Advances in deep neural networks have helped us build far more accurate voice bots. While adding a voice channel to a chatbot, there are two important services that are required:

1) TTS (text to Speech)

2) ASR/STT (Automatic Speech Recognition or Speech to Text)

For TTS, there are a lot of commerical cloud services available ---- few examples given below:

• https://aws.amazon.com/polly/
• https://azure.microsoft.com/en-us/services/cognitive-services/text-to-speech/#overview
• https://cloud.google.com/text-to-speech
• https://www.nuance.com/en-gb/omni-channel-customer-engagement/voice-and-ivr/text-to-speech.html

Similary for speech recognition, there are many commercial services available:

• https://aws.amazon.com/transcribe/
• https://azure.microsoft.com/en-us/services/cognitive-services/speech-to-text/
• https://cloud.google.com/speech-to-text
• https://www.nuance.com/en-gb/dragon.html
• https://www.speechly.com/

If you are looking for open source solutions for TTS & STT, then the following open source projects look promising:

• MaryTTS: http://mary.dfki.de/
• Kaldi Speech recognition - https://kaldi-asr.org/doc/
• SoX sound processing - http://sox.sourceforge.net/
• Open Source Wrapper API - https://github.com/codeforequity-at/botium-speech-processing
• NVIDIA TTS - https://github.com/NVIDIA/tacotron2
• NVIDIA Speech Synthesis - https://github.com/NVIDIA/waveglow
• Mozilla TTS - https://github.com/mozilla/TTS
• Mozilla Speech to Text - https://github.com/mozilla/DeepSpeech
• Facebook STT - https://github.com/flashlight/wav2letter

A few articles that explain the setup:

• https://rasa.com/blog/how-to-build-a-voice-assistant-with-open-source-rasa-and-mozilla-tools/
• https://chatbotslife.com/full-blown-open-source-speech-processing-server-available-on-github-4fb88a54d338