Custom learning using Jetson Nano - How to setup the environment

There are some challenges in setting Jetson Nano device for carrying out custom learning on your dataset. The challenges stem from the incompatibilities between PyTorch, JetPack, TensorRT and other components in your environment. The purpose of this blog is to clear the air and provide a simple step by step guide on how to set up the environment right in the first place. 

Here are the steps:

Writing image to SD card

There are several SD card images that are available on different sites. The best one that works is the one that is provided by the official link (https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit#write) in preparing for environment. You can either go to https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit#write or click the following link to download the image:

https://developer.nvidia.com/jetson-nano-sd-card-image 

Follow along the instructions to flash the image to your SD card mentioned in the above link.

Preparing the Jetson Nano device

Once the image has been flashed to your SD card and you have booted up the Jetson Nano device, you can proceed with the following steps.

1. Run the Software Updater: This is to make sure any components that are updated after the official image is released, have the latest updates.

2. Validate JetPack version: Once you have run the Software Updater, it is time to validate that you have the correct version of Jetpack installed. For that you have to clone https://github.com/jetsonhacks/jetsonUtilities and download the code to be able to run jetsonInfo.py. 

Once you have cloned https://github.com/jetsonhacks/jetsonUtilities, run the following commands:

cd jetsonUtilities

./jetsonInfo.py

There are few things that we need to make note of. The JetPack version is 4.4. This is compatible with TensorRT version 7.1. In my experience most of the problems related to environment setup for custom learning stem from the incompatibility between these two components.

3. Install protobuf: In my experience, I found it helpful to install protobuf before installing any other componets.

Here is the command:

pip install protobuf # for python 2x

pipe3 install protobuf # for python 3x

Note:
If pip is not installed then install pip using the following command

sudo apt install python-pip # for python 2x

sudo apt install python3-pip  # for python 3

Since you might be using python3 for training your dataset and running AI tests, I highly recommend to install protobuf and pip for both python 2x and python 3x.

4. Setting up jetson-inference: The last step is to setup jetson-inference. The steps are clearly mentioned by Dustin Franklin's github repo at https://github.com/dusty-nv/jetson-inference/blob/master/docs/building-repo-2.md

Here are the set of commands as mentioned by Dustin Franklin.

$ sudo apt-get update
$ sudo apt-get install git cmake libpython3-dev python3-numpy
$ git clone --recursive https://github.com/dusty-nv/jetson-inference
$ cd jetson-inference
$ mkdir build
$ cd build
$ cmake ../
$ make -j$(nproc)
$ sudo make install
$ sudo ldconfig

During the steps mentioned in Dustin's set of commands, you will see the option to install PyTorch. If you are planning to train your dataset using PyTorch then you would need install PyTorch. 

Note: During the installation of PyTorch you will get the option top install PytTorch 1.6.0 for Python 3.6 as shown below:

Since the PyTorch is installed for Python 3.6, I used python3 for my custom learning.

Once the steps are complete you can head out to train your models using your datasets. Before that I recommend testing your environment using the steps mentioned in Dustin Franklin's post at: https://github.com/dusty-nv/jetson-inference/blob/master/docs/pytorch-cat-dog.md

Conclusion

In this article we have seen from start to finish how to setup your Jetson Nano environment to carry out custom learning for your models using your datasets. The goal of this article to address the compatibility issues that cause failed attempts to carryout custom learning of your models.

Create and Deploy Azure IoT Edge Module on NVIDIA Jetson device and send direct messages to the module

Introduction

In this tutorial we will see how we can deploy an IoT Edge Module on NVidia Jetson Nano device and send direct message to the newly created module and get its response. This tutorial can also be used on any Linux VM that Azure IoT Edge runtime supports. Just deploy the Azure IoT Edge runtime and use that VM as your device.

Link to GitHub post

Prerequisites

1. Azure IoT Hub. Steps to create Azure IoT Hub can be found here
2. IoT Edge Device registered. Details can be found here
3. Docker Image repository: This will be used to push the container images to. This tutorial requires access to already created Docker Image Repository with user name and password. Details on how to create Azure Container Registry can be found here
4. VS Code
5. Azure IoT Tools for VS Code: This should configured and connected to your IoT Hub. More details can be found here
6. Preconfigured Nvidia Jetson device with IoT Edge runtime installed and configured to use Azure IoT Edge. More details can be found here. Or Linux Virtual Machine with Azure IoT Edge runtime installed and configured. More details can be found here. In this case "Linux Virtual Machine" will act as a device.

Steps

Setting up the environment

Step 1. Create device identity as shown below:

az iot hub device-identity create --device-id [your-azure-iot-edge-device] --edge-enabled --hub-name [your-azure-iot-hub_name]

Step 2. On VS Code open command palette and enter command Azure IoT Edge: New IoT Edge solution.

Step 3. Choose the location for solution files.

Step 4. Choose name for solution. NvidiaJetsonEdgeSolution was selected for this tutorial.

Step 5. On "Select module template" question, choose "Python Module".

Step 6. Enter the name for "Python Module". For this tutorial "RequestProcessorModule" was chosen.

Step 7. For "Provide Docker image repository" question, choose a pre-existing image repository followed by name of your repository. Example: [your-docker-image-registry].azurecr.io/requestprocessormodule After the above step, VS Code will open a new window with the following view:

Step 8. Open the .env file and enter the user name and password for your docker image registry as shown below: 

Step 9. On VS Code open the command palette and enter command "Azure IoT Edge: Set Default Target Platform for Edge Solution".

Step 10. Select "arm64v8" or your correct architecture. You can find out the architecture of your device by running the following command on the device:

$ uname -m
aarch64

In this case "aarch64" corresponds to "arm64v8". Once the architecture is set the settings.json file would look like: 

Adding code

Step 1. Open "main.py"

Step 2 . Replace the code with the code mentioned below:

Deploy

Step 1. Right click "deployment.template.json and select "Build and Push IoT Edge Solution" as shown below: 

Step 2. The result of above step will be the creation of new folder called "config". The folder will contain a deployment json file corresponding to the default platform selected on step 8 under "Setting up the environment" section. For our Nvidia Jetson Nano device the architecture is arm64v8 as shown below:

Step 3. [Optional] Removing of SimulatedTemperatureSensor module. If you open the "deployment.arm64v8.json" file under config folder, you will notice that it comes with "RequestProcessorModule" and "SimulatedTemperatureSensor" module. This means if you deploy this deployment json file to the device, you will end up with additional SimulatedTemperatureSensor module. If you would like to not have that module added, then simple remove the section as shown below: 

Step 4. Open the "AZURE IOT HUB" section under the "Explorer" panel on VS Code.

Step 5. Select the target IoT Edge Device and right click.

Step 6. Select "Create Deployment for Single Device" menu item as shown below:

Step 7. This will open a dialog window to asking to select Edge Deployment Manifest. Select the correct deployment manifest file that corresponds to your device architecture under the config folder as shown below: 

This will result in the deployment of your edge module as shown below:

Step 8. Head over the Azure Portal and navigate to IoT Edge Device. This will show the newly created IoT Edge module as shown below: 

Step 9. To view the newly created IoT Edge module on the device, open the device terminal and run the following command:

$ sudo iotedge list

This will show the newly created IoT Edge module as shown below: 

Step 10. To view the log entries by the newly created IoT Edge module, run the following command on the device terminal:

$ sudo iotedge RequestProcessorModule

This will show the following result:

Test

Step 1. Head over to Azure Portal, select the IoT Edge device, click the "RequestProcessorModule".

Step 2. On the IoT Edge Module Details page, select "Direct Method" link. This will open up "Direct Method" page that is used to test.

Step 3. Execute the test as shown below:

Step 4. Head over to the device terminal and run the following command:

$ sudo iotedge RequestProcessorModule

This will show the following result:

Conclusion

In this tutorial we have seen how easy it is to create a new Azure IoT Edge module using python and deploy it using VS Code.

How to copy resource group tags between two resource groups using powershell

I recently ran into a requirement of copying resource group tags from one resource group to another resource group.

Here is the link to the github respository

CopyTag - Github

Source code:

The CopyTags.ps1 allows you to copy tags between resource groups. The source resource group is the resource group that we want the tags to be copied from and target resource group is the resource group that we intend the tag to be copied to.

Features

1. The script does not overwrite any tags that are already present in the target resource group.
2. The script will only copy those tags that are present in the source resource group and not present in the target resource group.
3. If the same tag (with same tag name) exists in both source and target resources groups, then the script will not overwrite the existing tag in the target resource group. This gives great flexibility of using this script without the concern of overwriting existing valuable tag information.

Usage

This script requires two parameters. These are sourceResourceGroup and targetResourceGroup.

Example:

.\CopyTags "[Source Resource Group Name]", "[Target Resource Group Name]"

Steps

1. Login to your subscription using powershell commands. Example: Connect-AzAccount -Subscription "[Your subscription name]"
2. Run the script as shown in the above example.

Cloud Native way of security

How to secure cloud in cloud native era

Introduction

When it comes to cloud security there are approaches that rely on same tools that worked for on premise infrastructure. Those approaches are commonly referred as on premise security posture. We are seeing that more companies are making cloud native applications. For cloud security, there is a need to have a shift from on premise security posture to a cloud native approach to security. We need to take an approach that relies more on native cloud primitives offered by the cloud provider.

Cloud native way of security

Cloud infrastructure has become the de facto infrastructure for hosting applications and workloads that enable a business to provide value. Cloud has proved to be a value multiplier and more and more businesses are trying to tap into the cloud value. In terms of the applications, companies are shifting to cloud native approach for application architecture. This shift towards cloud native applications is the right approach as this helps companies realize the cloud value. The cloud native way of security is taking the cloud native approach for the applications and applying it for cloud security. For example, the cloud native application is architected to use cloud platform as a resource and not just a hosting environment, similarly cloud native security is tapping into cloud platform for providing security posture.

Note: The left side of the cloud native security shots the conceptual view of security in depth. The implementation architecture might show each layer as a separate entity at the same level.

Why we need Cloud native way of security

It is all about the understanding and the approach that is the result of the understanding. There are two mindsets when it comes to cloud security. One mindset is about looking at what has worked at on premise data center and replicating it to cloud. The other mindset is looking at cloud as a platform for resources that provides components for cloud security. The on premise mindset typically involves looking the resources as virtual machines that are part of a network and are joined to a domain. The security tools that have evolved over the years for on premise resources essentially addresses the same scheme. These security tools involve (and not limited to):

•       Relying on host operating systems to provide security by managing and restricting incoming and outgoing traffic flow to/from ports. A good example is a windows firewall. 
•       Recording all the traffic that is flowing in and out of the network and then analyzing the traffic to flag any suspicious traffic flow. This also give us idea that security actions are typically thought of as reactive action and not proactive actions.

The above two points are just the two examples of security tools that are employed by on premise approach. The intention here is to point out the mindset where security is delegated to tools and not built in the resources/workloads that are part of an on premise establishment.

The reason that those tools were designed that way was because the cloud providers did not offered alternatives to support the cloud native mindset. If we go back a couple of years ago and image that we had to host a cloud native application, we still had to rely on the tools that worked before.

Now the time has changed. The cloud providers have addressed the lack of security tools as part of the cloud infrastructure. We have entered an era where native cloud security tools or cloud primitive tools have evolved to fully support the cloud native security. The need is to create cloud based solution architectures that leverage the cloud primitive security capabilities. For example:

·       Instead of relying on windows firewall, NSG (in case of Azure) should be employed. That will also mean to evolve the network architectures that designed for clouds to support that. 

·       Instead of using security tools that record all the traffic, tools like Azure Sentinel should be employed.

Case for Cloud native way of security

Shift-Left Security: With the cloud native way of security approach there will be more push towards achieving security using the shift-left approach. This means that instead of security being addressed something as wrapper, it will be addressed as a feature that is "baked in" into the architecture from the beginning. That is the very definition of the Shift-Left approach.

Security by platform: As more and more systems are being automated and moved to cloud, more and more threats are emerging. The threats are becoming multidimensional in nature. The cloud-native way of security is better equipped to cope with the present and emergent threats.

Defense in Depth: Cloud native way of security allows multiple mitigations applied against threats. If we replace the cloud native way of security with on premise way of security, then we would need to have multiple layers of security that we have to stand up and maintain either by ourselves or the vendors.

Scale: Cloud native way of security is platform based, so it inherently scales within the platform. To understand this, imagine we are back in 2000s. The data that was being generated at that time was not much in scale what we have now. For that time the on premise infrastructure was provisioned to cater the needs of that footprint of data. Now we are in an era of “big data” and to support the scale requirement for that enormous amount of data, we would need more dedicated resources.

Focus on business value: With Cloud native way of security, the focus of business is shifted from security and business to only business. This also means that the computing base that a business is responsible for has been reduce. This makes the systems more secure as there is less computing base that a business needs to protect.

Containerization (aka Kubernetes) revolution: With the advent of containerization more and more applications have shifted towards cloud native architecture. This shift means that the classical cloud security model needs a shift as well towards cloud native way of security.

Example Components of Cloud Native Security Architecture

Azure Sentinel

Each cloud provider has developed its own set of tools that can be defined in the realm of cloud native way of security. From Azure’s side, the tool that implements the cloud native way of security is “Azure Sentinel”. Azure Sentinel is a cloud-native SEIM that leverages AI for threat protection. The way Azure Sentinel works is that it uses Data Connectors. The Data Connectors hook into various Azure services such as Firewall. The Data Connectors then feed in data to Azure Sentinel. Azure Sentinel runs the security related tasks and then shows the security landscape onto a dashboard. More information on Azure Sentinel can be found here.

DDoS Protection

This is a platform level service that Azure offers. As mentioned earlier, the cloud providers are now answering to the call of providing cloud native security tools. Azure DDoS is one of the newer service that helps against the DDoS targeting cloud resources.

Firewalls

Firewalls is an Azure service that helps protect the virtual networks. Since it is a managed service offering there are advantages that come with it. These advantages include real time analysis of traffic and then feeding in the data to Azure Sentinel.

Web Application Firewall

For the applications that are hosted on Azure App services, the web application firewall is essential for it security. For a truly cloud native application that is hosted on App service, just adding the Web Application Firewall will make it cloud native for security.

Azure NSG

NSG is a primitive (native) service offering that helps us implement traffic rules for incoming and outgoing traffic. The beauty of NSG is that it acts like windows firewall but can be applied to various resources such as subnet. In case of subnet if NSG is added then the NSG rules will be enforced on all the VMs and other resources that are part of that particular subnet.

Conclusion

In this article we have looked at the two approaches to cloud security. As we have seen from the tools and the conceptual architecture there are Managed Services that are offered by the cloud providers. Using Managed Services for cloud security infrastructure is the step in the right direction to achieve cloud-native security.