开源软件名称（OpenSource Name）：PINTO0309/MobileNet-SSD-RealSense开源软件地址(OpenSource Url)：https://github.com/PINTO0309/MobileNet-SSD-RealSense开源编程语言(OpenSource Language)：Python 100.0%开源软件介绍(OpenSource Introduction)：MobileNet-SSD-RealSenseRaspberryPi3(Raspbian Stretch) or Ubuntu16.04/UbuntuMate + Neural Compute Stick(NCS/NCS2) + RealSense D435(or USB Camera or PiCamera) + MobileNet-SSD(MobileNetSSD) 【Notice】December 19, 2018 OpenVINO has supported RaspberryPi + NCS2 !! 【Dec 31, 2018】 My blog【Japanese Article1】 Table of contents1. Summary SummaryPerformance measurement result each number of sticks. (It is Detection rate. It is not a Playback rate.) Verification environment (1)
Result of detection rate (1)
Verification environment (2)
Result of detection rate (2)
Performance comparison as a mobile application (Based on sensory comparison)◯=HIGH, △=MEDIUM, ×=LOW
Change historyChange history
[July 14, 2018] Corresponds to NCSDK v2.05.00.02 [July 17, 2018] Corresponds to OpenCV 3.4.2 [July 21, 2018] Support for multiprocessing [MultiStickSSDwithRealSense.py] [July 23, 2018] Support for USB Camera Mode [MultiStickSSDwithRealSense.py] [July 29, 2018] Added steps to build learning environment [Aug 3, 2018] Background Multi-transparent mode implementation [MultiStickSSDwithRealSense.py] [Aug 11, 2018] CUDA9.0 + cuDNN7.2 compatible with environment construction procedure [Aug 14, 2018] Reference of MobileNetv2 Model added to README and added Facedetection Model [Aug 15, 2018] Bug Fixed. `MultiStickSSDwithRealSense.py` depth_scale be undefined. Pull Requests merged. Thank you Drunkar!! [Aug 19, 2018] 【Experimental】 Update Facedetection model [DeepFace] (graph.facedetectXX) [Aug 22, 2018] Separate environment construction procedure of "Raspbian Stretch" and "Ubuntu16.04" [Aug 22, 2018] 【Experimental】 FaceDetection model replaced [resnet] (graph.facedetection) [Aug 23, 2018] Added steps to build NCSDKv2 [Aug 25, 2018] Added "Detection FPS View" [MultiStickSSDwithRealSense.py] [Sep 01, 2018] FaceDetection model replaced [Mobilenet] (graph.fullfacedetection / graph.shortfacedetection) [Sep 01, 2018] Added support for MultiGraph and FaceDetection mode [MultiStickSSDwithRealSense.py] [Sep 04, 2018] Performance measurement result with 5 sticks is posted [Sep 08, 2018] To prevent thermal runaway, simple clustering function of stick was implemented. [Sep 16, 2018] 【Experimental】 Added Semantic Segmentation model [Tensorflow-UNet] (semanticsegmentation_frozen_person.pb) [Sep 20, 2018] 【Experimental】 Updated Semantic Segmentation model [Tensorflow-UNet] [Oct 07, 2018] 【Experimental】 Added Semantic Segmentation model [caffe-jacinto] (cityscapes5_jsegnet21v2_iter_60000.caffemodel) [Oct 10, 2018] Corresponds to NCSDK 2.08.01 [Oct 12, 2018] 【Experimental】 Added Semantic Segmentation model [Tensorflow-ENet] (semanticsegmentation_enet.pb) https://github.com/PINTO0309/TensorFlow-ENet.git [Dec 22, 2018] Only "USB Camera + single thread mode" correspondence with NCS 2 is completed [Dec 31, 2018] "USB Camera + MultiStick + MultiProcess mode" correspondence with NCS2 is completed [Jan 04, 2019] Tune performance four times. MultiStickSSDwithRealSense_OpenVINO_NCS2.py [Feb 01, 2019] Pull request merged. Fix Typo. Thanks, nguyen-alexa!! [Feb 09, 2019] Corresponds to PiCamera. [Feb 10, 2019] Added support for SingleStickSSDwithRealSense_OpenVINO_NCS2.py [Feb 10, 2019] Firmware v5.9.13 -> v5.10.6, RealSenseSDK v2.13.0 -> v2.16.5 [May 01, 2019] Corresponds to OpenVINO 2019 R1.0.1 [Nov 12, 2019] Corresponds to OpenVINO 2019 R3.0 Motion imageRealSense Mode about 6.5 FPS （Detection + Synchronous screen drawing / SingleStickSSDwithRealSense.py）
【YouTube Movie】 https://youtu.be/77cV9fyqJ1w RealSense Mode about 25.0 FPS （Asynchronous screen drawing / MultiStickSSDwithRealSense.py）
However, the prediction rate is fairly low.(about 6.5 FPS) USB Camera Mode MultiStick x4 Boosted 16.0 FPS+ （Asynchronous screen drawing / MultiStickSSDwithRealSense.py）
【YouTube Movie】 https://youtu.be/GedDpAc0JyQ RealSense Mode SingleStick about 5.0 FPS（Transparent background in real time / Asynchronous screen drawing / MultiStickSSDwithRealSense.py）
【YouTube Movie】 https://youtu.be/ApyX-mN_dYA USB Camera Mode MultiStick x3 Boosted （Asynchronous screen drawing / MultiGraph(SSD+FaceDetection) / FaceDetection / MultiStickSSDwithRealSense.py）
【YouTube Movie】 https://youtu.be/fQZpuD8mWok Simple clustering function (MultiStick / MultiCluster / Cluster switch cycle / Cluster switch temperature)
USB Camera Mode NCS2 SingleStick + RaspberryPi3（Synchronous screen drawing / SingleStickSSDwithUSBCamera_OpenVINO_NCS2.py）
【YouTube Movie】 https://youtu.be/GJNkX-ZBuC8 USB Camera Mode NCS2 SingleStick + Core i7（Synchronous screen drawing / SingleStickSSDwithUSBCamera_OpenVINO_NCS2.py）
【YouTube Movie】 https://youtu.be/1ogge90EuqI USB Camera Mode NCS2 x 1 Stick + Core i7（Asynchronous screen drawing / MultiStickSSDwithRealSense_OpenVINO_NCS2.py）
【YouTube Movie】 https://youtu.be/Nx_rVDgT8uY $ python3 MultiStickSSDwithRealSense_OpenVINO_NCS2.py -mod 1 -numncs 1
USB Camera Mode NCS2 x 1 Stick + RaspberryPi3（Asynchronous screen drawing / MultiStickSSDwithRealSense_OpenVINO_NCS2.py）
【YouTube Movie】 https://youtu.be/Xj2rw_5GwlI $ python3 MultiStickSSDwithRealSense_OpenVINO_NCS2.py -mod 1 -numncs 1
USB Camera Mode NCS2 x 1 Stick + LattePanda Alpha（Asynchronous screen drawing / MultiStickSSDwithRealSense_OpenVINO_NCS2.py）[48 FPS]
https://twitter.com/PINTO03091/status/1081575747314057219 PiCamera Mode NCS2 x 1 Stick + RaspberryPi3（Asynchronous screen drawing / MultiStickSSDwithPiCamera_OpenVINO_NCS2.py）
$ python3 MultiStickSSDwithPiCamera_OpenVINO_NCS2.py
USB Camera Mode NCS2 x 1 Stick + RaspberryPi4（Asynchronous screen drawing / MultiStickSSDwithUSBCamera_OpenVINO_NCS2.py）
Motion diagram of MultiStick
Environment1．RaspberryPi3 + Raspbian Stretch (USB2.0 Port) or RaspberryPi3 + Ubuntu Mate or PC + Ubuntu16.04 Firmware update with Windows 10 PC1．ZIP 2 types (1) Firmware update tool for Windows 10 (2) The latest firmware bin file Download and decompress Work with RaspberryPi3 (or PC + Ubuntu16.04 / RaspberryPi + Ubuntu Mate)1.NCSDK ver (Not compatible with NCS2)Use of Virtualbox is not strongly recommended 1.Execute the following $ sudo apt update;sudo apt upgrade
$ sudo reboot2.Extend the SWAP area (RaspberryPi+Raspbian Stretch / RaspberryPi+Ubuntu Mate Only) $ sudo nano /etc/dphys-swapfile
CONF_SWAPSIZE=2048
$ sudo /etc/init.d/dphys-swapfile restart;swapon -s3.Install NSCDK $ sudo apt install python-pip python3-pip
$ sudo pip3 install --upgrade pip
$ sudo pip2 install --upgrade pip
$ cd ~/ncsdk
$ make uninstall
$ cd ~;rm -r -f ncsdk
#=====================================================================================================
# [Oct 10, 2018] NCSDK 2.08.01 , Tensorflow 1.9.0
$ git clone -b ncsdk2 http://github.com/Movidius/ncsdk
#=====================================================================================================
$ cd ncsdk
$ nano ncsdk.conf
#MAKE_NJOBS=1
↓
MAKE_NJOBS=1
$ sudo apt install cython
$ sudo -H pip3 install cython
$ sudo -H pip3 install numpy
$ sudo -H pip3 install pillow
$ make install
$ cd ~
$ wget https://github.com/google/protobuf/releases/download/v3.5.1/protobuf-all-3.5.1.tar.gz
$ tar -zxvf protobuf-all-3.5.1.tar.gz
$ cd protobuf-3.5.1
$ ./configure
$ sudo make -j1
$ sudo make install
$ cd python
$ export LD_LIBRARY_PATH=../src/.libs
$ python3 setup.py build --cpp_implementation
$ python3 setup.py test --cpp_implementation
$ sudo python3 setup.py install --cpp_implementation
$ sudo ldconfig
$ protoc --version
# Before executing "make examples", insert Neural Compute Stick into the USB port of the device.
$ cd ~/ncsdk
$ make examples -j1【Reference】https://github.com/movidius/ncsdk 4.Update udev rule $ sudo apt install -y git libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
$ sudo apt install -y libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev
$ cd /etc/udev/rules.d/
$ sudo wget https://raw.githubusercontent.com/IntelRealSense/librealsense/master/config/99-realsense-libusb.rules
$ sudo udevadm control --reload-rules && udevadm trigger5.Upgrade to "cmake 3.11.4" $ cd ~
$ wget https://cmake.org/files/v3.11/cmake-3.11.4.tar.gz
$ tar -zxvf cmake-3.11.4.tar.gz;rm cmake-3.11.4.tar.gz
$ cd cmake-3.11.4
$ ./configure --prefix=/home/pi/cmake-3.11.4
$ make -j1
$ sudo make install
$ export PATH=/home/pi/cmake-3.11.4/bin:$PATH
$ source ~/.bashrc
$ cmake --version
cmake version 3.11.46.Register LD_LIBRARY_PATH $ nano ~/.bashrc
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
$ source ~/.bashrc7.Install TBB (Intel Threading Building Blocks) $ cd ~
$ wget https://github.com/PINTO0309/TBBonARMv7/raw/master/libtbb-dev_2018U2_armhf.deb
$ sudo dpkg -i ~/libtbb-dev_2018U2_armhf.deb
$ sudo ldconfig8.Uninstall old OpenCV (RaspberryPi Only) $ cd ~/opencv-3.x.x/build
$ sudo make uninstall
$ cd ~
$ rm -r -f opencv-3.x.x
$ rm -r -f opencv_contrib-3.x.x9.Build install "OpenCV 3.4.2" or Install by deb package. $ sudo apt update && sudo apt upgrade
$ sudo apt install build-essential cmake pkg-config libjpeg-dev libtiff5-dev \
libjasper-dev libavcodec-dev libavformat-dev libswscale-dev \
libv4l-dev libxvidcore-dev libx264-dev libgtk2.0-dev libgtk-3-dev \
libcanberra-gtk* libatlas-base-dev gfortran python2.7-dev python3-dev
$ cd ~
$ wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.4.2.zip
$ unzip opencv.zip;rm opencv.zip
$ wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.4.2.zip
$ unzip opencv_contrib.zip;rm opencv_contrib.zip
$ cd ~/opencv-3.4.2/;mkdir build;cd build
$ cmake -D CMAKE_CXX_FLAGS="-DTBB_USE_GCC_BUILTINS=1 -D__TBB_64BIT_ATOMICS=0" \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.4.2/modules \
-D BUILD_EXAMPLES=OFF \
-D PYTHON_DEFAULT_EXECUTABLE=$(which python3) \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D BUILD_opencv_python2=ON \
-D BUILD_opencv_python3=ON \
-D WITH_OPENCL=OFF \
-D WITH_OPENGL=ON \
-D WITH_TBB=ON \
-D BUILD_TBB=OFF \
-D WITH_CUDA=OFF \
-D ENABLE_NEON:BOOL=ON \
-D ENABLE_VFPV3=ON \
-D WITH_QT=OFF \
-D BUILD_TESTS=OFF ..
$ make -j1
$ sudo make install
$ sudo ldconfig9.2 Install by deb package (RaspberryPi Only) [I already activated VFPV3 and built it] $ cd ~
$ sudo apt autoremove libopencv3
$ wget https://github.com/PINTO0309/OpenCVonARMv7/raw/master/libopencv3_3.4.2-20180709.1_armhf.deb
$ sudo apt install -y ./libopencv3_3.4.2-20180709.1_armhf.deb
$ sudo ldconfig10.Install Intel® RealSense™ SDK 2.0 全部评论
专题导读
上一篇：hahwul/MobileHackersWeapons: Mobile Hacker's Weapons / A collection of cool ...发布时间：2022-08-30下一篇：Alfresco/alfresco-android-app发布时间：2022-08-30热门推荐
热门话题
 阅读排行榜
|
请发表评论