开源软件名称(OpenSource Name):ethz-asl/hierarchical_loc开源软件地址(OpenSource Url):https://github.com/ethz-asl/hierarchical_loc开源编程语言(OpenSource Language):Python 82.5%开源软件介绍(OpenSource Introduction):Hierarchical LocalizationThis repository contains the training and deployment code used in our paper Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization presented at CoRL 2018. This work introduces MobileNetVLAD, a mobile-friendly image retrieval deep neural network that significantly improves the performance of classical 6-DoF visual localization through a hierarchical search.
We introduce here two main features:
The modularity of our system allows to train a model and index a map on a powerful workstation while performing the retrieval on a mobile platform. Our code has thus been extensively tested on an NVIDIA Jetson TX2, widely used for robotics research.
DeploymentThe package relies on map primitives provided by maplab, but can be easily adapted to other SLAM frameworks. We thus do not release the code performing the local matching. The trained MobileNetVLAD is provided in InstallationBoth Ubuntu 14.04 and 16.04 are supported. First install the system packages required by maplab. Then setup the Catkin workspace: export ROS_VERSION=kinetic #(Ubuntu 16.04: kinetic, Ubuntu 14.04: indigo)
export CATKIN_WS=~/maplab_ws
mkdir -p $CATKIN_WS/src
cd $CATKIN_WS
catkin init
catkin config --merge-devel # Necessary for catkin_tools >= 0.4.
catkin config --extend /opt/ros/$ROS_VERSION
catkin config --cmake-args \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_TIMING=1 \
-DENABLE_STATISTICS=1 \
-DCMAKE_CXX_FLAGS="-fext-numeric-literals -msse3 -msse4.1 -msse4.2" \
-DCMAKE_CXX_STANDARD=14
cd src If you want to perform the inference on GPU (see the requirements of tensorflow_catkin), add: catkin config --append-args --cmake-args -DUSE_GPU=ON Finally clone the repository and build: git clone https://github.com/ethz-asl/hierarchical_loc.git --recursive
touch hierarchical_loc/catkin_dependencies/maplab_dependencies/3rd_party/eigen_catkin/CATKIN_IGNORE
touch hierarchical_loc/catkin_dependencies/maplab_dependencies/3rd_party/protobuf_catkin/CATKIN_IGNORE
cd $CATKIN_WS && catkin build global_loc Run the test examples: ./devel/lib/global_loc/test_inference
./devel/lib/global_loc/test_query_index IndexingGiven a VI map in ./devel/lib/global_loc/build_index \
--map_name <map_name> \
--model_name mobilenetvlad_depth-0.35 \
--proto_name <index_name.pb> As an example, we provide the Zurich map used in our paper. Several indexing options are available in place-retrieval.cc, such as subsampling or mission selection. RetrievalAn example of query is provided in test_query_index.cc. Descriptor indexes for the Zurich dataset are included in ./devel/lib/global_loc/time_query \
--map_name <map_name> \
--model_name mobilenetvlad_depth-0.35 \
--proto_name lindenhof_afternoon_aligned_mobilenet-d0.35.pb \
--query_mission f6837cac0168580aa8a66be7bbb20805 \
--use_pca --pca_dims 512 --max_num_queries 100 Use the same indexes to evaluate and visualize the retrieval: install retrievalnet, generate the Python protobuf interface, and refer to tango_evaluation.ipynb and tango_visualize_retrieval.ipynb. TrainingWe use distillation to compress the original NetVLAD model into a smaller MobileNetVLAD with mobile real-time inference capability. InstallationPython 3.5 is required. It is advised to run the following installation commands within a virtual environment. You will be prompted to provide the path to a data folder (subsequently referred as
Exporting the target descriptorsIf you wish to train MobileNetVLAD on the Google Landmarks dataset as done in our paper, you first need to download the index of images and then download the dataset itself with download_google_landmarks.py. The weights of the original NetVLAD model are provided by netvlad_tf_open and should be extracted in Finally export the descriptors of Google Landmarks:
Training MobileNetVLADExtract the MobileNet encoder pre-trained on ImageNet in python train.py config/mobilenetvlad_train_distill.yaml mobilenetvlad The training can be interrupted at any time using Exporting the model for deploymentpython export_model.py config/mobilenetvlad_train_distill.yaml mobilenetvlad will export the model in Evaluating on the NCLT datasetDownload the NCLT sequences in python export_descriptors.py configs/mobilenetvlad_export_nclt.yaml mobilenetvlad These can be used to evaluate and visualize the retrieval (see nclt_evaluation.ipynb and nclt_visualize_retrieval.ipynb). CitationPlease consider citing the corresponding publication if you use this work in an academic context:
|
2023-10-27
2022-08-15
2022-08-17
2022-09-23
2022-08-13
请发表评论