ids nxt vision apps – cnn manager€¦ · ids nxt: vision apps – cnn manager 2.2 actions fig....
TRANSCRIPT
IDS NXT
IDS Imaging Development Systems GmbHDimbacher Str. 6-8
D-74182 Obersulm, GermanyT: +49 7134 96196-0
E: [email protected]: http://www.ids-nxt.com
Vision Apps – CNN manager
November 2019 2
IDS NXT: Vision Apps – CNN manager
Contents
1 Preface ............................................................................................................................................. 3
2 Vision app "CNN manager" ............................................................................................................... 4
2.1 Overview ...................................................................................................................................... 4
2.2 Actions ....................................................................................................................................... 5
2.3 Settings ...................................................................................................................................... 5
2.4 Selecting ROI .............................................................................................................................. 5
2.5 Files ........................................................................................................................................... 6
2.6 Result display .............................................................................................................................. 6
3 Interface description vision app "CNN manager" .............................................................................. 7
3.1 /cnnmanager ............................................................................................................................... 7
3.2 /cnnmanager/rois/evaluationarea .................................................................................................... 7
3.3 /cnnmanager/images/default ......................................................................................................... 8
3.4 /cnnmanager/resultsources/last ..................................................................................................... 9
3.5 /cnnmanager/configurables .......................................................................................................... 10
3.6 /cnnmanager/actions/deletecurrentpackage .................................................................................. 11
3.7 /cnnmanager/files/cnnfile/data ..................................................................................................... 12
3.8 Error codes ................................................................................................................................ 12
Index ................................................................................................................................................. 14
November 2019 3
IDS NXT: Vision Apps – CNN manager
1 Preface
Introduction
IDS Imaging Development Systems GmbH has taken every possible care in preparing this manual. We howeverassume no liability for the content, completeness or quality of the information contained therein. The content ofthis manual is regularly updated and adapted to reflect the current status of the software. We furthermore donot guarantee that this product will function without errors, even if the stated specifications are adhered to.
Under no circumstances can we guarantee that a particular objective can be achieved with the purchase of thisproduct.
Insofar as permitted under statutory regulations, we assume no liability for direct damage, indirect damage ordamages suffered by third parties resulting from the purchase of this product. In no event shall any liabilityexceed the purchase price of the product.
Please note that the content of this manual is neither part of any previous or existing agreement, promise,representation or legal relationship, nor an alteration or amendment thereof. All obligations of IDS ImagingDevelopment Systems GmbH result from the respective contract of sale, which also includes the complete andexclusively applicable warranty regulations. These contractual warranty regulations are neither extended norlimited by the information contained in this manual. Should you require further information on this product, orencounter specific problems that are not discussed in sufficient detail in the manual, please contact your localdealer or system installer.
Trademarks
The IDS logo is a registered trademark of IDS Imaging Development Systems GmbH,registered for U.S. (Reg.No. 4,513,138) and other countries.
IDS NXT and uEye are registered trademarks of IDS Imaging Development SystemsGmbH. Microsoft and Windows are trademarks or registered trademarks of Microsoft Corporation. All otherproducts or company names mentioned in this manual are used solely for purposes of identification ordescription and may be trademarks or registered trademarks of the respective owners.
Copyright
© IDS Imaging Development Systems GmbH. All rights reserved. This manual may not be reproduced,transmitted or translated to another language, either as a whole or in parts, without the prior written permissionof IDS Imaging Development Systems GmbH.
Status: November 2019
Contact
Visit our web site http://www.ids-nxt.com where you will find all the latest information about our software andhardware products. Please contact your local IDS distributors for first level support in your language. For a listof IDS distributors worldwide please go to our website http://www.ids-nxt.com.
Address IDS Imaging Development Systems GmbHDimbacher Str. 6-8D-74182 Obersulm, Germany
T +49 7134 96196-0
W http://www.ids-nxt.com
November 2019 4
IDS NXT: Vision Apps – CNN manager
2 Vision app "CNN manager"The CNN manager vision app allows you to upload trained neural networks ("CNN package") for IDS NXTcameras and execute them on the camera. The vision app provides the results of the image analysis for othervision apps.
Symbols and hints
This symbol indicates hints with useful information for better understanding and using featuresand functions.
This symbol indicates important warnings for product safety to prevent damage.
2.1 Overview
Fig. 1: CNN manager
1. Selection of the vision app
2. Information about the vision app
3. Visio app actions
4. Vision app settings
5. Selection of the image detail (ROI - region of interest)
6. Upload CNN file in vision app
7. Result of image analysis
8. Selection of the image view (live image, image detail)
At the first start of the vision app the included CNN packages will be installed. Therefore, itmay take a few seconds until the vision app is ready for use.
November 2019 5
IDS NXT: Vision Apps – CNN manager
2.2 Actions
Fig. 2: Delete CNN package
Deletes the current active CNN package on the device. When deleting a CNN package, the image acquisitionis temporarily stopped. After deleting, the first CNN package from the CNN package list is activated.
At least one CNN package must always be present. If there is only 1 CNN package on thedevice, it cannot be deleted.
2.3 Settings
Fig. 3: Select CNN package
· Switches between different CNN packages.
· Displays the information about the loaded CNN package.
· Selection of the CNN within the CNN package.
2.4 Selecting ROI
Fig. 4: Set ROI
Here, you define the image detail (ROI - region of interest):
· Sets the complete image as ROI.
· Sets a ROI. This ROI can be moved, enlarged or reduced with the mouse.
November 2019 6
IDS NXT: Vision Apps – CNN manager
2.5 Files
Fig. 5: Upload CNN package
Here, you upload a CNN file that you have previously trained in IDS NXT lighthouse.
The IDS NXT Cockpit displays an error message if the CNN package is large and the uploadtakes a long time. The upload is still performed, however. After uploading, the CNN package isinstalled, which causes the vision app to take a few seconds to run (as it did the first time thevision app was started).
2.6 Result display
Fig. 6: Result display
· Switching between the entire image and an image detail
· Inference: Shows the best 4 hits (top 4).
· Probability Shows the probability of the best 4 hits. The probability can be between 0 ... 1.
· Inference time in milliseconds
November 2019 7
IDS NXT: Vision Apps – CNN manager
3 Interface description vision app "CNN manager"The description of the REST interface here is limited to the parameters/values that are specific for the CNNmanager. A more detailed description of the IDS NXT REST interface can be found in the manual "IDS NXT -REST interface".
REST queries of IDS NXT vision apps are only possible if the corresponding vision app is activated, as they aregenerated dynamically during runtime.
Depending on the action and parameters, the used HTTP methods are color-coded and assigned to thepermission of IDS NXT user profiles (see manual "IDS NXT - Setup").
3.1 /cnnmanager
/vapps/cnnmanager
· GET
· OPTIONS
Transfers information to the vision app "CNN manager".
Response format
The content of the HTTP response is transferred in JSON format (application/json).
Return values
· Activated (Boolean): Activation status of the vision app
· Brief (String): Short description of the vision app
· Description (String): Long description of the vision app
· Manufacturer (String): Manufacturer of the vision app
· Name (String): Name of the vision app
· Status (String): Runtime status of the vision app "NotRunning", "Starting", "Running", "Crashed","FailedToStart", "TimedOut", "ReadError", "WriteError", "UnknownError", "Incompatible"
· Title (String): Vision app title
· Type (String): Vision app type
· Version (String): Version of the vision app
Error response
See Error codes
Sample code
curl -i -X GET -u admin:ids "http://<ip>/vapps/cnnmanager"
3.2 /cnnmanager/rois/evaluationarea
/vapps/cnnmanager/rois/evaluationarea
· GET
November 2019 8
IDS NXT: Vision Apps – CNN manager
· OPTIONS
Transfers the current ROI settings.
Response format
The content of the HTTP response is transferred in JSON format (application/json).
Return values
· Brief (String): Short description of the ROI
· Description (String): Description of the ROI
· Height (Integer): ROI height
· OffsetX (Integer): The X coordinate of the upper left corner of the ROI.
· OffsetY (Integer): The Y coordinate of the upper left corner of the ROI.
· Title (String): ROI title
· Width (Integer): ROI width
Error response
See Error codes
Sample code for GET
curl -X GET http://<ip>/vapps/cnnmanager/rois/evaluationarea --user admin:ids
Sample output
{ "Brief": "", "Description": "", "Height": 295, "OffsetX": 595, "OffsetY": 306, "Title": "", "Width": 379}
Sample code for PATCH
Setting the ROI
curl -X PATCH http://<ip>/vapps/cnnmanager/rois/evaluationarea --useradmin:ids -i --data 'Width=1280&Height=1024&OffsetX=0&OffsetY=0'
3.3 /cnnmanager/images/default
/vapps/cnnmanager/images/default
· GET
· OPTIONS
Transfers the current ROI as JPEG.
Response format
The Accept Token is used to select the format in which the result image is transferred.
· image/jpeg: Transfers a JPEG image.
November 2019 9
IDS NXT: Vision Apps – CNN manager
Parameters
The result image can be further specified using additional parameters.
· Quality (Integer): JPEG quality of the result image in percent if the content type “image/jpeg” is set: 0…100
Return values
· message-body: Image data
Error response
See Error codes
Sample code
curl -H Accept:image/jpeg -X GET http://<ip>/vapps/cnnmanager/images/default?quality=100 --user admin:ids --output roi.jpg
3.4 /cnnmanager/resultsources/last
/vapps/cnnmanager/resultsources/last
· GET
· OPTIONS
Returns the result of the last image analysis.
Response format
The content of the HTTP response is transferred in JSON format (application/json).
Return values
· inference (String): Shows the best 4 hits (top 4).
· inference_propability (Double): Shows the probability of the best 4 hits. The probability can be between0...1.
· inferencetime (Integer): Inference time in milliseconds
Error response
See Error codes
Sample code
curl -X GET http://<ip>/vapps/cnnmanager/resultsources/last --user admin:ids
Sample output
{ "inference": { "Top1": "snowplow", "Top2": "ski", "Top3": "hand_blower", "Top4": "space_shuttle" }, "inference_propability": { "Top1": "0.16", "Top2": "0.11", "Top3": "0.06", "Top4": "0.03", }, "inferencetime": {
November 2019 10
IDS NXT: Vision Apps – CNN manager
"Content": "102" }}
3.5 /cnnmanager/configurables
/vapps/cnnmanager/configurables
· GET
· OPTIONS
· PATCH
Shows the active CNN package.
Response format
The content of the HTTP response is transferred in JSON format (application/json).
Return values
· packages (String): CNN package
Error response
See Error codes
Sample code for GET
curl -X GET http://<ip>/vapps/cnnmanager/configurables --user admin:ids
Sample output for GET
{ "packages": "ImageNet",}
Sample code for PATCH
curl -X PATCH http://<ip>/vapps/cnnmanager/configurables -i --user admin:ids--data 'packages=<New package from OPTIONS-->Range>'
Sample code for OPTIONS
curl -X OPTIONS http://<ip>/vapps/cnnmanager/configurables -i --user admin:ids
Sample output for OPTIONS
{ "GET": { "application/json": { "Description": "Read VApp Configurables.", "Values": { "packages": { "Range": [ "Blumen160d2a5", "Blumen192d2a1", "DoodleTector", "ImageNet", "MultiArchTest", "imagenet_1.0.2" ], "Title": "CNN Packages",
November 2019 11
IDS NXT: Vision Apps – CNN manager
"Type": "Enum", "ZIndex": "0" }, "activecnn": { "Range": [ "imagenet 1.0.2", "inception_v3_1000", "resnet50", "squeezenet_1000" ], "Title": "Active CNN", "Type": "Enum", "ZIndex": "0" } } } }, "Objects": [ ], "PATCH": { "application/x-www-form-urlencoded": { "Description": "Modify VApp Configurables.", "Values": { "packages": { "Range": [ "Blumen160d2a5", "Blumen192d2a1", "DoodleTector", "ImageNet", "MultiArchTest", "imagenet_1.0.2" ], "Title": "CNN Packages", "Type": "Enum", "ZIndex": "0" }, "activecnn": { "Range": [ "imagenet 1.0.2", "inception_v3_1000", "resnet50", "squeezenet_1000" ], "Title": "Active CNN", "Type": "Enum", "ZIndex": "0" } } } }}
3.6 /cnnmanager/actions/deletecurrentpackage
/vapps/cnnmanager/actions/deletecurrentpackage
· GET
November 2019 12
IDS NXT: Vision Apps – CNN manager
· OPTIONS
· POST
Deletes the current active CNN package on the device. After deleting, the first CNN package from the CNNpackage list is activated.
At least one CNN package must always be present. If there is only one CNN package on thedevice, it cannot be deleted.
Response format
The content of the HTTP response is transferred in JSON format (application/json).
Return values
· HTTP code 200
Error response
See Error codes
Sample code
curl -X POST http://<ip>/vapps/cnnmanager/actions/deletecurrentpackage -i --user admin:ids
3.7 /cnnmanager/files/cnnfile/data
/vapps/cnnmanager/files/cnnfile/data
· GET
· OPTIONS
· PUT
Adds a new CNN.
Response format
The content of the HTTP response is transferred in JSON format (application/json).
Return values
· HTTP code 200
Error response
See Error codes
Sample code
curl http://<ip>/vapps/cnnmanager/files/cnnfile/data -i --user admin:ids -T /<path>/<file name>.cnn -H "Content-Type: application/octet-stream"
3.8 Error codes
For the REST queries, HTTP response codes are used as return values. The following HTTP codes are possiblereturn values of the REST queries, in general any HTTP code would be possible:
HTTP code Description
200 OK
November 2019 13
IDS NXT: Vision Apps – CNN manager
HTTP code Description
404 The requested object could not be found (e. g. wrong path)
405 The query must be made using other HTTP methods, e. g. OPTIONS instead of GET.
500 Internal device error
503 Result not available (e.g. no code read yet)
November 2019 14
IDS NXT: Vision Apps – CNN manager
- C -
CNN manager
GUI 4
CNN package
delete 5, 11
display 10
information 5
upload 6
- E -
Error code 12
- I -
Image
analysis 6
detail 5
ROI 5
Image analysis
last 9
result 9
- N -
Neural network
select 5
switch 5
- R -
REST
query 7
Result
display 6
query 12
ROI 5
JPEG 8
position 7
size 7
- V -
Vision app
CNN manager 4, 7