ricoh’s second stage machine vision: empowering digital ... · ricoh’s second stage machine...
Post on 22-Jun-2018
217 Views
Preview:
TRANSCRIPT
Ricoh’s Second Stage Machine Vision:
EMPOWERING DIGITAL WORKPLACES
Ricoh has been providing automation and other labor-saving systems in many areas using machine
vision. Our machine vision activities are now in their second stage and we are working on how we
can elevate workplaces into a space for knowledge creation. Our system autonomously generates
rules, understands the situation, presents optimal methods, and helps people at work to make
decisions quickly. Ricoh aims to establish a smarter workplace through collaboration between
intelligent machine vision and knowledge workers.
20/11/2017 Version 1.0.0
White
Paper
1 © 2017 Ricoh Company, Ltd.
Table of Contents
1. Innovating Workplaces to Enable the Knowledge Creation .................................... 2
2. Delivering Machine Vision to All Workplaces .............................................................. 3
3. Ricoh’s Second Stage Machine Vision Activities .................................................... 4
3.1. Second stage machine vision in social infrastructure .............................................. 5
・Road Surface Inspection System Based on Stereo Cameras and AI ................. 5
・Public infrastructure inspection systems using drones ............................................... 9
・Autonomous flights of drones with a 3D vision system using an ultrawide-angle
stereo camera........................................................................................................................................ 12
3.2. Second-stage machine vision in Factory Automation (FA) .................................. 15
・Automatic random picking system using a stereo camera...................................... 15
・Image recognition and analysis technology for visual inspection based on
machine learning ................................................................................................................................ 17
3.3. Second stage machine vision in logistics ................................................................... 19
・Flow monitoring using stereo camera ................................................................................ 19
4. Tackling Second Stage Machine Vision with Open Innovation .................................. 20
2 © 2017 Ricoh Company, Ltd.
1. Innovating Workplaces to Enable the Knowledge Creation
With the rapid progress of digital innovation, workplaces are about to face a great
change. Productivity has been improved, costs have been cut, and operations have
been accelerated as we know it. The mode of corporate organizations has changed
dramatically in just a few years, and so has the awareness of the people working
there.
Digital innovation has freed people from the constraints of time and space, bringing
opportunities to create new value. The waves of innovation that permeate offices now
reach the frontlines of many industries—manufacturing, logistics, retail, healthcare,
education, and more.
A technology accelerating this movement is artificial intelligence (AI), which has
gained huge attention recently. Particularly, deep learning based on the multi-layer
neural networks, which is one of the technologies1 of machine learning, has found
applications in a variety of examples, enabling many people to recognize AI as
something familiar. Expectations are rising for the dawn of the new era when
machines (systems) are given human-like intelligence.
As systems gain “smartness,” they will be able to do things that people have been
doing or could not do in the past. A typical example is autonomous car driving. The
technology not only reduces the burden on the driver but also increases safety, and
is especially valuable in that regard. Efforts are underway to equip lighting and air-
conditioners with sensors, enabling automatic control of energy consumption
according to the movement of people, thus creating a more comfortable
environment.
Workplace innovation, however, cannot be accomplished by merely introducing a
1 A scheme of finding characteristic patterns out of big data through iterative learning (statistical processing)
3 © 2017 Ricoh Company, Ltd.
system that features “smartness.” Innovation requires the knowledge creation,
including the likes of value judgments and task setting, which cannot be replaced
by AI no matter how excellent it is. Workplaces are reformed through the consistent
knowledge creation by the people working there, and evolve into places for
producing new value. The value of a “smart” system is determined by how much it
contributes to the knowledge creation by people.
People perform a series of processes unconsciously in their daily tasks. A system
must organically coordinate these processes to assist in the knowledge creation in
a workplace. Knowledge creation in a workplace requires a wide scope of
technologies that encompass input, processing, and output, as well as know-how
in formulating an easy-to-use smart solution system.
2. Delivering Machine Vision to All Workplaces
Ricoh has been nurturing technologies for optics, image processing, and electronic
devices over many years. These technologies are integrated and can provide new
value in the areas of machine vision. Ricoh has already established a prominent
position in factory automation, (FA), automotive systems, security, distribution, and
more.
The concept is described in Ricoh’s Machine Vision white paper. That white paper
reveals Ricoh’s commitment to establishing “an intelligent technology that enables
machines to not only operate according to instructions from people but also to
quickly grasp the situation on behalf of a person and take appropriate action.”
Ricoh’s machine vision facilitates the visualization of things that are not otherwise
Input Output Processing
Collecting information by optimal devices
Analyzing and processing collected information and
converting it into intelligence
Providing results to systems and people in optimal form
Like sensory organs Like brain Like body response
4 © 2017 Ricoh Company, Ltd.
visible or automates a process that otherwise must depend upon human effort. With
machine vision, Ricoh will allow people to take part in activities that will be
increasingly productive and have higher added value.
Our commitment will require a “smart” system that autonomously processes the
entire scope of capture, analytics, and visualization. For Ricoh, this is the second
stage of machine vision. Activities have already begun to enable a system to derive
optimal judgment and action from vision-based information and to present it clearly.
The system will repeat learning on its own, thus improving the accuracy of the
information it presents.
Primary element technologies of second stage machine vision
Capture Analytics Visualize
Stereo camera (including
ultrawide-angle)
Multi-spectral camera
Extended depth-of-field
camera
Polarized camera
Spherical camera
Image processing
Voice processing
Natural language
processing
Data mining
Electronic devices
Printing
Display
Autonomous control
3D modeling
Second stage machine vision is not just an alternative to the human eye. It will
enhance people’s value judgments and task setting, enabling advanced decision-
making and reforming the workplace into a space for the knowledge creation.
Activities in the second stage have already begun, covering a wide range of areas
including social infrastructure, FA, logistics, and security. This white paper describes
the status quo of Ricoh’s second stage machine vision, and the value produced
from these activities.
3. Ricoh’s Second Stage Machine Vision Activities
Ricoh believes that its activities in this second stage of machine vision will lead to
smarter workplaces. The second stage has a vast range of applications, enabling
5 © 2017 Ricoh Company, Ltd.
solutions in a variety of fields. This document provides examples2 in the areas of
social infrastructure, FA and logistics.
3.1. Second stage machine vision in social infrastructure
・Road Surface Inspection System Based on Stereo Cameras and AI
Roads enable traffic of cars and pedestrians, which is an important role as part of
the infrastructure for social and economic activities and local lives. Yet road surfaces
suffer constant deterioration and damage due to external forces such as heavy
vehicle loads. If the roads are left unattended, the deterioration can result in
unwanted incidents, such as a car crash due to the poor surface condition affecting
the steering. To prevent such incidents, road surfaces must be inspected to find
deterioration at an early stage and provide necessary repairs accordingly.
Road surface conditions are evaluated according to three factors: the rate of cracks,
the depth of ruts, and flatness (lengthwise bumpiness). Generally, the inspection
requires a huge specialist vehicle provided with laser measuring instruments and
digital cameras. The specialist vehicle, however, is very costly—it is difficult to
increase their number, and thus to increase the inspection frequency or expand the
range. The vehicle cannot enter narrow roads, so inspection of community roads
has been deficient.
2 Some are under development.
Three factors of road surface conditions: rate of cracks, depth of ruts, and flatness
6 © 2017 Ricoh Company, Ltd.
Ricoh’s Road Surface Inspection System features multiple stereo cameras; yet it is
compact enough to be mounted on a standard motor vehicle. Unlike the
conventional specialist vehicles, it can inspect community roads. The system allows
automation of the work processes from capturing images by stereo cameras to
creating inspection records, reducing the burden of inspection tasks.
The greatest feature of the system
is that the three factors of the road
surface conditions can be
inspected with only the stereo
cameras3. The rate of cracks, for
instance, can be automatically
determined. The cameras capture
the images of the road surfaces
and the system creates two-
dimensional images by combining
feature points.
The automatic determination is based on machine learning4, which is a type of AI.
The created two-dimensional images are processed on a 50-cm grid, and the crack
level (the number of cracks) is automatically determined by AI. A machine learning
model has replaced human sight in the visual determination process, significantly
improving efficiency and accuracy.
3 Click here to view a movie of the road surface inspection using stereo cameras. 4 Machine learning: Ricoh sets machine learning as one of the element development themes for enhancing image recognition technologies and has been studying it and developing its applications.
Road surface monitoring system installed on a standard motor vehicle
7 © 2017 Ricoh Company, Ltd.
Where tires pass, ruts are caused by friction on the paved surface, sinking of the
roadbed, and transformation of the paving material, for instance. To measure their
depths, Ricoh’s inspection system uses multiple stereo cameras to scan the road
surface, automatically combines the scanned images along the width, and reveals
the three-dimensional shape of the cross section of the road every 20 meters.
Flatness of the paved surface influences the comfort of the drivers and passengers
on traveling vehicles. Ricoh’s inspection system continuously measures three-
dimensional shapes in the forward direction by capturing images by the stereo
cameras and combining them. This eliminates the need for different measuring
instruments or devices for measurement. We need only have the car-mounted
stereo cameras scan the road surface and let the system measure the conditions.
After measurement, the three factors of the road surface conditions need to be
comprehensively evaluated to determine the need for repairs and the locations to
be given priority. The evaluation should be based on the Maintenance Control Index
(MCI). Ricoh’s inspection system calculates an MCI quickly as it can measure the
three factors on a single shooting drive.
The MCI is indispensable for the comprehensive evaluation of the three factors. In
the past, the management of community roads depended on visual measurement
because the specialist vehicles could not enter and numerical data such as the MCI
was not available. Ricoh’s inspection system solves this problem.
Determining the rate of cracks by AI
8 © 2017 Ricoh Company, Ltd.
Ricoh’s inspection system visualizes the road surface conditions by mapping the
pavement conditions according to the measurement results. The mapped images
are colored according to the MCI values, allowing the need for repairs to be quickly
determined.
A demonstrative experiment of the inspection system started in June 2016 with
participation by the Ministry of Land, Infrastructure and Transport, Akita Prefecture,
Semboku City, and Ricoh. A general-use vehicle was used to measure the road
surface conditions for three kilometers of national, prefectural, and municipal roads
in Semboku City, Akita Prefecture. The measurement was done twice, before the
snow season (November 2016) and after the snow season (March 2017). The
inspection system compared well with the specialist vehicle regarding MCI values
for the three factors of road surface conditions. For the rate of cracks, high accuracy
comparable to visual determination was attained through machine learning of many
data samples.
In addition, the demonstrative experiment confirmed how the paved surfaces are
affected by snow, ice, snow clearing by snowplows, and snow-melting agents. The
Road surface condition mapping (example): Measured data is colored according to MCI
values.
9 © 2017 Ricoh Company, Ltd.
experiment revealed the need to grab changes regularly over time, for instance,
and demonstrated the effectiveness of the measurement by a general-use vehicle.
The results of the demonstrative experiment and the issues found were reported to
the mayor of Semboku City on August 4, 2017, at the final debriefing meeting of the
Consortium for the Demonstrative Experiment of the Road Surface Condition
Inspection System*. Ricoh will continue to overcome technical issues and will strive
to develop practical inspection systems that are accurate and easy to use. Ricoh is
committed to implementing safe and secure road maintenance and management.
*Consortium for the Demonstrative Experiment of the Road Surface Condition
Inspection System: A joint project unit established by the Ministry of Land,
Infrastructure and Transport, Akita Prefecture, Semboku City, and Ricoh for the
demonstrative experiment of the road condition inspection system.
・Public infrastructure inspection systems using drones
The aging social infrastructure has recently become a major problem for Japan’s
national and local governments. Many of the highways, bridges, tunnels, and
buildings constructed in and after the high economic growth period (1960’s) have
exceeded their service life of 50 years and need maintenance and repair. The
Cabinet Office has been conducting a national project called the Strategic
Innovation Creation Program (SIP). Under the theme of Infrastructure Maintenance,
Renovation, and Management Technologies (supervised by NEDO: New Energy
and Industrial Technology Development Organization), the Cabinet Office is
promoting development of new technologies for the maintenance, renovation, and
management of aged infrastructure.
10 © 2017 Ricoh Company, Ltd.
Supported by the program, Ricoh has been
developing bridge inspection system with drones5
that assist with close proximity visual inspection of
bridges and systems to develop inspection reports.
This has been done jointly with Tohoku University,
Chiyoda Engineering Consultants, Japan
AeroSpace Technology Foundation, and Tokyu
Construction. It is an attempt to reduce the burden
of inspection work on individuals; a camera is
mounted on a drone to photograph items that require inspection. According to a
2014 survey by the Road Bureau of the Ministry of Land, Infrastructure, Transport
and Tourism (MLIT), Japan has about 700,000 bridges longer than two meters, 20%
of which have been in service for more than 50 years. The percentage will continue
to rise. In July 2014, MLIT started to impose obligations on local governments to
inspect bridges every five years (MLIT Ordinance No. 39). In the implementation of
the ordinance, however, both the dramatic increase in workload as well as the
financial burden must be overcome.
Bridge maintenance and inspection processes are largely classified into field work
and office work. First, the inspection staff conduct visual inspections on-site, chalk
the damaged points, take pictures, and record findings in a field book. They then
return to the office, summarize the data according to the records in the field book,
and generate a detailed survey report on the damage, attaching photographs and
sketches. The workload is burdensome, and report results can vary depending on
the person.
Bridge inspections are often more difficult due to both location and structure. When
a bridge is difficult to access by an inspector, specialist inspection vehicles and
5 When a drone-mounted camera is used to inspect a bridge, the distance between the subject and the drone needs to be kept constant. The drone must enter a narrow space while avoiding beams and attached objects (such as cables, lights, and water pipes). So the drone is contained in a spherical frame about one meter in diameter; the frame comes in contact with the subject, keeping the drone at a distance and preventing direct collision. It was developed by Tadokoro Laboratory, Tohoku University.
Inspecting a bridge with a spherical-shell drone
11 © 2017 Ricoh Company, Ltd.
large-scale scaffolds are often required. Besides the inspection staff, traffic control
staff may also be required depending on the situation. In these instances, it may
also be necessary to address the influence of traffic control on the lives and
economy of people in the surrounding neighborhood. Drones can be a solution to
this issue. Drones with a photography equipment mounted on them can reduce the
required number of people, cut costs, and ensure greater safety.
Additionally, Ricoh’s bridge inspection solution greatly reduces the workload in
office work. The latest image processing technologies help eliminate the
troublesome and time-consuming tasks of capturing data from damaged locations
and generating reports. From photographs (close-up images6), a panoramic view of
the entire bridge is automatically reconstructed in three dimensions (3D), and a
6 When inspecting a bridge, a spherical-shell drone performs a flyby across the inspection area to take close-up photos of all inspection points.
Bridge inspection processes
12 © 2017 Ricoh Company, Ltd.
development image is generated to provide a bird’s-eye view of the points to be
inspected. The system makes it easy to determine (detect) the locations and
seriousness of any damage. In addition, Ricoh provides a document generation
support tool for summarizing the results in a report, conforming to the inspection
procedure recommended by MLIT. Automating the report generation process will
not only reduce the workload but also eliminate the variation in report quality,
reducing the number of man-hours required dramatically.
Beyond bridges, drones within a spherical shell can be safely used where
maintenance and inspection by individuals poses a risk e.g. public structures like
tunnels, super skyscrapers, and plant facilities. Ricoh will continue to develop the
automation of the report generation process, including the efficient identification of
damage that occurs in many different forms. Our goal is to provide a total solution
for infrastructure inspection.
・Autonomous flights of drones with a 3D vision system using an ultrawide-
angle stereo camera
Drones are expanding their applications in business areas thanks to rapid
improvements e.g. increased structural reliability, higher environmental resistance,
and enhanced navigation precision. As more and more drones are put into practical
use, people expect them to fly beyond the boundaries of conventional physical
restrictions. Demand has arisen for drones to undertake a stable flight outside the
Extracting damage points from photographed images and creating a survey report
13 © 2017 Ricoh Company, Ltd.
scope of the operator’s view or without access to global positioning systems (GPS).
Currently, most business-use drones require operation by an expert operator.
However with the expansion of applications, operator shortage is anticipated. This
is another reason why autonomous flight capability is required.
GPS does not help in complex topography because errors in positioning can be in
the order of meters. When drones are used inside a building or tunnel, GPS signals
are unstable or unavailable in the first place. Even in an open space, GPS
navigation is unsuitable for flight due to electromagnetic interference.
Through joint research with the University of Tokyo and Blue Innovation, Ricoh has
solved these issues and enabled autonomous flights and the avoidance of
obstacles by drones based on a 3D vision system. Combined with an inertial
measurement unit (IMU7), our proprietary 3D vision system, equipped with an
ultrawide-angle stereo camera, allows a drone to avoid obstacles autonomously
and fly stably without an operator. Through coordination between a wide-view “eye”
that captures the surrounding conditions and the “semicircular canals” that produce
the sense of balance during flight, the drone autonomously flies in a non-GPS
environment, accurately tracing the path predetermined by the user.
For the autonomous flight system, the University of Tokyo has developed flight
controls, and Ricoh has developed the 3D vision system8. The drone knows where
it is using the ultrawide-angle stereo camera of the 3D vision system, measuring its
distance to the surrounding objects in a wide view and forecasting its movement in
reference to characteristic points and their relative distances. This technology has
enabled stable autonomous flights where GPS signals are unavailable. The wide
viewing angle of the camera allows the drone to keep the characteristic points in
sight even when its attitude changes, thus forecasting its own movement stably.
7 A sensor that detects angular velocity (gyro) and acceleration 8 RICOH 3D Vision System × Drone (Movie here)
14 © 2017 Ricoh Company, Ltd.
Although the path is predetermined by the user, unexpected obstacles may exist on
the path. The 3D vision system generates a 3D map around the flight path
concurrently with movement forecast, so the drone can detect and automatically
avoid unexpected obstacles.
The 3D vision system is unique in that it handles a series of processes in real time9
inside the camera unit, from photography through to positioning forecasting and
generating 3D maps. This is a new technology produced by the fusion of excellent
optical design technology, advanced image processing technology, and elaborate
systems control.
Ricoh’s 3D vision system allows drones to be used without an operator both indoors
and outdoors. It autonomously avoids obstacles whilst flying. Ricoh will continue to
enhance the smartness of the 3D vision system, expanding the possibilities of drone
applications.
9 Fast processing is attained by implementing multiple processes on an FPGA (Field-Programmable Gate Array) package, an integrated circuit configurable to specific design objectives. The processes include image processing, such as characteristic point extraction, and parallax operation for the stereo camera.
Autonomous obstacle avoidance based on 3D vision system
15 © 2017 Ricoh Company, Ltd.
3.2. Second-stage machine vision in Factory Automation (FA)
・Automatic random picking system using a stereo camera
Ricoh has developed a fast and precise stereo camera primarily for industrial
applications. A stereo camera can capture the depth of a subject. Developing it
requires advanced calibration technology, parallax operation technology 10 ,
precision parts mounting technology, and more. Quality requirements are acute in
industrial fields, and developing a module for them requires particularly high skills
and ample experience.
Since the latter half of the 1970’s, Ricoh has been enhancing automation of its own
production facilities using FA and accumulating technical know-how in this area. We
have shared some of the results outside the Company e.g. machine vision, and
have collaborated with other FA manufacturers. Ricoh has been contributing to the
improvement of office productivity (Office Automation - OA) with multi-function
printers (MFPs) and single-function printers and, at the same time, has attained
many achievements in FA.
FA is prevalent in manufacturing, but some processes still depend on human labor.
Random picking is one of them. Parts that are randomly placed on a rack need to
be individually transferred to a part feeder or an assembly line. Their conditions
must be quickly determined while they are on the rack so that they can be picked
up efficiently and properly arranged.
Conventional automatic picking systems used multiple devices and robots, resulting
in poor space efficiency and high cost. Many of their cameras were 2D—they had
a low recognition rate and were not reliable enough to replace human operators.
10 Calibration is the adjustment for detecting and measuring subjects to be photographed, and parallax operation is the processing to recognize the positions and shapes of subjects to be photographed.
16 © 2017 Ricoh Company, Ltd.
Then came a projector-based 3D vision system, but it was too large and slow11 to
readily fit on production lines.
The picking systems based on the RICOH SV-M-S1 industrial-use stereo camera,
is the solution to this difficult issue. With the stereo camera capturing the subject in
real time in 3D, the picking systems can quickly recognize the shapes and
orientation of parts that conventional cameras cannot. Collaborating with a robot
manufacturer that has worked with us before, Ricoh has developed the picking robot
into a system that maximizes the use of the fast and precise stereo camera. Before
implementation, Ricoh Industrial Solutions, an experienced FA systems integrator
within the Ricoh Group, can now propose an effective system from the viewpoint of
the optimization of the whole process in collaboration with the customer. The system
has already been introduced to the production lines of multiple customers,
contributing to improved production efficiency.
At this dawn of the IoT age, the functions
required for the picking system are about to
change. The movement for PLM (Product
Lifecycle Management), a concept for
centrally managing the entire lifecycle of a
product, has become active internationally.
The conventional system based on batch
processing on the part feeder is incapable of
answering the needs sufficiently. Ricoh’s random picking system uses a stereo
camera to recognize individual parts, and respond to such requirements flexibly. By
developing “smart” system technologies that solve the issues of FA and providing
customer-oriented solutions, Ricoh will continue to contribute to production
innovation.
11 The projector-based system uses the phase shift method for 3D measurement, which requires extra processes: pattern projection on the projector and dot data processing on the camera.
Robot picking system
17 © 2017 Ricoh Company, Ltd.
・Image recognition and analysis technology for visual inspection based
on machine learning
The parts inspection process is considered the “last hurdle” in implementing FA in
production processes. Thanks to the recent improvements in photography
technology and image recognition technology, the applications of machine-vision-
based visual inspection systems have expanded. Nevertheless, their capabilities
have not been fully exploited. The delay is the difficulty in telling the difference
between good and defective products. In addition to damage and flaws, many
factors are included in making a comprehensive judgment based on a variety of
conditions. When the boundary between the good and defective products is
ambiguous, the judgment must be based on knowledge and experience.
Still, results tend to vary when inspections are based on human labor. Depending
on the parts, inspection tasks can burden the inspection staff (excessive use of
eyesight and transfer of heavy objects). Automation of the inspection processes is
a must in improving production efficiency, reducing costs, and effectively using staff
(reassigning people to higher added value tasks). To provide a solution in this area,
Ricoh has been conducting research and development into Image Recognition and
Analysis Technology for Visual Inspection based on machine learning.
Ricoh has been studying machine learning as an element technology for image
recognition. One result is anomaly detection based on a semi-supervised learning
method. This method lets the machine learn correct values (labels) only, so that
anything that contains any other data is detected as an anomaly. Accuracy
increases with the iteration of judgment (iterative learning), and the data quantity
(the number of sample images) can be smaller than the generally used supervised
learning (based on data sets in which all data items are labeled).
Semi-Supervised learning can be enhanced by applying image processing
18 © 2017 Ricoh Company, Ltd.
algorithms12 that calculate characteristic quantities from the photographed data of
the parts. Using the enhanced method, Ricoh is developing a precision visual
inspection system that automatically detects defective products. Our ample know-
how in image recognition contributes to the improved precision of machine learning.
In general, defective parts are far fewer than good parts, so it is unrealistic to
prepare learning samples of all defect types. Thus, the semi-supervised learning
method is suitable because it is based only on the characteristic data of good
products. This method can detect unknown defects, and has high precision in
inspecting parts where the shapes of good products vary greatly (parts in the
preliminary processing stage, for instance).
The image recognition and analysis technology for visual inspection based on semi-
supervised learning received an award for excellence over two consecutive years
(2014 and 2015) at the Visual Inspection
Algorithm Contest, sponsored by the
Technical Committee on Industrial
Application of Image Processing and the
Japan Society for Precision Engineering.
Machine learning can be applied to many
fields. Its value is increasing at this dawn
of the Big Data age. Ricoh will continue to
improve the learning precision and
12 SIFT (Scale-Invariant Feature Transform) to extract characteristic points from images (boundaries and edges) and SURF (Speeded-Up Robust Features) to calculate characteristic quantities (information robust against rotation, scale, and illumination)
Function blocks of visual inspection system
Sample detection result
(Visual Inspection Algorithm Contest 2014)
19 © 2017 Ricoh Company, Ltd.
processing speed, aiming to take image recognition and analysis technology to an
advanced level that compares to human judgment.
3.3. Second stage machine vision in logistics
・Flow monitoring using stereo camera
The advancement in machine vision has enabled mass image information to be
used in a variety of fields, but how to obtain useful information is another problem.
It can be solved by the visualization technology, which analyzes the mass
information as necessary and presents the results as visual information. Particularly,
the technology to visualize dynamic information in real time assists quick decision-
making, helping people discover things that are difficult to recognize in the visual
inspection of objects or the investigation of static images.
In a warehouse, for instance, there is a need to observe the motion (flow) of people
after changing the shelf layout to see if the change was appropriate. Ricoh has
developed a flow monitoring system using stereo cameras, and uses it to optimize
the layout in the logistics warehouse of Ricoh Europe SCM B.V. (Bergen op Zoom,
The Netherlands).
Visualizing the flow in a warehouse (heat map)
20 © 2017 Ricoh Company, Ltd.
The flow monitoring system has multiple stereo cameras to monitor the conditions
of the transport work in the warehouse in 3D in real time, and displays the results
visually. The information on the workflow—where people or loads tend to
concentrate, for instance—is classified and shown in shades of different colors,
allowing people to grasp the situation at a glance. The obtained information can be
analyzed from a range of viewpoints to figure out a layout that maximizes efficiency
and minimizes workload.
In the past, flow monitoring was done as fixed-point observation by a video camera.
The fixed-point method required a long time to review, and things tended to be
overlooked when the recording was fast-forwarded. The stereo cameras featured
in Ricoh’s flow monitoring system precisely capture and visualize the information,
such as positions and flows, in 3D, enabling quick and appropriate decision-making.
The visualized images can be analyzed from multiple angles under different
conditions as necessary to determine the optimal layout.
The application of the flow monitoring system is not limited to warehouses. In an
office, for instance, it can be used in an ecology system for controlling energy
according to the flow of people. It can also be used in a security system that issues
an alarm upon detection of an anomaly in the incoming and outgoing flows of people.
Ricoh will continue to further expand the applications, developing the monitoring
system into a “smart” decision-making support system with excellent capabilities,
including automatic analysis of the results obtained from monitoring.
4. Tackling Second Stage Machine Vision with Open
Innovation
This document has introduced Ricoh’s products in the second stage of machine
vision, including those under development. With each one, the developers look
directly at how to increase the intelligent productivity of people, rather than merely
seeking automation or labor-saving. Ricoh’s attempts to make “smart” systems has
21 © 2017 Ricoh Company, Ltd.
just begun. Performance is being improved steadily through on-site verification and
other efforts. Ricoh has accumulated numerous technologies over many years, not
to mention the optical, image processing, and precision mounting technologies, and
they are about to bring a shift in the status quo as a destructive innovation13.
Yet Ricoh will not conduct these efforts on its own. As seen in the examples
introduced in this document, many of our development tasks are conducted through
collaboration with research facilities, with multiple companies, and even with our
customers. We are striving to create new value by bringing about innovation through
the fusion between Ricoh’s proprietary technologies and know-how and the
technologies, knowledge, and know-how of our partners. Keep your eyes open for
Ricoh’s second stage machine vision, which is based on open innovation to
establish smarter workplaces.
13 Using innovative technology to bring dramatic changes to the existing business models and industrial structures
22 © 2017 Ricoh Company, Ltd.
Revision history
Ver 1.0.0 2017/11/20 First edition
top related