active learning methods for interactive image retrieval
TRANSCRIPT
1
Abstract
The project titled “ACTIVE LEARNING METHODS FOR
INTERACTIVE IMAGE RETRIEVAL” is designed using.NET
with Microsoft Visual Studio.Net 2005 as front end which works in .Net
framework version 2.0. The coding language used is C# .Net.
Active learning methods have been considered with increased interest
in the statistical learning community. Initially developed within a
classification framework, a lot of extensions are now being proposed to
handle multimedia applications. This paper provides algorithms within a
statistical framework to extend active learning for online content-based
image retrieval (CBIR). The classification framework is presented with
experiments to compare several powerful classification techniques in this
information retrieval context. Focusing on interactive methods, active
learning strategy is then described. The limitations of this approach for
CBIR are emphasized before presenting our new active selection process
RETIN. First, as any active method is sensitive to the boundary estimation
ACTIVE LEARNING METHODS FOR INTERACTIVE
IMAGE RETRIEVAL
2
between classes, the RETIN strategy carries out a boundary correction to
make the retrieval process more robust. Second, the criterion of
generalization error to optimize the active learning selection is modified to
better represent the CBIR objective of database ranking. Third, a batch
processing of images is proposed. Our strategy leads to a fast and efficient
active learning scheme to retrieve sets of online images (query concept).
Experiments on large databases show that the RETIN method performs well
in comparison to several other active strategies.
1. INTRODUCTION
Human interactive systems have attracted a lot of research interest in
recent years, especially for content- based image retrieval systems. Contrary
to the early systems, which focused on fully automatic strategies, recent
approaches have introduced human-computer interaction. In this paper, we
focus on the retrieval of concepts within a large image collection. We
assume that a user is looking for a set of images, the query concept, within a
database. The aim is to build a fast and efficient strategy to retrieve the
query Concept. In content-based image retrieval (CBIR), the search may be
initiated using a query as an example. The top rank similar images are then
presented to the user. Then, the interactive process allows the user to refine
his request as much as necessary in
3
a relevance feedback loop. Many kinds of interaction between the user and
the system have been proposed, but most of the time, user information
consists of binary labels indicating whether or not the image belongs to the
desired concept.
1.1 PROJECT DESCRIPTION
The project entitled as “Active Learning Methods for Interactive
Image Retrieval” developed using .NET using C#. Modules display as
follows.
RGB Projection
Image Utility
Comparable Image
Similarity Images
Result
4
MODULES DESCRIPTION:
1) RGB Projections:
The RGB color model is an additive color model in which red,
green, and blue light are added together in various ways to reproduce a
broad array of colors. The name of the model comes from the initials of the
three additive primary colors, red, green, and blue. The main purpose of the
RGB color model is for the sensing, representation, and display of images in
electronic systems, such as conventional photography.
In this module the RGB Projections is used to find the size of the image
vertically and horizontally.
2) Image Utility:
Whenever minimizing the error of classification is interesting for CBIR, this
criterion does not completely reflect the user satisfaction. Other utility
5
criteria Closer to this, such as precision, should provide more efficient
selections.
3) Comparable Image:
In this module a reselection technique to speed up the selection
process, which leads to a computational complexity negligible compared to
the size of the database for the whole active learning process. All these
components are integrated in our retrieval system, called RETIN and the
user gives new labels for images, and they are compared to the current
classification. If the user mostly gives relevant labels, the system should
propose new images for labeling around a higher rank to get more irrelevant
labels.
4) Similarity measure:
The results in terms of mean average precision according to the
training set size (we omit the KFD which gives results very close to
inductive SVMs) for both ANN and Corel databases. One can see that the
classification-based methods give the best results, showing the power of
statistical methods over geometrical approaches, like the one reported here
(similarity refinement method).
6
5) Result:
Finally, the image will take the relevant image what the user search.
One can see that we have selected concepts of different levels of
complexities. The performances go from few percentages of Mean average
precision to 89%. The concepts that are the most difficult to retrieve are very
small and/or have a much diversified visual content. The method which aims
at minimizing the error of generalization is the less efficient active learning
method. The most efficient method is the precision- oriented method.
Graph:
This module is used to determine relationships between the two Images. The
precision and recall values are measured by simulating retrieval scenario.
For each simulation, an image category is randomly chosen. Next, 100
images are selected using active learning and labeled according to the
chosen category. These labeled images are used to train a classifier, which
returns a ranking of the database. The average precision is then computed
7
using the ranking. These simulations are repeated 1000 times, and all values
are averaged to get the Mean average precision. Next, we repeat ten times
these simulations to get the mean and the standard deviation of the MAP
Input/Output:
The image will take the relevant image what the user search. one can see that
we have selected concepts of different levels of complexities. The
performances go from few percentages of Mean average precision to 89%.
The concepts that are the most difficult to retrieve are very small and/or have
a very diversified visual content
8
2. SYSTEM STUDY
2.1 FEASIBILITY STUDY
The feasibility of the project is analyzed in this phase and
business proposal is put forth with a very general plan for the project and
some cost estimates. During system analysis the feasibility study of the
proposed system is to be carried out. This is to ensure that the proposed
system is not a burden to the company. For feasibility analysis, some
understanding of the major requirements for the system is essential.
Three key considerations involved in the feasibility analysis are
ECONOMICAL FEASIBILITY
TECHNICAL FEASIBILITY
SOCIAL FEASIBILITY
9
ECONOMICAL FEASIBILITY
This study is carried out to check the economic impact that the
system will have on the organization. The amount of fund that the company
can pour into the research and development of the system is limited. The
expenditures must be justified. Thus the developed system as well within the
budget and this was achieved because most of the technologies used are
freely available. Only the customized products had to be purchased.
TECHNICAL FEASIBILITY
This study is carried out to check the technical feasibility, that is,
the technical requirements of the system. Any system developed must not
have a high demand on the available technical resources. This will lead to
high demands on the available technical resources. This will lead to high
10
demands being placed on the client. The developed system must have a
modest requirement, as only minimal or null changes are required for
implementing this system.
SOCIAL FEASIBILITY
The aspect of study is to check the level of acceptance of the system
by the user. This includes the process of training the user to use the system
efficiently. The user must not feel threatened by the system, instead must
accept it as a necessity. The level of acceptance by the users solely depends
on the methods that are employed to educate the user about the system and
to make him familiar with it. His level of confidence must be raised so that
he is also able to make some constructive criticism, which is welcomed, as
he is the final user of the system.
2.2 EXISTING SYSTEM
In the existing system the CBIR method faced a lot of disadvantage in case
of the image retrival. The following are the main disadvantage faced in case
of the medical field - Medical image description is an important problem in
content-based medical image retrieval. Hierarchical medical image semantic
features description model is proposed according to the main sources to get
11
semantic features currently. Hence we propose the new algorithim to over
come the existing system.
In existing system ,Images were first annotated with text and then searched
using a text-based approach from traditional database management systems.
2.3 PROPOSED SYSTEM
In case of the proposed system we use the following method to
improve the efficiency. They are as follows.
We implemented our models in a CBIR system for a specific
application domain, the retrieval of coats of arms. We implemented
altogether 19 features, including a color histogram, symmetry
features.
Content-based image retrieval, uses the visual contents of an image
such as color, shape, texture, and spatial layout to represent and index
the image
12
3. SYSTEM SPECIFICATION
3.1 HARDWARE CONFIGURATION
• SYSTEM : Pentium IV 2.4 GHz
• HARD DISK : 40 GB
• FLOPPY DRIVE : 1.44 MB
• MONITOR : 15 VGA colour
• MOUSE : Logitech.
• RAM : 256 MB
• KEYBOARD : 110 keys enhanced.
3.2 SOFTWARE CONFIGURATION
• Operating system :- Windows XP Professional
• Front End :- Microsoft Visual Studio .Net 2005
13
• Coding Language : - C# 2005.
4. LANGAUGE SPECIFICATION
4.1 FEATURES OF. NET
Microsoft .NET is a set of Microsoft software technologies for
rapidly building and integrating XML Web services, Microsoft Windows-
based applications, and Web solutions. The .NET Framework is a language-
neutral platform for writing programs that can easily and securely
interoperate. There’s no language barrier with .NET: there are numerous
languages available to the developer including Managed C++, C#, Visual
Basic and Java Script. The .NET framework provides the foundation for
components to interact seamlessly, whether locally or remotely on different
platforms. It standardizes common data types and communications protocols
so that components created in different languages can easily interoperate.
“.NET” is also the collective name given to various software
components built upon the .NET platform. These will be both products
14
(Visual Studio.NET and Windows.NET Server, for instance) and services
(like Passport, .NET My Services, and so on).
THE .NET FRAMEWORK
The .NET Framework has two main parts:
1. The Common Language Runtime (CLR).
2. A hierarchical set of class libraries.
The CLR is described as the “execution engine” of .NET. It provides the
environment within which programs run. The most important features are
Conversion from a low-level assembler-style language, called
Intermediate Language (IL), into code native to the platform
being executed on.
Memory management, notably including garbage collection.
Checking and enforcing security restrictions on the running
code.
15
Loading and executing programs, with version control and
other such features.
The following features of the .NET framework are also worth
description:
Managed Code
The code that targets .NET, and which contains certain extra
Information - “metadata” - to describe itself. Whilst both managed and
unmanaged code can run in the runtime, only managed code contains the
information that allows the CLR to guarantee, for instance, safe execution
and interoperability.
Managed Data
With Managed Code comes Managed Data. CLR provides
memory allocation and Deal location facilities, and garbage collection. Some
.NET languages use Managed Data by default, such as C#, Visual
Basic.NET and JScript.NET, whereas others, namely C++, do not. Targeting
CLR can, depending on the language you’re using, impose certain
constraints on the features available. As with managed and unmanaged code,
one can have both managed and unmanaged data in .NET applications - data
16
that doesn’t get garbage collected but instead is looked after by unmanaged
code.
Common Type System
The CLR uses something called the Common Type System (CTS) to
strictly enforce type-safety. This ensures that all classes are compatible with
each other, by describing types in a common way. CTS define how types
work within the runtime, which enables types in one language to
interoperate with types in another language, including cross-language
exception handling. As well as ensuring that types are only used in
appropriate ways, the runtime also ensures that code doesn’t attempt to
access memory that hasn’t been allocated to it.
Common Language Specification
The CLR provides built-in support for language interoperability. To
ensure that you can develop managed code that can be fully used by
developers using any programming language, a set of language features and
17
rules for using them called the Common Language Specification (CLS) has
been defined. Components that follow these rules and expose only CLS
features are considered CLS-compliant.
THE CLASS LIBRARY
.NET provides a single-rooted hierarchy of classes, containing
over 7000 types. The root of the namespace is called System; this contains
basic types like Byte, Double, Boolean, and String, as well as Object. All
objects derive from System. Object. As well as objects, there are value
types. Value types can be allocated on the stack, which can provide useful
flexibility. There are also efficient means of converting value types to object
types if and when necessary.
The set of classes is pretty comprehensive, providing
collections, file, screen, and network I/O, threading, and so on, as well as
XML and database connectivity.
18
The class library is subdivided into a number of sets (or
namespaces), each providing distinct areas of functionality, with
dependencies between the namespaces kept to a minimum.
LANGUAGES SUPPORTED BY .NET
The multi-language capability of the .NET Framework and
Visual Studio .NET enables developers to use their existing programming
skills to build all types of applications and XML Web services. The .NET
framework supports new versions of Microsoft’s old favorites Visual Basic
and C++ (as VB.NET and Managed C++), but there are also a number of
new additions to the family.
Visual Basic .NET has been updated to include many new and
improved language features that make it a powerful object-oriented
programming language. These features include inheritance, interfaces, and
overloading, among others. Visual Basic also now supports structured
exception handling, custom attributes and also supports multi-threading.
19
Visual Basic .NET is also CLS compliant, which means that
any CLS-compliant language can use the classes, objects, and components
you create in Visual Basic .NET.
Managed Extensions for C++ and attributed programming are
just some of the enhancements made to the C++ language. Managed
Extensions simplify the task of migrating existing C++ applications to the
new .NET Framework.
C# is Microsoft’s new language. It’s a C-style language that is
essentially “C++ for Rapid Application Development”. Unlike other
languages, its specification is just the grammar of the language. It has no
standard library of its own, and instead has been designed with the intention
of using the .NET libraries as its own.
Microsoft Visual J# .NET provides the easiest transition for
Java-language developers into the world of XML Web Services and
dramatically improves the interoperability of Java-language programs with
existing software written in a variety of other programming languages.
20
Active State has created Visual Perl and Visual Python, which
enable .NET-aware applications to be built in either Perl or Python. Both
products can be integrated into the Visual Studio .NET environment. Visual
Perl includes support for Active State’s Perl Dev Kit.
Other languages for which .NET compilers are available include
FORTRAN
COBOL
Eiffel
Fig1 .Net Framework
ASP.NET
XML WEB
SERVICES
Windows
Forms
21
Base Class Libraries
Common Language Runtime
Operating System
C#.NET is also compliant with CLS (Common Language Specification)
and supports structured exception handling. CLS is set of rules and
constructs that are supported by the CLR (Common Language Runtime).
CLR is the runtime environment provided by the .NET Framework; it
manages the execution of the code and also makes the development
process easier by providing services.
C#.NET is a CLS-compliant language. Any objects, classes, or
components that created in C#.NET can be used in any other CLS-
compliant language. In addition, we can use objects, classes, and
components created in other CLS-compliant languages in C#.NET .The
22
use of CLS ensures complete interoperability among applications,
regardless of the languages used to create the application.
CONSTRUCTORS AND DESTRUCTORS:
Constructors are used to initialize objects, whereas destructors are
used to destroy them. In other words, destructors are used to release the
resources allocated to the object. In C#.NET the sub finalize procedure is
available. The sub finalize procedure is used to complete the tasks that
must be performed when an object is destroyed. The sub finalize
procedure is called automatically when an object is destroyed. In
addition, the sub finalize procedure can be called only from the class it
belongs to or from derived classes.
GARBAGE COLLECTION
Garbage Collection is another new feature in C#.NET. The .NET
Framework monitors allocated resources, such as objects and variables.
In addition, the .NET Framework automatically releases memory for
reuse by destroying objects that are no longer in use.
In C#.NET, the garbage collector checks for the objects that are not
currently in use by applications. When the garbage collector comes
23
across an object that is marked for garbage collection, it releases the
memory occupied by the object.
OVERLOADING
Overloading is another feature in C#. Overloading enables us to define
multiple procedures with the same name, where each procedure has a
different set of arguments. Besides using overloading for procedures, we
can use it for constructors and properties in a class.
MULTITHREADING:
C#.NET also supports multithreading. An application that supports
multithreading can handle multiple tasks simultaneously, we can use
multithreading to decrease the time taken by an application to respond to
user interaction.
STRUCTURED EXCEPTION HANDLING
C#.NET supports structured handling, which enables us to
detect and remove errors at runtime. In C#.NET, we need to use
Try…Catch…Finally statements to create exception handlers. Using
24
Try…Catch…Finally statements, we can create robust and effective
exception handlers to improve the performance of our application.
THE .NET FRAMEWORK
The .NET Framework is a new computing platform that simplifies
application development in the highly distributed environment of the
Internet.
OBJECTIVES OF. NET FRAMEWORK
1. To provide a consistent object-oriented programming environment
whether object codes is stored and executed locally on Internet-
distributed, or executed remotely.
2. To provide a code-execution environment to minimizes software
deployment and guarantees safe execution of code.
3. Eliminates the performance problems.
There are different types of application, such as Windows-based
applications and Web-based applications.
25
5. SYSTEM DESIGN
5.1 MODULE DIAGRAM:
RGB Projections
Image Utility and
compare eg color
Computing Similarity
Measure
Average and graph values
26
5.2 UML Diagrams:
5.2.1 Use case Diagram:
27
5.2.2 Class Diagram:
28
5.2.3 Object Diagram:
RGB Projections
Image Utility and select
the image
Compare the source and
designation images
Find the average values
of the image Draw the graph
29
5.2.4 State Diagram:
30
5.2.5 Activity Diagram:
31
32
5.2.6 Sequence Diagram:
5.2.7 Collaboration Diagram:
33
5.2.8 Component Diagram:
34
5.3 E-R Diagram:
35
5.4 Data Flow Diagram:
Server System
Select File
Send the File
Client
Remove
process
Source
process
Source
Remove Destn
Processes
Zip
Unzip
User
create delete
36
5.5 Project Flow Diagram:
Check Image
RGB
Projections
Compare&
Utility
Draw the
Graph
Yes
No
New
Image
Similarity
Images
Exit
RGB
Projections
Image
Utility Compare
the Image
37
5.6 System Architecture:
38
6. SYSTEM TESTING AND MAINTENANCE
Testing is vital to the success of the system. System testing makes
a logical assumption that if all parts of the system are correct, the goal
will be successfully achieved. In the testing process we test the actual
system in an organization and gather errors from the new system operates
in full efficiency as stated. System testing is the stage of implementation,
which is aimed to ensuring that the system works accurately and
efficiently.
In the testing process we test the actual system in an organization
and gather errors from the new system and take initiatives to correct the
same. All the front-end and back-end connectivity are tested to be sure
that the new system operates in full efficiency as stated. System testing is
the stage of implementation, which is aimed at ensuring that the system
works accurately and efficiently.
The main objective of testing is to uncover errors from the
system. For the uncovering process we have to give proper input data to
the system. So we should have more conscious to give input data. It is
important to give correct inputs to efficient testing.
39
Testing is done for each module. After testing all the modules,
the modules are integrated and testing of the final system is done with the
test data, specially designed to show that the system will operate
successfully in all its aspects conditions. Thus the system testing is a
confirmation that all is correct and an opportunity to show the user that
the system works. Inadequate testing or non-testing leads to errors that
may appear few months later.
This will create two problems
Time delay between the cause and appearance of the problem.
The effect of the system errors on files and records within the system.
The purpose of the system testing is to consider all the likely
variations to which it will be suggested and push the system to its
limits.
The testing process focuses on logical intervals of the software
ensuring that all the statements have been tested and on the function
intervals (i.e.,) conducting tests to uncover errors and ensure that defined
inputs will produce actual results that agree with the required results.
Testing has to be done using the two common steps Unit testing and
Integration testing. In the project system testing is made as follows:
40
The procedure level testing is made first. By giving improper
inputs, the errors occurred are noted and eliminated. This is the final step
in system life cycle. Here we implement the tested error-free system into
real-life environment and make necessary changes, which runs in an
online fashion. Here system maintenance is done every months or year
based on company policies, and is checked for errors like runtime errors,
long run errors and other maintenances like table verification and reports.
6.1. UNIT TESTING
Unit testing verification efforts on the smallest unit of
software design, module. This is known as “Module Testing”. The
modules are tested separately. This testing is carried out during
programming stage itself. In these testing steps, each module is found to
be working satisfactorily as regard to the expected output from the
module.
6.2. INTEGRATION TESTING
Integration testing is a systematic technique for constructing
tests to uncover error associated within the interface. In the project, all
the modules are combined and then the entire programmer is tested as a
41
whole. In the integration-testing step, all the error uncovered is corrected
for the next testing steps.
7. SYSTEM IMPLEMENTATION
Implementation is the stage of the project when the theoretical design is
turned out into a working system. Thus it can be considered to be the most
critical stage in achieving a successful new system and in giving the user,
confidence that the new system will work and be effective.
The implementation stage involves careful planning, investigation of the
existing system and it’s constraints on implementation, designing of
methods to achieve changeover and evaluation of changeover methods.
Implementation is the process of converting a new system design into
operation. It is the phase that focuses on user training, site preparation and
file conversion for installing a candidate system. The important factor that
should be considered here is that the conversion should not disrupt the
functioning of the organization.
42
7.1 SCOPE FOR FUTURE ENHANCEMENTS
The project has covered almost all the requirements. Further
requirements and improvements can easily be done since the coding is
mainly structured or modular in nature. Improvements can be appended
by changing the existing modules or adding new modules.
8. CONCLUSION
In this paper, the RETIN active learning strategy for interactive learning in
content-based image retrieval context is presented. The classification
framework for CBIR is studied and powerful classification techniques for
information retrieval context are selected. After analyzing the limitation of
active learning strategies to the CBIR context, we introduce the general
RETIN active learning scheme, and the different components to deal with
this particular context. The main contributions concern the boundary
correction to make the retrieval process more robust, and secondly, the
introduction of a new criterion for image selection that better represents the
CBIR objective of database ranking. Other improvements, as batch
43
processing and speed-up process are proposed and discussed. Our strategy
leads to a fast and efficient active learning scheme to online retrieve query
concepts from a database. Experiments on large databases show that the
RETIN method gives very good results in comparison to several other active
strategies. The framework introduced in this article may be extended. We are
currently working on kernel functions for object classes retrieval, based on
bags of features: each image is no more represented by a single global
vector, but by a set of vectors. The implementation of such a kernel function
is fully compatible with the RETIN active learning scheme described in this
article, and the initial results are really encouraging.
44
9. LITERATURE REVIEW
There are alternative ways to avoid the scheduling latency issue described
above.
The main options are:
1) Bring the scheduler closer to the adapters;
2) Use provisioning (circuit switching);
3) Use a buffered switch core;
4) Eliminate the scheduler altogether.
Although one can attempt to locate the scheduler as close to the adapters as
possible, a certain distance determined by the system packaging limitations
and requirements will remain. Although the RTT can be minimized, the
fundamental problem of non-negligible RTTs remains valid. One can also do
without cell-level allocation and rely on provisioning to resolve contention.
Of course, this approach has several well-known drawbacks, such as a lack
of flexibility, inefficient use of resources, and long set-up times when a new
connection is needed, which make this approach unattractive for Parallel
computer interconnects. An alternative approach is to provide buffers in the
switch core and employ some form of link-level flow control (e.g.,credits) to
manage them. As long as an adapter has credits, it can send immediately
45
without having to go through a centralized scheduling process. However, as
optical buffering technology is currently neither practically nor
economically feasible and the key objective of OSMOSIS is to demonstrate
the use of optics, this is not an option.
The last alternative is the load-balanced Birkhoff–von-Neumann
switch, which eliminates the scheduler entirely. It consists of a distribution
and a routing stage, with a set of buffers at the inputs of the second stage.
Both stages are reconfigured periodically according to a sequence of
permutation matrices.The first stage uniformizes the traffic regardless of
destination, and the second stage performs the actual switching. Its main
advantage is that, despite being crossbar-based, no centralized scheduler is
required. Although this architecture has been shown to have 100%
throughput under a technical condition on the traffic, it incurs a worst-case
latency penalty of time slots: if a cell arrives at an empty VOQ just after the
VOQ had a service opportunity, it has to wait for exactly time slots for the
next opportunity. The mean latency penalty is time slots plus a minimum
transit latency intrinsically added by the second stage. Moreover,
missequencing can occur. This approach results in overall lower latency if
46
the total architecture-induced latency penalty can be expected to be less than
the control-path
latency In a traditional IQ switch. In the OSMOSIS system this is not the
case, hence we choose the centrally-scheduled architecture.
SPECULATIVE TRANSMISSION:
Our objective is to eliminate the control-path latency in the absence of
contention. To this end, we introduce a speculative transmission (STX)
scheme. The principle behind STX is related to that of the original ALOHA
and Ethernet protocols: Senders compete for a resource without prior
scheduling. If there is a collision, the losing sender(s) must retry their data
transmissions in a different time slot. However, the efficiency of ALOHA-
like protocols is very poor (18.4% for pure ALOHA and 36.8% for slotted
ALOHA) because under heavy load many collisions occur, reducing the
effective throughput. Therefore, we propose a novel method to combine
scheduled and speculative (non-scheduled) transmissions in a crossbar
switch. The objective is to achieve reduced latency at low utilization owing
to The speculative mode of operation and achieve high maximum
Throughput owing to the scheduled mode of operation.
47
We consider the presence of multiple receivers per output port, allowing up
to cells to arrive simultaneously. Although in OSMOSIS , we are interested
in the general case with here. We exploit this feature to improve the STX
success rate. The first receiver is for either a scheduled or a speculative cell.
The extra receivers can accommodate additional speculative cells.
Correspondingly, the STX arbitration can acknowledge multiple STX
requests per output per time slot. The following rules govern the design of
the STX scheme: Upon cell arrival, a request for scheduling (REQ) is issued
to the central scheduler. This request is processed by a bipartite graph
matching algorithm, and will eventually result in a corresponding scheduled
grant (GRT). An adapter is eligible to perform an STX in a given time slot if
it has no grant for a scheduled transmission in that time slot. Performing an
STX involves selecting a cell, sending it on the data path, and issuing a
corresponding speculative request (SRQ) on the control path. When multiple
cells collide, cells proceed and the remaining cells are dropped. If the
number of colliding cells is smaller than or equal to , all cells proceed. If
more than cells collide, a scheduled cell (if present) always proceeds.
Moreover, or (if a scheduled cell is present) randomly chosen speculative
cells proceed. Every cell may be speculatively transmitted at most once.
48
Every speculative cell remains stored in its input adapter until it is either
acknowledged as a successful STX or receives a scheduled grant. The
scheduler acknowledges every successful speculative cell to the sending
input by returning an acknowledgment (ACK). To this end, every cell, SRQ,
and ACK carries a sequence number. However, when a grant arrives before
theACK, a cell is transmitted a second time. These are called duplicate cells
as opposed to the pure cells, which are transmitted through grants but are not
duplicate. The corresponding grants are classified as duplicate and pure
accordingly.Every grant is either regular, spurious, or wasted. It is regular if
it is used by the cell that initiated it. A grant corresponding to a successfully
speculatively transmitted and acknowledged cell is spurious when used by
another cell residing in the same VOQ, resulting in a spurious transmission,
or wasted if the VOQ is empty.If it is wasted, the slot can be used for a
speculative transmission.
STX Policy
According to , an adapter performs an STX in a given time slot if it receives
no grant at and has an eligible cell. If it receives a grant, it performs the
corresponding scheduled transmission. allows the STX scheme to operate in
conjunction with regular scheduled transmissions, which take precedence?
49
over the speculative ones. Accordingly, we distinguish between scheduled
and speculative cells. When an adapter is eligible to perform an STX, it
selects a non-empty VOQ according to a specific STX policy, dequeue
its HOL cell and stores it in a retransmission buffer, marks the cell as
speculative, and sends it to the crossbar. On the control path, it sends an
SRQ indicating that a cell has been sent speculatively to the selected output.
Both the cell and the SRQ comprise a unique sequence number to enable
reliable, in-order, single-copy delivery. The STX policy defines which VOQ
the adapter selects when it is eligible to perform an STX. This policy can
employ, e.g., a random (RND), oldest-cell-first (OCF), or youngest-cell-first
(YCF) selection. First, we consider the OCF policy. It chooses the cell that
has been waiting longest at the input adapter for an STX opportunity.
Collisions:
An important consequence of STX is the occurrence of collisions in the
switch fabric: As STX cells are sent without prior arbitration, they may
collide with either other STX cells or scheduled cells destined to the same
output, and as a result they may be dropped. In OSMOSIS, it is possible to
always allow up to cells to “survive” the collision, because the colliding
cells do not share a physical medium until they arrive at the crossbar. The
50
scheduler knows about incoming STX cells from the accompanying SRQs
on the control path, and it also knows which scheduled cells have been
scheduled to arrive in the current time slot. Therefore, it can arbitrate
between arriving STX cells if necessary and configure the crossbar to allow
up to to pass, while dropping the others. Therefore, transmissions are always
successful, even in the case of a collision. This is an important difference to
ALOHA or Ethernet, where all colliding cells are lost. When multiple STX
cells collide, we can forward up to of them, but when a scheduled cell
collides with one or more STX cells, the scheduled cell always takes
precedence to ensure that STX does not interfere with the basic operation of
the underlying matching algorithm (see and ). Note also that the matching
algorithm ensures that collisions between scheduled cells can never occur.
The collision arbitration operates as follows. Before resolving contention
among SRQs destined to port , a scheduled matching for the time slot under
consideration must be ready. For every matched port , a number of SRQs are
randomly accepted and the others denied. For every unmatched port , a
number of SRQs are randomly accepted and the others denied. Granting
SRQs does not affect the operation of the matching algorithm, e.g., in the
case of -SLIP, the round-robin pointers are not updated. The scheduler
notifies the sender of a successful SRQ by means of an acknowledgment
51
(ACK). Of course, it also issues the regular grants according to the
matching. These grants may cause duplicate cell transmissions as described
in . The scheduler does not generate explicit negative acknowledgments
(NAK) for dropped cells.
Retransmission:
Collisions imply cell losses and out-of-order (OOO) delivery, which in turn
imply a need for link-level retransmissions and ACKs, as this loss
probability is orders of magnitude higher than that due to transmission
errors. Reliability and ordering can be restored by means of a reliable
delivery (RD) scheme. Any RD scheme requires that an STX cell remain in
the input adapter buffer until successfully transmitted. The ACKs are
generated by the scheduler for every successful STX cell and include the
sequence number of the acknowledged cell. specifies that a speculative cell
remains stored in the adapter until either of the following two events occurs:
52
• The cell is positively acknowledged, i.e., an ACK arrives with the
corresponding sequence number. The cell is dequeued and dropped.
• A grant for this output arrives and the cell is the oldest unacknowledged
STX cell. When a grant arrives and there are any unacknowledged STX cells
for the granted output, the oldest of these is dequeued and retransmitted.
Otherwise, the HOL cell of the VOQ is dequeued and transmitted, as usual.
This rule implies that unacknowledged STX cells take precedence over other
cells in the VOQ, to expedite their reliable, in-order delivery.
According to , unacknowledged STX cells are never eligible for STX,
because they have already been transmitted speculatively once. Allowing
only one STX attempt per cell reduces the number of STXs, which increases
their chance of success. Moreover, if an STX cell fails, the potential gain in
latency has been lost in any case, so retrying the same cell serves no
purpose. This is also the reason for not using explicit NAKs.
According to and , a non-wasted grant can be classified in two orthogonal
ways: It is either pure or duplicate, and it is either regular or spurious
depending on whether it is used by the cell that initiated it.
There are several methods of achieving reliable, in-order delivery in the
presence of STX, e.g., Go-Back-N (GBN) and Selective Retry (SR). First,
53
we consider SR. SR allows a predetermined maximum number of cells per
output to be unacknowledged at each input at any given time. STX cells are
stored in retransmission (RTX) queues (one RTX queue per VOQ). The
output adapter accepts cells in any order and performs resequencing to
restore the correct cell order. To this end, it has a resequencing queue (RSQ)
per input to store OOO cells until the missing ones arrive. The input adapter
accepts ACKs in any order. This implies that only the failed STX cells need
to be retransmitted, hence the name Selective Retry, as opposed to
retransmitting the entire RTX queue as is done with GBN. SR requires
resequencing logic and buffers at every output adapter. In addition, the RTX
queues require a random-out organization, because cells can be dequeued
from any point in the queue. However, SR minimizes the number of
retransmissions, thus improving performance.
Advantages:
It is used to reduce the computational time.
Application:
The computation time is also an important criterion for CBIR in
generalized applications, since people will not wait several minutes
54
between two feedback steps. Furthermore, a fast selection allows the
user to provide more labels in the same time. Thus, it is more
interesting to use a less efficient but fast method than a more efficient
but highly-computational one.
It will reduce the control-path latency incurred between issuance of a
request and arrival of the corresponding grant.
10. BIBLIOGRAPHY
1] R. Veltkamp, “Content-based image retrieval system: A survey,” Tech.
Rep., Univ. Utrecht, Utrecht, The Netherlands, 2002.
[2] Y. Rui, T. Huang, S. Mehrotra, and M. Ortega, “A relevance feedback
architecture for content-based multimedia information retrieval systems,”
in Proc. IEEE Workshop Content-Based Access of Image and
Video Libraries, 1997, pp. 92–89.
[3] E. Chang, B. T. Li, G. Wu, and K. Goh, “Statistical learning for effective
visual information retrieval,” in Proc. IEEE Int. Conf. Image
Processing, Barcelona, Spain, Sep. 2003, pp. 609–612.
[4] S. Aksoy, R. Haralick, F. Cheikh, and M. Gabbouj, “A weighted distance
55
approach to relevance feedback,” in Proc. IAPR Int. Conf. Pattern
Recognition, Barcelona, Spain, Sep. 3–8, 2000, vol. IV, pp. 812–815.
[5] J. Peng, B. Bhanu, and S. Qing, “Probabilistic feature relevance
learning for content-based image retrieval,” Comput. Vis. Image
Understand., vol. 75, no. 1-2, pp. 150–164, Jul.-Aug. 1999.
[6] N. Doulamis and A. Doulamis, “A recursive optimal relevance feedback
scheme for CBIR,” presented at the Int.Conf. Image Processing,
Thessaloniki, Greece, Oct. 2001.
[7] J. Fournier, M. Cord, and S. Philipp-Foliguet, “Back-propagation
algorithm for relevance feedback in image retrieval,” in Proc. Int.
Conf. Image Processing, Thessaloniki, Greece, Oct. 2001, vol. 1, pp.
686–689.
[8] O. Chapelle, P. Haffner, and V. Vapnik, “Svms for histogram
based image classification,” IEEE Trans. Neural Netw., vol. 10, pp.
1055–1064, 1999.
[9] N. Vasconcelos, “Bayesian models for visual information retrieval,”
Ph.D. dissertation, Mass. Inst. Technol., Cambridge, 2000.
[10] S.-A. Berrani, L. Amsaleg, and P. Gros, “Recherche approximative de
plus proches voisins: Application la reconnaissance d’images par
descripteurs
56
locaux,” Tech. Sci. Inf., vol. 22, no. 9, pp. 1201–1230, 2003.
[11] N. Najjar, J. Cocquerez, and C. Ambroise, “Feature selection for semi
supervised learning applied to image retrieval,” in Proc. IEEE Int. Conf.
Image Processing, Barcelona, Spain, Sep. 2003, vol. 2, pp. 559–562.
[12] X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-supervised learning
using gaussian fields and harmonic functions,” presented at the Int.
Conf. Machine Learning, 2003.
GOSSELIN AND CORD: ACTIVE LEARNING METHODS FOR
INTERACTIVE IMAGE RETRIEVAL 1211
[13] A. Dong and B. Bhanu, “Active concept learning in image databases,”
IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, pp. 450–466,
2005.
[14] N. Vasconcelos and M. Kunt, “Content-based retrieval from image
databases: Current solutions and future directions,” in Proc. Int. Conf.
Image Processing, Thessaloniki, Greece, Oct. 2001, vol. 3, pp. 6–9.
[15] Y. Chen, X. Zhou, and T. Huang, “One-class SVM for learning in
image retrieval,” in Proc. Int. Conf. Image Processing, Thessaloniki,
Greece, Oct. 2001, vol. 1, pp. 34–37.
[16] S.-A. Berrani, L. Amsaleg, and P. Gros, “Approximate searches:
k-neighbors + precision,” in Proc. Inf. Conf. Information and Knowledge
57
APPENDIX
SCREEN SHOTS
58
59
CODING:
MAIN FORM:
namespace Retrieval
{
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Diagnostics;
using System.Drawing;
using System.IO;
using System.Windows.Forms;
using Processing;
internal partial class Main : Form
{
private bool exit = false;
public Main()
{
this.InitializeComponent();
}
#region "GUI delegates"
private delegate void ProcessImagesDelegate(FileInfo[] files);
private delegate void SetMaximumDelegate(ProgressBar
progressBar, int value);
60
private delegate void UpdateOperationStatusDelegate(string
format, System.Windows.Forms.Label label, ProgressBar progressBar, int
value, DateTime startTime);
private delegate void
UpdateDataGridViewDelegate(BindingList<SimilarityImages> images,
DataGridView dataGridView, IList<Button> buttons, TextBox textBox);
private delegate void DeleteImageDelegate(FileInfo fileInfo);
private delegate void ShowGridDelegate(DataGridView
dataGridView);
#endregion
#region "Start/stop search"
//THE SEARCH BUTTON
private void FindButton_Click(object sender, EventArgs e)
{
visible();
string folder = this.folderTextBox.Text;
string filee = this.filetext.Text;
DirectoryInfo directoryInfo;
FileInfo[] files;
try
{
directoryInfo = new DirectoryInfo(folder );
files = directoryInfo.GetFiles("*.jpg",
SearchOption.AllDirectories);
this.exit = false;
this.findButton.Enabled = false;
this.cancelButton.Enabled = !this.findButton.Enabled;
this.folderTextBox.Enabled = this.findButton.Enabled;
}
catch (DirectoryNotFoundException)
{
MessageBox.Show("Path not valid.", "Invalid path",
MessageBoxButtons.OK, MessageBoxIcon.Error);
this.folderTextBox.Enabled = true;
return;
}
catch (ArgumentException)
{
MessageBox.Show("Path not valid.", "Invalid path",
MessageBoxButtons.OK, MessageBoxIcon.Error);
return;
}
ProcessImagesDelegate processImagesDelegate = new
ProcessImagesDelegate(ProcessImages);
processImagesDelegate.BeginInvoke(files, null, null);
}
private void CancelButton_Click(object sender, EventArgs e)
{
this.exit = true;
this.findButton.Enabled = true;
this.cancelButton.Enabled = !this.findButton.Enabled;
this.folderTextBox.Enabled = this.findButton.Enabled;
61
}
#endregion
#region "GUI delegates body"
private UpdateDataGridViewDelegate updateDataGridViewDelegate =
delegate(BindingList<SimilarityImages> images, DataGridView
dataGridView, IList<Button> buttons, TextBox textBox)
{
images.RaiseListChangedEvents = true;
dataGridView.DataSource = images;
dataGridView.Columns["Similarity"].DisplayIndex = 0;
dataGridView.Columns["Source"].DisplayIndex = 1;
dataGridView.Columns["Destination"].DisplayIndex = 2;
if (images.Count > 0)
dataGridView.Rows[0].Selected = true;
buttons[0].Enabled = true;
buttons[1].Enabled = !buttons[0].Enabled;
textBox.Enabled = buttons[0].Enabled;
};
SetMaximumDelegate setMaximumDelegate = delegate(ProgressBar
progressBar, int value)
{
progressBar.Maximum = value;
};
UpdateOperationStatusDelegate updateOperationStatusDelegate =
delegate(string format, System.Windows.Forms.Label label, ProgressBar
progressBar, int value, DateTime startTime)
{
progressBar.Value = value;
double percentage = Math.Round(((double)progressBar.Value /
(double)progressBar.Maximum), 3);
format += " {0}/{1} ({2}) Elapsed: {3} Estimated: {4}";
TimeSpan elapsed = DateTime.Now.Subtract(startTime);
elapsed = new TimeSpan(elapsed.Days, elapsed.Hours,
elapsed.Minutes, elapsed.Seconds, 0);
long estimatedTicks = (elapsed.Ticks / value) *
progressBar.Maximum;
TimeSpan estimated = new TimeSpan(estimatedTicks);
estimated = new TimeSpan(estimated.Days, estimated.Hours,
estimated.Minutes, estimated.Seconds, 0);
label.Text = string.Format(format, progressBar.Value,
progressBar.Maximum, percentage.ToString("P"), elapsed.ToString(),
estimated.ToString());
};
ShowGridDelegate showGridDelegate = delegate(DataGridView
dataGridView)
{
dataGridView.ResumeLayout();
62
dataGridView.Enabled = true;
};
#endregion
BindingList<SimilarityImages> similarityImages;
private void ProcessImages(FileInfo[] files)
{
List<ComparableImage> comparableImages;
comparableImages = new List<ComparableImage>();
this.Invoke(setMaximumDelegate, new object[]
{
this.workingProgressBar, files.Length
});
int index;
index = 0;
DateTime operationStartTime;
operationStartTime = DateTime.Now;
foreach (FileInfo file in files)
{
if (this.exit)
{
return;
}
ComparableImage comparableImage = new
ComparableImage(file);
//comparableImages comparebleimaaa = new
ComparableImage(file);
comparableImages.Add(comparableImage);
index++;
this.Invoke(updateOperationStatusDelegate, new object[]
{ "Processed images", workingLabel, workingProgressBar, index,
operationStartTime });
}
this.Invoke(this.setMaximumDelegate, new object[]
{
this.workingProgressBar, ( comparableImages .Count
+(comparableImages.Count -1)) / 2
});
index = 0;
List<SimilarityImages> similarityImagesSorted = new
List<SimilarityImages>();
operationStartTime = DateTime.Now;
// int counn = 0;
string[] selectedFiles = openFileDialog1.FileNames;
// for (int i = 0; i <= 6; i++)
63
//{
//int k = selectedFiles[i].Length;
//for (int i=0;i<selectedFiles
//for (int i = 0; i < comparableImages.Count; i++)
// {
//string b=comparableImages [0]
for (int j = 1; j < comparableImages.Count; j++)
{
if (this.exit)
{
return;
}
//string[] files =
Directory.GetFiles(openFileDialog1.FileName);
// if (comparableImages[0].ToString() == "rr.jpeg")
//{
int ss = index1;
ComparableImage source = comparableImages[ss];
ComparableImage destination =
comparableImages[j];
//double similarity =
destination.CalculateSimilarity(source);
double similarity =
source.CalculateSimilarity(destination);
SimilarityImages sim = new
SimilarityImages(source, destination, similarity);
similarityImagesSorted.Add(sim);
index++;
this.Invoke(updateOperationStatusDelegate, new
object[]
{
"Compared images", workingLabel,
workingProgressBar, index, operationStartTime }
);
}
similarityImagesSorted.Sort();
similarityImagesSorted.Reverse();
similarityImages = new
BindingList<SimilarityImages>(similarityImagesSorted);
List<Button> buttons2 = new List<Button>();
buttons2.Add(findButton);
buttons2.Add(cancelButton);
64
this.BeginInvoke(updateDataGridViewDelegate, new
object[] { similarityImages, imagesDataGridView, buttons2,
folderTextBox });
//}
}
private void ShowProjections(Graph graph, ComparableImage
comparable)
{
graph.ClearCurves();
graph.AddPlotCurve(Color.Red,
comparable.Projections.HorizontalProjection);
graph.AddPlotCurve(Color.Green,
comparable.Projections.VerticalProjection);
graph.Invalidate();
}
protected override void OnClosing(CancelEventArgs e)
{
exit = true;
}
private void linkLabel1_LinkClicked(object sender,
LinkLabelLinkClickedEventArgs e)
{
this.folderBrowserDialog.ShowDialog();
this.folderTextBox.Text =
this.folderBrowserDialog.SelectedPath;
}
private void exitToolStripMenuItem_Click(object sender,
EventArgs e)
{
}
private void ShowSelectedImages()
{
if (imagesDataGridView.SelectedRows.Count <= 0)
{
return;
}
SimilarityImages sim =
(SimilarityImages)imagesDataGridView.SelectedRows[0].DataBoundItem;
//FileStream streamSource = new
System.IO.FileStream(sim.Source.File.FullName, FileMode.Open,
FileAccess.Read);
FileStream streamDestination = new
System.IO.FileStream(sim.Destination.File.FullName, FileMode.Open,
FileAccess.Read);
Image source = Image.FromFile(openFileDialog1.FileName);
Image destination = Image.FromStream(streamDestination);
// streamSource.Close();
streamDestination.Close();
65
string infoFormat = "Resolution: {0}x{1}\nSize: {2}kb\nFull
path: {3}";
string resolutionFormat = "{0} ({1}x{2})";
sourcePictureBox.Image =
Image.FromFile(openFileDialog1.FileName);
//mainToolTip.SetToolTip(sourcePictureBox,
string.Format(infoFormat, source.Width, source.Height,
Math.Round((double)(sim.Source.File.Length / 1024), 1),
sim.Source.File.FullName));
sourceLabel.Text = string.Format(resolutionFormat,
"Source", source.Width, source.Height);
destinationPictureBox.Image = destination;
mainToolTip.SetToolTip(destinationPictureBox,
string.Format(infoFormat, destination.Width, destination.Height,
Math.Round((double)(sim.Destination.File.Length / 1024), 1),
sim.Destination.File.FullName));
destinationLabel.Text = string.Format(resolutionFormat,
"Destination", destination.Width, destination.Height);
// ShowProjections(sourceGraph, sim.Source);
ShowProjections(destinationGraph, sim.Destination );
ShowProjections(sourceGraph, sim.Source);
}
private void DeleteSourceLinkLabel_LinkClicked(object sender,
LinkLabelLinkClickedEventArgs e)
{
if (imagesDataGridView.SelectedRows.Count <= 0)
{
return;
}
DialogResult result = MessageBox.Show("Delete the image?",
"Confirm delete", MessageBoxButtons.OKCancel, MessageBoxIcon.Question);
if (result == DialogResult.Cancel)
{
return;
}
SimilarityImages sim =
(SimilarityImages)imagesDataGridView.SelectedRows[0].DataBoundItem;
DeleteImageDelegate deleteImageDelegate = new
DeleteImageDelegate(DeleteImage);
imagesDataGridView.SuspendLayout();
imagesDataGridView.Enabled = false;
deleteImageDelegate.BeginInvoke(sim.Source.File, new
AsyncCallback(ShowDataGrid), null);
}
private void ShowDataGrid(IAsyncResult result)
{
this.Invoke(showGridDelegate, imagesDataGridView);
}
66
private void DeleteDestinationLinkLabel_LinkClicked(object
sender, LinkLabelLinkClickedEventArgs e)
{
if (imagesDataGridView.SelectedRows.Count <= 0)
return;
DialogResult result = MessageBox.Show("Delete the image?",
"Confirm delete", MessageBoxButtons.OKCancel, MessageBoxIcon.Question);
if (result == DialogResult.Cancel)
return;
SimilarityImages sim =
(SimilarityImages)imagesDataGridView.SelectedRows[0].DataBoundItem;
DeleteImageDelegate deleteImageDelegate = new
DeleteImageDelegate(DeleteImage);
imagesDataGridView.SuspendLayout();
imagesDataGridView.Enabled = false;
deleteImageDelegate.BeginInvoke(sim.Destination.File, new
AsyncCallback(ShowDataGrid), null);
}
private void DeleteImage(FileInfo fileInfo)
{
try
{
List<SimilarityImages> toDelete = new
List<SimilarityImages>();
for (int index = 0; index < similarityImages.Count;
index++)
{
SimilarityImages item = similarityImages[index];
if
((item.Source.File.FullName.Equals(fileInfo.FullName,
StringComparison.InvariantCultureIgnoreCase)) ||
(item.Destination.File.FullName.Equals(fileInfo.FullName,
StringComparison.InvariantCultureIgnoreCase)))
toDelete.Add(item);
}
similarityImages.RaiseListChangedEvents = false;
for (int index = 0; index < toDelete.Count; index++)
{
SimilarityImages item = toDelete[index];
similarityImages.Remove(item);
}
similarityImages.RaiseListChangedEvents = true;
similarityImages.ResetBindings();
fileInfo.Delete();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
67
private void OpenSourceLinkLabel_LinkClicked(object sender,
LinkLabelLinkClickedEventArgs e)
{
if (imagesDataGridView.SelectedRows.Count <= 0)
return;
SimilarityImages sim =
(SimilarityImages)this.imagesDataGridView.SelectedRows[0].DataBoundItem
;
Process.Start(sim.Source.File.FullName);
}
private void OpenDestinationLinkLabel_LinkClicked(object
sender, LinkLabelLinkClickedEventArgs e)
{
if (this.imagesDataGridView.SelectedRows.Count <= 0)
{
return;
}
SimilarityImages sim =
(SimilarityImages)this.imagesDataGridView.SelectedRows[0].DataBoundItem
;
Process.Start(sim.Destination.File.FullName);
}
private void DataGridView1_SelectionChanged(object sender,
EventArgs e)
{
this.ShowSelectedImages();
}
private void selectFolderToSerachToolStripMenuItem_Click(object
sender, EventArgs e)
{
gbxFolder.Visible = true;
}
public void visible()
{
workingLabel.Visible = true;
imagesDataGridView.Visible = true;
workingProgressBar.Visible = true;
panel.Visible = true;
imagesLabel.Visible = true;
}
private void aboutToolStripMenuItem_Click(object sender,
EventArgs e)
{
about ab = new about();
ab.Show();
}
private void closeToolStripMenuItem_Click(object sender,
EventArgs e)
{
68
Application.Exit();
}
private void viewGraphToolStripMenuItem_Click(object sender,
EventArgs e)
{
label2.Visible = true;
label3.Visible = true;
sourceGraph.Visible = true;
destinationGraph.Visible = true;
}
private void hideGraphToolStripMenuItem_Click(object sender,
EventArgs e)
{
label2.Visible = false;
label3.Visible = false;
sourceGraph.Visible = false;
destinationGraph.Visible = false;
}
public string ss;
private void linkLabel1_LinkClicked_1(object sender,
LinkLabelLinkClickedEventArgs e)
{
if (openFileDialog1.ShowDialog() == DialogResult.OK)
{
this.filetext.Text = openFileDialog1.FileName;
sourcePictureBox.Image =
Image.FromFile(openFileDialog1.FileName);
// pboxAfter.Image =
Image.FromFile(openFileDialog1.FileName);
//openFileDialog1.ShowDialog();
ss = openFileDialog1.FileName.ToString();
this.filetext.Text = openFileDialog1.FileName;
//string[] selectedFiles = new string[20];
//selectedFiles =openFileDialog1 .FileNames ;
//for (int i = 0; i < selectedFiles.Length; i++)
//{
// // MessageBox.Show("rrrr", +i);
//}
// int selectedFilesCount = selectedFiles.Length;
// for (int i = 0; i < selectedFilesCount.c; i++)
// {
// MessageBox.Show("rrr"+i);
// }
}
}
private void Main_Load(object sender, EventArgs e)
{
}
private void sourceGraph_Click(object sender, EventArgs e)
{
69
}
public int index1;
private void openFileDialog1_FileOk(object sender,
CancelEventArgs e)
{
DirectoryInfo directoryInfo = new
DirectoryInfo(folderTextBox.Text);
FileInfo[] files = directoryInfo.GetFiles("*.jpg",
SearchOption.AllDirectories);
string searchImage = openFileDialog1.FileName.ToString();
int se = searchImage.LastIndexOf("\\");
int aaa = searchImage.Length - se;
string aa = searchImage.Substring(se + 1, aaa - 1);
int count = files.Length;
index1 = 0;
for (int i = 0; i <= count-1; i++)
{
if (aa == files[i].ToString())
{
index1 = i;
}
}
}
private void folderBrowserDialog_HelpRequest(object sender,
EventArgs e)
{
}
private void sourcePictureBox_Click(object sender, EventArgs e)
{
}
}
}
70
RGB PROJECTIONS:
namespace Processing
{
using System;
using System.Collections.Generic;
/// <summary>
/// Represents RGB projections, horizontal and vertical.
/// </summary>
public class RgbProjections
{
private double[] horizontalProjection;
private double[] verticalProjection;
internal RgbProjections(double[][] projections)
: this(projections[0], projections[1])
{
}
internal RgbProjections(double[] horizontalProjection, double[]
verticalProjection)
{
this.horizontalProjection = horizontalProjection;
this.verticalProjection = verticalProjection;
}
public double[] HorizontalProjection
{
get
{
return this.horizontalProjection;
}
}
public double[] VerticalProjection
{
get
{
return this.verticalProjection;
}
}
/// <summary>
/// Calculate the similarity between two RGB projections,
horizontal and vertical.
/// </summary>
/// <param name="compare">The RGB projection to compare
with.</param>
/// <returns>Return the max similarity value betweem horizontal
and vertical RGB projections.</returns>
public double CalculateSimilarity(RgbProjections compare)
{
71
double horizontalSimilarity =
CalculateProjectionSimilarity(this.horizontalProjection,
compare.horizontalProjection);
double verticalSimilarity =
CalculateProjectionSimilarity(this.verticalProjection,
compare.verticalProjection);
return Math.Max(horizontalSimilarity, verticalSimilarity);
}
/// <summary>
/// Calculate the similarity to another RGB projection.
/// </summary>
/// <param name="source">The source RGB projection.</param>
/// <param name="compare">The RGB projection to compare
with.</param>
/// <returns>Return a value from 0 to 1 that is the
similarity.</returns>
private static double CalculateProjectionSimilarity(double[]
source, double[] compare)
{
if (source.Length != compare.Length)
{
throw new ArgumentException();
}
Dictionary<double, int> frequencies = new
Dictionary<double, int>();
////Calculate frequencies
for (int i = 0; i < source.Length; i++)
{
double difference = (source[i] - compare[i]);
difference = (Math.Round(difference, 2));
difference = Math.Abs(difference);
if (frequencies.ContainsKey(difference))
{
frequencies[difference] = frequencies[difference] +
1;
}
else
{
frequencies.Add(difference, 1);
}
}
double deviation = 0;
////Calculate "weighted mean"
////http://en.wikipedia.org/wiki/Weighted_mean
foreach (KeyValuePair<double, int> value in frequencies)
{
deviation += (value.Key * value.Value);
}
deviation /= source.Length;
////Maximize scale
72
deviation = (0.5 - deviation) * 2;
return deviation;
}
}
}
IMAGE UTILITY: namespace Processing
{
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Drawing.Imaging;
using System.Diagnostics.CodeAnalysis;
/// <summary>
/// An utility class to do simple image processing.
/// </summary>
public static class ImageUtility
{
private static object syncRoot = new object();
/// <summary>
/// Resize an image in high resolution
/// </summary>
/// <param name="bitmap">The image to resize.</param>
/// <param name="width">The expected width.</param>
/// <param name="height">the expected height.</param>
/// <returns></returns>
[SuppressMessage("Microsoft.Design",
"CA1011:ConsiderPassingBaseTypesAsParameters")]
public static Bitmap ResizeBitmap(Bitmap bitmap, int width, int
height)
{
Bitmap result = new Bitmap(width, height);
using (Graphics graphic =
Graphics.FromImage((System.Drawing.Image)result))
{
graphic.InterpolationMode =
InterpolationMode.HighQualityBicubic;
graphic.DrawImage(bitmap, 0, 0, width - 1, height - 1);
}
return result;
}
/// <summary>
/// Calculate the RBG projection.
/// </summary>
/// <param name="bitmap">The image to process.</param>
/// <returns>Return horizontal RGB projection in value [0] and
vertical RGB projection in value [1].</returns>
public static double[][] GetRgbProjections(Bitmap bitmap)
{
int width = bitmap.Width - 1;
int height = bitmap.Width - 1;
73
double[] xProjection = new double[width];
double[] yProjection = new double[height];
BitmapData bitmapData1 = bitmap.LockBits(new Rectangle(0,
0, bitmap.Width, bitmap.Height), ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
int luminosity = 0;
unsafe
{
byte* imagePointer1 = (byte*)bitmapData1.Scan0;
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
byte blu = imagePointer1[0];
byte green = imagePointer1[1];
byte red = imagePointer1[2];
luminosity = (byte)(((0.2126 * red) + (0.7152 *
green)) + (0.0722 * blu));
xProjection[x] += luminosity;
yProjection[y] += luminosity;
////4 bytes per pixel
imagePointer1 += 4;
} ////end for j
////4 bytes per pixel
imagePointer1 += bitmapData1.Stride -
(bitmapData1.Width * 4);
}////end for i
}////end unsafe
MaximizeScale(ref xProjection, height);
MaximizeScale(ref yProjection, width);
double[][] projections = new double[][] { xProjection,
yProjection };
bitmap.UnlockBits(bitmapData1);
return projections;
}
/// <summary>
/// Optimize the range of values.
/// </summary>
/// <param name="projection">The array to process.</param>
/// <param name="max">The max value for the elements.</param>
private static void MaximizeScale(ref double[] projection,
double max)
{
double minValue = double.MaxValue;
double maxValue = double.MinValue;
74
for (int i = 0; i < projection.Length; i++)
{
if (projection[i] > 0)
{
projection[i] = projection[i] / max;
}
if (projection[i] < minValue)
{
minValue = projection[i];
}
if (projection[i] > maxValue)
{
maxValue = projection[i];
}
}
if (maxValue == 0)
{
return;
}
for (int i = 0; i < projection.Length; i++)
{
if (maxValue == 255)
{
projection[i] = 1;
}
else
{
projection[i] = (projection[i] - minValue) /
(maxValue - minValue);
}
}
}
}
}
COMPARABLE IMAGE:
namespace Processing
{
using System;
using System.Drawing;
using System.IO;
/// <summary>
/// Represents an image and its RGB projections.
/// </summary>
public class ComparableImage
{
private FileInfo file;
private RgbProjections projections;
75
public ComparableImage(FileInfo file)
{
if (file == null)
{
throw new ArgumentNullException("file");
}
if (!file.Exists)
{
throw new FileNotFoundException();
}
this.file = file;
using (Bitmap bitmap = ImageUtility.ResizeBitmap(new
Bitmap(file.FullName), 100, 100))
{
this.projections = new
RgbProjections(ImageUtility.GetRgbProjections(bitmap));
}
}
public FileInfo File
{
get
{
return this.file;
}
}
public RgbProjections Projections
{
get
{
return this.projections;
}
}
/// <summary>
/// Calculate the similarity to another image.
/// </summary>
/// <param name="compare">The image to compare with.</param>
/// <returns>Return a value from 0 to 1 that is the
similarity.</returns>
public double CalculateSimilarity(ComparableImage compare)
{
return
this.projections.CalculateSimilarity(compare.projections);
}
public override string ToString()
{
return file.Name;
}
}
}
76
SIMILARITY IMAGES:
namespace Processing
{
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
/// <summary>
/// Represents a comparable images class.
/// </summary>
public class SimilarityImages : IComparer<SimilarityImages>,
IComparable
{
private ComparableImage source;
private ComparableImage destination;
private double similarity;
public SimilarityImages(ComparableImage source, ComparableImage
destination, double similarity)
{
this.source = source;
this.destination = destination;
this.similarity = similarity;
}
public ComparableImage Source
{
get
{
return this.source;
}
}
public ComparableImage Destination
{
get
{
return this.destination;
}
}
public double Similarity
{
get
{
return Math.Round(this.similarity * 100, 1);
}
}
[SuppressMessage("Microsoft.Globalization",
"CA1305:SpecifyIFormatProvider", MessageId =
"System.String.Format(System.String,System.Object,System.Object,System.
Object)")]
public override string ToString()
{
77
return string.Format("{0}, {1} --> {2}",
this.source.File.Name, this.destination.File.Name, this.similarity);
}
#region IComparer<SimilarityImages> Members
public int Compare(SimilarityImages x, SimilarityImages y)
{
return x.similarity.CompareTo(y.similarity);
}
#endregion
#region IComparable Members
public int CompareTo(object obj)
{
SimilarityImages other = (SimilarityImages)obj;
return this.Compare(this, other);
}
#endregion
// override object.Equals
public override bool Equals(object obj)
{
if (obj == null || GetType() != obj.GetType())
{
return false;
}
SimilarityImages other = (SimilarityImages)obj;
bool equals =
this.Source.File.FullName.Equals(other.Source.File.FullName,
StringComparison.InvariantCultureIgnoreCase);
if (!equals)
{
return false;
}
equals =
this.Destination.File.FullName.Equals(other.Destination.File.FullName,
StringComparison.InvariantCultureIgnoreCase);
if (!equals)
{
return false;
}
return true;
}
// override object.GetHashCode
[SuppressMessage("Microsoft.Globalization",
"CA1305:SpecifyIFormatProvider", MessageId =
"System.String.Format(System.String,System.Object,System.Object)")]
78
public override int GetHashCode()
{
return string.Format("{0};{1}", this.Source.File.FullName,
this.Destination.File.FullName).GetHashCode();
}
public static int operator !=(SimilarityImages value,
SimilarityImages compare)
{
return value.CompareTo(compare);
}
public static int operator <(SimilarityImages value,
SimilarityImages compare)
{
return value.CompareTo(compare);
}
public static int operator ==(SimilarityImages value,
SimilarityImages compare)
{
return value.CompareTo(compare);
}
public static int operator >(SimilarityImages value,
SimilarityImages compare)
{
return value.CompareTo(compare);
}
}
}