interactive matlab platform for bioimaging informatics user...

TACTICS v3.x by R.S. 2010-2014

Appendix I

TACTICS Toolbox v3.x

User Guide

Interactive MATLAB Platform For Bioimaging informatics

TRACKING

MODULE


Appendix I -17-

Cell Tracking Module – 1 (user interface)

Once the images were successfully segmented, the next step in the processing scheme, is the

cell tracking and manual correction interface:

From the main menu, go to >Cell Tracking Module

The Cell Tracking Module is used to correct segmentation, label cells and tracking. The GUI has

several selection methods for manual image segmentation that allow the user to inspect and

correct cell segmentation manually, and allows for multiple objects tracking correction.

Navigation panel

Secondary Screen

Primary Screen

Segmentation panel

Mark cell division

Mark cell death

Trajectory panel

Editing tracks panel

Screen capture of the Cell Tracking Module interface

Video tutorial

3

Open new MATLAB figure

1. Run the Linker

2. Split cells after divisions

3. Save manual correction of cell associations

4. Show lineage tree

5. Mark cell division

6. Mark cell death

Load experimental data file

Save experimental data file

Drag zoom rectangle

Capture ROI to secondary screen

Zoom in

Zoom 100%

Zoom out

Pan

Error Zoom (temporary solution for Zoom issues)

Functions supported in the icon bar-


Appendix I -18-

Cell Tracking Module – 1 (user interface)

Change the channel and the display mode (raw,

filtered etc.) of the

Primary screen using this pull down menu

Channel and display mode of the

secondary screen

Read and upload images within

selected range to

RAM (dependent on the computer

memory)


Keyboard shortcuts: Uparrow - Next frame Downarrow - Previous frame Space - label frame again Enter - associate frame overpassing selective operator F - Format painting S - Segmentation mode M - Mark division mode E - Next frame T - Trajectory mode A - Association mode

D - Dead cell mode E - Next frame R - Remove segment mode U - uncheck/check P - Paint Tool mode


Appendix I -19-

Cell Tracking Module – 2 (cell labeling and tracking)

Instructions for cell labeling and tracking:

1.Load an experiment data file to TACTICS .

2.To label the cells- Go to >’Label cells’>> ‘Label cells’ . A labelling function will label each

segment from n to m (n and m are user input). Reference channel can be used to help to interpret

normalized pixel intensities in the quantified channel and allow to track two types of cells at the

same time. After running the Label Cells function, the cells will be detected, identified and labeled

including feature extraction. A bounding box will mark each segment, and all labeled frames will

be marked in red tab under the sliderbar:

3. To connect between the cells over time- Go to >’Label cells’>> ‘Label association‘. An input

box will ask for the range of frames to be associated and maximum distance that the cells can

travel (a new cell will be defined if the distance to a closer point in time (t-1) is above this criteria.

The input units are currently in pixels. Once finished, all associated frames will be marked in

green tab under the sliderbar (note that the first frame cannot be associated):

4.To track the cells, click on the ‘tracking’ icon or Go to >Track cells>> Hungarian

algorithm>>>Tracking linker. The associated points will be connected to create the cell`s

tracks. The user must input a cut off value that sets the possible minimum track length (a value of

1 is frequently acceptable).

------------------

As an alternative to label association followed by Hungarian algorithm, the user can utilize the

Crocker algorithm for tracking multiple objects Go to >Track cells>> Crocker algorithm. The

advantage is the ability to recover for missing points. However, the algorithm is slower and fails

when the number of cells or time points is large (dozens of cells for 1000 frames will fail the

algorithm).

------------------

•Once cells are tracked, trajectories of cell over time can be visualized. The location of the cells in

different time points is marked with a colored point, and the size of the point is proportional to the

timeframe of the track.

•To mark dividing cells, go to cell division mode and click on the cell at the point of division

(one time point before the appearance of two separated cells) with the middle mouse button. X

sign will be added/removed correspondingly.

•To mark dying cell, go to cell death mode and click on the cell at the last point of

appearance with the middle mouse button. + sign will be added/removed correspondingly.

IMPORTANT. Save the experiment file (overwrite or create new file) before exiting TACTICS (or

before loading new experiment file). Otherwise, any operation will be lost.


Appendix I -20-

Cell Tracking Module – 3 (cell labeling corrections)

A. Improving segmentation of a single frame using the secondary screen view

1. Click on the ‘Capture ROI to secondary screen’ icon . Once you have clicked on the icon

a window will come up with an image of the screen you have been working on. Hold down

mouse (left click) anywhere on the new screen. This will create a second figure from the

right side of the screen containing a magnified image of the ROI within the images that is

chosen for zoom.

2. While still holding down left click use the arrow keys to form a box around the cells that you

would like to work with in the secondary screen (Make sure the aspect ratio is 1:1 or you will

not be able to see the cells clearly or correct them with good accuracy). Once you are happy

with the size and placement of your box, double left click to confirm. The selected region will

be magnified to the secondary screen view and the borders of this region will be appeared in

the primary screen view.

3. When using the mouse-wheel up and down, the ROI will be shown in the primary screen

view. User can use the Drag zoom rectangle icon to move the ROI keeping the current

dimensions.

4. Go to Segmentation mode, and select ‘Segment current frame and save to HD’ option in the

popup menu.

5. To load the segmentation settings go to the File menu > import > (import segmentation settings

OR segmentation settings for current frame). The user can then adjust those settings to

improve segmentation of the current frame by altering values on the bottom right frame..

6. The user can remove individual pixels by clicking with middle mouse button (or mouse-wheel)

while in the ‘segmentation mode’.

Secondary screen view Primary screen view

Box containing the ROI

Secondary Screen showing the

zoomed-in ROI



Appendix I -21-


Separating merged cells.

TACTICS provides multiple options for manually separating merged cells, including:

1. Splitting a single object using watershed or intensity shell algorithms.

2. Splitting a single object by drawing a line through it or deleting individual pixels.

3. Improving segmentation of a single frame (as per previous page).

1. Splitting objects using watershed or intensity shell algorithms.

If a labeled object is clearly two or more adjoined cells, move the cursor over the box surrounding

the object, and left mouse click (watershed) or right mouse click (intensity shell) in the

primary screen view. If the algorithm succeeds, the bounding box should split into two boxes.

If the algorithm fails, the alternative algorithm can be attempted. After separating all the

adjoined cells in a frame scroll forward or back through the movie to save the changes.

2. Splitting objects by drawing a line or deleting pixels.

a. To separate between cells by drawing a line of deleting pixels use the Secondary Screen

(See the previous page for instructions on how to access the secondary screen using the

capture ROI icon ).

b. Draw a line where cells need to be divided. To do this, left click at the first point of where your

line should be and a yellow dot will appear. Left click again at the same point and drag the

blue line to divide your cells. An intensity line will go between the cells to give two labelled

cells.

c. Double left click on the blue line to accept changes.

d. Click middle button (wheel) or CTRL+A to save changes.

e. The properties of each segment that are saved in the .dat experiment file (described in

Appendix 1), and bounding box will appear for each segment (in the Primary Screen). In the

secondary screen, outlining of the cells by different colors indicates successful segmentation.

f. Run tracking algorithm again . This is required to incorporate the newly segmented

cells, and can be performed after resegmenting one or several frames.

I M P O R T A N T: Once this data file is created and loaded into Cell Tracking Module, any

modification in the data file will be logged in the loaded data file. Therefore it is necessary to

save the data file once processing is applied.

Initial problem cell

First point identified

Blue line drawn

Final product (two defined cells)


Appendix I -22-


Changed threshold value

3. Splitting objects by altering segmentation parameters.

Resegmenting the entire frame can also help with splitting two cells, although care must be taken

that further segmentation problems do not occur in other areas of the frame. Below is an example

where two joined cells were separated by increasing the threshold value as per instructions in A

(p 18).

Optional. Changing the LUT is a standard procedure that helps to distinguish between pixel

intensities. An example shown below shows that pixel intensities in the cell perimeter have low

value, causing over-segmentation by Otsu’s method.


Appendix I -23-


Removal of segmented pixels that are falsely identified as cells:

Sometimes a new object appears during segmentation correction that is incorrectly labeled as a

cell. An example of this is shown below:

Instructions for removal:

1. Zoom-in on the problem area in the secondary screen until each pixel is clearly visible.

2. Right click on one of the pixels to remove the colored dots indicating the segment edge.

3. You can now remove each unwanted pixel by right clicking on it. To replace a pixel that you

have deleted right click in the same place. Similarly, right clicking on a black pixel converts it

from 0 to1, and so can be used to link separated cells.

Cell Tracking Module incorrectly recognizes this

as a cell

Shot of secondary screen with pixels

clearly visible After one right click

After pixels had been removed.


Appendix I

The tracking accuracy depends on the quality of the raw data, the imaging system, the biological

system, and the computational approach to analyze the data. Common problems with tracking

are when a cell suddenly moves a large distance. Tracks can be changed manually using several

approaches, that either involve moving track vectors on the primary screen, or use a pull down

menu in which cells can be connected by numbers.

To correct cell association tracks by shifting connecting vectors:

1.Choose (n-1) in the secondary axes.

2.Click association button.

3.Drag the lines between the middle f the cell bounding-box (n-1) to its origin in n.

4.Click on the green icon in the icon panel to accept changes. Cell without match between n and

(n-1) marked by blue centroid. Cells with match are marked in red and id number. Watch the

video tutorial for more information.

Instructions for manual corrections for tracking:

1.Go to edit tracks mode.

2.Point with the mouse curser on required cell and click on the middle button. A selection window

will give several different options.

•IMPORTANT: the manual corrections are applied on the matrix of trajectories. Therefore, when

applying the linker again, the manual corrections are deleted. For this reason, it is always better

to apply manual corrections for cell association.

-24-

Cell Tracking Module – 4 (cell tracking corrections)

Video tutorial

4


Appendix I -25-

Cell Tracking Module –5 (marking cell division and death)

•Marking cell division

•Go to the (Mark cell division) mode

•Go to the last frame before division, where only one cells in visible

•In the primary screen, wheel click on the dividing cell

•Click on the (tracking splitter) icon

•Parental cells will have name ##-**, whereas ## is the name chosen by the user and ** is a

number by order of appearance.

•Daughter cells of parental cells will have name ##-**.1 and ##-**.2, whereas ##-** is the name of

the parental, and 1 and 2 symbolise daughter 1 and 2.

•For each additional cell division a dot and number 1 or 2 will be added. For instance:

##-**.1.2.1.2.2

1- Cell division mode

4-click tracking splitter


3- wheel click on cell,

Adding red x annotate the

cell as dividing cell

2- cell about to divide

5- (next frame) two cells

identified as two related

daughter cells and the

information is saved in

the experimental file

Frame n

Frame n

Frame n+1

Frame n+1

Frame n : Final frame before division


Appendix I

The painting tool allows manually to draw, fill, and remove pixels on any selected channels. This

can be very useful to correct segmentation when automated segmentation fails.

To apply the drawing tool:

1.Go to drawing tool mode.

2. Select show segmentation to “on”.

3.Select channel to apply segmentation, channel for visualization, and channel for tracking.

Zooming-in is recommended, but not a recruitment. The blue net will appear for each existing

pixel in the binary matrix. 4. Select the size of painting point. 5. Press mouse left click. Hold it and move the mouse curser to fill new pixels. 6. To remove pixels- Press mouse right click. Hold it and move the mouse curser. 7. To accept: click mouse wheel button. This will save the modified image as segmented one (overwrite) and will label again (relabeling should be saved to experimental file otherwise this will be lost). 8. To cancel (before acceptance): just use mouse wheel to refresh frame (go to the next or previous frames)

9. To cancel (after acceptance): currently TACTICS doesn't have this feature (is that useful?). Therefore need to segment the frame again.

-26-

Cell Tracking Module – 6 (drawing tool)

1- drawing tool mode

2- click show segment on

3- select channels

4- select point size


Appendix I

The parameter selection window ddisplay parameters such as intensity, and velocity for a particular selected cell, which allow to improve the cell tracking. To activate the parameter selection window :

1. Select cell. 2. Choose- ‘plot data for selected cell’ option. 3. New Manu will popup. Choose parameter to be plotted. (more parameters can be easily

added). 4. When using the mouse scroll wheel, the plot will be shown in the secondary figure view for the relevant time points.

-27-

Cell Tracking Module – 7 (Parameter selection)

1- select cell

2- select mode

3- select parameters in popup menu

4- parameter is plotted

Display number of objects in the current frame


Appendix I

Complicated switch in tracking can accrues when cells drift and tracking is incorrect for the following tracks. This can strongly influence the lineage tree and its analysis.

To correct tracks for lineage trees:

1. From main Go to >’Tools’>> ‘Generate Lineage tree’. A popup window shows the lineage for any selected cell. In this particular example it can be seen that a problem accrues in the tracking. Click on the lineage window interactively goes to the frame of selection in the Cell

Tracking Module. 2. In some cases other supporting tools can help to allocate events suspected as wrong. For

instance the cell tracks window. From main Go to >’Tools’>> ‘open cell tracks window’. Other tools can be useful Go to >’ Data inspection’ , such as show number of objects over

time, show trajectories matrix and search for undefined association. 3. Adjust the current settings showing for frame n and (n-1). 4. To visualize the association between cells at particular frame click on Centy button. In this

particular example, because (n-x) is set to -1, the cells in DIC in frame 382 but the tracks and bounding boxes are actually shown for frame 381. This provides the locations of the

cells in the previous frame. The Centy button shows that a new cell was defined (blue), and that the second daughter cell was wrongly labelled as cell 1.1.2.2.

-28-

Cell Tracking Module – 8 (corrections of lineage trees)

1- Generate Lineage tree 2- (optional) open cell tracks window

or use other inspection tools

4- Click on centy

3.a.- input frame to show to be -1

3.b.- input channels to

visualize frame n-1

3.c.- input channels to

track and visualize frame n


Appendix I

cell association tracks by shifting connecting vectors

5. The user can move the pink lines, so the pink star * touch the destination location in

frame 383. In this case a line was stretched from cell number 7 to 1.1.2.2, while the line that start. In this example, from the cell between 1.1.2.2 and 1.1.2 was pointed back on itself. This mean that this cell doesn't Have a much and will therefore recognized as a new cell. To accept click the cell association correction button.

6. After the correction, run the linker again. The lineage tree is now corrected. If the user regrets or want to repeat the process, labelling the frame again goes to initial association.

-29-

Cell Tracking Module – 8 (corrections of lineage trees)

5- drag lines to link between cells

6- lineage tree is corrected

Two-dimensional graphic visualization of lineage tree. Bifurcations in the tree represent cell divisions. X-axis

represents the cell index. Y-axis represents the frame (or time) that each cell appear.


Appendix I

Selective operator is method that allows the user to select what frames will be associate or

skipped. The user can select by frequency to be skipped (for instance every 3 rd frame) or/and

certain frames in the movie. The associated frames appear in green bars under the movie

sliderbar. Only the frames between following green bar coded frames are associated.

-30-

Cell Tracking Module – 9 (selective operator)

Selective operator

Number of cells in frame in

selective mode

Green bars shows the associated

frames

Trajectories matrix shows association

between every 10 frames

Selective operator

Modes

Show frames Show tracks

Show tracks

annotations

Non-Selective All frames Only in selected frames Only in selected frames

Selective All frames Only in selected frames Only in selected frames

Only Selective Skip to selected frames Skip to selected frames Skip to selected frames

Partly Selective All frames All frames Only in selected frames

interactive matlab platform for bioimaging informatics user...

Documents