nextgenerationcorrelatorworkshop@groningen nextgenerationcorrelatorworkshop@groningen 2006.6.28...
TRANSCRIPT
NextGenerationCorrelatorWorkshop@GroningenNextGenerationCorrelatorWorkshop@Groningen 2006.6.282006.6.28
Kondo, T. Kondo, T. *1*2*1*2 and T. Hobiger and T. Hobiger*2*2
*1 Kashima Space Research Center,*1 Kashima Space Research Center, National Institute of Information and National Institute of Information and Communications Technology, JapanCommunications Technology, Japan *2 Vienna University of Technology*2 Vienna University of Technology
Kondo, T. Kondo, T. *1*2*1*2 and T. Hobiger and T. Hobiger*2*2
*1 Kashima Space Research Center,*1 Kashima Space Research Center, National Institute of Information and National Institute of Information and Communications Technology, JapanCommunications Technology, Japan *2 Vienna University of Technology*2 Vienna University of Technology
An XF-type Software An XF-type Software Correlator Developed by Correlator Developed by
NICTNICT
An XF-type Software An XF-type Software Correlator Developed by Correlator Developed by
NICTNICT
ContentsContents
• Software correlators at KashimaSoftware correlators at Kashima• History of the processing speed of XF-tyHistory of the processing speed of XF-ty
pe software correlatorspe software correlators• Details of an XF-type software correlatoDetails of an XF-type software correlato
r for geodetic user for geodetic use• Distributed processingDistributed processing• ConclusionsConclusions
• Software correlators at KashimaSoftware correlators at Kashima• History of the processing speed of XF-tyHistory of the processing speed of XF-ty
pe software correlatorspe software correlators• Details of an XF-type software correlatoDetails of an XF-type software correlato
r for geodetic user for geodetic use• Distributed processingDistributed processing• ConclusionsConclusions
Software Correlators at NICTSoftware Correlators at NICT
*This software correlator is very fast, but not for geodetic use right now
K5/VSSP
K5/VSI
for K5/VSSP (geodetic use)
for K5/VSI (gigabit VLBI system)
FX-Type Correlator
(developed by Kimura-san)*
XF-Type Correlator (for 1bit AD data) (K5-cor)
FX-Type Correlator (for multi-bit AD data) (K5-fx_cor)
History of the processingHistory of the processingspeed of XF-type speed of XF-type
software correlatorssoftware correlators
History of Processing Speed (XF-type)History of Processing Speed (XF-type)(converted to 32-lag complex correlation)(converted to 32-lag complex correlation)
Thr
ough
put
(bps
)T
hrou
ghpu
t (b
ps)
YearYear1960 1970 1980 1990 2000 2010
1M
10M
100M
1G
100K
10K
1K
100
Mark-IMark-I
(IBM360/50)
K-1K-1
K-3K-3
(mini-computer)
CONTROL COMPUTER TELEPHONE LINESTELEPHONE LINES
COMPUTER MT
HOST COMPUTERHP-1000 45F
RECEIVER
K-3 FORMATTER
K-3 TAPE
K-3 DECODER(1Mbit BUFFER MEMORY)
MODEM
RECEIVER
K-3 FORMATTER
K-3 TAPE
K-3 DECODER(1Mbit BUFFER MEMORY)
CONTROL COMPUTER
CONTROL COMPUTER
COMPUTER MT
MODEM
KASHIMA MIYAZAKI
Historical e-VLBI Experiment using Software Correlator Historical e-VLBI Experiment using Software Correlator in JAPAN (1986) in JAPAN (1986)
1200bps
It took about 10 hours to get fringes!
KASHIMA
MIYAZAKI1000km
History of Processing Speed (XF-type)History of Processing Speed (XF-type)(converted to 32-lag complex correlation)(converted to 32-lag complex correlation)
Thr
ough
put
(bps
)T
hrou
ghpu
t (b
ps)
YearYear
Mark-IMark-I
K-1K-1
Moore's LawMoore's Law
K-3K-3(IBM360/50)
1960 1970 1980 1990 2000 2010
1M
10M
100M
1G
100K
10K
1K
100(mini-computer)
K-5K-5(PC)
Details of an Details of an
XF-type software correlator XF-type software correlator
for geodetic usefor geodetic use
Requirements for a Software Requirements for a Software Correlator for Geodetic UseCorrelator for Geodetic Use
• Compatible with conventional hardware correlators, such as K3, KSP correlators– Consistent definitions in delay, clock param
eters, etc.– PCAL signal detection– Checking bit slip or make
• Both K5 and Mark-5 data processing capability
Architecture of K5-cor CorrelatorArchitecture of K5-cor Correlator
ImagImagcorrcorr
X(t)X(t)
Y(t)Y(t)
fringe stoppingfringe stopping
cos corr.cos corr.N lagN lag
sin corr.sin corr.N lagN lag
RealRealcorrcorr
X-Pcal detectionX-Pcal detection
Y-Pcal detectionY-Pcal detection
X Pcal X Pcal phasephase
Y Pcal Y Pcal phasephase
header check
header check
)(cos t
)(sin t
delay trackingdelay tracking
)(t
How to get 8-lag cross-correlationHow to get 8-lag cross-correlation
1Byte（ 8bits) data
refer look-up table as a function of 1byte-integer(X) and 2byte-integer(Y)
1001 1 010MSBLSB
= 128+32+8+1= 169
0001 1 011
MSBLSB
= 32768+8192+4096+1024+128+64+16+4+1= 46293
X
Y
correlation counts for 0 lag = cortable0(169,46293) =6
1011 1 010
16bits data
×○ ○○○○×○
How to get 8-lag cross-correlationHow to get 8-lag cross-correlation
1Byte（ 8bits) data
refer look-up table as a function of 1byte-integer(X) and 2byte-integer(Y)
1001 1 010MSBLSB
= 128+32+8+1= 169
0001 1 011
MSBLSB
= 32768+8192+4096+1024+128+64+16+4+1= 46293
X
Y
correlation counts for 0 lag = cortable0(169,46293) =6 1 lag = cortable1(169,46293) =2
1011 1 010
16bits data
× ○× ○× × × ×
How to get 8-lag cross-correlationHow to get 8-lag cross-correlation
1Byte（ 8bits) data
refer look-up table as a function of 1byte-integer(X) and 2byte-integer(Y)
1001 1 010MSBLSB
= 128+32+8+1= 169
0001 1 011
MSBLSB
= 32768+8192+4096+1024+128+64+16+4+1= 46293
X
Y
correlation counts for 0 lag = cortable0(169,46293) =6 1 lag = cortable1(169,46293) =2 ……………………. 7 lag = cortable7(169,46293)=0
1011 1 010
16bits data
× × ×× × × × ×
Actual look-up table isActual look-up table iscortable(169,46293)=
table size (8lags) =28x216x4 bytes =67Mbytes
table size (16lags) =28x232x8 bytes =8.8Gbytes too large
0 001 0 101 .... . ... .... . ... 00.. . 00.
lag 0 lag 1 lag 7
32-bit integer
lag 2 lag 3 lag 4 lag 5 lag 6
Mark5 to K5 converter (m5tok5)Mark5 to K5 converter (m5tok5)
Analyze Analyze bit length,bit length,
bit position bit position vs track#vs track#
Analyze Analyze bit length,bit length,
bit position bit position vs track#vs track#
CIFCIFCIFCIF
VEX FileVEX File
Mark5 FileMark5 File
ConversionConversionInformationInformationFileFile
Mark5 FileMark5 File
CIFCIFCIFCIF
Get Get AD bits, fanout, AD bits, fanout,
track#, CH#track#, CH#informationinformation
Get Get AD bits, fanout, AD bits, fanout,
track#, CH#track#, CH#informationinformation
Mark5 Mark5 K5 K5conversionconversion
Mark5 Mark5 K5 K5conversionconversion
K5 FileK5 File
information fileinformation filecreation modecreation modeinformation fileinformation filecreation modecreation mode
conversion modeconversion modeconversion modeconversion mode
・・・・・・・0t 1t 2t 3t 4t
・・・・・・・
LSB bit#0
MSB bit#31
bit #n
Mark5 Data File FormatMark5 Data File Format
Trk #k
ｔ・・・
Mark IV
・・・
header160bits
data20000bits
20160bits/frame
VLBA
・・・
sampled data
header160bits
data19840bits
20000bits/frame
Mark IV Header block format
block#
word#
byte#
bit# 1
2
3
4
5
7
6
8
P
1 2 3 4
1
1 2 3 4
2
1 2 3 4
3
1 2 3 4
5
1 2 3 4
4
1 2
1・・・ 5
・・・・
AuxiliaryData
(16 hexa)
Syncword
(all “1”)
Y10
0D D
10H
10D
H M10
M
S10
S
0.01
S0.
1S
0.00
1S
CRC-12
code
Sampleddata
(20byte)
・・・・
odd parity even (“0”) odd parity odd
Auxiliary field Sync word Time field
Header blockDatablock
FRAME BORDER
VLBA Header block format
block#
word#
byte#
bit# 1
2
3
4
5
7
6
8
P
1 2 3 4
1
1 2 3 4
2
1 2 3 4
3
1 2 3 4
5
1 2 3 4
4
1 2
1・・・ 5
・・・・
AuxiliaryData
(16 hexa)
Syncword
(all “1”)
10M
JD
1000
0S
1000
S
MJD
100S S
10S
0.01
S0.
1S
0.00
01S
0.00
1S
CRC -16
code
Sampleddata
(20byte)
・・・・
odd parity even (“0”) odd parity odd
Auxiliary field Sync word Time field
Header blockDatablock
100M
JD
FRAME BORDER
K5/VSSP Data File FormatK5/VSSP Data File Format
word#
byte#
bit# 1
2
3
4
5
7
6
8
1 2 3 4
1
1 2 3 4
2
1 2 3 4
3
1 2 3 4
4
Syncword
(all “1”)
tim
e of
day
(se
c)(1
7bit
) chsa
mpl
e fr
eqA
Dbi
ts
sync
byt
e (“
8Bh”
)
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
sampled data
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
ch1
ch2
ch3
ch4
header data
ｔ
・・・・
(in case of 4ch data)
K5/VSSP Data File FormatK5/VSSP Data File Format
sampled dataheader data
ｔ
word#
byte#
bit# 1
2
3
4
5
7
6
8
1 2 3 4
1
1 2 3 4
2
1 2 3 4
3
1 2 3 4
4
Syncword
(all “1”)
tim
e of
day
(se
c)(1
7bit
) chsa
mpl
e fr
eqA
Dbi
ts
sync
byt
e (“
8Bh”
)
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
ch1
・・・・
(in case of 1ch data)
K5K5－－ Mark5 FringesMark5 FringesOct. 15, 2002
Kashima - Westford
X band S band
4C39.25
Distributed processingDistributed processing
VLBI@homeVLBI@homeDistributed Correlation Processing
Server
huge VLBI data
huge VLBI data
developed byTakeuchi-san
GSI has developed similar distributed system
Improvement Factor Improvement Factor Total Processing Speed using N PCs
Total Processing Speed using one PC=
• Assumptions– Data are transferred from a server– Data transfer does not affect the speed of
correlation processing – Correlation processing is possible with a data
transfer for a different scan– Speed of data transfer reduces to 1/N in case
of N parallel data transfers
Improvement Factor Improvement Factor Im
prov
emen
t F
acto
r
k (=network speed / processing speed)
10 scansN=20
N=10
N=2
100Mbps1Gbps
Total Processing Speed using N PCs
Total Processing Speed using one PC=
Total Processing SpeedTotal Processing Speedin case of the use of in case of the use of NN PCs PCs
Network Speed >> Processing Speed
Network Speed ~ Processing Speed
N1 32 4N times faster
same as one PC
Current Status of XF Software Current Status of XF Software CorrelatorCorrelator
• Correlation processing speed of K5-corCorrelation processing speed of K5-cor– 17Mbps17Mbps for 32-lag complex correlation for 32-lag complex correlation– 34Mbps for 16-lag* complex correlation34Mbps for 16-lag* complex correlation
• Network speed for distributed processingNetwork speed for distributed processing– 100Mbps ~ 1Gbps (at office), 100Mbps ~ 1Gbps (at office), 10Gbps10Gbps (available)(available)
• Total processing speed of distributed processing Total processing speed of distributed processing can becan be– ~~170Mbps170Mbps (32-lag, use of 10 PCs) (32-lag, use of 10 PCs)– ~~340Mbps 340Mbps (32-lag, use of 20 PCs) (32-lag, use of 20 PCs)– ~~1.7Gbps1.7Gbps (32-lag, use of 100 PCs) (32-lag, use of 100 PCs)– ~~3.4Gbps 3.4Gbps (32-lag, use of 200 PCs) (32-lag, use of 200 PCs)
• Correlation processing speed of K5-corCorrelation processing speed of K5-cor– 17Mbps17Mbps for 32-lag complex correlation for 32-lag complex correlation– 34Mbps for 16-lag* complex correlation34Mbps for 16-lag* complex correlation
• Network speed for distributed processingNetwork speed for distributed processing– 100Mbps ~ 1Gbps (at office), 100Mbps ~ 1Gbps (at office), 10Gbps10Gbps (available)(available)
• Total processing speed of distributed processing Total processing speed of distributed processing can becan be– ~~170Mbps170Mbps (32-lag, use of 10 PCs) (32-lag, use of 10 PCs)– ~~340Mbps 340Mbps (32-lag, use of 20 PCs) (32-lag, use of 20 PCs)– ~~1.7Gbps1.7Gbps (32-lag, use of 100 PCs) (32-lag, use of 100 PCs)– ~~3.4Gbps 3.4Gbps (32-lag, use of 200 PCs) (32-lag, use of 200 PCs)
* 16-lag is good enough for geodetic VLBI
more realistic
1980 2000 20101990 2020 20301M
1G
1T
1P
LANWAN
YEAR
SP
EE
D (
bps)
Road Map of Network SpeedRoad Map of Network Speedand K5-cor Processing Speedand K5-cor Processing Speed
10G
100G
WAN & LAN fromhttp://www8.cao.go.jp/cstp/project/super/haihu02/siryo3-sanko.pdf
K5-cor
?
Expectation at 2010Expectation at 2010
• Correlation processing speed of K5-corCorrelation processing speed of K5-cor– ~100Mbps for 32-lag complex correlation~100Mbps for 32-lag complex correlation
• Network speed for distributed processingNetwork speed for distributed processing– ~100Gbps~100Gbps
• Total processing speed of distributed processingTotal processing speed of distributed processing– ~ ~ 1Gbps1Gbps (32-lag, use of 10 PCs) (32-lag, use of 10 PCs)– ~ ~ 2Gbps2Gbps (32-lag, use of 20 PCs) (32-lag, use of 20 PCs)– ~ ~ 10Gbps10Gbps (32-lag, use of 100 PCs) (32-lag, use of 100 PCs)– ~ ~ 20Gbps20Gbps (32-lag, use of 200 PCs) (32-lag, use of 200 PCs)
• Correlation processing speed of K5-corCorrelation processing speed of K5-cor– ~100Mbps for 32-lag complex correlation~100Mbps for 32-lag complex correlation
• Network speed for distributed processingNetwork speed for distributed processing– ~100Gbps~100Gbps
• Total processing speed of distributed processingTotal processing speed of distributed processing– ~ ~ 1Gbps1Gbps (32-lag, use of 10 PCs) (32-lag, use of 10 PCs)– ~ ~ 2Gbps2Gbps (32-lag, use of 20 PCs) (32-lag, use of 20 PCs)– ~ ~ 10Gbps10Gbps (32-lag, use of 100 PCs) (32-lag, use of 100 PCs)– ~ ~ 20Gbps20Gbps (32-lag, use of 200 PCs) (32-lag, use of 200 PCs)
more realistic
ConclusionsConclusions• Processing speed of software correlator Processing speed of software correlator
(K5-cor : XF type) for geodetic use is (K5-cor : XF type) for geodetic use is 17Mbps17Mbps for 32-lag complex correlation for 32-lag complex correlation
• 170Mbps~340Mbps170Mbps~340Mbps for distributed for distributed processing with 10~20 PCs will be processing with 10~20 PCs will be available soonavailable soon
• About About 10~20Gbps10~20Gbps for distributed for distributed processing with 100~200 PCs will be processing with 100~200 PCs will be expected by expected by 2010, 2010, if we don’t care about if we don’t care about “costs” and “save-energy”.“costs” and “save-energy”.
• Processing speed of software correlator Processing speed of software correlator (K5-cor : XF type) for geodetic use is (K5-cor : XF type) for geodetic use is 17Mbps17Mbps for 32-lag complex correlation for 32-lag complex correlation
• 170Mbps~340Mbps170Mbps~340Mbps for distributed for distributed processing with 10~20 PCs will be processing with 10~20 PCs will be available soonavailable soon
• About About 10~20Gbps10~20Gbps for distributed for distributed processing with 100~200 PCs will be processing with 100~200 PCs will be expected by expected by 2010, 2010, if we don’t care about if we don’t care about “costs” and “save-energy”.“costs” and “save-energy”.