national radio astronomy observatory - cv.nrao.eduaroshi/beamformer/summer... · arnab pramanik (...

National Radio Astronomy Observatory

By

Arnab Pramanik

D. Anish Roshi

William Shillue

07/15/15 – 08/15/15

1

Arnab Pramanik ([email protected]) 08-12-2015


Index:

INTRODUCTION …………………………………………………………………… 02

BEAMFORMER BACK-END ……………….…………………………………..….. 02

SOFTWARE PACKETIZER ………………………………………………………... 03

TEST RESULTS …………….………………...…………………………………….. 06

APPENDIX …………………………………………………………………………... 08

mailto:[email protected]

2



INTRODUCTION

Phased Array feed (PAF) are specialized feed when placed at the focus of a parabolic dish antenna can be used to form multiple beams. The beams are formed by combining signals from different elements in the feed array. A PAF and a hardware backend for forming beams (referred to as beamformer) are being built for the Green Bank Telescope. The PAF operates at L band and it has an operating frequency range from 1.3 GHz to 1.8 GHz. The PAF has 19 dual polarization dipoles and the front end system provides 38 signals from the two polarization (See figure: 1)

Figure 1: Beam former of Phased Array Feed

Beamformer Back-end :

The back end 40 channels (38 channels + 2 extra) which are connected to the signals coming from the front end. The back end system consists of five ROACH-II boards with ADCs. Each board gets 8 channels from the PAF. Each ROACH-II has a Virtex 6 FPGA. The FPGA performs 512 point PFB and produces 512 frequency channels. From the 512 frequency channels we take 500 channels and sent them to 10 GPUs to form beams. The 500 frequency channels are distributed to 10 GPUs thus each GPU processes 50 frequency channels (See Figure: 12). The FPGA based packetizer packetizes samples from the PFB output and sent them to 10 GPUs over an 10 Gbps Ethernet link (See figure: 2). As the FPGA based packetizer is still under development we needed a simulator of the packetizer to let rest of work of the beamformer going at the GPU side. The project was to design a software packetizer which performs similar to the hardware packetizer.


3



Figure 2: Block diagram of FPGA based packetizer

Software packetizer:

The software packetizer was developed in C. The software packetizer has no physical input. So we send a sine signal with variable phase in a user defined frequency channel. The channel and the phase of the sine signal can both be varied by the user.


4




5



Figure 3: Packet format of the packetizer

Figure: 3 is the packet format defined by BYU. The packet consists of 8008 bytes. In that 8008 bytes 8 bytes is the header and rest of the 8000 bytes are data. The header consist 44 bits of packet count, 4 bits of cal, 8 bits of ROACH-II number and 8 bits of GPU number. The PFB outputs are complex values, represented with 1byte real and 1byte imaginary. So for a single GPU we get 800 bytes from the 8 inputs channels connected to the ROACH II (2 x 8 x 50). As we combine 10 time samples in a packet we get 8000 bytes (800x10) of data.

Figure 4: Multithreaded software packetizer

Figure: 4 shows the architecture of the program. The software runs five different threads to simulate five different ROACH-II packetizers. Each thread can be switched ON and OFF by the user depending upon the usage. The user also gets the flexibility to choose the number of the GPUs which the data can be transmitted to. Each thread creates a socket to send the data using UDP protocol. Each GPU receives data from all the five thread. To test our software we have used personal computers to transmit and receive the data.


6



Figure 5: packet transmission through network at different OSI layers

Figure: 5 shows the encapsulation and de-capsulation of the packet at different OSI layers. The data get packetized in the server side and get transmitted through the network. The client side receives the packet and de-packetize removing each header at different level of the OSI layer and provides the data to the user.

Test Result

We have tested the system for two different configurations. For the first configuration we have transmitted the data and have received in the same PC. Figure: 6 shows the plot of the packet count with respect to the number of the packets received. We can see the packet counts increases linearly with the number of packets without any discontinuity for the five threads. The plots also show that the threads are synchronized and start and stop at the same time. The packetcount from each thread is plotted with an offset for clarity.

Figure 6: Plot of pack count with respect to number of packets for configuration I


7



For the second configuration we have transmitted the data to a remote desktop through NRAO network and performed the same test as above. The plot of packet count vs received packet number is shown in figure 7. In this case there are packet loss, which is due to the network traffic. In the application for testing beamformer we have a dedicated network and so packet loss is expected to be negligible. But the 2 PC test confirms the packetizer works as expected and all the 5 threads are operating synchronously.

Figure 7: Plot of pack count with respect to number of packets for configuration II


8



Appendix:

bfpacketizer_params.h

#define PACKET_SIZE 8008 // Packet Size in Bytes #define PORT_NUM 50000 // Port Number for UDP Server #define Ninputs 40 // Number of analog inputs #define Nfengines 5 // Number of Roach-II boards used as F-engines #define Nxengines 10 // Number of GPUs used for correlations #define Ntime_per_packet 10 // Number of time stamps in a packet #define Nbins 500 // Number of freq channels used for correlations #define Nfft 512 // Number of PFB freq channels #define Nin_per_f() Ninputs/Nfengines; // Number of analog inputs to each Roach-II board #define Nbin_per_x() Nbins/Nxengines; // Number of freq channels processed in a GPU // Ip address of the GPUs const char *ipaddr[] = {"10.12.97.179", "10.12.96.248", "192.168.4.254", "192.168.4.254", "192.168.4.254", "192.168.4.254", "192.168.4.254", "192.168.4.254", "192.168.4.254", "192.168.4.254"};

C code of the Packetizer with UDP Server

/*Anish & Arnab July 2015 * * This code emulates 5 Roach board sending packets as defined * by BYU to Kmax GPU machines. The packets are send by 5 threads. * The ipaddress of the GPUs are defined in bfpacketizer_params.h * * Arnab & Anish Aug 2015 */ #include<stdio.h> #include<stdlib.h> #include<math.h> #include<complex.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <netdb.h> #include <signal.h> #include <sys/times.h> #include <sys/time.h> #include <time.h> #include <unistd.h> #include <getopt.h> #include <arpa/inet.h>


9



#include <pthread.h> #include "bfpacketizer_params.h" #define TOTAL_DATA 10.0 unsigned long long packet_count = 0x0; //Global packet_count for thread synch struct hostent *hh[Nxengines]; struct sockaddr_in ip_addr[Nxengines]; int slen; int sock[Nfengines]; float wait_cyc=0; float total_data = TOTAL_DATA; int StartSendingPacket=0; int packet_size = PACKET_SIZE; int endian=0; int run=1; void cc(int sig) { run=0; } void usage() { fprintf(stderr, "Usage: bfpacketizer0.1 (options) \n" "Options:\n" " -p nn, --port=nn Port number (%d)\n" " -s nn, --packet-size=nn Packet size, bytes (%d)\n" " -d nn, --total-data=nn Total amount to send, GB (%.1f)\n" " -q, --quiet More compact output\n" " -w nn, --wait=nn Wait 1000*nn cycles (0)\n" " -c, --sine_channel Channel number for sine wave (0 to 499)\n" " -a, --sine_phase Phase of the sine wave in rad \n" " -e, --endian Byte-swap seq num\n" , PORT_NUM, PACKET_SIZE, TOTAL_DATA); } //Thread Variable for the packetizer struct send_packet_data{ unsigned char m; unsigned char cal; int Kmax; int SineCh; float SinePh; int SinePer; int UseGlobalPacketCount; }; //Byte swap void byte_swap(unsigned long long *d) { unsigned long long tmp; char *ptr1, *ptr2; ptr1 = (char *)d; ptr2 = (char *)&tmp + 7; int i; for (i=0; i<8; i++) { ptr1 = (char *)d + i; ptr2 = (char *)&tmp + 7 - i; memcpy(ptr2, ptr1, 1);


10



} *d = tmp; } //Header generation function void header_gen(unsigned char *packet_header, unsigned long long packet_count, unsigned char m, unsigned char k, unsigned char cal) { int p; unsigned long long header; cal = (cal & 0xF); //Fill header with packet_count, cal, m and k header = packet_count; header = ((header << 4) | cal); header = ((header << 8) | m); header = ((header << 8) | k); //Fill packet_header with the header. for (p = 7;p >= 0; p--){ packet_header[p] = (unsigned char)(header); header = (header >> 8); } } char quantize(double trig) { return (char) round(trig*127); } //Payload generation void payload_gen(char *data, unsigned char K, int SineCh, float SinePh, int SinePer) { int t, i, cc, ch; int Nin_per_f = Nin_per_f(); /* Number of inputs per ROACH */ int nbins = Nbins; int ntime_per_packet = Ntime_per_packet; int ContChan = 5; /* Contiguous channels to be packed */ int ChanInc=50; /* Contiguous channels are incremented in this step */ int SetOfChan, DatIndx, DatSizeForAtime, PaylSize, DatSizeForNin,chan,DatSizeForContChan; SetOfChan = nbins/ChanInc; /* Number of contiguous set of channels in payload */ DatSizeForAtime = Nin_per_f*ContChan*SetOfChan*2; /*Total data size in byte for a time stamp */ DatSizeForContChan = Nin_per_f*ContChan*2; DatSizeForNin = Nin_per_f * 2; /*Total data size in byte for all inputs per ROACH */ PaylSize = DatSizeForAtime * ntime_per_packet; /*Total payload size in bytes */ for(t = 0; t < ntime_per_packet; t++){ /* Loop through time samples */ for(cc = 0; cc < SetOfChan; cc++){ /* Loop through contiguous set of channels*/ for(ch = 0; ch < ContChan; ch++){ /* Loop through contiguous channels */ chan = ContChan*K + ch + cc*ChanInc; /* Channel number */ for(i = 0; i < Nin_per_f; i++){ DatIndx = i*2 + ch*DatSizeForNin + cc*DatSizeForContChan + t*DatSizeForAtime; //printf("%d %d\n",chan,DatIndx); if (chan == SineCh){ data[DatIndx] = quantize(cos(SinePh)); data[DatIndx+1] = quantize(sin(SinePh));


11



} else{ data[DatIndx] = 0; data[DatIndx+1] = 0; } } } } } } //Send Packet of 8008 Bytes each void* send_packet(void *td) { struct send_packet_data* ThreadData = (struct send_packet_data*) td; char *packet = (char *)malloc(sizeof(char)*packet_size); int k, rv; unsigned long long lpacket_count=0; double byte_count=0; while(StartSendingPacket == 0){ if(run == 0){ exit(0); } } while (run && (byte_count<total_data || total_data<=0)) { for(k=0; k < ThreadData->Kmax; k++){ //header generation function call header_gen(packet, lpacket_count, ThreadData->m, (unsigned char) k, ThreadData->cal); //printf("%d %x\n", (int)ThreadData->m, *(packet+6)); //payload generation function call payload_gen(packet+8, (unsigned char) k, ThreadData->SineCh, ThreadData->SinePh, ThreadData->SinePer); rv = sendto(sock[ThreadData->m], packet, (size_t)packet_size, 0, (struct sockaddr *)&ip_addr[k], slen); if (rv==-1) { perror("sendto"); exit(1); } if(ThreadData->UseGlobalPacketCount){ packet_count++; lpacket_count = packet_count; } else{ lpacket_count++; } int i; for (i=0; i<(int)(1000.0*wait_cyc); i++) { __asm__("nop;nop;nop"); } } byte_count += (double)packet_size; } free(packet); }


12



void main(int argc, char *argv[]) { int Kmax = Nxengines; //1; /*Maximum no. of Xengines used*/ int Mmax = Nfengines; //5; /*Maximum no. of Fengines used*/ unsigned char cal = 0x0; int mcnt, hcnt; int port_num = PORT_NUM; int SineCh = 0; float SinePh = 0; int SinePer = 0; int rv; signal(SIGINT, cc); /* Cmd line */ static struct option long_opts[] = { {"help", 0, NULL, 'h'}, {"port", 1, NULL, 'p'}, {"packet-size", 1, NULL, 's'}, {"total-data", 1, NULL, 'd'}, {"quiet", 0, NULL, 'q'}, {"wait", 1, NULL, 'w'}, {"endian", 0, NULL, 'e'}, {"sine_channel", 1, NULL, 'c'}, {"sine_phase", 1, NULL, 'a'}, {0,0,0,0} }; int quiet=0; int opt, opti; while ((opt=getopt_long(argc,argv,"hp:s:d:qw:ec:a:",long_opts,&opti))!=-1) { switch (opt) { case 'p': port_num = atoi(optarg); break; case 's': packet_size = atoi(optarg); break; case 'd': total_data = atof(optarg); break; case 'q': quiet=1; break; case 'w': wait_cyc = atof(optarg); break; case 'c': SineCh = atoi(optarg); break; case 'e': endian=1; break; case 'a': SinePh = atof(optarg); break; case 'h':


13



default: usage(); exit(0); break; } } if(!quiet){ printf("Mmax = %d , Kmax = %d\n", Mmax, Kmax); } /* Send packets */ if(!quiet){ printf("Sine Channel = %d \n", SineCh); printf("Sine Phase (rad) = %f \n", SinePh); printf("Sine Per = %d \n", SinePer); printf("cal = %x \n", cal); printf("total_data (in GB) = %f \n", total_data); } for (hcnt=0; hcnt<Kmax; hcnt++) { /* Resolve hostname */ hh[hcnt] = gethostbyname(ipaddr[hcnt]); if ((hh[hcnt])==NULL) { herror("gethostbyname"); exit(1); } /* Set up recvr address */ ip_addr[hcnt].sin_family = AF_INET; ip_addr[hcnt].sin_port = htons(port_num); memcpy(&ip_addr[hcnt].sin_addr, hh[hcnt]->h_addr, sizeof(struct in_addr)); if(!quiet){ printf("k=%d ipaddr=%s\n", hcnt, inet_ntoa(ip_addr[hcnt].sin_addr)); } } slen = sizeof(ip_addr[0]); //Mmax Thread ids pthread_t thread_send_packet[Mmax]; struct send_packet_data data_thread[Mmax]; int RoachUsingGlobalPacketCount = -1; for (mcnt=0; mcnt<Mmax; mcnt++) { /* Create socket */ sock[mcnt] = socket(PF_INET, SOCK_DGRAM, 0); if (sock[mcnt]==-1) { perror("socket"); exit(1); } //Data for each thread is set here data_thread[mcnt].m = mcnt; data_thread[mcnt].cal = cal;


14



data_thread[mcnt].Kmax = Kmax; data_thread[mcnt].SineCh = SineCh; data_thread[mcnt].SinePh = SinePh; data_thread[mcnt].SinePer = SinePer; if(mcnt == RoachUsingGlobalPacketCount){ data_thread[mcnt].UseGlobalPacketCount = 1; } else{ data_thread[mcnt].UseGlobalPacketCount = 0; } //Create Thread pthread_create(&thread_send_packet[mcnt], NULL, (void*)send_packet, &data_thread[mcnt]); } total_data *= 1024.0*1024.0*1024.0; /* change to bytes */ StartSendingPacket=1; int thread_done=1, ftc; //Thread Join to Synchronize for(ftc = 0; ftc < Mmax; ftc ++){ thread_done = pthread_join(thread_send_packet[ftc], NULL); if(thread_done != 0){ printf("Abnormal exist of thread %d \n", ftc); } } }

C code of the UDP Client

/* udp_recv.c * Receiver for UDP tests. * Paul Demorest 12-15-2007. */ #define _GNU_SOURCE 1 /* sched stuff doesn't work w/o this... */ #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <netdb.h> #include <signal.h> #include <unistd.h> #include <fcntl.h> #include <errno.h> #include <sys/times.h> #include <sys/time.h> #include <poll.h> #include <getopt.h> #include <pthread.h> #include <sched.h> #include "bfpacketizer_params.h" void usage() {


15



fprintf(stderr, "Usage: udp_recv (options) sender_hostname\n" "Options:\n" " -p nn, --port=nn Port number (%d)\n" " -s nn, --packet-size=nn Packet size, bytes (%d)\n" " -b nn, --buffer-size=nn Receiver buffer size, packets (2)\n" " -d file, --disk-output=file Output raw data to file\n" " -c n, --cpu=n Use only CPU #n\n" " -q, --quiet More compact output\n" " -e, --endian Byte-swap seq num\n" " -w, --wait Wait for packet 0\n" " -i n, --incr=n Seq num increment per packet (1)\n" " -a, --print Print packet seq nums\n" " -t nn, --timeout=nn Receive timeout, ms (1000)\n" " -n nn, --npacket=nn Stop after n packets\n" , PORT_NUM, PACKET_SIZE); } /* Use Ctrl-C for stop */ int run=1; void cc(int sig) { run=0; } struct write_info { //int fd; FILE *fd; char *buf; size_t nbytes; }; void *write_to_disk(void *args) { struct write_info *inf = (struct write_info *)args; size_t rv; size_t remain = inf->nbytes; char *wptr = inf->buf; while (remain>0) { //rv = write(inf->fd, wptr, remain); rv = fwrite(wptr, 1, remain, inf->fd); if (rv<0) { perror("write"); exit(1); } remain -= rv; wptr += rv; } pthread_exit(NULL); } void byte_swap(unsigned long long *d) { unsigned long long tmp; char *ptr1, *ptr2; ptr1 = (char *)d; ptr2 = (char *)&tmp + 7; int i; for (i=0; i<8; i++) { ptr1 = (char *)d + i; ptr2 = (char *)&tmp + 7 - i; memcpy(ptr2, ptr1, 1); } *d = tmp; }


16



int main(int argc, char *argv[]) { int rv; /* Cmd line */ static struct option long_opts[] = { {"help", 0, NULL, 'h'}, {"port", 1, NULL, 'p'}, {"packet-size", 1, NULL, 's'}, {"buffer-size", 1, NULL, 'b'}, {"disk-output", 1, NULL, 'd'}, {"cpu", 1, NULL, 'c'}, {"quiet", 0, NULL, 'q'}, {"endian", 0, NULL, 'e'}, {"incr", 1, NULL, 'i'}, {"print", 0, NULL, 'a'}, {"timeout", 1, NULL, 't'}, {"npacket", 1, NULL, 'n'}, {"wait", 0, NULL, 'w'}, {0,0,0,0} }; int port_num = PORT_NUM; int packet_size = PACKET_SIZE; int buffer_size = 2; int disk_out=0; char ofile[1024]; int cpu_idx=-1; int quiet=0, endian=0, print_all=0; int poll_timeout=1000, npacket=0; int wait_for_0=0; unsigned long long packet_incr=1; int opt, opti; while ((opt=getopt_long(argc,argv,"hp:s:qb:d:c:ei:at:n:w",long_opts,&opti))!=-1) { switch (opt) { case 'p': port_num = atoi(optarg); break; case 's': packet_size = atoi(optarg); break; case 'q': quiet=1; break; case 'b': buffer_size = atoi(optarg); break; case 'd': disk_out=1; strncpy(ofile,optarg,1024); break; case 'c': cpu_idx = atoi(optarg); break; case 'e': endian=1; break; case 'i': packet_incr = atoll(optarg); break;


17



case 'a': print_all=1; break; case 't': poll_timeout = atoi(optarg); break; case 'n': npacket = atoi(optarg); break; case 'w': wait_for_0=1; break; case 'h': default: usage(); exit(0); break; } } /* check args */ if (optind==argc) { usage(); exit(1); } /* Set CPU affinity */ cpu_set_t cpuset, cpuset_orig; if (cpu_idx>=0) { /* get current list */ sched_getaffinity(0, sizeof(cpu_set_t), &cpuset_orig); /* blank out new list */ CPU_ZERO(&cpuset); /* Add requested CPU */ CPU_SET(cpu_idx, &cpuset); /* use new settings */ rv = sched_setaffinity(0, sizeof(cpu_set_t), &cpuset); if (rv<0) { perror("sched_setaffinity"); exit(1); } } /* Create socket */ int sock = socket(PF_INET, SOCK_DGRAM, 0); if (sock==-1) { perror("socket"); exit(1); } /* Test that buffer_size is reasonable */ long long buffer_size_bytes = (long long)packet_size * (long long)buffer_size; if (buffer_size<0) { fprintf(stderr, "Buffer size is negative!\n"); exit(1); } if (buffer_size % 2) { buffer_size++; } if (buffer_size_bytes>1024*1024*1024) { /* 1 GB max */


18



fprintf(stderr, "Max buffer size is 1 GB\n"); exit(1); } /* Init buffer, use first 4 bytes as packet count */ char *buf = (char *)malloc(sizeof(char)*packet_size*buffer_size); /* Resolve hostname */ struct hostent *hh; hh = gethostbyname(argv[optind]); if (hh==NULL) { herror("gethostbyname"); exit(1); } //printf("ipaddr=%s\n", inet_ntoa(*(struct in_addr *)hh->h_addr)); /* Bind to local address */ struct sockaddr_in local_ip; local_ip.sin_family = AF_INET; local_ip.sin_port = htons(port_num); local_ip.sin_addr.s_addr = INADDR_ANY; unsigned slen=sizeof(local_ip); rv = bind(sock, (struct sockaddr *)&local_ip, slen); if (rv==-1) { perror("bind"); exit(1); } /* Set up address to recieve from */ struct sockaddr_in ip_addr; ip_addr.sin_family = AF_INET; //ip_addr.sin_port = htons(port_num); memcpy(&ip_addr.sin_addr, hh->h_addr, sizeof(struct in_addr)); //rv = connect(sock, (struct sockaddr *)&ip_addr, sizeof(ip_addr)); if (rv==-1) { perror("connect"); exit(1); } fprintf(stderr, "ipaddr=%s\n", inet_ntoa(*(struct in_addr *)&ip_addr.sin_addr)); /* Make recvs non-blocking, set up for polling */ fcntl(sock, F_SETFL, O_NONBLOCK); struct pollfd pfd; pfd.fd = sock; pfd.events = POLLIN; /* Open output file if needed */ int first_write=1; pthread_t write_tid; struct write_info out; if (disk_out) { //out.fd = open(ofile, O_RDWR | O_CREAT | O_LARGEFILE); //out.fd = open(ofile, O_RDWR | O_CREAT, 0666); out.fd = fopen(ofile, "wb"); if (out.fd==NULL) { fprintf(stderr, "you lose.\n"); exit(1); } out.nbytes = buffer_size_bytes/2; }


19



/* clock stuff */ clock_t time0, time1, time_cur, time_last; struct tms t0, t1, tt; long int tps = sysconf(_SC_CLK_TCK); time0 = times(&t0); time_cur = time0; /* Recieve packets */ double byte_count=0; unsigned long long packet_count=0; unsigned long long sent_count=0; unsigned long long out_of_order_count=0; int drop_count=0; unsigned long long packet_0=0, packet_num=0, last_packet_num=2048; signal(SIGINT, cc); int first=1, timeout=0; slen = sizeof(ip_addr); int bufctr=0; char *bufptr=buf; while (run) { rv = poll(&pfd, 1, poll_timeout); if (rv > 0) { //rv = recv(sock, bufptr, packet_size, MSG_TRUNC); #if 1 rv = recvfrom(sock, bufptr, packet_size, MSG_TRUNC, (struct sockaddr *)&ip_addr, &slen); #endif if (rv==-1) { if (errno!=EAGAIN) { perror("recvfrom"); exit(1); } } else { /* Get packet number */ packet_num = *((unsigned long long *)bufptr); if (endian) byte_swap(&packet_num); /* If we're waiting for packet 0, continue */ if (wait_for_0) { if (packet_num <= last_packet_num-2048) wait_for_0 = 0; else continue; } /* Basic timing stuff */ if (first) { time0 = times(&t0); time_cur = time0; time_last = time0; sent_count = packet_num/packet_incr; packet_0 = packet_num/packet_incr; fprintf(stderr, "Receiving data (packet size=%d).\n", rv); if (rv != packet_size) { fprintf(stderr, " Unexpected packet size: data will be %s.\n", (rv > packet_size) ? "truncated" : "padded");


20



} //fprintf(stderr, "ipaddr=%s\n", // inet_ntoa(*(struct in_addr *)&ip_addr.sin_addr)); first=0; } else { time_last = time_cur; time_cur = times(&tt); //drop_count += *((unsigned int *)buf) - (packet_num+1); if (packet_num/packet_incr>sent_count) sent_count=packet_num/packet_incr; } /* Test, print packet_num */ if (print_all) { int i; for (i=0; i<8; i++) { printf("%2.2X ", *(unsigned char *)&bufptr[i]); } printf("%20lld (diff=%lld, %.3fs)\n", packet_num, packet_num-last_packet_num, (double)(time_cur-time_last)/(double)tps); } /* Update counters, pointers */ packet_count++; byte_count += (packet_size > rv) ? (double)rv : (double)packet_size; bufctr++; if (bufctr >= buffer_size) { bufctr = 0; bufptr = buf; } else { bufptr += packet_size; } if (packet_count>1) { if ((packet_num-last_packet_num) % packet_incr) { fprintf(stderr, "Inconsistent packet_incr (count=%lld, send_count=%lld)?\n", packet_count, sent_count-packet_0); fprintf(stderr, "diff=%lld (diff mod incr)=%lld\n", packet_num-last_packet_num, (packet_num-last_packet_num)%packet_incr); } if (packet_num<last_packet_num) out_of_order_count++; } last_packet_num=packet_num; /* Disk stuff */ if (disk_out && ((bufctr==0) || (bufctr==buffer_size/2)) ){ /* Wait for last write thread to finish */ if (!first_write) { pthread_join(write_tid, NULL); } /* Launch new write thread */ if (bufctr==0) { out.buf = buf+buffer_size_bytes/2; } else { out.buf = buf; } rv = pthread_create(&write_tid, NULL, write_to_disk, (void *)&out); if (rv) { perror("pthread_create");


21



exit(1); } first_write=0; } /* Check if we need to stop */ if ((npacket>0) && (packet_count>=npacket)) run = 0; } } else if (rv==0) { if (first==0) { run=0; timeout=1; } /* No data for 1 sec => quit */ } else { if (errno==EINTR) { run=0; } else { perror("poll"); exit(1); } } } // wait for write thread if (disk_out) { pthread_join(write_tid,NULL); //close(out.fd); fclose(out.fd); } time1 = times(&t1); sent_count -= packet_0; double time_sec = (double)(time1-time0)/(double)tps; if (timeout && (poll_timeout>0)) { time_sec -= (double)poll_timeout/1000.0; } double load = (double)(t1.tms_utime+t1.tms_stime-t0.tms_utime-t0.tms_stime) / (double)(time1-time0); double rate = (double)byte_count/time_sec; double srate = (double)sent_count * (double)packet_size / time_sec; drop_count = sent_count - packet_count; if (quiet) { printf("%5d %8.1f %8.3f %.3e %5.3f %d %d R:%s\n", packet_size, byte_count/(1024.0*1024.0), rate/(1024.0*1024.0), (double)drop_count/(double)sent_count, load, drop_count, cpu_idx, argv[optind]); } else { printf("Receiving from %s\n", argv[optind]); printf("Packet size %d B\n", packet_size); printf("Total time %.3f s\n", time_sec); printf("Got %.1f MB (%lld packets)\n", byte_count/(1024.0*1024.0), packet_count); printf("Recv rate %.3f MB/s\n", rate/(1024.0*1024.0)); printf("Send rate %.3f MB/s\n", srate/(1024.0*1024.0)); printf("Dropped %d packets\n", drop_count); printf("Drop rate %.3e\n", (double)drop_count/(double)sent_count); printf("Out-of-order %lld packets\n", out_of_order_count); printf("Avg load %.3f\n", load); }


22



exit(0); }


national radio astronomy observatory - cv.nrao.eduaroshi/beamformer/summer... · arnab pramanik (...

Documents