Hi folks,
Has anyone got decent performance out of SRB <http://www.sdsc.edu/srb/>
and its Sput utility?
The main loop looks like this
while ((nbytes = fread(buf, 1, BUFSIZE,(FILE *)infile_fd ))>0)
{
wbytes = srbObjWrite(conn, out_fd, buf, nbytes);
if (wbytes != nbytes) {
fprintf(stderr, "Sput: write error in srbObject %s/%s: %i\n",
collName,dataName,wbytes);
fprintf(stderr, "Sput: Sput: Exiting \n");
srbObjClose (conn, out_fd);
fclose(infile_fd);
free(buf);
return -1;
}
}
free(buf);
srbObjClose (conn, out_fd);
fclose(infile_fd);
where srbObjWrite() does a RPC to the server to ask it to request
the next block from this client.
res = clFunct(conn, F_SRBO_WRITE,(char *) &retval,&resultLen,1,argv,2);
where clFunc has an RTT to the server
mySrbResult = clReadServerReturn (conn, resultBuf, actualResultLen,
resultIsInt);
I think this raises some interesting issues as round-trip time increases
(ie, clients in Australia).
Cheers,
Glen
--
Glen Turner Tel: (08) 8303 3936 or +61 8 8303 3936
Australia's Academic & Research Network www.aarnet.edu.au
--- Begin Message ---
You might be able to provide them with some tuning advice.
Mark.
Begin forwarded message:
From: Glenn Moloney <glenn@xxxxxxxxxxxxxxxxxxxxxx>
Date: 17 January 2006 11:58:50 AM
To: George McLaughlin <George.McLaughlin@xxxxxxxxxxxxx>
Cc: Geoffrey Taylor <g.taylor@xxxxxxxxxxxxxxxxxxxxxx>, Bruce Morgan
<Bruce.Morgan@xxxxxxxxxxxxx>, smt-l@xxxxxxxxxxxxx, Edwin Wong
<Edwin.Wong@xxxxxxxxxxxxx>, Greg Chenhall <gmc@xxxxxxxxxxxxxx>
Subject: [smt-l] RE: UofM - AARNet Member Agreement signed
Hi George,
After working around some bottlenecks at our end - I am pretty
confident
we doing things correctly at our end.
I have asked KEK to check their end - but we are seeing no improvement
so far on the transfer rate we used to see to KEK:
Sput -f -M -V -S gl01-unix-srb debian-testing-i386-netinst.iso
LOCAL:debian-testing-i386-netinst.iso->SRB:debian-testing-i386-
netinst.iso | 146.758 MB | 1.263 MB/s | 116.15 s | 2006.01.17
12:14:51 | 16 thr
So - we see 1.263 MB/s to gl01.cc.kek.jp from unimelb. The same
transfer
to anusf sees about 20 times that rate. This is with 16 transfer
threads
operating in parallel.
Any hints or tips?
I'm happy to run any transfer test you'd like to see.
cheers,
glenn.
PS.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.749 ms 0.811 ms
0.596
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.800 ms 1.256 ms
1.235 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
1.928 ms 1.231 ms 0.756 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.038 ms
0.917
ms 1.452 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.248 ms
13.083
ms 13.452 ms
6 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 13.113 ms
13.313
ms 13.046 ms
7 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 107.749 ms
107.817 ms 107.404 ms
8 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 158.408 ms
158.535 ms 158.227 ms
9 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135)
183.082
ms 183.610 ms 183.181 ms
10 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) 283.692 ms 283.701 ms
283.217 ms
11 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) 283.262 ms 283.453 ms
283.410 ms
12 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) 284.735 ms
284.126 ms
283.951 ms
13 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) 289.553 ms 289.410 ms
289.091 ms
14 kekgw-0.sinet.ad.jp (150.99.197.2) 289.872 ms 289.569 ms
289.338
ms
15 keksw1-kekgw.kek.jp (130.87.4.2) 289.649 ms 288.962 ms
289.322 ms
16 130.87.4.38 (130.87.4.38) 289.221 ms 289.105 ms 289.091 ms
17 130.87.5.203 (130.87.5.203) 289.489 ms 289.482 ms 289.466 ms
18 * * *
On Thu, 2006-01-12 at 13:53 +1100, George McLaughlin wrote:
Many thanks Geoff
you should now be seeing an SXTransPORT path to Seattle (and from
there a 2.5Gbps path on SINET). This is not yet true for APAC.
If you have some Terabyte datasets at UniMelb can you try sending
them
to Tsukuba.
Bear in mind that once there is fat pipes on the backbone and
internationally, the likelihood is that other bottlenecks will
show up
(end user equipment, local or far end networks, stack tuning issues,
protocol issues, etc – remember all those talk about end-to-end
performance at the HEP meeting in Daegu Geoff?). So it will be
interesting to see what happens.
good luck
George
George McLaughlin
Director, International Developments, AARNet
Home Office: (02) 4849 4315
Mobile: 0411 256 370
_____________________________________________________________________
_
From:Geoffrey Taylor [mailto:g.taylor@xxxxxxxxxxxxxxxxxxxxxx]
Sent: Thursday, 12 January 2006 12:11 PM
To: George McLaughlin
Subject: Re: UofM - AARNet Member Agreement signed
Thanks George.
Happy New Year
Great news on the Taiwan connection.
Keep up the excellent support for Australian HEP. Its much
appreciated.
Regards
Geoff
On 12/01/2006, at 11:52 AM, George McLaughlin wrote:
Many thanks Greg
Can you please confirm that the SXTransPORT Schedule
(Schedule
5 of the Agreement) is also signed, and if by any chance not,
then arrange for that to be executed at the earliest
opportunity.
Bruce, Mark and Steve – please note wrt SXTransPORT and the
Australia-KEK (Tsukuba) data transfers.
Glenn and Geoff – We now have Jumbo Frames peering between
SXTransPORT and TWAREN/TANET2 in Seattle, so a jumbo-frame
high capacity path should now be available to the Academia
Sinica TEIR1 LHC site in Taiwan – we need to talk more about
this and with Simon Lin and his colleagues. We’ll probably
see
Simon in Tokyo.
Edwin – can you please let me know of any other specific
SXTransPORT requirements over and above those we are already
liaising with the respective parties on
regards
George
From: Greg Chenhall [mailto:gmc@xxxxxxxxxxxxxx]
Sent: Wednesday, 11 January 2006 10:37 AM
To: Mary Fleming; Don Robertson; George McLaughlin
Cc: Edwin Wong; Colin Blythe
Subject: UofM - AARNet Member Agreement signed
Dear Mary, Don and George,
The UofM's legal team are sending the signed AARNet Member
Agreement back through James Flowers, and I expect James will
receive that by courier this week.
Research staff are keen to begin testing on Sxtransport. If
there are any technical or other barriers to the use of
SXTransport could you please advise me.
Many Thanks,
Greg Chenhall
Director, Information Infrastructure
Information Division
Thomas Cherry Building, Level 4
The University of Melbourne, Victoria 3010 Australia
Phone: +61 3 8344 7888 Fax: +61 3 9341 6077
Internal Phone: 47888 Email: gmc@xxxxxxxxxxxxxx
This e-mail and any attachments may contain personal
information or information that is otherwise confidential or
the subject of copyright. Any use, disclosure or copying of
any part of it is prohibited. The University does not warrant
that this email or any attachments are free from viruses or
defects. Please check any attachments for viruses and defects
before opening them. If this e-mail is received in error,
please notify me by return e-mail and delete the message.
______________________
Professor Geoffrey N. Taylor
Head, School of Physics, The University of Melbourne
Melbourne, Victoria, 3010 AUSTRALIA
g.taylor@xxxxxxxxxxxxxxxxxxxxxx
Ph: +61 3 8344 5420 Mobile: +61 419 533 080 Fax: +61 3 93494912
Begin forwarded message:
From: Glenn Moloney <glenn@xxxxxxxxxxxxxxxxxxxxxx>
Date: 17 January 2006 1:16:41 PM
To: George McLaughlin <George.McLaughlin@xxxxxxxxxxxxx>
Cc: Geoffrey Taylor <g.taylor@xxxxxxxxxxxxxxxxxxxxxx>, Bruce Morgan
<Bruce.Morgan@xxxxxxxxxxxxx>, smt-l@xxxxxxxxxxxxx, Edwin Wong
<Edwin.Wong@xxxxxxxxxxxxx>, Greg Chenhall <gmc@xxxxxxxxxxxxxx>
Subject: [smt-l] RE: UofM - AARNet Member Agreement signed
Hi George (and otehrs),
Actually, the route has now changed (see below), and we are seeing
lower
bandwidth.
Running iperf between unimelb and KEK I have seen very large
fluctuations in the bandwidth in the last few hours. Up to 60 MBit/s,
and now down to 5 MBit/s.
Is something happening with the routing?
cheers,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms 49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms 108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms 241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms 261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms 238.802 ms
15 * * *
16 *
On Tue, 2006-01-17 at 12:28 +1100, Glenn Moloney wrote:
Hi George,
After working around some bottlenecks at our end - I am pretty
confident
we doing things correctly at our end.
I have asked KEK to check their end - but we are seeing no
improvement
so far on the transfer rate we used to see to KEK:
Sput -f -M -V -S gl01-unix-srb debian-testing-i386-netinst.iso
LOCAL:debian-testing-i386-netinst.iso->SRB:debian-testing-i386-
netinst.iso | 146.758 MB | 1.263 MB/s | 116.15 s | 2006.01.17
12:14:51 | 16 thr
So - we see 1.263 MB/s to gl01.cc.kek.jp from unimelb. The same
transfer
to anusf sees about 20 times that rate. This is with 16 transfer
threads
operating in parallel.
Any hints or tips?
I'm happy to run any transfer test you'd like to see.
cheers,
glenn.
PS.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.749 ms 0.811 ms
0.596
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.800 ms 1.256 ms
1.235 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
1.928 ms 1.231 ms 0.756 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.038 ms
0.917
ms 1.452 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.248 ms
13.083
ms 13.452 ms
6 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 13.113 ms
13.313
ms 13.046 ms
7 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 107.749 ms
107.817 ms 107.404 ms
8 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 158.408 ms
158.535 ms 158.227 ms
9 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135)
183.082
ms 183.610 ms 183.181 ms
10 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) 283.692 ms
283.701 ms
283.217 ms
11 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) 283.262 ms 283.453 ms
283.410 ms
12 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) 284.735 ms
284.126 ms
283.951 ms
13 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) 289.553 ms 289.410 ms
289.091 ms
14 kekgw-0.sinet.ad.jp (150.99.197.2) 289.872 ms 289.569 ms
289.338
ms
15 keksw1-kekgw.kek.jp (130.87.4.2) 289.649 ms 288.962 ms
289.322 ms
16 130.87.4.38 (130.87.4.38) 289.221 ms 289.105 ms 289.091 ms
17 130.87.5.203 (130.87.5.203) 289.489 ms 289.482 ms 289.466 ms
18 * * *
On Thu, 2006-01-12 at 13:53 +1100, George McLaughlin wrote:
Many thanks Geoff
you should now be seeing an SXTransPORT path to Seattle (and from
there a 2.5Gbps path on SINET). This is not yet true for APAC.
If you have some Terabyte datasets at UniMelb can you try sending
them
to Tsukuba.
Bear in mind that once there is fat pipes on the backbone and
internationally, the likelihood is that other bottlenecks will
show up
(end user equipment, local or far end networks, stack tuning issues,
protocol issues, etc – remember all those talk about end-to-end
performance at the HEP meeting in Daegu Geoff?). So it will be
interesting to see what happens.
good luck
George
George McLaughlin
Director, International Developments, AARNet
Home Office: (02) 4849 4315
Mobile: 0411 256 370
____________________________________________________________________
__
From:Geoffrey Taylor [mailto:g.taylor@xxxxxxxxxxxxxxxxxxxxxx]
Sent: Thursday, 12 January 2006 12:11 PM
To: George McLaughlin
Subject: Re: UofM - AARNet Member Agreement signed
Thanks George.
Happy New Year
Great news on the Taiwan connection.
Keep up the excellent support for Australian HEP. Its much
appreciated.
Regards
Geoff
On 12/01/2006, at 11:52 AM, George McLaughlin wrote:
Many thanks Greg
Can you please confirm that the SXTransPORT Schedule
(Schedule
5 of the Agreement) is also signed, and if by any chance
not,
then arrange for that to be executed at the earliest
opportunity.
Bruce, Mark and Steve – please note wrt SXTransPORT and the
Australia-KEK (Tsukuba) data transfers.
Glenn and Geoff – We now have Jumbo Frames peering between
SXTransPORT and TWAREN/TANET2 in Seattle, so a jumbo-frame
high capacity path should now be available to the Academia
Sinica TEIR1 LHC site in Taiwan – we need to talk more about
this and with Simon Lin and his colleagues. We’ll
probably see
Simon in Tokyo.
Edwin – can you please let me know of any other specific
SXTransPORT requirements over and above those we are already
liaising with the respective parties on
regards
George
From: Greg Chenhall [mailto:gmc@xxxxxxxxxxxxxx]
Sent: Wednesday, 11 January 2006 10:37 AM
To: Mary Fleming; Don Robertson; George McLaughlin
Cc: Edwin Wong; Colin Blythe
Subject: UofM - AARNet Member Agreement signed
Dear Mary, Don and George,
The UofM's legal team are sending the signed AARNet Member
Agreement back through James Flowers, and I expect James
will
receive that by courier this week.
Research staff are keen to begin testing on Sxtransport. If
there are any technical or other barriers to the use of
SXTransport could you please advise me.
Many Thanks,
Greg Chenhall
Director, Information Infrastructure
Information Division
Thomas Cherry Building, Level 4
The University of Melbourne, Victoria 3010 Australia
Phone: +61 3 8344 7888 Fax: +61 3 9341 6077
Internal Phone: 47888 Email: gmc@xxxxxxxxxxxxxx
This e-mail and any attachments may contain personal
information or information that is otherwise confidential or
the subject of copyright. Any use, disclosure or copying of
any part of it is prohibited. The University does not
warrant
that this email or any attachments are free from viruses or
defects. Please check any attachments for viruses and
defects
before opening them. If this e-mail is received in error,
please notify me by return e-mail and delete the message.
______________________
Professor Geoffrey N. Taylor
Head, School of Physics, The University of Melbourne
Melbourne, Victoria, 3010 AUSTRALIA
g.taylor@xxxxxxxxxxxxxxxxxxxxxx
Ph: +61 3 8344 5420 Mobile: +61 419 533 080 Fax: +61 3 93494912
Begin forwarded message:
From: Bruce Morgan <Bruce.Morgan@xxxxxxxxxxxxx>
Date: 17 January 2006 1:36:30 PM
To: Glenn Moloney <glenn@xxxxxxxxxxxxxxxxxxxxxx>
Cc: George McLaughlin <George.McLaughlin@xxxxxxxxxxxxx>, Geoffrey
Taylor <g.taylor@xxxxxxxxxxxxxxxxxxxxxx>, smt-l@xxxxxxxxxxxxx,
Edwin Wong <Edwin.Wong@xxxxxxxxxxxxx>, Greg Chenhall
<gmc@xxxxxxxxxxxxxx>
Subject: [smt-l] Re: UofM - AARNet Member Agreement signed
Hi Glenn,
What do you see now?
It should be taking SXTransport.
regards
Bruce
On 17/01/2006, at 10:46 AM, Glenn Moloney wrote:
Hi George (and otehrs),
Actually, the route has now changed (see below), and we are seeing
lower
bandwidth.
Running iperf between unimelb and KEK I have seen very large
fluctuations in the bandwidth in the last few hours. Up to 60 MBit/s,
and now down to 5 MBit/s.
Is something happening with the routing?
cheers,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms
49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms
108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms 241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms
261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms 238.802 ms
15 * * *
16 *
On Tue, 2006-01-17 at 12:28 +1100, Glenn Moloney wrote:
Hi George,
After working around some bottlenecks at our end - I am pretty
confident
we doing things correctly at our end.
I have asked KEK to check their end - but we are seeing no
improvement
so far on the transfer rate we used to see to KEK:
Sput -f -M -V -S gl01-unix-srb debian-testing-i386-netinst.iso
LOCAL:debian-testing-i386-netinst.iso->SRB:debian-testing-i386-
netinst.iso | 146.758 MB | 1.263 MB/s | 116.15 s | 2006.01.17
12:14:51 | 16 thr
So - we see 1.263 MB/s to gl01.cc.kek.jp from unimelb. The same
transfer
to anusf sees about 20 times that rate. This is with 16 transfer
threads
operating in parallel.
Any hints or tips?
I'm happy to run any transfer test you'd like to see.
cheers,
glenn.
PS.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.749 ms 0.811 ms
0.596
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.800 ms 1.256 ms
1.235 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
1.928 ms 1.231 ms 0.756 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.038 ms
0.917
ms 1.452 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.248 ms
13.083
ms 13.452 ms
6 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 13.113 ms
13.313
ms 13.046 ms
7 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 107.749 ms
107.817 ms 107.404 ms
8 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 158.408 ms
158.535 ms 158.227 ms
9 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135)
183.082
ms 183.610 ms 183.181 ms
10 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) 283.692 ms
283.701 ms
283.217 ms
11 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) 283.262 ms 283.453 ms
283.410 ms
12 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) 284.735 ms
284.126 ms
283.951 ms
13 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) 289.553 ms 289.410 ms
289.091 ms
14 kekgw-0.sinet.ad.jp (150.99.197.2) 289.872 ms 289.569 ms
289.338
ms
15 keksw1-kekgw.kek.jp (130.87.4.2) 289.649 ms 288.962 ms
289.322 ms
16 130.87.4.38 (130.87.4.38) 289.221 ms 289.105 ms 289.091 ms
17 130.87.5.203 (130.87.5.203) 289.489 ms 289.482 ms 289.466 ms
18 * * *
On Thu, 2006-01-12 at 13:53 +1100, George McLaughlin wrote:
Many thanks Geoff
you should now be seeing an SXTransPORT path to Seattle (and from
there a 2.5Gbps path on SINET). This is not yet true for APAC.
If you have some Terabyte datasets at UniMelb can you try
sending them
to Tsukuba.
Bear in mind that once there is fat pipes on the backbone and
internationally, the likelihood is that other bottlenecks will
show up
(end user equipment, local or far end networks, stack tuning
issues,
protocol issues, etc – remember all those talk about end-to-end
performance at the HEP meeting in Daegu Geoff?). So it will be
interesting to see what happens.
good luck
George
George McLaughlin
Director, International Developments, AARNet
Home Office: (02) 4849 4315
Mobile: 0411 256 370
___________________________________________________________________
___
From:Geoffrey Taylor [mailto:g.taylor@xxxxxxxxxxxxxxxxxxxxxx]
Sent: Thursday, 12 January 2006 12:11 PM
To: George McLaughlin
Subject: Re: UofM - AARNet Member Agreement signed
Thanks George.
Happy New Year
Great news on the Taiwan connection.
Keep up the excellent support for Australian HEP. Its much
appreciated.
Regards
Geoff
On 12/01/2006, at 11:52 AM, George McLaughlin wrote:
Many thanks Greg
Can you please confirm that the SXTransPORT Schedule
(Schedule
5 of the Agreement) is also signed, and if by any chance
not,
then arrange for that to be executed at the earliest
opportunity.
Bruce, Mark and Steve – please note wrt SXTransPORT and the
Australia-KEK (Tsukuba) data transfers.
Glenn and Geoff – We now have Jumbo Frames peering between
SXTransPORT and TWAREN/TANET2 in Seattle, so a jumbo-frame
high capacity path should now be available to the Academia
Sinica TEIR1 LHC site in Taiwan – we need to talk more
about
this and with Simon Lin and his colleagues. We’ll
probably see
Simon in Tokyo.
Edwin – can you please let me know of any other specific
SXTransPORT requirements over and above those we are
already
liaising with the respective parties on
regards
George
From: Greg Chenhall [mailto:gmc@xxxxxxxxxxxxxx]
Sent: Wednesday, 11 January 2006 10:37 AM
To: Mary Fleming; Don Robertson; George McLaughlin
Cc: Edwin Wong; Colin Blythe
Subject: UofM - AARNet Member Agreement signed
Dear Mary, Don and George,
The UofM's legal team are sending the signed AARNet Member
Agreement back through James Flowers, and I expect James
will
receive that by courier this week.
Research staff are keen to begin testing on Sxtransport. If
there are any technical or other barriers to the use of
SXTransport could you please advise me.
Many Thanks,
Greg Chenhall
Director, Information Infrastructure
Information Division
Thomas Cherry Building, Level 4
The University of Melbourne, Victoria 3010 Australia
Phone: +61 3 8344 7888 Fax: +61 3 9341 6077
Internal Phone: 47888 Email: gmc@xxxxxxxxxxxxxx
This e-mail and any attachments may contain personal
information or information that is otherwise
confidential or
the subject of copyright. Any use, disclosure or copying of
any part of it is prohibited. The University does not
warrant
that this email or any attachments are free from viruses or
defects. Please check any attachments for viruses and
defects
before opening them. If this e-mail is received in error,
please notify me by return e-mail and delete the message.
______________________
Professor Geoffrey N. Taylor
Head, School of Physics, The University of Melbourne
Melbourne, Victoria, 3010 AUSTRALIA
g.taylor@xxxxxxxxxxxxxxxxxxxxxx
Ph: +61 3 8344 5420 Mobile: +61 419 533 080 Fax: +61 3 93494912
AARNet, Perth Office, Western Australia
Ph: +618 6436 8912
Fax: +618 6436 8585
www.aarnet.edu.au
Begin forwarded message:
From: Glenn Moloney <glenn@xxxxxxxxxxxxxxxxxxxxxx>
Date: 17 January 2006 2:00:28 PM
To: Bruce Morgan <Bruce.Morgan@xxxxxxxxxxxxx>
Cc: George McLaughlin <George.McLaughlin@xxxxxxxxxxxxx>, Geoffrey
Taylor <g.taylor@xxxxxxxxxxxxxxxxxxxxxx>, smt-l@xxxxxxxxxxxxx,
Edwin Wong <Edwin.Wong@xxxxxxxxxxxxx>, Greg Chenhall
<gmc@xxxxxxxxxxxxxx>
Subject: [smt-l] Re: UofM - AARNet Member Agreement signed
G'day Bruce,
On Tue, 2006-01-17 at 11:06 +0800, Bruce Morgan wrote:
What do you see now?
It should be taking SXTransport.
Yep - we are back on SXTransport.....
cheers & thanks,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms 49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms 108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms 241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms 261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms 238.802 ms
15 * * *
16 *
regards
Bruce
On 17/01/2006, at 10:46 AM, Glenn Moloney wrote:
Hi George (and otehrs),
Actually, the route has now changed (see below), and we are seeing
lower
bandwidth.
Running iperf between unimelb and KEK I have seen very large
fluctuations in the bandwidth in the last few hours. Up to 60
MBit/s,
and now down to 5 MBit/s.
Is something happening with the routing?
cheers,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms
49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms
108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms
241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms
261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms 238.802 ms
15 * * *
16 *
On Tue, 2006-01-17 at 12:28 +1100, Glenn Moloney wrote:
Hi George,
After working around some bottlenecks at our end - I am pretty
confident
we doing things correctly at our end.
I have asked KEK to check their end - but we are seeing no
improvement
so far on the transfer rate we used to see to KEK:
Sput -f -M -V -S gl01-unix-srb debian-testing-i386-netinst.iso
LOCAL:debian-testing-i386-netinst.iso->SRB:debian-testing-i386-
netinst.iso | 146.758 MB | 1.263 MB/s | 116.15 s | 2006.01.17
12:14:51 | 16 thr
So - we see 1.263 MB/s to gl01.cc.kek.jp from unimelb. The same
transfer
to anusf sees about 20 times that rate. This is with 16 transfer
threads
operating in parallel.
Any hints or tips?
I'm happy to run any transfer test you'd like to see.
cheers,
glenn.
PS.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.749 ms 0.811 ms
0.596
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.800 ms 1.256 ms
1.235 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au
(202.158.200.249)
1.928 ms 1.231 ms 0.756 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.038 ms
0.917
ms 1.452 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.248 ms
13.083
ms 13.452 ms
6 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 13.113 ms
13.313
ms 13.046 ms
7 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 107.749 ms
107.817 ms 107.404 ms
8 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 158.408 ms
158.535 ms 158.227 ms
9 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135)
183.082
ms 183.610 ms 183.181 ms
10 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) 283.692 ms
283.701 ms
283.217 ms
11 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) 283.262 ms 283.453 ms
283.410 ms
12 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) 284.735 ms
284.126 ms
283.951 ms
13 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) 289.553 ms 289.410 ms
289.091 ms
14 kekgw-0.sinet.ad.jp (150.99.197.2) 289.872 ms 289.569 ms
289.338
ms
15 keksw1-kekgw.kek.jp (130.87.4.2) 289.649 ms 288.962 ms
289.322 ms
16 130.87.4.38 (130.87.4.38) 289.221 ms 289.105 ms 289.091 ms
17 130.87.5.203 (130.87.5.203) 289.489 ms 289.482 ms 289.466 ms
18 * * *
On Thu, 2006-01-12 at 13:53 +1100, George McLaughlin wrote:
Many thanks Geoff
you should now be seeing an SXTransPORT path to Seattle (and from
there a 2.5Gbps path on SINET). This is not yet true for APAC.
If you have some Terabyte datasets at UniMelb can you try sending
them
to Tsukuba.
Bear in mind that once there is fat pipes on the backbone and
internationally, the likelihood is that other bottlenecks will
show up
(end user equipment, local or far end networks, stack tuning
issues,
protocol issues, etc – remember all those talk about end-to-end
performance at the HEP meeting in Daegu Geoff?). So it will be
interesting to see what happens.
good luck
George
George McLaughlin
Director, International Developments, AARNet
Home Office: (02) 4849 4315
Mobile: 0411 256 370
__________________________________________________________________
__
__
From:Geoffrey Taylor [mailto:g.taylor@xxxxxxxxxxxxxxxxxxxxxx]
Sent: Thursday, 12 January 2006 12:11 PM
To: George McLaughlin
Subject: Re: UofM - AARNet Member Agreement signed
Thanks George.
Happy New Year
Great news on the Taiwan connection.
Keep up the excellent support for Australian HEP. Its much
appreciated.
Regards
Geoff
On 12/01/2006, at 11:52 AM, George McLaughlin wrote:
Many thanks Greg
Can you please confirm that the SXTransPORT Schedule
(Schedule
5 of the Agreement) is also signed, and if by any chance
not,
then arrange for that to be executed at the earliest
opportunity.
Bruce, Mark and Steve – please note wrt SXTransPORT and
the
Australia-KEK (Tsukuba) data transfers.
Glenn and Geoff – We now have Jumbo Frames peering between
SXTransPORT and TWAREN/TANET2 in Seattle, so a jumbo-frame
high capacity path should now be available to the Academia
Sinica TEIR1 LHC site in Taiwan – we need to talk more
about
this and with Simon Lin and his colleagues. We’ll
probably see
Simon in Tokyo.
Edwin – can you please let me know of any other specific
SXTransPORT requirements over and above those we are
already
liaising with the respective parties on
regards
George
From: Greg Chenhall [mailto:gmc@xxxxxxxxxxxxxx]
Sent: Wednesday, 11 January 2006 10:37 AM
To: Mary Fleming; Don Robertson; George McLaughlin
Cc: Edwin Wong; Colin Blythe
Subject: UofM - AARNet Member Agreement signed
Dear Mary, Don and George,
The UofM's legal team are sending the signed AARNet Member
Agreement back through James Flowers, and I expect James
will
receive that by courier this week.
Research staff are keen to begin testing on
Sxtransport. If
there are any technical or other barriers to the use of
SXTransport could you please advise me.
Many Thanks,
Greg Chenhall
Director, Information Infrastructure
Information Division
Thomas Cherry Building, Level 4
The University of Melbourne, Victoria 3010 Australia
Phone: +61 3 8344 7888 Fax: +61 3 9341 6077
Internal Phone: 47888 Email: gmc@xxxxxxxxxxxxxx
This e-mail and any attachments may contain personal
information or information that is otherwise
confidential or
the subject of copyright. Any use, disclosure or
copying of
any part of it is prohibited. The University does not
warrant
that this email or any attachments are free from
viruses or
defects. Please check any attachments for viruses and
defects
before opening them. If this e-mail is received in error,
please notify me by return e-mail and delete the message.
______________________
Professor Geoffrey N. Taylor
Head, School of Physics, The University of Melbourne
Melbourne, Victoria, 3010 AUSTRALIA
g.taylor@xxxxxxxxxxxxxxxxxxxxxx
Ph: +61 3 8344 5420 Mobile: +61 419 533 080 Fax: +61 3 93494912
AARNet, Perth Office, Western Australia
Ph: +618 6436 8912
Fax: +618 6436 8585
www.aarnet.edu.au
Begin forwarded message:
From: Bruce Morgan <Bruce.Morgan@xxxxxxxxxxxxx>
Date: 17 January 2006 2:52:30 PM
To: Glenn Moloney <glenn@xxxxxxxxxxxxxxxxxxxxxx>
Cc: George McLaughlin <George.McLaughlin@xxxxxxxxxxxxx>, Geoffrey
Taylor <g.taylor@xxxxxxxxxxxxxxxxxxxxxx>, smt-l@xxxxxxxxxxxxx,
Edwin Wong <Edwin.Wong@xxxxxxxxxxxxx>, Greg Chenhall
<gmc@xxxxxxxxxxxxxx>
Subject: [smt-l] Re: UofM - AARNet Member Agreement signed
Hi Glen,
Was the traceroute provided correct at the time. It indicates the
155Mbps path rather than the 10Gbps path.
It should be something like:
1 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 4 msec 0
msec 0 msec
2 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 12 msec 12
msec 12 msec
3 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 12 msec 12
msec 12 msec
4 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 108 msec 104
msec 108 msec
5 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 160 msec 160
msec 264 msec
6 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135) 228
msec 180 msec 184 msec
7 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) [AS 2907] 284 msec
280 msec 284 msec
8 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) [AS 2907] 284 msec 284
msec 280 msec
9 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) [AS 2907] 284 msec
284 msec 284 msec
10 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) [AS 2907] 288 msec 288
msec 288 msec
11 kekgw-1.sinet.ad.jp (150.99.197.6) [AS 2907] 288 msec 288 msec
288 msec
12 keksw1-kekgw.kek.jp (130.87.4.2) [AS 2505] 292 msec 288 msec 288
msec
On 17/01/2006, at 11:30 AM, Glenn Moloney wrote:
G'day Bruce,
On Tue, 2006-01-17 at 11:06 +0800, Bruce Morgan wrote:
What do you see now?
It should be taking SXTransport.
Yep - we are back on SXTransport.....
cheers & thanks,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms
49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms
108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms 241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms
261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms 238.802 ms
15 * * *
16 *
regards
Bruce
On 17/01/2006, at 10:46 AM, Glenn Moloney wrote:
Hi George (and otehrs),
Actually, the route has now changed (see below), and we are seeing
lower
bandwidth.
Running iperf between unimelb and KEK I have seen very large
fluctuations in the bandwidth in the last few hours. Up to 60
MBit/s,
and now down to 5 MBit/s.
Is something happening with the routing?
cheers,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au
(202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms
49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms
108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms
241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms
261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms 238.802 ms
15 * * *
16 *
On Tue, 2006-01-17 at 12:28 +1100, Glenn Moloney wrote:
Hi George,
After working around some bottlenecks at our end - I am pretty
confident
we doing things correctly at our end.
I have asked KEK to check their end - but we are seeing no
improvement
so far on the transfer rate we used to see to KEK:
Sput -f -M -V -S gl01-unix-srb debian-testing-i386-netinst.iso
LOCAL:debian-testing-i386-netinst.iso->SRB:debian-testing-i386-
netinst.iso | 146.758 MB | 1.263 MB/s | 116.15 s | 2006.01.17
12:14:51 | 16 thr
So - we see 1.263 MB/s to gl01.cc.kek.jp from unimelb. The same
transfer
to anusf sees about 20 times that rate. This is with 16 transfer
threads
operating in parallel.
Any hints or tips?
I'm happy to run any transfer test you'd like to see.
cheers,
glenn.
PS.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.749 ms 0.811 ms
0.596
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.800 ms 1.256 ms
1.235 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au
(202.158.200.249)
1.928 ms 1.231 ms 0.756 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.038 ms
0.917
ms 1.452 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.248 ms
13.083
ms 13.452 ms
6 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 13.113 ms
13.313
ms 13.046 ms
7 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 107.749 ms
107.817 ms 107.404 ms
8 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 158.408 ms
158.535 ms 158.227 ms
9 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135)
183.082
ms 183.610 ms 183.181 ms
10 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) 283.692 ms
283.701 ms
283.217 ms
11 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) 283.262 ms
283.453 ms
283.410 ms
12 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) 284.735 ms
284.126 ms
283.951 ms
13 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) 289.553 ms
289.410 ms
289.091 ms
14 kekgw-0.sinet.ad.jp (150.99.197.2) 289.872 ms 289.569 ms
289.338
ms
15 keksw1-kekgw.kek.jp (130.87.4.2) 289.649 ms 288.962 ms
289.322 ms
16 130.87.4.38 (130.87.4.38) 289.221 ms 289.105 ms 289.091 ms
17 130.87.5.203 (130.87.5.203) 289.489 ms 289.482 ms
289.466 ms
18 * * *
On Thu, 2006-01-12 at 13:53 +1100, George McLaughlin wrote:
Many thanks Geoff
you should now be seeing an SXTransPORT path to Seattle (and from
there a 2.5Gbps path on SINET). This is not yet true for APAC.
If you have some Terabyte datasets at UniMelb can you try sending
them
to Tsukuba.
Bear in mind that once there is fat pipes on the backbone and
internationally, the likelihood is that other bottlenecks will
show up
(end user equipment, local or far end networks, stack tuning
issues,
protocol issues, etc – remember all those talk about end-to-end
performance at the HEP meeting in Daegu Geoff?). So it will be
interesting to see what happens.
good luck
George
George McLaughlin
Director, International Developments, AARNet
Home Office: (02) 4849 4315
Mobile: 0411 256 370
_________________________________________________________________
___
__
From:Geoffrey Taylor [mailto:g.taylor@xxxxxxxxxxxxxxxxxxxxxx]
Sent: Thursday, 12 January 2006 12:11 PM
To: George McLaughlin
Subject: Re: UofM - AARNet Member Agreement signed
Thanks George.
Happy New Year
Great news on the Taiwan connection.
Keep up the excellent support for Australian HEP. Its much
appreciated.
Regards
Geoff
On 12/01/2006, at 11:52 AM, George McLaughlin wrote:
Many thanks Greg
Can you please confirm that the SXTransPORT Schedule
(Schedule
5 of the Agreement) is also signed, and if by any chance
not,
then arrange for that to be executed at the earliest
opportunity.
Bruce, Mark and Steve – please note wrt SXTransPORT
and the
Australia-KEK (Tsukuba) data transfers.
Glenn and Geoff – We now have Jumbo Frames peering
between
SXTransPORT and TWAREN/TANET2 in Seattle, so a jumbo-
frame
high capacity path should now be available to the
Academia
Sinica TEIR1 LHC site in Taiwan – we need to talk more
about
this and with Simon Lin and his colleagues. We’ll
probably see
Simon in Tokyo.
Edwin – can you please let me know of any other specific
SXTransPORT requirements over and above those we are
already
liaising with the respective parties on
regards
George
From: Greg Chenhall [mailto:gmc@xxxxxxxxxxxxxx]
Sent: Wednesday, 11 January 2006 10:37 AM
To: Mary Fleming; Don Robertson; George McLaughlin
Cc: Edwin Wong; Colin Blythe
Subject: UofM - AARNet Member Agreement signed
Dear Mary, Don and George,
The UofM's legal team are sending the signed AARNet
Member
Agreement back through James Flowers, and I expect James
will
receive that by courier this week.
Research staff are keen to begin testing on
Sxtransport. If
there are any technical or other barriers to the use of
SXTransport could you please advise me.
Many Thanks,
Greg Chenhall
Director, Information Infrastructure
Information Division
Thomas Cherry Building, Level 4
The University of Melbourne, Victoria 3010 Australia
Phone: +61 3 8344 7888 Fax: +61 3 9341 6077
Internal Phone: 47888 Email: gmc@xxxxxxxxxxxxxx
This e-mail and any attachments may contain personal
information or information that is otherwise
confidential or
the subject of copyright. Any use, disclosure or
copying of
any part of it is prohibited. The University does not
warrant
that this email or any attachments are free from
viruses or
defects. Please check any attachments for viruses and
defects
before opening them. If this e-mail is received in error,
please notify me by return e-mail and delete the message.
______________________
Professor Geoffrey N. Taylor
Head, School of Physics, The University of Melbourne
Melbourne, Victoria, 3010 AUSTRALIA
g.taylor@xxxxxxxxxxxxxxxxxxxxxx
Ph: +61 3 8344 5420 Mobile: +61 419 533 080 Fax: +61 3 93494912
AARNet, Perth Office, Western Australia
Ph: +618 6436 8912
Fax: +618 6436 8585
www.aarnet.edu.au
AARNet, Perth Office, Western Australia
Ph: +618 6436 8912
Fax: +618 6436 8585
www.aarnet.edu.au
Begin forwarded message:
From: Glenn Moloney <glenn@xxxxxxxxxxxxxxxxxxxxxx>
Date: 17 January 2006 3:16:07 PM
To: Bruce Morgan <Bruce.Morgan@xxxxxxxxxxxxx>
Cc: George McLaughlin <George.McLaughlin@xxxxxxxxxxxxx>, Geoffrey
Taylor <g.taylor@xxxxxxxxxxxxxxxxxxxxxx>, smt-l@xxxxxxxxxxxxx,
Edwin Wong <Edwin.Wong@xxxxxxxxxxxxx>, Greg Chenhall
<gmc@xxxxxxxxxxxxxx>
Subject: [smt-l] Re: UofM - AARNet Member Agreement signed
Hi Bruce,
I just checked again and the traceroute is shown below. It does appear
now to match your suggested correct route.
cheers,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.464 ms 0.445 ms
0.426
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 1.023 ms 0.878 ms
1.328 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
1.080 ms 1.122 ms 1.178 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 0.776 ms
0.990
ms 1.252 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 12.887 ms
13.251
ms 12.701 ms
6 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 12.945 ms
13.378
ms 13.236 ms
7 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 107.767 ms
107.865 ms 107.432 ms
8 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 158.319 ms
158.537 ms 158.748 ms
9 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135)
183.148
ms 183.134 ms 182.991 ms
10 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) 283.191 ms 283.577 ms
283.274 ms
11 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) 283.325 ms 283.493 ms
283.343 ms
12 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) 284.003 ms
284.116 ms
283.553 ms
13 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) 289.251 ms 289.220 ms
289.074 ms
14 kekgw-0.sinet.ad.jp (150.99.197.2) 289.441 ms 289.169 ms
288.955
ms
15 keksw1-kekgw.kek.jp (130.87.4.2) 289.153 ms 289.247 ms
289.174 ms
On Tue, 2006-01-17 at 12:22 +0800, Bruce Morgan wrote:
Hi Glen,
Was the traceroute provided correct at the time. It indicates the
155Mbps path rather than the 10Gbps path.
It should be something like:
1 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 4 msec 0 msec
0 msec
2 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 12 msec 12
msec 12 msec
3 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 12 msec 12
msec 12 msec
4 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106) 108 msec 104
msec 108 msec
5 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110) 160 msec 160
msec 264 msec
6 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135) 228
msec 180 msec 184 msec
7 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) [AS 2907] 284 msec
280 msec 284 msec
8 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) [AS 2907] 284 msec 284
msec 280 msec
9 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) [AS 2907] 284 msec
284 msec 284 msec
10 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) [AS 2907] 288 msec 288
msec 288 msec
11 kekgw-1.sinet.ad.jp (150.99.197.6) [AS 2907] 288 msec 288 msec 288
msec
12 keksw1-kekgw.kek.jp (130.87.4.2) [AS 2505] 292 msec 288 msec
288 msec
On 17/01/2006, at 11:30 AM, Glenn Moloney wrote:
G'day Bruce,
On Tue, 2006-01-17 at 11:06 +0800, Bruce Morgan wrote:
What do you see now?
It should be taking SXTransport.
Yep - we are back on SXTransport.....
cheers & thanks,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au (202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms
49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms
108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms
241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms
261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms 238.802 ms
15 * * *
16 *
regards
Bruce
On 17/01/2006, at 10:46 AM, Glenn Moloney wrote:
Hi George (and otehrs),
Actually, the route has now changed (see below), and we are seeing
lower
bandwidth.
Running iperf between unimelb and KEK I have seen very large
fluctuations in the bandwidth in the last few hours. Up to 60
MBit/s,
and now down to 5 MBit/s.
Is something happening with the routing?
cheers,
glenn.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38 byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.582 ms 0.663 ms
0.446
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.917 ms 1.329 ms
1.548 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au
(202.158.200.249)
0.717 ms 0.866 ms 1.033 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.447 ms
0.996
ms 1.229 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.208 ms
12.653
ms 13.170 ms
6 gigabitethernet3-0.bb3.a.syd.aarnet.net.au (202.158.194.114)
13.072
ms 13.388 ms 13.383 ms
7 pos2-0.bb1.a.suv.aarnet.net.au (202.158.194.82) 49.418 ms
49.393
ms 49.884 ms
8 pos1-0.bb1.b.hnl.aarnet.net.au (202.158.194.86) 108.849 ms
108.907
ms 109.424 ms
9 202.158.204.254 (202.158.204.254) 108.668 ms 108.545 ms
108.726
ms
10 tppr-pos5-0.jp.apan.net (203.181.248.174) 241.674 ms
241.285 ms
241.373 ms
11 tpr4-10gi2-1-0-6.jp.apan.net (203.181.249.99) 260.649 ms
261.865
ms 263.857 ms
12 203.181.249.69 (203.181.249.69) 238.720 ms 238.746 ms
239.096 ms
13 130.87.4.38 (130.87.4.38) 238.954 ms 238.496 ms 238.670 ms
14 130.87.5.203 (130.87.5.203) 240.275 ms 239.169 ms
238.802 ms
15 * * *
16 *
On Tue, 2006-01-17 at 12:28 +1100, Glenn Moloney wrote:
Hi George,
After working around some bottlenecks at our end - I am pretty
confident
we doing things correctly at our end.
I have asked KEK to check their end - but we are seeing no
improvement
so far on the transfer rate we used to see to KEK:
Sput -f -M -V -S gl01-unix-srb debian-testing-i386-netinst.iso
LOCAL:debian-testing-i386-netinst.iso->SRB:debian-testing-i386-
netinst.iso | 146.758 MB | 1.263 MB/s | 116.15 s | 2006.01.17
12:14:51 | 16 thr
So - we see 1.263 MB/s to gl01.cc.kek.jp from unimelb. The same
transfer
to anusf sees about 20 times that rate. This is with 16 transfer
threads
operating in parallel.
Any hints or tips?
I'm happy to run any transfer test you'd like to see.
cheers,
glenn.
PS.
traceroute to gl01.cc.kek.jp (130.87.208.81), 30 hops max, 38
byte
packets
1 rtr-500192.unimelb.edu.au (128.250.50.1) 0.749 ms 0.811 ms
0.596
ms
2 rtr2-4.unimelb.edu.au (128.250.5.219) 0.800 ms 1.256 ms
1.235 ms
3 gigabitethernet1.er1.unimelb.cpe.aarnet.net.au
(202.158.200.249)
1.928 ms 1.231 ms 0.756 ms
4 ge-1-0-5.bb1.a.mel.aarnet.net.au (202.158.200.241) 1.038 ms
0.917
ms 1.452 ms
5 so-0-1-0.bb1.b.syd.aarnet.net.au (202.158.194.34) 13.248 ms
13.083
ms 13.452 ms
6 ge-0-0-0.bb1.a.syd.aarnet.net.au (202.158.194.45) 13.113 ms
13.313
ms 13.046 ms
7 so-0-1-0.bb1.a.hnl.aarnet.net.au (202.158.194.106)
107.749 ms
107.817 ms 107.404 ms
8 so-0-1-0.bb1.a.sea.aarnet.net.au (202.158.194.110)
158.408 ms
158.535 ms 158.227 ms
9 sinet-1-is-jmb-776.lsanca.pacificwave.net (207.231.241.135)
183.082
ms 183.610 ms 183.181 ms
10 nii-gate2-P1-0.sinet.ad.jp (150.99.199.13) 283.692 ms
283.701 ms
283.217 ms
11 nii-S1-P8-0.sinet.ad.jp (150.99.198.21) 283.262 ms
283.453 ms
283.410 ms
12 tokyo-core1-P8-0.sinet.ad.jp (150.99.197.21) 284.735 ms
284.126 ms
283.951 ms
13 kek-S1-P4-0.sinet.ad.jp (150.99.197.34) 289.553 ms
289.410 ms
289.091 ms
14 kekgw-0.sinet.ad.jp (150.99.197.2) 289.872 ms 289.569 ms
289.338
ms
15 keksw1-kekgw.kek.jp (130.87.4.2) 289.649 ms 288.962 ms
289.322 ms
16 130.87.4.38 (130.87.4.38) 289.221 ms 289.105 ms 289.091 ms
17 130.87.5.203 (130.87.5.203) 289.489 ms 289.482 ms
289.466 ms
18 * * *
On Thu, 2006-01-12 at 13:53 +1100, George McLaughlin wrote:
Many thanks Geoff
you should now be seeing an SXTransPORT path to Seattle (and
from
there a 2.5Gbps path on SINET). This is not yet true for APAC.
If you have some Terabyte datasets at UniMelb can you try
sending
them
to Tsukuba.
Bear in mind that once there is fat pipes on the backbone and
internationally, the likelihood is that other bottlenecks will
show up
(end user equipment, local or far end networks, stack tuning
issues,
protocol issues, etc – remember all those talk about end-to-end
performance at the HEP meeting in Daegu Geoff?). So it will be
interesting to see what happens.
good luck
George
George McLaughlin
Director, International Developments, AARNet
Home Office: (02) 4849 4315
Mobile: 0411 256 370
________________________________________________________________
__
__
__
From:Geoffrey Taylor [mailto:g.taylor@xxxxxxxxxxxxxxxxxxxxxx]
Sent: Thursday, 12 January 2006 12:11 PM
To: George McLaughlin
Subject: Re: UofM - AARNet Member Agreement signed
Thanks George.
Happy New Year
Great news on the Taiwan connection.
Keep up the excellent support for Australian HEP. Its much
appreciated.
Regards
Geoff
On 12/01/2006, at 11:52 AM, George McLaughlin wrote:
Many thanks Greg
Can you please confirm that the SXTransPORT Schedule
(Schedule
5 of the Agreement) is also signed, and if by any chance
not,
then arrange for that to be executed at the earliest
opportunity.
Bruce, Mark and Steve – please note wrt SXTransPORT and
the
Australia-KEK (Tsukuba) data transfers.
Glenn and Geoff – We now have Jumbo Frames peering
between
SXTransPORT and TWAREN/TANET2 in Seattle, so a jumbo-
frame
high capacity path should now be available to the
Academia
Sinica TEIR1 LHC site in Taiwan – we need to talk more
about
this and with Simon Lin and his colleagues. We’ll
probably see
Simon in Tokyo.
Edwin – can you please let me know of any other specific
SXTransPORT requirements over and above those we are
already
liaising with the respective parties on
regards
George
From: Greg Chenhall [mailto:gmc@xxxxxxxxxxxxxx]
Sent: Wednesday, 11 January 2006 10:37 AM
To: Mary Fleming; Don Robertson; George McLaughlin
Cc: Edwin Wong; Colin Blythe
Subject: UofM - AARNet Member Agreement signed
Dear Mary, Don and George,
The UofM's legal team are sending the signed AARNet
Member
Agreement back through James Flowers, and I expect James
will
receive that by courier this week.
Research staff are keen to begin testing on
Sxtransport. If
there are any technical or other barriers to the use of
SXTransport could you please advise me.
Many Thanks,
Greg Chenhall
Director, Information Infrastructure
Information Division
Thomas Cherry Building, Level 4
The University of Melbourne, Victoria 3010 Australia
Phone: +61 3 8344 7888 Fax: +61 3 9341 6077
Internal Phone: 47888 Email: gmc@xxxxxxxxxxxxxx
This e-mail and any attachments may contain personal
information or information that is otherwise
confidential or
the subject of copyright. Any use, disclosure or
copying of
any part of it is prohibited. The University does not
warrant
that this email or any attachments are free from
viruses or
defects. Please check any attachments for viruses and
defects
before opening them. If this e-mail is received in
error,
please notify me by return e-mail and delete the
message.
______________________
Professor Geoffrey N. Taylor
Head, School of Physics, The University of Melbourne
Melbourne, Victoria, 3010 AUSTRALIA
g.taylor@xxxxxxxxxxxxxxxxxxxxxx
Ph: +61 3 8344 5420 Mobile: +61 419 533 080 Fax: +61 3 93494912
AARNet, Perth Office, Western Australia
Ph: +618 6436 8912
Fax: +618 6436 8585
www.aarnet.edu.au
AARNet, Perth Office, Western Australia
Ph: +618 6436 8912
Fax: +618 6436 8585
www.aarnet.edu.au
--
Chief Technology Officer AARNet Pty Ltd
Tel: +61 8 8303 6334 Level 7, 10 Pulteney Street
Fax: +61 8 8303 4400 The University of Adelaide
Web: http://www.aarnet.edu.au South AUSTRALIA, 5005
--- End Message ---
_______________________________________________
Discussion mailing list
Discussion@xxxxxxxxxx
http://internal.web100.org/mailman/listinfo/discussion
|