aai map

University of Kansas

Telecommunications & Information Sciences Laboratory

line

AAI Performance Experiments

line

Abstract

Initial experiements that indicate that we may have some problems with small TCP segment sizes on the AAI, due to the fact that most current operating systems do not understand supernetting (grouping blocks of Class C addresses). Small segment sizes have a surprisingly large impact on performance. We implemented the correct solution with a quick hack to the OSF/1 v3.0 kernel, and the test results were successful (large segments across Class C boundaries, leading to high throughput). There are reasonably nice workarounds (per-route controls) in OSF/1, IRIX, and Linux. There is an extremely ugly workaround (large default MSS) for SunOS and Solaris.

Summary

  • measurements still difficult to perform
  • PVCs between ARL have only been up since approximately 22 April 1995
  • no other PVCs between KU and other AAI sites are up as of 30 April 1995
  • KU personnel have accounts at only two non-MAGIC sites
  • initial tests indicate performance less than expected
  • results can be partially explained by lack of operating system support for supernetting
  • workaround available in many cases

    • Experiment Series 1: KU to ARL

    Notes

  • experimental conditions: window = 51200, buffer = 8k, 29 ms RTT
  • with 51200 byte windows, should see about 14 Mb/s between KU
  • pacing at 10 Mb/s and 20 Mb/s on OTTO interface ineffective
  • some cell losses seen from KU to ARL when pacing > 10 Mb/s

    • Experimental Data

    Source Destination Pacing Throughput Cell Losses
    mauchly.ukans.aai.net arl.aai.net 10 Mb/s 1.5 Mb/s 0%
    mauchly.ukans.aai.net arl.aai.net 20 Mb/s 1.5 Mb/s 0.04% (1)
    arl.aai.net mauchly.ukans.aai.net - 12 Mb/s -

    Comments

    1. 17 errored packets on 37041 packets transferred (0.04%)

    Experiment Series 2: KU to TIOC on Different Subnets

    Notes

  • test throughput in known environment
  • pacing at 25 Mb/s on OTTO interface

    • Experimental Data

    Source Destination Throughput
    mauchly.ukans.magic.net houdini.tioc.magic.net 25 Mb/s
    mauchly.ukans.magic.net houdini.tioc.aai.net 2.5 Mb/s
    mauchly.ukans.aai.net houdini.tioc.magic.net 2.5 Mb/s
    mauchly.ukans.aai.net houdini.tioc.aai.net 2.5 Mb/s

    Experiment Series 3: Segment Sizes and Operating Systems

    Notes

  • ttcp tests between various sites across Class C subnet boundaries
  • recorded segment sizes using atmtimes on OSF/1
  • inferred segment sizes from number of bytes and packets received in other cases
  • if supernetting is not supported, other Class C subnets are assumed to be different physical networks
  • small segments used because IP assumes anything not on the local subnet passes through a router, hence default Maximum Segment Size (536 bytes) used

    • Experimental Data

  • small segments (approximately 512-576 bytes) observed
  • OSF/1 to/from OSF/1, IRIX, SunOS, Linux
  • IRIX to/from SunOS
  • SunOS to/from Solaris
  • other Solaris tests inconclusive due to non-standard configuration

    • Experiment Series 4: Supernetting Kernel Patch

    Notes

  • need support for classless subnets such as the 16 Class C subnets in AAI
  • prevents use of default MSS on same physical network
  • supernetting patch to OSF/1 kernel made and tested

    • Experimental Data

    Prior to OSF/1 Kernel Patch
    Source Destination Window Buffer MSS RTT Pacing Throughput Cell Losses
    tioc.aai.net ukans.aai.net 131072 bytes 8192 bytes 576 bytes 1.95 ms - 20 Mb/s 0%
    ukans.aai.net tioc.aai.net 131072 bytes 8192 bytes 576 bytes 1.95 ms 25 Mb/s 10 Mb/s 0%
    After OSF/1 Kernel Patch
    Source Destination Window Buffer MSS RTT Pacing Throughput Cell Losses
    tioc.aai.net ukans.aai.net 131072 bytes 8192 bytes 576 bytes 1.95 ms - 20 Mb/s 0%
    ukans.aai.net tioc.aai.net 131072 bytes 8192 bytes 576 bytes 1.95 ms 25 Mb/s 24.5 Mb/s 0%

    Experiment Series 5: Experiments with Per-Route MTU/MSS

    Notes

  • pre-route MTU/MSS workaround for supernetting
  • multihomed interfaces may cause performance oddities
  • KU configuration options
  • 204.235.73.2 (aai.net) as primary, 198.207.141.121 (magic.net) as alias
  • 198.207.141.121 (magic.net) as primary, 204.235.73.2 (aai.net) as alias
  • performance depends on configuration - OSF/1 problem?

    • Experimental Data

    Primary Interface at KU End: aai.net
    Source Destination Window Buffer MSS RTT Pacing Throughput Cell Losses
    tioc.aai.net ukans.aai.net 131072 bytes 65536 bytes 9180 bytes 1.95 ms - 76.4 Mb/s 0%
    ukans.aai.net tioc.aai.net 131072 bytes 65536 bytes 9180 bytes 1.95 ms 70 Mb/s 65.9 Mb/s 0%
    tioc.aai.net ukans.aai.net 131072 bytes 65536 bytes 536 bytes 1.95 ms - 22.4 Mb/s 0%
    ukans.aai.net tioc.aai.net 131072 bytes 65536 bytes 536 bytes 1.95 ms 70 Mb/s 2.7 Mb/s 0%
    Primary Interface at KU End: magic.net
    Source Destination Window Buffer MSS RTT Pacing Throughput Cell Losses
    tioc.aai.net ukans.aai.net 131072 bytes 65536 bytes 9180 bytes 1.95 ms - 71.2 Mb/s 0%
    ukans.aai.net tioc.aai.net (1) 131072 bytes 65536 bytes 9180 bytes 1.95 ms 70 Mb/s 49.4 Mb/s 0%
    ukans.aai.net tioc.aai.net (2) 131072 bytes 65536 bytes 9180 bytes 1.95 ms 70 Mb/s 65.2 Mb/s 0%

    Comments

    1. ipgateway off - interleaved 48 byte pdus from tioc.aai.net when ukans.aai.net transmitting
    2. ipgateway on - no interleaved small pdus from tioc.aai.net

    Experiment Series 6: Throughput Between Various AAI Subnets

    Notes

  • test throughput in various environments
  • develop baseline for further experiments

    • Experimental Data

    Source Destination Window Buffer MSS RTT Throughput Cell Losses
    arl.aai.net tioc.aai.net 51200 bytes 8192 bytes ~512 bytes 28 ms 7.3 Mb/s 0%
    arl.aai.net tioc.aai.net 51200 bytes 8192 bytes ~512 bytes 28 ms 2.2 Mb/s 0.76% (1)
    arl.aai.net edc.aai.net 51200 bytes 8192 bytes ~512 bytes 35 ms 9.1 Mb/s 0%
    arl.aai.net edc.aai.net 51200 bytes 8192 bytes ~512 bytes 35 ms 6.3 Mb/s 0.15% (2)
    edc.aai.net arl.aai.net 51200 bytes 8192 bytes ~8000 bytes 35 ms 9.8 Mb/s 0.10% (3)
    cewes.aai.net tioc.aai.net 65536 bytes 8192 bytes ~512 bytes 30 ms 7.6 Mb/s 0%
    tioc.aai.net cewes.aai.net 65536 bytes 8192 bytes ~512 bytes 30 ms 0.9 Mb/s 0.79% (4)
    arl.aai.net cewes.aai.net 51200 bytes 8192 bytes ~512 bytes 37 ms 1.9 Mb/s 0.25% (5)
    cewes.aai.net arl.aai.net 51200 bytes 8192 bytes ~512 bytes 37 ms 6.1 Mb/s 0%

    Comments

    1. 256 errored packets on 32496 packets transferred (0.76%)
    2. 52 packet errors on 34197 packets transferred (0.15%)
    3. 2 packet errors on 2098 packets transferred (0.10%)
    4. 264 errored packets on 33227 packets transferred (0.79%)
    5. 82 packet errors on 32916 packets transferred (0.25%)

    Conclusions

  • segment size workaround available in OSF/1, IRIX, Linux
  • specify MTU or MSS on "route" command line
  • OSF/1: route add -net 204.235.73.0 204.235.71.x -interface -mtu 9180
  • IRIX: route add net 204.235.73.0 204.235.71.2 0 -mtu 9180
  • Linux: route add -net 204.235.73.0 dev atm0 mss 9180
  • no workaround seems to be available in SunOS, Ultrix except for setting the global default MSS to a large value, which is unfriendly to routers and may be unacceptable
  • not clear if workaround available in Solaris
  • KU supernetting kernel patch available in OSF/1
    • line

    Author

    Joseph B. Evans, <evans@eecs.ku.edu>