Internet Topology Collection
This page is maintained by Lucas Wang (lucas at cs.ucla.edu)
Home    |   Data sources    |   File format    |   Download data    |   Publications    |   Contact

Daily AS topology maps with annotated AS relationships (maps.YYYYMMDD.gz)

AS classification in business types (ases.YYYYMMDD.gz)

Prefix geolocation using Maxmind Geolite (geo.YYYYMMDD.gz)

Please refer to the paper "Quantifying the Completeness of the Observed Internet AS-level Structure" for a description of the relationship inference and AS classification and see the file format details bellow.

Announcements

2010/11/10: The internal problem mentioned below (on 2010/10/22) is fixed, and all historical data from 2009/12/19 are regenerated. If you have any questions or concerns, please contact Jonathan (jpark@cs.ucla.edu).

2010/10/22: Our topology data from 2009/12/19 to 2010/10/20 did not include historical data due to an internal error, and thus, may miss many p2p links. As of 2010/10/21, this missing links/nodes problem is fixed for links, nodes, weights, and origins files. We plan to fix the other files (ases and maps) by the end of next week (2010/10/29). If you were using these files prior to 2010/10/21, please discard the downloaded files and re-download. Please contact Jonathan Park (jpark@cs.ucla.edu) for any questions. We apologize for any inconvenience.

Collecting Internet AS-level topology

 Typically, an AS-level topology estimate is derived from BGP routing tables. Each entry in the routing table lists the path of ASes used to reach a destination prefix, and thus each entry implicitly lists AS connectivity information. Estimates of the Internet AS-level topology are commonly obtained by taking the union of all AS paths found in BGP routing tables available. The conventional technique only captures the AS links currently used in some BGP paths and can potentially miss lots of backup links not currently in routing tables. We found that many of these backup AS links can be observed from routing updates over time. Our goal is to capture the AS-level topology to the furthest extent using existing resources.

Data sources

 We update daily the topology map using the following data sources. NOTE: At this time we do not use data collected through traceroute for AS-level topology discovery, due to the inability to accurately converting router paths to AS paths. We plan to incorporate traceroute data once this problem is solved. For more information about this problem, please read these papers:
  • H. Chang, S. Jamin, and W. Willinger, "Inferring AS-level Internet topology from router-level path traces," in Proceeding of SPIE ITCom 2001, Denver, CO, August 2001 (ps).
  • Towards an Accurate AS-Level Traceroute Tool by Z. Morley Mao, Jennifer Rexford, Jia Wang, and Randy Katz, ACM SIGCOMM 2003 (pdf).
  • "On Third-party Addresses in Traceroute Paths" authored by Young Hyun, Andre Broido, and kc claffy. Presented at the Passive and Active Measurement Workshop in 2003 (pdf).

File format

 For each day, we mantain a file with the AS nodes and AS links observed so far. Each node/link is annotated with the time it was first observed and the time it was last observed. Furthermore, we also include information about the location of each link in the ASPATH.

Link file format: 
AS0      AS1      time_first_seen      time_last_seen     link_position		last_BGP_update
Times are in the UNIX timestamp format (seconds since midnight January 1, 1970 GMT/UTC).

Link position:
0: if it was seen at the beginning of an ASPATH
1: if it was seen at the end of an ASPATH
2: if it was seen at the middle of an ASPATH
3: cannot determine position of link, e.g. links extracted from looking glasses

Example: in ASPATH [7018 3356 11537 52], 7018-3356 is type 0, 3356-11537 is type 2, and 11537-52 is type 1. In ASPATH [701 29], link 701-29 is type 1.

Node file format
AS      time_first_seen      time_last_seen      node_position
Node position:
0: if it was seen at the beginning of an ASPATH
1: if it was seen at the end of an ASPATH
2: if it was seen at the middle of an ASPATH
3: cannot determine position of node


Weight file format: 
AS0	AS1	w0	w0_avg	w1 	w1_avg	scope
w0: number of routes using link AS0-AS1 when AS0 is the monitor (source)
w0_avg: value of w0 averaged over time w0_avg(t) = 0.8 * w0_avg(t-1) + 0.2 * w0(t)
w1: number of routes using link AS0-AS1 as viewed by monitors w/ full tables; if there are N such monitors, this weight is divided by N
w1_avg: value of w1 averaged over time w1_avg(t) = 0.8 * w1_avg(t-1) + 0.2 * w1(t)
scope: number of monitors with full tables using the link AS0-AS1


Map file format: 
AS0	AS1	relationship
Relationship:
p2p: if AS0 and AS1 are peers
c2p: if AS0 is a customer of AS1
p2c: if AS1 is a customer of AS0


AS file format: 
AS AS_type degree provider_tree customer_tree
AS_type:
Tier-1: if AS has no providers
Large ISP: if AS customer tree >=50
Small ISP: if AS customer tree <50 and >=5
Stub: if AS customer tree <5


Geo file format: 
Prefix Country City


Download data
Your name (optional):   Your affiliation (optional):

Publications
  • Pei-chun Cheng, Xin Zhao, Beichuan Zhang, Lixia Zhang, "Longitudinal Study of BGP Monitor Session Failures", ACM SIGCOMM Computer Communication Review (CCR), April 2010. (pdf)

  • Ricardo Oliveira, Dan Pei , Walter Willinger, Beichuan Zhang, Lixia Zhang, "The (in)Completeness of the Observed Internet AS-level Structure", IEEE/ACM Transactions on Networking, February 2010. (pdf)

  • Ricardo Oliveira, Dan Pei, Walter Willinger, Beichuan Zhang, Lixia Zhang, "In Search of the Elusive Ground Truth: The Internet's AS-level Connectivity Structure", ACM SIGMETRICS, Annapolis, Maryland, June 2008. (pdf,slides)

  • Ricardo Oliveira, Beichuan Zhang, Lixia Zhang, "Observing the Evolution of Internet AS Topology", ACM SIGCOMM, Kyoto, Japan, August 2007. (pdf)

  • Ricardo Oliveira, Ying-Ju Chi, Ioannis Pefkianakis, Mohit Lad, Lixia Zhang, "Visualizing Internet Topology Dynamics with Cyclops", ACM SIGCOMM (poster session), Kyoto, Japan, August 2007. (NANOG slides)

  • Beichuan Zhang, Raymond Liu, Daniel Massey, Lixia Zhang, "Collecting the Internet AS-level Topology," ACM SIGCOMM Computer Communication Review (CCR), special issue on Internet Vital Statistics, January, 2005. (pdf)

Contact

 Please send any comments/questions/bug reports to Lucas Wang (lucas at cs.ucla.edu), Jonathan Park (jpark at cs.ucla.edu), Ricardo Oliveira (rveloso at cs.ucla.edu), or Beichuan Zhang (bzhang at cs.arizona.edu).