Received: from pizza by PIZZA.BBN.COM id aa08109; 23 Sep 93 10:50 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa08105; 23 Sep 93 10:48 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa11332; 23 Sep 93 10:48 EDT Received: by ginger.lcs.mit.edu id AA29370; Thu, 23 Sep 93 10:48:25 -0400 Date: Thu, 23 Sep 93 10:48:25 -0400 From: Noel Chiappa Message-Id: <9309231448.AA29370@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Verify Cc: jnc@ginger.lcs.mit.edu Hi, this is just to let everyone who's on so far know they got added OK. "If you didn't get this message, you need to resend your request to be added!" :-) Noel   Received: from pizza by PIZZA.BBN.COM id aa25705; 27 Sep 93 17:41 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa25701; 27 Sep 93 17:38 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa20893; 27 Sep 93 17:37 EDT Received: by ginger.lcs.mit.edu id AA01056; Mon, 27 Sep 93 17:37:14 -0400 Date: Mon, 27 Sep 93 17:37:14 -0400 From: Noel Chiappa Message-Id: <9309272137.AA01056@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: OK, here we go... Cc: jnc@ginger.lcs.mit.edu Hi all, a few things to get the ball rolling. First, the old Nimrod I-D that I wrote some years ago has been put back online, as "draft-chiappa-routing-01.txt". It's fairly long, and pretty dense, so don't expect to whiz through it, but if you want to participate heavily in the discussions (as opposed to listen with one ear), it's a good idea to read this. This is identical to the one of a couple of years ago, so if you read that one, you don't need to read this one, although it might be good to scan it quickly to swap it all back in. It contains basically three things: some stuff about generic theory of routing in very large scale networks, a drafty architecture based on those thoughts, and some ideas on a deployment plan. These are all a bit dated; if I find time (hard with a 2 week old baby at home :-) I'll update it to include the last several years of Big-I induced thinking, which have clarified *lots* of stuff I didn't understand well (or at all :-) when I wrote this. Second, I am attempting to get the technical content of the BBN proposal released, so everyone will have an idea what the direction of the funded stuff looks like. BBN has also drafted a short note which explains the relationship of the funded BBN contract work to the WG, and I hope that will be released soon. I'll paraphrase here my understanding of what their position is (lest anyone have a paranoia attack from lack of info :-). However, this is my impression, and the final word will have to come from BBN. My understanding is that they intend to be active participants in the WG, just ones who have a lot more time and energy to devote to the work of design and document writing than the bulk of the other members will. (This does not mean other members can't write; if you want to volunteer, you may well get recruited! :-) It is intended that the WG be the focus of Nimrod design work; i.e. this WG does not exist solely to review "done" specs, but rather points will be debated here as the design goes forward. In other words, BBN is not going to go off in their own direction, ignoring what the WG does. BBN does have contractual obligations to their funder, and they may be forced to take a divergent path temporarily to meet an obligation, but they will get try and back into synchronization with the WG's direction if that happens. Again, this is just my impression, not an official BBN position. Third, a WG charter has been drafted, and was submitted some time ago: I'll send a copy of the draft charter to the WG mailing list. If anyone has comments, please send them to me directly. Fourth, a glance at the charter will show we have a lot to do, and not a whole lot of time. As such, we are going to have to be crisp. I'm going to insist on a certain amount of "democratic centralism"; i.e., once things are decided, they generally stay decided. This does not mean I think we shouldn't go back and look at things if it seems there's a better way, or we went wrong, but if a point gets argued to death, and something is picked, attempts to reopen the point are going to be frowned on, unless there's a good reason to repoen them. (E.g., someone had a new idea, or saw a previously unseen problem.) At any time, I encourage anyone who doesn't like something, either procedural or technical, to pick up the telephone and call me. (The number is 1-804-898-8183, 10AM-5PM or 8PM-11PM EST.) I will be more than happy to hear your point, and discuss it with you. I may not agree, but if I think your point has any merit, I will probably usually refer it to the WG and see what the general opinion is. I'm very concerned that we balance on the exact point on the progress/openness spectrum which is most productive in the long run. Anyway, thanks for the interest, and here we go. I will send out a message tonight detailing the first couple of steps I'd like to see happen. Noel   Received: from pizza by PIZZA.BBN.COM id aa27658; 28 Sep 93 6:59 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa27647; 28 Sep 93 6:57 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03200; 28 Sep 93 6:41 EDT Received: by ginger.lcs.mit.edu id AA03486; Tue, 28 Sep 93 02:03:13 -0400 Date: Tue, 28 Sep 93 02:03:13 -0400 From: Noel Chiappa Message-Id: <9309280603.AA03486@ginger.lcs.mit.edu> To: Hal.Miller@mel.dit.csiro.au, nimrod-wg@BBN.COM Subject: Re: Nimrod WG Cc: jnc@ginger.lcs.mit.edu Please add Hal. Noel -------- I had planned to join today. Should I do so formally, or are you taking informal requests? (If so, please add me!)   Received: from pizza by PIZZA.BBN.COM id aa27639; 28 Sep 93 6:59 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa27635; 28 Sep 93 6:56 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03175; 28 Sep 93 6:41 EDT Received: by ginger.lcs.mit.edu id AA03842; Tue, 28 Sep 93 02:45:25 -0400 Date: Tue, 28 Sep 93 02:45:25 -0400 From: Noel Chiappa Message-Id: <9309280645.AA03842@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Terminology (I: Big-I) Here are the terms developed by Big-I. Noel -------- "interface locator" - The structured name of an interface, i.e. a host's attachment point to the internetwork. The structure is used by the routing to make its job a little easier. Things which have related locators must usually be topologically related, or else the routing will break down. The destination locator might or might not appear in all packets (see "selector"). "element" - An individual component of a structured, presumably heirarchical, locator. "locator" - Usually alternative, short, term for the first term above. "locator" used on its own often means an "interface locator", but it can also mean the structured name of an aggregate, be it a single net, a group of nets, or whatever. This is usually formed by dropping the least significant elements from a locator. "host locator" - Some people believe that hosts, as well as interfaces, need names which indicate where they are. This is otherwise identical to an "interface locator", but names a host, not an interface. Again, in an architecture which has them, the destination locator might or might not appear in all packets (see "selector"). "forwarding selector", "selector" - The field(s) in the packets which the routers look at to decide where to send the packet next. Forwarding selectors can exist at different layers; i.e. the link level and the internetwork. The selector might or might not be, or include, a locator. For example, in X.25, the selector is the VC-ID, in IPv4, the 'destination IP address'; in a flow network, it would be the flow id. "host identifier" - A name (not *necessarily* an ASCII string; i.e. not a DNS name), probably globally unique, and not a locator, for a host. The current IPv4 'address' has this as one of its current functions (e.g. in the TCP pseudo-header). Much the same as an EID, for those who understand that term. "hostname" - A DNS name.   Received: from pizza by PIZZA.BBN.COM id aa27684; 28 Sep 93 6:59 EDT Received: from BBN.COM by PIZZA.BBN.COM id ac27657; 28 Sep 93 6:57 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03238; 28 Sep 93 6:41 EDT Received: by ginger.lcs.mit.edu id AA02379; Mon, 27 Sep 93 21:20:30 -0400 Date: Mon, 27 Sep 93 21:20:30 -0400 From: Noel Chiappa Message-Id: <9309280120.AA02379@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Draft charter for WG Cc: jnc@ginger.lcs.mit.edu Here's a a copy of the latest version of draft charter. If anyone has comments, please send them to me directly. Noel ---------------- WG Charter: New Internet Routing and Addressing Architecture (Nimrod) Chair(s): J. Noel Chiappa Isidro Castineyra Routing Area Director(s) Bob Hinden Mailing lists: General Discussion: nimrod-wg@bbn.com To Subscribe: nimrod-wg-request@bbn.com Archive: Description of Working Group: The goal of the WG is to design, specify, implement and test a flexible new routing and addressing architecture suitable for very large scale internets. The basic architecture for computation of routes will be based on distribution of network topology maps, with source-specified route selection, and local (i.e. not hop by hop) computation of routes. The architecture will provide a single homogenous framework for all routing, including both inter-domain and intra-domain. It will include a new network component naming abstraction hierarchy (i.e. addressing scheme), starting from network attachment points, and based in actual connectivity, but taking into consideration policy requirements. These new addresses will be variable length, with a variable number of levels of abstraction; they will not appear in most packets, though. Actual packet forwarding will be based both on retained non-critical state in the switches (via flow setup, for long-lived communications), and source-route type instructions in individual packets (for one-shot communications). Although the general design and algorithms will be usable in any internetworking protocol family, the initial detailed protocol specifications and implementation are currently planned for deployment with IPv4, but support for another packet format may be substituted or added, depending on the situation in the Internet in the future. Interoperabilty with existing unmodified IPv4 hosts will be achieved by re-interpreting the existing source and destination fields in IPv4 packets as endpoint identifiers. An effort to take into account support for mobility, multicast and resource allocation will be made when designing the Nimrod architecture, provided it is not burdensome. Nimrod itself is not an attempt to solve these problems, although it may include mechanisms to facilitate providing these capabilities. Goals and Milestones: October 1993 Commence project. January 1994 Complete the review and discussion of the fundamentals of the routing and addressing architecture. February 1994 Produce an architecture document, which will also serve as an in-depth introduction to Nimrod. May 1994 Complete the design of the basic routing and addressing architecture and protocols, i.e. expand and refine the existing high-level design. June 1994 Produce a first version of the protocol specification, which embodies the completed basic routing and addressing architecture. September 1994 Finish design of all the detailed mechanisms, including sample algorithms for those parts which are outside the core specification. October 1994 Produce an algorithms supplement document which describes the sample algorithms. July 1995 Finish an initial prototype protocol implementation, suitable for experimentation within the Internet, to allow field trials. October 1995 Complete an initial field trial of the prototype protocol and implementation. January 1996 After assessing the performance of the protocols and sample algorithms, based on operational experience, release an updated protocol specification and sample algorithms.   Received: from pizza by PIZZA.BBN.COM id aa27654; 28 Sep 93 6:59 EDT Received: from BBN.COM by PIZZA.BBN.COM id ad27635; 28 Sep 93 6:57 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03207; 28 Sep 93 6:41 EDT Received: by ginger.lcs.mit.edu id AA03770; Tue, 28 Sep 93 02:27:22 -0400 Date: Tue, 28 Sep 93 02:27:22 -0400 From: Noel Chiappa Message-Id: <9309280627.AA03770@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Oops... Cc: jnc@ginger.lcs.mit.edu Sorry about that last, everyone; should have been -request, obviously... Noel   Received: from pizza by PIZZA.BBN.COM id aa27643; 28 Sep 93 6:59 EDT Received: from BBN.COM by PIZZA.BBN.COM id ab27635; 28 Sep 93 6:56 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03187; 28 Sep 93 6:41 EDT Received: by ginger.lcs.mit.edu id AA03831; Tue, 28 Sep 93 02:42:04 -0400 Date: Tue, 28 Sep 93 02:42:04 -0400 From: Noel Chiappa Message-Id: <9309280642.AA03831@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: First moves... Cc: jnc@ginger.lcs.mit.edu OK, here's what I reckon the initial plan of action should be. First, a list of "open architectural issues" in Nimrod has been put together (from stuff in the old I-D, points raised on Big-I, questions asked at the Nimrod Bof at the Boston IETF, etc). These things are places where the initial vision in the documents is incomplete, or the documents give a number of alternatives, but don't pick one, etc, etc. It is not low level, or open- ended things like "what is the database exchange protocol", but major open single issues like "will Nimrod aggregate flows, or keep them separate". I will clean this list up, and send it to the list tomorrow; what I would like to hear first are suggestions for things that are missing from the list. Once we have a complete list in hand, we can go down it, one issue at a time, discussing these points. Hopefully, at the end of two months or so, we will have a much better idea of what the basic architecture looks like, and we can start writing the architecture document. Second, we need to have clear, consistent, well-defined and universally understood terminology. I propose we start with the basic terminology set which was recently developed on Big-I, with suggestions from a number of people; I assume that will be acceptable to everyone? (I will forward a copy separately.) Nimrod needs a more extensive set of terms than just these, however, to describe some of the more involved ways in which Nimrod handles abstraction issues when policy and scaling are involved. I'll put together a list of my current suggestions for terminology, and send it in, and everyone can shoot arrows at it! :-) Finally, I'm not sure quite what to do about the term "address". The Big-I list could never settle on a definition everyone liked, and people persisted in using it for conflicting definitions, which made debates really confusing. I see two possibilities for the Nimrod arena; the first is to make it *exactly* equivalent to "locator" (see the Big-I jargon list), and keep to that, and the second is to simply Never, Ever, use it. What does everyone think? Noel   Received: from pizza by PIZZA.BBN.COM id aa28158; 28 Sep 93 8:29 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa28154; 28 Sep 93 8:28 EDT Received: from pluto.dss.com by BBN.COM id aa06985; 28 Sep 93 8:29 EDT Received: by pluto.dss.com (4.1/SMI-4.0) id AA13836; Tue, 28 Sep 93 08:26:15 EDT Date: Tue, 28 Sep 93 08:26:15 EDT From: Mike Davis Message-Id: <9309281226.AA13836@pluto.dss.com> To: jnc@ginger.lcs.mit.edu Cc: nimrod-wg@BBN.COM In-Reply-To: Noel Chiappa's message of Tue, 28 Sep 93 02:42:04 -0400 <9309280642.AA03831@ginger.lcs.mit.edu> Subject: First moves... Noel, Finally, I'm not sure quite what to do about the term "address". The Big-I list could never settle on a definition everyone liked, and people persisted in using it for conflicting definitions, which made debates really confusing. I see two possibilities for the Nimrod arena; the first is to make it *exactly* equivalent to "locator" (see the Big-I jargon list), and keep to that, and the second is to simply Never, Ever, use it. What does everyone think? As you correctly observe, immense amounts of confusion have resulted from the (ab)use of the word "address" in connection with routing. As I believe in using precise terms where they exist, I would propose that we Never, Ever, use it. Of course, that means you'll have to change the name of the paper to "A New IP Selecting and ID'ing Architecture." -- Mike Davis mike@dss.com + Ceci n'est pas une .sig Datability Communications Research Center + 261 West Central Street, Natick, MA, 02160 +   Received: from pizza by PIZZA.BBN.COM id aa28541; 28 Sep 93 9:51 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa28537; 28 Sep 93 9:50 EDT Received: from ietf.cnri.reston.va.us by BBN.COM id aa11678; 28 Sep 93 9:48 EDT Received: from [127.0.0.1] by IETF.CNRI.Reston.VA.US id aa02857; 28 Sep 93 9:43 EDT To: Noel Chiappa cc: nimrod-wg@BBN.COM Subject: Re: Terminology (I: Big-I) In-reply-to: Your message of "Tue, 28 Sep 93 02:45:25 EDT." <9309280645.AA03842@ginger.lcs.mit.edu> Date: Tue, 28 Sep 93 09:43:46 -0400 From: "Vinton G. Cerf" Message-ID: <9309280943.aa02857@IETF.CNRI.Reston.VA.US> Noel, I think we need a way to identify a host without binding that identifier to a specific interface. We also need a way to refer to interfaces. Where interfaces link to very distinct networks, there is an obvious problem binding a host identifier to one or more interfaces and this is often handled by means of ARP-like methods. I don't see much functional difference between host identifiers and host names in this context, do you? Vint   Received: from pizza by PIZZA.BBN.COM id aa28698; 28 Sep 93 10:21 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa28694; 28 Sep 93 10:19 EDT Received: from MITCHELL.CIT.CORNELL.EDU by BBN.COM id aa13543; 28 Sep 93 10:20 EDT Received: from by mitchell.cit.cornell.edu (4.1/1.34/Honig-1.3) id AB29675; Tue, 28 Sep 93 10:20:04 EDT Message-Id: <9309281420.AB29675@mitchell.cit.cornell.edu> X-Sender: swb@nr-tech.cit.cornell.edu Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 28 Sep 1993 10:19:27 -0400 To: Noel Chiappa , nimrod-wg@BBN.COM From: Scott W Brim Subject: Re: OK, here we go... Cc: jnc@ginger.lcs.mit.edu >if a point gets argued to death, and something is picked, attempts to >reopen the point are going to be frowned on Anything on the scale of Nimrod has to have, written down, perhaps in an analysis document, the reasons why particular paths were taken and why others were *not* taken. Please let's be sure to do that along the way. Besides being the only way to avoid repeating arguments, that sort of documentation is a necessary part of our institutional memory.   Received: from pizza by PIZZA.BBN.COM id aa28805; 28 Sep 93 10:45 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa28801; 28 Sep 93 10:44 EDT Received: from atlas.xylogics.com by BBN.COM id aa14589; 28 Sep 93 10:44 EDT Received: by atlas.xylogics.com id AA28296 (5.65c/UK-2.1-930726); Tue, 28 Sep 1993 10:45:46 -0400 From: Gary Scott Malkin Date: Tue, 28 Sep 1993 10:45:46 -0400 Message-Id: <28296.199309281445@atlas.xylogics.com> To: jnc@ginger.lcs.mit.edu Cc: nimrod-wg@BBN.COM In-Reply-To: Noel Chiappa's message of Tue, 28 Sep 93 02:42:04 -0400 <9309280642.AA03831@ginger.lcs.mit.edu> Subject: First moves... We should definitely not use "address", even as an alias. It simply has too much historical baggage associated with it. ---------------------------------------------------------------------- Gary Malkin Cheap, Fast, Good (617) 272-8140 Pick two!   Received: from pizza by PIZZA.BBN.COM id aa28820; 28 Sep 93 10:49 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa28816; 28 Sep 93 10:47 EDT Received: from mcigateway.mcimail.com by BBN.COM id aa14752; 28 Sep 93 10:49 EDT Received: from mcimail.com by MCIGATEWAY.MCIMail.com id ae10012; 28 Sep 93 14:13 GMT Date: Tue, 28 Sep 93 14:15 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: First moves... Message-Id: <55930928141555/0003858921NA3EM@mcimail.com> Mike Davis said: >I believe in using precise terms where they exist, I would propose >that we Never, Ever, use it. Since some approximation of the English language is spoken here, and none of us EVER want to be accused of legalesse (at least I hope not), we can never ever not use the 'A' word. I think that the use of the work 'address' should exactly correspond with its common usage. My house has an address: 15210 Sutherland Oak Park, MI 48237. If you are not familiar with Oak Park, MI, you will never find my house. But don't worry, the US mailman knows where I live and will deliver any mail to my home. UPS and FED-ex have gotten there too. I have also supplied routing directions to friends and family coming from different addresses, themselves. Also, there is only one house in the world with this address. There are no duplicates anyplace. Oh, I suppose, I really should have added my country of USA to that address. So to should 'address' by used in Networking technology. Oh, some other examples of 'addresses' and routing. Never tell someone that you are at the corner of Southfield and Outer Drive in Detroit, Mi. They cross twice. Be careful about telling someone driving south on Lee Road in Cleveland Hights that you are north of South Park. When they cross South Park before they see North Park they tend to get confused (North and South Park Roads cross over for about 1/2 mile...). Bob Moskowitz Chrysler Corp   Received: from pizza by PIZZA.BBN.COM id aa29020; 28 Sep 93 11:09 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29016; 28 Sep 93 11:07 EDT Received: from alpha.Xerox.COM by BBN.COM id aa15535; 28 Sep 93 11:07 EDT Received: from skylark.parc.xerox.com ([13.2.116.7]) by alpha.xerox.com with SMTP id <11764(3)>; Tue, 28 Sep 1993 08:07:07 PDT Received: from localhost by skylark.parc.xerox.com with SMTP id <12171>; Tue, 28 Sep 1993 08:07:02 -0700 To: Noel Chiappa Cc: nimrod-wg@BBN.COM, deering@parc.xerox.com Subject: Re: Draft charter for WG In-reply-to: Your message of "Mon, 27 Sep 93 18:20:30 PDT." <9309280120.AA02379@ginger.lcs.mit.edu> Date: Tue, 28 Sep 1993 08:06:52 PDT Sender: Steve Deering From: Steve Deering Message-Id: <93Sep28.080702pdt.12171@skylark.parc.xerox.com> > An effort to take into account support for mobility, multicast and > resource allocation will be made when designing the Nimrod architecture, > provided it is not burdensome. Nimrod itself is not an attempt to solve these > problems, although it may include mechanisms to facilitate providing these > capabilities. Noel, It seems unlikely to me that a new routing and addr*ssing architecture could punt on those issues and still be relevent. I can see how mobility might be left as a "bag on the side" (as the IP mobility work has shown), but multicast routing and resource-sensitive routing seem fundamental to me, not things that can be tacked on as an afterthought. Steve   Received: from pizza by PIZZA.BBN.COM id aa29207; 28 Sep 93 11:32 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29203; 28 Sep 93 11:31 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa16643; 28 Sep 93 11:31 EDT Received: by ginger.lcs.mit.edu id AA06911; Tue, 28 Sep 93 11:30:49 -0400 Date: Tue, 28 Sep 93 11:30:49 -0400 From: Noel Chiappa Message-Id: <9309281530.AA06911@ginger.lcs.mit.edu> To: jnc@ginger.lcs.mit.edu, vcerf@cnri.reston.va.us Subject: Re: Terminology (I: Big-I) Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM I think we need a way to identify a host without binding that identifier to a specific interface. Vint, I think this is fairly widely agreed upon, although I would not call it "general acceptance" yet. (Not sure I want to get into a big debate over this; Big-I has worn itself out on it innumerable times. :-) In any case, Nimrod as a routing architecture doesn't care about this issue, obviously, since this is really not a routing issue per se. Nimrod as a deployment plan will do exactly this, as mentioned in the WG Charter. Obviously, I've agreed with this point for a *long* time (since the conversations with Jerry which led to his 1982 paper :-), and have campaigned vigorously for just such a separate namespace. However, on a suggestion from Dave Oran, I think it ought to name something a little more abstract than a host, i.e. the endpoint (definition coming in Teminology II :-). We also need a way to refer to interfaces. Yes. The abstraction naming heirarchy, i.e. "locators", in Nimrod (what used to be "addressing scheme", before we canned the A-word :-) will definitely be a separate namespace, and it will name interfaces. (Reasons given when we get to discussing the form of locators, one of the first of the open issues.) Where interfaces link to very distinct networks, there is an obvious problem binding a host identifier to one or more interfaces and this is often handled by means of ARP-like methods. Ah. There are two parts to this. The first is "how do you get the list of the interfaces which connect to a given host identifier", and the second is "given this list, how do I pick one"? Both of these are in the issues list. The first is needed mechanism, but is a reasonably straightforward problem. The second is more complex: you need to think of this decision (which interface to a host to use) as a routing decision, just like decisions on which nets to use to get there. If you need a 20 Mbit/sec stream, you need to know that one interface is Ethernet and can't hack it, and the other is FDDI and can. Looked at as a routing decision, it becomes part of an existing (admittedly hard :-) problem, not a separate one. I don't see much functional difference between host identifiers and host names in this context, do you? In this particular context (as something to feed into the binding system to get out the list of interfaces), no, but in general, there is a difference. In case that wasn't what you meant, the host identifiers are used at all layers in the protocol stack, e.g. to name the entities at each end of a reliable stream, for the duration of the stream. Host names have no such property; the are used at the start of the transaction, and then no more. In addition, it is perfectly feasible to run an application without a hostname, by typing in the appropriate number, so hostnames aren't very central to the system. Noel   Received: from pizza by PIZZA.BBN.COM id aa29403; 28 Sep 93 12:01 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29399; 28 Sep 93 11:59 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa18125; 28 Sep 93 11:59 EDT Received: by ginger.lcs.mit.edu id AA07127; Tue, 28 Sep 93 11:59:14 -0400 Date: Tue, 28 Sep 93 11:59:14 -0400 From: Noel Chiappa Message-Id: <9309281559.AA07127@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM, swb1@cornell.edu Subject: Re: OK, here we go... Cc: jnc@ginger.lcs.mit.edu Anything on the scale of Nimrod has to have, written down, perhaps in an analysis document, the reasons why particular paths were taken and why others were *not* taken. Please let's be sure to do that along the way. Besides being the only way to avoid repeating arguments, that sort of documentation is a necessary part of our institutional memory. Excellent point. Perhaps we can use the Host Requirements style, with "Discussion" and "Implementation" sections. Noel   Received: from pizza by PIZZA.BBN.COM id aa29454; 28 Sep 93 12:08 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29450; 28 Sep 93 12:07 EDT Received: from hermes.bos.locus.com by BBN.COM id aa18558; 28 Sep 93 12:08 EDT Received: from medusa.edec.locus.com by hermes.edec.Locus.COM id aa04509; 28 Sep 93 12:02 EDT Received: by locus.com (5.65/DEC-OSF/1) id AA04006; Tue, 28 Sep 1993 12:07:54 -0400 Message-Id: <9309281607.AA04006@locus.com> To: Noel Chiappa Cc: nimrod-wg@BBN.COM Subject: Re: First moves... In-Reply-To: Your message of "Tue, 28 Sep 93 02:42:04 EDT." <9309280642.AA03831@ginger.lcs.mit.edu> Date: Tue, 28 Sep 93 12:07:53 -0400 From: avri@locus.com X-Mts: smtp Re: the word address. I don't believe that you can ever set a formal definition on a common word and have it stick. That is because you never know, in a conversation, whether one means address in the formal sense or in the folk sense. This is unless you specifically make statements like "address, i mean in the formal sense of locator". If you do this, however, you will start to sound like a philosopher (a fate, surely, to be avoided). I also don't believe that you can ever ban a word from the lexicon. I do believe that you can ban the word from the architecture document, and you can treat any usage of the word as refering to the general class of names that refer to entities such as locators, identifiers and selectors. I.e. you can treat it as a meta-term instead of a architectural term. avri   Received: from pizza by PIZZA.BBN.COM id aa29564; 28 Sep 93 12:38 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29560; 28 Sep 93 12:36 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa19943; 28 Sep 93 12:36 EDT Received: by ginger.lcs.mit.edu id AA07468; Tue, 28 Sep 93 12:36:50 -0400 Date: Tue, 28 Sep 93 12:36:50 -0400 From: Noel Chiappa Message-Id: <9309281636.AA07468@ginger.lcs.mit.edu> To: deering@parc.xerox.com, jnc@ginger.lcs.mit.edu Subject: Re: Draft charter for WG Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM > An effort to take into account support for mobility, multicast and > resource allocation will be made when designing the Nimrod architecture, > provided it is not burdensome. Nimrod itself is not an attempt to solve > these problems, although it may include mechanisms to facilitate > providing these capabilities. It seems unlikely to me that a new routing and addr*ssing architecture could punt on those issues and still be relevent. I can see how mobility might be left as a "bag on the side" (as the IP mobility work has shown), but multicast routing and resource-sensitive routing seem fundamental to me, not things that can be tacked on as an afterthought. Steve, the draft meant what it said; we will try hard to do these things, because we recognize that they *are* important. However, there are components of the problem (particular in the resource allocation area) which are outside the scope of Nimrod, and to tie Nimrod too closely to that might be to drag it down. Let's look at the easy one first, resource-sensitive routing. The ORWG looked at this (see the Appendix to RFC-1126), and decided (correctly, I think) that it was too dynamic to include *directly* in the routing (i.e. the way the old ARPANet used to) in a very large system. The general consensus was that resource allocation needed to be a separate subsystem, with *ties* to routing, sure, but not totally integrated. Map based, source routed, systems (such as Nimrod) obviously have a tremendous advantage here, since it's easy to see how to put info from the resource allocation subsystem into the route selection process, along with topology information. So, I think this will be a pretty natural thing to see in Nimrod. Note also that the route selection algorithm is not a *fundamental* part of Nimrod (as currently envisioned, anyway), but rather a local decision. So, it would be easy to incrementally "field-swap" in a new route selection algorithm which *does* do resource-sensitive routing, even if the first deployed one did not. Likewise for "attributes" which say something about the utilization of the link; since the attribute set is incrementally expandable, that too can be added, if we don't know enough to do it first pass. The more interesting one is multicast. Half of the time I think it should be a separate sub-layer (architecturally, if not in implementation, the way Van J's fast TCP smooshes TCP and IP together) on top of the basic packet layer, and half of the time I think it makes sense to integrate it fully into the basic packet layer. The obvious concern here is the routing aspects of multicast. I think that in small systems it *does* work to integrate multicast group support into the routing; it's just one more destination to track, sort of. However, as the multicast world is now discovering, scaling is a Big Problem (surprise, surprise :-). Groups with a few widely scattered members, in a very large internet, seem to be best supported with entirely different mechanisms from groups with many, many members. Once again (as with resource allocation), larger systems mean separate mechanisms are needed (a general system design principle I talked about on Big-I). I don't pretend to have anything but the faintest glimmers of how to deal with multicast. I will not be surprised if it turns out that multi-cast *is* best handled (both route selection *and* forwarding) by a separate subsystem from unicast traffic, one which, like resource allocation, has ties to the routing system (particularly in the route selection phase, where you are computing the distribution tree). Note that the point above about scaling necessitating separate mechanisms applies even to the basic packet layer; Nimrod is pulling apart forwarding and route selection for unicast traffic, so don't be surprised if the same comes true of multicast. However, this is just speculation. Rest assured, however, it was *already* in the issues list! :-) The interaction with resource allocation wasn't, and I have added it; thaks for the tip. Noel   Received: from pizza by PIZZA.BBN.COM id aa29702; 28 Sep 93 12:58 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29698; 28 Sep 93 12:56 EDT Received: from inet-gw-1.pa.dec.com by BBN.COM id aa20821; 28 Sep 93 12:57 EDT Received: by inet-gw-1.pa.dec.com; id AA02491; Tue, 28 Sep 93 09:57:26 -0700 Received: by sneezy.lkg.dec.com (5.65/DEC-Ultrix/4.3) id AA14490; Tue, 28 Sep 1993 12:57:23 -0400 To: Noel Chiappa Cc: vcerf@cnri.reston.va.us, nimrod-wg@BBN.COM Subject: Re: Terminology (I: Big-I) In-Reply-To: <9309281530.AA06911@ginger.lcs.mit.edu> References: <9309281530.AA06911@ginger.lcs.mit.edu> X-Mailer: Poste 2.1 From: David Oran Date: Tue, 28 Sep 93 12:57:22 -0400 Message-Id: <930928125722.11119@sneezy.lkg.dec.com.-v> Encoding: 34 TEXT, 6 TEXT SIGNATURE > I think we need a way to identify a host without binding that identifier > to a specific interface. > > Vint, I think this is fairly widely agreed upon, although I would not call it > "general acceptance" yet. (Not sure I want to get into a big debate over this; > Big-I has worn itself out on it innumerable times. :-) In any case, Nimrod as a > routing architecture doesn't care about this issue, obviously, since this is > really not a routing issue per se. > At the risk of re-opening a very large can of creepy crawly things, I'd like to propose that "hosts" do not need identifiers for the purpose of identifying the endpoint of individual packets. Hosts need NAMES so that people and applications can refer to collections of application instances, and so that managers can have a handle on a physical "Box" or set of boes that fails as a unit. So, why have I kept arguing that packets need endpoint identifiers in them somewhere? Because you need a very efficient way to: a) tell whether a packet has arrived at the correct destination b) deliver the packet to the right "application" at that destination. I propose that somewhere in the lower layers there has to be an "endpoint identifier" which allows a very fast and efficient way to deliver data from network adapters to the address space of the ultimate consumer without creating a serial bottleneck through a single CPU or kernel instance. Some data is already available to support this view [c.f. the paper on user protocol stack implementation by the U. Washington folks] who used the link-level packet identifier scheme to put packets straight into the right address space. Dave. -+-+-+-+-+-+-+ David R. Oran Phone: + 1 508 486-7377 Digital Equipment Corporation Fax: + 1 508 486-5279 LKG 1-2/A19 Email: oran@lkg.dec.com 550 King Street Littleton, MA 01460   Received: from pizza by PIZZA.BBN.COM id aa29872; 28 Sep 93 13:28 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29868; 28 Sep 93 13:27 EDT Received: from mcigateway.mcimail.com by BBN.COM id aa22082; 28 Sep 93 13:27 EDT Received: from mcimail.com by MCIGATEWAY.MCIMail.com id ai18029; 28 Sep 93 17:07 GMT Date: Tue, 28 Sep 93 17:09 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: Terminology (I: Big-I) Message-Id: <15930928170951/0003858921NA4EM@mcimail.com> Noel said: >Ah. There are two parts to this. The first is "how do you get the list of >the interfaces which connect to a given host identifier", and the second is >"given this list, how do I pick one"? Both of these are in the issues list. This second one is much nastier than you indicate. Issues like load leveling, fault tolerance (ah, gee that interface just died, let me try another and not break this connection), capacity (as you indicated which is a special case of load leveling), and perhaps billing come to mind. Not that NIMROD should tackle these things, but rather make it possible for others to do them. Bob Moskowitz   Received: from pizza by PIZZA.BBN.COM id aa29880; 28 Sep 93 13:28 EDT Received: from BBN.COM by PIZZA.BBN.COM id ac29868; 28 Sep 93 13:27 EDT Received: from mcigateway.mcimail.com by BBN.COM id ab22086; 28 Sep 93 13:28 EDT Received: from mcimail.com by MCIGATEWAY.MCIMail.com id al18029; 28 Sep 93 17:07 GMT Date: Tue, 28 Sep 93 17:10 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: First moves... Message-Id: <31930928171013/0003858921NA4EM@mcimail.com> Gary Malkin said: >We should definitely not use "address", even as an alias. It simply >has too much historical baggage associated with it. That is EXACTLY why we must at least equate "address" in all of its nuances to the new naming. Only in doing so will we bring the non-Nimrod educated people into the framework being created here. It may be nothing more than a topological mapping as an appendix... Bob Moskowitz   Received: from pizza by PIZZA.BBN.COM id aa00135; 28 Sep 93 13:44 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa00131; 28 Sep 93 13:42 EDT From: Lyman Chapin Subject: Re: First moves... To: avri@locus.com Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM In-Reply-To: <9309281607.AA04006@locus.com> Date: Tue, 28 Sep 93 13:22:25 EDT Mail-System-Version: >To: Noel Chiappa >Cc: nimrod-wg@BBN.COM >Subject: Re: First moves... >Date: Tue, 28 Sep 93 12:07:53 -0400 >From: avri@locus.com > >Re: the word address. > > >I don't believe that you can ever set a formal definition on a common word and >have it stick. That is because you never know, in a conversation, whether one >means address in the formal sense or in the folk sense. This is unless you >specifically make statements like "address, i mean in the formal sense of >locator". If you do this, however, you will start to sound like a >philosopher (a fate, surely, to be avoided). > >I also don't believe that you can ever ban a word from the lexicon. I do >believe that you can ban the word from the architecture document, and you can >treat any usage of the word as refering to the general class of names that >refer to entities such as locators, identifiers and selectors. I.e. you can >treat it as a meta-term instead of a architectural term. > >avri Avri, This is exactly what we had to do when we were defining the network layer architecture for OSI - we used formal terms (Network address, Network service access point address, subnetwork address, and network entity title) in the architecture specs, but didn't try to keep people from saying "host" (when they "really" meant to say "end system") or "host name" (when they "really" meant to say "end system network entity title"), or even "address" (...NSAP address) in less formal contexts. People who tried to actually speak in the formal language of the network layer naming and addressing architecture tended to sound not like philosophers but like idiots... - Lyman   Received: from pizza by PIZZA.BBN.COM id aa00191; 28 Sep 93 13:46 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa00187; 28 Sep 93 13:44 EDT Received: from quern.epilogue.com by BBN.COM id aa22965; 28 Sep 93 13:44 EDT To: oran@sneezy.lkg.dec.com CC: nimrod-wg@BBN.COM In-reply-to: David Oran's message of Tue, 28 Sep 93 12:57:22 -0400 <930928125722.11119@sneezy.lkg.dec.com.-v> Subject: Terminology (I: Big-I) Reply-to: dab=replies@epilogue.com Date: Tue, 28 Sep 93 13:43:53 EDT From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9309281343.aa07983@quern.epilogue.com> References: <9309281530.AA06911@ginger.lcs.mit.edu> X-Mailer: Poste 2.1 From: David Oran Date: Tue, 28 Sep 93 12:57:22 -0400 Encoding: 34 TEXT, 6 TEXT SIGNATURE At the risk of re-opening a very large can of creepy crawly things, I'd like to propose that "hosts" do not need identifiers for the purpose of identifying the endpoint of individual packets. Hosts need NAMES so that people and applications can refer to collections of application instances, and so that managers can have a handle on a physical "Box" or set of boes that fails as a unit. I'd take your premisses and argue to a different conclusion. Those "collections of application instances" are what need the names, not the hosts. Whether hosts get identifiers then depends on whether the routing and forwarding system finds them useful. Since I see flow IDs as a good selector and a globally unique host ID turns a host specific flow ID into a globally unique flow ID, I see host IDs as useful. I propose that somewhere in the lower layers there has to be an "endpoint identifier" which allows a very fast and efficient way to deliver data from network adapters to the address space of the ultimate consumer without creating a serial bottleneck through a single CPU or kernel instance. I like this idea. Would the flow ID be appropriate for this or am I missing something? Dave Bridgham   Received: from pizza by PIZZA.BBN.COM id aa29878; 28 Sep 93 13:28 EDT Received: from BBN.COM by PIZZA.BBN.COM id ab29868; 28 Sep 93 13:27 EDT Received: from mcigateway.mcimail.com by BBN.COM id aa22086; 28 Sep 93 13:28 EDT Received: from mcimail.com by MCIGATEWAY.MCIMail.com id aj18029; 28 Sep 93 17:07 GMT Date: Tue, 28 Sep 93 17:10 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: Terminology (I: Big-I) Message-Id: <30930928171003/0003858921NA4EM@mcimail.com> Vint said: >I think we need a way to identify a host without binding >that identifier to a specific interface. This is the way the 'rest of the world' works. I can go into my front door or my back door. It is still my house. If I am bringing the groceries in, I'll perfer to use the back door, right into the kitchen. If I'm bringing in a guest, I'll use the front door so they don't see the kitchen (or is the living room worst :). We are now doing a major implementation of MVS/TCP. We have just completed our first major capacity test of 5,000 connections which is the limit of the software today, but not enough for our near-terms needs. What we are having to do to allow for multiple interfaces and multiple MVS/TCP address spaces to create a 'industrial strength' environment is a joke. Major changes are needed here as we all know... Bob Moskowitz   Received: from pizza by PIZZA.BBN.COM id aa01457; 28 Sep 93 16:23 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa01453; 28 Sep 93 16:21 EDT Received: from inet-gw-1.pa.dec.com by BBN.COM id aa03328; 28 Sep 93 16:23 EDT Received: by inet-gw-1.pa.dec.com; id AA25341; Tue, 28 Sep 93 13:23:03 -0700 Received: by sneezy.lkg.dec.com (5.65/DEC-Ultrix/4.3) id AA15232; Tue, 28 Sep 1993 16:23:01 -0400 To: dab=replies@epilogue.com Subject: Re: Terminology (I: Big-I) In-Reply-To: <9309281343.aa07983@quern.epilogue.com> References: <9309281343.aa07983@quern.epilogue.com> Cc: nimrod-wg@BBN.COM X-Mailer: Poste 2.1 From: David Oran Date: Tue, 28 Sep 93 16:23:00 -0400 Message-Id: <930928162300.11119@sneezy.lkg.dec.com.-v> >> > I propose that somewhere in the lower layers there has to be an > "endpoint identifier" which allows a very fast and efficient way to > deliver data from network adapters to the address space of the > ultimate consumer without creating a serial bottleneck through a > single CPU or kernel instance. > > I like this idea. Would the flow ID be appropriate for this or am I > missing something? > It depends on whether flows get aggregated and whether the integrity of a flow identifier is preserved end-to-end. End-to-end significance of a flow identifier is not necessary just to accomplish the main goal of flows which is to isolate traffic that is not supposed to interact with other traffic. If the flow architecture preserves flow-ids end to end, and the number of flows is expected to be able scale to be ~number of communicating address spaces, then I'd say one identifier could serve both purposes. Dave.   Received: from pizza by PIZZA.BBN.COM id aa01572; 28 Sep 93 16:44 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa01568; 28 Sep 93 16:42 EDT Received: from pluto.dss.com by BBN.COM id aa04430; 28 Sep 93 16:42 EDT Received: by pluto.dss.com (4.1/SMI-4.0) id AA18505; Tue, 28 Sep 93 16:38:57 EDT Date: Tue, 28 Sep 93 16:38:57 EDT From: Mike Davis Message-Id: <9309282038.AA18505@pluto.dss.com> To: avri@locus.com Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM In-Reply-To: avri@locus.com's message of Tue, 28 Sep 93 12:07:53 -0400 <9309281607.AA04006@locus.com> Subject: First moves... I don't want to sound like a lawyer, idiot, or philosopher (actually that would not be so bad), but, again. I think we should avoid the A-word. Avri is right that we cannot set a formal definition and make it stick, but we can define our terms and maybe eliminate some of the "When you say 'address' do you mean X or Y or what?" We will have to differentiate the various uses of address somehow, anyway, so having a convention from the start would be useful. -- Mike Davis mike@dss.com + Ceci n'est pas une .sig Datability Communications Research Center + 261 West Central Street, Natick, MA, 02160 +   Received: from pizza by PIZZA.BBN.COM id aa02007; 28 Sep 93 17:48 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa02003; 28 Sep 93 17:45 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa07332; 28 Sep 93 17:45 EDT Received: by ginger.lcs.mit.edu id AA10932; Tue, 28 Sep 93 17:45:09 -0400 Date: Tue, 28 Sep 93 17:45:09 -0400 From: Noel Chiappa Message-Id: <9309282145.AA10932@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: The "A-word" Cc: jnc@ginger.lcs.mit.edu From: mike@pluto.dss.com (Mike Davis) As I believe in using precise terms where they exist Good point. We have precisely defined terms for all the various functions and meanings different people use "address" for, so we will just have to bite the bullet and use them. "locator" may not be *quite* as euphonious as "address", but it's not exactly totally clunky. From: "Robert G. Moskowitz" <0003858921@mcimail.com> I think that the use of the work 'address' should exactly correspond with its common usage. Well, for what it's worth, I personally agree with you. However, that and 25c will get us a cup of coffee! The problem seems to be that "address" has come to mean *three* different things in computer networking, based on its usage, first in the ARPANet 1822/HHP, and later in TCP/IP. It means i) "locator", since it usualy tells you where the thing is, ii) "selector" since it's the field in the packet the routers look at, and iii) "host identifier" since TCP and other end-end protocol use it for that. That is EXACTLY why we must at least equate "address" in all of its nuances to the new naming. Only in doing so will we bring the non-Nimrod educated people into the framework being created here. I assume by this you mean the kind of thing above, where it is pointed out that "address" in IPv4 and elsewhere means all three of those distinct concepts? Or did I miss your point... From: Gary Scott Malkin We should definitely not use "address"... It simply has too much historical baggage associated with it. Sigh, sounds like we are gonna have to do this. Too many people, when they hear the term, assume it's going to be in all the packets too (meaning ii), etc. From: avri@locus.com I do believe that you can ban the word from the architecture document Hmm, we may have to, and not just that document, all Nimrod documents... and you can treat any usage of the word as refering to the general class of names that refer to entities such as locators, identifiers and selectors. Can't imagine what use there is for such a broad term; I think we'll have to obsolete it along with IPv4, which broke it! From: Lyman Chapin we used formal terms ... in the architecture specs, but didn't try to keep people from saying ... "address" ... in less formal contexts I'd prefer that it be used informally as a synonym for "locator", and if I use it informaly, that's what I mean by it. It is clearly closer to that individual meaning than either or the other two (selector or host-identifier), and as Bob points out, that's also the real world meaning. Noel   Received: from pizza by PIZZA.BBN.COM id aa02215; 28 Sep 93 18:24 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa02211; 28 Sep 93 18:23 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa08732; 28 Sep 93 18:24 EDT Received: by ginger.lcs.mit.edu id AA11275; Tue, 28 Sep 93 18:24:05 -0400 Date: Tue, 28 Sep 93 18:24:05 -0400 From: Noel Chiappa Message-Id: <9309282224.AA11275@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Endpoints Cc: jnc@ginger.lcs.mit.edu From: David Oran At the risk of re-opening a very large can of creepy crawly things, I'd like to propose that "hosts" do not need identifiers for the purpose of identifying the endpoint of individual packets. Hosts need NAMES ... So, why have I kept arguing that packets need endpoint identifiers in them somewhere? ... I propose that somewhere in the lower layers there has to be an "endpoint identifier"... Err, I think I already opened just this can! (Also, I agree completely.) From my reply to Vint: [I] have campaigned vigorously for just such a separate namespace. However, on a suggestion from Dave Oran, I think it ought to name something a little more abstract than a host, i.e. the endpoint... My personal view is that the protocol architecture ought to have separate namespaces for interfaces and endpoint; Nimrod proposes that we have locators for the former, and that's about as far as Nimrod the routing architeture. Nimrod the deployment plan calls for reinterpreting the 32 bit source and dest fields in IPv4 packets as endpoint identifiers. It depends on whether flows get aggregated Another open point which is already in the list. From: dab@epilogue.com I'd take your premisses and argue to a different conclusion. Those "collections of application instances" are what need the names, not the hosts. You're speaking of something which is more a resource naming and location issue, right? This is assuming you mean "string names" when you say "name", not the more formal computer science meaning of "label". If not, and you are referring to the EID (endpoint identifier) 'name', we went round on this for a while on Big-I. My problem with naming collective entities is that the semantics are much more complex. One of the key parts of the definition of "endpoint" is that it is a fate-sharing region (i.e. it's all either live or dead), and it is this way for a reason. That's the fundamental building block. Collections of endpoints can exhibit more complex behaviour; e.g. one can fail in a way which is visible to the clients. (If you have a collection of things which *always* acts like a single thing, then yeah, maybe it is an endpoint.) We may have a use for an aggregate entity, but this is still very new territory, and I'd be uncomfortable putting something that unfamiliar in as a basic object, as opposed to building it on as an abstraction one layer (or half-layer) up. Since I see flow IDs as a good selector and a globally unique host ID turns a host specific flow ID into a globally unique flow ID, I see host IDs as useful. The statement "Since ... a globally unique endpoint ID turns an endpoint specific flow ID into a globally unique flow ID, I see endpoint IDs as useful" is equally true, no? Can we not debate this particular point right now, please, lest we go into melt-down here? (Only one debate at a time, please, gentlebeings! :-) I've added both of these to the "open points" list (should Nimrod have separate namespaces for interfaces and endpoints, and are flow-ID's globally unique or not, and how are they constructed), and they'll get dragged out and fought over in due course. Noel   Received: from pizza by PIZZA.BBN.COM id aa04246; 29 Sep 93 8:15 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa04242; 29 Sep 93 8:12 EDT Received: from bells.cs.ucl.ac.uk by BBN.COM id aa16146; 29 Sep 93 7:55 EDT Received: from sartre.cs.ucl.ac.uk by bells.cs.ucl.ac.uk with local SMTP id ; Wed, 29 Sep 1993 08:56:48 +0100 To: David Oran cc: dab=replies@epilogue.com, nimrod-wg@BBN.COM Subject: Re: Terminology (I: Big-I) In-reply-to: Your message of "Tue, 28 Sep 93 16:23:00 EDT." <930928162300.11119@sneezy.lkg.dec.com.-v> Date: Wed, 29 Sep 93 08:56:45 +0100 From: Jon Crowcroft >End-to-end significance of a flow identifier is not necessary just to >accomplish the main goal of flows which is to isolate traffic that is not >supposed to interact with other traffic. depends if you want to use them as a rendezvous point for multiple flows, like class D addresses could be used as (effectively) flow id's for audio conferences... c.f. Another Terrible Mistake... jon   Received: from pizza by PIZZA.BBN.COM id aa04259; 29 Sep 93 8:17 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa04255; 29 Sep 93 8:15 EDT Received: from world.std.com by BBN.COM id aa16615; 29 Sep 93 8:02 EDT Received: by world.std.com (5.65c/Spike-2.0) id AA04377; Wed, 29 Sep 1993 08:01:40 -0400 From: Howard C Berkowitz Message-Id: <199309291201.AA04377@world.std.com> Subject: Nimrod usability/pity the user To: nimrod-wg@BBN.COM Date: Wed, 29 Sep 1993 08:01:39 -0400 (EDT) Cc: hberkowi@world.std.com X-Mailer: ELM [version 2.4 PL21] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 3454 We argue a lot about how and why to do things, as evidenced by the discussion about the A-word. There's been a valuable suggestion to keep track of the motivations for our decisions. We are also people who have a great deal of theoretical and practical knowledge about routing. We need to remember that our knowledge level is not typical of many commercial network administrators, who will have to use Nimrod implementations. Oone of my main interests is the operational usability of products and services. I would like to see, probably in an informational which need to be administered by the grunt network administrator, and which should be more restricted to the view of "core" network operators and product implementors. Such guidance certainly is not intended to stifle implementor creativity, but help point to areas where configuration tools will be a practical necessity for Nimrod to be usable. These tools can be algorithms built into products, expert systems, etc., but, outside of prototypes they can't be even more complex configuration files or extremely long sequences of menu selections. My work is divided between network design consulting at one end, and managing/delivering router product training at the other. One is the agony and the other the ecstasy, but the roles change. In any event, the training part lets me see the understanding level of typical network administrators who eventually will connect their networks to Nimrod- based architectures. They are having serious troubles with the complexity of existing protocols and [the word that has to do with using A-words :-)]. They have trouble grasping non-octet-aligned FIXED subnet masks, autonomous systems, and default routes. Remember that we are designing for a population that goes through the sequence (well supplemented with graphics), in a router product course: [Principle of OSPF lecture, Day D-1] "OSPF routing configurations (note that I did _not_ say topological data bases) are different that those previously discussed, because they contain information only about your local area and possibly the backbone (we are avoiding talking about virtual links). The typical router knows only about one non-zero area." [OSPF Configuration lecture, D-Day, H-1 hour] "When you do your configurations, remember that you don't want more than two areas to be in any router. If there are two, one should be the backbone." [-30 minutes] "Remember when I said I'd stamp my foot when something was important? *stamp, stamp* If you have any areas in your configuration other than yours and the backbone, you've done something wrong." [-10 minutes] "Let me review using _Lost in Space_ principles: Danger! Danger, Will Robinson! If you have more than one nonzero area in the same configuration, you will screw up the lab! Danger! Danger!" [-5 minute] "This is important! Only one router per area (we have two) should be configured to know about the backbone; all others just know about their own area." [just before beginning lab] "Remember, the most common error in this lab is putting the other side of the room's area in your configuration file. If you are in Area 21, you should NOT mention Area 26, and vice versa. One router per area will mention Area 0." I have never taught this lab without someone proceeding to configure two nonzero areas in the same router. ------- Howard   Received: from pizza by PIZZA.BBN.COM id aa04452; 29 Sep 93 8:43 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa04448; 29 Sep 93 8:41 EDT Received: from world.std.com by BBN.COM id aa17468; 29 Sep 93 8:13 EDT Received: by world.std.com (5.65c/Spike-2.0) id AA06067; Wed, 29 Sep 1993 08:12:36 -0400 From: Howard C Berkowitz Message-Id: <199309291212.AA06067@world.std.com> Subject: Re: Terminology (I: Big-I) To: nimrod-wg@BBN.COM Date: Wed, 29 Sep 1993 08:12:35 -0400 (EDT) Cc: Howard C Berkowitz In-Reply-To: <15930928170951/0003858921NA4EM@mcimail.com> from "Robert G. Moskowitz" at Sep 28, 93 05:09:00 pm X-Mailer: ELM [version 2.4 PL21] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1808 > > Noel said: > > >Ah. There are two parts to this. The first is "how do you get the list of > >the interfaces which connect to a given host identifier", and the second is > >"given this list, how do I pick one"? Both of these are in the issues list. > > This second one is much nastier than you indicate. > > Issues like load leveling, fault tolerance (ah, gee that interface just > died, let me try another and not break this connection), capacity (as you > indicated which is a special case of load leveling), and perhaps billing > come to mind. > > Not that NIMROD should tackle these things, but rather make it possible for > others to do them. > > Bob Moskowitz > > I'm not sure whether this fits into this discussion or the terminology thread, but there is a practical need to have some construct that identifies a given "physical interface thing." By that, I mean one step below the "interface in a list" Noel describes, which, since it is mapped to a host name/identifier, is probably a network layer locator. What I refer to is an identifier or locator, in a router, that describes the physical port traffic comes in or goes out on. MAC address doesn't do this, because ports also go to serial lines. They may very well have implementation-dependent identifier values (e.g., "serial 1"), but they do need a place in the structure. ---- Howard ---- PS--going back to a past offline discussion with Noel, I returned to my old chemist days to solve :-) this whole terminology question. We seem to agree that the hierarchical pieces of an A-thing can generically be called elements. It is therefore obvious that we are talking about network molecules. We can talk about empirical formulas to describe general types of routing, and structural formulas to describe syntax. :-)   Received: from pizza by PIZZA.BBN.COM id aa04735; 29 Sep 93 9:19 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa04731; 29 Sep 93 9:17 EDT Received: from babyoil.ftp.com by BBN.COM id aa21599; 29 Sep 93 9:13 EDT Received: by ftp.com id AA11934; Wed, 29 Sep 93 09:13:11 -0400 Date: Wed, 29 Sep 93 09:13:11 -0400 Message-Id: <9309291313.AA11934@ftp.com> To: deering@parc.xerox.com, jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM Subject: Mcast, Mobility, etc, (wasRe: Draft charter for WG) From: Frank Kastenholz Reply-To: kasten@ftp.com > > An effort to take into account support for mobility, multicast and > > resource allocation will be made when designing the Nimrod architecture, > > provided it is not burdensome. Nimrod itself is not an attempt to solve > > these problems, although it may include mechanisms to facilitate > > providing these capabilities. > > It seems unlikely to me that a new routing and addr*ssing architecture > could punt on those issues and still be relevent. I can see how mobility > might be left as a "bag on the side" (as the IP mobility work has shown), > but multicast routing and resource-sensitive routing seem fundamental > to me, not things that can be tacked on as an afterthought. > > Steve, the draft meant what it said; we will try hard to do these things, > because we recognize that they *are* important. However, there are components > of the problem (particular in the resource allocation area) which are outside > the scope of Nimrod, and to tie Nimrod too closely to that might be to drag it > down. Noel, I sort of agree with Steve here. We must investigate these issues. We can then decide to either A) punt them (i.e. not Nimrod's job) or B) integrate them into Nimrod. If we decide to punt them then we must have a good, documented reason (see Scott's earlier post) for punting them and then we must leave the right hooks (whatever that means) for doing these things, _and_ be able to show that we can do multicast and mobility as well-designed add-ons. We can not simply decide that these are not our job and then ignore them. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa04739; 29 Sep 93 9:19 EDT Received: from BBN.COM by PIZZA.BBN.COM id ab04731; 29 Sep 93 9:17 EDT Received: from babyoil.ftp.com by BBN.COM id aa21597; 29 Sep 93 9:13 EDT Received: by ftp.com id AA11920; Wed, 29 Sep 93 09:13:01 -0400 Date: Wed, 29 Sep 93 09:13:01 -0400 Message-Id: <9309291313.AA11920@ftp.com> To: jnc@ginger.lcs.mit.edu Subject: Re: Terminology (I: Big-I) From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM Noel, A couple of nits.... First, with respect to your definitions of locators -- I assume that whenever you use the term "host" you also mean "router"? > "interface locator" - The structured name of an interface, i.e. a host's > attachment point to the internetwork. The structure is used by the > routing to make its job a little easier. Things which have related > locators must usually be topologically related, or else the routing > will break down. The destination locator might or might not appear > in all packets (see "selector"). > "locator" - Usually alternative, short, term for the first term above. > "locator" used on its own often means an "interface locator", but it > can also mean the structured name of an aggregate, be it a single net, > a group of nets, or whatever. This is usually formed by dropping the > least significant elements from a locator. This is the most general form -- all "interface locators" are "locators", but not all "locators" are "interface locators". One of the problems that we have had with the A-Word is that it means different things at different times to different people depending on the context and the phase of the moon. I would strongly urge that when we mean a -locator, we ALWAYS use the full term -- -locator. We should use the term "locator" by itself only when we explicitly intend to use the general term -- i.e. when it does not matter whether we are talking about a host-locator, an interface-locator, etc.... As long as we have specific forms of locators, should we define a specific, "Aggregate-Locator": "Aggregate locator" - The structured name of an Aggregate. Aggregates which have related Aggregate Locators must be topologically related.... > "host locator" - Some people believe that hosts, as well as interfaces, need > names which indicate where they are. This is otherwise identical to an > "interface locator", but names a host, not an interface. Again, in an > architecture which has them, the destination locator might or might > not appear in all packets (see "selector"). If a host has multiple interfaces, connected to multiple, different, aggregates then one might say that the host appears at multiple, different, topological positions on the network. Given that Locators are based on topological location on the network, what would that host's host-locator be? >"host identifier" - A name (not *necessarily* an ASCII string; i.e. not a > DNS name), probably globally unique, and not a locator, for a host. > The current IPv4 'address' has this as one of its current functions > (e.g. in the TCP pseudo-header). Much the same as an EID, for those who > understand that term. I presume that the host-identifier is explicitly non-structured, at least as far as routing and network-topology are concerned. Any structure would probably be administrative in nature, to aid in delegation of assignment authority. Anyway, the important part is that the network-layer sees this as a simple, flat, space. What differences are there between host-identifiers and EIDs? -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa04743; 29 Sep 93 9:19 EDT Received: from BBN.COM by PIZZA.BBN.COM id ac04731; 29 Sep 93 9:17 EDT Received: from babyoil.ftp.com by BBN.COM id aa21602; 29 Sep 93 9:13 EDT Received: by ftp.com id AA11928; Wed, 29 Sep 93 09:13:06 -0400 Date: Wed, 29 Sep 93 09:13:06 -0400 Message-Id: <9309291313.AA11928@ftp.com> To: nimrod-wg@BBN.COM, jnc@ginger.lcs.mit.edu Subject: To Address or Not to Address, that is the question. From: Frank Kastenholz Reply-To: kasten@ftp.com It seems that one of Nimrod's goals is to develop fundamentally new (at least with respect to the current IP/etc thinking) methods of doing network layer stuff. Since we are trying to do things that are fundamentally new, we ought not use the "old" term since A) using the "old" word could lead to unconscious attempts to equate the new concepts with the old ones and B) as we throw the new terms around, a newcomer, not seeing the old, familiar, comfortable, terms will realize that we are doing something new and different here and won't be tempted to assume that he/she knows what's going on and will ask "what are you talking about". I think that this should address the issue sufficiently :-) -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa04749; 29 Sep 93 9:19 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa04745; 29 Sep 93 9:17 EDT Received: from babyoil.ftp.com by BBN.COM id aa21614; 29 Sep 93 9:13 EDT Received: by ftp.com id AA11943; Wed, 29 Sep 93 09:13:16 -0400 Date: Wed, 29 Sep 93 09:13:16 -0400 Message-Id: <9309291313.AA11943@ftp.com> To: oran@sneezy.lkg.dec.com Subject: Re: Terminology (I: Big-I) From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: jnc@ginger.lcs.mit.edu, vcerf@cnri.reston.va.us, nimrod-wg@BBN.COM Dave, Perhaps I am being extraordinarily dense here, but this note seems to me to say that we do not need these identifiers, and then goes on to say that we do need these identifiers.... Could you please elaborate? > At the risk of re-opening a very large can of creepy crawly things, I'd > like to propose that "hosts" do not need identifiers for the purpose of > identifying the endpoint of individual packets. Hosts need NAMES so > that people and applications can refer to collections of application instances, > and so that managers can have a handle on a physical "Box" or set of boes > that fails as a unit. > > So, why have I kept arguing that packets need endpoint identifiers in them > somewhere? Because you need a very efficient way to: > a) tell whether a packet has arrived at the correct destination > b) deliver the packet to the right "application" at that > destination. > > I propose that somewhere in the lower layers there has to be an "endpoint > identifier" which allows a very fast and efficient way to deliver data > from network adapters to the address space of the ultimate consumer > without creating a serial bottleneck through a single CPU or kernel > instance. > > Some data is already available to support this view [c.f. the paper on > user protocol stack implementation by the U. Washington folks] who used the > link-level packet identifier scheme to put packets straight into the right > address space. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa04871; 29 Sep 93 9:28 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa04867; 29 Sep 93 9:27 EDT Received: from babyoil.ftp.com by BBN.COM id aa22303; 29 Sep 93 9:21 EDT Received: by ftp.com id AA12266; Wed, 29 Sep 93 09:21:36 -0400 Date: Wed, 29 Sep 93 09:21:36 -0400 Message-Id: <9309291321.AA12266@ftp.com> To: hcb@world.std.com Subject: Re: Nimrod usability/pity the user From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM, hberkowi@world.std.com Howard, One of the sort-of-widely-but-perhaps-not-universally (or not, maybe kind of sort of) goals of the next generation of IP is easier configuration on the part of the end users. The Holy Grail in this regard is complete auto-configuration on the part of routers and hosts (you take it out of the box, plug it in, turn it on and you have complete network connectivity). I would expect that Nimrod will attempt to address these issues. I am sure that one of the points that it, along with all other contenders for IPtng, will be evaluated on by the market will be ease of use/ease of configuration. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa05941; 29 Sep 93 11:58 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa05937; 29 Sep 93 11:55 EDT Received: from OPAL.ACC.COM by BBN.COM id aa01841; 29 Sep 93 11:55 EDT Received: by opal.acc.com (4.1/SMI-4.0) id AA28612; Wed, 29 Sep 93 08:55:25 PDT Date: Wed, 29 Sep 93 08:55:25 PDT From: Art Berggreen Message-Id: <9309291555.AA28612@opal.acc.com> To: nimrod-wg@BBN.COM Subject: Re: Terminology (I: Big-I) > >Perhaps I am being extraordinarily dense here, but this note seems to me >to say that we do not need these identifiers, and then goes on to say >that we do need these identifiers.... Could you please elaborate? The way I interpreted Dave's remarks was that we do not really communicate with "hosts", but rather with some service entity via the "endpoint(s)" of some communication link. We have all gotten into the habit of equating the "host" with the service which happens to reside at that location, but for the purposes of this group, we need to be very specific in semantics. Art   Received: from pizza by PIZZA.BBN.COM id aa05949; 29 Sep 93 11:59 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa05945; 29 Sep 93 11:58 EDT Received: from world.std.com by BBN.COM id aa01957; 29 Sep 93 11:58 EDT Received: by world.std.com (5.65c/Spike-2.0) id AA05832; Wed, 29 Sep 1993 11:56:22 -0400 From: Howard C Berkowitz Message-Id: <199309291556.AA05832@world.std.com> Subject: Re: Nimrod usability/pity the user To: kasten@ftp.com Date: Wed, 29 Sep 1993 11:56:21 -0400 (EDT) Cc: nimrod-wg@BBN.COM, hberkowi@world.std.com In-Reply-To: <9309291321.AA12266@ftp.com> from "Frank Kastenholz" at Sep 29, 93 09:21:36 am X-Mailer: ELM [version 2.4 PL21] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2179 > > Howard, > > One of the sort-of-widely-but-perhaps-not-universally (or not, maybe > kind of sort of) goals of the next generation of IP is easier > configuration on the part of the end users. The Holy Grail in this > regard is complete auto-configuration on the part of routers and hosts > (you take it out of the box, plug it in, turn it on and you have > complete network connectivity). > > I would expect that Nimrod will attempt to address these issues. I am > sure that one of the points that it, along with all other contenders for > IPtng, will be evaluated on by the market will be ease of use/ease of > configuration. > I agree that autoconfiguration is a goal, and there are certainly strides in doing so (e.g., ES-IS) once information at a higher hierarchical level is known. In today's terms, that higher level involves subnet number, network number, autonomous system number, and posssibly area. In the very near term, it includes positioning in the CIDR structure. NSAPs, of course, provide an additional level of complexity/power. I think you helped me clarify my concern as more associated with administrator-level network planning than with assignment of individual interface addresses. Among the most frequent classes I get is "OK, now I know how to set up my address and mask. How do I decide how many subnet bits to use? Should I have small subnets and use secondary addresses to have multiple subnets per cable?" I don't see the current usage of autoconfiguration including that planning level. If the abstractions used in the routing architecture are even more complex than they are now, which seems likely, they are even more likely to be beyond the capability of many network administrators. I am prototyping a tool for setting up a current-technology IP addressing plan. As I define rules, I realize that many of them are not understandable by the typical user. It's a humbling and challenging experience. I would like to see Nimrod include requirements for both "autoconfiguration" and "autoplanning." The latter might very well be realized as a knowledge base for an expert system, not tied to any specific implementation.   Received: from pizza by PIZZA.BBN.COM id aa06019; 29 Sep 93 12:06 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa06015; 29 Sep 93 12:05 EDT Received: from world.std.com by BBN.COM id aa02324; 29 Sep 93 12:06 EDT Received: by world.std.com (5.65c/Spike-2.0) id AA07853; Wed, 29 Sep 1993 12:03:19 -0400 From: Howard C Berkowitz Message-Id: <199309291603.AA07853@world.std.com> Subject: Re: Terminology (I: Big-I) To: kasten@ftp.com Date: Wed, 29 Sep 1993 12:03:19 -0400 (EDT) Cc: oran@sneezy.lkg.dec.com, jnc@ginger.lcs.mit.edu, vcerf@cnri.reston.va.us, nimrod-wg@BBN.COM In-Reply-To: <9309291313.AA11943@ftp.com> from "Frank Kastenholz" at Sep 29, 93 09:13:16 am X-Mailer: ELM [version 2.4 PL21] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1053 I Dave Crocker wrote (having lost the original post), > > I propose that somewhere in the lower layers there has to be an "endpoint > > identifier" which allows a very fast and efficient way to deliver data > > from network adapters to the address space of the ultimate consumer > > without creating a serial bottleneck through a single CPU or kernel > > instance. > > > > Some data is already available to support this view [c.f. the paper on > > user protocol stack implementation by the U. Washington folks] who used the > > link-level packet identifier scheme to put packets straight into the right > > address space. A while back on B-I, I suggested this "end identifier" might very well map to the POSIX.4 model of threads, or perhaps to an "acceptor of threads." I recognize that not everyone agrees that threads are the optimal model for operating systems, but it could be a useful test of conceptual end identifiers to see if the thread identifier scheme could be accommodated. -- Howard Berkowitz PSC International (703)998-5819   Received: from pizza by PIZZA.BBN.COM id aa06167; 29 Sep 93 12:26 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa06163; 29 Sep 93 12:25 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03447; 29 Sep 93 12:26 EDT Received: by ginger.lcs.mit.edu id AA16981; Wed, 29 Sep 93 12:25:58 -0400 Date: Wed, 29 Sep 93 12:25:58 -0400 From: Noel Chiappa Message-Id: <9309291625.AA16981@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Meltdown.... Cc: jnc@ginger.lcs.mit.edu Errr, everyone, I'm pleased there's so much energy out there, but I'm worried we've gone into what I think of "new-topic-mailing-list-mode", i.e. tons of messages on 17 different threads all going at once. I fear that the explosion of traffic is i) going to freak people out, so they bail off this list (which has obviously gone berserk, and will take too many of their scarce cycles :-), ii) in the confusion, some valuable topics will be discussed, but get passed over by many, and iii) it's going to be a long haul over the next couple of months to get the basic architecture nailed down, and I'm concerned that we are going to burn all our energy here in a short, brief blast. Soooo, can we please keep it down to a dull roar, and one or two topics only? I promise, all these very critical technical points (such as easy configuration) are getting saved up, to go over in a methodical and complete way. If you like, you can send me private mail and I'll add stuff to the "to be discussed file". For the moment, can we stick to just terminology issues, and leave aside everything else? If the mailing list (and IETF, groan) subside to a dull roar, I'll clean up and send in the complete "open issues list", so you can all tell me what I've left off. :-) Noel   Received: from pizza by PIZZA.BBN.COM id aa06311; 29 Sep 93 12:53 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa06302; 29 Sep 93 12:52 EDT Received: from babyoil.ftp.com by BBN.COM id aa04634; 29 Sep 93 12:52 EDT Received: by ftp.com id AA22098; Wed, 29 Sep 93 12:52:34 -0400 Date: Wed, 29 Sep 93 12:52:34 -0400 Message-Id: <9309291652.AA22098@ftp.com> To: hcb@world.std.com Subject: Re: Terminology (I: Big-I) From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: oran@sneezy.lkg.dec.com, jnc@ginger.lcs.mit.edu, vcerf@cnri.reston.va.us, nimrod-wg@BBN.COM > A while back on B-I, I suggested this "end identifier" might very > well map to the POSIX.4 model of threads, or perhaps to an > "acceptor of threads." > > I recognize that not everyone agrees that threads are the optimal > model for operating systems, but it could be a useful test of > conceptual end identifiers to see if the thread identifier > scheme could be accommodated. POSIX threads, in and of themselves, should not be the definer of endpoints. This is a note that I just sent privately to Dave Oran, so instead of retyping it i'll send it to the group, and you should read "POSIX thread" wherever my final response says "Address Space", it should be applicable. I said... > Perhaps I am being extraordinarily dense here, but this note seems to me > to say that we do not need these identifiers, and then goes on to say > that we do need these identifiers.... Could you please elaborate? Dave said... > I'm saying hosts don't need identifiers, but address spaces on hosts do. I responded... > Ah. I see. What you are getting to is that the piece of bent sheet > metal, PC cards, and slivers of silicon do not need identifiers, but > rather the stuff that is executing on said sheet metal, etc, does. > > I would not want to limit it to something like address spaces, nor > would I want to specifically exclude it from things like bent sheet > metal... > > Can an address space have >1 id (in which case we are not really id-ing > the address space, but rather some constituent element of the address > space)? Can >1 address spaces have the same identifier (in which case > it is sort of acting as a multi-cast or maybe well-known, any-cast, > identifier)? > > What happens when the address space which "instantiates" the end > point is itself created only as a result of receiving a packet with that > eid in it (ala inetd)? > > Also, can pure-hardware have eids? ( For example, suppose I build a > box that is a high-precision clock. It has a complete hardware implementation > of a UDP/IP stack. When it receives a UDP packet on a pre-wired port it > generates a UDP response with the current time of day. There is no program > running, no software, no address space, etc etc. This thing should be able > to be an endpoint and therefore have an endpoint id) > > What I think an endpoint is is the ultimate destination/source of a packet. > It might be a process or thread. It might be an address space, it might > not. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa07004; 29 Sep 93 15:05 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa07000; 29 Sep 93 15:02 EDT Received: from mts-gw.pa.dec.com by BBN.COM id aa15568; 29 Sep 93 15:01 EDT Received: by mts-gw.pa.dec.com; id AA28974; Wed, 29 Sep 93 12:01:04 -0700 Received: by sneezy.lkg.dec.com (5.65/DEC-Ultrix/4.3) id AA18920; Wed, 29 Sep 1993 15:01:02 -0400 To: kasten@ftp.com Subject: Re: Terminology (I: Big-I) In-Reply-To: <9309291652.AA22089@ftp.com> References: <9309291652.AA22089@ftp.com> Cc: nimrod-wg@BBN.COM X-Mailer: Poste 2.1 From: David Oran Date: Wed, 29 Sep 93 15:00:58 -0400 Message-Id: <930929150058.11119@sneezy.lkg.dec.com.-v> Encoding: 88 TEXT, 6 TEXT SIGNATURE > Ah. I see. What you are getting to is that the piece of bent sheet > metal, PC cards, and slivers of silicon do not need identifiers, but > rather the stuff that is executing on said sheet metal, etc, does. > > I would not want to limit it to something like address spaces, nor > would I want to specifically exclude it from things like bent sheet > metal... > > Can an address space have >1 id (in which case we are not really id-ing > the address space, but rather some constituent element of the address > space)? Can >1 address spaces have the same identifier (in which case > it is sort of acting as a multi-cast or maybe well-known, any-cast, > identifier)? > I'm trying to solve a practical engineering problem here rather than have an abstract idendifier for a modelling abstraction of interest only to computer scientists. In many computer systems today, the memory architecture makes it expensive to copy data or to change the mapping of physical memory to the place the data resides in (possibly multiple) virtual address spaces. In conventional machines, the three places are the network adapter memory, kernel memory, and process/user memory. Ususally the cost of accessing data in memory is much lower when the needed data is in the already-mapped address space of the ultimate consumer. The goal I'd like to achive is to be able to have data go directly from the memory on the network adapter (or directly off the serial/parallel hardware if the DMA engine is smart enough) to memory that can be accessed cheaply by the consumer. The "conventional" machine architecures on which this would be helpful are the normal RISC/CISC virtual memory architectures, where such an identifier can avoid placing data in "kernel buffers" and then either copying or remapping the data for the consuming address space. On non-conventional architecures, like NUMA machines, the win could be even bigger by bypassing any serial bottleneck and directing data striaght to the local memory of the processor which wants to consume it. All of the above arguments relate to the receive path, but can be inverted for handling the transmit path. > What happens when the address space which "instantiates" the end > point is itself created only as a result of receiving a packet with that > eid in it (ala inetd)? > Then there is no big performance win since the cost of instantiating the address space is probably much greater than the cost of data copy or remapping. IOW: EIDs have to have a useful lifetime over many packet transmissions/receptions in order to have the properties I desire. > Also, can pure-hardware have eids? ( For example, suppose I build a > box that is a high-precision clock. It has a complete hardware implementation > of a UDP/IP stack. When it receives a UDP packet on a pre-wired port it > generates a UDP response with the current time of day. There is no program > running, no software, no address space, etc etc. This thing should be able > to be an endpoint and therefore have an endpoint id) > Again, sure this is possible, but then I don't see the need for these kind of identifiers from a performance perspective. We'd have to justify them by showing how the level of indirection between Host Name and "attachement point / locator / address" - pick your favorite term - solves a subtantial problem in performance, usability, auto-configurability, robustness, etc. I can think of one simple robustness argument, and probably a few for autoconfiguration. The robustness argument is the one I mentioned earlier: if you allow rapid rebinding of the state that represents the conventional notion of "host" to what the internet thinks the host's "attachment point" is, then you will have to verify this binding on each received packet to avoid deliveing data to the wrong destination. Such verification can be expensive (and even set off false alarms if done by cryptographic means) unless there is a semi-permanent identifier which is semantically associated with the host name, but cheaper to transmit and check on reception. Whew...hope people could follow that... > What I think an endpoint is is the ultimate destination/source of a packet. > It might be a process or thread. It might be an address space, it might > not. > I don't think of threads as "destinations for data". Threads are the active things which operate on system state, which for many interesting applications has to be stored *somewhere*. I think it important to minimize the cost of placing the data where it is cheapest for the thread to operate on it. Dave. P.S. - from the above, you can figure out where I stand on the "state versus process" duality in describing systems. -+-+-+-+-+-+-+ David R. Oran Phone: + 1 508 486-7377 Digital Equipment Corporation Fax: + 1 508 486-5279 LKG 1-2/A19 Email: oran@lkg.dec.com 550 King Street Littleton, MA 01460   Received: from pizza by PIZZA.BBN.COM id aa07345; 29 Sep 93 16:04 EDT Received: from BBN.COM by PIZZA.BBN.COM id ab07338; 29 Sep 93 16:03 EDT Received: from ibm1.scri.fsu.edu by BBN.COM id aa20662; 29 Sep 93 16:04 EDT Received: by ibm1.scri.fsu.edu id AA196980 (5.65c/IDA-1.4.4 for nimrod-wg@BBN.COM); Wed, 29 Sep 1993 16:02:56 -0400 Date: Wed, 29 Sep 1993 16:02:56 -0400 From: Ken Hays Message-Id: <199309292002.AA196980@ibm1.scri.fsu.edu> To: David Oran Cc: kasten@ftp.com, nimrod-wg@BBN.COM In-Reply-To: <930929150058.11119@sneezy.lkg.dec.com.-v> Subject: Re: Terminology (I: Big-I) Reply-To: Ken Hays Dave, In a follow-up to Frank Kasten, in part, you wrote: >On non-conventional architecures, like NUMA machines, the win could >be even bigger by bypassing any serial bottleneck and directing data >striaght to the local memory of the processor which wants to consume it. What is NUMA? Just guessing, is the A for Architecture? Later, Ken --------------------------------------------------------------------------- Kenneth M. Hays, Assistant Director hays@scri.fsu.edu Supercomputer Computations Research Institute scri::hays (HEPnet/SPAN) Florida State University voice=904-644-7053 400 Dirac Science Center Library fax=904-644-0098 Tallahassee, Florida 32306-4052 ----------------------------------------------------------------------------   Received: from pizza by PIZZA.BBN.COM id aa09138; 29 Sep 93 23:56 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa09134; 29 Sep 93 23:54 EDT Received: from dreggs.cisco.com by BBN.COM id aa27112; 29 Sep 93 23:56 EDT Received: from sl-chasm-16.cisco.com by dreggs.cisco.com with TCP; Wed, 29 Sep 93 20:56:04 -0700 Date: Wed, 29 Sep 93 20:56:04 -0700 Message-Id: <9309300356.AA15410@dreggs.cisco.com> X-Sender: pfortin@dreggs.cisco.com Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: nimrod-wg@BBN.COM From: Pierre Fortin Subject: Re: the "A" word I'd like to address the addressing of the "address" issue... If we don't address the problem of addressing addresses by eliminating the address word, then we will all need to address addresses of addressable entities from a new address: one which addresses the improper addressing of our address-thought-processes (nut-house). I would like to address this address issue by addressing my vote on the address-the-address issue in the addressable direction of the final and ultimate address for such addressable issues: the Trash address! Pierre this addressable ex-end-user is through addressing this issue (hopefully with proper address) and is re-addressing his attention to the lurk-mode address... [sorry if I contorted the English language... :^] -- Pierre Fortin Office: 1.214.770.5565 direct: 770.5554 Fax: 770.5559 ciscoSystems Inc., 15851 N. Dallas Pkwy, Suite 1055, Dallas, TX 75248   Received: from pizza by PIZZA.BBN.COM id aa09728; 30 Sep 93 3:19 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa09724; 30 Sep 93 3:17 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03676; 30 Sep 93 3:18 EDT Received: by ginger.lcs.mit.edu id AA24288; Thu, 30 Sep 93 03:18:46 -0400 Date: Thu, 30 Sep 93 03:18:46 -0400 From: Noel Chiappa Message-Id: <9309300718.AA24288@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Open issues Cc: jnc@ginger.lcs.mit.edu OK, everyone, here it is (under separate cover). It is sorted out into vague areas such as Addr..., err Locators, EID's, Multicast, etc. Can you send in suggestions for topics which are missing from this list, and, *pleeez*, *no debating*, since with this many things to argue about, we could go totally bonkers! :-) I *promise*, as soon as the list is updated, we will start in on all these fascinating goodies one at a time, but I have to make sure we are efficient and crisp without using too much bandwidth, since we have a very aggressive schedule, and I don't want to overload people; it's a long road, we have to pace it. Please keep your submissions as specific as possible: instead of "how will we distribute topology/connectivity info", I would like to see things like "how far will info be distributed automatically", or "do we use a probabilistic or reliable update"? These more specific questions will be easier to bring to a close quickly in list email discussions. Also, items marked with a "*" are more or less already decided (call them Nimrod Axioms :-), and in the charter (to allow me to rule endless debates on them out of order :-), but I've included them to allow a short round of debate on them, so i) everyone understands why they are axioms, and ii) to check and make sure I haven't blown a gasket by making them axioms. :-) (Feel free to add other fundamental axioms to the list, these were just the ones I though of.) Noel   Received: from pizza by PIZZA.BBN.COM id aa09742; 30 Sep 93 3:20 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa09738; 30 Sep 93 3:18 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa03926; 30 Sep 93 3:20 EDT Received: by ginger.lcs.mit.edu id AA24295; Thu, 30 Sep 93 03:20:02 -0400 Date: Thu, 30 Sep 93 03:20:02 -0400 From: Noel Chiappa Message-Id: <9309300720.AA24295@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Open issue list Addr.. err, Locators: * - What do Nimrod addresses look like (i.e. are they fixed length (64 bits?) or variable length, with a variable number of levels)? - If variable length, what about each level, is it fixed size, or can it grow too? - Address basing (i.e. do addresses grow up, down or both) - Strict reduction (i.e. can level K area only contain K-1 objects, or K-2 .... 0 objects as well) - 5.1 - Do we keep ARP (i.e. do Nimrod addresses look like NSAP's", i.e. do they simply specify a binding region, inside which the routing has to track endpoints individually, or what)? - Do Nimrod addresses contain MAC addresses? (Beaten to death on Big-I already... :-) - Multiple addresses (i.e. do we allow these, and what uses do people see for these) EID's: * - Do we have separate namespaces for endpoints and interfaces (in other words, do we have separate locators and EID's at all, or just classical addresses). - What do endpoint identifiers look like (i.e. fixed/variable length, how long, how are they assigned, etc) - How will Nimrod map from identifiers to addresses? Will it use an extended version of the DNS (and hence, we will work with the DNS working group)? Or will it be a separate system but have an architecture like the DNS? - How will this mapping work for hosts with multiple interfaces? How will it pick one? - How will we assign endpoint identifiers to processes (that may migrate between hosts)? Flows: - Are flow ID's globally unique, or just local? - Will Nimrod aggregate flows, or keep them separate? Deployment: - Deployment plan (i.e. what holes do people see in this that I have missed). In addition to what's in the proposal, Bob Smart point out that we can do a lot with ARP in terms of when 32-bit EID's start getting moved around, so you can't route on them anymore. - New packet format (i.e. do people believe Van and think we should put this off as long as possible, and if not, why not) - Do we wrap packets (how do we add addresses to packets which need them, and are packets in flows modified or wrapped) - 5.7 Optimizations: * - Do we have a hop-by-hop mode, or just flows and source-routed packets? - Entry point optimization (i.e. of the three suggested methods for handling people with multiple service providers, which looks most promising) - 6.3 Multicasting: - Do we design the Nimrod protocols to handle multicasting explicitly? (This will affect routing generation, path setup, and routing information distribution functionality.) Or do we say that multicasting is not part of Nimrod proper? - What about multicast policy routing? Odds'n'ends: * - Do we flush the IGP/EGP split? - Mobile hosts (i.e. do we map addresses to addresses, or what, and how far back toward the source do we go with the news that the address has changed) - How do we handle partitions? - Area representation issues (i.e. is an area a node or a simpler piece of topology, and are area nodes always separated by routers) - 5.1 - Default abstraction algorithm (model an area as a pseudo node with individual links to border routers, or N^2 border router direct interconnections) - 5.4 - What does the interaction with resource allocation look like?   Received: from pizza by PIZZA.BBN.COM id aa10027; 30 Sep 93 4:52 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa10023; 30 Sep 93 4:51 EDT Received: from mitsou.inria.fr by BBN.COM id aa07817; 30 Sep 93 4:49 EDT Received: by mitsou.inria.fr (5.65c8/IDA-1.2.8) id AA07497; Thu, 30 Sep 1993 09:51:39 +0100 Message-Id: <199309300851.AA07497@mitsou.inria.fr> To: Noel Chiappa Cc: nimrod-wg@BBN.COM Subject: Open issue -- Atomicity In-Reply-To: Your message of "Thu, 30 Sep 93 03:20:02 -0400." <9309300720.AA24295@ginger.lcs.mit.edu> Date: Thu, 30 Sep 93 09:51:37 +0100 From: Christian Huitema Noel, One of the issues evidenced by the discussion between Frank K. and Dave O. is what would call "atomicity" -- what are the "atoms" in the network, are they pieces of hardware (hosts) or mere processes? This is a question which has some importance, as "process mobility" is something we see happening already (distributed OSes, knowbots) and which has enormous potential growth. Not being able to send data to the process themselves is the equivalent of "three legs routing" towards mobile, and is clearly suboptimal as it introduces an unnecessary point of failure. On the other hand, having to name short lived entities such as processes clearly has an undesirable effect on the name space. Workable compromizes abound, and have in fact already been studied in the "mobility" framework -- e.g. the possibility to fine track a process or a mobile within some restricted domains (e.g. nominal and current) and the need to three-leg-route outside of these domains. But I guess this point should be in your "open issues" list. Christian Huitema   Received: from pizza by PIZZA.BBN.COM id aa10935; 30 Sep 93 9:12 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa10931; 30 Sep 93 9:11 EDT Received: from mts-gw.pa.dec.com by BBN.COM id aa14990; 30 Sep 93 9:08 EDT Received: by mts-gw.pa.dec.com; id AA02610; Thu, 30 Sep 93 06:08:28 -0700 Received: by sneezy.lkg.dec.com (5.65/DEC-Ultrix/4.3) id AA21682; Thu, 30 Sep 1993 09:08:25 -0400 To: Ken Hays Cc: kasten@ftp.com, nimrod-wg@BBN.COM Subject: Re: Terminology (I: Big-I) In-Reply-To: <199309292002.AA196980@ibm1.scri.fsu.edu> References: <199309292002.AA196980@ibm1.scri.fsu.edu> X-Mailer: Poste 2.1 From: David Oran Date: Thu, 30 Sep 93 09:08:24 -0400 Message-Id: <930930090824.11119@sneezy.lkg.dec.com.-v> > Dave, > > In a follow-up to Frank Kasten, in part, you wrote: > > >On non-conventional architecures, like NUMA machines, the win could > >be even bigger by bypassing any serial bottleneck and directing data > >striaght to the local memory of the processor which wants to consume it. > > What is NUMA? > Just guessing, is the A for Architecture? > Non Uniform Memory Access i.e., a machine, unlike a shared memory machine, where the cost of accessing memory varies depending on which processor does the access. Examples: BBN Butterfly CM5 Batcher-Banyan Switched memories   Received: from pizza by PIZZA.BBN.COM id aa11128; 30 Sep 93 9:43 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa11120; 30 Sep 93 9:41 EDT Received: from ftp.com by BBN.COM id aa17363; 30 Sep 93 9:41 EDT Received: by ftp.com id AA19749; Thu, 30 Sep 93 09:41:20 -0400 Date: Thu, 30 Sep 93 09:41:20 -0400 Message-Id: <9309301341.AA19749@ftp.com> To: jnc@ginger.lcs.mit.edu Subject: Re: Open issue list From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > Flows: How do flows get set up? Torn down? Managed? Rerouted around failures? How do attributes get assigned to flows (i.e. how do we define the policies which a flow is to satisfy). > Odds'n'ends: What modifications to higher-layer software are absolutely required in order to make Nimrod work? What modifications are possible in order to optimize a higher-layers' use of Nimrod? The whole configuration/setup issue -- how do hosts and routers get configured? How do we do auto-configuration (how much auto- configuration is possible?) Security? -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa11133; 30 Sep 93 9:43 EDT Received: from BBN.COM by PIZZA.BBN.COM id ab11120; 30 Sep 93 9:41 EDT Received: from ftp.com by BBN.COM id aa17362; 30 Sep 93 9:41 EDT Received: by ftp.com id AA19752; Thu, 30 Sep 93 09:41:23 -0400 Date: Thu, 30 Sep 93 09:41:23 -0400 Message-Id: <9309301341.AA19752@ftp.com> To: Christian.Huitema@sophia.inria.fr Subject: Re: Open issue -- Atomicity From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM Christian, To make sure I understand what you are saying, are you asking what the things are that we can assign an EID to? >One of the issues evidenced by the discussion between Frank K. and Dave O. is >what would call "atomicity" -- what are the "atoms" in the network, are they >pieces of hardware (hosts) or mere processes? This is a question which has some >importance, as "process mobility" is something we see happening already >(distributed OSes, knowbots) and which has enormous potential growth. Not >being able to send data to the process themselves is the equivalent of "three >legs routing" towards mobile, and is clearly suboptimal as it introduces an >unnecessary point of failure. On the other hand, having to name short >lived entities such as processes clearly has an undesirable effect on the name >space. Workable compromizes abound, and have in fact already been studied in >the "mobility" framework -- e.g. the possibility to fine track a process or a >mobile within some restricted domains (e.g. nominal and current) and the need >to three-leg-route outside of these domains. But I guess this point should be >in your "open issues" list. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from pizza by PIZZA.BBN.COM id aa11707; 30 Sep 93 11:10 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa11703; 30 Sep 93 11:09 EDT Received: from mitsou.inria.fr by BBN.COM id aa22642; 30 Sep 93 11:09 EDT Received: by mitsou.inria.fr (5.65c8/IDA-1.2.8) id AA08394; Thu, 30 Sep 1993 16:11:15 +0100 Message-Id: <199309301511.AA08394@mitsou.inria.fr> To: kasten@ftp.com Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM Subject: Re: Open issue -- Atomicity In-Reply-To: Your message of "Thu, 30 Sep 93 09:41:23 -0400." <9309301341.AA19752@ftp.com> Date: Thu, 30 Sep 93 16:11:14 +0100 From: Christian Huitema => Christian, => => To make sure I understand what you are saying, are you asking what the => things are that we can assign an EID to? => Yes. Are they processes, and if yes how do we assign EIDs to these short lived beasties, or only variations of what we usually call "hosts", in which case we have to explain how we track processes efficiently. Note that the second case would open a design campaign for the "son of UDP".. Christian Huitema   Received: from pizza by PIZZA.BBN.COM id aa12653; 30 Sep 93 14:24 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa12649; 30 Sep 93 14:22 EDT Received: from alpha.Xerox.COM by BBN.COM id aa03125; 30 Sep 93 14:23 EDT Received: from skylark.parc.xerox.com ([13.2.116.7]) by alpha.xerox.com with SMTP id <11705(2)>; Thu, 30 Sep 1993 11:23:20 PDT Received: from localhost by skylark.parc.xerox.com with SMTP id <12171>; Thu, 30 Sep 1993 11:23:08 -0700 To: Noel Chiappa Cc: nimrod-wg@BBN.COM, deering@parc.xerox.com Subject: Re: Draft charter for WG In-reply-to: Your message of "Tue, 28 Sep 93 09:36:50 PDT." <9309281636.AA07468@ginger.lcs.mit.edu> Date: Thu, 30 Sep 1993 11:22:57 PDT Sender: Steve Deering From: Steve Deering Message-Id: <93Sep30.112308pdt.12171@skylark.parc.xerox.com> > It seems unlikely to me that a new routing and addr*ssing architecture > could punt on those issues and still be relevent. I can see how mobility > might be left as a "bag on the side" (as the IP mobility work has shown), > but multicast routing and resource-sensitive routing seem fundamental > to me, not things that can be tacked on as an afterthought. > > Steve, the draft meant what it said; we will try hard to do these things, Noel, I'm glad to hear that. What the draft said was that you would take those issues into account "provided it is not burdensome", which didn't sound like "trying hard" to me. I appreciate your clarification. > The general consensus was that resource allocation needed to be a separate > subsystem, with *ties* to routing, sure, but not totally integrated. > ... > The more interesting one is multicast. Half of the time I think it should > be a separate sub-layer ... on top of the basic packet layer, and > half of the time I think it makes sense to integrate it fully into the basic > packet layer. You clearly have some distinction in your mind about what it means to be "fully integrated" vs. "a separate subsystem or sublayer" that is not obvious to me, probably because I have forgotten or never knew exactly what the scope of Nimrod is (is it an architecure, a service, a protocol?). However, that's a level of detail deeper than my concern -- I just wanted to be assured that those issues are not neglected in any new routing architecture for the Internet. > The obvious concern here is the routing aspects of multicast. I think that in > small systems it *does* work to integrate multicast group support into the > routing; it's just one more destination to track, sort of. However, as the > multicast world is now discovering, scaling is a Big Problem (surprise, > surprise :-). You are insulting the "multicast world" by implying that it has not previously considered scaling to be an important issue and a difficult problem. At least this member of the multicast world has been concerned with scaling from the beginning. (Read my dissertation for evidence). > Groups with a few widely scattered members, in a very large > internet, seem to be best supported with entirely different mechanisms from > groups with many, many members. That's a research question whose answer is far from clear at this point. Steve   Received: from pizza by PIZZA.BBN.COM id aa13015; 30 Sep 93 15:22 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa13011; 30 Sep 93 15:20 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa06665; 30 Sep 93 15:17 EDT Received: by ginger.lcs.mit.edu id AA27981; Thu, 30 Sep 93 15:17:26 -0400 Date: Thu, 30 Sep 93 15:17:26 -0400 From: Noel Chiappa Message-Id: <9309301917.AA27981@ginger.lcs.mit.edu> To: deering@parc.xerox.com, jnc@ginger.lcs.mit.edu Subject: Re: Draft charter for WG Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM What the draft said was that you would take those issues into account "provided it is not burdensome", which didn't sound like "trying hard" to me. The "burdensome" language was there because people were concerned that Nimrod as a routing architecture could fail to make the progress it needed to if it made complete solution of the resource allocation, multicast etc problems a condition of success. I can undertake to wordsmith the language a little to make it plain just how interested we are in providing the support needed for solutions to these problem. How about: A substantial effort to take into account support for mobility, multicast and resource allocation will be made when designing the Nimrod architecture; provided that so doing is neither impossible because of incomplete work outside the scope of Nimrod, or cause of very substantial delays in the first iteration of the protocol design. You clearly have some distinction in your mind about what it means to be "fully integrated" vs. "a separate subsystem or sublayer" that is not obvious to me Well, I'm not sure I can exactly define it, but "I know it when I see it". :-) Here's an example: "fully integrated" support for routing around congested areas in the ARPAnet meant that the metrics used in all the route selection calculations included a congestion/delay factor; i.e. it's all twined in together in a way where the two functions cannot be approached separately. "A separate subsystem" would be an approach that separates out the topology distribution from the route selection into two *other* separate subsystems, and then uses input from the resource allocation subsystem (e.g. feedback from flow setup requests which are rejected because of insufficient resources along the first route selected, causing selection of an alternate route) into the route selection subsystem. probably because I have forgotten or never knew exactly what the scope of Nimrod is (is it an architecure, a service, a protocol?). All three, and more! It's also a deployment plan for the completed protocols. However, that's a level of detail deeper than my concern -- I just wanted to be assured that those issues are not neglected in any new routing architecture for the Internet. It's clear to both the BBN crew and I that a solution which does not handle these things is simply not going to be very useful (or interesting, or likely to be adopted, or successful in the long run) in the Internet. Since being all of these is very important to us, you can be sure we are thinking very heavily about them! You are insulting the "multicast world" by implying that it has not previously considered scaling to be an important issue and a difficult problem. Sorry, I was putting a quick gloss on my admittedly less than absolutely total comprehension of the current state of multicast. The impression I received from what I was told about it was that people had not yet come to complete agreement on how to do multicast in all its aspects, and from what I heard, it seemed like having the correct scaling properties was a large part of the reason. We'll be wanting to know a lot more when we get around to tackling multicast. Noel   Received: from pizza by PIZZA.BBN.COM id aa20325; 2 Oct 93 7:13 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa20321; 2 Oct 93 7:09 EDT Received: from shark.mel.dit.CSIRO.AU by BBN.COM id aa11523; 2 Oct 93 7:11 EDT Received: from squid.mel.dit.CSIRO.AU by shark.mel.dit.csiro.au with SMTP id AA23399 (5.65c/IDA-1.4.4/DIT-1.3 for nimrod-wg@bbn.com); Sat, 2 Oct 1993 21:11:39 +1000 Message-Id: <199310021111.AA23399@shark.mel.dit.csiro.au> To: nimrod-wg@BBN.COM Subject: Multicast routing (was Re: Draft charter for WG) In-Reply-To: Your message of "Tue, 28 Sep 1993 08:06:52 PDT." <93Sep28.080702pdt.12171@skylark.parc.xerox.com> Date: Sat, 02 Oct 1993 21:11:22 +1000 From: Bob Smart >but multicast routing and resource-sensitive routing seem fundamental >to me, not things that can be tacked on as an afterthought. The t-shirt says "IP over everything", and IP packets are already being broadcast over cable-tv wires. It seems obvious that IP packets can be just poured into the air like radio or TV broadcasting and that that might be particularly suitable for popular multicast traffic. Since this is a one-way medium it will always be used in conjunction with other media. Those who can't successfully pick up the air-waves packets will have to use their other link if it has enough bandwidth. A relatively common situation in the future is that networks will be connected to the world by a relatively low-speed 2-way link but have inward access to one or more high-speed sources of packets. These high speed packet broadcasts will not only be for multicast. Any future routing architecture needs to be able to use such shared efficient high-speed broadcast media when it has spare bandwidth and fall back to the unshared low-speed media when the broadcast media has high-priority traffic as well as when the particular destination is having physical-layer trouble receiving the broadcast traffic.   Received: from pizza by PIZZA.BBN.COM id aa13016; 7 Oct 93 15:53 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa13012; 7 Oct 93 15:50 EDT Received: from pluto.dss.com by BBN.COM id aa01202; 7 Oct 93 15:49 EDT Received: by pluto.dss.com (4.1/SMI-4.0) id AA24475; Thu, 7 Oct 93 15:46:31 EDT Date: Thu, 7 Oct 93 15:46:31 EDT From: Mike Davis Message-Id: <9310071946.AA24475@pluto.dss.com> To: nimrod-wg@BBN.COM Subject: Archives Is there an archive for this mailing list. That line is blank in the WG charter. Thanks, -- Mike Davis mike@dss.com + Ceci n'est pas une .sig Datability Communications Research Center + 261 West Central Street, Natick, MA, 02160 +   Received: from pizza by PIZZA.BBN.COM id aa14199; 7 Oct 93 20:02 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa14195; 7 Oct 93 20:01 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa11850; 7 Oct 93 20:02 EDT Received: by ginger.lcs.mit.edu id AA01333; Thu, 7 Oct 93 20:02:04 -0400 Date: Thu, 7 Oct 93 20:02:04 -0400 From: Noel Chiappa Message-Id: <9310080002.AA01333@ginger.lcs.mit.edu> To: mike@pluto.dss.com, nimrod-wg@BBN.COM Subject: Re: Archives Cc: jnc@ginger.lcs.mit.edu Is there an archive for this mailing list. Yes there is, but due to internal stuff at BBN (I don't want to know the details) it's not yet on a machine which is anon-FTP-accessible. This is supposed to happen Real Soon Now. Noel PS: Sorry for the delay in getting rolling, folks, but a friend of mine in the vintage car restoration business is going down the tubes and I'm spending my time trying to sort that out. We'll get started here in a day or so, hopefully.   Received: from pizza by PIZZA.BBN.COM id aa16431; 8 Oct 93 9:30 EDT Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa16427; 8 Oct 93 9:26 EDT To: nimrod-wg@BBN.COM Subject: archives Date: Fri, 08 Oct 93 09:28:23 -0400 From: Martha Steenstrup The archives for the nimrod working group are available via anonymous ftp from bbn.com. They are in the file entitled nimrod-wg.archive in the pub/nimrod-wg directory.   Received: from PIZZA.BBN.COM by BBN.COM id aa27382; 11 Oct 93 11:35 EDT Received: from pizza by PIZZA.BBN.COM id aa26528; 11 Oct 93 11:18 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa26524; 11 Oct 93 11:16 EDT Received: from mcigateway.mcimail.com by BBN.COM id aa26897; 11 Oct 93 11:17 EDT Received: from mcimail.com by MCIGATEWAY.MCIMail.com id bh17223; 11 Oct 93 15:05 GMT Date: Mon, 11 Oct 93 15:05 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: Noel Chiappa To: nimrod wg Subject: Re: The "A-word" Message-Id: <34931011150543/0003858921NA2EM@mcimail.com> I just got back from Holiday time off and have over 500 messages to plow through. I know that a lot of water has gone over the dam on this, but I want to respond to this one from Noel. I hope that it is the only one that I will respond to on this subject... >I'd prefer that it be used informally as a synonym for "locator", and if I >use it informaly, that's what I mean by it. It is clearly closer to that >individual meaning than either or the other two (selector or >host-identifier), and as Bob points out, that's also the real world >meaning. That will do. As that is the real world... Bob   Received: from PIZZA.BBN.COM by BBN.COM id aa06381; 11 Oct 93 17:25 EDT Received: from pizza by PIZZA.BBN.COM id aa27531; 11 Oct 93 17:10 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa27527; 11 Oct 93 17:08 EDT Received: from mcigateway.mcimail.com by BBN.COM id aa05993; 11 Oct 93 17:10 EDT Received: from mcimail.com by MCIGATEWAY.MCIMail.com id ai03629; 11 Oct 93 21:01 GMT Date: Mon, 11 Oct 93 20:58 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: Terminology (I: Big-I) Message-Id: <34931011205843/0003858921NA1EM@mcimail.com> Art Berggreen said: >The way I interpreted Dave's remarks was that we do not really communicate >with "hosts", but rather with some service entity via the "endpoint(s)" of >some communication link. We have all gotten into the habit of equating >the "host" with the service which happens to reside at that location, but >for the purposes of this group, we need to be very specific in semantics. This is the way our SNA network works. The users only know about the names we have created for their applications. They have no way, and typically, no interest in where they are running. VTAM figures that out for them dynamically and routes the BIND accordingly... Bob   Received: from PIZZA.BBN.COM by BBN.COM id aa06748; 12 Oct 93 7:45 EDT Received: from pizza by PIZZA.BBN.COM id aa29487; 12 Oct 93 7:29 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29483; 12 Oct 93 7:27 EDT Received: from mcigateway.mcimail.com by BBN.COM id aa06213; 12 Oct 93 7:28 EDT Received: from mcimail.com by MCIGATEWAY.MCIMail.com id am29090; 12 Oct 93 11:16 GMT Date: Tue, 12 Oct 93 11:10 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: Open issue -- Atomicity Message-Id: <41931012111014/0003858921NA4EM@mcimail.com> Christian Huitema said: >One of the issues evidenced by the discussion between Frank K. and Dave O. >is what would call "atomicity" -- what are the "atoms" in the network, are >they pieces of hardware (hosts) or mere processes? This is a question which >has some importance, as "process mobility" is something we see happening >already (distributed OSes, knowbots) and which has enormous potential >growth. We have had 'process mobility' in our SNA world for a few years now. An OLTP process (CICS region or IMS region) can be moved between 'hosts' for load balancing or disaster recovery sometimes within 15 minutes of the decision to move it. VTAM finds the new home of the OLTP and directs all new connections to it; old connections do get broken during such a move. So perhaps process mobility might be a grandular thing. OLTPs should be addressable, i.e. have EIDs. But a process controlled by a scheduler smart enough to find a host with available resources does not need an EID. Bob   Received: from PIZZA.BBN.COM by BBN.COM id aa16350; 12 Oct 93 10:26 EDT Received: from pizza by PIZZA.BBN.COM id aa00215; 12 Oct 93 10:10 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa00211; 12 Oct 93 10:07 EDT Received: from nutmeg-gw.NTU.EDU.AU by BBN.COM id aa14970; 12 Oct 93 10:07 EDT Received: from localhost (coleman@localhost) by nutmeg (8.3/8.3) id XAA01536; Tue, 12 Oct 1993 23:38:25 +0930 From: James Coleman Message-Id: <199310121408.XAA01536@nutmeg> Subject: Re: Open issue -- Atomicity To: nimrod-wg@BBN.COM Date: Tue, 12 Oct 1993 23:38:25 +0930 (CST) Cc: coleman@nutmeg.ntu.edu.au In-Reply-To: <41931012111014/0003858921NA4EM@mcimail.com> from "Robert G. Moskowitz" at Oct 12, 93 11:10:00 am X-Mailer: ELM [version 2.4 PL5] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1324 The purpose of a network is not to connect two hosts (machines) together. This can be done with a piece of string and comes out cheaper than wires etc. Therefore the purpose is to be able to exchange data between the machines. The question is then, what is this data? I believe that the data is of two forms: a) the contents of a file or data store of some form b) messages to control a remote operation So far, networking seems to only take the b) view, even for file operation where extracting data is a matter of sending a message to a process on the remote host and getting the host to send the data. Perhaps this has been a mistake because it has blinded us to different ways of looking at networking. Consider the following: Let us go to the opposite extreme [Plan 9} and model everything as a file. If everything was a file, and there was a super file system in the sky (fsis) (well network) then everything would be part of the file system and mobility as a concept would not exist. Of course the actual location could be mobile so long as the fsis can keep track of it. The user can only acces those files that they have access permission for. Routing A******ing [in all its conotations]then becomes a matter of getting to a particular part of the fsis. James [from the School of Way-Out Ideas]   Received: from PIZZA.BBN.COM by BBN.COM id aa08894; 18 Oct 93 10:53 EDT Received: from pizza by PIZZA.BBN.COM id aa25123; 18 Oct 93 10:33 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa25117; 18 Oct 93 10:29 EDT Received: from MITCHELL.CIT.CORNELL.EDU by BBN.COM id aa07441; 18 Oct 93 10:30 EDT Received: from by mitchell.cit.cornell.edu (4.1/1.34/Honig-1.3) id AB00199; Mon, 18 Oct 93 10:30:15 EDT Message-Id: <9310181430.AB00199@mitchell.cit.cornell.edu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Mon, 18 Oct 1993 10:29:48 -0400 To: Noel Chiappa , nimrod-wg@BBN.COM From: Scott W Brim Subject: Re: Open issue list Which open issues is the mailing list supposed to be working on at this time? It feels like the list sort of fizzled, or else Noel and BBN are off talking among themselves (or else my mail is being fed to sheep in Tasmania?). Thanks ... Scott   Received: from PIZZA.BBN.COM by BBN.COM id aa06541; 19 Oct 93 3:02 EDT Received: from pizza by PIZZA.BBN.COM id aa29042; 19 Oct 93 2:47 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29038; 19 Oct 93 2:45 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa06261; 19 Oct 93 2:46 EDT Received: by ginger.lcs.mit.edu id AA12606; Tue, 19 Oct 93 02:46:34 -0400 Date: Tue, 19 Oct 93 02:46:34 -0400 From: Noel Chiappa Message-Id: <9310190646.AA12606@ginger.lcs.mit.edu> To: jnc@ginger.lcs.mit.edu, swb@nr-tech.cit.cornell.edu Subject: Re: Open issue list Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM It feels like the list sort of fizzled, or else Noel and BBN are off talking among themselves Noel is off recovering from a two week spike impulse in his time consumption due to one of my friends just about losing his vintage car restoration business in a nasty mess which almost turned into major legal action. I hope to start getting back on track in a few days, apologies for the drop-out. (or else my mail is being fed to sheep in Tasmania?). Of course, this doesn't mean you mail *also* isn't beign fed to sheep! Noel   Received: from ABERI.BBN.COM by BBN.COM id aa06606; 26 Oct 93 17:23 EDT Received: from pizza by PIZZA.BBN.COM id aa12366; 26 Oct 93 16:01 EDT To: nimrod-wg@BBN.COM Subject: Can clusters in Nimrod intersect? Date: Tue, 26 Oct 93 16:57:30 -0400 From: Ram Ramanathan Here are some ideas on "clustering" that some of us have been throwing around in the recent past. One of the many issues that has come up is whether clusters can overlap without being nested. That is, can there be two clusters A and B such that neither A is entirely contained within B nor B is contained entirely within A, but there are entities that are in both A and B? Before we can start a meaningful discussion on the pros and cons of cluster overlap, it is important that everybody have an uniform notion of what we mean by a cluster. So I am giving below a tentative notion that we have been considering. This is by no means finalized, and will probably need to be refined further if not changed as things become better understood. 1. What is a Cluster? A CLUSTER is an aggregation of network entities or of clusters. Thus, one can form a cluster out of a set of routers, form a cluster of such clusters and so on ad infinitum. A CLUSTERING refers to one such hierarchical organization of entities into clusters. One way to form clusters is to define a relation among internetwork entities and to aggregate entities according to their relationship. Some example relations are "belonging to the same country", "belonging to the same university or corporation", and "attached to the same service provider". Another way to form clusters is to aggregate entities so as to minimize some cost function. Some example cost functions are "minimize expected forwarding table size" and "minimize the average number of neighboring clusters". Some other properties of a cluster : - All entities within a cluster must satisfy at least one relation - "topological connectivity". That is, any two entities within a cluster must be connected by at least one path that lies entirely within the cluster. - Entities form the bottom level clusters. An entity is a kind of degenerate cluster - a cluster that cannot contain other clusters. (A related question is whether it is worthwhile, and in fact meaningful to assign a "level" to a cluster. For instance, consider a cluster Z that a router R. Suppose our level numbering scheme starts from 1 upwards. Then, it is not clear whether the level of R is 1 or one less than the level of Z. Note that the two quantities may be different. Arguments can be made in favor of each of the two possibilities. - A cluster can be addressed as a whole and may be mobile. The address of a cluster or of an entity is specified in terms of all of the clusters that contain it. For example, for clusters X, Y, Z, and the universal cluster U, if X is contained in Y, Y is contained in Z, then X has the address U.Z.Y.X. Since all clusters are contained in U, we can omit the U in the address, and hence X's address becomes Z.Y.X. 2. Can clusters overlap without nesting? By definition, every cluster except the universal cluster is nested within another cluster. So every cluster actually overlaps with its ancestor clusters. However, the situation we are interested in here concerns the case when the set intersection of two clusters X and Y is not empty but neither X nor Y is a proper subset of the other, as shown in the figure below : ------------------------------------------------------ | Z | | ----------------------\ | | | X ------------------------- | | | | | Y | | | | | | | | | | | | | | | | ------------------------| | | -----------------------/ | | | ------------------------------------------------------- One advantage to allowing clusters to intersect is that we can better represent some situations. For instance if some routers are shared between two administrative domains (for example if the two administrations are doing a joint project) they may be in both AD clusters. Also an entity e in X /\ Y (X intersection Y) is addressable as Z.X.e as well as Z.Y.e and not allowing overlaps would probably "hide" such information that could be of value to routing. However, allowing cluster overlap introduces increased complexity especially in the context of policies. For instance, in the above figure, if Y had special requirements about what traffic can go through it, then, not only Y but also X has the responsibility of policing the traffic, ie, it must have checks at the line separating X only and X /\ Y. It is always possible however, to convert a clustering with intersecting clusters to one without. I can think of at least 3 ways to do it, say for the figure shown above. + Form three clusters W, X' and Y' where W = X /\ Y, X' = X minus W and Y' = Y minus W. That is form a separate cluster at the same level of the intersecting part. + Form two clusters X and Y' where X is as before and Y' is Y minus (X /\ Y). That is, give the intersecting part to one of the clusters. + Form 1 cluster Z = X \/ Y (X union Y) and force policies etc to be uniform throughout. But if we do that are we hiding some valuable information? Conventional clustering algorithms generally produce disjoint clusters and also I haven't seen any previous works on hierarchical networks consider intersecting clusters. Should we? So in summary, some of the questions we have to answer are : * Is the notion of clustering as briefly described above adequate? * What are other pros and cons of cluster intersection? * Are there situations where we simply *cannot* do without allowing clusters to intersect? * If we do allow cluster intersection, how can we solve the "fuzzy boundary" problem when it comes to imposing policies (amongst other things). Are there any other problems if clusters intersect? Perhaps this is something we all should think about and discuss in the upcoming IETF. - Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa06754; 26 Oct 93 17:27 EDT Received: from pizza by PIZZA.BBN.COM id aa08207; 26 Oct 93 17:02 EDT To: nimrod-wg@BBN.COM Subject: Can clusters in Nimrod intersect? Date: Tue, 26 Oct 93 16:57:30 -0400 From: Ram Ramanathan Here are some ideas on "clustering" that some of us have been throwing around in the recent past. One of the many issues that has come up is whether clusters can overlap without being nested. That is, can there be two clusters A and B such that neither A is entirely contained within B nor B is contained entirely within A, but there are entities that are in both A and B? Before we can start a meaningful discussion on the pros and cons of cluster overlap, it is important that everybody have an uniform notion of what we mean by a cluster. So I am giving below a tentative notion that we have been considering. This is by no means finalized, and will probably need to be refined further if not changed as things become better understood. 1. What is a Cluster? A CLUSTER is an aggregation of network entities or of clusters. Thus, one can form a cluster out of a set of routers, form a cluster of such clusters and so on ad infinitum. A CLUSTERING refers to one such hierarchical organization of entities into clusters. One way to form clusters is to define a relation among internetwork entities and to aggregate entities according to their relationship. Some example relations are "belonging to the same country", "belonging to the same university or corporation", and "attached to the same service provider". Another way to form clusters is to aggregate entities so as to minimize some cost function. Some example cost functions are "minimize expected forwarding table size" and "minimize the average number of neighboring clusters". Some other properties of a cluster : - All entities within a cluster must satisfy at least one relation - "topological connectivity". That is, any two entities within a cluster must be connected by at least one path that lies entirely within the cluster. - Entities form the bottom level clusters. An entity is a kind of degenerate cluster - a cluster that cannot contain other clusters. (A related question is whether it is worthwhile, and in fact meaningful to assign a "level" to a cluster. For instance, consider a cluster Z that a router R. Suppose our level numbering scheme starts from 1 upwards. Then, it is not clear whether the level of R is 1 or one less than the level of Z. Note that the two quantities may be different. Arguments can be made in favor of each of the two possibilities. - A cluster can be addressed as a whole and may be mobile. The address of a cluster or of an entity is specified in terms of all of the clusters that contain it. For example, for clusters X, Y, Z, and the universal cluster U, if X is contained in Y, Y is contained in Z, then X has the address U.Z.Y.X. Since all clusters are contained in U, we can omit the U in the address, and hence X's address becomes Z.Y.X. 2. Can clusters overlap without nesting? By definition, every cluster except the universal cluster is nested within another cluster. So every cluster actually overlaps with its ancestor clusters. However, the situation we are interested in here concerns the case when the set intersection of two clusters X and Y is not empty but neither X nor Y is a proper subset of the other, as shown in the figure below : ------------------------------------------------------ | Z | | ----------------------\ | | | X ------------------------- | | | | | Y | | | | | | | | | | | | | | | | ------------------------| | | -----------------------/ | | | ------------------------------------------------------- One advantage to allowing clusters to intersect is that we can better represent some situations. For instance if some routers are shared between two administrative domains (for example if the two administrations are doing a joint project) they may be in both AD clusters. Also an entity e in X /\ Y (X intersection Y) is addressable as Z.X.e as well as Z.Y.e and not allowing overlaps would probably "hide" such information that could be of value to routing. However, allowing cluster overlap introduces increased complexity especially in the context of policies. For instance, in the above figure, if Y had special requirements about what traffic can go through it, then, not only Y but also X has the responsibility of policing the traffic, ie, it must have checks at the line separating X only and X /\ Y. It is always possible however, to convert a clustering with intersecting clusters to one without. I can think of at least 3 ways to do it, say for the figure shown above. + Form three clusters W, X' and Y' where W = X /\ Y, X' = X minus W and Y' = Y minus W. That is form a separate cluster at the same level of the intersecting part. + Form two clusters X and Y' where X is as before and Y' is Y minus (X /\ Y). That is, give the intersecting part to one of the clusters. + Form 1 cluster Z = X \/ Y (X union Y) and force policies etc to be uniform throughout. But if we do that are we hiding some valuable information? Conventional clustering algorithms generally produce disjoint clusters and also I haven't seen any previous works on hierarchical networks consider intersecting clusters. Should we? So in summary, some of the questions we have to answer are : * Is the notion of clustering as briefly described above adequate? * What are other pros and cons of cluster intersection? * Are there situations where we simply *cannot* do without allowing clusters to intersect? * If we do allow cluster intersection, how can we solve the "fuzzy boundary" problem when it comes to imposing policies (amongst other things). Are there any other problems if clusters intersect? Perhaps this is something we all should think about and discuss in the upcoming IETF. - Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa28092; 27 Oct 93 10:38 EDT Received: from pizza by PIZZA.BBN.COM id aa11304; 27 Oct 93 10:20 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa11300; 27 Oct 93 10:18 EDT Received: from nsco.network.com by BBN.COM id aa27181; 27 Oct 93 10:20 EDT Received: from anubis-e1.network.com by nsco.network.com (5.61/1.34) id AA14206; Wed, 27 Oct 93 09:24:03 -0500 Received: from gramarye.network.com by anubis.network.com (4.1/SMI-4.1) id AA26370; Wed, 27 Oct 93 09:19:27 CDT Date: Wed, 27 Oct 93 09:19:27 CDT From: Joel Halpern Message-Id: <9310271419.AA26370@anubis.network.com> To: nimrod-wg@BBN.COM Subject: Re: Can clusters in Nimrod intersect? On the topic of whether there is existing work on overlapping clusters, it is worth noting that IDRP (OSI InterDomain Routing Protocol) allows overlapping confederations. The rules for when information could be aggregated had to be tuned several times to get the proper behavior for those cases. The point is that while overlapping confederations (clusters) are not pretty, ISO/ANSI thought that they were important enough to go to some trouble to put in. Thank you, Joel M. Halpern jmh@network.com Network Systems Corporation   Received: from PIZZA.BBN.COM by BBN.COM id aa07036; 27 Oct 93 13:15 EDT Received: from pizza by PIZZA.BBN.COM id aa11970; 27 Oct 93 12:59 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa11966; 27 Oct 93 12:53 EDT Received: from vela.acs.oakland.edu by BBN.COM id aa05999; 27 Oct 93 12:54 EDT Received: from via.rmm3.merit.edu by vela.acs.oakland.edu with SMTP id AA11348 (5.65c+/IDA-1.4.4); Wed, 27 Oct 1993 12:54:39 -0400 Date: Wed, 27 Oct 93 11:10:51 EDT From: William Allen Simpson Message-Id: <1594.bill.simpson@um.cc.umich.edu> To: nimrod-wg@BBN.COM Reply-To: bsimpson@morningstar.com Subject: Re: Can clusters in Nimrod intersect? > From: Ram Ramanathan > Here are some ideas on "clustering" that some of us have been throwing > around in the recent past. One of the many issues that has come up is > whether clusters can overlap without being nested. That is, can there > be two clusters A and B such that neither A is entirely contained within > B nor B is contained entirely within A, but there are entities that are > in both A and B? > This comes up in multiple venues. I have done some thought on this. My thinking is both yes and no. A "cluster" has to have some edge definition. In the network, the edge is defined at a router which provides the semi-permeable membrane. By definition, some kinds of nodes (hosts) can't talk to other nodes without going through a router. Thus, hosts are always within a cluster. The router is communicating between clusters. It therefore is part of both clusters. However, some "half-routers" could be each in a separate cluster. I do not believe that a cluster could surround a router which is also the border to another cluster. That is, a router within a bubble. It might be done, but I think that we can specifically disallow such cases in our protocol definitions, or provide a transform to another mapping. We MAY need non-euclidean geometry to describe these maps. Bill.Simpson@um.cc.umich.edu   Received: from PIZZA.BBN.COM by BBN.COM id aa11767; 27 Oct 93 14:40 EDT Received: from pizza by PIZZA.BBN.COM id aa12466; 27 Oct 93 14:21 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa12462; 27 Oct 93 14:18 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa10557; 27 Oct 93 14:19 EDT Received: by ginger.lcs.mit.edu id AA22514; Wed, 27 Oct 93 14:19:25 -0400 Date: Wed, 27 Oct 93 14:19:25 -0400 From: Noel Chiappa Message-Id: <9310271819.AA22514@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Status of things Cc: jnc@ginger.lcs.mit.edu I hope to have a few cycles here to get get things moving again, my apologies for the dropout. Here's what's up: - The WG charter has gone through one round of review with the AD, and hopefully we'll have that done soon. - An agenda should be coming out later today for the meetings at the IETF. - Before the WG meeting at the IETF I will definitely be sending out an updated "Open Issues" list, since this is one of the things we will want to discuss at the meeting. - Hopefully, later this week I'll be able to send out another set of "Nimrod specific" terminology for some of the more abstract routing concepts we'll be talking about. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa13717; 27 Oct 93 15:08 EDT Received: from pizza by PIZZA.BBN.COM id aa12586; 27 Oct 93 14:49 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa12582; 27 Oct 93 14:45 EDT Received: from quern.epilogue.com by BBN.COM id aa12348; 27 Oct 93 14:47 EDT To: nimrod-wg@BBN.COM In-reply-to: William Allen Simpson's message of Wed, 27 Oct 93 11:10:51 EDT <1594.bill.simpson@um.cc.umich.edu> Subject: Can clusters in Nimrod intersect? Reply-to: dab=replies@epilogue.com Date: Wed, 27 Oct 93 14:47:17 EDT From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9310271447.aa15564@quern.epilogue.com> Date: Wed, 27 Oct 93 11:10:51 EDT From: William Allen Simpson Reply-To: bsimpson@morningstar.com A "cluster" has to have some edge definition. In the network, the edge is defined at a router which provides the semi-permeable membrane. The router is communicating between clusters. It therefore is part of both clusters. However, some "half-routers" could be each in a separate cluster. Routers may or may not be part of two clusters. But I'll argue that we don't need or want overlapping clusters. The effect of overlapping clusters is that interfaces within the overlap have multiple locators. The reason I've heard advanced for wanting this is so someone who has two service providers doesn't have to pick one, they can be in both. This should only be an issue if we're using the locators as routes, the no-brainer routing option. I think we should move away from equating locators and routes as firmly as possible and discourage no-brainer routing from the beginning. Maybe there are other reasons for wanting overlapping clusters and we don't want to disallow the possibility. But this is the only reason I've heard so far. A router can still be in two clusters of course because it has multiple interfaces and I've only said that interfaces should have only one locator not that all interfaces on a machine should have locators from the same cluster. We MAY need non-euclidean geometry to describe these maps. Actually, one of the concerns I have with the roadmap analogy is that we're certainly not dealing with planer maps here. One thing you do when looking up a route on a roadmap is estimate the distance. Now you start looking in the area for roads that go roughly in the right direction or if it's a long trip you head for a fast road like an Interstate. Given your estimated as-the-bird-flies distance you have a maximum distance you'll go the wrong way in looking for a faster road. But in network maps we have wormholes. This makes it much harder to come up with an estimated distance to your destination. Also, on a planer map the concept of "in the right direction" is pretty easy. Not so in network maps. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa17913; 27 Oct 93 15:54 EDT Received: from pizza by PIZZA.BBN.COM id aa12809; 27 Oct 93 15:28 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa12805; 27 Oct 93 15:27 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa16287; 27 Oct 93 15:27 EDT Received: by ginger.lcs.mit.edu id AA23524; Wed, 27 Oct 93 15:27:04 -0400 Date: Wed, 27 Oct 93 15:27:04 -0400 From: Noel Chiappa Message-Id: <9310271927.AA23524@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Re: Terminology (I: Big-I) Cc: jnc@ginger.lcs.mit.edu Date: Tue, 28 Sep 93 17:09 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> > The first is "how do you get the list of the interfaces which connect to > a given host identifier", and the second is "given this list, how do I > pick one"? Both of these are in the issues list. This second one is much nastier than you indicate. Issues like load leveling, fault tolerance (ah, gee that interface just died, let me try another and not break this connection), capacity (as you indicated which is a special case of load leveling), and perhaps billing come to mind. Not that NIMROD should tackle these things, but rather make it possible for others to do them. Yes and no. First, there are a lot of things which are outside the Nimrod "core" specification (such as the route selection algorithm, etc.. actually I should make a list of them all :-), but which we will still need reasonable examples of before we can deploy anything for field test. Second, there are a whole bunch of things (sigh, another list) which aren't specifically part of Nimrod, but which we need to make sure Nimrod has the right tools and hooks to allow reasonable solutions and mechanisms to be crafted, and this is one example. Sigh, this is really gonna be a giant job... :-( Date: Tue, 28 Sep 93 17:10 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> VCerf> I think we need a way to identify a host without binding that VCerf> identifier to a specific interface. This is the way the 'rest of the world' works. I can go into my front door or my back door. It is still my house. ... We are now doing a major implementation of MVS/TCP. ... What we are having to do to allow for multiple interfaces and multiple MVS/TCP address spaces to create a 'industrial strength' environment is a joke. Major changes are needed here as we all know... Well, I agree with this sentiment, and I think careful distinctions between the namespaces of computing entities, and interfaces, are a key part of getting this, but not everyone is convinced. This split is on the open issue list, and we''ll spend some time on it (not that I expect we'll convince everyone, but I want to make sure it is tackled openly). Date: Wed, 29 Sep 1993 08:12:35 -0400 (EDT) From: Howard C Berkowitz >> The first is "how do you get the list of the interfaces which connect >> to a given host identifier", and the second is "given this list, how do >> I pick one"? BobM> Issues like load leveling, fault tolerance ..., capacity ..., and BobM> perhaps billing come to mind. Not that NIMROD should tackle these BobM> things, but rather make it possible for others to do them. ... there is a practical need to have some construct that identifies a given "physical interface thing." By that, I mean one step below the "interface in a list" Noel describes, which, since it is mapped to a host name/identifier, is probably a network layer locator. I'm not sure I understand the difference between a "physical interface thing" and an "interface"; I thought they were the same thing. Also, the naming system for the latter will be a internetwork level naming system, but my assumption is it will be fully capable (at the lowest layers) of directly including local link layer naming (i.e. "addressing") information; e.g. the lowest part of an Ethernet interface's Nimrod locator (below the locator of the physical network itself) would be the 48-bit number of the interface. What I refer to is an identifier or locator, in a router, that describes the physical port traffic comes in or goes out on. MAC address doesn't do this, because ports also go to serial lines. They may very well have implementation-dependent identifier values (e.g., "serial 1"), but they do need a place in the structure. Your wording here is a bit confusing. I thought the definition in Nimrod of an "interface" ("a place to which the last hop router can send packets") was more or less what you state above. I'll add an entry for "interface" in the definitions list to make this explicit. Also, I didn't quite understand the reference to MAC addresses; the lowest level of locators, i.e. those for interfaces to a given media, are i) relative to that locator of that media, and ii) in the namespace of that media, as described above. To the extent that you want to name (i.e. provide a locator) for the media to which the router is attached (e.g. for a specific physical Ethernet), you do so. Finally, the router's interfaces can have their own locators, and these can be used in route specifications. To make all that abstraction concrete, here are some specific cases. If you have a router with interfaces to two private X.25 nets, you can tell which one to go to based on the high-order part of the locator; the low order part will be what to put in the call-setup packet to actually get to the next hop. If you have a router with two interfaces to a single network, if the entity selecting the route cared about which one the traffic should flow through, it can say so in the route specification. If you have a router with N serial lines, there is an open question, which is can we have un-named topology elements (e.g. unnamed point-point links between routers). If so, the each serial can have a separate network locator, but need not. The interface at each end can have an interface locator, but this may not be very useful; you just fire packets down the link the thing on the other end. If the link is to a host, it's a little trickier, since how do you create a locator for the host without a locator for the media it is connected to? Hmm, one more thing for the open issue list. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa18382; 27 Oct 93 16:02 EDT Received: from pizza by PIZZA.BBN.COM id aa12902; 27 Oct 93 15:43 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa12898; 27 Oct 93 15:40 EDT Received: from noc.sura.net by BBN.COM id aa17166; 27 Oct 93 15:40 EDT Received: by noc.sura.net for nimrod-wg@BBN.COM (5.65b/(SURAnet $Revision: 1.29 $)) id AA17311; Wed, 27 Oct 93 15:40:39 -0400 Date: Wed, 27 Oct 93 15:40:39 -0400 From: Rob Coltun Message-Id: <9310271940.AA17311@noc.sura.net> To: dab=replies@epilogue.com, nimrod-wg@BBN.COM Subject: Re: Can clusters in Nimrod intersect? Interesting conversation. >Routers may or may not be part of two clusters. But I'll argue that >we don't need or want overlapping clusters. The effect of overlapping >clusters is that interfaces within the overlap have multiple locators. >The reason I've heard advanced for wanting this is so someone who has >two service providers doesn't have to pick one, they can be in both. >This should only be an issue if we're using the locators as routes, >the no-brainer routing option. I think we should move away from >equating locators and routes as firmly as possible and discourage >no-brainer routing from the beginning. I aggree that you need multiple locators. A couple of reasons that you may want overlapping clusters. One is for policy. As an example, the Navy may have a research branch that may be doing work with other research organizations. So policy may be for the research divisions to be able to communicate freely with other members of the research community. But the Navy as a whole or that building that is housing the research would also have to do more private communication. Having overlapping secure and research clusters would help this. Another reason is for resource partitioning. One company that I know of supports this in their ATM routing. For example, they may have several switches in a geographic region whose resources on some subset of the switches are shared between say MCI and a local provider. I see this as doable using overlapping clusters. Thanks, ---rob   Received: from PIZZA.BBN.COM by BBN.COM id aa20785; 27 Oct 93 16:39 EDT Received: from pizza by PIZZA.BBN.COM id aa13086; 27 Oct 93 16:19 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa13082; 27 Oct 93 16:18 EDT Received: from quern.epilogue.com by BBN.COM id aa19693; 27 Oct 93 16:19 EDT To: nimrod-wg@BBN.COM In-reply-to: Rob Coltun's message of Wed, 27 Oct 93 15:40:39 -0400 <9310271940.AA17311@noc.sura.net> Subject: Can clusters in Nimrod intersect? Reply-to: dab=replies@epilogue.com Date: Wed, 27 Oct 93 16:18:27 EDT From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9310271618.aa15871@quern.epilogue.com> Date: Wed, 27 Oct 93 15:40:39 -0400 From: Rob Coltun I aggree that you need multiple locators. A couple of reasons that you may want overlapping clusters. One is for policy. This is a good example of why I think that multiple locators and overlapping clusters are *not* a good idea. I think you should use the policy mechanism that Nimrod provides with tagging links on the maps instead of inventing your own with multiple locators. However, policy has another side (at least one more) and that's enforcement. It may be that I'm not seeing how having multiple locators is useful in enforcing a policy. Nimrod's maps and flow setup do nothing for enforcement. If this becomes an argument for overlapping clusters then it may be that Nimrod needs to add policy enforcement to its agenda. I would prefer that this stay an orthogonal issue to Nimrod itself though. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa20988; 27 Oct 93 16:43 EDT Received: from pizza by PIZZA.BBN.COM id aa13112; 27 Oct 93 16:22 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa13108; 27 Oct 93 16:20 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa19889; 27 Oct 93 16:22 EDT Received: by ginger.lcs.mit.edu id AA25552; Wed, 27 Oct 93 16:22:18 -0400 Date: Wed, 27 Oct 93 16:22:18 -0400 From: Noel Chiappa Message-Id: <9310272022.AA25552@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: IETF meeting agenda Cc: jnc@ginger.lcs.mit.edu The New Internet Routing and Addressing Architecture BOF (nimrod) will meet on Thursday, November 4th, from 0930 to 1200 and from 1330 to 1530. Following is the proposed agenda: - Agenda bashing (10 minutes) - Review of proposed charter (30 minutes) - Review of existing and proposed new terminology (1 hour) - Debate on some items from "open architectural issues" list (2 hours) - Work plan for immediate future (30 minutes) Exactly which issues from the "open issues" list we tackle will have to be decided; we can't do the whole thing, obviously. If anyone has suggestions for things which have been missed, can you please send them to me? Thanks... Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa08658; 28 Oct 93 6:24 EDT Received: from pizza by PIZZA.BBN.COM id aa16105; 28 Oct 93 6:07 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa16101; 28 Oct 93 6:06 EDT Received: from zephyr.isi.edu by BBN.COM id aa08340; 28 Oct 93 6:08 EDT Received: from zephyr.isi.edu by zephyr.isi.edu (5.65c/5.61+local-13) id ; Thu, 28 Oct 1993 03:08:07 -0700 Message-Id: <199310281008.AA20647@zephyr.isi.edu> To: nimrod-wg@BBN.COM Subject: Can clusters in Nimrod intersect? Reply-To: hotz@isi.edu Date: Thu, 28 Oct 93 03:08:07 PDT From: Steve Hotz From: dab@epilogue.com >> From: Rob Coltun >> >> I agree that you need multiple locators. A couple of reasons that >> you may want overlapping clusters. One is for policy. > > This is a good example of why I think that multiple locators and > overlapping clusters are *not* a good idea. I think you should use > the policy mechanism that Nimrod provides with tagging links on the > maps instead of inventing your own with multiple locators. I concur with Rob. Starting with the premise that clusters are our mechanism for aggregation (be it for purposes of scaling or administratively- convenient grouping), a cluster is then an indication that some subset of links are "equivalently tagged". Admittedly it's been awhile since i've read the nimrod tome ;-), but i believe the notion is that one doesn't need detailed global information, but would acquire more details (about clusters) as they started to look "promising". Hence, both tagged links and clusters (overlapped or not) are needed. The interesting discussion is then what abstraction of the policy (tagged links) within a cluster, is presented to the outside world (perhaps for use as hints about which clusters would be useful to explore in more detail). What will come back and bite us (in the "cluster"? ;-) is trying to present a meaningful abstraction of clusters X & Y (which may have radically different policies) when there is a subset (X^Y) common to each. To do so, one may have to split X^Y off into it's own separate cluster (per one of the three suggestions in original posters message), however, this has two problems: (1) scaling may be negatively impacted if this is done on a widespread basis (2) administrative types might not be inclined to let part of it's zone be "yanked" from the parent cluster (consider the research lab example as proposed by Rob). xxoo, steve FYI: At the risk of gratuitously hawking one of our papers (yet to appear anywhere), we (rekhter, estrin, & self) have a somewhat-related tech report available. How it's related: systematic treatment of clustering issue (approach might be useful for our discussion) && motivating problem/example/use for overlapping clusters How it not (a big not): focus was on link-state HOP-BY-HOP protocols && the problems/constraints imposed by clustering in/on them If interested (& you haven't already reviewed it), ftp'able from ... catarina.usc.edu:pub/papers/ls-cluster.v58.ps   Received: from PIZZA.BBN.COM by BBN.COM id aa18103; 28 Oct 93 10:04 EDT Received: from pizza by PIZZA.BBN.COM id aa16917; 28 Oct 93 9:41 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa16913; 28 Oct 93 9:39 EDT Received: from ftp.com by BBN.COM id aa16618; 28 Oct 93 9:41 EDT Received: by ftp.com id AA25344; Thu, 28 Oct 93 09:41:09 -0400 Date: Thu, 28 Oct 93 09:41:09 -0400 Message-Id: <9310281341.AA25344@ftp.com> To: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM Subject: Re: Terminology (I: Big-I) From: Frank Kastenholz Reply-To: kasten@ftp.com > If you have a router with N serial lines, there is an open question, which is > can we have un-named topology elements (e.g. unnamed point-point links between > routers). If so, the each serial can have a separate network locator, but need > not. The interface at each end can have an interface locator, but this may not > be very useful; you just fire packets down the link the thing on the other > end. If the link is to a host, it's a little trickier, since how do you create > a locator for the host without a locator for the media it is connected to? > Hmm, one more thing for the open issue list. I am not so sure how big an issue this is. First, either we allow unnumbered elements, or we don't. There are no other options so the discussion really centers around these two. If we do not allow unnumbered Elements (that is, every Element in the network MUST have a Locator assigned to it) then there is no problem. The two big reasons in IPv4 for having unnumbered links are A) ease of administration and B) conservation of IPv4 address space. These two are irrelevant for IPng efforts -- a goal of IPng is easier configuration and administration, removing point A. Also, IPng should have a much much larger (perhaps unbounded) space of numbers for use as Locators (i.e. we fix the IPv4 address space depletion problem) so point B vaporizes. Now, the alternative is to allow unnumbered Elements. Well, we will not REQUIRE that some, or for that matter, any, Elements be unnumbered -- that is, if you wish to assign a number to something then you may. Furthermore, there will be Elements in the architecture for which numbers (i.e. Locators) are required; e.g. interfaces. So, for those cases where there is a host at the end of a serial link (Noel's example) then the local administrator would have to assign Locators to the host and the link. (I do not see how we can have "unnumbered hosts" and expect to be able to communicate with them :-). Now, unnumbered elements will certainly add complexity to the various protocols that implement the architecture. Why go through this complexity? Mostly that any IPng effort (Nimrod included) will need to coexist with IPv4 for some time and if IPv4 has unnumbered links, so during coexistance IPng might also have to. Also, a goal of Nimrod is to "fix" the IPv4 routing and addressing problems "in the routing" -- i.e. not change the IPv4 protocols, packets, etc. Therefore, Nimrod would have to exist with some of IPv4's problems -- such as relatively difficult configuration and the IPv4 Address. So, given this analysis, it seems to me that we might be best to allow unnumbered elements to exist. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa26641; 28 Oct 93 12:40 EDT Received: from pizza by PIZZA.BBN.COM id aa17869; 28 Oct 93 12:16 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa17865; 28 Oct 93 12:14 EDT Received: from quern.epilogue.com by BBN.COM id aa25051; 28 Oct 93 12:12 EDT To: nimrod-wg@BBN.COM In-reply-to: Steve Hotz's message of Thu, 28 Oct 93 03:08:07 PDT <199310281008.AA20647@zephyr.isi.edu> Subject: Can clusters in Nimrod intersect? Reply-to: dab=replies@epilogue.com Date: Thu, 28 Oct 93 12:11:30 EDT From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9310281211.aa01332@quern.epilogue.com> Reply-To: hotz@isi.edu Date: Thu, 28 Oct 93 03:08:07 PDT From: Steve Hotz Starting with the premise that clusters are our mechanism for aggregation (be it for purposes of scaling or administratively- convenient grouping), a cluster is then an indication that some subset of links are "equivalently tagged". Then wouldn't those links that are equivalently tagged just be shown as a single link or node on the map one level up? It seems like what you're trying to do with clusters is much more easily done with the nodes in abstracted/aggregated maps. Oh oh. I've just realized that I'm not working with a solid definition for cluster. And having realized that I can see two possible meanings. I've been assuming that clusters meant levels in the locator heirarchy. That is, overlapping clusters means interfaces in the overlap have two locators. It could also be that by cluster you mean an abstracted node in a map. In that case, I don't exactly know what overlapping clusters mean but it's probably a fine way to set up policy. I think. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa09376; 28 Oct 93 16:13 EDT Received: from pizza by PIZZA.BBN.COM id aa18995; 28 Oct 93 15:57 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa18991; 28 Oct 93 15:54 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa08004; 28 Oct 93 15:52 EDT Received: by ginger.lcs.mit.edu id AA07278; Thu, 28 Oct 93 15:51:58 -0400 Date: Thu, 28 Oct 93 15:51:58 -0400 From: Noel Chiappa Message-Id: <9310281951.AA07278@ginger.lcs.mit.edu> To: kasten@ftp.com Subject: Re: Terminology (I: Big-I) Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM The two big reasons in IPv4 for having unnumbered links are A) ease of administration and B) conservation of IPv4 address space. These two are irrelevant for IPng efforts ... Now, unnumbered elements will certainly add complexity to the various protocols that implement the architecture. Why go through this complexity? Good points. I've added this to the "Open Issues" list, to avoid getting a thread, separate from the area issues, going at the same time. Also, a goal of Nimrod is to "fix" the IPv4 routing and addressing problems "in the routing" -- i.e. not change the IPv4 protocols, packets, etc. Therefore, Nimrod would have to exist with some of IPv4's problems -- such as relatively difficult configuration and the IPv4 Address. I had hoped to move fairly quickly to a situation where the IPv4 "address" fields are only used as EID's, and much of the routing, etc was in terms of the new locators. As long as we have the existing IPv4 addresses being used as locators, it's going to present certain limitations. For instance, do we design Nimrod protocols to deal with topology maps where some objects are named with IPv4 locators, or can we dispense with that added complexity (which is purely for limited backward compatabilty)? I'd much prefer a situation where all objects in Nimrod maps have Nimrod locators, even if you have places (a la OSPF) where the Nimrod map just has a single node for a largish area where, say, RIP is being run, and there is no internal connectivity info available, just a single Nimrod locator for that whole area. Locators for interfaces, etc inside that are would thus consist of the Nimrod locator for the area concatenated with the IPv4 address within the area, and the internal routers would route on the IPv4 address. (This is more or less the interoperable deployment plan for Nimrod in the routers.) I'll add this to the Open Issue list too. Now, back to areas/clusters.. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa16679; 28 Oct 93 17:55 EDT Received: from pizza by PIZZA.BBN.COM id aa19583; 28 Oct 93 17:32 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa19579; 28 Oct 93 17:30 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa15426; 28 Oct 93 17:30 EDT Received: by ginger.lcs.mit.edu id AA23803; Thu, 28 Oct 93 17:30:26 -0400 Date: Thu, 28 Oct 93 17:30:26 -0400 From: Noel Chiappa Message-Id: <9310282130.AA23803@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM, ramanath@BBN.COM Subject: Re: Can clusters in Nimrod intersect? Cc: jnc@ginger.lcs.mit.edu (I'd been thinking about where to start on the "open issues" list, and was sort of circling in on areas, so this is a timely thread. Speaking of "areas", do preople prefer that term, or "clusters"? Reponses to me, and I'll summarize; this is some of the new "extended jargon" I promised.) One of the many issues that has come up is whether clusters can overlap without being nested. Turns out this is the same question as "do we allow multiple addr.. err, locators" (it was listed as that in the first "open issues" list..) One way to form clusters is to define a relation among internetwork entities and to aggregate entities according to their relationship. ... Another way to form clusters is to aggregate entities so as to minimize some cost function. We will want to do both. One mostly does the latter to provide routing whose overhead costs scales less than linearly with the size of the network, so we will have to do that. We will also do the former, since the only way to express policies concisely is to be able to refer to topology aggregates across which a given policy applies. - All entities within a cluster must satisfy at least one relation - "topological connectivity". That is, any two entities within a cluster must be connected by at least one path that lies entirely within the cluster. Err, yes and no. This rule implies that no area is partitioned. I reckon it's simply infeasible to make this rule, and we need a way to handle partitioned areas, which may or may not include relaxing this rule. However handling parititions is a separate problem (already in the list :-), so let's pas over that for now. A related question is whether it is worthwhile, and in fact meaningful to assign a "level" to a cluster. ... The address of a cluster or of an entity is specified in terms of all of the clusters that contain it. This is more about locators, and do they grow up or down, than about areas specifically. There are a number of similar questions: can a K level area contain on K-1 level objects, or can it contain K-n level objects too? Then there are all the subsidiary questions about how such things would be represented in locators, but let's skip them all for now (they're all in the list too :-( Can clusters overlap without nesting? ... One advantage to allowing clusters to intersect is that we can better represent some situations. ... It is always possible however, to convert a clustering with intersecting clusters to one without. Conventional clustering algorithms generally produce disjoint clusters I used to be against allowing overlapping areas, since most people seemed to want to use multiple locators to do policy; e.g. if you want to get to D via free networks, use locator L1 for D, and L2 otherwise. I think this is really wrong, locators should simply name places in the internetwork in a unique and easy to use fashion, and policy should be done elsewhere. Still, that's a different diatribe, so back on track.... Anyway, I was really down on overlapping areas (I used to think they were just one more piece of unneeded flexibility that the unwise would overuse, to their detriment), but now I think we may need them. The particular application I have in mind is icnrementally and interoperably reassigning locators (which you will need to do to keep the abstraction hierarchy reasonably efficient as the topology changes), which is hard to do without allowing multiple locators, hence overlapping areas. For example, company X used to be served via provider A, and is switching to B; it presumably wants to be attached to both for a while, rather than throwing a giant switch. This means that for a while, devices inside company X will want to have two locators, of the form A.X and B.X, so areas A and B are overlapping over X. I would be interested in hearing of more applications for multiple locators/overlapping areas. Conversely, are there reasons (other than ugliness :-) not to allow them? As a general way of deciding this topic, if we find just one thing which cannot easily be done without them, that's all we really need to settle the question of whether or not we should have them, no? I haven't seen any previous works on hierarchical networks consider intersecting clusters. Most previous works on hierarchical networks are pretty limited, and not well tuned to the real world. For instance, most of them use a single metric model. So, it's not surprising they are out of touch on this one too.. * Are there situations where we simply *cannot* do without allowing clusters to intersect? Exactly. Noel   Received: from ABERI.BBN.COM by BBN.COM id aa16835; 28 Oct 93 17:58 EDT Received: from pizza by PIZZA.BBN.COM id aa12591; 28 Oct 93 16:33 EDT To: nimrod-wg@BBN.COM Subject: Re: Can clusters in Nimrod intersect? In-reply-to: Your message of Wed, 27 Oct 93 15:40:39 -0400. <9310271940.AA17311@noc.sura.net> Date: Thu, 28 Oct 93 17:31:10 -0400 From: Ram Ramanathan In a previous message Rob Coltun writes : I aggree that you need multiple locators. A couple of reasons that you may want overlapping clusters. One is for policy. As an example, the Navy may have a research branch that may be doing work with other research organizations. So policy may be for the research divisions to be able to communicate freely with other members of the research community. But the Navy as a whole or that building that is housing the research would also have to do more private communication. Having overlapping secure and research clusters would help this. I am not convinced that having overlapping secure and research clusters would help. Perhaps I am missing something here, but I think that even if we configure the clusters as overlapping, it would *effectively* be either disjoint or "separated" (the intersection as a separate cluster). Let me explain. Suppose two companies Alpha and Beta want their research organizations to communicate freely between each other and configure their clusters as shown below (lower case are the research divisions of the upper case): ----------------------\ | A A ------------------------- | a a B B B | | A A | | b b B B | | ------------------------| -----------------------/ Since we want some kind of restriction in traffic flow, we need to do some "police-ing" at appropriate boundaries. If we represent the policeing boundaries by lines in a cluster diagram, we get the representation of what I call the "effective" clustering. What is the effective clustering that results from the constraints imposed on the traffic? We have two scenarios : Scenario 1 : Alpha wants traffic to flow freely between entities A and a, Beta similarly between B and b and both agree that traffic must be free between a and b. Both require that their internal traffic not go outside their domain. If A sends private communication to a and a and b can exchange messages freely, the communication may find its way to b and B. So a has to make sure that only certain messages find its way to b. Hence, we need police-ing function between a and b. Furthermore, there must be no police-ing function between A and a. Drawing the lines for "effective" clustering we have : ----------------------\ | A A ------------------------- | a a | B B B | | A A ------| | | | b b B B | | ------------------------| -----------------------/ So with these requirements, we effectively end up with disjoint clustering. Scenario 2 : Alpha wants to let a and b be as collaborative as possible and decides to make sure private messges do not go to a. (It may however, let messages come un-policed in the opposite direction). Similarly for B. In this case, you effectively end up having to look at traffic between your research and non-research divisions anyway and get something like : ----------------------\/---\------------------ | A A | | | |a a | B B B | | A A | | | | | b b| B B | | | | ---------------------/ \---/-------------------| That is, we get three separate clusters. So it seems to me that while overlapping the clusters looks good on the surface, in effect we have to do what we would do if we had disjoint clusters. While police-ing is not the only criterion for cluster boundary delineation, it is certainly an important factor. I am not saying that it is unreasonable to ask for research divisions (for instance) to be able to freely interact. But is cluster overlapping the only way or the best way to solve the problem? Is it possible to represent that need in some other manner, while making the clusters disjoint (either 2 clusters or as 3 clusters)?. The paper by Yakov Rekhter, Steve Hotz, Deborah Estrin also gives a similar example to motivate the need for clustering. Are there situations that absolutely cannot do without cluster overlap? - Ram. -------------------------------------------------------------- Ram Ramanathan Systems and Technologies Bolt, Beranek and Newman Inc. 10 Moulton Street, Cambridge, MA 02138 Phone : (617) 873-2736 INTERNET : ramanath@bbn.com   Received: from PIZZA.BBN.COM by BBN.COM id aa20754; 28 Oct 93 19:39 EDT Received: from pizza by PIZZA.BBN.COM id aa20283; 28 Oct 93 19:18 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa20279; 28 Oct 93 19:16 EDT Received: from babyoil.ftp.com by BBN.COM id aa19798; 28 Oct 93 19:17 EDT Received: by ftp.com id AA26594; Thu, 28 Oct 93 19:17:49 -0400 Date: Thu, 28 Oct 93 19:17:49 -0400 Message-Id: <9310282317.AA26594@ftp.com> To: jnc@ginger.lcs.mit.edu Subject: Re: Can clusters in Nimrod intersect? From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > One of the many issues that has come up is whether clusters can overlap > without being nested. > > Turns out this is the same question as "do we allow multiple addr.. err, > locators" (it was listed as that in the first "open issues" list..) If you wish to have overlaping clusters then you seem to require multiple locators. On the other hand, if you allow multiple locators, you do not require overlapping clusters (I think). > - All entities within a cluster must satisfy at least one relation - > "topological connectivity". That is, any two entities within a cluster > must be connected by at least one path that lies entirely within the > cluster. > > Err, yes and no. This rule implies that no area is partitioned. I reckon it's > simply infeasible to make this rule, and we need a way to handle partitioned > areas, which may or may not include relaxing this rule. However handling > parititions is a separate problem (already in the list :-), so let's pas > over that for now. I would suggest that as a rule of nominal network operation, no aggregate can be defined so that it is partitioned (though perhaps allowing two separate partitions of an aggregate to be 'unified' via a virtual connection of some kind is probably desireable). I think that a partitioned aggregate is something that we can look at as a problem condition, to be dealt with via "special case code" and not the nominal operating condition. Consider, this is a real issue when both parts of the aggregate still retain connectivity. I feel that most aggregates will be defined so that this can not really happen, for example, FTP might be an aggregate with multiple connections to the Internet, but both of those connections are probably going to be two serial lines, from two service providers, into a single router, or two routers that are one-on-top-of-the-other in the same rack, both connected to the same Ethernet... It would be difficult if not impossible to partition this network such that both partitions maintain connectivity. The other common failure would be something like when the router that connects the engineering subnet to the rest of the company blows a fuse; FTP's aggregate has partitioned, but there is no connectivity for one of those partitions, so repairing partitions simply is not an issue here. > A related question is whether it is worthwhile, and in fact meaningful > to assign a "level" to a cluster. ... The address of a cluster or of an > entity is specified in terms of all of the clusters that contain it. > > This is more about locators, and do they grow up or down, than about areas > specifically. There are a number of similar questions: can a K level area or can elements be inserted into them in the middle -- can I turn ftp.foo into ftp.west_coast_office.foo? I think this is needed -- but you might want to add it to the list anyway. > Can clusters overlap without nesting? I am not sure whether this is the right way to look at it. There is a question here. Are aggregates actually contained within other, higher-level, aggregates (like countries contain states contain towns)? Or is it more of a tree structure? That is, if I draw a picture of the net, do I get a set of nested blobs: +--------------------+ | +----------------+ | | | +------------+ | | | | | +--------+ | | | | | | | +----+ | | | | | | | | | | | | | | | | | | +----+ | | | | | | | +--------+ | | | | | +------------+ | | | +----------------+ | +--------------------+ Or do I get a series of interconnected blobs: +----+ | | +----+ / \ +----+ +----+ | | | | +----+ +----+ / \ / +----+ +----+ | | | | +----+ +----+ > I would be interested in hearing of more applications for multiple > locators/overlapping areas. Conversely, are there reasons (other than ugliness > :-) not to allow them? As a general way of deciding this topic, if we find > just one thing which cannot easily be done without them, that's all we really > need to settle the question of whether or not we should have them, no? Also, if you want multiple providers you need them -- in effect, the static version of your other example. This is important if you consider geographically dispersed companies that have an internal network. For example, FTP Software has two offices, one on the east coast and one on the west coast. We have an internal link connecting the two. We might also want the east coast office to get service from Nearnet and the west coast office to get service from Barrnet. So, we would have our internal network with nodes having locators something like ftp.east.mass_ave_bridge and ftp.west.golden_gate_bridge, plus the "public" access to the nodes; barrnet.ftp.west.golden_gate_bridge and nearnet.ftp.east.mass_ave_bridge. This could be even more complex -- for example, we might want all of our east-coast nodes to be in the "west coast/barrnet" aggregate and all west-coast nodes to be in the "east-coast/nearnet" aggregate. So, we would also have locators: nearnet.ftp.west.golden_gate_bridge and barrnet.ftp.east.mass_ave_bridge. If one wanted to send packets from MIT to the node "golden_gate_bridge" then one would send them to nearnet.ftp.west.golden_gate_bridge -- the packets would enter ftp's aggregate at our nearnet connection and then traverse internal paths (that internal link) to get to the west coast office...) Multiple locators can also give you emergency fail-over capability. Suppose, again, that FTP Software is its own aggregate. Let's say that it gets service through Nearnet but, it has a hot-standby link to PSI. All FTP nodes have two locators: nearnet.ftp.x and psi.ftp.x. However, only Nearnet advertises routes to nearnet.ftp.x and psi.ftp.x. PSI advertises nothing -- we neither recieve routes to the outside world, nor advertise them to the outside world via the PSI connection. Some crazed terrorist blows up BBN and Nearnet ceases to exist. We detect the loss of connectivity via Nearnet so our routers (special, aggregate border routers that can do this sort of magic) start advertising routes to nearnet.ftp.x and psi.ftp.x via the PSI connection. Traffic then switches over to the PSI link. Earlier you wrote.... > free networks, use locator L1 for D, and L2 otherwise. I think this is really > wrong, locators should simply name places in the internetwork in a unique and > easy to use fashion, and policy should be done elsewhere. Still, that's a > different diatribe, so back on track.... Note that my example does not necessarily mean that you are using locators as policy handles... I think that the nodes CAN APPEAR TO BE in two places on the net. I do not see anything "wrong" with this. The nodes at FTP are at two places at once (maybe the n-dimesional physical geography on which the network exists is crumpled up in n+1 dimensional space, so that San Francisco Cal and North Andover Mass are touching each other) -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa25651; 28 Oct 93 21:37 EDT Received: from pizza by PIZZA.BBN.COM id aa20875; 28 Oct 93 21:22 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa20871; 28 Oct 93 21:21 EDT Received: from quern.epilogue.com by BBN.COM id aa25124; 28 Oct 93 21:23 EDT To: nimrod-wg@BBN.COM In-reply-to: Frank Kastenholz's message of Thu, 28 Oct 93 19:17:49 -0400 <9310282317.AA26594@ftp.com> Subject: Can clusters in Nimrod intersect? Reply-to: dab=replies@epilogue.com Date: Thu, 28 Oct 93 21:22:48 EDT From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9310282122.aa01414@quern.epilogue.com> Date: Thu, 28 Oct 93 19:17:49 -0400 From: Frank Kastenholz Reply-To: kasten@ftp.com Also, if you want multiple providers you need them [multiple locators per interface] . . . Multiple locators can also give you emergency fail-over capability. I think this assumes we're using no-brainer routing. That is, our routing primarily follows the locator heirarchy. Perhaps with occasional exceptions like FTP's internal link sideways between two clusters but that'd be the exception. If that's the best we can do then you're right, multiple providers require multiple locators. And multiple layers above the provider makes the number of required locators grow geometrically. I think, hope, we can do better. I'll see if I can explain how I see this working. Undoubtedly that'll make it clearer to me as well. I start with the assumption that each interface has only one locator. Probably that locator is clustered in with things along my primary network link. If I have two then I arbitrarily pick one since it shouldn't matter. So FTP would have barrnet.ftp-west.golden-gate and nearnet.ftp-east.mass-ave. When ftp-east passes its map up to nearnet, it tells nearnet that it also has a link over to barrnet.ftp-west. All the rest of the internals of ftp-east can be aggregated but the external connections shouldn't be (more on this later). Now when someone from outside wants to talk to nearnet.ftp-east.mass-ave it picks up nearnet's map. It sees that ftp-east has been aggregated to a single point so there's no reason there to need more detailed information but it also sees the link to barrnet.ftp-west. It now has the information that barrnet might provide an interesting path and so can pick up its map. Note that this allows for the backup path to kick in if needed. It allows for other west coast sites to reach nearnet.ftp-east through ftp's cross country line if they decide it's better. FTP of course might put a policy on that link that doesn't allow this but that's another discussion. Since nearnet's map shows the link it even allows barrnet.tgv.surfer to get to nearnet sites through FTP's link. More policy considerations. I havn't needed multiple locators on any interface but I have required that the sideways link be passed up. How far up? Well, as soon as you go above the level where the two sites' locator prefixes are the same. Above that you can abstract that link away with no loss of information. Below that, if you abstract the link away someone might not find it and so either take a non-optimal path of fail to connect even though there *is* a functional network path if you just could find it. I havn't thought a lot about routing algorithms which "work harder" at finding paths. My previous example doesn't have a deep enough heirarcy to allow the cross country link to be abstracted safely. So, through the magic of bottom up heirarchies I add a couple new layers and we have sol.terra.us.nearnet.ftp-east.mass-ave and sol.terra.us.barrnet.ftp-west.golden-gate. The map for sol.terra doesn't have to show the link. If the map for sol.terra.us does show it then everyone can find it easily. If it doesn't, you have to look harder and there's nothing that tells you that looking at sol.terra.us.nearnet's map might be interesting. Now the more links that don't follow the heirarchy the faster your maps grow. If it gets too bad you have to start dropping links at a lower layer than you'd like. Here's where the option of an automatic system to rebalance the locator heirarchy comes in. Queue the automatic renumbering discussion for later. A possible problem with this shows up in the multiple provider case. It's not a technical problem but a political one. Suppose FTP's cross country link was provided by a competitor and the net has degraded to having usage charges. Nearnet might succumb to the temptation to edit out that link when they pass on the map so the traffic goes through Nearnet's links for which they get paid. Now I'm certainly not making disparaging remarks about Nearnet here, they provide my net connection, do an excellent jobs of it, and are fine people who know better than to charge by the packet. The example was just a convenient one. Having said all this, I'm not necessarily against multiple locators per interface. I just havn't heard an example where they're a useful. I have heard examples where I think they shouldn't be used. My how I go on. I think it's past my bed time. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa01624; 29 Oct 93 7:41 EDT Received: from pizza by PIZZA.BBN.COM id aa22695; 29 Oct 93 7:27 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa22691; 29 Oct 93 7:25 EDT Received: from mwunix.mitre.org by BBN.COM id aa01020; 29 Oct 93 7:25 EDT Return-Path: Received: from mwmgate2.mitre.org by mwunix.mitre.org (5.65c/SMI-2.2) id AA12760; Fri, 29 Oct 1993 07:25:58 -0400 Message-Id: <199310291125.AA12760@mwunix.mitre.org> Date: Fri, 29 Oct 93 07:26:03 EDT From: m_fidler%W036_NW@mwmgate1.mitre.org To: nimrod-wg@BBN.COM Subject: Re: Re: Can clusters in Nimrod intersect? Short question: >Suppose two companies Alpha and Beta want their research organizations >to communicate freely between each other and configure their clusters as >shown below (lower case are the research divisions of the upper case): > > ----------------------\ > | A A ------------------------- > | a a B B B | > | A A | > | b b B B | > | ------------------------| > -----------------------/ > I am concerned that this looks too much like a real physical topology so I have to ask if the following could be possible (a more realistic problem.) ----------------------\ | A A ------------------------- | a a a B B B | | A A b | | b b B B b | | ------------------------| -----------------------/ - Mike F.   Received: from PIZZA.BBN.COM by BBN.COM id aa15018; 29 Oct 93 10:06 EDT Received: from pizza by PIZZA.BBN.COM id aa23304; 29 Oct 93 9:45 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa23300; 29 Oct 93 9:42 EDT Received: from babyoil.ftp.com by BBN.COM id aa12472; 29 Oct 93 9:32 EDT Received: by ftp.com id AA09345; Fri, 29 Oct 93 09:32:28 -0400 Date: Fri, 29 Oct 93 09:32:28 -0400 Message-Id: <9310291332.AA09345@ftp.com> To: dab=replies@epilogue.com Subject: Re: Can clusters in Nimrod intersect? From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM Good morning Dave, > > Also, if you want multiple providers you need them [multiple > locators per interface] > . . . > Multiple locators can also give you emergency fail-over capability. > > I think this assumes we're using no-brainer routing. I wasn't assuming any particular type of routing protocol/algorithm whatsoever. My thoughs went from Noel's request: "What can be done with multiple-locators/overlapping aggregates" to something that we could do with them... Let's say that this demonstrates a compelling need and we end up supporting multiple-locators -- then a subsequent step would be to develop the necessary routing and other control protocols to do the things we want. > That is, our > routing primarily follows the locator heirarchy. Perhaps with I view locators as being derived from the service provider(s) that one uses to connect to the rest of the Internet. So, if I wish to send a packet to someone else "on the Internet" then, necessarily, I need to send that packet through my service provider (or one of them). Similarly, if that person wishes to send a packet to me, the packet must necessarily go through my service provider to get to me. > occasional exceptions like FTP's internal link sideways between two > clusters but that'd be the exception. If that's the best we can do > then you're right, multiple providers require multiple locators. And > multiple layers above the provider makes the number of required > locators grow geometrically. > > I think, hope, we can do better. I'll see if I can explain how I see > this working. Undoubtedly that'll make it clearer to me as well. I > start with the assumption that each interface has only one locator. > Probably that locator is clustered in with things along my primary > network link. If I have two then I arbitrarily pick one since it > shouldn't matter. So FTP would have barrnet.ftp-west.golden-gate and > nearnet.ftp-east.mass-ave. > > When ftp-east passes its map up to nearnet, it tells nearnet that it > also has a link over to barrnet.ftp-west. All the rest of the > internals of ftp-east can be aggregated but the external connections > shouldn't be (more on this later). > > Now when someone from outside wants to talk to > nearnet.ftp-east.mass-ave it picks up nearnet's map. It sees that > ftp-east has been aggregated to a single point so there's no reason > there to need more detailed information but it also sees the link to > barrnet.ftp-west. It now has the information that barrnet might > provide an interesting path and so can pick up its map. But how does a node at mit (let's say its locator is nearnet.mit.lcs.frotz) that wishes to talk to barrnet.ftp-west.golden-gate know to look under the map for nearnet.ftp-east? By allowing multiple locators, you provide indications to the rest of the network as to how to get here from there -- "golden-gate", by having two locators (nearnet.ftp.west.... and barrnet.ftp.west...) is essentially saying to the world (via DNS or whatever) that there are two ways to get there. FTP's private link is not advertised, as such, to anyone. The router that connects the local network on the east coast to the link simply advertises into the east-coast local network that it can get to nearent.ftp.west and barrnet.* . Similarly, the router connecting the west-coast local network to the link advertises into the west- coast local network that it can get to barrnet.ftp.east and nearnet.* These advertisements could then be carried up the hierarchy, out of the east/west-coast local networks. The router connecting the FTP's east coast office to Nearnet would advertise into Nearnet that it can get to nearnet.ftp (nearnet.ftp.east and nearnet.ftp.west would be aggregated as just nearnet.ftp) AND that it can get to barrnet.ftp (barrnet.ftp. east and barrnet.ftp.west also get aggregated) Now, let's suppose that ANS is the backbone for the entire US Internet. THus, all the Locators used so far are actually ans.nearnet... or ans.barrnet... Then, when Nearnet advertises what it can get to up into the ANS backbone, it would aggregate ans.nearnet.ftp and ans.nearnet. epilogue and ans.nearnet.mit and ans.nearnet.wellfleet and so on and advertise ans.nearnet.* into the ANS backbone. However, it also has this route to barrnet.ftp floating around, but since it can not aggregate barrnet.ftp with anything else, it ALSO advertises barrnet.ftp into the ANS backbone. > Note that this allows for the backup path to kick in if needed. It > allows for other west coast sites to reach nearnet.ftp-east through > ftp's cross country line if they decide it's better. FTP of course > might put a policy on that link that doesn't allow this but that's > another discussion. Since nearnet's map shows the link it even allows > barrnet.tgv.surfer to get to nearnet sites through FTP's link. More > policy considerations. The backup-path issue that I am concerned with is not so much between ftp's east and west coast sites. I am concerned with real commercial customers. What happens when the stock markets and stock brokers are on the network, you call your broker and place an order to sell some stock and the broker says "Sorry Dave, my service provider just got blown up and I lost connectivity". Operations that involve real money will demand to have alternate service provisions, backups, hot-standbys, etc etc etc. A long time ago I had a job to architect networks for newspapers to use in the production of the paper -- and they spent more, per node, on the network (minimizing single points of failure, ensuring that multiple paths were available, and so on) than they did on the workstations themselves. I think that I've noticed fundamental difference in approach between you and I to these problems. I think that you see the routing protocols and algorithms as necessarily distributing topology information in the form of maps, perhaps showing partucular links, etc etc. and then working from that perspective. I think that I see things differently -- I think that I am looking at the problem the other way around -- I am looking for a set of procedures, principles, whatevers, that will allow these named blobs -- ftp, nearnet, ans, barrnet, epilogue, etc, -- to talk to one-another and once I get the general process right _then_ I'll worry about the specific protocols to use. On the other hand, my morning caffeine infusion might not be working yet... > A possible problem with this shows up in the multiple provider case. > It's not a technical problem but a political one. Suppose FTP's cross > country link was provided by a competitor and the net has degraded to > having usage charges. Nearnet might succumb to the temptation to edit > out that link when they pass on the map so the traffic goes through > Nearnet's links for which they get paid. Now I'm certainly not making > disparaging remarks about Nearnet here, they provide my net > connection, do an excellent jobs of it, and are fine people who know > better than to charge by the packet. The example was just a > convenient one. These sorts of issues are always possible, and as you say, not technical. I would imagine that the carrying of routes and advertisements and so on might end up being a contractual matter between the service provider and its customer. Perhaps bundled into our "subscription" with Nearnet is advertisement to the rest of the net of a route to ans.nearnet.ftp. This would be a part of the base service since all ans.nearnet. routes would be aggregated together into just ans.nearnet and advertised as such into ans. As an option for $X more, they could advertise another route -- perhaps to ans.barrnet.ftp -- or perhaps to just ftp, that way FTP can have its own locator, without reference to its service providers. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa22139; 29 Oct 93 11:46 EDT Received: from pizza by PIZZA.BBN.COM id aa23980; 29 Oct 93 11:29 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa23976; 29 Oct 93 11:27 EDT Received: from quern.epilogue.com by BBN.COM id aa20823; 29 Oct 93 11:26 EDT To: nimrod-wg@BBN.COM In-reply-to: Frank Kastenholz's message of Fri, 29 Oct 93 09:32:28 -0400 <9310291332.AA09345@ftp.com> Subject: Can clusters in Nimrod intersect? Reply-to: dab=replies@epilogue.com Date: Fri, 29 Oct 93 11:26:45 EDT From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9310291126.aa02975@quern.epilogue.com> Date: Fri, 29 Oct 93 09:32:28 -0400 From: Frank Kastenholz Reply-To: kasten@ftp.com Good morning Dave, Mornin' Frank, > That is, our > routing primarily follows the locator heirarchy. Perhaps with I view locators as being derived from the service provider(s) that one uses to connect to the rest of the Internet. So, if I wish to send a packet to someone else "on the Internet" then, necessarily, I need to send that packet through my service provider (or one of them). Similarly, if that person wishes to send a packet to me, the packet must necessarily go through my service provider to get to me. This is where we go separate ways. I don't agree that packets to my locator have to come through the service provider that's mentioned in the locator. In fact, the way I'm thinking that's one of my axioms. I'll see if I can explain why. If you have a different locator for each of your service providers then you're equating addressing and routing. You select your route by picking a different address, I mean locator. I see two major problems with this. The first is a problem we have today with multi-homed hosts. You, as a random luser out in the wide net, have no information as to which locator you should use. In the Internet now we just pick one of the addresses and hope the routing will do the right thing. With Nimrod's maps you'd have the opportunity at least to pick up the necessary maps to trace the routes to all locators of your destination and then select the locator that gives you the best route. It makes me nervous equating addressing and routing this way. The other problem is one of scaling. Sure, most sites are only going to have one or two service providers. The really studly places will have three. So you have three addresses per interface. Big deal. The problem I see is that you then could have overlapping clusters in the layer above service providers and more in layers above that. Suppose I have two service providers one of whom is very well connected (for reliability) to three long hauls and the other is my backup and is connected to two long hauls. So now each of my interfaces has five addresses. I don't like the way this is going and I think this path is inevitable once you equate addresses and routes. There's a third problem I see too. Once you've equated addresses and routes you've taken away the incentive to do better routing. If we have Nimrod's maps you theoretically have the ability to go look up routes that don't follow the locator heirarchy but why bother if you get all possible combinations from the locator list of your destination? But how does a node at mit (let's say its locator is nearnet.mit.lcs.frotz) that wishes to talk to barrnet.ftp-west.golden-gate know to look under the map for nearnet.ftp-east? This is just the reverse direction of the example I gave previously. barrnet.ftp-west advertises in its maps its link to nearnet.ftp-east. When you pick up the map for barrnet you'll see the link and you now have the information that you might be interersted in looking at the maps for nearnet and nearnet.ftp-east. By allowing multiple locators, you provide indications to the rest of the network as to how to get here from there -- "golden-gate", by having two locators (nearnet.ftp.west.... and barrnet.ftp.west...) is essentially saying to the world (via DNS or whatever) that there are two ways to get there. FTP's private link is not advertised, as such, to Yes, I see how multiple locators can pass on the information that there's another path. I just think the maps are a better way to do it. The backup-path issue that I am concerned with is not so much between ftp's east and west coast sites. I am concerned with real commercial customers. I'm obviously missing your point. My conclusion was that backup paths worked without multiple locators per interface. They should work as well for commercial customers as FTP. I think that I've noticed fundamental difference in approach between you and I to these problems. I think that you see the routing protocols and algorithms as necessarily distributing topology information in the form of maps, perhaps showing partucular links, etc etc. and then working from that perspective. I think that I see things differently -- I think that I am looking at the problem the other way around -- I am looking for a set of procedures, principles, whatevers, that will allow these named blobs -- ftp, nearnet, ans, barrnet, epilogue, etc, -- to talk to one-another and once I get the general process right _then_ I'll worry about the specific protocols to use. On the other hand, my morning caffeine infusion might not be working yet... Hmmm. In the structure that I see for Nimrod, my looking at specific algorithms isn't that I'm pushing those algorithms as such. I've started with the premiss that I'm not going to equate routing and addressing, that is I don't want multiple locators per interface (for all the reasons I gave above). Now I need to prove that routing algorithms exist which can do the right thing given that premiss. So I came up with the algorithm that I described yesterday. Is it the best one? Is it what we'll actually use? I don't care. Nimrod allows everyone to run their own routing algorithm. I just needed an existance proof that there is an implementable algorithm that can provide all the functions that multiple locators do. Well, at least all the functions that I've heard so far. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa23450; 29 Oct 93 12:06 EDT Received: from pizza by PIZZA.BBN.COM id aa24082; 29 Oct 93 11:45 EDT Received: from MARENGO.BBN.COM by PIZZA.BBN.COM id aa24078; 29 Oct 93 11:44 EDT To: nimrod-wg@BBN.COM Subject: Re: Can clusters in Nimrod intersect? In-reply-to: Your message of Fri, 29 Oct 93 07:26:03 -0400. <199310291125.AA12760@mwunix.mitre.org> Date: Fri, 29 Oct 93 11:36:32 -0400 From: Ram Ramanathan >Short question: >>Suppose two companies Alpha and Beta want their research organizations >>to communicate freely between each other and configure their clusters as >>shown below (lower case are the research divisions of the upper case): >> >> ----------------------\ >> | A A ------------------------- >> | a a B B B | >> | A A | >> | b b B B | >> | ------------------------| >> -----------------------/ >> >I am concerned that this looks too much like a real physical topology so >I have to ask if the following could be possible (a more realistic problem.) > >----------------------\ >| A A ------------------------- >| a a a B B B | >| A A b | >| b b B B b | >| ------------------------| -----------------------/ >- Mike F. In my representation schema, the above two are topologically isomorphic. It was not my intention, while drawing the first figure above, to indicate any geographical/physical location of things such as all a's are to the east of A's or anything like that - I did it merely because it is convenient to draw the lines in the later part of my message, if all a's and b's are "shown" together. In other words, yes, the research division routers could be sitting in the middle of the non-research ones. But since we are giving A's and a's (or B's and b's) differing treatments, we can draw lines around them. In Mike F's diagram, such lines would be as follows : ----------------------\ | A A ------------------------- | ----------------| | | | a a a | B B B | | ---------------------| |------- | A A -------------- ____ b | | | | b b B B | b| | | ------------------------| -----------------------/ (Well, the limitations of ascii drawings make it difficult to show things for B and b but you know what I mean!). This can be topologically transformed into the picture given in the latter part of my earlier message...... Maybe this kind of representation is inherently amibguous. If anybody knows of a better way to represent and talk about clusters, I would be eager to hear it. Mike, hope the answer to your question is there somewhere above. - Ram -------------------------------------------------------------- Ram Ramanathan Systems and Technologies Bolt, Beranek and Newman Inc. 10 Moulton Street, Cambridge, MA 02138 Phone : (617) 873-2736 INTERNET : ramanath@bbn.com   Received: from PIZZA.BBN.COM by BBN.COM id aa05569; 29 Oct 93 15:05 EDT Received: from pizza by PIZZA.BBN.COM id aa24952; 29 Oct 93 14:46 EDT Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa24948; 29 Oct 93 14:43 EDT To: dab=replies@epilogue.com cc: nimrod-wg@BBN.COM Subject: Re: Can clusters in Nimrod intersect? In-reply-to: Your message of Wed, 27 Oct 93 14:47:17 -0400. <9310271447.aa15564@quern.epilogue.com> Date: Fri, 29 Oct 93 14:43:30 -0400 From: Isidro Castineyra Dave, You wrote: >> I think we should move away from equating locators and routes as >> firmly as possible and discourage no-brainer routing from the >> beginning. >> In a hierarchical system like Nimrod, I don't think it is possible to completely separate locators from routes. After all, when a group of entities have been clustered together, and, therefore, have, say, addresses that start with the same prefix, if the network has been designed sanely, it should mean that you can get to any of the entities by "about" the same route. If that is not the case, the hierarchy is not helping to minimize the amount of information one has to send around. This does not mean that a locator has to imply a "no brainer" route. I agree that we should quickly forget about no-brainers. Isidro Isidro Castineyra (isidro@bbn.com) Bolt Beranek and Newman, Incorporated (617) 873-6233 10 Moulton Street, Cambridge, MA 02138 USA   Received: from PIZZA.BBN.COM by BBN.COM id aa08499; 29 Oct 93 15:31 EDT Received: from pizza by PIZZA.BBN.COM id aa25149; 29 Oct 93 15:05 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa25145; 29 Oct 93 15:01 EDT Received: from quern.epilogue.com by BBN.COM id aa05202; 29 Oct 93 15:00 EDT To: nimrod-wg@BBN.COM In-reply-to: Isidro Castineyra's message of Fri, 29 Oct 93 14:43:30 -0400 <9310291445.aa03943@quern.epilogue.com> Subject: Can clusters in Nimrod intersect? Reply-to: dab=replies@epilogue.com Date: Fri, 29 Oct 93 15:00:29 EDT From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9310291500.aa04051@quern.epilogue.com> Date: Fri, 29 Oct 93 14:43:30 -0400 From: Isidro Castineyra In a hierarchical system like Nimrod, I don't think it is possible to completely separate locators from routes. After all, when a group of entities have been clustered together, and, therefore, have, say, addresses that start with the same prefix, if the network has been designed sanely, it should mean that you can get to any of the entities by "about" the same route. If that is not the case, the hierarchy is not helping to minimize the amount of information one has to send around. You're right of course. Since we've said that clusters contain things that can reach each other without leaving the cluster, obviously the locator heirarchy tells you about one route. May not be the best route and may not be working today but it's certainly a route you shouldn't ignore. This does not mean that a locator has to imply a "no brainer" route. I agree that we should quickly forget about no-brainers. I thought the route along the locator tree _was_ the "no brainer" route. That's the route where you don't have to really know anything but your address and your destination's address. I guess I'm just afraid that people see the locators and just route along the heirarchy like we've always done. "No brainer" routing is easy; it's what networks do now so people are familiar with it. But it leads to people using multiple locators to signify multiple routes and all the problems I outlines in my last message. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa14988; 29 Oct 93 17:08 EDT Received: from pizza by PIZZA.BBN.COM id aa25680; 29 Oct 93 16:50 EDT Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa25676; 29 Oct 93 16:48 EDT To: kasten@ftp.com cc: dab=replies@epilogue.com, nimrod-wg@BBN.COM Subject: Re: Can clusters in Nimrod intersect? In-reply-to: Your message of Fri, 29 Oct 93 09:32:28 -0400. <9310291332.AA09345@ftp.com> Date: Fri, 29 Oct 93 16:48:42 -0400 From: Isidro Castineyra Frank, You wrote: But how does a node at mit (let's say its locator is nearnet.mit.lcs.frotz) that wishes to talk to barrnet.ftp-west.golden-gate know to look under the map for nearnet.ftp-east? By allowing multiple locators, you provide indications to the rest of the network as to how to get here from there -- "golden-gate", by having two locators (nearnet.ftp.west.... and barrnet.ftp.west...) is essentially saying to the world (via DNS or whatever) that there are two ways to get there. FTP's private link is not advertised, as such, to anyone. The router that connects the local network on the east coast to the link simply advertises into the east-coast local network that it can get to nearent.ftp.west and barrnet.* . Similarly, the router connecting the west-coast local network to the link advertises into the west- coast local network that it can get to barrnet.ftp.east and nearnet.* A different approach to the problem is to think in terms of "routing hints" rather than multiple locators. (This is an idea of Dave Clark, though I do not think he used this name.) Suppose when you look for Dave's machine, say, ftp.dave's in the equivalent of Nimrod's DNS you get a list of route "suffixes": nearnet.ftp, and farnet.ftp. If you were to query another machine at ftp, one that happens to be at the west coast, you would get the reverse list: farnet.ftp, nearnet.ftp. You would try the first entry of that list as the tail-end of the route to get to ftp.dave. If the route derived from that entry fails, you would use the second one, etc. The locator stays the same, say, nearnet.ftp.dave's, the hints in the DNS help you get there. In the end, this might be the moral equivalent of having multiple locators, but it might be useful to separate naming and clustering from routing. Isidro Isidro Castineyra (isidro@bbn.com) Bolt Beranek and Newman, Incorporated (617) 873-6233 10 Moulton Street, Cambridge, MA 02138 USA   Received: from PIZZA.BBN.COM by BBN.COM id aa22802; 30 Oct 93 1:06 EDT Received: from pizza by PIZZA.BBN.COM id aa27353; 30 Oct 93 0:53 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa27349; 30 Oct 93 0:51 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa22232; 30 Oct 93 0:46 EDT Received: by ginger.lcs.mit.edu id AA09499; Sat, 30 Oct 93 00:46:52 -0400 Date: Sat, 30 Oct 93 00:46:52 -0400 From: Noel Chiappa Message-Id: <9310300446.AA09499@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Terminology Cc: jnc@ginger.lcs.mit.edu I have prepared (as promised :-) a list of more, Nimrod-specific, terminology, which includes a fairly large number of terms; I will be sending that along under separate cover. Also, attached to this message are some minor (and hopefully non-contentious :-) updates to the existing terminology which came from Big-I. The principal change is to produce a definition of "interface", and to make a clearer definition of "locator" (in the general sense, not as shorthand for "interface locator"). We will be discussing all of these at the Nimrod WG meeting at the IETF, so please don't sidetrack the mailing list by discussing them there (lets stay focused on the area discussion there). Just print out this list and think about what problems you have, etc, and bring it along. For those of you who aren't coming, feel free to send me private email about concerns. I will update this list with the results of the IETF WG, and then submit it to the mailing list for a second round, and we can discuss any remaining issues at that point. Noel -------- "locator" - A structured name for network objects and aggregates. The structure is used by the routing to make its job feasible. Things which have related locators must usually be topologically related, or else the routing will break down. "locator" used on its own is often jargon shorthand for an "interface locator" (see below), but it can also mean the structured name of an aggregate, be it a single net, a group of nets, or whatever. Such locators are usually formed by dropping the least significant elements from an interface locator. "element" - An individual component of a structured, presumably heirarchical, locator. "interface" - A host's attachment point to the internetwork. Alternatively, the 'place' to which the last hop router delivers the packets, or a distinctly nameable 'place' (at the link layer) to which packets may be sent. Thus, even though several distinct machines may share a physical piece of hardware which connects them to some sort of communication mesh, usually there is some way (in the link-level packet headers) to distinguish which of the hosts the packet is for; each thus has a separate interface. "interface locator" - The structured name of an interface. The destination interface locator might or might not appear in all packets (see "selector").   Received: from PIZZA.BBN.COM by BBN.COM id aa25548; 30 Oct 93 2:03 EDT Received: from pizza by PIZZA.BBN.COM id aa27492; 30 Oct 93 1:51 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa27488; 30 Oct 93 1:50 EDT Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa24236; 30 Oct 93 1:50 EDT Received: by ginger.lcs.mit.edu id AA09538; Sat, 30 Oct 93 01:50:53 -0400 Date: Sat, 30 Oct 93 01:50:53 -0400 From: Noel Chiappa Message-Id: <9310300550.AA09538@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Terminology (Nimrod-specific) Cc: jnc@ginger.lcs.mit.edu "endpoint" - An endpoint is one participant of an end-end communication; i.e. the fundamental agent of end-end communications. It is the entity which is performing a reliable communication on an end-end basis. In most circumstances, this concept maps in a fairly direct and straightforward way to the concept of "host". Another way to characterize an endpoint uses the design principle of "fatesharing". An "endpoint" may thus be alternately viewed as a "fatesharing region"; i.e. a boundary drawn around a set of state and/or computations such that it lives or dies as a unit. "EID" - an 'endpoint identifier'. A name for an endpoint which has the characteristics that it is globally unique, 'flat' (i.e. contains no information as to where the endpoint is), fixed length and reasonably short. "flow" - A unreliable stream of traffic from one host to another, which is explicity recognized by the network. Any of the packets which make up a flow may be delayed, damaged, duplicated, or reordered by the network. There may be state stored in the network relating to a flow, but it is not 'critical state'; i.e. it can be destroyed without warning, and resupplied by the source of the flow. "flow ID" - A 'flow identifier'. A name for a flow. "node" - The computing object which an interface to the network leads to; in other words, the machine where packets sent to that interface wind up. This includes both nodes which are the sources and destinations of packets only (i.e. they do not foward traffic), as well as those which send traffic from one physical network to another. "endnode" - A node which is only the source and/or destination of packets, i.e. not one which forwards traffic. "router" - A node which does forward traffic; i.e. all nodes which are not endnodes are routers, and vice versa. "host" - An alternative name which mostly has the connotation of endnode, but note that routers usually contain some functions (such as remote login) which are normally associated with endnodes; in so doing, they are the sources and destinations of packets. When routers are performing these functions, they are hosts as far as the network is concerned. "network" - A shared communciation system which allows a group of nodes, connected to that system via interfaces, to all communicate directly with each other. "link" - A point to point link, i.e. a network with only two interfaces. "basic entities" - The lowest level items in a network topology; i.e. interfaces, nodes, and networks. "area", "cluster" - The actual network topology can be modelled as a graph, containing nodes and networks, with the former connected to the latter via interfaces. An area is a (usually contiguous) section of the graph. Areas may recursively contain other areas. Areas have names, which are the elements in locators. "entities" - Graph elements, including basic entities and areas. "attributes" - Entities may have traits which are either inherent (such as bandwidth, delay, etc), which often go under the name of 'Quality of Service', or externally applied (such as cost and usage restrictions), which are usually described as 'policies'. An "attribute" is a descriptor of such a trait, applied to a entity. Since the two different types of trait are essentially indistinguishable as far as the routing is concerned, they have only a single name. "route", "path" - A sequence of entities which leads from a source entity to a destination entity. "policies" - External contraints of various kinds; all known policies fall into one of following three classes. "transit policies", "access control policies" - A policy applied to an entity; e.g. only certain classes of traffic are allowed to use a given link. "source policies" - A policy applied to the selection of a route; e.g. only certain types of link are acceptable in creating a given route. "information hiding" - A policy applied to the distribution of information about the details of the topolology of some part of the network. "abstraction" - A process which reduces the amount of information available for making decisions about routes, to reduce overhead. In a map based system, this usually involves substituting a simplified map of an area for the fully detailed map of the area. "thinning" - An abstraction proces in which some routes differ after the abstraction has happened; i.e. some useful info is discarded. In other words, the routes are now less optimal, and in policy routing, perhaps impossible to find, even though a valid route does exist. "compression" - An abstraction process in which all routes are the same after the abstraction has happened as before; i.e. no 'useful' information is discarded. All instances of abstraction can be divided into two non-overlapping sets, compression and thinning. "aggregation" - An term meaning approximately the same thing as abstraction, but more limited. It refers to the replacement of a number of routing table entries (in a routing table based system such as all Destination Vector systems) with a single routing table entry. This is obviously not quite as flexible as abstraction in map based system, where the representation of an abstracted area need not be a single node, but may be a simpler map than the original. Abstraction in a map based system thus allows degrees of thinning which are intermediate between the simple "yes/no" decisions of aggregation in a routing-table based system. "area boundary", "abstraction naming boundary" - The boundary of an area on the map of the network topology. "scope" - The section of the network over which the detailed representation of an area, as opposed to its abstraction, is normally available. "abstraction action boundary" - A term very similar to the above, but used mostly in Destination vector systems. It refers to the boundary at which routes to individual elements of an aggregate are replaced by a single route to the aggregate. -------- PS: A "tip of the hatly hat" to Bill Simpson for the suggestion on nodes, end-nodes, etc.   Received: from PIZZA.BBN.COM by BBN.COM id aa07708; 30 Oct 93 13:26 EDT Received: from pizza by PIZZA.BBN.COM id aa29105; 30 Oct 93 13:14 EDT Received: from BBN.COM by PIZZA.BBN.COM id aa29101; 30 Oct 93 13:12 EDT Received: from iesd.auc.dk by BBN.COM id aa07233; 30 Oct 93 13:11 EDT Received: from turing.iesd.auc.dk by iesd.auc.dk with SMTP id AA05458 (5.65c8/IDA-1.5/MD for ); Sat, 30 Oct 1993 18:09:58 +0100 Received: by turing.iesd.auc.dk id AA06499 (5.65c8/IDA-CLIENT/MD for nimrod-wg@BBN.COM); Sat, 30 Oct 1993 18:10:19 +0100 Date: Sat, 30 Oct 1993 18:10:19 +0100 From: Erick Herring Message-Id: <199310301710.AA06499@turing.iesd.auc.dk> To: nimrod-wg@BBN.COM Subject: Unsubscribe request: herring@iesd.auc.dk Please unsubscribe `herring@iesd.auc.dk' Thanks, Erick   Received: from PIZZA.BBN.COM by BBN.COM id aa26993; 31 Oct 93 17:53 EST Received: from pizza by PIZZA.BBN.COM id aa03015; 31 Oct 93 17:41 EST Received: from BBN.COM by PIZZA.BBN.COM id aa03011; 31 Oct 93 17:38 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa26347; 31 Oct 93 17:38 EST Received: by ginger.lcs.mit.edu id AA02735; Sun, 31 Oct 93 17:37:59 -0500 Date: Sun, 31 Oct 93 17:37:59 -0500 From: Noel Chiappa Message-Id: <9310312237.AA02735@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: More terminology? Cc: jnc@ginger.lcs.mit.edu Also, if people have suggestion for more terms that need definitions, please send them in to me. I have the following list of terms which I intend to provide definitions for, in addition to the ones we already have: "routing" "routing architecture" "forwarding" "distributed routing", "centralized/unitary routing" "per-packet routing", "setup routing" "critical state" "soft state" "hints" "mobil host" "portable host" Obviously a lot of these are not very Nimrod specific, I just want to make sure that when we use them, we all have the same meaning in mind. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa25096; 1 Nov 93 2:54 EST Received: from pizza by PIZZA.BBN.COM id aa04391; 1 Nov 93 2:41 EST Received: from BBN.COM by PIZZA.BBN.COM id aa04387; 1 Nov 93 2:39 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa24425; 1 Nov 93 2:38 EST Received: by ginger.lcs.mit.edu id AA16312; Mon, 1 Nov 93 02:38:00 -0500 Date: Mon, 1 Nov 93 02:38:00 -0500 From: Noel Chiappa Message-Id: <9311010738.AA16312@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Open Issues List Cc: jnc@ginger.lcs.mit.edu Hi all, coming next is a new (and substantially expanded and reorganized :-) version of the open issues list. Thanks to everyone who sent in things to add to the list... We'll spend 10 minutes at the start of the "Open Issues" section of the WG meeting scanning this list to decide which ones to try and tackle there. I've tried to pull apart some of the larger categories (such as Addresses :-) into several smaller ones (e.g. Areas, Topology Map, Locators-abstract and Locators-representation_and_implementation). I've also tried to put the categories in the approximate order we should tackle them in; I've put a number of restricted topics which bear on locators (e.g. areas) before the locators themselves, in the hope that by the time we get to locators we'll have a fair amount of stuff worked out, so the locator discussion won't be quite so aimless and wide-ranging as I fear it otherwise might. As an aid to concentrating on the important ones (since the list is so darn long), I've also gone through and tried to mark things which are "implementation issues" (by which I mean pretty trivial things that any good engineer, given the overall architecture, could answer with a modest amount of thought) from the big "architectural issues", which are truly wide open. E.g., the map distribution mechanism might be more of an implementation issue, whereas the question of "can areas overlap" is more architectural. It's a pretty long list, and I hope it won't discourage you all. I actually think it's good that we've gotten these things all written down, and (hopefully) arranged in a way that allows orderly progress in closing open topics, rather than going around in circles endlessly. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa25182; 1 Nov 93 2:57 EST Received: from pizza by PIZZA.BBN.COM id aa04412; 1 Nov 93 2:45 EST Received: from BBN.COM by PIZZA.BBN.COM id aa04408; 1 Nov 93 2:44 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa24545; 1 Nov 93 2:40 EST Received: by ginger.lcs.mit.edu id AA16344; Mon, 1 Nov 93 02:40:50 -0500 Date: Mon, 1 Nov 93 02:40:50 -0500 From: Noel Chiappa Message-Id: <9311010740.AA16344@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Open Issues (List) Nimrod Open Architectural Issues List Key: * = Issue which is sort of a tenet of the Nimrod faith, and thus already decided really, but included here to allow some discussion to i) allow everyone to understand why it is the way it is, and ii) make sure that a horrble mis-step hasn't been made! # = Issue which is more of an implementation detail than a true architectural issue. Areas - Can areas overlap, which is the same question as do we allow multiple locators for a single interface. (Technically, does each K level entity appear in at least and at most one K+1 level entity?) If so, what uses do people see for them? - Strict reduction (i.e. can level K area only contain K-1 objects, or K-2 .... 0 objects as well) - 5.1 - How do we handle partitions? Three possibilties are renumbering, advertising subcomponents, and virtual joining (a la IS-IS). # - Default abstraction algorithm (model an area as a pseudo node with individual links to border routers, or N^2 border router direct interconnections) - 5.4 # - How are we going to have an area configuration system which is either autoconfiguring, or, if done partially manually, impossible to screw up? Topology Map - Area representation issues (i.e. is an area a node or a simple piece of topology, and are area nodes always separated by routers) - 5.1 - Do the nodes in the graph represent interfaces or routers/networks; i.e. are the arcs interfaces of routers/networks? - What will be the form of the routing information generated and used by Nimrod? Simply topology details, or something more complex, such as route fragments? # - Exactly what info will be provided about topology entities, other than the list of neighbouring entities, and the attributes of the entity? - In addition to areas, what other sort of high-level entities will be needed? Virtual links are talked about in the document, and virtual routers seem like they are useful for much the same reasons as virtual links. - Are policies related to interfaces or routers or both? - Are point-links nodes or just arcs between routers? # - How do attributes get expressed; i.e. what is the syntax and semantics of attributes? Map Distribution # - What will be the mechanisms for distributing/obtaining Nimrod routing information? Do we use a probabilistic or reliable update? # - How much information will be distributed automatically, and how far? # - How will a route selector decide when to ask for more information than it already has in its map? # - How do routers figure out where their neighbours are, especially without configuration? Addr.. err, Locators (abstract): * - Do interfaces have locators? - Locator basing (i.e. do locators grow up, down or both) - 5.2 - Is the label of each abstraction globally unique, or only within the enclosing abstraction? - Do we keep ARP; i.e. do Nimrod locators look like NSAP's, i.e. do they simply specify a binding region, inside which the routing has to track endpoints individually, or what? - Do Nimrod locators contain MAC addresses? (Beaten to death on Big-I already... :-) - Are partial locators possible? - Can we have topology elements (e.g. point to point links) without locators? - Do routers have locators? Locators (representation and implementation): - Representation versus abstraction; i.e. what do locators look like written down, or in packets, versus what is their abstract syntax, etc. - Can you tell by looking at a locator what kind of thing it names (an abstraction, a network, an interface, etc?) * - What do Nimrod locators look like: are they fixed length (e.g. 64 bits?) or variable length, and with a fixed or variable number of levels? - If variable length, what about each level, is it fixed size, or can it grow too? - 5.2 # - Is there some way for hosts to pick up most of the locators from their neighbour routers? This will help with autoconfiguration, not to mention address reassignment. EID's (abstract): * - Do we have separate namespaces for endpoints and interfaces: in other words, do we have separate locators and EID's at all, or just classical addresses? - Do Nimrod interfaces have unique identifiers? I.e, is there a use for being able to look at two locators and know that they refer to the same interface? * - What is the smallest level of thing an EID can name; is it a host, or somethign smaller, like a mobile process? EID's (representation and implementation): - What do endpoint identifiers look like; i.e. are they fixed or variable length, if fixed how long are they, how are they assigned, etc.. # - How will Nimrod map from identifiers to locators? Will it use an extended version of the DNS (and hence, we will work with the DNS working group)? Or will it be a separate system but have an architecture like the DNS? # - How will this mapping work for hosts with multiple interfaces? How will it pick one? # - How will we assign endpoint identifiers to processes (ones which may migrate between hosts)? # - Can we use link level ID's (assuming a flow to link equivalence) to avoid sending EID's over slow links? Flows: - Are flow ID's globally unique, or just locally? - Will Nimrod aggregate flows, or keep them separate? # - How do flows get set up? Torn down? Managed? # - How do flows get rerouted around failures? When it is rerouted, how do we make sure the new route meets the flow's policy requirements. Optimizations: * - Do we have a hop-by-hop mode, or just flows and source-routed packets? - Entry point optimization / hidden path finding (i.e. of the three suggested methods for handling people with multiple service providers, which looks most promising) - 6.3 # - We need to decide what source routes look like. We can't include lists of Nimrod locators; that would be way too space-hungry. They need to be fairly compact, perhaps looking something like PIP source routes. Multicasting: - Do we design the Nimrod protocols to handle multicasting explicitly? (This will affect routing generation, which is a local matter, and path setup, which is not. Is routing information distribution functionality affected at all?) - What about multicast policy routing? Mobility: # - How do we handle mobile hosts; i.e. do we map locators to locators, or what, and how far back toward the source do we go with the news that the locator has changed? - Can areas be mobile; i.e. can you take a whole piece of the topology and plug it in somewhere else. How does this work? Does it need different support from mobile hosts? Configuration: # - What sort of support can we offer to make re-addressing easier? # - Configuration in general needs to be as automatic as possible. What can we design in the way of tools to help people with configuration, setup, etc? # - How much auto-configuration is possible? Odds'n'ends: * - Do we flush the IGP/EGP split? - What does the interaction with resource allocation look like? # - What does the security look like? Presumably the protocol has hooks to allow Byzantine security? # - What debugging tools do we need? # - What fault analysis mechanisms do we need? Deployment: # - Deployment plan; i.e. what holes do people see in this that I have missed? In addition to what's in the proposal, Bob Smart point out that we can do a lot with ARP in terms of when 32-bit EID's start getting moved around, so you can't route on them anymore. # - New packet format (i.e. do people believe Van and think we should put this off as long as possible, and if not, why not) # - Do we wrap packets (how do we add locators to packets which need them, and are packets in flows modified or wrapped) - 5.7 - Do we allow topology elements to be named with IPv4 locators, or does Nimrod handle only Nimrod type locators? # - Do we want an IPv4 flow ID option for the transition? # - What modifications to higher-layer software are absolutely required in order to make Nimrod work? # - What modifications are possible in order to optimize a higher-layers' use of Nimrod? # - What DNS support is needed?   Received: from PIZZA.BBN.COM by BBN.COM id aa28628; 5 Nov 93 0:51 EST Received: from pizza by PIZZA.BBN.COM id aa29968; 5 Nov 93 0:36 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29964; 5 Nov 93 0:33 EST Received: from mcigateway.mcimail.com by BBN.COM id aa28286; 5 Nov 93 0:35 EST Received: from mcimail.com by MCIGATEWAY.MCIMail.com id aa16230; 5 Nov 93 5:25 GMT Date: Fri, 5 Nov 93 05:25 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: More terminology? Message-Id: <72931105052527/0003858921NA2EM@mcimail.com> Noel said: > Also, if people have suggestion for more terms that need >definitions, please send them in to me. I have the following list of terms >which I intend to provide definitions for, in addition to the ones we >already have: > "mobil host" > "portable host" What about "mobile clusters"? You already have this with airplanes, and as I said, expect it with cars... Bob   Received: from PIZZA.BBN.COM by BBN.COM id aa28640; 5 Nov 93 0:52 EST Received: from pizza by PIZZA.BBN.COM id aa29974; 5 Nov 93 0:36 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29970; 5 Nov 93 0:33 EST Received: from mcigateway.mcimail.com by BBN.COM id aa28288; 5 Nov 93 0:35 EST Received: from mcimail.com by MCIGATEWAY.MCIMail.com id ad16230; 5 Nov 93 5:25 GMT Date: Fri, 5 Nov 93 05:25 GMT From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: Open Issues (List) Message-Id: <63931105052536/0003858921NA2EM@mcimail.com> Noel said: > - Can areas be mobile; i.e. can you take a whole piece of the topology > and plug it in somewhere else. How does this work? Does it need > different support from mobile hosts? For the case of the car, it might not have a 'home'. Trucks, on the other hand might have the company as their 'home'. And then there are 'Mobile homes' ;-) Bob   Received: from PIZZA.BBN.COM by BBN.COM id aa17785; 8 Nov 93 18:51 EST Received: from pizza by PIZZA.BBN.COM id aa20561; 8 Nov 93 18:34 EST Received: from BBN.COM by PIZZA.BBN.COM id aa20557; 8 Nov 93 18:31 EST Received: from quern.epilogue.com by BBN.COM id aa15747; 8 Nov 93 18:26 EST To: nimrod-wg@BBN.COM Subject: on interfaces as nodes in the nimrod map Reply-to: dab=replies@epilogue.com Date: Mon, 8 Nov 93 18:26:05 EST From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9311081826.aa17486@quern.epilogue.com> I've had some more thoughts on what nimrod maps ought to look like and why I liked Martha's suggestion to make the nodes be interfaces and areas. All the other suggestions had areas as nodes and interfaces as links. This is certainly how we tend to draw network maps. But in the abstraction heirarchy, interfaces are simply a degenerate case of areas. Leads one to think that perhaps they really ought to appear in the map the same way. It's easy to make interfaces links but not areas, therefore interfaces ought to be nodes too. The problem with interfaces as nodes is that you get O(n^2) links between the interfaces on a network and O(n^2) links between the interfaces on a router. There're two ways to fix this. One way is to add concentrator nodes to the map. These exist purely for the purpose of reducing O(n^2) connectivity to O(n). They make the map a little simpler. They would sit in the topology where a router or multidrop network exists. They would not have names. Lots of people will hop up here and say that the concentrator node sitting in for the network could have the name of that network's area. It could but it's not necessary. Don't equate networks and areas just yet because I don't think they're necessarily going to be the same. The other fix for O(n^2) is to hide it all in an area. Make an area for the network and then it's reduced to a single node, the complexity isn't seen outside, so it doesn't matter. You can do this for either a network or a router but if you do both you get lots of overlapping areas. [Can you have an area with no interfaces in it? Then you could show the router as one of these and put all the interfaces in the area for the network. I don't think this makes sense but it's something to think about some night when you can't get to sleep.] But why make the first level (counting up) area stop when it hits a router? The size of an area is limited by the number of destinations the area routing can handle. For routing technology today this is in the 1E4 to 1E5 range. That's how many interfaces can be in a first level area. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa06535; 9 Nov 93 1:44 EST Received: from pizza by PIZZA.BBN.COM id aa21803; 9 Nov 93 1:23 EST Received: from BBN.COM by PIZZA.BBN.COM id aa21799; 9 Nov 93 1:21 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa05116; 9 Nov 93 1:21 EST Received: by ginger.lcs.mit.edu id AA07041; Tue, 9 Nov 93 01:21:11 -0500 Date: Tue, 9 Nov 93 01:21:11 -0500 From: Noel Chiappa Message-Id: <9311090621.AA07041@ginger.lcs.mit.edu> To: dab=replies@epilogue.com Subject: Re: on interfaces as nodes in the nimrod map Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM I've had some more thoughts on what nimrod maps ought to look like Me, too.. here goes. Martha's suggestion to make the nodes be interfaces and areas. All the other suggestions had areas as nodes and interfaces as links. Err, can we call them "arcs", to avoid confusion with point-point links? I'd love to find out why OSPF has network and router nodes; was this discussed extensively during the OSPF design? This is certainly how we tend to draw network maps. I think that the fact that we tend to draw maps in that way should not be whizzed past so quickly. An architecture should provide simple mappings between its abstractions and the real world, for a number of reasons. First, people will find it easier to adapt to it (and I *don't* want Nimrod to be solely an academic exercise), and second it will be easier to use. (That's the whole point of the policy mechanisms that IDPR/Nimrod use; they are good maps to real world needs.) The second is particularly important: we need to build a system that the average person can easily comprehend, and maps with routers and networks in will help this. Of course, these are not sufficient reasons for one choice or another, but let's keep them in mind. There may be some very elegant engineering we can do if we think of everything as 0-N interfaces which will outweigh the utility of the router/network paradigm, but let's look at things in a little more detail before deciding. But in the abstraction heirarchy, interfaces are simply a degenerate case of areas. Leads one to think that perhaps they really ought to appear in the map the same way. Interfaces are the degenerate case of *everything*. However, I get *very* nervous brushing *real* differences under a tautological rug for the sake of producing a uniform basic paradigm. Maybe we need to not lose track of the differences? What are the advantages and disadvantages of such a uniform paradigm? Two important questions are i) "are there uses for distinguishing among the various different kinds of entities you can build out of interfaces; i.e. routers, networks and areas", and ii) "are there composite entities for which there is no reasonable use in being able to decompose them"? If the answer to the first is "yes", then we need to be able to distinquish between them *somehow*. I think the answer to this is "yes". There are things you can do with routers (e.g. open an SNMP connection) which you don't do with networks and areas. Likewise, there are operations which only make sense on networks and areas. True, we could provide attributes which say "this compound entity is a router", etc, to meet this need. I'm not sure about the second. Are we ever going to be able to split a router into two independant functioning halves? Maybe... but you can treat this as the creation of a new node, with attendant changes to the map. Same thing for a network: yes, nets can partition, but there are three ways of handling parititions (readdress, tunnel-tie, and advertise the pieces) and only the latter necessitates being able to decompose the pieces. It can also lead to a tremendous amount of overhead; e.g. if A.P.X splits into two pieces, and half is left connected to B.Q.Y, advertising the individual hosts which are connected to B.Q.Y at the top level is probably not the most efficient way of handling that partition. I am going to have to turn this over in my brain for a long time until I think I have a good grip on the pros and cons.... It's easy to make interfaces links but not areas, therefore interfaces ought to be nodes too. If you buy the argument that interfaces and areas should be treated identically, I guess this makes sense. This would make the arcs on the graph represent connectivity. Actually, I was sort of assuming that the "interface arcs" in the router/network node map would have attributes (e.g. locator, bandwidth - an interface might have less bandwidth than the network it is connected to). Looked at that way, you can perhaps consider them *yet another* kind of basic entity node (along with routers and networks), and in that map (sort of a hybrid between the two approaches) arcs would still denote connectivity. You'd have to have an interface node between a router node and a network node, and you couldn't have two network nodes connected directly, etc. Hmm, now that I think about it.... wouldn't you want to have the same rules in an interface/area nodes map? I.e., a network can't be directly connected to another network, whether or not you represent them as areas or special nodes? Wouldn't you want to be able to look for, and disallow, this kind of confused representation? The problem with interfaces as nodes is that you get O(n^2) links between the interfaces on a network and O(n^2) links between the interfaces on a router. Yup. I think that's what was in my mind when I started thinking about this years ago. It can get pretty bad; imagine a large Ethernet with, say, 1000 hosts on it.. that's a lot of arcs... There're two ways to fix this. One way is to add concentrator nodes to the map. In other words, the interface nodes, instead of being directly connected to each other via arcs, would connect to the concentrator node, yes? These exist purely for the purpose of reducing O(n^2) connectivity to O(n). They make the map a little simpler. They would sit in the topology where a router or multidrop network exists. But these are simply router and network nodes! The only difference is you've now replaced the complex arcs I assumed to represent interfaces (which can have attrributes) with triples consisting of arc/interface-node/arc... basically the same info, just stored in more bits? They would not have names. They have to have names, otherwise you can't transmit the map to somebody else. Lots of people will hop up here and say that the concentrator node sitting in for the network could have the name of that network's area. It could but it's not necessary. Don't equate networks and areas just yet because I don't think they're necessarily going to be the same. True, you could possibly make them separate. But, how would the two differ? In what circumstances would it make a difference which one, the network area or the network concentrator node, we had? The other fix for O(n^2) is to hide it all in an area. In other words, draw a circle so that all the interfaces are inside the circle? Make an area for the network and then it's reduced to a single node, the complexity isn't seen outside, so it doesn't matter. Wouldn't some map still have to have the full complexity, though? If not, and you have an "un-decomposable area node" for the network, how is this any different from what I'm calling a "network node"? Wouldn't you still want the rule that a "network area node" could not be directly connected to another "network area node", etc, etc, etc? You can do this for either a network or a router but if you do both you get lots of overlapping areas. Right, and if you only have a map with the network and router abstractions the arcs represent.... you guessed it, interfaces! Can you have an area with no interfaces in it? I don't think this makes sense. I think an area consists of a part of the topology; i.e. some collection of nodes, as well as the arcs which fall completely within the area boundary. By definition, an area with no nodes it cannot be... Then you could show the router as one of these and put all the interfaces in the area for the network. So, once again there are router nodes, and network nodes, and the arcs represent the interfaces.... gee, wonder why we always keep coming up with this model? :-) Look, I don't mean to be pouring cold water all over this, but there is a real physical topology out there, made out of real routers and networks, and the question we have to consider is what is the "best" way to represent that real topology. Now, obviously, "best" is a very loaded word, and I don't have a good definition, but I think conciseness, and ease of understanding, should figure, as well as generality and flexibility. I don't know the answer to "what is the best representation model". This idea of making interfaces the basic object has some intriguing aspects, and I'd like to mull it over for a while. I don't think this makes sense but it's something to think about some night when you can't get to sleep. Actually, this is more likely to keep me awake than put me to sleep! (But then again, I have a wierd mind... :-) But why make the first level (counting up) area stop when it hits a router? The size of an area is limited by the number of destinations the area routing can handle. For routing technology today this is in the 1E4 to 1E5 range. That's how many interfaces can be in a first level area. This is starting to sound like IS-IS level 1! Sure, it's possible, but only in limited circumstances; e.g. if the nets are different technologies, they probably have different speeds, etc, (i.e. things which are attributes). Also, the bandwidths available in each net represent separate resource pools, etc. Sure, Nimrod can't *stop* it; you can always hide it from the higher layers if you are doing this, using information hiding. Before I add it to the Nimrod architecture, though, I'd like to know: what exactly does this do for you? Mobility (albeit a very limited kind)? What does is cost you? If it's an incomplete solution to various desires, is is worth it in increased complexity? It's very late. I have no more brain cells. Hope there's nothing stupid in this.. :-) Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa06534; 9 Nov 93 1:44 EST Received: from pizza by PIZZA.BBN.COM id aa21822; 9 Nov 93 1:26 EST Received: from BBN.COM by PIZZA.BBN.COM id aa21818; 9 Nov 93 1:24 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa05389; 9 Nov 93 1:25 EST Received: by ginger.lcs.mit.edu id AA07051; Tue, 9 Nov 93 01:25:37 -0500 Date: Tue, 9 Nov 93 01:25:37 -0500 From: Noel Chiappa Message-Id: <9311090625.AA07051@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: WG meetings at the IETF Cc: jnc@ginger.lcs.mit.edu Here's a brief report on what happened at the WG meetings at the IETF. A fuller set of minutes will be done in the not too distant future. Noel PS: Apologies to those who were listening to the multicast when the back and forth got so fast we started missing the mike; when you have 5 people all going at once, and saying single sentences rapidly back and forth, it's hard to handle without each having their own mike. --------- The group reviewed the current draft working group charter and the latest proposed terminology list, and general satisfaction was expressed with the curent state of both. Discussion then moved on to some of the open architectural issues. Among the points discussed were: - Can areas overlap? - Are abstraction levels identified explicitly? - Do the nodes in the graph represention of the network represent interfaces or routers/networks? - Do interfaces have locators? - Are the labels which elements of locators globally unique? - Do locators grow up, down, and can they be expanded in the middle? - Are partial locators possible? - Do routers have locators? - Do we have separate namespaces for interfaces and endpoints? - What is the smallest thing which can be an endpoint? - Do we have a hop-by-hop mode, or just source routed packets and flows? - Do we retain the EGP/IGP split? - When do we tackle multicast? The following action items were decided on: - Next IETF meetings to be scheduled for Tue and Wed morning if possible. - All new open issues raised during the WG meeting are to be sent to the WG mailing list. - The chair will include the new points, re-sort the list into priority order, add a new category of "local" for issues, and resubmit. - A document showing the outcome of the discussions on the open items will be preapred and sent to the list. - A moderated list discussion will take on remaining open issues. - Scheduling a Boston interim meeting will be investigated. - The WG agreed to have a draft of the architecture RFC prepared by the end of January, 1994, for final examination at the March IETF.   Received: from PIZZA.BBN.COM by BBN.COM id aa11513; 9 Nov 93 10:06 EST Received: from pizza by PIZZA.BBN.COM id aa23490; 9 Nov 93 9:47 EST Received: from BBN.COM by PIZZA.BBN.COM id aa23486; 9 Nov 93 9:46 EST Received: from babyoil.ftp.com by BBN.COM id aa09609; 9 Nov 93 9:46 EST Received: by ftp.com id AA01008; Tue, 9 Nov 93 09:44:58 -0500 Date: Tue, 9 Nov 93 09:44:58 -0500 Message-Id: <9311091444.AA01008@ftp.com> To: dab=replies@epilogue.com, jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM Subject: Re: on interfaces as nodes in the nimrod map From: Frank Kastenholz Reply-To: kasten@ftp.com What is the advantage of having the interfaces be the graph nodes? I can see several disadvantages, but no advantages. Dave and Noel both pointed out the disadvantages so I won't belabor the point. Another way to look at the problem is to first ask what the requirements are. That is, what is the information that needs to be dissemenated outside of a "local" aggregate in order for "remote" hosts to send traffic to a "local" host and vice versa? Now, I have always seen locators as being structured something like: ........ that is, sort of like an IPv4 Address, but with more numbers on the left. For packets to move through nimrod-net, the first-hop router will find the lowest area (root being highest) in its maps that matches the ...... part of the destination locator. This router then "does the right thing" to forward the packet on -- it might be hop-by-hop, where each hop represents an area in the map, or it might be source-routed, where each element in the source-route is an area. Eventually the packet gets to the destination's local area. The routers in this area then would use a mechanism, internal to that area, to get the packet to the destination host's subnet and then the host itself. So, with my model in mind, outside of an area the only information that needs to be advertised to the rest of the world is the ...... stuff. Aggregation will occur because the intermediate areas will only advertise themselves, and not all the areas that are subordinate to them. This brings me to another point, it seems to me that routing within an area should be solely a matter for that area to worry about. If area A is sending a packet to area C and it is going through area B then A should not be concerned with B's internal routing or topology -- A should simply pass the packet to B along with the destination and a set of constraints and policies to observe in forwarding the packet, and B should have the responsibility of getting the packet to the destination within the stated constraints and policies. (The constraints and policies could be tagged onto each packet or they could be part of a flow, which the packet associates itself with via a flow-id, or it could be algorithmically determined by examination of the packet's data, or it could be by magic -- I don't care.) -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000 To: dab=replies@epilogue.com Subject: Re: on interfaces as nodes in the nimrod map From: kasten@baby.ftp.com (Frank Kastenholz) Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > To: nimrod-wg@BBN.COM > Subject: on interfaces as nodes in the nimrod map > Reply-To: dab=replies@epilogue.com > Date: Mon, 8 Nov 93 18:26:05 EST > From: dab@epilogue.com > > I've had some more thoughts on what nimrod maps ought to look like and > why I liked Martha's suggestion to make the nodes be interfaces and > areas. All the other suggestions had areas as nodes and interfaces as > links. This is certainly how we tend to draw network maps. But in > the abstraction heirarchy, interfaces are simply a degenerate case of > areas. Leads one to think that perhaps they really ought to appear in > the map the same way. It's easy to make interfaces links but not > areas, therefore interfaces ought to be nodes too. > > The problem with interfaces as nodes is that you get O(n^2) links > between the interfaces on a network and O(n^2) links between the > interfaces on a router. There're two ways to fix this. > > One way is to add concentrator nodes to the map. These exist purely > for the purpose of reducing O(n^2) connectivity to O(n). They make > the map a little simpler. They would sit in the topology where a > router or multidrop network exists. They would not have names. Lots > of people will hop up here and say that the concentrator node sitting > in for the network could have the name of that network's area. It > could but it's not necessary. Don't equate networks and areas just > yet because I don't think they're necessarily going to be the same. > > The other fix for O(n^2) is to hide it all in an area. Make an area > for the network and then it's reduced to a single node, the complexity > isn't seen outside, so it doesn't matter. You can do this for either > a network or a router but if you do both you get lots of overlapping > areas. > > [Can you have an area with no interfaces in it? Then you could show > the router as one of these and put all the interfaces in the area for > the network. I don't think this makes sense but it's something to > think about some night when you can't get to sleep.] > > But why make the first level (counting up) area stop when it hits a > router? The size of an area is limited by the number of destinations > the area routing can handle. For routing technology today this is in > the 1E4 to 1E5 range. That's how many interfaces can be in a first > level area. > > Dave > -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa24531; 9 Nov 93 12:40 EST Received: from pizza by PIZZA.BBN.COM id aa24406; 9 Nov 93 12:21 EST Received: from BBN.COM by PIZZA.BBN.COM id aa24402; 9 Nov 93 12:19 EST Received: from quern.epilogue.com by BBN.COM id aa22753; 9 Nov 93 12:18 EST To: kasten@ftp.com CC: nimrod-wg@BBN.COM In-reply-to: Frank Kastenholz's message of Tue, 9 Nov 93 09:44:58 -0500 <9311091444.AA01008@ftp.com> Subject: on interfaces as nodes in the nimrod map Reply-to: dab=replies@epilogue.com Date: Tue, 9 Nov 93 12:18:48 EST From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9311091218.aa19036@quern.epilogue.com> Date: Tue, 9 Nov 93 09:44:58 -0500 From: Frank Kastenholz Reply-To: kasten@ftp.com What is the advantage of having the interfaces be the graph nodes? It might work? Whenever I've really tried to draw up how the maps are going to work I've gotten stuck. I now think the problem was that areas, which are aggregations of interfaces, were not represented the same as interfaces even though otherwise they should be used the same way. I don't know that this fixes it and it does bring it's own set of problems. But it lets us stay true to the definitions we set up. The other possibility is to change some of those definitions. I'll expend on this more in my next message. I can see several disadvantages, but no advantages. Dave and Noel both pointed out the disadvantages so I won't belabor the point. I thought I was pointing out advantages. Oh well. For packets to move through nimrod-net, the first-hop router will find the lowest area (root being highest) in its maps that matches the ...... part of the destination locator. This router then "does the right thing" to forward the packet on -- it might be hop-by-hop, where each hop represents an area in the map, or it might be source-routed, where each element in the source-route is an area. Remember, this is no-brainer routing. You're routing only along the naming abstraction heirarchy. This is also what we realized would happen if you did the nimrod loose source route routing and simply specified the destination as the only hop in the loose source route. In other words, this is nimrod's hop-by-hop routing. I sincerely hope that we'll field good enough nimrod route selection code that no-one uses this just because it seems so easy. This brings me to another point, it seems to me that routing within an area should be solely a matter for that area to worry about. It could do this but since we're talking about source selected routing, the source also has the option of pinning the route down at certain points along the way. (The constraints and policies could be tagged onto each packet or they could be part of a flow, which the packet associates itself with via a flow-id, or it could be algorithmically determined by examination of the packet's data, or it could be by magic -- I don't care.) I'd claim that this is better left to the source to decide. I think you'll have troubles if you think you can define today all the possible policies you might want to represent. The problem is that the core of nimrod is going to be fielded everywhere and therefore difficult to change. The more we can throw out of this core, the more flexibility we'll have later to chagne those parts when we find we don't know everything yet. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa26412; 9 Nov 93 13:12 EST Received: from pizza by PIZZA.BBN.COM id aa24613; 9 Nov 93 12:54 EST Received: from BBN.COM by PIZZA.BBN.COM id aa24609; 9 Nov 93 12:52 EST Received: from quern.epilogue.com by BBN.COM id aa25240; 9 Nov 93 12:53 EST To: jnc@ginger.lcs.mit.edu CC: nimrod-wg@BBN.COM In-reply-to: Noel Chiappa's message of Tue, 9 Nov 93 01:21:11 -0500 <9311090621.AA07041@ginger.lcs.mit.edu> Subject: on interfaces as nodes in the nimrod map Reply-to: dab=replies@epilogue.com Date: Tue, 9 Nov 93 12:52:11 EST From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9311091252.aa19211@quern.epilogue.com> Date: Tue, 9 Nov 93 01:21:11 -0500 From: Noel Chiappa This is certainly how we tend to draw network maps. I think that the fact that we tend to draw maps in that way should not be whizzed past so quickly. This bumped me just enough to go off in yet another direction; not a different topic, just a change to our starting point that might work better. Maybe this is nimrod heresy but I suggest that areas should be aggregations of endpoints instead of interfaces. Sometime back Noel, I remember you saying that we route to interfaces. I think we route via interfaces but I'm interested in routing _to_ an endpoint. So the leaves of the locator tree would now be endpoints. Interfaces would be arcs on the map but probably don't need names of their own. Another thought on why whatever it is that we aggregate into areas ought to be represented as nodes in the map. If you don't then area boundries must cut nodes in half. Dave   Received: from ABERI.BBN.COM by BBN.COM id aa03460; 9 Nov 93 15:04 EST Received: from pizza by PIZZA.BBN.COM id aa00981; 9 Nov 93 13:36 EST To: nimrod-wg@BBN.COM Subject: Re: on interfaces as nodes in the nimrod map In-reply-to: Your message of Tue, 09 Nov 93 12:52:11 -0500. <9311091252.aa19211@quern.epilogue.com> Date: Tue, 09 Nov 93 14:33:41 -0500 From: Ram Ramanathan >Maybe this is nimrod heresy but I suggest that areas should be >aggregations of endpoints instead of interfaces. Sometime back Noel, >I remember you saying that we route to interfaces. I think we route >via interfaces but I'm interested in routing _to_ an endpoint. >So the leaves of the locator tree would now be endpoints. Interfaces >would be arcs on the map but probably don't need names of their own. >Another thought on why whatever it is that we aggregate into areas >ought to be represented as nodes in the map. If you don't then area >boundries must cut nodes in half. Funny you should mention this too - it had occurred to me as a method of uniformity over the entire range and hence a method of unsurpassed elegance :-). But we should remember that endpoints can be mobile whereas interfaces cannot. Furthermore, the basic entity aggregated in the clusters should probably be something that is routable to within the Nimrod scope. Does Nimrod concern itself with routing to a router and let the local machinery worry about getting it to the endpoint or not? I thought the former, but maybe only I am unanimous in this. Nevertheless, I am not saying that because the endpoint is mobile, we cannot have it as the basic entitiy - why, clusters can move, and when they move, we would make a change in the hierarchical structure. We *could* let the hierarchy change itself whenever an endpoint moves, but that is probably not practical. - Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa04676; 9 Nov 93 15:23 EST Received: from pizza by PIZZA.BBN.COM id aa25381; 9 Nov 93 15:02 EST Received: from BBN.COM by PIZZA.BBN.COM id aa25377; 9 Nov 93 14:58 EST Received: from quern.epilogue.com by BBN.COM id aa03131; 9 Nov 93 14:59 EST To: nimrod-wg@BBN.COM In-reply-to: Ram Ramanathan's message of Tue, 09 Nov 93 14:33:41 -0500 <9311091449.aa20266@quern.epilogue.com> Subject: on interfaces as nodes in the nimrod map Reply-to: dab=replies@epilogue.com Date: Tue, 9 Nov 93 14:58:14 EST From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9311091458.aa20314@quern.epilogue.com> Date: Tue, 09 Nov 93 14:33:41 -0500 From: Ram Ramanathan But we should remember that endpoints can be mobile whereas interfaces cannot. You're right but I don't think it's a problem. If we aggregate interfaces then we have the following: an endpoint is related to one or more interfaces named by interface locators. Nimrod routes on these locators. If the endpoint moves then it gets new interface locators that reflect its new location in the heirarchy. If we aggregate endpoints then endpoints have both an endpoint-ID and an endpoint locator. The endpoint locator is the heirarchical endpoint name and comes from the abstraction heirarchy. If the endpoint moves its locator changes, just like the interface locator changes in the other scheme. I think the two approaches are nearly identical on this point. Dave   Received: from ABERI.BBN.COM by BBN.COM id aa14336; 9 Nov 93 17:27 EST Received: from pizza by PIZZA.BBN.COM id aa01031; 9 Nov 93 16:09 EST To: nimrod-wg@BBN.COM Subject: Re: on interfaces as nodes in the nimrod map In-reply-to: Your message of Tue, 09 Nov 93 14:58:14 -0500. <9311091458.aa20314@quern.epilogue.com> Date: Tue, 09 Nov 93 17:06:57 -0500 From: Ram Ramanathan >If we aggregate endpoints then endpoints have both an endpoint-ID and >an endpoint locator. The endpoint locator is the heirarchical >endpoint name and comes from the abstraction heirarchy. If the >endpoint moves its locator changes, just like the interface locator >changes in the other scheme. I think the two approaches are nearly >identical on this point. Theoretically, I agree they are both very similar, but practically, I think they are different. Moving a cluster from one place in the hierarchy to another induces two kinds of overheads : - Reconfiguration overhead. We need to ask somebody for a new name for that cluster (this would have to be manual), note it with DNS (could be automated) etc. - Routing advertisement overhead. We need to tell people how to get to the cluster now that it has moved. In your suggestion of endpoints as degenerate clusters, mobile endpoints would require the above changes on *each* move. While the routing overhead above may not be substantial (an equivalent thereof in *some* form or the other has to be done anyway, because, hey, stuff has *moved*), the reconfiguration overhead is probably not acceptable. Presumably, with *interfaces* as degenerate clusters and an EID to interface mapping, if endpoint e moves from A.B.C to X.Y.Z, we simply need to send a message to DNS changing the association. Or do the equivalent using forwarding etc. in the context of some "mobility handling system". No manual interference is needed on the move. All the three choices for the basic indecmposable clustering entity, namely, router, interface and endpoint make sense. There are tradeoffs of each of the three. At some point, I guess we (rather I) should make a list of tradeoffs and put it out for general perusal.... - Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa24733; 10 Nov 93 10:56 EST Received: from pizza by PIZZA.BBN.COM id aa29985; 10 Nov 93 10:14 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa29981; 10 Nov 93 10:12 EST Date: Wed, 10 Nov 93 10:14:20 EST From: Martha Steenstrup To: nimrod-wg@BBN.COM Subject: stuff Hi guys, Here is one view of clustering in an internetwork. (In fact, it is only a partial view - lots of things are not addressed here.) Fire away. 1 Definitions A "cluster" is a collection of one or more "interfaces" together with the "arcs" that connect the interfaces, such that a "path", consisting of interfaces and connecting arcs, between any two interfaces in the cluster lies entirely within that cluster. Clusters may be stationary or mobile. (I will send a separate note about mobility.) An "interface" is a point of connection between a router or a host and the communications medium (e.g., a network or a point-to-point link). An "arc" represents that portion of an external communications medium (e.g., a network or point-to-point link) or an internal communications medium (e.g., a router's backplane) that connects two interfaces. Arcs and interfaces are not decomposable into smaller components. They are the elemental building blocks of clusters. A single interface is a cluster; in fact, it is a nondecomposable cluster. New clusters may be built by clustering together other clusters. One may choose to cluster interfaces according to relationships between them, such as "administered by the same authority", or according to an objective function, such as "minimize the expected amount of routing information known globally". 2 HIerarchies Repeating clustering of interfaces may produce a hierarchy of clusters, such that clusters are disjoint, nested, or may be overlapping. (I will send a separate note about overlapping clusters.) Moreover, different portions of the hierarchy may be formed using different clustering criteria. A router can be viewed as a cluster, and a network can be viewed as a cluster. The internal organization of a router can be viewed as a complete graph of its interfaces and connecting arcs, and the internal organization of a network can be viewed as a complete graph of its interfaces and connecting arcs. However, from the perspective of other clusters within the same parent cluster, the router or the network may be viewed as a single cluster with no detail about its interior. When drawing pictures of an internetwork and the clustering imposed over the internetwork, one may draw the topology in terms of the routers, links, networks, and hosts and then may superimpose the clustering over the set of interfaces. Both views of the internetwork are useful and should be retained. However, the clustering criteria may not always make cluster boundaries that fall neatly on router, network, link, or host boundaries. As long as one knows the mapping between the physical entities and the virtual cluster entities, there should not be a problem in maintaining multiple views of an internetwork. 3 Connecting points and virtual links Each cluster has a set of "connecting points" through which it can communicate with neighboring clusters. A connecting point is a collection of one or more interfaces within the cluster such that each interface in a connecting point has an arc to at least one interface in a connecting point in a given neighboring cluster. A pair of connecting points is directly connected through a set of "virtual links". A virtual link is a collection of one or more arcs between the set of interfaces that constitute connecting points. Two neighboring clusters are directly connected by one or more "external" (or inter-cluster) virtual links. Two connecting points within the same cluster are directly connected by one or more "internal" (or intra-cluster) virtual links. Each internal and external virtual link has associated with it a set of attributes describing the qualities of service (e.g. delay, loss, throughput) together with the restrictions on access to the service (e.g. according to time of day, source and destination of traffic, ability to pay, type of traffic). Each connecting point enforces the access restrictions applied to the services associated with the virtual links that terminate at that connecting point. Each cluster has a local label that is unique within the set of labels for its sibling clusters. Each cluster has a globally unique label formed by the concatenation of the labels of all of its ancestor clusters and itself, beginning with the universal cluster that contains all other clusters. There are many ways of labelling virtual links. The following is useful if one considers a route to be an alternating sequence of external and internal virtual links, E_1 I_2, E_2, ... , E_n-1, I_n, E_n. Each external virtual link has a local label that is unique within the set of labels for virtual links connecting the two neighboring clusters. Each external virtual link has a globally unique label formed by the concatenation of the labels of the two neighboring clusters, the virtual link local label, and the attributes of the link. Each internal virtual link has a globally unique label formed by the concatenation of the labels of the external virtual links it connects and the attributes of the link. Hence, an external virtual link between clusters C_i and C_i+1 may be defined by the parameters C_i, C_i+1, L_i , and E_i_attr. L_1 is the local label for the external virtual link and E_i_attr are its attributes. An internal virtual link across cluster C_i connecting two external virtual links is defined by the parameters C_i-1, L_i-1, C_i, I_i_attr, L_i, and C_i+1. I_i_attr are the attributes of the internal virtual link. Thus, a route among sibling clusters could be conpactly represented as a sequence of alternating internal and external virtual links labelled as: (C_1) (L_1, E_1_attr) (C_2, I_2_attr) (L_2, E_2_attr) ... (C_n). If the attributes of all of the internal virtual links within a cluster are identical, the cluster need not advertise each internal virtual link separately. Rather, one may use a compact representation listing the attributes of one (and therefore all) internal virtual links and the set of external virtual links connected by internal virtual links with these attributes. Thus, using interfaces as the entities to cluster does not by itself increase the amount of routing information about a cluster. What determines the amount of routing information are the number of different virtual links across a cluster. Using interfaces as the elemental clusters permits one to define a router or a network with different criteria for moving between incoming and outgoing interfaces on the router or the network and hence different internal virtual links. These would be useful for controlling access to certain services from certain entry points. 4 Endpoints Endpoints are instantiations of applications and may be mobile. They are associated with but not contained within clusters. Hence, an endpoint currently residing in a host with two interfaces, one on network A and one on network B, is automatically associated with two different clusters: the interface to network A and the interface to network B. Each endpoint has a globally unique identifier. At a given point in time, one may locate an endpoint according to the labels of the cluster(s) with which it is associated. TIme to stop. More later. m   Received: from PIZZA.BBN.COM by BBN.COM id aa00750; 10 Nov 93 12:27 EST Received: from pizza by PIZZA.BBN.COM id aa00649; 10 Nov 93 11:57 EST Received: from BBN.COM by PIZZA.BBN.COM id aa00645; 10 Nov 93 11:55 EST Received: from quern.epilogue.com by BBN.COM id aa28613; 10 Nov 93 11:56 EST To: ramanath@BBN.COM CC: nimrod-wg@BBN.COM In-reply-to: Ram Ramanathan's message of Tue, 09 Nov 93 17:06:57 -0500 <9311091717.aa21186@quern.epilogue.com> Subject: on interfaces as nodes in the nimrod map Reply-to: dab=replies@epilogue.com Date: Wed, 10 Nov 93 11:55:53 EST From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9311101155.aa07462@quern.epilogue.com> Date: Tue, 09 Nov 93 17:06:57 -0500 From: Ram Ramanathan Presumably, with *interfaces* as degenerate clusters and an EID to interface mapping, if endpoint e moves from A.B.C to X.Y.Z, we simply need to send a message to DNS changing the association. Or do the equivalent using forwarding etc. in the context of some "mobility handling system". No manual interference is needed on the move. I'm afraid I'm not seeing something. This is identical to how I'd expect it to happen with endpoints as degenerate clusters too. I'd still have EIDs; they'd just map to an endpoint locator instead of an interface locator. You'd send the same mesage to DNS changing the association... Except of course we talked about how DNS probably can't be the naming system for nimrod so whatever naming system we invent in its place. God, that makes me uneasy, but it's a topic for another day. All the three choices for the basic indecmposable clustering entity, namely, router, interface and endpoint make sense. There are tradeoffs of each of the three. At some point, I guess we (rather I) should make a list of tradeoffs and put it out for general perusal.... My biggest concern with clustering endpoints is that I think I'm going to end up wanting to put an area boundry through the middle of one. Either that or you have to pick which area an endpoint lives in when there's not always an easy answer. With interface clustering, a router has multiple interfaces and each interface can go in the right area. You're almost never tempted to put an interface into two areas. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa08013; 10 Nov 93 14:38 EST Received: from pizza by PIZZA.BBN.COM id aa01580; 10 Nov 93 14:09 EST Received: from BBN.COM by PIZZA.BBN.COM id aa01576; 10 Nov 93 14:04 EST Received: from ftp.com by BBN.COM id aa06022; 10 Nov 93 14:04 EST Received: by ftp.com id AA17164; Wed, 10 Nov 93 14:03:18 -0500 Date: Wed, 10 Nov 93 14:03:18 -0500 Message-Id: <9311101903.AA17164@ftp.com> To: ramanath@BBN.COM Subject: Re: on interfaces as nodes in the nimrod map From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > Does Nimrod concern itself with routing to a router and let the local > machinery worry about getting it to the endpoint or not? I thought the > former, but maybe only I am unanimous in this. I would say that it routes to the final area/cluster/aggregate and then local technology (i.e. within the area and limited in scope to the area) takes over. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa08179; 10 Nov 93 14:40 EST Received: from pizza by PIZZA.BBN.COM id aa01587; 10 Nov 93 14:09 EST Received: from BBN.COM by PIZZA.BBN.COM id aa01583; 10 Nov 93 14:05 EST Received: from ftp.com by BBN.COM id aa06039; 10 Nov 93 14:05 EST Received: by ftp.com id AA17169; Wed, 10 Nov 93 14:03:21 -0500 Date: Wed, 10 Nov 93 14:03:21 -0500 Message-Id: <9311101903.AA17169@ftp.com> To: dab=replies@epilogue.com Subject: Re: on interfaces as nodes in the nimrod map From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > You're right but I don't think it's a problem. If we aggregate > interfaces then we have the following: an endpoint is related to one > or more interfaces named by interface locators. Nimrod routes on > these locators. If the endpoint moves then it gets new interface > locators that reflect its new location in the heirarchy. yeah. in the real world, it probably means that i've taken my laptop from here and moved it (along with its interface card) to there. as a result, the interface gets a new locator. if interfaces are the lowest form of areas/clusters/aggregates then that sort of implies that areas/clusters/aggregates can get new locators -- (just to take something to its logical extreme -- this means that subnets, networks, regionals, backbones, planets, star-systems, galaxies, herds, and universes can all be mobile in the network sense). > If we aggregate endpoints then endpoints have both an endpoint-ID and > an endpoint locator. The endpoint locator is the heirarchical > endpoint name and comes from the abstraction heirarchy. If the > endpoint moves its locator changes, just like the interface locator > changes in the other scheme. I think the two approaches are nearly > identical on this point. and -- for something like a router or a multi-homed host, this implies that all locators, for all interfaces, apply to the same endpoint. no? -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa08216; 10 Nov 93 14:41 EST Received: from pizza by PIZZA.BBN.COM id aa01593; 10 Nov 93 14:09 EST Received: from BBN.COM by PIZZA.BBN.COM id aa01589; 10 Nov 93 14:05 EST Received: from ftp.com by BBN.COM id aa06042; 10 Nov 93 14:05 EST Received: by ftp.com id AA17172; Wed, 10 Nov 93 14:03:25 -0500 Date: Wed, 10 Nov 93 14:03:25 -0500 Message-Id: <9311101903.AA17172@ftp.com> To: ramanath@BBN.COM Subject: Re: on interfaces as nodes in the nimrod map From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > Theoretically, I agree they are both very similar, but practically, I > think they are different. > > Moving a cluster from one place in the hierarchy to another induces > two kinds of overheads : > > - Reconfiguration overhead. We need to ask somebody for a new name for > that cluster (this would have to be manual), note it with DNS (could > be automated) etc. > > - Routing advertisement overhead. We need to tell people how to get to > the cluster now that it has moved. i see this as "simply" changing/adding/deleting a number from a routing advertisement. i think that it is exactly the same routing overhead as moving a host. (however it is done for a host) the routing adverts are changed to say that this thing is now over there. i do not know enough about how mobile-host stuff is done -- perhaps a m-h person should chime in here... if clusters are opaque to all higher-level clusters that contain them and to all lower-level clusters that they contain (i.e if cluster A contains cluster B which contains cluster C, A and C have no knowledge of the internal structure of B). sources of traffic within a cluster (e.g. contained hosts and clusters) could merely know should merely know to send traffic "that way (with appropriate policy)" to get something outside of the cluster. this opacity means that if a cluster moves then all the stuff inside of it need not be told. the contained stuff certainly need not know the new 'external' topology. contained things might have to get new locator prefixes (e.g. if ftp moves to europe, it would change from nsfnet.nearnet.ftp to ebone.ftp) but this could be done in the same manner that clusters (and contained things) learn the prefix initially. in this case, i see an advertisement from a containing cluster into the contained cluster (recursively, ending at interfaces) that says "your locator prefix is xxxxxxxx", > Presumably, with *interfaces* as degenerate clusters and an EID to interface > mapping, if endpoint e moves from A.B.C to X.Y.Z, we simply need to send > a message to DNS changing the association. Or do the equivalent using > forwarding etc. in the context of some "mobility handling system". No > manual interference is needed on the move. what about active sessions? i know of a company that makes cellular radio networks for use in buildings -- they are used in things like large stores, warehouses, and the like. people have small computers with radios that are connected to this cellular network. as these people wander through the store (taking inventory or whatever) they will move from one cell to another. i know that they want the ability to have cells be individual subnets (in ipv4 parlance). this means that the person would have an active session to the store's central computer that has to be kept open while that person wanders through the store doing their job -- this person could even be at the border of two subnet/cells and moving back and forth; one packet is from subnet a, the next from subnet b and back and forth and back and forth. this can not be done via fondling dns. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa08528; 10 Nov 93 14:48 EST Received: from pizza by PIZZA.BBN.COM id aa01599; 10 Nov 93 14:09 EST Received: from BBN.COM by PIZZA.BBN.COM id aa01595; 10 Nov 93 14:05 EST Received: from ftp.com by BBN.COM id aa06049; 10 Nov 93 14:05 EST Received: by ftp.com id AA17177; Wed, 10 Nov 93 14:03:29 -0500 Date: Wed, 10 Nov 93 14:03:29 -0500 Message-Id: <9311101903.AA17177@ftp.com> To: dab=replies@epilogue.com Subject: Re: on interfaces as nodes in the nimrod map From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > For packets to move through nimrod-net, the first-hop router will > find the lowest area (root being highest) in its maps that matches > the ...... part of the > destination locator. This router then "does the right thing" to > forward the packet on -- it might be hop-by-hop, where each hop > represents an area in the map, or it might be source-routed, where > each element in the source-route is an area. > > Remember, this is no-brainer routing. You're routing only along the > naming abstraction heirarchy. This is also what we realized would > happen if you did the nimrod loose source route routing and simply > specified the destination as the only hop in the loose source route. > In other words, this is nimrod's hop-by-hop routing. I sincerely hope > that we'll field good enough nimrod route selection code that no-one > uses this just because it seems so easy. first, yes, i realize that it is no-brainer. second, i believe (with no facts to back me up) that this will be a majority of the cases on the network -- in other words, no-brainer might be adequate for (eg) 80% of the cases. third, i see no-brainer as the 'default' route, to be used only if some point along the path can not get a better route. if an area along the way can see a "better" route than the n.b. route, let that area take the responsibility of using that better route. for instance; if someone in a.b.c.d wants to send a packet to a.e.f.g, the nb route is to send the packet all the way up the hierarchy and then all the way back down. however, if when the packet gets to b, b finds that it has a direct route to g (say a private b-g link that is unadvertised to the rest of the world) then let b take the responsibility of sending the packet directly to g. this, of course, is ignoring policy concerns. > This brings me to another point, it seems to me that routing within > an area should be solely a matter for that area to worry about. > > It could do this but since we're talking about source selected > routing, the source also has the option of pinning the route down at > certain points along the way. but why mandate that the source MUST select the route? in the example in the previous paragraph, if the originator wishes to select a route (at whatever granularity that is made available in the routing information available to the originator) then it should be able to. however, the originator should also be able to hand the packet to the network and say "deal with it". in the current internet we have waffled back and forth between having hosts know about routing or not (e.g. should they listen to rip packets, etc). it seems to me that we pretty much believe that unless a host has a particular reason to be concerned with routing, it should not know about routing. this allows network infrastructure (i.e. routers) to change, be upgraded, change protocols, etc, without requiring that hosts be changed as well. it seems to me that this is a good principle that should be carried into nimrod. > (The constraints and policies could be tagged onto each packet or > they could be part of a flow, which the packet associates itself > with via a flow-id, or it could be algorithmically determined by > examination of the packet's data, or it could be by magic -- I > don't care.) > > I'd claim that this is better left to the source to decide. I think > you'll have troubles if you think you can define today all the > possible policies you might want to represent. The problem is that regardless, i still need a common representation of the policies, whatever they may be. hosts (or route servers) will need to examine route maps, or query routers, as to what policies they support and the routers must answer in the same 'language'. any particular policy must be represented by the same bits on the wire* throughout an internet. so, the bits that are put in packets that name policies must be the same bits, regardless of the mechanism for getting policy information around the network, and how that information is selected. these bits could be as tags in a datapacket that say "please route this packet according to these policies" or they could be in a flow-setup packet that say "please associate flow 'x' with these policies" or they could be in a "do you support these policies" query packet or they could be in a "i support these policies" advertisement packet. * -- unless you do area-hop-by-hop routing and translate policy terms in packets on the fly as they go across area boundaries. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa09345; 10 Nov 93 15:02 EST Received: from pizza by PIZZA.BBN.COM id aa01605; 10 Nov 93 14:09 EST Received: from BBN.COM by PIZZA.BBN.COM id ab01595; 10 Nov 93 14:05 EST Received: from ftp.com by BBN.COM id aa06051; 10 Nov 93 14:05 EST Received: by ftp.com id AA17182; Wed, 10 Nov 93 14:03:32 -0500 Date: Wed, 10 Nov 93 14:03:32 -0500 Message-Id: <9311101903.AA17182@ftp.com> To: msteenst@BBN.COM Subject: Re: stuff From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > 1 Definitions > > A "cluster" is a collection of one or more "interfaces" together with > the "arcs" that connect the interfaces, such that a "path", > consisting of interfaces and connecting arcs, between any two > interfaces in the cluster lies entirely within that cluster. Clusters > may be stationary or mobile. (I will send a separate note about > mobility.) > > An "interface" is a point of connection between a router or a host > and the communications medium (e.g., a network or a point-to-point > link). > > An "arc" represents that portion of an external communications medium > (e.g., a network or point-to-point link) or an internal > communications medium (e.g., a router's backplane) that connects two > interfaces. > > Arcs and interfaces are not decomposable into smaller components. > They are the elemental building blocks of clusters. > > A single interface is a cluster; in fact, it is a nondecomposable > cluster. New clusters may be built by clustering together other > clusters. One may choose to cluster interfaces according to > relationships between them, such as "administered by the same > authority", or according to an objective function, such as "minimize > the expected amount of routing information known globally". if i read this properly, it says that the interface-cluster will (or perhaps can) exist in at least two higher-level clusters. one such cluster will be the collection of interfaces connected to the same physical network, and the other will be the set of interfaces on the same router. this leads to an interesting thought: if interfaces a, b, c, d, e, f, and g are all on the same physical network (or router), can i make clusters containing just {a, b, c} and {d, e, f, g}? can i make clusters {a,b,c,d} and {d,e,f,g} (i.e. with overlap)? would this be (very roughly) like having multiple ipv4 subnets on a single medium today? > Repeating clustering of interfaces may produce a hierarchy of > clusters, such that clusters are disjoint, nested, or may be > overlapping. (I will send a separate note about overlapping > clusters.) Moreover, different portions of the hierarchy may be > formed using different clustering criteria. does this answer my interesting thought? i think so. > > A router can be viewed as a cluster, and a network can be viewed as a > cluster. The internal organization of a router can be viewed as a > complete graph of its interfaces and connecting arcs, and the > internal organization of a network can be viewed as a complete graph > of its interfaces and connecting arcs. However, from the perspective > of other clusters within the same parent cluster, the router or the > network may be viewed as a single cluster with no detail about its > interior. > > When drawing pictures of an internetwork and the clustering imposed > over the internetwork, one may draw the topology in terms of the > routers, links, networks, and hosts and then may superimpose the > clustering over the set of interfaces. Both views of the internetwork > are useful and should be retained. However, the clustering criteria > may not always make cluster boundaries that fall neatly on router, > network, link, or host boundaries. As long as one knows the mapping > between the physical entities and the virtual cluster entities, there > should not be a problem in maintaining multiple views of an > internetwork. > > 3 Connecting points and virtual links > > Each cluster has a set of "connecting points" through which it can > communicate with neighboring clusters. A connecting point is a > collection of one or more interfaces within the cluster such that > each interface in a connecting point has an arc to at least one > interface in a connecting point in a given neighboring cluster. this c/would be drawn as: ==================+ +=================== | | *--------------------* | | Cluster x | | Cluster y | | *--------------------* | | ==================+ +=================== (* -> Connecting Point) this sort of says that cluster boundaries fall on networks or on router backplanes? > A pair of connecting points is directly connected through a set of > "virtual links". A virtual link is a collection of one or more arcs > between the set of interfaces that constitute connecting points. Two > neighboring clusters are directly connected by one or more "external" > (or inter-cluster) virtual links. Two connecting points within the > same cluster are directly connected by one or more "internal" (or > intra-cluster) virtual links. so, i guess that this says that conecting points are nodes of the graph and arcs are cp->cp links. clusters can also be represented as nodes on the graph, yes? if so, then all external virtual links which connect to the cluster would connect to the graph-node by which the cluster is represented. > There are many ways of labelling virtual links. The following is > useful if one considers a route to be an alternating sequence of > external and internal virtual links, E_1 I_2, E_2, ... , E_n-1, I_n, > E_n. internal virtual links are in effect synonyms for the set of policies that a cluster should follow when moving the packet across that cluster. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa14295; 10 Nov 93 15:57 EST Received: from pizza by PIZZA.BBN.COM id aa02031; 10 Nov 93 15:22 EST Received: from BBN.COM by PIZZA.BBN.COM id aa02027; 10 Nov 93 15:19 EST Received: from quern.epilogue.com by BBN.COM id aa10229; 10 Nov 93 15:16 EST To: nimrod-wg@BBN.COM Subject: open issues Reply-to: dab=replies@epilogue.com Date: Wed, 10 Nov 93 15:15:47 EST From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9311101515.aa08772@quern.epilogue.com> Noel said send new open issues to the list so here are a few more that came up in hallway discussions in Houston. - Destination policy and preference. Nimrod's architecture give the source wide latitude in how it decides which route's fit its model of what's a good way to go. Some way to let the destination in on this selection process would be nice. I say "preference" as well because it's easy to forget in talking about policy that sometimes a policy may be more complicated than just a "go"/"no go" decision. - Are policy identifiers purely global? This is Noel's wording. I'd state it more generally as we need to talk about how policies are named. - Nimrod requirements of the network layer. I don't know that this one really wants to get added to the open issues list. It may become important depending on what happens on the IPng front. If we're not given the option of staying with IPv4 until we know what we're doing, we might want to be able to come up with a wish list that we can give to the folks who are building us a new network layer packet header that says what nimrod needs of them. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa29921; 12 Nov 93 17:06 EST Received: from pizza by PIZZA.BBN.COM id aa13505; 12 Nov 93 16:42 EST Received: from BBN.COM by PIZZA.BBN.COM id aa13501; 12 Nov 93 16:38 EST Received: from quern.epilogue.com by BBN.COM id aa26813; 12 Nov 93 16:39 EST To: kasten@ftp.com CC: nimrod-wg@BBN.COM In-reply-to: Frank Kastenholz's message of Wed, 10 Nov 93 14:03:29 -0500 <9311101903.AA17177@ftp.com> Subject: on interfaces as nodes in the nimrod map Reply-to: dab=replies@epilogue.com Date: Fri, 12 Nov 93 16:39:04 EST From: dab@epilogue.com Sender: dab@epilogue.com Message-ID: <9311121639.aa17911@quern.epilogue.com> Date: Wed, 10 Nov 93 14:03:29 -0500 From: Frank Kastenholz Reply-To: kasten@ftp.com first, yes, i realize that it is no-brainer. second, i believe (with no facts to back me up) that this will be a majority of the cases on the network -- in other words, no-brainer might be adequate for (eg) 80% of the cases. This is something that I fear; I think it would be a bad idea to let no-brainer routing become the default. That's why I worded my statement as "I hope we get good enough code written and out there from the start that no-one is ever tempted to use no-brainer routing". this, of course, is ignoring policy concerns. Enough policy in the net could be the driving force that ensures we don't have anyone in the net who can't do better than no-brainer routing. Sheesh, I hate to encourage more policy nonsense. but why mandate that the source MUST select the route? in the example in the previous paragraph, if the originator wishes to select a route (at whatever granularity that is made available in the routing information available to the originator) then it should be able to. however, the originator should also be able to hand the packet to the network and say "deal with it". Yes and no. I'd expect the host that feels this way to say to the local route server, "hey you, give me a route to here", and get back something it just stuffs in the packets and it's happy. This implicitly will pick up the site's policy requirements because the route server is maintained by the site network administrator. The IPv4 transition will have the first hop router doing this on behalf of ignorant hosts. Maybe that mechanism will continue past transition. regardless, i still need a common representation of the policies, whatever they may be. hosts (or route servers) will need to examine route maps, or query routers, as to what policies they support and the routers must answer in the same 'language'. any particular policy must be represented by the same bits on the wire* throughout an internet. I agree that the map format needs to be specified by the core nimrod. This will have to include a way to tag policies on some part of the map (once we figure out what that looks like) therefore it needs a way to name those policies. Whether this implies you have to be able to machine read the policy is an open question I claim. And a discussion for another time. Dave   Received: from PIZZA.BBN.COM by BBN.COM id aa14690; 15 Nov 93 11:09 EST Received: from pizza by PIZZA.BBN.COM id aa26315; 15 Nov 93 10:43 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa26311; 15 Nov 93 10:38 EST To: nimrod-wg@BBN.COM Subject: Paperman (flimsier than a strawman) Date: Mon, 15 Nov 93 10:38:32 -0500 From: Isidro Castineyra Paperman for Nimrod Isidro Castineyra November 15, 1993 1 Introduction This is an attempt to come up with the most basic---read primitive---version of Nimrod I can think of. I wanted to have something we can criticize and use as initial point of reference. It is most certainly full of holes and bad ideas. I am explicitly not claiming that the design decisions taken here would be part of any instantiation of Nimrod. That is, this is not supposed to be a minimal design, only a simple one. I have numbered the stuff below so you can refer easily to whatever irks you. Please start shooting. 2 Basic Principles 1. There are two kinds of basic objects: endpoints and entities. 2. An endpoint is a fate-sharing region---e.g., a process. 3. Each endpoint has an endpoint-ID (eid). Eids are globally unique 4. There are two kind of entities: interfaces and arcs. 5. An interface corresponds to a physical interface. 6. An arc is a configured entity. It can consist of several physical links. 3 Clusters 1. A cluster is defined by its contents and its structure. 1 2. The contents of a cluster is the set of all entities enveloped by the cluster. 3. Between any two entities in the contents of a cluster there should be a path that connects them, such that this path is entirely within the cluster. 4. A cluster is organized in sub-clusters. The sub-clusters split among themselves the contents of the parent cluster. That is, the union of the contents of the sub-clusters is the contents of the parent cluster; also, the contents of the sub-clusters are non-overlapping. Each sub-cluster has also its own structure. 5. A cluster can be organized in many different ways. Each structure has a name. The different structures of a cluster are numbered. The number of bits necessary to number the structures of a cluster will probably be small---certainly, no more than eight. 6. There exists a top-level universal cluster U. U is unique. One cannot build clusters above or at the same level of it. 7. Other than the universal cluster U, all clusters belong to another cluster. A cluster belongs only to one other cluster: its owner. 8. Given an structure, the set of clusters that are owned by a given cluster are referred to as the members of that cluster. 9. Not every cluster makes practical sense, though. For example, a cluster that uses internally vanilla IP for its routing should not be separated into different clusters because there is no way to enforce nimrod-style routing. Specifically, if A is such a cluster, see figure 1, once a packet has entered A, one cannot actually enforce routing directives that request that a path enter cluster A at the entry point marked and also not touch elements in A2. 4 Naming 1. Clusters have names that are not necessarily globally unique. In a given structure of a cluster, two different clusters owned by the same parent cluster cannot have the same name. In this paper, Here, cluster names are ascii strings. 2. A cluster's locator is given by a dot separated series of pairs (cluster name, structure number). This series starts with the universal cluster U and satisfies that every cluster is a member of the preceding one in the given structure. In figure 2, cluster C is owned by cluster B, which is owned by cluster A, which is owned by U, nobody owns U. 2 A ****************************************************** * A1 * * ******************** A2 * * * * ******************** * * * X destination* * * * ......entry>* * * * * * * * * * * * * ******************** * * * * ******************** * * ^ Boundaries not enforceable* * * ******************************************************* Figure 1: Example of Bad Clustering 3. An end-point can have more than one locator. The number of locators depends on how the enclosing clusters are organized. DNS:TNG provides a mapping service between endpoints and their locators. U,2 / \ / \ / ? A,1 / \ B,1 ? / \ C,1 ? Figure 2: Example of Hierarchy 5 Maps 1. With every structure of a cluster there is associated an internal map. The internal map of a cluster is given by the identities of the clusters that belong to it, and by the connections (arcs) between these. 3 2. The physical realization of an advertised arc can vary from arc to arc. For example, while an advertised arc can consist of a single point-to-point link, it also can represent multiple parallel physical links. When there are more than one arc between two given clusters, they are distinguished by assigning to them different numbers. 3. Every element of a map---arc or cluster---can have associated with it multiple attributes. This attributes correspond to things like policy constraints, resource availability, available transit qos parameters, etc. 6 Forwarding 1. Forwarding enforcement is done by routers. A requirement is that the forwarding database that these routers have be in agreement with the maps used to generate the packet headers---i.e., if the header of a packet was generated using an out-of-date map, the routers will not be able to implement that header. These databases must also be consistente with the distributed databases of the DNS:TNG. More on this in the section on reconfiguration, see section 9. 2. If a cluster is organized in more than one way, the routers in the cluster will keep a forwarding database for each structure. 3. A packet contains in its header a loose source route. This route could be as loose as including only a locator for the destination eid, or as tight as defining every nimrod entity through which the packet is required to pass. 4. A packet can also include a flow-id which maps into state that has been previously set-up in routers along the way. Packet forwarding when there is a flow-id established is relatively simple: follow the instructions in the routers state. 5. The loose source route in a packet is the only way to indicate policy within a packet. For example, when the loose source route includes only the destination eid, this means the the source does not care how the packet gets there. 6. The path of a packet that contains a loose source route that partially says, say, ...arc1:clusterA.org#:arc2... is restricted to remain inside cluster A.org# while going from arc1 to arc2. Similarly, the path of a packet whose header ends in arc1:clusterA.org#:eid1 is restricted to remain within A when going from arc1 to the entity assigned eid1. 7. A cluster that does not divulge its internal map can work internally any way it pleases as long as it satisfies its external characterization as given in its nimrod map advertisements. This means 4 that the EGP/IGP split effectively is left in place, sort of. 7 Map Distribution 1. A cluster can request the map of any other cluster/structure. This request may or may not be granted. 2. Internal topologies for a cluster are always given in their entirety. There is no way to obtain only part of the map of a cluster. If desired, the internal map of a component cluster can be requested---recursively, ad follia. A given cluster can always refuse to give its internal structure. 3. Maps have an expiration time. It is assumed that all nimrod routing entities have a common accurate clock---for example, by using NTP. This clock is used to determine the validity of maps. 4. A cluster can also ``subscribe'' to the map of a second cluster. This means that the second cluster will send to it topology updates periodically. The update could be as brief as indicating that the time-out of the last full topology received should be extended until an indicated time. 8 Routing 1. Routing generates the right packet header, or the right instructions to set-up state in intermediate routers, given an origin, a destination and a set of policies and qos requirements. 2. A cluster has a top level map of the cluster it is a member of. It also has a map of the top level of every cluster it is a descendent of. For example, a cluster D with locator U,1.A,1.B,1.C,1.D,1, has top level maps of clusters C.1, B,1, A,1, and U,1. Notice that this implies that there is no information hiding ``downwards,'' that is, a cluster cannot hide its top level map from any of its descendants. It also implies that two clusters that are owned by the same cluster have a consistent view of the network ``upwards.'' 3. A cluster might have other maps according to the availability of the information and its requirements. 4. When a packet header contains only a locator for the destination, we have what we know today as hop-by-hop routing. Routing loops are avoided by forwarding the packet in a direction that minimizes the number of hops to the destination. This is called ``vanilla datagram'' forwarding. Consistency of the maps is required for this to work. 5 9 Reconfiguring The multiple-structure mechanism is used when a cluster needs to reconfigure. Suppose a cluster U,1.A,1.B,1 has become too large and we want to split it. This can be done by creating a new structure for A. In this new structure, U,1.A,2, there are two clusters U,1.A,2.B,1 and U,1.A,2.C,2. The entities previously associated with U,1.A,1.B,1 are split between the two new clusters. 6   Received: from PIZZA.BBN.COM by BBN.COM id aa19524; 15 Nov 93 12:27 EST Received: from pizza by PIZZA.BBN.COM id aa26806; 15 Nov 93 11:54 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa26802; 15 Nov 93 11:51 EST Date: Mon, 15 Nov 93 11:52:39 EST From: Martha Steenstrup To: nimrod-wg@BBN.COM Subject: more stuff Hello, At the recent Nimrod working group meeting during IETF, there was some discussion of "overlapping" clusters and a perception that they were a necessary part of Nimrod. However, at that meeting, it was not clear (at least to me) that all present had a consistent view of what constitutes "overlapping". Some equated "overlapping" with having more than one locator per indecomposable entity, while others did not. The following discussion is an attempt to sort some of this out, using definitions and examples. 1 Assumptions. Nimrod must accommodate an internetwork in which: (a) Two organizations share some network resources. For example, two companies working on a joint project pool some of their network resources to form a network (or set of networks) dedicated to activities for the project. This special network may have access limited to traffic from the two cooperating companies only, while the other portions of the two companies' networks may have no such usage restrictions. (b) Reorganizations occur and not all portions of the internetwork learn about the reorganization at the same time. Hence, multiple views of the same internetwork will coexist for a while. For example, two companies merge into a conglomerate and their networking resources are merged as well. Management of the merged network resources is according to the guidelines of the conglomerate, which may be different than those of the individual companies. Hence, usage restrictions of the merged network may be different from those of the companies' networks prior to merger. During the transition, some portions of the internetwork will have the pre-merger view while others will have the post-merger view. (c) Multiple organizations coexist. For example, one may want to organize an internetwork in two different ways in order to determine which organization generates smaller forwarding information databases. One organization might be along geopolitical boundaries while the other might be along network ownership boundaries. A country may contain many separately administered networks, and a single network may span international boundaries, so neither organization is a finer granularity version of the other. Do you guys think these assumptions are reasonable? (Throughout the remainder of this note, I assume the "interface" model of clustering in Nimrod, as a point of reference.) 2 Definitions. For a given hierarchical clustering of an internetwork, two clusters X and Y are: - Disjoint, if they share no common interfaces in common. That is, the set intersection of X and Y is empty. - Nested, if one completely contains but is not equal to the other. That is, either X is a proper subset of Y or Y is a proper subset of X. - Overlapping, if they share common interfaces but are not nested. That is, the set intersection of X and Y is non-empty, and either - X is equal to Y or - both the set difference of X and Y and the set difference of Y and X are non-empty. Each cluster has a label that is locally unique among the sibling clusters of the parent cluster in which it is nested. Moreover, each cluster has a globally unique label, its locator, formed as an ordered list of the local labels of the clusters in which it is nested. This list begins with the label of the universal cluster and ends with the label of the given cluster. 3 Overlapping clusters. The following is a suggestion. For Nimrod, clusters within a given hierarchy may be disjoint or nested, but they may not overlap. Hence, within a given hierarchy, an interface has exactly one locator. With the interface model of clusters, a router may be a cluster of interfaces and a network may be a cluster of interfaces. However, given a clustering hierarchy for an internetwork and an interface, in that internetwork, connecting a router to a network, that interface has a single parent cluster. That parent cluster may be a router, a network, or a hybrid such as all of the interfaces to the network plus a subset of the interfaces of some of the routers connected to that network. In any case, the clustering procedure will not make a router cluster and a network cluster that are siblings and that both contain the given interface. Why preclude overlapping clusters? Overlapping clusters increase the amount of location information that must be maintained about entities in an internetwork. In fact, cluster overlaps at multiple levels within a given hierarchy may lead to an exponential increase in the amount of location information that must be maintained. Moreover, route generation must be able to generate routes consistent with the many locators per entity. At first, precluding overlapping clusters in Nimrod may appear to violate assumption (a) above. However, the overlapping portion of a set of clusters has all the characteristics of a cluster and may end up being treated as such. Suppose there are two disjoint adjoining clusters, X and Y, that decide to share network resources. There are two cases to consider. (1) If X and Y have exactly the same characteristics, i.e., qualities of service and restrictions on access to resources, then they could have been combined into a single larger cluster to begin with. There is no reason to "overlap" X and Y so that they may share resources. X and Y may freely share resources without "overlapping" as they have exactly the same characteristics. If X and Y are combined into a single larger cluster, one may still wish to keep them as separate clusters within the larger cluster, simply for size reasons, i.e., to contain the number of clusters within the larger cluster. In any case, X and Y need not "overlap" to share resources. (2) If X and Y have different characteristics, particularly restrictions on access to service, then the connecting points at the boundaries of these clusters will enforce these access restrictions. When X and Y decide to share resources, the shared resources may be managed in one of the following two ways. If there is a clear boundary between which shared resources belong to X and which belong to Y, then the two clusters can remain disjoint. Each cluster may in turn form a subcluster within itself consisting of those of its resources that are to be shared with the other cluster. Hence, X may form X', and Y may form Y', such that the union of X' and Y' are the shared resources. Both X' and Y' may have special access restrictions that apply to these shared resources. If there is not a clear boundary between which shared resources belong to which cluster, i.e., if X and Y agree on mutual ownership of the shared resources, then one cannot simply keep X and Y disjoint. The boundaries of these shared resources must at least respect the access restrictions of X and Y and may also have their own special access restrictions of their own. Hence, the shared resources appear as their own cluster, Z. Thus, we may form clusters Z, X' = X - Z (where "-" is set difference), and Y' = Y - Z, thus increasing the number of clusters under the given parent cluster by one. To reduce the number of clusters under that parent, one may create a single cluster containing X', Y', and Z. Hence, I don't believe that shared resources require overlapping clusters, and I do believe that overlapping clusters unnecessarily complicate Nimrod. 4 Multiple hierarchies. Assumptions (b) and (c) indicate that we may wish to maintain more than one hierarchy at the same time. In the case of (b), two hierarchies will differ only in that portion of the hierarchy being reorganized. In the case of (c), two hierarchies may differ completely. These hierarchies behave as "ships in the night", that is, there is no sharing of routing or location information among clusters in different hierarchies. Hence, although clusters in two different hierarchies may contain some common interfaces, they do not "overlap". When routing traffic from one endpoint to another, one must select the hierarchy that forms the "routing context" for that traffic. Hence, one could apply labels to hierarchies in order to distinguish them. Although this approach makes sense for multiple hierarchies in the case of (c), it is rather awkward for the case of (b), as we discuss below. The remainder of the note is concerned with multiple hierarchies in the sense of (b). Note that in a large internetwork, reorganization of a given hierarchy may simultaneously occur in multiple positions in the hierarchy. We expect that the frequency of reorganization will increase as one descends lower into the hierarchy, i.e., local changes within small clusters of interfaces will be much more common that reorganizations among large clusters. During a reorganization, there is a succession of hierarchies which are each a snapshot of how the hierarchy appears at the given time. One could label each version of the hierarchy by the time at which it came into being. This implies a consistent notion of time throughout the internetwork, a reasonable assumption, but one that I don't think I need to make. Moreover, this assumes that all of the internetwork knows about all of the changes currently taking place within a giving hierarchy. This I think is an unreasonable assumption for a large internetwork. A given reorganization of a hierarchy may only affect the hierarchy at certain clusters. All other clusters may be completely unaffected by the reorganization. Hence, instead of assigning timestamps to each snapshot of the hierarchy, one may assign timestamps to each snapshot of a cluster. When a cluster changes, it assigns itself a new timestamp that will be associated with its state at the given time. Distributed routing and location information for the cluster will bear the new timestamp. These timestamps need not represent absolute time but rather act as sequence numbers. Hence, they need not come from an internetwork-wide consistent clock, but may be independently generated by the given cluster. During a reorganization, not all of the entities involved are likely to be reorganized simultaneously. Moreover, entities throughout the internetwork are unlikely to learn of the reorganization simultaneously. When selecting routes and forwarding messages, an internetwork entity uses the most recent routing and location information that it has. However, this information may no longer represent the current state of the internetwork. To prevent large numbers of routing failures during a reorganization, we suggest that Nimrod entities in clusters affected by the reorganization maintain multiple states representing the new as well as the old view of the hierarchy. Old states should not live forever, but should time out after a "reasonable" duration. This will limit routing failures during a reorganization. After the timeout, entities using outdated routing and location information will experience routing failures and will subsequently update their databases with the newer information. Isidro has an elegant representation of the reorganization problem, that he will distribute to the list. What do you guys think about all of this? m   Received: from PIZZA.BBN.COM by BBN.COM id aa15259; 17 Nov 93 11:08 EST Received: from pizza by PIZZA.BBN.COM id aa10411; 17 Nov 93 10:43 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa10407; 17 Nov 93 10:37 EST To: nimrod-wg@BBN.COM Subject: On maps and physical realizations Date: Wed, 17 Nov 93 10:38:17 -0500 From: Isidro Castineyra Hi guys, These are additions/modifications to the message I sent a couple of days ago. Section 5 is a modification. Section 6 is new. Isidro ------------------ 5 Maps 1. With every structure of a cluster there is associated an internal map. The internal map of a cluster is given by the identities of the clusters that belong to it, and by the connections (arcs) between these. 2. Arcs are uni-directional. A bi-directional connection is represented by two uni-directional arcs, one in each direction. 3. Arcs are identified by giving locators ot its the source and destination clusters. When there are more than one arc between two given clusters, they are distinguished by assigning to them different numbers. 4. Every element of a map---arc or cluster---can have associated with it multiple attributes. This attributes correspond to things like policy constraints, resource availability, available transit qos parameters, etc. 5. When giving the internal map of a cluster, each arc that connects this cluster to another should be shown as terminating in one of the top-level component clusters. 6 Physical Realization 1. The physical realization of an advertised arc can vary from arc to arc. For example, while an advertised arc can consist of a single point-to-point link, it also can represent multiple parallel physical links. Similarly, a single physical transmission link can be represented as multiple arcs. 2. A cluster can represent any physical structure that is deemed useful. There is no undecomposable physical structure---i.e., there is no minimum cluster. A consequence of this, is that interfaces do not appear in maps, or are mentioned by the architecture. Examples of clustering are, (a) A cluster can represent a corporate network. (b) A cluster can represent a 1993 workstation. This cluster can have an internal map. That is, there can be clusters inside a cluster that represents a workstation---for example, a set of processes within the workstation can be enveloped by a cluster boundary This architecture is agonostic about the usefulness of a given clustering of a physical network.   Received: from PIZZA.BBN.COM by BBN.COM id aa08175; 17 Nov 93 17:15 EST Received: from pizza by PIZZA.BBN.COM id aa13059; 17 Nov 93 16:56 EST Received: from BBN.COM by PIZZA.BBN.COM id aa13055; 17 Nov 93 16:54 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa07094; 17 Nov 93 16:53 EST Received: by ginger.lcs.mit.edu id AA19841; Wed, 17 Nov 93 16:53:50 -0500 Date: Wed, 17 Nov 93 16:53:50 -0500 From: Noel Chiappa Message-Id: <9311172153.AA19841@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: High-level take on Nimrod Cc: jnc@ginger.lcs.mit.edu The following high level take on "what Nimrod is" might prove interesting. You can view Nimrod as two semi-separate things, with the second being a "client" of the first: - A distributed, replicated, database for (potentially abstracted) topology information. - A two part system for handling user data traffic, with the route selection and the forwarding being (potentially) separated. (I have a new way of doing datagram traffic which is not source-routed, but isn't classic hop-by-hop; a note on this will be out shortly.) The reason this view is important is that it makes us focus on the fact that *other* clients may wish to use the first part; e.g. resource management/ traffic control, network management, etc, etc. Viewed in this light, it might be useful to think of the first part as exactly a database problem, and make sure that it has the kinds of capabilities and extension tools that such a database ought to provide. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa10155; 17 Nov 93 18:02 EST Received: from pizza by PIZZA.BBN.COM id aa13407; 17 Nov 93 17:51 EST Received: from BBN.COM by PIZZA.BBN.COM id aa13403; 17 Nov 93 17:49 EST Received: from bridge2.NSD.3Com.COM by BBN.COM id aa09606; 17 Nov 93 17:48 EST Received: from remmel.NSD.3Com.COM by bridge2.NSD.3Com.COM with SMTP id AA01575 (5.65c/IDA-1.4.4nsd for ); Wed, 17 Nov 1993 14:48:25 -0800 Received: from localhost.NSD.3Com.COM by remmel.NSD.3Com.COM with SMTP id AA07035 (5.65c/IDA-1.4.4-910725); Wed, 17 Nov 1993 14:48:24 -0800 Message-Id: <199311172248.AA07035@remmel.NSD.3Com.COM> To: Noel Chiappa Cc: nimrod-wg@BBN.COM Subject: Re: High-level take on Nimrod In-Reply-To: Your message of "Wed, 17 Nov 93 16:53:50 EST." <9311172153.AA19841@ginger.lcs.mit.edu> Date: Wed, 17 Nov 93 14:48:23 -0800 From: tracym@nsd.3com.com > - A distributed, replicated, database for (potentially abstracted) topology > information. > > - A two part system for handling user data traffic, with the route selection > and the forwarding being (potentially) separated. (I have a new way of Bravo! This is an excellent, concise, description of the overall problem. I'd guess that the "working" Nimrod folks have no problems with the distinction between database maintenance and database usage, but I'm sure many besides myself have occasionally had the experience of explaining that link-state information flooding does not immediately imply Dykstra SPF, and vice-versa, and neither of these is related to the use of Patricia pattern matching in a routing table, which is itself (potentially) unrelated to individual packet forwarding decisions (none of which is Nimrod, of course;-). > The reason this view is important is that it makes us focus on the fact that > *other* clients may wish to use the first part; e.g. resource management/ > traffic control, network management, etc, etc. Viewed in this light, it might > be useful to think of the first part as exactly a database problem, and make > sure that it has the kinds of capabilities and extension tools that such a > database ought to provide. I'd second this notion. I'm sure BBN isn't the only place that's tapped a management system into a routing database via a protocol connection. Some standard database terminology might be also prove useful (serialization, replication, commitment, checkpointing, etc.), and the perspective might even provide some algorithmic help for massaging large amounts of data (e.g. extract the set of all clusters with transit paths for policy X). Tracy   Received: from PIZZA.BBN.COM by BBN.COM id aa02522; 17 Nov 93 23:34 EST Received: from pizza by PIZZA.BBN.COM id aa14813; 17 Nov 93 23:15 EST Received: from nic.near.net by PIZZA.BBN.COM id aa14808; 17 Nov 93 23:11 EST Received: from nic.near.net by nic.near.net id aa15068; 17 Nov 93 23:13 EST To: Noel Chiappa cc: nimrod-wg@nic.near.net Subject: Re: High-level take on Nimrod In-reply-to: Your message of Wed, 17 Nov 1993 16:53:50 -0500. <9311172153.AA19841@ginger.lcs.mit.edu> Date: Wed, 17 Nov 1993 23:13:46 -0500 From: John Curran -------- ] From: Noel Chiappa ] Subject: High-level take on Nimrod ] Date: Wed, 17 Nov 93 16:53:50 -0500 ] ] The following high level take on "what Nimrod is" might prove ] interesting. You can view Nimrod as two semi-separate things, with the second ] being a "client" of the first: ] ] - A distributed, replicated, database for (potentially abstracted) topology ] information. "topology information" or "topology and policy information" I suspect the former, but could use confirmation... /John   Received: from PIZZA.BBN.COM by BBN.COM id aa23122; 18 Nov 93 9:59 EST Received: from pizza by PIZZA.BBN.COM id aa17155; 18 Nov 93 9:43 EST Received: from BBN.COM by PIZZA.BBN.COM id aa17151; 18 Nov 93 9:40 EST Received: from ftp.com by BBN.COM id aa21634; 18 Nov 93 9:35 EST Received: by ftp.com id AA20050; Thu, 18 Nov 93 09:35:07 -0500 Date: Thu, 18 Nov 93 09:35:07 -0500 Message-Id: <9311181435.AA20050@ftp.com> To: jnc@ginger.lcs.mit.edu Subject: Re: High-level take on Nimrod From: Frank Kastenholz Reply-To: kasten@ftp.com Cc: nimrod-wg@BBN.COM > The following high level take on "what Nimrod is" might prove > interesting. You can view Nimrod as two semi-separate things, with the second > being a "client" of the first: > > - A distributed, replicated, database for (potentially abstracted) topology > information. > > - A two part system for handling user data traffic, with the route selection > and the forwarding being (potentially) separated. (I have a new way of > doing datagram traffic which is not source-routed, but isn't classic > hop-by-hop; a note on this will be out shortly.) > > The reason this view is important is that it makes us focus on the fact that > *other* clients may wish to use the first part; e.g. resource management/ > traffic control, network management, etc, etc. Viewed in this light, it might > be useful to think of the first part as exactly a database problem, and make > sure that it has the kinds of capabilities and extension tools that such a > database ought to provide. by resource management/traffic control do you mean stuff like braden/et al's rsvp flows and so on? i think that this stuff is fundamentally a part of the second bullet (which i usually refer to by the general term of 'forwarding'). you can think of all traffic through the network as having associated with it a certain set of policies, attributes, resource requirements, and so on; some traffic has certain explicitly stated requirements, the rest does not and would be treated in the 'default' manner. i think that if you consider these as two fundamentally different types of beast then you might end up complicating things when you do not have to. the split is a good thing. it represents a nice partitioning of the problem. ideally, either of the two components can be changed without significantly effecting the other. -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa26635; 18 Nov 93 17:54 EST Received: from pizza by PIZZA.BBN.COM id aa20152; 18 Nov 93 17:30 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa20148; 18 Nov 93 17:28 EST To: nimrod-wg@BBN.COM Subject: routing Date: Thu, 18 Nov 93 17:30:13 -0500 From: Martha Steenstrup Hello Isidro, I have a few questions about your Paperman. > 3. Maps have an expiration time. It is assumed that all nimrod routing > entities have a common accurate clock---for example, by using NTP. This > clock is used to determine the validity of maps. While a common clock is nice to have and a reasonable thing to request (given the existence of NTP), do we want to require such a clock for Nimrod? For map validity, "non-wrapping" sequence numbers on maps and map aging mechanisms based on local clocks may be sufficient. 4. A cluster can also ``subscribe'' to the map of a second cluster. This means that the second cluster will send to it topology updates periodically. The update could be as brief as indicating that the time-out of the last full topology received should be extended until an indicated time. For subscription, do we want to impose a periodic nature on map updating? Certainly, we should allow periodic updating, but should we mandate it? > 8 Routing > 2. A cluster has a top level map of the cluster it is a member of. It > also has a map of the top level of every cluster it is a descendent of. > For example, a cluster D with locator U,1.A,1.B,1.C,1.D,1, has top > level maps of clusters C.1, B,1, A,1, and U,1. Notice that this > implies that there is no information hiding ``downwards,'' that is, a > cluster cannot hide its top level map from any of its descendants. It > also implies that two clusters that are owned by the same cluster have > a consistent view of the network ``upwards.'' > 4. When a packet header contains only a locator for the destination, we > have what we know today as hop-by-hop routing. Routing loops are > avoided by forwarding the packet in a direction that minimizes the > number of hops to the destination. This is called ``vanilla datagram'' > forwarding. Consistency of the maps is required for this to work. Do we really want to require a cluster to have top-level maps of all of its ancestor clusters? To route throughout a hierarchical internetwork, I don't think we need that much information. It seems that one can reduce the locality of the information that a cluster needs about its surroundings. For example, the following information is sufficient (but may not be necessary) to do routing in a hierarchical internetwork. Each cluster knows: - the set of child clusters (if any) that it contains and information about how to reach them; - the set of sibling clusters and how to reach them; - the set of sibling clusters which are exit points of its parent cluster. With this information, each cluster can send traffic up to its parent, across to its siblings, and down to its children. This is "strict hierarchical" routing. In fact, one could use this as the Nimrod "hop-by-hop" mode. For example, suppose that an endpoint in cluster U.X.Y.Z.W wants to send traffic to an endpoint in cluster U.X.Y.T.V. By comparing labels, a Nimrod routing entity in the source cluster, U.X.Y.Z.W, knows that it shares a common cluster, U.X.Y, with the destination cluster, U.X.Y.T.V. It need only direct the traffic "up" to its parent, U.X.Y.Z, through sibling clusters to an exit point of its parent, for example, cluster U.X.Y.Z.P. This exit point of U.X.Y.Z then routes the traffic through a set of clusters in U.X.Y until it reaches U.X.Y.T. The entry point of U.X.Y.T, for example, cluster U.X.Y.T.R then routes the traffic across sibling clusters to U.X.Y.T.V. m   Received: from PIZZA.BBN.COM by BBN.COM id aa03441; 21 Nov 93 1:50 EST Received: from pizza by PIZZA.BBN.COM id aa01747; 21 Nov 93 1:37 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa01743; 21 Nov 93 1:33 EST To: nimrod-wg@BBN.COM Subject: endpoints Date: Sun, 21 Nov 93 01:36:12 -0500 From: Martha Steenstrup A while ago, in a previous message, I included some information about endpoints. Among other things, I said that an endpoint is associated with one or more clusters but is not contained within any cluster. I would like to make clear what I mean by an endpoint/cluster association. 1 Definitions. An endpoint is "associated with" a cluster if there exists at least one Nimrod entity within the cluster that, given only the endpoint identifier, can forward packets toward that endpoint through resources that belong only to that cluster or to no cluster at all. A Nimrod "entity" performs some or all of the Nimrod functions, including route generation, routing information distribution, identifier/locator mapping, and packet forwarding. 2 Endpoint Locators. An endpoint may be associated with more that one cluster. However, this does not imply that the clusters, with which the endpoint is associated, overlap. For example, suppose a host is connected to two distinct networks, each of which has been designated as a cluster in the given hierarchy. These two clusters, X and Y, are disjoint. Furthermore, suppose there are several different applications running on the host and communicating with peers somewhere else in the Internet. If the host is also part of a separate cluster Z, distinct from its two connected networks, then the application peers resident in the host are associated with Z. If the host is not in any cluster, then the application peers resident in the host are associated with both clusters X and Y. A locator for an endpoint consists of the locator for a cluster with which that endpoint is associated plus the identifier for that endpoint. The cluster's locator is the sequence of labels of ancestral clusters, beginning with the universe and ending with the given cluster. When a Nimrod entity performs, on behalf of the source of a traffic flow, a mapping between endpoint identifier and locator, that mapping yields a set of locators of clusters associated with that endpoint. The Nimrod entity selects one of these locators and routes packets, destined for that endpoint, to the cluster indicated by the locator. In particular, the packets travel via Nimrod routing, passing through other clusters in order to reach the cluster associated with the endpoint. This ultimate cluster is the final "jumping off place". The Nimrod entity that, on behalf of the source of the traffic flow, selected the route to the cluster associated with the endpoint has no control on packet forwarding once packets reach that cluster. From that point on, the cluster associated with the endpoint has the responsibility for getting the packets to the designated endpoint. Comments? (I will be out of touch for about a week.) m   Received: from PIZZA.BBN.COM by BBN.COM id aa14364; 22 Nov 93 21:06 EST Received: from pizza by PIZZA.BBN.COM id aa11404; 22 Nov 93 20:47 EST Received: from BBN.COM by PIZZA.BBN.COM id aa11400; 22 Nov 93 20:45 EST Received: from wizard-gw.qualcomm.com by BBN.COM id aa13953; 22 Nov 93 20:47 EST Received: from guru.qualcomm.com by qualcomm.com; id AA09223 sendmail 5.65/QC-main-2.1a via SMTP Mon, 22 Nov 93 17:47:07 -0800 for isidro@BBN.COM Received: by guru; id AA01640 sendmail 5.67/QC-subsidiary-2.2 Mon, 22 Nov 93 17:47:06 -0800 for nimrod-wg@BBN.COM Date: Mon, 22 Nov 93 17:47:06 -0800 From: Greg Noel Message-Id: <9311230147.AA01640@guru> To: isidro@BBN.COM, nimrod-wg@BBN.COM Subject: Re: Paperman (flimsier than a strawman) I've finally had time to go over the paperman. I have one point, which may well be an implementation detail rather than a design-level distinction, but I thought I'd mention it anyway.... > 3. A packet contains in its header a loose source route. This route could > be as loose as including only a locator for the destination eid, or as > tight as defining every nimrod entity through which the packet is > required to pass. > > 4. A packet can also include a flow-id which maps into state that has been > previously set-up in routers along the way. Packet forwarding when > there is a flow-id established is relatively simple: follow the > instructions in the routers [sic] state. I view this somewhat differently. Instead of an optional flow-id, I'd require that some minimum set of default, always available flow-ids be implemented in each router. One of the default flow-ids would be "route to the next source-route element however you wish". Packet forwarding then is always "follow the instructions for the current flow-id" rather than a special case for when the flow-id is not specified. In keeping with the Nimrod philosophy, I'd like the number of default policy flows to be as small as possible. It's possible that only the one special case would be required. > 5. The loose source route in a packet is the only way to indicate policy > within a packet. For example, when the loose source route includes > only the destination eid, this means the the source does not care how > the packet gets there. Thus, instead of the loose source route indicating the policy, the flow-ids would indicate the policy. -- Greg   Received: from PIZZA.BBN.COM by BBN.COM id aa25505; 23 Nov 93 14:51 EST Received: from pizza by PIZZA.BBN.COM id aa15628; 23 Nov 93 14:39 EST Received: from BBN.COM by PIZZA.BBN.COM id aa15624; 23 Nov 93 14:35 EST Received: from ftp.com by BBN.COM id aa24829; 23 Nov 93 14:36 EST Received: by ftp.com id AA03192; Tue, 23 Nov 93 14:36:26 -0500 Date: Tue, 23 Nov 93 14:36:26 -0500 Message-Id: <9311231936.AA03192@ftp.com> To: nimrod-wg@BBN.COM Subject: blasts from the past From: Frank Kastenholz Reply-To: kasten@ftp.com Hi I've been going over the archives to router requirements and an interesting point was threaded through the discussions. I think that it might have applicability to Nimrod (or any other routing protocol development). Strict vs Advisory TOS/Policy A big thread of the discussions were on TOS -- should it be strict (i.e. if you can not fulfill my requested TOS, do not route the packet) or should it be advisory (if you have my requested TOS then route accordingly, otherwise, do the best you can). There were N people arguing the points, N/2 of them argued one side and the other N/2 argued the other. It seems to me that this is an issue that we will face here in Nimrod-land. Are the policies, etc, that are in the routing, and that hosts request when sending packets (or setting up flows or whatever) mandatory or advisory? Or will we ignore the problem and allow the policy terms to be tagged as either advisory or mandatory? (of course, when we get to that point, we will have to define what happens mandatory policy terms can not be fulfilled.) -- Frank Kastenholz FTP Software 2 High Street North Andover, Mass. USA 01845 (508)685-4000   Received: from PIZZA.BBN.COM by BBN.COM id aa08201; 23 Nov 93 18:41 EST Received: from pizza by PIZZA.BBN.COM id aa16889; 23 Nov 93 18:24 EST Received: from BBN.COM by PIZZA.BBN.COM id aa16885; 23 Nov 93 18:22 EST Received: from TIEO.SAIC.COM by BBN.COM id aa07610; 23 Nov 93 18:23 EST Received: by tieo.saic.com (4.1/1.34) id AA01101; Tue, 23 Nov 93 18:25:22 EST Date: Tue, 23 Nov 93 18:25:22 EST From: Kenneth Carlberg Message-Id: <9311232325.AA01101@tieo.saic.com> To: kasten@ftp.com Subject: Re: blasts from the past Cc: nimrod-wg@BBN.COM > Are the policies, etc, that are in the routing, and that > hosts request when sending packets (or setting up flows or whatever) > mandatory or advisory? my opinion is that the policy terms should be elastic enough to convey if the policy is mandatory or advisory. TOS presents a harder problem because it is more of a yes/no issue. I believe that the policy terms, as presented by Dave Clark (can't remember the RFC #), could be used to convey mandatory/advisory status. But this leads to another question. If NIMROD is considered a next generation of IDPR, then does this imply that the same structure (specification ?) used for IDPR's policy terms are to be used for NIMROD's policy terms ? Note: given the early development of NIMROD, I'll concede to arguements that structure/specification of policy terms is premature at this point. -ken   Received: from PIZZA.BBN.COM by BBN.COM id aa09576; 27 Nov 93 19:57 EST Received: from pizza by PIZZA.BBN.COM id aa01286; 27 Nov 93 19:47 EST Received: from BBN.COM by PIZZA.BBN.COM id aa01282; 27 Nov 93 19:43 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa09367; 27 Nov 93 19:42 EST Received: by ginger.lcs.mit.edu id AA25208; Sat, 27 Nov 93 02:43:59 -0500 Date: Sat, 27 Nov 93 02:43:59 -0500 From: Noel Chiappa Message-Id: <9311270743.AA25208@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: More terminology Cc: jnc@ginger.lcs.mit.edu If we don't have globally unique labels on areas/clusters, but rather only locally unique ones, we are almost certainly (modulo things like being able to handle incomplete locators, etc) going to have to deal with problems of how do we find the correct "place" in which to interpret them. To aid in this discussion, for those of you to whom terms like "binding", "context" and "closure" aren't second nature, here's a section of an (incomplete) ID on EID's which explains most of the relevant terms. Noel -------- Terminology Before proceeding to the technical details, it is important to understand some definitions about objects, names, and related subjects. The terms "object" and "name" are hopefully self-explanatory. The association between a name and an object is called a "binding". It is important to realize that a single instance of an object (i.e. a member of an "object class") may have more than one name. Moreover, each of those names may be drawn from the same class of names (or "namespace"), i.e. a many-to-one binding; alternatively, each of the names may be from a different namespace, i.e. multiple one-one bindings. For example, the person writing this document is an instance of the object class of humans, and has several names, from different kinds of namespaces, such as my 'name' name ('J. Noel Chiappa'), my Social Security Number (a government supplied personal identification number in the U.S.A.), etc. However, all the names refer to the same object. The *form* of the names in a namespace is usually driven by the use to which the name will be put. Any structure in the name is usually intended to make some operation on the name (be it looking it up in a directory, or whatever) easier. For example, Unix filenames and DNS names have hierarchical levels, indicated in the name, to allow them to be looked up in hierachical directories. IP addresses have structure in them to help them be routed. Names may also have multiple "representations", or ways of encoding the same individual name. For instance, the IP address "18.8.0.8" (in decimal, printed notation) is the same exact name as the bit string "00010010 00001000 00000000 00001000" in an IP packet header; they are just different ways of encoding the exact same information, just as 12 base 8, and 10 base 10, are the same number. A collection of bindings in which a system records and looks up the connections is called a "context" (sometimes loosely referred to as a "directory", although this term has other meanings). A context may either map from names for objects to a direct handle on the object, or from one class of name for an object to another kind of name for the same object, or perhaps some other useful mapping.   Received: from PIZZA.BBN.COM by BBN.COM id aa14771; 3 Dec 93 14:11 EST Received: from pizza by PIZZA.BBN.COM id aa08022; 3 Dec 93 13:38 EST Received: from BBN.COM by PIZZA.BBN.COM id aa08018; 3 Dec 93 13:35 EST Received: from THYME.LCS.MIT.EDU by BBN.COM id aa12965; 3 Dec 93 13:33 EST Received: by thyme.LCS.MIT.EDU id AA13634; Fri, 3 Dec 93 13:33:37 EST Date: Fri, 3 Dec 93 13:33:37 EST From: Noel Chiappa Message-Id: <9312031833.AA13634@thyme.LCS.MIT.EDU> To: nimrod-wg@BBN.COM Subject: Abstraction taxonomy Cc: jnc@thyme.lcs.mit.edu Dear fellow graph-theory aficionados, err, Nimrod-WG'ers: In response to Isidro's "straw man", I've been thinking about the multi-level graph representation of the network a lot recently, and in a phone discussion with Isidro we've come up with yet another, fairly obscure, but nonetheless quite important, distinction in abstraction models, which I will try to explain (hopefully without being as absolutely unintelligible as my rantings usually are :-). To make it easier to visualize, let's use an example. (You may want to draw pictures; I had to do this for a many years before I got to the point that words were good enough.) I'm going to use the terminology of single metrics for simplicity, but it generalizes to full-blown attributes (trust me :-). Imagine we have a network graph (leaving aside for the moment the question of exactly *what* the nodes and arcs represent, a still *extremely* vexing question :-). (Draw a picure with a bunch of dots and random arcs connecting them, here.) Now, draw an area boundary around a (contiguous) part of the graph, intending to provide an abstraction for the part of the graph inside the boundary; call this area A. The boundary line bisects (i.e. crosses), say, 6 arcs, the "external" arcs, and no nodes. The external arcs connect to six (or fewer, if several external arcs connect to the same node) "border" nodes. The area fully contains at least 3 "internal" nodes, which are not connected to any of the external arcs, and thus, by definition, also fully contains at least 3 "internal" arcs. (It's pretty obvious why: each of the external arcs must connect to a node *inside* the area, and since the area is contiguous, each of the internal nodes must have at least one arc which connects it to a border, or other internal, node.) I can visualize many different possible abstractions (i.e. models) of the topology inside the area boundary. Among them, in approximate order of increasing complexity are: #1: A single node, to which the six "external" arcs connect. #2: The several border nodes, which are connected in some irregular fashion, but with complete internal connectivity. #3: The several border nodes, which are fully interconnected internally; e.g. 15 arcs (from the (N^2 - N)/2 formula, assuming 6 border nodes), allowing exact metrics of all internal inter-border paths to be modelled. #4: Several border nodes, each of which connects to a center node; this allows different metrics to be put on each arc to the center node, giving a less uniform view of the internal connections between the six entry arcs than #1, but not as optimal as #3. #5: Several border nodes, which are connected internally in some irregular fashion, using a number of internal nodes. #6: The null abstraction (i.e. the original topology). In thinking of Nimrod's abstraction system, I had imagined that it would allow any of these. (Note that we might use more than one of these at the same time; things near A in the graph might wish to use, say, abstraction #5, whereas things far away might be happy with #1. Things really far away might not even have A in the graph at all, but only a node representing a larger area which encloses A.) However, there is an important division (actually, it's a multi-way split, but let's hold the details for a bit) that can be made among possible abstractions: not all *possible* abstractions can be produced by what I call a "recursive strict abstraction" rule, in which *neither* nodes *nor* arcs can be created out of thin air. Here's how that rule works. To create an abstraction, follow the process outlined above; you take a graph, draw a line which encloses a contiguous part of the graph, bisecting only arcs, not nodes (I don't yet know why this restriction is the Right Thing, but I think it is; I'll go off and think about it). You then have your choice of representing that area in the graph as either: - the complete original topology, or - a single node, to which are connected all the arcs which cross the area boundary. You can repeat this process, i.e. apply the rule recursively. This has the advantage of being a really simple rule, but it's actually quite powerful in practise, when applied *recursively*. It is, for example, *always* possible (as far as I can tell) to produce model #4 using this rule recursively, and many instances of #2 and #5 as well. Here's how to do #2. Draw a set of area boundaries (call the resulting areas B0...Bn), each of which includes *one* of the border nodes, and making sure that all internal nodes are in one of the areas B0...Bn. Now, redraw the topology with areas B0...Bn replaced by nodes, and the area boundary for A as before. The topology inside A is now exactly #2. Here's how you do #4. Draw an area boundary around all the internal nodes of area A. Call this area B. Now, redraw the topology with area B replaced by a node, and the area boundary for A as before. The topology inside A is now exactly #4, except that some of the border nodes may have multiple arcs to B. (If you have a rule which allows you to drop arcs, or group ones which join the same pair of nodes into a single arc, then it is exactly #4.) Here's how to do #5. Draw an area boundary around two of the internal nodes of area A. Call this area B. Now, redraw the topology with area B replaced by a node, and the area boundary for A as before. The topology inside A is now exactly #5. So, it's actually quite powerful in practise. It's easy to see how it works, and it provides easy naming for the elements of the graph (since each abstraction is made up of a specific set of real graph elements, each of those elements is named within that abstraction). However, there two ways in which the rule can be relaxed. Remember, the rule said that neither nodes nor arcs can be created out of thin air. This would usually prevent, for instance, abstractions such as #3. We can relax either of these restrictions independantly (e.g. allow the creation of arcs, but not nodes), or we can relax them both, allowing creation of "virtual" nodes and arcs which do not correspond to a specific disjoint set of lower level objects. Note the use of the word "disjoint". The rule I stated effectively does not allow overlapping areas. Once you start allowing area boundaries to overlap, all sorts of chaos breaks out, which is why I really don't like them. If you do allow overlapping area boundaries, you can produce models like #3, albeit with some hassle. I can't quite see a rule in which we allow creation of "virtual" nodes, but not arcs, although I guess it's feasible to visualize such a rule (in action). So, we have 4 allowed abstractions models: strict, virtual-node, virtual-arc, and virtual (in which both arcs and nodes can be virtual). Exactly what to allow is a good question. I like arguments that say we should be flexible, but I'm concerned that the result not be too complex and hard to understand. This is definitely true if the extra complexity turns out to be something that you almost never use in practise. Quite often there is a way to make a simpler model do everything the more complex one does, albeit by making the more unusual cases more complex to model. However, if the unusual cases are rare, the tradeoff is probably worth it, and the simple model is the right choice. So, I don't see much use for virtual nodes (or at least enough to make it worth allowing them). There is probably a certain amount of real-world use for virtual arcs, so perhaps we should allow those. I admit it's hard to make a definite choice without knowing what it is arcs and nodes represent! So, back to thinking about what it is that those nodes and arcs *actually* represent! This choice may become a little clearer then. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa16682; 3 Dec 93 14:45 EST Received: from pizza by PIZZA.BBN.COM id aa08306; 3 Dec 93 14:25 EST Received: from BBN.COM by PIZZA.BBN.COM id aa08300; 3 Dec 93 14:23 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa15306; 3 Dec 93 14:21 EST Received: by ginger.lcs.mit.edu id AA23931; Fri, 3 Dec 93 14:21:38 -0500 Date: Fri, 3 Dec 93 14:21:38 -0500 From: Noel Chiappa Message-Id: <9312031921.AA23931@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: More open points Cc: jnc@ginger.lcs.mit.edu In the process of creating the last message, I added the following open points: - Do we allow an area to be represented by a single abstraction, or can an area have multiple abstractions? (I think this is probably the same question as do we allow overlapping areas, but I'm not positive yet.) - Can area boundaries bisect arcs, nodes, or both? (This is more or less the same question as "can an area boundary run through a router", but not quite). - Does the abstraction algorithm allow only strict abstraction, or can you create virtual nodes and/or arcs as well? (This is basically the same question as "are virtual links and routers needed", but stated in a more graph-theory way.) - Can multiple arcs which join two nodes be grouped into a single arc? - Can arcs and nodes be dropped from the map other than via abstraction? Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa17750; 3 Dec 93 15:05 EST Received: from pizza by PIZZA.BBN.COM id aa08439; 3 Dec 93 14:48 EST Received: from BBN.COM by PIZZA.BBN.COM id aa08435; 3 Dec 93 14:46 EST Received: from nsco.network.com by BBN.COM id aa16722; 3 Dec 93 14:46 EST Received: from anubis-e1.network.com by nsco.network.com (5.61/1.34) id AA01418; Fri, 3 Dec 93 13:49:34 -0600 Received: from gramarye.network.com by anubis.network.com (4.1/SMI-4.1) id AA09622; Fri, 3 Dec 93 13:44:53 CST Date: Fri, 3 Dec 93 13:44:53 CST From: Joel Halpern Message-Id: <9312031944.AA09622@anubis.network.com> To: nimrod-wg@BBN.COM Subject: Re: More open points Cc: jnc@ginger.lcs.mit.edu On the question of what a boundary may bisect, I suggest that an area/aggregation/abstraction boundary should be able to bisect a node. To Explore this further, one must think about what a node and a link represent. I tend to follow the old school that nodes generally represent attached devices, or aggregates of some kind, while arcs represent links/interfaces/connections. Now, one of the reasons I do think this way is that a device with multiple interfaces is an aggregate, and it seems clear that we want to use nodes to represent aggregates. Having said that, we run into the fact that while a router is an aggregate, it is a physical aggregate. The logical notion of what is "within" it is rather more interesting. It may well be that half the router is managed by one organization, and half by another. It would seem to me that the aggregation boundary possibly should run down the middle of the router. Having said that, I will note that the alternative is to require that any "node" which you want to "split" be represented as more than one "node" in the most detailed graph. Then, one is never running a boundary through the middle of the graph. (Of course, one has created a virtual node and a virtual link, and probably confused the local site administrator, who is not used to thinking in those terms. (If the router is truly set up to be separately managed, then it probably is correct to model it as two nodes. However, I think that this situation may arise without the degree of modelling/support within the router, based on cooperation between the controlling entitites. Thank you, Joel M. Halpern jmh@network.com Network Systems Corporation PS Given the existence of several sets of "multi-link" procedures, operating already at several layers of abstraction, the ability to combine several "equivalent" arcs into one arc would seem a natural abstraction. Care clearly must be taken in such cases.   Received: from PIZZA.BBN.COM by BBN.COM id aa14222; 4 Dec 93 11:54 EST Received: from pizza by PIZZA.BBN.COM id aa11894; 4 Dec 93 11:42 EST Received: from BBN.COM by PIZZA.BBN.COM id aa11890; 4 Dec 93 11:40 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa14067; 4 Dec 93 11:41 EST Received: by ginger.lcs.mit.edu id AA01937; Sat, 4 Dec 93 11:41:39 -0500 Date: Sat, 4 Dec 93 11:41:39 -0500 From: Noel Chiappa Message-Id: <9312041641.AA01937@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Virtual nodes Cc: jnc@ginger.lcs.mit.edu Oh, in case you think you need virtual nodes to do virtual routers (a la IDPR), maybe not. Imagine a scenario in which you have two areas, A and B, with three routers between them in parallel. You could draw a line around the three routers and make them a single node. In general, the reason I'm not sure we need the ability to create virtual nodes is that the whole point of areas is to reduce the number of nodes (and arcs) in the map. Virtual links are demonstrably useful as they help in modelling an area as simply the border nodes, and connectivity between them. Sure, there may be some circumstances in which being able to create nodes is useful, but I'm wondering how much use they real are. An example would be if interfaces can be nodes. In which case, you'd model the typical full connectivity between the N interfaces which form a network as (N^2 - N)/2 arcs; creating a virtual node for the network (assuming we don't have network nodes, as yet undecided :-) would allow you to reduce this to N arcs. Of course, if networks could be real live nodes, this application isn't needed. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa12453; 6 Dec 93 19:47 EST Received: from pizza by PIZZA.BBN.COM id aa20257; 6 Dec 93 19:26 EST Received: from TTL.BBN.COM by PIZZA.BBN.COM id aa20253; 6 Dec 93 19:24 EST To: Noel Chiappa cc: nimrod-wg@BBN.COM Subject: Re: Abstraction taxonomy In-reply-to: Your message of Fri, 03 Dec 93 13:33:37 -0500. <9312031833.AA13634@thyme.LCS.MIT.EDU> Date: Mon, 06 Dec 93 19:19:44 -0500 From: Ram Ramanathan Noel, I have some questions/comments on the abstraction taxonomy you wrote about. > I can visualize many different possible abstractions (i.e. models) of >the topology inside the area boundary. Among them, in approximate order of >increasing complexity are: > >#1: A single node, to which the six "external" arcs connect. >#2: The several border nodes, which are connected in some irregular >fashion, but with complete internal connectivity. >#3: The several border nodes, which are fully interconnected internally; >e.g. 15 arcs (from the (N^2 - N)/2 formula, assuming 6 border nodes), >allowing exact metrics of all internal inter-border paths to be >modelled. >#4: Several border nodes, each of which connects to a center node; this >allows different metrics to be put on each arc to the center node, >giving a less uniform view of the internal connections between the six >entry arcs than #1, but not as optimal as #3. >#5: Several border nodes, which are connected internally in some irregular > fashion, using a number of internal nodes. >#6: The null abstraction (i.e. the original topology). It seems to me that #3 is just a special case of #2; and #4 and #6 are special cases of the more general #5. For instance, if border nodes are "fully interconnected internally" (#3), then they are also "interconnected in some irregular fashion"(#2). Thus, we need to worry about the recursive abstraction scheme only for #1, #2 and #5. If we can fix that, the others follow. Actually, there is another abstraction scheme that may or may not fit in with your taxonomy. In the situation that in #3 above, many of the arcs have the same metrics, we can group the arcs so that all arcs within a group have the same metrics. In many cases, we might have one large group and a few small ones. Then, we can specify the few small groups and just say "for the rest, the metrics are....". This does not lose information and cuts down the number of arcs that must be represented greatly. However, it assumes some uniformity amongs the arcs. > However, there is an important division (actually, it's a multi-way >split, but let's hold the details for a bit) that can be made among possible >abstractions: not all *possible* abstractions can be produced by what I call a >"recursive strict abstraction" rule, in which *neither* nodes *nor* arcs can >be created out of thin air. ..... >Here's how you do #4. Draw an area boundary around all the internal >nodes of area A. Call this area B. Now, redraw the topology with area B >replaced by a node, and the area boundary for A as before. The topology inside >A is now exactly #4, except that some of the border nodes may have multiple >arcs to B. (If you have a rule which allows you to drop arcs, or group ones >which join the same pair of nodes into a single arc, then it is exactly #4.) I guess I am not understanding the rule. Maybe your answer to the following concern will dispel my incomprehension : When you use abstraction #4, say 2 times within an area, it seems that you will have 2 concentric circles around a node. Right? Call the node X, the circle around it area B and the outermost circle area A. Now there is an arc from a border node on A to a border node on B and another arc from a border node on B to the node X. But to have abstraction #4 for the area, you need to have an arc from a border node in A directly to X. The only way I can see this happening is to create a virtual arc using the two real arcs. So it seems that one would have to create arcs "out of thin air"?? - Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa11402; 8 Dec 93 12:23 EST Received: from pizza by PIZZA.BBN.COM id aa29143; 8 Dec 93 11:52 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29139; 8 Dec 93 11:49 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa09630; 8 Dec 93 11:47 EST Received: by ginger.lcs.mit.edu id AA24530; Wed, 8 Dec 93 11:47:39 -0500 Date: Wed, 8 Dec 93 11:47:39 -0500 From: Noel Chiappa Message-Id: <9312081647.AA24530@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Abstractions again.. Cc: jnc@ginger.lcs.mit.edu One interesting question is "how many different abstractions of a given piece of toplogy do you allow"? I don't ask this as "do we allow diferent abstractions", since by definition you can usually have more than one, viz: Take a graph containing a bunch of K-1 level "area" nodes B0...Bn, grouped into a K level area A. (I number from the bottom level up! :-) Now, one way to represent that area in your map is as a single node, A. You could also represent it as the complete set of K-1 level nodes, so you see all of the Bi as individual nodes. You can also expand one or all of the Bi, so that your map now contains many K-2 level nodes Cij, where Ci0...Cin represent the constituents of Bi. Etc, etc, etc. So, even the most rigid abstraction model already allows multiple abstractions. What I'm speaking of is something more flexible. Consider area A. Even using the "recursive strict abstraction" rule (in which any K level node represents a fixed list of K-1..0 level nodes, to put it in a different, and more concise way, from the "picture/word" definition in my previous message), you could have two different sets of B area boundaries, which give two different abstractions of the underlying topology when you look at the details of A. If you allow abstractions which permit virtual nodes and/or virtual arcs, it gets even more open, since you may advertise one set of internal links (among border nodes) to some people, and a different set to another. Finally, Martha has this idea (from IDPR) where an area has a representation which is sort of intermediate between i) a single simple K-level node, (A) and ii) the set of K-1 level nodes (B0...Bn) and their associated arcs. In it, a node consists of a list of entries into the node, a set of attributes which apply to all pair-wise connections across the node, and an exception list which provides, for each entry in source-dest pair list, the attributes of picking that particular pair for a transit path. Of course, this is simply a more compact representation of a graph model of the area which has these characteristics, and perhaps the extra efficiency of this design is not worth the complexity, but it's something we should think about. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa12091; 8 Dec 93 12:36 EST Received: from pizza by PIZZA.BBN.COM id aa29273; 8 Dec 93 12:14 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29269; 8 Dec 93 12:13 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa10851; 8 Dec 93 12:11 EST Received: by ginger.lcs.mit.edu id AA24695; Wed, 8 Dec 93 12:11:10 -0500 Date: Wed, 8 Dec 93 12:11:10 -0500 From: Noel Chiappa Message-Id: <9312081711.AA24695@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Model conflicts Cc: jnc@ginger.lcs.mit.edu I've been thinking about what do the nodes and arcs *represent*, and I'm starting to realize that we have an inherent conflict. We have to have them represent real, physical assets (and things about them, such as the fact that all the interfaces to a single physical network can generally communicate directly) since these real things are what the traffic flows over. We also want to have them model various kinds of people-oriented abstractions such as policy regions, and there often is an inherent conflict between these two goals. Getting one abstraction hierarchy, and thus one set of names (i.e. locators, since the locators come from the abstraction hierarchy) to do both can be tricky. Consider, for example, the situation where we have a single network N, which is the place when some organization's external links attach. It has, e.g. two attached routers, E which is external (i.e. owned by the service provider, and connects to the rest of the internet), and I, which is internal (i.e. connect to the rest of the site). You want E to be part of the provider's "abstraction", but its interface to N cannot be part of that abstraction since it needs to be "associated" with the N abstraction if it is to have a locator which is useful. You cannot make the interface part of the N abstraction however, since you wish to be able to show (perhaps by means of an abstraction) that it, together with the other interfaces, form a functional unit, E. So, the interface is neither fish nor fowl; the two different, and conflicting, requirements don't allow a choice. It's almost as if such an interface *cannot* be a node, but must be an arc: at the very least, it must be a node with the very special property that its locator is derived from the node it is connected to. This, of course, leads to thinking about routers themselves. My original thinking was that they did not have locators; only the networks had locators, and interfaces derived their locators from the network they attached to. Hmm. This all requires some thought, but I have a feeling that progress is being made, believe it or not! :-) Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa00983; 8 Dec 93 18:43 EST Received: from pizza by PIZZA.BBN.COM id aa01135; 8 Dec 93 18:26 EST Received: from BBN.COM by PIZZA.BBN.COM id aa01131; 8 Dec 93 18:25 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa00350; 8 Dec 93 18:25 EST Received: by ginger.lcs.mit.edu id AA27584; Wed, 8 Dec 93 18:24:13 -0500 Date: Wed, 8 Dec 93 18:24:13 -0500 From: Noel Chiappa Message-Id: <9312082324.AA27584@ginger.lcs.mit.edu> To: jmh@anubis.network.com, nimrod-wg@BBN.COM Subject: Re: More open points Cc: jnc@ginger.lcs.mit.edu MMDF-Warning: Parse error in original version of preceding line at BBN.COM On the question of what a boundary may bisect, I suggest that an area/aggregation/abstraction boundary should be able to bisect a node. Every time I think about this, my head hurts. It's yet another aspect of the point I just made, that sometimes we want to put the boundaries in different places for different reasons, and they clash. Here, you put one boundary around the thing that became the node, and want another one across it. I'd like to be able do describe an area by enumerating the nodes which are in it, which makes partial node inclusion problematic. (I'm assuming that whatever it is you are trying to do can't be handled by having the two areas overlap so that the whole node is in both areas.) What I am coming down to saying is that you can't put an area boundary across a node. What you *can* do is, as you point out later on, provide a more complex representation of the thing that the node represents (i.e. a number of nodes and arcs), and draw the area boundaries in such a way that one area includes some by not all of the nodes which made up the node you wanted to "split", and so on for the other area. Does this do what you want in all cases? To Explore this further, one must think about what a node and a link represent. Ah yes, the eternal question! :-) I tend to follow the old school that nodes generally represent attached devices, or aggregates of some kind, while arcs represent links/interfaces/connections. Yup, I'm heading in that direction (again, see the previous message), but there are difficulties. For instance, if interfaces have attributes, then we are faced with having, e.g., router nodes which have (either as an attribute, or part of the basic "router" datatype [whether routers are done as a separate basic node type, or an attribute]) some number of interfaces, in addition to whatever other attributes the router has. Now, if interfaces have attributes, we are faced with attributes having attributes! (Was it DAB who said that all non-trivial protocols should send S-expressions? I'm starting to sympathize!) Having said that, we run into the fact that while a router is an aggregate, it is a physical aggregate. The logical notion of what is "within" it is rather more interesting. It may well be that half the router is managed by one organization, and half by another. It would seem to me that the aggregation boundary possibly should run down the middle of the router. Yup, again, this is a case of different applications of the map wanting to have the abstraction boundaries in different places. Having said that, I will note that the alternative is to require that any "node" which you want to "split" be represented as more than one "node" in the most detailed graph. Then, one is never running a boundary through the middle of the graph. You mean "middle of the node", right? (Of course, one has created a virtual node and a virtual link, and probably confused the local site administrator, who is not used to thinking in those terms. (If the router is truly set up to be separately managed, then it probably is correct to model it as two nodes. However, I think that this situation may arise without the degree of modelling/support within the router, based on cooperation between the controlling entitites. Yes, but again, the problem is that if you do this, you are in conflict with the desire and ability to model that unitary physical box as a single object. Given the existence of several sets of "multi-link" procedures, operating already at several layers of abstraction, the ability to combine several "equivalent" arcs into one arc would seem a natural abstraction. Care clearly must be taken in such cases. Yes, there are many uses for such a thing; load-sharing, invisible redundancy, etc, etc. If the various arcs represent interfaces with varying attributes, combining them could be difficult, though. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa00261; 9 Dec 93 9:16 EST Received: from pizza by PIZZA.BBN.COM id aa03409; 9 Dec 93 8:49 EST Received: from BBN.COM by PIZZA.BBN.COM id aa03405; 9 Dec 93 8:45 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id ab28075; 9 Dec 93 8:47 EST Received: by ginger.lcs.mit.edu id AA00105; Thu, 9 Dec 93 08:47:10 -0500 Date: Thu, 9 Dec 93 08:47:10 -0500 From: Noel Chiappa Message-Id: <9312091347.AA00105@ginger.lcs.mit.edu> To: ramanath@BBN.COM Subject: Re: Abstraction taxonomy Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM > I can visualize many different possible abstractions (i.e. models) of > the topology inside the area boundary. Among them, in approximate order > of increasing complexity are: > #1: A single node, to which the six "external" arcs connect. > #2: The several border nodes, which are connected in some irregular > fashion, but with complete internal connectivity. > #3: The several border nodes, which are fully interconnected internally; > e.g. 15 arcs (from the (N^2 - N)/2 formula, assuming 6 border nodes), > allowing exact metrics of all internal inter-border paths to be > modelled. > #4: Several border nodes, each of which connects to a center node; this > allows different metrics to be put on each arc to the center node, > giving a less uniform view of the internal connections between the > six entry arcs than #1, but not as optimal as #3. > #5: Several border nodes, which are connected internally in some > irregular fashion, using a number of internal nodes. > #6: The null abstraction (i.e. the original topology). It seems to me that #3 is just a special case of #2; and #4 and #6 are special cases of the more general #5. For instance, if border nodes are "fully interconnected internally" (#3), then they are also "interconnected in some irregular fashion"(#2). Yes. My list was not intended to a division of all possible abstractions into fundamental classes, but simply a list of examples, so that people could have some idea of the range of possible abstractions. Thus, we need to worry about the recursive abstraction scheme only for #1, #2 and #5. If we can fix that, the others follow. Not quite. The "recursive strict abstraction" rule cannot produce #3, unless the original topology was fully connected. The problem lies in the loose definitions of #2 and #5; if you make the rule that these irregular connections are a subset of the original topology (as implied by the recursive strict abstraction rule), then #3 is no longer a special case of #2. Actually, there is another abstraction scheme that may or may not fit in with your taxonomy. In the situation that in #3 above, many of the arcs have the same metrics, we can group the arcs so that all arcs within a group have the same metrics. In many cases, we might have one large group and a few small ones. Then, we can specify the few small groups and just say "for the rest, the metrics are....". This does not lose information and cuts down the number of arcs that must be represented greatly. However, it assumes some uniformity amongs the arcs. I'm not sure it is a different abstraction scheme so much as it is a different, and more efficient (as long as some arcs have the same attributes) representation for a certain class of abstraction. The topology you present is still #3, just represented more efficiently. If we decide this representation is useful, we would of course have to add it to Nimrod at the time the protocols are designed. As far as which abstraction classes (of the four) we allow, I'm starting to think that in the same way that Nimrod, in some sense, cannot "prevent" use of another routing algorithm inside an area which does not disclose details of the internal connectivity, we cannot prevent use of any particular class of abstraction. Even if we did not support virtual arcs, say, an area could always provide an abstraction which included them, and refuse to provide the underlying toplogy which would show that no such arcs existed. However, we still to have to decide if we wish to support virtual arcs and nodes, since I expect that a certain amount of support (e.g. a "virtul" attribute, to indicate that you cannot dive past the abstraction to the underlying entities) would be useful. > However, there is an important division (actually, it's a multi-way > split, but let's hold the details for a bit) that can be made among > possible abstractions: not all *possible* abstractions can be produced > by what I call a "recursive strict abstraction" rule, in which *neither* > nodes *nor* arcs can be created out of thin air. ..... Here's how you > do #4. Draw an area boundary around all the internal nodes of area A. > Call this area B. Now, redraw the topology with area B replaced by a > node, and the area boundary for A as before. The topology inside A is now > exactly #4, except that some of the border nodes may have multiple arcs > to B. (If you have a rule which allows you to drop arcs, or group ones > which join the same pair of nodes into a single arc, then it is exactly > #4.) I guess I am not understanding the rule. Another way to look at the rule is that for each area node, you can enumerate the nodes which are within it (i.e. the nodes which compose that area node). The area boundary is in some sense equivalent (i.e. it contains the identical information) to this list of contained nodes. If any of these nodes are themselves area nodes (as opposed to basic nodes), you can repeat this process. Also, I'm not proposing this for the abstraction rule in Nimrod; it's just a simple and clear one I like to use as an example. Maybe your answer to the following concern will dispel my incomprehension: When you use abstraction #4, say 2 times within an area, it seems that you will have 2 concentric circles around a node. Right? Well, three, since the outermost area (A) will still include it as well as the two inner ones (B and your new one). One of the original nodes would be part of three areas, instead of two, and thus potentially displayable in the map as either i) "part of" three different area nodes (actually, subsumed within those nodes), depending on which abstraction you chose to have in your map, or ii) its original self (assuming you chose to dive past all the abstractions which include that node). Call the node X, the circle around it area B and the outermost circle area A. You've reused a label. A was the name for the area in which this was all happening... Now there is an arc from a border node on A to a border node on B and another arc from a border node on B to the node X. Not necessarily. If I have a set of basic nodes, and draw an area boundary around them, and then draw a new area boundary around that area and one other basic node, all the external arcs from the original area (except any which go to the node which was just included in the new area) are now bisected by two area boundaries; the original one and the new one. To put it another way, if a basic node was a border node of the first area when the area was drawn, drawing a second area around the first will leave it a border node of the second area if the original node has an arc which is bisected by the second area boundary; i.e. the second area did not avoid bisecting the node's arcs by including all the previously unincluded nodes to which that original node was connected. If this abstract description has confused everyone, let me know and I'll send out an example you can draw up. Anyway, so there is not necessarily a "border node on B" which is a different node from X. But to have abstraction #4 for the area, you need to have an arc from a border node in A directly to X. I don't see how this follows. Go back to my original example, and assume that there is a node, I, inside A, but which has no arcs to any of the border nodes of A. Thus, when I draw the area boundary for B, there is no arc which is connected to I which is bisected by that area boundary. Thus, I find it perfectly plausible to have nodes in A which don't need to have an arc to a border node in A. We appear to be severely non-communicating, but let me put #4 a different way, and see if this helps. Inside A, we have a set of non-border nodes. If I draw an area boundary B, inside A, which includes all the non-border nodes of A, then no matter how many further area boundaries I draw inside that area boundary, the list of basic nodes which constitute that area B cannot change, right? Even though I may draw another area C, around some of the nodes in B (turns out it doesn't matter if you include border nodes or not, actually....), so that the constituent node list for B now contains C and some subset of the original contained node list, when I recursively expand C I still get back the same list. The only way I can see this happening is to create a virtual arc using the two real arcs. So it seems that one would have to create arcs "out of thin air"?? I'm afraid I'm so lost at this point that I don't see your point. Let's try this one again... Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa12047; 9 Dec 93 12:31 EST Received: from pizza by PIZZA.BBN.COM id aa04289; 9 Dec 93 11:42 EST Received: from BBN.COM by PIZZA.BBN.COM id aa04285; 9 Dec 93 11:40 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa08772; 9 Dec 93 11:33 EST Received: by ginger.lcs.mit.edu id AA01074; Thu, 9 Dec 93 11:33:33 -0500 Date: Thu, 9 Dec 93 11:33:33 -0500 From: Noel Chiappa Message-Id: <9312091633.AA01074@ginger.lcs.mit.edu> To: kasten@ftp.com, nimrod-wg@BBN.COM Subject: Re: blasts from the past Cc: jnc@ginger.lcs.mit.edu From: Frank Kastenholz I've been going over the archives to router requirements and an interesting point ... that ... might have applicability to Nimrod... A big thread of the discussions were on TOS -- should it be strict ... or should it be advisory ... It seems to me that this is an issue that we will face here in Nimrod-land. Are the policies, etc, that are in the routing, and that hosts request when sending packets (or setting up flows or whatever) mandatory or advisory? Or will we ignore the problem and allow the policy terms to be tagged as either advisory or mandatory? Well, remember, the policies which Nimrod will be advertising in the topology database are not requests from clients (which is what the RReq was talking about), but statements from suppliers. I'm not sure I see how such policies can be anything but mandatory; if I as a suppler have a policy, but it's advisory, what it anything happens if someone ignores the advice? Isn't such a policy a NOP? Can you think of a counter-example? Yes, we will have to consider the issue when it comes to route selection; then, the user will have to make plain to the agent computing the route (assuming they are two separate entities) what its mandatory and advisory policies are. However, unless we are going to define an interface between a client and a route selection agent, we won't get into that, and in any case, route selection is a local issue. (of course, when we get to that point, we will have to define what happens mandatory policy terms can not be fulfilled.) Yup, but again, that's local. From: Kenneth Carlberg my opinion is that the policy terms should be elastic enough to convey if the policy is mandatory or advisory. TOS presents a harder problem because it is more of a yes/no issue. Gee, I would have put it the other way round! If my application needs a certain amount of bandwidth, or a certain maximum delay, I need to be able to quantify that numerically, no? I believe that the policy terms, as presented by Dave Clark (can't remember the RFC #), could be used to convey mandatory/advisory status. RFC-1102, "Policy Routing in Internet Protocols". I haven't fully swapped this in (it's been a long time since I read it), but do recall that his model seemed to imply a different balance as to who decided what about the path between the intermediate routers and the source (or agent of the source) than Nimrod seems to. Still, it's a good pointer, and it's probably worth fully rereading; it might be useful for my new datagram mode. But this leads to another question. If NIMROD is considered a next generation of IDPR, then does this imply that the same structure (specification ?) used for IDPR's policy terms are to be used for NIMROD's policy terms? I don't think there is any such restriction. We certainly want to draw on the valuable experience of IDPR wherever possible, and I also assume they learned some things about what not do do, but I don't have an a priori model that says we do or do not have to follow them. Note: given the early development of NIMROD, I'll concede to arguements that structure/specification of policy terms is premature at this point. This too is also true! Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa03994; 9 Dec 93 18:24 EST Received: from pizza by PIZZA.BBN.COM id aa06276; 9 Dec 93 17:58 EST Received: from BBN.COM by PIZZA.BBN.COM id aa06272; 9 Dec 93 17:55 EST Received: from TIEO.SAIC.COM by BBN.COM id aa02906; 9 Dec 93 17:52 EST Received: by tieo.saic.com (4.1/1.34) id AA00894; Thu, 9 Dec 93 17:54:40 EST Date: Thu, 9 Dec 93 17:54:40 EST From: Kenneth Carlberg Message-Id: <9312092254.AA00894@tieo.saic.com> To: nimrod-wg@BBN.COM Subject: mobility and NIMROD Hello, The following are some thoughts about mobility within the context of NIMROD. It was originally sent to Martha, but she asked that it be distributed to the list. Some of this is a rehash of previous discussions, but it seemed helpful to clump it into one note. ------------------------------------------ Synopsis: - I feel that the impact of movement should be localized as much as possible - I'm a strong proponent in using classical approaches, such as reverse path forwarding, as a means of updating entities that communicate with a mobile EID - I feel that resolvers, as opposed to routing updates, should be one of the primary means of distributing the mobile EID <-> locator association - I would like the responsibility of supporting mobility placed on the routing architecture as opposed to the hosts ------------------------------------------ The following complement and/or add to the above 1. Premise #1 is that movement should be classified as: 1) entities that are PORTABLE 2) entities that are MOBILE This is a position that I first brought up on the Big-Internet list last year. The reason I thought that movement should be broken into these two classifications are as follows... Rationale: From my perspective, support for mobility will entail more effort than support for portability. With portability, one only (yes, I use this term lightly ;-) needs a resolver to translate an EID with the current routing location that is assigned to it. Ideally, this resolver would work within the confines of a distributed schema much in the way DNS operates. (note: as it currently stands, I don't think the current incarnation of DNS will be able to support the EID <-> locator association). In reality, it would seem that this same DNS-like functionality is needed for any normal static (non-moving) entity that wishes to forward traffic to a given destination EID. Other issues like caching time (for portable EIDs) and explicit disconnect notification of the portable EID are items that are more of an engineering issue and could be hammered out at a later date. On the other hand, support for mobility (which in my mind should be considered a superset of portability) is really a balancing act that weighs: (a) notification of movement with (b) minimizing system overhead in its attempt to maintain end to end connections as the entity/EID moves. I'll expand on this below, but for now I'll mention that I have always been a fan of reverse path forwarding as a primary means of dealing with the above balancing act. 2. Premise #2 is that the impact of mobility should be localized. First, a couple of definitions: (a) authoritative cluster - a cluster that has authority over some group of clusters (eg. SAIC.COM has authority over TIEO.SAIC.COM) (a) system wide notification - explicit notifications (or redirects, for lack of a better term) that are sent to other authoritative clusters in order to notify other entities that the destination EID has moved to another routing location. I'm not a fan of proposals for supporting mobility that requires system wide notifications _EACH_ time an entity moves from one routing location to another*. And while I believe the impact of mobility should be localized, I don't think one can totally eliminate system wide impact of mobility in that once an entity moves from one authoritative cluster to another, some type of notification will need to be made. *sidenote: Please don't take this comment as a direct attack against the current draft in the mobile-IP wg which is constrained by IPv4 As an example, if we consider Cellular-1 as one cluster serving the mid-atlantic region of the US, then a car that moves within Cellular-1's cluster does not cause system wide notification each time it moves between cells, base stations, cities, counties, or even states. But when the car moves out of Cellular-1's operating area into Bell-South's operating area, then some type of notification/update would be generated. The point being that through the prudent use of a hierarchical network, one should be able to localize mobility so that if say my machine moves within SAIC.COM, then no entity outside of SAIC.COM needs to be informed about my movement. But this leads to how detailed the connectivity maps should be when an entity, say at BBN, is requesting where to route traffic destined to an EID that happens to be at SAIC. It would be my belief that the most generalized map of SAIC would be the one that is distributed outside of SAIC. Thus updates for the mobile EID remain localized within SAIC. The distributed map, of course, could be more generalized so that CERFnet (SAIC's provider) becomes the routing location associated with the mobile EID, or it could be more specific such that TIEO.SAIC.COM is the associated routing location. I won't get into the tradeoffs in this note. 3. Routing: source vs. hop-by-hop One interesting item is, how does one route to the mobile EID ? Right now I don't have an absolute answer one way or the other. The use of setup and teardown protocols to establish source routes would seem to be expensive for mobile EIDs. If source routes are to be used, then I would envision them being established via the use of a recorded route that was generated by traffic sent from the mobile entity. The recorded route could simply be a partial one that at a minimum contains the authority cluster that "contains" the mobile entity/EID. Furthermore, within the authoritative cluster, perhaps hop-by-hop forwarding (or source routing sans a setup procedure) could be used. Again, I'm not totally commited one way or the other. 4. Who supports mobility ? In the synopsis, I stated that I would like the responsibility of supporting mobility placed on the routing architecture as opposed to the hosts. This thought stems more from a configuration/ease-of-use desire as opposed to any specific technical criteria. Simply put, I would rather have one basic set of internetworking software in my host that can work on ANY platform (from a Newton, or a Sun, or...) instead of having to install add-ons to a host that _might_ be portable/mobile. Well, that my $0.02, -ken   Received: from PIZZA.BBN.COM by BBN.COM id aa21646; 10 Dec 93 5:48 EST Received: from pizza by PIZZA.BBN.COM id aa08082; 10 Dec 93 5:24 EST Received: from BBN.COM by PIZZA.BBN.COM id aa08078; 10 Dec 93 5:22 EST Received: from shark.mel.dit.CSIRO.AU by BBN.COM id aa20912; 10 Dec 93 5:08 EST Received: by shark.mel.dit.csiro.au id AA07310 (5.65c/IDA-1.4.4/DIT-1.3 for nimrod-wg@bbn.com); Fri, 10 Dec 1993 21:08:50 +1100 From: "Harold A. Miller" Message-Id: <199312101008.AA07310@shark.mel.dit.csiro.au> Subject: re: mobility To: nimrod-wg@BBN.COM Date: Fri, 10 Dec 1993 21:08:50 +1100 (EST) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 4848 push lurk() Ken Carlberg's recent note has me interested enough to stick my neck out. I agree with most of his points, yet disagree with how to put them together. A very quick groundwork: the project I'm playing with deals with a non- hierarchical dynamic net, entailing hosts that jump not only between routing forwarders but between protocol stacks. Gets a little beyond most of what NIMROD does, but still analogous in many ways. > 1. Premise #1 is that movement should be classified as: > > 1) entities that are PORTABLE > 2) entities that are MOBILE Agree fully. To go slightly out of Ken's order, ... > On the other hand, support for mobility (which in my mind should be > considered a superset of portability) ... This is the part I don't like at all. In my view, portability is a host breaking physical connectivity, relocating to a potentially completely different environment, then re-establishing communication. This may occur with sufficient speed in a given instance that, if the relocation didn't involve change of protocol stack, it might appear that the host always maintained it's connection. Taking that to the extreme, given no extenuating circumstances this may be exactly what happens--the host actually is able to maintain communications sessions while relocating. This we call "mobility". Thus in my view, mobility is a special case of portability. > From my perspective, support for mobility will entail more > effort than support for portability. I agree with this. In the context of my previous paragraph, ensuring that all connections are kept operational will take more work than restarting things once the host is reestablished. > With portability, one only [...] needs a resolver to translate an EID > with the current routing location ... Not sure I agree here. I think that what we (the NIMROD-routing host) need to ask is "what do I do with this data unit now?" May not sound too different, but I think it is. Where (Ken please correct me if I put inappropriate words into your mouth here) Ken says above that the need is to find where our target is now, I think our need is just to find what physical entity we need to send data to that is going to end up getting our communication to the target eventually. Kind of like a store-and-forward train of thought, but it makes more sense in my world of inter-network-types than in the IP world where it's easier to "assume full connectivity". In my context it makes sense to look for a protocol gateway, which may not be necessary if you're in the same net type, but in that case you really don't need to go beyond the local routing capabilities already in place. Effectively, I'm working with an EID-to NextHop approach. This should apply to an intra-IP situation just as well. Incidentally, I agree with Ken regarding the limitation of DNS to support the EID<-> locator association. And I'm a DNS fan. > 2. Premise #2 is that the impact of mobility should be localized. Again, I agree wholeheartedly. Fits well into my world of inter-network- type portability, where anything intra-net need not escape, and a jump between net-type-A and net-type-B need not interest net-type-C, so long as the inter-net-type gateways can find the rascal when necessary. Besides, as I think Ken is saying, overhead will kill us (wait until you try doing this without a stable hierarchy!) > 4. Who supports mobility ? > > [...] I would like the responsibility of supporting mobility placed > on the routing architecture as opposed to the hosts. While I'm less concerned about configuration/ease-of-use *at this point in time*, I still agree. Hosts moving around will not be able (in general) to keep sufficient track of the changing topology to make good routing decisions. This is even more true in my case where a host might be jumping from IP to XNS to X.25 during a single day. > Simply put, I would rather have one basic set of internetworking > software in my host that can work on ANY platform [...] instead of > having to install add-ons ... Agree fully. Any host ought to be able to go relocating, whether portable or mobile, regardless of which is a subset of which, OR to be able to stay where it is. This also should (someday) apply regardless of the protocol stack upon which NIMROD (or any other internetworking protocol) sits. I guess my reason for writing is to suggest that we ought not to tie our thinking to the IP-type world. I think we ought to consider "routing" between unlike network types, then look to applying those more general rules to each of the network types for near-term design. > Well, that's my $0.02, > > -ken Believe it or not, your note is what finally allowed me to put NIMROD in perspective (or so I think). Well done! And thanks. OK, shoot me full of holes. pop lurk() and go back quiet(). -- HM   Received: from PIZZA.BBN.COM by BBN.COM id aa26037; 10 Dec 93 8:17 EST Received: from pizza by PIZZA.BBN.COM id aa08486; 10 Dec 93 8:00 EST Received: from BBN.COM by PIZZA.BBN.COM id aa08482; 10 Dec 93 7:58 EST Received: from TIEO.SAIC.COM by BBN.COM id aa25101; 10 Dec 93 7:55 EST Received: by tieo.saic.com (4.1/1.34) id AA01041; Fri, 10 Dec 93 07:57:37 EST Date: Fri, 10 Dec 93 07:57:37 EST From: Kenneth Carlberg Message-Id: <9312101257.AA01041@tieo.saic.com> To: Hal.Miller@mel.dit.csiro.au Subject: re: mobility Cc: nimrod-wg@BBN.COM Hal, > > On the other hand, support for mobility (which in my mind should be > > considered a superset of portability) ... > > This is the part I don't like at all. In my view, portability is a host > breaking physical connectivity, relocating to a potentially completely > different environment, then re-establishing communication. I agree. I guess I did not elaborate enough on my distinction between the two. By "superset" I meant that where portability dictates the need to discover where the destination device has "popped up", mobility must take this one step further (ie. the discovery process) and attempt to maintain connections as the mobile entity moves. > > With portability, one only [...] needs a resolver to translate an EID > > with the current routing location ... > > Not sure I agree here. I think that what we (the NIMROD-routing host) > need to ask is "what do I do with this data unit now?" May not sound > too different, but I think it is. A model I had in mind goes something like this... o the host transmits a packet, to some default gateway, using an EID as the destination o the Nimrod router (be it the first or some other default gateway) "sees" the destination EID and resolves it into the current location and forwards the packet via a source route In a sense, this is similar to the work done by IDPR in that traffic from the source domain was forwarded to a policy gateway who in turn resolved the destination IP address into a destination domain identifier. This identifier was then used as the basis for generating/establishing a source route to the destination domain. So perhaps we are in agreement with your notion of EID-to-NextHop approach. -ken   Received: from PIZZA.BBN.COM by BBN.COM id aa01600; 10 Dec 93 9:43 EST Received: from pizza by PIZZA.BBN.COM id aa08799; 10 Dec 93 9:18 EST Received: from BBN.COM by PIZZA.BBN.COM id aa08795; 10 Dec 93 9:17 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa29448; 10 Dec 93 9:16 EST Received: by ginger.lcs.mit.edu id AA08384; Fri, 10 Dec 93 09:16:04 -0500 Date: Fri, 10 Dec 93 09:16:04 -0500 From: Noel Chiappa Message-Id: <9312101416.AA08384@ginger.lcs.mit.edu> To: carlberg@tieo.saic.com, nimrod-wg@BBN.COM Subject: Re: mobility and NIMROD Cc: jnc@ginger.lcs.mit.edu From: Kenneth Carlberg The following are some thoughts about mobility within the context of NIMROD. Well, I'm glad someone has been thinking about this! I certainly haven't had time; been thinking about graphs a lot! :-) Let me take the opportunity to encourage people to take topics like this that interest them, and go off and think about them in a similar way. A few comments: - I feel that the impact of movement should be localized as much as possible If this means "info about the move being kept near the thing which moved, the things which want to talk to it, and in some minimal way in a database which mediates between the two", yes. If this means "kept near the thing which moved", I'm not sure I agree. Here's the problem: S is talking to D, and D moves a long way. S may wish to set up a new path directly from itself to D, but to do this the knowledge that D has moved has to be propogated to S. - I feel that resolvers, as opposed to routing updates, should be one of the primary means of distributing the mobile EID <-> locator association Yes, absolutely. Each destination the routing is forced to handle adds to the load on the routing over the scope which that destination needs to be visible in. In a world-wide network, there's simply no way to have the routing support mobile/portable entities. (I'll get to local scope mobility below.) - I would like the responsibility of supporting mobility placed on the routing architecture as opposed to the hosts Doesn't this conflict with the statement above? How exactly is the routing going to do it, if this data is not in routing updates? To go back to my example above, how can you reroute the data without getting S into the act? In the "real world", when people move, the "system" doesn't handle it all for ever; the people who want to talk to you, send mail, etc do eventually have to find out that you have moved... movement should be classified as: 1) entities that are PORTABLE 2) entities that are MOBILE This .. first brought up on the Big-I list Yes, I liked that distinction. support for mobility will entail more effort than support for portability. Yes. In the portability case, one can assume that a static source will go through some system database before trying to reach the destination; this gives you the opportunity to modify that database as a single-point (modulo redundancy) way of effecting the change. In the mobility case, there is no system database which records who is currently talking to the destination; the destination has to cause them all to be notified. With portability, one only ... needs a resolver to translate an EID with the current routing location that is assigned to it. Well, I've always had the vision that you'd translate a DNS name into the (EID, locator) tuple, and carry them around together from then on. This handles portability, but not mobility. However, mobility has to involve the destination, so that's OK. I'm a little bit leery of having to maintain an EID-> translation system, except as a debugging/fault investigation aid. Since the EID's are allegedly "flat", this might be hard to do. E.g., imagine a database which translated IEEE 48-bits to anything... Ideally, this resolver would work within the confines of a distributed schema much in the way DNS operates. (note: as it currently stands, I don't think the current incarnation of DNS will be able to support the EID <-> locator association). Well, it's not impossible; it would have to be done as a "white pages" type service by some trusted agency (e.g. the ISOC). What about the DNS -> (EID, locator) translation? Could DNS handle that? The real problem that I see there is not so much changing the database entries when that mapping changes (any entry which can change has to have a short timeout; after you change the entry, you have to make sure that change propogates to all authoritative copies, then you wait for the timeout before you comsummate the move, since you can then be sure that nobody has a copy of the old mapping any more), as doing that securely. Can this be done within the context of DNS, perhaps with some mods, or do we need a whole new system? (My friends who are DNS experts say DNS is totally screwed up anyway, so maybe we will get the chance to get our stuff in!) Other issues like caching time ... and explicit disconnect notification ... are items that are more of an engineering issue Doing that last one securely is hard to do securely without authentication. Also, maybe I'm slow this morning, but what exactly are the problems you see with maintaining an EID-> mapping? On the other hand, support for mobility ... is really a balancing act that weighs: (a) notification of movement with (b) minimizing system overhead in its attempt to maintain end to end connections as the entity/EID moves. I'd put it differently. It's a balancing act between how much of the mechanism you put in the hosts (where it's an easy problem) and how much you expect the system to do for you automagically (where it becomes harder). There are of course intermediate approaches where, with some loss of efficiency and flexibility, you get some reasonable compromise. The base station approach, where the packets take two sides of a triangle, is one. Notifying the first hop router is another, and both allow you to keep the static host out of it. Still, there's no avoiding the fact that if the static host was involved in picking the path (a key capability in Nimrod, although you don't have to use it), there's *no* solution whcih gives you back the full power. So, these two together (difficulty, and loss of capability) drive me towards involving the static host. It's not that much of an upheaval, really, and it makes the whole thing much cleaner. I'll expand on this below, but for now I'll mention that I have always been a fan of reverse path forwarding as a primary means of dealing with the above balancing act. For those of us who are out of touch, could you just give a sentence or two on how RPF is used in the mobility context? (a) system wide notification - explicit notifications (or redirects, for lack of a better term) that are sent to other authoritative clusters in order to notify other entities that the destination EID has moved to another routing location. This is a slightly unfortunate choice of terminology. Explicit notifications that a host has moved, to some (smallish) set of database sites which are authoritative for a particular translation, is not necessarily anything like as expensive as the term "system wide" would imply... They are certainly a heck of a lot cheaper than getting the routing involved! I'm not a fan of proposals for supporting mobility that requires system wide notifications _EACH_ time an entity moves from one routing location to another*. Why not (assuming that "system wide" in this context means updating authoritative translation database sites)? Getting all the routers which are potential last host to remember that the host has moved isn't any cheaper, really, for example? I don't think one can totally eliminate system wide impact of mobility in that once an entity moves from one authoritative cluster to another, some type of notification will need to be made. ... The point being that through the prudent use of a hierarchical network, one should be able to localize mobility so that if say my machine moves within SAIC.COM, then no entity outside of SAIC.COM needs to be informed about my movement. Ah, I see what you want to do. You want to associate an EID with a routing scope, and, once tarffic for that EID reaches that scope, have the routing inside the scope responsible for tracking the destination (sort of the way IS-IS allows you to make several LAN's a level 1 area, and hosts can move from one to another with nobody the wiser). Here's my chief bug with that approach. The larger you make the scope, the more expensive it gets to support this capability; as I mentioned, the load on the routing gets larger, since while the cost of tracking a single destination grows only slowly as the scope gets larger (the total costs per destination grow substantially, but that cost is amortized over all the switches), the larger scope will contain more and more of these things, and that's what blows the costs. So, that limits the scope size, and in fact the smaller you make the scope (to reduce those costs), the more moves turn into extra-scope moves. Now, for moves *outside* the scope, we have to have some other mechanism, right? Presumably, this mechanism gets invoked enough that it has to be reasonably efficient? So, why have the first mechanism? Why not use the wide-area mechanism everywhere? Here's a *sample* cost-benefit analysis. (I say sample because I'm going to pull the numbers out of thin air, in the best system architect style! :-) Let's assume the system wide mechanism is twice as expensive (ignoring the extra overhead in the routing, which is substantial), and 50% of all moves are extra-scope. (I don't think that's unreasonable; portable computers are *very* common now, and people fly around a lot. Most people do that more than they move their workstation inside their company. Heck, it might be *more* than 50%!) So, if we have both mechanisms, we will save only 25% on overhead, versus if we had only the system-wide mechanism. Is that enough savings to justify the complexity, etc of two mechanisms? Not in my book. There are secondary consideration, such as: suppose the networks inside the scope do not have identical QOS, but the static host outside has specific QOS requirements? It might not be possible for the routing inside the scope to operate on the static hosts's behalf. Then there are resource allocation issues. No, I think that if something moves, we can't hide it under the sheets; we have to let the far end know. Of course, there may be people who don't want to let info about these kind of moves go outside their scope. For them, they have to build a mechanism which does just what you suggest. It would be my belief that the most generalized map of SAIC would be the one that is distributed outside of SAIC. Thus updates for the mobile EID remain localized within SAIC. The map may be widely distributed outside SAIC, but the location of the EID only has to be distributed to things which wish to communicate with that EID. The use of setup and teardown protocols to establish source routes would seem to be expensive for mobile EIDs. Depends on how mobile they are. If you get off 2 packets before they move, yah it's probably too expensive. If you get off 100, you've got a pretty good amortization going there... If source routes are to be used... The recorded route could simply be a partial one ... within the authoritative cluster, perhaps hop-by-hop forwarding ... could be used. I need to finish the note on my proposed support for datagram mode. Basically, packet would contain the destination locator, a pointer into the locator (for correctness, as well as efficient, reasons), a flow-id which was *not set by the source, but bashed about by intermediate routers*, and maybe a few other bits. So, as long as the static host has the up-to-date locator, it could use this the whole way. Of course, a source route is also an option, as you mention. In the synopsis, I stated that I would like the responsibility of supporting mobility placed on the routing architecture as opposed to the hosts. Yes, but suppose the *total* overhead of doing that is larger than getting the hosts involved? This thought stems more from a configuration/ease-of-use desire as opposed to any specific technical criteria. Simply put, I would rather have one basic set of internetworking software in my host that can work on ANY platform ... instead of having to install add-ons to a host that _might_ be portable/mobile. This is a good point. There are two phases. First, we can't have a solution in which you have to add software to static hosts to interoperate with mobile hosts. However, if it's simple enough (and I believe it is), I think it reasonable to include that code with all hosts. Second, the point you make; you'd like to not have to futz around with the code in the potentially portable/mobile host to actually make it portable/mobile. However, what's wrong with simply putting such an IP layer in all hosts which are even potentially mobile? Heck, for all I know, it's simple enough code that maybe we can just put that too, in all hosts. I mean, don't all BSD derivatives include an IP layer which has basic support for being a router in it (even though 99.9% of such boxes don't use it)? Be bold! Don't assume we can't go into the hosts, *especially* if the Right Solution involves them. Sure, we have to have an interoperability scheme for deployed boxes, but we have to have that anyway for basic Nimrod, and remember, Nimrod will eventually mean changes to most hosts anyway. I'd rather pay the price, and do it right. In my experience, any time a system designer doesn't do it right, it eventually winds up costing more than if you'd bit the bullet.... Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa15036; 10 Dec 93 13:48 EST Received: from pizza by PIZZA.BBN.COM id aa09915; 10 Dec 93 13:27 EST Received: from BBN.COM by PIZZA.BBN.COM id aa09911; 10 Dec 93 13:25 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa13664; 10 Dec 93 13:21 EST Received: by ginger.lcs.mit.edu id AA09958; Fri, 10 Dec 93 13:20:56 -0500 Date: Fri, 10 Dec 93 13:20:56 -0500 From: Noel Chiappa Message-Id: <9312101820.AA09958@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM Subject: Open issue wrt Nimrod paths Cc: jnc@ginger.lcs.mit.edu Here are a couple of things I found scribbled on a small piece of paper: - What are the elements of a Nimrod path - nets or routers? (In other words, you can specify a path either by specifying the nodes or the arcs; which do we do?) - Can a Nimrod path be incomplete in the middle; i.e., can it contain detail near the source, and near the destination, but not say anything about the path in the middle? - How do we do "downward" routing; i.e. if a path is created using a map in which only a very high level abstraction is shown for the destination, will some agent closer to the destination make up the rest of the detail? Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa06855; 11 Dec 93 18:12 EST Received: from pizza by PIZZA.BBN.COM id aa14075; 11 Dec 93 17:58 EST Received: from BBN.COM by PIZZA.BBN.COM id aa14071; 11 Dec 93 17:55 EST Received: from Gropo.TGV.COM by BBN.COM id aa06494; 11 Dec 93 17:57 EST Received: by gropo.tgv.com (5.65/DEC-Ultrix/4.3) id AA24339; Sat, 11 Dec 1993 14:56:50 -0800 Date: Sat, 11 Dec 1993 14:56:50 -0800 Message-Id: <9312112256.AA24339@gropo.tgv.com> To: carlberg@tieo.saic.com Cc: nimrod-wg@BBN.COM In-Reply-To: Kenneth Carlberg's message of Thu, 9 Dec 93 17:54:40 EST <9312092254.AA00894@tieo.saic.com> Subject: Re: mobility and NIMROD Reply-To: karl@empirical.com Sender: karl@gropo.tgv.com From: karl@gropo.tgv.com Repository: empirical.com Originating-Client: gropo > The following are some thoughts about mobility within the context of Having wrestled with the notion of nomadic computing for quite some time, I've come to the conclusion that this is a problem for the application layer, not the network layer. That's a pretty radical conclusion, so I ought to say why and how I got there... It all started when I started using the pcmail protocol. That protocol uses names to identify clients to a mail repository. Every time the client establishes a connection to the repository, there is a handshake which says, in essence, "hello, my name is foo. Let's reconcile our views of the repository state". If a connection breaks under pcmail, then the client can just rebuild the transport connection and re-perform the handshake. The import of this is that the network layer, i.e. the routing protocols, don't have to assume that a mobile host is keeping the same IP address. And there is no need to keep transport connections alive during migration. The mobile device can adapt an address, point of attachement name, call it what you will, appropriate for the location in which it is currently operating. Of course, the mobile device must have a unique, never-changed name, but that is known only to the various applications (and presumably some authentication service.) This approach does require that applications build what ammounts to an association state which transcends transport state. It also requires a resonbly efficient means for a mobile device to acquire a useful local address. However, I claim these to be a lesser task than that of keeping track of a moving host, leaving forwarding pointers, trying to short-circuit multiple forwarding pointers into a single pointer, etc (not to mention the potential routing update traffic that might be engendered.) (In addition, I think we need to come up with an efficient means to configure IP hosts on the fly anyway to deal with the fact that TCP/IP is, for many folks, just too painful to configure.) In summary, perhaps nimrod ought to punt mobility up to the applications. --karl--   Received: from PIZZA.BBN.COM by BBN.COM id aa16005; 12 Dec 93 14:20 EST Received: from pizza by PIZZA.BBN.COM id aa16495; 12 Dec 93 14:06 EST Received: from BBN.COM by PIZZA.BBN.COM id aa16491; 12 Dec 93 14:04 EST Received: from TIEO.SAIC.COM by BBN.COM id aa15670; 12 Dec 93 14:05 EST Received: by tieo.saic.com (4.1/1.34) id AA01505; Sun, 12 Dec 93 14:07:41 EST Date: Sun, 12 Dec 93 14:07:41 EST From: Kenneth Carlberg Message-Id: <9312121907.AA01505@tieo.saic.com> To: jnc@ginger.lcs.mit.edu Subject: Re: mobility and NIMROD Cc: nimrod-wg@BBN.COM Noel, > - I feel that the impact of movement should be localized as much as > possible > >If this means "info about the move being kept near the thing which moved, the >things which want to talk to it, and in some minimal way in a database which >mediates between the two", Well...., sort of yes. I'd like to delay, as much as possible, your second item, "the things which want to talk to it". I'll explain below. > - I would like the responsibility of supporting mobility placed on the > routing architecture as opposed to the hosts > >Doesn't this conflict with the statement above? Ugh, a brain lapse. The above is a remnant of personal biases from a previous proposal I had in mind on how to support mobility, so instead of expanding on it, I'll bring up a previous arguement that you had talked about, namely, treating existing IPv4 as EIDs. If this were to be done, then would not the routing architecture (or some component thereof) have to do some type of translation from the EID (IPv4 address) to the current Nimrod representation of the destination entity ? Thus, if the IPv4 entity is mobile, then the routing components of Nimrod would be responsible for supporting mobility. > On the other hand, support for mobility ... is really a balancing act > >I'd put it differently. It's a balancing act between how much of the >mechanism you put in the hosts and how much you expect the ... >system to do for you automagically .... There are of >course intermediate approaches ... The base station approach, >where the packets take two sides of a triangle, is one. The only reason I'd support the use of triangle routing is because I'd want to hide from the source where my mobile entity "really" is (a rationale brought up by Tony Li at the last IETF). But other than that, I'd really like to avoid this approach for IPng because it has the potential for being extremely inefficient. As an example, picture yourself traveling to Japan, "connecting" your portable computer, and having all traffic sent to you by way of Boston (which is especially acute when the originator of the traffic is from Japan)! >Notifying the first hop router is another, and both allow you to keep >the static host out of it. This approach has been presented in the mobile-IP working group as a means of avoiding triangle routing. I'll defer any specifics to the authors or editor of their draft proposal. >Still, there's no avoiding the fact that if >the static host was involved in picking the path (a key capability in >Nimrod, although you don't have to use it), there's *no* solution whcih >gives you back the full power. a good point. >This is a slightly unfortunate choice of terminology. Explicit notifications >that a host has moved, to some (smallish) set of database sites which are >authoritative for a particular translation, is not necessarily anything like >as expensive as the term "system wide" would imply... They are certainly a >heck of a lot cheaper than getting the routing involved! yes, but this goes back to my desire of localizing the impact of mobility. (btw, Yakov pointed me to a draft that he and John Penners wrote titled Simple Mobile-IP [draft-penners-mobileip-smip-00.txt] that touches on this topic). >Ah, I see what you want to do. You want to associate an EID with a routing >scope, and, once tarffic for that EID reaches that scope, have the routing >inside the scope responsible for tracking the destination (sort of the way >IS-IS allows you to make several LAN's a level 1 area, and hosts can move >from one to another with nobody the wiser). Yes. But the scope itself could (and I would rather expect it to) be layered to further limit the impact of mobility. For example lets assume cluster A is the scope, and it is subdivided into A.X, A.Y, A.M. Furthermore, A.M is subdivided into A.M.1, A.M.2 and A.M.3, where objects are only mobile within this third level of cluster A. Then the "tracking" is only done within A.M and A, as well as the source, remains ignorant of these changes. When and _if_ the object moves outside of A, then the source, and any appropriate databases, would be notified. The objective is primarily to delay this notification process, not to eliminate it. A slight digresion...In my mind, I tend to view mobile objects spending _most_ of their time moving within a specific scope. If I had a wireless palmtop or laptop the scope of my movement would more that likely be restricted to the speed that "I" am traveling. This movement could be fast within the context of a floor in my building (ie. moving from office to office) but in the context of my building or an internetwork spanning the world, its inconsequential. Now if you were to present the example of a car or an airplane, then I would say that the scope might be different but it is still a scope. In the case of a car, its scope could be bounded within the operating area of a cellular network provider (whose scope tends to span counties/states). And in the case of a plane, the scope can encompass whole regions of the earth through the use of satelite footprints. The common aspect in these three examples is that the hardware has already provided us with a _loosely_ defined scope. >Here's my chief bug with that approach. The larger you make the scope, the >more expensive it gets to support this capability; ... and in fact >the smaller you make the scope (to reduce those costs), the more moves >turn into extra-scope moves. > >Now, for moves *outside* the scope, we have to have some other mechanism, >right? Presumably, this mechanism gets invoked enough that it has to be >reasonably efficient? So, why have the first mechanism? Why not use the >wide-area mechanism everywhere? The rationale for two mechanisms stems from my opinion that each mechanism has its drawbacks in supporting mobility. Thus, instead of settling on one method, I would rather try to use the strengths of each to support mobility as opposed to being limited by the weakness of one method. I should say perceived weakness since this is rather my perception of things. I expect that the mobile-IP wg will in time be able to provide us with some interesting working experience as well as some simulation efforts that may be triggered by their efforts. >Be bold! Don't assume we can't go into the hosts, *especially* if the Right >Solution involves them. You're right, I do want to avoid going into the hosts. But if it is to be done, then I'd rather do it once _with_ the deployment of IPng as opposed to afterwards. -ken   Received: from PIZZA.BBN.COM by BBN.COM id aa01764; 13 Dec 93 9:48 EST Received: from pizza by PIZZA.BBN.COM id aa19561; 13 Dec 93 9:28 EST Received: from BBN.COM by PIZZA.BBN.COM id aa19557; 13 Dec 93 9:25 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa29778; 13 Dec 93 9:23 EST Received: by ginger.lcs.mit.edu id AA27159; Mon, 13 Dec 93 09:23:28 -0500 Date: Mon, 13 Dec 93 09:23:28 -0500 From: Noel Chiappa Message-Id: <9312131423.AA27159@ginger.lcs.mit.edu> To: karl@gropo.tgv.com Subject: Re: mobility and NIMROD Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM Having wrestled with the notion of nomadic computing for quite some time, I've come to the conclusion that this is a problem for the application layer, not the network layer. I've heard this idea batted around. It's true that attempting to provide a given function at too low a layer is a Bad Thing, but it's also true that providing it at too high a layer results in a lot of useless duplicate implementation of some function. (My favourite example of this is application response timeouts, i.e. you send a command down the TCP stream, and the thing on the other side swallows it, so TCP is happy, but you never get back a response. When doing the Host Req RFC, we had to specify over and over again that applications implement these; oh how we wished the TCP/IP stack had an optional "timed response" layer these applications could use.) When it comes to mobility, I'm not convinced that doing it at the internetwork layer is absolutely the Right Thing, but, on the other hand, I haven't heard what I consider either i) a concrete case that doing it at the internetwork level is an incomplete solution, or ii) a good analysis of why doing it at the transport/application level is more efficient that doing it at the internet layer. It sure would make life a lot easier if this function were provided at the internet layer; all subsequent layers could avoid having code to deal with the issue. For example, it's true that after a move you might not be able to create a path with the same QOS as you had before, but then again, the network topology might change (with a non-mobile host) to make the same thing true. However, nobody has suggested moving routing into the application layer! So, I have a hard time finding something where putting mobility at the internet layer forces a more incomplete solution. Note that I don't include portability, since by definition with portability there is always a new connection. The internetwork layer is only involved to the extent that the portable host wants to operate as a server; if it's just a client, nobody needs to know it has moved, and there no other ramifications. the pcmail protocol ... uses names to identify clients ... The import of this is that the network layer, i.e. the routing protocols, don't have to assume that a mobile host is keeping the same IP address. This is an unavoidable consequence of the fact that in IPv4, the IP "address" is *both* a locator *and* a host-identification name. So, if you change where you are in the network, you have to get a new IPv4 address, and to TCP it looks like a different host. The pcmail "name" is the only invariant label around, so that's what get used. Nimrod explicitly breaks that IPv4 naming function into several parts, precisely to allow the same transport connection (identified by the EID's, which aren't really part of the Nimrod routing architecture) to move from place to place in the network (identified by the locator). Nimrod thus expends *zero* energy in the routing protocols in keeping track of mobile hosts, since Nimrod only concerns tiself with locators. Nimrod (as a routing arch) doesn't even know mobile hosts exist! If a connection breaks under pcmail, then the client can just rebuild the transport connection and re-perform the handshake. Sure, but there's a certain amount of mechanism needed to do this. Multiply that by all the applications which want to be mobile, and you've got a lot, compared to the cost of doing it in the internetwork layer. Plus, applications which *don't* go to the effort aren't mobile; if it's in the internet layer, all applications get it for free. Now, if for some reason mobility *can't* be done right at the internet layer, that would be different... The mobile device can adapt an address, point of attachement name, call it what you will, appropriate for the location in which it is currently operating. But this is just what Nimrod does... Of course, the mobile device must have a unique, never-changed name, but that is known only to the various applications This is what the EID does... but it's at the transport layer. Of course, all applications can use that name if it is good enough for them. This approach does require that applications build what ammounts to an association state which transcends transport state. Yup, and that's mechanism and work, replicated in each such application. It also requires a resonbly efficient means for a mobile device to acquire a useful local address. Yes, but all mobility scheme require this. owever, I claim these to be a lesser task than that of keeping track of a moving host, leaving forwarding pointers, trying to short-circuit multiple forwarding pointers into a single pointer, etc Maybe; as I said, it's the multiplication factor of many applications that gets you. And I'm not sure it is less complex, actually; mail applications already have a lot of what's needed lying around, but others might not. (not to mention the potential routing update traffic that might be engendered.) But there is no such traffic... (In addition, I think we need to come up with an efficient means to configure IP hosts on the fly anyway to deal with the fact that TCP/IP is, for many folks, just too painful to configure.) Amen to that. In summary, perhaps nimrod ought to punt mobility up to the applications. Perhaps, but I still don't hear a good case for either i) why it can't be done at the internet layer, or ii) is more efficiently done higher up. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa00534; 13 Dec 93 16:13 EST Received: from pizza by PIZZA.BBN.COM id aa21410; 13 Dec 93 15:50 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id ab21403; 13 Dec 93 15:46 EST Date: Mon, 13 Dec 93 15:47:04 EST From: Martha Steenstrup To: nimrod-wg@BBN.COM Subject: routing Some opinions about network-layer versus application-layer routing. I am also an advocate of concentrating as much of the routing functionality as possible into the network layer and of minimizing the amount of routing necessary at the application layer. However, this is independent of whether the internetwork contains fixed or mobile elements. Clearly, we don't want to impose redundant routing functions on multiple protocol layers. This unnecessarily complicates protocols, making them harder to manage and less efficient. Moreover, network-layer protocols can quickly learn about the state of the network and thus are in the best position to make control decisions based upon the network state. The delay incurred in delivering network-layer information to the application layer and in the application layer acting upon this information may make fine-grained routing control difficult if not impossible at the application layer. However, there are applications that do wish to perform some of the routing functions. For example, the X.400 Message Transfer Agents (MTAs) act as application-layer routers, using application-layer information contained in the envelope of the X.400 message as well as information about neighboring MTAs in order to select the next-hop MTA for the message. In this case, application-layer and network-layer routing are complementary. The application layer determines the coarse-grained sequence of MTAs, while the fine-grained routing necessary to get an X.400 message from one MTA to the next are left to the network layer. In general, application-layer routing (for a particular application) may wish to factor in the performance of the application or of a set of hosts, in selecting a peer and the "best" route to that peer. Information about host or application performance is not part of the network-layer information but is instead known by the application or obtained through network management. In this case, the application layer may request from the network layer a set of routes to various hosts, providing the network layer with sufficient information to select routes and make forwarding decisions that meet the service requirements of the application. Based upon the characteristics of the routes provided by the network layer and the performance information for the hosts, the application selects a particular peer with which to communicate. So the network layer and the application layer cooperate to provide peer-to-peer routes. m   Received: from PIZZA.BBN.COM by BBN.COM id aa03076; 13 Dec 93 16:58 EST Received: from pizza by PIZZA.BBN.COM id aa21661; 13 Dec 93 16:43 EST Received: from BBN.COM by PIZZA.BBN.COM id ab21651; 13 Dec 93 16:41 EST Received: from black-ice.cc.vt.edu by BBN.COM id aa02214; 13 Dec 93 16:41 EST Received: from localhost (valdis@localhost) by black-ice.cc.vt.edu (8.6.4/8.6.4) id QAA16586; Mon, 13 Dec 1993 16:41:11 -0500 Message-Id: <199312132141.QAA16586@black-ice.cc.vt.edu> To: Martha Steenstrup cc: nimrod-wg@BBN.COM Subject: Re: routing In-reply-to: Your message of "Mon, 13 Dec 1993 15:47:04 EST." <199312132124.QAA15015@black-ice.cc.vt.edu> From: Valdis.Kletnieks@vt.edu MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Mon, 13 Dec 1993 16:41:11 +22306256 Sender: valdis@black-ice.cc.vt.edu On Mon, 13 Dec 1993 15:47:04 EST, you said: > In general, application-layer routing (for a particular application) > may wish to factor in the performance of the application or of a set of > hosts, in selecting a peer and the "best" route to that peer. See SMTP MX records and the Archie system for what are probably the 2 best-known examples of this. Note that these, as well as X.400 ADMD/PRMD foolishness, are not "routing" per se, but rather are *TARGET SELECTION*. In other words, we aren't doing any routing, we're simply deciding what host to route TO. Now, this is quite an interesting topic in and of itself, but opens quite a few cans of worms. For instance, it WOULD be nice if Archie was able to sort selections based on network distance metrics from the user, but that would involve an Archie server on Suranet knowing the paths from a user in Sesquinet to all the archive sites of interest. Of course, selecting the correct metrics is a problem - should it present "fastest if all links are up", "fastest biased via the last 30 days uptime", "fastest with this instant's topology", or... This can of worms should not be confused with on-the-fly routing to a set of hosts. As far as I can tell, this requires mobility on the part of at least one endpoint - if you open an SMTP connection to one of a set of 4 mailgates, if you wish to change the routing to another one in mid-mail, the connection and all its state has to be able to follow along. My brain hurts thinking about this - moving the connection is enough hassle - managing to move the partially sent mailgram as well is more hassle. ;) Bottom line - we dont know what application-layer routing means in sufficient detail to specify protocol to implement it. Valdis Kletnieks Computer Systems Engineer Virginia Tech   Received: from PIZZA.BBN.COM by BBN.COM id aa09959; 13 Dec 93 19:29 EST Received: from pizza by PIZZA.BBN.COM id aa22290; 13 Dec 93 19:07 EST Received: from BBN.COM by PIZZA.BBN.COM id aa22284; 13 Dec 93 19:04 EST Received: from uu2.psi.com by BBN.COM id aa09173; 13 Dec 93 19:06 EST Received: from port12.sunnyvale.ca.pub-ip.psi.net by uu2.psi.com (5.65b/4.0.071791-PSI/PSINet) via SMTP; id AA18230 for nimrod-wg@BBN.COM; Mon, 13 Dec 93 19:06:16 -0500 Received: by aland.bbn.com (4.1/3.1.090690-BBN) id AA13740; Mon, 13 Dec 93 16:05:27 PST Message-Id: <9312140005.AA13740@aland.bbn.com> To: Valdis.Kletnieks@vt.edu Cc: Martha Steenstrup , nimrod-wg@BBN.COM Subject: Re: routing In-Reply-To: Your message of Mon, 13 Dec 93 16:41:11 -0800. <199312132141.QAA16586@black-ice.cc.vt.edu> From: Craig Partridge Date: Mon, 13 Dec 93 16:05:26 -0800 Sender: craig@aland.bbn.com > Bottom line - we dont know what application-layer routing means > in sufficient detail to specify protocol to implement it. Well, I'm not quite sure. For instance, anycasting appears to addresses the problem of "find me the nearest Archie server", so that you're not telneting from Boston to Helsinki when there's a server in New Haven. Craig   Received: from PIZZA.BBN.COM by BBN.COM id aa11728; 13 Dec 93 20:33 EST Received: from pizza by PIZZA.BBN.COM id aa22539; 13 Dec 93 20:10 EST Received: from nic.near.net by PIZZA.BBN.COM id aa22533; 13 Dec 93 20:08 EST Received: from nic.near.net by nic.near.net id aa15543; 13 Dec 93 20:10 EST To: Craig Partridge cc: Valdis.Kletnieks@vt.edu, Martha Steenstrup , nimrod-wg@nic.near.net Subject: Re: routing In-reply-to: Your message of Mon, 13 Dec 1993 16:05:26 -0800. <9312140005.AA13740@aland.bbn.com> Date: Mon, 13 Dec 1993 20:09:37 -0500 From: John Curran -------- ] From: Craig Partridge ] Subject: Re: routing ] Date: Mon, 13 Dec 93 16:05:26 -0800 ] ] > Bottom line - we dont know what application-layer routing means ] > in sufficient detail to specify protocol to implement it. ] ] Well, I'm not quite sure. For instance, anycasting appears to addresses ] the problem of "find me the nearest Archie server", so that you're not ] telneting from Boston to Helsinki when there's a server in New Haven. I have heard many folks in the application area of the IETF looking for a mechanism to find the "closest" FOO on the network. However, there appears to be many different definitions of _closeness_ being used: o Geographically Close -- I'd like to connect to closest resource of type X, even if it means 20 hops to get back to New England. I don't care if the next hop is London; the weather servers over there don't interest me. o Administratively Close -- I'd like the nearest foo server *which I am administratively allowed to make connections to*. No, don't bother to connect me to the closest newsserver; it doesn't allow posting. o Topologically Close -- I'd like the ABC server which is closest when measured in hops. No, I'd like the one with the least delay. Err, how about the one with the biggest pipes to where I am? Ah, not quite. The one with the most available capacity? What's exactly is the measure of "available capacity" on given service? I'm sorry, but application-layer [routing exchange, service location, resource discovery] is a large problem space, and I'll have to side with the "we don't know yet" crowd on this issue. /John   Received: from PIZZA.BBN.COM by BBN.COM id aa12243; 13 Dec 93 20:55 EST Received: from pizza by PIZZA.BBN.COM id aa22608; 13 Dec 93 20:33 EST Received: from nic.near.net by PIZZA.BBN.COM id aa22602; 13 Dec 93 20:32 EST Received: from uu2.psi.com by nic.near.net id aa17023; 13 Dec 93 20:34 EST Received: from port12.sunnyvale.ca.pub-ip.psi.net by uu2.psi.com (5.65b/4.0.071791-PSI/PSINet) via SMTP; id AA00247 for jcurran@nic.near.net; Mon, 13 Dec 93 20:33:57 -0500 Received: by aland.bbn.com (4.1/3.1.090690-BBN) id AA14158; Mon, 13 Dec 93 17:33:14 PST Message-Id: <9312140133.AA14158@aland.bbn.com> To: John Curran Cc: Craig Partridge , Valdis.Kletnieks@vt.edu, Martha Steenstrup , nimrod-wg@nic.near.net Subject: Re: routing In-Reply-To: Your message of Mon, 13 Dec 93 20:09:37 -0500. <9312140110.AA24678@uu2.psi.com> From: Craig Partridge Date: Mon, 13 Dec 93 17:33:13 -0800 Sender: craig@aland.bbn.com John: I'll make a bold statement. Anycasting simply allows a host to advertise (with whatever routing metrics) that it supports a service. If you can solve the problem of routing with different metrics (e.g., you find some way to distinguish different types of closeness on a per application basis) then anycasting works just fine for all values of "close." In the interim, you're stuck with topologically close (which is what current routing protocols permit). That's still better in most cases than not close... Craig   Received: from PIZZA.BBN.COM by BBN.COM id aa12517; 13 Dec 93 21:07 EST Received: from pizza by PIZZA.BBN.COM id aa22643; 13 Dec 93 20:46 EST Received: from nic.near.net by PIZZA.BBN.COM id aa22637; 13 Dec 93 20:44 EST Received: from nic.near.net by nic.near.net id aa17651; 13 Dec 93 20:46 EST To: Craig Partridge cc: Valdis.Kletnieks@vt.edu, Martha Steenstrup , nimrod-wg@nic.near.net Subject: Re: routing In-reply-to: Your message of Mon, 13 Dec 1993 17:33:13 -0800. <9312140133.AA14158@aland.bbn.com> Date: Mon, 13 Dec 1993 20:46:32 -0500 From: John Curran -------- ] From: Craig Partridge ] Subject: Re: routing ] Date: Mon, 13 Dec 93 17:33:13 -0800 ] ] John: ] ] I'll make a bold statement. Anycasting simply allows a host to advertise ] (with whatever routing metrics) that it supports a service. Anycasting certainly is one way to do this. Clearly, multicasting can also be used to advertise such information (as the current service location WG is considering), as can DNS or even TCPMUX. ] If you can solve the problem of routing with different metrics (e.g., ] you find some way to distinguish different types of closeness on a per ] application basis) then anycasting works just fine for all values of ] "close." In the interim, you're stuck with topologically close (which ] is what current routing protocols permit). That's still better in most ] cases than not close... Perhaps we're in agreement: Anycasting is yet another way of doing service advertisement and discovery, presuming that someone figures out which set of metrics should be used. One might claim that the hardest part of the job is figuring out the right set(s) of metrics, and until that is done, "we just don't know" enough about application routing. /John   Received: from PIZZA.BBN.COM by BBN.COM id aa24440; 14 Dec 93 15:07 EST Received: from pizza by PIZZA.BBN.COM id aa26339; 14 Dec 93 14:37 EST Received: from BBN.COM by PIZZA.BBN.COM id aa26335; 14 Dec 93 14:34 EST Received: from nsco.network.com by BBN.COM id aa22304; 14 Dec 93 14:30 EST Received: from anubis-e1.network.com by nsco.network.com (5.61/1.34) id AA16963; Tue, 14 Dec 93 13:33:58 -0600 Received: from blefscu.network.com by anubis.network.com (4.1/SMI-4.1) id AA23739; Tue, 14 Dec 93 12:35:06 CST Date: Tue, 14 Dec 93 12:35:06 CST From: Andrew Molitor Message-Id: <9312141835.AA23739@anubis.network.com> To: nimrod-wg@BBN.COM Subject: Misc remarks - Some miscellaneous remarks. I should note that I had a chat with Joel Halpern here at NSC about some of this, so anything smart in the following should be attributed to him, and anything dumb to me. How does a Nimrod host bootstrap a route? That is, if I at blefscu.network.com wish to reach some host, moink.nmsu.edu, what do I do? There's some DNS style lookups, and some getting of some maps at various levels of granularity, right? In order to get these maps, I need to talk to some root mapping authorities, and then presumably 'map-boss.nmsu.edu' or something, and I'll be needing to get a route to there. Remember, any solution has got to scale, so requiring a number of transactions exponential in the number of levels of heirarchy is naughty! How does a Nimrod system recover from the case where flow state in the system gets thrown away for whatever reason? Again, this is one of those solvabale problems that needs to get looked at, and needs to scale well. I suppose you can propagate the flush back along the path or some such thing, until it reaches the source, which cheerfully sends a full source-route and a request to rebuild? A router that runs out of gas and starts thrashing its flow cache would be *bad*. I have some thoughts on clusters and whatnot that I will inflict on y'all shortly. Andrew Molitor   Received: from PIZZA.BBN.COM by BBN.COM id aa25818; 14 Dec 93 15:33 EST Received: from pizza by PIZZA.BBN.COM id aa26532; 14 Dec 93 15:02 EST Received: from BBN.COM by PIZZA.BBN.COM id aa26528; 14 Dec 93 15:00 EST Received: from nsco.network.com by BBN.COM id aa23746; 14 Dec 93 14:53 EST Received: from anubis-e1.network.com by nsco.network.com (5.61/1.34) id AA17187; Tue, 14 Dec 93 13:56:25 -0600 Received: from blefscu.network.com by anubis.network.com (4.1/SMI-4.1) id AA24960; Tue, 14 Dec 93 13:51:32 CST Date: Tue, 14 Dec 93 13:51:32 CST From: Andrew Molitor Message-Id: <9312141951.AA24960@anubis.network.com> To: nimrod-wg@BBN.COM Subject: A slightly different take on clusters - In what follows, I use the term 'end-thing' to mean the last place the network cares about the packet. This might be an interface or an end-point, I am unsure which. Indeed, I am somewhat dubious about the whole idea of 'interface' as a separate concept. I suspect that 'end-thing' is an EID, interfaces go away, and the way you represent the fact that EID "argle" has a FDDI interface and an ethernet interface is that you toss it in two different clusters, one named "fddi" and one named "ethernet-9." But enough of that.. It appears to me that areas or clusters could most handily be used to arbitrarily group things together. With overlapping areas, the whole notion of 'links' and 'networks' can go away, in fact. There are only clusters, which may or may not have an attribute indicating that they're internally connected. Of course, for generating routes, you're only interested in the connected ones. The problem of routing then reduces to finding a sequence of bottom-level clusters, each of which intersects with the next one along, and each of which enjoys the attribute 'connected' as well as whatever other attributes are desired by your policy. From the point of view of routing, there are only clusters. (this does neatly sweep under the rug the issue of getting across the bottom level clusters, but this is not something high-level routing worries about, since it depends on the stuff in the cluster. A cluster containing a single point-to-point link will solve this differently from a cluster containing a bunch of hosts on an ethernet. Bottom level 'connected' clusters are, by my definition, where the routing issues become a local problem.) Multicast groups can be represented as clusters which do not have the 'connected' attribute, but which do intersect lots of other clusters. The current notion of locator is really a special case of the more general concept of a 'filter' (<-- math geek term, introduced here because it's appropriate). The 'filter' describing a thing in the network would be the set of all clusters which contain the thing, or which contain a cluster that contains the thing, and so on up the hierarchy. To be more precise: Definition: An L0 cluster is a set of end-things, and a set of attributes. An Ln cluster for n > 0 is a set of L(n-1) clusters, and a set of attributes, mostly derived from the union of the attributes of its members (see note below on upward propagation of attributes.) Definition: The filter associated with an end-thing A is the set of all clusters defined as follows: F0 = { the set of L0 clusters which include A } Fn = { the set of all Ln clusters which include some element of F(n-1) } The filter, F_A, is the union of all the Fn's. One might think of it as all possible locators for A. If you write down some examples, one finds that this is actually a pretty small thing, normally. Assuming the lookup system is set up to be able to thin it down a bit based on attributes one wants, it gets smaller. For example, the filter for blefscu.network.com might be: { { engineering-net-15, multicast-group-otters, sparc-10s }, // F0 { engineering, nntp-clients }, // F1 { network-systems-corp }, // F2 { MR-net, OTHER-net }, // F3, providers, // in this case { USA, confederation-of-free-regional-nets }, // F4 { world } // F5 } If OTHER-net was actually in Europe, you'd add "Europe" in the F4 list, and under network-systems-corp somewhere would probably be a little L0 cluster containing a transatlantic point-to-point link. If you can get the complete filter of connected clusters for the desired destination, and the attributes for them, and if you can generate queries of the form 'what things are in cluster Foo' and 'what clusters intersect cluster Foo', and 'what attributes does cluster Foo have', then you can generate routes. Getting the list of things in a cluster is termed 'exploding' it, and the list itself is 'the explosion.' To build a route, you do this: 1) get the filter for your destination end-thing. 2) get your own filter. 3) find the lowest level cluster the two filters share. 4) explode that cluster, and find a sequence of intersecting clusters in the explosion with the right attributes. 5) recursively explode each of these clusters until you hit the bottom. 6) you're done. Note: this assumes a couple of things. Firstly, all (or most) attributes a cluster has are propagated up to enclosing clusters. Secondly, all attributes are phrased in a positive sense: 'I have some high speed links in me' as opposed to 'I don't have any high speed links', so upward propagation makes sense. A source looking for a fast path to a destination would ignore clusters w/o the 'hi-speed' attribute, but would not be guaranteed to find a fast path in every cluster it looks in. I think this models what we're after pretty well. Endpoints aren't very interesting, neither are wires. What is interesting is that these 7 end-thing are connected together by wires, and can talk to one another. Let's throw them in a box labeled 'connected'! As an added bonus, you don't have to worry about what the nodes and the arcs in your graphs mean, since you don't have them any more ;) Comments welcome, of course. Andrew Molitor   Received: from PIZZA.BBN.COM by BBN.COM id aa25974; 14 Dec 93 15:36 EST Received: from pizza by PIZZA.BBN.COM id aa26639; 14 Dec 93 15:15 EST Received: from TTL.BBN.COM by PIZZA.BBN.COM id aa26635; 14 Dec 93 15:14 EST To: nimrod-wg@BBN.COM Subject: Re: mobility and NIMROD Date: Tue, 14 Dec 93 15:12:22 -0500 From: Ram Ramanathan This is on the topic of localizing mobility related information, as advocated by Ken Carlberg in a previous mail. I too am a proponent of keeping mobility related information as local as possible. For instance, in Nimrod parlance, if a host moves from cluster A.B.C.D to cluster A.B.E.F, I would like that the move is transparent to all clusters not contained within the lowest cluster containing the move, in this case, A.B. I don't know whether this is possible at all or not, but it seems to me that this is probably a good idea for both scalability and efficiency. Noel Chiappa writes, comparing system wide updates and localized : >Here's a *sample* cost-benefit analysis. (I say sample because I'm going to >pull the numbers out of thin air, in the best system architect style! :-) >Let's assume the system wide mechanism is twice as expensive (ignoring the >extra overhead in the routing, which is substantial), and 50% of all moves are >extra-scope. (I don't think that's unreasonable; portable computers are *very* >common now, and people fly around a lot. Most people do that more than they >move their workstation inside their company. Heck, it might be *more* than >50%!) So, if we have both mechanisms, we will save only 25% on overhead, >versus if we had only the system-wide mechanism. Is that enough savings to >justify the complexity, etc of two mechanisms? Not in my book. While "portable" communications may be the order of the day, I would expect that, with the boom in personal communication networks etc., in the future, you would have all these handheld data terminals zipping about (real "mobile" communications). I read yesterday that Arthur D. Little has predicted the number of such equipment would be 60 million by the year 2005. While that is a scary number, it is likely that a very large fraction of mobility would local (office to office, office to subway to home, cars etc.). In that case, localizing mobility information makes sense; in fact I would argue that it is almost necessary to do that. Nimrod, as a hierarchical architecture, provides the perfect platform for such (localizing mobility) a design. For those interested in more quantitative comparison of the two aproches..... Noel's figures prompted me to spend some time looking at the two approaches a little more quantitatively. I considered two approaches. 1) in which every move causes the information to be propagated to all the levels of the hierarchy and 2) in which a move causes the information to be propagated only upto the the lowest cluster containing the move. I also considered two extreme models of mobility. (in what follows, I use the abbreviation LLCM to denote the "lowest level containing the move", which is the level of the cluster which immediately contains the origin and destination clusters of the move. In a L-level hierarchy the LLCM of a move can be between 1 and L). a) the probability of going from one cluster to another is same throughout, that is, LLCM of a move is a pure random number between 1 and L. b) the probability of going from one cluster to another decreases geometrically as the hierarchical distance between the two clusters. In particular, the probability that the LLCM is i is exactly half that of the LLCM of i-1. For example, if the cluster hierarchy is as follows : U / \ A B / \ / \ C D E F then, the probability that a host in C would move to D is half the probability that it would move within itself and the probability that it would move to E is one-fourth the probability that it would move within itself. I think that the reality would be somewhere in between. Comments? I am not going to give the details of the analysis, which essentially involves summing up a harmonic series (thanks to Isidro Castineyra, BBN, for that), but what I came up with is as follows : With Model (a), the cost of approach #2 is of the same order of complexity as approach #1, though better by factor of 2. That is, the growth rate is the same, but the absolute cost with #2 is half the absolute cost wiht #1. With Model (b), the cost of approach #2 is a lower order of complexity (O( 2(1 - (L/2^L)) ), compared to the cost of approach #1 (O(L)). In fact, with model (b), for large L, the cost remains almost constant as the number of levels increases. I would argue that for large networks with a mobility profile something like (b), the savings would definitely justify the complexity of having a non-system wide mechanism. My $0.03141. - Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa03087; 14 Dec 93 17:11 EST Received: from pizza by PIZZA.BBN.COM id aa27257; 14 Dec 93 16:54 EST Received: from BBN.COM by PIZZA.BBN.COM id aa27248; 14 Dec 93 16:52 EST Received: from black-ice.cc.vt.edu by BBN.COM id aa00405; 14 Dec 93 16:47 EST Received: from localhost (valdis@localhost) by black-ice.cc.vt.edu (8.6.4/8.6.4) id QAA14350; Tue, 14 Dec 1993 16:47:38 -0500 Message-Id: <199312142147.QAA14350@black-ice.cc.vt.edu> To: Ram Ramanathan cc: nimrod-wg@BBN.COM Subject: Re: mobility and NIMROD In-reply-to: Your message of "Tue, 14 Dec 1993 15:12:22 EST." <199312142043.PAA17854@black-ice.cc.vt.edu> From: Valdis.Kletnieks@vt.edu MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 14 Dec 1993 16:47:37 +22306256 Sender: valdis@black-ice.cc.vt.edu On Tue, 14 Dec 1993 15:12:22 EST, you said: > > a) the probability of going from one cluster to another is same throughout, > that is, LLCM of a move is a pure random number between 1 and L. > b) the probability of going from one cluster to another decreases geometrical ly > as the hierarchical distance between the two clusters. In particular, > the probability that the LLCM is i is exactly half that of the LLCM of i-1. c) the probability of going from one cluster to another decays as the hierarchical distance between the clusters, but at a higher rate. For instance, in our environment, we often pick up a workstation for a demo and move it to the next room, or recable a machine between segments. Much less often, a machine migrates across campus, and we don't have many machines at all that make trips to the west coast.... What does your model say if it decays at rate 1/(n**2) or 1/(n**3)? Valdis Kletnieks Computer Systems Engineer Virginia Tech   Received: from PIZZA.BBN.COM by BBN.COM id aa11156; 14 Dec 93 21:43 EST Received: from pizza by PIZZA.BBN.COM id aa28469; 14 Dec 93 21:27 EST Received: from TTL.BBN.COM by PIZZA.BBN.COM id aa28465; 14 Dec 93 21:25 EST To: Valdis.Kletnieks@vt.edu cc: nimrod-wg@BBN.COM Subject: Re: mobility and NIMROD In-reply-to: Your message of Tue, 14 Dec 93 16:47:37 -0500. <199312142147.QAA14350@black-ice.cc.vt.edu> Date: Tue, 14 Dec 93 21:22:34 -0500 From: Ram Ramanathan >> a) the probability of going from one cluster to another is same throughout, >> that is, LLCM of a move is a pure random number between 1 and L. >> b) the probability of going from one cluster to another decreases geometrica >> as the hierarchical distance between the two clusters. In particular, >> the probability that the LLCM is i is exactly half that of the LLCM of i-1. >c) the probability of going from one cluster to another decays >as the hierarchical distance between the clusters, but at a higher >rate. I think I should elaborate (b) first. It indicates the series 1/2, 1/4, 1/8 ... (ie, probability is 1/2 that it moves within level 1, 1/4 that it moves within level 2 and so on). If n is the number of the lowest level it moves within, the decay is propor tional to 1/(2**n). That is pretty high, and do you really think the decay is would be at a *even* higher rate? Just clarifying. >For instance, in our environment, we often pick up a workstation >for a demo and move it to the next room, or recable a machine between >segments. Much less often, a machine migrates across campus, >and we don't have many machines at all that make trips to the west >coast.... >What does your model say if it decays at rate 1/(n**2) or 1/(n**3)? Interesting cases. This is what sort of what I meant when I said the reality would be somewhere in between the linear (model (a)) and the exponential (model (b)) decays. Well, this is what I got : For 1/(n**2), the cost grows as O(log n), which is significantly lower than (a) which is O(n) and not much higher than (b) which is O(1) for large n. For 1/(n**3), I have encountered the harmonic series \Sigma(1/(k**2)), and I am wimping out of deriving it for now :-). However, this is an "exercise" problem in the Algorithms book by Cormen et al. The problem says "Show that the series \Sigma(1/(k**2)), k = 0 to infinity is bounded above by a constant". So it seems that for decay rates greater than 1/(n**3), the cost tapers off to almost constant. We don't even have to assume the extreme of model (b) for the "localization" scheme to be justified - 1/(n**3) is enough. Most interesting..... . -Ram. P.S. I should mention that my modelling and analysis are not very rigorous in nature and make use of many simplifying assumptions in regard to how the updates are distributed within a cluster and the hierarchy. So please take these for what they are meant to be - a macroscopic view of the nature of dependencies between cost and mobility.   Received: from PIZZA.BBN.COM by BBN.COM id aa05163; 15 Dec 93 0:01 EST Received: from pizza by PIZZA.BBN.COM id aa28899; 14 Dec 93 23:49 EST Received: from BBN.COM by PIZZA.BBN.COM id aa28895; 14 Dec 93 23:47 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa04910; 14 Dec 93 23:48 EST Received: by ginger.lcs.mit.edu id AA09755; Tue, 14 Dec 93 23:48:19 -0500 Date: Tue, 14 Dec 93 23:48:19 -0500 From: Noel Chiappa Message-Id: <9312150448.AA09755@ginger.lcs.mit.edu> To: carlberg@tieo.saic.com, jnc@ginger.lcs.mit.edu Subject: Re: mobility and NIMROD Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM I'll bring up a previous arguement that you had talked about, namely, treating existing IPv4 as EIDs. If this were to be done, then would not the routing architecture (or some component thereof) have to do some type of translation from the EID (IPv4 address) to the current Nimrod representation of the destination entity? Depends on what you mean by "routing architecture". The definition I use is the system which i) distributes information about that topology, ii) calculates paths from one location in that topology to another, and iii) gets traffic along that path. The latter often goes under the name "forwarding". (In the current Destination Vector routing architecture, i) and ii) are performed in an intermingled way, at the time routing table entries are created. I don't consider the looking up of such entries at the time packets are forwarded to really encompass ii), although some people use the term "routing" to describe the process of looking up a route in a database.) Clearly you have to have *somone* do the IPv4 EID to locator translation, but in the cases where the first hop router does it, is that lookup part of the "forwarding"? (It clearly doesn't fit into either of the the other two stages.) I don't think so (what about cases where it is done by the host), although I suppose it's a matter of definition. Anyway, I've always viewed IPv4 support as a "bag on the side" of Nimrod; it's more of an "interoperability module" than anything else... Thus, if the IPv4 entity is mobile, then the routing components of Nimrod would be responsible for supporting mobility. Again, definitions. My vision is that if thing A moves from locator X to locator Y, *none* of what I think of when I think of "the real Nimrod", i.e. the topology distribution, route calculation and forwarding support, has an integral role to play in supporting this (although they are used directly). Sure, if you're cutting out the triangle, the thing which is sending packets to A needs to calculate a path to Y, and make sure that the packets get forwarded to there (or if you're using the Base Station model, the base station has to get the new locator, and send packets there), but this is how the rest of the system uses Nimrod, not Nimrod. Don't get me wrong, I know that these usage modes need to be thought about, and specified, and maybe this is just a useless discussion about terms, but I see some messages later on that seem to indicate people are confused about this. My feeling is that the Nimrod protocols (as we will specify them) do not directly support mobility (e.g. the way IS-IS does); they provide tools (such as the separate EID and locator namespaces) to allow mobility. Also, I realize we need an interim solution, either base stations, or the first hop router, or something, but I distinguish between interim interoperability stuff (which is allowed to be kludgy and inefficient), and the final mechanism (which must be neither)! > I'd put it differently. It's a balancing act between how much of the > mechanism you put in the hosts and how much you expect the ... system > to do for you automagically .... There are of course intermediate > approaches ... The base station approach, where the packets take two > sides of a triangle, is one. The only reason I'd support the use of triangle routing is because I'd want to hide from the source where my mobile entity "really" is (a rationale brought up by Tony Li at the last IETF). But other than that, I'd really like to avoid this approach for IPng because it has the potential for being extremely inefficient. I mention triangle routing (with a base station) since I know it's quite popular. The big advantage I see is a short-term, practical-deployment one, in that it does not require any support at the source (either in the hosts or routers). This allows changes to support mobility to be localized to the mobile host and its home routers. I prefer the simple, powerful solution where we just tell the source that the mobile thing is at a new locator, but I realize that this is not an instantly achievable one; it would have to be a long-term goal. >Notifying the first hop router is another, and both allow you to keep >the static host out of it. This approach has been presented in the mobile-IP working group as a means of avoiding triangle routing. Yes, but it does require the first-hop router to support this feature! Of course, you could backtrack from the mobile host's home location toward the source (using Reverse Path Fowarding - is that what you meant by your reference to it?), until you find a router which doesn't support the mobility upgrades, and chop the triangle there. Alternatively, somehow start from the source, and go forward until you find one which does, which is more optimal but probably harder to do.. > This is a slightly unfortunate choice of terminology. Explicit > notifications that a host has moved, to some (smallish) set of database > sites which are authoritative for a particular translation, is not > necessarily anything like as expensive as the term "system wide" would > imply... They are certainly a heck of a lot cheaper than getting the > routing involved! yes, but this goes back to my desire of localizing the impact of mobility. Hrrm. If the authoritative translation sites are colocated with the home location of the mobile host, as would be common, then the impact would be localized. (This, of course, is modulo any open connections, but for a variety of reasons, such as source policies, QOS, resource allocations, etc, it's probably not a good idea to try and get away from notifying them...) > Ah, I see what you want to do. You want to associate an EID with a > routing scope, and, once tarffic for that EID reaches that scope, have > the routing inside the scope responsible for tracking the destination > [a la IS-IS] But the scope itself could (and I would rather expect it to) be layered to further limit the impact of mobility. ... the "tracking" is only done within A.M and A, as well as the source, remains ignorant of these changes. Oh, I was assuming that; you'd limit the scope to that over which the object was mobile. However, my point about overhead is still valid; the cost is a sum over all mobile destinations, and the cost for each individial mobile destination gets larger as the scope over which you allow it to roam is made bigger. You're back in the same box where limiting the scopes, to limit costs, makes more moves extra-scope. When and _if_ the object moves outside of A, then the source, and any appropriate databases, would be notified. The objective is primarily to delay this notification process, not to eliminate it. Now we're into an optimization/efficiency argument, and it's hard to know what the Right Thing is there without a much better model of both the traffic, and costs. E.g., if delaying the notification saves you 1% overall, you probably don't do it, but if it saves you 99%, you do. Maybe we should think about what Nimrod needs to be able to support a variety of different mobility models (arrgh, kitchen sink design, I hate it), since we don't really know which one is best? Still, a good point; I'll add this "hiding mobility in the routing" as an open point, since that particular solution will require protocol support from the "Nimrod routing". A slight digresion...In my mind, I tend to view mobile objects spending _most_ of their time moving within a specific scope. If I had a wireless palmtop or laptop the scope of my movement would more that likely be restricted to the speed that "I" am traveling. This movement could be fast within the context of a floor in my building (ie. moving from office to office) but in the context of my building or an internetwork spanning the world, its inconsequential. Aren't you likely to still be in the same wireless cell, here, though? In that case, your locator probably hasn't changed at all.. Now if you were to present the example of a car or an airplane, then I would say that the scope might be different but it is still a scope. In the case of a car, its scope could be bounded within the operating area of a cellular network provider (whose scope tends to span counties/states). And in the case of a plane, the scope can encompass whole regions of the earth through the use of satelite footprints. The common aspect in these three examples is that the hardware has already provided us with a _loosely_ defined scope. Yes, but it's a scope that's large enough that it may be more efficient to do the notification process (which has basically fixed costs, no matter how far the move) than hide the move in the routing (which has costs which are related to the distance of the move). Perhaps moves are either local enough that you don't even change the locator (human mobility), or so non-local that you can't hide them (mechanical mobility)? Once again, we are grappling with the issue of "what are the costs going to be". Are we really sure that a large enough percentage of moves are going to be over a small enough scope that the hiding mechanism will be useful? My intuition, as a system designer, is that a mechanism which handles extra-scope moves will be cheap enough that we don't want the complexity, etc, of a local optimization, for the share of cases that could use it. Of course, I'll cheerfully admit that this intuition could be wrong! :-) > Now, for moves *outside* the scope, we have to have some other mechanism, > right? Presumably, this mechanism gets invoked enough that it has to be > reasonably efficient? So, why have the first mechanism? Why not use the > wide-area mechanism everywhere? The rationale for two mechanisms stems from my opinion that each mechanism has its drawbacks in supporting mobility. Well, I understand the drawbacks of hiding mobility in the routing, and I really don't like it! The routing is going to have a tough enough time keeping track of the static network topology, without tracking dynamic objects too! What exactly do you see as the drawbacks to the notification mechanism (understanding that this is a long-range solution, not an immediately deployable one)? Thus, instead of settling on one method, I would rather try to use the strengths of each to support mobility as opposed to being limited by the weakness of one method. There are times when multiple parallel mechanisms are the right approach, I won't deny it. (I mean, Nimrod has the potential for multiple parallel mechanisms all over the place, so I'm not a priori biased against them!) Note, however, that in most such places use of a different mechanism has only local consequences. If we decide to built support for local mobility into Nimrod, it'll cause complexity everywhere... All I can say is that there are many cases in which (due to source policies, QOS, resource allocation, etc) you *cannot* hide even local moves, and we still need a non-local mechanism. So, the utility of the second method is limited; these things together make me look very askance at this optimization. Nimrod is complex enough without this bell and whistle! I should say perceived weakness since this is rather my perception of things. Exactly; could you provide some more detail on what you see as the problems of notification? I'm not trying to be difficult; I genuinely feel that this is the most direct, and thus most powerful and likely to be successful in the long run, way to do this. However, if it has a problem, I really, really want to know about it; leaving out a critical mechanisms will not help Nimrod, and I know that. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa05924; 15 Dec 93 0:32 EST Received: from pizza by PIZZA.BBN.COM id aa29042; 15 Dec 93 0:21 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29035; 15 Dec 93 0:19 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa05579; 15 Dec 93 0:20 EST Received: by ginger.lcs.mit.edu id AA09982; Wed, 15 Dec 93 00:20:37 -0500 Date: Wed, 15 Dec 93 00:20:37 -0500 From: Noel Chiappa Message-Id: <9312150520.AA09982@ginger.lcs.mit.edu> To: msteenst@BBN.COM, nimrod-wg@BBN.COM Subject: Re: routing Cc: jnc@ginger.lcs.mit.edu Clearly, we don't want to impose redundant routing functions on multiple protocol layers. This unnecessarily complicates protocols, making them harder to manage and less efficient. Moreover, network-layer protocols can quickly learn about the state of the network and thus are in the best position to make control decisions based upon the network state. The delay incurred in delivering network-layer information to the application layer and in the application layer acting upon this information may make fine-grained routing control difficult if not impossible at the application layer. The delay need not be large, though, no? If the host's internet layer is being notified anyway, notifying the application need not be remarkably more expensive in time. I would say that a bigger reason is that the applications usually just don't care about all those useless (to them :-) details. I mean, they may have *requirements* they want to pass along, and they may want to get told if you can't meet those requirements, but in general they seem to be happier if they can just treat all that complex routing stuff as a black box they don't have to mess with, and get on with their own thing. "Functional modularity", and all that stuff, yah? (Maybe this is just a restatement of your first point...) However, there are applications that do wish to perform some of the routing functions. For example, the X.400 Message Transfer Agents (MTAs) act as application-layer routers, using application-layer information contained in the envelope of the X.400 message as well as information about neighboring MTAs in order to select the next-hop MTA for the message. I've always been suspicious of this "feature"; it has always felt like incipient "49-layer disease" to me, although perhaps in a world where security is provided by application level gateways, it makes sense. (I guess it also makes operation of one monolithic mail system across multiple transport protocols easier, too.) Are there any other good reasons for this capability? In general, application-layer routing (for a particular application)... Information about host or application performance is not part of the network-layer information ... the application layer may request from the network layer a set of routes to various hosts ... the application selects a particular peer with which to communicate. So the network layer and the application layer cooperate to provide peer-to-peer routes. Yes, if you were going to do it, this is the way to do it. What I don't understand is, modulo reasons such as the ones above (security barriers, and providing a monolithic application across multiple transport layers), where there is no other way to do it, why would you want to do this? Leaving aside cases where going through various application level boxes leaves behind side-effects (things like updating a replicated database), or where there is an availability problem (hosts A and B are never up at the same time, so they do their thing through C), I don't see the point of application level routing. Note that both of these exceptions are very application specififc, so a general mechanism isn't going to be possible.. You want to get some interaction between S and D; if you have a ubiqitous internetwork connecting them, why not just go directly, and do all your routing in one fell swoop, rather than doing it in two layers? The walls between the two layers are going to produce all sorts of problems in trying to perform what ought to be a monolithic function; i.e. route selection. If the intermediate boxes are just forwarding the data, they aren't doing anything a router can't do... Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa06574; 15 Dec 93 1:01 EST Received: from pizza by PIZZA.BBN.COM id aa29167; 15 Dec 93 0:47 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29163; 15 Dec 93 0:46 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa06231; 15 Dec 93 0:47 EST Received: by ginger.lcs.mit.edu id AA10059; Wed, 15 Dec 93 00:46:00 -0500 Date: Wed, 15 Dec 93 00:46:00 -0500 From: Noel Chiappa Message-Id: <9312150546.AA10059@ginger.lcs.mit.edu> To: craig@aland.bbn.com, jcurran@nic.near.net Subject: Re: routing Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM From: Craig Partridge For instance, anycasting appears to addresses the problem of "find me the nearest Archie server" Craig, this is yet one more capability I think should be provided by a separate mechanism which *draws upon* the routing, but is not *in* the routing. In other words, I don't think anycasting should be a fundamental capability of the system. Rather, it should be built up from more primitive components; a bunch of subsystems which some higher level entity draws upon to provide a service. I.e., you go get a list of the servers for service X (through some mechanism which selects, in some way I don't understand, and which is per application anyway, a subset of all the available servers for that service which is likely to contain one which you wil like), then go get paths from yourself to each of them, then look at the attributes of the paths, and decide based on that. Remember, any time you get some other subsystem to make complicated choices for you, either i) you need horrendously complex interface semantics so you can tell that interface exactly what you want, or ii) you have to live with whatever crummy choice it makes for you... This model of subsystems, each doing one simple thing well, is the future of the Internet, I feel. Of course, we have to make sure we have a set of subsystems which provide the right facilities, and which can interact in the right ways, but that's the fun of it... :-) From: Craig Partridge Anycasting simply allows a host to advertise (with whatever routing metrics) that it supports a service. ... If you can solve the problem of routing with different metrics ... then anycasting works just fine for all values of "close." In the interim, you're stuck with topologically close... This is not the chief problem, from my point of view. Once again, I think overloading this function into the routing is simply going to be too expensive. The routing is gonna have its hands full keeping track of the actual network topology, without trying to be a service location protocol too... From: John Curran Anycasting certainly is one way to do this. Clearly, multicasting can also be used to advertise such information (as the current service location WG is considering), as can DNS or even TCPMUX. Exactly what mechanism is best depends on a number of factors, but in general the overhead of locating a service it just that, overhead, and we want to minimize it, but a large group of clients may allowe the costs to be amortized to a low enough level on even an expensive mechanism. Of course, there's the usual cost/benfit knob to set (more expensive methods may give you better response) too... Anycasting is yet another way of doing service advertisement and discovery, presuming that someone figures out which set of metrics should be used. My intuition is that anycasting will not be a viable method over any but local scopes. Noel   Received: from PIZZA.BBN.COM by BBN.COM id ab06660; 15 Dec 93 1:06 EST Received: from pizza by PIZZA.BBN.COM id aa29217; 15 Dec 93 0:54 EST Received: from BBN.COM by PIZZA.BBN.COM id ab29209; 15 Dec 93 0:53 EST Received: from necom830.cc.titech.ac.jp by BBN.COM id aa06374; 15 Dec 93 0:53 EST Received: by necom830.cc.titech.ac.jp (5.65+/necom-mx-rg); Wed, 15 Dec 93 14:48:19 +0900 From: Masataka Ohta Return-Path: Message-Id: <9312150548.AA28643@necom830.cc.titech.ac.jp> Subject: Re: mobility and NIMROD To: Noel Chiappa Date: Wed, 15 Dec 93 14:48:16 JST Cc: karl@gropo.tgv.com, jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM In-Reply-To: <9312131423.AA27159@ginger.lcs.mit.edu>; from "Noel Chiappa" at Dec 13, 93 9:23 am X-Mailer: ELM [version 2.3 PL11] The effect of mobility should be localized as much as possible, of course. The current scheme of mobile WG requires nothing to the immobile hosts communicating with mobile hosts. It can have optimization to make immobile hosts directly communicate with mobile hosts. The point is that such optimization should be localized to the immobile end hosts. It should not further be globalized to evolve routers between immobile and mobile hosts. Isn't it obvious? > When it comes to mobility, I'm not convinced that doing it at the internetwork > layer is absolutely the Right Thing, I think the network layer solution is absolutely right. > the pcmail protocol ... uses names to identify clients ... The import of > this is that the network layer, i.e. the routing protocols, don't have to > assume that a mobile host is keeping the same IP address. I'm afraid it requires the network wide broadcasting or simething like that so that only PC softwares which wrongly assumes very small networks are doing so. > Nimrod explicitly breaks that IPv4 naming function into several parts, I think mobility issues has nothing to do with IPv4 specific properties including DNS. DNS is used to point to the unchanging IP address of a moble host and, thus, its net- and sunet- part give the information of the location of the home agent (which gives information of the actual location of the mobile host), which is perfectly good. > It also requires a resonbly efficient means for a mobile device to acquire > a useful local address. > > Yes, but all mobility scheme require this. No, mobility schemes such as discussed in the mobility WG do NOT require any local IP addresses. > In summary, perhaps nimrod ought to punt mobility up to the applications. In summay, I think mobility has notihg to do with nimrod. Masataka Ohta   Received: from PIZZA.BBN.COM by BBN.COM id aa06914; 15 Dec 93 1:17 EST Received: from pizza by PIZZA.BBN.COM id aa29297; 15 Dec 93 1:02 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29291; 15 Dec 93 1:00 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa06580; 15 Dec 93 1:02 EST Received: by ginger.lcs.mit.edu id AA10094; Wed, 15 Dec 93 01:01:23 -0500 Date: Wed, 15 Dec 93 01:01:23 -0500 From: Noel Chiappa Message-Id: <9312150601.AA10094@ginger.lcs.mit.edu> To: Valdis.Kletnieks@vt.edu, msteenst@BBN.COM Subject: Re: routing Cc: jnc@ginger.lcs.mit.edu, nimrod-wg@BBN.COM For instance, it WOULD be nice if Archie was able to sort selections based on network distance metrics from the user, but that would involve an Archie server on Suranet knowing the paths from a user in Sesquinet to all the archive sites of interest. Of course, selecting the correct metrics is a problem... Ouch. The answer seems to be "well, you can't get there from here". All the Archie server can do is hand the list to the client, and let the client figure it out.... This is an interesting question, though. Giving the client all the basic info, and letting them go from there, gives the maximum flexibility to the client, but it has many downsides. First, it means you can't amortize any of the cost of doing that selection over more than one client. Well, here in the future, computes are chearp. Then there's the question of "if the server has 17 zillion locations which can provide that data, how does it select which set to try and tell the client about"? Maybe we wind up with some sort of heuristic balance between the extremes (making the choice for the client, and leaving it all up to the client), where the server picks a "likely set" and says "OK, here they are, but there are more if you don't like any of these". This can of worms should not be confused with on-the-fly routing to a set of hosts. ... if you open an SMTP connection to one of a set of 4 mailgates, if you wish to change the routing to another one in mid-mail, the connection and all its state has to be able to follow along. Is this really what you are doing (changing from one to another 'cause you feel like it), or is is providing redundancy? It gets complicated, yes. Either i) you want to have a set of fate-sharing regions act like they are one, so the state has to be replicated enough so that you can fail over from one to another invisibly to the client, or ii) you use something more complex for the client interface (like a two-phase commit with checkpoints) which gets the client into the game explicitly too. I suspect that the latter is the optimal way to go; in general, invisible things seem to be more expensive than getting a certain amount of help (shades of mobility :-). Any of these are so application specific that they are probably going to stay that way for a long time, though.   Received: from PIZZA.BBN.COM by BBN.COM id aa07318; 15 Dec 93 1:36 EST Received: from pizza by PIZZA.BBN.COM id aa29345; 15 Dec 93 1:20 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29339; 15 Dec 93 1:19 EST Received: from necom830.cc.titech.ac.jp by BBN.COM id aa06972; 15 Dec 93 1:20 EST Received: by necom830.cc.titech.ac.jp (5.65+/necom-mx-rg); Wed, 15 Dec 93 15:16:03 +0900 From: Masataka Ohta Return-Path: Message-Id: <9312150616.AA28846@necom830.cc.titech.ac.jp> Subject: Re: routing To: Martha Steenstrup Date: Wed, 15 Dec 93 15:16:02 JST Cc: nimrod-wg@BBN.COM In-Reply-To: <9312132113.AA21464@necom830.cc.titech.ac.jp>; from "Martha Steenstrup" at Dec 13, 93 3:47 pm X-Mailer: ELM [version 2.3 PL11] > Moreover, network-layer protocols can quickly learn about the state of > the network and thus are in the best position to make control decisions > based upon the network state. The delay incurred in delivering > network-layer information to the application layer and in the > application layer acting upon this information may make fine-grained > routing control difficult if not impossible at the application layer. Why you think finely grained routing control is necessary at the application layer, at all? > However, there are applications that do wish to perform some of the > routing functions. Then, the application is broken, though an application may want to select the best server between multiple servers. > Information about host or application performance is not part of the > network-layer information but is instead known by the application or > obtained through network management. Information about host or application performace could be propagated through network layer to be able to accomodate performance of links in between. I'm sure you don't want to talk to a high performance application which is unreachable from you. Masataka Ohta   Received: from PIZZA.BBN.COM by BBN.COM id aa09235; 15 Dec 93 2:25 EST Received: from pizza by PIZZA.BBN.COM id aa29499; 15 Dec 93 2:09 EST Received: from BBN.COM by PIZZA.BBN.COM id aa29495; 15 Dec 93 2:08 EST Received: from GINGER.LCS.MIT.EDU by BBN.COM id aa08874; 15 Dec 93 2:09 EST Received: by ginger.lcs.mit.edu id AA10574; Wed, 15 Dec 93 02:09:25 -0500 Date: Wed, 15 Dec 93 02:09:25 -0500 From: Noel Chiappa Message-Id: <9312150709.AA10574@ginger.lcs.mit.edu> To: nimrod-wg@BBN.COM, ramanath@BBN.COM Subject: Re: mobility and NIMROD Cc: jnc@ginger.lcs.mit.edu I too am a proponent of keeping mobility related information as local as possible. For instance ... if a host moves from cluster A.B.C.D to cluster A.B.E.F, I would like that the move is transparent to all clusters not contained within the lowest cluster containing the move, in this case, A.B. This seems to assume that the routing is involved. Sure, if you want to do something which is *invisible* to everyone, you have to involve the routing, (in which case you want to limit the scope over which the move is visible to the smallest possible scope, to limit the routing overhead). However, I don't see why the support has to be in the routing, and I claim there are good arguments to *not* get the routing involved. Any such mechanism will have to limit the scope over which it functions (due to overhead in the routing which is a function of the scope), which means we *have* to have an alternative mechanism to handle those long moves, and I think we should use that one everywhere. I mean, if thing A has moved from locator X to locator Y, why should the routing care? You just have to use locator Y instead of X if you want to get to A. Whether it's explicit notification of affected entities (optimal in the long run, in my view), or performed on their behalf by some router (better from the deployability standpoint; Nimrod does use this as a deployment path :-), this is the way to go, I reckon. There are four cases: portable/mobile time client/server (by which I mean that connections will want to come in, as well as out). Here's how they look (assuming explicit notification is used): #1 - Portability, client: Nobody has to be notified, the host just has to find out its new locator before it can communicate. #2 - Portability, server: Translation databases which are authoritative for that host have to be updated with the host's new locator. There is no telling where these databases are; a host moving from A.B.C.D to P.Q.R.S might have to notify a database at A.B.C.x, or a host moving to A.B.C.y might have to notify a database at P.Q.R.S. #3 - Mobility, client: Hosts with active connections to the mobile host have to be notified of the new locator; these hosts can obviously be anywhere in the system. #4 - Mobility, server: Combination of #2 and #3. Obviously, solutions where you hide the move (with the aid of some router) produce similar matrices. Noel ... writes, comparing system wide updates and localized : > Let's assume the system wide mechanism is twice as expensive (ignoring > the extra overhead in the routing, which is substantial), and 50% of all > moves are extra-scope. ... So, if we have both mechanisms, we will save > only 25% on overhead, versus if we had only the system-wide mechanism. I seem to have been unclear in my terminology. By "system-wide", I did not mean a mechanism which distributed notices system-wide, but rather a system which allows a *move* system-wide (i.e. not over some local scope) by distributing explicit notification of the move to affected parties (or some selected agents acting on their behalf, such as base stations, or first-hop routers, or whatever). (Also, the "extra overhead in the routing" which I referred to was for the "localized" case, i.e. one in which the move was hidden by the routing; i.e. the routing tracks EID's directly inside a scope, or the object kept its original locator after it moved; the overhead is pretty much identical. It's not at all clear to me, when all is said and done, that explicit notification is more expensive, even in cases where mobility is only supported over fairly local scopes.) Even if we do "hidden" mobility, I simply don't think we are going to be able to do it for moves anywhere in the system. We are going to have to use notification for those. It's those two mechanisms I was thinking of in that back-of-the-envelope calculation above. In that case, localizing mobility information makes sense; in fact I would argue that it is almost necessary to do that. I'd take it one step further. I'd say it's necessary to get the routing out of the business of supporting mobility altogether. Put the costs directly on the mobile devices... make them be the ones to notify everyone. For those interested in more quantitative comparison of the two approaches ... For what it's worth, assuming the number of translation databases which are authoritative for a given host is a approximately constant over time (probably a reasonable bet), and the number of open connections for an average mobile host is constant over time (maybe not quite so good an assumption, but I'll bet it's a pretty small growth rate) the cost of the explicit notification grows as O(1) as the network gets larger; i.e. it doesn't. (I'm leaving out second order effects, like the diameter of the network increasing meaning the number of hops notifications to distant hosts have to traverse increases...) This is a real win. 1) in which every move causes the information to be propagated to all the levels of the hierarchy I'm not sure I completely understand this one, unless you mean than a move from A.B.C.D to A.B.C.E is visible to *, A.*, etc? With Model (a), the cost of approach #2 is of the same order of complexity as approach #1, though better by factor of 2. That is, the growth rate is the same, but the absolute cost with #2 is half the absolute cost wiht #1. This is not absolutely intuitive, but it does't surprise me a lot. I guess I need to understand your approach #1 in more detail.. With Model (b), the cost of approach #2 is a lower order of complexity (O( 2(1 - (L/2^L)) ), compared to the cost of approach #1 (O(L)). In fact, with model (b), for large L, the cost remains almost constant as the number of levels increases. That last sentence refers to approach #2 only, right? I'm not sure I believe that this is a good model of reality. The exponential decrease in probability is what produces the constant cost, but I don't think it'll be exponential. In which case, the cost will not be constant... and we're back to my hypothesis, which is that we won't be able to use "invisible" mobility over large scopes... I would argue that for large networks with a mobility profile something like (b), the savings would definitely justify the complexity of having a non-system wide mechanism. There are a couple of points here. First, as you point out, it assumes a profile like (b); if distant moves are more common, you probably move further away from the almost constant cost. Second, even if it is a constant cost, the absolute value might in fact still be higher than the cost of explicit notification (which is what I meant by "system-wide"). Without a much more complete design, I reckon it's hard to quantify these costs, though.. Noel   Received: from PIZZA.BBN.COM by BBN.COM id aa08626; 15 Dec 93 13:25 EST Received: from pizza by PIZZA.BBN.COM id aa02037; 15 Dec 93 12:55 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa02033; 15 Dec 93 12:52 EST To: nimrod-wg@BBN.COM Subject: a few comments on mobility Date: Wed, 15 Dec 93 12:54:33 -0500 From: Martha Steenstrup Mobility need not be confined to endpoints. Whole clusters may move. Examples include present-day tactical military networks and packet radios within a packet radio networks, and in the not-to-distant future networks on commuter trains or planes. Mobility of clusters may be of the "discrete" or "continuous" type. With discrete mobility, a network moves and reconnects at a new location, but during the move, endpoints on that network do not communicate with endpoints outside of that network. With continuous mobility, endpoints on a mobile network continue to communicate with endpoints outside of that network while that network moves. In either case, confining mobility support to the mapping between endpoint identifier and location in the internetwork is not sufficient. One also needs to know to which clusters the mobile cluster is currently connected. This information cannot be deduced from the new locator for the given endpoint. It must come from the information about cluster interconnectivity supplied by routing. Hence, routing will have to be involved to reflect the change in cluster interconnectivity. m   Received: from PIZZA.BBN.COM by BBN.COM id aa10731; 15 Dec 93 14:02 EST Received: from pizza by PIZZA.BBN.COM id aa02248; 15 Dec 93 13:43 EST Received: from BBN.COM by PIZZA.BBN.COM id aa02241; 15 Dec 93 13:41 EST Received: from necom830.cc.titech.ac.jp by BBN.COM id aa09569; 15 Dec 93 13:39 EST Received: by necom830.cc.titech.ac.jp (5.65+/necom-mx-rg); Thu, 16 Dec 93 03:34:56 +0859 From: Masataka Ohta Return-Path: Message-Id: <9312151835.AA02561@necom830.cc.titech.ac.jp> Subject: Re: a few comments on mobility To: Martha Steenstrup Date: Thu, 16 Dec 93 3:34:54 JST Cc: nimrod-wg@BBN.COM In-Reply-To: <9312151823.AA02502@necom830.cc.titech.ac.jp>; from "Martha Steenstrup" at Dec 15, 93 12:54 pm X-Mailer: ELM [version 2.3 PL11] > Mobility need not be confined to endpoints. Whole clusters may move. > Examples include present-day tactical military networks and packet > radios within a packet radio networks, and in the not-to-distant > future networks on commuter trains or planes. Why each endpoint in the clusters could not be treated as mobile separately? > One also needs to know to which clusters the mobile > cluster is currently connected. This information cannot be deduced > from the new locator for the given endpoint. The cluster information could also be supplied from a home agent of the entire cluster, I think. Masataka Ohta   Received: from PIZZA.BBN.COM by BBN.COM id aa21144; 15 Dec 93 16:38 EST Received: from pizza by PIZZA.BBN.COM id aa03104; 15 Dec 93 16:17 EST Received: from BBN.COM by PIZZA.BBN.COM id aa03098; 15 Dec 93 16:14 EST Received: from black-ice.cc.vt.edu by BBN.COM id aa19168; 15 Dec 93 16:08 EST Received: from localhost (valdis@localhost) by black-ice.cc.vt.edu (8.6.4/8.6.4) id QAA16907; Wed, 15 Dec 1993 16:08:51 -0500 Message-Id: <199312152108.QAA16907@black-ice.cc.vt.edu> To: Martha Steenstrup cc: nimrod-wg@BBN.COM Subject: Re: a few comments on mobility In-reply-to: Your message of "Wed, 15 Dec 1993 12:54:33 EST." <199312151839.NAA16936@black-ice.cc.vt.edu> From: Valdis.Kletnieks@vt.edu MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Wed, 15 Dec 1993 16:08:50 +22306256 Sender: valdis@black-ice.cc.vt.edu On Wed, 15 Dec 1993 12:54:33 EST, you said: > Mobility need not be confined to endpoints. Whole clusters may move. > Examples include present-day tactical military networks and packet > radios within a packet radio networks, and in the not-to-distant > future networks on commuter trains or planes. Hmm.. is a commuter train a good example of what we'd want as a mobile cluster, given that 98% of the people there would want to be connecting to their home office and not to somebody else on the train? In particular, I think we want to be *very* careful here in regards to how this interacts with any security model we may wish to consider. There's an implicit assumption in the current version of IP that all hosts on a given subnet are (if not totally trusted) at least are not being TOO actively hostile. However, a commuter train full of people who work for competing companies... Packet sniffing, traffic analysis, denial of service (remember, a train can't easily have a fiberoptic link - assume it has a 64kb radio link. Amazing what an FTP of the X11R5 source tree to /dev/null will do to the response time of that guy from the competing brokerage across the street. ;) Yes, I know that most universities already have war zone subnets where they have students with too much time on their hands. This doesn't mean we shouldn't try to define mobile clusters to do some damage control. For that matter, has anybody done serious thinking about how stationary clusters look? Are we able to do something elegant to minimize the equivalents of ICMP bombs, ARP attacks, ICMP redirect hacks, DNS hacks, and all the rest? > Mobility of clusters may be of the "discrete" or "continuous" type. > With discrete mobility, a network moves and reconnects at a new > location, but during the move, endpoints on that network do not > communicate with endpoints outside of that network. With continuous > mobility, endpoints on a mobile network continue to communicate with > endpoints outside of that network while that network moves. A case could be made that the military would want a 'discrete' mobility, but all those suits riding the 7:45 into the city would be more interested in continuous. ;) /Valdis   Received: from PIZZA.BBN.COM by BBN.COM id aa11867; 16 Dec 93 7:53 EST Received: from pizza by PIZZA.BBN.COM id aa05971; 16 Dec 93 7:37 EST Received: from BBN.COM by PIZZA.BBN.COM id aa05967; 16 Dec 93 7:33 EST Received: from mcigateway.mcimail.com by BBN.COM id aa11357; 16 Dec 93 7:33 EST Received: from mcimail.com by MCIGATEWAY.MCIMail.com id aa29711; 16 Dec 93 12:24 GMT Date: Thu, 16 Dec 93 07:24 EST From: "Robert G. Moskowitz" <0003858921@mcimail.com> To: nimrod wg Subject: Re: a few comments on mobility Message-Id: <25931216122452/0003858921NA3EM@mcimail.com> Valdis.Kletnieks said: >Hmm.. is a commuter train a good example of what we'd want as >a mobile cluster, given that 98% of the people there would want >to be connecting to their home office and not to somebody else >on the train? What about the train itself? And the computers controlling it? Those will go back to the owner of the train, even if it is running on a competitors line. Yes? Bob   Received: from PIZZA.BBN.COM by BBN.COM id aa20104; 16 Dec 93 19:44 EST Received: from pizza by PIZZA.BBN.COM id aa08913; 16 Dec 93 19:29 EST Received: from TTL.BBN.COM by PIZZA.BBN.COM id aa08909; 16 Dec 93 19:28 EST To: nimrod-wg@BBN.COM cc: Noel Chiappa Subject: Re: mobility and NIMROD In-reply-to: Your message of Wed, 15 Dec 93 02:09:25 -0500. <9312150709.AA10574@ginger.lcs.mit.edu> Date: Thu, 16 Dec 93 19:21:07 -0500 From: Ram Ramanathan There have been some discussions in this working group on whether or not mobility should involve routing. Noel writes : >I mean, if thing A has moved from locator X to locator Y, why should the >routing care? You just have to use locator Y instead of X if you want to get This has prompted the following high-level comments, after which I will return to earth. Before the era of mobility, the function of "routing" was mainly to get a packet from one entity to another *in the presence of node failures, link failures, changing link loads etc*. Routing would hardly have been the problem it was in the absence of the part within the *'s in the above sentence. Looking at it another way, routing addressed the problem of adapting to a dynamic topology ( a graph in which edge weights are a function of time, if you will). Node/link failures and changing link loads are one way by which the topology is dynamic. But not the only way. With the advent of mobility, it seems that there is yet another. In other words, mobility is nothing but the deletion of one link and the creation of another. Is it not natural then that there be a single component/layer that handles dynamic topology, namely routing, irrespective of what caused the dynamics? If wireless data networks had come into vogue before wired (we would have called THEM "ETHERnets" then, wouldn't we? :-)), would we not have designed one layer to handle anything to do with dynamic topologies? I think routing should care. Anyway, since mobility is being introduced in the context of "classical" routing, we need to look at it differently. Assuming that the transport layer should be oblivious to mobility, the layering would be something like this? TRANSPORT LAYER MOBILITY ADAPTATION LAYER ROUTING (NETWORK) LAYER DATA LINK LAYER So now we have the question of how to build the mobility adaptaion layer. It seems that the following suggestions have been put forward, for Nimrod at least. - On a move, notify appropriate translation services (DNS TNG) of the new EID to locator mapping. - On a move, notify source of traffic to the mobile host, if any, of the new location. Noel writes : >What exactly do you see as the drawbacks to the notification mechanism >(understanding that this is a long-range solution, not an immediately >deployable one)? Let me try. The (percieved) disadvantages are : 1. Scalability : If we are considering source notifications, and there are say 10000 moves (changes in network (wire/wireless) attachment points) per second (not improbable in 5-10 years), and if each notification is 128 bytes in length, we have a bandwidth requirement of 1.28 Mbytes/s. Note that if another source now wants to talk to the host that has moved, it will have to use the old locator and there will be triangle routing. With DNS notifications, there is this and the additional DNS consistency update messages generated. Maybe I am alone here, but I find it difficult to accept the fact that if I am in my office telnetting to California and move to a different "cell" with my laptop, an update message goes across the country or (in the case of DNS) many messages go across. 2. I presume that hosts/routers will use a EID-locator cache for the hosts they most talk to, for efficiency. This cache would have to be updated much more frequently if mobility entails a change in the DNS, thereby increasing the load on the system. 3. As Martha pointed out in an earlier message, whole clusters can move and we need to get routing involved then, anyway. 4. Global effect of a move. Maybe I am alone here, but I find it difficult to accept the fact that if I am in my office telnetting to California and move to a different room ("cell") with my laptop, an update message goes across the country or (in the case of DNS) many messages go across. Of course, I violently agree that this approach has a major advantage - simplicity. I am just worried that we do not defeat one of the purposes for which Nimrod is being designed - Scalability. An alternative is to hide the movement outside the cluster in which it occured. In this case, with the California example above, the host in California still thinks I am at the old locator. However, when it enters the BBN or even BBN 5th floor cluster, a cluster "representative" (router)that has been notified of the move directs the packet to the right room. This has disadvantages too, like complexity, but helps in scalability because now the number of updates in a cluster is proportional to the number of moves between immediate descendant clusters of that cluster instead of being proportional to the number of moves within the entire network. This is typically several orders of magnitude lower - for instance, there may only be about 20 moves per second in the BBN cluster. Noel writes : >authoritative for a given host is a approximately constant over time (probably >a reasonable bet), and the number of open connections for an average mobile >host is constant over time (maybe not quite so good an assumption, but I'll >bet it's a pretty small growth rate) the cost of the explicit notification >grows as O(1) as the network gets larger; i.e. it doesn't. (I'm leaving out ^^^^^^^^^^^^^^^^^^^ >second order effects, like the diameter of the network increasing meaning the >number of hops notifications to distant hosts have to traverse increases...) >This is a real win. I think the scalability issue is not about how large the network is, but how many hosts move (change locators) per unit time. If we have a network with a million zillion nodes and nobody moves, we obviously don't have a problem. It may be that the capacity of the networks in the future outpaces the degree of the mobility, in which case, doing explicit notifications makes a lot of sense. - Ram. -------------------------------------------------------------- Ram Ramanathan Systems and Technologies Bolt, Beranek and Newman Inc. 10 Moulton Street, Cambridge, MA 02138 Phone : (617) 873-2736 INTERNET : ramanath@bbn.com   Received: from PIZZA.BBN.COM by BBN.COM id aa11999; 16 Dec 93 23:46 EST Received: from pizza by PIZZA.BBN.COM id aa09517; 16 Dec 93 23:32 EST Received: from BBN.COM by PIZZA.BBN.COM id aa09513; 16 Dec 93 23:31 EST Received: from necom830.cc.titech.ac.jp by BBN.COM id aa05258; 16 Dec 93 23:30 EST Received: by necom830.cc.titech.ac.jp (5.65+/necom-mx-rg); Fri, 17 Dec 93 13:25:51 +0859 From: Masataka Ohta Return-Path: Message-Id: <9312170426.AA11707@necom830.cc.titech.ac.jp> Subject: Re: mobility and NIMROD To: Ram Ramanathan Date: Fri, 17 Dec 93 13:25:49 JST Cc: nimrod-wg@BBN.COM, jnc@ginger.lcs.mit.edu In-Reply-To: <9312170043.AA10190@necom830.cc.titech.ac.jp>; from "Ram Ramanathan" at Dec 16, 93 7:21 pm X-Mailer: ELM [version 2.3 PL11] > So now we have the question of how to build the mobility adaptaion layer. > It seems that the following suggestions have been put forward, for Nimrod > at least. > > - On a move, notify appropriate translation services (DNS TNG) of the > new EID to locator mapping. Though too many people expect too much for DNS, DNS is not a magic. > Of course, I violently agree that this approach has a major advantage - > simplicity. You can make anythng simple with simple-minded assumption that DNS TNG can do anything. Masataka Ohta   Received: from PIZZA.BBN.COM by BBN.COM id aa20372; 17 Dec 93 1:52 EST Received: from pizza by PIZZA.BBN.COM id aa09878; 17 Dec 93 1:34 EST Received: from nic.near.net by PIZZA.BBN.COM id aa09874; 17 Dec 93 1:32 EST Received: from nic.near.net by nic.near.net id aa14679; 17 Dec 93 1:33 EST To: nimrod-wg@nic.near.net cc: Masataka Ohta , Ram Ramanathan , jnc@ginger.lcs.mit.edu Subject: Re: mobility and NIMROD In-reply-to: Message of Fri, 17 Dec 1993 13:25:49 +0200. <9312170426.AA11707@necom830.cc.titech.ac.jp> Date: Fri, 17 Dec 1993 01:33:44 -0500 From: John Curran -------- ] From: Masataka Ohta ] Subject: Re: mobility and NIMROD ] Date: Fri, 17 Dec 93 13:25:49 JST ] ] > So now we have the question of how to build the mobility adaptaion layer. ] > It seems that the following suggestions have been put forward, for Nimrod ] > at least. ] > ] > - On a move, notify appropriate translation services (DNS TNG) of the ] > new EID to locator mapping. ] ] Though too many people expect too much for DNS, DNS is not a magic. Correct. ] > Of course, I violently agree that this approach has a major advantage - ] > simplicity. ] ] You can make anythng simple with simple-minded assumption that DNS TNG ] can do anything. Given one tool of infinite power, it is possible to remap almost any problem so that the tool may be applied. The question of EID->locator mapping has discussed many times. You definitely want to have a local cache of mappings from EID->locators, you'd like forward and backward notification if possible, local mobility at both the datalink (e.g. radio cells) and network layer (e.g. ESIS level 1 areas), etc. The real question comes down to what tool provides the mapping service when all of the above optimizations do not apply. There have been a number of interesting suggestions on big-internet over the last year and half: o Network-wide broadcast -- Right... N hosts times any update size is still a _very large_ number. o DNS -- Everyone's favorite tool for the problem at hand. One method that has not been fully disucussed is the use of hierarchical multicast for this purpose. Taking that as my tool of choice, I ask you consider the following: Presume (big leap of faith) that we have a hierarchical multicast system which has 64-bit group Ids. Assume further that we have administratively assigned EID's which are 64 bits in length, and assigned hierarchically for convenience sake. Systems have an nvram/eeprom which stores that value of this EID, and autoconfigure their network address using a netowrk prefix and unique interface value similiar to CLNP. Now here comes the magic: Presume that when a system is connected to the network, it joins the hierachical multicast group which has a one-to-one correspondence with it's EID. 1400 hosts in a building (for now, presuming similiar administrative EID prefixes) would be listening on their groups for "location requests". Any location information requrests for a given host would be sent to the group which corresponded to the hosts EID. Obviously, this does not scale. Let's see if we can make it scale: Presume that listening to a given multicast group costs a certain amount of $$ for given period of time. Further presume that it is possible to listen to a "higher" multicast group, thereby receiving all of the traffic for those multicast groups which are subsumed. (Listing to a higher group causes more location request traffic to be received, listening to "*" would result in an amazing amount of traffic.) Now here comes reality: Add to each "leaf" on the Internet, a single knob. This knob would control how large a multicast group that site would aggregate into a single join request. For example, BBN (which has thousands of systems whose EID's begin with the 32-bit prefix "234.156.193.024." would perhaps decide to aggregate all of the internal multicast group joins into a single join for this prefix. When hearing requests for individual systes, this aggregator would have to send the request downstream as necessary. BBN's motivation for aggregating is to avoid the $$ cost of 1000+ little group listeners. BBN does not join a larger prefix because it doesn't want the needless traffic. Let's say hypothetically that there are about 40 MIT systems at BBN, due project work and joint activities. This systems have their own EID prefix, and may even come predominently from one or two groups at MIT (which may easily have their own subprefix for EID assignments). The displaced-MIT systems will obtain topologically-correct addresses at startup (since they're on tbe BBN internal net), but each need to join the corresponding multicast group so that it can receieve requests for location and return the correct replies. These 40 annnouncements are expensive... they need to be tracked by "higher" multicast servers, and in effect, are 40 times the burden of joining the the single BBN prefix group. Clearly, it would be possible for the BBN location aggregator to join just the MIT EID prefix group, but who really want all of the location traffic for all of MIT hosts? Potentially, the various tradoffs could result in BBN joining a two MIT prefixes for the pair of MIT departments which have the most displaced systems, and perhaps 3 or 4 individual prefixes to cover the loose systems. Now, some interesting details: It turns out that for aggregation to work, it's crucial to know not just the EIDs that are active in a given leaf, but to also know the EID's which _have not been assigned_. This is becuase we cannot afford to have multiple sources of any given EID (multiple sources cannot be aggregated, they must all be kept seperate.) and we cannot afford have gaps which inhibit aggragation. For example, if the BBN shipping dept has a range of 32 continguous EIDs for its use but only 13 systems, we'd really like to hear EID-group-joinings for each system that is presently on the network, and then a single message that says that the following 19 eids are effectively "known", i.e. the local aggregator knows that these 19 eids are not in use elsewhere in the world, and therefore is announcing these to prevent "holes" from occuring. (such an announcement could look very similiar to normal system announcements, only with a metric of "known-not-in-use". It would be necessary to allow the individual announcements to contain prefixes so that the 19 systems could be represented via just 4 eid-prefix-groups in the ideal case). The BBN main aggregator should be able to take the sum of all internal EID- prefix-joins and build a complete prefix announcement (minus all those systems whcih do exist but are not presently connected at BBN). Because of the knowledge of known holes, it is posible to safely aggregate across such ranges, and the resulting set of announcements to completely cover the EID space (even without turning a knob to aggregate over unknown systems) should be rather small. Each hole represents a system which is currently elsewhere on the network, or is not available locally. Note that in the case of BBN joining MIT department prefix groups, there would be multiple listers to be tracked by the "higher" multicast tables, but these could combine to a single prefix at the next layer up (regional or national) in some cases. One real test case for an facility like this would be an event such as the InteropNET, with 1000 organizations each bringing 1 to 20 systems to the show, and (say) 10000 individual users with mobile devices. The provider for such a network will have problems with aggregation; with a bit work of it should be possible to get the group joins down to one per exhibitor (the charge for listening to the exhibitors EID prefix would be added to the exhibitor costs) and the individuals would have to pay to have their system EID listened to. Note that the free market could do wonders with this model... if a user doesn't want to get charged for having the network track his prefix, he can always have his home network simply announce it and buy a mobility server which answers location requests. When he moves to a new network, his startup procedures contacts the server and says: "I'm here, right now". This is done transparent to the current network he's connected to and it's EID-prefix-groups being listened to. Enough for now, /John   Received: from PIZZA.BBN.COM by BBN.COM id aa08132; 17 Dec 93 8:24 EST Received: from pizza by PIZZA.BBN.COM id aa10913; 17 Dec 93 8:14 EST Received: from BBN.COM by PIZZA.BBN.COM id aa10909; 17 Dec 93 8:12 EST Received: from TIEO.SAIC.COM by BBN.COM id aa07754; 17 Dec 93 8:13 EST Received: by tieo.saic.com (4.1/1.34) id AA02755; Fri, 17 Dec 93 08:15:13 EST Date: Fri, 17 Dec 93 08:15:13 EST From: Kenneth Carlberg Message-Id: <9312171315.AA02755@tieo.saic.com> To: ramanath@BBN.COM, mohta@necom830.cc.titech.ac.jp Subject: Re: mobility and NIMROD Cc: nimrod-wg@BBN.COM, jnc@ginger.lcs.mit.edu From Martha... > Mobility need not be confined to endpoints. Whole clusters may move. Excellent point. One example beyond those you cited has been brought up by Robert Moskowitz who has talked about Chryslers plan (or rather proposal) of configuring each of its cars to be a LAN. From Noel... > could you provide some more detail on what you see as the problems > of notification? Let me add to what Ram said in the _percieved_ drawbacks of using just a notification mechanism. (note: I grant you, some of the following could be classified as nits). o Cost - today, most of us on the Internet use a flat fee for unlimited service (forwarding/receiving of packets). In the future, I would assume that another type of charging system would be made available (for the bean counters of a company ;-) that would be based on a "charge per packet" fee. Now I *freely admit* that the amount of traffic used to update where I am would be expected to be very small, but the point is that I still get penalized for simply moving and I think that is a bad model to aim for. Another cost relates to the need of authenticating my notification. My presumption is that some type of certificate system would need to be put in place to verify that the notification is valid. And I would assume that any usage of a security apparatus will incur more costs (i.e., more penalties for simply moving) o pipe size - The pipe size, of both the wireless system as well as _access_ to the Internet, will always be a factor in traffic overhead. Again, this is possibly another nit, but the delay incurred from the pipe size helps in assuring that some percentage of the updates are already stale when they reach their destination. This leads to the next item... o forwarding from "previous location" - in accepting the fact that a percentage of update notifications will not be valid (eg., the update says I'm in A.B.X.1 when I'm really in A.B.X.2), then either (a) traffic will be dropped, or (b) some type of _local_ tracking/updating will have to be employed to minimize dropped traffic. If (a) is acceptable to the end user, fine, if not then (b) puts us back into using two schemes. From Masataka... > Though too many people expect too much for DNS, DNS is not a magic... > ..You can make anythng simple with simple-minded assumption that DNS TNG > can do anything. No, contrary to popular belief, DNS is not magic. But it is a good example to build upon so that a possible next generation DNS could be used within the context of Nimrod. -ken   Received: from PIZZA.BBN.COM by BBN.COM id aa16069; 17 Dec 93 10:39 EST Received: from pizza by PIZZA.BBN.COM id aa11466; 17 Dec 93 10:23 EST Received: from TTL.BBN.COM by PIZZA.BBN.COM id aa11462; 17 Dec 93 10:21 EST To: Masataka Ohta cc: nimrod-wg@BBN.COM Subject: Re: mobility and NIMROD In-reply-to: Your message of Fri, 17 Dec 93 13:25:49 +0200. <9312170426.AA11707@necom830.cc.titech.ac.jp> Date: Fri, 17 Dec 93 10:14:24 -0500 From: Ram Ramanathan >> Of course, I violently agree that this approach has a major advantage - >> simplicity. >You can make anythng simple with simple-minded assumption that DNS TNG >can do anything. I don't think I or anyone else who has written recently has made the "simple-minded" assumption that DNS TNG can do "anything". What I was referring to in "simplicity" was that within the context of Nimrod (ie, for the designers of Nimrod), sending updates to DNS TNG for each move and asking DNS TNG for a locator each time you want a locator appears to be simpler (FOR NIMROD) than involving routing. What would Nimrod need of DNS in this case? It seems that we need the following three things : 1) on the receipt of a new EID - locator mapping, update the old mapping. 2) synchronize the update to all distributed DNS entities. 3) given a query with an EID, return the corresponding locator. I am no DNS expert, but the functionality *as such* is not so demanding? However, the hidden devil is that the sheer number of such requests that would be generated in a future mobile environment would be enough to give any distributed database designer nightmares. It is for such "scalabililty" reasons that I advocated the use of localizing movement information and having a minimal amount of DNS updates, if any. -Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa17611; 17 Dec 93 11:06 EST Received: from pizza by PIZZA.BBN.COM id aa11607; 17 Dec 93 10:49 EST Received: from BBN.COM by PIZZA.BBN.COM id aa11603; 17 Dec 93 10:48 EST Received: from nsco.network.com by BBN.COM id aa16521; 17 Dec 93 10:47 EST Received: from anubis-e1.network.com by nsco.network.com (5.61/1.34) id AA11013; Fri, 17 Dec 93 09:51:22 -0600 Received: from blefscu.network.com by anubis.network.com (4.1/SMI-4.1) id AA24473; Fri, 17 Dec 93 09:46:25 CST Date: Fri, 17 Dec 93 09:46:25 CST From: Andrew Molitor Message-Id: <9312171546.AA24473@anubis.network.com> To: nimrod-wg@BBN.COM Subject: Re: mobility and NIMROD I've been thinking about mobility for a while. Does it not require fairly strong authentication, in general? Lest I kill host A, and then have my laptop pretend that it's host A by persuading the directory authorities that host A has just moved. This is, of course, general to mobility, but if Nimrod is going to provide support for this, some thought needs to be given to the mechanisms for authenticating moves, in the context of Nimrod. Andrew Molitor   Received: from PIZZA.BBN.COM by BBN.COM id aa22970; 17 Dec 93 12:44 EST Received: from pizza by PIZZA.BBN.COM id aa12056; 17 Dec 93 12:24 EST Received: from TTL.BBN.COM by PIZZA.BBN.COM id aa12052; 17 Dec 93 12:22 EST To: Andrew Molitor cc: nimrod-wg@BBN.COM Subject: Re: A slightly different take on clusters - In-reply-to: Your message of Tue, 14 Dec 93 13:51:32 -0600. <9312141951.AA24960@anubis.network.com> Date: Fri, 17 Dec 93 12:16:39 -0500 From: Ram Ramanathan Some questions/comments on Andrew Molitor's article on a new way of cluster representation and route generation using them : > It appears to me that areas or clusters could most handily be >used to arbitrarily group things together. With overlapping areas, the >whole notion of 'links' and 'networks' can go away, in fact. There are Overlapping clusters (non-nested overlapping to be precise) generate some problems like management of multiple locators, policy enforcement at boundaries etc., which were discussed a few months back in this WG. Morover, overlapping clusters, even if they are allowed in Nimrod, would be more the exception than the rule. However, any scheme that can handle overlapping clusters can handle non-overlapping (as a special case). But it is probably not a good idea to have route generation *require* clusters to overlap so that a route can be expressed solely in terms of clusters. >because it's appropriate). The 'filter' describing a thing in the >network would be the set of all clusters which contain the thing, or >which contain a cluster that contains the thing, and so on up the >hierarchy. And if clusters don't overlap, a filter and locator are one and the same, right? >have', then you can generate routes. Getting the list of things in a >cluster is termed 'exploding' it, and the list itself is 'the >explosion.' To build a route, you do this: > 1) get the filter for your destination end-thing. > 2) get your own filter. > 3) find the lowest level cluster the two filters > share. There may be more than one lowest level cluster the filters share, no? > 4) explode that cluster, and find a sequence of intersecting > clusters in the explosion with the right attributes. > 5) recursively explode each of these clusters until you > hit the bottom. > 6) you're done. If the attributes are propagated up the enclosing clusters as has been assumed later, item #5 may not be necessary. Suppose there are three clusters A, B, C. Cluster A contains a cluster D and cluster C contains a cluster E. Suppose further that to get from A to C, you have to go through B. Say we want to go from an end-thing in D to an end-thing in E. If the (propagated) attributes of B are okay for the end-thing in D, it need not recursively explode B. We just use B as a "transit cluster" and our route contains B, without actually containing the details, which are filled in, during path setup time at cluster B itself. In fact, it seems to me that the ability to explode all the way to the bottom for all the clusters enroute to the destination presumes the "detailed" map which is one of the things Nimrod is trying to hide. I found this alternate take on clusters interesting. In fact some of us had been tossing around some ideas with some amount of similarity with what was written. I will probably post a "yet another take on clusters" soon. - Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa29875; 17 Dec 93 13:44 EST Received: from pizza by PIZZA.BBN.COM id aa12269; 17 Dec 93 13:14 EST Received: from BBN.COM by PIZZA.BBN.COM id aa12265; 17 Dec 93 13:12 EST Received: from nsco.network.com by BBN.COM id aa28303; 17 Dec 93 13:11 EST Received: from anubis-e1.network.com by nsco.network.com (5.61/1.34) id AA12483; Fri, 17 Dec 93 12:14:57 -0600 Received: from blefscu.network.com by anubis.network.com (4.1/SMI-4.1) id AA26001; Fri, 17 Dec 93 12:10:01 CST Date: Fri, 17 Dec 93 12:10:01 CST From: Andrew Molitor Message-Id: <9312171810.AA26001@anubis.network.com> To: nimrod-wg@BBN.COM Subject: Re: A slightly different take on clusters - Ram Ramanathan writes: >And if clusters don't overlap, a filter and locator are one and the same, >right? Exactly. I didn't say it explicitly, because I hadn't thought it through enough, but a filter is not only a raw list of clusters, but contains structuring information, about what level each listed cluster is, and which of the listed clusters are contained in which. For us math geeks, a filter is a partially ordered set of clusters, while a traditional locator is a linearly ordered set. Traditional Filter Level 3 foo foo wubba blub | \ / | Level 2 bar bar baz | \ / Level 1 blub otterpaws Where a line indicates that the higher cluster contains the lower. The traditional is just a degenerate case of the general thing. Any system which allows overlapping clusters is stuck with these things, I suspect. > > 1) get the filter for your destination end-thing. > > 2) get your own filter. > > 3) find the lowest level cluster the two filters > > share. > > There may be more than one lowest level cluster the filters share, no? Oh my, good point! Umm, choose randomly amoung the lowest level shared clusters with the right attributes? (handwave, handwave) > > > 4) explode that cluster, and find a sequence of intersecting > > clusters in the explosion with the right attributes. > > 5) recursively explode each of these clusters until you > > hit the bottom. > > 6) you're done. > > If the attributes are propagated up the enclosing clusters as has been > assumed later, item #5 may not be necessary. Suppose there are three clusters > A, B, C. Cluster A contains a cluster D and cluster C contains a cluster E. > Suppose further that to get from A to C, you have to go through B. > > Say we want to go from an end-thing in D to an end-thing in E. If the > (propagated) attributes of B are okay for the end-thing in D, it need not > recursively explode B. We just use B as a "transit cluster" and our route > contains B, without actually containing the details, which are filled in, > during path setup time at cluster B itself. If you're trying to actually move a packet via the higher-level cluster B, it's not clear to me how that actually maps to answers to the questions 'what MAC address do I, Generic Router, need to heave this packet at right now?'. Oh, hmm. Were you suggesting that one could build a looser source route by putting higher-level clusters in it, and having the routers do the detailed map exploding along the way, at connection setup time? So the source doesn't bother to explode B, in your example, and depends on things along the way to figure out how to get across B. I find this very appealing. It should improve performance, by moving the more detailed map lookups closer to the owners of the detailed maps, at (perhaps) some cost in policy control. It implies depending on remote systems to implement your policy needs, at the very least. >I found this alternate take on clusters interesting. In fact some of us >had been tossing around some ideas with some amount of similarity with what >was written. I will probably post a "yet another take on clusters" soon. My thanks! I look forward to your remarks. Andrew Molitor >- Ram.   Received: from PIZZA.BBN.COM by BBN.COM id aa07191; 17 Dec 93 15:37 EST Received: from pizza by PIZZA.BBN.COM id aa12903; 17 Dec 93 15:26 EST Received: from KARIBA.BBN.COM by PIZZA.BBN.COM id aa12899; 17 Dec 93 15:24 EST To: Masataka Ohta cc: Ram Ramanathan , nimrod-wg@BBN.COM, jnc@ginger.lcs.mit.edu Subject: Re: mobility and NIMROD In-reply-to: Your message of Fri, 17 Dec 93 13:25:49 +0200. <9312170426.AA11707@necom830.cc.titech.ac.jp> Date: Fri, 17 Dec 93 15:23:39 -0500 From: Isidro Castineyra >>> So now we have the question of how to build the mobility adaptaion layer. >>> It seems that the following suggestions have been put forward, for Nimrod >>> at least. >>> >>> - On a move, notify appropriate translation services (DNS TNG) of the >>> new EID to locator mapping. >> >>Though too many people expect too much for DNS, DNS is not a magic. >> If the DNS, or something just like the DNS, provides the correct solution for our problem, and the current DNS would not be able to deal with the load, we should upgrade it. Isidro Isidro Castineyra (isidro@bbn.com) Bolt Beranek and Newman, Incorporated (617) 873-6233 10 Moulton Street, Cambridge, MA 02138 USA   Received: from PIZZA.BBN.COM by BBN.COM id aa13443; 17 Dec 93 17:46 EST Received: from pizza by PIZZA.BBN.COM id aa13552; 17 Dec 93 17:27 EST Received: from TTL.BBN.COM by PIZZA.BBN.COM id aa13548; 17 Dec 93 17:24 EST To: nimrod-wg@BBN.COM Subject: Re: A slightly different take on clusters - In-reply-to: Your message of Tue, 14 Dec 93 13:51:32 -0600. <9312141951.AA24960@anubis.network.com> Date: Fri, 17 Dec 93 17:22:32 -0500 From: Ram Ramanathan This is YET another take on clustering in Nimrod. Let me begin by quoting the last few lines of Andrew Molitor's message of Tue, 14 Dec : >another. Let's throw them in a box labeled 'connected'! As an added >bonus, you don't have to worry about what the nodes and the arcs in >your graphs mean, since you don't have them any more ;) In this writeup, I have described what I hope is a consistent representation scheme ALSO not using nodes/arcs explicitly as the basic entities doing the representation. The idea is to have more generality to be able to accommodate within the same framework, different requirements - such as interface being an arc or a node. Both options can be expressed within this framework and the user can suit him/her self as to what he/she does. What is here is not much different from what Martha wrote about to this this working group a while ago (11/10). Clusters however, have been generalized here to not be built not from just interfaces/routers as basic entities, but with anything. Also, I have tried to draw parallels to the node/arc scheme wherever possible. ABSTRACT of the writeup (for those who don't have time to read the whole thing) There are four entities that are used for representing topology in (this scheme of) Nimrod. These are : cluster, boundary point, intercluster link, intracluster link. A "cluster" is a Nimrod addressable entity. Data flows into and out of a cluster by means of "intercluster links" incident to that cluster at "boundary points". An intercluster link connects two boundary points in distinct clusters and represents the fact that data can flow between these two clusters without going through any other cluster. The characteristics of a cluster are expressed by "intracluster links" between a pair of boundary points belonging to the same cluster. A route typically consists of a sequence of inter/intra cluster links. An intracluster link abstracts out the details of a cluster in terms of attributes of the cluster. All of these entities have "locators" associated with them. An entity's locator is composed using labels and/or locators of other entities. For instance, the intercluster link from cluster A.B to cluster A.C can be written as (A.B -> A.C). ELABORATION (for those who do have the time) 1. Representation details A set of clusters and the links between them may be aggregated to form a cluster. The former clusters are then said to be "contained" within the latter cluster. Recursive aggregation of clusters in this manner results in a (cluster) "hierarchy". Boundary points effect an association between intercluster and intracluster links. The entire transit characteristics of the cluster are expressed in the individual characteristics of the intracluster links between each pair of boundary points. Very informally, clusters represent network elements that are to be treated somewhat uniformly. Intercluster links represent connectivity between clusters and intracluster links represent the transit characteristics of that cluster. Thus, they are quite different things and have different locator construction schemes (explained later). Note that if there is an intercluster link L between clusters A and B and both A and B are contained within cluster C, then L is *not* an intracluster link for C. Intracluster links always run between boundary points of the same cluster. [Remark : In a sense, the names "intra" and "inter" are misleading and should probably be changed, although I cannot think of anything better at the moment]. The following is an example of a map, illustrated. Efficient data structures to store and manipulate this is an interesting topic. Note that there may be more than one intracluster link in a cluster, denoting different ways of transiting that cluster. Here, boxes are clusters --- lines are intercluster links ... lines are intracluster links x are boundary points Attributes not represented. A B C ----------- ----------- ---------- | | | | | | | x-----x.........x----x........x | | |. | _/ |. | | | | ........x/ | . | ----------- ----------- ---x------ [Comment 1 : No "basic" entities? Note that no mention has been made of any "basic" entities. There is no basic entity other than a cluster. Within a cluster, a "local mechanism" determines transfer of data entering the cluster to physical entities within the cluster. These entities may be whatever you want - interfaces, routers, networks, processes and so on. From Nimrod's viewpoint, its job is to get it to the destination cluster indicated, it does not need to worry about the basic entities within the cluster.] [Comment 2 : No "nodes"? Note also that there is no representational entity "node". Nodes and arcs are an useful framework for many routing problems and we are used to graphs, but traditional graphs are not necessarily the representational panacea for *all* routing problems. We can, without any undue difficulty, generate a route from cluster A to cluster B using only the four representational entities I mentioned : cluster, boundary point, intercluster links and intracluster links. Typically, the inter and intra cluster links will behave as "edges/arcs" and the boundary points as "nodes" (if one wants to use the graph theoretical paradigm, not a necessary view in my opinion). However, if clusters have uniform attributes between its boundary points, then nodes might well be clusters . So what "nodes" are is dependent on what route generation wants.] [Comment 3 : The N^2 problem] Here we have the famous N^2 problem, ie, the possibility of the number of intracluster links growing as square of the boundary points. If all the intracluster links do have different attributes and if all of the information has to be necessarily expressed, then we can do no better than N^2. However, neither of these conditions is true typically, and we can group the information (that are common) and smooth it (hide slight differences) to reduce the complexity. Such condensation of information is one form of abstraction (IMHO). A simple grammar for such a thing is used by IDPR (based on Dave Clark's policy representation scheme - refer recent note on this mailing list by Ken Carlberg 11/23). 2. How is a route generated and what does it look like? The input to route generation is a map containing clusters, boundary points, intracluster and intercluster links with corresponding attributes. The route for a given unicast destination is a sequence of inter/intra links. How detailed the route is depends on how detailed the map is. A cluster may simply advertise its intracluster links (low detail) or advertise the actual connectivity between the clusters it contains. The optimality of the route generated would typically be proportional to the detail in the map. If the details are recursively gotten to the bottommost level, all intracluster links dissappear and we can write down the route in terms of intercluster links between the bottom level clusters/end-things alone. For instance, an entity in cluster D below may have (the less detailed) map #1 or the more detailed map #2. Map #1 : A B C ----------- ----------- ---------- | | | | | | | |-----|.........|----| | | | | | | | | | | | | | ----------- ----------- ---------- Route with Map #1 = (...interClink A-B, inraClink B, interClink B-C....) Map #2 : A B C ----------- ----------- ---------- | | |D__ E__ | | | | |-----|-|.|-|..||----| | | | | --- -- | | | | | | | | | ----------- ----------- ---------- Route with Map #2 = (.. interClink A-D, intraClink D, interClink D-E, intraClink E, interClink E-C ....) Note that the boundary points involved in the route are implicitly specified in a route consisting of links. 3. How can locators be assigned to the representational entities? All of the entities discussed above, namely, clusters, boundary points, intercluster links and intracluster links are associated with a unique "locator" that identifies the location of the entity within a particular cluster hierarchy. The basic constituent of a locator is a "label". The locator for an entity, typically, combines in some manner, labels or other locators. The rules for forming the locators could be as follows : Cluster ::- l1.l2.l3...lk, such that l1 is U the universal cluster. lk is the label assigned to the concerned cluster and for all n from 1 to k, ln is contained in l(n-1). Boundary point :- ,[