EtherType	Protocol
0x0800	IP Internet Protocol (IPv4)
0x0806	Address Resolution Protocol (ARP)
0x8035	Reverse Address Resolution Protocol (RARP)
0x809b	AppleTalk (Ethertalk)
0x80f3	Appletalk Address Resolution Protocol (AARP)
0x8137	Novell IPX (alt)
0x8138	Novell
0x86DD	Internet Protocol, Version 6 (IPv6)

IEEE 802.x defined the 16 bit field after the MAC addresses as a length field again. As Ethernet I framing is no longer used, this allows software to determine whether a frame is an Ethernet II frame or an IEEE 802.x frame, allowing the coexistence of both standards on the same physical medium. All 802.x frames have an LLC field. By examining the LLC field, it is possible to determine whether it is followed by a SNAP field.

Novell's "raw" 802.3 frame format was based on early IEEE 802.3 work. Novell used this as a starting point to create the first implementation of its own IPX Network Protocol over Ethernet. They did not use any LLC header but started the IPX packet directly after the length field. In principle this is not interoperable with the other later variants of 802.x Ethernet, but since IPX has always FF at the first byte (while LLC has not), this mostly coexists on the wire with other Ethernet implementations (with the notable exception of some early forms of DECnet which got confused by this).

Novell Netware used this frame type by default until the mid nineties, and since Netware was very widespread back then (while IP was not) at some point in time most of the world's Ethernet traffic ran over "raw" 802.3 carrying IPX. Since Netware 4.10 Netware now defaults to IEEE 802.x with LLC (Netware Frame Type Ethernet_802.2) when using IPX. There is a Classical Series of Usenet Postings (http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&c2coff=1&frame=right&th=887b61494cf0c72a&seekm=1993Sep17.191208.13580%40novell.com#link1) by Novell's Don Provan that have found their way into numerous FAQs and are widely considered the definitive answer to the Novell Frame Type jungle.

The 802.x variants of Ethernet are not in widespread use on common networks currently, with the exception of large corporate Netware installations that have not yet migrated to Netware over IP. The most common type used today is Ethernet Version 2, as it is used by most Internet Protocol-based networks, with its EtherType set to 0x0800. There exists a well defined standard (http://www.ietf.org/rfc/rfc1042.txt) for encapsulating IP traffic in IEEE 802.3 frames with LLC/SNAP headers, but it is commonly not supported.

IP Version 6 over Ethernet is also standardized based on IEEE 802.x. with LLC/SNAP.

Novell IPX is probably the only network protocol that supports all 4 current frame types.

Varieties of Ethernet

Other than the framing types mentioned above, most of the other differences between Ethernet varieties have all been variations on speed and wiring. Therefore, in general, network protocol stack software will work identically on most of the following types.

The following sections provide a brief summary of all the official ethernet media types. In addition to these official standards, many vendors have implemented proprietary media types for various reasons—often to support longer distances over fiber optic cabling.

Many Ethernet cards and switch ports support multiple speeds, using auto-negotiation to set the speed and duplex for the best values supported by both connected devices. If auto-negotiation fails, a multiple speed device will sense the speed used by its partner, but will assume half-duplex. A 10/100 Ethernet port supports 10BASE-T and 100BASE-TX. A 10/100/1000 Ethernet port supports 10BASE-T, 100BASE-TX, and 1000BASE-T.

Some early varieties of Ethernet

• Xerox Ethernet -- the original Ethernet implementation, which in turn had two versions, Version 1 and Version 2, during its development. The version 2 framing format is still in common use.
• 10BROAD36 -- Obsolete. An early standard supporting ethernet over longer distances. It utilized broadband modulation techniques similar to those employed in cable modem systems, and operated over coaxial cable.
• 1BASE5 -- An early attempt to standardize a low-cost LAN solution, it operates at 1 Mbit/s and was a commercial failure.
• StarLAN 1 -- The first implementation of Ethernet on twisted pair wiring.

10 Mbit/s (10 Mbps) Ethernet

• 10BASE5 (also called Thickwire or Yellow Cable) -- this is the original 10Mbit/s implementation of Ethernet. The early IEEE standard uses a single 50-ohm coaxial cable of a type designated RG-8, of maximum length 500 metres, into which the transceiver was connected by a so-called "vampire tap", which attached itself by drilling into the cable to connect to the core and screen. An AUI cable then connected the transceiver to the Ethernet device. Largely obsolete, though due to its widespread deployment in the early days, some systems may still be in use. It requires precise termination at each end of the cable.
• 10BASE2 (also called Thinwire or Cheapernet) -- 50-ohm RG-58 coaxial cable, of maximum length 200 metres, connects machines together, each machine using a T-adaptor to connect to its NIC, which has a BNC connector. Requires terminators at each end. For many years this was the dominant 10 Mb/s ethernet standard.
• StarLAN 10 -- First implementation of Ethernet on twisted pair wiring at 10 Mbit/s. Later evolved into 10BASE-T.
• 10BASE-T -- runs over 4 wires (two twisted pairs) on a cat-3 or cat-5 cable. A hub or switch sits in the middle and has a port for each node. This is also the configuration used for 100BASE-T and Gigabit ethernet. 10 Mbit/s.
• FOIRL -- Fiber-optic inter-repeater link. The original standard for ethernet over fibre.
• 10BASE-F -- A generic term for the new family of 10 Mbit/s ethernet standards: 10BASE-FL, 10BASE-FB and 10BASE-FP. Of these only 10BASE-FL is in widespread use.
• 10BASE-FL -- An updated version of the FOIRL standard.
• 10BASE-FB -- Intended for backbones connecting a number of hubs or switches, it is now obsolete.
• 10BASE-FP -- A passive star network that required no repeater, it was never implemented

Fast Ethernet (100 Mbit/s)

• 100BASE-T -- A term for any of the three standards for 100 Mbit/s ethernet over twisted pair cable. Includes 100BASE-TX, 100BASE-T4 and 100BASE-T2.
• 100BASE-TX -- Similar star-shaped configuration to 10BASE-T. It also uses two pairs, but requires cat-5 cable. 100Mbit/s.
• 100BASE-T4 -- 100 Mbit/s ethernet over cat-3 cabling (as used for 10BASE-T installations). Uses all four pairs in the cable. Now obsolete, as cat-5 cabling is the norm. Limited to half-duplex.
• 100BASE-T2 -- No products exist. 100 Mbit/s ethernet over cat-3 cabling. Supports full-duplex, and uses only two pairs. It is functionally equivalent to 100BASE-TX, but supports old cable.
• 100BASE-FX -- 100 Mbit/s ethernet over fibre.

Gigabit Ethernet

• 1000BASE-T -- 1 Gbit/s over cat-5e or cat-6 copper cabling.
• 1000BASE-SX -- 1 Gbit/s over multi-mode fiber (up to 550 m).
• 1000BASE-LX -- 1 Gbit/s over multi-mode fiber (up to 550 m). Optimized for longer distances (up to 10 km) over single-mode fiber.
• 1000BASE-LH -- 1 Gbit/s over single-mode fiber (up to 100 km). A long-haul solution.
• 1000BASE-CX -- A short-haul solution (up to 25 m) for running 1 Gbit/s Ethernet over special copper cable. Predates 1000BASE-T, and now obsolete.

10 Gigabit Ethernet

The new 10 gigabit Ethernet standard encompasses seven different media types for LAN, MAN and WAN. It is currently specified by a supplementary standard, IEEE 802.3ae, and will be incorporated into a future revision of the IEEE 802.3 standard.

• 10GBASE-SR -- designed to support short distances over deployed multi-mode fiber cabling, it has a range of between 26 m and 82 m depending on cable type. It also supports 300 m operation over a new 2000 MHz.km multi-mode fiber.
• 10GBASE-LX4 -- uses wavelength division multiplexing to support ranges of between 240 m and 300 m over deployed multi-mode cabling. Also supports 10 km over single-mode fiber.
• 10GBASE-LR and 10GBASE-ER -- these standards support 10 km and 40 km respectively over single-mode fiber.
• 10GBASE-SW, 10GBASE-LW and 10GBASE-EW. These varieties use the WAN PHY, designed to interoperate with OC-192 / STM-64 SONET/SDH equipment. They correspond at the physical layer to 10GBASE-SR, 10GBASE-LR and 10GBASE-ER respectively, and hence use the same types of fiber and support the same distances. (There is no WAN PHY standard corresponding to 10GBASE-LX4.)

10 gigabit Ethernet is very new, and it remains to be seen which of the standards will gain commercial acceptance.

Related standards

These networking standards are not part of the IEEE 802.3 Ethernet standard, but support the ethernet frame format, and are capable of interoperating with it.

• 100BaseVG -- An early contender for 100 Mbit/s ethernet. It runs over Category 3 cabling. Uses four pairs. Commercial failure.
• TIA 100BASE-SX -- Promoted by the Telecommunications Industry Association. 100BASE-SX is an alternative implementation of 100 Mbit/s ethernet over fiber; it is incompatible with the official 100BASE-FX standard. Its main feature is interoperability with 10BASE-FL, supporting autonegotiation between 10 Mbit/s and 100 Mbit/s operation -- a feature lacking in the official standards due to the use of differing LED wavelengths. It is targeted at the installed base of 10 Mbit/s fiber network installations.
• TIA 1000BASE-TX -- Promoted by the Telecommunications Industry Association, it was a commercial failure, and no products exist. 1000BASE-TX uses a simpler protocol than the official 1000BASE-T standard, but requires Category 6 cabling.

(Back to Top)

FrameRelay

Frame relay is an efficient data transmission technique used to send digital information quickly and cheaply to one or many destinations from one point. It can be used for voice, data, local area network (LAN), and wide area network (WAN) traffic. Each frame relay end user gets a private line to a frame relay node. The frame relay network handles the transmission to its other end users over a path which is always changing and is invisible to the end users.

Frame relay is a packet switched Telecommunications network, commonly used at the data link layer (layer 2) of the Open Systems Interconnection reference model. Generally the concept of permanent virtual circuits (PVCs) is used to form logical end-to-end links mapped over a physical network. Switched virtual circuits (SVCs), analogous to circuit switching in the public switched telephone network, are also part of the Frame relay specification but are rarely applied in the real world. Frame relay was originally developed as a stripped-down version of the X.25 protocol.

Datalink Connection Identifiers or DLCIs are a locally significant numeric value to represent each end point. Multiple PVCs can be mapped to the same physical end point. It is often provisioned with a Committed Information Rate (CIR) and a burstable component sometimes known as Extended Information Rate (EIR).

Frame relay was designed to make more efficient use of existing physical resources, thereby allowing the overprovisioning of data services to telco customers, as most clients are unlikely to be utilising a data service 100 percent of the time. Frame relay has acquired a bad reputation in some markets because of excessive bandwidth overbooking by telcos.

Frame relay is/was often sold by Telecommunications companies to businesses looking for a cheaper alternative than dedicated lines, its use in different areas depending on governmental and telecommunication's companies policies.

Frame relay is being displaced by ATM and native IP based products, including IP virtual private networks.

(Back to Top)

FTP

The File Transfer Protocol (FTP) is a software standard for transferring computer files between machines with widely different operating systems. It belongs to the application layer of the Internet protocol suite.

FTP is an 8-bit client-server protocol, capable of handling any type of file without further processing, such as MIME or Uuencode. However, FTP has extremely high latency; that is, the time between beginning the request and starting to receive the required data can be quite long, and a sometimes-lengthy login procedure is required.

FTP is commonly run on two ports, 20 and 21. Port 20 is a data stream which transfers the data between the client and the server. Port 21 is the control stream and is the port where commands are passed to the ftp server. While data is being transferred via the data stream, the control stream sits idle. This can cause problems with large data transfers through firewalls which time out sessions after lengthy periods of idleness. While the file may well be successfully transferred, the control session can be disconnected by the firewall, causing an error to be generated.

The objectives of FTP are:

1. To promote sharing of files (computer programs and/or data).
2. To encourage indirect or implicit use of remote computers.
3. To shield a user from variations in file storage systems among different hosts.
4. To transfer data reliably and efficiently.

Disadvantages are:

1. Passwords and file contents are sent in clear text, allowing eavesdropping which may be unwanted.
2. It is hard to filter active mode FTP traffic on the client side by using a firewall, since the client must open a random port in order to make the connection. This problem is largely resolved by using passive mode FTP.
3. It is possible to tell a server to send to an arbitrary port of a third computer.
4. FTP, though usable directly by a user at a terminal, is designed mainly for use by FTP client programs.

Many sites that run FTP servers enable so-called "anonymous ftp". Under this arrangement, users do not need an account on the server. By default, the account name for the anonymous access is 'anonymous'. This account does not need a password. Although users are commonly asked to send their email addresses as their passwords for authentication, usually there is trivial or no verification, depending on the FTP server and its configuration.

Active and passive mode

There are two modes that can be used for FTP: active and passive. Active mode requires both the client and the server to open a port and listen on it in order to establish an FTP session. As this often causes problems with firewalls on the client computer, passive mode was created. Passive mode requires only the server to have a process listen on a port, and thus it bypasses firewall issues on the client computer.

An active mode FTP connection is established in the following manner:

1. A random unprivileged port (a port with a number above 1024, in this case we will call it x) is opened on the client and the FTP program connects to port 21, known as the command port, on the server. The source port is now the random port number x on the client and the destination port is port 21 on the server.
2. The client starts listening to port (x+1) and sends a PORT command to the server (via the server's command port 21) that tells the server the port number it is listening to and that it is ready to receive data on this port. This port is known as the data port.
3. The server opens source port 21 and connects to the client's data port. The source port is port 21, and the data port (x+1) is the destination port.
4. The client opens a connection to port 21 on the server via the client's data port and then sends the server an acknowledgment that it has established the connection.

FTP and web browsers

Most recent web browsers and file managers can connect to FTP servers. This allows manipulation of remote files over FTP through an interface similar to that used for local files. This is done via an FTP URL, which takes the form ftp:// (e.g., ftp://ftp.gimp.org/). A password can optionally be given in the URL, e.g.: ftp://:@. Most web-browsers require the use of passive mode FTP, which not all FTP servers are capable of handling.

(Back to Top)

HDLC

High-Level Data Link Control (HDLC) is a bit-oriented synchronous data link Layer 2 protocol developed by the International Organization for Standarization (ISO).

HDLC was imported by the ITU into the X.25 protocol stack. It was modified by IBM to become the SDLC protocol that became the layer 2 protocol for IBM's System Network Architecture (SNA). HDLC is now the basis for synchronous PPP (point to point protocol) used by many servers to connect to a wide area network, most commonly the Internet.

The frame begins and ends with the bit pattern 01111110 or hex 7E, called the frame delimiter.

Then there is an address and a control field followed by a data field that may be 0 to 5000 octets long. Then a frame sequence check (FSC) is added and then the frame delimiter is added to the end. The frame delimiter continues to be sent until another frame is sent.

(Back to Top)

HTTP

HTTP (for HyperText Transfer Protocol) is the primary method used to convey information on the World Wide Web. The original purpose was to provide a way to publish and receive HTML pages.

Development of HTTP was co-ordinated by the World Wide Web Consortium and working groups of the Internet Engineering Task Force, culminating in the publication of a series of RFCs, most notably RFC 2616, which defines HTTP/1.1, the version of HTTP in common use today.

HTTP is a request/response protocol between clients and servers. An HTTP client, such as a web browser, typically initiates a request by establishing a TCP/IP connection to a particular port on a remote host (port :80 by default). An HTTP server listening on that port waits for the client to send a request string, such as "GET / HTTP/1.1" (which would request the default page of that web server), optionally followed by an email-like MIME message which has a number of informational header strings that describe aspects of the request, followed by an optional body of arbitrary data. Upon receiving the request string (and message, if any), the server sends back a response string, such as "200 OK", and a message of its own, the body of which is perhaps the requested file, an error message, or some other information.

HTTP differs from other TCP-based protocols such as FTP, in that connections are usually terminated once a particular request (or related series of requests) has been completed. This design makes HTTP ideal for the World Wide Web, where pages regularly link to pages on other servers. It can occasionally pose problems for Web designers, as the lack of a persistent connection necessitates alternative methods of maintaining users' "state". Many of these methods involve the use of "cookies".

There is a secure version of HTTP called HTTPS that can use any given encryption method as long as it is understood by both sides.

The locations of HTTP (and HTTPS) pages are given as Uniform Resource Locators or URLs. This address location syntax was created for HTML linking.

Below is a sample conversation between an HTTP client and an HTTP server running on www.google.com, port 80.

Client request:

GET / HTTP/1.1
Host: www.google.com

(followed by a new line, in the form of a carriage return followed by a line feed.)

Server response:

HTTP/1.1 200 OK
Content-Length: 3059
Server: GWS/2.0
Date: Sat, 11 Jan 2003 02:44:04 GMT
Content-Type: text/html
Cache-control: private
Set-Cookie: PREF=ID=73d4aef52e57bae9:TM=1042253044:LM=1042253044:S=SMCc_HRPCQiqy
X9j; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com
Connection: keep-alive

In HTTP 1.0, a client sends a request to the server, the server sends a response back to the client. After this, the connection will be released. On the other hand, HTTP 1.1 supports persistent connections. This enables the client to send a request and get a response, and then send additional requests and get additional responses immediately. The TCP connection is not released for the multiple additional requests, so the relative overhead due to TCP is much less per request. It is also possible to send more than one (usually two) request before getting responses from previous requests. This technique is known as "pipelining".

(Back to Top)

ICMP

The Internet Control Message Protocol (ICMP) is part of the Internet protocol suite and defined in RFC 792. ICMP messages are typically generated in response to errors in IP datagrams (as specified in RFC1122 (http://www.ietf.org/rfc/rfc1122.txt)) or for diagnostic or routing purposes.

The version of ICMP for Internet Protocol version 4 is also known as ICMPv4, as it is part of IPv4. IPv6 has an equivalent protocol.

ICMP messages are constructed at the IP layer, usually from a normal IP datagram which has generated an ICMP response. IP encapsulates the appropriate ICMP message with a new IP header (to get the ICMP message back to the original sending host), and transmits the resulting datagram in the usual manner.

For example, every machine (such as intermediate routers) that forwards an IP datagram has to decrement the IP Time-to-live (TTL) field of the IP header by one; if the TTL reaches 0, an ICMP "Time to live exceeded in transit" message is sent to the source of the datagram.

Each ICMP message is encapsulated directly within a single IP datagram, and thus, like UDP, ICMP does not guarantee delivery.

Although ICMP messages are contained within standard IP datagrams, ICMP messages are usually processed as a special case distinguished from normal IP processing, rather than processed as a normal sub-protocol of IP. In many cases, it is necessary to inspect the contents of the ICMP message, and deliver the appropriate error message to the application which generated the original IP packet, the one which prompted the sending of the ICMP message.

Many commonly used network utilities are based on ICMP messages. The traceroute command is implemented by transmitting UDP datagrams with specially set IP TTL header fields, and looking for ICMP "Time to live exceeded in transit" (above) and "Destination unreachable" messages generated in response. The related ping utility (well known on Unix) is implemented using the ICMP "Echo" and "Echo reply" messages.

List of permitted control messages:
0 - Echo Reply
1 - Reserved
2 - Reserved
3 - Destination Unreachable
4 - Source Quench
5 - Redirect Message
6 - Alternate Host Address
7 - Reserved
8 - Echo Request
9 - Router Advertisement
10 - Router Solicitation
11 - Time Exceeded
12 - Parameter Problem
13 - Timestamp
14 - Timestamp Reply
15 - Information Request
16 - Information Reply
17 - Address Mask Request
18 - Address Mask Reply
19 - Reserved for security
20-29 - Reserved for robustness experiment
30 - Traceroute
31 - Datagram Conversion Error
32 - Mobile Host Redirect
33 - IPv6 Where-Are-You
34 - IPv6 Here-I-Am
35 - Mobile Registration Request
36 - Mobile Registration Reply
37 - Domain Name Request
38 - Domain Name Reply
39 - SKIP Algorithm Discovery Protocol
40 - Photuris, Security failures
41-255 - Reserved
(Please complete this list!)
(Source: IANA ICMP Parameters (http://www.iana.org/assignments/icmp-parameters))

(Back to Top)

IGMP

The Internet Group Management Protocol is a communications protocol used to manage the membership of Internet Protocol multicast groups. IGMP is used by IP hosts and adjacent multicast routers to establish multicast group memberships. It is an integral part of the IP multicast specification, like ICMP for unicast connections.

For more information on IGMP see IGMP on Wikipedia.

(Back to Top)

IP

The Internet Protocol (IP) is a data-oriented protocol used by source and destination hosts for communicating data across a packet-switched internetwork.

Data in an IP internetwork are sent in blocks referred to as packets or datagrams (the terms are basically synonymous in IP). In particular, in IP no setup is needed before a host tries to send packets to a host it has previously not communicated with.

The Internet Protocol provides an unreliable datagram service (also called best effort); i.e. it makes almost no guarantees about the packet. The packet may arrive damaged, it may be out of order (compared to other packets sent between the same hosts), it may be duplicated, or it may be dropped entirely. If the application needs reliability, this is added by the Transport layer.

Packet switches, or internetwork routers, forward IP datagrams across interconnected layer 2 networks. The lack of any delivery guarantees means that the design of packet switches is made much simpler. (Note that if the network does drop, reorder or otherwise damage a lot of packets, the performance seen by the user will be poor, so most network elements do try hard to not do these things - hence the best effort term. However, an occasional error will produce no noticeable effect.)

IP is the common element found in today's public Internet. The current and most popular network layer protocol in use today is IPv4; this version of the protocol is assigned version 4. IPv6 is the proposed successor to IPv4; the Internet is slowly running out of addresses, and IPv6 has 128-bit source and destination addresses, providing more addresses than IPv4's 32 bits. Versions 0 through 3 were either reserved or unused. Version 5 was used for an experimental stream protocol. Other version numbers have been assigned, usually for experimental protocols, but have not been widely used.

IP addressing and routing

Perhaps the most complex aspects of IP are addressing and routing. Addressing refers to how end hosts are assigned IP addresses and how subnetworks of IP host addresses are divided and grouped together. IP routing is performed by all hosts, but most importantly by internetwork routers, which typically use either interior gateway protocols (IGPs) or external gateway protocols (EGPs) to help make IP datagram forwarding decisions across IP connected networks.

IPv4

IPv4 is version 4 of the Internet Protocol (IP). It was the first version of the Internet Protocol to be widely deployed, and forms the basis for most of the current Internet (as of 2004).

It is described in IETF RFC 791, which was first published in September, 1981.

IPv4 uses 32-bit addresses, limiting it to 4,294,967,296 unique addresses, many of which are reserved for special purposes such as local networks or multicast addresses, reducing the number of addresses that can be allocated as public Internet addresses.

As the number of addresses available is consumed, an IPv4 address shortage appears to be inevitable in the long run.

This limitation has helped stimulate the push towards IPv6, which is currently in the early stages of deployment, and is hoped will eventually replace IPv4.

IPv4 addresses are usually written in Dot-decimal notation. Here's an example: 207.142.131.235. But it's also possible to write in the following formats:

Dotted Decimal (normal)	207.142.131.235
Dotted Hexadecimal	0xCF.0x8E.0x83.0xEB
Dotted Octal	0317.0216.0203.0353
Decimal	3482223595
Hexadecimal	0xCF8E83EB

The above ip addresses should work in most browsers and at the time of writing point to wikipedia.org.

IPv4 header format

+	0 - 3				4 - 7				8 - 15								16 - 18			19 - 31
0	Version				Header length				Type of Service (now DiffServ and ECN)								Total Length
32	Identification																Flags			Fragment Offset
64	Time to Live								Protocol								Header Checksum
96	Source Address
128	Destination Address
160	Options
192	Data

The first header field in an IPv4 packet is the 4-bit version field.

The second field is a 4-bit Internet Header Length (IHL) telling the number of 32-bit words in the IPv4 header. Since an IPv4 header may contain a variable number of options, this field essentially specifies the offset to the data portion of an IPv4 datagram. A minimum IPv4 header is 20 bytes long, so the minimum value in decimal in the IHL field would be 5.

In RFC 791, the following 8 bits were allocated to a Type of Service (ToS) field - now DiffServ and ECN. The original intention was for a sending host to specify a preference for how the datagram would be handled as it made its way through an internetwork. For instance, one host could set its IPv4 datagrams' ToS field value to prefer low delay, while another might prefer high reliability. In practice, the ToS field has not been widely implemented. However, a great deal of experimental, research and deployment work has focused on how to make use of these eight bits. These bits have been redefined and most recently through DiffServ working group in the IETF and the Explicit Congestion Notification codepoints (see RFC 3168).

The next 16-bit IPv4 field defines the entire datagram size, including header and data, in 8-bit bytes. The minimum-length datagram is 20 bytes and the maximum is 65535. The maximum size datagram which any host is required to be able to handle is 576 bytes, but most modern hosts handle much larger packets. Sometimes subnetworks impose further restrictions on the size, in which case datagrams must be fragmented. Fragmentation is handled in either the host or packet switch in IPv4 (see below).

The next 16-bit field is an identification field. This field is primarily used for uniquely identifying fragments of an original IP datagram. Some experimental work has suggested using the ID field for other purposes, such as for adding packet tracing information to datagrams in order to help trace back datagrams with spoofed source addresses.

A 3-bit field follows and is used to control or identify fragments. They are (in order, from high order to low order:

• Reserved, must be zero
• Don't Fragment (if set)
• More Fragments.

The fragment offset field is 13-bits long, and allows a receiver to determine the place of a particular fragment in the original IP datagram, measured in units of 8-byte blocks.

An 8-bit time to live (TTL) field helps prevent datagrams from persisting (e.g. going in circles) on an internetwork. Historically the TTL field limited a datagram's lifetime in seconds, but has come to be a hop count field. Each packet switch (or router) that a datagram crosses decrements the TTL field by one. When the TTL field hits zero, the packet is no longer forwarded by a packet switch and is discarded.

An 8-bit Protocol field follows. This field defines the next protocol used in the data portion of the IP datagram. The Internet Assigned Numbers Authority maintains a list of Protocol numbers. Common protocols and their decimal values include ICMP (1), TCP (6) and UDP (17).

The following field is a 16-bit checksum field for the IPv4 datagram header. Some values in a IPv4 datagram header may change at each packet switch hop, so the checksum must be adjusted on its way through an internetwork.

The checksum is followed by a 32-bit source address and 32-bit destination address respectively.

Additional header fields (called options) may follow the destination address field, but these are not often used. Note that the value in the IHL field must include enough extra 32-bit words to hold all the options (plus any padding needed to ensure that the header contains an integral number of 32-bit words). The list of options may be terminated with an EOL (End of Options List) option; this is only necessary if the end of the options would not otherwise coincide with the end of the header.

The use of the LSSR and SSRR (Loose and Strict Source and Record Route) options is discouraged because they create security concerns; packets with them are therefore usually blocked by routers.

Fragmentation and reassembly

IPv4 supports the use of network elements (e.g. point-point links) which support small packet sizes. Rather than mandate link-local fragmentation and reassembly, which would require the router at the far end of the link to collect the separate pieces and reassemble the packet (a complicated process, especially when pieces may be lost due to errors on the link), a router which discovers that a packet which it is processing is too big to fit on the next link is allowed to break it into fragments (separate IPv4 packets which each carry part of the data in the original IPv4 packet), using a standardized procedure which allows the destination host to reassemble the packet from the fragments, after they are separately received there.

When a large IPv4 packet is split up into smaller fragments (which is usually, but not always, done at a router in the middle of the path from the source to the destination), the fragments are all normal IPv4 packets; i.e. they have a full IPv4 header. The original packet's data portion is split into segments which are small enough (when appended to the requisite IPv4 header) to fit into the next link; one segment of the original data is placed in each fragment. Almost all the header fields have the same value in them as the corresponding field in the original packet. In particular, all the fragments will have the same "identification" field value. The differences are:

• The "total length" field will be smaller; it will be set to the size of each fragment
• The "more fragments" single bit flag will be one in all but the last fragment
• The "fragment offset" field will be non-zero in all but the first fragment

Note that at the destination, in any incoming packet, if either:

• The "more fragments" single bit flag is one, or
• The "fragment offset" field is non-zero

that packet is a fragment.

To reassemble the fragments back into the original packet at the destination, the host looks for incoming packets with the same value in the identification field: they all belong to the same original packet. The offset and total length fields tell it where each piece goes, and how much of the original packet it fills in. It can work out the total size of the original packet because in the packet with the "more fragments" flag clear, the value of the size field in that packet (less the length of the IPv4 header, of course), plus the value in the offset field (multiplied by the 8-byte fragmentation block size), gives the total length of the original packet.

Note that a router can repeat the fragmentation process even if it only has a single fragment (e.g. at a later router along the path) - it takes the fragment, splits it up in the same manner as above to create two (or more) new fragments, and adjusts the offset and total length fields as appropriate. The only complication is that if the "more fragments" flag was zero, it needs to set it to one in all but the last new fragment. (It is relatively simple to set up the process so that the router doesn't need to know whether the packet it is fragmenting is itself a fragment, or a complete packet.)

Note also that if a packet is fragmented, and some of the fragments were lost while sending them from the source to the destination, then if it is retransmitted with the same identification number, and this second copy is also fragmented (again, potentially with the loss of some fragments), then fragments from the second copy can be used to fill in the "blank spots" from the first one.

IPv6

IPv6 is version 6 of the Internet Protocol; it was initially called IP Next Generation (IPng) when it was picked as the winner in the IETF's IPng selection process. IPv6 is intended to replace the previous standard, IPv4, which only supports up to about 4 billion (4 × 109) addresses, whereas IPv6 supports up to about 3.4 × 1038 (3.4 dodecillion) addresses. This is the equivalent of 4.3 × 1020 addresses per inch² (6.7 × 1017 addresses/mm²) of the Earth's surface. It is expected that IPv4 will be supported until at least 2025, to allow time for bugs and system errors to be corrected.

The compelling reason behind the formation of IPv6 was lack of address space, especially in the heavily populated countries of Asia such as India and China. See the article IPv4 address exhaustion for more on this topic. However since the introduction of NAT this is not such a big problem any more. Currently the big drive for IPv6 is new uses, such as mobility, quality of service, privacy extension and so on.

IPv6 is the second version of the Internet Protocol to be formally adopted for general use. (There was also an IPv5, but it was not a successor to IPv4; rather, it was an experimental flow-oriented streaming protocol, intended to support voice, video, and audio.)

The plan is for IPv6 to form the basis for future expansion of the Internet. Although IPv6 was adopted by the IETF as the successor to IPv4 over ten years ago (in 1994), worldwide IPv6 deployment as a publicly-accessible internet is still only a few percent [1] (http://bgp.potaroo.net/index-v6.html) of the size of the worldwide IPv4 Internet [2] (http://bgp.potaroo.net/index-bgp.html).

IPv6 addressing

The most dramatic change from IPv4 to IPv6 is the length of network addresses. IPv6 addresses, as defined by RFC 2373 and RFC 2374, are 128 bits long; this corresponds to 32 hexadecimal digits, which are normally used when writing IPv6 addresses, as described in the following section.

The number of possible addresses in IPv6 is 2128 ˜ 3.4 x 1038. The number of IPv6 addresses can also be thought of as 1632 as each of the 32 hexadecimal digits can take 16 values (see combinatorics).

In many situations, IPv6 addresses are composed of two logical parts: a 64-bit network prefix, and a 64-bit host-addressing part, which is often automatically generated from the interface MAC address. The host part is called a EUI-64 (or 64-bit Extended Unique Identifier).

It is often argued that 128-bit addresses is overkill, and that the Internet will never need that much. It should be noted that the rationale for the 128-bit address space is not primarily to make sure that addresses never run out, but rather to ensure that routing can be handled smoothly by keeping the address space unfragmented, rather than as the current situation is with IPv4, where a great number of discrete netblocks can be, and often are, assigned to one organization.

Notation for IPv6 addresses

IPv6 addresses are 128 bits long but are normally written as eight groups of 4 hexadecimal digits each. For example,

2001:0db8:85a3:08d3:1319:8a2e:0370:7344

is a valid IPv6 address.

If a 4 digit group is 0000, it may be omitted. For example,

2001:0db8:85a3:0000:1319:8a2e:0370:7344  

is the same IPv6 address as

2001:0db8:85a3::1319:8a2e:0370:7344

Following this rule, if more than two consecutive colons result from this omission, they may be reduced to two colons, as long as there is only one group of two or more consecutive colons. Thus

2001:0DB8:0000:0000:0000:0000:1428:57ab 
2001:0DB8:0000:0000:0000::1428:57ab 
2001:0DB8:0:0:0:0:1428:57ab
2001:0DB8:0::0:1428:57ab
2001:0DB8::1428:57ab  

are all valid and mean the same thing, but

2001::25de::cade 

is invalid. (As it is ambiguous how many 0000 groups should be on each side.)

Also leading zeros in all groups can be omitted, thus

2001:0DB8:02de::0e13

is the same thing as

2001:DB8:2de::e13

If the address is an IPv4 address in disguise, the last 32 bits may be written in decimal; thus

::ffff:192.168.89.9 is the same as ::ffff:c0a8:5909, but not the same as ::192.168.89.9 or ::c0a8:5909.

The ::ffff:1.2.3.4 format is called an IPv4-mapped address, and is deprecated. The ::1.2.3.4 format is an IPv4-compatible address.

IPv4 addresses are easily convertible to IPv6 format. For instance, if the decimal IPv4 address was 135.75.43.52 (in hexadecimal, 0x874B2B34), it could be converted to 0000:0000:0000:0000:0000:0000:874B:2B34 or ::874B:2B34. Then again, one could use the hybrid notation (IPv4-compatible address, in which case the address would be ::135.75.43.52.

Special addresses

There are a number of addresses with special meaning in IPv6. This is a brief table of these, in CIDR notation – see the linked page for more information.

• ::/128 – the address with all zeroes is used to specify any address, and is only to be used in software.
• ::1/128 – the loopback address is a host-local address which echoes back all the packets sent to it (corresponding to 127.0.0.1 in IPv4).
• ::/96 – The IPv4-compatible address is used as a transition mechanism in dual IPv4/IPv6 networks.
• ::ffff:0:0/96 – The IPv4-mapped address is used as a transition mechanism in dual-stack hosts.
• fe80::/10 – The link-local prefix specifies that the address only is valid in the local physical link.
• fe90::/10 – The site-local prefix specifies that the address only is valid inside the local organization.
• ff00::/8 – The multicast prefix is used for multicast addresses.

IPv6 packet

The IPv6 packet is composed of two main parts; the header, and the payload.

The header is in the first 40 bytes of the packet and contains both source and destination addresses (128 bits each), as well as the version (4 bit IP version), traffic class (8 bit, Packet Priority), flow label (20 bits, QoS management), payload length (16 bit), next header (for backwards compatibility), and hop limit (8 bits, time to live). Next comes the payload, which must be at least 1280 bytes long, or 1500 bytes long in an environment with a flexible MTU size. The payload can go up to 65,535 in standard mode, or can be set to a "jumbo payload" option.

There have been two very slightly different versions of IPv6; the (now-obsolete) initial version, described in RFC 1883, differs from the current proposed standard version, described in RFC 2460, in one field. This is the traffic class, which has its size increased from 4 to 8 bits. All other differences are minor.

Fragmentation is handled in the host only in IPv6. In IPv6, options also move out of the standard header and are specified by a Next Protocol field, similar in function to IPv4's Protocol field. As a handwaving example, whereas in IPv4 one would add a Strict Source and Record Routing (SSRR) option to the IPv4 header itself in order to enforce a certain route for the packet, in IPv6 one would make the Next Header field indicate that a Routing header comes next. The Routing header would then specify the additional routing information for the packet, and then indicate that the, for example, TCP header comes next. This is analogous to the handling of AH and ESP in IPSec for IPv4 (which applies to IPv6 as well, of course).

IPv6 and the Domain Name System

IPv6 addresses are represented in the Domain Name System by AAAA records (so-called quad-A records) for forward lookups (by analogy with A records for IPv4); reverse lookups take place under ip6.arpa (previously ip6.int), where address space is delegated on nibble boundaries. This scheme is defined in RFC 3596.

The AAAA scheme was one of two proposals at the time the IPv6 architecture was being designed. The other proposal would have had A6 records for the forward lookup and a number of other innovations such as bit-string labels and DNAME records. It is defined in the experimental RFC 2874 and its references.

While the AAAA scheme is a simple generalisation of the IPv4 DNS, the A6 scheme was an overhaul of the DNS to be more general, and hence more complex:

• A6 records allowed a single IPv6 address to be broken across several records, perhaps held in different zones; this allowed in principle for rapid renumbering of networks, for example.
• Address delegation by use of NS records was largely replaced with DNAME records (analogous to the existing CNAME but renaming an entire tree). This permitted related forward and reverse components to be managed together.
• A new data type called the bit label was introduced to domain names, primarily for reverse lookups.

The AAAA scheme was effectively standardised on in August 2002 by RFC 3363 (with further discussion of the pros and cons of both schemes in RFC 3364).

IPv6 deployment

On 20 July 2004 ICANN announced[3] (http://icann.org/announcements/announcement-20jul04.htm) that the root DNS servers for the Internet had been modified to support both IPv6 and IPv4.

Disadvantages:

• the need for roll-out of pervasive support for IPv6 throughout the Internet and its connected devices
• to be reachable from the IPv4 universe during the transition phase, you still need an IPv4-address or some kind of NAT (=shared IP-address) in the gateway routers (IPv6<-->IPv4) which adds complexity there and means the large address space promised by the specification can't be immediately used effectively.
• remaining architectural problems, such as the lack of agreement for proper support for IPv6 multihoming.

To do:

• 6bone
• ICMPv6
• IPv6 multihoming

Transition mechanisms

Until IPv6 native connectivity becomes widely available and supported by the routing infrastructure, it is necessary to use transition mechanisms to tunnel IPv6 through IPv4 networks. That can be achieved with:

• configured static IPv6-in-IP tunnels between to dual-stack nodes, or
• 6to4, which is an automatic and asymmetric tunnel mechanism.

These tunnels work by encapsulating IPv6 packets into IPv4 packets with IP next-layer protocol number 41, hence the name proto-41. Similarly, ISATAP allows the transmission of IPv6 packets through an internal IPv4-only networking infrastructure. It also uses protocol number 41.

When IPv6 connectivity is desired from behind a NAT device, many of which do not forward proto-41 packets properly, one may use the Teredo protocol which encapsulates IPv6 over UDP over IPv4. It is also possible to use IPv6-to-IPv4 and IPv6-to-IPv6 proxies, though that is typically application-layer specific (eg. HTTP).

(Back to Top)

IPX

Internetwork Packet Exchange (IPX) is at the Network layer of the OSI model and is part of the IPX/SPX protocol stack.

Novell built on top of DEC and Xerox's XNS protocol, and is largely responsible for the use of IPX as a popular computer internetworking protocol as a result of its large marketshare of Network Operating System software (Novell NetWare) in the late 1980s through mid 1990s.

IPX usage is in general decline as the boom of the Internet has made TCP/IP nearly universal. Computers and networks can run multiple network protocols, so almost all IPX sites will be running TCP/IP as well to allow for Internet connectivity. It is also now possible to run Novell products without IPX, as they have supported both IPX and TCP/IP for a few versions now.

(Back to Top)

ISDN

Integrated Services Digital Network (ISDN) is a type of circuit switched telephone network system, designed to allow digital (as opposed to analog) transmission of voice and data over ordinary telephone copper wires, resulting in better quality and higher speeds, than is capable with analog systems. More broadly, ISDN is a set of protocols for establishing and breaking circuit switched connections, and for advanced call features for the end user.

Consumer and industry perspectives

There are two points of view into the ISDN world. The most common viewpoint is that of the end user who wants to get a digital connection into the telephone/data network from home, whose perforcemance would be a little better than a modem connection. They typical end-user's connection to the Internet is related to this point of view, and talk about the merits of various ISDN modems, carriers' offerings and tarriffing (features, pricing) are from this perspective. Much of the following discussion is from this point of view, but it should be noted that as a data connection service, ISDN has been mostly superseded by DSL..

There is however a second viewpoint, that of the telephone industry, where ISDN is not a dead issue. A telephone network can be thought of as a collection of wires strung between switching systems. The common electrical specification for the signals on these wires is T1 or E1. On a normal T1, the signalling is done with A&B bits to indicate on or off hook conditions and MF and DTMF tones to encode the destination number. ISDN is much better than this as messages can be sent much more quickly than by trying to encode numbers as long (100 ms per digit) tone sequences. This translated to much faster call setup times which is greatly desired by carriers who have to pay for line time and also by callers who hate to wait while their call hops from switch to switch.

It is also used as a smart network technology intended to add new services to the public switched telephone network (the PSTN) by giving users direct access to end-to-end circuit-switched digital services.

National services

ISDN has never gained popularity as a telephone network in the United States and today remains a niche product. In Japan, it became popular to some extent from around 1999 to 2001, but now that ADSL has been introduced, the number of subscribers is in decline.

In Japan, NTT, a dominant telephone company, provides an ISDN service with the names INS64 and INS1500, which are much less recognized than ISDN.

In the UK, British Telecom (BT) provides Home Highway and Business Highway, which are BRI ISDN services which offer connection from analog devices (such as normal phones) as well as ISDN devices (such as PCs equipped with terminal adapters). Home Highway has been bought by many home users, usually for Internet connection. Although not as fast as ADSL, it was available before ADSL, and in places where ADSL does not reach. BT also offers PRI ISDN.

In France, France Telecom offers ISDN services under their product name Numeris (2 B+D) of which a profesional Duo and home Itoo version is available. ISDN is generally known as RNIS in France and has widespread availability. The introduction of ADSL is reducing ISDN use for data transfer and internet access, although it is still common in more rural and outlying areas.

In Germany, ISDN is very popular with an installed base of 25 mio. channels (29% of all subscriber lines in Germany as of 2003 and 20% of all ISDN channels worldwide). Due to the success of ISDN, the number of installed analog lines is decreasing. Deutsche Telekom (DTAG) offers both BRI and PRI. Competing phone companies often offer ISDN only and no analog lines.

ISDN Configurations

In ISDN, there are two types of channels, B and D:

• B channels are used for data--this refers both to voice and data information, and
• D channels are intended for signalling and control (but can also be used for data).
• B stands for Bearer and D stands for Delta.

There are two kinds of access to ISDN:

• Basic Rate Interface (BRI) - consisting of two B channels, each with bandwidth of 64 kbit/s, and one D channel with a bandwitdth of 16 kbit/s. Together these three channels cna be designated as 2B+D, and,
• Primary Rate Interface (PRI) - containing a greater number of B channels and a D channel with a bandwidth of 64 kbit/s, The number of B channels varies based on the country:
• North America and Japan: 23B+1D, aggregate bit rate of 1.544 Mbit/s (T1)
• Europe, Australia: 30B+D, aggregate bit rate of 2.048 Mbit/s (E1)

Call data is transmitted over the data (B) channels, with the signalling (D) channels used for call setup and management. Once a call is set up, there is a simple 64 kbit/s synchronous bidirectional data channel between the end parties, lasting until the call is terminated. There can be as many calls as there are data channels, to the same or different end-points. Bearer channels may also be multiplexed into what may be considered single, higher-bandwidth channels via a process called B channel bonding.

The D channel can also be used for sending and receiving X.25 data packets, and connection to X.25 packet network. In practice, this was never widely implemented.

Reference points

A set of reference points are defined in the ISDN standard to refer to certain points between the telco and the end user ISDN equipment.

• R - defines the point between a non-ISDN device and a terminal adapter (TA) which provides translation to and from such a device
• S - defines the point between the ISDN equipment (or TA) and a Network Termination Type 2 (NT-2) device
• T - defines the point between the NT-2 and NT-1 devices1
• U - defines the point between the NT-1 and the telco switch2

1 Most NT-1 devices can perform the functions of the NT-2 as well, and so the S and T reference points are generally collapsed into the S/T reference point.

2 Inside North America, the NT-1 device is considered customer premises equipment and must be maintained by the customer, thus, the U interface is provided to the customer. In other locations, the NT-1 device is maintained by the telco, and the S/T interface is provided to the customer.

Types of communications handled

Amongst the kinds of data that can be moved over the 64 kbit/s channels are pulse-code modulated voice calls, providing access to the traditional voice PSTN. This information can be passed between the network and the user end-point at call set-up time.

In North America, ISDN is nowadays mostly used as an alternative to analog connection, most commonly for Internet access. Some of the services envisaged as being delivered over ISDN are now delivered over the Internet instead. In Europe, and in Germany in particular, ISDN has been successfully marketed as a phone with features, as opposed to a POTS (Plain Old Telephone Service) with little to no features. In North America, POTS phones have all of the features available to them marketed (Three Way Call, Call Forwarding, Caller ID, etc), but in Germany, the ISDN lines were marketed as being the phones for people who wanted advanced services.

Where an analog connection would require a modem an ISDN connection requires a terminal adapter (TA).

A sample ISDN call

The following is an example of a Primary Rate (PRI) ISDN call showing the Q.921/LAPD and the Q.931/Network message intermixed (i.e. exactly what was exchanged on the D-channel). The call is originating from the switch where the trace was taken and goes out to some other switch, possibly an end-office LEC, who terminates the call.

The first line format is . If the message is an ISDN level message, then a decoding of the message is attempted showing the various Information Elements that make up the message. All ISDN messages are tagged with an ID number relative to the switch that started the call (local/remote). Following this optional decoding is a dump of the bytes of the message in ... ... format.

The RR messages at the beginning prior to the call are the keep alive messages. Then you will see a SETUP message that starts the call. Each message is acknowledged by the other side with a RR.

10:49:47.33  21/1/24  R  RR
0000  02 01 01 a5                                          ....

10:49:47.34  21/1/24  T  RR
0000  02 01 01 b9                                          ....

10:50:17.57  21/1/24  R  RR
0000  02 01 01 a5                                          ....

10:50:17.58  21/1/24  T  RR
0000  02 01 01 b9                                          ....

10:50:24.37  21/1/24  T  SETUP
    Call Reference       : 000062-local
    Bearer Capability    : CCITT, Speech, Circuit mode, 64 kbit/s 
    Channel ID           : Implicit Interface ID implies current span, 
	                       21/1/5, Exclusive
    Calling Party Number : 8018023000 National number  User-provided, 
	                       not screened  Presentation allowed
    Called Party Number  : 3739120 Type: SUBSCRB
0000  00 01 a4 b8  08 02 00 3e  05 04 03 80  90 a2 18 03   .......>........
0010  a9 83 85 6c  0c 21 80 38  30 31 38 30  32 33 30 30   ...l.!.801802300
0020  30 70 08 c1  33 37 33 39  31 32 30                   0p..3739120

10:50:24.37  21/1/24  R  RR
0000  00 01 01 a6                                  ....

10:50:24.77  21/1/24  R  CALL PROCEEDING
    Call Reference       : 000062-local
    Channel ID           : Implicit Interface ID implies current 
	                       span, 21/1/5, Exclusive
0000  02 01 b8 a6  08 02 80 3e  02 18 03 a9  83 85 ...>......

10:50:24.77  21/1/24  T  RR
0000  02 01 01 ba                                  ....

10:50:25.02  21/1/24  R  ALERTING
    Call Reference       : 000062-local
    Progress Indicator   : CCITT, Public network serving local user,
	       In-band information or an appropriate 
           pattern is now available
0000  02 01 ba a6  08 02 80 3e  01 1e 02 82  88 ..>.....

10:50:25.02  21/1/24  T  RR
0000  02 01 01 bc                            ....

10:50:28.43  21/1/24  R  CONNECT
    Call Reference       : 000062-local
0000  02 01 bc a6  08 02 80 3e  07           .......>.

10:50:28.43  21/1/24  T  RR
0000  02 01 01 be                            ....

10:50:28.43  21/1/24  T  CONNECT_ACK
    Call Reference       : 000062-local
0000  00 01 a6 be  08 02 00 3e  0f           .......>.

10:50:28.44  21/1/24  R  RR
0000  00 01 01 a8                            ....

10:50:35.69  21/1/24  T  DISCONNECT
    Call Reference       : 000062-local
    Cause                : 16, Normal call clearing.
0000  00 01 a8 be  08 02 00 3e  45 08 02 8a  90....>E....

10:50:35.70  21/1/24  R  RR
0000  00 01 01 aa                            ....

10:50:36.98  21/1/24  R  RELEASE
    Call Reference       : 000062-local
0000  02 01 be aa  08 02 80 3e  4d           .......>M

10:50:36.98  21/1/24  T  RR
0000  02 01 01 c0                            ....

10:50:36.99  21/1/24  T  RELEASE COMPLETE
    Call Reference       : 000062-local
0000  00 01 aa c0  08 02 00 3e  5a           .......>Z

10:50:36.00  21/1/24  R  RR
0000  00 01 01 ac                            ....

10:51:06.10  21/1/24  R  RR
0000  02 01 01 ad                               ....

10:51:06.10  21/1/24  T  RR
0000  02 01 01 c1                               ....

10:51:36.37  21/1/24  R  RR
0000  02 01 01 ad                               ....

10:51:36.37  21/1/24  T  RR
0000  02 01 01 c1                               ....

(Back to Top)

NetBIOS

NetBIOS is an acronym for Network Basic Input/Output System. It generally refers to a programming API for local network communication.

NetBIOS History

NetBIOS is a networking protocol, co-developed by IBM and Sytec for the so-dubbed PC-Network in the early 1980s. Although only published through a technical reference book from IBM, the protocol's API became a de-facto standard.

As PC-Network hardware is no longer used, having been replaced by TokenRing and Ethernet networks, the NetBIOS protocol might not be needed anymore. However, because a lot of software was written for NetBIOS's API, it has since been adapted to work over various other protocols such as IPX/SPX and TCP/IP.

NetBIOS over TokenRing or Ethernet is now referred to as NetBEUI. It was still heavily used until the Microsoft Windows 98 operating system was released. NetBIOS over TCP/IP is referred to as NBT and has been standardized by RFCs 1001 and 1002. NBT intends to provide a NetBIOS-based PC-Network LAN emulation over a routed IP-based internetwork. It has been introduced with Microsoft Windows 2000 and is now the preferred NetBIOS transport.

NetBIOS Overview

NetBIOS, whatever the adaptation, provides three distinct services:

• Name service: name registration and resolution (name browsing service is part of SMB, an upper layer)
• Session service: reliable connection-oriented communication
• Datagram distribution service: unreliable connectionless communication

When NetBIOS was an OSI data link layer protocol, its functions were accessed via the 5Ch interrupt. Messages passed to these functions were formatted according to the NCB (Network Block Control) format.

NetBIOS and NetBEUI were intended for use on local networks only. Therefore, they have no support for routing and can only handle a maximum of 72 nodes or named devices. Usage of broadcast is intensive, especially for operations related to the name service.

NBT (NetBIOS over TCP/IP) uses one or more NBNS (NetBIOS Name Server(s)) to span name service over multiple subnets (while broadcast is limited to only one subnet). An NBNS is a sort of dynamic DNS. Microsoft's implementation of NBNS is called WINS. Moreover, in order to extend virtual NetBIOS networks across multiple IP subnetworks, the standard also introduced the use of one or more NBDD (NetBIOS Datagram Distribution) server(s). Unfortunately, Microsoft's implementation of NBDD never worked.

(Back to Top)

NSF

Network File System (NFS) is a protocol originally developed by Sun Microsystems in 1984 and defined in RFCs 1094, 1813, (3010) and 3530, as a file system which allows a computer to access files over a network as easily as if they were on its local disks.

Version 2 of the protocol originally operated entirely over UDP and was meant to keep the protocol stateless, with locking (for example) implemented outside of the core protocol.

Version 3 introduced support for using TCP as transport. Several vendors had already extended NFSv2 to support TCP as transport. Using TCP as transport made using NFS over a WAN more feasible (although not necessarily practical).

Version 4 includes performance improvements and introduces a stateful protocol.

NFS is strongly associated with UNIX systems, though it can be used on any platform such as Macintosh and Microsoft Windows operating systems. The server message block (SMB), a similar protocol, is the equivalent implementation of a network file system under Microsoft Windows.

(Back to Top)

RPC

A remote procedure call (RPC) is a protocol that allows a computer program running on one host to cause code to be executed on another host without the programmer needing to explicitly code for this. When the code in question is written using object-oriented principles, RPC is sometimes referred to as remote invocation or remote method invocation.

RPC is an easy and popular paradigm for implementing the client-server model of distributed computing. An RPC is initiated by the caller (client) sending a request message to a remote system (the server) to execute a certain procedure using arguments supplied. A result message is returned to the caller. There are many variations and subtleties in various implementations, resulting in a variety of different (incompatible) RPC protocols.

In order to allow servers to be accessed by differing clients, a number of standardized RPC systems have been created. Most of these use an interface description language (IDL) to allow various platforms to call the RPC. Examples of such systems include Sun RPC (sometimes called ONC RPC), the Distributed Computing Environment (DCE), Microsoft's DCOM (and ActiveX), which is based in part on DCE, and CORBA.

Recently a number of vendors have started using XML as the IDL, and HTTP as the network protocol. The advantage of this system, known as web services, is simplicity and standardization, the IDL is a text file that is widely understood, and HTTP is built into almost all modern operating systems. An example of such an RPC system is SOAP, developed in turn from XML-RPC.

(Back to Top)

OSPF

Open Shortest Path First (OSPF) is a link-state, hierarchical Interior Gateway Protocol (IGP) routing algorithm. The well known Dijkstra's algorithm is used to calculate the shortest path tree. It uses cost as its routing metric. A link state database is constructed of the network topology which is identical on all routers in the area.

An OSPF network can be broken up into smaller networks where a backbone always known as area zero can have any other arbitrarily numbered areas connecting directly to it. All areas must connect via area zero. If areas haven't direct physical connection to the backbone area, Virtual links must be used to connect them. It was often a general rule not to have more than fifty routers in one area. However, improvements in processor power can now allow over 400 sufficiently powerful routers to participate with no stability issues. Routers at the edge of two areas are known as 'Area Border Routers' or ABRs. Routes from other autonomous systems are inserted into OSPF via Autonomous System Boundary Routers (ASBRs).

Routers in the same broadcast domain or at each end of a point to point link form adjacencies when they have discovered each other . The routers elect a designated router (DR) and backup designated router (BDR) which act as hub to reduce traffic between routers. OSPF uses both unicast and multicast to send 'hello packets' and link state updates. Multicast addresses 224.0.0.5 and 224.0.0.6 are used. In contrast to RIP using UDP on port 520 or BGP using TCP on port 179, OSPF does not use a Layer 4 protocol but uses IP directly, using IP protocol 89.

OSPF can utilise authentication for forming adjacencies and exchanging routing information. OSPF was VLSM capable or classless from its inception. OSPF is capable of tagging external routes.

OSPF is the suggested successor to RIP in the Internet environment.

A new version of OSPF to support IP version 6 has been proposed, commonly called OSPFv3.

Multicast extensions to OSPF (MOSPF) was a largely unsuccessful (in terms of deployment success, not in technical terms) attempt to adapt OSPF for use as multicast routing protocol.

(Back to Top)

PPP

In computing, the Point-to-Point Protocol, or PPP, is commonly used to establish a direct connection between two nodes. Its primary use has been to connect computers using a phone line, though it is also occasionally used over broadband connections. Many ISPs use PPP when providing customers with dial-up access (e.g. to the Internet, where it has largely superseded an older protocol known as SLIP).

PPP is commonly used to act as a layer 2 (the "Data Link" layer of the OSI model) protocol for connection over synchronous and asynchronous circuits. PPP was designed to work with several network layer protocols, such as IP, IPX and AppleTalk, and as a replacement for the non-standard layer 2 protocol SLIP.

PPP was designed much later than the original HDLC specifications. As a result, the creators of PPP included many additional features that had not been seen in WAN data-link protocols up to that time. Enhanced error detection PPP uses FCS fields to determine if an individual frame has an error, however PPP monitors the frequency with which frames are received in error, and it can be configured to take down an interface if too many errors occur.

Looped link detection

LCP (Link Control Protocol, an integral part of PPP and defined in the same RFC) notices looped links using a feature involving magic numbers. When using PPP, the endpoint sends PPP LCP messages, these messages include a magic number which is different on each end point. If a line is looped, the end point receives an LCP message with its own magic number instead of getting a message with the other peer's magic number.

PPP provides hooks for automatically configuring the network interfaces at each end (setting an IP address, default gateway etc.) and for authentication.

PPP is described by IETF RFC 1661. Numerous documents on PPP have been published through the RFC process since July 1990, including various authentication, encryption and compression methods and the use of PPP in conjunction with other network protocols

RFC 1994 describes CHAP, the Challenge Handshake Authentication Protocol which is commonly used when establishing dialup connections with ISPs.

RFC 2516 describes PPPoE, a method for transmitting PPP over Ethernet which is sometimes used with DSL.

RFC 2364 describes PPPoA, a method for transmitting PPP over ATM Adaptation Layer 5 (AAL5) known as or PPPoATM for PPP over ATM.

(Back to Top)

RARP

Reverse address resolution protocol (RARP) is a protocol used to resolve an IP address from a given hardware address (such as an Ethernet address). The primary limitations were that each MAC had to be manually configured on a central server, and it was limited to only the IP address, leaving subnetting, gateways, and other information to be configured by hand. It was later obsoleted by BOOTP, which had a much greater feature set.

RARP is the dual of ARP. RARP is described in RFC 903

(Back to Top)

RIP

The Routing Information Protocol (RIP) is one of the most commonly used Interior Gateway Protocols on internal networks (and to a lesser extent, the internet), which helps routers dynamically adapt to changes of network connections by communicating information about which networks each router can reach and how far away those networks are. Although RIP is still actively used, it is generally considered to have been obsoleted by Link-state routing protocols such as OSPF EIGRP.

RIP was first developed in 1969 as part of ARPANET, and used the Bellman-Ford algorithm. RIP is a distance-vector routing protocol which employs hop count as a routing metric. The maximum number of hops allowed with RIP is 15. Each RIP router transmits full updates every 30 seconds by default, generating large amounts of network traffic in lower bandwidth networks. It runs above the network layer of the Internet protocol suite, using UDP port 520 to carry its data. A mechanism called split horizon with limited poison reverse is used to avoid routing loops. Routers of some brands also use a holddown mechanism known as heuristics, whose usefulness is arguable and is not a part of the standard protocol.

There are two versions of RIP, RIPv1 and RIPv2. RIPv1 uses classful routing. The routing updates do not carry subnet information, which means that a network's size is determined solely by the network class of its IP Address, and there is no way to split a network into smaller subnets, each routed along a different path.

Due to the original deficiencies in addressing, RIPv2 was developed in 1994 to use CIDR (Classless InterDomain Routing). However to maintain backwards compatibility the 15 hop count limit remained. Rudimentary Simple Text authentication was added to secure routing updates. RIP Routers may also implement MD5 authentication as specified in RFC 2082.

In many current networking environments RIP would not be the first choice of Routing as its convergence times and scalability are poor compared to OSPF or IS-IS, and the hop limit severely limits the size of network it can be used in. On the other hand, it is easier to configure.

RIPv1 is specified in RFC 1058. RIPv2 is specified in RFC 2453 or STD 56.

(Back to Top)

RPT

The Real-time Transport Protocol (or RTP) was developed by the Audio-Video Transport Working Group of the IETF and published in 1996 as RFC 1889 (http://www.ietf.org/rfc/rfc1889.txt).

RTP was also published by the ITU-T as H.225.0, but later removed once the IETF had a stable standards-track RFC published. It exists as an Internet Standard (STD 64) defined in RFC 3550 (http://www.ietf.org/rfc/rfc3550.txt) (which obsoletes RFC 1889) that specifies the protocol. RFC3551 (http://www.ietf.org/rfc/rfc3551.txt) (STD 65) (which obsoletes RFC 1890) defines a specific profile for Audio and Video Conferences with Minimal Control.

RTP defines a standardized packet format for delivering audio and video over the Internet. It was originally designed as a multicast protocol, but has since been applied in many unicast applications. It is frequently used in streaming media systems (in conjunction with RTSP) as well as videoconferencing and push to talk systems (in conjunction with H.323 or SIP), making it the technical foundation of the Voice over IP industry. It goes along with the RTP Control Protocol (RTCP) and it's built on top of User Datagram Protocol (in OSI model).

(Back to Top)

RTSP

RTSP is the Real Time Streaming Protocol developed by the IETF and published in 1998 as RFC 2326 (http://www.ietf.org/rfc/rfc2326.txt). RTSP is a protocol for use in streaming media systems which allows a client to remotely control a streaming media server, issuing VCR-like commands such as "play" and "pause", and allowing time-based access to files on a server.

RTSP servers typically use RTP as the protocol for the actual audio/video data.

(Back to Top)

SCP

Secure Copy or scp is a means of securely transferring computer files between a local and a remote host or between two remote hosts, using the Secure Shell (SSH) protocol.

The term scp can refer to one of two related things:

• A protocol -- basically identical to the BSD rcp protocol, but run over secure shell (SSH) rather than rsh. (compare sftp protocol)
• A command line program to perform secure copying. Whether the scp tool uses the scp protocol or the sftp protocol depends on the version and variant of the tool.

Scp is the secure analog of the rcp command. Unlike rcp, data is encrypted during transfer, to avoid potential packet sniffers extracting usable information from the data packets.

scp is a command line tool provided with SSH and OpenSSH. Alternative tools which also support scp are available.

In its basic form the syntax of scp is like the syntax of cp:

scp fileToMove user@host:folder/file

After entering the command the remote host will ask user's password and the copy process starts.

A more comprehensive tool/protocol for transferring files over SSH is sftp.

(Back to Top)

SCTP

SCTP, or Stream Control Transmission Protocol is a new transport layer protocol (2000) defined by the IETF. The protocol is defined in RFC 2960, and an introductory text is provided by RFC 3286.

As a transport protocol, SCTP is equivalent in a sense to TCP or UDP. Indeed it provides some similar services as TCP, ensuring reliable, in-sequence transport of messages. Whilst TCP is byte-oriented, SCTP deals with framed messages.

A major contribution of SCTP is multi-homing support, where one (or both) endpoints of a connection can consist of more than one IP address, enabling transparent fail-over between hosts or network cards.

SCTP was originally intended for the transport of telephony (SS7) protocols over IP, with the goal of duplicating some of the reliability attributes of the SS7 signaling network in IP. This IETF effort is known as SIGTRAN. In the meantime, other uses have been proposed for the protocol.

(Back to Top)

SMB

Server message block (SMB) is a network protocol mainly applied to share files, printers, serial ports, and miscellaneous communications between nodes on a network. It is mainly used by Microsoft Windows equipped computers.

SMB was originally invented by IBM but the most common version is modified heavily by Microsoft. Microsoft renamed SMB to CIFS (Common Internet FileSystem) in 1998 and added more features, including support for Symbolic link, Hard Link, and larger file sizes.

SMB works through a client-server approach, where a client makes specific requests and the file server responds accordingly.

The SMB servers make their file systems and other resources available to clients on the network. Client computers may have their own hard disks, which are not publicly shared, yet also want access to the shared file systems and printers on the server.

SMB was originally designed to run on top of one of the various NetBIOS implementations (usually NetBEUI or NBT), though it can also run on top of TCP/IP directly since Windows 2000.

Microsoft has added several features to its own SMB implementation that are not part of the original SMB protocol.

Samba is a free reimplementation of the SMB protocol and the Microsoft extensions to it.

(Back to Top)

SMTP

Simple Mail Transfer Protocol (SMTP) is the de facto standard for email transmission across the Internet.

SMTP is a relatively simple, text-based protocol, where one or more recipients of a message are specified (and in most cases verified to exist) and then the message text is transferred. It is quite easy to test a SMTP server using the telnet program. SMTP uses TCP port 25. To determine the SMTP server for a given domain name, the MX (Mail eXchange) DNS record is used.

SMTP started becoming widely used in the early 1980s. At the time, it was a complement to UUCP which was better suited to handle e-mail transfers between machines that were intermittently connected. SMTP, on the other hand, works best when both the sending and receiving machines are connected to the network all the time.

Sendmail was one of the first (if not the first) mail transfer agents to implement SMTP. As of 2001 there are at least 50 programs that implement SMTP as a client (sender of messages) or a server (receiver of messages). Some other popular SMTP server programs include Philip Hazel's exim, IBM's Postfix, D. J. Bernstein's Qmail, and Microsoft Exchange Server.

Since this protocol started out as purely ASCII text-based, it did not deal well with binary files. Standards such as MIME were developed to encode binary files for transfer through SMTP. Today, most SMTP servers support the 8BITMIME extension, permitting binary files to be transmitted almost as easily as plain text.

SMTP is a "push" protocol that does not allow one to "pull" messages from a remote server on demand. To do this a mail client must use POP3 or IMAP. Another SMTP server can trigger a delivery in SMTP using ETRN.

Example SMTP communication

After establishing a connection between the sender (the client) and the receiver (the server), the following is a legal SMTP session. In the following conversation, everything sent by the client is prefaced with "C:" and everything sent by the server is prefaced with "S:". On most computer systems, a connection can be established using the telnet command on the sending machine, for example

telnet www.example.com 25

which opens an SMTP connection from the sending machine to the host www.example.com.

S: 220 www.example.com ESMTP Postfix
C: HELO mydomain.com
S: 250 Hello mydomain.com
C: MAIL FROM: sender@mydomain.com
S: 250 Ok
C: RCPT TO: friend@example.com
S: 250 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: Subject: test message
C: From: sender@mydomain.com
C: To: friend@example.com
C:
C: Hello,
C: This is a test.
C: Goodbye.
C: .
S: 250 Ok: queued as 12345
C: quit
S: 221 Bye

Although optional, nearly all clients ask the server which SMTP extensions the server supports by using the EHLO greeting (rather than HELO as shown above) and the text body of the email (following DATA) is typically MIME formatted.

SMTP security and spamming

One of the limitations of the original SMTP is that it has no facility for authentication of senders. Therefore the SMTP-AUTH extension was defined.

In spite of this, E-mail spamming is still a major problem. Modifying SMTP extensively, or replacing it completely, is not believed to be practical, due to the network effects of the huge installed base of SMTP. Internet Mail 2000 is one such proposal for replacement.

For this reason, there are a number of proposals for sideband protocols that will assist SMTP operation. The Anti-Spam Research Group of the IRTF is working on a number of proposals for providing simple source authentication that is flexible, lightweight, and scalable. The most likely proposal to be accepted is the Sender Policy Framework protocol.

(Back to Top)

SNMP

The Simple Network Management Protocol (SNMP) forms part of the internet protocol suite as defined by the Internet Engineering Task Force. The protocol can support monitoring of network-attached devices for any conditions that warrant administrative attention.

Architecturally, the SNMP framework has three fundamental components:

1. master agents
2. subagents
3. management stations

Each Internet Protocol (IP) addressable system in a network, such as a node or a router, hosts a master agent for that system. A master agent typically limits its activity to parsing and to formatting of the protocol.

If the system has multiple manageable subsystems present, the master agent passes on the requests it receives to one or more subagents. These subagents model objects of interest within a subsystem and interface to that subsystem for monitoring and management operations. The role of the master agent and subagent can merge, in which case one can simply speak of an agent.

A clean separation of the protocol from the structure of management information has made it easy to use SNMP to monitor and often manage hundreds of different subsystems within a network. Within the SNMP architecture, a management information base (MIB) models each managed subsystem with a subsystem-specific definition. This MIB specifies precisely the management data and operations that a subagent makes possible. This model permits management across all layers of the OSI reference model and extending into applications such as databases, email, and the J2EE reference model.

The manager or management station provides the third component. It functions as the equivalent of a client in a client-server architecture. It issues requests for management operations on behalf of an administrator or application, and receives traps from agents as well.

The SNMP protocol operates at the application layer (layer 7) of the OSI model. It specified (in version 1) four core protocol data units (PDUs):

1. GET, used to retrieve a piece of management information.
2. GETNEXT, used iteratively to retrieve sequences of management information.
3. SET, used to make a change to a managed subsystem.
4. TRAP, used to report an alert or other asynchronous event about a managed subsystem.
5. Typically, SNMP uses UDP ports 161 for the agent and 162 for the manager.

Simple Network Management Protocol version 2 revises version 1 and includes improvements in the areas of performance, security, confidentiality, and manager-to-manager communications. It introduced GETBULK, an alternative to iterative GETNEXTs for retrieving large amounts of management data in a single request.

The Internet Engineering Task Force (IETF) recognizes Simple Network Management Protocol version 3 as defined by RFC3411-RFC3418 (also known as STD0062) as the current standard version of SNMP as of 2004. The IETF considers earlier versions as "Obsolete" or "Historical".

In practice, SNMP implementations often support multiple versions: typically SNMPv1, SNMPv2c, and SNMPv3. See RFC3584 "Coexistence between Version 1, Version 2, and Version 3 of the Internet-standard Network Management Framework" (link in the following section).

SNMP Usage Example

The output below exemplifies an snmpwalk performed on a router, and shows general information about the device.

snmpwalk -c public punch system
SNMPv2-MIB::sysDescr.0 = STRING: Cisco Internetwork Operating System Software IOS (tm) C2600 Software (C2600-IO3-M), Version 12.2(15)T5, RELEASE SOFTWARE (fc1)
TAC Support: http://www.cisco.com/tac
Copyright (c) 1986-2003 by cisco Systems, Inc.
Compiled Thu 12-Jun-03 15:49 by eaarm
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.9.1.187
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (835747999) 96 days, 17:31:19.99
SNMPv2-MIB::sysContact.0 = STRING: wikiuser
SNMPv2-MIB::sysName.0 = STRING: punch
SNMPv2-MIB::sysLocation.0 = STRING: test
SNMPv2-MIB::sysServices.0 = INTEGER: 78
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (0) 0:00:00.00


(Back to Top)

SPX

Sequenced Packet Exchange (SPX) is an old Novell protocol used to manage the Novell Internetwork Packet Exchange (IPX) that allowed Novell Servers and clients to communicate over LAN (Local Area Networks) and WAN (Wide Area Networks).

The advantages of one machine easily finding another machine on the LAN was a disavantage on a WAN since it needed so much overhead. TCP/IP is now a more common protocol stack for both WAN and LAN.

(Back to Top)

SSH

In computing, Secure shell, or SSH, is both a computer program and an associated network protocol designed for logging into and executing commands on a remote computer. It is intended to replace the earlier rlogin, telnet and rsh protocols, and provides secure encrypted communications between two untrusted hosts over an insecure network. X11 connections and arbitrary TCP ports can also be forwarded over the secure channel, and files can be transferred using the associated scp or sftp programs. The standard TCP port that an ssh server listens to is port 22.

The first version of the protocol (now called SSH-1) was designed, and the first software written, by Tatu Ylönen from Espoo, Finland in 1995. He soon formed a company called SSH Communications Security Oy to exploit this innovation. The original version of the SSH software used various pieces of free software, such as GNU libgmp, but later versions released by SSH Secure Communications evolved into increasingly proprietary software. SSH Communications Security subsequently relicensed SSH to F-Secure Oy (formerly known as Data Fellows Oy). SSH Secure Communications has a USA subsidiary in Palo Alto, California.

A later version of the protocol was released under the name SSH-2. It is being standardised by the IETF "secsh" working group, and features both security and feature improvements over SSH-1. Examples of the former are Diffie-Hellman key exchange and strong integrity checking via MACs; of the latter, the ability to run any number of shell sessions over a single SSH connection. [1] (http://www.snailbook.com/faq/ssh-1-vs-2.auto.html)

The program is a common Unix shell program for client connections, accompanied by a daemon for accepting remote connections. Implementations exist for most modern platforms, including Microsoft Windows (where one of the most popular is PuTTY) and Mac OS. There are commercial versions, freeware versions, and open source versions.

List of SSH implementations

• Lsh, the GNU Projects' implementation of SSH. (client and server)
• OpenSSH is an open source implementation of SSH. OpenSSH derives from the original, free implementation of SSH. (client and server)
• PuTTY (client)
• SSH Tectia Client [2] (http://www.ssh.com/products/tectia/client/)
• PenguiNet [3] (http://www.siliconcircus.com/penguinet/)
• SSHDOS [4] (http://sshdos.sourceforge.net/)
• WinSCP [5] (http://winscp.sourceforge.net/eng/) (file transfer client)
• JavaSSH [6] (http://javassh.org/)
• Dropbear (http://matt.ucc.asn.au/dropbear/dropbear.html), small client and server for POSIX systems

(Back to Top)

TCP

Transmission Control Protocol (TCP) is a connection-oriented, reliable delivery byte-stream transport layer communication protocol, currently documented in IETF RFC 793 [1] (http://www.ietf.org/rfc/rfc793.txt). It does the task of the transport layer in the simplified OSI model of computer networks.

In the Internet protocol suite, TCP is the intermediate layer between the Internet Protocol below it, and an application above it. Applications most often need reliable pipe-like connections to each other, whereas the Internet Protocol does not provide such streams, but rather only unreliable packets.

Applications send streams of 8-bit bytes to TCP for delivery through the network, and TCP divides the byte stream into appropriately sized segments (usually delineated by the maximum transmission unit (MTU) size of the data link layer of the network the computer is attached to). TCP then passes the resulting packets to the Internet Protocol, for delivery through an internet to the TCP module of the entity at the other end. TCP checks to make sure that no packets are lost by giving each byte a sequence number, which is also used to make sure that the data is delivered to the entity at the other end in the correct order. The TCP module at the far end sends back an acknowledgement for bytes which have been successfully received; a timer at the sending TCP will cause a timeout if an acknowledgement is not received within a reasonable round trip time, and the (presumably lost) data will then be re-transmitted. The TCP checks that no bytes are damaged by using a checksum; one is computed at the sender for each block of data before it is sent, and checked at the receiver.

TCP connections contain three phases: connection establishment, data transfer and connection termination. A 3-way handshake is used to establish a connection. A four-way handshake is used to tear-down a connection. During connection establishment, parameters such as sequence numbers are initialized to help ensure ordered delivery and robustness.

Connection establishment (3-way handshake)

While it is possible for a pair of end hosts to initiate a connection between themselves simultaneously, typically one end opens a socket and listens passively for a connection from the other. This is commonly referred to as a passive open, and it designates the server-side of a connection. The client-side of a connection initiates an active open by sending an initial SYN segment to the server as part of the 3-way handshake. The server-side should respond to a valid SYN request with a SYN/ACK. Finally, the client-side should respond to the server with an ACK, completing the 3-way handshake and connection establishment phase.

Data transfer

During the data transfer phase, a number of key mechanisms determine TCP's reliability and robustness. These include using sequence numbers for ordering received TCP segments and detecting duplicate data, checksums for segment error detection, and acknowledgements and timers for detecting and adjusting to loss or delay.

During the TCP connection establishment phase, initial sequence numbers (ISNs) are exchanged between the two TCP speakers. These sequence numbers are used to identify data in the byte stream, and are numbers that identify (and count) application data bytes. There are always a pair of sequence numbers included in every TCP segment, which are referred to as the sequence number and the acknowledgement number. A TCP sender refers to its own sequence number simply as the sequence number, while the TCP sender refers to receiver's sequence number as the acknowledgement number. To maintain reliability, a receiver acknowledges TCP segment data by indicating it has received up to some location of contiguous bytes in the stream. An enhancement to TCP, called selective acknowledgement (SACK), allows a TCP receiver to acknowledge out of order blocks.

Through the use of sequence and acknowledgement numbers, TCP can properly deliver received segments in the correct byte stream order to a receiving application. Sequence numbers are 32-bit, unsigned numbers, which wrap to zero on the next byte in the stream after 232-1. One key to maintaining robustness and security for TCP connections is in the selection of the ISN.

A 16-bit checksum, consisting of the one's complement of the one's complement sum of the contents of the TCP segment header and data, is computed by a sender, and included in a segment transmission. (The one's complement sum is used because the end-around carry of that method means that it can be computed in any multiple of that length - 16-bit, 32-bit, 64-bit, etc - and the result, once folded, will be the same.) The TCP receiver recomputes the checksum on the received TCP header and data. The complement was used (above) so that the receiver does not have to zero the checksum field, after saving the checksum value elsewhere; instead, the receiver simply computes the one's complement sum with the checksum in situ, and the result should be -0. If so, the segment is assumed to have arrived intact and without error.

Note that the TCP checksum also covers a 96 bit pseudo header containing the Source Address, the Destination Address, the Protocol, and TCP length. This provides protection against misrouted segments.

The TCP checksum is a quite weak check by modern standards. Data Link Layers with a high probability of bit error rates may require additional link error correction/detection capabilities. If TCP were to be redesigned today, it would most probably have a 32-bit cyclic redundancy check specified as an error check instead of the current checksum. The weak checksum is partially compensated for by the common use of a CRC or better integrity check at layer 2, below both TCP and IP, such as is used in PPP or the Ethernet frame. However, this does not mean that the 16-bit TCP checksum is redundant: remarkably, surveys of Internet traffic have shown that software and hardware errors that introduce errors in packets between CRC-protected hops are common, and that the end-to-end 16-bit TCP checksum catches most of these simple errors. This is the end-to-end principle at work.

Acknowledgements for data sent, or lack of acknowledgements, are used by senders to implicitly interpret network conditions between the TCP sender and receiver. Coupled with timers, TCP senders and receivers can alter the behavior of the flow of data. This is more generally referred to as flow control, congestion control and/or congestion avoidance. TCP uses a number of mechanisms to achieve both robustness and high performance. These mechanisms include the use of a sliding window, the slow-start algorithm, the congestion avoidance algorithm, the fast retransmit and fast recovery algorithms, and more. Enhancing TCP to reliably handle loss, minimize errors, manage congestion and go fast in very high-speed environments are ongoing areas of research and standards development.

Connection termination

The connection termination phase uses a four-way handshake, with each side of the connection terminating independently. Therefore, a typical teardown requires a pair of FIN and ACK segments from each TCP endpoint.

TCP ports

TCP uses the notion of port numbers to identify sending and receiving applications. Each side of a TCP connection has an associated 16-bit unsigned port number assigned to the sending or receiving application. Ports are categorized into three basic categories: well known, registered and dynamic/private. The well known ports are assigned by the Internet Assigned Numbers Authority (IANA) and are typically used by system-level or root processes. Well known applications running as servers and passively listening for connections typically use these ports. Some examples include: FTP (21), TELNET (23), SMTP (25) and HTTP (80). Registered ports are typically used by end user applications as ephemeral source ports when contacting servers, but they can also identify named services that have been registered by a third party. Dynamic/private ports can also be used by end user applications, but are less commonly so. Dynamic/private ports do not contain any meaning outside of any particular TCP connection. There are 65535 possible ports officially recognised.

TCP development

TCP is both a complex and evolving protocol. However, while significant enhancements have been made and proposed over the years, its most basic operation has not changed significantly since RFC 793, published in 1981. RFC 1122, Host Requirements for Internet Hosts, clarified a number of TCP protocol implementation requirements. RFC 2581, TCP Congestion Control, one of the most important TCP related RFCs in recent years, describes updated algorithms to be used in order to avoid undue congestion. In 2001, RFC 3168 was written to describe explicit congestion notification (ECN), a congestion avoidance signalling mechanism. In the early 21st century, TCP is typically used in approximately 95% of all Internet packets. Common applications that use TCP include HTTP/HTTPS (world wide web), SMTP/POP3/IMAP (email) and FTP (file transfer). Its widespread use is testimony to the original designers that their creation was exceptionally well done.

Recently, a new protocol compatible with conventional TCP has been developed called FAST TCP (Fast Active queue management Scalable Transmission Control Protocol) by scientists at Caltech. It uses up to 10 parallel routes to transmit data and can be thousands of times faster than traditional TCP.

TCP Software Limitations

Networking speed gradually raised over the years. What started as a protocol build for unreliable low speed networks (few Kbytes per second) is now required to run at 1 Gigabit per second. The TCP software implementations on host systems require extensive computing power. Gigabit TCP communication, alone, eats up 100% of a 2.4 GHz Pentium processor.

Hardware TCP Solutions

One way to overcome the processing power limitations of TCP is building hardware implementations of TCP, widely known as TCP Offload Engines (TOE). The main problem of TOEs is that apart from the improvement they bring to the system they are hard to integrate into computing systems, requiring extensive changes in the Operating System of the computer. A more robust solution are the Network Traffic Accelerators (NTA) or TCP accelerators. Those are hardware coprocessors that assist the main CPU in a system to process the TCP packets. Those solutions don't require changes in the Operating System and are very simple to implement.

Alternatives to TCP

However, TCP is not appropriate for many applications, and newer transport layer protocols are being designed and deployed to address some of the inherent weaknesses. For example, real-time applications often do not need, and will suffer from TCP's reliable delivery mechanisms. In those types of applications it is often better to deal with some loss, errors or congestion than to try to adjust for them. Example applications that do not typically use TCP include real-time streaming multimedia (such as Internet radio), real-time multiplayer games and voice over IP (VoIP). Any application that doesn't require reliability, or that wants to minimize functionality, may choose to avoid using TCP. In many cases, the User Datagram Protocol (UDP) may be used in place of TCP when just application multiplexing services are required.

(Back to Top)

Telnet

Telnet is a network protocol used on the Internet. IETF document STD 8 (aka RFC 854 (http://www.faqs.org/rfcs/rfc854.html) and RFC 855 (http://www.faqs.org/rfcs/rfc855.html)) states:

The purpose of the TELNET Protocol is to provide a fairly general, bi-directional, eight-bit byte oriented communications facility.

It is typically used to provide user oriented command line login sessions between hosts on the internet.

By extension, telnet is also the name of a program that a user can use to invoke a Telnet session to a remote host; the telnet program provides the client part of the protocol. Telnet clients have been available on most Unix systems for many years, and are available for virtually all types of computers.

"To telnet" is also used as a verb meaning to establish or use such a connection, as in, "If you need to change your password you need to telnet to the server and run the passwd command".

Telnet Protocol details

Telnet is a client-server protocol, based on TCP, and clients generally connect to port 23 on the host providing the service (though like many protocols in use on the Internet which port to use is fairly easy to change). Partly because of the design of the protocol and partly because of the flexibility typically provided by telnet programs, it is also possible to use a Telnet program to establish an interactive TCP connection to some other service on an internet host. A classic use of this is telnetting to port 25 (where typically an SMTP server is to be found) to debug a mail server.

The Telnet protocol can be divided into a core and a set of extensions. The core protocol is described by IETF documents RFC 854 and RFC 855 which are also collected together in STD 8, which defines fairly basic operating characteristics of the protocol and a means of defining and implementing extensions. There are many extensions, some of which have been adopted as Internet Standards, some of which haven't. IETF STD document numbers from 27 through to 32 define various Telnet extensions (most of which are extremely common). Of the remaining extensions the most useful ones are probably those that are on the IETF standards track as proposed standards; details can be found in STD 1.

This is explained later, but telnet is insecure - and in the modern day world should not be used.

Telnet Security

There are three main problems with Telnet, making it a bad choice for modern systems from the point of view of security:

Commonly used telnet daemons have several vulnerabilities discovered over the years, and probably several more still exist. Telnet does not encrypt any data sent over the connection (including passwords), and so it is trivial to eavesdrop on the communications and use the password later for malicious purposes.

Telnet lacks an authentication scheme that makes it possible to ensure that communication is carried out between the two desired hosts, and not intercepted in the middle.

In environments where security is important, such as on the public Internet, telnet should not be used. Telnet sessions are unencrypted. This means that anybody who has access to any router, switch, or gateway located on the network between the two hosts where telnet is being used can intercept the telnet packets passing by and easily obtain login and password information (and whatever else is typed) with any of several common utilities like tcpdump and Ethereal.

These flaws have seen the usage of the Telnet protocol drop rapidly in favor of a more secure and functional protocol called SSH, released in 1998. SSH provides all functionality present in telnet, with the addition of strong encryption to prevent sensitive data such as passwords from being intercepted, and public key authentication, to ensure that the remote computer is actually who it claims to be.

Experts in Computer Security, such as SANS Institute, and the members of the comp.os.linux.security newsgroup recommend that the use of Telnet for remote logins should be discontinued under all normal circumstances.

When telnet was being developed in the early 1980s (according to some sources (http://philip.greenspun.com/ancient-history/history-of-computer-science) in 1969), most users of networked computers were in the computer departments of academic institutions, or at large private and government research facilities. In this environment, security was not nearly as much of a concern as it became after the bandwidth explosion of the 1990s. With the exponential rise in the number of people with access to the Internet, and by extension, the number of people attempting to crack into other people's servers, telnet should generally not ever be used on networks with Internet connectivity.

Telnet clients are still occasionally used to manually "talk" to other services. It is sometimes used in debugging network services such as an SMTP or HTTP server, by serving as a simple way to send commands to the server and examine the responses. Telnet can also be used as a rudimentary IRC client if you know the protocol well enough.

Telnet is also heavily used for Multi User Dungeon games played over the internet.

(Back to Top)

TokenRing

Token ring is a local area network protocol which resides at the data link layer (DLL) of the OSI model. It uses a special three-byte frame called a token that travels unidirectionally around a star-wired logical ring. Token ring frames travel completely around the loop. The name 'Token Ring' is misleading since the physical topology is a loop.

Each station passes or repeats the special token frame around the ring to its nearest upstream neighbor. This token identifies a particular entity. This token-passing process is used to arbitrate access to the shared ring media. Stations that have data frames to transmit must first acquire the token before they can transmit them. Token ring LANs normally use differential Manchester encoding of bits on the LAN media.

IBM popularized the use of token ring LANs in the mid 1980s when it released its IBM token ring architecture based on active multi-station access units (MSAUs or MAUs) and the IBM Structured Cabling System. The Institute of Electrical and Electronics Engineers or IEEE (http://www.ieee.org) later standardized a token ring LAN system as IEEE 802.5 (http://www.8025.org).

Token ring LAN speeds of 4Mbps, 16Mbps, 100Mbps and 1Gbps have been standardized by the IEEE 802.5 working group.

Token ring networks had significantly superior performance and reliability compared to early shared-media implementations of Ethernet (IEEE 802.3), and were widely adopted as a higher-performance alternative to shared-media Ethernet.

However, with the development of switched Ethernet, token ring architectures lagged badly behind Ethernet in both performance and reliability. The higher sales of Ethernet allowed economies of scale which drove down prices further, and added a compelling price advantage to its other advantages over token ring.

Token ring networks have since declined in usage and the standards activity has since come to a standstill as switched Ethernet has dominated the LAN/layer 2 networking market.

The Token Frame

When no station is transmitting a data frame, a special token frame circles the loop. This special token frame is repeated from station to station until arriving at a station that needs to transmit data. When a station needs to transmit a data frame, it converts the token frame into a data frame for transmission. The special token frame consists of three bytes as follows:

Starting Delimiter - consists of a special bit pattern denoting the beginning of the frame. The bits from most significant to least significant are J,K,0,J,K,0,0,0. J and K are code violations. Since Manchester encoding is self clocking, and has a transition for every encoded bit 0 or 1, the J and K codings violate this, and will be detected by the hardware.

Access Control - this byte field consists of the following bits from most significant to least significant bit order: P,P,P,T,M,R,R,R. The P bits are priority bits, T is the token bit which when set specifies that this is a token frame, M is the monitor bit which is set by the Active Monitor (AM) station when it sees this frame, and R bits are reserved bits.

Ending Delimiter - The counterpart to the starting delimiter, this field marks the end of the frame and consists of the following bits from most significant to least significant: J,K,1,J,K,1,I,E. I is the intermediate frame bit and E is the error bit.

Token Ring Frame Format

A data token ring frame is an expanded version of the token frame that is used by stations to transmit medium access control (MAC) management frames or data frames from upper layer protocols and applications.

The token ring frame format is defined as follows:

• Starting Delimiter - as described above.
• Access Control - as described above.
• Frame Control - a one byte field that contains bits describing the data portion of the frame contents.
• Destination address - a six byte field used to specify the destination(s).
• Source address - a six byte field that is either the locall assigned address (LAA) or universally assigned address (UAA) of the sending station adapter.
• Data - a variable length field of 0 or more bytes, the maximum allowable size depending on ring speed containing MAC management data or upper layer information.
• Frame Check Sequence - a four byte field used to store the calculation of a CRC for frame integrity verification by the receiver.
• Ending Delimiter - as described above.
• Frame Status - a one byte field used as a primitive acknowledgement scheme on whether the frame was recognized and copied by its intended receiver.

Active and Standby Monitors

Every station in a token ring network is either an active monitor (AM) or standby monitor (SM) station. However, there can be only one active monitor on a ring at a time. The active monitor is chosen through an election or monitor contention process.

The monitor contention process is initiated when

1. a loss of signal on the ring is detected,
2. an active monitor station is not detected by other stations on the ring or
3. when a particular timer on an end station expires such as the case when a station hasn't seen a token frame in the past 7 seconds.

The station with the highest MAC address will win the election process. Every other station becomes a standby monitor. All stations must be capable of becoming an active monitor station if necessary.

The active monitor performs a number of ring administration functions. The first function is to operate as the master clock for the ring in order to provide synchronization of the signal for stations on the wire. Another function of the AM is to insert a 24-bit delay into all frame transmissions. A third function for the AM is to ensure that a frames on the ring is present or to detect a broken ring. Lastly, the AM is responsible for removing circulating frames from the ring.

Token Ring Insertion Process

Token ring stations must go through a 5-phase ring insertion process before being allowed to participate in the ring network. If any of these phases fail, the token ring station will not insert into the ring and the token ring driver may report an error.

• Phase 0 (Lobe Check) - A station first performs a lobe media check. A station is wrapped at the MSAU and is able to send 2000 test frames down its transmit pair which will loop back to its receive pair.

The station checks to ensure it can receive these frames without error.

• Phase 1 (Physical Insertion) - A station then sends a 5 volt signal to the MSAU to open the relay.
• Phase 2 (Address Verification) - A station then transmits MAC frames with its own MAC address in the destination address field of a token ring frame.

When the frame returns and if the address copied and frame recognized bits are set, the station knows there is another station using its MAC address on the ring and will then de-insert.

• Phase 3 (Participation in Ring Poll) - Before transmitting any data, the station must participate in the periodic (every 7 seconds) ring poll process. This is where stations identify themselves on the network as part of the MAC management functions.
• Phase 4 (Request Initialization) - Finally a station sends out a special request to a parameter server to obtain configuration information. This frame is sent to a special functional address, typically a token ring bridge, which may hold timer and ring number information with which to tell the new station about.

(Back to Top)

TLS

Secure Sockets Layer (SSL) and Transport Layer Security (TLS), its successor, are cryptographic protocols which provide secure communications on the Internet.

These protocols provide endpoint authentication and communications privacy over the Internet using cryptography. In typical use, only the server is authenticated (i.e. its identity is ensured) while the client remains unauthenticated; mutual authentication requires PKI deployment to clients. The protocols allow client/server applications to communicate in a way designed to prevent eavesdropping, tampering, and message forgery.

Both TLS and SSL involve a number of basic phases:

• Peer negotiation for algorithm support
• Public-key encryption-based key exchange and certificate-based authentication
• Symmetric cipher-based traffic encryption

TLS Applications

The SSL and TLS protocols run on layers beneath application protocols such as HTTP, SMTP and NNTP and above the TCP transport protocol, which forms part of the TCP/IP protocol suite. While both SSL and TLS can add security to any protocol that uses TCP, they occur most commonly used in the HTTPS access method. HTTPS serves to secure World Wide Web pages for applications such as Electronic commerce. Both the SSL and the TLS protocols use public key cryptography and public key certificates to verify the identity of endpoints.

While an increasing number of client and server products can support TLS or SSL natively, many still do not. In these cases, a user may wish to use standalone SSL products like Stunnel to provide SSL encryption.

History and development

Developed by Netscape, SSL version 3.0 was released in 1996, which later served as a basis to develop Transport Layer Security (TLS), an IETF standard protocol. The first definition of TLS appeared in RFC 2246: "The TLS Protocol Version 1.0". Visa, MasterCard, American Express and many leading financial institutions have endorsed TLS for commerce over the internet.

Like SSL (which provided its base), the TLS protocol operates in modular fashion: its authors designed it for extendability, with support for forwards and backwards compatibility and negotiation between peers.

TLS early weak keys

Some early implementations of SSL could use a maximum of only 40-bit symmetric keys because of US government restrictions on the export of cryptographic technology. The US government explicitly imposed a 40-bit keyspace small enough to be broken by brute-force search by law enforcement agencies wishing to read the encrypted traffic, while still presenting obstacles to less-well-funded attackers. A similar limitation applied to Lotus Software's 'Notes' product in export versions. After a several years of public controversy, a series of lawsuits, and eventual US government recognition of changes in the market availability of 'better' cryptographic products (within and without the US), the authorities relaxed some aspects of the export restrictions. The 40-bit key size limitation has mostly gone away. Modern implementations use 128-bit (or longer) keys for symmetric key ciphers.

(Back to Top)

UDP

The User Datagram Protocol (UDP) is a minimal message-oriented transport layer protocol that is currently documented in IETF RFC 768.

In the TCP/IP model, UDP provides a very simple interface between a network layer below and an application layer above. UDP provides no guarantees for message delivery and a UDP sender retains no state on UDP messages once sent onto the network. (For this reason UDP is sometimes expanded to "Unreliable Datagram Protocol"). UDP adds only application multiplexing and data checksumming on top of an IP datagram.

The UDP header consists of only 4 header fields of which two are optional. The source and destination port fields are 16-bit fields that identify the sending and receiving process. Since UDP is stateless and a UDP sender may not solicit replies, the source port is optional. If not used, the source port should be set to zero. The port fields are followed by a mandatory length field specified as bytes of the UDP datagram including the data. The minimum value of the length field is 8 (octets). The remaining header field is a 16-bit checksum field covering the header and data. The checksum is also optional, but almost always used in practice.

Lacking reliability, UDP applications must generally be willing to accept some loss, errors or duplication. Some applications such as TFTP may add rudimentary reliability mechanisms into the application layer as needed. Most often, UDP applications do not require reliability mechanisms and may even be hindered by them. Streaming media, real-time multiplayer games and voice over IP (VoIP) are examples of applications that often use UDP. If an application requires a high degree of reliability, a protocol such as the Transmission Control Protocol may be used instead.

Lacking any congestion avoidance and control mechanisms, network-based mechanisms are required to minimize potential congestion collapse effects of uncontrolled, high rate UDP traffic loads. In other words, since UDP senders cannot detect congestion, network-based elements such as routers using packet queueing and dropping techniques will often be the only tool available to slow down excessive UDP traffic. The Datagram Congestion Control Protocol (DCCP) is being designed as a partial solution to this potential problem by adding end host congestion control behavior to high-rate UDP streams such as streaming media.

While the total amount of UDP traffic found on a typical network is often on the order of only a few percent, numerous key applications use UDP. These include the Domain Name System (DNS), the simple network management protocol (SNMP), the Dynamic host configuration protocol (DHCP) and the Routing Information Protocol (RIP) to name just a few.

(Back to Top)

X.25

X.25 is an ITU-T standard protocol suite for WAN networks using the phone or ISDN system as the networking hardware. It defines standard physical layer, data link layer and network layers (layers 1 through 3) of the OSI model. The packet switching network was the common name given to the international collection of X.25 providers, typically the various national telephone companies. Their combined network had large global coverage during the 1980s and into the '90s, and it is still in use mainly in transaction systems.

X.25 was developed in the ITU Study Group VII based upon a number of emerging data network projects, such as the research project at the UK's National Physical Laboratory under the direction of Donald Davies who developed the concepts of packet switched networks. In the late 1960s a test network was started, and by 1974 a number of sites had been linked together to form SERCnet (Science and Engineering Research Council Network). SERCnet would later grow and be re-organized as JANET in 1984, which continues in service today, but as a TCP/IP network. Other contributions to the standardising process came from the ARPA project as well as French, Canadian, Japanese and Scandinavian projects emerging in the early 1970s. Various updates and additions were worked into the standard, eventually recorded in the ITU series of technical books describing the telecoms systems. These books were published every fourth year with different colored covers.

The general concept of X.25 was to create a universal and global packet-switched network on what was then the bit-error prone analog phone system. Much of the X.25 system is a description of the rigorous error correction needed to achieve this, a system known as LAP-B. The X.25 model was based on the concept of establishing "virtual calls" through the network, with "data terminating equipment" (DTE's) providing endpoints to users that looked like point-to-connections.

X.25 was developed in the era of dumb terminals connecting to host computers selected in an address protocol (or alternately, two hosts), so the idea of random routing from a single point as in TCP/IP was not considered. As a result, X.25 appears to be a circuit switched network, even though in fact the data itself is packet switched internally. Host machines were described by a phone-number-like address described in the X.121 Address Format, which consisted of a three-digit country code, a one-digit carrier, and a ten-digit National Terminal Number. Note the use of a single carrier digit, allowing for only 10 carriers per country, at the time considered more than enough. However the US soon had more than ten services, and shortened its country code to "31" to allow for up to 100 systems.

For much of its history X.25 was used for permanent virtual circuits (PVCs) to connect two host computers in a dedicated link. This was common for applications such as banking, where distant branch offices could be connected to central hosts for a cost that was considerably lower than a permanent long distance telephone call. X.25 was typically billed as a flat monthly service fee, and then a price-per-packet on top of this. Speeds increased during the years, typically up to 48 or 96 kbit/s.

Publicly-accessible X.25 networks (Compuserve, Tymnet, Euronet) were set up in most countries during the 1970s and 80s, to lower the cost of accessing various online services, in which the user would first interact with the network interface to set up the connection. Known as switched virtual circuits (SVCs) or "virtual calls" in public data networks', this use of X.25 disappeared from most places fairly rapidly as long distance charges fell in the 1990s and today's Internet started to emerge.

A number of systems were developed to directly use the underlying packet nature of X.25, back when it appeared that X.25 would become the single universal networking system. Many of these were "private" applications, but the X.400 e-mail system was also based on X.25 as a transmission layer. The basic idea was to develop a universal set of standards for "Open System Interconnect (OSI)", however, industry developments eventually led down the path to Internet.

With the widespread introduction of "perfect" quality digital phone services and error correction in modems, the overhead of X.25 was no longer worthwhile. The result was Frame Relay, essentially the X.25 protocol with the error correction systems removed, and somewhat better throughput as a result.

X.25 networks are still in use throughout the world, although in dramatic decline, being largely supplanted by newer layer 2 technologies such as frame relay, ISDN, ATM, POS, and the ubiquitous layer 3 Internet Protocol. They remain some of the only available reliable links in many portions of the third world however, where access to a PDN may be the most reliable and low cost way to access the internet.

(Back to Top)

XDR

eXternal Data Representation (XDR) is an implementation of the presentation layer in the OSI model. XDR allows data to be wrapped in an architecture independent matter so data can be transferred between heterogenous computer systems. Converting from the local representation to XDR is called encoding. Converting from XDR to the local representation is called decoding. XDR is implemented as a software library of functions that is portable between different operating systems and is also independent of the transport layer. Sun RPC uses XDR.

XDR Data Types

XDR defines the following data types:

• boolean
• char
• short
• int
• long
• float
• double
• enumeration
• structure
• string
• fixed length array
• variable length array
• union
• opaque data

(Back to Top)

XML

The Extensible Markup Language (XML) is a W3C recommendation for creating special-purpose markup languages. It is a simplified subset of SGML, capable of describing many different kinds of data. Its primary purpose is to facilitate the sharing of structured text and information across the Internet. Languages based on XML (for example, RDF, RSS, MathML, XSIL and SVG) are themselves described in a formal way, allowing programs to modify and validate documents in these languages without prior knowledge of their form.

XML Strengths and weaknesses

The features of XML that make it particularly appropriate for data transfer are:

• simultaneously human- and machine-readable format
• support for Unicode representing all current and many historical character sets
• the ability to represent the most general computer science data structures (records, lists and trees)
• the format is self-documenting in that it describes the structure and field names as well as specific values
• strict syntax makes the necessary parsing algorithms simple, fast and efficient.

XML is also heavily used as the format for document storage and processing, both online and offline, and offers several benefits:

• robust, logically-verifiable format based on international standards
• hierarchical structure suitable for most (but not all) types of document
• plain text files, unencumbered by licenses or restrictions
• platform-independent, and so relatively immune to changes in technology
• has already been in use (as SGML) for long over a decade, and is very popular by itself, so there is extensive experience and software available.

For certain applications, the format also has the following weaknesses:

• XML syntax is fairly verbose and partially redundant. This can hurt human readability and application efficiency, and yields higher storage costs. It can also make XML difficult to apply in cases where bandwidth is limited, though compression can reduce the problem in some cases.
• XML syntax contains a number of obscure features due to its legacy of SGML compatibility.
• XML still often requires further parsing to extract individual values.
• Modelling overlapping (non-hierarchical) data structures requires extra effort.
• Mapping XML to the relational or object oriented paradigms is often cumbersome.

Syntax rules in XML

An XML document is text, usually a particular encoding of Unicode such as UTF-8 or UTF-16, although other encodings may be used.

Unlike, for example, HTML, XML is highly dependent upon structure, content and integrity for its efficacy. In order for a document to be considered "well-formed" W3C Recommendation XML 1.0 (Third Edition) (http://www.w3.org/TR/REC-xml#sec-well-formed), it must conform (at the very least) to the following:

• It must have one (and only one) root element.
• Non-empty elements must be delimited by a start-tag and an end-tag. Empty elements may be marked with an empty-element tag.
• All attribute values must be quoted (either single (') or double (") quotes, but a single quote closes a single quote and a double quote a double quote. The other pair can then be used inside values.)
• Tags may be nested but may not overlap, that is each non-root element must be completely contained in another element.
• Element names in XML are case-sensitive: for example and are a well-formed matching pair whereas and are not.

Also, clever choice of XML element names allows the meaning of the data to be retained as part of the markup. This makes it more easily interpreted by humans while also consumable by software programs.

As a concrete example, a simple recipe expressed in an XML representation might be:

 <?xml version="1.0" encoding="UTF-8"?>
 <Recipe name="bread" prep_time="5 mins" cook_time="3 hours">
    <title>Basic bread</title>
    <ingredient amount="3" unit="cups">Flour</ingredient>
    <ingredient amount="0.25" unit="ounce">Yeast</ingredient>
    <ingredient amount="1.5" unit="cups">Warm Water</ingredient>
    <ingredient amount="1" unit="teaspoon">Salt</ingredient>
    <Instructions>
       <step>Mix all ingredients together, and knead thoroughly.</step>
       <step>Cover with a cloth, and leave for one hour in warm room.</step>
       <step>Knead again, place in a tin, and then bake in the oven.</step>
    </Instructions>
 </Recipe>

Giving logical names to elements and attributes allows an author unfamiliar with a particular document type to quickly grasp the meaning of elements and attributes without having to refer to documentation or having to spend several minutes studying a document to understand its structure. This can, however, also lead to excess verbosity (which can complicate authoring) and greatly increase file size (which decreases efficiency when content is transferred over a network).

An XML document that complies with an associated schema (such as a DTD) in addition to being well-formed is said to be "valid".

XML schema languages

Before the advent of generalised data description languages such as SGML and XML, software designers had to define special file formats or small languages to share data between programs. This required writing detailed specifications and special-purpose parsers and writers.

XML's regular structure and strict parsing rules allows software designers to leave parsing to standard tools, and since XML provides a general, data model-oriented framework for the development of application-specific languages, software designers need only concentrate on the development of schemas for their data, at relatively high levels of abstraction.

An XML schema is a description of a type of XML document, typically expressed in terms of constraints on the structure and content of documents of that type, above and beyond the basic constraints imposed by XML itself. A number of standard and proprietary XML schema languages have emerged for the purpose of formally expressing such schemas, and some of these languages are XML-based, themselves.

Well-tested tools exist to validate XML files against a schema in order to automatically verify whether the document conforms to constraints expressed in the schema. Other usages of schemas exist: XML editors, for instance, can use schemas to support the editing process.

The oldest XML schema format is the DTD (Document Type Definition), which is inherited from SGML. While DTD support is ubiquitous due to its inclusion in the XML 1.0 standard, it is seen as limited for the following reasons:

• No support for newer features of XML, most importantly namespaces.
• Lack of expressivity. Certain formal aspects of an XML document cannot be captured in a DTD.
• Custom non-XML syntax to describe the schema, inherited from SGML.

A newer XML schema language, described by the W3C as the successor of DTDs, is simply called XML Schema, also referred to as XML Schema Definition (XSD). XSD schemas are far more powerful than DTDs in describing XML languages. Additionally XSD uses an XML based format, which makes it possible to use the XML toolset to help process XML schema. It also becomes possible to write a schema for the schema language itself. Criticisms of XSD are:

• Standard is very large, which makes it difficult to understand and implement.
• XML-based syntax leads to verbosity in schema description, which makes XSDs harder to read and write.

Another XML popular schema language is RELAX NG. Initially standardized by OASIS and now also a ISO international standard (as part of DSDL), RELAX NG comes in two formats, an XML based syntax and a non-XML compact syntax. The compact syntax aims to increase readability and writability, but since there is a well-defined way to translate compact syntax to the XML syntax and back again the advantage of using standard XML tools is not lost. RELAX NG has a more compact definition which makes it easier to implement than XSD.

Some schema languages not only describe the structure of a particular XML format but also offer limited facilities to influence processing of individual XML files that conform to this format. DTDs and XSDs both have this ability; they can for instance provide attribute defaults. RELAX NG intentionally does not provide these facilities.

Displaying XML on the web

Extensible stylesheet language (XSL) is a further adjunct to XML that allows users to describe visual properties and transformations of XML data without embedding those instructions into the data itself. The resulting document can then be displayed by a browser in analogy to an HTML document which uses CSS for rendering. One way to achieve this, is to include the following line in the XML document:

which declares that the named XSLT style sheet should be used to transform the XML into HTML. This process may of course also occur on the server side as well as in the browser.

An XML document may also be rendered directly in some browsers such as e.g. Internet Explorer 5 or Mozilla with the stylesheet language CSS. This process is still not yet stable as of March 2004 in those browsers; in other browsers, such as Opera, this works very well. In order to allow CSS styling, the XML document must include a special reference to a style sheet:

Note that this differs greatly from the standard HTML way to call a stylesheet, where it is usually done by the element.

While browser-based XML rendering develops, the alternative is conversion into HTML or PDF or other formats on the server. Programs like Cocoon process an XML file against a stylesheet (and can perform other processing as well) and send the output back to the user's browser without the user needing to be aware of what has been going on in the background.

XML extensions

XPath It is possible to refer to individual components of an XML document using XPath. This allows stylesheets in (for example) XSL and XSLT to dynamically "cherry-pick" pieces of a document in any sequence needed in order to compose the required output. XQuery is to XML what SQL is to relational databases.

XML namespaces enable the same document to contain XML elements and attributes taken from different vocabularies, without any naming collisions occurring.

XML Signature defines the syntax and processing rules for creating digital signatures on XML content.

XML Encryption defines the syntax and processing rules for encrypting XML content.

Processing XML files

The APIs widely used in processing XML data by programming languages are SAX and DOM. SAX is used for serial processing whereas DOM is used for random-access processing. Another form of XML Processing API is data binding, where XML data is made available as a strongly typed programming language data structure, in constrast to the DOM. Example data binding systems are the Java Architecture for XML Binding (JAXB) [1] (http://java.sun.com/xml/jaxb/) and the Strathclyde Novel Architecture for Querying XML (SNAQue) [2] (http://www.cis.strath.ac.uk/research/snaque/).

A processor for languages in the extensible stylesheet language (XSL) family may be used to render an XML file for displaying or printing. One of these members, called XSL-FO, is used to render XML files for display. XSL-FO is basically a modern replacement for CSS. XSLT, another member in the XSL family, is for transforming to other formats, including HTML, other vocabularies of XML, and any other plain-text format. XQuery [3] (http://www.w3.org/TR/xquery/) is a W3C language for querying, constructing and transforming XML data. XPath [4] (http://www.w3.org/TR/xpath) is a path expression language for selecting data within an XML file. XPath is a sublanguage of both XQuery and XSLT.

The native file format of OpenOffice.org and AbiWord is XML. Some parts of Microsoft Office 11 will also be able to edit XML files with a user-supplied schema (but not a DTD). There are dozens of other XML editors available.

Versions of XML

The current version of XML is 1.1 (as of February 4, 2004). The first version XML 1.0 currently exists in its third revision. XML 1.0 and XML 1.1 differ in the requirements of characters used for element names, attribute names etc.: XML 1.0 only allows characters which are valid Unicode 2.0, which includes most world scripts, but excludes scripts which only entered in a later Unicode version, such as Mongolian, Cambodian, Amharic, Burmese, etc.. XML 1.1 only disallows certain control characters, which means that any other character can be used, even if the Unicode standard grows exponentially.

It should be noted here that the restriction present in XML 1.0 only applies to element/attribute names: both XML 1.0 and XML 1.1 allow for the use of full Unicode in the content itself. Thus XML 1.1 is only needed if in addition to using a script added after Unicode 2.0 you also wish to write element and attribute names in that script.

Other minor changes between XML 1.0 and XML 1.1 are that control characters are now allowed to be included but only when escaped, and two special 'form-feed' characters are included, which must be treated as whitespace.

XML 1.0 documents are well-formed XML 1.1 documents with one exception: XML documents that contain unescaped C1 control characters are now malformed: this is because XML 1.1 requires the C1 control characters to be escaped with numeric character references.

There are also discussions on an XML 2.0, although it remains to be seen if such will ever come about. XML-SW (SW for skunk works), written by one of the original developers of XML, contains some proposals for what an XML 2.0 might look like: elimination of DTDs from syntax, integration of namespaces, XML Base and XML Information Set into the base standard.