Network Working Group H. Clark
Request For Comments: 1856 BBN Planet
Category: Informational September 1995
The Opstat Client-Server Model for Statistics Retrieval
Status of this Memo
This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.
Abstract
Network administrators gather data related to the performance,
utilization, usability and growth of their data network. The amount
of raw data gathered is usually quite large, typically ranging
somewhere between several megabytes to several gigabytes of data each
month. Few (if any) tools exist today for the sharing of that raw
data among network operators or between a network service provider
(NSP) and its customers. This document defines a model and protocol
for a set of tools which could be used by NSPs and Network Operation
Centers (NOCs) to share data among themselves and with customers.
1.0 IntrodUCtion
Network administrators gather data related to the performance,
utilization, usability and growth of their data network. The primary
goal of gathering the data is to facilitate near-term problem
isolation and longer-term network planning within the organization.
The amount of raw data gathered is usually quite large, typically
ranging somewhere between several megabytes to several gigabytes of
data each month. From this raw data, the network administrator
produces various types of reports. Few (if any) tools exist today
for the sharing of that raw data among network operators or between a
network service provider (NSP) and its customers. This document
defines a model and protocol for a set of tools which could be used
by NSPs and Network Operation Centers (NOCs) to share data among
themselves and with customers.
1.1 The OPSTAT Model
Under the Operational Statistics model [1], there exists a common
model under which tools exist for the collection, storage, retrieval
and presentation of network management data.
This document defines a protocol which would allow a client on a
remote machine to retrieve data from a central server, which itself
retrieves from the common statistics database. The client then
presents the data to the user in the form requested (maybe to a X-
window, or to paper).
The basic model used for the retrieval methods defined in this
document is a client-server model. This architecture envisions that
each NOC (or NSP) should install a server which provides locally
collected information for clients. Using a query language the client
should be able to define the network object of interest, the
interface, the metrics and the time period to be examined. Using a
reliable transport-layer protocol (e.g., TCP), the server will
transmit the requested data. Once this data is received by the
client it could be processed and presented by a variety of tools
including displaying the data in a X window, sending postscript to a
printer, or displaying the raw data on the user"s terminal.
The remainder of this document describes how the client and server
interact, describes the protocol used between the client and server,
and discusses a variety of other issues surrounding the sharing of
data.
2.0 Client-Server Description
2.1 The Client
The basic function of the client is to retrieve data from the server.
It will accept requests from the user, translate those requests into
the common retrieval protocol and transmit them to the server, wait
for the server"s reply, and send that reply to the user.
Note that this document does not define how the data should be
presented to the user. There are many methods of doing this
including:
- use a X based tool that displays graphs (line, histogram, etc.)
- generate PostScript output to be sent to a printer
- dump the raw data to the user"s terminal
Future documents based on the Operational Statistics model may define
standard graphs and variables to be displayed, but this is work yet
to be done (as of this writing).
2.2 The Server
The basic function of the server is to accept connections from a
client, accept some series of commands from the client and perform a
series of actions based on the commands, and then close the
connection to the client.
The server must have some type of configuration file, which is left
undefined in this document. The configuration file would list users
that could Access the server along with the authentication they would
use. The configuration file should also allow the specification of
the data items that the user should be permitted to access (and, by
implication, not allowed to access). Server security concerns are
specifically addressed in Section 4.
3.0 Protocol Commands
This section defines the commands which may be transmitted to the
server and the server responses to those commands. The available
commands are:
LOGIN - accept new connection
EXIT - disconnect
LIST - show available variables
SELECT - mark data for retrieval
STATUS - show the state of the server
GET - download data to the client
In addition, a state machine describing specific actions by the
server is included. Server security concerns are addressed in
Section 4.
Note that in some of the descriptions below, the term <ASCII-STRING>
is used. This refers to printable ASCII characters, defined as all
letters, numbers, and special characters such as $, %, or *. It
specifically excludes all special control characters in the lower
parts of the character set (i.e., 0x00 - 0x1F), and any such
characters that are received by the server or client should be
ignored.
3.1 Command Return Codes
The responses a server will return to a client are of the form:
RETURN-INFO ::= <RETURN-CODE> " <ASCII-STRING> " <RETURN-CODE>
RETURN-CODE ::= <MAIN-CODE><COMMAND><SUB-CODE>
MAIN-CODE ::= 1..9
COMMAND ::= 1..9
SUB-CODE ::= 0..9
For each command sent to the server, the server returns a series of
three digit numeric codes which specifies the result of the
operation, plus optional ASCII text for humans. The value of MAIN-
CODE specifies what happened, as in:
1 Error
9 Success / Informational
The commands are encoded as:
1 LOGIN
2 SELECT
3 STATUS
4 LIST
5 GET
9 EXIT
The following specific error codes must be supported by all servers
and clients:
110 Login Invalid
113 Scanning Error during LOGIN
120 SELECT Failed
130 STATUS Failed
140 LIST Failed
141 Bad LIST encoding
150 GET Failed
151 GET doesn"t support that type of encoding
910 Login Accepted
920 SELECT successful
931 STATUS Output Starting
932 STATUS Output Done
941 LIST lookup successful, here comes the data!
942 LIST dump done!
951 GET lookup successful, here comes the data!
952 GET dump done!
990 Server closing connection after EXIT received
Other codes may be used, but may not be supported by all clients or
servers.
3.2 The LOGIN Command
The LOGIN command authenticates a user to the server. The format of
the LOGIN command is:
LOGIN-CMD ::= LOGIN <username> <auth-type>
USERNAME ::= " <ASCII-STRING> "
AUTH-TYPE ::= "none" "passWord" " <ASCII-STRING> "
CHAL-CMD ::= CHAL " <ASCII-STRING> "
AUTH-CMD ::= AUTH " <ASCII-STRING> "
The authentication types supported by each server will vary, but must
include "none" and "password". Note that a server may or may not
choose to allow logins via either of these methods, but it must
recognize the two special authentication types.
In processing a LOGIN command sequence, the server first checks the
username and authentication type requested. If the username is
invalid (e.g., there"s no such user known to the server) or the
authentication type requested is not supported by the server, then
the server must return a 110 error and close the connection after
faking the challenge/authentication process (see examples below).
After passing the username and authentication type checking, a
challenge must be sent. Note that the challenge will be specific to
the type of authentication requested, and the ASCII string may be an
empty string if no specific challenge is needed (such as in the
password-only case). The next command the client returns must be an
AUTH response, and if not, the server must close the connection.
After processing the authentication information, the server must
return a 910 code if the authentication process is successful, or a
110 error messsage if unsuccessful. Additionally, if the
authentication fails, the server must immediately close the
connection.
If, at any point, during the LOGIN sequence, a syntax error occurs (a
client doesn"t send the correct number of arguments in the LOGIN
command, for example), the server must return a 113 error and close
the connection.
If the special AUTH-TYPE of "none" is used, and the server allows the
specified username (such as anonymous) to login without
authentication, then the server should still send a "CHAL" response
to get additional information about the person logging in. The
server may then choose to allow or disallow the login based on the
information returned in the AUTH response.
An example of an invalid authentication type requested:
>LOGIN "cow" "s/key"
<CHAL "lo35098 98"
>AUTH "COW DOG BARK CAT MOO MEOW"
<110 "Login invalid"
The server didn"t support S/Key, but it made it appear to the user as
if it did. An example of an authentication failure:
>LOGIN "dog" "securid"
<CHAL "enter passcode"
>AUTH "103945"
<110 "Login invalid"
The user gave the wrong number for SecurID authentication. An
example of a successful login:
>LOGIN "cat" "password"
<CHAL "send the dumb clear-text password"
>AUTH "Foobar"
<910 "Login accepted"
or
>LOGIN "anonymous" "none"
<CHAL "tell me who you are anyway"
>AUTH "bessie@barn.farm.com"
<910 "Login accepted"
An example of a invalid username:
>LOGIN "mule" "skey"
<CHAL "78 lo39065"
>AUTH "COW DOG FRED LOG COLD WAR"
<110 "Login invalid"
The server should have some type of logging mechanism to record both
successful and unsuccessful login attempts for a system adminstrator
to peruse.
3.3 The EXIT Command
The EXIT command disconnects a current user from the server. The
format of the EXIT command is:
EXIT
Note that upon reception of an EXIT command, the server must always
close the connection, even if it would be appropriate to return an
ERROR return code.
A sample EXIT command:
>EXIT
<990 "OK, Bye now"
3.4 The SELECT Command
The SELECT command is the function used to tag data for retrieval
from the server. The SELECT command has the format:
SELECT-COM ::= SELECT <NETWORK> <DEVICE> <INTERFACE> <VARNAME>
<GRANULARITY> <START-DATE> <START-TIME> <END-DATE>
<END-TIME> <AGG> <SELECT-COND>
NETWORK ::= <ASCII-STRING>
DEVICE ::= <ASCII-STRING>
INTERFACE ::= <ASCII-STRING>
VARNAME ::= <ASCII-STRING>
GRANULARITY ::= <ASCII-STRING>
START-DATE ::= <DATE-TYPE>
END-DATE ::= <DATE-TYPE>
DATE-TYPE ::= YYYY-MM-YY
START-TIME ::= <TIME-TYPE>
END-TIME ::= <TIME-TYPE>
TIME-TYPE ::= HH:MM:SS
AGG ::= <AGG-TYPE> NULL
AGG-TYPE ::= TOTAL PEAK
SELECT-COND ::= <SELECT-STMT> NULL
SELECT-STMT ::= WITH DATA <COND-TYPE> <ASCII-STRING>
COND-TYPE ::= LE GE EQ NE LT GT
If any conditional within the SELECT does not match existing data
within the database (such as VARNAME, the S-DATE or E-DATE, or
GRANULARITY), the server must return an ERROR (and hopefully a
meaningful error message). The time values must be specified in GMT,
and hours are specified in the range from 0-23. The granularity
should always be specified in seconds. A sample query might be:
SELECT net rtr1 eth-0 ifInOctets 900 1992-01-01 00:00:00 1992-02-
01 23:59:59
which would select all data from network "net" device "rtr1"
interface "eth-0" from Jan 1, 1992 @ 00:00:00 to Feb 1, 1992 @
23:59:59.
Note that if the client requests some type of aggregation to be
performed upon the data, then the aggregation field specifies how to
perform the aggregration (i.e., total or peak) and the granularity
specifies to what interval (in seconds) to agggregate the data to.
For more details about the granularity types, see [1]. If the server
cannot perform the requested action, then it must return a 120 error.
The server may, if it wishes, use other error codes in the range
121-129 to convey more information about the specific error that
occured. In either case, its recommended that the server return
ASCII text describing the error.
Upon completion of the data lookup, the SELECT must return the an
indication of whether the lookup was successful and (if the search
was successful) the tag associated with that data. If the lookup was
successful, then information in the return code should be encoded as:
920 " TAG <ASCII-STRING> "
In this case, the use of the word TAG is used as a handle for the
selected data on the server. Note that this single handle may refer
to one or more specific SNMP variables (refer to [1] for a further
eXPlanation).
For example, if the tag "foobar" were assigned to the select example
above, then the OK would be as:
920 "TAG foobar"
It is recommended that the return tag string be less than 10 bytes
long (this gives many tag combinations), although the server (and
client) should be capable of handling arbitrary length strings.
There is no requirement that the TAG have any particular meaning and
may be composed of arbitrary strings.
The server must keep any internal information it needs during a
session so that all SELECT tags can be processed by GET or other
commands. If a server doesn"t have the resources to process the
given SELECT, it must return an error message.
It is the responsibility of the client to store information about the
data that a particular tag corresponds to, i.e., if the server had
returned a tag "1234" for ifInOctet data for October 1993, then the
client must store that information someplace as the variables which
correspond to that tag cannot be retrieved from the server.
3.5 The STATUS Command
The STATUS command shows the general state of the server plus listing
all data sets which have been tagged via the SELECT command. The
STATUS command has no arguments. The output from a STATUS command
is:
STATUS-DATA ::= <SERVER-STATUS> <SERVER-TAG-LIST>
SERVER-STATUS ::= "STATUS= " <STATUS-FIELDS>
STATUS-FIELDS ::= "OK" "NOT-OK"
SERVER-TAG-LIST ::= <SERVER-TAG> NULL
SERVER-TAG ::= "TAG" <TAG-ID> "SIZE" <NUMBER>
The number returned in the SIZE field represents the number of octets
of data represented by the particular TAG. The server must return a
931 message before the STATUS output starts, and a 932 message at the
end of the STATUS output. If any type of failure occurs, then a 130
error messages must be sent. If the server prefers, it may send a
message in the range of 131-139 if it wishes, but its recommended
that the server always return ASCII describing the enoutered error.
For example, a sample output might look like:
>STATUS
<931 "STATUS Command Starting"
<STATUS= OK
<TAG 1234 SIZE 123456
<TAG ABCD SIZE 654321
<932 "STATUS Command successful"
or
>STATUS
<130 "Can"t get STATUS right now, sorry."
or
>STATUS
<931 "STATUS Command Starting"
<STATUS= OK
<TAG 1234 SIZE 1
<131 "Oops, error reading TAG table, sorry."
3.6 The GET Command
The GET command actually retrieves the data chosen via a previous
SELECT command. The GET command has the format:
GET-CMD ::= GET <TAG> <TYPE>
TAG ::= <ASCII-STRING>
TYPE ::= 1404 <ASCII-STRING>
If the TAG matches a previously returned TAG from a SELECT statement,
then the previously tagged data is returned. If the TAG is invalid
(i.e., it hasn"t been previously assigned by the server), then the
server must return an error. The TYPE specifies the encoding of the
data stream. All servers must support "1404" encoding. Others forms
may be supported as desired.
If the server, while retrieving the data, cannot retrieve some
portion of the data (i.e., some of the data previously found
disappeared between the time of the SELECT and the time of the GET),
then the server must return a 150 error. If the client requests an
encoding type not supported by the server, then the server must
return a 151 error.
The format of the returned data is as follows:
RETURN-DATA-TYPE ::= START-DATA <RETURN-TYPE> <DATA> END-DATA
RETURN-TYPE ::= 1404 <ASCII-STRING>
An example would be:
>GET ABC 1404
<951 "OK, here it comes!"
<START-DATA 1404
1404 data stream here...
<END-DATA
<952 "All done!"
Error examples:
>GET ABC STRONG-CRYPT
<151 "Sorry, that encoding not available here"
or
>GET ABC 1404
<951 "OK, here it comes!"
<START-DATA 1404
1404 data stream here...
<END-DATA
<150 "Whoa, bad data..."
If any type of error code is returned by the server, the client must
discard all data received from the server.
3.7 The LIST Command
The LIST command allows the client to query the server about
available data residing on the server. The LIST command has the
format:
LIST-CMD ::= LIST <net> <dev> <intf> <var> <gran> <sdate> <stime>
<edate> <etime>
<net> ::= <ASCII-STRING> *
<dev> ::= <ASCII-STRING> *
<intf> ::= <ASCII-STRING> *
<var> ::= <ASCII-STRING> *
<gran> ::= <ASCII-STRING> *
<sdate> ::= <DATE-TYPE> *
<edate> ::= <DATE-TYPE> *
<stime> ::= <TIME-TYPE> *
<etime> ::= <TIME-TYPE> *
For example, to get a list of networks that the server has data for,
you would use the command:
LIST * * * * * * * * *
The command
LIST netx rtry * * * * * * *
will list all interfaces for rtry. The command
LIST netx rtry * ifInOctets * 1993-02-01 * * *
will get the list of interfaces on device "rtry" in network "netx"
which have values for the variable "ifInOctets" after the start date
of Februrary 1, 1993.
To process wildcards in a LIST command, follow these rules:
1) Only the leftmost wildcard will be serviced for a given
LIST command
2) If all fields to the right of the leftmost wildcard are
wildcards, then all values for the wildcard being processed
will be returned.
3) If specific values are given for fields to the right of the
wildcard being serviced, then the specific values must match
a known value
The output from the LIST command is formatted as follows:
LIST-RETURN ::= START-LIST <LIST-ENTRY> END-LIST
LIST-ENTRY ::= <net> <device> <intf> <var> <gran> <sdate> <stime>
<edate> <etime>
<net> ::= <ASCII-STRING>
<device> ::= <ASCII-STRING> <NULL>
<intf> ::= <ASCII-STRING> <NULL>
<var> ::= <ASCII-STRING> <NULL>
<gran> ::= <ASCII-STRING> <NULL>
<sdate> ::= <DATE-TYPE> <NULL>
<edate> ::= <DATE-TYPE> <NULL>
<stime> ::= <TIME-TYPE> <NULL>
<etime> ::= <TIME-TYPE> <NULL>
Note that only the fields with values in them will be returned by the
server. For example, the query to find the interfaces on rtry:
>LIST netx rtry * * * * * * *
<941 "OK, here comes the list..."
<START-LIST
<netx rtry intf1
<netx rtry intf2
<netx rtry intf3
<END-LIST
<942 "all done"
The query to find interfaces having ifInOctets data with a 15 minute
granularity:
>LIST netx rtry * ifInOctets 15min * * * *
<941 "OK, here comes the list..."
<START-LIST
<netx rtry intf1
<netx rtry intf2
<netx rtry intf3
<END-LIST
<942 "all done"
If, while processing a LIST command, the server encounters an error,
then the server must return a 140 error message. If the server
cannot process the LIST command (syntax error, etc.), then it must
return a 141 message. For example:
>LIST netx rtry
<141 "huh, bad list dude"
or
>LIST netx rtry * ifInOctets 15min * * * *
<941 "OK, here comes the list..."
<START-LIST
<netx rtry intf1
<netx rtry intf2
<netx rtry intf3
<END-LIST
<140 "Whoa, bad list dude, please ignore"
3.8 The Server State Machine
The state machine pictured below describes how a server should
interact with a client:
+------+
+--------> WAIT <-----+
+------+
New
Connect LOGIN Failure
EXIT /
Received +-------+
LOGIN -----+
+-------+

LOGIN Successful
/
+---------+
+-------- PROCESS <----+
+---------+
Process Commands

+----------+
The server normally stays in WAIT (after starting and initialization)
until a new connection is made from a client. The first command a
client must issue is a LOGIN command, otherwise the server must
immediately close the connection. If the login process fails in any
way (as described in 3.2), then the server must immediately close the
connection and return to the WAIT state.
Once a successful LOGIN is received, the server enters the PROCESS
state where it processes some number of LIST, GET, STATUS, and SELECT
commands. Any other command received while in this state must be
ignored, except for the EXIT command. Once an EXIT command is
received, the server exits immediately (after perfoming any needed
internal bookkeeping) and returns to the WAIT state. Any command a
server receives while processing a command (e.g., if you send an
"EXIT" while a large "GET" is being processed) will be ignored until
the command being processed completes.
If the data connection to the client closes for any reason while the
server is in the PROCESS state, the server must immediately close its
connection and do any associated internal cleanup and return to the
LOGIN state.
4.0 Security Issues
There are legal, ethical and political concerns of data sharing. For
this reason there is a need to insure integrity and confidentiality
of any shared data. Although not specified in this standard,
mechanisms to control a user"s access to specific data about specific
objects may need to be included in server implementations. This
could potentially be done in several ways, including a configuration
file that listed the objects a user was allowed to access or limiting
file access by using file permissions within a given file system. At
a minimum, the server should not allow default access to all data on
the server.
Additionally, the server should strictly follow the state diagram
shown in section 3.8. The server should be tested with arbitrary
strings in the command fields to ensure that no unexpected security
problems will be caused by the server. The server should
specifically discard illegal ASCII characters as discussed in section
3.0. If the server executes other programs, then the server must
verify that no unexpected side-effects will occur as the result of
the invocation or the arguments given to that program. The server
should always verify that all data is contained within the input
buffer, and that a long input string from a client will not cause
unexpected side-effects.
Finally, given the relative insecurity of the global Internet, and
the presence of packet-sniffing capability, several considerations
must be weighed. The authentication process via the LOGIN process
must be strictly adhered to, and the use of one-time authentication
is strongly encouraged. It is also suggested that the data returned
from the server be protected (such as through encryption) so that no
sensitive data is revealed by accident.
5.0 Summary
This document defines a protocol which could be used in a client-
server relationship to retrieve statistics from a remote database
server.
Much work remains to be done in the area of Operational Statistics
including questions such as:
- what "standard" graphs or "variables" should always be made
available to the user?
- what additions to the standard MIB would make the network
manager"s job easier?
6.0 References
[1] Stockman, B., "A Model for Common Operational Statistics", RFC
1404, NORDUnet/SUNET, January 1993.
Appendix A: Sample Client-Server Sessions
Session 1: Check available variables on device rtr1 interface eth0
>LOGIN "henry" "skey"
<CHAL "78 lo35098"
>AUTH "COW MOO DOG BARK CAT MEOW"
<910 "Login OK, what now?"
>LIST OARnet rtr1 eth0 * * * *
<941 "List lookup OK, here it comes!"
<START-LIST
<OARnet rtr1 eth0 ifInOctets
<OARnet rtr1 eth0 ifOutOctets
<OARnet rtr1 eth0 ifInErrors
<OARnet rtr1 eth0 ifOutErrors
<END-LIST
<942 "List done!"
>EXIT
<990 "OK, Bye now!"
Session 2: Retrieve a bit of data from the server
>LOGIN henryc "skey"
<CHAL "78 lo35098"
>AUTH "COW MOO DOG BARK CAT MEOW"
<910 "Login OK, what now?"
>SELECT OARnet rtr1 eth0 InBytes 15min 1993-02-01 00:00:00 1993-03-01 23:59:59
<920 "TAG blah"
>STATUS
<931 "here it comes..."
<STATUS= OK
<TAG blah SIZE 654321
<932 "all done"
>GET blah 1404
<951 "here it comes..."
<START-DATA 1404
1404 data here
<END-DATA
<952 "wow, all done"
>EXIT
<990 "OK, bye"
Author"s Address
Henry Clark
BBN Planet Corp.
150 Cambridge Park Dr.
Cambridge, MA 02140
Phone: (617) 873-4622