$page_title = "Introduction"

Synopsis



NetReg/NetMon was designed from the ground-up at Carnegie Mellon to be a very flexible, lightweight
system for management and centralization of network parameters. The goal is to automate much of the
machine registration and configuration information, while allowing a high degree of control by users.
We wanted to minimize the overhead in maintaining the registration system, and streamline the
management of the network, to maximize the time available to solve better problems.



Warning


<_margin>WARNING!</_margin>

<_default>
<p class=warn>WARNING WARNING WARNING WARNING WARNING WARNING WARNING</P>
<p>
  NetReg/NetMon was designed to be flexible and adaptable to individual
environments. However, you are working with the first public release. We
expect that there will be some hard-coded information that makes absolutely
no sense in your environment. We've tried to minimize this and extract 
the main bits that *need* to change into certain configuration files
(ie CMU/Netdb/config.pm, CMU/WebInt/config.pm, CMU/NetMon/Config.pm),
but you are warned.
<p> 
  We would appreciate receiving a summary of your experience installing NetReg, and
appreciate comments, suggestions, or patches. :) Please see the 
<A HREF="#1.3">Contacting Us</A> section for more information.

<p class=warn>WARNING WARNING WARNING WARNING WARNING WARNING WARNING</p>

</_default>


Overview



NetReg/NetMon is a collection of Perl modules and scripts. There are two main trees the modules are
organized under: <tt>CMU</tt> and <tt>DNS</tt>. Throughout this manual we may refer to <tt>CMU::Module</tt>
or <tt>DNS::Module</tt>, which, following the Perl structure, means that the actual file being discussed
is in <tt>lib/CMU/Module.pm</tt>.



Contacting Us/Downloading Software


<_header3>
Homepage
</_header3>


The canonical source for NetReg/NetMon source is:
<A HREF="http://www.net.cmu.edu/netreg">http://www.net.cmu.edu/netreg</A>.





NetReg/NetMon is maintained in CVS, and you can grab the latest version through anonymous CVS access.
Our policy on updating files to the HEAD of the CVS tree is that they should be at least minimally tested.
Extensive long-term development generally happens on a branch. However, this is not an absolute
guarantee that the CVS tree will be completely functional at any given time.



Further, please note that the default path to 'perl' we use is <tt>/usr/local/bin/perl5</tt>, due to
local installation details. In many cases you will need to change files ending in <tt>.pl</tt> to 
your path.



To check out the tree from our anonymous CVS server, run the commands below. Note that you will get a
few warnings about "Permission Denied" while accessing certain files. These files are not yet available
for public distribution, but your checkout should not be affected.



cvs -d :pserver:anoncvs@cvs.andrew.cmu.edu:/cvs login <br>
<em>(enter "anonymous" as the password)</em><br>
cvs -d :pserver:anoncvs@cvs.andrew.cmu.edu:/cvs co -d netdb src/netdb




<_header3>
Bugs
</_header3>

<_margin> </_margin>


If you have a bug, patch, question, comment, or suggestion, please send them to:
<A HREF="mailto:netreg-bugs@andrew.cmu.edu">netreg-bugs@andrew.cmu.edu</A>. We also have a bugzilla
server for bug tracking. It's at: <a href="http://bugzilla.andrew.cmu.edu">bugzilla.andrew.cmu.edu</a>.



<_header3>
Mailing List
</_header3>


A mailing list has been setup for the purpose of discussing the NetReg/NetMon project. The list is:<br>
<tt>netreg-users@lists.andrew.cmu.edu</tt>.<br>
To subscribe, send mail to 
<a href="mailto:majordomo@lists.andrew.cmu.edu">majordomo@lists.andrew.cmu.edu</a> with a body of:<br>
<tt>subscribe netreg-users</tt>




To unsubscribe, send mail to <a href="mailto:majordomo@lists.andrew.cmu.edu">majordomo@lists.andrew.cmu.edu</a>,
with a body of:<br>
<tt>unsubscribe netreg-users</tt>.



A web-accessible archive of the mailing list is available from:<br>
<a href="http://asg.web.cmu.edu/bb/archive.netreg-users">
http://asg.web.cmu.edu/bb/archive.netreg-users</a>.



Copyright/License



All versions of the Network Registration System source code are covered by
the following copyright:




<_code>
Copyright (c) 2000-2002 Carnegie Mellon University. All rights reserved.<br><br>

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
<br><br>
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
<br><br>
 2. Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in
    the documentation and/or other materials provided with the
    distribution.
<br><br>
 3. The name "Carnegie Mellon University" must not be used to
    endorse or promote products derived from this software without
    prior written permission. For permission or any legal
    details, please contact:<br>
      Office of Technology Transfer<br>
      Carnegie Mellon University<br>
      5000 Forbes Avenue<br>
      Pittsburgh, PA  15213-3890<br>
      (412) 268-4387, fax: (412) 268-7395<br>
      tech-transfer@andrew.cmu.edu<br>
<br>
 4. Redistributions of any form whatsoever must retain the following
    acknowledgment:
    "This product includes software developed by Computing Services
     at Carnegie Mellon University (http://www.cmu.edu/computing/)."
<br><br>
CARNEGIE MELLON UNIVERSITY DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS, IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY BE LIABLE
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
<br><br>
If you find this software useful and valuable in your work, we would
welcome any support you can offer toward continuing this work.

We gratefully accept contributions, whether intellectual or monetary.
Intellectual contributions in the form of code or constructive
collaboration can be directed to netreg-bugs@andrew.cmu.edu (even if it is not a bug.)


</_code>

<!--
[variables]
{
  $page_title = "Installation"
  $page_title_english = "Installation"
  $file_prev = "002.html"
  $file_next = "004.html"
  $real_page_number = "3"
  $page_number = "3"
}
-->

<_header1>
$page_title
</_header1>


Installation Basics



We are going to assume that you know all the requirements (see below)
and can get Apache built with SSL and mod_perl, Perl, DBI, and DBD::mysql
built, and mysql built, installed, and running. There is extensive
documentation with these packages, and we would only be duplicating the
work of others. Note that the CPAN module is very useful for getting 
modules out of CPAN.




Installing NetReg is not an easy task. The installation could be made vastly easier,
and telling us about your experiences installing it will be helpful in this regard.




Here's a brief synopsis of the installation:



<ol>
  <li>Review requirements/recommendations
  <li>Install the database schema/initial data
  <li>Run the <tt>netreg-install</tt> script to build the NetReg tree
  <li>Configure NetReg/NetMon
  <li>Configure the web server
  <li>Add the DNS zones of your DNS/DHCP servers (including parent zones).
  <li>Add the subnets that contain your DNS/DHCP servers.
  <li>Make sure to add permissions for the subnets and zones.
  <li>Register your DNS and DHCP servers.
  <li>Add DNS server configuration groups, zones to the config groups, and nameservers to the zones.
  <li>Add the DHCP service, add permissions, and add the DHCP servers to the service.
  <li>Test DNS/DHCP output
  <li>Add <tt>scheduled.pl</tt> cronjob.
</ul>



Requirements and Recommendations



The following additional software is required (or highly recommended) 
to run the NetReg/NetMon system. We assume that Perl, MySQL, the DBI modules, and Apache are installed
and tested for basic functionality prior to continuing with the installation.




<_default>
<table border=0
<tr>
  <th valign=top>Perl 5.005 or later<br>
    <em>www.perl.org</em></th>
  <td>
<UL>
  <LI>We recommend 5.6.1 or later. Less-than-5.6 allowed us to be a little
      bit sloppier with some references. 
  <LI>Has not been tested on perl 5.7 or 6.x.
</UL>
  </td>
</tr>

<tr>
  <th valign=top>Apache 1.3.x<br>
  <em>www.apache.org</em></th>
  <td>
<UL>
  <LI>Currently running 1.3.20
  <LI>Has not been tested on Apache 2.x.
</UL>
<td></tr>

<tr>
<th valign=top>MySQL 3.23.40+<br>
  <em>www.mysql.org</em></th>
  <td>
<UL>
  <LI>Currently using 3.23.40 in production, 3.23.42 is built and will
      be used here shortly.
  <LI>The system should be able to run on other DBMS; it depends mostly
      on a few features that may not be supported elsewhere. See below
      under "Database Notes" for more info.
</UL>
</td></tr>

<tr>
<th valign=top>DBI Perl Module & DBD::mysql<br>
  <em>www.cpan.org</em></th>
<td>

<UL>
  <li>We use <tt>Msql-Mysql-modules-1.2216</tt>.
</UL>
</td></tr>

<tr>
<th valign=top>Assorted Perl Modules<br>
  <em>www.cpan.org</em></th>
<td>
<UL>
  <LI>The following modules may be required:
    <UL>
      <LI>Config::General (required)
      <LI>Net::DNS (required for dynamic DNS updates)
      <LI>Expect
      <LI>IO::Pty
      <LI>IO::Tty
      <LI>FIXME (other required modules)
    </UL>
  </LI>
</UL>
</tr></tr>

<tr>
<th valign=top>mod-perl 1.26<br>
  <em>perl.apache.org</em></th>
<td>
<UL>
  <LI>Not strictly required, we don't run mod-perl on our devel machines
      because otherwise a server restart is sometimes required/recommended
      for testing. But, mod-perl on our production machine greatly increases
      responsiveness.
</UL>
</td>
</tr>

<tr>
<th valign=top>Authentication module for Apache<br>
<em>modules.apache.org</em>
</th>
<td>

<UL> 
  <LI>We use "pubcookie", a single signon system for the web 
      that integrates nicely with our campus infrastructure. However, NetReg
      should be flexible enough to allow for any Apache auth mechanism.

  <LI>We also use SSL Client Certificates, so the netreg code will look for 
      either the REMOTE_USER environment variable, or an assortment of SSL_*
      variables to determine the user identity.

</UL>
</td></tr>

<tr><th valign=top>OpenSSL, mod_ssl<br>
<em>www.openssl.org; www.modssl.org</em></th>
<td>

<UL>
   <LI>We're going to assume that you would prefer to run a system wherein
      you encrypt the interactions (at very least, the authentication), 
      to prevent information lossage.
   <LI>In particular, if you are planning on using DDNS, we HIGHLY RECOMMEND
      using encryption, since information lossage in this context can mean
      people having complete update privileges to your DNS servers.
</UL>
</td></tr>
<tr><th valign=top>ISC DHCP Server<br>
<em>www.isc.org</em></th>
<td>

<UL> 
  <LI>NetReg can generate the complete configuration file for your ISC DHCP server.
  <LI>DHCP v3.0+ is recommended; certain DHCP v3-only commands and command syntax 
      is used. 
</UL></td></tr>

<tr><th valign=top>ISC BIND Server<br>
<em>www.isc.org</em></th>
<td>
  
<UL>
  <LI>NetReg can generate the <tt>named.conf</tt>, zone files, and do DDNS updates
of ISC Bind 8 and Bind 9 servers.
  <LI>We recommend running DDNS on Bind 9 only.
</UL>
</td></tr>

<tr><th valign=top>FPing<br><em>www.fping.com</em></th>
<td>
<ul>
  <li>Used by NetMon; only needed if you are doing topology-generation
</td>
</tr>

<tr><th valign=top>rsync<br><em>rsync.samba.org</em></th>
<td>
<ul>
  <li>Used on the backend for copying files out to DHCP, DNS servers.
  <li>Not needed if your NetReg server is also DNS/DHCP.
</ul>
</td></tr>

</table>

</_default>


Database Installation



The NetReg database schema is located in <tt>doc/db/NETREG-COMPLETE.sql</tt>. We have developed an initial
set of NetReg data to demonstrate the various areas. This includes the database schema, so you can
load both quickly. It's <tt>doc/netdb.data</tt>. Just do:



<_code>
% mysql -u root -p < doc/netdb.data

</_code>


[Assuming you are in the root directory of the source distribution.] You will be prompted for the MySQL
root password. Once this is loaded, you should have a new database: <tt>netdb</tt>.




At this point, NetReg will have enough basic data to give you an idea how things fit together. However,
it will not have enough data to grant you access via the web. You should run the <tt>netreg-install</tt>
below, and it will take care of this step.



You will also need to create an initial user in your NetReg database for purpose of allowing you to login.
For example, if your username is 'admin' (as Apache would authenticate it), then you should add the initial user
by doing:





<tt>% mysql -u root -p netdb</tt>



Enter your MySQL root password. Then run:




<tt>insert into users (comment) values ('Admin Account'); </tt>
<br /><tt>insert into credentials (authid, user, description) values ('admin', 2, 'Mr. Admin'); </tt>
<br /><tt>insert into memberships (uid, gid) values (2, 1); </tt>




NetReg/NetMon Tree



We have written a script that should help in the installation of NetReg/NetMon. The script,
<tt>doc/netreg-install.pl</tt>, should be run as the root user. If you get a "Command not found" message,
run the script using your local path to perl explicitly: <tt>perl doc/netreg-install.pl</tt>.
The script will do the following:




<ul>
  <li>Create and populate the NetReg/NetMon tree. We use <tt>/home/netreg</tt> as the top of this tree. 
It will be populated with a number of directories. Throughout this document we'll use <tt>$NRHOME</tt>
to refer to this directory.

  <li>Create passwords for accessing the database. You should make sure MySQL is running prior to 
running the installation script.

</ul>



<_margin>Important!</_margin>


After running the script, your <tt>$NRHOME/stable/netdb</tt> will be a link to the <tt>netdb</tt> 
directory as you unpacked the distribution. Since we expect most people unpack downloaded software
in a relatively temporary location, you probably want to:




<ul>
  <li>Remove the <tt>$NRHOME/stable/netdb</tt> symbolic link
  <li>Copy the distribution to <tt>$NRHOME/stable/netdb</tt>.
</ul>



If you decide to checkout/stay updated to the CVS tree, you can check it out to <tt>$NRHOME/stable/netdb</tt>
and the links from, for example, <tt>$NRHOME/lib</tt>, will be maintained.


<_margin>New!</_margin>


New to version 1.1.6 is the ability for <tt>netreg-install.pl</tt> to automatically
configure the two configuration files for NetReg: <tt>lib/CMU/WebInt/config.pm</tt> and
<tt>lib/CMU/Netdb/config.pm</tt>. The last step in netreg-install will be a prompt to
begin configuring these files. The script is fairly self-explanatory; once complete you'll
have the option of having the new configuration written as <tt>config.pm.new</tt>, or 
be copied to <tt>config.pm</tt> itself.



<tt>netreg-install.pl</tt> now accepts the <tt>-config</tt> option. When specified, this
causes netreg-install to only configure the config.pm files. <b>Note!</b> If you are 
using a <tt>$NRHOME</tt> other than <tt>/home/netreg</tt>, and you are using -config,
you'll need to edit the <tt>NRHOME</tt> variable at the top of netreg-install.pl.



NetReg/NetMon Configuration



Now that the database and the NetReg tree are installed, you should set to work configuring the
components for your environment. It's probably a good time, though, if you are comfortable enough
with mysql, to poke around the <tt>netdb</tt> database.




Moving right along, you should visit <tt>$NRHOME/lib/CMU/Netdb/config.pm</tt> and
<tt>$NRHOME/lib/CMU/WebInt/config.pm</tt>, if you haven't configured them using 
<tt>netreg-install.pl</tt>.
Each of these modules contains extensive documentation on how things need to be adjusted. Additionally,
the "Configuration" section below also documents the various geek knobs available. We are trying to
remove any embedded Carnegie Mellon-specific configuration options from most of the source, and make
the various <tt>Config.pm</tt> modules provide the interface for tweaking these bits.



Web Server Configuration



NetReg and NetMon aren't very useful without the web front-end, so it's a good idea to get your web
server configured at this juncture. We use the Apache web server exclusively; you should be able to use
another server without relative difficulty, assuming it supports execution of Perl (CGI) scripts,
authentication, SSL, and can pass the authenticated user as an environment variable (or some other
generic method.)




We have not tried using Apache 2.x locally; if you have success or failure we'd be interested in
knowing.


<_header3>
Apache without mod_perl
</_header3>


The configuration without mod_perl is a little bit simpler, so we'll start without mod_perl, and then
it's just a few other directives to get mod_perl involved. We're going to assume the web server on this 
machine is only going to be serving NetReg/NetMon. If it's not, we'll assume you can make the necessary 
adjustments.



The basic outline is:




<ul>
  <li>Your DocumentRoot is going to be <tt>$NRHOME/htdocs</tt>. You need a Directory
statement for this directory. 
  <li><tt>$NRHOME/htdocs/bin</tt> contains the actual hook to start the bulk of the
web interfaces, so you'll need another Directory statement to require authentication/SSL to access
the <tt>bin</tt> directory.
  <li>You need a hook to execute <tt>.pl</tt> files as cgi scripts. This is done as: <br>
<tt>AddHandler cgi-script .pl</tt>
  <li><tt>index.pl</tt> needs to be added as a valid directory index file. This is done as: <br>
<tt>DirectoryIndex index.pl index.html</tt> (you should already have a DirectoryIndex statement)
</ul>



So the configuration bits we've discussed (with <tt>/home/netreg</tt> assumed as your <tt>$NRHOME</tt>):


<_code>
DocumentRoot /home/netreg/htdocs<br>
...<br>
&lt;Directory "/home/netreg/htdocs"><br>
&nbsp;&nbsp;Options FollowSymLinks ExecCGI<br><br>
&nbsp;&nbsp;AllowOverride none<br>
&nbsp;&nbsp;Order allow,deny<br>
&nbsp;&nbsp;Allow from all<br>
&lt;/Directory><br><br>
&lt;Directory "/home/netreg/htdocs/bin"><br>
&nbsp;&nbsp;Options FollowSymLinks ExecCGI<br><br>
&nbsp;&nbsp;SSLRequireSSL<br><br>
&nbsp;&nbsp;AuthType Basic<br>
&nbsp;&nbsp;AuthName NetReg/NetMon<br>
&nbsp;&nbsp; [[ other auth options, like AuthUserFile, require, etc. ]]<br>
&nbsp;&nbsp;AllowOverride none<br>
&nbsp;&nbsp;Order allow,deny<br>
&nbsp;&nbsp;Allow from all<br>
&lt;/Directory><br>
...<br>
DirectoryIndex index.pl index.html<br>
...<br>
AddHandler cgi-script .pl
</_code>


Your users should be directed to connect to <tt>http://machine.name.example.org</tt> (substituting
your machine name, clearly). The <tt>index.pl</tt> script in $NRHOME/htdocs will provide some basic
instructions to users (and you are encouraged to modify this file appropriately). This, as well as 
the <tt>nc.pl</tt> script in <tt>$NRHOME/htdocs</tt>, also clear users' cookies for your machine. At
CMU, our authentication mechanism uses short-life cookies. Clearing the cookies ensures fresh
credentials are required when accessing NetReg. <tt>nc.pl</tt> is the script pointed to by the 
"Signoff" link in NetReg.



<_header3>
Apache with mod_perl
</_header3>


Adding mod_perl into the equation makes your Apache setup slightly more complicated, but can yield
significant benefits in response time. We do <b>not</b> use a DBMS handle cache (due to us wanting to
just completely avoid any deadlock issues due to un-released locks, etc.) However, the NetReg modules
take a short but appreciable amount of time to load. mod_perl saves both loading time and memory,
since we set things up to load all our modules pre-fork, meaning we having one 10MB copy of NetReg
in memory rather than 12.



<ul>
  <li>Assuming mod_perl is installed with Apache, you should already have: <br>
<tt>LoadModule perl_module libexec/libperl.so</tt><br>
<tt>AddModule mod_perl.c</tt><br>
  <li>Add an appropriate PerlRequire to pre-load various libraries. Substitute $NRHOME appropriately:
<br><tt>PerlRequire $NRHOME/lib/startup.pl</tt>
  <li>Add the Perl InitHandler:<br><tt>PerlInitHandler Apache::StatINC</tt>
  <li>Setup the perl script handler (from the basic config, remove the AddHandler cgi-script .pl):<br>
<tt>&lt;Files ~ "\.pl$"><br>
SetHandler perl-script<br>
PerlHandler Apache::Registry<br>
Options +ExecCGI<br>
&lt;/Files>
</ul>



Stop... Beverage Break



You need to edit the <tt>$NRHOME/htdocs/index.pl</tt> file to make it applicable to your environment.
Basically, the first line needs to be changed to point to your Perl interpreter, and you probably want
to edit the footer to indicate your name.



Finally, edit <tt>$NRHOME/htdocs/bin/netreg.pl</tt> and <tt>$NRHOME/htdocs/bin/nc.pl</tt>
to point to your Perl interpreter (first line).



At this point, you should now be able to connect to NetReg, login, and see the initial machine list.
Under "Registered machines for", select "NetReg Administrators". After refreshing, you should see a
small list of machines. That working, we'll proceed. It's probably a good idea at this point to
read parts of "NetReg Design", including "Subnets", "DNS Zones", "DHCP Options", and "Machines".



That being said, it's time for a small break in the installation.



Add DNS Zones



Now we'll get to work configuring NetReg. The goal will be to get NetReg to the point that your DNS
and DHCP is generated. First, you need to add some DNS zones. At this point you'll need to add the
zones which contain your DHCP and DNS servers. We'll use <tt>EXAMPLE2.ORG</tt> for demonstrating this.




If your DNS servers are <tt>NS-A.EXAMPLE2.ORG</tt> and <tt>NS-B.EXAMPLE2.ORG</tt>, and DHCP is 
<tt>DHCP.EXAMPLE2.ORG</tt>, then you just need to add <tt>EXAMPLE2.ORG</tt>. Since <tt>ORG</tt> already
exists, you can easily do this (NetReg requires all zones from the root zone to every leaf zone used
be registered in the Zones area). If your domain was <tt>EXAMPLE.TV</tt>, though, you'll need to add
the <tt>TV</tt> zone first. (See the "Zones" section below for more details.)




In this case, you just visit the "Zones" area in NetReg, click "Add Zone", enter <tt>EXAMPLE2.ORG</tt>
as the zone name and a type of 'fw-toplevel'. You also need to define the SOA Email, SOA Host, and other
SOA parameters. Your SOA email could be "root.example2.org", for example. Fill in a valid hostname
for SOA Host, but it actually doesn't matter. The other fields have recommended values. Leave the
"DDNS Auth" field blank. 



Once added, you need to add permissions for the zone. Visit the "View/Update Protections" link on the
zone information page. From there, you could add 'netreg:admins' with level 9 ADD access, and this would
restrict any new registrations in the zone to only members of the netreg:admins group. Alternatively,
adding 'system:anyuser' with level 1 ADD would grant anyone the ability to add machines to the zone.



The situation is slightly more complicated if your nameservers are in a child zone to some toplevel
zone you control. Everything can be done via the interface, but a picture is likely worth 1000 words
here, so here's a diagram of our DNS layout (coming soon):




<ul>
  <li><b>toplevel</b>: Serving <tt>CMU.EDU</tt>, 'toplevel' is our unpublished master server, 'ns1.net',
'papaya.srv.cs', and 'cucumber.srv.cs', are published as the zone nameservers (all pulling from the master).
  <li><b>thirdlevel</b>: Serving <tt>ANDREW.CMU.EDU</tt> (and other third level zones), 'thirdlevel' is
our unpublished master server, 'res-ns', 'weh-ns', and 'campus-ns' are the designated secondaries.
  <li><b>"DotCom"</b>: We separate "commercial" endeavours of the university into distinct IP and
namespace. The 'ns1.cmu.net' nameserver handles these zones. It is a published master, with 'ns2.cmu.net'
as the published secondary.
</ul>



You are also going to need reverse zones for the IP ranges that your DNS/DHCP servers are in. If the IP 
of <tt>NS-A.EXAMPLE2.ORG</tt> is 192.168.0.4, you need the <tt>0.168.192.IN-ADDR.ARPA</tt> zone, as well
as the <tt>168.192.IN-ADDR.ARPA</tt> and <tt>192.IN-ADDR.ARPA</tt> zones. The latter two can be added
as type 'rv-permissible', and you don't need to fill in ANY of the SOA values for 'permissible' zones.
The former will need to be added 'rv-toplevel' with appropriate values.


<_margin>Note!</_margin>


Please see section 4.4.2 of this manual for instructions on adding
new top level domains (such as .US, .INFO, etc.) to the system.



Add Subnets



In the Subnet area of NetReg, you must define all subnets that machines will be registered into. For
most registrations, this makes absolute sense. You definitely want subnet definitions for your networks,
so that you can hand out DHCP options applicable to the subnet. However, you will also need to add a 
subnet definition for external networks that you may need to register machines into. For example, if
someone wants WWW.FOO.EXAMPLE.ORG (and you own example.org) to be a CNAME to an external machine, such
as WWW.BAR.ORG, then you will need to "register" www.bar.org (and add a zone for it). Technically you could
register the machine with mode 'reserved', and not specify an IP address. However, we prefer to 
register the machine correctly and make the "ownership" of the additional record the user that requested
the name.



In registering the subnet, specify the subnet name and abbreviation, base address and network mask, and
indicate if you want to permit any dynamics on the subnet (allow unregistered machines to get DHCP
leases), restrict the dynamics (allow only registered machines to get a dynamic address), or flat-out
deny all dynamic IP allocation on the subnet. 



The subnet name can be a free-form string (with spaces, etc.) that will be presented to users as the
name of the subnet. The abbreviation is a short, 8 character max, no spaces abbreviation. This is used
in some output formats -- basically, it provides an easily-parseable and unique identification for
the subnet for use in scripts, etc.



The base address and network mask should be specified in dotted-quad format (127.0.0.1 and 255.255.255.0, for example).
We would like to include some CIDR notation in the interface at some point, but really you just need
to be able to convert between them quickly.



For the time being, you can ignore the purge fields of the NetReg entry. The default values will be
supplied, making the subnet un-purgeable. The purge process if done from NetMon, anyway, so unless
you configure that portion, it won't mean anything.



After adding a new subnet, the system will dynamically calculate the valid "range of addresses" that 
will be usable on this subnet, given your base address/network mask. If the range presented differs
from what you believe is the valid range, double-check your base and mask. There have been a few cases
that we have specified base + mask on an invalid boundary -- ie, you can't start a /26 block on a base
address ending in '96'. 



If you feel like restricting the number of registrations for a particular user on a given subnet, you
can fill in values for the Static or Dynamic quota. Most users are allowed to only add machines with
mode 'static' or 'dynamic'. There are other modes, but these are restricted to administrators, and we
assume they don't need any help with quotas (in fact, Level 9 ADD on a particular subnet for any
user or group is sufficient for bypassing all quota enforcement.) A quota of '0' means 'unlimited'.



Each subnet has a few flags that can be set. Specify 'no_dhcp' if you do NOT want this subnet to appear
in your DHCP configuration. Any subnet that you don't control would be a good candidate for this option.
The 'no_static' flag disallows any static IP address registration for users without L9 ADD permissions.
Make sure your 'Dynamics' setting is 'permit' or 'restrict' if you want normal users to register on
this subnet. The 'delegated' flag is purely informational; the Subnet Map report uses this to change
the color of these subnets so that you get a nice picture of your IP space allocation.


<_header3>
Subnet Presence (Buildings)
</_header3>


After adding a new subnet (or modifying an existing one), you have a few other administrative tasks
that can be done with the subnet. First, if you have any buildings defined, you can add this subnet
to a particular building. It can appear in more than one building, but will appear on the subnet list
after a user selects "Register New Machine" and selects the building. The "Register" page also gives users
the option of selecting the subnet directly, or selecting a "Network" (see below for more information).


<_header3>
Allowed Domains
</_header3>


Within a given subnet, we only allow machine registrations (including by administrators) on domains that
are defined on the Subnet Information page. This provides some consistency in registrations. Note that
users are presented with a drop-down box of valid domains when registering their machine. They will
only see domains for which they have permission to add machines into. Thus, most users at CMU will see
2 or 3 (at most) domains for their registrations, while we register network gear in a different domain.
This domain is also a valid domain for the subnet.


<_header3>
DHCP Options
</_header3>


The subnet information page is also the location for adding subnet-specific DHCP options. You can
select 'Set Standard Options' under the DHCP Options section to automatically add the most common
options for a subnet (such as router, broadcast address, etc.)


<_header3>
Protections
</_header3>


After adding a subnet, you will need to specify some protections to allow registrations in this subnet.
The method is similar to specifying the protections on zones: namely, 'netreg:admins' with Level 9 ADD
would be the default to allow administrators registration on the subnet. 'system:anyuser' at Level 1 ADD
would grant registration privileges to any user.



Register DNS/DHCP Servers



Before proceeding, you should register all of your DNS and DHCP servers. This will require the zones 
to be present (with proper permissions) and the subnets to exist. You can register them as type
'reserved' if you do not know/care about the hardware address, or 'static' if you have this information.



DNS Configuration



Now that your nameservers are completely registered, it's time to create DNS configuration groups. 
For information on doing this, see the DNS section later in this manual.



DHCP Configuration



We use the new "Service" area to specify some parameters for your DHCP configuration. The sample data
includes a "DHCP Server Pool" specification. In the service area, go ahead and add a service of
type "DHCP Server Pool". This just requires specifying the name you want -- we use 'dhcp.net.cmu.edu'.
After adding a DHCP service, you want to add your DHCP servers into the service. Once this is 
complete, you should add the attributes for the servers and the group.


<_header3>
DHCP Service Attributes
</_header3>


The attributes we have defined for DHCP servers thus far includes:



<ul>
  <li><b>Dynamic Master Port</b>: Specifies the port to use on your master server in a failover 
configuration. The default is 519, and the scope is the service group.
  <li><b>Dynamic Secondary Port</b>: Specifies the port ot use on your secondary server in a failover
configuration. The default is 520, and the scope is the service group.
  <li><b>File Type</b>: This per-machine attribute specifies if you want a 'static' file (which contains
only fixed-address and subnet declarations -- no pools), or a 'dynamic' file (which contains all the
pools, etc.)
  <li><b>Dynamic Failover</b>: This can be set to 'master', 'secondary', or 'neither' on a per-machine basis.
This defines the servers involved in the default failover configuration.
</ul>



There are a few restrictions that you should keep in mind:
<ul>
  <li>You can only have one dynamic server, or two dynamic servers configured as failover peers, answering
your DHCP requests on a given wire. You could have multiple DHCP servers, although NetReg isn't quite
designed for that (it's something we'd like to see in the future).
  <li>Only one server can be a failover master, and one a failover secondary.
</ul>



Test DNS/DHCP Output



On your NetReg server, you can become the <tt>netreg</tt> user (or whatever user you specified in
the installation and are running your scheduled scripts as) to test the DNS/DHCP output. 
First, make the following directories: <tt>/tmp/zones</tt>, <tt>/tmp/dhcp</tt>.
Go into your <tt>$NRHOME/bin</tt> directory, and run <tt>./dhcp.pl -debug</tt>, 
<tt>./dns.pl -debug</tt>, and <tt>./dns-config.pl -debug</tt>.
This should create the requisite files in <tt>/tmp</tt> that you can then verify for accuracy. Namely,
you should have a <tt>named.conf.FOO</tt> for each nameserver, a <tt>dhcpd.conf.FOO</tt> for each
DHCP server, and a zone file for each zone configuration (name: <tt>EXAMPLE.ORG.zone</tt>).



Activate Scheduler



At this time, you also need to think about creating a cron job for running scheduled NetReg operations. 
In reality, you probably want to hold off doing this until you update the proto
data. But it logically fits here, so:



We have a cronjob that runs the <tt>$NRHOME/bin/scheduled.pl</tt> script every minute. While there aren't
always things that need to happen every minute, the <tt>scheduled</tt> keeps track of all the different
periodic operations and launches them as necessary. So it's more or less a mini-crond/mini-atd.



You want to add this cron job as user <tt>netreg</tt>, or another user of your choice. We do <b>not</b> recommend
you run the job as: <tt>root</tt>, whatever user your DBMS (ie MySQL) runs as, or the user that your 
web server runs as. Implicitly we're also recommending that your DBMS and web server run as different
non-root users. The theory here is that a compromise/exploit of one of the individual components does
not mean you have an immediate compromise of other components.



Done...



The NetReg installation is now done. You probably want to read the rest the NetReg Design section a few
times. If you are interested in granting users access to register machines, you will need to come up
with a method of loading the allowed users (see "passwd-load.pl" in the "Support Utilities" section).
NetReg requires some tedious initial configuration (and we're trying to reduce the amount of that..), but
once running it is fairly self-sufficient.



<!--
[variables]
{
  $page_title = "Configuration"
  $page_title_english = "Configuration"
  $file_prev = "003.html"
  $file_next = "005.html"
  $real_page_number = "4"
  $page_number = "4"
}
-->

<_header1>
$page_title
</_header1>


This section serves to expand upon settings in <tt>lib/CMU/Netdb/Config.pm</tt>,
<tt>lib/CMU/WebInt/Config.pm</tt>,
and <tt>lib/CMU/NetMon/Config.pm</tt>. You probably want to see the first few sections of "NetReg Design"
and "NetMon Design" to understand how these configurations work together.



Netdb Configuration



The following configuration parameters are defined in <tt>lib/CMU/Netdb/config.pm</tt>.


<_header4>
ADMIN_ADDRESS
</_header4>


Netreg is setup to mail the administrators about problems it
encounters - generally these are things that you probably want to
pay attention to. If you really don't want to receive email, set
the MAILER to ''. But REALLY, you WANT to receive this mail.





ADMIN_ADDRESS is the address to which this mail will be sent.


<_header4>
ADMIN_NAME
</_header4>


ADMIN_NAME is just the pretty name that will be used as the recipient
name on the outgoing administrative mail.


<_header4>
DHCP_SERVICE
</_header4>


This should be set to the name of your DHCP Server Pool service in
NetReg. The DHCP generation script will look for this service
when creating DHCP configurations. Default is <tt>dhcp</tt>.



<_header4>
MAILER
</_header4>


MAILER needs to specify a mail program (like sendmail) that can accept
a mail message on its STDIN, and takes as an argument the address to
send the message. Default is <tt>/usr/lib/sendmail</tt>.



<_header4>
NRHOME
</_header4>


Specifies the home directory for all of the NetReg modules, password cache, output area, administrative
binary area, etc. The default is <tt>/home/netreg</tt>.


<_header4>
SENDER_ADDRESS
</_header4>


SENDER_ADDRESS specifies the email address indicated as the sender of administrative mail. 
It's not strictly necessary, but helpful if you set it to something useful. That way you
can reply easily to the administrative mail (if you want to comment on an error, for example.)


<_header4>
SENDER_NAME
</_header4>


Outgoing administrative mail will indicate the sender's pretty name as SENDER_NAME.


<_header4>
SERVICE_FILE
</_header4>


Specify the location that you want the default services file to be written. The default is 
<tt>$NRHOME/etc/service-gen/services.sif</tt>. We request (recommend?) you at least specify a filename
with a .sif extension, as we'd like to standardize on Service-Information-Files having that extension.


<_header4>
DHCP_GENPATH
</_header4>


Specify the location of your DHCP configuration file generation. The default is 
<tt>$NRHOME/etc/dhcp-gen</tt>, which should be all setup for you.


<_header4>
DHCP_XFERPATH
</_header4>


Specify the location from which your DHCP configuration files are transferred to the DHCP servers
(after being copied from <tt>DHCP_GENPATH</tt>.) The default is <tt>$NRHOME/etc/dhcp-xfer</tt>.


<_header4>
DHCP_SERVICE
</_header4>


Specify the name of your DHCP service pool, as configured in the Services area of NetReg. We use
<tt>dhcp.net.cmu.edu</tt>, for example, but you probably want to change this to something more 
location-oriented.


<_header4>
DHCP_RR_DNS_SERV
</_header4>


The DHCP_RR_DNS_SERV array is a convenient way for you to specify a number of nameservers that you
want to hand out to clients. You <em>can</em> configure the nameservers on a per-subnet (or per-machine)
basis via NetReg, and in fact <tt>dhcp.pl</tt> will be smart enough to recognize when you do (on a 
per-subnet basis) and ignore this array. However, for subnets that do not contain name servers, they
will be filled in from entries on this list.



The actual method of filling in the nameservers is to do so in a round-robin fashion (based off the
NetReg subnet ID). Since most clients accept multiple nameservers, but most clients hit the first
nameserver a good percentage of the time (and in fact take several seconds to timeout to additional
nameservers), this method gives you decent loading across all your nameservers. However, if you know
that certain clients will be better served by a particular nameserver, we recommend you specify the
nameserver option explicitly on the subnet.



The default is: <tt>qw/128.2.4.21, 128.2.32.37, 128.2.64.2/</tt>, our nameservers. We <em><b>highly 
recommend</b></em> you change this to be a group of your own nameservers. If we decide to up and move
ours, your clients could break (aside from which, your queries will take much longer to resolve than 
if you use your own, or even your ISPs, servers).


<_header4>
DHCP_NSKEY
</_header4>


This specifies a path to the file that will contain additional zone and key information from your
DNS configuration generation that will be used in the DHCP configuration. DHCP can only update DNS
if it has the proper keys configured for the various zones (or the nameservers explicitly allow the 
DHCP servers' IP, but we prefer the key approach.) The default of:<br>
<tt>$NRHOME/etc/zone-config/dhcpd.conf.nsaux</tt> should be okay for most purposes.


<_header4>
DNS_GENPATH
</_header4>


This specifies the path in which your DNS zonefiles are written. The default is:<br>
<tt>$NRHOME/etc/zone-gen</tt>.


<_header4>
DNS_XFERPATH
</_header4>


This specifies the path from which your DNS zonefiles and configurations are copied to the DNS servers.
The default is:<br><tt>$NRHOME/etc/zone-xfer</tt>.


<_header4>
DNS_CONFPATH
</_header4>


This specifies the path in which your DNS configurations are written. The default is:<br>
<tt>$NRHOME/etc/zone-config</tt>.


<_header4>
DNS_TOP
</_header4>


The DNS_* configuration bits are specified so that your DNS configurations are correctly generated.
This needs to be the top directory of where you store your DNS related files <em>on your nameserver</em>.
For example, we have historically used <tt>/usr/domain</tt> as the homedirectory. Under that, we have
a <tt>db</tt> directory for storing zone files, a <tt>var</tt> directory for logs, an <tt>etc</tt>
directory for the actual configuration, and <tt>bin</tt> and <tt>sbin</tt> for the server itself
plus assorted utilities.



Specify the top directory here, and specify the remaining DNS_* variables as they fit into your
environment.


<_header4>
DNS_DBDIR
</_header4>


In our environment, <tt>/usr/domain/db</tt> is where we store zonefiles. If your zonefiles will be
in a different location, adjust this variable accordingly.


<_header4>
DNS_SBIN
</_header4>


The DNS_SBIN directory should be the location of the <tt>named</tt> binary as well as assorted 
support utilities (*ndc, named-xfer, dns*). 


<_header4>
DNS_VAR
</_header4>


The DNS_VAR directory is the location that we'll setup for your named process ID (named.pid) file.


<_header4>
DNS_ETC
</_header4>


The DNS_ETC directory should contain the <tt>named.hints</tt> file, and this is the location that
we use to put the <tt>named.conf</tt>.


<_header4>
RSYNC_PATH
</_header4>


We use <tt>rsync</tt> to copy our DNS zone/config files and our DHCP config files to the proper servers.
This utility is pretty flexible in allowing you to use different transport mechanisms while retaining
the easy of copying many files. <tt>RSYNC_PATH</tt> should specify the complete path to your <tt>rsync</tt>
utility. We use <tt>/usr/ng/bin/rsync</tt>, which probably isn't the proper location for you.


<_header4>
RSYNC_OPTIONS
</_header4>


Specify any options that you want to pass to <tt>rsync</tt> in the <tt>RSYNC_OPTIONS</tt> variable.
The default of:<br>
<tt>-a -v -v -v --rsync-path=$RSYNC_PATH</tt><br>
is probably sufficient, unless you have a special configuration.


<_header4>
RSYNC_REM_USER
</_header4>


Specify the remote user that you will use to rsync files with. We use the <tt>ftp</tt> user on our
DNS and DHCP servers, because it's an easy user that already had a local home directory configured.


<_header4>
RSYNC_RSH
</_header4>


The <tt>RSYNC_RSH</tt> variable controls what the environment variable <tt>RSYNC_RSH</tt> will be set
to while performing rsync operations. This is how we invoke SSH, for example, as the underlying
transport mechanism for syncing files. To use SSH we specify:<br>
<tt>/usr/local/lib/ssh/bin/ssh -x -i $NRHOME/etc/.identity-xfer</tt>.




We use an RSA keypair to authenticate the connection. The <tt>$NRHOME/etc/.identity-xfer</tt> files
needs to be owned by the <tt>netreg:netreg</tt>, and we set the permissions to 400.



WebInt Configuration


<_header4>
ADMIN_GROUP
</_header4>


This is the plaintext description of the group responsible for your installation of NetReg -- it will
be printed on the bottom of every NetReg page. We use "Carnegie Mellon Network Development", for
example.


<_header4>
BGCOLOR
</_header4>


The background color attribute is simply that: the background color of all NetReg pages. We use 'white'.


<_header4>
DEF_ITEMS_PER_PAGE
</_header4>


On most screens NetReg will try to paginate the displays - ie show you
only a subset of the entire dataset (limited to X rows). You can control
the parameters of that paging. The DEF_ITEMS_PER_PAGE variables specifies the
default number of items that will be presented on a single page. The default "default"
is 40.



<_header4>
DEF_MAX_PAGES
</_header4>


Another aspect of the NetReg paging is to specify the maximum number of links to pages that will 
be displayed. The links are always centered around the current page. Don't make this too high or the
pagination will take too much horizontal space. Our default is '15'.


<_header4>
ENABLE_CABLES_OUTLETS
</_header4>


If you want to use the cable/outlet interface, set this to '1'.
'0' is recommended unless you are interested in development of this section
and have read the "Outlets" section of the manual.



<_header4>
HDCOLOR
</_header4>


The Heading color is that which is used for major subheadings, delineating major sections of a page.
The default of '#c6d4ff' is a light blue. The color can (or should) be somewhat darker than your
TACOLOR/THCOLOR, but it still needs to allow black text on top to be readable.


<_header4>
LocalRealm
</_header4>


At Carnegie Mellon, we use a Kerberos-based authentication system, so the actual IDs passed from
Apache (in the REMOTE_USER environment variable) are: <tt>userid@ANDREW.CMU.EDU</tt>. In this case,
setting LocalRealm to 'ANDREW.CMU.EDU' causes the effective UserID to be stripped of the at-sign and
the realm. However, a user connecting as, for example, ju33@CS.CMU.EDU, would be treated as the full
string, including non-local realm. Set this to '' if the userIDs from Apache will lack any realm.


<_header4>
MACHINES_PER_PAGE
</_header4>


On the main NetReg page, both machines and outlets are displayed (if ENABLE_CABLES_OUTLETS is activated),
causing the default number of pagination entries to be too many. We recommend setting the MACHINES_PER_PAGE
value to half the DEF_ITEMS_PER_PAGE (so we set it to '20').


<_header4>
NetReg_Web_DB
</_header4>


The NetReg_Web_DB variables defines parameters for establishing the database connection. It should
be a reference to an array of 3 elements. The first should be the driver, the second the username to 
connect to the database as, and the third the password to use. The username is strictly a username
defined in the database server (it has <b>no</b> relationship with NetReg users). 



The driver parameter needs to specify the hostname as well. We use:<br> <tt>DBI:mysql:netdb:localhost</tt>.<br>
This instructs DBI to use the 'mysql' connection method to connect to the 'netdb' database on server
'localhost'. If you are installing the database and web server on the same machine (our setup), then
this should not need to be changed.



We use the 'netreg-web' user for the web accesses to the NetReg database. If you use the <tt>netreg-install</tt>
script, it should give you instructions for configuring this user to have access to the mysql database.



The password can either be specified in plaintext. Alternatively it can provide a reference to a file
that contains the password. The format for this reference is: <tt>file=/path/to/password/file</tt>.
The contents of this file MUST be the password only. If there is a trailing newline, it will be
sent along to the database with the newline. To prevent your standard editor from adding a newline,
we update the passwords as: <br>
<tt>perl -e 'print "PASSWORD"' > /path/to/password/file



Putting it all together, the value we set NetReg_Web_DB to is: <br>
<tt>['DBI:mysql:netdb:localhost', 'netreg-web', 'file=/home/netreg/etc/.password']</tt>
 


<_header4>
SuperUsers [Array]
</_header4>


The SuperUsers array should contain a list of all SuperUsers of the system. This is currently only
helpful for one case (which we haven't ever used). By setting the DB_OFFLINE key in the <tt>_sys_info</tt>
table you will lock out all users <b>except</b> SuperUsers. SuperUsers database access is still 
drawn out of the standard permissions, though. Default: blank.


<_header4>
SYSTEM_MAIN_URL
</_header4>


This should contain the full URL to the main page of your NetReg installation. It will be presented
to users in this manner. Default: 'http://FILL-ME-IN.EXAMPLE.ORG'


<_header4>
SYSTEM_NAME
</_header4>


Whatever you decide to call your system, fill it in here. Default: 'Network Registration System'.


<_header4>
TACOLOR
</_header4>


Most of NetReg's output is done via tables containing many rows of information. We use an "alternate"
color (other than "white") every other row, to make it easier to read entire rows. The default of
'#c0f7de' is an aqua color. We recommend a fairly light color.


<_header4>
THCOLOR
</_header4>


The Table Heading color is used for the headings to input fields. The default of 'lightyellow' is, well,
a light yellow. :) A light color is recommended.


<_header4>
USER_MAIL
</_header4>


Set the USER_MAIL variable to the email address that should be given to users for contacting you. This
email will be a link on the bottom of every NetReg page, so it should be your standard helpdesk/support
staff, but it's probably a bad idea to set this to the same value as your ADMIN_MAIL address.



Support Utility Configuration



We have tried to minimize the amount of configuration required for the support utilities. You should
definitely read about them below (NetReg Design) to determine which ones will need additional assistance.
However you should also definitely configure <tt>vars.sh</tt> and <tt>vars_l.pm</tt> appropriately.


<_header4>
NRHOME
</_header4>


The perpetual configuration required is to make sure everything knows about your NetReg home directory.
Make sure both vars and vars_l know about NRHOME.


<_header4>
NRLIB
</_header4>


vars_l.pm will assume NRLIB to be <tt>$NRHOME/lib</tt>, which is a good assumption if you used
<tt>netreg-install</tt>. Otherwise, change this.


<_header4>
NRUSER
</_header4>


Just leave NRUSER set to <tt>netreg</tt>.


<_header4>
MYSQL_PATH
</_header4>


This is used to quickly change the MYSQL and MYSQLDUMP variables below; set this to be the path to
your mysql/mysqldump binaries.


<_header4>
MYSQL
</_header4>


Specify the full path to your <tt>mysql</tt> binary.


<_header4>
MYSQLDUMP
</_header4>


Specify the full path to your <tt>mysqldump</tt> utility. This is used by dump-db.sh.



NetMon Configuration



It's probably a good idea to read the section on NetMon design prior to configuring it, because there 
are several concepts there that you should understand prior to configuration.



Configuration is done in the <tt>CMU/NetMon/Config.pm</tt> file. Like Netdb/WebInt, there are a number
of variables.


<_header3>
Hostname Conventions
</_header3>


NetMon automatically begins capturing ARP and CAM information from devices that match certain parameters
in NetReg. Thus, if a new switch is added to NetReg, NetMon will begin recording CAM table information
within a day or so.


<_header4>
GW_PATTERN
</_header4>


GW_PATTERN is specified as an array of patterns to match in looking for new routers ("gateway device").
Basically, it will look at the NetReg <tt>host_name</tt> field using the SQL LIKE operator with wildcards
around each pattern. You should make sure to include preceding dot separators to match a domain 
completely. For example, our array of patterns is: <br><tt>(.gw.cmu .gw.net.cmu)</tt> so a host_name of
<tt>foo.gw.cmu.net</tt> would match, while <tt>foogw.cmu.net</tt> would NOT (due to the dot in the
pattern.)



If a new GW is located in NetReg, it is added to the device table in NetMon. Additionally, device_timing
entries are added for types ARP, Ping, and DevMAC.


<_header4>
SW_PATTERN
</_header4>


SW_PATTERN is similar to GW_PATTERN except that it looks for switches (or other non-gateway devices)
and adds them to NetReg. It uses the same pattern matching syntax as GW_PATTERN. When added, new 
devices have device_timing entries for CAM, Ping, and DevMAC.


<_header4>
IGNORE_LOAD
</_header4>


It pays to standardize on your naming conventions when using NetMon, because you can easily exclude
certain patterns of devices (for example, aliases for the same box). You can specify an array of
patterns to exclude. Unlike GW_PATTERN, you MUST include wildcard ("%") expressions where necessary.
For example, we might have <tt>foo.gw.cmu.net</tt> and <tt>foo-vlan34.gw.cmu.net</tt>. By having 
<tt>%-vlan%</tt> in our IGNORE_LOAD list, we do not create a separate entry for foo-vlan34 in NetMon.



Note that GW_PATTERN, SW_PATTERN, and IGNORE_LOAD are used ONLY while adding new devices from NetReg --
even name changes will not be verified by these rules. (See the NetMon Design, "Device and Device Timings")
section for more information.


<_header4>
PERM_DEACTIVATE
</_header4>


PERM_DEACTIVATE applies to all existing NetMon devices. It is similar to IGNORE_LOAD in syntax, ie
an array of patterns (wildcards required). Any devices matching these patterns will always have all
device_timings for the device set to "Deactivated" state (ie, no capturing will occur). We use this
to mark off devices that are unreachable from the main network.


<_header3>
Capture Parameters
</_header3>


NetMon has a general Capture engine (see the NetMon Design section for more detail). The engine is 
more or less continuously running (it dies if there is nothing to do, but cron re-starts it every
minute, or 5 minutes, or whatever you decide.) 



During each iteration, it looks to see how much work needs to be done (ie, how many devices haven't been
captured in their configured captured interval, and thus need to be touched). It also identifies how
many processes are already running, trying to touch the devices. It will double the number of
collection agents <em>of a given type</em> if any devices have not been touched in 1.3 times their
collection interval -- we call this "falling behind". The parameters here control some aspects of this
collecting. After it has taken care of this work, it sleeps for some period of time before re-checking
the state of the world.



Each individual capture agent (children of the master capture process) will loop until there is no
more work (of its designated capture type) doing the following:




<ul>
  <li>If there is work to do, get some chunk of the work (and mark the <tt>last_begin</tt> time to
 <tt>now()</tt> to indicate such to other agents.
  <li>For each device in the chunk, try capturing from it.
</ul>


<_header4>
TIMEOUT
</_header4>


If we start capturing from a device, but for some reason don't finish, then <tt>last_begin</tt> will
be some time AFTER <tt>last_end</tt>. Once the difference between <tt>last_begin</tt> and <tt>now()</tt>
exceeds $TIMEOUT*(capture interval for this device and type), then we'll declare that the previous
capture attempt failed, and we'll try again. We set this value to '5', and it's basically here to 
make sure that two agents don't collide. Normally this would never happen, but devices that are down
or otherwise unreachable take some time to timeout.



<_header4>
FORKNUM
</_header4>


If the master capture process determines a capture type has available work, it will spawn this many
processes to start working. We set the value to '3'. The goal is that this many processes can always
keep up with the work required (and thus, we never need to start doubling the active agents.) You
definitely should set this value to something other than '1', unless you have some reason for not
wanting parallel work -- the time required to transit the network, even a very fast <1ms transit
network means you are wasting CPU cycles unless you have a couple running in parallel. 



Note that we also spawn a separate process for handling "slow" devices -- ie devices that we've identified
as down or otherwise unreachable (this determination is done by the capture agents and automatically
updated as we re-obtain or re-lose reachability). That way, your main processes are mostly handling
fast, responsive devices, and we trudge along with one process verifying each of the slow devices.


<_header4>
MAXFORK
</_header4>


While the master process will double the number of agents of a particular type as needed, it will never
exceed the MAXFORK value (plus the extra "slow" agent). This is to prevent the master process from
running away and killing your system with dozens of capturing agents. We set this value to '12', which
seems like a reasonable value. If the entire NetMon system is paused for awhile (which we do once in
awhile during testing), and then restarted, we quickly hit the MAXFORK value. While the system is slow,
it still handles the load.


<_header4>
DEV_PER_RUN
</_header4>


Each capture agent will select a chunk of DEV_PER_RUN devices to begin work on during each iteration.
The goal here is to reduce database hits (by grabbing a few devices that need work instead of just
one-at-a-time). However, going too high may mean that you don't enjoy the benefits of parallel work,
and also means you can't stop the whole system quite as fast. The agents will only check for the 
existence of <tt>/etc/nonetmon</tt> once every iteration, and especially with the slow-device agent,
it may take a few minutes to get through a reasonably large list of devices. We set this value to '5'.


<_header4>
SPIN_SEC
</_header4>


The master capture process will sleep for SPIN_SEC seconds at the end of each iteration of checking the scheduled
work and active agents. Setting this for too low means you are doing a fair bit of extra database and
system work identifying the number of agents, while going too high means a slower reaction time to
changes in the environment (new work, falling behind, etc.). We use a value of '10' seconds.


<_header3>
DNS Search List
</_header3>

<_header4>
SEARCH_LIST
</_header4>


This list is used to translate the names assigned to routers and switches into valid hostnames.
(As retrieved by the SNMP OID <tt>.1.3.6.1.2.1.1.5.0</tt>.) Basically we recommend that you create this
list of any domains used by any of your network devices.


<_header3>
DB Connections
</_header3>


NetMon has a database separate from NetReg, but requires a connection to the NetReg database for 
much of the useful cross-linking. Additionally, certain components of NetMon are designed to be installed
on machines other than the NetMon machine (for example, the DHCP server), so we have multiple database
connection strings. The format of the DB connection strings is exactly the same as the <tt>NetReg_Web_DB</tt>
variables (as configured in the WebInt section).



Note that MySQL's connections across the network are not currently encrypted (though the password 
exchange is done via a challenge-response). We have tried strategies involving SSH tunnels, but these have
been extremely unreliable. MySQL 4.0.0 is supposed to include support for SSL connections, though as of
this writing, 4.0 is not in 'beta' stage. We recommend your user table on both the NetReg and NetMon
server allow specific users access from specific IPs only (or 'localhost' for local connections). This will
minimize any ability for malicious password guessing. Also, the <tt>netreg-install.pl</tt> script has
a routine for easily generating some random strings to use as passwords.


<_header4>
NetReg_DB
</_header4>


This needs to be a connection string that grants NetMon read access to the entire <tt>netdb</tt> database
on your NetReg server. Note this does not need to be read-write. Our default string is:<br>
<tt>['DBI:mysql:netdb:netreg.net.cmu.edu', 'netmon', 'file=/home/netreg/etc/.password-s=netreg-u=netmon']</tt>.




<_header4>
NetReg_RW_DB
</_header4>


This connection string is used to establish read-write access to the NetReg database. Wherever possible,
the read-only connection is used (to minimize potential exposure). If you do not want to allow read-write
access from NetMon, some features (such as subnet purges) will be unavailable, as they must update
NetReg records.


<_header4>
NetMon_DB
</_header4>


This is the standard connection string used for connecting to the NetMon database. It should grant
read-write access. See the Installation section above for an exact definition of what 'read-write' needs
to encompass.


<_header4>
NetMon_Rem_RW_DB
</_header4>


This is the connection string used by NetMon components when they are accessing the NetMon database
remotely (for example, the DHCP lease information loader).


<_header3>
Syslog Components
</_header3>


NetMon uses syslog to record a variety of log information. This section allows you to tune some of
the logging parameters. In general, you should run: <tt>perldoc Sys::Syslog</tt> for a description
of the various terms, and how they need to be specified.


<_header4>
SYSLOG_FAC
</_header4>


The default facility is 'daemon', but you may want to adjust this depending upon your syslog configuration.


<_header4>
SYSLOG_OPT
</_header4>


By default we specify 'pid' as the only syslog option; this means that the process ID of the process
originating the log message will also be logged.


<_header4>
SYSLOG_LOGSOCK
</_header4>


If your syslog server uses a unix domain socket to listen for new syslog entries, the logsock should
be 'unix'. Again, consult the man page (see 'setsockopt') for more details.


<_header3>
Process Components
</_header3>


The processing engine takes the raw captured data and processes it. During this, extraneous or duplicate
information is eliminated, other data is condensed by using "begin", "end" and "last updated" times (and
thus eliminating data that continues a series). The variables here affect the processing.


<_header4>
Process_Modules
</_header4>


The Process_Modules variables defines the set of capture_types that need to be processed. The format
is an associative array of capture_type keys to a reference to an array of three elements. The
keys are, by convention, lower case and prefixed by a dash. You should verify that no key is a
substring of another (ie '-foo' and '-foobar' would conflict.)
 




<ul> 
  <li>A reference to a function that conforms to the processing API and processes the selected capture type
  <li>The name of a lock that will be used to ensure mutual exclusion. By convention we use the upper case
form of the capture type prefixed by 'RUN_'
  <li>The processing interval. This is the minimum elapsed time between sucessive runs of the processing
engine. The format must conform to that suitable for use with MySQL. Generally speaking, the format is
the word 'interval ', followed by the quantity, followed by the units. The units are typically
the singular form of traditional time spans (minute, hour, day, etc.)
</ul>



Note that there is no requirement that only valid capture types be used as keys for the processing
engine -- as long as the requirements are met (a valid function, for example), you could have a
processing routine to make coffee. (Perhaps there would be a few other requirements in this case..)



<_header4>
ROOT_DEVICES
</_header4>


The network topology generation code needs to have some idea of the root of your network. The format
is a list of devices that should be put at the top of your automatically generated network topology.


<_header4>
LOCK_OVERWRITE
</_header4>


If the processing engine for any component does not finish in LOCK_OVERWRITE seconds, the master
processing engine will start a new process agent and overwrite any locks of the previous agent.


<_header4>
MAX_PROCESS
</_header4>


MAX_PROCESS specifies the maximum number of entries to process at a given
time. You should set this to be more than the approximate number of MACs on
all devices, or it will probably thrash around trying to run topology and what-not.
It's mainly to prevent Process from being choked if Process doesn't run
for some period of time and is trying to recover.



<_header4>
SaveCAMForTopology
</_header4>


This variable triggers the CAM processing engine to save certain MAC addresses (those belonging to 
network devices known via DevMACs) so that the topology information can be completely generated.
However, we recommend you set this to '0' unless you are actively using Topology (see the section
under "NetMon Design"). Otherwise you'll get a number of entries in <tt>cam_capture</tt> that don't
go away.


<_header3>
Capture Components
</_header3>

<_header4>
Capture_Components
</_header4>


The VARIABLE Capture_Components contains a list of all the various capture types and the associated
methods for invoking them. There is a standard API for capture routines. The variable is 
specified as an associative array of "key" to a reference to an array of capture type information.
The capture type information contains a reference to the function that processes that type of data,
a generalized name for the capture type, and a printable name for the capture type. The various
types should be formatted as:



<ul>
  <li><b>Key</b>: The Key is used while counting capture agents, so it should be unique and not exist as
a substring to any other key. (IE, if you had -foo, you would NOT also want to have -foobar, because
they could be incorrectly counted as the same.) The convention is to have the capture type prefixed
by a dash, all lowercase.
  <li><b>Function</b>: This should be a reference to the function that accepts the standard API for
capture routines.
  <li><b>General Name</b>: The convention is to use the lower-case capture type plus the string '_capture'.
  <li><b>Printable Name</b>: This should just be a human-readable, nicely formatted name, like "ARP".
</ul> 



<_header3>
Web Components
</_header3>

<_header4>
StatFile
</_header4>


We set this to <tt>$NRHOME/etc/statfile</tt>. The statfile contains some statistics on how current
capturing is going (for each capture type, how many processes are running, how many devices want 
attention, etc.)


<_header4>
MAX_PAGE
</_header4>


This should be the maximum number of database rows to present on a single page table. We use '20'.Th


<_header4>
NETREG_URL
</_header4>


This should be set to the full path to your <tt>netreg.pl</tt> script (ie, not a pre-page, but the 
actual NetReg script). This will be used to create links from NetMon direct to NetReg pages. For 
example, we use <tt>http://netreg.net.cmu.edu/bin/netreg.pl.</tt>


<_header4>
NETMON_SERVER
</_header4>


This should be set to base URL of the NetMon server (ie <tt>http://netmon.net.cmu.edu</tt>).


<_header4>
NETMON_AUTH_URL
</_header4>


NETMON_AUTH_URL should contain the complete URL to your protected NetMon pages. We allow only
administrators access to these pages. Our value is, for example, <tt>$NETMON_SERVER/bin/netmon.pl</tt>.


<_header4>
NETMON_UNAUTH_URL
</_header4>


NETMON_UNAUTH_URL should contain the complete URL to your public NetMon pages (ie the network
debugger). Our value is, for example, <tt>$NETMON_SERVER/pbin/netmon.pl</tt>. The only difference
between "bin" and "pbin" is that "pbin" does not require or attempt authentication, and <tt>netmon.pl</tt>
will recognize the difference and handle the situation accordingly.


<_header4>
TREE_URL
</_header4>


This should contain the complete URL to the Topology Tree graphing engine. By default you might want
to use <tt>$NETMON_SERVER/bin/main.pl?id=</tt>.


<_header4>
TOPOLOGY_HEADER
</_header4>


This contains the complete path to your header file for the topology generation. You probably want
to use our supplied header and footer unless otherwise desired. The default is <br>
<tt>$NRHOME/htdocs/tree/code-top.html</tt>.


<_header4>
TOPOLOGY_FOOTER
</_header4>


This is the complete path to the topology footer file. By default this is:
<br><tt>$NRHOME/htdocs/tree/code-bot.html</tt>.


<_header4>
TOPOLOGY_TREEFILE
</_header4>


The file generated from the topology information will be written to TOPOLOGY_TREEFILE. By default
this is <tt>$NRHOME/htdocs/tree/code.html</tt>.


<_header4>
READ_LEV
</_header4>


The READ_LEV should specify the level being used to indicate read-only access to NetMon. This value
should be lower than the RW_LEV or ADMIN_LEV. We use '3', for example. This does not need
to be changed unless you have a good reason for doing so.


<_header4>
RW_LEV
</_header4>


The RW_LEV should specify the level being used to indicate read-write access to NetMon. This value
should be higher than READ_LEV and lower than ADMIN_LEV. We use '6', for example. This does not
need to be changed unless you have a good reason for doing so.


<_header4>
ADMIN_LEV
</_header4>


The ADMIN_LEV should specify the level being used to indicate administrative access to NetMon. This
value should be higher than RW_LEV and READ_LEV. We use '9', for example. This does not
need to be changed unless you have a good reason for doing so.


<_header3>
Error Reporting
</_header3>

<_header4>
EmailAddress
</_header4>


Similar to NetReg's ADMIN_MAIL, this should be an address to receive NetMon error reports. We use
a different address than NetReg, for ease in identifying the source.


<_header4>
ReportInterval
</_header4>


To reduce the likelihood of NetMon falling off its rocker and spamming you with email, we implemented
a duplicate-message-type suppression system. The interval specified in ReportInterval (in seconds) will
be the minimum time between reporting similar type errors to the EmailAddress. The error reports
will still be logged to the NM_ERR_DIR below. We use a value of 3600 seconds (1 hour).


<_header4>
NM_ERR_DIR
</_header4>


The NetMon Error Directory will receive a copy of all error reports, even those suppressed due to
time intervals. We use <tt>$NRHOME/etc/netmon-errors</tt>.



<_header3>
DHCP Reporting
</_header3>


NetMon has a number of features related to DHCP management. For complete details, read the "NetMon
Design" section. None of these configuration bits are relevant unless you are using the DHCP lease
loading/graphing capabilities.


<_header4>
DHCPStaticLog
</_header4>


This variable specifies the location of your DHCP static log. As described in the "NetMon Design" section,
this is a custom logfile that records fixed-address assignment. This variable needs to be set correctly
on all DHCP servers being used to load lease information. We use <tt>/var/log/dhcpd.static</tt>, 
for example.


<_header4>
DHCPLeasesFile
</_header4>


The DHCPLeasesFile variable should point to the location of your DHCPD leases file. This is typically
in <tt>/etc/dhcpd.leases</tt>, but your location may be different. This is the standard DHCP leases
file from ISC DHCP 3.0. This variable must be set on all DHCP servers being used to load lease information.


<_header4>
DHCPConf
</_header4>


A copy of your dynamic DHCP configuration (see the NetReg sections for our definition of "dynamic"
versus "static") must reside on the NetMon server. Using NetReg, it's easy to add your NetMon server
as a recipient of a DHCP configuration (and adds very little overhead to DHCP configuration generation).
Please specify the complete path to the DHCP configuration. You might use
<br><tt>$NRHOME/etc/dhcp-xfer/dhcpd.conf.HOSTNAME</tt>.


<_header4>
DHCPGraphInterval
</_header4>


NetMon must translate DHCP leases (specified with a start and end time) into a series of data points
representing "X users had leases at this time". In order to do that, it steps through time and generates
data points at certain intervals. The DHCPGraphInterval variable specifies that interval (in seconds). We use
a value of 300 seconds. Since we added the graphing after we started collecting lease information,
the initial graphing run took several hours as it stepped through time passed.


<_header4>
RRDToolPath
</_header4>


This variable should contain the location of your <tt>rrdtool</tt> utility. For example, we have it
installed in <tt>/usr/local/bin/rrdtool</tt>. 


<_header4>
RRDDBPath
</_header4>


This variable should contain the directory that you want to use to store your RRD database of DHCP
utilization information. We use <tt>$NRHOME/rrd</tt>.


<_header4>
DHCPImgPath
</_header4>


This provides the absolute path on disk to the directory your DHCP graphs should be created in.
Probably want to use <tt>$NRHOME/htdocs/dyn-img</tt>.


<_header4>
DHCPImgURL
</_header4>


This provides the path <b>as web clients would see</b> to your DHCP graphs. We use <tt>/dyn-img</tt>.


<_header4>
DHCPImgPeriods
</_header4>


Utilization graphs for all dynamic subnets will be constructed based on the DHCPImgPeriods array. One
graph will be created for each entry in this array. Each entry indicates a number of days to cover in
the graph. If you want reasonable titles over one month, you probably want to look at 
<tt>pdhcp_gen_graphs</tt> in <tt>CMU/NetMon/Process/DHCP.pm</tt>. The default is: <tt>(7, 28)</tt>.


<!--
[variables]
{
  $page_title = "NetReg Design"
  $page_title_english = "NetReg Design"
  $file_prev = "004.html"
  $file_next = "006.html"
  $real_page_number = "5"
  $page_number = "5"
}
-->

<_header1>
$page_title
</_header1>


The database schema is available in the distribution 
(see <tt>doc/db/NETREG-COMPLETE.sql</tt>).



Overview



NetReg can be broken into three logical components: 
<ul>
  <li><b>Database Schema</b>: NetReg relies on the backend database for every
operation, and the schema supports a flexible protections mechanism. We tried
to keep the schema as simple as possible, to allow for rapid understanding
and modification as needed.

  <li><b>NetDB</b>: The CMU::Netdb modules provide an API into the database.
They implement the protections checking mechanisms and provide intricate 
verification and augmentation to data being added or updated.

  <li><b>WebInt</b>: The CMU::WebInt modules form the web interface to the
database. The web interface uses the CMU::Netdb modules for most database
accesses.

</ul>




This design provides logical delineation points. The Netdb modules, for
example, know absolutely nothing about HTML or web pages. At the same time,
the WebInt module foists responsibility to the Netdb components for most
data verification.




Note that NetReg was designed on top of a MySQL database. MySQL does not
support foreign keys, so NetDB functions provide most of the foreign key
validation. For this reason, then, you should be <b>very, very careful</b>
about updating records in the database directly. For example, if you were to
change the IP address of a machine record, the subnet reference in the
record may need to be updated.




Users and Groups



NetReg is designed with a fairly standard concept of users and groups. 
Every user that will use NetReg must have an entry in the users table. While
we don't use realms, it would be fairly straightforward to support them. 
UserIDs are authenticated by standard authentication/authorization modules
in Apache, and Apache passes the UserID via the REMOTE_USER environment 
variable. The function <tt>CMU::WebInt::helper::getUserInfo</tt> is called to 
retrieve and parse the UserID. In theory, no other components should use
REMOTE_USER directly (and this is true of NetReg in the current state).




There are a few special users and groups. Our sample database comes with these
users and groups pre-configured to work correctly, but you should take note.
First, the <tt>netreg</tt> user is special. You <b>must</b> grant 'netreg'
full, level 9 read/write access to every record of every table.
(See Section 6.1.3. for more information.) In several places, we assume
'netreg' has this access to force certain queries or operations to succeed
regardless of protections entries. In practice, all of these instances are
safe: the system has already determined the user is authorized to perform
the operations, and we want to belay any further checking.




Because of the special nature of the 'netreg' user, this user is not
actually allowed to use the web interface to the database. This
is enforced in <tt>CMU::WebInt::helper::db_connect</tt>. We recommend
not removing this.




The <tt>system:anyuser</tt> group is not actually a group inasmuch as it
does not appear in the <tt>groups</tt> table. However, it is interpreted by
Netdb to represent the group of all users.




The <tt>netreg:admins</tt> group is distributed in our sample database,
however you could easily rename it. In the sample data, members of this group
form the elitist group of NetReg users: they have universal access to all
sections of the database.



In general, groups take the form of a prefix, a colon, and a name. We use
prefixes to logically determine the purpose of a group, and make it consistent
with other groups with the same prefix. The current prefixes in use are:



<_default>
<table border=1>
<tr>
  <th>netreg:</th>
  <td>These groups indicate elevated database access. We use a
<tt>netreg:fullread</tt> group, for example, to grant Help Desk users access
to view, but not change, all machine registrations.
  </td>
</tr>

<tr>
  <th>dept:</th>
  <td>Every machine in NetReg must have one (and only one) department
associated with it. Members of the Department group then have the ability to
view and update the registrations associated with the department.
  </td>
</tr>

<tr>
  <th>local:</th>
  <td>We are using the 'local' prefix to define groups of users with a
similar affiliation. In practice, we will have a 'dept:foo' group to define
the department administrators for the 'foo' department, and a 'local:foo'
group to define the actual users associated with that department. The 'local'
groups will be used to restrict registration privileges on certain domains
or subnets. The 'local' groups will not have rights on the actual machine
registrations, however. We chose the 'local' prefix because the source of 
data about users in certain groups will come from a simple filesystem
based design.
  </td>
</tr>

<tr>
  <th>ldap:</th>
  <td>The 'ldap' prefix will be used exactly the same as the 'local' prefix,
except the source of data will be our internal LDAP directory.
  </td>
</tr>
</table>

</_default>


Protections



NetReg boasts a flexible and extendable protections system. It can be frustra
ting at
times, because you will sometimes receive error messages of the sort "X doesn't 
exist",
when in fact X exists, but you don't have permission to use X (even if you are t
he
administrator). The protections system was designed to give maximum flexibility 
in
allowing users to add and update machines, while giving administrators finely gr
ained
controls over what domains, subnets, and so forth users are allowed to register 
machines
on.




All protections are controlled from the 'protections' table in NetReg. We will r
efer to
each line in this table as one protections entry. Each entry contains:
<ul>
  <li>Table: The NetReg table this record applies to.
  <li>Table ID: The ID in the specified table that this record applies to.
  <li>Identity: The User or Group this entry gives permissions to.
  <li>Level: The level of access granted the user or group on the Table/Table ID
.
  <li>Rights: The rights given the user or group on the Table/Table ID.
</ul>




The <tt>Table</tt> and <tt>Table ID</tt> fields specify the table name and ID wi
thin
that table that this entry refers to. If the Table ID field is '0', the entry ap
plies
to all records of that table.




The <tt>Identity</tt> field specifies the user or group that will be granted rig
hts to
the specified table and table ID. If the Identity field is positive, it refers t
o a 
User ID (users.id). If the field is negative, it refers to the (positive) Group 
ID 
(groups.id). If the field is '0', it is a special case and refers to all users.




The <tt>Level</tt> field specifies the level of access given to the user(s) on t
he
specifies table. Most average user permissions are granted at level 1. Administr
ative
privileges are generally granted at level 9. Department administrators (those in
a group with prefix 'dept:') are given additional privileges at level 5. The lev
el
of privileges works hand-in-hand with the rights (see below). If a user has leve
l 1
WRITE access to a machine, for example, they will be allowed to update basic inf
ormation,
like the MAC Address and Mode of the machine. However, users with level 9 WRITE 
access
will be allowed to change other information, such as the Abuse/Suspended status 
of the
machine, and so forth.




The <tt>Rights</tt> field specifies the rights granted to the specified users, a
t the
specified level, to the specified table/table ID. The valid rights are READ, WRI
TE,
and ADD. The READ and WRITE permissions are fairly self-explanatory: if you have
 
READ access to an individual record, or to the entire table, you will be able to
 read
those records at the specified access level. The WRITE permission operates the s
ame
way. 



The ADD permission has different meanings depending upon the table ID. For a 
table ID of '0' (ie "entire table"), the ADD permission grants you the ability t
o 
add new records to this table. However, with a non-zero table ID, the "ADD" perm
ission
grants you the ability to use the specified record as a reference in another tab
le. 
For example, the ADD permission on record 24 of the "subnets" table, for identit
y 0
(ie all users) enables all users to add machines to subnet #24.




Note that the permissions system is entirely up to the DB interface components
(the <tt>CMU::Netdb</tt> and <tt>CMU::WebInt</tt> modules) to enforce. In some
cases, protections are massaged a bit. For example, the ability to add DHCP
options and DNS resources to a machine doesn't require the ADD permission
on the machine (only WRITE). We would like to document all of these individual
cases, but do not have such documentation now. Doc patches are welcome. :)




The system <b>is</b> designed, however, to make protections implementation
fairly straightforward. The list, get, add, modify, and delete functions in
<tt>CMU::Netdb::primitives</tt> are used by most Netdb API functions and
implement protections checking at that level.




Note that by design, the DBMS is not performing individual user
authorization checking. The actual "user" seen by the DBMS is a single
user/password configured. Basically, the implication is: make sure the
machine hosting the web server is secure. You should see our section on
NetReg Security for more information.




DNS Zones and Resources


<_header3>
DNS Zone Types
</_header3>


There are six different types of zones represented in NetReg. All have the prefix 
designator 'fw-' or 'rv-'. The former indicates the zone is a forward zone (ie EXAMPLE.ORG, etc)
while the latter indicates a reverse zone (ie 0.128.10.IN-ADDR.ARPA, the zone for reverse
lookups of 10.128.0.x IP addresses.)



Within each prefix, there are three suffixes: 'toplevel', 'delegated', or 'permissible'. While NetReg does
not strictly enforce where you can have these different suffixes, you should follow these rules for
setting up the zones properly: 



<ul>
  <li>Any zone for which you want a zonefile to be generated should be "toplevel".
  <li>Any zone that you do not want a zonefile (ie, the zone is either a sub-domain of a "toplevel" zone, or you otherwise don't control), should be "permissible".
  <li>If you delegate authority for a zone <b>and you control the parent zone</b>, then the zone should be "delegated".
</ul>



A quick example should lend some clarity. We start at the top of the DNS system.



<ul>
  <li>Under the root zone (".") you have Top Level Domains (TLDs). These include ORG, NET, EDU, COM, MIL, GOV, a host of country codes, and new TLDs such as AERO, BIZ, MUSEUM, etc. These should all be "fw-permissible", because you don't control them (or if you do, we'd be pleasantly surprised!)
  <li>Under the TLDs are the 2LDs, such as EXAMPLE.ORG, CMU.EDU, etc. Assuming the zone is properly delegated to you, these zones should be "fw-toplevel".
  <li>Under your 2LD, you may have different 3LDs. For example, we have "ANDREW.CMU.EDU" and "CS.CMU.EDU". We control ANDREW, but delegate CS to the Computer Science
Department. But, we don't want the ANDREW records in the CMU.EDU zone, to limit the number of changes to CMU.EDU. Thus ANDREW is fw-toplevel, and CS is fw-delegated.
  <li>Under ANDREW.CMU.EDU, we also have the CYERT.ANDREW.CMU.EDU zone. However, while we want to be able to register
machines in CYERT.ANDREW.CMU.EDU, we don't care to have a separate zonefile -- the CYERT machines can just appear in the ANDREW
zone. Thus, CYERT is "fw-permissible".
</ul>
  


<_header3>
Adding DNS Zones
</_header3>


Adding a second-level domain (ie EXAMPLE.ORG) requires the parent zone (ie ORG) to exist. The default
data supplied contains the top level domains (TLDs): ORG, NET, COM, EDU. If you need to add
2LDs in other TLDs, you will need to manually add the TLD as follows. Connect to your DBMS
(ie "mysql -u root -p netdb", and enter your root password). Then enter:




<_code>
insert into dns_zone (name, type, parent) VALUES ('US', 'fw-permissible', 0);

</_code>


Substitute the TLD you wish to add for "US" in the previous example. Similarly, you will
need to manually add any missing zones 



DNS Configurations


<_header3>
Delegation Notes
</_header3>


If you are running ISC BIND 8 or earlier, we recommend that you
do not setup a name system that involves delegation from one
zone to another, where both zones are served by the same nameserver.



Prior to BIND 9, there were some issues involving leakage of zone
information between parent and child zones. This could cause incorrect
responses, and thus our recommendation. We have a small group of BIND 9
servers for DDNS, but mostly run BIND 8. Thus, we have a group of
servers for CMU.EDU and other 2LDs, and another group for
ANDREW.CMU.EDU and other 3LDs.



DNS Configuration is controlled through service groups: specifically, the <tt>DNS View Definition</tt>
and <tt>DNS Server Group</tt> groups.


<_header3>
DNS View Definition
</_header3>


The DNS View Definition groups are used to specify the parameters of a specific view. Note that the
name of the service group is <b>not</b> the name of the view as it will be written in DNS configurations.
The attributes of the service are:
<ul>
  <li>Server Version</li>: This attribute specifies the version of the software that this view 
definition applies to. You can have multiple view groups attached as members to a DNS Server Group
(see below) with the same view name. The options in the View Group will only be included on servers
with a version matching this parameter.
  <li>DNS Paramater</li>: This attribute allows you to specify free-form text in the view definition.
</ul>



<_header3>
DNS Server Group
</_header3>


The DNS Server Group groups are used to specify common settings for a group of zones. The members
of the group are:



<_header4>
Zones
</_header4>


The zones that are part of this group are added as members. The attributes of the zone membership
are:
<ul>
  <li>Zone In View</li>: Specify a particular view that the zone should be included in. If no
views are specified, the zone is included in all views.
  <li>Zone Parameter</li>: You can specify a free-form <tt>named</tt> configuration parameter to
be added in the <tt>zone {}</tt> block. There is <b>no</b> correctness checking done to these
parameters (although the backend servers can validate it prior to reloading the nameserver with
<tt>named-checkconf</tt>.
</ul>


<_header4>
Machines
</_header4>


The machines that are members of service groups are the DNS servers for the zones in the group. The 
attributes of these service members are:
<ul>
  <li>Server Version</li>:
  <li>Server Type</li>: This parameter specifies whether the server is a <tt>master</tt> or a
<tt>slave</tt> for the zones in this 
</ul>


<_header4>
Views
</_header4>


Other service groups (specifically the DNS View Groups) can be members of this group as views. The
view name is specified by the <tt>Service View Name</tt> parameter. "_default_" refers to the
global namespace (see below). The <tt>Service View Order</tt> parameter allows you to control the
order in which the views are written to the configuration. This is important, because clients will
be presented with the data in the first view that matches (not an aggregation of views they can see,
or other metric). The order field is specified as an integer, where '1' is the lowest or "first" 
view. You could specify the order of two views as '10' and '50', and the view of order '10' will be
printed in the configuration first. The behavior of the DNS configuration script with equal orders
(ie 10 and 10) is <b>not guaranteed</b>, and you should avoid this case.



Note that you can have a DNS server referenced in multiple server groups -- it might be the master
for a set of zones, the slave for others (with specific view configurations), and the slave for yet
another set, with other views. The configuration output script will deal intelligently with this
situation. In the case of a server referenced in multiple server groups, common view names are
combined -- that is, if you have an 'internal' view in one server group, and 'internal' in another,
they are treated as being identical.



The special view name <tt>_default_</tt> refers to the global option space. For example, our sample data
includes special views "view.default.bind8" and "view.default.bind9". When added as members of the
service group with name <tt>_default_</tt>, the options specified in these views (the "view.default.bind8" view),
the parameters are included in the global space.



Subnets and Networks



A core component of NetReg is the definition and configuration of subnets.
Specifically, the subnet table stores such information as the subnet's
base address, network mask, name, abbreviation, and several other related
pieces of information. 




While the system does not prevent configuring two overlapping subnets (due
to issues involved with moving subnets), you should <b>never</b> configure
overlapping subnets.




Users will not be allowed to register any machines on a particular subnet
(indeed, they won't even <em>see</em> the subnet) unless they are given
ADD permission (level 1) on the subnet. Even system administrators must
be given ADD access. Once they begin registration on a subnet, they will
only be allowed to register machines in the DNS zones you specify on the
subnet screen. They must also have ADD access on the DNS zone.
(One exception: If a machine is registered in zone FOO.ORG, and FOO.ORG does
not exist as a valid domain on the subnet, users will still be allowed to
update other information, such as the MAC address, of this machine, while
maintaining the FOO.ORG name. Indeed, WebInt does not actually give them
the list of valid zones in this case.




At the bottom of the subnet information page in WebInt is a list of DHCP
option for the subnet. Some common DHCP options you will need:
<ul>
  <li><tt>option routers</tt>: Specifies the IP address of the default gateway
 of this subnet.
  <li><tt>option broadcast-address</tt>: Specifies the broadcast address of
the subnet.
  <li><tt>option subnet-mask</tt>: Specifies the subnet mask handed out to 
DHCP clients.
</ul>





In reality, the only piece of information that cannot be gleaned from the
subnet configuration is the router address, and even that tends to be
fairly standardized. One notion has been to add a WebInt function that 
automatically adds these common options.



The "networks" concept is redundant, actually. With hundreds of subnets here,
including a few that were not entirely contained in a single building (our
campus-wide Wireless infrastructure, and DSL connections, for example), the
concept of "special" subnets was born. When registering a machine, users
begin by selecting either a building, a subnet, or a network. If a network
is selected, the system does the appropriate lookup to determine the subnet,
and proceeds with registration on that subnet. Every network represents one,
and only one, subnet. (Although a subnet could be aliased to multiple network
names.)



<_header3>
Shared Subnets
</_header3>


Shared subnet declarations were initially used as we transitioned from a mostly bridged network
to our currently mostly subnetted network. As the configuration was somewhat fixed by the DHCP
configuration generation script, this area of NetReg is not completely functional at this time.



Machines



While NetReg is certainly useful as a network management tool, much of the design is to facilitate 
end-user registration of machines (with proper restrictions). You must also register machines that will
be used as DNS servers.



Registration of machines is accomplished from the first page of NetReg (by selecting "Register New Machine").
The next screen requests you select the Network, Building, or Subnet to register this machine on. Each
machine is registered on one and only one subnet. Users may only register new
machines on subnets they have ADD access to.



After selecting the subnet, normal users can select
the hostname (and the domain name from a restricted list, based on the users ADD access to the zones
and the zone attached to the subnet, via the subnet information page.) Normal users must also select
the mode ("static" or "dynamic"), the MAC Address of the machine, and the department to which the 
machine is affiliated. Once the information is selected, the machine is registered. Machine registration
is very complicated; if you are planning on modifying this section of code in any way, we
recommend consulting with us first.



Administrative users have additional options (administrative defined as having level 9 ADD access to
the entire machine table (tid == 0)). They can specify additional modes: "reserved", "base", "broadcast",
or "pool". The modes are defined as:




<ul>
  <li><b>static</b>: The basic mode for registering a machine. The IP address is assigned by NetReg
at registration time (unless specified by an admin). All static registrations must have a hostname,
MAC Address, and IP address.
  <li><b>dynamic</b>: The basic mode for machines on subnets with IP address pools and obtaining a
dynamic IP address (as assigned by the DHCP server). All registrations must have a hostname 
(note: this is a recent change, not enforced yet) and MAC address. The IP Address will always be '0'
(although updating as an Administrator may require you to clear the IP address field first.)
  <li><b>reserved</b>: The 'reserved' mode is a mode for cases when the administrator wants a partial
registration for whatever reason. There is no restriction on IP, MAC Address, or Hostname existence.
However, as with all registrations, under <b>no circumstances</b> can you have duplicate IP addresses.
(See ANAMEs under DNS Resources if you think you need duplicates.) The IP, MAC, and Hostname fields
are treated like other registrations for purposes of duplicate suppression (thus, no duplicate IPs,
no duplicate hostnames, and no duplicate MAC address on the same subnet.)
  <li><b>base</b>: An registration for the base address of a subnet should be set to the "base" mode.
MAC Address is not required; IP and Hostname <b>are</b>. These will be handled slightly differently
in DHCP and DNS generation.
  <li><b>broadcast</b>: Similar to 'base', this mode is for addresses that are the broadcast addresses
for a subnet.
  <li><b>pool</b>: Any address defined as a 'pool' address is added to the DHCP configuration file
as a valid pool address for the subnet. If the subnet mode is 'restrict', only registered machines
with mode 'dynamic' will be allowed to obtain one of these pool address. If the subnet mode is 'permit',
ANY machine (registered or not) will be allowed to obtain one of these pool addresses. Pool addresses
must have a hostname (for legacy purposes). If you need/want to add a number of pool addresses,
see the "Support Utilities" section below.



Administrators can also set the DNS TTL value for the hostname and IP address (separately), as well as 
update a comment field visible only to them, and/or change the flags. The current machine flags are:
<ul>
  <li><b>abuse</b>: This is a flag to indicate the machine has been involved in an abuse incident.
When marked "abuse", the owner of the machine cannot update or delete the registration. They <b>cannot</b>
see the flag, however they will receive a general "permission denied" message when trying to perform
these operations.
  <li><b>suspend</b>: When set, this flag causes the machine to stop propogating to the DHCP and 
DNS servers. Generally this is used in conjunction with "abuse", but we could think of situations
where you would want control over them individually.
  <li><b>stolen</b>: When set, any attempt to register a machine with the same MAC Address will cause
an alert message to be sent to the administrators (the registration will fail with a "duplicate
registration" message).
</ul>


<_header3>
DNS Options
</_header3>


The DNS options area of the machine information page allows users and administrators to add additional
DNS records related to the machine (similar to zone options). To add a particular resource you must have
ADD access on the specific DNS record type. There are additional restrictions, as described here. Each
record can have a name, rname, TTL, 2 metrics, a port, and/or two text fields. Unless otherwise 
indicated, you can have multiple DNS records of the same type for the same resource.



<ul>
  <li><b>CNAME</b>: A CNAME provides a method for binding a hostname to another host at the time of the
DNS request. The <tt>name</tt> field is the alias hostname (ie WWW.EXAMPLE.ORG) while the 
<tt>rname</tt> is the real
name of the machine (and CNAMEs will appear on the information page of the <tt>rname</tt> machine.) 
Note that a single RR name cannot contain any type other than a CNAME, if one exists.
Thus, if you add a CNAME to a zone, the actual record generated will be an "A" record (the standard
hostname-to-IP-address record.) Thus CNAMEs are valid on machine, zone, and service records.

  <li><b>MX</b>: The mail exchanger record is valid for machines, zones, and service records. The
<tt>name</tt> field contains the record to apply the MX to (ie EXAMPLE.ORG), while the 'rname'
field indicates the mail exchanger for this record, and <tt>rmetric0</tt> supplies the metric
weighting (metrics are entirely dependent upon other metrics for the RR to have any meaning.) MX
records appear on the information page related to the <tt>name</tt> field.
 
  <li><b>NS</b>: Nameserver records create a delegation point for zones. They are ONLY applicable
to zones, and will appear on the information page of the child zone (though the parent zone will
identify the NS records which need to exist there). The <tt>name</tt> field is the zone, while
<tt>rname</tt> indicates the nameserver that is authoritative for the zone. 

  <li><b>TXT</b>: Text records allow for free-form text to be added to a zone. <tt>name</tt> indicates
the machine, zone, or service owning the text record, while <tt>text0</tt> stores the free-form text.

  <li><b>HINFO</b>: The host-info record is similar in nature to the TXT record.
  
  <li><b>ANAME</b>: The ANAME record is a creation of our own, and is used to deal with situations that
you want multiple hostnames to refer to the same IP address. For example, if you have a registered
machine FOO.EXAMPLE.ORG and want BAR.EXAMPLE.ORG to have the same IP address as FOO, then you would
add an ANAME to FOO. The <tt>name</tt> field would be BAR.EXAMPLE.ORG, while the <tt>rname</tt> field
would be FOO.EXAMPLE.ORG. When the zonefile is being written, ANAME records will be looked up by 
the name field, and an "A" record will be added from the <tt>name</tt> field to the IP address of
the machine in the <tt>rname</tt> field. You can also use ANAMEs to have simple round-robin load
balancing. For example, you could have FOO.EXAMPLE.ORG and BAR.EXAMPLE.ORG, and add ANAMEs to both
to WWW.EXAMPLE.ORG. Then WWW.EXAMPLE.ORG would be generated with two "A" records (thus 2 IP addresses).
You can even have an ANAME to the name of a pre-existing registration. In this case, you could have
WWW.EXAMPLE.ORG registered, and add a registration of BAR.EXAMPLE.ORG with an ANAME to WWW.EXAMPLE.ORG.

  <li><b>SRV</b>: We designed the system to support SRV records, but are not using any currently,
so no documentation is available.
</ul>


<_header3>
DHCP Options
</_header3>


See the section below "Machine DHCP Options" for more information.



DHCP Options



DHCP Options can apply to three scopes: "global", "machine", or "subnet".
Each is described here.


<_header3>
General Notes
</_header3>


Some DHCP Options must be enclosed in quotation marks. Otherwise you will
end up with a broken or non-functional DHCP configuration file. We would like to make
the handling of these options better in the long run (not require quotations in the
database, but add them at generation time.)


<_header3>
Global DHCP Options
</_header3>


Global DHCP Options are added to the global section of the DHCP
configuration file. We have tried to reduce the number of options
added directly via the configuration file generation script, so that
everything is controlled via the NetReg database/interface.



The standard database data provided contains a number of example 
global DHCP options. You can view and modify the options by select
the "DHCP" administrative link. The options included are:




<ul>
<li><tt>option domain-name</tt>: This is the default domain-name passed to clients,
in the absence of another. We <b>strongly</b> recommend this domain NOT be a zone that is
DDNS-updatable. (See below.)
<b>Note</b> that you must include quotations around the domain name.
<li><tt>option ntp-servers</tt>: This option provides clients with your Network Time
Protocol servers, if you have any. It also demonstrates that you can specify
multiple servers separated by commas (as the DHCP configuration will accept.)
<li><tt>option netbios-name-servers</tt>: This option is used by Windows clients to point at
a valid WINS server. See the ISC DHCP man pages for more information.
<li><tt>option netbios-node-type</tt>: A setting of '2' for the node type is the standard
we use to point clients at our WINS servers.
<li><tt>option local-printer-cfg</tt>: Recent versions of the ISC DHCP server have changed the method
of specifying vendor-specific options. Long term we would like to get DHCP options cleaned up
a bit (enabling vendor classes, etc. from NetReg). In the short term, in order to support printer configurations
we use the 'option local-printer-cfg', which maps to option 144. The mapping is done here in the global scope
by adding a value "code 144 = text" to the name.
<li><tt>next-server</tt>: The Next Server is used to specify the TFTP host for client images.
<li><tt>ddns-update-style</tt>: You should set the update style to "interim"; the DHCP configuration contains
more details but this is <b>probably</b> the style you want to use.
<ul>



The reason that we recommend the default domain-name not be dynamically
updatable is due to our policy of not accepting client-supplied hostname information.
Windows supplies the "client requested hostname" as that name which is given to the computer. In
several cases, users have selected names such as "Bob's Machine", and Windows supplies this hostname,
spaces and punctuation included. Thus, we recommend the default domain-name be a zone that is
not allowed to be dynamically updatable. If a client, through the proper registration mechanisms,
is to be assigned a particular hostname, then the domain-name will be specified for the client
explicitly in the DHCP configuration (See the section below under DHCP zone file generation.)


<_header3>
Subnet DHCP Options
</_header3>


Subnets form a large basis of the DHCP configuration, and logically have options specific to 
all machines within them. See the section above ("Subnets and Networks") for the typical DHCP options
needed. Additional DHCP options for a subnet can be added on the subnet information page, and are
passed into the DHCP configuration file verbatim.


<_header3>
Machine DHCP Options
</_header3>


Options given for a machine will be added to the DHCP configuration and apply to the machine only.
Some typical options include specifying a filename for booting; this option is commonly used by
printers and network devices for startup configurations/system images. Note that you also probably
want to specify the <tt>next-server</tt> option globally or for the machine, unless the DHCP server
is running an appropriate server for the files. (Though this is outside the scope of this document, we use
a next-server and filename options successfully here.)



QuickReg



QuickReg is a new feature as of the 1.1.4 release of the package. This feature enables virtually
effortless registration. The benefits in a University environment (such as residential network) are
substantial.



The design of the QuickReg system is the following: For each "real" IP subnet, the administrator defines
a secondary "registration" subnet, as well as a new subnet share declaration (in NetReg). The subnet
share is attached to both the real and registration subnet. The registration subnet is populated with
a number of dynamic addresses (using the Bulk IP Registration feature and a mode of 'pool'). The dynamic
restriction on the subnet is 'permit', meaning that anyone can get an IP lease on the registration subnet.



The registration subnet might contain DNS settings that point to a nameserver which responds with only
a single IP address for any query (thus directing all traffic at the NetReg server; even traffic for
<tt>www.yahoo.com</tt>. The registration subnet is most likely also ACLed to prevent off-campus
communication of any form.



When an unregistered machine comes up on the wire, they will receive a lease in the registration subnet
and be directed to NetReg. Upon entering NetReg, the system detects they are coming in from a registration
subnet. It then queries the DHCP server (using OMAPI) to find the MAC address of the machine.



Once the MAC address and real subnet are determined, the user is presented with a short information
screen and taken to a registration page. The user may optionally select a hostname, but the domain name,
MAC address, and subnet information are pre-selected. Clicking "Continue" will cause a registration to be
entered.



After registration is complete, the system takes some time to propogate the new information to DHCP
and DNS. Clients are instructed to restart after 30 minutes. Assuming all is correct, the DHCP servers
now return a lease in the real subnet.


<_header3>
QuickReg Subnet
</_header3>


An excerpt from the mailing list, where Kevin at CMU describes their QuickReg setup (in this case, a 
test setup):



Real subnet: 128.2.225.0/24, appropriate ADD access for netreg:admins, default registration mode: static, dynamic allow: restrict. There are 50 pool addresses in this subnet, but they aren't needed with static registrations. The DHCP options are just the standard ones you get if you use the "standard options" link on the subnet page. There was an appropriate domain allowed on the subnet.




Fake subnet: 192.168.12.0/24, protections had ADD for netreg:admins but not needed, default registration mode: dynamic, dynamic allow: permit. There are 50 pool addresses here (defined via the Bulk IP Registration interface, mode "pool"). The DHCP options are standard.



Both subnets were in a single subnet share. The config file bits I got were:



<_code>
shared-network shared-8 {<br>
&nbsp;        option domain-name-servers 128.2.32.37, 128.2.4.21, 128.2.64.2;<br>
&nbsp;        # Sysdev QuickReg test<br>
&nbsp;        subnet 128.2.225.0 netmask 255.255.255.0 {<br>
&nbsp;&nbsp;                max-lease-time 86400;<br>
&nbsp;&nbsp;                default-lease-time 86400;<br>
<br>
&nbsp;&nbsp;                option subnet-mask 255.255.255.0;<br>
&nbsp;&nbsp;     option broadcast-address 128.2.225.255;<br>
&nbsp;&nbsp;                option routers 128.2.225.1;<br>
&nbsp;&nbsp;                option domain-name-servers 128.2.64.2, 128.2.32.37, 128.2.4.21;<br>
&nbsp;&nbsp;                pool {<br>
&nbsp;&nbsp;&nbsp;                        failover peer "dhcp.net.cmu.edu";<br>
&nbsp;&nbsp;&nbsp;                        max-lease-time 7200;<br>
&nbsp;&nbsp;&nbsp;                        allow members of "quikreg";<br>
&nbsp;&nbsp;&nbsp;                        range 128.2.225.4 128.2.225.53;<br>
&nbsp;&nbsp;                }<br>
&nbsp;        }<br>
&nbsp;        # Sysdev QuickReg Test NonGlobal<br>
&nbsp;        subnet 192.168.12.0 netmask 255.255.255.0 {<br>
&nbsp;&nbsp;                max-lease-time 86400;<br>
&nbsp;&nbsp;                default-lease-time 86400;<br>
<br>
&nbsp;&nbsp;                option broadcast-address 192.168.12.255;<br>
&nbsp;&nbsp;                option routers 192.168.12.1;<br>
&nbsp;&nbsp;                option subnet-mask 255.255.255.0;<br>
&nbsp;&nbsp;                option domain-name-servers 128.2.32.37, 128.2.4.21, 128.2.64.2;<br>
&nbsp;&nbsp;                pool {<br>
&nbsp;&nbsp;&nbsp;                        failover peer "dhcp.net.cmu.edu";<br>
&nbsp;&nbsp;&nbsp;                        max-lease-time 300;<br>
&nbsp;&nbsp;&nbsp;                        deny dynamic bootp clients;<br>
&nbsp;&nbsp;&nbsp;                        range 192.168.12.4 192.168.12.53;<br>
&nbsp;&nbsp;                }<br>
&nbsp;        }<br>
}<br>

</_code>


Bringing up an unregistered machine, I went to NetReg (DNS games could be played, but I have no doubt that works regardless of this). Using the 'machine_unreg' QUICKREG_METHOD it pops up some verbiage, you can hit "continue", and get the machine registration page.


<_header3>
Configuration Variables
</_header3>


CMU/WebInt/config.pm contains some QuickReg configuration variables. If you have the OMAPI::DHCP 
Perl module (check the mailing list if you can't find it), you can define $HAVE_OMAPI_MODULE = 1, which
will make the MAC address lookups slightly faster. Setting it to '0' means it will use the 
<tt>omshell</tt> utility.



The $QUICKREG_METHOD variable can be either 'no_regs' or 'machine_unreg'. The former will present
the user with the QuickReg form only when the user has <b>no</b> registrations. The latter will
present the form when the current machine's MAC address is unregistered.



Buildings



While in the network realm we tend to think of subnets to logically divide our network, we assumed that
our users would first tend to think of networking logically by building. Thus, buildings are a key
component of NetReg.



When adding a building, you can then specify the subnets that exist in that building. When users go to
register a machine, three options are presented for finding the correct subnet: users can choose the
Network (See "Subnets and Networks"), the building, or the subnet. If they select the building and
only one subnet exists in the building, the system then presents the registration page for that subnet.
Otherwise (if there are more than one subnets in the building), the users can then select from the
shorter list of subnets in the building.



Buildings are also used for our outlet activation (See "Cables, Outlets, Activations").



Cables, Outlets, Activations




The cables, outlets, and activations areas of NetReg are used at Carnegie Mellon to automate the process of outlet activation. 

We have two basic types of outlets: those that are pre-connected to a network switch and just need to have the switch port enabled, and 
those that are terminated in a closet but not connected to an access layer device. The system supports the registration
of both types of these outlets (each "outlet" is connected to a "cable" in NetReg-speak). The Activations section
provides the technicians that need to physically connect an outlet with a list of pending
activation requests.



This section of NetReg is probably the most CMU-specific, as our
cable labelling strategy is most likely very different from most.

For this and other reasons, the default setting in the configuration file 
is to disable these areas. If you want to use them, visit the <tt>CMU::WebInt::config</tt>
file. We'd be interested in getting feedback if you are interested in trying.



Services



Services are a relatively new and interesting component of NetReg. We found
that several projects now and in the future will require certain data sources.
The data sources, for most part, can be described in terms of groups of
machines, subnets, or zones, and various attributes. One possible
implementation would be to extend NetReg via additional database tables with
the required fields for each new project. However, this would be unnecessarily
complex when a generic solution could be identified (and, in retrospect,
the DNS configuration portion could be redesigned as a service).




Thus, services were designed. Currently only groups of machines are
supported, but we'd like to extend this to include groups of subnets, zones,
etc. First, you must create a service template. Here, you name the service
and define the attributes.



<_header3>
Service Attributes
</_header3>


Attributes are where services become very flexible. You define the attribute
schema, and it pertains directly to the service template. When defining an
attribute specification, you specify:
<ul>
  <li>The attribute name and description (separately).
  <li>The scope of the attribute. For example, 'service' or 'service_membership'. 'Service' attributes apply to the entire service, while 'service_membership' attributes apply to individual members of services (ie, machines.)
  <li>The maximum number of times it can appear, within the scope. For example,
if you set the maxtimes to '1' for a service_membership attribute, it can
appear only once (0 or 1 times) for each machine in the service.
  <li>The attribute format. Here, you can define the exact format that the
attribute data must follow. This is very similar to defining the format of a 
database column, but is a meta-format (in that NetReg enforces the formatting
and stores the data raw in the database). We support a number of formats,
and it would be fairly easy to add new formats:
    <ul>
      <li>int: The attribute must be an integer.
      <li>uint: The attribute must be an unsigned integer. In practice, negative integers will be converted to their absolute value. If the data does not conform to an integer format, though, it will be rejected.
      <li>stringXXX: The data must be a string of length XXX or shorter (so you specify the format as <tt>string64</tt> and a maximum length of 64 chars is enforced.)
      <li>ustring: The data can be as long as needed.
      <li>set(X,Y,Z): You can define a free-form set of elements that the
attribute can conform to. For example, you could specify: <tt>set(dog,cat,turtle)</tt>. Then, data of <tt>dog,cat</tt>; <tt>dog,turtle</tt>; or <tt>cat</tt> would be acceptable, while <tt>dog,horse</tt> would be rejected. (Although why an attribute would be used to define animals is unknown.)
      <li>enum(X,Y,Z): Enum() acts like set(), except that the attribute must
be one, and only one, of the possible values.
    </ul>
</ul>




Additionally, attributes are further being used beyond services. We are 
in the process of adding User Preference screens. In this case, 
the scope of an attribute specification would be 'users', and the attributes
could be added to an individual user.




If you are interested in adding additional attributes format, look in
<tt>CMU::Netdb::validity</tt> to extend the routines which verify that the
format is understand, and verify that user data matches the attribute format.



<_header3>
Adding Services
</_header3>


System Tables


<_header5>
_sys_dberror

</_header5>


Currently unused. The plan was to use this table to log errors occurring during execution.


<_header5>
_sys_errors

</_header5>


This table can be updated to contain error text presented to users. Text can be 
customized for the particular error number, the location of the error, or the
fields involved in the error. The web interface looks to find the most specific
error message for the error, and returns the text.


<_header5>
_sys_info
</_header5>


The _sys_info table allows for generic storage of system information.
In practice, the table is used by the Scheduling mechanism to store locks
with atomicity. The table's schema contains a 'sys_key', and 'sys_value',
and contains an index guaranteeing uniqueness of <tt>sys_key</tt>s. 





The only other key we have defined (besides general lock keys) is <tt>DB_OFFLINE</tt>. If this value
is present, the value in <tt>sys_value</tt> is presented to ALL users when they attempt to login. This
is a mechanism for locking out users if you want or need to ensure maintenance can proceed without
interruption. Superusers (as defined in WebInt/Config.pm) are not locked out. Note this is <b>only</b>
enforced in the web interface.


<_header5>
_sys_scheduled
</_header5>


The "scheduling table" is used to keep track of what external processes
need to run at various intervals. Every minute, a cron job as user 'netreg'
runs <tt>support/bin/scheduled.pl</tt>. The Scheduler then queries NetReg
to see which jobs need to run, runs them, and updates the table to indicate
the next time of run. For example, we define DNS and DHCP generation runs
take place every 15 minutes, while tasks to load new users once per day.



Unfortunately this table does not have a NetReg interface (except to provide
the next run times on the Signoff screen). Eventually one will be added. Such
as screen would provide an easy way to update the intervals (currently we
just run raw SQL), or to "force" an immediate (or within 60 seconds) run.
In this vein, one of Kevin's favorite SQL commands is:
<tt>update _sys_scheduled set next_run = now() where id = 1;</tt>




Support Utilities



Over time we have written a number of scripts to deal with maintenance of NetReg. We have documented most
of them here. These are all in the <tt>support/bin</tt> subdirectory. Any of the utilities below that
have 'Vars!' in the margin are utilities that you need to configure a bit before using. The configuration
is generally a number of variables in the first few lines (after copyrights, etc.) in the utility.
If you have questions about configuration of these utilities, please let us know.



Utilities with 'Local!' in the margin are worse than those with 'Vars!'. These are utilities <em>heavily</em>
designed for our environment, and are probably most useful to you as a basis for other utilities
that perform similar operations. 'Okay!' means we thing (with proper configuration in canonical
locations) that it should work nicely for you.


<_margin>Local!</_margin>

<_header3 language="All">
afs-xfer.sh
</_header3>


This file is used to copy NetReg output onto AFS, our distributed filesystem. It contains special calls
to authenticate as the 'netreg' user to AFS. It could probably be easily rewritten to deal with NFS, but
you need to at least review it before using it anywhere, because it tests for existence of a particular
file in AFS to determine if AFS is "up".


<_margin>Okay!</_margin>

<_header3>
bulkupdate.pl
</_header3>


Bulkupdate allows you to update a number of machine registrations quickly. You supply the operations to
the STDIN in the following format:



<_code>
OperationType,TargetType,Args<br>
OR comment lines starting with #<br>
<br>
OperationType:<br>
delete: delete matching entries, via API calls<br>
seperator: change the seperator used for field delimination in the spec<br>
           file.<br>
add: Add an entry to a table<br>
<br>
TargetType:<br>
delete: table name<br>
seperator: new regexp to use for split<br>
add: table to add to, only machine supported at present<br>
<br>
Args<br>
delete: where clause to use for list_<foo> OR just a row ID.<br>
seperator: none<br>
 add: values of the fields for the entry.  The form of the values<br>
      should be fieldname=value.  Multiple fields are separated by the current<br>
      seperator (comma default, changed via 'seperator' command.).<br>
      Permissions are specified via "perm=groupname PERMS level" i.e.<br>
      "perm=dept:nginfra READ,WRITE 1"  (note that the seperator must not<br>
      be ',' for that to work)<br>

</_code>

<_margin>Okay!</_margin>

<_header3>
checker.pl
</_header3>


This is our consistency checker. It has fairly useful comand line help, and is run automatically from scheduled.pl.
Depending on the number of errors it finds, you will receive mail at the administrative email address
once per day from the consistency checker. The goal is, of course, not to find errors. :)



The consistency checker interfaces with <tt>CMU::Netdb::consistency</tt> for much of its operations.


<_margin>Local!</_margin>

<_header3>
cs-report.pl
</_header3>


This script generates some basic machine information for particular zones on campus; some groups that
we delegated zones to needed this information. Could be useful to use as a basis for other similar reports.


<_margin>Okay!</_margin>

<_header3>
delete-expired.pl
</_header3>


This deletes any machines with an 'expires' value of something other than '0' and earlier than today's
date. This is used in conjunction with NetMon/NetReg purges, which set the 'expires' field for machines
that haven't been seen lately. Run from scheduled.pl.


<_margin>Okay!</_margin>

<_header3>
delete-mach-subnet.pl
</_header3>


This script is used interactively to clear out machines on a particular subnet. You must specify the
subnet ID (which you can glean from the URL if you are looking at the subnet information; it's the XX in
<tt>sid=XX</tt>.) If you just specify the subnet ID, it will show you all the machines in the subnet
that it would otherwise delete. Specify with <tt>-yes</tt> and it will delete all of these registrations.
Specify with <tt>-ask</tt> and it will prompt with a Yes/No question for each machine, enter 'y' to
delete the machine.



This is an easy way to selectively delete a number of machines; we use it to clean up after certain users.


<_margin>Okay!</_margin>

<_header3>
dhcp.pl
</_header3>


The DHCP configuration file generator. When running interactively, you can specify the <tt>-debug</tt>
option if you just want to see what it's doing. When running in debug mode, it will NOT try copying
the configurations out to the servers, and the files will be written to <tt>/tmp/dhcp</tt> rather than
the configured location.



As with all of NetReg, we have tried reducing the amount of CMU-specific code in the DHCP configuration
file. The only major element still remaining is a set of configuration information for booting Sun
workstations (and beginning a network installation as directed.) This is removed from the package
as exported, but will be in our CVS tree. We also include it conditionally, if the script believes
it's running on a CMU-configured machine.



Once written, the configuration files are copied to your DHCP servers. You should read the NetReg Output
section for more information.


<_margin>Okay!</_margin>

<_header3>
dns-config.pl
</_header3>


The <tt>dns-config</tt> script generates the configuration files for your DNS servers, assuming they
are properly configured in the DNS area of NetReg. See the NetReg Output section for more details.
As with <tt>dhcp.pl</tt>, the dns-config script can also be run with <tt>-debug</tt> to enable debugging
mode. In this mode, configuration files are written to <tt>/tmp/zones</tt>.


<_margin>Okay!</_margin>

<_header3>
dns-xfer.pl
</_header3>


While DHCP has integrated the transfer process into the actual generation script, DNS invokes
a separate script, <tt>dns-xfer</tt>, to actually handle transferring files (zone files and
configurations) to your DNS servers. Again, look towards the NetReg Output section for more information.
Configuration information is culled from Netdb::config.


<_margin>Okay!</_margin>

<_header3>
dns.pl
</_header3>


<tt>dns.pl</tt> is the DNS zonefile generation script. It also uses the <tt>DNS</tt> utilities to
update your dynamic DNS servers as necessary. Again, configuration information is done from 
Netdb::config and more information in NetReg Output..


<_margin>Okay!</_margin>

<_header3>
domain-report.pl
</_header3>


The <tt>domain-report.pl</tt> is a general script for dumping all the machines in given DNS zones. 
We run this script from domain-reports.sh.


<_margin>Vars!</_margin>

<_header3>
domain-reports.sh
</_header3>


<tt>domain-reports</tt> is a wrapper around the Perl script of the same name to generate reports about
machines in a given DNS zone.


<_margin>Vars!</_margin>

<_header3>
dump-db.sh
</_header3>


The <tt>dump-db</tt> script is what we use to do backups. Via cron, once an hour dump-db simply uses
<tt>mysqldump</tt> to create an offline copy of the database. This is fairly fast (the entire database
is dumped in a couple seconds) and provides some good short-term history. Long term we would like
to incorporate a log of NetReg changes into the database (higher level than just the MySQL update
logs). Right now we're average 20MB per dump (including many more cables than are included in the 
sample data), and about 2GB saving 30 days worth of hourly backups (though they are also copied to
tape in that timeperiod).


<_margin>Local! (For Now)</_margin>

<_header3>
lbnamed.pl
</_header3>


This is our first test at a load balancing name server that uses Dynamic DNS and configuration
information from NetReg. As we get this into production, more documentation will be available. The
basic premise is that we'll use multiple A records that are updated fairly frequently using Dynamic DNS,
so that load balancing can be done via multiple mechanisms. For example, the <tt>DNS::KMemSrv</tt>
module implements a method for collecting data from our machines.


<_header3>
misc-reports.pl
</_header3>

<_margin>Vars!</_margin>


<tt>misc-reports</tt> is our script for generating different files that are useful for various tasks.
For example, we generate a <tt>hosts</tt> file (in the format expected by <tt>/etc/hosts</tt>). We also
generate a "bootptab". This is actually a derivative of our old BOOTP server configuration, but many
people have taken a liking to the existence of this file, with one line for MAC Address, IP Address,
and Hostname. If you find it useful, even better.


<_margin>Vars!</_margin>

<_header3>
passwd-load.pl
</_header3>


The passwd-load script is used to load users from the <tt>/etc/passwd</tt> (or equivalent) file into
NetReg. We don't use their passwords; that must be done via some other mechanism. However, NetReg wants
to know the UserID of every person using the system.


<_margin>Vars!</_margin>

<_header3 language="All">
portadmin.pl
</_header3>


The portadmin script is our backend procedure for activating and deactivating outlets via SNMP. Unless
you are using the outlet/cable infrastructure.


<_margin>Local!</_margin>

<_header3>
ptloader.pl
</_header3>


The ptloader script is our mechanism for loading some groups of users in our AFS cell to the NetReg
groups table.


<_margin>Vars!</_margin>

<_header3>
re_ip.pl
</_header3>


The re_ip code is used to move one subnet to another. It can either pack the hosts into the
bottom of the destination subnet, or do a 1 to 1 transfer (preserving the offset into the subnet.) It 
can also reposition the hosts by a specific offset to free space at the start of the destination subnet
and provide a predictable translation. This can also take a file of random IP addresses and move them
into the first available space in the new subnet. The file should have one IP address per line, lines
starting with a hash are ignored.


<_margin>Local!</_margin>

<_header3 language="Italian">
reparent.pl
</_header3>


This was an older script used to reparent DNS zones when we had some issues with DNS zone parenting.
If you believe your DNS zones are mis-parented (still somewhat likely, unfortunately), let us know. 
Mis-parenting in this case is the <tt>dns_zone.parent</tt> pointer is incorrect. This can manifest
itself as records appearing in the wrong zonefile, etc.


<_margin>OkaY!</_margin>

<_header3>
scheduled.pl
</_header3>


<tt>scheduled</tt> is the central clearinghouse for running periodic tasks with NetReg. See the 
NetReg Design section for more details.


<_margin>Okay!</_margin>

<_header3>
service-dump.pl
</_header3>


<tt>service-dump</tt> dumps out everything from the service groups to the <tt>SERVICE_FILE</tt>. Again,
look to Netdb::config and the NetReg Output section for more information.


<_margin>Vars!</_margin>

<_header3>
set-base-broadcast.pl
</_header3>


At one point we tried actually keeping the base and broadcast address on every subnet registered, and
the <tt>set-base-broadcast</tt> script is an automated way of registering base/broadcast addresses
that are not.


<_header3>
subnet-scan.sh
</_header3>


The subnet scan script handles the actual updating of machine records that are being purged for
non-activity on the network. This is still in the final stages of development, and more documentation
is forthcoming.


<_margin>Local!</_margin>

<_header3>
user_sync.pl
</_header3>


The user_sync script is similar to passwd_load in that it syncs the NetReg user database. The difference
is that user_sync is designed to use our LDAP database instead of the password file.


<_header3>
vars.sh
</_header3>


The <tt>vars.sh</tt> file is our attempt at standardizing configuration of support utilities. This
file is sourced by other shell scripts as needed, so that duplicate configuration of your NetReg
home, for example, is automated. <tt>vars</tt> should not execute any commands, just set variables.


<_header3>
vars_l.pm
</_header3>


The <tt>vars_l.pm</tt> file is our attempt at standardizing configuration of support utilities. This
file is used by other Perl scripts as needed, so that duplicate configuration of your NetReg home,
for example, is automated. The goal of <tt>vars_l.pm</tt> is just to allow support utilities to 
locate CMU::Netdb and the accompanying config module.


<!--
[variables]
{
  $page_title = "NetMon Design"
  $page_title_english = "NetMon Design"
  $file_prev = "005.html"
  $file_next = "007.html"
  $real_page_number = "6"
  $page_number = "6"
}
-->

<_header1>
$page_title
</_header1>


While NetReg manages what your network "should" look like, NetMon was designed to give more insight
into how things actually look. We like to think every machine will be configured with DHCP so things
will just "work". NetMon can tell you what machines aren't, and more. Using NetMon data we have begun
identifying machines that are using a different MAC address than registered, as well as finding unused
network registrations. The former allows us to contact users and get the registrations fixed, so that
when we sweep for the latter, we catch true dead registrations and reclaim IP space.



Much of this assumes you have a number of static (aka "fixed") addresses and don't simply run lots of
dynamic pools. However, NetMon has also been extended to keep track of DHCP lease information, and
provide tools for searching and graphing this data. In short, NetMon is sort of an always-changing
target of features, but the basic premise is to enable you to have better control of your network.



A work of caution, though, before proceeding: we are a fairly Cisco-centric campus. Some of the
SNMP MIBs used may not be compatible with other agents. We have, however, had fairly good success
collecting from the Lucent/ORiNOCO AP-1000 and AP-500's. Long term we'd like to make the vendor-specific
bits minimal and extracted for easy extensibility. One of many goals, though.



Master Capturing Agent



One of the central NetMon components is the master capture process. Invoked from cron (typically), the
master capture process:




<ul>
  <li>Identifies work that needs to be done (from device_timings)
  <li>Determines how many agents are collecting
  <li>Starts additional agents as necessary
</ul>



If there is no more work to be done, the process dies. It could be easily designed to stay around and
thus save the expense of re-invoking from cron, but we like having definite cycles. This also gives
you a chance to easily update NetMon code. In the <tt>lib/CMU/NetMon</tt> directory is a <tt>netmon.sh</tt>.
file. This is what we run from cron; it invokes the capture agent and, once done, invokes the processing
agent. You can start a new capture agent with the command line: <br>
<tt>perl -I$NRHOME -MCMU::NetMon::Capture -e 'run_capture()'</tt><br>
(substituting your NRHOME appropriately..)



Most configuration of the capturing agent is done in the <tt>Config.pm</tt> file. (See the Configuration
section above for more details.)



If <tt>/etc/nonetmon</tt> exists, <b>no</b> NetMon capturing or processing will take place (unless
you are running an agent on a remote machine, for example, and that machine does not contain the <tt>nonetmon</tt>
file.)



Master Processing Agent



The master processing agent is the hub for launching processing routines as needed. Unlike the capture
tasks, in which the master agent starts many subagents for parallel collection, the processing routines
are fairly serial in nature - generally they are consuming CPU time exchanging data with the SQL server
as well as doing a fair bit of processing work themselves. Thus, the process agent reviews the work
to be done, starts the tasks, and waits for completion before starting the next.



Most configuration of the processing agent is done in the <tt>Config.pm</tt> file. (See the Configuration
section above for more details.)



Device, Device Timings



A large part of NetMon is the collection of data from network devices -- routers and switches on your
network. Any device we collect from must be registered in the device table. The table also
contains additional fields related to the device:



<ul>
  <li><tt><b>read_comm</b></tt>: All collection is done via SNMP, and we need to know the SNMP "read"
string used on your device. The default for most devices is 'public', but you may/may want to restrict
access to your devices by setting a non-public read string.
  <li><tt><b>slow</b></tt>: The slow value can be 'Yes' or 'No', depending upon whether NetMon has had
recent trouble contacting this device. If slow, all collection will be done by separate collection
agents. This will allow our 'fast' agents to collect from responsive devices (and thus, collect from
them faster.)
  <li><tt><b>netreg_id</b></tt>: NetMon attempts to identify the NetReg machine record matching this
device. This is so that we can track changes in the hostname, and in general establish a more solid
link to NetReg.
  <li><tt><b>alias_dev</b></tt>: If this device is just an alias for another device (ie, multiple
interfaces on the same router), then NetMon will set the alias_dev field to the "real" device name.
  <li><tt><b>function</b></tt>: The function field can be 'Gateway' or 'Non-Gateway'. We use this
designation in the topology generation (see below).
  <li><tt><b>system_id</b></tt>: This is a text field that will be updated to contain information
about the device, as reported by itself through SNMP. This could be used, for example, to select
different collection routines depending upon the device type, version, etc.



As alluded to above, NetMon can dig through NetReg looking for new devices that you want to collect. 
The parameters for this searching are in the NetMon configuration (see above). 



CAM Tables



The CAM tables of network switches contain information about which port a given MAC address is connected
to. Fundamentally a CAM table is what makes a switch more than a hub: stateful information about
where traffic should be forwarded. By gathering the information about where MAC addresses appear in
CAM tables around the network, we have very useful information for tracking down machines. 




The fundamental data elements we deal with in capturing this information is a 4-element group:
<ul>
  <li>The network device we captured this MAC Address from
  <li>The MAC Address
  <li>The port that the MAC Address appeared on
  <li>The VLAN that this MAC Address appeared on
</ul>



Our capture of VLAN information is actually a bit more specific: we identify all the VLANs on a given
switch, and then walk the CAM table for each separately. Thus we can elegantly deal with multi-access
and/or trunked VLANs. This is extremely useful in generating the topology information, as VLANs are 
handled very nicely, being kept separate even if multiple VLANs use the same network devices.



The capture agents just suck all of the CAM table information from the device and put it into the
<tt>cam_capture</tt> table. It is then the job of the CAM processine engine to reduce the size of the
captured data and store it in the <tt>cam_tracking</tt> table.



The CAM processing engine makes the assumption that if a host appeared on a given switch/port/vlan an
hour ago, and is still there, it makes no sense to add another line to the <tt>cam_tracking</tt> table.
Instead, all we do is update a <tt>last_updated</tt> field to indicate that we saw the host again,
and move on (removing the entry from <tt>cam_capture</tt>). We also tie into NetReg to identify if
a host is unregistered at the time of processing. Thus, if a host was unregistered but then registered
in NetReg, a new <tt>cam_tracking</tt> entry will be created.



Each <tt>cam_tracking</tt> entry has a start and end time. Any "active" entries are defined to have the
end value set to '0'. NetMon is designed for maximum parallelism: we have made no assumptions about
the serialability of certain tasks. Thus, if a processing engine is processing records from a given 
device at the same time as a capturing agent is adding records, the behavior will be consistent and
correct.



As entries disappear from CAM tables, the <tt>end</tt> fields are updated, and the entry is no longer
considered active. If the device appears again on the same port/vlan/device, we'll add a new line.



You can then query NetMon for the known information about a given MAC address and receive a list of
everywhere that NetMon saw the host.



An additional feature that NetMon does with <tt>cam_tracking</tt> entries is the so-called "assumed
location" identification. In a typical network, if two hosts communicate with one another, the CAM
tables of all switches between them will be updated to reflect this. The MAC Addresses of the hosts
will appear on the uplink ports of non-common network infrastructure. Thus, NetMon will "see" a host
on the uplink port on a number of other network switches (other than those in the direct path to
the host from your root).



The "assumed location" feature takes a look at all the currently active locations that we see a current
host, and makes its best guess about the actual physical port the device is connected to. It does this
by counting the number of CAM table entries on all the ports that we see the host on. The port with
the fewest number of CAM table entries is then the "assumed location". This, in practice, is very
accurate.



ARP Tables



We also collect the ARP tables from all of our network routers. The ARP tables provide a layer-3 (IP)
to layer-2 (MAC Address) translation. Thus, the ARP tables give us some idea what IP address various
MAC Addresses are using. This is interesting data, because we can then compare the MAC Address registered
for a given IP address with the actual MAC Address being used. Where different, we can contact the user
and request they update the network registration. Additionally, while we are not currently doing this,
we could compare the IP information provided by DHCP with the actual IP/MAC Address in use.



All of this helps us provide good service for our users -- if we decide to renumber an IP subnet, for
example, we can identify the users that won't automatically work because of not using DHCP 
and contact them prior to the move.



The ARP table capturing agent simply grabs the tables from all of our routers and stores it in the
<tt>arp_capture</tt> table. Periodically the processing agent runs, eliminating duplicate IP and MAC
Address mappings (which should not happen unless you capture from more than 1 layer-3 device on the same
physical network). It updates the <tt>arp_tracking</tt> and <tt>arp_map</tt> tables with the mapping
information. To reduce the number of entries in <tt>arp_tracking</tt>, we record the actual location of
mappings being seen in <tt>arp_map</tt>.



As with CAM information, the ARP information can be searched in NetMon and used to validate the location
of hosts.



Ping/DevMAC



These two collection types are done mainly for housekeeping purposes, but they do play important roles
for the topology generation. The ping "capture" agent does not actually capture any data, but simply
sends out pings to all of your devices. The goal here is to make sure that the management interface
of the network devices is completely known by all upstream devices (and by "known" we mean "in the CAM or ARP 
table of..").



The DevMAC collector solves an issue we discovered during testing. Namely, most network devices have
several MAC addresses that technically belong to the device. For example, a 24 port switch might have
one MAC address for each port, as well as an address for the management interface. During normal network
activities, though, the exact MAC address used to answer ARPs and otherwise act as a network citizen
is sometimes different from device to device. Thus, the DevMAC collector simply grabs ALL MAC addresses
that belong to the device. Then in generating the Topology information, we understand that if we see
any of the MAC addresses, we still know what the actual device is.



Due to some relatively short CAM/ARP table timeouts, the Ping agent is generally run very frequently
(on the order of every 5 minutes). However, most devices do not dynamically change their MAC addresses
(at least, none that we have found), so the DevMAC collector typically runs once a day.



DHCP Lease Information



While tracking down abuse issues and other network problems, we frequently needed to identify the user
of a particular IP address at a given time. Rather than spending the rest of our lives grepping DHCP
logs (and thus, maintaining large unweildy logfiles), we decided to write a quick NetMon agent that
gathered DHCP lease information and loaded it into our NetMon database. Then we can quickly identify
the user of a given IP and easily generate some utilization graphs. Much more could be done with
the gathered lease data.


<_header3>
Capture Agent
</_header3>


To load your DHCP lease information into NetMon, you'll need to install some capturing agents on 
your DHCP servers. In practice we put a skeleton NetReg/NetMon install on our DHCP servers (though it
includes the entire <tt>lib</tt> tree.) We then have a script that launches the agents. The script is
designed to be put into cron and have it attempt to re-launch every hour or so. Upon startup it tries
to determine if any other agents are running and, if not, continue running. This is a basic failure
protection; it's not elegant, but we like our DHCP lease information.



The agent reads <tt>/etc/dhcpd.leases</tt> for dynamic lease assignment information. However,
fixed-addresses are not recorded in this file. Thus, we need to get information about static leases.
We <em>could</em> use syslog information, but we'd definitely encounter duplicates and it's a fairly
annoying format. Instead we've written a simple patch to the DHCP server to keep a separate logfile of
static lease assignments. This solution is not particularly elegant (hrm, there is an elegance trend
here), but works nicely. The patch is located in <tt>doc/dhcpd.patch</tt>. You should modify it to
write your logfile where you like (we use <tt>/var/log/dhcpd.static</tt>). If someone feels like
making the agent work using syslog, that would be useful for many, we're sure.



The script for starting the collection agent is in
<br><tt>support/bin/start-netmon-dhcp-agent.sh</tt>. As noted, we run this from cron every hour or so.



<_header3>
Graph Generation
</_header3>


Every hour (or whatever interval you specify in <tt>Config.pm</tt>, the DHCP lease processing engine 
fires off and updates your RRD graphs of DHCP utilization. It will add a data point every DHCPGraphInterval
 seconds
(where DHCPGraphInterval is also a value from <tt>Config.pm</tt>.) These graphs will then be available
from the standard NetMon web interface (in the "DHCP" section.)



The graph generation uses your DHCP configuration (also configured in <tt>Config.pm</tt>) to determine
the subnets with active dynamic pools. If your configuration is not generated from NetReg, your results
with the graphs may vary. The parser is fairly robust, but you definitely want to add one comment you
probably don't otherwise have -- namely, a description of the subnet (ie subnet name) prior to every
subnet declaration.



So the DHCP configuration files that NetReg generates looks like:



<_code>
# Wireless Network<br>
subnet 128.2.64.0 netmask 255.255.240.0 {<br>
&nbsp;&nbsp;...<br>
&nbsp;&nbsp;pool ...<br>
}<br>

</_code>


The utilization graphs are created and keyed on the base address of the subnet (the first IP address in 
the previous section). Thus, if you change the base address of a subnet and want to keep the 
utilization graphs going (with historical data), then you need to perform some non-trivial conversion.
We have not done this, but the basic outline is:



<ul>
  <li>Lock the NetMon DHCP Processing from happening (set the last run time of the <tt>RUN_DHCP</tt> key
in <tt>_sys_info</tt> to now()), and lock the DHCP configuration (or copy the config without the base address
change to somewhere safe)
  <li>As atomically as possible, kill your DHCP servers, force the NetMon collection agent to load
all the lease information (you can just start a new one), kill all the NetMon collection agents, and
restart your DHCP server (with the new base address configured). 
Do NOT restart the collection agents yet. It's okay, as they will lose no data 
as long as they aren't dead for longer than your shortest DHCP lease
time.
  <li>Run the NetMon DHCP processing agent (set the last run time of the RUN_DHCP key to at least an 
hour in the past). Make sure the DHCP config it has does NOT see the new base address.
  <li>Move the RRD database file to the new base address (the filenames in your RRD directory are
named after the base address of the subnet, so move it to the new base address).
  <li>Restart the NetMon agents on your DHCP servers.
  <li>At some point the NetMon DHCP Processor will update all the graphs, etc.
</ul>
  



Note, though, that if your base address moves and the old leases are no longer on the same network (ie if you are moving the entire
network),
your graphs will not include the old leases in the graphs. Your best bet in this case would be to shorten the lease time considerably to reduce the
overlap as clients move from one network to the other.



You should <b>never</b> change the <tt>DHCP_GRAPH_START</tt> variable in <tt>_sys_info</tt> unless you really
know what you are doing -- this can dramatically affect your RRD databases.


<_header3>
Web Access
</_header3>


The DHCP lease search page and DHCP graphs are available under the "DHCP" section on the administrative
access section of NetMon.



Topology



One of our later goals of NetMon was to be able to construct a dynamic graph of the network topology
with as few assumptions about the design as possible. Using the ARP and CAM collected information,
and an interesting topology algorithm, we have had some success in generating the topology. While are
not actively generating the topology currently (there have been some major changes in ARP and CAM
collection since the initial design, requiring it to be fixed), the hope is to have this working in
the near future.



The basic premise of the topology generation is that if the path to network device C is via A and B,
then "A" will know about the MAC address of "B" and "C" on the appropriate downlink port, and "B" will
know of "C" on a downlink port. The algorithm repeatedly iterates over a list of devices. For each
device we keep track of an active list of devices known by another device. Through each iteration,
devices known only in a one location are removed as we are able to place them in the network graph.
Following this, that device is removed from every other device's list, thus making more devices "known"
in one location.



If you are interested in the topology discovery algorithm and system, please let us know. There are
several interesting issues related to the topology discovery that we can discuss.



Web Access







Network Debugger



The network debugger was designed to give end-users (and our front-line support staff) a quick way
to diagnose common network problems. It provides a very limited interface to the NetMon data. .




<para condition="ARP Query">
Given
an IP address, it queries the ARP information to see how many unique mappings we identified (which can
quickly identify if a machine is misconfigured and stealing an IP). It also queries the DHCP lease
information to see how many leases were given (0 leases means no use of DHCP/no network connectivity..)


<para condition="CAM Query">
Given a MAC address, it queries the ARP information, as above. It also queries the CAM tables to see
how many switches knew of the host (0 switches knowing about the host implies no network connectivity
at very low levels).



Both queries also check NetReg for basic registration. We do not allow un-registered attainment of 
DHCP leases (except in a very few, protected locations.)



In all cases, we only report anything seen in the last 24 hours. Thus, historical access to the NetMon
data is not permitted. We also do not show any location-specific information (ie, this machine was seen
on switch X). The goal is to protect privacy but eliminate some load on the networking staff to answer
simple questions.



When the authenticated user has no access to NetMon, or no authentication was supplied (ie, Apache didn't
request any authentication), <tt>netmon.pl</tt> will default to providing users the Network Debugger
form. You should edit the text in <tt>CMU/NetMon/Web.pm</tt> (the <tt>netdebug_info</tt> function) to
suit your individual support system. 



Subnet Purge



As users come and go, the machine registrations are not always kept up-to-date. When we ran a completely
flat (ie non-subnetted) network, it was easy to just start assigning addresses from another pool to
additional machines. However, as subnetting has taken hold and additional routers added around the
network, when subnets fill up we have problems to deal with. In some cases, doubling the subnet size
is reasonable and done. In many cases, though, we started to wonder how many of the registrations were
actually in use.



The results were better (or worse) than we expected: some of our subnets had as much as 30% wasted to
inactive registrations. By removing these outdated registrations, we regained valuable IP space and
did not need to waste additional space by doubling already large subnets. We have currently run the
complete purge operation on a few subnets, and are planning on expanding this to encompass most subnets
in the near future.



Purging is done on a per-subnet basis. The purge parameters are fields in the NetReg <tt>subnet</tt>
table. There are four parameters (described in the NetReg design section). They are: purge interval,
purge days, grace time for NetReg updates, and grace time for network usage.



The purge is actually run as a NetMon component, but long term we will run the agent on NetReg out
of <tt>scheduled.pl</tt>, and it will remotely access the NetMon database (via the 
<tt>NetMon_Rem_RW_DB</tt> variable).



We expect to run the purge no more than once per day. It will identify any subnets that have not 
been purged in <tt>purge_interval</tt> days (we keep the <tt>purge_lastdone</tt> variable to remember
the last purge day). On each of these subnets, we get all the registered MAC Address from NetReg
(on that subnet) that have not been updated in <tt>purge_notupd</tt> days. We then query NetMon, and 
build a list of all MAC Addresses that have not been seen on the network (via the CAM tracking table)
in at least <tt>purge_notseen</tt> days. All of these machine registrations are then set to expire
in <tt>purge_explen</tt> days. 



For expiration purposes, the <tt>expires</tt> field on the machine registration is set to a non-zero 
date. Users are then mailed about all registrations they own that have just been set to expire (each
user should receive no more than one piece of mail per day). On the appointed expiration day, unless
the user has "Retain"ed the registration, the <tt>delete-expired.pl</tt> script deletes the registration.



We may also include a scan of the DHCP lease information in the determination of unused registrations.
We specifically excluded matching IP addresses against the ARP tracking table, because we would like
users to keep registrations up to date. To this end, we created the "Misregistered Devices" report
(see above.)



Users



NetMon supports a very basic user authorization system. It incorporates the concept of "levels" like
NetReg, but has only a single per-user level. Thus, a user can be set as having full read,
read-write, or administrative access. Administrative access allows the user to control other UserIDs
and levels, while read-write allows the user to change the capture settings of a device. As with 
NetReg, these permissions are enforced by the web interface; thus, having access to the database
password (via login access/permissions to the machine, for example) grants the user read-write access to the NetMon
database.


<!--
[variables]
{
  $page_title = "NetReg Output"
  $page_title_english = "NetReg Output"
  $file_prev = "006.html"
  $file_next = "008.html"
  $real_page_number = "7"
  $page_number = "7"
}
-->

<_header1>
$page_title
</_header1>


DNS



DNS generation in NetReg has seen several changes in the last six months. The biggest change is that
our DNS generation now has two methods: static zone files, or dynamic DNS update. The motivation for
this change is that we would like to offer dynamic update for dynamically assigned IP addresses. Long
term, we will allow the registration of a so-called "global dynamic" address. NetReg was designed around
our long-standing policy of requiring static registration for every machine. As we add dynamic IP pools
to many subnets around campus, users registering a machine to the "global dynamic" pool would be able
to have a hostname follow their active IP address, by updates from the DHCP server as the IP address
is assigned or expired.



Below we'll explain both methods of updating your DNS servers. We recommend that you NOT use the same
DNS server groups to serve both static and dynamic zones.


<_header3>
Static Zone Update
</_header3>


Static zones are fairly straightforward: the machine running NetReg will generate a file containing all
resource records that belong in the zone. The zone serial will be automatically incremented by the
zonefile generation script <tt>dns.pl</tt>. The static zone files are generated in 
<tt>$NRHOME/etc/zone-gen</tt>. The generation script then calls the <tt>dns-xfer.pl</tt> script.
dns-xfer copies all applicable zonefiles to the respective master DNS server. This copy routine
is done via SSH (using the <tt>rsync</tt> utility). 



Periodically another script runs on the master DNS server to load new zone files. BIND 8 and later use
the DNS NOTIFY protocol to then inform secondary servers of the new zonefiles, and a transfer is
initiated.



So the progression of the zonefiles is:
<ul>
  <li><tt>dns.pl</tt> writes <tt>$NRHOME/etc/zone-gen/ZONENAME.zone</tt> (so the zone EXAMPLE.ORG is
written to the file <tt>.../zone-gen/EXAMPLE.ORG.zone</tt>.
  <li><tt>dns-xfer.pl</tt> copies the appropriate zones from <tt>zone-gen/*.zone</tt> to
<tt>$NRHOME/etc/zone-xfer/*.zone</tt>.
  <li><tt>dns-xfer.pl</tt> executes <tt>rsync</tt> (which then calls <tt>scp</tt>) and copies the zone
files from the NetReg host (<tt>zone-xfer</tt> directory) to the master DNS server (the
<tt>$NRHOME/etc/zone-xfer</tt> directory on the DNS server).
  <li><tt>update-zones.sh</tt> on the DNS server copies the zone from <tt>$NRHOME/etc/zone-xfer</tt>
to <tt>$DNSZONEDIR</tt>. (You need to update the <tt>DNSZONEDIR</tt> variable in update-zones.sh.)
  <li>The nameserver is reloaded by <tt>update-zones.sh</tt> and picks up the new zone information,
informing the secondaries.
  <li>The secondaries transfer the zone information from the master servers (done via the standard
DNS XFER protocol; no special configuration is required.)
</ul>


<_header3 language="All">
Dynamic Zone Update

</_header3>


A note of caution before jumping into dynamic zone update: we are currently only testing this
method of updating zones. We do <b>NOT</b> recommend you use this on mission-critical zones. We'd
be interested in feedback, positive or negative, regarding your use of dynamic zone update.






<_header3>
Moving a Static Zone to Dynamic Update
</_header3>


Moving a zone from your static nameserver group to a dynamic group is a complicated process. The
problem is really that our method of updating nameserver configurations and zone information is too
efficient: you want the new zone information to propogate to the dynamic group (and have the current
zone contents available) prior to your static servers dropping the zone (so that you have no
interruption in DNS service.)



To this end, we've created a step-by-step guide to moving zones. This will require manual intervention,
but should guarantee no interruption in service. Note when we say "pin", we mean that you should
<tt>chown</tt> the files in question to <tt>root</tt>, or some other user (in this case, NOT the user
that generates the files. At CMU, 'netreg' writes the files, so in 'pinning' we change the owner to
root, and 'unpin' by changing it back to 'netreg'. Basically we want the files to be unchangeable by
the normal means. Assume $ZONE refers to the zone that is moving from static to dynamic.



<ol>
  <li>In <tt>$NRHOME/etc/zone-xfer</tt> PIN: the zonefile for $ZONE, 
the zonefile for the parent zone, and the configuration files for the static master and static
slaves (<tt>named.conf.$HOSTNAME</tt>).
  <li>Change $ZONE's nameservers in NetReg to the dynamic zone NSs, add the appropriate keys to the
DDNS Auth field (can be done earlier), and add the zone to your DDNS server group. Do <b>NOT</b> add
<tt>ddns:ena</tt> to the DDNS Auth field yet.
  <li>Run <tt>force-dns</tt> out of support-bin (or just wait until the zones are re-generated). In
any case, wait for the zone regen.
  <li>Wait for the nameserver configuration files to work their way out to the DDNS nameservers.
  <li>Copy <tt>$NRHOME/etc/zone-gen/$ZONE.zone</tt> and the to the zone directory on your DDNS master nameserver,
and reload the master. At this point the DDNS servers should be serving the zone (you can verify with:
<br><tt>% dig axfr $ZONE @NS</tt>
where NS refers to your DDNS nameserver.
  <li>Add <tt>ddns:ena</tt> to the DDNS auth field of $ZONE.
  <li><tt>force-dns</tt> again; verify it is succeeding (basically, if you see an <tt>nsupdate</tt>
process on the NetReg server for more than a few seconds, something is failing)
  <li>Verify the zone is being served from the DDNS servers again.
  <li>Unpin all the files.
  <li>Again, watch the propogation and verify the static servers drop the zone from their configs, the
parent zone is pushed out to direct requests to the DDNS servers, etc.
</ul>



DHCP



From the NetReg registrations and subnet information, a configuration file for the ISC DHCP server can
be generated. Note that the DHCP server configuration is far more complicated than DHCP configuration.
While it's fairly easy to specify options on a global, subnet, or machine basis, many DHCP features
are inaccessible. This is simply because they are complicated constructs. If you need features beyond
those that we generate by default, feel free to talk with us about the best way to add them to NetReg.
Our general goal is to make NetReg as generic as possible and allow complete administration from the
Web interface.



That being said, we write the DHCP server configuration files by writing:



<ul>
 <li>General, automatic options (such as server-identifier, etc.)
 <li>Auxiliary information from DNS (update keys, if used)
 <li>DHCP failover configuration, if configured. Note that we do not support more than one failover
pair. However, once service groups are extended to include the ability to group subnets and zones, it
will be fairly easy to use a service group with attributes and subnet members to have more than one
pair. 
 <li>Global DHCP options
 <li>Subnet configuration information (inside a subnet share as appropriate)
   <ul>
     <li>Basic parameters (base address and netmask) specified in configuring the subnet
     <li>Subnet DHCP Options
     <li>Dynamic pools -- We do not support a general mechanism for setting pool options (ie, one
pool for certain machines/users, etc.). Pool statements are generated directly from machines
registered on the subnet with mode "pool". We coalesce adjacent IP addresses into a single block
definition (ie <tt>10.1.1.4-10.1.1.42</tt>). 
     <li>A class that matches hardware addresses (for subnets with pools configured)
   </ul>
 <li>All registered machines with mode 'dynamic' are added as subclasses to the pool matching hardware
address. We add the <tt>ddns-hostname</tt> and <tt>domain-name</tt> options to match the machine's
hostname, unless these options are specified as DHCP options of the machine. The DHCP options
of the machine are added here.
 <li>All registered machines with mode 'static' are added in <tt>fixed-address</tt> statements. We then
add the DHCP options of the machine in the <tt>fixed-address</tt> blocks.



Historically our network has been solely fixed ("static") addresses. Thus, with 20,000+ hosts, our 
DHCP configuration file is on the order of 2-3MB. Currently this configuration file is written three
times per hour on our NetReg server. The generated configurations are then <tt>rsync</tt>'ed to the
DHCP servers. They check for new configurations every five minutes (by running the <tt>generate.sh</tt>
script). Once a new configuration is installed in <tt>/etc/dhcpd.conf</tt>, the <tt>rundhcpd</tt> 
program notices the change and restarts the DHCP server.



Our DHCP servers are relatively old machines: Sun Ultra 1's with 128-256MB of RAM. They handle our load
adequately (most subnets have 1 day leases; areas that we encourage dynamic addressing have 3-4 hour
leases). However, our up-to-thrice-hourly restarts are not insignificant. Thus, we were very happy to
move to a failover configuration; there should be very few cases of no service to DHCP clients in
this configuration.



The parameters of building the DHCP configuration files are all configurable in <tt>CMU/Netdb/config.pm</tt>.
By default the files are written as <tt>$NRHOME/etc/dhcp-gen/dhcpd.conf.$HOSTNAME</tt>. We generate
one file for each $HOSTNAME. Each member of your DHCP server service (configured in Config.pm) will
get a configuration file. If the File Type attribute is 'Dynamic', we'll write one with the subclasses and pools.
If the File Type attribute is 'Static', it will not contain this information.



Once generated, the files are <tt>rsync</tt>'ed to the appopriate servers, after being copied to
<tt>$NRHOME/etc/dhcp-xfer</tt>. Our <tt>generate.sh</tt> script runs via cron every five minutes on the
DHCP servers.



Services



The services area is still under some development, so the exact output may change over time. However,
the current output format is a single file that contains every service. This file is written as:
<tt>$NRHOME/etc/service-gen/services.sif</tt>. The file format is similar
to the format of ISC BIND's named.conf or the ISC DHCP server dhcpd.conf. Hash marks ('#') mark
comments. Blocks are contained in braces: { and }. We'll give an overview of the current blocks
that are output.



The design of this file is to provide an extensible, easily parseable format. In theory the file could
be written as separate components, perhaps as applicable to certain services. However, this file is
not currently publicly distributed at CMU (since it contains DDNS update keys), so we have no reason
to separate it at this time. If you are interested in separate files, please talk to us about development
of some standardized methods.


<_header3>
sif-header
</_header3>


The <tt>sif-header</tt> block prefixes the entire file. The contents of this block are not currently
used, but we created the header for future use. It's fairly self-explanatory; an example is:



<_code>
sif-header {<br>
&nbsp;&nbsp;originator NetReg;<br>
&nbsp;&nbsp;timestamp 1010293741;<br>
&nbsp;&nbsp;time-formatted Sun Jan  6 00:09:01 2002;<br>
};<br>

</_code>

<_header3>
machine
</_header3>


Every machine that exists as a member of a service group will have a block added in the services file.
This will provide basic machine information that could be used by services. With the desire to allow
subnets, zones, or other NetReg entities to become members of service groups, we expect similar blocks
will be generated for those members.



An example machine block is below. The 'id' and 'version' fields
are the NetReg table ID and version. The IP address field is represented as an unsigned 32-bit integer
(see <tt>CMU/Netdb/helper.pm:long2dot</tt> for help converting to the standard dotted-quad format.)
The respective TTL fields are for the DNS TTLs; a value of 0 is used to indicate "default" (inherits
from the DNS zone). "Created" is the date and time the actual record was created.


<_code>
machine DHCP1.NET.CMU.EDU {<br>
&nbsp;&nbsp;id 6635;<br>
&nbsp;&nbsp;version 20010825004103;<br>
&nbsp;&nbsp;ip_address 2147615746;<br>
&nbsp;&nbsp;mac_address 080020811637;<br>
&nbsp;&nbsp;host_name_ttl 0;<br>
&nbsp;&nbsp;ip_address_ttl 0;<br>
&nbsp;&nbsp;created 2000-08-12 12:39:49;<br>
&nbsp;&nbsp;flags ;<br>
};<br>

</_code>

<_header3>
service
</_header3>


The "meat" of the services file is the output of the actual service groups. Service attributes are 
added as <tt>attr &lt;ATTR NAME> &lt;ATTR VALUE>;</tt>. For maximum versatility, we are planning on changing
this to a format: <tt>attr &lt;ATTR NAME> = &lt;ATTR VALUE>;</tt>. Additionally, we are planning on 
enclosing the service name in quotations. Members of the group are added as <tt>member</tt> blocks. 
As we add additional service membership types, we will need to indicate the type of member. 



So, basically, this is a format which currently works, but will be changing soon. Below is the service
block for our DHCP servers. The "File Type" attribute specifies whether the DHCP configuration file
for the machine will contain dynamic pools, allowed dynamic machines, etc. In our case, <tt>dhcp1</tt>
and <tt>weh-ns</tt> are failover peers for both our dynamic pools and fixed-address ("static")
machines. <tt>dhcp2</tt> provides redundant service for fixed-addresses, so it does not get any
pool information. <tt>netmon</tt> does not actually provide DHCP service; instead it needs the full
dynamic DHCP configuration to generate graphs of utilization, etc.


<_code>
service dhcp.net.cmu.edu type DHCP Server Pool {<br>
&nbsp;&nbsp;description The main campus DHCP service.;<br>
&nbsp;&nbsp;version 20011129155545;<br>
<br>
&nbsp;&nbsp;attr Dynamic Secondary Port 520;<br>
&nbsp;&nbsp;attr Dynamic Master Port 519;<br>
&nbsp;&nbsp;member DHCP1.NET.CMU.EDU {<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr Dynamic Failover master;<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr File Type dynamic;<br>
&nbsp;&nbsp;};<br>
&nbsp;&nbsp;member DHCP2.NET.CMU.EDU {<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr File Type static;<br>
&nbsp;&nbsp;};<br>
&nbsp;&nbsp;member WEH-NS.NET.CMU.EDU {<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr Dynamic Failover secondary;<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr File Type dynamic;<br>
&nbsp;&nbsp;};<br>
&nbsp;&nbsp;member NS.FS.ANDREW.CMU.EDU {<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr File Type static;<br>
&nbsp;&nbsp;};<br>
&nbsp;&nbsp;member NETMON.NET.CMU.EDU {<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr Dynamic Failover neither;<br>
&nbsp;&nbsp;&nbsp;&nbsp;attr File Type dynamic;<br>
&nbsp;&nbsp;};<br>
};<br>

</_code>


Configuring NetReg File Transfers



DNS and DHCP require an easy mechanism for transferring files from your NetReg machine to the various
servers. We use the <tt>rsync</tt> utility, with configuration done from <tt>CMU::Netdb::config</tt>
(see above as well). We also use SSH as the underlying transport mechanism. It's fairly easy to
configure.



We use RSA keypairs for authenticating the SSH connection. You will need an unused user account on
your DHCP and DNS servers. We use the <tt>ftp</tt> user mostly because it was easy to setup in our
environment (with a centrally maintained <tt>passwd</tt> file, etc.). We recommend you just add another
user, possible <tt>netreg</tt>.



<ul>
  <li>Generate an RSA keypair by doing:<br>
<tt>% ssh-keygen -C netreg -f xfer</tt><br>
  <li>When asked for a passphrase, just hit enter twice. 
  <li>You should then have two files: <tt>xfer</tt> and <tt>xfer.pub</tt>. 
  <li>Copy <tt>xfer</tt> to <tt>$NRHOME/etc/.identity-xfer</tt> on your NetReg machine (or some other
file, and edit the <tt>RSYNC_RSH</tt> variable in Netdb/config.
  <li><tt>% chown netreg:netreg $NRHOME/etc/.identity-xfer</tt>
  <li><tt>% chmod 400 $NRHOME/etc/.identity-xfer</tt>
</ul>



Then, on all of your DNS/DHCP servers:
<ul>
  <li>Add the new user account as per above (we'll call the username $USER and the user home directory $USERHOME)
  <li><tt>% mkdir $USERHOME/.ssh; chown $USER $USERHOME/.ssh; chmod 755 $USERHOME/.ssh</tt>
  <li>Copy the <tt>xfer.pub</tt> file from above to <tt>$USERHOME/.ssh/authorized_keys</tt>
  <li><tt>% chown $USER $USERHOME/.ssh/authorized_keys; chmod 644 $USERHOME/.ssh/authorized_keys</tt>
</ul>
  



Finally, on each DNS and DHCP server you'll need to create a skeleton NetReg home directory. You should
do:
<ul>
  <li><tt>% mkdir $NRHOME; mkdir $NRHOME/etc</tt>
  <li>For DNS servers: <br><tt>% mkdir $NRHOME/etc/zone-xfer; chown $USER $NRHOME/etc/zone-xfer</tt>
  <li>For DHCP servers: <br><tt>% mkdir $NRHOME/etc/dhcp-xfer; chown $USER $NRHOME/etc/dhcp-xfer</tt>
</ul>


<!--
[variables]
{
  $page_title = "F.A.Q."
  $page_title_english = "F.A.Q."
  $file_prev = "007.html"
  $file_next = "009.html"
  $real_page_number = "8"
  $page_number = "8"
}
-->

<_header1>
$page_title
</_header1>


Here are some questions that have been posed on the mailing list and to the netreg-bugs
email address. The <a href=http://asg.web.cmu.edu/bb/archive.netreg-users>mailing list archive</a>
is also searchable.


<_header3>
Q: I'm getting lots of "XXX doesn't exist" ("subnet doesn't exist", "domain doesn't exist") errors, when clearly it does, because I just added it! What gives?

</_header3>


You probably need to add permissions for the resource (the subnet, DNS zone, or
whatever you are trying to use.) On the information page for the resource, click
"View/Update Protections", then either add "netreg:admins" with level 9 ADD access
(if you only want admins to be able to use this resource) or "system:anyuser" with
level 1 ADD access. Then try your operation again.


<_header3>
Q: I'm trying to debug DHCP/DNS (running ie "dns.pl -debug", "dhcp.pl -debug") and seeing "Cannot open services 
file: XXX".
</_header3>


Run <tt>$NRHOME/bin/service-dump.pl -debug</tt> first. The services file contains information used by
many of the backend scripts (such as dns.pl, dhcp.pl).


<_header3>
Q: What is the format of the input file for load-machines.pl (part of the loader collection)?
</_header3>


The machines load format is expected to be:


<_code>
#hostname|HW_address|IP_address|UID|Dept|status
</_code>


hostname, HW_address, and IP_address are fairly straightforward. 
UID is expected to be something like "kevinm" or "didier".



status should be 'C' for most normal machines and 'R' if the HW_address or IP_address fields are blank. 
There are a couple other valid 'status' codes, but those are the most common. 
('S' will get the machine flagged as stolen).



The IP_address field, if the IP is empty, should be '..' (or it's a fairly quick fix to 
load-machines to change this behavior).



Dept in our case was free form text, for example PRESIDENT'S OFFICE. 
load-machine uses a 'department mapfile' to map the free form text into a valid group id. 
The format of the mapfile is:


<_code>
#dept name|netreg group id|<br>
PRESIDENT'S OFFICE|34|<br>
</_code>


So in this case, group id #34 in the database would be something like dept:presoffice.



<_header3>
Q: How do I setup my DNS using the service groups?
</_header3>


Here's how CMU has it setup:



We have a 'view.default.bind9' service group, of type "DNS View Definition". 
There are several attributes of the group that are "DNS Parameter"s. They are:


<_code>
directory "/usr/domain/db"<br>
pid-file "/usr/domain/var/named.pid"<br>
auth-nxdomain yes<br>
fake-iquery yes<br>
transfer-format one-answer<br>
max-ncache-ttl 120<br>
</_code>


(Obviously these will vary from site to site, and I'd recommending reading the BIND9 ARM to 
understand the meaning of the various options.)




There is also a "Server Version" parameter (very important, it's compared with the server type 
to see if this view matters), of "bind9".



Then we have a "DDNS.dns" service group, type is "DNS Server Group". All of our DNS servers are members,
 and the attributes on them are "Server Type" ("master" or "slave") and "Server Version" of "bind9".



The view.default.bind9 group is also a member of this service; the attributes are:<br><br>
"Service View Name" is "_default_"<br>
"Service View Order" is "10".<br>



The service view name is important, since it specifies where in the zonefile to generate the view. 
_default_ is a "special" identifier, in that it appears as the "global" view.


<_header3>
Q: What are the various passwords in use? How do I get Apache authentication setup?
</_header3>


There are two sets of passwords here:
<ul>
<li>The passwords in /home/netreg/etc (/.password*) are used by the web system and support binaries to authenticate to the mysql server. These passwords are/should never be known by end users. In NetReg, the mysql server trusts the web system to do ALL access enforcement (ie, the web server is authenticated to the mysql server with an access level that allows it to add/modify/delete anything in the netdb database).
<li>The end user passwords used in authenticating to the system. Technically, you authenticate to the web server and NetReg trusts the web server to provide the proper credentials (it just reads the REMOTE_USER environment variable to pull the login name).
</ul>



A very basic test of authentication would be a .htaccess or <Directory> statement in httpd.conf that says:


<_code>
AuthType Basic<br>
AuthName NetReg Login<br>
AuthUserFile /path/to/userfile<br>
require valid-user<br>

</_code>


Then use 'htpasswd' to add an entry in '/path/to/userfile', and try logging in with this user and password. ( I suspect this is where you are. )



If NetReg doesn't present anything, verify that the username you are logging in with is in the netdb "users" table; a simple way to do this would be:


<_code>
mysql -u root -p netdb <br>
[enter your mysql root password] <br>
mysql> select * from users; <br>

</_code>


This will show you what users are configured in the system. If you don't have any users, then at least the initial administrative user needs to be added. This should be in the archive somewhere (http://asg.web.cmu.edu/bb/archive.netreg-users).



I'm still a bit unclear what you are seeing out of NetReg: most people get a "Unable to view user information" error when they are authenticated but the user is not in the mysql database."



Other things to check: look at your Apache access.log, and see what it says. Make sure that /bin/netreg.pl is actually returning a 200 response code, and that it shows your userid. For example:




<_code>
128.2.121.236 - kevinm@ANDREW.CMU.EDU [08/Oct/2001:11:22:32 -0400] "GET /bin/netreg.pl HTTP/1.0" 200 10804
</_code>


This shows my authentication. Using Basic Auth you won't have a @REALM.



Some codes and probable causes:
<ul>
<li>401: Generic authentication error?
<li>403: Incorrect directory permissions on $NRHOME/htdocs or htdocs/bin
<li>404: $NRHOME/htdocs/bin or $NRHOME/htdocs/bin/netreg.pl doesn't exist
<li>501: Could be an error with perl finding the right modules, etc. Check the Apache error log...
</ul>


<_header3>
Q: What are the Perl modules required to run the system?
</_header3>


We would hope the system requirements would cover this, but Cindy Smith compiled a comprehensive list
of the extra modules required on a RedHat 7.2 installation:



<ul>
<li>Convert-BER-1.31
<li>Crypt-DES-2.03
<li>Data-Compare-0.02
<li>Digest-HMAC-1.01
<li>Digest-SHA1-2.01
<li>Expect-1.15
<li>IO-Stty-.02
<li>IO-Tty-1.02
<li>Net-DNS-0.23
<li>Net-SNMP-4.0.2
<li>SNMP-4.2.0
<li>SNMP-MIB-Compiler-0.06
<li>SNMP-Util-1.8
<li>Test-Harness-2.26
<li>Test-More-0.08
<li>Test-Simple-0.45
</ul>


<_header3>
Q: What are the valid machine modes, and what do they all mean?
</_header3>


<b>Static</b>: This means that a hostname, MAC Address, and IP address are all present. DNS is updated with the hostname and IP, and DHCP has a fixed address statement to assign the machine an IP based on MAC address.



<b>Dynamic</b>: This means that a hostname and MAC Address are present. If the dynamic mode of a subnet (see the subnet information page) is 'restrict', then any machines registered as dynamic will be able to get a dynamic IP. DHCP can be configured to update DNS with the hostname of the record.



<b>Reserved</b>: A hostname must be present, but MAC address and IP are optional. If IP or MAC are specified, then the standard duplicate checking rules apply (no other machine can have that IP, and no other machine on the same subnet can have the MAC address).



<b>Base/Broadcast</b>: These are special modes reserved for a registration of the base or broadcast address of a subnet. This isn't required however.



<b>Pool</b>: These addresses must have an IP address, but no hostname or IP. They are used to provide IPs for dynamic IP allocation on a subnet. The subnet mode must be 'permit' or 'restrict' to allow ANY dynamic IP allocation ('permit' means that anyone, registered or not, will receive a dynamic address; 'restrict' means that the machine must be registered with type 'dynamic' to get an address).



<b>Secondary</b>: In the next NetReg release, the 'secondary' type will be available for a machine with multiple interfaces; the secondary addresses must have a MAC address matching another machine on the subnet (their "parent").


<_header3>
Q: How do the table permissions work?
</_header3>


Each entry in the protections table specifies:
<ul>
<li>User/Group: Values less than 0 indicate the (positive) group id. A value of 0 applies to all users.
<li>Table Name: The name of the table to which the protection applies.
<li>Table ID: The record ID in the table that the protection applies to. A value of 0 applies to all
records in the table.
<li>Rights: The rights can be 'READ', 'WRITE', and/or 'ADD'. See below.
<li>Level: The level of the access right. The valid values of level currently are 1, 5, and 9. Level 9
indicates the highest level of access.



Read/Write permission is additive between the "whole table" and "individual record" entries.
For example, if you had:


<_code>
netreg:admins, L9 READ on machine:*<br>
netreg:admins, L9 WRITE on machine:343

</_code>


then on record 343 netreg:admins would have both.



However, the ADD bit is special. If a user has ADD access to a particular table, they are authorized to add new entries to this table. For example, ADD access to the machine table authorizes a user to add machines.



ADD access on an individual record, however, means that the user is authorized to "use" this record elsewhere. For example, ADD access on a particular DNS zone means the user is authorized to add new machines into this zone.



We have a project underway to try and document more of the exact permissions combinations necessary to perform various functions. For example, adding a machine requires something like:



<ul>
<li>L1 ADD on machine:*
<li>L1 ADD on the forward DNS zone
<li>L1 ADD on the reverse DNS zone
<li>L1 ADD on the subnet
<li>plus the forward zone needs to be attached to the subnet
</ul>


<!--
[variables]
{
  $page_title = "Development"
  $page_title_english = "Development"
  $file_prev = "008.html"
  $file_next = ""
  $real_page_number = "9"
  $page_number = "9"
}
-->

<_header1>
$page_title
</_header1>


Known Issues



There are a few known issues that are being addressed:



<em>Please visit bugzilla.</em>



Database (DBMS) Notes



NetReg was designed using the MySQL DBMS. Why did we choose this platform?




<ol>
  <li>MySQL is relatively easy to build, install, and manage. We are
       primarily networking people, and don't want or need a fulltime
       database manager. MySQL can be easily managed as part of many other
       pieces we support.
  <li>MySQL is open source, and we like open source software.
  <li>MySQL offered the features we feel are critical.
</ol>



Some criticism is levelled against MySQL for not supporting foreign keys
or transactions. Recent versions of MySQL use the InnoDB table type to support
both of these methods. We have our database using InnoDB, as they are more resilient
to corruption in our experience. We have not yet made use of foreign keys or transactions
in NetReg yet, though NetMon is using transactions in some functions.




  Now, if you want to port NetReg to use another DBMS, here are some of our
thoughts:




<ul>
  <li>It should be relatively easy to use a different Perl DBD module. Just
    look in CMU/WebInt/config.pm and change the appropriate variable.
  <li>All in all, we believe most of the DB features used are fairly standard,
    but an exhaustive comparison to other DBMSs has not been done. However,
    knowing what we do, the following structures that NetReg uses may not
    be widely compatible:
    <ul>
       <li>We use statements such as<br>
        <tt> SELECT COUNT(DISTINCT()) </tt><br>
       Ironically this was supported in MySQL "Just in Time" for NetReg
       <li>The schema uses types such as TINYINT, SMALLINT, INT, and TEXT,
       which may be incompatible
     </ul>
</ul>




Change Log


<_header3>
Stable Release 1.1.7
</_header3>


Release 1.1.7 contains some under-the-hood improvements and bug fixes.



Noteworthy changes in 1.1.7 are:


<_header4>
Cascade Deletion
</_header4>


A new cascade deletion / update engine enables a high degree of data consistency in
an environment without foreign keys (MySQL pre-InnoDB). Even with DBMS foreign keys, the
system can enforce similar constraints at the places in our schema that can reference
one of several tables.



The benefit of this system will be greater consistency, as deletion of dependent elements
will either not be permitted or will be done automatically with the deletion of a master
element (for example, DNS resources of a zone being deleted will be cleaned up automatically).


<_header3>
Stable Release 1.1.6
</_header3>


Release 1.1.6 contains new features and bug fixes. If you are running release 1.1.4 or 1.1.5,
you should read these notes and use the <tt>support/convert/convert-1_1_5-1_1_6.pl</tt> script, and
test your upgrade plans on a non-production server prior to rolling it out.



The major changes to 1.1.6 are:


<_header4>
Installation Improvement
</_header4>


The netreg-install.pl script has been improved; it now has the ability to read
Netdb/config.pm and WebInt/config.pm and configure the variables in each.
Certain variables are flagged as important, and netreg-install can present only
the important variables, or all the variables, for configuration.


<_header4>
Improved Netdb Debugging
</_header4>


Throughout the Netdb modules, all error debugging is tagged with the file and
line number generating the output.


<_header4>
Protection Profiles
</_header4>


Adding new subnets, zones, and services results in automatic ADD access 
granted to netreg:admins. This is implemented by "protection profiles",
controlled via the Netdb/config.pm module.


<_header4>
DHCP Subnet Default Options
</_header4>


With this release, all of the options in the DHCP configuration file can be
controlled from within the web interface/database. Additionally, so-called
"subnet default" options can be specified. Assuming no subnet-specific
option, a subnet default option will be added to the configuration.


<_header4>
Other features/bug fixes
</_header4>


Minor features and bugfixes include: <ul>
  <li>QuickReg bugfixes - eval { } protection of omshell
  <li>Error messages reported to user can now be recorded in the Apache logfile
  <li>Improved basic error messages
  <li>CNAMEs may now be added to any zone permissible on the subnet by any user
(previously non-admins could only add CNAMEs to the same zone as the primary machine)
  <li>DHCP failover declarations use IP addresses, not hostnames.
</ul>



<_header3>
Interim Release 1.1.5
</_header3>


Release 1.1.5 corrects minor fixes in the 1.1.4 release; namely, several files were
sticky tagged in the CVS release sandbox that should not have been tagged.


<_header3>
Stable Release 1.1.4
</_header3>


Release 1.1.4 contains new features and bug fixes. If you are running release 1.1.3, you should read these
notes and use the <tt>support/convert/convert-1_1_3-1_1_4.pl</tt> script, and test your upgrade plans on
non-production server prior to rolling it out.



The new features of 1.1.4 are:


<_header4>
Secondary IP Support
</_header4>


Support for secondary IP address interfaces of machines has been added. This enables a single MAC address
to be registered to more than one IP address on a subnet. However, only the 'static' (primary)
registration will be propogated into DHCP. If you have a server providing services on multiple IPs
(for example, an SSL-protected web server with multiple virtual domains), this enables the secondary
IP addresses to be registered using mode 'secondary'


<_header4>
QuickReg Feature
</_header4>


QuickReg is a new feature that provides the fastest possible registrations to users. NetReg queries the
DHCP server for the MAC address of an unregistered host, and determines the correct registration
subnet. It's "one-click" registration.


<_header4>
"Hot Keys" in Web Interface
</_header4>


Using new HTML features, we have enable "hot key" support throughout NetReg. In NetReg forms, pressing
Alt-<Key>, where <Key> is the first letter of the field name, will toggle the cursor to this field.



<_header4>
Bulk IP Registration Fix
</_header4>


Bulk IP Registration (a link on the Subnet Information page) was introduced in prior versions. However,
there were fundamental bugs involving the user and department information attached to the new 
registrations. The web front-end provides fields to enter this information.


<_header4>
Removed DNS Server Software Section
</_header4>


Prior to having the DNS server configuration information derived from service groups, there was a
dedicated section to handle these operations. This section has been inaccessible in the past few
releases, but this release officially removes all of the relevant code.


<_header4>
Advanced Permissions Matrices
</_header4>


Access to perform certain features (such as change the IP address of a host without being an administrator)
has been implemented. See the <tt>doc/netreg-permissions.txt</tt> file for the exact permissions
required to perform certain operations. Not all operations are documented; we are hoping to improve
the completeness of this file with time.


<_header3>
Stable Release 1.1.3
</_header3>


Release 1.1.3 contains new features and bug fixes. If you are running release 1.1.2, you should read these
notes and use the <tt>support/convert/convert-1_1_2-1_1_3.pl</tt> script, and test your upgrade plans on a
non-production server prior to rolling it out.



The new features of 1.1.3 are:


<_header4>
"Switch User" capability
</_header4>


This feature enables administrators to look at NetReg and see exactly what another user sees. This is 
very useful in debugging and user education. When acting as another user, an orange bar appears at the
top of each web page, indicating the alternate user and providing a link for logging out. This is
implemented as an extra cookie. Verification is done on each page load that the true identity is
that of a SuperUser (users of this feature must be listed in the SuperUsers array in CMU/WebInt/config.pm).
Access to this function is via the "Users/Groups" area of NetReg.


<_header4>
Outlet Subnet Membership/Vlan capabilities
</_header4>


Outlets can now exist on a single subnet or multiple subnets. They can be configured as a trunk port, 
with appropriate settings. This feature is useful for enabling VoIP (using Cisco's "voice 
vlan" settings). The backend is done via Perl and Expect (in terms of actually setting vlans, etc.). 
Long term the back-end will most likely be extracted out completely.


<_header4>
NetMon InnoDB Tables
</_header4>


At this time, we highly recommend the use of MySQL's InnoDB table types for NetMon. We are currently running
version 3.23.49a of the server; 3.23.50 promises to solve a few bugs (keeping foreign key references
intact across an ALTER TABLE, for one). The InnoDB table offers substantially improved parallelism
with row-level locking. Additionally, it provides support for transactions and foreign keys. The code base
has changed to use the InnoDB features (such as the SELECT ... FOR UPDATE syntax), thus you will most likely
want to stay with 1.1.2 and non-InnoDB tables, or upgrade both.


<_header4>
NetMon Topology Improvements
</_header4>


The NetMon Topology code has undergone substantial improvements in determining the exact layer-three
network topology. Unfortunately, the code in 1.1.3 is most likely not going to provide any useful output,
as it is still undergoing heavy reorganization. However, if you want to see the latest and greatest,
check it out.


<_header4>
Improved Consistency Checker Routines
</_header4>


Several consistency checking routines have been added.


<_header4>
DNS Resource Searches
</_header4>


You can now search the DNS resources table from the web interface. There is a link to this search off
the "Search Machines" form. Additionally, traditional machine searches that would also be legal as a 
DNS resource search (ie, just searching by hostname) will generate a link to an equivalent DNS resource
search if no machine results are found.


<_header4>
DNS Load Balancer Available
</_header4>


Release 1.1.3 features our newest DNS load balancer. This load balancer uses DNS UPDATE to perform
updates against specific zone records based on input from the members of a pool. The method of collecting
this data (such as number of users, number of processes, etc.) is modular. We have written collectors
that use SNMP (standard Unix MIBs) and our in-house "KMemSrv" server. Updates are performed very rapidly
and react to the gathered statistics.


<_header4>
Permissions Updates
</_header4>


The file <tt>doc/netreg-permissions.txt</tt> now contains the beginnings of documentation about
the exact permissions required for various operations.


<_header3>
Stable Release 1.1.2
</_header3>


Release 1.1.2 contains a number of changes and new features. If you are running release 1.1.0 or 1.1.1,
you should make sure to read these notes, use the <tt>support/convert/convert-1_1_1-1_1_2.pl</tt> script, and
test your upgrade plans on a non-production server prior to rolling it out.




The new features of 1.1.2 are:



<_header4>
Service Group Membership
</_header4>


Service Groups previously were composed of member machines and attributes. We have extended the 
<tt>service_membership</tt> table and related functions to support membership of many other NetReg
structures, including subnets, zones, users, groups, and even other service groups. This provides
amazing flexibility in defining additional service types.


<_header4>
DNS Server Configuration
</_header4>


Previously DNS Server configuration (zones, master/slaves setup, etc.) was controlled through the
DNS Server area. This has been changed to be controlled as a number of service groups. While this is
slightly less intuitive, the additional flexibility outweighs this drawback. You can easily define 
additional attributes of DNS servers, and then just update the <tt>dns.pl</tt> output script to
translate your attribute into configuration bits -- no work to the <tt>Netdb</tt> or <tt>WebInt</tt>
trees are necessary.



Be sure to re-read the DNS server configuration sections for a full description of the current service
group implementation.


<_header4>
DHCP Option Validation, DHCP Classes
</_header4>


We have written a full recursive-decent parser to validate the format of all DHCP options. This should
make it safe to allow end users to define DHCP options for machines under their ownership. In previous
versions this was not recommended because an invalid option specification would break the DHCP server
configuration.



The syntax for the DHCP Option format closely mirrors that of the ISC DHCP server format (<tt>man dhcp-options</tt>.)
It is fully described above.



Additionally, we have added a DHCP Class service group for definining various DHCP classes. The only
attribute is the match parameter string. However, service groups were extended to allow DHCP Options
to be attached to them. The options attached to a DHCP Class service group are included in the class
definition in your DHCP server configuration.



Finally, all DHCP Option Type (specifications) now include a <tt>builtin</tt> parameter for specifying
whether a given option is known by the DHCP server, or not. Options that are not 'builtin' will have
the output format specification added to the configuration.


<_header4>
Scheduler Frontend
</_header4>


We added a web interface to control the scheduler's tasks. User/groups with Level 1 WRITE access will
be allowed to run the 'Force Now' or 'Back Off' operations to control scheduling of certain tasks. Users
with Level >= 5 WRITE access will be able to perform these operations, as well as change the name, 
default interval, or blocked time of a task. The blocked time is a hard "next run" time. The point of
the interface was to allow a wide(r) group of people have some control in critical periods of
DNS/DHCP propogation, while the smaller administrator can set a blocked time if work is being
done, etc


<_header3>
Interim Release 1.1.1
</_header3>


This release changed a few default configuration options for the use of the general public.


<_header3>
Initial Public Release: 1.1.0
</_header3>


This is the initial public release. The entire manual has been written for this purpose, as well as the
development of initial database data. We welcome all comments, suggestions, offers to fix things, etc.
There have been a number of changes since the initial release; too numerous to list.


<_header3>
Initial Release: 1.0.0
</_header3>


The initial release of August 14, 2000, was the first release to campus users of NetReg.



To-Do


<_header3>
In Progress
</_header3>


The following changes are in-progress:



<ul>
  <li>User Attributes: Change "service attributes" to also include user/group attributes (schema
changes already done). Allow user attributes and develop user preferences, to include persistent
sort order of machines/outlets, etc. Changes in progress, no target date.
  <li>Stop using the BIND 'nsupdate' utility for performing DNS updates; instead use the Perl
Net::DNS::Update package.
</ul>


<_header3>
In the Future
</_header3>


The following changes are desired, but no active development is currently occurring:



<ul>
  <li>User attribute: selectable disabling of paging on various screens
  <li>Make the outlet database more generic (not as specific to our labelling standard, etc.)
  <li>Implement CSS throughout NetReg; pull CSS from <tt>WebInt/interface.pm</tt> and put it into a separate styles.css file
  <li>Make the subnet map interface more generic (not pegged to 128.2 and 128.237 networks): base the list
on subnets
  <li>Cleanup dns_resource rname_tid usage -- make it the canonical reference where appropriate
</ul>



Carnegie Mellon Developers



If you need help with NetReg, please use one of the contact methods listed earlier (sending mail to
<a href="mailto:netreg-bugs@andrew.cmu.edu\">netreg-bugs@andrew.cmu.edu</a> is one). This will lead
to speedier resolution of your problems.


<_header5>
Kevin Miller
</_header5>


<ul><LI>Developed most of the main NetReg codebase
  <LI>Designed, developed most of the NetMon system
  <LI>Developed all DNS zonefile/DDNS update operations
  <LI>Overhauled DHCP and other output scripts
  <LI>Wrote this manual
  <LI>Developed attribute/attribute spec and formatting
  <LI>Developed several of the reports (subnet usage, subnet map)
  <LI>Developed DNS Load Balancing system
</UL>



<_header5>
Dave Nolan
</_header5>


<UL>
  <LI>Developed NetDB skeleton/ground work
  <LI>Extensive work to Telecom cable management system
  <LI>Developed outlet vlan configuration system
  <LI>Numerous patches for bugs, fixes, etc.
</UL>



<_header5>
Chaskiel Grundman
</_header5>


<UL>
  <LI>Wrote the consistency checker
</UL>



<_header5>
Kunal Trivedi
</_header5>


<UL>
  <LI>Subnet policy pages
  <LI>LDAP and local user group
</UL>




<_header5>
Ed Bardsley
</_header5>


<UL>
  <LI>Cleaned up DNS resource and DNS zone problems
  <LI>Developed secondary IP support
  <LI>Implemented hot keys throughout the web interface
</UL>



<_header5>
Russell Yount
</_header5>


<UL>
  <LI> Project Manager
  <LI> Initial database schema design
</UL>



<_header5>
Ros Neplokh
</_header5>


<UL>
  <LI>Initial DHCP configuration generation
  <LI>Initial DNS configuration file generation
</UL>



<_header5>
Jim Foraker
</_header5>


<UL>
  <LI>Designed initial outlet interface
</UL>


<_header5>
Ken Tew
</_header5>


<UL>
  <LI>Developed additional reports
</UL>