The Big Brother Systems and Network Monitor

| help

# Frequently Asked Questions
    		The Big Brother System and Network Monitor
    ------------------------------------------------------------------------
    			Frequently Asked Questions
    		    Version 1.9c  - May 15th 2002
    				LICENSE
    ------------------------------------------------------------------------

    1.0	About Big Brother

    	1.1	What is Big Brother?
    	1.2	Where can I get Big Brother?
    	1.3	What is the current version?
    	1.4	What do I need to run Big Brother?
    	1.5	How do I install Big Brother?
    	1.6	How can I upgrade?

    2.0	Debugging Big Brother

    	2.1	I get the message: "bb: CAN'T CONNECT TO bbd"
    	2.2.1	I get lots of processes, then bb dies!
    	2.2.2	bbnet test dies
    	2.3	I get garbage / my environment all over my screen
    	2.4	I get the message: "Can't open stream socket"
    	2.5	conn (connection) test is always red / not working...
    	2.6	http test is always red / not working...
    	* 2.7	bbnet is dumping core
    	2.8	Background color is always red / yellow / wrong
    	2.9	Pager problems
    	2.10	I've removed a host in bb-hosts and the screen is purple
    	2.11	The gifs aren't displayed properly
    	2.12	I'm not getting numeric pages
    	2.13	Generating trace code to debug the notification feature

    3.0	Using Big Brother

    	3.1	How can I monitor routers / things that have no hostname?
    	3.2	Can I monitor NT's, Novell servers, VAXEN with BB?
    	3.3	Can I monitor things outside my network?
    	3.4	Is Big Brother secure?  Do you have to be root to run it?
    	3.5	How can I monitor more TCP services?
    	3.6	How can I check password protected web pages?
    	3.7	Can BB restart processes that have failed?
    	3.8	Can BB show historical data?
    	3.9	How can I add my own tests ?
    	3.10	Can I check for errors in multiple log files ?
    	3.11	Starting BB at system boot
    	3.12	Why is availability report not working ?

    4.0 	Miscellaneous Big Brother questions

    	4.1	Where is the name from?
    	4.2	Do you write BB or bb?
    	4.3	Whose picture is that, and can I get rid of it?
    	4.4	Is there a Big Brother Mailing List?
    	4.5	Is there a Big Brother FTP site?
    	4.6	Where can I get more help?

    5.0      Security Considerations

    ------------------------------------------------------------------------
    This FAQ is © Copyright Quest Software, Inc.  1997-2002  All rights reserved.
    ------------------------------------------------------------------------

    THE BIG BROTHER SYSTEM AND NETWORK MONITOR
    ==========================================
    Version 1.9c
    May 15th 2002

    This program is Copyright (c) 1997-2002
    BB4 Technologies Inc.
    All Rights Reserved

    The Big Brother System and Network Monitor
    is not free software.  The terms and conditions
    under which you may use the Big Brother System
    and Network Monitor at no charge is detailed
    in our license below.  If you need a commercial license, you
    can get one online from http://bb4.com/

    We've put thousands of hours into our software, please
    take 30 seconds to read our license, it's not long, or
    complicated, but it is important,

    Thanks.

    ====================================
    License Agreement & Lack of Warranty
    ====================================

    By installing, copying or otherwise using this software
    product, you agree to abide by the following terms and
    conditions:

    1.	NON-COMMERCIAL USE

    	No part of the Big Brother System and Network Monitor
    	may be used for any commercial purpose after a 30
    	day evaluation period without obtaining a commercial
    	license from BB4 Technologies Inc. Commercial purpose
    	includes any activity engaged for the purpose of directly
    	generating revenues including the sale of the product,
    	use of the product to provide a service or in support
    	of a service for which you charge.

    2.	TRADEMARKS AND NOTICES

    	The software, graphics and documentation which
    	make up the Big Brother  System and Network
    	Monitor are Copyright (c) 1997-2002
    	BB4 Technologies Inc.  "Big Brother System
    	and Network Monitor" is a trademark of
    	BB4 Technologies Inc.  You agree to respect
    	these rights and leave all notices and product
    	references intact.

    3.	REDISTRIBUTION

    	This software may not be redistributed without prior
    	written permission from BB4 Technologies Inc.

    4.	REQUIRED LINKS

    	At least one link to the Big Brother System
    	and Network Monitor home site at http://bb4.com
    	must be available from the status pages
    	generated by the Big Brother  System and
    	Network Monitor so that others may obtain a
    	copy of this software.

    5.	LIMITATION OF LIABILITY AND DISCLAIMER OF WARRANTY

    	You understand that this software is provided as-is.
    	BB4 Technologies Inc. makes no claims towards its
    	suitability for any purpose and accepts absolutely
    	no liability for any damages the software may cause.
    	Use at your own risk.

    Questions? Comments?

    Contact:
    	BB4 Technologies Inc.
         	E-mail: info@bb4.com
          	Tel: +1 (514) 996-INET
          	Fax: +1 (514) 996-0326

    ------------------------------------------------------------------------

    Section 1: About Big Brother

    1.1	What is Big Brother?

    	Big Brother is a Web-based Systems and Network monitor written by
    	Sean MacGuire (sean@bb4.com) and Robert-Andre Croteau (robert@bb4.com).

    	Big Brother consists of simple shell scripts which periodically
    	monitor system conditions and network connectivity.  Disk space,
    	CPU, servers, and important processes can be kept track of.
    	Unix and NT systems are supported  by BB4 Technologies Inc.
    	but we've heard of clients for Netware, /AS/400 and VMS.

    	The Big Brother display is a Web page that presents a matrix of
    	machines and monitored functions, with color codes denoting the
    	current status.

    	Big Brother can notify administrators via a pager, e-mail or SMS.

    1.2	Where can I get Big Brother?

    	Big Brother is only available via the web at http://bb4.com/

    	If you don't have Web access, drop a note to sean@bb4.com
    	and a copy will be mailed to you.

    1.3	What's the current version?

    	The current version of BB is 1.9c / May 15th 2002.

    1.4	What do I need to run Big Brother?

    	Big Brother for Unix is written as Bourne Shell scripts (/bin/sh),
    	with a couple of C programs for client-server communications.  You'll
    	need:

    	* A C compiler to port BB
    	* A Web server to serve up the results

    	For Paging, we recommend:
    	* Kermit (http://www.kermit-project.org/) and a modem (for numeric pager communications)
    	* Qpage (http://www.qpage.org) or Sendpage for Alpha pagers

    	BB has been ported to and configuration files are available for
    	most Unix and linux-based systems.

    	Big Brother for NT requires Windows NT 4.0 with at least service
    	pack 3 installed.  It is available for the Intel and Alpha platforms.

    1.5	How do I install Big Brother?

    	*** READ THE README.SECURITY FILE BEFORE PROCEEDING ***

    	Unpack the archive, read the README.INSTALL and follow the
    	instructions.  Basically
    		cd install
    		./bbconfig
    		cd ../src
    		make
    		make install
    		cd ../etc/
    			edit etc/bb-hosts, bbdef.sh, bbwarnrules.cfg,
    			and bbwarnsetup.cfg
    		and start it
    			cd ..
    			./runbb.sh start

    	If you don't understand the above, read the README.

    1.6	How can I upgrade?

    	Save your old bb-hosts/bbwarnrules.cfg/bbwarnsetup.cfg
    	files... then...

    	Depends how much customization has been made to your version
    	of Big Brother.  Generally, all you should have to do is
    	recompile, make the above changes, and copy your old version
    	of etc/bb-hosts in. You may have to copy external scripts in
    	ext/ if you are using some.

    ------------------------------------------------------------------------

    Section 2: Debugging Big Brother

    2.1	I get the message: "bb: CAN'T CONNECT TO bbd"

    	This message indicates that an instance of bb can't
    	connect to the Big Brother daemon.  This might be because
    	bbd isn't running, or that bb can't determine where bbd
    	lives because of some troubles with the bb-hosts file.

    	So check the following things:

    	* bbd is actually running on your system...
    	* BBHOME is correctly set in runbb.sh...
    	* Your bb-hosts file is formatted correctly
    	* Your firewall isn't blocking port 1984

    	Some errors can be caught by:

    	cd etc
    	run ./bbchkcfg.sh
    	run ./bbchkhosts.sh

    2.2.1	I get lots of processes, then bb dies!

    	Define -DZOMBIE in the Makefile, recompile and run "make install".
    	This seems to happen on some Solaris machines, and will definitely
    	happen on SunOS 4.1.3, although 4.1.4 is OK!

    2.2.2	bbnet test hangs

    	Define -DSIGSETJMP in the Makefile, recompile and run "make install".
    	This seems to happen on some RedHat machines.  Signals aren't
    	handled properly.    Could also happen on other Linux distributions.

    2.3	I get garbage / my environment all over my screen

    	This is almost always due to a problem with the way your bb-hosts
    	file is laid out.  BB needs this file to be perfect to work, and
    	any little problem with it will cause BB to fail.

    	The most common cause of this problem is pop3 being defined as
    	pop-3 in /etc/services.  Make sure the spelling of all services
    	in bb-hosts matches /etc/services.

    	Make sure also that the BBDISPLAY/BBPAGER are defined only once in
    	the etc/bb-hosts file.  Also make sure that the hostnames defined
    	in etc/bb-hosts are the same as returned by 'uname -n'.

    2.4	I get the message: "Can't open stream socket"

    	This message is from bbd being unable to attach itself to port
    	1984 and begin listening.  Make sure there are no "bb" processes
    	running (bb, bbd).  If there are, kill them.

    	Make sure port 1984 is also not in use.  To check this issue
    	the following command:

    		netstat -an | grep 1984

    	If anything comes back, wait a few minutes and try again.
    	Once this command returns nothing, you should be able to
    	start up Big Brother.

    2.5	conn (connection) test is always red / not working...

    	The connections column is generated from the machine defined
    	as BBNET in bb-hosts.  This machine tries to ping every IP
    	address listed in the bb-hosts file. BB looks to see that the
    	reply from ping contains the string "bytes from".

    	Check that PING and PINGPARS are set correctly in etc/bbsys.sh
    	or etc/bbsys.local.

    2.6	http test is always red / not working...

    	This is usually because the http test isn't for the same
    	machine as defined on that line in the bb-hosts file, i.e.:

    	Wrong: 204.101.110.101 fred.bobo.com # http://youre.bobo.com/
    	Right: 204.101.110.101 fred.bobo.com # http://fred.bobo.com/

    2.7 *	bbnet is dumping core

    	Add a trailing slash at the end of the URL.  This is a programming
    	bug by the author.   Fixed as of v.1.04g thanks to Doug White
    	<dwhite@gdi.uoregon.edu>

    2.8	Background color is always red / yellow / wrong

    	The background color should reflect the most serious state
    	on your network at any given time.  If it's not doing this, or
    	the background color is wrong, it's because there are some
    	leftover log file in the $BBLOGS directory (BBLOGS is
    	defined in etc/bbinc.sh).  To check this, hit the
    	VIEW button on the main web screen, and the offending
    	entries should become visible.  Delete them.  They live in
    	$BBLOGS and an HTMLized version is in www/html.

    2.9	Pager problems

    	The paging subsystem is really time sensitive.  It's possible
    	that the timing be either too long or two short for your pager.

    	The following comes from  Don Carney <dcarney@fmi.fujitsu.com>:
    	In the etc/numeric.scr where it actually dials the number.
    	the command is something like
    		dial /@[3],,,,,,,,/@[4]
    	my fix was to remove a few of the commas, and everything worked
    	fine.  (Commas are generally used by modems for short delays).

    	Similarly if you're using one of those 800 number paging services
    	you'll probably have to embed these commas in the pager number
    	itself, something like:
    		PAGER="1800PAGENET,,,,,,7777"
    	where 7777 is your account number.

    2.10	I've removed a host in etc/bb-hosts and the screen is purple

    	After you remove a host(s) in etc/bb-hosts, you must remove the
    	corresponding files in $BBLOGS, www/html and $BBHIST. BBLOGS and
    	BBHIST are defined in etc/bbinc.sh

    	cd www
    	rm logs/thedeletedhost*
    	rm html/thedeletedhost*
    	rm hist/thedeletedhost*

    2.11	The gifs aren't displayed properly

    	Make sure that the BBWEB value defined in bbdef.sh is
    	correct.  Check spelling and location. Defined it
    	BBWEB="/bb" where /bb is the location in your web server
    	docs directory.

    2.13	Generating trace code to debug the notification feature

    	If you're having problems with the notification feature and
    	do not understand why it's not working, you can enable trace
    	code to help you figure you what's going on.  Here's the
    	steps to enable the trace code:

    	in bbpage.c, right after the #include definitions, add

    	#define DEBUG 1

    	then

    	make bbd
    	cd ..
    	./runbb.sh stop
    	rm BBOUT
    	cp src/bbd bin/

    	<make sure all BB processes are killed>

    	./runbb.sh start

    	Then when the next notification should have been sent out or
    	whatever problem with notification has occured, take a look
    	at BBOUT and follow the trace code.  You should get an idea
    	of what's going on.

    	Don't forget to remove the "#define DEBUG 1" and recompile
    	bbd when you're all done or your BBOUT file will grow quite rapidly.

    ------------------------------------------------------------------------

    Section 3: Using Big Brother

    3.1	How can I monitor routers and things that have no hostname?

    	Just put a line in the bb-hosts file and make up a name
    	for your router.

    3.2	Can I monitor Novell servers, VAXEN, AS/400 with BB?

    	You can monitor them from the outside, but not from the
    	inside.  That means the bbnet tests that check for connectivity
    	and servers should work, but the bb-local tests which monitor
    	processes and disk space won't because there is no BB client
    	for these systems.

    	Without a client program for each platform you can still monitor
    	connectivity and IP services.  Clients have been written for
    	these platforms but BB4 Technologies Inc. doesn't support these
    	3rd-party clients.  Please check with the mailing list for
    	more info on these (http://support.bb4.com/).

    3.3	Can I monitor things outside my network?

    	Yup.  Just put the appropriate lines in the bb-hosts file and
    	that's all.  However it is good form to ask permission, just
    	because the remote admin may get curious about repeated accesses
    	from the same addresses every 5 minutes, 24 hours a day.

    3.4	Is Big Brother secure?  Do you have to be root to run it?

    	A certain amount of effort has been made to make sure that
    	BB is reasonably secure.  We also recommend running bb as
    	it's own, non-root, user.

    3.5	How can I monitor more TCP services?

    	Add the TCP service in the list defined by the BBNETSVCS
    	variable in etc/bbdef.sh.  That service must be in
    	/etc/services or you must define it with its port number
    	in the bb-hosts file.  That's it, that's all.  Simple.

    	Stop and start BB. It'll magically have a new column
    	watching your service. The TCP service that is checked
    	should always return something at connection time to
    	properly be checked.

    	Only add TCP services as UDP services aren't supported yet.

    3.6	How can I check password protected web pages?

    	Paul Venezia had the answer for this one:
    	I've gotten around this by specifying LYNX to be
    	/usr/contrib/bin/lynx -dump -auth <username>:<password>

    3.7	Can BB restart processes that have failed?

    	No, that is your job. BB will tell you about the problem, you
    	solve it.  The philosophy is simple, BB will monitor and notify,
    	that's all.  The reason behind this is simple, doing more than
    	that makes BB exponentially more complex to run, configure and
    	support.

    3.8	Can BB show historical data?

    	Yes.  The history is in the $BBHIST directory: the file contains
    	the date of the last color change.  If you drill down to a
    	specific host.service, you can click on the history button, and
    	it'll show your the last 24hr statistics and a log of the last
    	50 status changes.

    3.9     How can I add my own tests ?

            You can easily add your own tests.  Start with the template
            available at ext/ext-proto, add your code.  Look at bb-local.sh,
            bb-network.sh for an example on how to send data to BB.
            Specify the name of your script in the etc/bb-bbexttab
            file. This file defines all scripts to run for each hosts.
            Start with the etc/bb-bbexttab.DIST file, copy to
            etc/bb-bbexttab and configure. Typical entries in
            etc/bb-bbexttab:

            www.bobo.com : script1 script2;3600 script3;900 script4
            www.baba.com : script5 script6

            make sure each defined script exists in the ext directory
            of the client install.  See below for an explanation of
            the script2;3600 entry.

            Restart BB and your test should be running.  But
            before you use it within BB, I suggest you test it for errors
            by using the method:

            cd /home/bb (or wherever your BB is located)
            BBHOME=/home/bb
            export BBHOME
            . ./etc/bbdef.sh
            cd ext
            ./yourexternaltest

            Look for errors, fix them, rerun your test until you're
            satisfied, then update bbdef.sh.  Note that all temporary
            files should be created in $BBTMP and make sure you remove them
            after use.   Also, remember that you don't have to deal with
            sending notification messages, the 'bb' process send a 'page'
            type message to the BBPAGER host when the status color is
            found in the PAGELEVELS variable defined in bbdef.sh.

            You can also set the frequency of the test by specifying
            the interval it should run at.  In ext/bb-bbexttab, set an
            interval qualifier with the script name by appending the
            qualifier to the script name. Here's an example entry in
            etc/bb-bbexttab:

            www.bobo.com : script1 script2;3600 script3;900 script4

            script1 and script4 will run every 5mins (the default)
            while script2 and and script3 will run every 60 and 15
            minutes respectively.  You specify the interval in seconds.
            Note that the delimiter is ';'.

            When it's ready don't forget to update the svcerrlist token
            in the bbwarnsetup.cfg file on your BBPAGER host.  You must
            assign a numeric code to your column name.

            You can also set the lifetime of the status sent by your
            script by following the instructions described in
            section 6.8: "Setting a Time To Live to a status message".

    3.10    Can I check for errors in multiple log files ?

    	You can set multiple log file names in the MSGFILE variable
    	in etc/bbsys.local.  The log files will be checked that
    	they are readable and not empty.  The empty test is done
    	because some hackers sometimes link log files to /dev/null.

    3.11     Starting BB at system boot

            Depending on U*X operation system version (Linux/BSD/Solaris/...)
            your startup procedures will vary from OS to OS.  In short, you
            have to create a startup script that has a start/stop
            capabilities.  Under a lot of OSes, you'll want to create your
            script in the init.d directory and create an S89bb link in rc3.d that
            links to init.d/S89bb (you may also want to create a K11bb link in
            rc3.d that'll be used when the system shuts down).  Use an existing
            startup script has an example and substitute with these commands:

            To start BB, use this command:

            su - <bbuser> -c "cd <BBHOME>;./runbb.sh start"
                    or
            su - <bbuser> -c "cd <BBHOME>;./runbb.sh restart"

            To stop BB, use this command:

            su -  <bbuser> -c "cd <BBHOME>;./runbb.sh stop"

            <bbuser> is the user that BB will execute has
                    make sure that bbuser has all permissions under BBHOME
            <BBHOME> is the location of your BB install

            e.g. su - bb -c "cd /home/bb;./runbb.sh start"

    3.12    Why is availability reporting not working ?

    	If you get a "Page not found" when running the availability
    	reporting feature, you probably have an invalid group name
    	set for $BBHOME/www/rep.

    	The group name of $BBHOME/www/rep must be set to the group
    	id of the user that the web server is running as.  Do not
    	set the $BBHOME/www/rep permissions to 777 as this may
    	represent a security risk.  Only set the group name of
    	$BBHOME/www/rep.

    ------------------------------------------------------------------------

    Section 4: Miscellaneous Big Brother questions

    4.1	Where is the name from?

    	Big Brother is named for George Orwell's novel Nineteen Eighty Four.
    	Big Brother is the head of a totalitarian regime, INGSOC, where
    	everyone is watched.  "... the poster with the enormous face
    	gazed from the wall.  It was one of those pictures which are
    	so contrived that the eyes follow you about when you move.
    	BIG BROTHER IS WATCHING YOU, the caption beneath it ran."

    	Doubleplus ungood for people.  Doubleplus good for networks.

    4.2	Do you write BB or bb?

    	Doesn't really matter.  bb tends to be used when denoting
    	programs (i.e. bbnet) whereas BB tends to be used when
    	discussing the entire Big Brother system.

    4.3	Whose picture is that and can I get rid of it?

    	That picure is of the creator of Big Brother, Sean MacGuire,
    	doing his best to do justice to George Orwell.  It's supposed
    	to be scary.... it now lives exclusively at the bottom of the
    	BB help files.

    	If you don't like it, feel free to change it to something more
    	neutral.  Change the file $BBHOME/www/gifs/bb.gif.  Please leave
    	a link back to the BB site, though.

    4.4	Is there a Big Brother Mailing List?

    	Yes, please subscribe by sending e-mail to MajorDomo@bb4.com.

    	In the body of the e-mail message (not the Subject line),
    	place the statement:

    		subscribe bb <INSERT-YOUR-EMAIL-ADDRESS-HERE>

    	An archive of the Big Brother mailing list available.
    	It can be found at the URL: http://support.bb4.com/

    4.4	Is there a Big Brother FTP site?

    	Yes, Adam Goryachev has set up an ftp site with user
    	contributed tools.  You can find it at:

    	ftp://ftp.deadcat.net/pub/BB
    		or
    	http://www.deadcat.net/

    4.5	Where can I get more help?

    	Run the tests as outlined on the install and debug web pages.

    	Subscribe to the mailing list.  Check the archives of the
    	mailing list to see if you're question has already been
    	answered.  Send a message to the mailing list, and as a
    	last resort, mail sean@bb4.com

    ------------------------------------------------------------------------

    Section 5: Security Considerations

    We care about security, and have a "full disclosure" policy.  That
    means if a security problem is discovered, we'll disclose it promptly
    to the BB mailing list, Bugtraq, and Freshmeat.net.  The reason for
    this is simple, if we know about it, so do the bad guys, and they're
    already exploiting it.

    If we issue a security alert, please follow the instructions if
    you're at risk.  If you discover a hole, please let us know
    immediately, and we'll fix it right away.  You will earn our
    undying gratitiude.

    The following suggestions are mostly targeted as BB display and pager hosts.
    These are the Big Brother daemons, and as such are higher risk than
    the simple clients.  If you have any additional suggestions, please
    pass them along!

    * Never install network software without considering the security
      implications.  If you have a security person, discuss it with them.
      If not, talk nicely to your Sys Admin.  If you're the Sys Admin,
      feel free to give us a shout on the BB mailing list if you have any
      questions.

    * Since you're probably running a Web server on the BBDISPLAY machine
      you might consider making sure it's secure.  Even Apache has gotten
      broken into, just from a misconfigured web server.  Beware!

    * BB does not need to run as root.  We suggest creating a user 'bb'
      and running bb as that user.

    * BB has the ability to restrict incoming connections to those IP
      addresses (and networks) listed in the etc/security file.  Use it.

    * If you're in an environment with a firewall, we suggest running
      two instances of BB, one on this inside of the firewall, and one
      on the outside.  This keeps things clean, and doesn't require any
      unnecessary holes in the firewall.

    * The usual warnings about scripts in the cgi-bin directory... make
      sure that your webserver isn't running as root, and be careful what
      can be seen and run by outsiders.

    * We recommend password-protecting the Big Brother web pages

    * Don't compile in the "notes" and "disable/enable" features of
      the BB display/pager hosts unless you understand the implications.
      Refer to the documentation for more information.

    * Subscribe to the BB support mailing for support, security updates
      and other news:

    	mailto: majordomo@bb4.com
    	in the text of the message: subscribe bb

    	There's also a developer's mailing list:
    	mailto: majordomo@bb4.com
    	in the text of the message: subscribe bbd

###### © Copyright Quest Software, Inc. 1997-2002 All rights reserved.