Finding text strings in binary files

Ever wondered what's inside some of those binary files on your system (binary executables or binary data)? Several times I've gotten error messages from some command in the Solaris system, but I couldn't tell where the error was coming from because it was buried in some binary executable file.

The Solaris "strings" command lets you look at the ASCII text buried inside of executable files, and can often help you troubleshoot problems. For
instance, one time I was seeing error messages like this when a user was trying to log in:

    Could not set ULIMIT

I finally traced the problem down to the /bin/login command by running the "strings" command like this:

    root> strings /bin/login | more

The strings command lists ASCII character sequences in binary files, and help me determine that the "Could not set ULIMIT" error was coming from
this file. Once I determined that the error message I was seeing was coming from this file, solving the problem became a simple matter.

A vi macro to display line numbers

While the vi editor has been known to be a little rough around the edges, it still has some pretty nice features. One of those features is the ability to define cool macros. Here we'll show you how to create two macros--one to display line numbers of the file you're editing, and one to hide the line numbers.

First, create a file named .exrc in your home directory (or edit the current file if it already exists). This is the configuration file that vi reads when it is started. Put the following two lines into this file:

    :map #1 :set number^M
    :map #2 :set nonumber^M

(A very important note: create the ^M characters in this file by typing the key sequence [CTRL-V][CTRL-M]. This key sequence embeds an actual ^M
character (the carriage return) into the file.)

Now, save this file and re-start vi. From vi's command-mode, you'll now be able to display line numbers beside your file contents by hitting the [F1]
function key, and clear line numbers by hitting the [F2] key. If you like these macros, create your own powerful macros by following this same
technique!

Use traceroute to find network problems

With the Internet and TCP/IP networks all around us, it's important to know a little bit about TCP/IP network troubleshooting. The Solaris "traceroute"
command lets you trace the route your packets are taking to get from your current workstation to a remote workstation you're trying to reach.

For instance, suppose you're on the Internet and you're not getting an HTTP response from a remote server named www.zdtips.com. You try to "ping" the
remote site like this:

    ping www.zdtips.com

but get no answer. Is the remote server down, or is there a broken link between you and the remote site? Issue the following command to see where
the problem lies:

    traceroute www.zdtips.com

The traceroute command works its way through the network, and tells you the path it's taking to get to the destination site as it goes along. Watch for
the point traceroute fails to learn more about the network segment that has failed. Of course you can also try this on working connections to learn more about how your Internet packets get from one site to another.

Using telnet's command mode

When you use the Solaris telnet command to log in from one site to another, don't forget that you can enter telnet's command mode at any time during
your session, usually by entering the following key sequence:

    [CTRL] ] (the control key and right-bracket key at the same time)

When this is successful, you'll see the following prompt:

    telnet>

Then, from the "telnet>" prompt, enter "?" to learn more about the available telnet commands. A few commands that will give you more help at the prompt are shown below:

    display        displays many current Telnet settings
    send ?        displays commands you can "send" to the remote site
    set ?             displays variables that can be set

One of my favorites (mostly when fooling around) is:

    send ayt

which means "send this message: are you there?".

Keep commands running after you leave with nohup

It's 3 p.m., and you want to start a long job running. Unfortunately, you can't be sure that the job will finish by 5 p.m. when you need to leave, and the company is very strict about making sure you log off when you leave. However, if you log off the system, the job will be stopped. What can you do?

On Solaris systems you can use the "nohup" (no hang-up) command to keep jobs running long after you log off the system. Using nohup tells the system not to "hang-up" on your job after you've logged off the system.

Here's how to run the job, and keep it running after you log off:

root> nohup my-long-job &

This creates a file named "nohup.out" in the current directory that contains the standard output of the command ("my-long-job") you're running.
Everyone is happy because the job keeps running, you get to leave at 5 p.m., and you're properly logged off the system.

Use rsh to run commands on somebody else's computer

Here's a hypothetical situation for you: On the "Sesame Street Network", you're logged into a Solaris workstation named "elmo", and you need to look
at the man pages for the tar command. Unfortunately the man pages aren't installed on this workstation.

If you're on a network of Solaris computers like this, and you know that the man pages are installed on another workstation named "bigbird", the solution is simple. Simply run the man command on bigbird, like this:

elmo> rsh bigbird "man ksh" | more

This runs the "man ksh" command on bigbird, and pipes the output of the command into the "more" command on elmo.

Here's a few other "rsh" commands you can run on remote workstations:

rsh bigbird "who"
rsh ernie "ps -ef"
rsh grover "ls -al /home"

Use CDPATH to traverse filesystems faster

If you're like many Solaris users and administrators, you spend a lot of time moving back and forth between directories in similar locations. For
instance, you might often work in your home directory (such as "/home/al"), the /usr/local directories, web page directories, or other user's home
directories in /home.

If you're often moving back-and-forth between the same directories, and you use the Bourne shell (sh) or Korn shell (ksh) as your login shell, you can
use the CDPATH shell variable to save yourself a lot of typing, and quickly move between directories.

Here's a quick demo. First move to the root directory:

    cd /

Next, if it's not set already, set your CDPATH shell variable as follows:

CDPATH=/usr/spool

Then, type this cd command:

    cd cron

What happens? Type this and see what happened:

    pwd

The result should be "/usr/spool/cron".

When you typed "cd cron", the shell looked in your local directory for a sub-directory named "cron". When it didn't find one, it searched the CDPATH
variable, and looked for a "cron" sub-directory. When it found a sub-directory named cron in the /usr/spool directory, it moved you there.

You can set your CDPATH variable just like your normal PATH variable:

    CDPATH=/home/al:/usr/local:/usr/spool:/home

Group commands together with parentheses

Have you ever needed to run a series of commands, and pipe the output of all of those commands into yet another command?

For instance, what if you wanted to run the "sar", "date", "who", and "ps -ef" commands, and wanted to pipe the output of all three of those commands
into the "more" command? If you tried this:

    sar -u 1 5; date; who; ps -ef | more

you'll quickly find that it won't work. Only the output of the "ps -ef" command gets piped through the "more" command, and the rest of the output
scrolls off the screen.

Instead, group the commands together with a pair of parentheses (and throw in a few echo statements for readability) to get the output of all these
commands to pipe into the more command:

(sar -u 1 5; echo; who; echo; ps -ef; echo; date; echo) | more

Use CDPATH to traverse file systems faster

If you're like many Solaris users and administrators, you spend a lot of time moving back and forth between directories in similar locations. For
instance, you might often work in your home directory (such as "/home/al"), the /usr/local directories, web page directories, or other user's home
directories in /home.

If you're often moving back-and-forth between the same directories, and you use the Bourne shell (sh) or Korn shell (ksh) as your login shell, you can
use the CDPATH shell variable to save yourself a lot of typing, and quickly move between directories.

Here's a quick demo. First move to the root directory:

    cd /

Next, if it's not set already, set your CDPATH shell variable as follows:

CDPATH=/usr/spool

Then, type this cd command:

    cd cron

What happens? Type this and see what happened:

    pwd

The result should be "/usr/spool/cron".

When you typed "cd cron", the shell looked in your local directory for a sub-directory named "cron". When it didn't find one, it searched the CDPATH
variable, and looked for a "cron" sub-directory. When it found a sub-directory named cron in the /usr/spool directory, it moved you there.

You can set your CDPATH variable just like your normal PATH variable:

    CDPATH=/home/al:/usr/local:/usr/spool:/home

A shell function to get one character at a time from the end user

Many times people writing shell programs want to prompt an end-user to "Hit Any Key to Continue" during the middle of a shell program. However, they
often settle for lesser functionality, because they think it's not easy to get one keystroke at a time from the end user.

The "GetKeystroke" function shown below shows exactly how to get one character at a time from your end-users in your Bourne shell/Korn shell
programs:

-----------------------------------------------------------------
#!/bin/sh

GetKeystroke () {
trap "" 2 3
oldSttySettings=`stty -g`
stty -echo raw
echo "`dd count=1 2> /dev/null`"
stty $oldSttySettings
trap 2 3
}


#----------------------------------------------#
# Here's the main program. Hit "x" to exit. #
#----------------------------------------------#

keyStroke=""
while [ "$keyStroke" != "x" ]
do
clear
echo "HIT ANY KEY TO CONTINUE: \c"
keyStroke=`GetKeystroke`
done

-----------------------------------------------------------------

To test this program, try the following:

1. Save the code between the lines in a file named /tmp/keytest
2. Make the program executable: chmod +x /tmp/keytest
3. Run the program: /tmp/keytest

You'll see that you can hit one character at a time, without the need to hit the [Enter] key. When you're finished testing, type "x" to leave the
program.

Use the "at" command to run jobs some other time

Many times it's necessary to schedule programs to run at a later time. For instance, if your computer system is very busy during the day, you may need
to run jobs late at night when nobody is logged on the system.

Solaris makes this very easy with the "at" command. You can use the "at" command to run a job at almost any time--later today, early tomorrow...whenever.

Suppose you want to run the program "my_2_hour_program" at ten o'clock tonight. Simply tell the at command to run the job at 10 p.m. (2200):

    /home/al> at 2200
    at> my_2_hour_program > /tmp/2hour.out
    at> <CTRL><D>
warning: commands will be executed using /bin/ksh
job 890193600.a at Tue Mar 17 22:00:00 1998

Or suppose you'd like to run a find command at five o'clock tomorrow morning:

/home/al> at 0500 tomorrow
at> find /home > /tmp/find.out
at> <CTRL><D>
warning: commands will be executed using /bin/ksh
job 890215200.a at Wed Mar 18 05:00:00 1998

When you're at the "at" prompt, just type the command you want to run. Try a few tests with the at command until you become comfortable with the way
it works.

Head, tail, and sed

Want to see the first 5 lines of the /etc/passwd file? Pretty easy, just use the "head" command:

    head -5 /etc/passwd

Want to see the last 20 lines of the /etc/passwd file? Again, pretty easy, just use the "tail" command:

    tail -20 /etc/passwd

But what if you only want to see lines 10-15 of a given file? Neither the "head" nor the "tail" commands alone will do. Instead, use the "sed"
command to print the range of lines you want to see:

    sed -n '10,15p' /etc/passwd

This command works great, even if you want to see a few lines somewhere in the middle of a 100,000 line file.

Rearranging columns with awk

Have you ever had a column-oriented text file, similar to a spreadsheet, but the columns weren't in the order you wanted? For instance, suppose you
had the following information in a file named "checkbook.orig":

    COST         DATE            BALANCE
    10.00         040198           1000.00
    20.00         040298             980.00
    30.00         040298             950.00

The information is good, but you'd prefer to have the DATE column first, followed by the COST information in the second column, and the BALANCE
column third.

Using awk, you can easily rearrange the columns. The following command reads the data from the file named "checkbook.orig", and writes the data to
a file named "checkbook.new":

    awk '{print $2, $1, $3}' checkbook.orig > checkbook.new

This brief awk command reads each line of the original file, and for each line it reads, it writes an output line to the "new" file. As it writes
each record to the new file, it rearranges the order of the columns, so that the columns now appear in the desired order!

If you prefer a little more control of the printed output, awk also has a "printf" function that's very similar to printf in the "C" programming
language. Here's the same example, with a tab character in-between each column of the output:

    awk '{printf ("%s\t%s\t%s\n", $2, $1, $3) }' checkbook.orig > checkbook.new

The awk command is a powerful programming utility that takes care of things like opening files and reading each line automatically, so all you have to
do is tell awk how to process each line as it goes by.

Add spice to interactive shell programs with tput

When you're writing interactive shell programs, you often want to add a little more spice to your user interface - those special nuances that make
your application more appealing. The Solaris "tput" command can be used to enhance your user interface.

Here's a couple of quick tput commands that can spice up your user interface:

The tput command can be used to make text appear bold on terminals that support a bold appearance. Making text appear bold works very well for
titles, or at times when you want a word or phrase to stand out from the rest of the text. Try these three commands at your command line to make
the "[Enter]" portion of the following echo statement output appear bold:

bold=`tput smso`
norm=`tput rmso`
echo "Hit the ${bold}[Enter]${norm} key to continue: \c"

You can also position the cursor on-screen with the tput command. Type this command at the command line to see what happens:

    tput cup 10 40

I once wrote a crude interactive screen editor using tput cup to properly position the cursor when the user hit the various arrow keys.

Create a directory and move into it at the same time

Question: How often do you create a new directory and then move into that directory in your next command?  Answer: Almost always.

I realized this trend in my own work habits, so I created a simple shell function to do the hard work for me.

     md () {
          mkdir -p $1  &&  cd $1
     }

This is a Bourne shell function named "md" that works for Bourne and Korn shell users. It can be easily adapted for C shell users.

Taking advantage of the -p option of the mkdir command, the function easily creates multi-level subdirectories, and moves you into the lowest level of the directory structure.  You can use the command to create one subdirectory like this:

     /home/al> md docs
     /home/al/docs> _

or you can create an entire directory tree and move right into the new directory like this:

     /home/al> md docs/memos/internal/solaris8
     /home/al/docs/memos/internal/solaris8>

Easily convert man pages to text documents

Have you ever wanted to convert a man page into a plain text document?
I do this occasionally when I want to share information via an email or other document format.

I used to think this was difficult, but then I discovered a simple way to do it. Here's the wrong way to write the man page for the ls command into a text file named ls.bad:

     man ls > ls.bad

This keeps all of the formatting characters in your document, which is generally not what you want.  Here's a better way that eliminates those formatting characters:

     man ls | col -b > man.txt

The col command with the -b option removes the undesirable backspace characters from the text stream, so the only thing left in your document is the text you want, in the format you want.

How to page more than one command at a time

Have you ever wanted to group a bunch of commands into a paging program like "page" or "more", but didn't know how?

As a system administrator, I always worry about certain things, like who's doing what, what processes are running, what the network traffic looks like, etc.  One day I decided to create a simple alias that would combine all the commands I wanted into one big chunk of information.  Then I realized that it wouldn't all fit into one screen. 
Fortunately I knew how to group all of the commands together, so the "more" command could handle them as one set of input.

First, here's the wrong way to try to page a sequence of four commands:

     date; netstat -i; whodo; ps -ef | more

The only command that gets paged properly here is the "ps -ef" command
--the rest of them scroll off the screen before you can read them.

Here's the correct way to page four commands so they're all controlled
by "more":

     (date; netstat -i; whodo; ps -ef) | more

Once you find the commands you want to group together, you can combine them into an alias or shell program.  I recommend a shell program for this, because a few "echo" statements sure make it easier to see where one command ends and the next command begins!

Initializing log files

Like all Unix file systems, open log files can cause a real problem when they get too large and need to be deleted. The problem is, if you delete an open file, the link is removed, but all of the inodes are lost. Even worse, if the program continues to log to the file, the link never re-appears, and additional inodes are lost and are unrecoverable. I suggest two solutions to the above problems.

If you have lost inodes, a simple reboot (make sure FSCK is run on startup) will recover lost inodes and missing filespace.

To empty (or zero out) an open log file, simply issue the following command:

date > logfile

This will 'empty' the file and insert as the first line the output from the date command. If you want a completely empty file, don't enter date,

just > logfile.

This works great on apache and other web server logs, without ever stopping the service.

Don't forget the options that make ls work better for you

Generally speaking, most users type ls or ls -al to see their directory listings. But don't forget that there are a few other cool options that make it easier to read your directory listings.

The -aCF options are my next-favorite listing combination.  The following command:

     ls -aCF

lists (a) all files in columns with special characters appended to the end of each name to show whether the file is a normal file, directory, executable file, or link.

The -m option lets you list files in a comma-separated list, which can be useful if you're going to be exporting the list to a Perl program or shell script.

To sort the listing by file size, try

     ls -al | sort -4n

To reverse the filesize listing, use

     ls -al | sort -4nr

instead.

Changing a file's owner and group at the same time

Did you know that in Solaris 2.x you can change both the user and group ownership on a file at the same time by running /usr/ucb/chown?

For example, suppose you have a file called /export/home/grahamc/myfile that is owned by user 'grahamc' and group 'users'. To change the ownerships, you could type the following as root:

    /usr/ucb/chown grahamc:users /export/home/grahamc/myfile

rather than typing:

    /usr/bin/chown grahamc /export/home/grahamc/myfile
    /usr/bin/chgrp users   /export/home/grahamc/myfile

You can also recursively change ownerships with the -R option.

For more information, try:   man -s 1b chown

Using the which command

I think a good command in addition to the 'type' is the 'which' command. You can really see where the executable come from. This is very helpful to find out if there is an alias set to the command and if so - which one.

Example:   # type ls
           ls is a tracked alias for /usr/bin/ls
           # which ls
           ls:     aliased to ls -aF

Legal Disclaimer
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS". WE EXPRESSLY DISCLAIMS ALL REPRESENTATIONS AND WARRANTIES OF ANY KIND REGARDING THE CONTENTS OR USE OF THE INFORMATION INCLUDING, BUT NOT LIMITED TO, EXPRESS AND IMPLIED WARRANTIES OF ACCURACY, COMPLETENESS, MERCHANTABILITY, FITNESS FOR A PARTICULAR USE, OR NON-INFRINGEMENT. IN NO EVENT SHALL WE BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, LOST BUSINESS OR LOST DATA, RESULTING FROM THE USE OR RELIANCE UPON THE INFORMATION, EVEN IF WE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION MAY NOT APPLY TO YOU.