How to grep logfile via SSH on multiple servers?

Question

Sometime, I need to find out specific log content(such as api call) from log files, those log files exists in several Linux servers.

I have to use SSH to login every server and use grep searching the log. Right now, I plan to write a shell to finish this job automatically. But when I tried to put the "ssh" in a shell, I execute it, it always need password input and show the message on screen.

Is there anyway to make sure there is no alert message about the password input? Or is there is anyway to support using "grep" on multiple servers?

I can't use SSH key authentication as I can do this change on production servers

Because I also plan to provide the result on web page(like JSP,PHP. ) and execute that shell by Java at backend.

Solution 1

An alternative to public key authentication or kerberos authentication is to use master connections. This is typically done by adding this into the ~/.ssh/config file on the client:

Host *
ControlMaster auto
ControlPath ~/.ssh/master-%r@%h:%p

After that, if you are already connected to a server, then opening another session will not ask your password and will reuse the first connection. So if you keep open connections to all of your servers, then your script will never need any password input.

See the ControlMaster, ControlPath, ControlPersist options in ssh_config(5) or, alternatively, the -O, -S options of ssh(1).

Solution 2

I'm really late to answer this question but you could use Fabric. You could write a fabfile.py like:

from fabric.api import *

env.roledefs = {
    'dev':  ['dev_a', 'dev_b'],
    'pro':  ['pro_a', 'pro_b', 'pro_c', 'pro_d']
} 

env.user = 'foobar'
env.shell = 'rbash -l -c'
env.disable_known_hosts = True
logfile = '/path/to/logfile.txt'

def g(pattern):
    with settings(warn_only=True):
        run('grep -H --color "{0}" {1}'.format(pattern, logfile))

The script above defines a new command called g the accepts a pattern as input, you can run the script with:

fab -R dev -p my_pwd g:"some pattern"

to grep "some pattern" on hosts defines by role dev.

Solution 3

just use grep4j ( https://code.google.com/p/grep4j/ ) on your backend and display the results on your jsp/php

Solution 4

Since you can't use public key authentication you are probably better off using the method described in the first answer to avoid having to repeat the login process. Making a script to connect to multiple servers should be relatively straightforward, though:

#!/usr/bin/env bash

hosts=(ipaddr1 ipaddr2 ipaddr3) # or host names
user=username                   # your user on host
log=/tmp/mylog                  # result of grepping

for host in ${hosts[@]}; do
    echo "== $host ==" >> $log
    ssh -l $user $host "grep some_pattern /path/to/logfile 2>&1" >> $log
done
less $log                       # view result

I have not tested this, but it should work. Please tell me if this is not what you need.

View more solutions

Comments

[Flexo]

See "ssh key authentication"

[Honza Hála]

providing commands which you are using will improve chances to get useful answers

[Oliver Salzburg]

Please explain how to use the tool instead of just dropping a link.

[Balaji Boggaram Ramanarayan]

where does it take password here ?

[mtak]

Kudo's for a creative approach!