Home How to use shell to continuously process tail -f new lines in real time?
Reply: 2

How to use shell to continuously process tail -f new lines in real time?

Germinate
1#
Germinate Published in 2018-02-14 02:26:11Z

There is a file on the linux server, which new lines appended to it aperiodically. I need to process new line, parse it and use other command or script to process it . Check the file periodically is not acceptable, I need a real time solution. No other language (Python, Perl) is available on the server, only shell.

Now, I'm trying to assign the new line to a shell variable, and then process it. But can not find a good way to do that. Another problem is I need to rely on some result former when processing new lines. For example, when I process the 11th line, maybe the 5th line result is required. So some variables needed to store the result before, and I need to use them in the loop.

Any solution or any better suggestion for my case?

shuerguo
2#
shuerguo Reply to 2018-02-14 03:33:45Z

Try the code below

#!/bin/bash
process_new_line() {
    local var= '' #declare some local variables here 
    read
    while true
    do
        #process $REPLY 
        #The new line content is in the variable $REPLY 
        #store the result in the local variable according to your rules.
        read
    done
}

tail -f the_file | process_new_line

Use a function will solve your problem. You could use local variable to store the result, and $REPLY hold the new line content.

John1024
3#
John1024 Reply to 2018-02-14 02:34:00Z

Try:

tail -f filename | while read -r line; do
    : # process the line
done

This will be real-time except to the extent that lines are buffered. If the buffering bothers you, there are utilities like stdbuf to shrink the buffers.

You didn't mention whether your system has awk. It is required by the POSIX standard for Unix systems. If you have it, it is excellent for "calculating, replacing some characters", etc.:

tail -f filename | awk 'awk code'
You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.302644 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO