Please help on GREP command of LINUX

hfwang

Use 'grep -E "`date +"%Y-%m-%d"`.*Total sending status: Sent =" /app/emagine/gateway/log/smpp-tx-service.l*'
I can find the log of current day broken by hour, like:
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 00:14:56,998 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1148498 Success = 1081861 Throttling = 59805 Failur
e = 6832 Rate = 0.03/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 01:14:57,847 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1149014 Success = 1082367 Throttling = 59815 Failur
e = 6832 Rate = 0.14/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 02:14:58,694 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1149025 Success = 1082378 Throttling = 59815 Failur
e = 6832 Rate = 0.00/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 03:14:59,544 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1149039 Success = 1082392 Throttling = 59815 Failur
e = 6832 Rate = 0.00/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 04:15:00,394 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1149055 Success = 1082408 Throttling = 59815 Failur
e = 6832 Rate = 0.00/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 05:15:01,243 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1149083 Success = 1082436 Throttling = 59815 Failur
e = 6832 Rate = 0.01/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 06:15:02,093 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1149135 Success = 1082488 Throttling = 59815 Failur
e = 6832 Rate = 0.01/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 07:15:02,942 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1149265 Success = 1082618 Throttling = 59815 Failur
e = 6832 Rate = 0.04/s
/app/emagine/gateway/log/smpp-tx-service.log:2013-06-25 08:15:03,791 INFO   Thr:
                          Timer  ceiver.sms.transmitter.TimerWorkerThread:52
Total sending status: Sent = 1493179 Success = 1375684 Throttling = 105690 Failu
re = 11581 Rate = 81.39/s



But I want to display the hourly log of LAST 24 HOURS, rather than the current day as above.

How can I modify the GREP to get that? Or the whole shell script need to be modified?

Many many Thanks.

hfwang

求助，非常感激 。

xiaomao8754

Use GNU awk

awk 'BEGIN { t = systime() } { s = $1 " " $2; gsub(/[-:]/, " ", s) } t - mktime(s) < 86400' log_file

Note: variable s needs to be adjusted depending on your log file data format

mlg_tech

Can't you just remove the date and just grep the last 24 lines? I assume each line equal to 1 hour.

mlg_tech

E.g.
grep -E ".*Total sending status: Sent =" /app/emagine/gateway/log/smpp-tx-service.l* | tail -n 24

hfwang

xiaomao8754 发表于 2013-6-25 20:31
Use GNU awk

awk 'BEGIN { t = systime() } { s = $1 " " $2; gsub(/[-:]/, " ", s) } t - mktime(s) < 86 ...

谢谢,不过不知道S 是 放什么变量的 ？
LOG FILE 里的 DATA  FORMAT ？ 不理解。
那请问哪里是 GREP KEY  WORD 的 呢 ？ I.E Total sending status: Sent =...

非常感谢

hfwang

mlg_tech 发表于 2013-6-25 21:36
E.g.
grep -E ".*Total sending status: Sent =" /app/emagine/gateway/log/smpp-tx-service.l* | tail -n ...

不行 ，有很多 LOG FILE..谢谢

mlg_tech

It's really depends your log file size, quantity and how often you run the script. If you are running the script every hour against a lot of 1MB logs, then it should be able do it's job well, with that being said. You can definitely alter the script to increase the performance.

xiaomao8754

hfwang 发表于 2013-6-25 20:50
谢谢,不过不知道S 是 放什么变量的 ？
LOG FILE 里的 DATA  FORMAT ？ 不理解。
那请问哪里是 GREP KEY   ...

s是log里面存date那行
假如其中一行log data format如下
2013-02-05 10:05:30 bla bla
那么s就是 2013-02-05 10:05:30.还不懂去查awk文档或者google

grep KEYWORD I awk

combine你的grep command和awk就可以了，awk做的就是list last 24 hours log

lingg

hfwang 发表于 2013-6-25 20:50
谢谢,不过不知道S 是 放什么变量的 ？
LOG FILE 里的 DATA  FORMAT ？ 不理解。
那请问哪里是 GREP KEY   ...

1. awk 'BEGIN { t = systime() } { s = $1 " " $2; gsub(/[-:]/, " ", s) } t - mktime(s) < 86400'

复制代码

我对AWK不是很熟，大概帮你读一下，systime，gsub 和 mktime 都是gawk支持的function,
awk和grep一样，也是逐行处理的。

赋值变量

1. { t = systime() }

复制代码

由于你的log里，时间在 第一列和第二列，定义变量s 为第一列+第二列，中间用空格连接，

然后做字符替换(gsub)，确保字符串 格式 正确

1. { s = $1 " " $2; gsub(/[-:]/, " ", s) }

复制代码

这个看成一个IF，如果TRUE就显示此行

1. t - mktime(s) < 86400'

复制代码