i have a webserver running with some applications. I do a grep from the server in order to see all the IP's that have been connected to the server. I achieved it but the HTML looks like this:
172.17.100.37 172.17.100.45 172.17.222.158
And I would like to see it like this:
172.17.100.37
172.17.100.45
172.17.222.158
To get these values I do this:
cmd = "grep -o '^[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' /var/log/apache2/access.log | sort | uniq "
Any idea about what could I do?
EDIT: The access.log shows like this:
172.17.222.158 - - [30/Jan/2014:09:33:11 +0000] "GET /cgi-bin/right.cgi HTTP/1.1" 204 219 "http://172.17.223.72/index2.html?fname=172.17.223.75" "Mozilla/5.$
172.17.222.158 - - [30/Jan/2014:09:33:11 +0000] "GET /cgi-bin/stop.cgi HTTP/1.1" 204 218 "http://172.17.223.72/index2.html?fname=172.17.223.75" "Mozilla/5.0$
172.17.222.158 - - [30/Jan/2014:09:33:12 +0000] "GET /cgi-bin/right.cgi HTTP/1.1" 204 218 "http://172.17.223.72/index2.html?fname=172.17.223.75" "Mozilla/5.$
172.17.222.158 - - [30/Jan/2014:09:33:12 +0000] "GET /cgi-bin/stop.cgi HTTP/1.1" 204
and so on...
As I understand you paste the ouput of your grep-command into a HTML-file.
To get the linefeeds you may use the pre-tag:
<pre> (your output goes here) </pre>
This will show your linefeeds even in HTML.
Using tr
echo "172.17.100.37 172.17.100.45 172.17.222.158" | tr ' ' '\n'
172.17.100.37
172.17.100.45
172.17.222.158
Using awk
cat file
172.17.222.158 - - [30/Jan/2014:09:33:11 +0000] "GET /cgi-bin/right.cgi HTTP/1.1" 204 219 "http://172.17.223.72/index2.html?fname=172.17.223.75" "Mozilla/5.$
172.17.222.158 - - [30/Jan/2014:09:33:11 +0000] "GET /cgi-bin/stop.cgi HTTP/1.1" 204 218 "http://172.17.223.72/index2.html?fname=172.17.223.75" "Mozilla/5.0$
172.17.222.158 - - [30/Jan/2014:09:33:12 +0000] "GET /cgi-bin/right.cgi HTTP/1.1" 204 218 "http://172.17.223.72/index2.html?fname=172.17.223.75" "Mozilla/5.$
172.17.222.158 - - [30/Jan/2014:09:33:12 +0000] "GET /cgi-bin/stop.cgi HTTP/1.1" 204
awk -F/ '{print $8}' file
172.17.223.72
172.17.223.72
172.17.223.72
Related
info Launching Dev Tools...
error Google Chrome exited with error:, Error: spawn cmd ENOENT
::ffff:127.0.0.1 - - [09/Apr/2019:11:35:16 +0000] "GET /launch-js-devtools HTTP/1.1" 200 - "-" "okhttp/3.12.1"
::ffff:127.0.0.1 - - [09/Apr/2019:11:35:17 +0000] "GET /onchange HTTP/1.1" - - "-" "okhttp/3.12.1"
::ffff:127.0.0.1 - - [09/Apr/2019:11:35:32 +0000] "GET /onchange HTTP/1.1" - - "-" "okhttp/3.12.1"
::ffff:127.0.0.1 - - [09/Apr/2019:11:35:47 +0000] "GET /onchange HTTP/1.1" - - "-" "okhttp/3.12.1"
::ffff:127.0.0.1 - - [09/Apr/2019:11:36:02 +0000] "GET /onchange HTTP/1.1" - - "-" "okhttp/3.12.1"
when I am running JS remotely, the error is showed in CMD. and the emulator is hanging.
I am using OpenStack Ocata release installed on my own servers. Long time all worked well.
A few days ago OpenStack dashboard starts frequently sign out users. And I can't figure out what is wrong.
Why httpd return 302 redirect to the login page? And how to debug what is wrong?
Httpd access logs:
10.0.0.2 - - [21/Mar/2018:08:29:26 +0000] "POST /dashboard/auth/login/ HTTP/1.1" 302 - "http://dashboard.example.com/dashboard/auth/login/?next=/dashboard/" "Mozilla/5.0 ... Firefox/59.0"
10.0.0.2 - - [21/Mar/2018:08:29:27 +0000] "GET /dashboard/ HTTP/1.1" 302 - "http://dashboard.example.com/dashboard/auth/login/?next=/dashboard/" "Mozilla/5.0 ... Firefox/59.0"
10.0.0.2 - - [21/Mar/2018:08:29:27 +0000] "GET /dashboard/identity/ HTTP/1.1" 200 53953 "http://dashboard.example.com/dashboard/auth/login/?next=/dashboard/" "Mozilla/5.0 ... Firefox/59.0"
193.169.81.251 - - [21/Mar/2018:08:29:29 +0000] "GET /dashboard/i18n/js/horizon+openstack_dashboard/ HTTP/1.1" 200 2372 "http://dashboard.example.com/dashboard/identity/" "Mozilla/5.0 ... Firefox/59.0"
10.0.0.2 - - [21/Mar/2018:08:29:33 +0000] "GET /dashboard/project/ HTTP/1.1" 302 - "http://dashboard.example.com/dashboard/identity/" "Mozilla/5.0 ... Firefox/59.0"
10.0.0.2 - - [21/Mar/2018:08:29:33 +0000] "GET /dashboard/auth/login/?next=/dashboard/project/ HTTP/1.1" 200 9041 "http://dashboard.example.com/dashboard/identity/" "Mozilla/5.0 ... Firefox/59.0"
10.0.0.2 - - [21/Mar/2018:08:29:34 +0000] "GET /dashboard/i18n/js/horizon+openstack_dashboard/ HTTP/1.1" 200 2372 "http://dashboard.example.com/dashboard/auth/login/?next=/dashboard/project/" "Mozilla/5.0 ... Firefox/59.0"
Httpd error logs:
[Wed Mar 21 08:29:26.646941 2018] [:error] [pid 41571] Attempted scope to domain default failed, will attemptto scope to another domain.
[Wed Mar 21 08:29:26.851412 2018] [:error] [pid 41571] Login successful for user "exampeuser", remote address 10.0.0.2.
[Wed Mar 21 08:29:27.161127 2018] [authz_core:error] [pid 25877] [client 10.0.0.2:44688] AH01630: client denied by server configuration: /usr/bin/keystone-wsgi-public, referer: http://dashboard.example.com/dashboard/auth/login/?next=/dashboard/
The problem was related to Memcached.
In my case, Memcached was DoSed from externally.
To resolve this I bind Memcached to local management interface instead any (0.0.0.0)
On Front End below screen comes up when we enter into kuali Student login UI with admin/admin account.
HTTP Status 404 - /ks-with-rice-bundled/kew/ActionList.do
type Status report
message /ks-with-rice-bundled/kew/ActionList.do
description The requested resource is not available.
This is the error I get in the log file:
192.168.0.134 - - [21/Jan/2016:15:33:00 +0530] "POST /ks-with-rice-bundled-2.0.3-cm/org.kuali.student.lum.lu.ui.main.LUMMain/rpcservices/ServerPropertiesRpcService HTTP/1.1" 200 46
192.168.0.134 - - [21/Jan/2016:15:33:00 +0530] "POST /ks-with-rice-bundled-2.0.3-cm/org.kuali.student.lum.lu.ui.main.LUMMain/rpcservices/SecurityRpcService HTTP/1.1" 200 21
192.168.0.134 - - [21/Jan/2016:15:33:00 +0530] "POST /ks-with-rice-bundled-2.0.3-cm/org.kuali.student.lum.lu.ui.main.LUMMain/rpcservices/MetadataRpcService HTTP/1.1" 200 4745
192.168.0.134 - - [21/Jan/2016:15:33:00 +0530] "POST /ks-with-rice-bundled-2.0.3-cm/org.kuali.student.lum.lu.ui.main.LUMMain/rpcservices/ServerPropertiesRpcService HTTP/1.1" 200 355
192.168.0.134 - - [21/Jan/2016:15:33:00 +0530] "POST /ks-with-rice-bundled-2.0.3-cm/org.kuali.student.lum.lu.ui.main.LUMMain/rpcservices/ServerPropertiesRpcService HTTP/1.1" 200 179
192.168.0.134 - - [21/Jan/2016:15:33:00 +0530] "POST /ks-with-rice-bundled-2.0.3-cm/org.kuali.student.lum.lu.ui.main.LUMMain/rpcservices/SecurityRpcService HTTP/1.1" 200 204
192.168.0.134 - - [21/Jan/2016:15:33:00 +0530] "GET /ks-with-rice-bundled/kew/ActionList.do HTTP/1.1" 404 1027
192.168.0.134 - - [21/Jan/2016:15:33:01 +0530] "POST /ks-with-rice-bundled-2.0.3-cm/org.kuali.student.lum.lu.ui.main.LUMMain/rpcservices/SecurityRpcService HTTP/1.1" 200 287
Cataline.out logs gives below one:
2016-01-21 15:33:00,760 [http-bio-8080-exec-2] u:/d: INFO org.kuali.student.common.ui.server.gwt.ServerPropertiesRpcGwtServlet - Property not found, looking in Context: ks.rice.actionList.serviceAddress with value: http://kuali.localdomain:8080/ks-with-rice-bundled/kew/ActionList.do
2016-01-21 15:33:00,804 [http-bio-8080-exec-5] u:/d: INFO org.kuali.student.common.ws.ServletWrappingController - handleRequestInternal : SecurityGwtServlet
2016-01-21 15:33:00,818 [http-bio-8080-exec-7] u:/d: INFO org.kuali.student.common.ws.ServletWrappingController - handleRequestInternal : SecurityGwtServlet
I can see in the error, what is it, however I don't know how to solve it.
All of your successful POSTs are to /ks-with-rice-bundled-2.0.3-cm/... but the 404 is on /ks-with-rice-bundled. Rice configuration is leading to a redirect to the incorrect location.
I would either deploy the application to /ks-with-rice-bundled or add
<param name="app.code">ks-with-rice-bundled-2.0.3-cm</param>
to your local ks-config.xml.
How can one split existing apache logfiles into separate files by month?
I've scoured the web and I can't find anything. Yes, I know about logrotate and cronolog and all that. But nothing I've found helps me with splitting existing files.
Is there an awk script or something?
Here's a snippet of the data:
124.115.5.11 - - [30/May/2011:23:21:37 -0500] "GET / HTTP/1.0" 200 206492 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322;TencentTraveler)"
58.61.164.39 - - [31/May/2011:00:36:35 -0500] "GET / HTTP/1.0" 200 206492 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322;TencentTraveler)"
114.80.93.55 - - [31/May/2011:01:42:17 -0500] "GET / HTTP/1.0" 200 206492 "-" "Sosospider+(+http://help.soso.com/webspider.htm)"
114.80.93.73 - - [31/May/2011:02:03:44 -0500] "GET / HTTP/1.0" 200 206492 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322;TencentTraveler)"
123.125.71.98 - - [31/May/2011:12:33:30 -0500] "GET / HTTP/1.1" 103 24576 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
220.181.108.187 - - [31/May/2011:12:33:55 -0500] "GET / HTTP/1.1" 103 24576 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
123.125.71.117 - - [31/May/2011:13:27:56 -0500] "GET / HTTP/1.1" 103 24576 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
123.125.71.78 - - [31/May/2011:16:45:48 -0500] "GET /node/54 HTTP/1.1" 200 3219 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
124.115.1.8 - - [31/May/2011:19:59:58 -0500] "GET / HTTP/1.1" 200 206492 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
123.125.71.69 - - [31/May/2011:22:05:46 -0500] "GET / HTTP/1.1" 200 206492 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
Here's my solution, greatly inspired by Steve's answer below:
One way using awk:
awk 'BEGIN {
split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec ", months, " ")
for (a = 1; a <= 12; a++)
m[months[a]] = a
}
{
split($4,array,"[:/]");
year = array[3]
month = sprintf("%02d", m[array[2]])
print > FILENAME"-"year"_"month".txt"
}' incendiary.ws-2009
This will output files like:
incendiary.ws-2010-2010_04.txt
incendiary.ws-2010-2010_05.txt
incendiary.ws-2010-2010_06.txt
incendiary.ws-2010-2010_07.txt
Against a 150 MB log file, the Accepted Answer by chepner took 70 seconds on an 3.4 GHz 8 Core Xeon E31270, while this method took 5 seconds.
Original inspiration: https://stackoverflow.com/a/11714105/430062
One way using awk:
awk '{ split($4,array,"/"); print > array[2] ".txt" }' file.txt
This will output files like:
May.txt
June.txt
July.txt
etc
EDIT:
Perhaps you would like to keep the years separate:
awk '{ split($4,array,"[:/]"); print > array[2] array[3] ".txt" }' file.txt
This will output files like:
May2011.txt
May2012.txt
July2011.txt
etc
Great answer #steve, I just reversed the terms in your example to get the year to be the first part of the resulting filename, so the file ordering was at least semi-OK.
awk '{ split($4,array,"[:/]"); print > array[3] "-" array[2] ".log" }' file.txt
This is what you get (not ideal)
2021-Apr.log
2021-Aug.log
2021-Dec.log
2021-Feb.log
2021-Jan.log
2021-Jul.log
2021-Jun.log
2021-Mar.log
2021-May.log
2021-Nov.log
2021-Oct.log
2021-Sep.log
Ideally, I would order by %Y-%m, not %Y-%b. Maybe an "awk wizard" can pick up the challenge and do that for us by processing the resulting file names with a second script?
TIP: Unix convert Month name to number
I have a repository in here http://repos.joomlaguruteam.com/
I can browse it but I can clone it.
Every time I clone it I have this error
hg clone http://repos.joomlaguruteam.com/hello
destination directory: hello
requesting all changes
abort: HTTP Error 404: Not Found
and the access log have that
115.5.95.59 - - [10/Feb/2011:04:20:33 -0600] "GET /hello?pairs=0000000000000000000000000000000000000000-0000000000000000000000000000000000000000&cmd=between HTTP/1.1" 200 1 "-" "mercurial/proto-1.0"
115.5.95.59 - - [10/Feb/2011:04:20:34 -0600] "GET /hello?cmd=heads HTTP/1.1" 200 41 "-" "mercurial/proto-1.0"
115.5.95.59 - - [10/Feb/2011:04:20:34 -0600] "GET /hello?cmd=changegroup&roots=0000000000000000000000000000000000000000 HTTP/1.1" 404 597 "-" "mercurial/proto-1.0"
What is the problem.
I really hope somebody can help me with that.
Thanks,
Yuan
I could clone this by using uncompressed transfer.
If you are using TortoiseHg, then check the box Use uncompressed transfer
If you are using command-line, then use --uncompressed flag
hg clone --uncompressed http://repos.joomlaguruteam.com/hello