Servers Hang

[desmondok]

I've got two servers hosting in the datacenter, but both server hang and i need a hard reboot in order to fix the problem, here's the log and the halt down happened at Jan 31
06:42:05 host2 named[2845]: exiting (due to assertion failure), and
fixed until a hard reboot requested to the datacenter at Feb 1
00:34:32 host2 shutdown: shutting down for system halt.


Do anyone knows what causing this problem?
Thx!
Desmond

Jan 31 06:38:09 host2 ipsec__plutorun: 003
"private-or-clear#0.0.0.0/0": ERROR: netlink XFRM_MSG_NEWPOLICY
response for flow %trap included errno 17: File exists
Jan 31 06:38:09 host2 ipsec__plutorun: 025
"private-or-clear#0.0.0.0/0": could not route
Jan 31 06:38:09 host2 ipsec__plutorun: ...could not route conn
"private-or-clear"
Jan 31 06:39:03 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:03 host2 last message repeated 2 times
Jan 31 06:39:05 host2 named[2845]: client 202.14.67.4#32768: error
sending response: unexpected error
Jan 31 06:39:05 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:07 host2 last message repeated 7 times
Jan 31 06:39:08 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:08 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:10 host2 last message repeated 2 times
Jan 31 06:39:11 host2 named[2845]: client 202.14.67.4#32768: error
sending response: unexpected error
Jan 31 06:39:12 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:12 host2 last message repeated 2 times
Jan 31 06:39:12 host2 named[2845]: client 202.14.67.4#32768: error
sending response: unexpected error
Jan 31 06:39:12 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:13 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:13 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:39:13 host2 named[2845]: internal_send: 202.64.76.10#953: No
such process
Jan 31 06:39:13 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:39:13 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:39:13 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:39:13 host2 named[2845]: internal_send: 202.64.76.10#953: No
such process
Jan 31 06:39:13 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:39:13 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:39:13 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:39:13 host2 named[2845]: internal_send: 202.64.76.10#953: No
such process
Jan 31 06:39:13 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:39:13 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:39:13 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:39:13 host2 named[2845]: internal_send: 202.64.76.10#953: No
such process
Jan 31 06:39:13 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:39:13 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:39:15 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:15 host2 last message repeated 2 times
Jan 31 06:39:16 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:16 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:16 host2 named[2845]: client 202.14.67.4#32768: error
sending response: unexpected error
Jan 31 06:39:17 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:17 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:20 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:21 host2 named[2845]: client 202.14.67.4#32768: error
sending response: unexpected error
Jan 31 06:39:22 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:26 host2 last message repeated 2 times
Jan 31 06:39:26 host2 named[2845]: client 202.14.67.4#32768: error
sending response: unexpected error
Jan 31 06:39:27 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:31 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:31 host2 last message repeated 2 times
Jan 31 06:39:33 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:39:33 host2 named[2845]: internal_send: 202.14.67.92#32774:
No such process
Jan 31 06:39:33 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:39:33 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:39:33 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:36 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:39:42 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:46 host2 last message repeated 3 times
Jan 31 06:39:48 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:39:48 host2 named[2845]: internal_send: 202.64.76.10#953: No
such process
Jan 31 06:39:48 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:39:48 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:39:48 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:39:48 host2 named[2845]: internal_send: 202.64.76.10#953: No
such process
Jan 31 06:39:48 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:39:48 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:39:56 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:39:56 host2 named[2845]: client 202.14.67.14#32768: error
sending response: unexpected error
Jan 31 06:40:01 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:40:03 host2 last message repeated 3 times
Jan 31 06:40:06 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:40:06 host2 named[2845]: internal_send: 202.14.67.92#32774:
No such process
Jan 31 06:40:06 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:40:06 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:40:06 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:40:37 host2 last message repeated 8 times
Jan 31 06:41:07 host2 last message repeated 7 times
Jan 31 06:41:35 host2 named[2845]: socket.c:1111: unexpected error:
Jan 31 06:41:35 host2 named[2845]: internal_send: 202.14.67.92#32774:
No such process
Jan 31 06:41:35 host2 named[2845]: errno2result.c:109: unexpected error:
Jan 31 06:41:35 host2 named[2845]: unable to convert errno to
isc_result: 3: No such process
Jan 31 06:41:35 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:41:37 host2 named[2845]: client 202.14.67.92#32774: error
sending response: unexpected error
Jan 31 06:42:05 host2 named[2845]: socket.c:1564:
INSIST(!sock->pending_send) failed
Jan 31 06:42:05 host2 named[2845]: exiting (due to assertion failure)
Feb 1 00:34:32 host2 shutdown: shutting down for system halt
Feb 1 00:34:32 host2 init: Switching to runlevel: 0
Feb 1 00:34:33 host2 messagebus: messagebus -TERM succeeded
Feb 1 00:34:34 host2 atd: atd shutdown succeeded
Feb 1 00:34:34 host2 webmin: Stopping Webmin server in /usr/libexec/webmin
Feb 1 00:34:34 host2 rc: Stopping webmin: succeeded
Feb 1 00:34:34 host2 xfs[3223]: terminating
Feb 1 00:34:34 host2 xfs: xfs shutdown succeeded
Feb 1 00:34:34 host2 mysqld: Stopping MySQL: failed
Feb 1 00:36:59 host2 syslogd 1.4.1: restart.
Feb 1 00:36:59 host2 syslog: syslogd startup succeeded

 

[samyu]

I have the same problem .