Sep 19 07:02:31 test02 kernel: [1448508.176136] __ratelimit: 28 callbacks suppressed
Sep 19 07:02:31 test02 kernel: [1448508.176139] mysqld invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 19 07:02:31 test02 kernel: [1448508.176142] mysqld cpuset=/ mems_allowed=0
Sep 19 07:02:31 test02 kernel: [1448508.176144] Pid: 6438, comm: mysqld Not tainted 2.6.32-bpo.5-amd64 #1
Sep 19 07:02:31 test02 kernel: [1448508.176146] Call Trace:
Sep 19 07:02:31 test02 kernel: [1448508.176154]  [<ffffffff810b60f0>] ? oom_kill_process+0x7f/0x23f
Sep 19 07:02:31 test02 kernel: [1448508.176156]  [<ffffffff810b6614>] ? __out_of_memory+0x12a/0x141
Sep 19 07:02:31 test02 kernel: [1448508.176158]  [<ffffffff810b676b>] ? out_of_memory+0x140/0x172
Sep 19 07:02:31 test02 kernel: [1448508.176160]  [<ffffffff810ba4b4>] ? __alloc_pages_nodemask+0x4e5/0x5f5
Sep 19 07:02:31 test02 kernel: [1448508.176163]  [<ffffffff810bba19>] ? __do_page_cache_readahead+0x9b/0x1b4
Sep 19 07:02:31 test02 kernel: [1448508.176165]  [<ffffffff810bbb4e>] ? ra_submit+0x1c/0x20
Sep 19 07:02:31 test02 kernel: [1448508.176168]  [<ffffffff810b483e>] ? filemap_fault+0x17d/0x2f6
Sep 19 07:02:31 test02 kernel: [1448508.176172]  [<ffffffff810ca786>] ? __do_fault+0x54/0x3c3
Sep 19 07:02:31 test02 kernel: [1448508.176174]  [<ffffffff810ccaa5>] ? handle_mm_fault+0x3b8/0x80f
Sep 19 07:02:31 test02 kernel: [1448508.176179]  [<ffffffff8104a442>] ? default_wake_function+0x0/0x9
Sep 19 07:02:31 test02 kernel: [1448508.176184]  [<ffffffff812fead6>] ? do_page_fault+0x2e0/0x2fc
Sep 19 07:02:31 test02 kernel: [1448508.176186]  [<ffffffff812fc975>] ? page_fault+0x25/0x30
Sep 19 07:02:31 test02 kernel: [1448508.176187] Mem-Info:
Sep 19 07:02:31 test02 kernel: [1448508.176188] Node 0 DMA per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.176190] CPU    0: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176191] CPU    1: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176192] CPU    2: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176193] CPU    3: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176194] CPU    4: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176195] CPU    5: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176202] CPU    6: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176203] CPU    7: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176204] Node 0 DMA32 per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.176205] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176207] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176208] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176209] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176210] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176211] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176212] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176213] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176214] Node 0 Normal per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.176215] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176216] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176217] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176218] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176219] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176220] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176221] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176222] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.176225] active_anon:2170956 inactive_anon:350467 isolated_anon:32
Sep 19 07:02:31 test02 kernel: [1448508.176226]  active_file:122 inactive_file:28 isolated_file:0
Sep 19 07:02:31 test02 kernel: [1448508.176227]  unevictable:8 dirty:0 writeback:40 unstable:0
Sep 19 07:02:31 test02 kernel: [1448508.176227]  free:13675 slab_reclaimable:4379 slab_unreclaimable:3573
Sep 19 07:02:31 test02 kernel: [1448508.176228]  mapped:171 shmem:4 pagetables:10576 bounce:0
Sep 19 07:02:31 test02 kernel: [1448508.176229] Node 0 DMA free:15808kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15220kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.176235] lowmem_reserve[]: 0 3512 10077 10077
Sep 19 07:02:31 test02 kernel: [1448508.176237] Node 0 DMA32 free:30716kB min:4472kB low:5588kB high:6708kB active_anon:2822852kB inactive_anon:564344kB active_file:420kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596504kB mlocked:0kB dirty:0kB writeback:12kB mapped:4kB shmem:0kB slab_reclaimable:3736kB slab_unreclaimable:1144kB kernel_stack:168kB pagetables:10604kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.176243] lowmem_reserve[]: 0 0 6565 6565
Sep 19 07:02:31 test02 kernel: [1448508.176245] Node 0 Normal free:8176kB min:8360kB low:10448kB high:12540kB active_anon:5860972kB inactive_anon:837524kB active_file:68kB inactive_file:176kB unevictable:32kB isolated(anon):128kB isolated(file):0kB present:6722560kB mlocked:32kB dirty:0kB writeback:148kB mapped:680kB shmem:16kB slab_reclaimable:13780kB slab_unreclaimable:13132kB kernel_stack:2104kB pagetables:31700kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:30 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.176251] lowmem_reserve[]: 0 0 0 0
Sep 19 07:02:31 test02 kernel: [1448508.176253] Node 0 DMA: 4*4kB 2*8kB 4*16kB 3*32kB 2*64kB 3*128kB 1*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15808kB
Sep 19 07:02:31 test02 kernel: [1448508.176266] Node 0 DMA32: 42*4kB 84*8kB 105*16kB 93*32kB 77*64kB 49*128kB 25*256kB 10*512kB 1*1024kB 1*2048kB 0*4096kB = 31288kB
Sep 19 07:02:31 test02 kernel: [1448508.176271] Node 0 Normal: 914*4kB 79*8kB 54*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 9184kB
Sep 19 07:02:31 test02 kernel: [1448508.176275] 22976 total pagecache pages
Sep 19 07:02:31 test02 kernel: [1448508.176276] 22625 pages in swap cache
Sep 19 07:02:31 test02 kernel: [1448508.176277] Swap cache stats: add 10304453, delete 10281828, find 4517870/5118908
Sep 19 07:02:31 test02 kernel: [1448508.176278] Free swap  = 0kB
Sep 19 07:02:31 test02 kernel: [1448508.176279] Total swap = 5242872kB
Sep 19 07:02:31 test02 kernel: [1448508.194186] 2621440 pages RAM
Sep 19 07:02:31 test02 kernel: [1448508.194187] 54834 pages reserved
Sep 19 07:02:31 test02 kernel: [1448508.194188] 6100 pages shared
Sep 19 07:02:31 test02 kernel: [1448508.194189] 2551576 pages non-shared
Sep 19 07:02:31 test02 kernel: [1448508.194199] Out of memory: kill process 19529 (mysqld_safe) score 793921 or a child
Sep 19 07:02:31 test02 kernel: [1448508.194655] Killed process 6403 (mysqld)
Sep 19 07:02:31 test02 kernel: [1448508.200116] mysqld invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 19 07:02:31 test02 kernel: [1448508.200119] mysqld cpuset=/ mems_allowed=0
Sep 19 07:02:31 test02 kernel: [1448508.200121] Pid: 6438, comm: mysqld Not tainted 2.6.32-bpo.5-amd64 #1
Sep 19 07:02:31 test02 kernel: [1448508.200122] Call Trace:
Sep 19 07:02:31 test02 kernel: [1448508.200131]  [<ffffffff810b60f0>] ? oom_kill_process+0x7f/0x23f
Sep 19 07:02:31 test02 kernel: [1448508.200133]  [<ffffffff810b6614>] ? __out_of_memory+0x12a/0x141
Sep 19 07:02:31 test02 kernel: [1448508.200134]  [<ffffffff810b676b>] ? out_of_memory+0x140/0x172
Sep 19 07:02:31 test02 kernel: [1448508.200137]  [<ffffffff810ba4b4>] ? __alloc_pages_nodemask+0x4e5/0x5f5
Sep 19 07:02:31 test02 kernel: [1448508.200151]  [<ffffffff810bba19>] ? __do_page_cache_readahead+0x9b/0x1b4
Sep 19 07:02:31 test02 kernel: [1448508.200153]  [<ffffffff810bbb4e>] ? ra_submit+0x1c/0x20
Sep 19 07:02:31 test02 kernel: [1448508.200156]  [<ffffffff810b483e>] ? filemap_fault+0x17d/0x2f6
Sep 19 07:02:31 test02 kernel: [1448508.200160]  [<ffffffff810ca786>] ? __do_fault+0x54/0x3c3
Sep 19 07:02:31 test02 kernel: [1448508.200162]  [<ffffffff810ccaa5>] ? handle_mm_fault+0x3b8/0x80f
Sep 19 07:02:31 test02 kernel: [1448508.200168]  [<ffffffff8104a442>] ? default_wake_function+0x0/0x9
Sep 19 07:02:31 test02 kernel: [1448508.200173]  [<ffffffff812fead6>] ? do_page_fault+0x2e0/0x2fc
Sep 19 07:02:31 test02 kernel: [1448508.200175]  [<ffffffff812fc975>] ? page_fault+0x25/0x30
Sep 19 07:02:31 test02 kernel: [1448508.200176] Mem-Info:
Sep 19 07:02:31 test02 kernel: [1448508.200177] Node 0 DMA per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.200179] CPU    0: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200180] CPU    1: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200181] CPU    2: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200183] CPU    3: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200184] CPU    4: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200185] CPU    5: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200186] CPU    6: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200187] CPU    7: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200188] Node 0 DMA32 per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.200189] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200190] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200191] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200193] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200194] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200195] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200196] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200197] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200198] Node 0 Normal per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.200199] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200200] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200201] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200202] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200203] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200204] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200205] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200207] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.200209] active_anon:2171066 inactive_anon:350394 isolated_anon:32
Sep 19 07:02:31 test02 kernel: [1448508.200210]  active_file:122 inactive_file:28 isolated_file:0
Sep 19 07:02:31 test02 kernel: [1448508.200211]  unevictable:8 dirty:0 writeback:40 unstable:0
Sep 19 07:02:31 test02 kernel: [1448508.200211]  free:13675 slab_reclaimable:4379 slab_unreclaimable:3573
Sep 19 07:02:31 test02 kernel: [1448508.200212]  mapped:171 shmem:4 pagetables:10576 bounce:0
Sep 19 07:02:31 test02 kernel: [1448508.200213] Node 0 DMA free:15808kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15220kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.200220] lowmem_reserve[]: 0 3512 10077 10077
Sep 19 07:02:31 test02 kernel: [1448508.200222] Node 0 DMA32 free:30716kB min:4472kB low:5588kB high:6708kB active_anon:2822852kB inactive_anon:564344kB active_file:420kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596504kB mlocked:0kB dirty:0kB writeback:12kB mapped:4kB shmem:0kB slab_reclaimable:3736kB slab_unreclaimable:1144kB kernel_stack:168kB pagetables:10604kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.200228] lowmem_reserve[]: 0 0 6565 6565
Sep 19 07:02:31 test02 kernel: [1448508.200230] Node 0 Normal free:8176kB min:8360kB low:10448kB high:12540kB active_anon:5861412kB inactive_anon:837232kB active_file:68kB inactive_file:176kB unevictable:32kB isolated(anon):128kB isolated(file):0kB present:6722560kB mlocked:32kB dirty:0kB writeback:148kB mapped:680kB shmem:16kB slab_reclaimable:13780kB slab_unreclaimable:13132kB kernel_stack:2104kB pagetables:31700kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.200248] lowmem_reserve[]: 0 0 0 0
Sep 19 07:02:31 test02 kernel: [1448508.200249] Node 0 DMA: 4*4kB 2*8kB 4*16kB 3*32kB 2*64kB 3*128kB 1*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15808kB
Sep 19 07:02:31 test02 kernel: [1448508.200261] Node 0 DMA32: 42*4kB 84*8kB 105*16kB 93*32kB 77*64kB 49*128kB 25*256kB 10*512kB 1*1024kB 1*2048kB 0*4096kB = 31288kB
Sep 19 07:02:31 test02 kernel: [1448508.200266] Node 0 Normal: 915*4kB 80*8kB 54*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 9196kB
Sep 19 07:02:31 test02 kernel: [1448508.200272] 22976 total pagecache pages
Sep 19 07:02:31 test02 kernel: [1448508.200273] 22622 pages in swap cache
Sep 19 07:02:31 test02 kernel: [1448508.200274] Swap cache stats: add 10304455, delete 10281833, find 4517872/5118917
Sep 19 07:02:31 test02 kernel: [1448508.200275] Free swap  = 0kB
Sep 19 07:02:31 test02 kernel: [1448508.200276] Total swap = 5242872kB
Sep 19 07:02:31 test02 kernel: [1448508.219028] 2621440 pages RAM
Sep 19 07:02:31 test02 kernel: [1448508.219029] 54834 pages reserved
Sep 19 07:02:31 test02 kernel: [1448508.219030] 6101 pages shared
Sep 19 07:02:31 test02 kernel: [1448508.219031] 2551572 pages non-shared
Sep 19 07:02:31 test02 kernel: [1448508.219033] Out of memory: kill process 19529 (mysqld_safe) score 114957 or a child
Sep 19 07:02:31 test02 kernel: [1448508.219485] Killed process 6405 (mysqld)
Sep 19 07:02:31 test02 kernel: [1448508.224599] php5 invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
Sep 19 07:02:31 test02 kernel: [1448508.224605] php5 cpuset=/ mems_allowed=0
Sep 19 07:02:31 test02 kernel: [1448508.224608] Pid: 6669, comm: php5 Not tainted 2.6.32-bpo.5-amd64 #1
Sep 19 07:02:31 test02 kernel: [1448508.224609] Call Trace:
Sep 19 07:02:31 test02 kernel: [1448508.224618]  [<ffffffff810b60f0>] ? oom_kill_process+0x7f/0x23f
Sep 19 07:02:31 test02 kernel: [1448508.224620]  [<ffffffff810b6614>] ? __out_of_memory+0x12a/0x141
Sep 19 07:02:31 test02 kernel: [1448508.224622]  [<ffffffff810b676b>] ? out_of_memory+0x140/0x172
Sep 19 07:02:31 test02 kernel: [1448508.224626]  [<ffffffff810c8404>] ? congestion_wait+0x74/0x80
Sep 19 07:02:31 test02 kernel: [1448508.224630]  [<ffffffff81064f0a>] ? autoremove_wake_function+0x0/0x2e
Sep 19 07:02:31 test02 kernel: [1448508.224633]  [<ffffffff810ba4b4>] ? __alloc_pages_nodemask+0x4e5/0x5f5
Sep 19 07:02:31 test02 kernel: [1448508.224636]  [<ffffffff810cc967>] ? handle_mm_fault+0x27a/0x80f
Sep 19 07:02:31 test02 kernel: [1448508.224639]  [<ffffffff810d1c98>] ? do_brk+0x227/0x307
Sep 19 07:02:31 test02 kernel: [1448508.224644]  [<ffffffff812fead6>] ? do_page_fault+0x2e0/0x2fc
Sep 19 07:02:31 test02 kernel: [1448508.224646]  [<ffffffff812fc975>] ? page_fault+0x25/0x30
Sep 19 07:02:31 test02 kernel: [1448508.224648] Mem-Info:
Sep 19 07:02:31 test02 kernel: [1448508.224649] Node 0 DMA per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.224650] CPU    0: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224652] CPU    1: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224653] CPU    2: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224654] CPU    3: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224655] CPU    4: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224657] CPU    5: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224658] CPU    6: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224659] CPU    7: hi:    0, btch:   1 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224660] Node 0 DMA32 per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.224662] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224663] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224664] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224666] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224667] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224668] CPU    5: hi:  186, btch:  31 usd:  30
Sep 19 07:02:31 test02 kernel: [1448508.224669] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224670] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224671] Node 0 Normal per-cpu:
Sep 19 07:02:31 test02 kernel: [1448508.224673] CPU    0: hi:  186, btch:  31 usd:  22
Sep 19 07:02:31 test02 kernel: [1448508.224674] CPU    1: hi:  186, btch:  31 usd:  21
Sep 19 07:02:31 test02 kernel: [1448508.224675] CPU    2: hi:  186, btch:  31 usd:  15
Sep 19 07:02:31 test02 kernel: [1448508.224676] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224677] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224678] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224680] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224681] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:31 test02 kernel: [1448508.224684] active_anon:2171066 inactive_anon:350462 isolated_anon:32
Sep 19 07:02:31 test02 kernel: [1448508.224685]  active_file:122 inactive_file:28 isolated_file:0
Sep 19 07:02:31 test02 kernel: [1448508.224685]  unevictable:8 dirty:0 writeback:40 unstable:0
Sep 19 07:02:31 test02 kernel: [1448508.224686]  free:13897 slab_reclaimable:4264 slab_unreclaimable:3573
Sep 19 07:02:31 test02 kernel: [1448508.224687]  mapped:171 shmem:4 pagetables:10576 bounce:0
Sep 19 07:02:31 test02 kernel: [1448508.224688] Node 0 DMA free:15808kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15220kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 19 07:02:31 test02 kernel: [1448508.224695] lowmem_reserve[]: 0 3512 10077 10077
Sep 19 07:02:31 test02 kernel: [1448508.224697] Node 0 DMA32 free:30916kB min:4472kB low:5588kB high:6708kB active_anon:2822852kB inactive_anon:564344kB active_file:420kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596504kB mlocked:0kB dirty:0kB writeback:12kB mapped:4kB shmem:0kB slab_reclaimable:3736kB slab_unreclaimable:1144kB kernel_stack:168kB pagetables:10604kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.224705] lowmem_reserve[]: 0 0 6565 6565
Sep 19 07:02:31 test02 kernel: [1448508.224706] Node 0 Normal free:8864kB min:8360kB low:10448kB high:12540kB active_anon:5861412kB inactive_anon:837504kB active_file:68kB inactive_file:176kB unevictable:32kB isolated(anon):128kB isolated(file):0kB present:6722560kB mlocked:32kB dirty:0kB writeback:148kB mapped:680kB shmem:16kB slab_reclaimable:13320kB slab_unreclaimable:13132kB kernel_stack:2104kB pagetables:31700kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:31 test02 kernel: [1448508.224713] lowmem_reserve[]: 0 0 0 0
Sep 19 07:02:31 test02 kernel: [1448508.224715] Node 0 DMA: 4*4kB 2*8kB 4*16kB 3*32kB 2*64kB 3*128kB 1*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15808kB
Sep 19 07:02:31 test02 kernel: [1448508.224720] Node 0 DMA32: 42*4kB 87*8kB 105*16kB 86*32kB 70*64kB 46*128kB 24*256kB 11*512kB 2*1024kB 1*2048kB 0*4096kB = 31536kB
Sep 19 07:02:31 test02 kernel: [1448508.224725] Node 0 Normal: 822*4kB 116*8kB 79*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 9512kB
Sep 19 07:02:31 test02 kernel: [1448508.224730] 22976 total pagecache pages
Sep 19 07:02:31 test02 kernel: [1448508.224731] 22623 pages in swap cache
Sep 19 07:02:31 test02 kernel: [1448508.224732] Swap cache stats: add 10304488, delete 10281865, find 4517872/5118917
Sep 19 07:02:31 test02 kernel: [1448508.224733] Free swap  = 4kB
Sep 19 07:02:31 test02 kernel: [1448508.224734] Total swap = 5242872kB
Sep 19 07:02:31 test02 kernel: [1448508.243351] 2621440 pages RAM
Sep 19 07:02:31 test02 kernel: [1448508.243352] 54834 pages reserved
Sep 19 07:02:31 test02 kernel: [1448508.243353] 6114 pages shared
Sep 19 07:02:31 test02 kernel: [1448508.243354] 2551298 pages non-shared
Sep 19 07:02:31 test02 kernel: [1448508.243356] Out of memory: kill process 19529 (mysqld_safe) score 104511 or a child
Sep 19 07:02:31 test02 kernel: [1448508.243857] Killed process 6427 (mysqld)
Sep 19 07:02:37 test02 kernel: [1448514.538706] Connection attempt (UNPRIV): IN=eth0 OUT= MAC=52:54:00:95:8a:88:00:09:0f:09:00:04:08:00 SRC=184.105.247.220 DST=85.239.120.148 LEN=29 TOS=0x00 PREC=0x00 TTL=55 ID=38679 DF PROTO=UDP SPT=38880 DPT=53413 LEN=9 
Sep 19 07:02:40 test02 collectd[17910]: mysql_ping failed for mysql: MySQL server has gone away
Sep 19 07:02:40 test02 collectd[17910]: mysql plugin: Successfully connected to database <none> at server Localhost via UNIX socket (server version: 5.5.20-55, protocol version: 10)
Sep 19 07:02:51 test02 kernel: [1448528.098039] iotop invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 19 07:02:51 test02 kernel: [1448528.098043] iotop cpuset=/ mems_allowed=0
Sep 19 07:02:51 test02 kernel: [1448528.098046] Pid: 5954, comm: iotop Not tainted 2.6.32-bpo.5-amd64 #1
Sep 19 07:02:51 test02 kernel: [1448528.098047] Call Trace:
Sep 19 07:02:51 test02 kernel: [1448528.098055]  [<ffffffff810b60f0>] ? oom_kill_process+0x7f/0x23f
Sep 19 07:02:51 test02 kernel: [1448528.098058]  [<ffffffff810b6614>] ? __out_of_memory+0x12a/0x141
Sep 19 07:02:51 test02 kernel: [1448528.098060]  [<ffffffff810b676b>] ? out_of_memory+0x140/0x172
Sep 19 07:02:51 test02 kernel: [1448528.098062]  [<ffffffff810ba4b4>] ? __alloc_pages_nodemask+0x4e5/0x5f5
Sep 19 07:02:51 test02 kernel: [1448528.098065]  [<ffffffff810bba19>] ? __do_page_cache_readahead+0x9b/0x1b4
Sep 19 07:02:51 test02 kernel: [1448528.098067]  [<ffffffff810bbb4e>] ? ra_submit+0x1c/0x20
Sep 19 07:02:51 test02 kernel: [1448528.098070]  [<ffffffff810b483e>] ? filemap_fault+0x17d/0x2f6
Sep 19 07:02:51 test02 kernel: [1448528.098074]  [<ffffffff810ca786>] ? __do_fault+0x54/0x3c3
Sep 19 07:02:51 test02 kernel: [1448528.098078]  [<ffffffff8103fa28>] ? __wake_up+0x30/0x44
Sep 19 07:02:51 test02 kernel: [1448528.098080]  [<ffffffff810ccaa5>] ? handle_mm_fault+0x3b8/0x80f
Sep 19 07:02:51 test02 kernel: [1448528.098082]  [<ffffffff8103fa28>] ? __wake_up+0x30/0x44
Sep 19 07:02:51 test02 kernel: [1448528.098086]  [<ffffffff8102cdcc>] ? pvclock_clocksource_read+0x3a/0x8b
Sep 19 07:02:51 test02 kernel: [1448528.098090]  [<ffffffff8106c473>] ? ktime_get_ts+0x68/0xb2
Sep 19 07:02:51 test02 kernel: [1448528.098095]  [<ffffffff812fead6>] ? do_page_fault+0x2e0/0x2fc
Sep 19 07:02:51 test02 kernel: [1448528.098097]  [<ffffffff812fc975>] ? page_fault+0x25/0x30
Sep 19 07:02:51 test02 kernel: [1448528.098098] Mem-Info:
Sep 19 07:02:51 test02 kernel: [1448528.098099] Node 0 DMA per-cpu:
Sep 19 07:02:51 test02 kernel: [1448528.098101] CPU    0: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098102] CPU    1: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098103] CPU    2: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098105] CPU    3: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098106] CPU    4: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098107] CPU    5: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098108] CPU    6: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098109] CPU    7: hi:    0, btch:   1 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098110] Node 0 DMA32 per-cpu:
Sep 19 07:02:51 test02 kernel: [1448528.098112] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098113] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098114] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098115] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098116] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098117] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098118] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098120] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098121] Node 0 Normal per-cpu:
Sep 19 07:02:51 test02 kernel: [1448528.098122] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098123] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098124] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098125] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098126] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098127] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098128] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:51 test02 kernel: [1448528.098129] CPU    7: hi:  186, btch:  31 usd:   4
Sep 19 07:02:51 test02 kernel: [1448528.098132] active_anon:2171659 inactive_anon:350608 isolated_anon:0
Sep 19 07:02:51 test02 kernel: [1448528.098133]  active_file:0 inactive_file:1 isolated_file:0
Sep 19 07:02:51 test02 kernel: [1448528.098134]  unevictable:8 dirty:0 writeback:2 unstable:0
Sep 19 07:02:51 test02 kernel: [1448528.098134]  free:13642 slab_reclaimable:3635 slab_unreclaimable:3559
Sep 19 07:02:51 test02 kernel: [1448528.098135]  mapped:160 shmem:4 pagetables:10562 bounce:0
Sep 19 07:02:51 test02 kernel: [1448528.098136] Node 0 DMA free:15808kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15220kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:51 test02 kernel: [1448528.098143] lowmem_reserve[]: 0 3512 10077 10077
Sep 19 07:02:51 test02 kernel: [1448528.098145] Node 0 DMA32 free:30544kB min:4472kB low:5588kB high:6708kB active_anon:2824936kB inactive_anon:565116kB active_file:20kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596504kB mlocked:0kB dirty:0kB writeback:0kB mapped:12kB shmem:0kB slab_reclaimable:2840kB slab_unreclaimable:1136kB kernel_stack:424kB pagetables:8788kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Sep 19 07:02:51 test02 kernel: [1448528.098151] lowmem_reserve[]: 0 0 6565 6565
Sep 19 07:02:51 test02 kernel: [1448528.098153] Node 0 Normal free:8216kB min:8360kB low:10448kB high:12540kB active_anon:5861700kB inactive_anon:837316kB active_file:0kB inactive_file:88kB unevictable:32kB isolated(anon):0kB isolated(file):0kB present:6722560kB mlocked:32kB dirty:0kB writeback:8kB mapped:628kB shmem:16kB slab_reclaimable:11700kB slab_unreclaimable:13084kB kernel_stack:1584kB pagetables:33460kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:96 all_unreclaimable? no
Sep 19 07:02:51 test02 kernel: [1448528.098160] lowmem_reserve[]: 0 0 0 0
Sep 19 07:02:51 test02 kernel: [1448528.098161] Node 0 DMA: 4*4kB 2*8kB 4*16kB 3*32kB 2*64kB 3*128kB 1*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15808kB
Sep 19 07:02:51 test02 kernel: [1448528.098166] Node 0 DMA32: 19*4kB 118*8kB 106*16kB 84*32kB 52*64kB 42*128kB 19*256kB 9*512kB 3*1024kB 2*2048kB 0*4096kB = 30748kB
Sep 19 07:02:51 test02 kernel: [1448528.098171] Node 0 Normal: 1000*4kB 10*8kB 5*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8192kB
Sep 19 07:02:51 test02 kernel: [1448528.098176] 25331 total pagecache pages
Sep 19 07:02:51 test02 kernel: [1448528.098177] 25154 pages in swap cache
Sep 19 07:02:51 test02 kernel: [1448528.098178] Swap cache stats: add 10795511, delete 10770357, find 4524412/5129110
Sep 19 07:02:51 test02 kernel: [1448528.098179] Free swap  = 0kB
Sep 19 07:02:51 test02 kernel: [1448528.098180] Total swap = 5242872kB
Sep 19 07:02:51 test02 kernel: [1448528.116516] 2621440 pages RAM
Sep 19 07:02:51 test02 kernel: [1448528.116517] 54834 pages reserved
Sep 19 07:02:51 test02 kernel: [1448528.116518] 6275 pages shared
Sep 19 07:02:51 test02 kernel: [1448528.116519] 2551863 pages non-shared
Sep 19 07:02:51 test02 kernel: [1448528.116521] Out of memory: kill process 6836 (mysqld) score 945072 or a child
Sep 19 07:02:51 test02 kernel: [1448528.117013] Killed process 6836 (mysqld)
Sep 19 07:02:52 test02 kernel: [1448528.796162] tailf invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 19 07:02:52 test02 kernel: [1448528.796166] tailf cpuset=/ mems_allowed=0
Sep 19 07:02:52 test02 kernel: [1448528.796169] Pid: 5594, comm: tailf Not tainted 2.6.32-bpo.5-amd64 #1
Sep 19 07:02:52 test02 kernel: [1448528.796170] Call Trace:
Sep 19 07:02:52 test02 kernel: [1448528.796178]  [<ffffffff810b60f0>] ? oom_kill_process+0x7f/0x23f
Sep 19 07:02:52 test02 kernel: [1448528.796189]  [<ffffffff810b6614>] ? __out_of_memory+0x12a/0x141
Sep 19 07:02:52 test02 kernel: [1448528.796191]  [<ffffffff810b676b>] ? out_of_memory+0x140/0x172
Sep 19 07:02:52 test02 kernel: [1448528.796195]  [<ffffffff810c8404>] ? congestion_wait+0x74/0x80
Sep 19 07:02:52 test02 kernel: [1448528.796199]  [<ffffffff81064f0a>] ? autoremove_wake_function+0x0/0x2e
Sep 19 07:02:52 test02 kernel: [1448528.796201]  [<ffffffff810ba4b4>] ? __alloc_pages_nodemask+0x4e5/0x5f5
Sep 19 07:02:52 test02 kernel: [1448528.796204]  [<ffffffff810bba19>] ? __do_page_cache_readahead+0x9b/0x1b4
Sep 19 07:02:52 test02 kernel: [1448528.796206]  [<ffffffff810bbb4e>] ? ra_submit+0x1c/0x20
Sep 19 07:02:52 test02 kernel: [1448528.796209]  [<ffffffff810b483e>] ? filemap_fault+0x17d/0x2f6
Sep 19 07:02:52 test02 kernel: [1448528.796212]  [<ffffffff810ca786>] ? __do_fault+0x54/0x3c3
Sep 19 07:02:52 test02 kernel: [1448528.796214]  [<ffffffff810ccaa5>] ? handle_mm_fault+0x3b8/0x80f
Sep 19 07:02:52 test02 kernel: [1448528.796219]  [<ffffffff812fbd3e>] ? do_nanosleep+0x6d/0xa3
Sep 19 07:02:52 test02 kernel: [1448528.796221]  [<ffffffff812fead6>] ? do_page_fault+0x2e0/0x2fc
Sep 19 07:02:52 test02 kernel: [1448528.796223]  [<ffffffff812fc975>] ? page_fault+0x25/0x30
Sep 19 07:02:52 test02 kernel: [1448528.796225] Mem-Info:
Sep 19 07:02:52 test02 kernel: [1448528.796226] Node 0 DMA per-cpu:
Sep 19 07:02:52 test02 kernel: [1448528.796228] CPU    0: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796229] CPU    1: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796230] CPU    2: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796231] CPU    3: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796232] CPU    4: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796233] CPU    5: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796235] CPU    6: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796236] CPU    7: hi:    0, btch:   1 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796237] Node 0 DMA32 per-cpu:
Sep 19 07:02:52 test02 kernel: [1448528.796238] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796239] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796241] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796242] CPU    3: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796243] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796244] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796245] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796246] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796247] Node 0 Normal per-cpu:
Sep 19 07:02:52 test02 kernel: [1448528.796248] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796250] CPU    1: hi:  186, btch:  31 usd:  51
Sep 19 07:02:52 test02 kernel: [1448528.796251] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796252] CPU    3: hi:  186, btch:  31 usd:   6
Sep 19 07:02:52 test02 kernel: [1448528.796253] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796254] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796255] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796256] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:52 test02 kernel: [1448528.796259] active_anon:2171478 inactive_anon:350627 isolated_anon:0
Sep 19 07:02:52 test02 kernel: [1448528.796260]  active_file:80 inactive_file:122 isolated_file:0
Sep 19 07:02:52 test02 kernel: [1448528.796261]  unevictable:8 dirty:0 writeback:51 unstable:0
Sep 19 07:02:52 test02 kernel: [1448528.796261]  free:13630 slab_reclaimable:3642 slab_unreclaimable:3560
Sep 19 07:02:52 test02 kernel: [1448528.796262]  mapped:0 shmem:4 pagetables:10518 bounce:0
Sep 19 07:02:52 test02 kernel: [1448528.796263] Node 0 DMA free:15808kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15220kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 19 07:02:52 test02 kernel: [1448528.796269] lowmem_reserve[]: 0 3512 10077 10077
Sep 19 07:02:52 test02 kernel: [1448528.796272] Node 0 DMA32 free:30588kB min:4472kB low:5588kB high:6708kB active_anon:2824496kB inactive_anon:565068kB active_file:112kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596504kB mlocked:0kB dirty:0kB writeback:0kB mapped:24kB shmem:0kB slab_reclaimable:2840kB slab_unreclaimable:1132kB kernel_stack:304kB pagetables:8604kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:41 all_unreclaimable? no
Sep 19 07:02:52 test02 kernel: [1448528.796278] lowmem_reserve[]: 0 0 6565 6565
Sep 19 07:02:52 test02 kernel: [1448528.796280] Node 0 Normal free:8124kB min:8360kB low:10448kB high:12540kB active_anon:5861416kB inactive_anon:837440kB active_file:208kB inactive_file:496kB unevictable:32kB isolated(anon):0kB isolated(file):0kB present:6722560kB mlocked:32kB dirty:0kB writeback:296kB mapped:0kB shmem:16kB slab_reclaimable:11728kB slab_unreclaimable:13092kB kernel_stack:1592kB pagetables:33468kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:61 all_unreclaimable? no
Sep 19 07:02:52 test02 kernel: [1448528.796290] lowmem_reserve[]: 0 0 0 0
Sep 19 07:02:52 test02 kernel: [1448528.796291] Node 0 DMA: 4*4kB 2*8kB 4*16kB 3*32kB 2*64kB 3*128kB 1*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15808kB
Sep 19 07:02:52 test02 kernel: [1448528.796296] Node 0 DMA32: 102*4kB 87*8kB 135*16kB 78*32kB 46*64kB 44*128kB 19*256kB 9*512kB 3*1024kB 2*2048kB 0*4096kB = 30976kB
Sep 19 07:02:52 test02 kernel: [1448528.796306] Node 0 Normal: 1086*4kB 20*8kB 2*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8568kB
Sep 19 07:02:52 test02 kernel: [1448528.796311] 24642 total pagecache pages
Sep 19 07:02:52 test02 kernel: [1448528.796312] 24341 pages in swap cache
Sep 19 07:02:52 test02 kernel: [1448528.796313] Swap cache stats: add 10796955, delete 10772614, find 4524684/5129522
Sep 19 07:02:52 test02 kernel: [1448528.796315] Free swap  = 0kB
Sep 19 07:02:52 test02 kernel: [1448528.796315] Total swap = 5242872kB
Sep 19 07:02:52 test02 kernel: [1448528.814573] 2621440 pages RAM
Sep 19 07:02:52 test02 kernel: [1448528.814574] 54834 pages reserved
Sep 19 07:02:52 test02 kernel: [1448528.814575] 6263 pages shared
Sep 19 07:02:52 test02 kernel: [1448528.814576] 2551685 pages non-shared
Sep 19 07:02:52 test02 kernel: [1448528.814578] Out of memory: kill process 6898 (mysqld) score 841725 or a child
Sep 19 07:02:52 test02 kernel: [1448528.815007] Killed process 6898 (mysqld)
Sep 19 07:02:57 test02 kernel: [1448533.883234] apache2 invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 19 07:02:57 test02 kernel: [1448533.883237] apache2 cpuset=/ mems_allowed=0
Sep 19 07:02:57 test02 kernel: [1448533.883240] Pid: 14602, comm: apache2 Not tainted 2.6.32-bpo.5-amd64 #1
Sep 19 07:02:57 test02 kernel: [1448533.883241] Call Trace:
Sep 19 07:02:57 test02 kernel: [1448533.883250]  [<ffffffff810b60f0>] ? oom_kill_process+0x7f/0x23f
Sep 19 07:02:57 test02 kernel: [1448533.883253]  [<ffffffff810b6614>] ? __out_of_memory+0x12a/0x141
Sep 19 07:02:57 test02 kernel: [1448533.883255]  [<ffffffff810b676b>] ? out_of_memory+0x140/0x172
Sep 19 07:02:57 test02 kernel: [1448533.883257]  [<ffffffff810ba4b4>] ? __alloc_pages_nodemask+0x4e5/0x5f5
Sep 19 07:02:57 test02 kernel: [1448533.883262]  [<ffffffff812fb356>] ? io_schedule+0x93/0xb7
Sep 19 07:02:57 test02 kernel: [1448533.883265]  [<ffffffff810bba19>] ? __do_page_cache_readahead+0x9b/0x1b4
Sep 19 07:02:57 test02 kernel: [1448533.883268]  [<ffffffff81064f38>] ? wake_bit_function+0x0/0x23
Sep 19 07:02:57 test02 kernel: [1448533.883271]  [<ffffffff810bbb4e>] ? ra_submit+0x1c/0x20
Sep 19 07:02:57 test02 kernel: [1448533.883281]  [<ffffffff810b483e>] ? filemap_fault+0x17d/0x2f6
Sep 19 07:02:57 test02 kernel: [1448533.883285]  [<ffffffff810ca786>] ? __do_fault+0x54/0x3c3
Sep 19 07:02:57 test02 kernel: [1448533.883288]  [<ffffffff810ccaa5>] ? handle_mm_fault+0x3b8/0x80f
Sep 19 07:02:57 test02 kernel: [1448533.883291]  [<ffffffff8106c473>] ? ktime_get_ts+0x68/0xb2
Sep 19 07:02:57 test02 kernel: [1448533.883294]  [<ffffffff812fead6>] ? do_page_fault+0x2e0/0x2fc
Sep 19 07:02:57 test02 kernel: [1448533.883296]  [<ffffffff812fc975>] ? page_fault+0x25/0x30
Sep 19 07:02:57 test02 kernel: [1448533.883297] Mem-Info:
Sep 19 07:02:57 test02 kernel: [1448533.883298] Node 0 DMA per-cpu:
Sep 19 07:02:57 test02 kernel: [1448533.883300] CPU    0: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883302] CPU    1: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883303] CPU    2: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883304] CPU    3: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883305] CPU    4: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883307] CPU    5: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883308] CPU    6: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883309] CPU    7: hi:    0, btch:   1 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883310] Node 0 DMA32 per-cpu:
Sep 19 07:02:57 test02 kernel: [1448533.883312] CPU    0: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883313] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883314] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883315] CPU    3: hi:  186, btch:  31 usd:  31
Sep 19 07:02:57 test02 kernel: [1448533.883317] CPU    4: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883318] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883319] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883320] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883321] Node 0 Normal per-cpu:
Sep 19 07:02:57 test02 kernel: [1448533.883322] CPU    0: hi:  186, btch:  31 usd:  30
Sep 19 07:02:57 test02 kernel: [1448533.883324] CPU    1: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883325] CPU    2: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883326] CPU    3: hi:  186, btch:  31 usd:  71
Sep 19 07:02:57 test02 kernel: [1448533.883327] CPU    4: hi:  186, btch:  31 usd:  45
Sep 19 07:02:57 test02 kernel: [1448533.883328] CPU    5: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883330] CPU    6: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883331] CPU    7: hi:  186, btch:  31 usd:   0
Sep 19 07:02:57 test02 kernel: [1448533.883334] active_anon:2171285 inactive_anon:351098 isolated_anon:0
Sep 19 07:02:57 test02 kernel: [1448533.883335]  active_file:93 inactive_file:137 isolated_file:64
Sep 19 07:02:57 test02 kernel: [1448533.883336]  unevictable:8 dirty:0 writeback:46 unstable:0
Sep 19 07:02:57 test02 kernel: [1448533.883336]  free:13672 slab_reclaimable:3607 slab_unreclaimable:3555
Sep 19 07:02:57 test02 kernel: [1448533.883337]  mapped:236 shmem:4 pagetables:10466 bounce:0
Sep 19 07:02:57 test02 kernel: [1448533.883339] Node 0 DMA free:15808kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15220kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 19 07:02:57 test02 kernel: [1448533.883346] lowmem_reserve[]: 0 3512 10077 10077
Sep 19 07:02:57 test02 kernel: [1448533.883348] Node 0 DMA32 free:30660kB min:4472kB low:5588kB high:6708kB active_anon:2823204kB inactive_anon:566900kB active_file:120kB inactive_file:228kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596504kB mlocked:0kB dirty:0kB writeback:0kB mapped:148kB shmem:0kB slab_reclaimable:2808kB slab_unreclaimable:1132kB kernel_stack:96kB pagetables:8596kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:77 all_unreclaimable? no
Sep 19 07:02:57 test02 kernel: [1448533.883355] lowmem_reserve[]: 0 0 6565 6565
Sep 19 07:02:57 test02 kernel: [1448533.883357] Node 0 Normal free:8220kB min:8360kB low:10448kB high:12540kB active_anon:5861936kB inactive_anon:837492kB active_file:252kB inactive_file:320kB unevictable:32kB isolated(anon):0kB isolated(file):256kB present:6722560kB mlocked:32kB dirty:0kB writeback:188kB mapped:796kB shmem:16kB slab_reclaimable:11620kB slab_unreclaimable:13072kB kernel_stack:1560kB pagetables:33268kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:591 all_unreclaimable? yes
Sep 19 07:02:57 test02 kernel: [1448533.883364] lowmem_reserve[]: 0 0 0 0
Sep 19 07:02:57 test02 kernel: [1448533.883366] Node 0 DMA: 4*4kB 2*8kB 4*16kB 3*32kB 2*64kB 3*128kB 1*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15808kB
Sep 19 07:02:57 test02 kernel: [1448533.883371] Node 0 DMA32: 97*4kB 143*8kB 115*16kB 76*32kB 46*64kB 43*128kB 19*256kB 9*512kB 3*1024kB 2*2048kB 0*4096kB = 30892kB
Sep 19 07:02:57 test02 kernel: [1448533.883376] Node 0 Normal: 1018*4kB 33*8kB 2*16kB 3*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8368kB
Sep 19 07:02:57 test02 kernel: [1448533.883381] 27264 total pagecache pages
Sep 19 07:02:57 test02 kernel: [1448533.883382] 26991 pages in swap cache
Sep 19 07:02:57 test02 kernel: [1448533.883383] Swap cache stats: add 10809615, delete 10782624, find 4528347/5134185
Sep 19 07:02:57 test02 kernel: [1448533.883385] Free swap  = 0kB
Sep 19 07:02:57 test02 kernel: [1448533.883385] Total swap = 5242872kB
Sep 19 07:02:57 test02 kernel: [1448533.902453] 2621440 pages RAM
Sep 19 07:02:57 test02 kernel: [1448533.902454] 54834 pages reserved
Sep 19 07:02:57 test02 kernel: [1448533.902455] 6277 pages shared
Sep 19 07:02:57 test02 kernel: [1448533.902456] 2551734 pages non-shared
Sep 19 07:02:57 test02 kernel: [1448533.902458] Out of memory: kill process 6655 (php5) score 56993 or a child
Sep 19 07:02:57 test02 kernel: [1448533.902804] Killed process 6669 (php5)
Sep 19 07:03:00 test02 collectd[17910]: mysql_ping failed for mysql: MySQL server has gone away
Sep 19 07:03:00 test02 collectd[17910]: mysql plugin: Failed to connect to database <none> at server localhost: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
Sep 19 07:03:00 test02 collectd[17910]: read-function of plugin `mysql' failed. Will suspend it for 20 seconds.
Sep 19 07:03:01 test02 /USR/SBIN/CRON[6938]: (root) CMD (   php5    /var/www/xres_jt/infx2cache/update2cache.php 1 > /dev/null 2>&1)
Sep 19 07:03:20 test02 collectd[17910]: mysql plugin: Failed to connect to database <none> at server localhost: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
Sep 19 07:03:20 test02 collectd[17910]: read-function of plugin `mysql' failed. Will suspend it for 40 seconds.
Sep 19 07:04:00 test02 collectd[17910]: mysql plugin: Failed to connect to database <none> at server localhost: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
Sep 19 07:04:00 test02 collectd[17910]: read-function of plugin `mysql' failed. Will suspend it for 80 seconds.