{"id":735,"date":"2018-03-29T10:43:12","date_gmt":"2018-03-29T02:43:12","guid":{"rendered":"https:\/\/blog.jsjs.org\/?p=735"},"modified":"2018-03-29T10:43:12","modified_gmt":"2018-03-29T02:43:12","slug":"how-to-enable-kdump-on-rhel-7-and-centos-7","status":"publish","type":"post","link":"https:\/\/blog.jsjs.org\/?p=735","title":{"rendered":"How to enable Kdump on RHEL 7 and CentOS 7"},"content":{"rendered":"

by\u00a0Pradeep Kumar<\/a><\/span>\u00a0<\/span>\u00b7 Published\u00a0

<\/div>\n
\n
\n

Kdump is a kernel feature which is used to capture crash dumps when the system or kernel crash. For enabling kdump we have to reserve some portion of physical RAM which will be used to execute kdump kernel in the event of kernel panic or crash.<\/p>\n

When a kernel crash or kernel panic occurs then running kernel runs \u2018kexec(kdump kernel)<\/strong>\u2018 and it loads kdump kernel from reserve memory and then contents of RAM and Swap is copied to vmcore file either on local disk or on remote disk and finally reboot the box.<\/p>\n

By analyzing the crash dumps we can find the reason or the root case of system failure. If you have OS support then you can share the crash dumps to the vendor for analysis.<\/p>\n

In this article we will demonstrate how to enable kdump on RHEL 7 and CentOS 7<\/p>\n

Step:1 Install \u2018kexec-tools\u2019 using yum command<\/h4>\n

Use the below yum command to install \u2018kexec-tools\u2019 package in case it is not installed.<\/p>\n

[root@cloud ~]# yum install kexec-tools<\/pre>\n
Step:2 Update the GRUB2 file to Reserve Memory for Kdump kernel<\/h4>\n
Edit the GRUB2 file (\/etc\/default\/grub<\/strong>), add the parameter \u2018crashkernel=<Reserved_size_of_RAM><\/strong>\u2018 in the line beginning with \u2018GRUB_CMDLINE_LINUX<\/strong>\u2018<\/p>\n
GRUB_CMDLINE_LINUX=\"rd.lvm.lv=centos\/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=centos\/root crashkernel=128M vconsole.keymap=us rhgb quiet\"<\/pre>\n
\"grub-file-centos7\"<\/p>\n
Execute the below command to regenerate grub2 configuration.<\/strong><\/p>\n
[root@cloud ~]# grub2-mkconfig -o \/boot\/grub2\/grub.cfg<\/pre>\n
In case of UEFI firmware, use the below command<\/strong><\/p>\n
[root@cloud ~]# grub2-mkconfig -o \/boot\/efi\/EFI\/redhat\/grub.cfg<\/pre>\n
Above command will inform bootlaoder to reserve 128 MB RAM after reboot.<\/p>\n
Reboot the box now using below command :<\/strong><\/p>\n
[root@cloud ~]# shutdown -r now<\/pre>\n
Step:3 Update the dump location & default action in the file (\/etc\/kdump.conf)<\/h4>\n
To store crash dump or vmcore file on a local file system, edit the file \u2018\/etc\/kdump.conf<\/strong>\u2018 and specify the location as per your setup. In my case i am using a separate local file system (\u00a0\/var\/crash<\/strong>). It is recommended that size of file system should be equivalent to the size of your system\u2019s RAM or file system should have free space equivalent to the size of RAM. Kdump allows to compress the dump data using \u2018core collector\u2019 option (core_collector makedumpfile -c<\/strong>\u00a0) where -c is used for compression.<\/p>\n
In case if kdump fails to store the dump file to specified location then default action will be performed which is mention in the default directive. In my case default action is reboot.<\/p>\n
Update the below three directives in kdump.conf file.<\/strong><\/p>\n
[root@cloud ~]# vi \/etc\/kdump.conf\n\npath \/var\/crash\ncore_collector makedumpfile -c\ndefault reboot<\/pre>\n
Different Options to store dump :<\/strong><\/p>\n
\"options-store-dump\"<\/a><\/p>\n
Step:4 Start and enable kdump service<\/h4>\n
[root@cloud ~]# systemctl start kdump.service\n[root@cloud ~]# systemctl enable kdump.service\n[root@cloud ~]#<\/pre>\n
Step:5 Now Test Kdump by manually crashing the system<\/h4>\n
Before crashing your system , please verify whether the kdump service is running or not using below command.<\/p>\n
[root@cloud crash]# systemctl is-active kdump.service\n[root@cloud crash]# service kdump status<\/pre>\n
To test our kdump configuration we will manually crash our system with below commands.<\/p>\n
[root@cloud ~]# echo 1 > \/proc\/sys\/kernel\/sysrq ; echo c > \/proc\/sysrq-trigger<\/pre>\n
This will create a crash dump file (vmcore<\/strong>\u00a0) under \u2018\/var\/crash<\/strong>\u2018 file system.<\/p>\n
[root@cloud ~]# ls -lR \/var\/crash\n\/var\/crash:\ntotal 0\ndrwxr-xr-x. 2 root root 42 Mar 4 03:02 127.0.0.1-2016-03-04-03:02:17\n\n\/var\/crash\/127.0.0.1-2016-03-04-03:02:17:\ntotal 135924\n-rw-------. 1 root root 139147524 Mar 4 03:02 vmcore\n-rw-r--r--. 1 root root 35640 Mar 4 03:02 vmcore-dmesg.txt\n[root@cloud ~]#<\/pre>\n
Step:6 Use \u2018crash\u2019 command to analyze and debug crash dumps<\/h4>\n
Crash is the utility or command to debug and analyze the crash dump or vmcore file.<\/p>\n
To use the crash, make sure two packages are installed : \u2018crash & kernel-debuginfo<\/strong>\u2018<\/p>\n
[root@cloud ~]# yum install crash<\/pre>\n
To install \u2018kernel-debuginfo\u2019 package , first enable debug repo. Edit the repo file\u00a0\/etc\/yum.repos.d\/CentOS-Debuginfo.repo<\/p>\n
change \u2018enbled=0\u2019 to \u2018enabled=1\u2019<\/p>\n
[root@cloud ~]# yum install kernel-debuginfo<\/pre>\n
Once the kernel-debuginfo is installed , then try to execute below crash command, it will give us a crash prompt where we can run commands to find process info , list of open files when the system got crashed.<\/p>\n
[root@cloud ~]# crash \/var\/crash\/127.0.0.1-2016-03-04-14\\:20\\:06\/vmcore \/usr\/lib\/debug\/lib\/modules\/`uname -r`\/vmlinux\n\ncrash><\/pre>\n
Type \u2018ps<\/strong>\u2018 command to list the Process which were running when the system got crashed.<\/p>\n
crash> ps<\/pre>\n
\"process-list-during-system-crash\"<\/a><\/p>\n
To view the files that were open when system got crashed , type \u2018files\u2019 command at crash prompt.<\/p>\n
crash> files\nPID: 5577 TASK: ffff88007b44f300 CPU: 0 COMMAND: \"bash\"\nROOT: \/ CWD: \/root\n FD FILE DENTRY INODE TYPE PATH\n 0 ffff880036b85000 ffff8800796fa540 ffff88007966f4d0 CHR \/dev\/pts\/0\n 1 ffff880036b73900 ffff880068c409c0 ffff8800794a8d10 REG \/proc\/sysrq-trigger\n 2 ffff880036b85000 ffff8800796fa540 ffff88007966f4d0 CHR \/dev\/pts\/0\n 10 ffff880036b85000 ffff8800796fa540 ffff88007966f4d0 CHR \/dev\/pts\/0\n255 ffff880036b85000 ffff8800796fa540 ffff88007966f4d0 CHR \/dev\/pts\/0\ncrash><\/pre>\n
Type\u00a0\u2018sys\u2019 command<\/strong>\u00a0to list the system info when it got crashed.<\/p>\n
crash> sys\n KERNEL: \/usr\/lib\/debug\/lib\/modules\/3.10.0-327.10.1.el7.x86_64\/vmlinux\n DUMPFILE: \/var\/crash\/127.0.0.1-2016-03-04-14:20:06\/vmcore\n CPUS: 1\n DATE: Fri Mar 4 14:20:01 2016\n UPTIME: 00:02:00\nLOAD AVERAGE: 0.75, 0.48, 0.19\n TASKS: 115\n NODENAME: cloud.linuxtechi.com\n RELEASE: 3.10.0-327.10.1.el7.x86_64\n VERSION: #1 SMP Tue Feb 16 17:03:50 UTC 2016\n MACHINE: x86_64 (2388 Mhz)\n MEMORY: 2 GB\n PANIC: \"SysRq : Trigger a crash\"\ncrash><\/pre>\n
To get help of any command on crash prompt , type \u2018help <command><\/strong>\u2018 , example is shown below.<\/p>\n
\"help-crash-commands\"<\/a><\/p>\n
That\u2019s conclude the article, Please don\u2019t hesitate to share it if you have enjoyed.<\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
by\u00a0Pradeep Kumar\u00a0\u00b7 Published\u00a0March 6, 2016\u00a0\u00b7 Updated\u00a0August 3, 2017 Kdump is a kernel feature which is used to capture crash dumps […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[1],"tags":[],"class_list":["post-735","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=\/wp\/v2\/posts\/735","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=735"}],"version-history":[{"count":0,"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=\/wp\/v2\/posts\/735\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=735"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=735"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.jsjs.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=735"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}