Implementing a new system call in Kernel version 2.6.32

A system call is used by application or user programs to request service from the operating systems. Since the user programs does not have direct access to the kernel whereas the OS has the direct access. OS can access the hardware through system calls only.

The following files has to be modified for implementing a system call

1. /usr/src/linux-2.6.32.5/arch/x86/kernel/syscall_table_32.S
2. /usr/src/linux-2.6.32.5/arch/x86/include/asm/unistd_32.h
3. /usr/src/linux-2.6.32.5/include/linux/syscalls.h
4. /usr/src/linux-2.6.32.5/Makefile

New set of files to be created

1. Create a new directory newcall/ inside the path “/usr/src/linux-2.6.32.5/”
2. Create new files Makefile, newcall.c and put them in the /usr/src/linux-2.6.32.5/newcall/ folder
3. Create new user files (in any folder of Linux) to test the system call
   testnewcall.c, testnewcall.h (created in /home/pradeepkumar)

syscall_table_32.S

Find the file /usr/src/linux-2.6.32.5/arch/x86/kernel/syscall_table_32.S and add the following line at the end
".long sys_newcall"  (add without double quotes, but the preceding . should)

unistd_32.h
open the file /usr/src/linux-2.6.32.5/arch/x86/include/asm/unistd_32.h
(all the system calls will be defined in this file using #define macro)

This file contains the system call number that is passed to the kernel through the register (EAX) when a system call is invoked.

Add "#define __NR_mycall <Last_System_Call_Num + 1>" at the end of the list.
If the last system call defined here is:
"#define __NR_vmsplice   336", then add:
"#define __NR_newcall   337" at the end of the list. (337 is the new system call number)

Increment the "NR_syscalls" by 1. So, if NR_syscalls is defined as:
"#define NR_syscalls 337", then change it to:
"#define NR_syscalls 338"  (Since we added a new kernel, so the total number of system calls should be incremented)

syscalls.h
open the file /usr/src/linux-2.6.32.5/include/linux/syscalls.h
Add the following line at the end of the file:
"asmlinkage long sys_newcall(int i);"  (without double quotes)

Makefile
Full path of the file - /usr/src/linux-2.6.32.5/Makefile
Create a new directory newcall/ under the folder /usr/src/linux-2.6.32.5
and include that path to /usr/src/linux-2.6.32.5/Makefile

open the /usr/src/linux-2.6.32.5/Makefile
and find the "core-y += " and append newcall/ to the path (please see the image below)

newcall.c
Create a new file called newcall.c with full path: /usr/src/linux-2.6.32.5/newcall/newcall.c
/*---Start of newcall.c----*/
#include <linux/linkage.h>
asmlinkage long sys_newcall(int i)
{
return i*10; //the value passed from the user program will be multiplied by 10
}
/*---End of newcall.c------*/

Makefile
Create a new file called Makefile with full path: /usr/src/linux-2.6.32.5/newcall/Makefile
and paste the following line
obj-y := newcall.o

Create userspace files to test the system call
create two files testnewcall.c and testnewcall.h and the full path of the files are
/home/pradeepkumar/testnewcall.c
/home/pradeepkumar/testnewcall.h

testnewcall.c

#include <stdio.h>
#include "testnewcall.h"
int main(void)
{
printf("%d\n", newcall(15)); // since 15 is passed, the output should be 15*10=150
return 0;
}

testnewcall.h

#include<linux/unistd.h>
#define __NR_newcall 337

long newcall(int i)
{
return syscall(__NR_newcall,i);
}

Note: "_syscall1(long, mycall, int, i)" this can be added instead of

long newcall(int i)
{
return syscall(__NR_newcall,i);
}

Macro _syscall1()

_syscall1(long, newcall, int, i)
the importance of the above syscall is

• The name of the system call is newcall.
• It takes one argument.
• The argument is an int named number.
• It returns an long.

Testing the new system call
Step 1: Recompile and install the new kernel so that our system call becomes available to the operating system. go to the kernel folder and give command make

Step 2: Reboot the system

Step 3: Compile and execute the user space C file (testnewcall.c) that we created above. (gcc testnewcall.c and then execute ./a.out)

RESULT: You should see the output as 150. This has been tested on kernel 2.6.32.5.

Source: http://tldp.org/HOWTO/html_single/Implement-Sys-Call-Linux-2.6-i386/ (The above link uses kernel version 2.6.17 and it uses different set of files)

My post uses a recent kernel 2.6.32.5 and it modifies different set of files.

Any doubts, please query through the comments….