Trouble Calling ioctl from user-space C

Question

I'm trying to implement a program to access memory on an embedded system. I need to access some control register so I think that ioctl is the best way to do it. I have added the ioctl to the fops:

struct file_operations aes_fops = {
  read: aes_read,
  write: aes_write,
  unlocked_ioctl: aes_ioctl,
  open: aes_open,
  release: aes_release
};

And have set up the function:

int aes_ioctl(struct inode *inode,  
     struct file *file, 
     unsigned int ioctl_num,    
     unsigned long ioctl_param){

     printk(KERN_INFO "in ioctl\n");
....
}

But I am not getting inside of this function. Here is my user space code. Please help me understand if I am doing this totally wrong.

int main(int argc, char* argv[]){
    int fd = fopen("/dev/aes", "r+");
    ioctl(fd, 0, 1);
    fclose(fd);
}

Some of the code is apparently for older kernels, because I am compiling for an embedded system where an older version of Linux has been modified.

Solution 1

The problem with your code is the request number you are using - 0. The kernel has some request number reserved for internal use. The kernel regards the request number as a struct, separates it to fields and calls the right subsystem for it.

See Documentation/ioctl/ioctl-number.txt (from Linux 3.4.6):

Code  Seq#(hex) Include File            Comments
========================================================
0x00    00-1F   linux/fs.h              conflict!
0x00    00-1F   scsi/scsi_ioctl.h       conflict!
0x00    00-1F   linux/fb.h              conflict!
0x00    00-1F   linux/wavefront.h       conflict!
0x02    all     linux/fd.h
0x03    all     linux/hdreg.h
...

Depending on what you are during, you'd have to follow the kernel guidelines for adding new ioctls():

If you are adding new ioctl's to the kernel, you should use the _IO
macros defined in <linux/ioctl.h>:

    _IO    an ioctl with no parameters
    _IOW   an ioctl with write parameters (copy_from_user)
    _IOR   an ioctl with read parameters  (copy_to_user)
    _IOWR  an ioctl with both write and read parameters.

See the kernel's own Documentation/ioctl/ioctl-decoding.txt document for further details on how those numbers are structured.

In practice, if you pick Code 1, which means starting from 0x100 up until 0x1ff, you'd be fine.

Solution 2

Is your setup of the aes_fops structure correct? I've never seen it done that way. All the code I have is:

.unlocked_ioctl = aes_ioctl,

rather than:

unlocked_ioctl: aes_ioctl,

Colons within a structure (as you have in your setup) fields are used for bit fields as far as I'm aware (and during definition), not initialising the individual fields.

In other words, try:

struct file_operations aes_fops = {
    .read            = aes_read,
    .write           = aes_write,
    .unlocked_ioctl  = aes_ioctl,
    .open            = aes_open,
    .release         = aes_release
};

---

Note: It appears that gcc once did allow that variant of structure field initialisation but it's been obsolete since gcc 2.5 (see here, straight from the GCC documentation). You really should be using the proper method (ie, the one blessed by the ISO standard) to do this.

Solution 3

Without knowing the error returned, it's hard to say... My first though is your permissions on your file descriptor. I've seen similar issues before. First, you can get some more information about the failure if you take a look at the return of the ioctl:

#include <errno.h>
int main(int argc, char* argv[])
{
    long ret;     
    int fd = fopen("/dev/aes", "r+");
    ret = ioctl(fd, 0, 1);
    if (ret < 0)
        printf("ioctl failed. Return code: %d, meaning: %s\n", ret, strerror(errno));
    fclose(fd); 
} 

Check the return values and this should help give you something to search on. Why check? See the bottom of the post... Next in order to check if it is permissions issue, run the command:

ls -l /dev/aes

if you see something like:

crw------- 1 root root 10, 57 Aug 21 10:24 /dev/aes

Then just issue a:

sudo chmod 777 /dev/aes

And give it a try again. I bet it will work for you. (Note I ran that with root permissions since root is the owner of my version of your mod)

If the permissions are already OK, then I have a few more suggestions:

1) To me, the use of fopen/fclose is strange. You really only need to do:

int fd = open("/dev/aes");
close(fd);

My system doesn't even let your code compile as is.

2) Your IOCTL parameter list is old, I don't know what kernel version your compiling on, but recent kernels use this format:

long aes_ioctl(struct file *file, unsigned int ioctl_num, unsigned long ioctl_param){   

Note the removal of the inode and the change of the return type. When I ran your code on my system, I made these changes.

Best of luck!

Note: Why should we check the return when we're "not getting into the ioctl"? Let me give you an example:

//Kernel Code:
//assume include files, other fops, exit, miscdev struct, etc. are present

long hello_ioctl(struct file *file, unsigned long ioctl_num, unsigned long ioctl_param) {
    long ret = 0;
    printk("in ioctl");
    return ret;
}

static const struct file_operations hello_fops = {
    owner: THIS_MODULE,
    read: hello_read,
    unlocked_ioctl: hello_ioctl,
};

static int __init hello_init(void) {
    int ret;
    printk("hello!\n");
    ret = misc_register(&hello_dev); //assume it worked...
    return ret;
}

User space code:

//assume includes

void main() {
  int fd;
  long ret;
  fd = open("/dev/hello");
  if(fd) {
    c = ioctl(fd, 0, 1);
    if (c < 0)
      printf("error: %d, errno: %d, meaning: %s\n", c, errno, strerror(errno));
    close(fd);
  }
  return;
}

So what's the output? Lets assume bad file permissions on /dev/hello (meaning our user space program can't access /dev/hello).

The dmesg | tail shows:

[ 2388.051660] Hello!

So it looks like we didn't "get in to" the ioctl. What's the output from the program?

error: -1, errno: 9, meaning: Bad file descriptor

Lots of useful output. Clearly the ioctl call did something, just not what we wanted. Now changing the permissions and re-running we can see the new dmesg:

[ 2388.051660] Hello!
[ 2625.025339] in ioctl

Comments

[Dan Aloni]

The C99 standard is backward compatible, and he can use the old initialization method if he likes (unless the driver is submitted back to the kernel maintainers). It has nothing to do with the problem at hand, though.

[paxdiablo]

The c99 standard, and earlier ISO ones, make no mention of that method. I assume you meant gcc 'standard', ie the extension. You may well be right on the actual cause, in which case your answer will no doubt blow mine out of the water :-) I'll still leave mine here though since I think it's a good idea to use proper language features.

[Dan Aloni]

See this question.

[paxdiablo]

@Don, I'm aware that C99 has designated initialisers but the field: value variant is a gcc extension (now obsoleted), not part of the standard. It actually infers as much in the linked document you provided in the other answer. That form is obsoleted, because the standard now provides a way to do it. That's the point I was making, not that your description of designated initialisers itself was somehow deficient.

[paxdiablo]

In fact, @DanAloni, since your answer is no doubt more correct than my as to the actual cause of the problem, here's an upvote for you :-) Don't reciprocate, I want to make sure the best answers rise to the top.

[Stuart]

I don't believe I am getting into the ioctl function, so there wouldn't be any return value. I have the proper permissions too. Can you further explain the "open" function. Is that proper for C and can you point me to some documentation? I updated the OP explaining this is for an older version of Linux for an embedded system.

[Mike]

Did you try checking? If your code compiles and runs then you did "get in" to the ioctl call, you just may not have made it all the way to your aes_ioctl handler. If this is the case, the return is not something you set, it's the system's way of letting you know what/how you're not accessing the function. For example you might get a EBADF (bad file handler), or EFAULT (references in inaccessible memory). Very good indications of why your ioclt handler isn't being called. for open() and close() those are just C system calls link

[Stuart]

So, I would need to put a definition for a _IO macro into linux/ioctl.h like: #define IOCTL_AES_<DEF> _IOW(0x01, 0x01, int) and then I can create other definitions by increasing the middle parameter?

[Stuart]

I don't think the user space ioctl is calling the right function. I have a printk in the ioctl function that isn't showing up in the console, so that is why I assume the two functions just aren't connected yet. The "open" function has 3 parameters, right? What is the advantage over "fopen"?

[Mike]

Check out my edit as to why you should be checking the output even if you think it's not working. This will give us more of a hint into what's wrong. open/fopen won't be the cause of your problem, there's no major difference wrt your issue. I just like open better because it's faster.

[Stuart]

Also, I am using nested Makefiles. How can I determine which "linux/ioctl.h" to add the definitions to?

[Dan Aloni]

The alternative is to keep the declaration of the ioctl number local in your headers. At any case, if you are modifying Linux headers you should modify the ones your kernel module builds against, and if you are not building the kernel itself too there's no point in modifying Linux headers anyway.

[Stuart]

If I want to write to the "/dev/aes" file should I be using fopen?