Using sk_buff to add an Ethernet frame header

Question

I have a kernel module that captures outgoing Internet traffic(Netfilter hook: LOCAL_OUT) At this hook, there's still no Ethernet header.

I built the Ethernet header and it's ready to use, but how can I attach it to the skb so that I can send the whole skb struct to dev_queue_xmit() ?

Is there any guide on how to manipulate sk_buff data that you can provide me for further information?

As a sample, I try to do what I want to do on all ECHO ICMP traffic; this is the code I have. But when checking with Wireshark on the other machine, I get a working Ethernet header but an EMPTY IP packet...how come?

static unsigned int post_icmp_check(unsigned int hooknum,
                 struct sk_buff *skb,
                 const struct net_device *in,
                 const struct net_device *out,
                 int (*okfn)(struct sk_buff *))
{
  struct iphdr *iph;
  struct icmphdr *icmph;
  struct sk_buff *sb;
  struct ethhdr *ethh;
  unsigned char *frame = NULL; //Used just to copy the skb->data and replace it, that's all
  unsigned char *ptr;

  /* Allocations and initial assignations  */
  sb = skb_copy(skb, GFP_ATOMIC); /* Making a private copy because we cannot modify all fields in the public one */
  iph = ip_hdr(sb);

  if(iph->protocol == IPPROTO_ICMP){
      printk(KERN_ALERT"POST ICMP traffic!\n");

      icmph = icmp_hdr(sb);
      if(icmph->type == ICMP_ECHO){
          frame = (unsigned char *)kmalloc(ntohs(iph->tot_len) + (ETH_HLEN*8), GFP_KERNEL);
          ptr = (frame +  (ETH_HLEN * 8));

          ethh = (struct ethhdr*) frame;      //00:1a:80:7e:96:ba
          ethh->h_source[0] = 0x00;   
          ethh->h_source[1] = 0x1a;           
          ethh->h_source[2] = 0x80;
          ethh->h_source[3] = 0x7e;
          ethh->h_source[4] = 0x96;
          ethh->h_source[5] = 0xba;
          ethh->h_dest[0] = 0xff;
          ethh->h_dest[1] = 0xff;
          ethh->h_dest[2] = 0xff;
          ethh->h_dest[3] = 0xff;
          ethh->h_dest[4] = 0xff;
          ethh->h_dest[5] = 0xff;
          ethh->h_proto = htons(ETH_P_IP);
          memcpy(ptr, skb->data, ntohs(iph->tot_len)); /* Copying the rest of the data from the skb to sb */
          sb->data = frame;

          //icmph = icmp_hdr(sb);

          //icmph->un.echo.sequence = htons(i++);         
          //icmph->checksum = 0;
          //icmph->checksum =  in_cksum((unsigned short *)icmph, ntohs(iph->tot_len) - sizeof(struct iphdr));
          sb->dev = out;
          printk(KERN_ALERT"%s\n",sb->dev->name);
          if(sb->dev == NULL)
              printk(KERN_ALERT"dev is NULL!");           
          dev_queue_xmit(sb);
          return NF_STOLEN;
}

EDIT : And yes, I am hardcoding my own MAC and BROADCASTING the frame, for the sake of simplicity only.

EDIT2: So I am giving this function the original sk_buff caught by net_filter and the appropriate net_device I tried to follow what you said and read a little guide about the functions for the sK_buff. Still, when I dev_queue_xmit() the returned sk_buff of this function, I get a complete kernel crash.

static struct sk_buff* set_skb(struct sk_buff *org_skb, struct net_device *dev)
{
    int header_size;
    unsigned char *data;
    struct icmphdr *icmph;
    struct iphdr *iph;      
    struct icmphdr *temp_icmph;
    struct iphdr *temp_iph;         
    unsigned char myaddr[] = {0x00,0x1a,0x80,0x7e,0x96,0xba};
    unsigned char addr [] = {0xff,0xff,0xff,0xff,0xff,0xff};    
    struct sk_buff *skb;

    temp_iph = ip_hdr(org_skb); 
    temp_icmph = icmp_hdr(org_skb);

    skb = alloc_skb(ntohs((temp_iph->tot_len) + (ETH_HLEN*8)), GFP_KERNEL); /* Allocating enough space for everything */

    /* icmp_hlen which is always 8, ip_hlen */
    header_size = 8 + ntohs(temp_iph->ihl*4);


    /* reserve headroom This actually allocates the IP and ICMP headers memory */
    skb_reserve(skb, header_size);

    /* payload */
    data = skb_put(skb, org_skb->data_len);/* This function allocates room for the data */
    memcpy(data, org_skb->data, org_skb->data_len);

    icmph = (struct icmphdr*)skb_push(skb, 8); 
    /* set up icmp header here */
    memcpy(icmph, temp_icmph, 8);   

    iph = (struct iphdr*)skb_push(skb, temp_iph->ihl*4);
    /* set up ip header here */
    memcpy(iph, temp_iph, temp_iph->ihl*4); 

    /*
     * This function sets up the ethernet header,
     * destination address addr, source address myaddr
     */
    dev_hard_header(skb, dev, ETH_P_IP, addr, myaddr, dev->addr_len);
    return skb;
}

EDIT3:

Apr 20 23:44:09 DHS-1022CYB kernel: [ 1139.520104] Loading Test module...
Apr 20 23:44:09 DHS-1022CYB kernel: [ 1139.520168] *********************HERE WE GO**********************
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.760448] PGD 114952067 PUD 12d932067 PMD 0 
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.760762] CPU 0 
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.760809] Modules linked in: myModule(P) binfmt_misc rfcomm ppdev sco bridge stp bnep l2cap joydev fbcon tileblit font bitblit softcursor vga16fb vgastate snd_hda_codec_realtek pcmcia arc4 snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event iwlagn snd_seq iwlcore snd_timer radeon ttm drm_kms_helper snd_seq_device mac80211 tifm_7xx1 led_class usbhid yenta_socket hid rsrc_nonstatic btusb bluetooth uvcvideo videodev v4l1_compat v4l2_compat_ioctl32 snd drm tifm_core i2c_algo_bit psmouse sony_laptop pcmcia_core serio_raw cfg80211 soundcore snd_page_alloc video output intel_agp lp parport ohci1394 ieee1394 r8169 mii
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.762642] Pid: 3423, comm: ping Tainted: P           2.6.32-33-generic #70-Ubuntu VGN-CR31Z_R
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.762753] RIP: 0010:[<ffffffffa042c081>]  [<ffffffffa042c081>] post_icmp_check+0x81/0x1e0 [myModule]
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.762891] RSP: 0018:ffff88011494f948  EFLAGS: 00010246
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.762965] RAX: 0000000000000000 RBX: ffff88013bb62000 RCX: 00000000000000d0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763059] RDX: 0000000000000000 RSI: ffff88013bb62000 RDI: 0000000000000000
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763153] RBP: ffff88011494f988 R08: 00000000ffffffff R09: 0000000000000001
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763248] R10: ffffffff81591000 R11: ffff880137c9c9ef R12: ffff880130ba8100
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763346] R13: ffffffff81831070 R14: ffff880130ba8100 R15: 0000000000000003
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763441] FS:  00007f3377ef6700(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763546] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763626] CR2: 0000000000000000 CR3: 0000000114863000 CR4: 00000000000006f0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763721] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763815] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.763910] Process ping (pid: 3423, threadinfo ffff88011494e000, task ffff880139b6c4d0)
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.764052]  ffff88011494fde8 0000000000000040 0000000000609420 0000000000000000
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.764195] <0> ffff88011494fa10 0000000080000000 ffffffff81831070 ffff880130ba8100
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.768380] <0> ffff88011494f9d8 ffffffff81486f1c ffff88013bb62000 0000000000000000
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81486f1c>] nf_iterate+0x6c/0xb0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81491b20>] ? dst_output+0x0/0x20
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81486fd4>] nf_hook_slow+0x74/0x100
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81491b20>] ? dst_output+0x0/0x20
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81493c3f>] __ip_local_out+0x9f/0xb0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81493c66>] ip_local_out+0x16/0x30
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81493f0a>] ip_push_pending_frames+0x28a/0x410
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff8148cd02>] ? ip_route_output_flow+0x82/0xb0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff814b40b5>] raw_sendmsg+0x3a5/0x5c0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff814be5b9>] inet_sendmsg+0x29/0x60
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81285389>] ? apparmor_socket_sendmsg+0x19/0x20
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff8145004b>] sock_sendmsg+0x10b/0x140
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81084cc0>] ? autoremove_wake_function+0x0/0x40
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff8144f5b4>] ? move_addr_to_kernel+0x64/0x70
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff8145a706>] ? verify_iovec+0x66/0xd0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff814504c3>] sys_sendmsg+0x233/0x3a0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff811131c9>] ? __do_fault+0x439/0x500
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff81541845>] ? _spin_lock_irq+0x15/0x20
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff810787ae>] ? do_sigaction+0x13e/0x1e0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff815444a8>] ? do_page_fault+0x158/0x3b0
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  [<ffffffff810121b2>] system_call_fastpath+0x16/0x1b
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.770015]  RSP <ffff88011494f948>
Apr 20 23:44:12 DHS-1022CYB kernel: [ 1142.887370] ---[ end trace 404e94883c3bf110 ]---

And that was the kernel log result after the crash.

EDIT4: To make things clearer, this is the part of the code that actually calls my set_skb()

if(icmph->type == ICMP_ECHO){
            memcpy(dev, out, sizeof(*out)); 
            sb = set_skb(skb, dev);
            printk(KERN_ALERT"%s\n",sb->dev->name);           
            dev_queue_xmit(sb);
            return NF_STOLEN;
        }

Solution

Wow, you don't do that. Don't mess with skb internal pointers directly. Just allocate an skb and copy the headers and payload to the new one.

Let's assume you want to add ICMP, IP and ethernet headers and copy payload from orig_skb. Do it like this:

struct skbuff *skb = skb_alloc(full_len, GFP_KERNEL);

/* icmp_hlen, ip_hlen and payload_size should be known */
int header_size = icmp_hlen + ip_hlen;

/* reserve headroom */
skb_reserve(skb, header_size);

/* payload */
unsigned char *data = skb_put(skb, payload_size);
memcpy(data, orig_skb->data, payload_size);

struct icmphdr *icmph = skb_push(skb, icmp_hlen);
/* set up icmp header here */

struct iphdr *iph = skb_push(skb, ip_hlen);
/* set up ip header here */

/*
 * This function sets up the ethernet header,
 * destination address addr, source address myaddr
 */
dev_hard_header(skb, dev, ETH_P_IP, addr, myaddr, dev->addr_len);

You can push multiple headers if you want to add transport, network etc headers.

Comments

[Fingolfin]

Hmmm I thought I wasn't messing directly with the skb because I did make a copy of it, and edited the copy, or that's also wrong? Also, can you please explain your code a little a bit more? Sorry. The first line allocates an SKB Then? skb_reserve adds a room for the header? Which header? and what does the last function do? Excuse my ignorance..

[ldx]

See my edit. After allocation, you reserve headroom, copy the payload, and then push headers into the skb (in reverse order, so first ICMP header, then IP header). In the end you add the link layer header via dev_hard_header().

[Fingolfin]

Sir, bear with me, I am sorry. Can you take a look at my EDIT2 please? I tried to follow what you've said and well, I am now getting a kernel crash. What did I do wrong up there?

[ldx]

Looks good to me. Do you get a trackback or an address where the crash happens?

[Fingolfin]

Take a look at the edit. Also, the kernel crashed before even sending out the traffic, Wireshark at the other end didn't receive a thing.

[ldx]

Can you fire up gdb and see where post_icmp_check+0x81 is? You might need to recompile your module with EXTRA_CFLAGS=-g.