Showing posts with label kernel. Show all posts
Showing posts with label kernel. Show all posts

Tuesday, June 18, 2013

Kung-f00: surviving in the kernel

There are two level of programming in an environment like the Linux Kernel, imho: survival and release. Survival programming is what you will do when you need to prototype or a proof-of-concept. Release programming (or refactoring) takes place when you want your code to be accepted in the kernel as a patch or another driver. Here are some ancient, half-forgotten programming techniques that could help you survive.
Initialization

If you have been raised in OOP, you may need to un-learn a few things to survive in the kernel. Especially your habits on constructors. Every time you can define an initial value at compile time rather than at run-time, you save yourself from the need to check your initialisation has indeed happened, and you avoid the risk that some mistake you’ve made could prevent the system from booting altogether. You don’t necessarily need to pollute the top-level name space for that:

{
   static int reported=0;
   do_swap_page(p);
   if (!reported && my_page(p)) {
       printk("my page got swapped.\n");
       reported=1;
   }
}

The static reported variable will have block-limited scope (so you could have as many such block as you need, and always use the name ‘reported’), but program-wide lifetime (and thus its value will persist across function calls). I believe this saves us from many __init functions.

just useless ...

Do you need a reference to some kernel object X? Save yourself the problem of figuring out whether that object will be ready when your __init function would be called. Use a NULL value as initialisation and ensure that you properly use lazy initialization
   if (!backing_device) backing_device=find_swap_device(0);
Downside is that you’ll have to repeat this line in every access point of your code. That’s unacceptable at the scope of a corporate business logic, but perfectly maintainable at the size of a prototype.

Do you need N bytes of memory to be available and can’t afford using kmalloc because your lazy initialization could take place at a moment where it could require memory that isn’t available right now (yup, even the kernel could be low on memory) ? Well, just be plain 8-bit style and go
   byte mySpace[N];
that will be set in the bss segment of your kernel, wiped with zeroes by the loader, and don’t take extra space on the vmlinux kernel image.

Probe System Call

Ideally, when you want a feature to be optional in the Linux Kernel, you wrap it in a module that can be loaded and unloaded. The price to pay for that is a more sophisticated template to follow and significant attention to memory management to ensure whatever your module doesn’t let dangling pointers when it’s gone. Core kernel functionalities typically get enabled/disabled through system control variable in /proc, just like support for the “magic SysRq key”. That’s the good way to go for a release. During prototyping, we can be old-school and define a new system call that we’ll use for invoking late initialisation in contexts where the initialisation, and otherwise enable communication from our user-world.

The /proc filesystem requires a significant amount of support data structure to work properly, yet if using /proc/kernel/myprobe is fine and if a mere int is enough for your communication needs, it comes down to
  • locate the appropriate place to add a entry in kern_table[], with .procname=”myprobe”;
  • mimmic sysrq_sysctl_handler, and add your own static int to receive the value decoded by proc_dointvec;
  • use a clone of the sysrq_sysctl_handler() function: let proc_dointvec deal with the buffer that comes from user-space, and just launch your code logic if the user is writing to your control variable.
Dynamic Allocation within a Table

You’re used to linked list, hash tables, trees, stacks and other dynamic data structures but realise that allocating cells wouldn’t be a wise idea in the kernel ? Congrats. Kernel developer put significant effort in crafting the slab and kernel object caches to reduce allocation costs, but at a prototype scale, it isn’t clear it’s worth learning how to master it. The “16-bit style” solution to this, given that you have a static table of N elements and that “N shall be enough for everybody and we guarantee there will be at least N slots available” is to embed a linked list of free slots within your table.

You don’t even need pointers for that: simply one field in your table slot that can hold an integer index to the next free slot.
int free_slot=-2;
int allocate_slot() {
   int al;
   if (free_slot==-1) return -1; // all slots used
   if (free_slot==-2) { // lazy init: chain all entries
      for (int i=0;i<N-1;i++) table[i].next=i+1;
      table[N-1].next=-1;
      free_slot=0;
   }
   al=free_slot; free_slot=table[al].next;
   return al;
}
Now you can focus on the real problem of guaranteeing that this table won’t suffer concurrent allocation issues :P

Saturday, July 09, 2011

per-socket statistics

Think twice before you pick the source of information about network behaviour…
Especially, /proc/$PID/net and friends are _not_ process-dependent as you might think. At least, not until your process got a dedicated _namespace_ to work with. So yes, every `struct sock` has a sk_net field that eventually leads to some statistics (`sock->sk_net->mib.net_statistics`), but if you look closely enough, e.g. by hooking one of the less frequent system calls and insert the following:

      {  // DEBUG CHECK
    struct net* n = sk?sk->sk_net:0;
    struct inet_connection_sock* ics = inet_csk(sk); // see net/inet_connection_sock.h
    struct tcp_sock* ts=tcp_sk(sk); // see linux/tcp.h

    printk(KERN_ERR “socket %p (#%i) has net@%p and stats@%p\n”,
    sk, index, n, n?n->mib.net_statistics:(void*)0xdeadbeef);

    }
You’ll see in your logs reports of various sockets addresses that all share the same net@xxxxxxxx and stats@yyyyyyyy … in other words, whatever you read there is not specific. Only if two sockets are in different namespaces (which could occur due to virtualization, chrooting and similar effects), they will have distinct statistics … if they’re from distinct process, that is. All the connections from your firefox browser will still share the same “fate” here.

So what should we look for ?

Well, give a look at the `tcp_get_info()` function, for instance. At the request of some `getsockopt()` call from user-land, it starts filling a `tcp_info` structure with information coming from the different “layers” of the `sock` structure. TCP retransmissions, for instance, are present in `inet_connection_sock.icsk_retransmits`. Running estimation of the RTT (yes, TCP does that for you) can be found in `tcp_sock.rcv_rtt_est`, and so on.

Always hunt for first-hand information.

Monday, November 03, 2008

Hacking gspca for Logitech QuickCam E1000 support.

Hop, je fais le saut: j'achète une webcam. Pas chère (25 francs) avec oreillette inclue "für skype" ... Mon choix s'est arrêté sur la QuickCam E1000 de Logitech, dans un rayon qui semble comporter une douzaine d'autres webcam qui se ressemblent toutes, en ce qui me concerne, hormis peut-être leur prix.

Je branche ça sur mon portable linux ... et évidemment ça ne marche pas tout seul. Le CD ne comporte que des drivers pour windows, ce qui n'est pas une surprise non-plus. Haa... on a pas encore fini de changer le monde.

Bref. Un petit tour de google, et je tombe là-dessus : http://doc.kubuntu-fr.org/spca5xx "installer gspca sous ubuntu, la version facile" ... je prends. je suis les instructions à la lettre, le module (comprenez "le driveur" pour les windoziens) s'installe. c'est bien. Sauf que point de webcam.

I just blindly bought the Logitech QuickCam E1000 this week-end (since it was the cheapest webcam around) and of course realised that it did not came with any Linux driver (not that it's surprising in any way) nor does it seems to be supported by any "out-of-the-box" driver for Ubuntu gutsy (unless i'm proved wrong). Hacking around, checking lsusb and reading the sources of the gspca driver, i figured out that it was yet-another-generic-driver that has a "white list" of supported devices. I thus added the product-id to that white list and now proudly have a working webcam. See compiled driver (which might no longer work with Labtec Webcam Pro, because i've really been *hacking* instead of *patching*) and modified sources down the page.

Un petit "lsusb" me donne au moins la référence "technique" du produit : Bus 001 Device 012: ID 046d:08af Logitech, Inc.

Ces deux numéros magiques (vendor-id=046d et product-id=08af) me permettent de voir (dans les sources du driver, en bas à gauche de l'image) que si pas mal d'autres webcam logitech sont supportées, par contre, la mienne n'est visiblement pas là. Comme le "gspca" est un driver "générique" (à savoir qu'il est en fait valide avec toutes les webcams 'standard'), j'ai 9 chances sur 10 pour que le code puisse effectivement faire fonctionner la webcam si j'arrive à expliquer au driver que "si, si, je t'assure, tu la connais aussi, celle-là". Un peu comme si votre clé-télécommande devait reconnaître la plaque minéralogique de votre voiture pour pouvoir la faire démarrer.

Bref. Premier jeu, donc, ajouter dans gspca_core.c l'identité de notre caméra. Perso, je parasite n'importe quelle entrée dans device_table[], mais il serait plus propre d'y ajouter {USB_DEVICE(0x046d, 0x08af)}, /* Logitech QuickCam E1000*/ et d'éditer en conséquence clist[] et l'énumération des caméras.

Avec ça, le driver peut dire au noyau que la webcam est pour lui, mais ça ne suffit pas. La fonction spcaDetectCamera() est appelée chaque fois qu'une nouvelle webcam est branchée, histoire de lui souhaiter la bienvenue, de prendre un verre avec les autres périphériques USB ... euh ... ou pas. C'est là que l'on va retrouver un gros "switch" qui va règler les options de notre driver générique pour chaque modèle de caméra (peut-être pas si générique que ça, donc, en fait :P) Je fais bêtement le pari que ma caméra "8af" sera probablement juste une réédition de la "8ae" (QuickCam for Notebooks), et donc :

   case 0x08ae:
case 0x08af: // uber-experimental.
    spca50x->desc = QuickCamNB;
    spca50x->bridge = BRIDGE_ZC3XX;
    spca50x->sensor = SENSOR_HDCS2020;
    break;
Et hop! ça marche! J'adore linux!

le module recompilé pour gutsy : gspca_e1000.ko le fichier gspca_core modifié pour la QuickCam E1000 : gspca_core_e1000.c (soyons open-source jusqu'au bout ;)

Et pour ceux qui trouvent que "Aarhgh! Skype, c'est le mal", je propose la lecture de l'édifiant "Castle in the Skype" de Fabrice DESCLAUX, puis je leur dirai que MSN, c'est pas forcément mieux ;)

edit: cette manipulation est valable sur Gutsy uniquement. Sous Hardy Heron (et probablement les versions ultérieures), les drivers pré-compilés supportent directement la caméra.

Thursday, April 10, 2008

Buggy Beetle

un bug vraiment très bête, en fait. Genre, deux lignes inversées dans un module noyau, et vlan, tout plante ... Je ne suis pas trop rouillé en Assembleur et en noyau linux, heureusement. A partir du crashdump sur le terminal (zd1201_usbrx+0x717), j'ai réussi malgré tout à retrouver le bout du driver pour ma clé wifi USB qui a provoqué un beau gros plantage de Gutsy Gibbon, la release ubuntu installée sur mon nouveau beetle. Il aura fallu attendre le passage à Hardy Heron pour que la correction arrive dans Ubuntu. Je suis aussi arrivé à la conclusion (après un peu plus de chipotage), que la ligne
 skb->dev->last_rx = jiffies; 
supposée indiquer dans l'état du driver l'instant où le dernier paquet avait été reçu tentait de modifier un "device" qui n'a jamais été initialisé auparavant ("thou shall not follow thee NULL pointer", comme dirait Cyril). J'aurais bien rajouté à la main un "skb->dev = zd->dev", mais j'ai fini par faire une petite recherche google qui m'a appris beaucoup plus rapidement qu'en fait, ce champs-là est initialisé par la fonction appelée juste en-dessous (eth_type_trans, pour les intimes) :P Le code correct lit donc
  memcpy(skb_put(skb, len), data, len);
  skb->protocol = eth_type_trans(skb, zd->dev);
  skb->dev->last_rx = jiffies;
  zd->stats.rx_packets++;

Bon, par contre, pas trouvé le moyen de recompiler juste le module foireux (voir plus bas). Il est teatime, et je lance ma première recompilation de noyau sur mon dual-c0re tout neuf à 4GB de RAM... ... 1/2 heure plus tard, la recompilation est terminée, mais évidemment, les batteries de ma DS n'ont pas attendu jusque là. 'faudra essayer tout ça demain :P here's a recompiled zd1201.ko module for the default gutsy kernel reusing local configuration (dual core, just in case). 

Install in /lib/modules/2.6.22-14-generic/kernel/drivers/net/wireless/ and make sure you have fresh firmware files (available on http://linux-lc100020.sourceforge.net/, to be installed in /lib/firmware/2.6.22-14-generic/) 2.6.22-14-generic/kernel/drivers/net/> md5sum wireless/zd12* 5e109dc87a1fa80d7c67834ce0d7a11c wireless/zd1201.ko 0ed15b4dab3e4f52f978a031ae102e02 wireless/zd1201.ko.original (i'd suggest that you *don't* replace your wireless/zd1201.ko unless with mine unless you happen to have the same md5 as wireless/zd1201.ko.original) edit: ça marche. c'est reparti pour le homebrew du temps de midi ^_^ edit+: oh, au fait: tout ça parce que je voulais vérifier que mon installation du dernier devkitpro (r21) était au poil... edit: Thanks to this post by linuxtilti, i figured out how to make a small Makefile that lets me rebuild just the module i need

obj-m = zd1201.o
KVERSION = $(shell uname -r)

all:
 make -C /lib/modules/$(KVERSION)/build M=$(PWD) modules
clean:
 make -C /lib/modules/$(KVERSION)/build M=$(PWD) clean
Okay, that was damn obvious, but i'm always confused by kernel makefiles, somehow. They export all their regular duties to those called builders, just pointing out "here is where i stand". I suspect that only obj-m is then imported by the builder ...