UNIX Socket FAQ

A forum for questions and answers about network programming on Linux and all other Unix-like systems

You are not logged in.

#1 2010-09-27 12:02 PM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

Hi,

I doubt this is the correct place to ask since it concerns kernel space but I can't find decent information.

I'm trying to write a line discipline to handle the protocol specific timing for DMX. (lighting control protocol, RS-485 based)

To mark the start of a new universe as it is called, a 513 byte bittrain that runs over the serial, very precise timing is required. The line should go down for 175 usecs (called the break), go up for 35 usecs (called the mark after break) and then the bittrain is sent.

I enabled high res timers in the kernel and checked that high res timers are enabled. But the precision of the timing in my dmx_tty driver (just a name for the line discipline) is miserable. The break varies from 175 usecs up to 250 usecs and the mark after break is equally imprecise.

I find it hard to believe that I can't get better precision on a 400 MHz CPU so I hope that somebody can tell me how timing like this is generally handled or point me to drivers that have similar timing requirements that I can study.

BTW. the kernel version I use is 2.6.32.

Offline

#2 2010-09-27 04:13 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

Take what we say with a grain of salt, we might have it wrong.

That said, some things do come to mind:

- Have you looked at existing implementations?

http://www.pcdmx.de/gb/software/drivers/linux/linux.htm
http://llg.cubic.org/dmx4linux/

- Is the timing difference a practical problem? That is, if the timings aren't
that narrow, just minimum values, then the time overruns aren't a real problem.

If it should be very close then I'd just send a bitsream of zeroes and ones
totalling those timings (amount sent depends on line bitrate of course).
That way the line timing is independend of scheduling.

If it's almost good enough then it's possible to figure out how to get the timings
better, but generally you don't get too much guarantees with sleep functions.
E.g. disable interrupts, use udelay, make sure you're using a decent clocksource,
etc. High resolution timers shouldn't make a difference because generally
you don't do such short delays with timers, but use udelay() instead. That
you mention them probably means you're using timers, in which case using
udelay() should probably fix your problem.

As for the reason for the "big" overruns: It could be anything. If the cpu went
into sleep mode, it can take a lot of time to get out of it again. An interrupt
might happen at the wrong time. If you use timers, multiple timers might be
grouped together when they're near enough, or the clocksource isn't precise
enough, or the code you use to measure time isn't precise enough or takes
too long, etc, etc.

Offline

#3 2010-09-27 05:43 PM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

Thx for the hefty reply!

i3839;30892 wrote:

Take what we say with a grain of salt, we might have it wrong.

That said, some things do come to mind:

- Have you looked at existing implementations?

http://www.pcdmx.de/gb/software/drivers/linux/linux.htm
http://llg.cubic.org/dmx4linux/

I took a look at this code, but it seems to be for a specific type of interface card. There is no code for managing timings so I guess the card takes care of that.


- Is the timing difference a practical problem? That is, if the timings aren't
that narrow, just minimum values, then the time overruns aren't a real problem.

I wish it wasn't a practical problem. I first noticed the problem when I was trying out my utterly cool dmx solution with some real world equipment. The signal seemed to stutter and hence no decent lighting.

If it should be very close then I'd just send a bitsream of zeroes and ones
totalling those timings (amount sent depends on line bitrate of course).
That way the line timing is independend of scheduling.

If it's almost good enough then it's possible to figure out how to get the timings
better, but generally you don't get too much guarantees with sleep functions.
E.g. disable interrupts, use udelay, make sure you're using a decent clocksource,
etc. High resolution timers shouldn't make a difference because generally
you don't do such short delays with timers, but use udelay() instead. That
you mention them probably means you're using timers, in which case using
udelay() should probably fix your problem.

what I don't get is that you have nanosecond resolution but the accuracy something in the range of 50 usecs. Just doesn't register. I even have this line in the boot sequence:
tcb_clksrc: tc0 at 16.012 MHz

If I understand correctly this means that my clock ticks every 63 nanosecs. So why is the accuracy so bad when I use the timers. Perhaps it's like you say below. Interrupts that mess up my timing.

As for the reason for the "big" overruns: It could be anything. If the cpu went
into sleep mode, it can take a lot of time to get out of it again. An interrupt
might happen at the wrong time. If you use timers, multiple timers might be
grouped together when they're near enough, or the clocksource isn't precise
enough, or the code you use to measure time isn't precise enough or takes
too long, etc, etc.

I measured the breaks and mark after break with an external tool. (very old scope in single shot mode... who would have thought I would need it again)

Perhaps the zero bittrain works, I don't know. At least not yet ;) I'm definitely going to try that one too. It's just not so easy since the ldisc uses dma for its writes.

I just can't believe there are no other drivers that need microsecond accuracy and not just a very high resolution. Perhaps any pointers?

Offline

#4 2010-09-27 08:57 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

Your clock is probably external, it takes time to read its value and to program it,
generating a timer interrupt and handling it takes time too, etc, etc. At least
on x86, if you're running ARM as I suspect it's more a matter of getting out of
an idle/sleep state or such things than the above.

If you are using timers then switch to what most drivers that need such precise
timing do: use udelay(). I'm pretty sure that will fix it enough for you, assuming
the delay between issuing a dma request and data transfer actually happening
isn't too big.

If that sometimes doesn't work then also disable interrupts when you're doing
timing sensitive things, but keep in mind that will add latency to other things
that might be going on at the same time. That is, your awful latencies might
also be caused by another driver disabling interrupts for too long. This is as
likely as other interrupt handlers keeping your cpu busy.

If you weren't using dma it would be hard to get any timing right when sending
the zero bittrain, but you got dedicated hardware that does cycle precise
timing for you. So all you have to do is calculate how much zeroes to send and
instead of waiting, just send them. Seems very simple to me, it's basically
totally timing insensitive code, because all timing is taken care of by the
hardware.

Offline

#5 2010-09-28 01:45 PM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

Well, I've been looking into your proposal to use the bit-train itself to manage the break and mark after break (mab) time. And as with all great ideas it is not that simple.
Simply because of the stop bits after each 8 bits. To pull this off I should lower the baud rate to a level where writing 1 zero byte would last as long as the break time. Similar for the mark after break after which the baud rate should be set back to normal.

But what I failed to mention was that my little setup will receive quite some network traffic which will cause a lot of interrupts which is why the timing was so fluctuating in the first place. And I can't afford to miss packets so disabling interrupts (although this fixed my problem beautifully) causes the system to loose channel data. This can result in e.g. stuttering i.s.o. smooth lamp movement.

Ugh,... I never realized that kernel space was this troublesome.

Offline

#6 2010-09-28 01:52 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

Can't you temporarily disable stop bits?

Alternatively, as a compromise, you could do the pause with udelay and
interrupts enabled, and only disable interrupts when doing the (I assume)
more timing sensitive 35us delay. You should have enough buffering in the
NIC to not lose any packets when doing that.

But if you're doing such high timing sensitive things on such a high load
machine you might be better of with a real-time linux kernel. Or at least
make sure NAPI is enabled for your network driver.

Offline

#7 2010-09-28 02:19 PM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

Regarding the stop bits. I think not... but I have to take a look.

I was reading up on the NAPI thingy you mention. Looks interesting. But I can't find how to enable that stuff. A search for NAPI in menuconfig only gives me results for Tulip family devices...

Offline

#8 2010-09-28 02:40 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

It used to be a per driver option, but now it's always enabled if the driver
supports it. So make sure the NIC driver supports NAPI.

I must say that the whole fixed timing things strikes me as odd, is it baud rate
independent?

Offline

#9 2010-09-28 03:20 PM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

The baud rate is fixed at 250k and no you're right, the DMX512 spec does not define strict timing. But some DMX512 hardware requires this kind of strict values which is what I ran into.

Besides, it is kind of ugly when people monitor the signal and see the timing fluctuate the way it does with my code. And I'm still learning this kernel land stuff as I go, but now I'm really fixated on getting it right. (at least acceptable)

I will try to find out if the driver for the AT91SAM9G20 chip that I use supports NAPI. Perhaps that can lower the amount of interrupts.

I still wonder what causes the timing accuracy to become worse when I enable the high res timers even though the granularity increases up to 1 nanosec...

BTW any pointers on debugging a soft lockup? :D

[edit]
hmm... if it is enabled by default when support is available, I can stop looking for it I guess. It won't change anything anyway.
[/edit]

Offline

#10 2010-09-28 03:36 PM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

can I assume that NAPI is enabled when the packet count of my NIC is way higher than the interrupt count?

I get the packet count from ifconfig and the interrupt count from /proc/interrupts.

Offline

#11 2010-09-28 06:26 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

Generally, yes.

Also, AT91SAM9G20 is handled by arch/arm/mach-at91/at91sam9260*,
and it uses the macb ethernet driver, which has NAPI support.

Enable lockdep, or the hangcheck timer, or better, read your code and figure
out what's wrong that way?

Looking at atmel_serial.*, it seems there is hardware support for doing breaks:
struct tty_operations.break_ctl()
(Nothing about timing though.)

I still wonder what causes the timing accuracy to become worse when I enable the high res timers even though the granularity increases up to 1 nanosec...


Well, did you enable NOHZ too? High res timers are only for timers, nothing
else. Without it, the best granularity you get is HZ.

Besides that, accuracy and granularity are independent of each other. You
can have a very precise 1kHz clock, and a widely inaccurate 1GHz clock.

Offline

#12 2010-09-30 09:25 AM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

I got the timing stable as hell now. I decided to give your bittrain proposal a try and rewrote the break ctrl function of the atmel driver.
So no more timers or locks needed.
This also happened to fix the soft lockup issue.
Thx again for your time. I'd send you a box of belgian chocolates but alas can't mail them through this forum :D

Offline

#13 2010-09-30 02:33 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

Great to hear you got it working! Can you post the diff for future reference?
I'd like to see what you did. Other people might be interested in this too. You
could even push this upstream, you're probably not the only one needing such
timings and this seems a much better way than disabling irqs.

You can mail those delicious chocolates through regular mail, I'm only one
country away. :P

Offline

#14 2010-09-30 03:55 PM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

I hardly doubt that this stuff is acceptable for upstream but anyway, here goes...

I actually hacked the atmel_break_ctl to behave the way I want by passing it the breaktime.
From now on the mar after break is hard linked with the breakt time. I use 9 bits to mimic the break and use the stopbits as mark after break.

so the baudrate = (9 * 1000000)/breaktime

e.g. a breaktime of 180 usecs results in a baudrate of 50000 and a mark after break of 40 usecs.

I set the new baudrate, disable the interrupt marking the end of transmission and start polling the status register myself. After that I restore the interupt mask and the original baudrate. When I return from this back in the line discipline I simply signal the DMA to take it from there.

This way my breaktime is as stable as can be. But I still have some latency on the mark after break, but well within boundaries. So good enough for me, probably not good for anybody else. :D

static void atmel_break_ctl(struct uart_port *port, int break)
{
  struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);

  unsigned long baud, quot, frac_quot;
  unsigned char brtime = break;
  unsigned long orig_irqm = UART_GET_IMR(port);

  unsigned long orig_brgr = UART_GET_BRGR(port);

  baud = (9 * 1000000) / brtime;
  
  quot = (port->uartclk / (16 * baud));
  frac_quot = (((port->uartclk * 8) / (16 * baud))) % 8;

  // set new baudrate that is calculated from the breaktime
  UART_PUT_BRGR(port, (quot | (frac_quot << 16)));
  UART_PUT_IDR(port, ATMEL_US_TXEMPTY | ATMEL_US_ENDTX | ATMEL_US_TXBUFE);

  if(atmel_port->auto_rts_mode)
  {
    UART_PUT_CR(port, ATMEL_US_RTSDIS);
  }

  UART_PUT_CR(port, ATMEL_US_TXEN);

  // write zero to mimic a break, the startbit  with 8 databits 
  // are regarded as the break while the mark after break will 
  // be represented  by the 2 stopbits
  UART_PUT_CHAR(port, 0);

  // poll the endtx status register 
  while(1)
  {
    if(UART_GET_CSR(port) & ATMEL_US_TXEMPTY)
      break;
  }

  UART_PUT_IER(port, orig_irqm);
  // restore baudrate to original
  UART_PUT_BRGR(port, orig_brgr);
}

PS. If you'd really want to I'll send the chocolats to Germany, Netherlands or France. Unless you're from Luxembourg of course! :D

Offline

#15 2010-10-18 10:24 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

Took a while longer than expected, with the server getting rooted and us moving
to different hosting and software, but oh well.

Well, that code doesn't work with multiple users of the terminal, I think it will go
quite wrong then. Then again, it will go wrong either way in that case.

I think I would have gone for disabling stop bits and just sending the raw zeroes
and ones depending on the current baud rate, that way you can always keep
sending the data and the timing is cycle precise. Only downside is that you either
have to add them yourself, or it's impossible to disable stop bits altogether.

wmoors wrote:

PS. If you'd really want to I'll send the chocolats to Germany, Netherlands or France. Unless you're from Luxembourg of course! :D

Thank you very much for your offer, but that would be too kind. I'll cherish the
thought though!

Offline

#16 2010-10-19 09:30 AM

wmoors
Member
Registered: 2010-09-03
Posts: 13

Re: Taking care of timing in a serial line discipline

I improved the stability of the MAB bit btw.
After I sent the byte that mimics the break and MAB I enable the TX empty interrupt and send the start code from the interrupt handler. This way I have less of a chance to be interrupted.

And yes this is not usable for multiple users, but since it is a dedicated solution that doesn't bother me so much. As I said, this stuff is probably not usable for anyone else.

I looked into disabling the stopbits before, but that is not an option because the uart always sends at least 1 stop bit.

BTW. the new look and feel of the forum is gorgeous! Congrats on that.

Last edited by wmoors (2010-10-20 09:27 AM)

Offline

#17 2010-10-21 01:14 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Taking care of timing in a serial line discipline

Yeah, that's a good improvement, and also makes your code less objectionable.

I'm puzzled by your frac_quot thing though, what's that supposed to do? It seems safer to call atmel_set_termios() and let that do what you need, or at least look at what it does, because it seems to do things quite differently... A comment there says that BRGR is 16-bit, so it seems that your whole frac_quot is ignored?

Glab you like the new forum. I'm uncertain about the blackness of the code blocks, but other than that I'm quite happy with it. It certainly is a lot faster than the old one.

(Edit: I meant "objectionable" in the sense of mainline inclusion, not as some kind of insult of the code, incase there was any doubt...)

Last edited by i3839 (2010-10-25 05:05 PM)

Offline

Board footer

Powered by FluxBB