You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-14 - 21:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Fri Aug 14, 2026 12:43:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3583466995547,5cyclictest3675784-21mailstats10:41:041
3583466995547,5cyclictest3675784-21mailstats10:41:041
3583465995132,16cyclictest12650irq/32-eth010:16:010
3583467994940,5cyclictest0-21swapper/211:45:012
358346799475,37cyclictest3745662-21ssh12:36:002
3583465994738,6cyclictest2799158-21cups-browsed12:31:120
3583465994638,4cyclictest3628672-21cut09:25:010
3583467994437,3cyclictest0-21swapper/210:45:552
3583467994437,3cyclictest0-21swapper/210:45:552
3583467994435,5cyclictest1-21systemd11:50:022
3583467994433,7cyclictest420-21avahi-daemon10:06:502
3583467994432,4cyclictest0-21swapper/208:45:562
3583467994425,16cyclictest351ktimers/211:20:572
3583467994424,15cyclictest351ktimers/212:10:562
3583466994435,7cyclictest3636317-21switchtime09:36:081
3583466994435,7cyclictest3636317-21switchtime09:36:081
3583466994435,6cyclictest3660945-21head10:16:081
3583467994336,3cyclictest0-21swapper/209:26:032
3583467994333,7cyclictest2799158-21cups-browsed11:53:372
3583467994333,7cyclictest2799158-21cups-browsed11:53:372
3583466994334,4cyclictest0-21swapper/112:10:551
3583467994232,7cyclictest3662258-21sendmail-msp10:20:012
358346699424,31cyclictest3657323-21ssh10:10:571
3583466994233,5cyclictest3634788-21munin-plugin-st09:35:011
3583466994232,7cyclictest3711009-21latency_hist11:40:021
3583466994225,3cyclictest171rcu_preempt12:06:031
358346699422,34cyclictest3704750-21ssh11:29:461
3583467994132,6cyclictest3683487-21idleruntime-cro10:55:002
3583466994127,11cyclictest0-21swapper/111:45:581
3583466994032,5cyclictest3698706-21ssh11:20:011
3583466994032,5cyclictest3640905-21cron09:45:001
3583466994032,5cyclictest3640905-21cron09:45:001
3583465994034,2cyclictest0-21swapper/011:36:060
3583465994031,6cyclictest421-21dbus-daemon09:46:060
3583465994029,8cyclictest3590835-21unixbench_multi07:31:110
3583465994022,11cyclictest12650irq/32-eth007:26:040
3583467993932,5cyclictest3617978-21kworker/u8:10+xprtiod09:35:022
3583467993931,5cyclictest3625623-21systemd-run09:20:022
3583467993931,4cyclictest0-21swapper/212:21:012
3583466993929,7cyclictest3604454-21sendmail08:20:011
3583465993933,3cyclictest0-21swapper/008:05:020
3583465993930,5cyclictest0-21swapper/011:00:590
358346799383,26cyclictest3663406-21ssh10:20:562
3583467993832,3cyclictest0-21swapper/208:00:012
3583467993827,7cyclictest3698908-21latency_hist11:20:022
3583466993829,5cyclictest3741409-21grep12:30:011
3583466993828,7cyclictest3611969-21sh08:45:021
3583466993827,7cyclictest2799158-21cups-browsed07:40:471
3583466993810,4cyclictest171rcu_preempt11:20:571
3583465993831,4cyclictest0-21swapper/011:55:570
3583465993818,12cyclictest12650irq/32-eth009:55:560
3583467993731,3cyclictest0-21swapper/207:20:012
3583467993728,6cyclictest0-21swapper/207:40:012
3583466993731,3cyclictest0-21swapper/110:51:031
3583466993729,5cyclictest3692655-21systemd-run11:10:011
3583466993729,5cyclictest3644601-21ssh09:50:471
3583466993729,5cyclictest3590973-21systemd-run07:35:011
3583466993729,4cyclictest0-21swapper/112:20:591
358346699371,9cyclictest28-21ksoftirqd/108:25:561
3583465993728,6cyclictest3669382-21needreboot10:30:540
3583465993726,9cyclictest265-21systemd-journal10:57:060
3583465993713,14cyclictest151ktimers/008:50:550
3583467993630,3cyclictest0-21swapper/212:16:072
3583467993629,3cyclictest0-21swapper/211:05:552
3583467993628,5cyclictest3595454-21idleruntime-cro07:50:012
3583467993625,8cyclictest958-21sendmail-mta11:57:312
3583467993620,12cyclictest351ktimers/209:50:582
3583466993630,3cyclictest0-21swapper/110:25:011
3583466993628,5cyclictest3631725-21cron09:30:011
3583466993628,5cyclictest3605957-21cron08:25:011
3583466993627,3cyclictest0-21swapper/112:16:031
358346699362,25cyclictest3742544-21ssh12:30:581
3583466993621,3cyclictest171rcu_preempt08:36:021
358346699360,10cyclictest28-21ksoftirqd/108:56:011
358346599363,28cyclictest3663450-21ssh10:20:580
358346599363,25cyclictest3733441-21ssh12:15:580
3583465993630,4cyclictest3720132-21ssh11:55:020
3583465993630,4cyclictest3720132-21ssh11:55:020
3583465993630,4cyclictest3691156-21timerandwakeup11:06:060
3583465993629,4cyclictest3639301-21sendmail_mailtr09:41:040
3583465993629,4cyclictest3639301-21sendmail_mailtr09:41:040
3583465993625,8cyclictest3694205-21timerandwakeup11:11:060
358346799354,6cyclictest0-21swapper/208:40:562
3583467993528,3cyclictest0-21swapper/208:20:012
3583467993527,5cyclictest3589480-21systemd-run07:30:012
3583467993526,6cyclictest3628665-21latency_hist09:25:012
3583466993529,2cyclictest0-21swapper/109:10:021
3583466993525,6cyclictest3673103-21unixbench_singl10:36:081
3583466993525,6cyclictest2799158-21cups-browsed10:27:121
3583466993525,6cyclictest2799158-21cups-browsed08:45:551
3583466993519,11cyclictest421-21dbus-daemon10:45:551
3583466993519,11cyclictest421-21dbus-daemon10:45:551
358346699350,8cyclictest28-21ksoftirqd/107:20:541
3583465993529,3cyclictest0-21swapper/012:25:570
3583465993524,8cyclictest3623468-21latency09:15:500
3583465993517,10cyclictest12650irq/32-eth009:10:590
3583467993427,5cyclictest3583464-21cyclictest12:05:482
3583467993425,6cyclictest3619378-21head09:06:062
3583467993425,5cyclictest421-21dbus-daemon10:36:002
358346799341,3cyclictest171rcu_preempt08:36:052
358346799341,27cyclictest3703030-21ssh11:25:592
358346699349,8cyclictest28-21ksoftirqd/110:00:561
3583466993428,3cyclictest1003-21mta-sts-daemon11:31:451
3583466993426,5cyclictest3650037-21cron10:00:011
358346699341,29cyclictest3623780-21ssh09:15:591
3583465993426,5cyclictest3637862-21latency_hist09:40:010
3583465993426,5cyclictest3637862-21latency_hist09:40:010
3583465993416,11cyclictest151ktimers/009:51:000
3583467993325,5cyclictest3622505-21cron09:15:012
3583467993325,5cyclictest3596822-21unixbench_multi07:51:062
3583467993325,4cyclictest0-21swapper/212:31:042
358346699339,7cyclictest28-21ksoftirqd/108:10:571
358346699337,9cyclictest28-21ksoftirqd/110:56:001
3583466993327,4cyclictest1002-21mta-sts-daemon12:04:351
3583466993327,3cyclictest0-21swapper/110:10:011
3583466993324,6cyclictest3642189-21sh09:46:011
3583466993324,6cyclictest1003-21mta-sts-daemon08:50:561
3583466993322,7cyclictest421-21dbus-daemon11:42:371
3583465993326,4cyclictest3631758-21ssh09:30:020
3583465993324,6cyclictest3737055-21unixbench_multi12:21:090
3583465993322,9cyclictest0-21swapper/010:06:080
3583465993322,6cyclictest0-21swapper/009:31:000
3583465993320,10cyclictest3677414-21munin-plugin-st10:45:000
3583465993320,10cyclictest3677414-21munin-plugin-st10:45:000
358346799326,20cyclictest3693822-21ssh11:10:562
3583467993226,3cyclictest0-21swapper/210:00:012
3583467993226,3cyclictest0-21swapper/207:45:022
3583467993224,3cyclictest0-21swapper/208:25:572
358346699327,6cyclictest28-21ksoftirqd/109:05:011
3583466993226,3cyclictest0-21swapper/111:55:011
3583466993226,3cyclictest0-21swapper/111:55:011
3583466993225,4cyclictest0-21swapper/107:55:011
3583466993225,3cyclictest3622532-21kworker/u8:2-nfsiod11:05:011
3583466993224,5cyclictest3600484-21fschecks_count08:05:501
358346699322,25cyclictest3721351-21ssh11:56:001
3583465993222,7cyclictest3653179-21latency_hist10:05:020
3583467993124,4cyclictest3677414-21munin-run10:45:002
3583467993124,4cyclictest3677414-21munin-run10:45:002
3583467993123,5cyclictest3685245-21grep10:56:102
3583467993123,5cyclictest3671339-21munin-plugin-st10:35:012
3583467993123,5cyclictest3657854-21unixbench_multi10:11:092
3583467993122,6cyclictest2799158-21cups-browsed08:04:172
358346799312,13cyclictest2799158-21cups-browsed09:36:012
358346799312,13cyclictest2799158-21cups-browsed09:36:012
3583467993116,11cyclictest1000-21mta-sts-daemon10:26:032
358346699313,10cyclictest28-21ksoftirqd/109:10:561
3583466993123,6cyclictest3585440-21sh07:15:481
3583466993123,5cyclictest3693669-21missed_timers11:10:511
3583466993123,5cyclictest3592325-21unixbench_multi07:36:081
3583466993123,4cyclictest0-21swapper/112:40:011
358346699312,23cyclictest2898839-21munin-node07:11:001
3583466993120,8cyclictest1003-21mta-sts-daemon10:31:061
3583466993110,3cyclictest171rcu_preempt07:46:051
358346599314,19cyclictest428-21mta-sts-daemon09:00:570
3583465993124,4cyclictest3680444-21cron10:50:000
3583465993124,4cyclictest3680444-21cron10:50:000
3583465993122,6cyclictest3703550-21unixbench_singl11:26:080
358346599312,22cyclictest151ktimers/008:26:010
3583467993024,3cyclictest0-21swapper/211:40:012
3583467993024,3cyclictest0-21swapper/210:05:012
3583467993022,5cyclictest3739442-21needreboot12:25:542
3583467993022,5cyclictest1-21systemd07:35:022
3583467993018,4cyclictest3595468-21kworker/u8:0-xprtiod09:48:072
3583467993017,6cyclictest171rcu_preempt07:21:012
358346699304,8cyclictest3588909-21ntpq07:26:021
3583466993021,4cyclictest0-21swapper/109:20:591
3583466993021,4cyclictest0-21swapper/107:55:591
3583466993016,3cyclictest171rcu_preempt08:31:021
358346599304,8cyclictest171rcu_preempt12:36:030
358346599304,20cyclictest3666581-21ntpq10:26:050
3583465993025,3cyclictest3712734-21grep11:41:090
3583465993024,4cyclictest3682070-21grep10:51:070
3583465993024,3cyclictest0-21swapper/010:10:500
358346599302,22cyclictest3715196-21ssh11:45:570
3583465993020,8cyclictest3598483-21systemd-run08:00:020
3583465993016,9cyclictest151ktimers/007:11:000
3583467992923,3cyclictest0-21swapper/212:01:012
3583467992921,6cyclictest351ktimers/208:56:002
3583467992920,5cyclictest420-21avahi-daemon11:03:102
3583465992920,6cyclictest3672803-21/usr/sbin/munin10:36:060
3583465992917,6cyclictest3727265-21sh12:05:550
3583465992914,10cyclictest12650irq/32-eth011:15:560
3583467992822,3cyclictest0-21swapper/207:10:012
3583467992820,5cyclictest3638442-21head09:40:452
3583467992820,5cyclictest3638442-21head09:40:452
3583467992820,5cyclictest3583968-21fschecks_count07:10:482
358346699288,8cyclictest28-21ksoftirqd/108:00:491
3583465992820,5cyclictest2799158-21cups-browsed07:36:540
3583465992819,4cyclictest0-21swapper/011:20:490
3583465992818,7cyclictest0-21swapper/008:06:090
3583465992812,12cyclictest428-21mta-sts-daemon07:45:580
3583467992721,2cyclictest0-21swapper/211:30:482
3583467992718,6cyclictest1-21systemd08:10:022
3583465992722,3cyclictest1000-21mta-sts-daemon12:10:500
3583465992720,4cyclictest3587957-21cron07:25:010
3583465992716,8cyclictest1003-21mta-sts-daemon11:30:290
358346599271,21cyclictest3585814-21kworker/0:1+events_power_efficient07:15:570
3583467992615,3cyclictest171rcu_preempt08:21:002
3583465992622,2cyclictest0-21swapper/008:45:020
3583465992620,4cyclictest3601914-21cpuspeed08:10:470
3583465992620,3cyclictest0-21swapper/012:05:010
3583465992620,3cyclictest0-21swapper/009:05:560
3583465992616,6cyclictest0-21swapper/007:40:580
3583467992518,4cyclictest0-21swapper/208:10:572
358346599257,12cyclictest151ktimers/008:21:040
3583465992520,3cyclictest421-21dbus-daemon07:51:030
358346599251,6cyclictest171rcu_preempt08:15:590
3583467992412,6cyclictest0-21swapper/209:05:022
3583467992412,5cyclictest2799158-21cups-browsed08:50:322
358346599242,16cyclictest3615839-21ntpq08:56:010
358346799238,11cyclictest351ktimers/208:31:002
3583466992310,3cyclictest171rcu_preempt07:10:011
358346599231,16cyclictest3612919-21ntpq08:46:000
358346599227,11cyclictest3607940-21df08:30:470
3583465992216,3cyclictest1001-21mta-sts-daemon08:38:470
3583465992111,7cyclictest265-21systemd-journal07:10:010
358346899171,11cyclictest0-21swapper/311:20:513
358346899161,7cyclictest0-21swapper/308:56:013
358346899152,8cyclictest0-21swapper/309:45:513
358346899152,6cyclictest0-21swapper/309:20:483
358346899151,8cyclictest0-21swapper/311:30:563
358346899151,7cyclictest0-21swapper/312:10:563
358346899151,7cyclictest0-21swapper/311:40:553
358346899151,7cyclictest0-21swapper/310:40:593
358346899151,7cyclictest0-21swapper/310:40:593
358346899151,7cyclictest0-21swapper/310:10:563
358346899151,7cyclictest0-21swapper/309:10:593
358346899151,7cyclictest0-21swapper/307:25:573
358346899151,6cyclictest0-21swapper/309:00:593
358346899143,6cyclictest0-21swapper/309:51:083
358346899143,5cyclictest0-21swapper/308:35:583
358346899142,8cyclictest0-21swapper/310:45:553
358346899142,8cyclictest0-21swapper/310:45:553
358346899142,8cyclictest0-21swapper/310:00:583
358346899142,7cyclictest0-21swapper/312:01:013
358346899142,7cyclictest0-21swapper/310:06:023
358346899142,6cyclictest0-21swapper/310:25:463
358346899142,6cyclictest0-21swapper/308:20:013
358346899142,4cyclictest0-21swapper/308:50:543
358346899141,8cyclictest0-21swapper/312:15:563
358346899141,8cyclictest0-21swapper/311:55:543
358346899141,8cyclictest0-21swapper/311:10:543
358346899141,8cyclictest0-21swapper/311:00:493
358346899141,8cyclictest0-21swapper/309:30:583
358346899141,7cyclictest0-21swapper/311:45:593
358346899141,7cyclictest0-21swapper/311:05:563
358346899141,6cyclictest0-21swapper/307:36:013
358346899135,6cyclictest0-21swapper/310:15:543
358346899132,5cyclictest0-21swapper/308:00:573
358346899131,7cyclictest0-21swapper/312:25:453
358346899131,7cyclictest0-21swapper/310:21:033
358346899131,7cyclictest0-21swapper/309:56:003
358346899131,7cyclictest0-21swapper/309:15:553
358346899131,6cyclictest0-21swapper/308:40:553
358346899131,6cyclictest0-21swapper/308:10:583
358346899131,6cyclictest0-21swapper/307:51:003
358346899131,6cyclictest0-21swapper/307:30:583
358346899131,6cyclictest0-21swapper/307:15:563
358346899125,5cyclictest0-21swapper/312:21:013
358346899123,6cyclictest0-21swapper/311:26:023
358346899123,5cyclictest0-21swapper/312:05:593
358346899123,4cyclictest0-21swapper/307:20:583
358346899122,7cyclictest0-21swapper/312:35:513
358346899122,7cyclictest0-21swapper/310:55:013
358346899121,8cyclictest0-21swapper/310:30:543
358346899121,7cyclictest0-21swapper/312:31:083
358346899121,7cyclictest0-21swapper/311:50:553
358346899121,7cyclictest0-21swapper/311:50:553
358346899121,7cyclictest0-21swapper/310:55:583
358346899121,7cyclictest0-21swapper/309:35:463
358346899121,7cyclictest0-21swapper/309:35:463
358346899121,7cyclictest0-21swapper/308:10:013
358346899121,6cyclictest0-21swapper/308:00:013
358346899120,6cyclictest0-21swapper/308:45:543
358346899113,6cyclictest0-21swapper/311:35:503
358346899113,6cyclictest0-21swapper/309:40:463
358346899113,6cyclictest0-21swapper/309:40:463
358346899112,7cyclictest0-21swapper/309:25:523
358346899111,6cyclictest0-21swapper/309:05:543
358346899111,6cyclictest0-21swapper/308:30:563
358346899111,5cyclictest0-21swapper/310:35:063
358346899111,5cyclictest0-21swapper/308:25:563
358346899110,6cyclictest0-21swapper/308:21:053
358346899102,6cyclictest0-21swapper/311:15:543
358346899101,6cyclictest0-21swapper/307:11:023
358346899101,5cyclictest0-21swapper/307:45:553
358346899101,5cyclictest0-21swapper/307:40:473
358346899101,5cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional