You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 19:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Aug 13, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
908600994844,3cyclictest1065188-21apt-get09:40:133
908589994743,3cyclictest1023402-21apt-get09:00:120
908600993935,2cyclictest934839-21kworker/u8:2+events_unbound07:45:003
908589993932,5cyclictest1179762-21diskstats11:30:130
908600993734,2cyclictest1147541-21kworker/u8:4+events_unbound11:05:203
908600993734,2cyclictest1023163-21kworker/u8:3+flush-259:009:10:163
908600993733,3cyclictest1023163-21kworker/u8:3+flush-259:009:05:233
908597993730,5cyclictest1174586-21diskstats11:25:152
908589993729,5cyclictest335557-21systemd-journal07:10:020
908595993633,2cyclictest1049435-21kworker/u8:0+events_unbound09:55:181
908589993629,5cyclictest924104-21http07:25:010
908600993532,2cyclictest1174198-21kworker/u8:1+flush-259:011:40:013
908600993530,3cyclictest966512-21apt-get08:05:093
908600993430,3cyclictest1023163-21kworker/u8:3+flush-259:009:20:153
908600993428,4cyclictest955475-21latency_hist07:55:003
908597993428,4cyclictest1028822-21cat09:05:012
908595993430,3cyclictest1184556-21kworker/u8:3+flush-259:012:20:211
908589993427,5cyclictest1049394-21latency_hist09:25:000
908600993330,2cyclictest1147541-21kworker/u8:4+flush-259:011:25:253
908595993328,3cyclictest987241-21apt-get08:25:111
908589993327,4cyclictest1158033-21cp11:10:010
908600993228,3cyclictest908591-21kworker/u8:3+flush-259:007:15:133
908597993227,3cyclictest1106523-21apt-get10:20:122
908595993229,2cyclictest950322-21kworker/u8:1+flush-259:008:35:181
908589993226,4cyclictest935057-21/usr/sbin/munin07:35:140
908600993128,2cyclictest1184556-21kworker/u8:3+flush-259:012:35:203
908600993128,2cyclictest1043945-21kworker/u8:1+events_unbound10:10:233
908589993123,6cyclictest1158481-21cp11:10:010
908600993027,2cyclictest1147541-21kworker/u8:4+events_unbound11:15:003
908595993027,2cyclictest924121-21kworker/u8:1+flush-259:007:30:181
908595993027,2cyclictest1070077-21kworker/u8:3+events_unbound10:05:261
908595993026,3cyclictest908591-21kworker/u8:3+flush-259:007:20:211
908589993023,4cyclictest1106257-21latency_hist10:20:010
908600992925,3cyclictest1023163-21kworker/u8:3+events_unbound09:30:123
908597992925,3cyclictest1147541-21kworker/u8:4+flush-259:011:20:012
908597992924,3cyclictest335557-21systemd-journal08:30:232
908595992927,2cyclictest971632-21kworker/u8:3+events_unbound08:13:221
908600992825,3cyclictest908591-21kworker/u8:3+flush-259:007:20:243
908600992825,2cyclictest987002-21kworker/u8:2+events_unbound08:40:213
908600992825,2cyclictest1236897-21kworker/u8:2+flush-259:012:28:493
908600992824,3cyclictest1043945-21kworker/u8:1+events_unbound09:38:513
908597992822,4cyclictest1147541-21kworker/u8:4+flush-259:011:25:012
908595992824,3cyclictest976837-21kworker/u8:4+flush-259:008:15:441
908589992822,4cyclictest950732-21diskstats07:50:140
908589992821,5cyclictest1173269-21ntpq11:20:210
90858999280,26cyclictest1200553-21true11:50:010
908600992724,2cyclictest987002-21kworker/u8:2+flush-259:009:00:293
908600992724,2cyclictest1147541-21kworker/u8:4+events_unbound11:20:003
908597992722,3cyclictest1231724-21/usr/sbin/munin12:20:212
908597992721,4cyclictest1002701-21/usr/sbin/munin08:40:222
908595992724,2cyclictest944824-21kworker/u8:0+events_unbound07:54:421
908595992724,2cyclictest1216043-21kworker/u8:0+flush-259:012:05:011
908595992724,2cyclictest1147541-21kworker/u8:4+flush-259:011:35:241
908595992723,3cyclictest950322-21kworker/u8:1+events_unbound08:51:511
908595992723,3cyclictest950322-21kworker/u8:1+events_unbound08:24:441
908595992722,4cyclictest1142575-21/usr/sbin/munin10:55:141
908595992722,4cyclictest1132293-21apt-get10:45:111
908595992720,5cyclictest1199642-21ntpq11:45:191
908595992720,5cyclictest1163415-21latency_hist11:15:001
908600992623,2cyclictest971632-21kworker/u8:3+flush-259:008:10:253
908600992623,2cyclictest1174198-21kworker/u8:1+flush-259:012:10:013
908595992623,2cyclictest950322-21kworker/u8:1+flush-259:008:10:001
908595992623,2cyclictest1147541-21kworker/u8:4+flush-259:011:00:261
908595992623,2cyclictest1070077-21kworker/u8:3+events_unbound10:00:131
90858999262,22cyclictest1194980-21munin-run11:45:010
908589992620,4cyclictest1247333-21apt-get12:35:130
908589992620,4cyclictest1211103-21/usr/sbin/munin12:00:150
908589992620,4cyclictest1179344-21latency_hist11:30:010
908600992522,2cyclictest1095512-21kworker/u8:2+events_unbound10:15:203
908600992521,3cyclictest1189993-21/usr/sbin/munin11:40:153
908597992521,3cyclictest950322-21kworker/u8:1+flush-259:008:05:102
908597992521,2cyclictest1205918-21kworker/u8:2+events_unbound11:55:182
908597992518,5cyclictest1251418-21ntpq12:35:222
908595992522,2cyclictest1147541-21kworker/u8:4+events_unbound11:41:511
908595992522,2cyclictest1095512-21kworker/u8:2+flush-259:010:24:591
908595992522,2cyclictest1012862-21kworker/u8:0+events_unbound08:59:491
908595992521,3cyclictest1226615-21apt-get12:15:131
908595992521,3cyclictest1023163-21kworker/u8:3+events_unbound09:22:511
908600992421,2cyclictest950322-21kworker/u8:1+flush-259:008:00:273
908597992419,3cyclictest924313-21store07:25:012
908595992421,2cyclictest1054832-21apt-get09:30:101
908595992421,2cyclictest1023163-21kworker/u8:3+flush-259:009:10:231
908595992421,2cyclictest1012862-21kworker/u8:0+flush-259:009:05:201
908595992420,3cyclictest960650-21latency_hist08:00:011
908595992420,3cyclictest1095512-21kworker/u8:2+events_unbound10:15:161
908589992420,3cyclictest944824-21kworker/u8:0+flush-259:007:55:200
908600992321,2cyclictest950322-21kworker/u8:1+events_unbound08:35:263
908600992320,2cyclictest982072-21apt-get08:20:123
908600992320,2cyclictest950322-21kworker/u8:1+flush-259:008:50:173
908600992320,2cyclictest950322-21kworker/u8:1+flush-259:008:30:173
908600992320,2cyclictest924121-21kworker/u8:1+flush-259:007:25:213
908600992320,2cyclictest1049435-21kworker/u8:0+events_unbound09:55:013
908600992320,2cyclictest1043945-21kworker/u8:1+events_unbound10:40:223
908600992320,2cyclictest1043945-21kworker/u8:1+events_unbound10:40:213
908600992320,2cyclictest1043945-21kworker/u8:1+events_unbound10:30:193
908597992319,3cyclictest1043945-21kworker/u8:1+flush-259:009:25:012
908595992320,2cyclictest1147541-21kworker/u8:4+events_unbound11:15:211
908595992320,2cyclictest1043945-21kworker/u8:1+flush-259:009:40:271
908595992320,2cyclictest1002087-21kworker/u8:0+events_unbound08:40:141
908595992319,3cyclictest1095512-21kworker/u8:2+events_unbound10:25:231
908589992320,2cyclictest1085807-21apt-get10:00:120
908589992319,3cyclictest1095512-21kworker/u8:2+events_unbound10:10:280
908589992316,5cyclictest1246140-21ntpq12:30:180
908600992220,2cyclictest1095512-21kworker/u8:2+flush-259:010:45:253
908600992219,2cyclictest1205918-21kworker/u8:2+events_unbound11:55:103
908600992217,4cyclictest1012862-21kworker/u8:0+flush-259:008:55:163
908597992215,5cyclictest1162515-21ntpq11:10:202
908595992218,3cyclictest1147541-21kworker/u8:4+flush-259:011:20:211
908589992217,4cyclictest1043945-21kworker/u8:1+flush-259:010:50:250
908589992216,4cyclictest1115695-21ntpq10:25:210
908589992213,7cyclictest335557-21systemd-journal08:50:010
908600992119,2cyclictest950322-21kworker/u8:1+events_unbound08:45:083
908600992119,1cyclictest1043945-21kworker/u8:1+events_unbound10:50:273
908600992118,2cyclictest1147541-21kworker/u8:4+events_unbound11:20:193
908600992118,2cyclictest1043945-21kworker/u8:1+events_unbound09:45:073
908600992117,3cyclictest1174198-21kworker/u8:1+flush-259:012:15:263
908600992117,3cyclictest1070077-21kworker/u8:3+flush-259:010:05:213
908595992118,2cyclictest1095512-21kworker/u8:2+flush-259:010:35:101
908595992118,2cyclictest1023163-21kworker/u8:3+events_unbound09:15:241
908595992117,3cyclictest950322-21kworker/u8:1+events_unbound08:45:101
908595992117,3cyclictest1070077-21kworker/u8:3+events_unbound09:50:511
908595992116,3cyclictest1070292-21/usr/sbin/munin09:45:141
908595992115,5cyclictest913750-21latency_hist07:15:011
908589992118,2cyclictest950322-21kworker/u8:1+flush-259:008:50:210
908589992116,3cyclictest335557-21systemd-journal10:09:480
908600992017,3cyclictest1132069-21kworker/u8:3+events_unbound10:56:503
908600992017,2cyclictest908591-21kworker/u8:3+events_unbound07:35:173
908600992016,3cyclictest945415-21apt-get07:45:123
908600992016,3cyclictest1607760-21fwupd11:31:493
908597992015,4cyclictest335557-21systemd-journal08:55:502
908597992013,4cyclictest1184524-21latency_hist11:35:012
908597992013,4cyclictest1095877-21latency_hist10:10:012
908595992016,3cyclictest1184556-21kworker/u8:3+flush-259:012:30:001
908595992016,3cyclictest1060034-21apt-get09:35:121
908589992017,2cyclictest1080362-21idleruntime-cro09:54:590
908589992013,5cyclictest509-21rs:main0
908600991915,3cyclictest950322-21kworker/u8:1+events_unbound07:58:423
908600991914,4cyclictest1195543-21cat11:45:013
908597991916,2cyclictest1197923-21kthreadcore11:45:162
908597991915,3cyclictest1194977-21kworker/u8:0+flush-259:011:55:012
908597991912,4cyclictest1152691-21latency_hist11:05:002
908595991916,2cyclictest903119-21kworker/u8:0+events_unbound07:19:031
908595991915,3cyclictest1137445-21apt-get10:50:121
908595991914,3cyclictest335557-21systemd-journal10:12:491
908589991915,3cyclictest950322-21kworker/u8:1+flush-259:008:05:220
908589991915,3cyclictest1184556-21kworker/u8:3+flush-259:012:05:210
908589991913,4cyclictest981804-21latency_hist08:20:000
908600991814,3cyclictest1095512-21kworker/u8:2+events_unbound10:35:223
908597991815,2cyclictest934839-21kworker/u8:2+flush-259:007:40:212
908595991816,2cyclictest1023163-21kworker/u8:3+events_unbound09:25:131
908595991815,2cyclictest976837-21kworker/u8:4+flush-259:008:30:211
908595991815,2cyclictest1174198-21kworker/u8:1+events_unbound11:34:501
908595991814,3cyclictest924121-21kworker/u8:1+events_unbound07:26:431
908595991814,3cyclictest908591-21kworker/u8:3+events_unbound07:35:111
908595991814,3cyclictest1184556-21kworker/u8:3+flush-259:012:10:161
908595991814,3cyclictest1043945-21kworker/u8:1+events_unbound10:40:161
908595991814,3cyclictest1043945-21kworker/u8:1+events_unbound10:40:161
908589991814,3cyclictest950322-21kworker/u8:1+flush-259:008:01:110
908589991814,2cyclictest1070077-21kworker/u8:3+flush-259:009:45:180
908589991812,5cyclictest335557-21systemd-journal08:37:500
908600991714,3cyclictest1095512-21kworker/u8:2+events_unbound10:25:323
908600991714,2cyclictest1049435-21kworker/u8:0+flush-259:009:55:203
908600991713,3cyclictest1184556-21kworker/u8:3+events_unbound11:50:033
908600991711,4cyclictest1039305-21/usr/sbin/munin09:15:133
908597991713,3cyclictest1070077-21kworker/u8:3+flush-259:009:50:212
908597991713,3cyclictest1043945-21kworker/u8:1+flush-259:010:20:002
908595991714,2cyclictest987002-21kworker/u8:2+flush-259:009:00:201
908595991713,3cyclictest1043945-21kworker/u8:1+events_unbound10:30:231
908589991714,2cyclictest61150irq/133-enp0s29f1:tx-012:25:190
908589991714,2cyclictest0-21swapper/010:45:000
908589991714,2cyclictest0-21swapper/010:44:590
908589991713,3cyclictest1049435-21kworker/u8:0+flush-259:009:35:150
908589991712,4cyclictest908591-21kworker/u8:3+flush-259:007:45:150
908589991712,3cyclictest1043945-21kworker/u8:1+flush-259:009:35:010
908589991710,5cyclictest335557-21systemd-journal08:10:420
908600991614,2cyclictest976837-21kworker/u8:4+flush-259:008:15:113
908600991613,2cyclictest875713-21kworker/u8:2+flush-259:007:10:173
908600991613,2cyclictest1184556-21kworker/u8:3+events_unbound12:10:263
908600991613,2cyclictest1095512-21kworker/u8:2+events_unbound10:20:213
908597991612,3cyclictest1028353-21systemd-run09:05:002
908595991614,2cyclictest1174198-21kworker/u8:1+events_unbound12:30:261
908595991613,2cyclictest1147541-21kworker/u8:4+flush-259:011:25:201
908595991612,3cyclictest335557-21systemd-journal11:05:291
908595991612,3cyclictest1205918-21kworker/u8:2+events_unbound11:59:521
908595991612,3cyclictest1184556-21kworker/u8:3+events_unbound11:51:421
90858999169,5cyclictest1084678-21ntpq09:55:210
908589991613,2cyclictest60350irq/132-enp0s29f1:rx-011:15:140
908589991613,2cyclictest60350irq/132-enp0s29f1:rx-011:00:000
908589991612,3cyclictest908591-21kworker/u8:3+flush-259:007:30:250
908589991612,3cyclictest1043945-21kworker/u8:1+flush-259:010:30:220
908589991611,4cyclictest1200798-21apt-get11:50:110
908589991610,4cyclictest996396-21ntpq08:30:200
908600991511,3cyclictest950322-21kworker/u8:1+flush-259:008:25:533
908597991512,2cyclictest0-21swapper/208:35:182
908597991512,1cyclictest0-21swapper/209:15:002
908597991511,3cyclictest1192325-21tr11:40:182
908595991512,2cyclictest1205918-21kworker/u8:2+events_unbound12:05:171
908589991514,1cyclictest60350irq/132-enp0s29f1:rx-007:19:010
908589991513,2cyclictest60350irq/132-enp0s29f1:rx-011:55:490
908589991513,1cyclictest60350irq/132-enp0s29f1:rx-009:15:060
908589991512,2cyclictest60350irq/132-enp0s29f1:rx-010:35:130
908589991511,3cyclictest1043945-21kworker/u8:1+flush-259:009:30:000
908589991510,3cyclictest335557-21systemd-journal11:37:490
908589991510,3cyclictest11325802sleep010:45:150
908600991412,2cyclictest1043945-21kworker/u8:1+events_unbound09:27:583
908600991411,2cyclictest1243151-21tr12:30:163
908600991410,3cyclictest1184556-21kworker/u8:3+events_unbound12:22:243
90859799149,4cyclictest1095512-21kworker/u8:2+flush-259:010:35:462
908597991411,1cyclictest0-21swapper/211:00:002
908597991410,3cyclictest950322-21kworker/u8:1+flush-259:008:45:232
908597991410,3cyclictest1184556-21kworker/u8:3+flush-259:012:10:182
908597991410,3cyclictest1096141-21apt-get10:10:122
908589991412,2cyclictest60350irq/132-enp0s29f1:rx-012:10:170
908589991412,1cyclictest60350irq/132-enp0s29f1:rx-008:25:520
908589991411,2cyclictest60350irq/132-enp0s29f1:rx-011:00:120
908589991411,2cyclictest60350irq/132-enp0s29f1:rx-009:42:240
908589991411,2cyclictest60350irq/132-enp0s29f1:rx-007:25:240
908589991410,2cyclictest60350irq/132-enp0s29f1:rx-010:20:130
908600991310,1cyclictest0-21swapper/312:05:013
90859799139,3cyclictest908591-21kworker/u8:3+flush-259:007:35:252
90859799139,3cyclictest1049435-21kworker/u8:0+flush-259:009:35:222
90859799137,4cyclictest960834-21cp08:00:012
908597991310,2cyclictest971632-21kworker/u8:3+flush-259:008:10:242
908595991310,2cyclictest944824-21kworker/u8:0+events_unbound07:48:321
908589991312,1cyclictest60350irq/132-enp0s29f1:rx-009:09:550
908589991311,1cyclictest60350irq/132-enp0s29f1:rx-008:55:230
908589991311,1cyclictest60350irq/132-enp0s29f1:rx-008:40:120
908589991311,1cyclictest60350irq/132-enp0s29f1:rx-007:43:070
908589991310,2cyclictest60350irq/132-enp0s29f1:rx-012:20:160
90859799129,2cyclictest981397-21unixbench_singl08:15:252
90859799129,2cyclictest0-21swapper/210:05:002
90859799129,1cyclictest0-21swapper/212:30:122
90859799128,3cyclictest1095512-21kworker/u8:2+flush-259:010:45:002
90859799128,3cyclictest1095512-21kworker/u8:2+flush-259:010:45:002
908597991210,1cyclictest1039474-21tr09:15:122
90859599129,3cyclictest950322-21kworker/u8:1+events_unbound08:03:491
90858999129,2cyclictest60350irq/132-enp0s29f1:rx-009:10:210
90858999129,2cyclictest60350irq/132-enp0s29f1:rx-008:20:250
90860099118,2cyclictest924121-21kworker/u8:1+events_unbound07:33:273
90859799119,1cyclictest0-21swapper/211:40:012
90859799118,2cyclictest0-21swapper/208:55:012
90859599118,2cyclictest908591-21kworker/u8:3+events_unbound07:40:111
9085979996,2cyclictest914015-21apt07:15:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional