You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 15:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Tue Jun 30, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3371286996258,3cyclictest3361042-21kworker/u8:3+flush-259:007:20:003
3371278995548,5cyclictest3661422-21latency_hist11:50:001
3371281995145,4cyclictest3687705-21latency_hist12:15:002
3371278995042,5cyclictest3666615-21latency_hist11:55:001
3371273995048,1cyclictest1-21systemd08:20:000
3371281994844,3cyclictest3603902-21gpgv10:55:002
3371278994741,4cyclictest335557-21systemd-journal07:39:591
3371278994642,3cyclictest3496014-21kworker/u8:0+events_unbound10:40:141
3371281994540,3cyclictest3651119-21latency_hist11:40:002
3371286994437,5cyclictest3391831-21latency_hist07:30:003
3371281994239,2cyclictest3496014-21kworker/u8:0+flush-259:010:35:152
3371278994239,2cyclictest3688355-21kworker/u8:2+flush-259:012:17:101
3371281994034,4cyclictest3677336-21latency_hist12:05:002
3371278994036,3cyclictest3661661-21kworker/u8:1+events_unbound12:13:111
3371278994036,3cyclictest3651139-21kworker/u8:0+events_unbound12:01:001
3371278994036,3cyclictest3496014-21kworker/u8:0+events_unbound11:03:011
3371278994036,3cyclictest3453938-21kworker/u8:1+events_unbound10:14:041
3371278994034,4cyclictest3412405-21latency_hist07:50:011
3371281993935,3cyclictest3402336-21kworker/u8:4+flush-259:008:35:002
3371278993935,3cyclictest3557516-21kworker/u8:3+events_unbound10:32:001
3371278993935,3cyclictest3210542-21kworker/u8:0+events_unbound07:28:271
3371281993835,2cyclictest3677571-21kworker/u8:4+flush-259:012:30:152
3371278993835,2cyclictest3557516-21kworker/u8:3+events_unbound10:15:241
3371278993834,3cyclictest3496014-21kworker/u8:0+events_unbound10:55:231
3371278993834,3cyclictest3361042-21kworker/u8:3+events_unbound07:33:011
3371278993734,2cyclictest3412636-21kworker/u8:3+events_unbound07:55:241
3371278993734,2cyclictest3402336-21kworker/u8:4+events_unbound07:40:151
3371278993733,3cyclictest3651139-21kworker/u8:0+events_unbound11:59:291
3371278993733,3cyclictest3562920-21kworker/u8:4+events_unbound10:24:071
3371278993731,4cyclictest3464453-21latency_hist08:40:011
3371281993630,4cyclictest3516396-21latency_hist09:29:592
3371278993633,2cyclictest3453938-21kworker/u8:1+events_unbound09:29:001
3371278993632,3cyclictest3333430-21kworker/u8:2+events_unbound07:13:261
3371278993630,4cyclictest3593836-21/usr/sbin/munin10:45:151
3371273993629,5cyclictest3453938-21kworker/u8:1+flush-259:009:35:160
3371281993532,2cyclictest3496014-21kworker/u8:0+flush-259:011:20:002
3371278993532,2cyclictest3629624-21kworker/u8:3+events_unbound11:40:251
3371278993532,2cyclictest3496014-21kworker/u8:0+events_unbound09:10:151
3371278993532,2cyclictest3402336-21kworker/u8:4+events_unbound09:23:521
3371278993531,2cyclictest3496014-21kworker/u8:0+events_unbound09:55:171
3371273993529,4cyclictest3661661-21kworker/u8:1+flush-259:012:10:240
3371286993429,3cyclictest3516637-21/usr/sbin/munin09:30:133
3371278993431,2cyclictest3333430-21kworker/u8:2+events_unbound07:20:001
3371278993430,3cyclictest3698495-21kworker/u8:1+events_unbound12:29:361
3371278993430,3cyclictest3698495-21kworker/u8:1+events_unbound12:29:361
3371278993430,3cyclictest3630683-21kworker/u8:4+events_unbound11:20:231
3371278993430,3cyclictest3496014-21kworker/u8:0+events_unbound10:39:101
3371278993429,3cyclictest3557516-21kworker/u8:3+events_unbound10:27:001
3371273993426,6cyclictest3603873-21latency_hist10:55:010
3371286993329,3cyclictest3588487-21kworker/u8:2+flush-259:011:45:163
3371278993330,2cyclictest3402336-21kworker/u8:4+events_unbound09:05:151
3371278993329,3cyclictest3661661-21kworker/u8:1+events_unbound12:08:001
3371278993329,3cyclictest3402336-21kworker/u8:4+events_unbound08:28:441
3371273993329,3cyclictest3588487-21kworker/u8:2+flush-259:011:45:010
3371281993229,2cyclictest3698706-21kworker/u8:3+flush-259:012:35:152
3371278993229,2cyclictest3630683-21kworker/u8:4+events_unbound11:25:181
3371273993228,3cyclictest3698495-21kworker/u8:1+flush-259:012:25:160
3371273993228,3cyclictest3698495-21kworker/u8:1+flush-259:012:25:160
3371273993228,3cyclictest3402336-21kworker/u8:4+flush-259:009:05:180
3371273993227,4cyclictest3551487-21ntpq10:00:200
3371281993126,4cyclictest3443453-21latency_hist08:20:002
3371278993127,3cyclictest3470792-21kworker/u8:0+events_unbound08:45:221
3371278993127,3cyclictest3210542-21kworker/u8:0+events_unbound08:05:151
3371273993125,4cyclictest3660572-21ntpq11:45:210
3371281993026,3cyclictest3573012-21kworker/u8:1+flush-259:010:25:002
3371281993026,3cyclictest3542126-21kworker/u8:2+flush-259:010:10:172
3371281993024,4cyclictest3532238-21diskstats09:45:132
3371278993027,2cyclictest3598740-21kworker/u8:1+events_unbound11:10:201
3371278993025,2cyclictest3412636-21kworker/u8:3+events_unbound08:30:131
3371273993026,3cyclictest3402336-21kworker/u8:4+flush-259:009:10:170
3371273993023,5cyclictest3682543-21latency_hist12:10:000
3371286992923,4cyclictest3635659-21latency_hist11:25:013
3371281992926,2cyclictest3333430-21kworker/u8:2+flush-259:007:45:252
3371281992925,3cyclictest3629624-21kworker/u8:3+events_unbound11:49:232
3371278992923,4cyclictest3501180-21apt-get09:15:121
3371273992922,5cyclictest3452798-21ntpq08:25:190
3371286992824,3cyclictest3598740-21kworker/u8:1+flush-259:011:10:163
3371286992823,4cyclictest3562920-21kworker/u8:4+flush-259:010:24:063
3371286992822,4cyclictest3526672-21latency_hist09:40:003
3371281992824,3cyclictest3562920-21kworker/u8:4+events_unbound10:19:042
3371281992824,3cyclictest3453938-21kworker/u8:1+events_unbound08:41:022
3371281992723,3cyclictest3677571-21kworker/u8:4+events_unbound12:19:112
3371281992723,3cyclictest3453938-21kworker/u8:1+flush-259:008:56:012
3371281992723,3cyclictest3453938-21kworker/u8:1+events_unbound09:00:212
3371278992724,2cyclictest3588487-21kworker/u8:2+events_unbound11:35:141
3371278992722,3cyclictest3703905-21apt-get12:30:141
3371273992722,4cyclictest3407532-21apt-get07:45:110
3371273992721,4cyclictest3665698-21ntpq11:50:200
3371286992620,4cyclictest3454132-21diskstats08:30:133
3371278992623,2cyclictest3698706-21kworker/u8:3+events_unbound12:37:101
3371273992620,5cyclictest3625549-21apt-get11:15:110
3371273992620,4cyclictest3568091-21/usr/sbin/munin10:20:140
3371286992521,3cyclictest3333430-21kworker/u8:2+flush-259:007:50:003
3371281992522,2cyclictest3402336-21kworker/u8:4+flush-259:008:40:012
3371278992521,3cyclictest3402336-21kworker/u8:4+events_unbound09:00:251
3371278992520,4cyclictest3537217-21apt-get09:50:111
3371286992421,2cyclictest3688354-21kworker/u8:0+events_unbound12:25:003
3371286992421,2cyclictest3688354-21kworker/u8:0+events_unbound12:25:003
3371286992420,3cyclictest3557516-21kworker/u8:3+events_unbound10:44:103
3371286992419,4cyclictest3371491-21apt-get07:10:113
3371281992420,3cyclictest3402336-21kworker/u8:4+events_unbound09:17:012
3371281992419,4cyclictest3453938-21kworker/u8:1+events_unbound10:04:032
3371278992421,2cyclictest3453938-21kworker/u8:1+events_unbound09:30:231
3371278992420,3cyclictest3402336-21kworker/u8:4+flush-259:009:39:011
3371273992417,5cyclictest3371193-21latency_hist07:10:000
3371286992319,3cyclictest3496014-21kworker/u8:0+events_unbound10:49:103
3371286992317,4cyclictest335557-21systemd-journal10:10:003
3371281992319,3cyclictest3629624-21kworker/u8:3+events_unbound11:40:172
3371281992319,3cyclictest3496014-21kworker/u8:0+flush-259:011:05:242
3371281992319,3cyclictest3402336-21kworker/u8:4+events_unbound09:42:012
3371281992318,4cyclictest3453938-21kworker/u8:1+events_unbound09:58:022
3371281992318,4cyclictest3402336-21kworker/u8:4+events_unbound08:21:022
3371278992320,2cyclictest3402336-21kworker/u8:4+events_unbound09:40:261
3371273992318,4cyclictest3412636-21kworker/u8:3+flush-259:008:50:170
3371273992317,4cyclictest3557767-21/usr/sbin/munin10:10:210
3371286992219,2cyclictest3661661-21kworker/u8:1+flush-259:012:00:003
3371286992218,3cyclictest507-21in:imuxsock10:35:003
3371286992218,3cyclictest3453938-21kworker/u8:1+flush-259:008:55:203
3371286992218,3cyclictest3361042-21kworker/u8:3+events_unbound07:35:193
3371286992217,4cyclictest335557-21systemd-journal12:14:553
3371286992216,4cyclictest3496014-21kworker/u8:0+flush-259:011:05:243
3371286992216,4cyclictest3495777-21latency_hist09:10:013
3371286992215,5cyclictest3666798-21cp11:55:003
3371281992219,2cyclictest3496014-21kworker/u8:0+flush-259:010:55:252
3371281992219,2cyclictest3402336-21kworker/u8:4+flush-259:009:05:162
3371281992218,3cyclictest3210542-21kworker/u8:0+events_unbound07:16:012
3371281992217,3cyclictest3667304-21apt-get11:55:122
3371281992216,4cyclictest3598713-21latency_hist10:50:012
3371278992218,3cyclictest3496014-21kworker/u8:0+events_unbound09:48:011
3371278992217,4cyclictest3614825-21apt-get11:05:111
3371273992218,3cyclictest3583577-21apt-get10:35:110
3371273992216,4cyclictest3406370-21ntpq07:40:210
3371273992215,4cyclictest3417561-21latency_hist07:55:010
3371286992118,2cyclictest3677571-21kworker/u8:4+flush-259:012:30:243
3371286992117,3cyclictest3496014-21kworker/u8:0+events_unbound09:55:223
3371286992117,3cyclictest3412636-21kworker/u8:3+events_unbound07:55:223
3371281992117,3cyclictest3210542-21kworker/u8:0+events_unbound07:23:002
3371281992116,4cyclictest3402336-21kworker/u8:4+events_unbound09:38:592
3371281992116,3cyclictest3630683-21kworker/u8:4+events_unbound11:20:252
3371281992115,4cyclictest3552650-21/usr/sbin/munin10:05:212
3371278992118,2cyclictest3402336-21kworker/u8:4+events_unbound08:20:261
3371278992117,3cyclictest3453938-21kworker/u8:1+events_unbound08:57:001
3371278992116,3cyclictest335557-21systemd-journal12:24:111
3371278992116,3cyclictest335557-21systemd-journal12:24:101
337127399211,19cyclictest1-21systemd11:34:000
3371286992018,1cyclictest3496014-21kworker/u8:0+events_unbound09:50:233
3371286992016,3cyclictest3698495-21kworker/u8:1+events_unbound12:25:233
3371286992016,3cyclictest3698495-21kworker/u8:1+events_unbound12:25:223
3371286992016,3cyclictest3402336-21kworker/u8:4+events_unbound09:10:203
3371286992015,3cyclictest335557-21systemd-journal11:36:003
337128699200,18cyclictest3474824-21unin-run08:50:013
3371281992016,3cyclictest3496014-21kworker/u8:0+events_unbound09:50:152
3371281992016,3cyclictest3470792-21kworker/u8:0+events_unbound08:45:232
3371281992016,3cyclictest3453938-21kworker/u8:1+flush-259:009:30:182
3371281992016,3cyclictest3427912-21kworker/u8:2+flush-259:008:25:212
3371281992015,3cyclictest3562920-21kworker/u8:4+events_unbound10:25:132
3371278992018,2cyclictest3453938-21kworker/u8:1+events_unbound08:50:281
3371278992017,2cyclictest3412430-21kworker/u8:1+events_unbound08:00:011
3371278992016,3cyclictest3453938-21kworker/u8:1+events_unbound08:43:451
3371273992015,3cyclictest3376631-21apt-get07:15:120
3371286991916,2cyclictest3396963-21sh07:35:003
3371286991915,3cyclictest3629624-21kworker/u8:3+events_unbound11:40:193
3371286991915,3cyclictest3496014-21kworker/u8:0+flush-259:011:00:003
3371286991915,3cyclictest3412636-21kworker/u8:3+events_unbound08:05:223
3371286991915,3cyclictest3402336-21kworker/u8:4+flush-259:008:35:113
3371281991916,2cyclictest3598740-21kworker/u8:1+events_unbound11:11:012
3371281991916,2cyclictest3210542-21kworker/u8:0+flush-259:008:05:192
3371281991915,3cyclictest3630683-21kworker/u8:4+events_unbound11:25:282
3371281991915,3cyclictest3496014-21kworker/u8:0+flush-259:011:00:232
3371281991915,2cyclictest3583320-21sh10:35:002
3371278991916,2cyclictest3427912-21kworker/u8:2+events_unbound08:18:001
3371278991916,2cyclictest3333430-21kworker/u8:2+events_unbound07:17:041
3371278991915,3cyclictest3412636-21kworker/u8:3+events_unbound07:54:561
3371278991911,5cyclictest3604296-21cat10:55:011
3371273991912,5cyclictest3697655-21ntpq12:20:220
3371273991912,5cyclictest3697655-21ntpq12:20:220
3371286991813,4cyclictest3609662-21/usr/sbin/munin11:00:213
3371281991816,1cyclictest3496014-21kworker/u8:0+events_unbound09:10:142
3371281991814,3cyclictest3333430-21kworker/u8:2+events_unbound07:13:312
3371281991813,4cyclictest3412430-21kworker/u8:1+events_unbound07:50:232
3371278991815,2cyclictest3598740-21kworker/u8:1+events_unbound11:16:411
3371278991815,2cyclictest3427912-21kworker/u8:2+events_unbound08:10:231
3371273991815,2cyclictest335557-21systemd-journal12:00:000
3371273991814,3cyclictest3511371-21latency_hist09:24:590
3371273991813,4cyclictest3464716-21apt-get08:40:100
3371273991812,4cyclictest3473906-21ntpq08:45:210
3371273991811,5cyclictest3707871-21ntpq12:30:220
3371286991714,2cyclictest3688354-21kworker/u8:0+flush-259:012:15:113
3371286991714,2cyclictest3661661-21kworker/u8:1+events_unbound12:05:143
3371286991714,2cyclictest3453938-21kworker/u8:1+flush-259:009:20:213
3371286991713,3cyclictest3412636-21kworker/u8:3+flush-259:008:10:233
3371286991713,3cyclictest3412430-21kworker/u8:1+flush-259:008:00:253
3371286991713,3cyclictest0-21swapper/311:30:003
3371281991715,1cyclictest3210542-21kworker/u8:0+events_unbound07:25:142
3371281991714,2cyclictest3651139-21kworker/u8:0+flush-259:011:50:152
3371281991713,3cyclictest3688355-21kworker/u8:2+flush-259:012:24:402
3371281991713,3cyclictest3688355-21kworker/u8:2+flush-259:012:24:402
3371281991713,2cyclictest3480341-21latency_hist08:55:012
3371278991714,2cyclictest3629624-21kworker/u8:3+events_unbound11:33:501
3371278991713,3cyclictest3453938-21kworker/u8:1+events_unbound10:07:221
3371273991714,2cyclictest60350irq/132-enp0s29f1:rx-007:30:190
3371273991714,2cyclictest60350irq/132-enp0s29f1:rx-007:23:440
3371273991713,3cyclictest60350irq/132-enp0s29f1:rx-011:00:130
3371273991713,2cyclictest0-21swapper/007:30:000
3371273991712,4cyclictest335557-21systemd-journal08:10:000
3371273991710,5cyclictest335557-21systemd-journal12:36:100
3371273991710,4cyclictest3427893-21latency_hist08:05:000
3371286991613,2cyclictest3562920-21kworker/u8:4+flush-259:010:25:213
3371286991612,3cyclictest3470792-21kworker/u8:0+flush-259:008:55:013
3371286991612,3cyclictest3453938-21kworker/u8:1+events_unbound10:00:173
3371286991612,3cyclictest3402336-21kworker/u8:4+flush-259:008:42:063
3371286991611,4cyclictest3624862-21apt-get11:15:013
3371286991610,4cyclictest3531813-21latency_hist09:45:003
337128199169,5cyclictest335557-21systemd-journal12:06:592
3371281991612,3cyclictest3588487-21kworker/u8:2+events_unbound11:30:272
3371281991611,4cyclictest3402336-21kworker/u8:4+flush-259:007:45:002
3371278991613,2cyclictest3542126-21kworker/u8:2+events_unbound10:02:481
3371273991614,1cyclictest60350irq/132-enp0s29f1:rx-010:47:060
3371273991614,1cyclictest60350irq/132-enp0s29f1:rx-010:15:260
3371273991614,1cyclictest60350irq/132-enp0s29f1:rx-009:17:250
3371273991613,2cyclictest60350irq/132-enp0s29f1:rx-011:05:230
3371273991613,2cyclictest60350irq/132-enp0s29f1:rx-009:54:300
3371273991613,2cyclictest3516401-21turbostat09:30:000
3371273991612,3cyclictest507-21in:imuxsock11:00:000
3371286991511,3cyclictest3588487-21kworker/u8:2+events_unbound10:50:173
3371286991511,3cyclictest3210542-21kworker/u8:0+events_unbound07:20:153
3371286991510,3cyclictest335557-21systemd-journal08:23:003
3371281991510,3cyclictest3412636-21kworker/u8:3+flush-259:008:00:162
3371281991510,3cyclictest3412636-21kworker/u8:3+events_unbound08:00:002
337127399159,4cyclictest3577245-21ntpq10:25:220
3371273991513,2cyclictest60350irq/132-enp0s29f1:rx-009:58:520
3371273991513,1cyclictest60350irq/132-enp0s29f1:rx-011:10:380
3371273991513,1cyclictest60350irq/132-enp0s29f1:rx-009:30:140
3371273991513,1cyclictest60350irq/132-enp0s29f1:rx-009:00:240
3371273991512,2cyclictest60350irq/132-enp0s29f1:rx-010:30:220
3371273991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:150
3371273991511,3cyclictest60350irq/132-enp0s29f1:rx-012:00:110
3371273991511,3cyclictest60350irq/132-enp0s29f1:rx-008:55:230
3371273991511,3cyclictest3630683-21kworker/u8:4+flush-259:011:20:240
3371281991410,3cyclictest3427912-21kworker/u8:2+flush-259:008:10:112
337127399149,4cyclictest3588724-21apt-get10:40:110
337127399148,4cyclictest335557-21systemd-journal12:20:000
3371273991412,1cyclictest61150irq/133-enp0s29f1:tx-007:35:210
3371273991412,1cyclictest60350irq/132-enp0s29f1:rx-008:35:150
3371273991411,2cyclictest60350irq/132-enp0s29f1:rx-008:20:120
3371273991411,2cyclictest3371901-21irqcore07:10:160
3371273991410,3cyclictest60350irq/132-enp0s29f1:rx-007:55:210
337128699139,3cyclictest3713927-21gpgv12:40:013
337128699139,3cyclictest3453938-21kworker/u8:1+flush-259:009:25:243
3371286991310,2cyclictest3402336-21kworker/u8:4+events_unbound09:15:223
337128199139,3cyclictest3361042-21kworker/u8:3+events_unbound07:35:162
337128199139,2cyclictest0-21swapper/210:40:172
3371281991310,2cyclictest3506432-21cpuspeed_turbos09:20:132
337127399131,2cyclictest3646245-21apt-get11:35:120
3371273991311,2cyclictest60350irq/132-enp0s29f1:rx-008:14:370
3371273991310,2cyclictest60350irq/132-enp0s29f1:rx-011:25:240
3371273991310,2cyclictest60350irq/132-enp0s29f1:rx-010:08:450
3371273991310,2cyclictest0-21swapper/008:35:010
337128699129,2cyclictest3588487-21kworker/u8:2+flush-259:011:30:203
337128699129,2cyclictest0-21swapper/310:14:593
3371273991210,1cyclictest60350irq/132-enp0s29f1:rx-009:40:170
337128699106,2cyclictest481rcuc/310:15:183
337128199106,3cyclictest3698706-21kworker/u8:3+flush-259:012:27:262
337128199106,3cyclictest3698706-21kworker/u8:3+flush-259:012:27:262
33712869996,2cyclictest17650irq/131-nvme0q408:20:003
33712819990,7cyclictest3396965-21latency_hist07:35:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional