You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 23:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sun Sep 20, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
737924995753,3cyclictest1065524-21apt-get12:25:120
737927995551,3cyclictest919813-21kworker/u8:2+flush-259:010:20:001
737927995047,2cyclictest930126-21kworker/u8:3+flush-259:012:25:241
737927994640,4cyclictest826091-21apt-get08:35:121
737927994539,4cyclictest888542-21base6409:35:011
737927994539,4cyclictest867727-21latency_hist09:15:011
737932994441,2cyclictest805112-21kworker/u8:3+flush-259:008:30:272
737932994337,4cyclictest935918-21cp10:20:012
737927994239,2cyclictest857376-21kworker/u8:4+flush-259:009:25:241
737932994035,4cyclictest941125-21apt-get10:25:122
737932993834,3cyclictest888164-21kworker/u8:1+flush-259:009:50:112
737932993834,3cyclictest1065276-21kworker/u8:4+flush-259:012:29:592
737924993831,5cyclictest764012-21apt-get07:35:130
737924993831,4cyclictest1060072-21latency_hist12:20:010
737936993732,4cyclictest1029077-21kworker/u8:2+flush-259:012:00:243
737936993631,4cyclictest930126-21kworker/u8:3+flush-259:011:40:003
737924993531,2cyclictest920082-21apt-get10:05:110
737932993330,2cyclictest794841-21kworker/u8:0+flush-259:008:44:292
737936993226,4cyclictest722512-21kworker/u8:3+flush-259:007:15:173
737932993229,2cyclictest857376-21kworker/u8:4+flush-259:009:25:262
737932993228,3cyclictest794841-21kworker/u8:0+flush-259:008:15:242
737932993227,3cyclictest810558-21apt-get08:20:112
737932993127,3cyclictest704986-21kworker/u8:2+events_unbound07:20:112
737932993125,4cyclictest992898-21latency_hist11:15:012
737927993127,3cyclictest1029077-21kworker/u8:2+flush-259:012:00:111
737927993126,4cyclictest800202-21apt-get08:10:131
737932993027,2cyclictest930126-21kworker/u8:3+flush-259:010:50:252
737932993027,2cyclictest930126-21kworker/u8:3+flush-259:010:50:252
737932993026,3cyclictest794841-21kworker/u8:0+events_unbound11:00:012
737932993025,3cyclictest335557-21systemd-journal09:49:242
737927993027,2cyclictest930126-21kworker/u8:3+events_unbound11:55:011
737927993027,2cyclictest794841-21kworker/u8:0+events_unbound08:20:001
737927993026,3cyclictest1018985-21apt-get11:40:121
737927993025,4cyclictest893941-21latency_hist09:40:011
737936992925,3cyclictest889078-21apt-get09:35:143
737932992925,3cyclictest773866-21kworker/u8:0+events_unbound07:50:242
737927992926,2cyclictest857376-21kworker/u8:4+flush-259:009:40:161
737936992823,3cyclictest753393-21kworker/u8:1+flush-259:008:00:213
737932992825,2cyclictest956378-21kworker/u8:1+events_unbound11:25:282
737932992825,2cyclictest758778-21kworker/u8:4+events_unbound07:40:232
737927992825,2cyclictest2873070-21kworker/u8:0+events_unbound07:10:151
737927992724,2cyclictest704986-21kworker/u8:2+flush-259:007:20:211
737924992720,5cyclictest794806-21latency_hist08:05:010
737936992622,2cyclictest805097-21turbostat08:15:003
737932992623,2cyclictest722512-21kworker/u8:3+events_unbound07:55:192
737927992623,2cyclictest919813-21kworker/u8:2+flush-259:010:05:201
737927992623,2cyclictest898976-21kworker/u8:2+flush-259:009:45:221
737927992622,3cyclictest335557-21systemd-journal07:48:241
737924992622,3cyclictest841418-21kworker/u8:1+flush-259:009:05:000
737932992520,4cyclictest794841-21kworker/u8:0+flush-259:010:35:202
737932992520,3cyclictest986774-21ntpq11:05:212
737927992522,2cyclictest753393-21kworker/u8:1+events_unbound07:55:251
737927992522,2cyclictest1049994-21kworker/u8:2+flush-259:012:10:211
737927992520,4cyclictest335557-21systemd-journal10:55:001
737927992520,4cyclictest335557-21systemd-journal10:55:001
737932992421,2cyclictest930126-21kworker/u8:3+flush-259:011:55:162
737932992421,2cyclictest930126-21kworker/u8:3+events_unbound12:07:252
737932992421,2cyclictest857376-21kworker/u8:4+flush-259:009:40:252
737932992421,2cyclictest794841-21kworker/u8:0+events_unbound10:04:462
737932992420,3cyclictest794841-21kworker/u8:0+events_unbound10:40:252
737932992420,3cyclictest794841-21kworker/u8:0+events_unbound08:25:192
737927992421,3cyclictest930126-21kworker/u8:3+events_unbound11:30:291
737927992421,2cyclictest1013562-21kworker/u8:2+events_unbound11:35:181
737936992318,3cyclictest773866-21kworker/u8:0+flush-259:007:45:003
737932992320,2cyclictest758778-21kworker/u8:4+flush-259:008:10:152
737932992319,3cyclictest930126-21kworker/u8:3+events_unbound12:10:072
737932992319,3cyclictest930126-21kworker/u8:3+events_unbound11:15:272
737932992319,3cyclictest888164-21kworker/u8:1+flush-259:009:35:272
737932992319,3cyclictest805112-21kworker/u8:3+flush-259:008:50:162
737932992319,3cyclictest794841-21kworker/u8:0+events_unbound08:35:212
737927992321,2cyclictest794841-21kworker/u8:0+events_unbound10:49:251
737927992320,2cyclictest805112-21kworker/u8:3+flush-259:008:50:211
737927992320,2cyclictest1059776-21kworker/u8:0+events_unbound12:39:591
737927992319,3cyclictest977655-21apt-get11:00:121
737927992319,3cyclictest836488-21apt-get08:45:121
737924992317,4cyclictest987706-21latency_hist11:10:010
737936992217,3cyclictest987687-21kworker/u8:0+flush-259:011:35:003
737936992217,3cyclictest857375-21kworker/u8:2+flush-259:009:30:003
737936992216,4cyclictest909739-21apt-get09:55:113
737936992215,5cyclictest762868-21ntpq07:30:223
737932992218,3cyclictest888164-21kworker/u8:1+events_unbound10:06:262
737927992219,2cyclictest987687-21kworker/u8:0+flush-259:012:05:261
737927992219,2cyclictest930126-21kworker/u8:3+events_unbound10:25:241
737927992219,2cyclictest753393-21kworker/u8:1+events_unbound08:00:151
737927992217,3cyclictest335557-21systemd-journal09:09:241
737936992117,3cyclictest758778-21kworker/u8:4+flush-259:008:00:003
737936992117,2cyclictest956378-21kworker/u8:1+flush-259:011:25:193
737936992116,4cyclictest815703-21kworker/u8:2+flush-259:008:34:593
737936992116,3cyclictest335557-21systemd-journal08:29:253
737936992115,4cyclictest862994-21diskstats09:10:133
737932992118,2cyclictest930126-21kworker/u8:3+flush-259:011:40:212
737932992118,2cyclictest758778-21kworker/u8:4+events_unbound08:05:192
737932992118,2cyclictest1065276-21kworker/u8:4+events_unbound12:36:262
737932992117,3cyclictest987687-21kworker/u8:0+events_unbound12:01:252
737932992112,2cyclictest930099-21sh10:15:002
737927992118,3cyclictest946259-21kworker/u8:4+flush-259:010:30:211
737927992118,2cyclictest704986-21kworker/u8:2+events_unbound07:29:261
737927992118,2cyclictest1023652-21kworker/u8:1+events_unbound11:45:201
737927992117,3cyclictest722512-21kworker/u8:3+events_unbound07:40:341
737924992117,3cyclictest904288-21idleruntime-cro09:50:010
737924992117,3cyclictest794841-21kworker/u8:0+flush-259:010:55:180
737924992117,3cyclictest1060345-21apt-get12:20:110
737936992016,3cyclictest930126-21kworker/u8:3+events_unbound11:15:173
737936992014,4cyclictest888164-21kworker/u8:1+flush-259:010:05:003
737936992014,4cyclictest1045018-21diskstats12:05:143
737936992014,4cyclictest1024147-21apt-get11:45:123
737932992017,2cyclictest794841-21kworker/u8:0+flush-259:010:15:172
737932992017,2cyclictest1065275-21kworker/u8:2+flush-259:012:34:352
737932992016,3cyclictest753609-21/usr/sbin/munin07:25:202
737932992016,3cyclictest722512-21kworker/u8:3+events_unbound07:15:182
737927992017,2cyclictest841418-21kworker/u8:1+events_unbound08:55:251
737927992014,4cyclictest335557-21systemd-journal10:10:241
737924992018,1cyclictest784437-21idleruntime-cro07:55:000
737924992016,2cyclictest1-21systemd11:02:240
737924992015,4cyclictest335557-21systemd-journal12:10:000
737932991916,2cyclictest805112-21kworker/u8:3+flush-259:009:00:262
737932991916,2cyclictest2873070-21kworker/u8:0+flush-259:007:10:162
737932991915,3cyclictest857375-21kworker/u8:2+events_unbound09:15:162
737932991915,3cyclictest722512-21kworker/u8:3+events_unbound07:39:152
737927991917,2cyclictest773866-21kworker/u8:0+events_unbound07:50:221
737927991916,2cyclictest794841-21kworker/u8:0+flush-259:009:00:161
737927991916,2cyclictest794841-21kworker/u8:0+events_unbound08:25:241
737927991916,2cyclictest1029077-21kworker/u8:2+events_unbound11:59:201
737927991915,3cyclictest930126-21kworker/u8:3+events_unbound12:15:131
737927991915,3cyclictest888164-21kworker/u8:1+events_unbound09:54:431
737927991915,3cyclictest820917-21apt-get08:30:141
737927991915,3cyclictest1059776-21kworker/u8:0+flush-259:012:20:171
737936991814,3cyclictest930126-21kworker/u8:3+flush-259:011:00:243
737936991814,3cyclictest919813-21kworker/u8:2+flush-259:010:05:243
737936991813,3cyclictest1059776-21kworker/u8:0+flush-259:012:20:163
737932991815,2cyclictest857376-21kworker/u8:4+events_unbound09:20:152
737932991815,2cyclictest857376-21kworker/u8:4+events_unbound09:10:212
737932991814,3cyclictest930126-21kworker/u8:3+events_unbound10:45:132
737932991812,4cyclictest335557-21systemd-journal09:35:012
737927991815,2cyclictest930126-21kworker/u8:3+events_unbound11:05:171
737927991815,2cyclictest758778-21kworker/u8:4+flush-259:008:05:201
73792499185,3cyclictest1017822-21ntpq11:35:200
73792499183,5cyclictest878973-21sed09:25:190
73792499181,6cyclictest852318-21date09:00:010
737924991816,1cyclictest1080790-21cron12:39:590
737924991812,4cyclictest335557-21systemd-journal08:54:590
73793699179,5cyclictest992023-21ntpq11:10:233
737936991713,3cyclictest794841-21kworker/u8:0+flush-259:008:50:003
737936991713,3cyclictest1020006-21kthreadcore11:40:173
737932991715,1cyclictest722512-21kworker/u8:3+events_unbound07:45:132
737932991714,2cyclictest930126-21kworker/u8:3+events_unbound11:30:222
737932991714,2cyclictest837510-21kthreadcore08:45:172
737932991714,2cyclictest805112-21kworker/u8:3+events_unbound08:58:232
737932991714,2cyclictest722512-21kworker/u8:3+flush-259:007:30:212
737932991713,3cyclictest930126-21kworker/u8:3+events_unbound10:30:322
737932991713,3cyclictest857376-21kworker/u8:4+events_unbound09:07:252
737932991713,2cyclictest0-21swapper/210:00:002
737927991714,2cyclictest930126-21kworker/u8:3+flush-259:011:10:211
737927991713,3cyclictest914649-21latency_hist10:00:001
737927991712,3cyclictest335557-21systemd-journal08:42:241
73792499177,1cyclictest1012692-21ntpq11:30:220
73792499173,4cyclictest760270-21cat07:30:170
73792499173,4cyclictest1000719-21cut11:20:160
737924991714,2cyclictest60350irq/132-enp0s29f1:rx-009:30:210
737924991714,2cyclictest60350irq/132-enp0s29f1:rx-009:10:230
737924991713,3cyclictest930126-21kworker/u8:3+flush-259:012:15:000
73793699169,4cyclictest768909-21latency_hist07:39:593
737936991610,4cyclictest857784-21diskstats09:05:143
737936991610,4cyclictest1065275-21kworker/u8:2+flush-259:012:35:173
737932991613,2cyclictest982219-21kworker/u8:2+events_unbound11:04:562
737932991613,2cyclictest930126-21kworker/u8:3+events_unbound11:24:562
737932991613,2cyclictest1049994-21kworker/u8:2+events_unbound12:16:242
737932991612,3cyclictest930126-21kworker/u8:3+flush-259:011:48:352
737927991614,2cyclictest794841-21kworker/u8:0+events_unbound10:37:311
737927991613,2cyclictest956378-21kworker/u8:1+events_unbound11:25:201
737927991613,2cyclictest805112-21kworker/u8:3+events_unbound08:20:201
737927991612,3cyclictest961755-21store10:45:001
737927991612,3cyclictest930126-21kworker/u8:3+events_unbound10:20:151
73792499165,7cyclictest0-21swapper/010:00:180
73792499163,3cyclictest954141-21cut10:35:200
73792499162,4cyclictest873634-21sed09:20:160
73792499161,5cyclictest895223-21kthreadcore09:40:180
73792499161,4cyclictest1008594-21date11:30:010
737924991614,1cyclictest60350irq/132-enp0s29f1:rx-011:50:160
737924991614,1cyclictest60350irq/132-enp0s29f1:rx-011:40:120
737924991614,1cyclictest60350irq/132-enp0s29f1:rx-009:15:200
737924991614,1cyclictest60350irq/132-enp0s29f1:rx-008:35:170
737924991614,1cyclictest60350irq/132-enp0s29f1:rx-007:49:400
737924991614,1cyclictest0-21swapper/010:55:000
737924991614,1cyclictest0-21swapper/010:55:000
737924991613,2cyclictest60350irq/132-enp0s29f1:rx-008:34:060
73792499160,6cyclictest817996-21cat08:25:190
737936991513,1cyclictest0-21swapper/310:45:003
737936991512,2cyclictest899796-21sed09:45:183
737936991510,3cyclictest956378-21kworker/u8:1+flush-259:011:25:003
73793299152,6cyclictest1018718-21munin-run11:40:012
737932991512,2cyclictest758778-21kworker/u8:4+flush-259:008:04:592
737927991513,2cyclictest956378-21kworker/u8:1+events_unbound11:20:231
737927991513,2cyclictest857375-21kworker/u8:2+events_unbound09:19:321
737927991512,2cyclictest1065275-21kworker/u8:2+events_unbound12:33:251
737927991511,3cyclictest857375-21kworker/u8:2+flush-259:009:20:211
73792499153,3cyclictest893064-21ntpq09:35:230
73792499151,5cyclictest964919-21grep10:45:200
73792499151,4cyclictest833776-21grep08:40:200
73792499151,4cyclictest814631-21ntpq08:20:220
737924991514,1cyclictest60350irq/132-enp0s29f1:rx-012:00:180
737924991513,1cyclictest60350irq/132-enp0s29f1:rx-010:25:160
737924991513,1cyclictest60350irq/132-enp0s29f1:rx-008:09:210
737924991512,2cyclictest60350irq/132-enp0s29f1:rx-007:55:190
73792499150,5cyclictest930169-21cat10:15:000
73792499150,5cyclictest907584-21grep09:50:180
73792499150,13cyclictest1-21systemd07:11:240
73793699149,3cyclictest930126-21kworker/u8:3+flush-259:011:55:253
737936991410,3cyclictest1065275-21kworker/u8:2+flush-259:012:30:253
737936991410,2cyclictest0-21swapper/312:10:253
737936991410,2cyclictest0-21swapper/309:40:243
737932991411,2cyclictest1029077-21kworker/u8:2+flush-259:011:52:352
737927991410,3cyclictest722512-21kworker/u8:3+events_unbound07:37:121
73792499146,6cyclictest857768-21tr09:05:120
73792499144,3cyclictest0-21swapper/010:15:200
73792499143,2cyclictest837471-21irqstats08:45:170
73792499143,2cyclictest747273-21ntpq07:15:200
73792499141,4cyclictest937643-21kthreadcore10:20:170
73792499141,2cyclictest992015-21ntpq11:10:220
73792499141,2cyclictest757706-21ntpq07:25:220
737924991412,1cyclictest60350irq/132-enp0s29f1:rx-012:32:350
737924991412,1cyclictest60350irq/132-enp0s29f1:rx-011:48:250
737924991412,1cyclictest60350irq/132-enp0s29f1:rx-010:40:170
73792499140,4cyclictest748617-21diskstats07:20:140
73793699139,3cyclictest977399-21idleruntime-cro11:00:003
73793699139,3cyclictest930126-21kworker/u8:3+flush-259:010:45:173
73793699139,3cyclictest704986-21kworker/u8:2+flush-259:007:20:183
73793699139,2cyclictest0-21swapper/309:35:003
73793699134,1cyclictest8712532kthreadcore09:15:193
737936991310,2cyclictest935283-21apt-get10:20:023
737927991311,2cyclictest758778-21kworker/u8:4+events_unbound07:30:201
737927991310,2cyclictest915334-21irqcore10:00:161
73792499131,7cyclictest769462-21fw_forwarded_lo07:40:140
73792499131,2cyclictest913772-21ntpq09:55:200
737924991312,1cyclictest60350irq/132-enp0s29f1:rx-011:17:430
73793699129,2cyclictest810969-21irqcore08:20:163
73793699126,4cyclictest335557-21systemd-journal09:21:243
73793699125,5cyclictest847117-21cat08:55:013
737936991210,1cyclictest0-21swapper/310:10:173
73793299129,2cyclictest1044610-21kworker/u8:1+events_unbound12:20:372
73792799128,3cyclictest2873070-21kworker/u8:0+events_unbound07:19:451
73792499127,3cyclictest950298-21ntpq10:30:210
73792499122,2cyclictest809396-21ntpq08:15:220
73793699117,2cyclictest0-21swapper/310:35:273
73792799117,3cyclictest930126-21kworker/u8:3+events_unbound11:15:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional