You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 22:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Mon Jun 29, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
177801994945,3cyclictest75-21kswapd012:25:251
177801993934,3cyclictest219163-21apt-get07:50:101
177799993834,3cyclictest442171-21apt-get11:25:020
177799993834,2cyclictest211rcuc/012:15:000
177799993834,2cyclictest211rcuc/012:15:000
177805993730,5cyclictest468691-21cp11:50:002
177805993730,5cyclictest468691-21cp11:50:002
177801993734,2cyclictest188083-21kworker/u8:2+flush-259:008:00:161
177801993731,4cyclictest251088-21latency_hist08:20:001
177799993734,2cyclictest436915-21kworker/u8:0+events_unbound12:15:180
177809993630,4cyclictest266905-21apt-get08:35:123
177805993530,3cyclictest505171-21/usr/sbin/munin12:25:142
177801993530,3cyclictest436915-21kworker/u8:0+flush-259:011:25:001
177809993429,4cyclictest463197-21latency_hist11:45:003
177809993429,3cyclictest380487-21latency_hist10:25:013
177805993428,4cyclictest318176-21awk09:25:002
177801993428,4cyclictest479127-21latency_hist12:00:001
177799993430,3cyclictest354904-21kworker/u8:4+flush-259:010:45:010
177809993327,4cyclictest442771-21cat11:25:013
177809993326,5cyclictest208626-21latency_hist07:40:013
177805993229,2cyclictest75-21kswapd012:30:112
177805993227,3cyclictest359859-21latency_hist10:05:012
177801993227,3cyclictest335557-21systemd-journal12:15:001
177801993227,3cyclictest335557-21systemd-journal12:15:001
177799993227,4cyclictest224038-21apt-get07:55:010
177805993125,4cyclictest335557-21systemd-journal09:35:462
177805993125,4cyclictest240195-40mandb08:09:272
177799993127,3cyclictest335557-21systemd-journal08:15:010
177805993027,2cyclictest271799-21kworker/u8:0+flush-259:008:50:002
177805993024,4cyclictest207728-21ntpq07:35:222
177799993028,1cyclictest1-21systemd12:24:400
177799993025,4cyclictest335557-21systemd-journal08:05:000
177805992926,2cyclictest344614-21apt-get09:50:122
177805992922,5cyclictest380906-21diskstats10:25:142
177799992925,3cyclictest224689-21kworker/u8:3+flush-259:009:00:250
177805992824,3cyclictest375318-21latency_hist10:20:012
177805992823,4cyclictest335557-21systemd-journal07:25:012
177801992724,2cyclictest442382-21kworker/u8:1+flush-259:011:25:151
177801992724,2cyclictest380712-21kworker/u8:1+flush-259:010:25:211
177801992724,2cyclictest354904-21kworker/u8:4+flush-259:010:30:171
177801992723,3cyclictest354904-21kworker/u8:4+flush-259:010:15:011
177801992723,3cyclictest182803-21kworker/u8:0+flush-259:007:55:171
177801992722,3cyclictest183139-21/usr/sbin/munin07:15:171
177801992721,4cyclictest335557-21systemd-journal07:23:401
177801992623,2cyclictest188083-21kworker/u8:2+flush-259:007:30:161
177801992622,3cyclictest323530-21kworker/u8:2+flush-259:009:50:001
177801992621,4cyclictest365059-21kworker/u8:2+events_unbound10:50:221
177801992619,4cyclictest297502-21latency_hist09:05:001
177799992622,3cyclictest365059-21kworker/u8:2+events_unbound10:45:250
177799992622,3cyclictest335557-21systemd-journal08:09:260
177799992621,4cyclictest281104-21ntpq08:45:200
177809992522,2cyclictest282052-21kworker/u8:1+flush-259:008:55:003
177801992523,1cyclictest436915-21kworker/u8:0+events_unbound11:40:131
177801992523,1cyclictest224689-21kworker/u8:3+events_unbound08:55:151
177801992522,2cyclictest375360-21kworker/u8:0+events_unbound10:20:251
177801992522,2cyclictest256269-21kworker/u8:5+flush-259:008:45:311
177801992522,2cyclictest256269-21kworker/u8:5+events_unbound08:40:141
177801992521,3cyclictest224689-21kworker/u8:3+events_unbound08:35:301
177801992519,4cyclictest349749-21/usr/sbin/munin09:55:131
177799992522,2cyclictest380712-21kworker/u8:1+flush-259:010:25:170
177809992421,2cyclictest354904-21kworker/u8:4+events_unbound10:10:003
177809992420,3cyclictest509136-21fwupd12:25:223
177805992419,3cyclictest478265-21ntpq11:55:222
177805992419,3cyclictest432185-21apt-get11:15:132
177799992422,2cyclictest499817-21kworker/u8:2+flush-259:012:35:410
177799992422,2cyclictest323530-21kworker/u8:2+events_unbound09:54:410
177799992421,2cyclictest436915-21kworker/u8:0+flush-259:012:30:140
177799992421,2cyclictest224689-21kworker/u8:3+events_unbound08:55:220
177799992420,3cyclictest468951-21base6411:50:010
177799992420,3cyclictest436915-21kworker/u8:0+events_unbound11:22:410
177799992419,3cyclictest385835-21/usr/sbin/munin10:30:140
177809992318,4cyclictest437295-21apt-get11:20:123
177809992318,4cyclictest335557-21systemd-journal11:05:013
17780599238,13cyclictest213743-21cron07:45:002
177801992320,2cyclictest271799-21kworker/u8:0+flush-259:010:00:161
177801992320,2cyclictest256269-21kworker/u8:5+flush-259:008:25:241
177801992320,2cyclictest224689-21kworker/u8:3+flush-259:009:10:261
177801992319,3cyclictest354904-21kworker/u8:4+flush-259:011:30:161
177801992319,3cyclictest256269-21kworker/u8:5+events_unbound08:50:151
177801992318,3cyclictest390975-21/usr/sbin/munin10:35:151
177801992316,5cyclictest431054-21ntpq11:10:231
17780199231,20cyclictest401059-21munin-run10:45:011
177799992320,2cyclictest499817-21kworker/u8:2+flush-259:012:25:210
177799992320,2cyclictest271799-21kworker/u8:0+events_unbound09:20:170
177799992320,2cyclictest188083-21kworker/u8:2+flush-259:007:25:000
177799992320,2cyclictest188083-21kworker/u8:2+events_unbound07:30:230
177809992218,3cyclictest256269-21kworker/u8:5+flush-259:008:45:173
177809992213,8cyclictest276501-21unixbench_singl08:40:233
177805992217,3cyclictest256269-21kworker/u8:5+flush-259:009:05:182
177805992216,4cyclictest224040-21latency_hist07:55:012
177801992219,2cyclictest271799-21kworker/u8:0+events_unbound09:20:251
177801992218,3cyclictest354904-21kworker/u8:4+flush-259:010:15:161
177799992219,2cyclictest452807-21kworker/u8:2+flush-259:011:35:160
177799992219,2cyclictest365059-21kworker/u8:2+flush-259:010:15:250
177799992219,2cyclictest256269-21kworker/u8:5+events_unbound09:35:160
177799992219,2cyclictest239449-21kworker/u8:1+flush-259:008:20:000
177799992219,2cyclictest224689-21kworker/u8:3+events_unbound09:16:410
177799992218,3cyclictest462830-21kworker/u8:3+events_unbound11:45:240
177799992218,3cyclictest462830-21kworker/u8:3+events_unbound11:45:240
177799992218,3cyclictest213758-21latency_hist07:45:010
177809992118,2cyclictest271799-21kworker/u8:0+flush-259:009:40:153
177809992115,5cyclictest343489-21ntpq09:45:203
177809992113,7cyclictest2971-21cron12:05:003
17780599217,12cyclictest250214-21ntpq08:15:212
177805992117,3cyclictest442382-21kworker/u8:1+flush-259:012:05:242
177805992117,3cyclictest442382-21kworker/u8:1+flush-259:012:05:242
177801992119,2cyclictest188083-21kworker/u8:2+flush-259:007:45:001
177801992118,2cyclictest426811-21kworker/u8:1+events_unbound11:15:251
177801992118,2cyclictest323530-21kworker/u8:2+flush-259:009:30:211
177801992118,2cyclictest163527-21kworker/u8:1+events_unbound07:10:141
177801992117,3cyclictest354904-21kworker/u8:4+flush-259:010:05:261
177801992117,3cyclictest256269-21kworker/u8:5+flush-259:009:35:161
177799992119,2cyclictest354904-21kworker/u8:4+events_unbound10:23:400
177799992118,2cyclictest436915-21kworker/u8:0+events_unbound11:55:160
177799992117,3cyclictest436915-21kworker/u8:0+events_unbound11:40:150
177799992115,4cyclictest335557-21systemd-journal12:10:000
177799992115,4cyclictest335557-21systemd-journal12:10:000
177809992011,8cyclictest514228-21needreboot12:30:183
177809992010,9cyclictest250184-21ntp_kernel_pll_08:15:213
17780999200,10cyclictest436176-21ld11:15:233
177805992016,3cyclictest442382-21kworker/u8:1+flush-259:012:00:262
177805992015,4cyclictest255367-21ntpq08:20:202
177805992011,7cyclictest0-21swapper/211:15:012
17780599200,19cyclictest338791-21munin-run09:45:002
177801992017,2cyclictest271799-21kworker/u8:0+flush-259:009:10:001
177801992017,2cyclictest198336-21cron07:30:001
177801992017,2cyclictest188083-21kworker/u8:2+flush-259:008:10:001
177801992016,3cyclictest436915-21kworker/u8:0+events_unbound12:15:201
177801992014,4cyclictest520420-21latency_hist12:40:011
177799992017,2cyclictest442382-21kworker/u8:1+events_unbound12:00:250
177799992017,2cyclictest172651-21kworker/u8:3+events_unbound07:19:420
177799992016,3cyclictest172651-21kworker/u8:3+flush-259:007:15:000
17780599195,4cyclictest507-21in:imuxsock11:45:002
17780599191,5cyclictest504932-21sh12:25:002
17780599191,5cyclictest335557-21systemd-journal10:25:002
177805991915,3cyclictest313022-21kworker/u8:1+flush-259:009:25:162
177805991914,4cyclictest271799-21kworker/u8:0+flush-259:009:15:172
177801991916,2cyclictest436915-21kworker/u8:0+events_unbound12:30:191
177801991916,2cyclictest421612-21idleruntime-cro11:05:011
177801991916,2cyclictest301rcuc/112:10:001
177801991916,2cyclictest301rcuc/112:10:001
177801991915,3cyclictest365059-21kworker/u8:2+events_unbound10:55:281
177799991917,2cyclictest224689-21kworker/u8:3+flush-259:008:40:210
177799991916,2cyclictest365059-21kworker/u8:2+flush-259:010:50:160
177809991814,3cyclictest468355-21apt-get11:50:003
177809991814,3cyclictest468355-21apt-get11:50:003
177809991814,3cyclictest335557-21systemd-journal11:32:413
177809991812,4cyclictest411361-21latency_hist10:55:013
177809991810,7cyclictest0-21swapper/310:35:183
17780599186,2cyclictest245878-21latency_hist08:15:012
17780599183,4cyclictest443002-21df11:25:112
177801991814,3cyclictest462830-21kworker/u8:3+flush-259:011:50:161
177801991812,5cyclictest504083-21ntpq12:20:221
177799991815,3cyclictest245357-21kworker/u8:4+events_unbound08:31:410
177799991815,2cyclictest354904-21kworker/u8:4+flush-259:011:30:250
177799991815,2cyclictest271799-21kworker/u8:0+flush-259:010:00:160
177799991815,2cyclictest224689-21kworker/u8:3+events_unbound08:35:170
177799991814,3cyclictest354904-21kworker/u8:4+events_unbound10:38:420
17780999178,7cyclictest350413-21cut09:55:163
177809991714,2cyclictest224689-21kworker/u8:3+flush-259:008:00:003
177809991713,3cyclictest380745-21apt-get10:25:113
177809991713,3cyclictest172651-21kworker/u8:3+flush-259:007:20:273
177809991712,4cyclictest335557-21systemd-journal08:30:003
177809991711,4cyclictest370208-21latency_hist10:15:013
17780599178,7cyclictest0-21swapper/208:25:192
17780599175,2cyclictest0-21swapper/208:55:182
17780599174,4cyclictest420783-21ntpq11:00:202
17780599174,3cyclictest386326-21sed10:30:172
17780599174,2cyclictest0-21swapper/211:25:002
17780599171,5cyclictest370218-21sleep10:14:592
17780599171,4cyclictest307826-21munin-plugin-st09:15:002
17780599171,3cyclictest520411-21awk12:40:002
17780599171,3cyclictest395880-21cron10:40:002
17780599171,3cyclictest344358-21munin-plugin-st09:50:002
17780599171,3cyclictest335557-21systemd-journal08:45:012
177805991713,3cyclictest271799-21kworker/u8:0+flush-259:009:30:172
177805991712,4cyclictest224669-21cp07:55:012
17780599170,4cyclictest203512-21awk07:35:012
177801991714,2cyclictest271799-21kworker/u8:0+events_unbound09:15:231
177801991714,2cyclictest188083-21kworker/u8:2+flush-259:008:10:191
177801991713,3cyclictest462830-21kworker/u8:3+events_unbound11:48:411
177801991713,3cyclictest462830-21kworker/u8:3+events_unbound11:48:411
177801991713,3cyclictest318427-21apt-get09:25:101
17779999179,7cyclictest172651-21kworker/u8:3+flush-259:007:35:260
177799991715,1cyclictest370219-21idleruntime-cro10:15:000
177799991714,3cyclictest224689-21kworker/u8:3+events_unbound09:13:410
177799991714,2cyclictest323530-21kworker/u8:2+events_unbound09:34:420
177799991714,2cyclictest188083-21kworker/u8:2+events_unbound07:45:120
177799991713,3cyclictest182803-21kworker/u8:0+events_unbound07:50:110
17780999162,11cyclictest333694-21cat09:40:013
177809991613,2cyclictest504919-21cron12:25:003
177809991613,2cyclictest265796-21ntp_offset08:30:213
17780999161,13cyclictest187209-21ntpq07:15:233
17780999161,12cyclictest355230-21grep10:00:143
17780999160,14cyclictest473956-21idleruntime-cro11:55:003
17780999160,10cyclictest240262-21latency_hist08:10:013
17780599167,8cyclictest0-21swapper/211:05:172
17780599166,1cyclictest0-21swapper/208:55:002
17780599165,2cyclictest2150712sleep207:45:172
17780599165,2cyclictest0-21swapper/208:05:012
17780599165,1cyclictest0-21swapper/210:55:202
17780599161,5cyclictest494604-21idleruntime-cro12:15:002
17780599161,5cyclictest494604-21idleruntime-cro12:15:002
17780599161,4cyclictest354656-21cron09:59:592
17780599161,3cyclictest365035-21apt10:10:002
177805991611,3cyclictest224689-21kworker/u8:3+flush-259:008:35:172
17780599160,5cyclictest411342-21cron10:55:012
17780599160,3cyclictest495290-21ps12:15:172
17780599160,3cyclictest193893-21sed07:25:142
177801991613,2cyclictest188083-21kworker/u8:2+events_unbound07:45:141
177801991612,3cyclictest271799-21kworker/u8:0+events_unbound09:40:191
177799991612,3cyclictest426811-21kworker/u8:1+events_unbound11:15:010
177799991612,3cyclictest323530-21kworker/u8:2+flush-259:009:45:210
177799991612,3cyclictest271799-21kworker/u8:0+flush-259:009:40:460
177799991610,4cyclictest256250-21latency_hist08:25:010
17780999157,7cyclictest230547-21tune2fs08:00:143
17780999156,8cyclictest251571-21fschecks_time08:20:133
17780999156,8cyclictest245861-21sh08:15:003
17780999156,7cyclictest475611-21kthreadcore11:55:183
17780999155,8cyclictest348115-21kthreadcore09:50:193
17780999151,13cyclictest297522-21idleruntime-cro09:05:003
17780999151,12cyclictest457191-21ntpq11:35:223
17780999151,12cyclictest193892-21grep07:25:143
17780999150,14cyclictest218885-21cron07:50:003
17780999150,13cyclictest494607-21latency_hist12:15:003
17780999150,13cyclictest401394-21grep10:45:143
17780999150,12cyclictest370467-21apt-get10:15:133
17780999150,12cyclictest318557-21tr09:25:133
17780999150,12cyclictest307965-21latency_hist09:15:013
17780599152,2cyclictest292633-21apt-get09:00:092
17780599151,5cyclictest405323-21ntpq10:45:222
17780599151,5cyclictest184143-21/usr/sbin/munin07:15:182
17780599151,4cyclictest401019-21cron10:45:002
17780599151,3cyclictest182901-21sh07:14:592
17780599150,5cyclictest453529-21/usr/sbin/munin11:35:172
17780599150,4cyclictest451935-21tail11:30:202
177801991512,2cyclictest452807-21kworker/u8:2+flush-259:011:35:211
177801991511,3cyclictest365059-21kworker/u8:2+flush-259:011:05:201
177799991513,1cyclictest60350irq/132-enp0s29f1:rx-010:55:250
177799991513,1cyclictest60350irq/132-enp0s29f1:rx-008:25:190
177799991511,3cyclictest354904-21kworker/u8:4+events_unbound11:00:140
177799991511,3cyclictest349536-21kworker/u8:3+events_unbound09:55:170
17780999147,6cyclictest4897112sleep312:10:123
17780999142,10cyclictest287552-21grep08:55:143
177809991412,1cyclictest412644-21kthreadcore10:55:183
17780999141,12cyclictest426319-21/usr/sbin/munin11:05:253
17780999141,11cyclictest427034-21/usr/sbin/munin11:10:123
17780999140,6cyclictest0-21swapper/307:50:113
17780999140,13cyclictest386274-21sed10:30:163
17780999140,13cyclictest181932-21head07:10:203
17780999140,12cyclictest515728-21/usr/sbin/munin12:35:163
17780999140,12cyclictest495287-21sed12:15:173
17780999140,12cyclictest396571-21grep10:40:173
17780999140,12cyclictest324544-21/usr/sbin/munin09:30:173
17780999140,12cyclictest313416-21df09:20:143
17780999140,12cyclictest297887-21cpuspeed_turbos09:05:133
17780999140,12cyclictest198728-21grep07:30:143
17780999140,11cyclictest484668-21grep12:05:133
17780999140,11cyclictest484668-21grep12:05:133
17780999140,11cyclictest209910-21sort07:40:183
17780599143,2cyclictest832-21runrttasks11:50:202
17780599140,5cyclictest265769-21ntpq08:30:202
17780199149,4cyclictest335557-21systemd-journal08:25:001
177801991411,3cyclictest323530-21kworker/u8:2+flush-259:009:50:161
177801991411,2cyclictest172651-21kworker/u8:3+events_unbound07:35:161
17779999149,4cyclictest302701-21latency_hist09:10:010
177799991413,1cyclictest60350irq/132-enp0s29f1:rx-008:50:060
177799991412,1cyclictest60350irq/132-enp0s29f1:rx-010:05:140
177799991411,3cyclictest172651-21kworker/u8:3+events_unbound07:25:230
177799991411,2cyclictest365059-21kworker/u8:2+flush-259:011:05:260
17780999130,11cyclictest308256-21sort09:15:143
17780599137,4cyclictest177726-21latency_hist07:10:012
177799991311,2cyclictest60350irq/132-enp0s29f1:rx-009:25:250
177799991310,2cyclictest426811-21kworker/u8:1+events_unbound11:15:170
17780199128,3cyclictest442382-21kworker/u8:1+events_unbound12:00:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional