You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 07:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jun 04, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
280687994340,2cyclictest499081-21fwupd23:30:403
280683994337,4cyclictest316162-21latency_hist20:10:002
280687994134,5cyclictest315516-21latency_hist20:05:003
280678993934,4cyclictest515180-21/usr/sbin/munin23:50:131
280678993934,3cyclictest483285-21gpgv23:20:011
280678993834,3cyclictest265272-21kworker/u8:1+flush-259:020:50:001
280687993732,4cyclictest494024-21gpgv23:30:003
280683993734,2cyclictest472671-21kworker/u8:0+flush-259:023:35:212
280678993733,2cyclictest316205-21kworker/u8:3+flush-259:020:20:141
280687993427,5cyclictest409668-21diskstats22:10:143
280678993430,3cyclictest419731-21gpgv22:20:001
280687993327,4cyclictest335557-21systemd-journal23:05:013
280687993327,4cyclictest335557-21systemd-journal23:05:003
280683993329,3cyclictest350996-21kworker/u8:1+flush-259:022:10:172
280687993227,4cyclictest297832-21cat19:35:013
280673993227,3cyclictest494592-21apt-get23:30:120
280683993128,2cyclictest461969-21kworker/u8:1+events_unbound00:00:002
280683993128,2cyclictest265272-21kworker/u8:1+flush-259:019:20:162
280678993127,3cyclictest299068-21apt-get19:40:081
280687993025,3cyclictest404389-21apt-get22:05:123
280678993024,4cyclictest535597-21latency_hist00:10:001
280678993024,4cyclictest345159-21latency_hist21:00:001
280683992926,2cyclictest377819-21kworker/u8:2+flush-259:021:40:002
280683992926,2cyclictest316205-21kworker/u8:3+events_unbound20:20:002
280678992926,2cyclictest419499-21kworker/u8:0+flush-259:022:55:151
280687992821,5cyclictest477629-21latency_hist23:15:003
280678992823,3cyclictest425462-21/usr/sbin/munin22:25:141
280673992823,3cyclictest361915-21latency_hist21:25:010
280673992822,4cyclictest378220-21diskstats21:40:120
280683992724,2cyclictest327485-21kworker/u8:0+flush-259:021:05:002
280683992721,4cyclictest483169-21latency_hist23:20:002
280673992723,3cyclictest393542-21cp21:55:010
280687992619,4cyclictest525515-21/usr/sbin/munin00:00:123
280683992623,2cyclictest546232-21apt-get00:20:112
280683992623,2cyclictest514747-21kworker/u8:0+flush-259:023:50:242
280683992623,2cyclictest488994-21apt-get23:25:102
280683992623,2cyclictest430594-21kworker/u8:3+flush-259:023:20:162
280683992621,3cyclictest335557-21systemd-journal19:38:352
280678992623,2cyclictest350996-21kworker/u8:1+events_unbound22:31:361
280678992623,2cyclictest328014-21apt-get20:30:071
280678992622,3cyclictest419499-21kworker/u8:0+flush-259:022:40:071
280673992623,2cyclictest514747-21kworker/u8:0+flush-259:000:15:230
280673992621,4cyclictest488735-21latency_hist23:25:000
280687992522,2cyclictest509805-21cut23:45:003
280687992521,3cyclictest335557-21systemd-journal20:33:353
280683992522,2cyclictest350125-21kworker/u8:3+flush-259:021:10:012
280683992522,2cyclictest332881-21kworker/u8:3+events_unbound20:40:202
280683992522,2cyclictest2388469-21kworker/u8:2+events_unbound19:30:102
280683992520,3cyclictest472196-21base6423:10:012
280678992522,2cyclictest350125-21kworker/u8:3+flush-259:021:10:111
280678992519,4cyclictest425229-21latency_hist22:25:001
280673992522,2cyclictest545605-21kworker/u8:3+flush-259:000:35:260
280673992521,3cyclictest350996-21kworker/u8:1+events_unbound22:35:360
280673992521,3cyclictest292197-21kworker/u8:0+events_unbound20:05:350
280673992519,4cyclictest308814-21ntpq19:50:160
280687992419,4cyclictest350125-21kworker/u8:3+flush-259:021:15:003
280683992421,2cyclictest377819-21kworker/u8:2+flush-259:022:20:252
280683992420,3cyclictest461969-21kworker/u8:1+flush-259:000:15:162
280678992421,2cyclictest2388469-21kworker/u8:2+flush-259:019:40:011
280678992418,4cyclictest296385-21ntpq19:30:161
280673992420,3cyclictest530699-21apt-get00:05:110
280673992420,3cyclictest350996-21kworker/u8:1+flush-259:021:45:160
280687992318,3cyclictest399203-21/usr/sbin/munin22:00:153
280687992317,4cyclictest451701-21apt-get22:50:123
280687992316,5cyclictest451436-21latency_hist22:50:013
280683992320,2cyclictest461969-21kworker/u8:1+events_unbound00:30:142
280683992320,2cyclictest451676-21/usr/sbin/munin22:50:202
280683992320,2cyclictest327485-21kworker/u8:0+flush-259:020:30:162
280683992319,3cyclictest446275-21kworker/u8:2+flush-259:023:00:012
280678992320,2cyclictest350996-21kworker/u8:1+flush-259:021:30:001
280678992319,3cyclictest461969-21kworker/u8:1+flush-259:000:30:151
280678992319,3cyclictest377819-21kworker/u8:2+events_unbound21:50:151
280678992319,3cyclictest332881-21kworker/u8:3+events_unbound20:38:371
280678992319,3cyclictest292197-21kworker/u8:0+events_unbound19:56:271
280673992320,2cyclictest461969-21kworker/u8:1+flush-259:000:34:590
280673992319,3cyclictest461969-21kworker/u8:1+flush-259:000:00:110
280673992319,3cyclictest292197-21kworker/u8:0+flush-259:019:30:360
280673992318,3cyclictest514945-21latency_hist23:50:010
280687992217,3cyclictest372670-21latency_hist21:35:013
28068799221,20cyclictest525296-21true00:00:003
280683992219,2cyclictest545605-21kworker/u8:3+flush-259:000:35:202
280683992218,3cyclictest316205-21kworker/u8:3+events_unbound20:10:092
280683992218,3cyclictest2388469-21kworker/u8:2+flush-259:019:15:002
280678992219,2cyclictest514747-21kworker/u8:0+events_unbound00:10:221
280678992219,2cyclictest461969-21kworker/u8:1+flush-259:023:00:251
280678992219,2cyclictest461969-21kworker/u8:1+flush-259:023:00:241
280678992219,2cyclictest419499-21kworker/u8:0+events_unbound22:50:231
280678992219,2cyclictest377821-21kworker/u8:4+events_unbound21:40:201
280678992219,2cyclictest316205-21kworker/u8:3+flush-259:020:10:121
280678992218,3cyclictest461969-21kworker/u8:1+flush-259:000:35:151
280673992219,2cyclictest461969-21kworker/u8:1+flush-259:023:25:200
280673992219,2cyclictest461969-21kworker/u8:1+flush-259:000:25:160
280673992218,3cyclictest350996-21kworker/u8:1+events_unbound22:15:250
280673992215,5cyclictest483318-21cat23:20:010
28067399220,20cyclictest1-21systemd23:41:390
280683992118,2cyclictest430594-21kworker/u8:3+events_unbound22:45:182
280683992118,2cyclictest377819-21kworker/u8:2+events_unbound22:00:142
280683992118,2cyclictest350996-21kworker/u8:1+flush-259:022:15:242
280683992117,3cyclictest419499-21kworker/u8:0+events_unbound23:00:022
280683992117,3cyclictest419499-21kworker/u8:0+events_unbound23:00:022
280683992117,3cyclictest361921-21apt-get21:25:022
280678992118,3cyclictest472671-21kworker/u8:0+events_unbound23:23:391
280678992118,2cyclictest499091-21kworker/u8:3+events_unbound23:45:181
280678992118,2cyclictest399008-21kworker/u8:0+events_unbound22:00:211
280678992118,2cyclictest377819-21kworker/u8:2+flush-259:022:05:211
280673992118,2cyclictest265272-21kworker/u8:1+events_unbound19:20:100
280673992117,3cyclictest535861-21apt-get00:10:120
280673992116,3cyclictest335557-21systemd-journal22:12:350
280687992015,3cyclictest285745-21kworker/u8:3+flush-259:019:25:003
280683992017,2cyclictest430594-21kworker/u8:3+flush-259:022:40:162
28068399201,18cyclictest248157-21munin-node21:55:012
280678992017,3cyclictest430594-21kworker/u8:3+events_unbound23:10:151
280678992016,3cyclictest327485-21kworker/u8:0+events_unbound21:31:491
280673992017,2cyclictest316205-21kworker/u8:3+flush-259:020:10:160
280673992017,2cyclictest292197-21kworker/u8:0+events_unbound19:48:370
280673992017,2cyclictest265272-21kworker/u8:1+events_unbound19:25:350
280673992016,3cyclictest419499-21kworker/u8:0+events_unbound22:50:220
280687991913,4cyclictest298904-21base6419:40:013
280687991912,5cyclictest420467-21diskstats22:20:143
280683991916,2cyclictest514747-21kworker/u8:0+events_unbound00:13:392
280683991915,3cyclictest472679-21/usr/sbin/munin23:10:132
280683991915,3cyclictest292197-21kworker/u8:0+events_unbound20:01:352
280683991914,4cyclictest335557-21systemd-journal21:00:002
280678991916,2cyclictest553715-21kthreadcore00:25:181
280678991916,2cyclictest350996-21kworker/u8:1+flush-259:022:10:251
280678991916,2cyclictest332881-21kworker/u8:3+flush-259:020:40:161
280673991916,2cyclictest350125-21kworker/u8:3+events_unbound21:25:230
280673991915,3cyclictest419499-21kworker/u8:0+events_unbound22:48:360
280673991915,3cyclictest419499-21kworker/u8:0+events_unbound22:28:380
280673991915,3cyclictest265272-21kworker/u8:1+events_unbound20:16:360
280673991911,7cyclictest0-21swapper/023:15:000
280687991812,4cyclictest309707-21latency_hist19:55:003
280683991815,2cyclictest377819-21kworker/u8:2+events_unbound22:25:192
280683991815,2cyclictest350996-21kworker/u8:1+events_unbound21:20:122
280683991814,3cyclictest285745-21kworker/u8:3+events_unbound19:26:232
280683991813,3cyclictest286369-21latency_hist19:20:002
280678991815,2cyclictest514747-21kworker/u8:0+events_unbound00:00:171
280678991815,2cyclictest419499-21kworker/u8:0+events_unbound22:38:361
280678991815,2cyclictest350996-21kworker/u8:1+flush-259:021:20:001
280678991815,2cyclictest338942-21kworker/u8:2+events_unbound20:53:371
280673991815,2cyclictest377819-21kworker/u8:2+flush-259:022:24:590
280673991815,2cyclictest377819-21kworker/u8:2+flush-259:022:05:000
280673991815,2cyclictest321996-21kworker/u8:2+events_unbound20:32:360
280673991815,2cyclictest292197-21kworker/u8:0+flush-259:020:04:060
280673991815,2cyclictest285745-21kworker/u8:3+flush-259:019:18:060
280673991815,2cyclictest0-21swapper/022:45:000
280673991814,3cyclictest520360-21apt-get23:55:110
280673991812,4cyclictest310357-21latency_hist20:00:000
280687991713,3cyclictest472367-21apt-key23:10:013
280683991714,2cyclictest461969-21kworker/u8:1+events_unbound00:25:182
280683991714,2cyclictest350996-21kworker/u8:1+flush-259:022:35:302
280683991713,3cyclictest514747-21kworker/u8:0+events_unbound00:03:352
280683991713,3cyclictest332881-21kworker/u8:3+events_unbound20:35:352
280683991713,3cyclictest265272-21kworker/u8:1+events_unbound19:50:082
280683991712,4cyclictest335557-21systemd-journal22:35:002
280678991714,2cyclictest285745-21kworker/u8:3+events_unbound19:17:361
280678991713,3cyclictest514747-21kworker/u8:0+events_unbound23:55:401
280678991713,3cyclictest338942-21kworker/u8:2+flush-259:021:00:201
280678991713,3cyclictest327485-21kworker/u8:0+events_unbound21:39:361
280678991712,4cyclictest357016-21apt-get21:20:081
280673991714,2cyclictest472671-21kworker/u8:0+flush-259:023:35:190
280673991714,2cyclictest377819-21kworker/u8:2+flush-259:021:55:250
280673991714,2cyclictest338942-21kworker/u8:2+flush-259:020:59:060
280673991713,3cyclictest430594-21kworker/u8:3+flush-259:023:05:010
280673991713,3cyclictest430594-21kworker/u8:3+flush-259:023:05:000
280673991713,3cyclictest419499-21kworker/u8:0+events_unbound22:55:120
280673991713,3cyclictest316205-21kworker/u8:3+flush-259:020:20:170
280687991613,2cyclictest535646-21kworker/u8:2-events_unbound00:29:593
280687991612,2cyclictest0-21swapper/300:39:593
280687991610,4cyclictest335557-21systemd-journal21:46:503
280683991612,3cyclictest514747-21kworker/u8:0+events_unbound00:05:192
280683991612,3cyclictest356339-21kworker/u8:2+events_unbound21:15:362
280683991612,3cyclictest292197-21kworker/u8:0+events_unbound19:56:372
280678991613,2cyclictest545069-21ntp_kernel_pll_00:15:221
280678991613,2cyclictest446275-21kworker/u8:2+events_unbound23:05:241
280678991613,2cyclictest265272-21kworker/u8:1+flush-259:020:06:351
280678991612,3cyclictest499697-21/usr/sbin/munin23:35:141
280678991612,3cyclictest350125-21kworker/u8:3+events_unbound21:06:321
280678991612,3cyclictest335557-21systemd-journal00:25:001
280678991612,3cyclictest2388469-21kworker/u8:2+events_unbound19:14:351
280673991613,2cyclictest61150irq/133-enp0s29f1:tx-022:05:130
280673991613,2cyclictest60350irq/132-enp0s29f1:rx-023:05:130
280673991613,2cyclictest535646-21kworker/u8:2+flush-259:000:20:140
280673991613,2cyclictest327485-21kworker/u8:0+flush-259:021:35:190
280673991612,3cyclictest514747-21kworker/u8:0+flush-259:023:50:250
280673991611,3cyclictest327485-21kworker/u8:0+flush-259:021:30:110
280687991511,2cyclictest327762-21sh20:30:003
280683991512,2cyclictest350996-21kworker/u8:1+flush-259:021:49:002
280683991511,3cyclictest472671-21kworker/u8:0+events_unbound23:34:182
280683991510,4cyclictest408879-21kworker/u8:3+events_unbound22:05:482
280678991512,2cyclictest265272-21kworker/u8:1+flush-259:019:50:161
280678991511,3cyclictest398964-21latency_hist22:00:011
280678991510,4cyclictest265272-21kworker/u8:1+events_unbound19:20:141
280673991513,1cyclictest60350irq/132-enp0s29f1:rx-021:10:260
280673991513,1cyclictest60350irq/132-enp0s29f1:rx-021:00:200
280673991512,2cyclictest265272-21kworker/u8:1+flush-259:020:50:140
280673991512,2cyclictest265272-21kworker/u8:1+flush-259:019:10:130
280673991511,3cyclictest332881-21kworker/u8:3+events_unbound20:39:370
280673991510,4cyclictest280586-21latency_hist19:10:010
28068799141,3cyclictest446248-21awk22:44:593
280687991411,2cyclictest461253-21unixbench_multi22:55:233
280687991411,2cyclictest377788-21idleruntime-cro21:40:013
280687991411,2cyclictest283146-21kthreadcore19:10:183
280683991410,3cyclictest265272-21kworker/u8:1+events_unbound20:21:272
280678991412,1cyclictest0-21swapper/123:30:001
280678991411,2cyclictest265272-21kworker/u8:1+flush-259:020:00:151
280673991412,1cyclictest60350irq/132-enp0s29f1:rx-022:30:230
280673991412,1cyclictest60350irq/132-enp0s29f1:rx-020:40:110
280673991412,1cyclictest60350irq/132-enp0s29f1:rx-019:38:250
280673991411,2cyclictest60350irq/132-enp0s29f1:rx-019:40:030
28068799139,2cyclictest0-21swapper/319:30:003
28068799137,4cyclictest414899-21diskstats22:15:153
28068799136,4cyclictest335557-21systemd-journal20:06:253
280687991310,2cyclictest425612-21tr22:25:113
28068399139,3cyclictest338942-21kworker/u8:2+events_unbound20:46:162
28068399138,3cyclictest327485-21kworker/u8:0+events_unbound20:29:112
280683991310,2cyclictest292197-21kworker/u8:0+events_unbound19:48:432
28067899139,3cyclictest350996-21kworker/u8:1+events_unbound21:45:191
280678991310,3cyclictest265272-21kworker/u8:1+events_unbound19:46:361
280678991310,2cyclictest461969-21kworker/u8:1+flush-259:023:40:201
280678991310,2cyclictest430594-21kworker/u8:3+flush-259:022:45:121
280678991310,2cyclictest0-21swapper/120:20:001
28067399139,3cyclictest350125-21kworker/u8:3+events_unbound21:05:360
28067399139,3cyclictest327485-21kworker/u8:0+flush-259:020:29:060
280673991311,1cyclictest60350irq/132-enp0s29f1:rx-021:16:060
280673991311,1cyclictest60350irq/132-enp0s29f1:rx-020:48:300
28068799129,2cyclictest361915-21latency_hist21:25:023
28068799125,5cyclictest499950-21uniq23:35:153
28068799125,5cyclictest335557-21systemd-journal20:45:013
28068799121,9cyclictest440710-21awk22:39:593
28068799121,3cyclictest540799-21awk00:15:003
28068799121,2cyclictest345153-21awk21:00:013
28068399128,3cyclictest350996-21kworker/u8:1+events_unbound21:40:122
28068399128,3cyclictest350125-21kworker/u8:3+events_unbound21:34:492
28067899129,2cyclictest285745-21kworker/u8:3+events_unbound19:29:361
28068799119,1cyclictest0-21swapper/323:45:213
28068799118,2cyclictest514747-21kworker/u8:0+flush-259:000:06:103
28068799117,3cyclictest265272-21kworker/u8:1+flush-259:020:46:483
28068799111,3cyclictest350321-21latency_hist21:05:003
28068799110,3cyclictest286378-21awk19:20:003
28067899119,1cyclictest0-21swapper/120:30:001
28067899118,2cyclictest461969-21kworker/u8:1+flush-259:023:30:251
28068799105,3cyclictest561306-21sshd00:33:023
28068799103,5cyclictest335557-21systemd-journal20:00:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional