You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 16:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri Apr 17, 2026 12:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
376728822060,3sleep375-21kworker/3:1-events11:01:173
361308921880,6sleep30-21swapper/308:26:153
353595899149141,4cyclictest3766021-21apt-get11:00:032
353595599132127,3cyclictest0-21swapper/111:00:031
37061082920,2chrt75-21kworker/3:1-events09:56:283
3535953996053,5cyclictest2696061-21kworker/u8:3+flush-8:007:51:260
3535953995447,5cyclictest280-21systemd-journal11:20:000
3535953995440,13cyclictest0-21swapper/011:00:030
3535955995345,6cyclictest3617360-21latency_hist08:30:011
3535955995143,6cyclictest3622299-21latency_hist08:35:011
3535953995142,6cyclictest3815681-21latency_hist11:50:000
3535955994942,5cyclictest3557643-21latency_hist07:30:011
3535953994940,7cyclictest3596631-21ntpq08:06:230
3535953994940,6cyclictest280-21systemd-journal12:00:000
37379042470,5sleep20-21swapper/210:31:182
3535953994738,7cyclictest280-21systemd-journal12:20:010
3535955994638,6cyclictest3721571-21latency_hist10:15:011
3535955994534,8cyclictest3666752-21latency_hist09:20:011
3535955994534,8cyclictest3666752-21latency_hist09:20:011
3535953994539,4cyclictest3687325-21apt-get09:41:100
3535955994440,3cyclictest3536115-21kworker/u8:1+events_unbound08:16:121
3535953994440,3cyclictest3536115-21kworker/u8:1+flush-8:008:56:260
3535955994333,8cyclictest3740440-21ntpq10:31:221
3535953994339,3cyclictest3602694-21kworker/u8:0+flush-8:009:16:280
3535953994339,3cyclictest3602694-21kworker/u8:0+flush-8:009:16:280
3535953994333,8cyclictest3851414-21diskstats12:26:140
3535955994239,2cyclictest3602694-21kworker/u8:0+events_unbound10:17:231
3535955994036,3cyclictest3724760-21kworker/u8:3+events_unbound10:58:391
3535955994034,4cyclictest3786010-21sh11:20:001
3535955994030,8cyclictest3814850-21ntpq11:46:241
3535955994026,6cyclictest161rcu_preempt12:31:141
3535953994031,6cyclictest3602461-21latency_hist08:15:010
3535953994030,8cyclictest3702352-21diskstats09:56:140
3535955993935,3cyclictest3692162-21kworker/u8:3+events_unbound09:47:221
3535955993834,3cyclictest3602694-21kworker/u8:0+events_unbound11:44:381
3535955993832,4cyclictest3781078-21latency_hist11:15:011
3535953993829,6cyclictest3835926-21latency_hist12:10:000
353595899379,9cyclictest3566979-21ntpq07:36:242
3535955993733,3cyclictest2696061-21kworker/u8:3+events_unbound08:01:351
3535955993730,5cyclictest280-21systemd-journal09:21:211
3535953993730,5cyclictest280-21systemd-journal08:39:380
3535953993727,8cyclictest3611554-21ntpq08:21:240
3535958993611,6cyclictest3861050-21apt-get12:36:132
3535958993611,6cyclictest3861050-21apt-get12:36:132
3535955993632,3cyclictest3790692-21kworker/u8:1+events_unbound11:29:421
3535955993632,3cyclictest3602694-21kworker/u8:0+events_unbound08:40:381
3535955993632,3cyclictest3536115-21kworker/u8:1+events_unbound07:47:041
3535955993630,4cyclictest3726864-21apt-get10:21:101
3535955993629,5cyclictest3692162-21kworker/u8:3+flush-8:009:56:271
3535955993629,5cyclictest280-21systemd-journal12:03:211
3535955993629,5cyclictest280-21systemd-journal12:03:211
3535955993531,3cyclictest3846111-21kworker/u8:2+events_unbound12:38:591
3535955993531,3cyclictest3846111-21kworker/u8:2+events_unbound12:38:591
3535955993531,3cyclictest3805024-21kworker/u8:2+events_unbound11:54:381
3535955993531,3cyclictest3536115-21kworker/u8:1+events_unbound09:25:541
3535955993531,3cyclictest3536115-21kworker/u8:1+events_unbound07:55:081
3535955993530,3cyclictest3845853-21awk12:20:011
3535955993528,3cyclictest3841117-21kworker/u8:3+events_unbound12:22:211
3535955993526,7cyclictest3707198-21aten2_r5power_v10:01:151
3535953993530,4cyclictest3536115-21kworker/u8:1+flush-8:007:20:010
3535953993529,4cyclictest3536115-21kworker/u8:1+flush-8:007:26:230
3535953993526,6cyclictest3736396-21latency_hist10:30:010
353566523521,9sleep10-21swapper/107:08:251
3535958993425,6cyclictest3776116-21turbostat.cron11:10:002
3535958993412,12cyclictest3692349-21aten2_r5power_v09:46:122
3535955993429,4cyclictest3841117-21kworker/u8:3+events_unbound12:27:461
3535955993428,4cyclictest3563168-21apt-get07:36:101
3535955993421,5cyclictest3568153-21apt-get07:41:101
3535953993428,4cyclictest3612431-21kworker/u8:3+flush-8:009:10:520
3535953993423,9cyclictest3761687-21diskstats10:56:160
3535955993329,3cyclictest3825273-21kworker/u8:1+events_unbound11:59:141
3535955993329,3cyclictest3536115-21kworker/u8:1+events_unbound09:00:501
3535955993329,3cyclictest3536115-21kworker/u8:1+events_unbound08:23:421
3535955993329,3cyclictest3530924-21kworker/u8:0+events_unbound07:22:581
3535955993329,3cyclictest2696061-21kworker/u8:3+events_unbound08:12:511
3535953993325,6cyclictest3536115-21kworker/u8:1+flush-8:007:46:160
353595899328,6cyclictest3606573-21ntpq08:16:242
3535955993228,3cyclictest3612431-21kworker/u8:3+events_unbound08:53:181
3535955993228,3cyclictest3536115-21kworker/u8:1+events_unbound08:05:471
3535955993223,6cyclictest3755272-21ntpq10:46:231
3535953993228,3cyclictest3536115-21kworker/u8:1+flush-8:009:26:250
3535953993226,5cyclictest3724760-21kworker/u8:3+flush-8:011:13:260
3535953993224,5cyclictest3731155-21grep10:21:300
353595899316,11cyclictest3780302-21sh11:11:252
353595899315,13cyclictest0-21swapper/212:21:242
3535958993122,7cyclictest3815778-21cp11:50:002
3535955993128,2cyclictest3612431-21kworker/u8:3+events_unbound08:36:011
3535955993128,2cyclictest3602694-21kworker/u8:0+events_unbound09:32:321
3535955993127,3cyclictest3724760-21kworker/u8:3+events_unbound11:20:291
3535955993127,3cyclictest3677107-21kworker/u8:2+events_unbound09:35:411
3535955993127,3cyclictest3602694-21kworker/u8:0+events_unbound12:13:081
3535955993127,3cyclictest3602694-21kworker/u8:0+events_unbound12:09:111
3535955993127,3cyclictest2696061-21kworker/u8:3+events_unbound07:54:351
3535955993126,3cyclictest3687355-21aten2_r5power_c09:41:121
3535955993121,8cyclictest280-21systemd-journal10:42:211
3535953993126,3cyclictest3866007-21dpkg12:40:010
3535953993126,3cyclictest3866007-21dpkg12:40:000
3535953993126,3cyclictest3602694-21kworker/u8:0+flush-8:010:10:520
3535953993125,5cyclictest401-21dbus-daemon10:10:000
3535953993125,4cyclictest3602694-21kworker/u8:0+flush-8:009:21:260
353595899309,5cyclictest3656260-21python309:06:292
353595899309,5cyclictest3656260-21python309:06:292
353595899306,11cyclictest3632684-21/usr/sbin/munin08:46:132
353595899304,13cyclictest3622634-21apt-get08:36:122
3535958993016,9cyclictest3796265-21apt-get11:31:132
3535958993015,9cyclictest3548108-21apt-get07:21:132
353595599303,17cyclictest3536536-21tune2fs07:11:151
3535955993026,3cyclictest3741040-21kworker/u8:2+events_unbound10:35:411
3535955993026,3cyclictest3692162-21kworker/u8:3+events_unbound09:52:501
3535953993027,2cyclictest3592620-21systemd-run08:05:000
3535953993025,4cyclictest3741040-21kworker/u8:2+flush-8:010:52:520
3535953993013,14cyclictest280-21systemd-journal11:51:210
353595899299,7cyclictest3666798-21dpkg09:20:012
353595899299,7cyclictest3666798-21dpkg09:20:012
353595899296,11cyclictest3715843-21ntpq10:06:242
3535958992920,7cyclictest3755477-21find10:46:282
353595599297,10cyclictest30-21ksoftirqd/111:31:521
3535955992925,3cyclictest3602694-21kworker/u8:0+events_unbound11:38:131
3535955992925,3cyclictest3602694-21kworker/u8:0+events_unbound08:49:321
3535955992911,4cyclictest3562572-21seq07:31:511
3535953992924,3cyclictest3741040-21kworker/u8:2+events_unbound10:50:010
3535953992924,3cyclictest3602694-21kworker/u8:0+events_unbound10:16:120
3535953992923,4cyclictest3724760-21kworker/u8:3+flush-8:011:31:190
3535953992920,7cyclictest3686777-21gpgv09:40:010
353595899287,10cyclictest0-21swapper/207:51:152
353595899286,10cyclictest3611562-21ntpq08:21:242
353595899286,10cyclictest3601613-21ntpq08:11:232
353595899285,11cyclictest3856104-21apt-get12:31:122
353595899285,11cyclictest3576904-21ntpq07:46:242
3535958992818,4cyclictest3592934-21apt-get08:06:112
3535958992817,5cyclictest3821447-21apt-get11:56:112
3535955992824,3cyclictest3612431-21kworker/u8:3+flush-8:009:05:251
3535955992824,3cyclictest3612431-21kworker/u8:3+flush-8:009:05:241
3535955992824,3cyclictest3612431-21kworker/u8:3+events_unbound09:10:341
3535953992822,4cyclictest3790692-21kworker/u8:1+flush-8:011:41:310
3535953992822,4cyclictest3692162-21kworker/u8:3+events_unbound10:01:260
353595899276,5cyclictest3617699-21apt-get08:31:112
353595899274,11cyclictest3732703-21irqstats10:26:182
3535958992719,3cyclictest3568153-21apt-get07:41:112
3535958992718,4cyclictest3746607-21apt-get10:41:082
353595599278,6cyclictest3547620-21seq07:18:311
3535955992713,2cyclictest0-21swapper/110:06:191
3535953992724,2cyclictest3696857-21idleruntime-cro09:50:000
3535953992717,8cyclictest3652525-21diskstats09:06:140
3535953992717,8cyclictest3652525-21diskstats09:06:140
353595899265,8cyclictest3840131-21ntp_states12:11:262
353595899264,11cyclictest3616535-21ntpq08:26:242
3535958992623,2cyclictest3845521-21unixbench_singl12:16:302
3535958992617,4cyclictest3791308-21apt-get11:26:122
3535958992612,11cyclictest3720741-21/usr/sbin/munin10:11:242
3535955992612,2cyclictest161rcu_preempt08:58:401
3535953992618,6cyclictest3746582-21/usr/sbin/munin10:41:200
3535953992617,6cyclictest3587032-21python307:56:290
353595899256,8cyclictest3786410-21cut11:21:122
353595899254,5cyclictest3759918-21cut10:51:202
353595899253,5cyclictest3670928-21memory09:21:222
3535958992521,3cyclictest3830654-21unixbench_multi12:01:292
3535958992521,3cyclictest3830654-21unixbench_multi12:01:292
3535958992521,2cyclictest3721963-21aten2_r5power_e10:16:122
3535958992521,2cyclictest3711347-21unixbench_singl10:01:342
3535958992521,2cyclictest3648069-21grep09:01:192
3535958992521,2cyclictest3553152-21aten2_r5power_v07:26:132
353595599252,9cyclictest30-21ksoftirqd/110:55:011
3535955992517,2cyclictest161rcu_preempt10:30:011
3535953992521,3cyclictest3621974-21unixbench_singl08:31:300
3535953992519,4cyclictest3541006-21install07:15:010
38032162240,2sleep375-21kworker/3:1+events11:36:203
353595899245,9cyclictest0-21swapper/209:51:172
353595899245,4cyclictest3546948-21ntpq07:16:252
353595899244,9cyclictest0-21swapper/209:10:422
3535958992420,3cyclictest3687589-21fschecks_count09:41:152
3535958992420,3cyclictest3592595-21idleruntime-cro08:05:002
3535958992420,2cyclictest3820497-21unixbench_multi11:51:302
3535958992420,2cyclictest3637744-21fschecks_count08:51:152
3535958992419,3cyclictest3766172-21latency_hist11:00:002
3535958992419,3cyclictest3631805-21unixbench_multi08:41:322
3535958992415,5cyclictest3806640-21/usr/sbin/munin11:41:182
3535958992414,8cyclictest3854976-21/usr/sbin/munin12:26:242
3535958992411,4cyclictest0-21swapper/212:06:102
353595599240,6cyclictest3771728-21chrt11:06:111
3535953992420,3cyclictest3536115-21kworker/u8:1+flush-8:007:41:270
3535953992419,3cyclictest0-21swapper/007:25:010
3535953992418,4cyclictest3602694-21kworker/u8:0+flush-8:012:21:170
3535953992417,1cyclictest0-21swapper/011:06:100
353595399241,20cyclictest3558121-21sshd07:30:210
35357842243,3sleep00-21swapper/007:09:570
353595899235,9cyclictest0-21swapper/211:16:312
353595899235,9cyclictest0-21swapper/209:32:192
353595899235,9cyclictest0-21swapper/207:11:312
3535958992316,3cyclictest3562032-21ntpq07:31:262
3535958992315,4cyclictest3705923-21ntpq09:56:252
3535958992313,8cyclictest3685909-21/usr/sbin/munin09:36:252
3535958992312,8cyclictest3730813-21/usr/sbin/munin10:21:282
3535953992319,3cyclictest3612431-21kworker/u8:3+flush-8:008:51:280
3535953992318,4cyclictest3602694-21kworker/u8:0+flush-8:008:29:520
3535953992316,5cyclictest280-21systemd-journal11:21:210
353595399231,20cyclictest3697430-21df09:51:140
353595899225,9cyclictest0-21swapper/210:36:312
353595899225,7cyclictest0-21swapper/209:25:502
353595899224,8cyclictest0-21swapper/208:59:382
3535958992210,4cyclictest0-21swapper/211:36:162
3535953992218,3cyclictest3737261-21sort10:31:160
3535953992218,3cyclictest3677107-21kworker/u8:2+flush-8:009:31:420
3535953992218,3cyclictest3651616-21unixbench_singl09:01:330
3535953992218,2cyclictest3536115-21kworker/u8:1+events_unbound08:16:260
3535953992217,4cyclictest3724760-21kworker/u8:3+flush-8:010:38:520
3535953992213,7cyclictest280-21systemd-journal12:01:210
3535953992213,7cyclictest280-21systemd-journal12:01:210
353595899215,9cyclictest0-21swapper/207:56:302
3535953992117,3cyclictest3804136-21kthreadcore11:36:200
3535953992117,2cyclictest0-21swapper/007:36:120
3535953992018,1cyclictest0-21swapper/012:11:280
3535953992016,2cyclictest0-21swapper/008:46:170
3535953992015,3cyclictest3602694-21kworker/u8:0+flush-8:011:26:310
35309322202,8sleep30-21swapper/307:05:023
35316512194,9sleep20-21swapper/207:06:182
36635122180,2sleep3481ktimers/309:16:193
36635122180,2sleep3481ktimers/309:16:193
3535953991814,3cyclictest3841117-21kworker/u8:3+flush-8:012:31:260
3535953991813,4cyclictest3612431-21kworker/u8:3+events_unbound08:41:250
36980752142,8sleep30-21swapper/309:51:203
35885922142,8sleep30-21swapper/308:01:203
35717832142,8sleep30-21swapper/307:41:223
36876982130,3sleep3481ktimers/309:41:153
36502112132,7sleep30-21swapper/309:01:243
36371032132,7sleep30-21swapper/308:50:003
35408942132,6sleep30-21swapper/307:15:023
38612982122,6sleep30-21swapper/312:36:163
38612982122,6sleep30-21swapper/312:36:163
38558602122,6sleep30-21swapper/312:30:023
38477092122,6sleep30-21swapper/312:21:193
38362732122,6sleep30-21swapper/312:11:123
38359142123,5sleep30-21swapper/312:10:003
38309552122,6sleep30-21swapper/312:04:223
38309552122,6sleep30-21swapper/312:04:213
37953342122,6sleep30-21swapper/311:26:303
37901202123,5sleep30-21swapper/311:21:243
37470982122,6sleep30-21swapper/310:41:153
37166282122,5sleep30-21swapper/310:10:013
37160212122,6sleep30-21swapper/310:06:283
36918982122,5sleep30-21swapper/309:45:023
36764512122,6sleep30-21swapper/309:26:323
36713312122,6sleep30-21swapper/309:21:303
36427012122,6sleep30-21swapper/308:56:133
36374582122,6sleep30-21swapper/308:51:123
36233452122,6sleep30-21swapper/308:36:203
36174232122,5sleep30-21swapper/308:30:023
35981982122,6sleep30-21swapper/308:11:163
35926222122,6sleep30-21swapper/308:05:013
35582172122,6sleep30-21swapper/307:31:153
35483532120,8sleep30-21swapper/307:21:163
38516372112,5sleep30-21swapper/312:26:173
38412002112,5sleep30-21swapper/312:16:103
38223652112,5sleep30-21swapper/311:56:193
38159402112,5sleep30-21swapper/311:50:013
38149462112,6sleep30-21swapper/311:46:283
38060372112,5sleep30-21swapper/311:40:023
37998372112,5sleep30-21swapper/311:31:233
37816602112,5sleep30-21swapper/311:16:153
37798972112,5sleep30-21swapper/311:11:203
37614212112,5sleep30-21swapper/310:56:123
37563632112,5sleep30-21swapper/310:50:023
37515482112,5sleep30-21swapper/310:46:133
37413442112,5sleep30-21swapper/310:35:013
37366292112,5sleep30-21swapper/310:30:123
37359732112,5sleep30-21swapper/310:26:313
37268682112,5sleep30-21swapper/310:21:133
37257862112,5sleep30-21swapper/310:16:283
37069622112,5sleep30-21swapper/310:00:023
36819542112,5sleep30-21swapper/309:35:023
36809442112,5sleep30-21swapper/309:31:253
36617932112,6sleep30-21swapper/309:15:003
36519522112,5sleep30-21swapper/309:05:013
36519522112,5sleep30-21swapper/309:05:003
36278982112,5sleep30-21swapper/308:41:173
36076692112,5sleep30-21swapper/308:20:023
36031752112,5sleep30-21swapper/308:16:173
35829092112,5sleep30-21swapper/307:55:023
35783282111,6sleep30-21swapper/307:51:133
35773022112,5sleep30-21swapper/307:46:323
35631722112,5sleep30-21swapper/307:36:123
35575462113,5sleep30-21swapper/307:29:023
35400042112,5sleep30-21swapper/307:11:243
37753922102,5sleep30-21swapper/311:06:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional