You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 09:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Fri Aug 07, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
368361521720,2sleep3575-21kworker/3:2+events22:07:573
364749821200,6chrt0-21swapper/221:32:572
36416892830,3chrt0-21swapper/221:28:002
3504195994740,5cyclictest3645269-21latency_hist21:30:011
3504197994626,18cyclictest3561020-21munin-run20:05:002
3504190994436,6cyclictest3515919-21latency_hist19:20:000
3504190994436,6cyclictest3515919-21latency_hist19:20:000
3504190994233,6cyclictest3531161-21base6419:35:010
3504195994031,6cyclictest3625481-21latency_hist21:10:011
3504197993925,12cyclictest3770212-21aten2_r5power_v23:37:482
3504197993924,13cyclictest3601557-21fschecks_time20:47:502
3504195993930,6cyclictest3814336-21latency_hist00:20:001
3504190993935,3cyclictest3526785-21kworker/u8:0+flush-8:019:57:520
3504190993934,3cyclictest3526785-21kworker/u8:0+flush-8:019:43:030
3504190993932,5cyclictest3526785-21kworker/u8:0+flush-8:019:52:510
3504190993931,6cyclictest3789810-21/usr/sbin/munin23:58:070
3504197993823,13cyclictest3775249-21cpuspeed_turbos23:42:492
3504195993828,2cyclictest161rcu_preempt23:07:541
3504190993832,4cyclictest3719981-21kworker/u8:1+flush-8:000:02:500
3504190993733,3cyclictest3601024-21kworker/u8:2+flush-8:021:27:560
3504190993733,2cyclictest3555838-21kworker/u8:2+flush-8:020:17:540
3504195993626,8cyclictest3565401-21ntpq20:08:011
3504190993630,4cyclictest3601024-21kworker/u8:2+flush-8:020:47:500
3504190993626,8cyclictest3585205-21ntpq20:28:000
3504197993518,15cyclictest0-21swapper/220:20:002
3504197993518,15cyclictest0-21swapper/219:40:002
3504190993531,3cyclictest3719957-21idleruntime-cro22:45:000
3504190993531,3cyclictest2686082-21kworker/u8:1+events_unbound19:48:230
3504190993529,4cyclictest3526785-21kworker/u8:0+flush-8:020:13:060
3504190993526,7cyclictest3645709-21/usr/sbin/munin21:33:000
3504190993525,8cyclictest3694220-21ntpq22:18:000
3504197993418,14cyclictest3710827-21fschecks_count22:37:512
3504195993429,3cyclictest3745968-21irqcore23:12:551
3504190993430,3cyclictest3814585-21kworker/u8:2+events_unbound00:22:580
350419799332,7cyclictest3808421-21ntpq00:13:002
350419799332,7cyclictest3808421-21ntpq00:13:002
3504197993319,12cyclictest3824799-21aten2_r5power_e00:32:502
3504195993328,3cyclictest3765434-21fschecks_time23:32:511
3504195993328,3cyclictest3755994-21irqrtprio23:22:581
3504195993316,3cyclictest0-21swapper/119:17:561
3504195993316,3cyclictest0-21swapper/119:17:561
3504190993325,6cyclictest3784438-21latency_hist23:50:010
3504190993325,5cyclictest3630417-21latency_hist21:15:010
350419799322,7cyclictest3814823-21apt-get00:22:492
3504197993227,3cyclictest3709830-21cron22:35:002
3504197993225,5cyclictest275-21systemd-journal21:55:002
3504197993216,14cyclictest3690630-21aten2_r5power_v22:17:492
350419799321,12cyclictest3550790-21apt-get19:55:032
3504195993228,2cyclictest3611141-21aten2_r5power_c20:57:461
3504195993222,7cyclictest275-21systemd-journal00:08:191
3504195993222,7cyclictest275-21systemd-journal00:08:191
3504190993228,3cyclictest3601024-21kworker/u8:2+flush-8:021:23:030
3504190993227,3cyclictest3699940-21kworker/u8:2+events_unbound22:35:000
3504190993227,3cyclictest3256672-21kworker/u8:4+events_unbound19:23:060
3504190993225,5cyclictest275-21systemd-journal19:29:470
350419799312,12cyclictest3829650-21apt-get00:37:502
3504197993118,11cyclictest3504797-21fschecks_count19:12:522
3504197993115,14cyclictest3665091-21idleruntime-cro21:50:002
3504197993115,14cyclictest3665091-21idleruntime-cro21:50:002
3504195993127,2cyclictest3650231-21systemd-run21:35:001
3504195993123,6cyclictest3700359-21/usr/sbin/munin22:28:001
3504195993118,9cyclictest275-21systemd-journal20:03:171
3504195993115,3cyclictest0-21swapper/122:33:061
3504190993126,4cyclictest3675464-21kworker/u8:0+events_unbound22:12:470
3504190993123,6cyclictest3744681-21latency_hist23:10:010
3504190993120,9cyclictest3755523-21diskstats23:22:530
350419799303,8cyclictest3725388-21apt-get22:52:492
350419799303,8cyclictest3621003-21apt-get21:07:482
350419799302,6cyclictest3580287-21ntpq20:23:012
3504197993012,2cyclictest0-21swapper/223:23:032
3504197993012,2cyclictest0-21swapper/219:47:462
350419799301,11cyclictest3590192-21ntpq20:33:012
350419799301,11cyclictest3590192-21ntpq20:33:012
350419599303,13cyclictest3790100-21fschecks_count23:57:521
3504195993014,9cyclictest0-21swapper/121:17:561
3504190993026,2cyclictest3615608-21unin-run21:00:010
3504190993025,4cyclictest3601024-21kworker/u8:2+flush-8:020:52:530
3504190993025,3cyclictest3808380-21needreboot00:13:000
3504190993025,3cyclictest3808380-21needreboot00:12:590
3504190993025,3cyclictest3560954-21idleruntime-cro20:05:000
3504190993025,3cyclictest3526785-21kworker/u8:0+flush-8:020:38:040
3504190993022,6cyclictest3788617-21ntpq23:52:590
3504190993022,6cyclictest3699940-21kworker/u8:2+flush-8:023:38:010
350339923019,6sleep30-21swapper/319:08:033
350419799293,5cyclictest3591308-21apt-get20:37:482
350419799292,6cyclictest3629756-21ntpq21:13:012
350419799292,6cyclictest3570405-21ntpq20:13:032
3504197992923,4cyclictest3794059-21unixbench_multi23:58:092
3504197992914,13cyclictest3740525-21fschecks_time23:07:512
3504197992911,2cyclictest0-21swapper/222:57:572
350419599296,8cyclictest30-21ksoftirqd/100:26:521
3504195992924,3cyclictest3716059-21irqcore22:42:541
3504195992918,7cyclictest275-21systemd-journal19:49:511
3504195992918,3cyclictest161rcu_preempt20:33:041
3504195992918,3cyclictest161rcu_preempt20:33:041
3504190992925,3cyclictest3590864-21kworker/u8:3+flush-8:021:02:480
3504190992924,3cyclictest3601024-21kworker/u8:2+events_unbound21:57:560
3504190992922,5cyclictest3724910-21cron22:50:000
350419799282,7cyclictest3565401-21ntpq20:08:012
350419799282,6cyclictest3670686-21apt-get21:57:492
350419799282,5cyclictest3754059-21ntpq23:17:592
3504197992823,3cyclictest3689794-21unixbench_multi22:13:072
350419799281,9cyclictest3656173-21ps21:42:532
350419799281,9cyclictest3656173-21ps21:42:532
350419799281,9cyclictest3606179-21apt-get20:52:482
3504197992811,3cyclictest0-21swapper/222:48:092
350419799281,12cyclictest3585152-21ntpq20:27:582
350419799281,11cyclictest3619949-21ps21:03:032
3504195992823,3cyclictest3628101-21kthreadcore21:12:561
3504195992822,4cyclictest3780689-21irqrtprio23:47:551
3504195992820,2cyclictest161rcu_preempt22:12:551
3504190992825,2cyclictest3833360-21needreboot00:38:000
3504190992823,4cyclictest3794566-21kworker/u8:0+flush-8:000:08:060
3504190992823,4cyclictest3794566-21kworker/u8:0+flush-8:000:08:060
3504190992816,9cyclictest275-21systemd-journal00:35:000
356094022717,6sleep30-21swapper/320:05:013
350419799273,6cyclictest3809864-21apt-get00:17:482
350419799273,6cyclictest3635858-21apt-get21:22:462
350419799273,6cyclictest3520197-21ntpq19:23:022
350419799272,7cyclictest3550104-21ntpq19:53:002
3504197992711,2cyclictest0-21swapper/223:53:062
350419799271,10cyclictest3798734-21ps00:03:052
350419799271,10cyclictest3735856-21sed23:02:542
3504195992720,5cyclictest3640328-21latency_hist21:25:011
3504195992720,2cyclictest161rcu_preempt00:03:061
3504195992714,2cyclictest161rcu_preempt20:22:551
3504190992723,3cyclictest3650643-21/usr/sbin/munin21:37:490
3504190992723,3cyclictest3601024-21kworker/u8:2+flush-8:021:08:050
3504190992721,4cyclictest3699940-21kworker/u8:2+events_unbound22:27:530
3504190992718,7cyclictest3699168-21ntpq22:23:000
3504190992718,7cyclictest3590123-21ntpq20:32:590
3504190992718,7cyclictest3590123-21ntpq20:32:590
350419799268,6cyclictest3653336-21tr21:37:572
350419799265,8cyclictest3682987-21cut22:07:572
350419799265,7cyclictest3700405-21cut22:27:492
350419799262,7cyclictest3515189-21ntpq19:18:002
350419799262,7cyclictest3515189-21ntpq19:17:592
350419799262,6cyclictest3763911-21ntpq23:28:012
350419799262,6cyclictest3540271-21ntpq19:43:022
3504197992623,2cyclictest3695507-21aten2_r5power_e22:22:502
350419799261,6cyclictest3611110-21apt-get20:57:472
3504197992611,3cyclictest3719963-21turbostat.cron22:45:002
3504197992610,6cyclictest3677679-21cut22:02:562
3504197992610,3cyclictest3804233-21/usr/sbin/munin00:10:012
3504197992610,3cyclictest3804233-21/usr/sbin/munin00:10:002
3504197992610,3cyclictest3526488-21df19:32:502
3504197992610,2cyclictest3525099-21ntpq19:28:022
350419599269,6cyclictest3694954-21cron22:20:001
350419599266,13cyclictest3597853-21kthreadcore20:42:571
350419599265,12cyclictest30-21ksoftirqd/121:52:581
350419599263,10cyclictest0-21swapper/122:38:021
3504195992621,3cyclictest3834136-21cron00:40:001
350419599262,13cyclictest3506731-21kthreadcore19:12:571
3504195992618,2cyclictest161rcu_preempt22:03:011
3504195992618,2cyclictest161rcu_preempt21:57:591
3504195992614,9cyclictest275-21systemd-journal19:44:071
3504195992614,4cyclictest849-21mta-sts-daemon20:49:491
3504195992613,2cyclictest0-21swapper/119:27:561
3504195992611,6cyclictest3699604-21ls22:23:071
3504195992611,4cyclictest0-21swapper/120:37:501
3504195992610,9cyclictest0-21swapper/122:07:541
350419599260,15cyclictest3777577-21grep23:42:561
350419599260,14cyclictest3536089-21date19:40:011
3504190992620,4cyclictest3719981-21kworker/u8:1+events_unbound23:43:050
3504190992618,6cyclictest3660118-21latency_hist21:45:000
3504190992618,6cyclictest3660118-21latency_hist21:45:000
350419799252,6cyclictest3596288-21apt-get20:42:502
3504197992521,3cyclictest3631197-21fschecks_time21:17:512
350419599252,13cyclictest3530884-21apt-key19:35:011
3504195992520,3cyclictest3787056-21kthreadcore23:52:561
3504195992519,4cyclictest0-21swapper/121:40:001
3504195992518,5cyclictest3566924-21fschecks_time20:12:531
3504195992517,2cyclictest161rcu_preempt23:37:561
3504195992516,3cyclictest161rcu_preempt22:52:591
3504195992511,8cyclictest0-21swapper/121:43:021
3504195992511,8cyclictest0-21swapper/121:43:011
3504195992511,2cyclictest0-21swapper/123:27:571
350419599250,15cyclictest3607898-21grep20:52:561
350419599250,14cyclictest3731677-21grep22:57:581
3504190992520,4cyclictest3508813-21unixbench_multi19:13:080
3504190992519,4cyclictest3675464-21kworker/u8:0+events_unbound22:03:080
350419599247,3cyclictest161rcu_preempt19:58:011
350419599244,2cyclictest0-21swapper/120:27:531
3504195992421,2cyclictest3754998-21latency_hist23:20:011
350419599240,14cyclictest3808479-21ld00:13:021
350419599240,14cyclictest3808479-21ld00:13:021
350419599240,14cyclictest3573138-21grep20:17:551
350419599240,14cyclictest3546513-21grep19:52:491
350419599240,14cyclictest3518737-21grep19:22:571
350419599240,13cyclictest3738271-21grep23:02:571
3504190992422,1cyclictest0-21swapper/022:58:080
3504190992420,3cyclictest3580977-21kworker/u8:1+flush-8:020:43:120
3504190992420,3cyclictest3555838-21kworker/u8:2+flush-8:020:08:080
3504190992418,4cyclictest3725363-21/usr/sbin/munin22:53:070
3504190992418,4cyclictest3719981-21kworker/u8:1+flush-8:023:28:060
3504190992418,3cyclictest3620561-21kworker/u8:1+events_unbound21:55:000
3504190992417,5cyclictest3699940-21kworker/u8:2+events_unbound22:35:030
3504190992416,6cyclictest0-21swapper/000:27:580
34995162243,3sleep00-21swapper/019:06:510
350419799231,9cyclictest3823723-21ps00:28:052
350419599234,11cyclictest30-21ksoftirqd/122:46:391
350419599234,11cyclictest30-21ksoftirqd/121:46:001
350419599234,11cyclictest30-21ksoftirqd/121:46:001
350419599230,12cyclictest3620332-21unixbench_singl21:03:051
3504190992319,3cyclictest3655420-21kworker/u8:3+flush-8:022:07:230
3504190992318,4cyclictest3601024-21kworker/u8:2+events_unbound21:18:030
37838102220,6sleep3575-21kworker/3:2+mm_percpu_wq23:48:043
350419599223,7cyclictest3814678-21taskset00:21:151
3504195992215,1cyclictest161rcu_preempt00:30:121
350419099225,16cyclictest3571103-21kworker/0:1+events20:22:560
350419099222,14cyclictest3531468-21chrt19:37:480
3504190992218,3cyclictest3661508-21kthreadcore21:47:570
3504190992218,3cyclictest3661508-21kthreadcore21:47:570
3504190992218,3cyclictest1-21systemd23:05:010
3504190992217,3cyclictest0-21swapper/023:12:560
34994772224,12sleep10-21swapper/119:06:201
3504190992117,2cyclictest0-21swapper/000:17:540
3504190992115,4cyclictest3750351-21tr23:17:480
35033652193,10sleep20-21swapper/219:08:022
350419799189,7cyclictest3748217-21tr23:12:582
350419099180,8cyclictest3765127-21/usr/sbin/munin23:32:480
3504197991712,3cyclictest0-21swapper/223:33:062
36198972168,5sleep30-21swapper/321:03:023
3504197991612,2cyclictest0-21swapper/223:50:002
38296242155,6sleep30-21swapper/300:37:483
35804842155,6sleep30-21swapper/320:23:063
38055792145,6sleep30-21swapper/300:12:563
38055792145,6sleep30-21swapper/300:12:563
36892342142,8sleep30-21swapper/322:12:593
37597432132,6sleep30-21swapper/323:25:033
37048652132,6sleep30-21swapper/322:30:013
36766232132,6sleep30-21swapper/322:02:563
36061512132,7sleep30-21swapper/320:52:473
38247422122,6sleep30-21swapper/300:32:503
37996732122,6sleep30-21swapper/300:06:423
37996732122,6sleep30-21swapper/300:06:423
37985702123,5sleep30-21swapper/300:03:013
37942132122,6sleep30-21swapper/323:59:003
37844592122,5sleep30-21swapper/323:50:013
37700722122,6sleep30-21swapper/323:37:453
37689082122,6sleep30-21swapper/323:33:003
37546142122,6sleep30-21swapper/323:18:073
37457882122,6sleep30-21swapper/323:12:543
37414432122,6sleep30-21swapper/323:07:563
37388682122,6sleep30-21swapper/323:02:593
37153302122,6sleep30-21swapper/322:41:203
36950102122,6sleep30-21swapper/322:20:023
36926962122,6sleep30-21swapper/322:17:573
36639282122,6sleep30-21swapper/321:48:003
36639282122,6sleep30-21swapper/321:48:003
36553032122,5sleep30-21swapper/321:40:013
36553032122,5sleep30-21swapper/321:40:013
36403202122,5sleep30-21swapper/321:25:013
36362992122,6sleep30-21swapper/321:22:523
36305522122,5sleep30-21swapper/321:15:023
36303452123,5sleep30-21swapper/321:14:173
36251712122,6sleep30-21swapper/321:08:083
35911662122,6sleep30-21swapper/320:36:183
35751372122,6sleep30-21swapper/320:18:043
35663552122,6sleep30-21swapper/320:10:023
35619092121,7sleep30-21swapper/320:07:533
35508012122,5sleep30-21swapper/319:55:013
35365062122,6sleep30-21swapper/319:42:493
35323512122,6sleep30-21swapper/319:37:563
35258022122,6sleep30-21swapper/319:30:023
35168972122,7sleep30-21swapper/319:22:553
35114172122,6sleep30-21swapper/319:17:463
35114172122,6sleep30-21swapper/319:17:463
35083002121,7sleep30-21swapper/319:13:003
38195242112,6sleep30-21swapper/300:25:083
38152032112,5sleep30-21swapper/300:22:553
38098382112,5sleep30-21swapper/300:17:463
37747922112,5sleep30-21swapper/323:40:023
37592502112,5sleep30-21swapper/323:23:093
37303192112,5sleep30-21swapper/322:57:493
37252462112,5sleep30-21swapper/322:51:193
37199892112,5sleep30-21swapper/322:45:013
37102942112,5sleep30-21swapper/322:35:023
37069762112,5sleep30-21swapper/322:32:583
36704572112,6sleep30-21swapper/321:55:103
36700482112,5sleep30-21swapper/321:53:093
36505002113,5sleep30-21swapper/321:35:563
36454942112,5sleep30-21swapper/321:30:023
36109852112,6sleep30-21swapper/320:56:283
36012152112,5sleep30-21swapper/320:47:453
35962402112,6sleep30-21swapper/320:42:323
35861972112,5sleep30-21swapper/320:30:523
35861972112,5sleep30-21swapper/320:30:523
35811742112,5sleep30-21swapper/320:25:023
35460842112,6sleep30-21swapper/319:50:123
35411322112,5sleep30-21swapper/319:45:023
35210992102,5sleep30-21swapper/319:25:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional