You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 00:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat Aug 01, 2026 12:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2996191994741,4cyclictest275-21systemd-journal08:40:010
2996193994540,4cyclictest3265728-21kworker/u8:0+flush-8:012:37:561
2996191994436,6cyclictest3320027-21latency_hist12:35:010
2996193994336,5cyclictest3067857-21/usr/sbin/munin08:23:091
32653702420,4sleep20-21swapper/211:38:112
2996191994235,5cyclictest3265728-21kworker/u8:0+flush-8:011:47:560
2996193994031,7cyclictest3305229-21latency_hist12:20:011
2996191994031,6cyclictest3062460-21latency_hist08:15:010
2996193993933,4cyclictest3001253-21kworker/u8:3+events_unbound07:55:011
2996193993932,5cyclictest3226389-21/usr/sbin/munin11:03:111
2996193993930,7cyclictest3017913-21latency_hist07:30:001
2996193993831,5cyclictest3121826-21kworker/u8:1+flush-8:009:27:581
2996193993731,4cyclictest2985946-21kworker/u8:2+flush-8:007:13:061
2996193993729,6cyclictest3137381-21/usr/sbin/munin09:33:101
2996193993728,6cyclictest3225931-21latency_hist11:00:011
2996193993727,7cyclictest3146862-21latency_hist09:40:001
2996193993631,4cyclictest3265728-21kworker/u8:0+flush-8:012:22:571
2996191993628,6cyclictest3186398-21latency_hist10:20:010
2996191993626,7cyclictest3141874-21latency_hist09:35:010
2996193993530,4cyclictest3097132-21kworker/u8:2+events_unbound09:05:001
2996193993530,4cyclictest3001253-21kworker/u8:3+flush-8:007:37:581
2996193993529,4cyclictest2985946-21kworker/u8:2+flush-8:008:18:101
2996193993527,5cyclictest275-21systemd-journal08:26:421
2996193993526,6cyclictest3057504-21latency_hist08:10:011
2996193993525,7cyclictest275-21systemd-journal12:32:421
2996191993528,5cyclictest3151805-21kworker/u8:2+flush-8:011:42:560
2996191993528,5cyclictest3151805-21kworker/u8:2+flush-8:011:42:560
2996191993524,8cyclictest275-21systemd-journal12:40:000
2996197993426,5cyclictest275-21systemd-journal07:45:002
2996193993428,4cyclictest3009229-21irqrtprio07:22:571
2996193993427,5cyclictest3132014-21cron09:25:001
2996191993425,7cyclictest3131355-21ntpq09:23:060
2996197993329,3cyclictest3111804-21kworker/u8:3+flush-8:010:18:082
2996193993328,3cyclictest3161701-21systemd-run09:55:001
2996191993326,5cyclictest3265728-21kworker/u8:0+flush-8:012:23:010
29957212333,8sleep10-21swapper/107:08:371
2996193993228,2cyclictest3060143-21kthreadcore08:13:021
2996193993227,4cyclictest3018141-21kworker/u8:0+flush-8:007:33:091
2996193993227,3cyclictest3057520-21kworker/u8:1+flush-8:008:52:561
2996193993225,5cyclictest3241221-21apt-get11:17:541
2996193993224,6cyclictest3206728-21aten2_r5power_v10:42:571
2996191993227,3cyclictest3176393-21kworker/u8:1+flush-8:010:33:130
2996191993227,3cyclictest3088431-21irqrtprio08:42:580
2996197993125,5cyclictest401-21dbus-daemon12:40:012
2996197993121,8cyclictest3310873-21cpuspeed_turbos12:27:542
2996193993125,5cyclictest3151805-21kworker/u8:2+flush-8:012:03:131
2996193993125,4cyclictest3275576-21idleruntime-cro11:50:001
2996193993125,4cyclictest3111804-21kworker/u8:3+flush-8:010:08:081
2996191993125,4cyclictest3225927-21idleruntime-cro11:00:000
2996191993124,5cyclictest3290378-21cat12:05:000
2996191993124,5cyclictest3097579-21apt-get08:52:510
2996191993124,5cyclictest2985946-21kworker/u8:2+flush-8:007:20:000
2996191993121,8cyclictest275-21systemd-journal11:40:000
2996191993118,4cyclictest3157318-21aten2_r5power_p09:52:530
2996197993021,7cyclictest3107781-21fschecks_count09:02:572
2996193993025,3cyclictest3213730-21kthreadcore10:48:021
2996193993024,4cyclictest414-21systemd-logind08:33:491
2996193993024,4cyclictest3235649-21kworker/u8:3+flush-8:011:32:561
2996193993024,4cyclictest3111804-21kworker/u8:3+events_unbound09:20:001
2996193993020,8cyclictest3037029-21ntpq07:48:091
2996191993025,4cyclictest3151805-21kworker/u8:2+flush-8:011:58:080
2996193992925,3cyclictest3057520-21kworker/u8:1+flush-8:008:38:101
2996193992924,4cyclictest3265728-21kworker/u8:0+flush-8:012:27:591
2996193992924,3cyclictest3248560-21kthreadcore11:23:011
2996193992924,3cyclictest3001253-21kworker/u8:3+events_unbound08:02:561
2996193992923,4cyclictest3043870-21irqrtprio07:58:001
2996191992926,2cyclictest3051811-21needreboot08:03:020
2996191992924,3cyclictest3113016-21irqcore09:08:000
2996191992923,4cyclictest3012764-21unixbench_singl07:23:100
299619199290,25cyclictest571-21snmpd10:47:530
2996197992825,2cyclictest3077977-21cpuspeed_turbos08:32:552
2996193992823,4cyclictest3151805-21kworker/u8:2+events_unbound12:15:011
2996193992823,3cyclictest3156791-21systemd-run09:50:011
2996193992822,4cyclictest3166408-21unixbench_multi09:58:111
2996193992821,5cyclictest3235649-21kworker/u8:3+events_unbound11:28:081
2996191992823,3cyclictest3299483-21needreboot12:13:030
2996191992822,4cyclictest3265728-21kworker/u8:0+flush-8:012:28:140
2996191992822,4cyclictest3001179-21latency_hist07:15:000
2996197992724,2cyclictest3126818-21unixbench_singl09:18:112
2996197992721,5cyclictest2985946-21kworker/u8:2+flush-8:007:18:472
2996197992721,4cyclictest3121845-21cron09:15:002
2996193992723,3cyclictest3235649-21kworker/u8:3+flush-8:011:38:101
2996193992720,5cyclictest2174223-21kworker/u8:1+events_unbound07:18:071
2996193992719,2cyclictest0-21swapper/110:32:561
2996193992717,8cyclictest3121181-21ntpq09:13:051
2996191992723,3cyclictest3250947-21latency_hist11:25:000
2996191992722,4cyclictest3072158-21unixbench_singl08:23:110
2996191992722,3cyclictest3190781-21ps10:23:080
2996191992721,4cyclictest275-21systemd-journal10:26:420
2996191992718,7cyclictest3067401-21latency_hist08:20:010
2996191992718,6cyclictest3206130-21latency_hist10:40:000
2996197992621,3cyclictest3231672-21fschecks_time11:07:562
2996197992621,3cyclictest3196778-21aten2_r5power_c10:32:542
2996197992620,4cyclictest275-21systemd-journal08:00:002
2996197992618,6cyclictest3266467-21fschecks_count11:42:562
2996197992618,6cyclictest3266467-21fschecks_count11:42:562
2996193992621,3cyclictest0-21swapper/108:50:011
2996193992618,5cyclictest3270668-21latency_hist11:45:001
2996193992618,5cyclictest3270668-21latency_hist11:45:001
2996193992618,5cyclictest3097132-21kworker/u8:2+events_unbound09:08:071
2996193992616,8cyclictest3106280-21ntpq08:58:041
2996191992622,3cyclictest3176524-21idleruntime-cro10:10:000
2996191992622,3cyclictest3151805-21kworker/u8:2+flush-8:012:07:560
2996191992621,3cyclictest3181462-21idleruntime-cro10:15:000
2996191992621,2cyclictest844-21mta-sts-daemon11:11:180
2996191992620,4cyclictest3276770-21irqrtprio11:53:010
2996191992617,7cyclictest3097115-21latency_hist08:50:010
2996197992521,2cyclictest0-21swapper/210:27:582
2996197992520,3cyclictest3092997-21fschecks_time08:47:572
2996197992520,3cyclictest3013736-21fschecks_time07:27:552
2996193992520,3cyclictest3203826-21kthreadcore10:38:011
2996193992520,2cyclictest0-21swapper/109:43:101
2996193992519,5cyclictest3265728-21kworker/u8:0+events_unbound11:53:081
2996193992518,5cyclictest3176393-21kworker/u8:1+events_unbound10:28:121
2996193992518,5cyclictest3151805-21kworker/u8:2+flush-8:010:53:051
299619199254,19cyclictest3122438-21sshd09:16:420
2996191992520,3cyclictest3230623-21unixbench_multi11:03:130
2996191992520,3cyclictest3146859-21apt09:39:590
2996191992520,3cyclictest3141738-21kworker/u8:0+flush-8:009:55:050
2996191992519,4cyclictest401-21dbus-daemon09:00:000
2996191992519,4cyclictest3300736-21apt-get12:17:530
2996191992519,4cyclictest3246069-21apt-config11:20:010
2996191992519,4cyclictest3017723-21unixbench_singl07:28:100
2996191992515,7cyclictest275-21systemd-journal08:34:590
2996197992421,2cyclictest3255825-21cron11:30:002
2996197992421,2cyclictest3087976-21fschecks_count08:42:552
2996197992420,3cyclictest3177303-21fschecks_time10:12:562
2996197992420,2cyclictest3063244-21fschecks_time08:17:572
2996197992419,3cyclictest3286196-21fschecks_count12:02:572
2996197992419,3cyclictest3152543-21fschecks_count09:47:562
2996197992416,6cyclictest3027074-21needreboot07:38:042
2996193992420,3cyclictest3265728-21kworker/u8:0+flush-8:011:57:561
2996193992420,3cyclictest3077303-21kworker/u8:0+flush-8:008:42:571
2996193992420,2cyclictest844-21mta-sts-daemon10:18:101
2996193992420,2cyclictest0-21swapper/110:13:021
2996193992418,4cyclictest3151805-21kworker/u8:2+events_unbound12:07:531
299619199242,21cyclictest0-21swapper/009:28:080
299619199242,14cyclictest0-21swapper/008:05:510
2996191992420,2cyclictest0-21swapper/007:43:000
2996191992419,3cyclictest0-21swapper/010:53:020
2996191992419,3cyclictest0-21swapper/007:38:020
2996191992417,5cyclictest3206701-21aten2_r5power_p10:42:570
2996191992415,2cyclictest421rcuog/209:47:530
2996197992320,2cyclictest3216768-21fschecks_count10:52:562
2996197992319,2cyclictest3067913-21aten2_r5power_c08:22:532
2996193992318,3cyclictest3176393-21kworker/u8:1+events_unbound11:08:111
2996193992318,3cyclictest3032318-21mailstats07:43:101
2996191992319,2cyclictest0-21swapper/009:12:550
2996191992319,2cyclictest0-21swapper/007:32:540
2996191992318,3cyclictest0-21swapper/010:05:010
2996191992317,4cyclictest3150523-21kthreadcore09:43:020
2996191992317,4cyclictest3047604-21cron08:00:000
31196742220,2sleep3481ktimers/309:13:003
299619799222,14cyclictest3097579-21apt-get08:52:532
299619799222,14cyclictest3038215-21apt-get07:52:532
2996197992219,2cyclictest3187098-21cpuspeed_turbos10:22:542
2996197992218,2cyclictest3221685-21cpuspeed_turbos10:57:562
2996197992214,6cyclictest3310187-21cat12:25:012
2996197992214,6cyclictest3057505-21cat08:10:002
2996193992217,2cyclictest0-21swapper/111:12:571
2996191992217,4cyclictest3235649-21kworker/u8:3+flush-8:011:27:470
299619799212,13cyclictest3172035-21apt-get10:07:562
2996197992117,2cyclictest3048185-21aten2_r5power_p08:02:522
2996197992116,3cyclictest3058054-21aten2_r5power_p08:12:552
2996197992113,6cyclictest3320029-21apt-get12:35:002
2996197992112,7cyclictest3165397-21cut09:58:012
299619399218,4cyclictest0-21swapper/110:22:451
2996191992117,2cyclictest849-21mta-sts-daemon07:48:030
2996191992117,2cyclictest0-21swapper/008:28:000
2996191992116,4cyclictest401-21dbus-daemon11:10:000
2996191992116,3cyclictest0-21swapper/007:52:540
31617592200,3chrt575-21kworker/3:2+events09:55:013
299619799202,3cyclictest0-21swapper/212:07:342
2996197992017,2cyclictest3008541-21aten2_r5power_c07:22:522
2996197992016,3cyclictest20823-21cron11:55:002
2996197992012,6cyclictest3137058-21sed09:30:002
2996197992012,6cyclictest3102511-21munin-node08:57:512
2996197992012,6cyclictest3022041-21sed07:33:032
2996197992011,7cyclictest3302935-21cut12:18:022
2996197992011,7cyclictest3139031-21cut09:33:012
2996197992011,7cyclictest2997272-21sed07:12:582
2996193992015,3cyclictest0-21swapper/110:02:581
2996191992016,2cyclictest0-21swapper/009:02:570
29909412204,9sleep20-21swapper/207:05:012
299619799198,9cyclictest3113854-21cut09:08:012
2996197991916,2cyclictest3201737-21aten2_r5power_e10:37:532
2996197991915,2cyclictest0-21swapper/209:37:562
2996197991911,6cyclictest3128332-21sed09:23:002
2996197991911,6cyclictest3034062-21sed07:48:022
2996197991910,7cyclictest3073504-21sed08:27:572
299619799190,8cyclictest32721651kthreadcore11:48:002
2996191991913,4cyclictest2996214-21apt-key07:10:010
29911452192,9sleep30-21swapper/307:05:033
2996197991814,3cyclictest3256560-21fschecks_count11:32:552
2996197991814,2cyclictest3147558-21cpuspeed_turbos09:42:542
2996197991814,2cyclictest0-21swapper/211:01:442
2996197991814,2cyclictest0-21swapper/210:03:082
2996197991811,5cyclictest3211562-21awk10:47:522
2996197991810,6cyclictest3282213-21cut11:57:592
2996197991810,6cyclictest3236877-21sed11:12:592
2996191991814,2cyclictest0-21swapper/011:33:010
299619799178,7cyclictest3297347-21cut12:13:002
2996197991712,3cyclictest0-21swapper/208:38:132
299619799170,8cyclictest31573192tr09:52:542
299619799169,5cyclictest3250164-21awk11:23:042
2996197991612,2cyclictest0-21swapper/211:19:132
2996197991611,3cyclictest0-21swapper/210:43:032
31814412154,8sleep30-21swapper/310:15:003
30283522152,9sleep30-21swapper/307:42:543
32313122145,6sleep30-21swapper/311:07:523
32126132145,6sleep30-21swapper/310:48:023
32508902132,6sleep30-21swapper/311:25:023
31819012132,7sleep30-21swapper/310:17:533
31413992134,5sleep30-21swapper/309:33:113
31364842132,7sleep30-21swapper/309:28:103
30772702132,7sleep30-21swapper/308:30:003
30138912133,6sleep30-21swapper/307:27:573
33212092122,6sleep30-21swapper/312:37:593
33151002122,5sleep30-21swapper/312:30:013
33009842122,6sleep30-21swapper/312:17:573
32903602122,5sleep30-21swapper/312:05:013
32899222122,6sleep30-21swapper/312:03:133
32806622122,6sleep30-21swapper/311:55:023
32720152122,6sleep30-21swapper/311:48:003
32615552122,6sleep30-21swapper/311:37:573
32571782121,6sleep30-21swapper/311:32:593
31976312122,6sleep30-21swapper/310:33:013
31922592122,6sleep30-21swapper/310:28:003
31864702122,6sleep30-21swapper/310:20:013
31762622122,6sleep30-21swapper/310:08:143
31715512122,5sleep30-21swapper/310:05:013
31617592121,6sleep30-21swapper/309:55:023
31565552122,6sleep30-21swapper/309:48:123
31222852121,6sleep30-21swapper/309:15:023
31151462122,6sleep30-21swapper/309:08:043
30969782122,6sleep30-21swapper/308:48:263
30915512122,6sleep30-21swapper/308:43:083
30786402122,6sleep30-21swapper/308:33:013
30705692122,6sleep30-21swapper/308:23:023
30579742122,6sleep30-21swapper/308:12:553
30395852123,5sleep30-21swapper/307:53:023
30129382122,5sleep30-21swapper/307:25:013
30011852122,5sleep30-21swapper/307:15:013
29966142122,6sleep30-21swapper/307:12:533
33155242112,6sleep30-21swapper/312:32:293
33053512112,5sleep30-21swapper/312:20:023
33002502112,6sleep30-21swapper/312:15:003
32762842112,6sleep30-21swapper/311:52:583
32659332112,5sleep30-21swapper/311:40:033
32659332112,5sleep30-21swapper/311:40:023
32461982113,5sleep30-21swapper/311:20:323
32421032112,5sleep30-21swapper/311:18:033
32365502112,6sleep30-21swapper/311:12:573
32262082112,6sleep30-21swapper/311:00:343
32254772112,5sleep30-21swapper/310:58:133
32166642112,6sleep30-21swapper/310:52:553
32064222112,6sleep30-21swapper/310:40:443
32015082112,6sleep30-21swapper/310:35:583
31477242111,6sleep30-21swapper/309:42:583
31426192112,5sleep30-21swapper/309:37:573
31273342112,6sleep30-21swapper/309:20:523
31070392112,5sleep30-21swapper/309:00:013
31022952112,6sleep30-21swapper/308:55:043
30973372112,5sleep30-21swapper/308:50:023
30825352112,6sleep30-21swapper/308:35:543
30627562112,6sleep30-21swapper/308:16:003
30530332112,6sleep30-21swapper/308:07:533
30480432112,6sleep30-21swapper/308:02:283
30430562112,6sleep30-21swapper/307:57:023
30335352112,5sleep30-21swapper/307:47:593
30231462113,5sleep30-21swapper/307:35:333
30181862112,5sleep30-21swapper/307:30:423
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional