You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 03:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun Dec 07, 2025 00:39:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
318180799533,23cyclictest657-21snmpd21:04:192
3181805995339,7cyclictest151ktimers/019:49:240
3181805995234,8cyclictest12450irq/32-eth022:59:210
3181805995234,8cyclictest12450irq/32-eth022:59:210
318180799512,25cyclictest3346021-21ssh00:09:292
3181807994940,6cyclictest1-21systemd00:00:022
3181807994939,7cyclictest3217789-21diskmemload21:10:022
3181806994840,4cyclictest0-21swapper/122:34:201
3181807994741,3cyclictest0-21swapper/222:59:132
3181807994741,3cyclictest0-21swapper/222:59:132
3181807994636,6cyclictest3299929-21systemd-run23:05:012
3181807994621,6cyclictest3184827-21kworker/u8:9+xprtiod19:45:012
3181807994621,6cyclictest3184827-21kworker/u8:9+xprtiod19:45:012
3181805994631,5cyclictest3196815-21systemd-run20:00:020
3181806994534,7cyclictest421-21avahi-daemon23:40:131
3181807994433,4cyclictest0-21swapper/223:44:262
3181805994418,7cyclictest3355206-21sh00:22:040
3181805994417,10cyclictest3222581-21sh21:16:130
3181807994335,5cyclictest422-21dbus-daemon00:30:012
3181807994335,5cyclictest422-21dbus-daemon00:30:012
3181805994316,11cyclictest3258610-21sh22:07:300
3181807994230,8cyclictest3342696-21switchtime00:04:372
3181807994228,6cyclictest0-21swapper/220:24:232
3181807994219,17cyclictest351ktimers/220:34:222
3181807994219,17cyclictest351ktimers/220:34:222
3181806994236,3cyclictest0-21swapper/122:39:221
3181806994234,5cyclictest3250332-21latency_hist21:55:021
318180599427,29cyclictest3281091-21ssh22:39:230
3181805994225,10cyclictest3187302-21ntpq19:29:260
3181805994218,7cyclictest3217789-21diskmemload22:28:380
3181805994217,7cyclictest3256199-21/usr/sbin/munin22:04:180
3181805994216,7cyclictest3358270-21sh00:26:000
3181805994216,7cyclictest3358270-21sh00:26:000
3181807994134,4cyclictest956-21mta-sts-daemon23:49:292
3181807994133,5cyclictest0-21swapper/219:40:012
3181807994132,3cyclictest0-21swapper/222:29:232
3181806994134,3cyclictest3307103-21kworker/u8:7-nfsiod00:00:031
3181806994132,6cyclictest3335400-21sed23:54:341
3181805994122,15cyclictest3214151-21ntpq20:59:210
3181805994118,7cyclictest3315012-21sh23:26:010
3181805994117,8cyclictest3365627-21sh00:38:210
3181805994116,7cyclictest3324423-21sh23:39:340
3181805994116,6cyclictest3297087-21sh23:01:200
3181807994032,5cyclictest3367497-21grep00:40:002
3181807994030,7cyclictest955-21mta-sts-daemon22:34:262
3181807994030,7cyclictest0-21swapper/223:29:202
3181807994029,7cyclictest1315733-21cups-browsed21:56:532
3181806994032,5cyclictest657-21snmpd00:30:021
3181806994032,5cyclictest657-21snmpd00:30:021
3181806994032,5cyclictest3296290-21awk23:00:011
3181806994032,5cyclictest3296290-21awk23:00:011
3181805994028,7cyclictest151ktimers/019:54:210
3181805994027,8cyclictest3203767-21ntpq20:24:230
3181805994027,10cyclictest3268629-21ssh22:21:160
3181805994023,14cyclictest3332055-21ssh23:49:450
3181805994015,9cyclictest3344779-21sh00:08:160
3181805994015,7cyclictest3326422-21sh23:42:490
3181807993930,4cyclictest0-21swapper/223:09:262
3181807993930,4cyclictest0-21swapper/223:09:262
3181807993929,6cyclictest3246818-21cron21:50:012
3181807993929,6cyclictest1315733-21cups-browsed22:49:192
3181806993931,5cyclictest3298340-21aten_r2power_cu23:04:101
3181805993927,8cyclictest3182671-21ntpq19:14:190
3181805993926,8cyclictest3242610-21ntpq21:44:310
3181805993926,8cyclictest3191799-21ntpq19:44:270
3181805993926,8cyclictest3191799-21ntpq19:44:270
3181805993925,12cyclictest3300441-21ssh23:05:430
3181805993925,12cyclictest3300441-21ssh23:05:430
3181805993923,14cyclictest3250536-21ssh21:55:040
3181805993922,14cyclictest3262404-21ssh22:13:030
3181805993921,12cyclictest3249633-21ps21:54:310
3181805993916,7cyclictest3309050-21sh23:19:030
3181807993831,4cyclictest3348975-21aten_r2power_fr00:14:192
3181807993830,5cyclictest3183278-21cron19:15:012
3181807993817,17cyclictest3284698-21ssh22:44:252
3181807993817,17cyclictest3284698-21ssh22:44:252
3181806993830,5cyclictest3221863-21systemd-run21:15:021
3181806993829,5cyclictest3314543-21latency_hist23:25:021
3181805993831,4cyclictest3184833-21systemd-run19:20:010
3181805993826,8cyclictest3212675-21ntpq20:54:210
3181805993824,8cyclictest3215673-21ntpq21:04:200
3181805993823,8cyclictest3338642-21ntpq23:59:250
3181805993823,11cyclictest3225758-21ssh21:20:130
3181805993823,10cyclictest3286624-21ssh22:47:150
3181805993821,15cyclictest3312542-21ssh23:23:380
3181805993821,14cyclictest3351740-21ssh00:17:360
3181805993821,14cyclictest3266084-21ssh22:18:320
3181805993820,7cyclictest3275974-21ssh22:32:050
3181805993820,15cyclictest3304162-21ssh23:11:010
3181805993820,15cyclictest3304162-21ssh23:11:010
3181805993818,10cyclictest3340955-21ssh00:02:530
3181805993815,7cyclictest3218453-21ssh21:10:500
3181805993814,17cyclictest12450irq/32-eth022:54:240
3181805993814,17cyclictest12450irq/32-eth022:54:240
3181806993731,3cyclictest0-21swapper/121:39:251
3181806993726,8cyclictest3257385-21munin-plugin-st22:05:021
3181805993729,4cyclictest3211257-21ntpq20:49:250
3181805993726,7cyclictest3202282-21ntpq20:19:260
3181805993724,8cyclictest3349655-21ntpq00:14:320
3181805993724,5cyclictest3200669-21ntpq20:14:230
3181805993723,11cyclictest3245260-21aten_r2power_cu21:49:090
3181805993722,12cyclictest3209486-21latency20:44:190
3181805993721,14cyclictest3333750-21ssh23:53:160
3181805993715,4cyclictest3284440-21memory22:44:180
3181805993715,4cyclictest3284440-21memory22:44:180
3181807993628,5cyclictest956-21mta-sts-daemon23:38:292
3181807993627,6cyclictest1-21systemd22:15:022
3181807993621,9cyclictest3353470-21mailstats00:19:302
318180699368,11cyclictest3355466-21sh00:22:401
3181806993628,5cyclictest3367498-21cut00:40:001
3181806993628,5cyclictest3321231-21grep23:34:321
3181805993626,6cyclictest3199227-21ntpq20:09:220
3181805993625,7cyclictest3217257-21ntpq21:09:240
3181805993624,8cyclictest3197704-21ntpq20:04:220
3181805993624,7cyclictest3188663-21ntpq19:34:200
3181805993623,8cyclictest3238919-21ntpq21:39:200
3181805993621,13cyclictest3230162-21ssh21:26:520
3181805993612,8cyclictest3318879-21ssh23:31:440
318180799358,18cyclictest3363335-21ssh00:34:302
318180799358,18cyclictest3363335-21ssh00:34:302
318180799354,8cyclictest3235493-21ssh21:34:252
318180799353,24cyclictest3356930-21ssh00:24:262
3181807993524,8cyclictest3213283-21cut20:55:012
3181807993524,7cyclictest422-21dbus-daemon23:22:012
318180799352,25cyclictest3256628-21ssh22:04:282
318180799352,24cyclictest657-21snmpd21:54:282
318180799351,24cyclictest3231854-21awk21:29:202
318180699358,9cyclictest3345873-21ntpq00:09:231
3181806993528,4cyclictest3253785-21munin-plugin-st22:00:011
3181806993526,5cyclictest3225433-21munin-run21:20:011
3181806993523,8cyclictest3291235-21aten_r2power_vo22:54:111
3181806993523,8cyclictest3291235-21aten_r2power_vo22:54:111
318180699352,29cyclictest3324193-21ssh23:39:281
3181805993526,6cyclictest3185181-21/usr/sbin/munin19:24:100
3181805993522,8cyclictest3206793-21ntpq20:34:270
3181805993522,8cyclictest3206793-21ntpq20:34:270
3181805993522,10cyclictest3363346-21ntpq00:34:310
3181805993522,10cyclictest3363346-21ntpq00:34:310
3181805993517,4cyclictest3233298-21sh21:30:520
318180799346,19cyclictest437-21mta-sts-daemon19:54:252
318180799343,22cyclictest3306186-21ssh23:14:212
318180799343,22cyclictest3306186-21ssh23:14:212
3181807993423,7cyclictest0-21swapper/221:24:312
3181806993426,5cyclictest3288657-21unixbench_multi22:49:321
3181806993426,5cyclictest3285200-21grep22:44:351
3181806993426,5cyclictest3285200-21grep22:44:351
3181806993426,5cyclictest3181810-21systemd-run19:10:021
3181806993422,9cyclictest3228672-21timerandwakeup21:24:311
3181805993424,7cyclictest3189804-21diskstats19:39:130
3181805993423,8cyclictest3205161-21ntp_kernel_err20:29:180
3181807993325,5cyclictest3201709-21aten_r2power_vo20:19:122
3181807993323,5cyclictest0-21swapper/222:54:112
3181807993323,5cyclictest0-21swapper/222:54:112
3181807993322,8cyclictest3320525-21needreboot23:34:172
318180799332,24cyclictest3334999-21ssh23:54:272
318180799332,22cyclictest1315733-21cups-browsed22:19:242
318180799331,24cyclictest3239015-21ssh21:39:242
318180699333,25cyclictest3317221-21ssh23:29:261
3181806993326,3cyclictest0-21swapper/119:49:241
3181806993320,8cyclictest3205220-21ntpq20:29:211
3181806993320,8cyclictest3200652-21ntpq20:14:221
3181805993320,9cyclictest3208166-21ntpq20:39:190
3181805993320,9cyclictest3208166-21ntpq20:39:190
318180799323,20cyclictest657-21snmpd20:39:192
318180799323,20cyclictest657-21snmpd20:39:192
3181807993224,4cyclictest0-21swapper/221:19:172
3181807993223,6cyclictest3196674-21unixbench_multi19:59:322
3181807993222,6cyclictest3201252-21head20:14:352
318180799322,24cyclictest3260116-21ssh22:09:262
3181807993221,8cyclictest1315733-21cups-browsed19:19:082
3181806993226,3cyclictest0-21swapper/123:15:011
3181806993226,3cyclictest0-21swapper/123:15:011
3181806993225,4cyclictest3310716-21cron23:20:001
3181806993225,4cyclictest0-21swapper/121:50:021
3181806993224,5cyclictest3204315-21systemd-run20:25:011
3181806993223,6cyclictest3216289-21cron21:05:011
3181806993219,7cyclictest3208260-21ntpq20:39:241
3181806993219,7cyclictest3208260-21ntpq20:39:241
3181806993217,11cyclictest3269313-21ssh22:22:541
3181806993216,9cyclictest3231865-21ntpq21:29:211
3181806993215,12cyclictest3274144-21ntpq22:29:261
3181807993123,3cyclictest0-21swapper/222:39:222
3181807993121,8cyclictest3214773-21cron21:00:012
3181807993117,10cyclictest3219958-21sshd21:13:412
318180699313,23cyclictest3242441-21ssh21:44:211
3181806993123,5cyclictest684-21munin-node00:01:141
3181806993123,5cyclictest3354061-21idleruntime-cro00:20:011
3181806993123,5cyclictest3263055-21/usr/sbin/munin22:14:151
3181806993122,4cyclictest0-21swapper/122:19:231
3181806993119,7cyclictest3235436-21ntpq21:34:231
3181806993118,8cyclictest3185737-21ntpq19:24:211
3181806993118,7cyclictest3196191-21ntpq19:59:211
3181806993110,6cyclictest28-21ksoftirqd/123:46:191
3181807993023,4cyclictest3166104-21kworker/u8:17-xprtiod19:10:022
3181807993023,3cyclictest0-21swapper/223:19:222
318180799301,26cyclictest351ktimers/219:24:242
3181806993023,3cyclictest0-21swapper/123:09:201
3181806993023,3cyclictest0-21swapper/123:09:201
3181806993022,5cyclictest3198651-21aten_r2power_fr20:09:111
3181806993021,6cyclictest3202789-21idleruntime-cro20:20:011
3181806993020,7cyclictest3192121-21wc19:44:331
3181806993020,7cyclictest3192121-21wc19:44:331
3181806993019,7cyclictest3194626-21/usr/sbin/munin19:54:181
3181806993018,8cyclictest3190194-21ntpq19:39:221
3181806993017,9cyclictest3209035-21sshd20:41:471
3181806993017,9cyclictest3207048-21/usr/sbin/munin20:34:331
3181806993017,9cyclictest3207048-21/usr/sbin/munin20:34:331
3181806993017,8cyclictest3184213-21ntpq19:19:201
3181806993015,11cyclictest3257731-21ssh22:05:211
3181806993015,11cyclictest3211222-21ntpq20:49:241
3181807992921,5cyclictest3187798-21munin-plugin-st19:30:012
3181807992918,7cyclictest422-21dbus-daemon22:21:052
3181806992921,5cyclictest3186842-21fschecks_count19:29:141
3181806992916,8cyclictest3182663-21ntpq19:14:181
3181806992915,11cyclictest3348015-21ssh00:12:181
3181806992915,11cyclictest3212766-21ntpq20:54:261
3181807992820,5cyclictest3198842-21cut20:09:142
3181807992819,7cyclictest3241795-21ssh21:44:122
3181806992817,7cyclictest3217075-21missed_timers21:09:181
3181806992817,7cyclictest3188671-21ntpq19:34:201
3181806992816,8cyclictest3197032-21sshd20:01:411
3181806992813,12cyclictest3361498-21ssh00:31:001
3181806992813,12cyclictest3361498-21ssh00:31:001
3181807992716,8cyclictest3205794-21munin-plugin-st20:30:022
3181805992715,7cyclictest3181761-21/usr/sbin/munin19:10:020
3181807992618,5cyclictest3198246-21unixbench_singl20:04:332
3181807992617,5cyclictest0-21swapper/220:44:292
318180699268,3cyclictest28-21ksoftirqd/120:58:411
3181807992418,2cyclictest0-21swapper/220:49:212
3181807992410,10cyclictest341rcuc/219:34:262
3181807992215,5cyclictest0-21swapper/219:49:142
318180899173,7cyclictest0-21swapper/300:19:203
318180899172,7cyclictest0-21swapper/319:29:203
318180899171,9cyclictest0-21swapper/321:54:263
318180899164,8cyclictest0-21swapper/323:29:213
318180899163,6cyclictest0-21swapper/319:19:213
318180899162,9cyclictest0-21swapper/321:39:243
318180899162,9cyclictest0-21swapper/300:04:293
318180899162,8cyclictest0-21swapper/322:39:213
318180899161,8cyclictest0-21swapper/323:14:263
318180899161,8cyclictest0-21swapper/323:14:263
318180899161,7cyclictest0-21swapper/322:44:183
318180899161,7cyclictest0-21swapper/322:44:183
318180899153,6cyclictest0-21swapper/321:59:203
318180899152,9cyclictest0-21swapper/321:44:323
318180899152,8cyclictest0-21swapper/323:19:243
318180899152,8cyclictest0-21swapper/322:29:223
318180899152,8cyclictest0-21swapper/322:19:233
318180899152,8cyclictest0-21swapper/321:34:313
318180899152,7cyclictest0-21swapper/323:44:133
318180899152,7cyclictest0-21swapper/300:34:303
318180899152,7cyclictest0-21swapper/300:34:303
318180899151,9cyclictest0-21swapper/323:04:433
318180899151,8cyclictest0-21swapper/322:14:253
318180899151,8cyclictest0-21swapper/322:09:263
318180899151,8cyclictest0-21swapper/300:09:113
318180899145,6cyclictest0-21swapper/321:19:223
318180899144,6cyclictest0-21swapper/322:24:223
318180899143,7cyclictest0-21swapper/300:14:213
318180899143,6cyclictest0-21swapper/323:34:263
318180899143,6cyclictest0-21swapper/322:49:263
318180899142,8cyclictest0-21swapper/323:59:243
318180899142,8cyclictest0-21swapper/321:14:163
318180899142,6cyclictest0-21swapper/320:44:303
318180899141,8cyclictest0-21swapper/323:39:253
318180899141,8cyclictest0-21swapper/323:09:353
318180899141,8cyclictest0-21swapper/323:09:353
318180899141,8cyclictest0-21swapper/322:04:253
318180899141,7cyclictest0-21swapper/322:54:193
318180899141,7cyclictest0-21swapper/322:54:193
318180899141,7cyclictest0-21swapper/320:54:243
318180899140,7cyclictest0-21swapper/320:39:193
318180899140,7cyclictest0-21swapper/320:39:193
318180899134,5cyclictest0-21swapper/321:09:253
318180899133,6cyclictest0-21swapper/300:39:223
318180899132,7cyclictest0-21swapper/321:24:203
318180899132,7cyclictest0-21swapper/300:29:163
318180899132,7cyclictest0-21swapper/300:29:163
318180899131,8cyclictest0-21swapper/323:24:293
318180899131,8cyclictest0-21swapper/322:59:103
318180899131,8cyclictest0-21swapper/322:59:103
318180899131,8cyclictest0-21swapper/322:34:113
318180899131,8cyclictest0-21swapper/321:49:243
318180899131,8cyclictest0-21swapper/300:25:013
318180899131,7cyclictest0-21swapper/323:54:143
318180899131,7cyclictest0-21swapper/321:29:293
318180899131,7cyclictest0-21swapper/320:14:233
318180899131,6cyclictest0-21swapper/320:29:213
318180899131,6cyclictest0-21swapper/320:24:193
318180899131,6cyclictest0-21swapper/319:14:183
318180899121,7cyclictest0-21swapper/319:59:263
318180899121,6cyclictest0-21swapper/323:49:103
318180899121,6cyclictest0-21swapper/320:49:213
318180899121,6cyclictest0-21swapper/320:04:233
318180899121,6cyclictest0-21swapper/319:54:103
318180899120,6cyclictest0-21swapper/320:09:213
318180899111,6cyclictest0-21swapper/321:04:103
318180899111,6cyclictest0-21swapper/320:34:233
318180899111,6cyclictest0-21swapper/320:34:233
318180899111,6cyclictest0-21swapper/319:39:203
318180899111,5cyclictest0-21swapper/319:49:133
318180899101,6cyclictest0-21swapper/320:59:133
318180899101,6cyclictest0-21swapper/320:19:143
318180899101,5cyclictest0-21swapper/319:35:023
31818089991,6cyclictest0-21swapper/319:10:023
31818089991,5cyclictest0-21swapper/319:44:153
31818089991,5cyclictest0-21swapper/319:44:153
31818089991,5cyclictest0-21swapper/319:24:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional