You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 07:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Mon Jun 29, 2026 00:44:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
35517842780,3chrt0-21swapper/223:30:322
3287229996155,4cyclictest3357920-21kworker/u8:0+flush-8:022:00:300
3287229995950,6cyclictest1704890-21systemd-journal00:30:260
3287232995450,3cyclictest3377826-21kworker/u8:2+flush-8:021:05:311
3287232995348,4cyclictest3377826-21kworker/u8:2+flush-8:022:15:301
3287229995346,5cyclictest3377826-21kworker/u8:2+events_unbound21:35:330
3287229995343,7cyclictest3312898-21ntpq19:30:260
3287232995248,3cyclictest3478410-21kworker/u8:1+flush-8:022:45:321
3287232995247,4cyclictest3308920-21kworker/u8:3+flush-8:020:15:321
35976892510,1chrt3597690-21/usr/sbin/munin00:20:162
3287229995047,2cyclictest3357920-21kworker/u8:0+events_unbound20:35:310
3287232994836,9cyclictest3382502-21ntpq20:40:251
3287232994743,3cyclictest3319034-21kworker/u8:1+flush-8:020:35:311
3287232994636,8cyclictest1704890-21systemd-journal19:45:271
3287232994535,7cyclictest1704890-21systemd-journal21:45:001
3287234994434,8cyclictest1704890-21systemd-journal20:40:012
3287232994440,3cyclictest2994733-21kworker/u8:2+flush-8:020:20:301
3287232994438,5cyclictest3143449-21kworker/u8:4+flush-8:019:10:281
3287232994436,6cyclictest3377826-21kworker/u8:2+flush-8:021:10:281
3287229994440,3cyclictest3319034-21kworker/u8:1+events_unbound20:12:280
32925012430,3chrt201rcuc/019:15:020
328723499438,18cyclictest3508180-21apt-get22:50:142
3287232994339,3cyclictest3601852-21kworker/u8:2+events_unbound00:25:321
3287232994339,3cyclictest3357920-21kworker/u8:0+flush-8:000:10:261
3287232994238,3cyclictest3357920-21kworker/u8:0+flush-8:022:40:311
3287232994235,5cyclictest3433052-21latency_hist21:35:001
3287229994233,6cyclictest3442994-21latency_hist21:45:000
3287232994132,6cyclictest1704890-21systemd-journal19:58:251
3287232994036,3cyclictest3377826-21kworker/u8:2+flush-8:022:30:311
3287232994028,9cyclictest3457090-21ntpq21:55:251
3287229994036,3cyclictest3377826-21kworker/u8:2+events_unbound21:22:540
3287232993931,5cyclictest1704890-21systemd-journal19:35:261
3287232993931,5cyclictest1704890-21systemd-journal19:35:251
3287229993935,3cyclictest3357920-21kworker/u8:0+events_unbound20:34:290
3287232993833,4cyclictest3388044-21kworker/u8:1+flush-8:020:55:321
3287232993832,4cyclictest2994733-21kworker/u8:2+events_unbound20:05:311
3287232993826,9cyclictest1704890-21systemd-journal21:39:271
3287229993834,3cyclictest3388044-21kworker/u8:1+events_unbound20:49:270
3287229993830,6cyclictest3552419-21latency_hist23:35:000
3287229993828,7cyclictest3337920-21ntpq19:55:260
3287232993732,4cyclictest3478410-21kworker/u8:1+flush-8:023:45:281
3287232993731,4cyclictest3552644-21kworker/u8:3+flush-8:000:00:261
3287229993731,4cyclictest1704890-21systemd-journal22:08:010
3287229993731,4cyclictest1704890-21systemd-journal22:08:010
3287229993727,7cyclictest3302983-21ntpq19:20:240
33889542360,3sleep3471rcuc/320:50:173
3287229993632,3cyclictest3478410-21kworker/u8:1+events_unbound22:54:390
3287232993530,4cyclictest2994733-21kworker/u8:2+events_unbound19:25:201
3287232993528,5cyclictest3602203-21latency_hist00:25:011
3287232993528,5cyclictest3319034-21kworker/u8:1+flush-8:020:10:281
3287232993527,6cyclictest1704890-21systemd-journal21:05:011
3287229993532,2cyclictest3493212-21kworker/u8:3+events_unbound22:44:380
3287229993531,3cyclictest3357920-21kworker/u8:0+events_unbound22:48:120
3287232993428,4cyclictest3517792-21latency_hist23:00:001
3287232993427,5cyclictest1704890-21systemd-journal22:05:281
3287232993427,5cyclictest1704890-21systemd-journal22:05:281
3287232993425,7cyclictest3294235-21/usr/sbin/munin19:15:251
3287229993431,2cyclictest3522388-21kworker/u8:2+events_unbound23:11:270
3287229993430,3cyclictest3552644-21kworker/u8:3+events_unbound00:23:330
3287229993429,3cyclictest3587716-21fschecks_time00:10:170
3287229993427,5cyclictest3379499-21irqrtprio20:40:210
3287229993419,2cyclictest161rcu_preempt00:35:340
3287234993322,8cyclictest3577190-21awk00:00:002
3287234993310,6cyclictest3357730-21ntpq20:15:272
3287232993326,5cyclictest3597027-21http00:20:011
3287232993326,5cyclictest3597027-21http00:20:011
3287232993326,5cyclictest3368825-21apt-get20:30:131
3287229993330,2cyclictest3377826-21kworker/u8:2+events_unbound21:14:390
3287229993330,2cyclictest3319034-21kworker/u8:1+events_unbound19:47:340
3287229993330,2cyclictest2994733-21kworker/u8:2+events_unbound19:14:110
3287229993329,3cyclictest3552644-21kworker/u8:3+events_unbound00:02:140
3287229993329,3cyclictest3388044-21kworker/u8:1+events_unbound20:55:560
3287229993329,3cyclictest3357920-21kworker/u8:0+events_unbound22:28:390
3287229993329,3cyclictest3357920-21kworker/u8:0+events_unbound21:48:050
328723499329,11cyclictest3453764-21perl21:55:172
328723499328,12cyclictest3392452-21ntpq20:50:262
3287232993228,3cyclictest3388044-21kworker/u8:1+events_unbound20:50:301
3287232993228,3cyclictest3294854-21kworker/u8:0+events_unbound19:20:271
3287232993228,3cyclictest2994733-21kworker/u8:2+flush-8:019:40:371
3287232993221,8cyclictest1704890-21systemd-journal23:50:251
3287229993229,2cyclictest3522388-21kworker/u8:2+events_unbound23:19:310
3287229993229,2cyclictest3478410-21kworker/u8:1+events_unbound23:53:100
3287229993229,2cyclictest3478410-21kworker/u8:1+events_unbound23:44:590
3287229993229,2cyclictest3478410-21kworker/u8:1+events_unbound00:07:510
3287229993229,2cyclictest3478410-21kworker/u8:1+events_unbound00:07:500
3287229993229,2cyclictest3357920-21kworker/u8:0+events_unbound22:17:360
3287229993229,2cyclictest3357920-21kworker/u8:0+events_unbound21:30:510
3287229993229,2cyclictest3357920-21kworker/u8:0+events_unbound20:51:350
3287229993228,3cyclictest3377826-21kworker/u8:2+events_unbound22:31:360
3287229993228,3cyclictest3367830-21kworker/u8:3+events_unbound20:29:160
3287229993228,3cyclictest3357920-21kworker/u8:0+events_unbound00:28:150
3287229993228,3cyclictest3308920-21kworker/u8:3+events_unbound19:37:240
3287229993228,3cyclictest3308920-21kworker/u8:3+events_unbound19:37:240
3287229993226,4cyclictest3358232-21unixbench_multi20:15:350
3287229993225,5cyclictest1712307-21ntpd21:50:290
3287229993225,2cyclictest3478410-21kworker/u8:1+events_unbound23:24:010
3287229993223,2cyclictest161rcu_preempt20:05:130
3287229993213,5cyclictest0-21swapper/019:40:170
32824012323,9sleep13282427-21apt-get19:05:121
3287234993126,3cyclictest3314201-21aten2_r5power_p19:35:162
3287234993126,3cyclictest3314201-21aten2_r5power_p19:35:152
3287234993122,4cyclictest3532895-21apt-get23:15:132
3287232993128,2cyclictest3339740-21irqcore20:00:201
3287232993128,2cyclictest3339740-21irqcore20:00:201
3287232993127,3cyclictest3478410-21kworker/u8:1+flush-8:023:55:331
3287232993126,4cyclictest3357920-21kworker/u8:0+flush-8:022:25:251
3287232993126,3cyclictest3412443-21kworker/u8:3+events_unbound21:20:181
328722999318,14cyclictest3330605-21cut19:50:210
3287229993128,2cyclictest3357920-21kworker/u8:0+events_unbound20:23:410
3287229993127,3cyclictest3522388-21kworker/u8:2+events_unbound23:03:110
3287229993127,3cyclictest3478410-21kworker/u8:1+events_unbound23:46:280
3287229993127,3cyclictest3478410-21kworker/u8:1+events_unbound23:06:440
3287229993127,3cyclictest3442642-21kworker/u8:1+events_unbound21:56:170
3287229993123,6cyclictest3308486-21unixbench_singl19:25:340
328723499309,5cyclictest3379156-21hddtemp_smartct20:40:172
328723499309,5cyclictest3363890-21apt-get20:25:142
3287234993020,5cyclictest3458328-21apt-get22:00:142
3287234993018,8cyclictest3309878-21/usr/sbin/munin19:30:212
3287234993010,5cyclictest3567567-21apt-get23:50:132
3287232993025,4cyclictest3493212-21kworker/u8:3+events_unbound22:35:211
3287232993023,5cyclictest3319034-21kworker/u8:1+flush-8:019:50:281
3287232993022,6cyclictest3478410-21kworker/u8:1+flush-8:023:40:301
3287232993022,6cyclictest3442642-21kworker/u8:1+flush-8:021:45:281
3287229993027,2cyclictest3522388-21kworker/u8:2+events_unbound23:26:170
3287229993027,2cyclictest3493212-21kworker/u8:3+events_unbound22:57:240
3287229993027,2cyclictest3493212-21kworker/u8:3+events_unbound22:35:560
3287229993027,2cyclictest3412443-21kworker/u8:3+events_unbound21:17:590
3287229993027,2cyclictest3377826-21kworker/u8:2+events_unbound21:01:300
3287229993027,2cyclictest3357920-21kworker/u8:0+events_unbound21:06:200
3287229993026,3cyclictest3377826-21kworker/u8:2+events_unbound22:14:350
3287229993022,6cyclictest3428120-21latency_hist21:30:000
328723499299,5cyclictest3476994-21ntpq22:15:262
3287234992924,3cyclictest3472505-21unixbench_multi22:10:352
3287234992917,8cyclictest3553050-21/usr/sbin/munin23:35:172
3287234992916,10cyclictest3287341-21/usr/sbin/munin19:10:122
3287234992913,13cyclictest3498888-21sed22:40:222
3287232992925,3cyclictest3357920-21kworker/u8:0+events_unbound22:00:311
3287232992922,5cyclictest3518067-21apt-get23:00:111
3287232992921,6cyclictest3551468-21ntpq23:30:241
3287229992926,2cyclictest3552644-21kworker/u8:3+events_unbound23:36:310
3287229992926,2cyclictest3377826-21kworker/u8:2+events_unbound22:23:180
3287229992926,2cyclictest3377826-21kworker/u8:2+events_unbound22:23:170
3287229992926,2cyclictest2994733-21kworker/u8:2+events_unbound20:01:360
3287229992926,2cyclictest2994733-21kworker/u8:2+events_unbound20:01:350
3287229992925,3cyclictest3478410-21kworker/u8:1+events_unbound23:59:330
3287234992824,2cyclictest3562948-21fschecks_time23:45:182
3287234992824,2cyclictest3369175-21fschecks_time20:30:182
3287234992823,4cyclictest3465480-21kthreadcore22:05:242
3287234992823,4cyclictest3465480-21kthreadcore22:05:232
3287234992818,8cyclictest3437543-21grep21:35:342
3287234992816,9cyclictest3503411-21/usr/sbin/munin22:45:152
3287234992816,10cyclictest3478721-21fschecks_time22:20:192
3287234992816,10cyclictest3478721-21fschecks_time22:20:192
3287234992815,8cyclictest3383716-21apt-get20:45:132
3287234992813,12cyclictest3532454-21chrt23:12:542
3287232992824,3cyclictest3552644-21kworker/u8:3+events_unbound00:05:361
3287232992824,3cyclictest3552644-21kworker/u8:3+events_unbound00:05:351
3287232992824,3cyclictest3377826-21kworker/u8:2+flush-8:021:50:271
3287232992823,3cyclictest3388379-21cron20:50:011
3287232992823,3cyclictest3377826-21kworker/u8:2+flush-8:021:25:281
328723499279,4cyclictest3592058-21munin-run00:15:002
3287234992722,3cyclictest3294374-21aten2_r5power_v19:15:162
3287234992718,4cyclictest3299219-21apt-get19:20:132
3287234992715,9cyclictest3413670-21/usr/sbin/munin21:15:152
3287234992713,12cyclictest3542413-21seq23:23:272
3287234992713,11cyclictest3447892-21seq21:49:492
3287232992724,2cyclictest3612295-21cron00:35:001
3287232992723,3cyclictest3493212-21kworker/u8:3+flush-8:022:50:281
3287232992723,3cyclictest3357920-21kworker/u8:0+events_unbound23:05:301
3287232992720,5cyclictest3532895-21apt-get23:15:121
3287232992720,5cyclictest3319034-21kworker/u8:1+events_unbound20:25:321
3287234992622,2cyclictest3592630-21fschecks_time00:15:182
3287234992622,2cyclictest3592630-21fschecks_time00:15:172
3287234992621,3cyclictest3423454-21aten2_r5power_c21:25:162
3287234992617,6cyclictest3513368-21/usr/sbin/munin22:55:162
3287234992617,6cyclictest3394121-21/usr/sbin/munin20:55:192
3287234992617,4cyclictest3398587-21apt-get21:00:142
3287234992616,8cyclictest3546819-21/usr/sbin/munin23:25:322
3287234992616,8cyclictest3492613-21/usr/sbin/munin22:30:332
3287234992616,7cyclictest3362701-21/usr/sbin/munin20:20:272
3287234992616,7cyclictest3324283-21/usr/sbin/munin19:45:172
3287234992616,6cyclictest3483383-21apt-get22:25:132
3287234992615,8cyclictest3344423-21sort20:05:162
3287234992615,8cyclictest3334631-21/usr/sbin/munin19:55:192
3287234992615,8cyclictest3308760-21runrttasks19:28:262
3287234992614,9cyclictest3582713-21fschecks_count00:05:172
3287234992614,9cyclictest3582713-21fschecks_count00:05:162
3287234992614,9cyclictest3408901-21sort21:10:192
3287234992612,12cyclictest3579467-21grep00:00:232
3287234992612,11cyclictest3420075-21cut21:20:232
3287234992612,10cyclictest3602462-21apt-get00:25:132
3287234992611,12cyclictest3350309-21cut20:10:212
3287232992622,2cyclictest3547502-21systemd-run23:30:011
3287232992622,2cyclictest0-21swapper/100:40:011
3287232992620,4cyclictest3527937-21apt-get23:10:131
3287229992621,4cyclictest0-21swapper/000:20:020
3287229992621,4cyclictest0-21swapper/000:20:010
36123242250,7sleep20-21swapper/200:35:002
3287234992517,6cyclictest3442087-21ntpq21:40:272
3287234992517,5cyclictest3557662-21/usr/sbin/munin23:40:172
3287234992516,7cyclictest3428886-21/usr/sbin/munin21:30:202
3287234992516,7cyclictest3322864-21/usr/sbin/munin19:40:272
3287234992516,6cyclictest3493638-21/usr/sbin/munin22:35:192
3287234992513,9cyclictest3448649-21awk21:50:192
3287234992512,11cyclictest3518882-21cut23:00:192
3287232992522,2cyclictest3468981-21irqrtprio22:10:221
32869532253,3sleep00-21swapper/019:09:040
3287234992415,7cyclictest3526776-21/usr/sbin/munin23:05:282
3287234992415,7cyclictest3341318-21ps20:00:232
3287234992415,7cyclictest3341318-21ps20:00:232
3287234992415,6cyclictest3403567-21apt-get21:05:132
3287234992414,7cyclictest3612720-21/usr/sbin/munin00:35:152
3287234992413,8cyclictest3329133-21base6419:50:022
3287232992420,2cyclictest3483144-21systemd-run22:25:001
3287232992420,2cyclictest3483144-21systemd-run22:25:001
3287232992420,2cyclictest0-21swapper/123:40:001
3287232992419,3cyclictest3357920-21kworker/u8:0+events_unbound21:15:151
3287232992317,5cyclictest3478410-21kworker/u8:1+flush-8:023:20:291
3287232992317,4cyclictest3309157-21apt-get19:30:131
3287234992110,7cyclictest3287293-21latency_hist19:10:012
32870132205,8sleep30-21swapper/319:09:533
34035702152,9sleep30-21swapper/321:05:143
35488192140,2sleep3481ktimers/323:30:213
34833572142,8sleep30-21swapper/322:25:133
33229192142,8sleep30-21swapper/319:40:303
35374542132,7sleep30-21swapper/323:18:263
35152912132,7sleep30-21swapper/322:55:223
35032302133,6sleep30-21swapper/322:45:143
34769002132,7sleep30-21swapper/322:15:253
33778352132,7sleep30-21swapper/320:35:333
33606112132,7sleep30-21swapper/320:20:243
36071552122,5sleep30-21swapper/300:30:013
36029502122,6sleep30-21swapper/300:25:203
35983902122,6sleep30-21swapper/300:20:213
35920372123,6sleep30-21swapper/300:15:013
35727492122,6sleep30-21swapper/323:55:173
35680662122,6sleep30-21swapper/323:50:203
35415282122,6sleep30-21swapper/323:20:263
35235162122,6sleep30-21swapper/323:05:213
35180412122,6sleep30-21swapper/323:00:113
34932512122,6sleep30-21swapper/322:35:163
34824692122,6sleep30-21swapper/322:20:353
34824692122,6sleep30-21swapper/322:20:343
34691512122,6sleep30-21swapper/322:10:243
34665852122,6sleep30-21swapper/322:05:253
34665852122,6sleep30-21swapper/322:05:253
34588222122,6sleep30-21swapper/322:00:213
34479332122,5sleep30-21swapper/321:50:003
34446242122,6sleep30-21swapper/321:45:243
34424282122,6sleep30-21swapper/321:40:343
34286292122,6sleep30-21swapper/321:30:193
34198312122,6sleep30-21swapper/321:20:223
34143472122,6sleep30-21swapper/321:15:213
34124202122,6sleep30-21swapper/321:10:323
34032442122,6sleep30-21swapper/321:04:253
33982882122,6sleep30-21swapper/321:00:013
33873072122,6sleep30-21swapper/320:45:233
33785482122,6sleep30-21swapper/320:40:013
33727142122,6sleep30-21swapper/320:30:313
33575262122,6sleep30-21swapper/320:15:263
33487882122,6sleep30-21swapper/320:10:003
33380542122,6sleep30-21swapper/319:55:323
33294672122,6sleep30-21swapper/319:50:183
33240002122,6sleep30-21swapper/319:45:123
32923462122,5sleep30-21swapper/319:15:013
32914242122,6sleep30-21swapper/319:10:323
36129392112,5sleep30-21swapper/300:35:183
35822062112,5sleep30-21swapper/300:05:013
35822062112,5sleep30-21swapper/300:05:003
35774732112,5sleep30-21swapper/300:00:163
35625952112,5sleep30-21swapper/323:45:033
35580352112,5sleep30-21swapper/323:40:203
35526832112,5sleep30-21swapper/323:35:153
35427802112,6sleep30-21swapper/323:25:123
35317212112,5sleep30-21swapper/323:10:273
35086882112,5sleep30-21swapper/322:50:213
34982532112,5sleep30-21swapper/322:40:183
34891682112,5sleep30-21swapper/322:30:213
34579882112,6sleep30-21swapper/321:59:283
34333212112,5sleep30-21swapper/321:35:153
34234282112,6sleep30-21swapper/321:25:173
33674792112,5sleep30-21swapper/320:25:243
33528592112,5sleep30-21swapper/320:10:283
33389032112,5sleep30-21swapper/320:00:013
33389032112,5sleep30-21swapper/320:00:003
33138382112,5sleep30-21swapper/319:35:013
33138382112,5sleep30-21swapper/319:35:013
33095352112,5sleep30-21swapper/319:30:183
33039772112,5sleep30-21swapper/319:25:023
32991862112,6sleep30-21swapper/319:20:083
35922782102,5sleep30-21swapper/300:15:053
35922782102,5sleep30-21swapper/300:15:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional