You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 03:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri Nov 14, 2025 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
34147762660,7sleep30-21swapper/322:36:253
3208947996651,12cyclictest3259409-21ntpq19:56:291
3208943996054,5cyclictest3369462-21kworker/u8:2+flush-8:022:36:370
3208943995951,5cyclictest3063011-21kworker/u8:2+events_unbound21:21:330
32475002570,2sleep3586-21kworker/3:2-events19:46:253
32475002570,2sleep3586-21kworker/3:2-events19:46:253
3208943995747,7cyclictest3369462-21kworker/u8:2+events_unbound00:06:360
3208943995650,4cyclictest3354185-21kworker/u8:3+flush-8:021:46:330
3208943995646,7cyclictest3309701-21apt-get20:50:010
3208950995345,6cyclictest3378278-21ntpq21:56:292
3208947995344,6cyclictest3379131-21latency_hist22:00:011
3208947995244,5cyclictest3518165-21latency_hist00:20:011
33764902510,3sleep00-21swapper/021:56:250
33309422510,5chrt0-21swapper/321:11:253
3208947995131,17cyclictest3409141-21apt-get22:31:161
3208943995144,5cyclictest3523096-21apt-get00:25:010
3208947994927,20cyclictest282-21systemd-journal23:45:001
3208943994842,4cyclictest3063011-21kworker/u8:2+events_unbound19:56:330
3208943994838,7cyclictest3384058-21latency_hist22:05:000
3208943994838,7cyclictest3384058-21latency_hist22:05:000
3208947994729,16cyclictest3221708-21irqcore19:21:241
3208943994742,4cyclictest2854935-21kworker/u8:4+flush-8:020:06:360
3208943994738,7cyclictest3422764-21ntpq22:41:300
3208943994738,6cyclictest3463321-21latency_hist23:25:010
3208943994735,9cyclictest282-21systemd-journal19:25:000
3208943994639,5cyclictest3478536-21apt-get23:41:180
3208943994636,7cyclictest3235570-21latency_hist19:35:000
3208947994527,16cyclictest3334676-21systemd-run21:15:001
3208947994427,14cyclictest282-21systemd-journal19:45:441
3208947994427,14cyclictest282-21systemd-journal19:45:441
3208943994440,3cyclictest3170818-21kworker/u8:0+flush-8:019:15:060
3208943994434,8cyclictest3358500-21ntpq21:36:280
3208947994324,5cyclictest282-21systemd-journal22:36:451
3208943994338,4cyclictest3063011-21kworker/u8:2+flush-8:019:41:340
3208943994338,3cyclictest3235519-21kworker/u8:1+flush-8:020:21:300
3208947994234,6cyclictest3290288-21apt-get20:31:181
3208947994222,7cyclictest3323904-21ntpq21:01:321
3208943994237,4cyclictest3063011-21kworker/u8:2+events_unbound21:31:390
3208943994236,4cyclictest3329922-21kworker/u8:1+flush-8:022:11:300
3208943994232,8cyclictest3437852-21ntpq22:56:320
3208947994134,5cyclictest3063011-21kworker/u8:2+flush-8:021:21:361
3208947994133,5cyclictest3349774-21/usr/sbin/munin21:31:311
3208947994123,5cyclictest282-21systemd-journal20:20:451
3208943994136,4cyclictest3063011-21kworker/u8:2+flush-8:020:01:200
3208943994136,3cyclictest3398334-21kworker/u8:3+flush-8:000:16:240
3208943994134,5cyclictest3329922-21kworker/u8:1+flush-8:023:26:340
34693212400,2sleep3586-21kworker/3:2-events23:31:263
3208950994030,8cyclictest3503149-21cp00:05:002
3208947994032,6cyclictest3325075-21apt-get21:06:161
3208947994031,6cyclictest3497983-21latency_hist00:00:011
3208947994017,8cyclictest282-21systemd-journal00:35:001
3208943994031,7cyclictest282-21systemd-journal23:10:000
3208950993931,6cyclictest3498020-21awk23:59:592
3208947993922,15cyclictest3534739-21runrttasks00:36:281
3208943993930,7cyclictest3369579-21apt-get21:51:170
3208943993929,8cyclictest3345109-21diskstats21:26:190
3208943993913,22cyclictest3483147-21kworker/0:0+events23:56:180
3208947993831,5cyclictest3423967-21apt-get22:46:181
3208947993825,10cyclictest282-21systemd-journal19:20:001
3208943993833,4cyclictest3063011-21kworker/u8:2+flush-8:020:31:410
3208943993831,4cyclictest3354185-21kworker/u8:3+events_unbound21:41:380
3208947993729,6cyclictest3270482-21/usr/sbin/munin20:11:341
3208943993731,5cyclictest3235519-21kworker/u8:1+flush-8:020:11:350
3208943993632,3cyclictest3063011-21kworker/u8:2+events_unbound21:16:200
3208943993631,4cyclictest2854935-21kworker/u8:4+flush-8:019:51:220
3208943993630,4cyclictest3329922-21kworker/u8:1+flush-8:022:31:210
3208943993626,8cyclictest3229765-21ntpq19:26:280
3208943993626,7cyclictest3245823-21/usr/sbin/munin19:46:280
3208943993626,7cyclictest3245823-21/usr/sbin/munin19:46:280
3208950993529,5cyclictest3063011-21kworker/u8:2+flush-8:021:06:332
3208947993529,4cyclictest3235571-21munin-plugin-st19:35:001
3208947993528,5cyclictest3305223-21grep20:46:211
3208947993527,6cyclictest282-21systemd-journal20:30:011
3208947993526,7cyclictest3298961-21ntpq20:36:351
3208943993531,3cyclictest3369462-21kworker/u8:2+events_unbound23:51:250
3208943993529,4cyclictest3398334-21kworker/u8:3+flush-8:023:40:000
320894399350,33cyclictest3458433-21unin-run23:20:000
3208947993429,4cyclictest3398334-21kworker/u8:3+flush-8:000:05:001
3208943993431,2cyclictest3063011-21kworker/u8:2+events_unbound20:56:340
3208943993430,3cyclictest3398334-21kworker/u8:3+events_unbound22:51:410
3208943993430,3cyclictest3369462-21kworker/u8:2+flush-8:022:21:360
3208943993430,3cyclictest2854935-21kworker/u8:4+events_unbound20:36:340
3208943993429,4cyclictest3369462-21kworker/u8:2+events_unbound00:11:410
3208943993429,3cyclictest3369462-21kworker/u8:2+events_unbound23:31:270
3208943993424,5cyclictest3369462-21kworker/u8:2+events_unbound23:11:310
3208950993323,8cyclictest3523086-21awk00:25:002
3208943993328,4cyclictest3324750-21kworker/u8:0+events_unbound21:11:190
3208943993327,4cyclictest3369462-21kworker/u8:2+events_unbound23:01:280
3208943993326,5cyclictest3235519-21kworker/u8:1+events_unbound20:20:000
3208943993326,5cyclictest3170818-21kworker/u8:0+events_unbound19:11:350
3208943993227,4cyclictest3369462-21kworker/u8:2+flush-8:000:01:350
3208943993226,4cyclictest3398334-21kworker/u8:3+flush-8:000:31:370
3208943993226,4cyclictest3329922-21kworker/u8:1+flush-8:022:06:310
3208947993127,2cyclictest3520673-21kthreadcore00:21:291
3208947993126,3cyclictest3329922-21kworker/u8:1+events_unbound22:16:251
3208947993125,5cyclictest3398334-21kworker/u8:3+events_unbound23:06:251
3208947993123,6cyclictest3226070-21apt-get19:26:151
3208947993122,7cyclictest3512345-21ntpq00:11:321
3208947993122,7cyclictest3402997-21ntpq22:21:291
3208943993127,3cyclictest3369462-21kworker/u8:2+flush-8:022:46:300
3208943993127,3cyclictest3235519-21kworker/u8:1+events_unbound20:26:380
3208943993127,3cyclictest3063011-21kworker/u8:2+flush-8:021:06:350
3208943993126,4cyclictest3329922-21kworker/u8:1+events_unbound22:16:210
3208950993024,4cyclictest3309952-21gpgv20:50:012
3208947993017,10cyclictest282-21systemd-journal20:42:451
3208943993026,3cyclictest3533447-21kworker/u8:0+flush-8:000:36:260
3208943993025,3cyclictest3304771-21latency_hist20:45:010
3208943993024,4cyclictest3369462-21kworker/u8:2+events_unbound23:46:350
3208943993022,6cyclictest2854935-21kworker/u8:4+events_unbound20:51:350
3208943993021,7cyclictest3239681-21ntpq19:36:280
3208950992923,4cyclictest3259911-21unixbench_multi19:56:362
3208950992921,6cyclictest282-21systemd-journal00:35:002
3208947992925,2cyclictest3251481-21irqrtprio19:51:251
3208943992925,3cyclictest2854935-21kworker/u8:4+events_unbound21:01:250
3208950992819,6cyclictest282-21systemd-journal23:35:002
3208947992824,3cyclictest3454109-21cut23:16:211
3208947992823,3cyclictest3389034-21systemd-run22:10:001
3208947992821,5cyclictest282-21systemd-journal23:25:001
3208943992826,1cyclictest3482847-21kworker/u8:1+events_unbound00:20:080
3208950992722,3cyclictest3349136-21unixbench_singl21:26:352
3208950992721,4cyclictest3487743-21unixbench_multi23:46:372
3208947992721,4cyclictest0-21swapper/120:06:261
3208947992720,5cyclictest3467371-21grep23:26:321
3208947992719,6cyclictest282-21systemd-journal00:30:001
3208950992623,2cyclictest3478176-21basename23:40:002
3208950992622,2cyclictest3221406-21fschecks_count19:21:222
3208950992621,3cyclictest3462944-21unixbench_multi23:21:362
3208947992622,3cyclictest3438660-21munin-plugin-st23:00:001
3208947992622,3cyclictest18912-21cron20:04:591
3208943992622,3cyclictest3369462-21kworker/u8:2+events_unbound22:26:210
3208950992522,2cyclictest3467906-21unixbench_multi23:26:402
3208950992521,2cyclictest3300234-21aten2_r5power_e20:41:202
3208950992520,3cyclictest3429622-21sed22:51:262
3208947992521,2cyclictest3280044-21cron20:20:001
3208947992521,2cyclictest0-21swapper/123:36:351
3208947992521,2cyclictest0-21swapper/123:01:191
3208947992520,4cyclictest3063011-21kworker/u8:2+flush-8:019:41:261
3208947992520,3cyclictest3487673-21wc23:46:371
3208947992520,3cyclictest282-21systemd-journal23:15:001
3208947992519,4cyclictest3433461-21latency_hist22:55:011
320869822510,9sleep10-21swapper/119:09:321
3208950992421,2cyclictest3364674-21aten2_r5power_c21:46:172
3208947992420,3cyclictest3423304-21unixbench_singl22:41:371
3208947992420,2cyclictest3214052-21systemd19:15:011
3208947992418,4cyclictest3339291-21unixbench_singl21:16:381
3208947992418,4cyclictest18912-21cron21:40:001
3208947992415,7cyclictest282-21systemd-journal22:15:011
3208947992414,8cyclictest3318937-21ntpq20:56:281
32085552242,16sleep00-21swapper/019:07:430
320895099233,14cyclictest3389356-21apt-get22:11:192
3208950992320,2cyclictest3513571-21aten2_r5power_c00:16:162
320895099231,10cyclictest32934632kthreadcore20:31:292
3208947992319,3cyclictest610-21sshd22:26:441
3208950992219,2cyclictest3330152-21aten2_r5power_v21:11:192
3208950992219,2cyclictest3241229-21fschecks_time19:41:212
3208950992218,2cyclictest3512871-21unixbench_singl00:11:392
3208950992218,2cyclictest3453853-21aten2_r5power_e23:16:182
3208950992218,2cyclictest0-21swapper/219:20:002
3208947992217,3cyclictest0-21swapper/100:06:271
3208950992117,2cyclictest0-21swapper/222:16:272
3208947992117,2cyclictest0-21swapper/121:41:221
3208947992115,4cyclictest0-21swapper/121:51:291
3208947992115,4cyclictest0-21swapper/120:51:201
320895099202,13cyclictest3478536-21apt-get23:41:202
3208950992016,3cyclictest3324413-21unixbench_singl21:01:392
3208950992016,3cyclictest3275548-21aten2_r5power_p20:16:212
3208950992015,4cyclictest3369462-21kworker/u8:2+flush-8:022:26:352
3208950992011,7cyclictest3318040-21tr20:56:262
3208947992014,4cyclictest0-21swapper/121:46:331
32084672204,9sleep20-21swapper/219:06:362
320895099199,8cyclictest282-21systemd-journal00:08:452
3208950991916,2cyclictest3265657-21aten2_r5power_p20:06:212
3208950991915,2cyclictest0-21swapper/223:01:232
3208950991915,2cyclictest0-21swapper/222:06:192
3208950991914,3cyclictest0-21swapper/219:40:002
320895099191,13cyclictest3230983-21apt-get19:31:182
3208950991910,6cyclictest3355866-21tr21:36:242
320895099190,4cyclictest3340095-21cpu21:21:192
3208947991915,2cyclictest0-21swapper/123:55:001
3208947991915,2cyclictest0-21swapper/123:31:291
3208947991915,2cyclictest0-21swapper/122:05:001
3208947991915,2cyclictest0-21swapper/122:04:591
3208947991915,2cyclictest0-21swapper/121:26:191
32042452193,8sleep30-21swapper/319:06:173
320895099189,7cyclictest3296348-21tr20:36:302
320895099182,15cyclictest0-21swapper/220:51:372
3208950991815,2cyclictest3523757-21aten2_r5power_p00:26:222
3208950991815,2cyclictest3285407-21aten2_r5power_e20:26:182
3208950991815,2cyclictest3280474-21aten2_r5power_e20:21:162
3208950991815,2cyclictest3260746-21aten2_r5power_v20:01:172
3208950991813,4cyclictest0-21swapper/219:55:002
3208950991811,5cyclictest282-21systemd-journal21:43:392
3208947991812,4cyclictest0-21swapper/119:36:201
320895099178,7cyclictest3446419-21tr23:06:252
3208950991714,2cyclictest0-21swapper/222:45:492
3208950991713,2cyclictest0-21swapper/222:41:192
3208950991713,2cyclictest0-21swapper/219:46:292
3208950991713,2cyclictest0-21swapper/219:46:292
3208950991712,3cyclictest0-21swapper/222:21:252
320895099171,12cyclictest3488428-21apt-get23:51:192
320895099171,12cyclictest3334996-21apt-get21:16:182
320895099162,2cyclictest0-21swapper/222:58:442
320895099162,2cyclictest0-21swapper/221:53:262
3208950991613,2cyclictest0-21swapper/222:30:462
3208950991613,2cyclictest0-21swapper/221:31:052
3208950991613,2cyclictest0-21swapper/200:36:262
3208950991612,2cyclictest0-21swapper/222:36:242
3208950991612,2cyclictest0-21swapper/222:01:382
3208950991612,2cyclictest0-21swapper/222:01:382
3208950991612,2cyclictest0-21swapper/219:15:012
3208950991611,3cyclictest0-21swapper/219:26:242
3208950991510,3cyclictest0-21swapper/223:11:232
3208950991510,3cyclictest0-21swapper/220:11:322
35124582142,8sleep30-21swapper/300:11:363
33633672142,8sleep30-21swapper/321:41:323
33141242142,8sleep30-21swapper/320:51:363
34553302132,7sleep30-21swapper/323:16:263
34497142132,7sleep30-21swapper/323:11:253
34324652132,7sleep30-21swapper/322:51:303
34236042132,7sleep30-21swapper/322:45:003
34001302132,7sleep30-21swapper/322:21:253
32923832133,7sleep30-21swapper/320:31:293
35334892122,6sleep30-21swapper/300:35:443
35326352122,6sleep30-21swapper/300:31:393
35231012122,5sleep30-21swapper/300:25:013
35133212122,6sleep30-21swapper/300:15:013
34972532122,6sleep30-21swapper/323:56:363
34881062122,6sleep30-21swapper/323:50:013
34839302122,6sleep30-21swapper/323:46:243
34633342122,5sleep30-21swapper/323:25:013
34485292122,7sleep30-21swapper/323:10:003
34246792122,6sleep30-21swapper/322:46:293
34086772122,6sleep30-21swapper/322:28:433
33942392122,5sleep30-21swapper/322:16:153
33890362122,6sleep30-21swapper/322:10:013
33788142122,6sleep30-21swapper/321:56:373
33695832122,6sleep30-21swapper/321:51:193
33643152122,5sleep30-21swapper/321:45:013
33573692122,6sleep30-21swapper/321:36:263
33541362122,6sleep30-21swapper/321:32:173
33453612122,6sleep30-21swapper/321:26:233
33395752122,6sleep30-21swapper/321:20:013
33237352122,6sleep30-21swapper/321:01:313
33162832122,6sleep30-21swapper/320:56:253
33039062122,6sleep30-21swapper/320:41:313
32891062123,6sleep30-21swapper/320:26:313
32800512122,5sleep30-21swapper/320:20:013
32793082122,6sleep30-21swapper/320:16:363
32705232122,6sleep30-21swapper/320:11:223
32612182122,6sleep30-21swapper/320:01:233
32306642122,5sleep30-21swapper/319:30:013
32268682122,6sleep30-21swapper/319:26:243
32205002122,6sleep30-21swapper/319:17:013
32096002122,6sleep30-21swapper/319:11:233
35194072112,5sleep30-21swapper/300:21:283
35036012112,6sleep30-21swapper/300:06:153
34984182112,6sleep30-21swapper/300:01:173
34785002112,6sleep30-21swapper/323:41:113
34734672112,5sleep30-21swapper/323:35:453
34627342112,5sleep30-21swapper/323:21:363
34390222112,5sleep30-21swapper/323:01:183
34362702112,5sleep30-21swapper/322:56:283
33844332112,5sleep30-21swapper/322:06:203
33839222112,6sleep30-21swapper/322:03:313
33839222112,6sleep30-21swapper/322:03:313
33399542112,6sleep30-21swapper/321:21:173
33291432112,5sleep30-21swapper/321:06:363
33047892112,5sleep30-21swapper/320:45:013
32952302112,5sleep30-21swapper/320:36:233
32659552112,5sleep30-21swapper/320:06:253
32568812112,5sleep30-21swapper/319:56:263
32529632112,5sleep30-21swapper/319:51:283
32417982112,5sleep30-21swapper/319:41:253
32360062111,6sleep30-21swapper/319:36:193
32213692112,5sleep30-21swapper/319:21:223
34090212102,5sleep30-21swapper/322:30:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional