You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 07:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Thu Jul 23, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3885669995547,6cyclictest4093596-21latency_hist22:45:011
3885667995346,5cyclictest4163282-21latency_hist23:55:010
3885669994739,6cyclictest4153624-21kworker/u8:2+flush-8:000:02:551
3885677994440,3cyclictest3838737-21kworker/u8:2+events_unbound21:37:502
3885669994434,7cyclictest4062693-21latency_hist22:05:011
3885667994439,4cyclictest3907536-21kworker/u8:1+flush-8:019:42:430
3885677994235,5cyclictest3972618-21apt-get20:35:002
3885669994236,4cyclictest3907536-21kworker/u8:1+flush-8:020:32:551
3885677994136,4cyclictest4183721-21kworker/u8:1+events_unbound00:37:562
3885677994136,4cyclictest4183721-21kworker/u8:1+events_unbound00:32:472
3885677994036,3cyclictest3972446-21kworker/u8:3+events_unbound22:17:502
3885677994036,3cyclictest3838737-21kworker/u8:2+flush-8:021:13:022
3885667994033,5cyclictest278-21systemd-journal20:32:400
3885667994031,7cyclictest278-21systemd-journal22:55:000
3885677993935,3cyclictest3838737-21kworker/u8:2+flush-8:021:22:432
3885677993934,4cyclictest4103715-21kworker/u8:0+flush-8:022:57:562
3885677993934,4cyclictest4023079-21kworker/u8:1+events_unbound21:27:462
3885677993934,4cyclictest4023079-21kworker/u8:1+events_unbound21:27:462
3885677993834,3cyclictest4103715-21kworker/u8:0+events_unbound23:12:422
3885669993833,4cyclictest3972446-21kworker/u8:3+flush-8:021:12:511
3885667993832,4cyclictest4555-21cron00:35:010
3885667993829,7cyclictest4148439-21latency_hist23:40:000
38809782384,9sleep10-21swapper/119:06:251
3885677993733,3cyclictest3907536-21kworker/u8:1+events_unbound19:47:562
3885667993732,3cyclictest4153624-21kworker/u8:2+events_unbound00:10:000
3885667993727,8cyclictest3913041-21/usr/sbin/munin19:37:480
3885667993726,8cyclictest278-21systemd-journal21:35:000
388566799371,34cyclictest3932437-21munin-run19:55:010
3885677993631,4cyclictest3972446-21kworker/u8:3+events_unbound20:57:562
3885677993628,6cyclictest3942747-21apt-get20:05:002
3885669993630,4cyclictest3932396-21apt-get19:55:001
3885667993632,3cyclictest4138534-21kworker/u8:3+flush-8:000:12:560
3885667993632,3cyclictest4087616-21kworker/u8:1+flush-8:023:22:560
3885667993631,3cyclictest3972446-21kworker/u8:3+flush-8:020:37:500
3885677993531,3cyclictest4153624-21kworker/u8:2+events_unbound23:52:552
3885669993530,4cyclictest3972446-21kworker/u8:3+flush-8:020:42:431
3885669993530,3cyclictest3908826-21irqrtprio19:32:431
3885667993531,3cyclictest3907536-21kworker/u8:1+events_unbound19:32:540
3885677993430,3cyclictest3972446-21kworker/u8:3+events_unbound21:55:022
3885669993429,4cyclictest4153624-21kworker/u8:2+flush-8:000:27:411
3885669993423,8cyclictest278-21systemd-journal23:57:591
3885667993429,4cyclictest4138534-21kworker/u8:3+flush-8:000:22:400
3885667993429,4cyclictest4103715-21kworker/u8:0+flush-8:023:57:420
3885667993428,4cyclictest3943428-21apt-get20:07:370
3885667993427,5cyclictest4173018-21grep00:02:570
3885677993328,3cyclictest4087616-21kworker/u8:1+events_unbound23:45:012
3885667993328,4cyclictest3907536-21kworker/u8:1+flush-8:019:57:410
3885667993326,5cyclictest4183691-21latency_hist00:20:020
3885667993326,3cyclictest3982832-21munin-run20:45:020
3885667993323,8cyclictest4091602-21ntpq22:32:490
3885667993319,5cyclictest3838737-21kworker/u8:2+events_unbound19:10:000
3885677993228,3cyclictest3972446-21kworker/u8:3+events_unbound22:52:492
3885677993228,3cyclictest3069895-21kworker/u8:3+events_unbound19:27:502
3885669993227,4cyclictest4183721-21kworker/u8:1+flush-8:000:32:401
3885669993227,3cyclictest4003861-21irqcore21:07:431
3885669993226,4cyclictest4103715-21kworker/u8:0+flush-8:023:32:551
3885669993226,4cyclictest3902651-21cron19:25:001
3885667993228,2cyclictest4123482-21latency_hist23:15:010
3885667993227,3cyclictest4183721-21kworker/u8:1+events_unbound00:37:440
3885667993226,4cyclictest3988373-21apt-get20:52:340
3885667993225,5cyclictest4077938-21apt-get22:22:360
3885669993125,4cyclictest3907536-21kworker/u8:1+flush-8:019:37:411
3885669993122,6cyclictest4133369-21latency_hist23:25:001
3885669993120,8cyclictest278-21systemd-journal20:12:491
3885667993126,4cyclictest3838737-21kworker/u8:2+flush-8:019:27:570
38809222314,10sleep20-21swapper/219:05:422
3885677993024,4cyclictest278-21systemd-journal00:04:092
3885669993025,4cyclictest4138534-21kworker/u8:3+flush-8:023:42:421
3885667993025,3cyclictest4082275-21kworker/u8:0+events_unbound22:27:540
3885667993024,4cyclictest4022616-21latency_hist21:25:000
3885667993023,5cyclictest3967680-21latency_hist20:30:010
3885667993023,5cyclictest3967680-21latency_hist20:30:000
3885667993021,7cyclictest4042059-21ntpq21:42:530
3885667993020,8cyclictest4011944-21ntpq21:12:550
41720912290,3sleep3577-21kworker/3:2+events00:02:483
40305142290,2sleep3481ktimers/321:32:473
3885677992925,3cyclictest3972446-21kworker/u8:3+events_unbound22:12:492
3885677992925,3cyclictest3838737-21kworker/u8:2+events_unbound19:10:032
3885669992925,2cyclictest4184739-21irqcore00:22:421
3885669992924,4cyclictest3838737-21kworker/u8:2+flush-8:022:52:551
3885669992924,4cyclictest3838737-21kworker/u8:2+flush-8:020:28:101
3885669992924,4cyclictest3838737-21kworker/u8:2+flush-8:020:28:101
3885669992924,3cyclictest3973614-21fschecks_time20:37:421
3885669992921,6cyclictest3957799-21cron20:20:001
3885667992924,4cyclictest4028630-21kworker/u8:4+flush-8:021:52:560
3885667992924,3cyclictest4028630-21kworker/u8:4+flush-8:021:57:520
3885667992924,3cyclictest3948897-21irqcore20:12:450
3885667992923,4cyclictest4103715-21kworker/u8:0+events_unbound23:27:440
3885667992923,4cyclictest3838737-21kworker/u8:2+flush-8:019:17:520
3885667992918,8cyclictest4192854-21ntpq00:27:490
3885677992824,3cyclictest4028630-21kworker/u8:4+events_unbound21:32:482
3885677992824,3cyclictest3972446-21kworker/u8:3+events_unbound20:39:502
3885669992825,2cyclictest4037795-21systemd-run21:40:011
3885669992824,3cyclictest4183720-21true00:20:001
3885669992824,3cyclictest4138509-21cron23:30:001
3885669992824,3cyclictest3838737-21kworker/u8:2+events_unbound19:45:011
3885669992823,4cyclictest402-21systemd-logind21:32:361
3885669992823,4cyclictest3907536-21kworker/u8:1+events_unbound20:22:411
3885669992823,3cyclictest4138534-21kworker/u8:3+events_unbound23:37:401
3885669992823,3cyclictest4017644-21idleruntime-cro21:20:011
3885669992823,3cyclictest3938982-21irqrtprio20:02:451
3885669992822,5cyclictest3069895-21kworker/u8:3+flush-8:019:12:531
3885669992820,6cyclictest3892906-21/usr/sbin/munin19:17:481
3885667992824,3cyclictest4103715-21kworker/u8:0+events_unbound23:32:410
3885667992822,5cyclictest3972446-21kworker/u8:3+flush-8:021:47:500
3885667992822,4cyclictest3942747-21apt-get20:05:000
3885667992821,5cyclictest3972446-21kworker/u8:3+events_unbound22:02:510
3885667992819,6cyclictest3927453-21latency_hist19:50:010
3885677992723,3cyclictest4038440-21cpuspeed_turbos21:42:412
3885677992723,2cyclictest4058793-21sed22:02:462
3885677992722,3cyclictest3927937-21aten2_r5power_c19:52:402
3885677992717,8cyclictest4053000-21cp21:55:012
3885669992723,3cyclictest4159144-21cut23:52:421
3885669992723,2cyclictest4078666-21irqrtprio22:22:431
3885669992722,4cyclictest4103715-21kworker/u8:0+flush-8:023:07:431
3885669992722,3cyclictest4063352-21cpuspeed_turbos22:07:391
388566799272,24cyclictest4022761-21kworker/0:1+events22:18:390
3885667992722,3cyclictest4103715-21kworker/u8:0+flush-8:023:02:450
3885667992721,4cyclictest4103715-21kworker/u8:0+flush-8:023:17:500
3885667992721,4cyclictest4103715-21kworker/u8:0+events_unbound23:47:560
3885667992718,7cyclictest278-21systemd-journal21:15:490
3885677992623,2cyclictest3978616-21fschecks_time20:42:432
3885677992622,3cyclictest4043528-21fschecks_time21:47:432
3885677992621,4cyclictest4092074-21unixbench_multi22:32:562
3885677992621,4cyclictest3838737-21kworker/u8:2+events_unbound20:27:552
3885677992621,4cyclictest3838737-21kworker/u8:2+events_unbound20:27:552
3885677992621,3cyclictest4103715-21kworker/u8:0+events_unbound00:05:122
3885677992617,1cyclictest421rcuog/219:32:432
3885669992622,2cyclictest0-21swapper/100:37:381
3885669992621,3cyclictest4020279-21kthreadcore21:22:471
3885667992622,3cyclictest3838737-21kworker/u8:2+events_unbound19:24:310
3885667992621,3cyclictest4072368-21sed22:12:570
3885667992620,4cyclictest4114089-21aten2_r5power_c23:07:400
3885667992620,4cyclictest4093825-21gpgv22:45:020
3885667992620,4cyclictest3972446-21kworker/u8:3+events_unbound22:07:560
3885667992615,9cyclictest278-21systemd-journal21:02:460
3885677992520,3cyclictest4128125-21unixbench_multi23:17:562
3885677992520,3cyclictest4083190-21fschecks_time22:27:422
3885669992521,3cyclictest4092371-21systemd-run22:35:001
3885669992521,2cyclictest4104871-21irqrtprio22:57:451
3885669992521,2cyclictest0-21swapper/120:52:411
3885669992520,3cyclictest4158076-21unixbench_multi23:47:581
3885669992520,3cyclictest4093005-21systemd-run22:40:011
3885669992519,4cyclictest868-21mta-sts-daemon23:02:501
3885669992519,4cyclictest4123984-21aten2_r5power_e23:17:401
3885669992519,4cyclictest278-21systemd-journal22:15:501
3885667992520,3cyclictest0-21swapper/020:17:440
3885667992518,5cyclictest4023079-21kworker/u8:1+flush-8:021:30:000
3885667992518,5cyclictest4023079-21kworker/u8:1+flush-8:021:30:000
3885667992512,9cyclictest3958283-21sshd20:22:380
388566799250,24cyclictest4022761-21kworker/0:1+events21:39:230
3885677992421,2cyclictest4139257-21fschecks_count23:32:422
3885677992420,3cyclictest3918363-21fschecks_time19:42:432
3885677992420,3cyclictest3907536-21kworker/u8:1+events_unbound19:37:532
3885677992419,3cyclictest3988591-21tr20:52:372
3885677992413,9cyclictest3897557-21cp19:20:002
3885669992421,2cyclictest3942766-21gpgv20:05:011
3885669992420,2cyclictest0-21swapper/121:42:571
3885669992420,2cyclictest0-21swapper/119:57:531
3885669992420,2cyclictest0-21swapper/100:12:581
3885669992419,3cyclictest4177907-21unixbench_multi00:07:571
3885669992419,3cyclictest3987694-21unixbench_multi20:47:571
3885669992418,4cyclictest4043468-21fschecks_count21:47:411
3885669992418,4cyclictest3972446-21kworker/u8:3+events_unbound20:57:521
3885677992320,2cyclictest4184439-21cut00:22:402
3885677992318,3cyclictest3982792-21apt-get20:45:042
388567799231,7cyclictest3952898-21taskset20:15:002
3885677992311,3cyclictest161rcu_preempt20:00:012
3885677992310,2cyclictest161rcu_preempt23:57:402
3885669992319,2cyclictest3999238-21kthreadcore21:02:451
3885669992319,2cyclictest0-21swapper/122:12:441
3885669992319,2cyclictest0-21swapper/121:50:501
3885669992317,3cyclictest0-21swapper/119:50:021
3885669992315,3cyclictest0-21swapper/122:47:381
3885669992312,6cyclictest868-21mta-sts-daemon19:27:551
388566799232,20cyclictest4022761-21kworker/0:1+events22:56:550
3885667992320,2cyclictest4151059-21kthreadcore23:42:460
3885667992319,3cyclictest3972446-21kworker/u8:3+events_unbound20:57:580
3885667992312,8cyclictest278-21systemd-journal22:45:000
3885677992219,2cyclictest4128975-21aten2_r5power_v23:22:412
3885677992217,3cyclictest4114291-21cpuspeed_turbos23:07:412
3885677992216,4cyclictest21170-21cron22:50:002
3885669992218,2cyclictest0-21swapper/122:27:401
3885669992217,3cyclictest0-21swapper/123:12:481
3885669992216,4cyclictest4053961-21irqrtprio21:57:441
388566799226,15cyclictest3952894-21kworker/0:1+events20:46:430
3885667992220,1cyclictest3069895-21kworker/u8:3+events_unbound19:12:580
3885667992218,3cyclictest3972446-21kworker/u8:3+events_unbound21:07:540
388567799219,2cyclictest161rcu_preempt23:02:462
3885677992118,2cyclictest3943515-21aten2_r5power_p20:07:392
3885677992116,3cyclictest4153387-21turbostat.cron23:44:592
3885677992113,6cyclictest4006266-21tr21:07:462
388567799203,3cyclictest161rcu_preempt20:17:392
388567799203,2cyclictest161rcu_preempt00:12:522
3885677992017,2cyclictest0-21swapper/222:40:002
3885677992016,3cyclictest3838737-21kworker/u8:2+events_unbound22:40:022
388567799200,11cyclictest371rcuc/223:37:482
3885669992016,2cyclictest0-21swapper/121:27:431
3885669992016,2cyclictest0-21swapper/121:27:421
388567799192,4cyclictest161rcu_preempt00:20:002
388567799192,12cyclictest4133377-21apt-get23:25:032
3885677991914,4cyclictest3838737-21kworker/u8:2+events_unbound21:15:172
388567799185,2cyclictest421rcuog/200:27:402
388567799182,10cyclictest4063135-21apt-get22:07:382
3885677991814,3cyclictest3972446-21kworker/u8:3+events_unbound21:00:232
3885677991814,3cyclictest3907536-21kworker/u8:1+flush-8:020:20:562
3885677991810,2cyclictest161rcu_preempt22:22:432
388567799180,10cyclictest3898495-21uniq19:22:422
38808832184,8sleep30-21swapper/319:05:113
3885667991711,4cyclictest4092971-21munin-run22:40:000
40878372162,8sleep30-21swapper/322:32:383
41012242142,8sleep30-21swapper/322:52:493
39827712142,7sleep30-21swapper/320:45:013
41434852132,6sleep30-21swapper/323:35:003
41326252132,7sleep30-21swapper/323:22:533
40819362133,6sleep30-21swapper/322:22:553
40627132132,6sleep30-21swapper/322:05:013
40554552132,7sleep30-21swapper/321:57:473
40218992132,7sleep30-21swapper/321:22:543
40070962134,5sleep30-21swapper/321:07:523
39771882132,7sleep30-21swapper/320:37:553
39596802132,7sleep30-21swapper/320:22:463
39569162132,7sleep30-21swapper/320:17:493
39522432132,7sleep30-21swapper/320:12:563
39168022132,7sleep30-21swapper/319:37:503
39031042132,7sleep30-21swapper/319:27:383
50512122,6sleep30-21swapper/300:35:233
41936362122,5sleep30-21swapper/300:30:013
41797192122,6sleep30-21swapper/300:12:483
41636072123,6sleep30-21swapper/323:56:233
41560332122,6sleep30-21swapper/323:47:483
41526352122,6sleep30-21swapper/323:42:513
41444642122,6sleep30-21swapper/323:37:453
41239312122,6sleep30-21swapper/323:17:393
41199792121,6sleep30-21swapper/323:12:463
41177992122,6sleep30-21swapper/323:07:533
41113182122,7sleep30-21swapper/323:02:473
40730412123,6sleep30-21swapper/322:17:413
40582062122,6sleep30-21swapper/322:02:413
40389502122,6sleep30-21swapper/321:42:473
40241852122,6sleep30-21swapper/321:27:463
40241852122,6sleep30-21swapper/321:27:463
40020882122,6sleep30-21swapper/321:02:513
39936132122,6sleep30-21swapper/320:57:453
39676912122,6sleep30-21swapper/320:30:013
39641992122,6sleep30-21swapper/320:27:443
39641992122,6sleep30-21swapper/320:27:443
39422572122,6sleep30-21swapper/320:02:573
39330762122,6sleep30-21swapper/319:57:393
39187032122,6sleep30-21swapper/319:42:463
39085962122,6sleep30-21swapper/319:32:433
38968762122,6sleep30-21swapper/319:17:553
41890612112,5sleep30-21swapper/300:27:363
41879682112,5sleep30-21swapper/300:22:513
41833222112,6sleep30-21swapper/300:15:293
41775442112,6sleep30-21swapper/300:07:543
41590412112,5sleep30-21swapper/323:52:423
41347622112,5sleep30-21swapper/323:27:463
41040702112,5sleep30-21swapper/322:57:013
40947752112,6sleep30-21swapper/322:47:423
40927492112,6sleep30-21swapper/322:37:233
40829352112,5sleep30-21swapper/322:27:403
40704052112,5sleep30-21swapper/322:12:483
40479352113,5sleep30-21swapper/321:50:573
40436472111,6sleep30-21swapper/321:47:443
40330822112,6sleep30-21swapper/321:35:533
40169022112,6sleep30-21swapper/321:17:513
39881682112,6sleep30-21swapper/320:50:153
39871612112,5sleep30-21swapper/320:47:503
39427712110,6sleep30-21swapper/320:05:013
39229452112,5sleep30-21swapper/319:47:323
38981602112,5sleep30-21swapper/319:22:403
38863012112,6sleep30-21swapper/319:12:413
40933372102,5sleep30-21swapper/322:41:483
40079912102,5sleep30-21swapper/321:10:053
39276802102,5sleep30-21swapper/319:50:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional