You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 15:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Wed May 27, 2026 12:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
408087723550,2sleep3481ktimers/308:45:423
3984400996251,8cyclictest4183883-21ntpq10:25:531
3984398995849,6cyclictest4081045-21diskstats08:45:430
3984400995747,8cyclictest20569-21apt-get11:00:011
3984400995649,5cyclictest2615400-21kworker/u8:1+flush-8:008:10:581
838792540,2sleep0161rcu_preempt12:00:500
838792540,2sleep0161rcu_preempt12:00:490
3984398995449,3cyclictest3147316-21kworker/u8:2+events_unbound09:00:570
1026802520,2sleep03984392-21cyclictest12:20:480
3984400995145,4cyclictest15842-21kworker/u8:3+flush-8:011:30:511
3984398995146,4cyclictest3147316-21kworker/u8:2+events_unbound07:10:490
398439899511,47cyclictest1-21systemd09:20:000
3984398995042,6cyclictest20572-21latency_hist11:00:010
3984400994934,12cyclictest1704890-21systemd-journal09:10:011
3984400994836,9cyclictest4014936-21ntpq07:35:531
3984398994842,4cyclictest4015846-21kworker/u8:3+events_unbound08:55:480
3984400994639,5cyclictest4015846-21kworker/u8:3+flush-8:008:01:001
3984400994637,6cyclictest4054585-21ntpq08:15:521
3984398994641,4cyclictest3147316-21kworker/u8:2+flush-8:007:16:030
3984398994641,3cyclictest4104885-21kworker/u8:1+flush-8:010:10:520
3984400994538,5cyclictest90541-21apt-get12:10:401
3984400994533,8cyclictest1704890-21systemd-journal07:11:501
3984398994538,4cyclictest1-21systemd08:15:000
3984400994439,3cyclictest3147316-21kworker/u8:2+flush-8:007:15:441
3984400994437,5cyclictest21126-21apt-get11:00:401
3984398994432,9cyclictest1704890-21systemd-journal09:34:590
3984400994337,4cyclictest1704890-21systemd-journal09:40:501
3984400994336,5cyclictest4015846-21kworker/u8:3+events_unbound08:50:531
3984400994334,7cyclictest110286-21latency_hist12:30:011
3984398994332,6cyclictest4105211-21true09:10:010
3984398994331,10cyclictest4059527-21ntpq08:20:520
3984398994237,3cyclictest4104885-21kworker/u8:1+events_unbound09:20:430
3984398994236,4cyclictest4040687-21latency_hist08:05:000
3984398994233,7cyclictest4128916-21kworker/u8:3+events_unbound09:56:010
3984398994233,6cyclictest4015861-21latency_hist07:40:010
3984400994134,5cyclictest94852-21kworker/u8:2+flush-8:012:30:491
3984398994136,3cyclictest4104885-21kworker/u8:1+events_unbound09:45:520
3984403994034,4cyclictest3979356-21kworker/u8:0+flush-8:007:30:582
3984400994034,5cyclictest4115343-21kworker/u8:0+flush-8:010:35:561
3984400994033,5cyclictest4026137-21apt-get07:50:401
3984398994035,3cyclictest4115343-21kworker/u8:0+flush-8:011:05:440
3984400993933,4cyclictest80024-21kworker/u8:1+events_unbound12:35:521
3984398993933,4cyclictest2615400-21kworker/u8:1+flush-8:007:50:440
3984398993928,9cyclictest80024-21kworker/u8:1+flush-8:012:36:030
3984403993828,8cyclictest95935-21diskstats12:15:432
3984400993832,4cyclictest54811-21kworker/u8:2+events_unbound11:50:501
3984400993829,7cyclictest4115343-21kworker/u8:0+flush-8:011:40:561
3984400993829,6cyclictest4095301-21latency_hist09:00:011
3984398993832,4cyclictest2615400-21kworker/u8:1+flush-8:007:40:440
39840762384,31sleep20-21swapper/207:08:112
3984400993732,4cyclictest4015846-21kworker/u8:3+events_unbound08:39:591
3984400993730,5cyclictest4115343-21kworker/u8:0+flush-8:012:05:561
3984400993724,10cyclictest29811-21ntpq11:05:511
3984398993732,4cyclictest54811-21kworker/u8:2+events_unbound11:55:530
3984400993632,3cyclictest4179667-21awk10:25:001
3984400993632,3cyclictest4179667-21awk10:24:591
3984400993630,4cyclictest2615400-21kworker/u8:1+flush-8:008:05:461
3984398993629,5cyclictest4115343-21kworker/u8:0+events_unbound11:10:510
39841152363,9sleep10-21swapper/107:08:421
3984403993529,4cyclictest54811-21kworker/u8:2+flush-8:011:50:472
3984400993530,4cyclictest3147316-21kworker/u8:2+flush-8:007:21:001
3984400993528,5cyclictest15842-21kworker/u8:3+flush-8:012:20:531
3984400993528,5cyclictest15842-21kworker/u8:3+flush-8:011:25:451
3984400993527,5cyclictest1704890-21systemd-journal09:25:511
3984400993527,5cyclictest1704890-21systemd-journal08:25:511
3984398993531,3cyclictest4104885-21kworker/u8:1+flush-8:010:30:580
3984403993428,4cyclictest95154-21apt-get12:15:012
3984400993430,3cyclictest15842-21kworker/u8:3+flush-8:011:55:581
3984400993429,3cyclictest4115343-21kworker/u8:0+events_unbound10:40:471
3984400993425,6cyclictest1704890-21systemd-journal09:56:511
3984398993430,3cyclictest15842-21kworker/u8:3+events_unbound12:15:520
3984398993430,3cyclictest110284-21apt-get12:30:010
3984398993427,5cyclictest15608-21latency_hist10:55:010
3984400993329,3cyclictest4104885-21kworker/u8:1+events_unbound09:15:531
3984400993327,4cyclictest4105489-21apt-get09:10:391
3984400993326,5cyclictest4104885-21kworker/u8:1+flush-8:009:45:531
3984400993326,5cyclictest15842-21kworker/u8:3+flush-8:011:35:541
3984398993328,3cyclictest60620-21cron11:40:000
3984398993327,4cyclictest15842-21kworker/u8:3+events_unbound11:30:440
3984398993324,7cyclictest4055495-21latency_hist08:20:000
3984400993228,2cyclictest4170724-21irqrtprio10:15:451
3984400993228,2cyclictest4170724-21irqrtprio10:15:451
3984400993227,4cyclictest4104885-21kworker/u8:1+flush-8:010:50:581
3984400993225,5cyclictest4165070-21apt-get10:10:391
3984398993228,3cyclictest3147316-21kworker/u8:2+flush-8:007:46:220
3984398993228,3cyclictest2615400-21kworker/u8:1+events_unbound07:55:570
3984398993227,3cyclictest3147316-21kworker/u8:2+flush-8:008:25:490
3984398993220,9cyclictest1704890-21systemd-journal09:45:000
3984403993125,4cyclictest20611-21gpgv11:00:012
3984403993120,8cyclictest110694-21cat12:30:022
3984400993126,4cyclictest4015846-21kworker/u8:3+events_unbound08:30:451
3984400993126,4cyclictest3147316-21kworker/u8:2+flush-8:007:49:211
3984400993126,3cyclictest70510-21awk11:50:001
3984400993126,3cyclictest401-21dbus-daemon09:05:011
3984400993125,4cyclictest4035757-21sendmail08:00:001
3984398993126,4cyclictest4104885-21kworker/u8:1+flush-8:009:35:530
3984403993023,5cyclictest4104885-21kworker/u8:1+flush-8:009:25:562
3984403993020,8cyclictest769-21awk10:40:012
3984400993026,2cyclictest4085431-21turbostat.cron08:49:591
3984400993024,5cyclictest80024-21kworker/u8:1+flush-8:012:00:531
3984400993024,5cyclictest80024-21kworker/u8:1+flush-8:012:00:521
3984400993024,4cyclictest4104885-21kworker/u8:1+events_unbound09:20:521
3984400993021,1cyclictest161rcu_preempt09:30:431
3984398993026,3cyclictest4104885-21kworker/u8:1+events_unbound09:10:420
3984398993023,5cyclictest94282-21ntpq12:10:540
39839502302,6sleep00-21swapper/007:06:320
3984403992925,2cyclictest4045288-21unixbench_singl08:06:002
3984403992925,2cyclictest36035-21aten2_r5power_p11:15:422
3984403992923,4cyclictest89832-21unixbench_multi12:05:572
3984403992919,7cyclictest11223-21diskstats10:50:422
3984400992925,3cyclictest4128916-21kworker/u8:3+events_unbound10:01:001
3984400992924,4cyclictest4104885-21kworker/u8:1+events_unbound10:30:521
3984400992922,5cyclictest4115343-21kworker/u8:0+flush-8:011:10:511
3984398992924,4cyclictest4128916-21kworker/u8:3+flush-8:010:05:540
3984398992924,4cyclictest4104885-21kworker/u8:1+flush-8:009:25:590
398440399289,3cyclictest40285702cut07:50:492
3984403992824,2cyclictest4020538-21unixbench_singl07:41:012
3984400992824,3cyclictest4115343-21kworker/u8:0+flush-8:011:20:571
3984400992822,4cyclictest4128916-21kworker/u8:3+flush-8:010:05:481
3984400992822,4cyclictest4015846-21kworker/u8:3+flush-8:008:40:551
3984400992822,4cyclictest4015846-21kworker/u8:3+flush-8:008:40:551
3984400992821,5cyclictest4104885-21kworker/u8:1+events_unbound09:50:011
3984398992824,3cyclictest54811-21kworker/u8:2+flush-8:011:50:510
3984398992824,3cyclictest4115343-21kworker/u8:0+events_unbound10:35:540
3984398992823,4cyclictest4115343-21kworker/u8:0+events_unbound11:40:570
3984398992822,5cyclictest4128916-21kworker/u8:3+flush-8:009:50:550
3984398992822,4cyclictest2615400-21kworker/u8:1+flush-8:007:25:510
3984398992821,5cyclictest4079661-21ntpq08:40:560
3984398992821,5cyclictest4079661-21ntpq08:40:560
3984403992722,4cyclictest3147316-21kworker/u8:2+flush-8:009:01:002
3984403992722,3cyclictest4114659-21unixbench_multi09:16:012
398440099277,12cyclictest0-21swapper/107:25:251
3984400992723,3cyclictest4104885-21kworker/u8:1+events_unbound09:35:471
3984400992722,4cyclictest3979356-21kworker/u8:0+flush-8:007:30:501
3984400992720,5cyclictest4060344-40mandb08:23:521
3984400992720,5cyclictest35951-21apt-get11:15:401
3984400992717,8cyclictest1704890-21systemd-journal12:15:511
3984398992723,3cyclictest4115343-21kworker/u8:0+events_unbound10:45:550
3984398992723,3cyclictest15842-21kworker/u8:3+flush-8:011:25:590
3984398992723,3cyclictest15842-21kworker/u8:3+flush-8:011:19:510
3984403992619,5cyclictest4154837-21journal-offline09:59:502
3984398992622,3cyclictest4015846-21kworker/u8:3+flush-8:008:50:570
3984398992622,2cyclictest0-21swapper/008:34:590
3984398992620,4cyclictest4158888-21needreboot10:00:520
3984398992620,4cyclictest3979356-21kworker/u8:0+events_unbound07:30:540
3984403992521,2cyclictest66204-21fschecks_time11:45:442
3984403992521,2cyclictest4115748-21fschecks_time09:20:422
398439899256,3cyclictest161rcu_preempt12:30:460
3984398992522,2cyclictest90264-21systemd-run12:10:000
3984398992520,4cyclictest4184441-21unixbench_singl10:25:590
3984398992520,3cyclictest54811-21kworker/u8:2+events_unbound11:50:000
3984398992516,7cyclictest1704890-21systemd-journal08:10:000
398440399246,3cyclictest39954792switchtime07:15:582
3984403992421,2cyclictest51094-21aten2_r5power_p11:30:432
3984403992421,2cyclictest4136095-21kthreadcore09:40:472
3984403992421,2cyclictest4015478-21unixbench_singl07:36:002
3984403992420,2cyclictest4066206-21fschecks_count08:30:462
3984403992419,4cyclictest4128916-21kworker/u8:3+flush-8:009:30:582
3984403992418,4cyclictest50104-21unixbench_singl11:26:002
3984398992420,3cyclictest4128916-21kworker/u8:3+flush-8:010:20:470
3984398992420,3cyclictest4128916-21kworker/u8:3+flush-8:010:20:470
3984398992415,7cyclictest4170254-21diskstats10:15:420
3984398992415,7cyclictest4170254-21diskstats10:15:410
3984403992320,2cyclictest0-21swapper/209:50:012
3984403992318,4cyclictest0-21swapper/208:40:002
3984403992318,3cyclictest4046023-21aten2_r5power_p08:10:432
3984403992317,4cyclictest10257-21unixbench_multi10:45:592
3984398992319,3cyclictest4115343-21kworker/u8:0+events_unbound11:20:580
3984398992318,3cyclictest0-21swapper/008:40:010
3984398992316,5cyclictest4115343-21kworker/u8:0+flush-8:010:40:520
3984403992219,2cyclictest79966-21unixbench_singl11:55:582
3984403992218,3cyclictest0-21swapper/209:15:002
3984403992218,2cyclictest0-21swapper/210:30:582
3984403992212,7cyclictest1704890-21systemd-journal11:45:002
3984400992215,5cyclictest3147316-21kworker/u8:2+flush-8:007:40:561
742722210,3sleep375-21kworker/3:1+events11:50:493
398440399212,18cyclictest0-21swapper/208:41:002
398440399212,18cyclictest0-21swapper/208:41:002
3984403992117,2cyclictest4061277-21tr08:25:472
3984403992112,7cyclictest4145269-21apt-get09:50:392
3984400992116,4cyclictest4115343-21kworker/u8:0+events_unbound10:45:511
3984398992116,4cyclictest4104885-21kworker/u8:1+flush-8:011:00:490
3984403992017,2cyclictest0-21swapper/212:00:582
3984403992017,2cyclictest0-21swapper/212:00:582
3984403992015,4cyclictest4015846-21kworker/u8:3+flush-8:008:45:452
3984403992015,3cyclictest4160251-21aten2_r5power_v10:05:422
3984403992015,3cyclictest0-21swapper/210:30:002
3984403992011,7cyclictest3985909-21cut07:10:492
398439899203,4cyclictest589-21runrttasks07:20:430
398440399192,16cyclictest0-21swapper/208:55:582
398440399192,16cyclictest0-21swapper/208:15:582
3984403991916,2cyclictest1300-21cpuspeed_turbos10:40:422
3984403991915,3cyclictest4115343-21kworker/u8:0+events_unbound10:15:562
3984403991915,3cyclictest4115343-21kworker/u8:0+events_unbound10:15:562
3984403991911,6cyclictest4134439-21tail09:35:582
3984403991910,6cyclictest120785-21cut12:40:002
398440399188,7cyclictest4179935-21cat10:25:012
398440399188,7cyclictest4179935-21cat10:25:002
3984403991813,3cyclictest4031318-21cpuspeed_turbos07:55:432
3984403991813,3cyclictest0-21swapper/211:10:482
3984403991813,2cyclictest0-21swapper/211:20:472
3984403991810,6cyclictest4100563-21cut09:05:412
3984403991810,6cyclictest4058283-21tr08:20:482
39840772183,9sleep30-21swapper/307:08:123
3984403991714,2cyclictest4086079-21fschecks_time08:50:432
3984403991713,2cyclictest4155447-21fschecks_count10:00:452
3984403991713,2cyclictest0-21swapper/211:00:522
3984403991712,3cyclictest0-21swapper/212:30:482
3984403991712,3cyclictest0-21swapper/207:45:472
3984403991712,3cyclictest0-21swapper/207:25:482
398440399171,14cyclictest30827-21latency_hist11:10:002
398440399170,2cyclictest1036341kthreadcore12:20:482
3984403991612,2cyclictest0-21swapper/208:05:012
3984403991611,3cyclictest0-21swapper/211:35:492
41452662152,8sleep30-21swapper/309:50:403
40199912156,6sleep30-21swapper/307:40:553
3984403991511,2cyclictest0-21swapper/207:20:432
952602142,7sleep30-21swapper/312:15:023
3984403991410,2cyclictest0-21swapper/210:10:472
495952132,8sleep30-21swapper/311:25:533
41612272133,6sleep30-21swapper/310:05:493
41402652133,6sleep30-21swapper/309:45:423
40111992136,5sleep30-21swapper/307:35:403
307212132,6sleep30-21swapper/311:10:003
208722132,6sleep30-21swapper/311:00:023
1146602132,7sleep30-21swapper/312:30:553
902292122,6sleep30-21swapper/312:10:013
759682122,6sleep30-21swapper/311:55:433
60452122,6sleep30-21swapper/310:45:423
556632122,5sleep30-21swapper/311:35:013
509322122,6sleep30-21swapper/311:30:043
41897782122,6sleep30-21swapper/310:35:013
41801362122,6sleep30-21swapper/310:25:433
41737712122,6sleep30-21swapper/310:15:523
41737712122,6sleep30-21swapper/310:15:513
41690762122,6sleep30-21swapper/310:10:593
41499552122,5sleep30-21swapper/309:55:003
41249862122,5sleep30-21swapper/309:30:013
41241712122,6sleep30-21swapper/309:25:553
41155602122,6sleep30-21swapper/309:20:423
41042082122,6sleep30-21swapper/309:05:543
40960092122,6sleep30-21swapper/309:00:483
40910452122,7sleep30-21swapper/308:55:463
40710462122,6sleep30-21swapper/308:35:443
40661662123,6sleep30-21swapper/308:30:473
40505782122,5sleep30-21swapper/308:15:013
40491582122,6sleep30-21swapper/308:10:523
40409882122,6sleep30-21swapper/308:05:423
40357412122,6sleep30-21swapper/308:00:013
40261412122,6sleep30-21swapper/307:50:413
40219092122,6sleep30-21swapper/307:45:483
40066332122,6sleep30-21swapper/307:30:473
40010122122,5sleep30-21swapper/307:25:013
39987162122,6sleep30-21swapper/307:20:523
263422122,6sleep30-21swapper/311:05:443
145002122,6sleep30-21swapper/310:50:503
13352122,6sleep30-21swapper/310:40:443
910672112,5sleep30-21swapper/312:10:493
803772112,5sleep30-21swapper/312:00:013
803772112,5sleep30-21swapper/312:00:013
696082112,5sleep30-21swapper/311:45:533
653492112,6sleep30-21swapper/311:42:313
454772112,6sleep30-21swapper/311:24:223
41850322112,6sleep30-21swapper/310:30:173
41747132112,5sleep30-21swapper/310:20:013
41747132112,5sleep30-21swapper/310:20:003
41555902112,5sleep30-21swapper/310:00:473
41355422112,5sleep30-21swapper/309:40:443
41301662112,5sleep30-21swapper/309:35:393
41110022111,6sleep30-21swapper/309:15:493
41056502112,5sleep30-21swapper/309:10:433
40860602112,5sleep30-21swapper/308:50:443
40755822112,5sleep30-21swapper/308:40:013
40755822112,5sleep30-21swapper/308:40:003
40645942112,5sleep30-21swapper/308:25:563
40593312112,5sleep30-21swapper/308:20:513
404092112,6sleep30-21swapper/311:17:523
40398082112,5sleep30-21swapper/308:00:573
39911902112,6sleep30-21swapper/307:15:063
39848452112,5sleep30-21swapper/307:10:453
181452112,5sleep30-21swapper/310:55:493
1196382112,5sleep30-21swapper/312:35:573
1093022112,5sleep30-21swapper/312:25:533
1006012112,5sleep30-21swapper/312:20:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional