You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 14:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun Apr 26, 2026 12:43:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
984714996653,10cyclictest1173977-21munin-plugin-st12:20:000
984714996033,17cyclictest0-21swapper/011:45:430
984714996033,17cyclictest0-21swapper/011:45:430
984715995843,9cyclictest171855-21cups-browsed11:30:491
984714995747,4cyclictest0-21swapper/011:15:430
984714995648,5cyclictest1-21systemd11:45:010
984714995648,5cyclictest1-21systemd11:45:010
98471699555,41cyclictest1122981-21ssh11:15:502
984714995136,12cyclictest1162004-21latency_hist12:05:020
984716995039,7cyclictest1-21systemd09:25:012
984714995036,11cyclictest0-21swapper/011:20:490
984714995024,23cyclictest1175157-21needreboot12:20:410
984716994942,3cyclictest0-21swapper/208:40:002
98471599483,33cyclictest1090672-21ssh10:35:481
984714994834,6cyclictest12350irq/32-eth010:55:440
984714994834,6cyclictest12350irq/32-eth010:55:440
98471499481,14cyclictest990130-21ntpq07:25:460
98471699474,32cyclictest1175304-21ssh12:20:462
984715994739,5cyclictest1004217-21systemd-run08:15:011
984715994738,6cyclictest421-21dbus-daemon10:10:511
98471499472,23cyclictest1066403-21ssh10:05:430
98471499464,32cyclictest1042327-21ssh09:35:470
984715994535,7cyclictest439-21systemd-logind07:30:021
984715994534,7cyclictest1029938-21needreboot09:20:401
984714994523,19cyclictest1020700-21diskmemload11:25:380
98471499452,15cyclictest997673-21ntpq07:50:480
984714994520,23cyclictest0-21swapper/009:55:380
984714994520,23cyclictest0-21swapper/009:55:380
98471499451,11cyclictest1034137-21ssh09:25:440
984716994435,6cyclictest1061738-21aten_r2power_en10:00:332
984716994432,9cyclictest1077258-21munin-run10:20:012
98471599447,26cyclictest1082613-21ssh10:25:471
984714994424,17cyclictest420-21avahi-daemon12:30:410
984716994332,7cyclictest421-21dbus-daemon12:38:552
984716994332,7cyclictest171855-21cups-browsed11:30:492
984715994335,5cyclictest1183876-21unixbench-2d12:30:581
984714994332,6cyclictest151ktimers/010:30:490
984714994323,16cyclictest0-21swapper/010:25:020
984714994322,18cyclictest12450irq/33-eth011:05:450
984714994321,19cyclictest1102154-21cpu10:50:350
984714994315,25cyclictest1032842-21cron09:25:000
984714994229,8cyclictest151ktimers/010:45:460
984714994223,16cyclictest1090747-21ssh10:35:500
984714994216,4cyclictest0-21swapper/012:35:370
98471499421,18cyclictest1151024-21ntpq11:50:410
98471499421,18cyclictest1151024-21ntpq11:50:410
98471499421,17cyclictest1110797-21ntpq11:00:450
98471499421,12cyclictest1021937-21ssh09:10:440
984716994133,5cyclictest421-21dbus-daemon09:45:022
984716994133,5cyclictest1109531-21ssh11:00:002
984716994133,5cyclictest1109531-21ssh11:00:002
984714994131,7cyclictest171855-21cups-browsed10:27:560
984714994126,12cyclictest1020693-21cron09:10:010
98471499412,17cyclictest1155109-21ssh11:55:430
984714994116,23cyclictest12450irq/33-eth009:15:440
98471499411,15cyclictest1050414-21ntpq09:45:510
984716994032,5cyclictest1166760-21fschecks_time12:10:372
984716994032,5cyclictest1166760-21fschecks_time12:10:372
984716994032,3cyclictest0-21swapper/211:55:372
984715994032,4cyclictest1137251-21kworker/u8:13-writeback12:30:011
984715994031,6cyclictest1158924-21latency12:00:401
98471499408,10cyclictest421-21dbus-daemon10:40:500
98471499406,27cyclictest1118803-21ssh11:10:430
98471499403,11cyclictest1054389-21ssh09:50:470
98471499403,11cyclictest1054389-21ssh09:50:470
98471499402,17cyclictest1062396-21ntpq10:00:450
98471499401,19cyclictest1004192-21cron08:15:010
984714994011,8cyclictest1017087-21ntp_kernel_pll_08:55:440
98471499401,16cyclictest1138536-21ssh11:35:350
98471499401,16cyclictest1138536-21ssh11:35:350
98471499401,16cyclictest1045900-21fschecks_count09:40:370
98471499401,13cyclictest1074506-21ntpq10:15:450
984714994010,10cyclictest12350irq/32-eth012:10:490
984714994010,10cyclictest12350irq/32-eth012:10:490
984716993931,5cyclictest1182078-21latency_hist12:30:002
984716993930,5cyclictest1083164-21unixbench_singl10:25:562
984715993931,5cyclictest1081355-21latency_hist10:25:021
984715993930,6cyclictest1066817-21switchtime10:05:561
984715993929,6cyclictest171852-21cupsd10:57:161
984715993929,6cyclictest171852-21cupsd10:57:161
98471499394,8cyclictest1011178-21ntpq08:35:470
984714993930,6cyclictest1165944-21true12:10:010
98471499392,12cyclictest989215-21/usr/sbin/munin07:25:020
98471499391,6cyclictest1012734-21proc_pri08:40:490
98471499391,6cyclictest1012734-21proc_pri08:40:490
984714993915,4cyclictest987211-21proc_pri07:15:490
984714993913,3cyclictest0-21swapper/010:10:510
984716993825,4cyclictest0-21swapper/210:20:582
984715993830,5cyclictest439-21systemd-logind11:20:011
984715993821,3cyclictest171rcu_preempt12:25:011
984714993826,9cyclictest1-21systemd07:45:010
98471499382,16cyclictest1038198-21sh09:30:440
984714993814,4cyclictest0-21swapper/011:31:030
98471499381,11cyclictest1020700-21diskmemload12:29:420
984715993729,5cyclictest1110662-21needreboot11:00:401
984715993728,6cyclictest1099243-21unixbench_multi10:45:561
984714993713,4cyclictest12450irq/33-eth007:30:520
984716993630,3cyclictest0-21swapper/211:50:012
984716993630,3cyclictest0-21swapper/211:50:012
984716993630,3cyclictest0-21swapper/210:15:012
984716993628,5cyclictest1163805-21grep12:05:572
984716993624,10cyclictest1-21systemd09:39:032
984716993621,11cyclictest1038157-21sh09:30:432
98471599363,26cyclictest1187443-21ssh12:35:491
984715993630,3cyclictest0-21swapper/112:10:011
984715993628,5cyclictest420-21avahi-daemon09:42:041
98471499361,17cyclictest1006630-21ntpq08:20:440
984716993529,3cyclictest0-21swapper/208:05:012
984716993528,4cyclictest1057208-21latency_hist09:55:022
984716993528,4cyclictest1057208-21latency_hist09:55:022
984716993527,5cyclictest1-21systemd12:05:022
984716993527,5cyclictest1110440-21awk11:00:372
984716993526,6cyclictest439-21systemd-logind07:45:012
98471599354,25cyclictest1034041-21ssh09:25:411
984715993527,5cyclictest1045094-21systemd-run09:40:011
984715993527,3cyclictest0-21swapper/111:22:581
984715993525,7cyclictest171852-21cupsd11:41:071
984715993525,7cyclictest171852-21cupsd11:41:071
98471599351,9cyclictest1151195-21ssh11:50:481
98471599351,9cyclictest1151195-21ssh11:50:481
984714993527,6cyclictest421-21dbus-daemon08:00:020
984714993516,2cyclictest0-21swapper/007:10:520
98471699345,4cyclictest171rcu_preempt09:45:502
98471699343,25cyclictest1139092-21ssh11:35:472
98471699343,25cyclictest1139092-21ssh11:35:472
984716993428,3cyclictest0-21swapper/210:40:012
984716993426,5cyclictest988239-21fschecks_count07:20:352
984716993425,7cyclictest695-21cron09:10:012
984716993424,7cyclictest1105465-21munin-plugin-st10:55:022
984716993423,8cyclictest1098257-21idleruntime10:45:382
984715993428,3cyclictest0-21swapper/112:00:001
984715993428,3cyclictest0-21swapper/109:00:011
984715993427,4cyclictest1053581-21acpi09:50:331
984715993427,4cyclictest1053581-21acpi09:50:331
984715993427,4cyclictest0-21swapper/107:20:011
984715993426,5cyclictest1119396-21unixbench_multi11:10:541
984715993426,5cyclictest1088951-21ssh10:34:211
984715993425,6cyclictest420-21avahi-daemon11:46:371
984715993425,6cyclictest420-21avahi-daemon11:46:371
984715993420,5cyclictest998227-21wc07:55:021
98471499341,29cyclictest659-21snmpd08:45:420
984716993326,3cyclictest0-21swapper/212:30:592
984716993326,3cyclictest0-21swapper/209:15:552
984716993324,6cyclictest998214-21systemd-run07:55:012
984716993324,6cyclictest1057950-21gltestperf09:55:362
984716993324,6cyclictest1057950-21gltestperf09:55:362
984715993327,3cyclictest0-21swapper/110:05:001
984715993326,4cyclictest0-21swapper/110:20:011
984715993325,5cyclictest1020697-21cron09:10:011
984715993324,7cyclictest1141836-21sh11:40:021
984715993324,7cyclictest1141836-21sh11:40:021
984715993324,6cyclictest171852-21cupsd09:33:051
984716993226,3cyclictest0-21swapper/211:50:412
984716993226,3cyclictest0-21swapper/211:50:412
984716993223,6cyclictest420-21avahi-daemon09:30:012
984716993126,2cyclictest0-21swapper/208:20:012
984716993123,5cyclictest990590-21unixbench_multi07:25:552
984716993123,5cyclictest1095944-21ssh10:42:142
984716993123,5cyclictest1001726-21fschecks_count08:05:372
984716993122,6cyclictest1143229-21proc_pri11:40:532
984716993122,6cyclictest1143229-21proc_pri11:40:532
984716993121,6cyclictest1018219-21fschecks_count09:00:342
984715993123,5cyclictest993694-21cut07:40:011
984715993123,5cyclictest1167799-21head12:10:551
984715993123,5cyclictest1167799-21head12:10:551
984715993122,6cyclictest1094023-21aten_r2power_vo10:40:321
984715993122,5cyclictest171852-21cupsd10:51:461
984715993120,8cyclictest1061260-21latency_hist10:00:011
984715993120,8cyclictest1061260-21latency_hist10:00:011
984716993024,3cyclictest0-21swapper/211:30:012
984716993023,3cyclictest0-21swapper/211:20:512
984716993023,3cyclictest0-21swapper/210:30:462
984715993023,3cyclictest0-21swapper/107:20:411
984715993022,5cyclictest171855-21cups-browsed08:50:041
98471599301,23cyclictest1050518-21grep09:45:541
984716992922,5cyclictest1051-21mta-sts-daemon07:58:442
984716992921,5cyclictest1170625-21aten_r2power_vo12:15:332
984716992920,6cyclictest420-21avahi-daemon11:11:562
98471599294,5cyclictest171rcu_preempt09:15:461
984715992923,3cyclictest0-21swapper/108:10:011
984715992922,4cyclictest1114488-21kernelversion11:05:381
984715992921,5cyclictest999294-21mailstats07:55:511
984715992921,5cyclictest1024866-21latency_hist09:15:021
984715992921,5cyclictest1010204-21idleruntime-cro08:35:001
984714992919,5cyclictest151ktimers/008:30:440
984714992911,3cyclictest1002724-21kworker/u8:2-xprtiod08:15:250
98471699284,15cyclictest993214-21ps07:35:482
984716992821,4cyclictest992388-21latency_hist07:35:022
984716992821,4cyclictest1065830-21aten_r2power_po10:05:342
984716992820,5cyclictest996656-21head07:45:542
984716992820,5cyclictest1012226-21fschecks_count08:40:362
984716992820,5cyclictest1012226-21fschecks_count08:40:362
98471699281,21cyclictest1114794-21ssh11:05:442
984715992821,5cyclictest271ktimers/108:45:411
984715992821,3cyclictest0-21swapper/111:25:431
984715992818,7cyclictest1005041-21ntp_kernel_err08:15:421
984714992814,12cyclictest151ktimers/007:35:440
984716992718,6cyclictest1010113-21unixbench_singl08:30:542
98471599273,19cyclictest1171340-21ssh12:15:491
984715992718,7cyclictest1001235-21sh08:05:011
984714992718,7cyclictest1018573-21ntp_kernel_pll_09:00:400
984716992619,4cyclictest1016416-21/usr/sbin/munin08:55:452
984716992615,7cyclictest420-21avahi-daemon07:10:452
984716992613,8cyclictest171855-21cups-browsed08:25:422
984715992619,4cyclictest0-21swapper/108:40:001
984715992618,5cyclictest421-21dbus-daemon07:50:011
984715992618,5cyclictest171855-21cups-browsed07:14:151
984715992617,7cyclictest994078-21aten_r2power_po07:40:341
984715992613,9cyclictest421-21dbus-daemon08:45:011
984715992613,9cyclictest421-21dbus-daemon08:45:011
984714992619,4cyclictest1007727-21fschecks_count08:25:360
984716992517,5cyclictest1021521-21fschecks_count09:10:372
984716992516,7cyclictest1014047-21ntp_kernel_err08:45:422
984715992519,3cyclictest0-21swapper/109:05:021
984715992517,5cyclictest1007212-21idleruntime-cro08:25:011
98471599250,9cyclictest99107150telnet07:30:321
984714992514,8cyclictest0-21swapper/007:50:000
984716992418,4cyclictest987793-21latency_hist07:20:022
98471499241,9cyclictest1001133-21unixbench_singl08:00:550
984716992316,4cyclictest1003108-21aten2.4-expect08:10:332
984716992316,4cyclictest0-21swapper/208:50:542
984715992316,4cyclictest1007551-21aten2.4-expect08:25:331
98471699227,3cyclictest171rcu_preempt08:20:322
98471499222,16cyclictest1015647-21ntp_states08:50:480
984714992216,3cyclictest427-21polkitd08:05:310
984715992116,2cyclictest0-21swapper/107:10:011
98471799202,9cyclictest0-21swapper/309:10:453
98471799182,10cyclictest0-21swapper/310:25:463
98471799181,9cyclictest0-21swapper/308:05:463
98471799181,6cyclictest0-21swapper/308:15:463
98471699181,3cyclictest984815-21ls07:10:022
98471799173,9cyclictest0-21swapper/311:00:453
98471799172,9cyclictest0-21swapper/309:50:473
98471799172,9cyclictest0-21swapper/309:50:473
98471799172,9cyclictest0-21swapper/309:30:443
98471799172,10cyclictest0-21swapper/311:30:493
98471799171,9cyclictest0-21swapper/310:10:463
98471799171,7cyclictest0-21swapper/307:35:453
984714991711,4cyclictest12350irq/32-eth007:10:020
98471799165,8cyclictest0-21swapper/312:35:493
98471799163,9cyclictest0-21swapper/309:55:453
98471799163,9cyclictest0-21swapper/309:55:453
98471799162,9cyclictest0-21swapper/311:10:443
98471799162,9cyclictest0-21swapper/311:05:443
98471799162,9cyclictest0-21swapper/310:55:483
98471799162,9cyclictest0-21swapper/310:55:483
98471799162,9cyclictest0-21swapper/310:05:443
98471799162,9cyclictest0-21swapper/309:45:543
98471799162,9cyclictest0-21swapper/309:20:443
98471799162,8cyclictest0-21swapper/312:25:423
98471799162,8cyclictest0-21swapper/312:15:473
98471799162,8cyclictest0-21swapper/310:20:443
98471799161,9cyclictest0-21swapper/310:50:433
98471799161,9cyclictest0-21swapper/309:25:403
98471799161,8cyclictest0-21swapper/312:10:453
98471799161,8cyclictest0-21swapper/312:10:453
98471799161,7cyclictest0-21swapper/307:30:463
98471799161,10cyclictest0-21swapper/312:20:463
98471799160,7cyclictest0-21swapper/307:20:423
98471799153,8cyclictest0-21swapper/310:30:433
98471799153,7cyclictest0-21swapper/311:35:473
98471799153,7cyclictest0-21swapper/311:35:473
98471799153,5cyclictest0-21swapper/307:10:423
98471799152,8cyclictest0-21swapper/311:40:493
98471799152,8cyclictest0-21swapper/311:40:493
98471799152,8cyclictest0-21swapper/310:35:423
98471799152,7cyclictest0-21swapper/312:05:403
98471799152,7cyclictest0-21swapper/311:55:363
98471799151,9cyclictest0-21swapper/312:30:473
98471799151,8cyclictest0-21swapper/311:45:433
98471799151,8cyclictest0-21swapper/311:45:433
98471799151,8cyclictest0-21swapper/311:15:433
98471799151,8cyclictest0-21swapper/309:35:483
98471799151,6cyclictest0-21swapper/308:25:453
98471799150,9cyclictest0-21swapper/310:15:483
98471799144,6cyclictest0-21swapper/310:40:413
98471799144,6cyclictest0-21swapper/310:00:533
98471799142,8cyclictest0-21swapper/311:50:433
98471799142,8cyclictest0-21swapper/311:50:433
98471799142,8cyclictest0-21swapper/311:25:513
98471799142,6cyclictest0-21swapper/308:40:443
98471799142,6cyclictest0-21swapper/308:40:443
98471799141,8cyclictest0-21swapper/311:20:443
98471799141,8cyclictest0-21swapper/309:40:443
98471799141,6cyclictest0-21swapper/307:50:453
98471799132,8cyclictest0-21swapper/310:45:393
98471799132,7cyclictest0-21swapper/312:00:333
98471799132,6cyclictest0-21swapper/308:40:013
98471799132,5cyclictest0-21swapper/308:55:343
98471799131,6cyclictest0-21swapper/309:15:333
98471799131,6cyclictest0-21swapper/309:00:413
98471799124,6cyclictest0-21swapper/307:25:433
98471799122,5cyclictest0-21swapper/308:45:463
98471799121,7cyclictest0-21swapper/309:10:013
98471799121,6cyclictest0-21swapper/308:30:443
98471799121,6cyclictest0-21swapper/308:00:353
98471799111,6cyclictest0-21swapper/308:50:423
98471799111,6cyclictest0-21swapper/308:10:433
98471799111,6cyclictest0-21swapper/307:55:453
98471799101,5cyclictest0-21swapper/308:20:323
98471799101,5cyclictest0-21swapper/307:45:333
98471799101,5cyclictest0-21swapper/307:10:033
98471799100,6cyclictest0-21swapper/307:15:333
9847179991,5cyclictest0-21swapper/307:40:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional