You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 10:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun May 24, 2026 00:43:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1052374995833,18cyclictest12350irq/32-eth000:21:090
1052376995736,15cyclictest0-21swapper/221:46:032
1052376995535,16cyclictest1-21systemd00:25:012
1052375995342,5cyclictest0-21swapper/120:25:501
1052376995139,9cyclictest1225116-40gzip00:00:012
1052375995143,5cyclictest1237325-21latency_hist00:15:021
1052375995141,6cyclictest421-21dbus-daemon20:00:011
1052374995130,14cyclictest0-21swapper/022:10:560
1052374994833,12cyclictest1209046-21latency_hist23:40:020
1052375994739,3cyclictest0-21swapper/123:25:521
1052376994638,5cyclictest1144508-21awk22:20:012
105237699463,34cyclictest1238732-21ssh00:16:042
105237699462,15cyclictest1054081-21/usr/sbin/munin19:16:112
1052374994636,6cyclictest1234007-21aten_r2power_vo00:10:490
1052374994616,25cyclictest1145874-21ssh22:20:580
105237699452,13cyclictest1166056-21ssh22:46:002
1052374994537,4cyclictest151ktimers/020:56:000
105237499452,34cyclictest1109697-21ssh21:35:590
105237499452,34cyclictest1109697-21ssh21:35:590
1052376994437,4cyclictest1172786-21latency_hist22:55:022
105237699443,35cyclictest1250721-21ssh00:30:592
105237699441,11cyclictest1082567-21/usr/sbin/munin20:51:102
1052376994335,5cyclictest1156632-21ssh22:35:012
1052376994320,19cyclictest420-21avahi-daemon20:27:062
105237699431,10cyclictest1230631-21ntpq00:06:022
105237699431,10cyclictest1230631-21ntpq00:06:022
105237599435,30cyclictest1158104-21ssh22:36:031
1052375994337,3cyclictest0-21swapper/121:35:021
1052375994336,3cyclictest0-21swapper/122:35:001
1052375994336,3cyclictest0-21swapper/100:05:591
1052375994336,3cyclictest0-21swapper/100:05:591
1052375994334,6cyclictest3459337-21cups-browsed23:18:321
1052374994336,4cyclictest1200960-21kworker/u8:1-nfsiod23:31:050
105237699423,35cyclictest1066799-21ntpq19:56:012
105237699421,4cyclictest1097707-21ntpq21:20:582
105237699421,10cyclictest1059342-21ntpq19:31:012
105237699421,10cyclictest1059342-21ntpq19:31:012
1052375994234,5cyclictest1148549-21ssh22:25:011
1052375994233,6cyclictest261-21systemd-journal00:40:011
1052376994134,4cyclictest0-21swapper/220:50:022
1052376994133,5cyclictest1128343-21sendmail22:00:012
1052376994131,6cyclictest3459337-21cups-browsed23:36:422
1052376994129,8cyclictest0-21swapper/223:20:002
105237699412,6cyclictest1077202-21ntpq20:30:552
105237699412,6cyclictest1077202-21ntpq20:30:552
105237699412,5cyclictest1057788-21ntpq19:25:592
105237699411,9cyclictest1196426-21ssh23:24:032
105237699411,6cyclictest1068268-21ntpq20:01:002
1052375994134,4cyclictest0-21swapper/123:56:021
1052375994133,5cyclictest1243262-21unixbench_multi00:21:171
1052375994133,5cyclictest1074970-21latency_hist20:25:021
1052375994131,6cyclictest1058384-21systemd-run19:30:011
105237499412,34cyclictest1210355-21ssh23:40:570
1052376994032,5cyclictest1108924-21aten_r2power_cu21:35:462
1052376994032,5cyclictest1108924-21aten_r2power_cu21:35:462
105237699402,5cyclictest1072818-21ntpq20:16:042
1052376994019,18cyclictest1181874-21if_wlan023:05:512
1052376994016,21cyclictest1133793-21ntpq22:06:012
105237699401,4cyclictest1087704-21ntpq21:05:562
105237699401,13cyclictest1234712-21ntpq00:11:042
105237699401,11cyclictest1115890-21ssh21:44:062
105237699401,11cyclictest1115890-21ssh21:44:062
105237699401,10cyclictest1103744-21ssh21:28:582
105237599402,31cyclictest1194315-21ssh23:21:011
1052374994028,9cyclictest1192909-21ls23:20:000
105237699397,29cyclictest1091478-21ssh21:13:242
105237699393,6cyclictest1145906-21ssh22:20:592
1052376993931,5cyclictest1174695-21unixbench-2d22:56:122
1052376993930,5cyclictest1150504-21unixbench_multi22:26:112
1052376993930,5cyclictest1150504-21unixbench_multi22:26:112
105237699392,5cyclictest1218431-21ntpq23:50:592
105237699392,4cyclictest1069702-21ntpq20:05:552
105237699392,4cyclictest1060753-21ntpq19:35:582
105237699392,4cyclictest1060753-21ntpq19:35:582
105237699391,9cyclictest1186898-21/usr/sbin/munin23:11:112
105237699391,8cyclictest1139370-21ssh22:12:572
105237699391,5cyclictest1210506-21ntpq23:41:052
105237699391,4cyclictest1074266-21ntpq20:20:592
105237699391,4cyclictest1071259-21ntpq20:10:572
105237699391,4cyclictest1062282-21ntpq19:41:002
105237699391,14cyclictest1129742-21sh22:01:012
105237699391,13cyclictest1214411-21ntpq23:46:002
105237699391,13cyclictest1214411-21ntpq23:46:002
105237699391,13cyclictest1157933-21ntpq22:35:562
105237699391,12cyclictest1198379-21ntpq23:26:022
105237699391,11cyclictest1063710-21ntpq19:45:562
105237699391,10cyclictest1245824-21ssh00:25:372
105237699391,10cyclictest1203824-21ssh23:32:502
105237699391,10cyclictest1122457-21ssh21:51:202
1052375993933,3cyclictest0-21swapper/119:35:001
1052375993933,3cyclictest0-21swapper/119:35:001
1052375993931,5cyclictest421-21dbus-daemon19:25:011
1052375993931,5cyclictest1112318-21systemctl21:40:001
1052375993931,5cyclictest1112318-21systemctl21:40:001
1052375993931,5cyclictest1080872-21systemd-run20:45:011
105237499394,29cyclictest1088346-21diskmemload21:40:570
105237499394,29cyclictest1088346-21diskmemload21:40:570
1052374993933,4cyclictest1144524-21sendmail22:20:010
1052374993932,5cyclictest1052373-21cyclictest21:14:010
1052376993828,4cyclictest0-21swapper/200:35:592
105237699382,5cyclictest1065267-21ntpq19:50:592
105237699382,4cyclictest1086201-21ntpq21:00:552
105237699381,9cyclictest1177154-21ssh23:00:172
105237699381,9cyclictest1084725-21ntpq20:56:002
105237699381,10cyclictest1225759-21ssh00:00:462
105237699381,10cyclictest1225759-21ssh00:00:462
105237699381,10cyclictest1160919-21ssh22:40:052
105237699381,10cyclictest1107196-21ssh21:32:562
105237699381,10cyclictest1094615-21sshd21:16:422
105237599383,29cyclictest3459334-21cupsd21:51:001
105237599383,28cyclictest1250682-21ssh00:30:581
1052375993829,6cyclictest420-21avahi-daemon22:14:161
1052375993829,6cyclictest1221198-21latency_hist23:55:021
1052375993829,6cyclictest1166701-21unixbench_singl22:46:131
1052375993827,7cyclictest420-21avahi-daemon22:16:071
1052374993831,4cyclictest1129734-21fake-hwclock22:01:010
1052374993828,7cyclictest1225113-21cron00:00:000
1052374993826,9cyclictest0-21swapper/021:20:010
105237699371,6cyclictest1078703-21ntpq20:35:552
105237699371,6cyclictest1078703-21ntpq20:35:552
105237699371,5cyclictest1056238-21ntpq19:20:552
105237699371,4cyclictest1053277-21ntpq19:10:572
1052375993731,3cyclictest0-21swapper/100:20:001
1052375993730,3cyclictest0-21swapper/122:28:361
1052375993730,3cyclictest0-21swapper/122:28:361
1052375993728,5cyclictest0-21swapper/123:36:121
1052375993728,5cyclictest0-21swapper/123:00:591
105237599372,31cyclictest1214473-21rm23:46:051
105237599372,31cyclictest1214473-21rm23:46:051
1052374993729,5cyclictest421-21dbus-daemon19:10:010
1052374993723,11cyclictest0-21swapper/022:51:050
1052374993721,13cyclictest151ktimers/023:11:000
1052374993721,10cyclictest12350irq/32-eth022:55:570
105237699362,3cyclictest1080215-21ntpq20:40:592
1052375993630,3cyclictest0-21swapper/123:05:561
1052375993630,3cyclictest0-21swapper/120:55:021
1052375993627,6cyclictest1055-21mta-sts-daemon23:44:411
1052375993626,6cyclictest3459337-21cups-browsed21:25:271
1052375993613,7cyclictest62450lldpd21:55:101
1052374993626,6cyclictest1101441-21latency21:25:530
1052375993527,5cyclictest1086859-21idleruntime-cro21:05:011
1052375993527,5cyclictest1054-21mta-sts-daemon00:01:151
1052375993527,5cyclictest1054-21mta-sts-daemon00:01:151
1052375993524,8cyclictest1173361-21/usr/sbin/munin22:55:581
1052375993524,8cyclictest1055-21mta-sts-daemon21:41:001
1052375993524,8cyclictest1055-21mta-sts-daemon21:41:001
1052374993523,9cyclictest1056869-21sh19:25:010
1052374993512,19cyclictest1246761-21ssh00:26:020
105237599343,27cyclictest1089751-21sh21:11:021
1052375993426,5cyclictest1060384-21fschecks_time19:35:501
1052375993426,5cyclictest1060384-21fschecks_time19:35:501
1052375993426,4cyclictest0-21swapper/122:55:011
1052375993423,8cyclictest1161357-21aten_r2power_fr22:40:471
1052375993423,8cyclictest1064538-21latency_hist19:50:031
1052374993428,4cyclictest1150270-21switchtime22:26:100
1052374993428,4cyclictest1150270-21switchtime22:26:100
1052374993426,6cyclictest3459337-21cups-browsed21:52:170
1052374993426,5cyclictest420-21avahi-daemon00:35:010
105237599336,17cyclictest428-21mta-sts-daemon19:51:031
105237599334,21cyclictest1186268-21ssh23:10:591
1052375993327,3cyclictest0-21swapper/119:20:011
1052374993327,4cyclictest1128330-21munin-run22:00:010
1052374993326,4cyclictest1073364-21idleruntime-cro20:20:010
1052374993325,5cyclictest1157908-21ntp_kernel_err22:35:550
1052374993320,10cyclictest421-21dbus-daemon19:15:020
1052374993318,10cyclictest12350irq/32-eth020:46:000
105237499331,28cyclictest1194501-21ssh23:21:070
1052375993225,4cyclictest0-21swapper/122:01:011
1052375993225,3cyclictest0-21swapper/122:06:111
1052375993224,4cyclictest0-21swapper/121:21:021
1052375993220,10cyclictest1084646-21needreboot20:55:571
105237599321,25cyclictest1117843-21ssh21:46:041
1052374993227,3cyclictest1153328-21aten_r2power_po22:30:480
1052374993226,3cyclictest0-21swapper/022:05:500
1052374993225,4cyclictest1241283-21munin-run00:20:000
1052374993225,4cyclictest1056-21mta-sts-daemon23:46:450
1052374993225,4cyclictest1056-21mta-sts-daemon23:46:450
105237499321,25cyclictest1230596-21ssh00:06:000
105237499321,25cyclictest1230596-21ssh00:06:000
1052375993124,3cyclictest0-21swapper/100:25:471
1052375993123,5cyclictest1088346-21diskmemload23:35:011
105237599312,21cyclictest1093646-21ssh21:15:581
1052375993121,7cyclictest1068855-21sh20:05:011
1052374993125,3cyclictest0-21swapper/020:45:000
1052374993123,6cyclictest1066375-21fschecks_time19:55:510
1052374993122,5cyclictest0-21swapper/023:26:080
105237699301,8cyclictest1052399-21latency_hist19:10:022
1052375993020,7cyclictest1082342-21/usr/sbin/munin20:50:021
1052374993024,4cyclictest1058295-21unixbench_singl19:26:090
1052374993023,5cyclictest3570788-21munin-node19:46:110
1052374993023,4cyclictest1100427-21latency_hist21:25:030
1052374993022,5cyclictest1054-21mta-sts-daemon00:35:210
1052374993019,9cyclictest1070344-21cron20:10:010
1052374993017,11cyclictest1077778-21unixbench_singl20:31:080
1052374993017,11cyclictest1077778-21unixbench_singl20:31:080
1052375992922,5cyclictest1088388-21latency_hist21:10:021
105237499293,20cyclictest1178220-21ssh23:01:000
1052374992923,3cyclictest0-21swapper/023:05:460
1052374992922,4cyclictest1088346-21diskmemload21:10:010
105237499291,23cyclictest1166194-21sh22:46:050
1052375992820,5cyclictest3459337-21cups-browsed19:44:201
105237499282,23cyclictest1218538-21ssh23:51:030
105237499282,22cyclictest1105862-21ssh21:31:040
105237499282,21cyclictest1117715-21ssh21:45:590
1052374992821,3cyclictest0-21swapper/019:30:570
1052374992821,3cyclictest0-21swapper/019:30:570
105237499281,22cyclictest1226493-21ssh00:00:570
105237499281,22cyclictest1226493-21ssh00:00:570
1052375992719,5cyclictest1052880-21fschecks_count19:10:491
1052374992720,5cyclictest1161739-21kernelversion22:40:520
1052374992720,4cyclictest1086739-21unixbench_multi21:01:080
1052374992619,4cyclictest1055351-21cron19:20:010
1052375992515,7cyclictest420-21avahi-daemon20:31:411
1052375992515,7cyclictest420-21avahi-daemon20:31:411
105237499254,13cyclictest428-21mta-sts-daemon19:36:020
105237499254,13cyclictest428-21mta-sts-daemon19:36:020
1052374992521,2cyclictest0-21swapper/020:10:560
1052375992418,3cyclictest0-21swapper/120:40:001
1052375992418,3cyclictest0-21swapper/120:40:001
1052375992416,6cyclictest1069911-21awk20:06:031
1052375992415,6cyclictest1073339-21cron20:20:011
1052375992317,3cyclictest0-21swapper/120:10:471
105237499239,11cyclictest0-21swapper/020:25:530
105237499232,15cyclictest1068268-21ntpq20:01:000
1052374992318,2cyclictest0-21swapper/020:40:000
1052374992318,2cyclictest0-21swapper/020:40:000
1052374992310,10cyclictest1064836-21df19:50:490
1052374992215,5cyclictest420-21avahi-daemon20:20:420
1052375992115,3cyclictest0-21swapper/119:10:011
1052374992115,4cyclictest1048504-21kworker/u8:0-xprtiod19:41:020
105237799203,8cyclictest0-21swapper/323:05:553
105237799202,11cyclictest0-21swapper/323:46:003
105237799202,11cyclictest0-21swapper/323:46:003
105237499203,11cyclictest428-21mta-sts-daemon20:50:570
105237799192,10cyclictest0-21swapper/323:25:583
105237799183,8cyclictest0-21swapper/323:50:563
105237799183,11cyclictest0-21swapper/320:25:503
105237799182,9cyclictest0-21swapper/323:36:043
105237799182,9cyclictest0-21swapper/300:10:593
105237799182,11cyclictest0-21swapper/322:46:053
105237799182,10cyclictest0-21swapper/322:06:033
105237799182,10cyclictest0-21swapper/321:51:003
105237799182,10cyclictest0-21swapper/321:15:593
105237799181,8cyclictest0-21swapper/319:11:003
105237799181,11cyclictest0-21swapper/322:40:563
105237799173,9cyclictest0-21swapper/323:21:053
105237799173,9cyclictest0-21swapper/322:50:553
105237799172,9cyclictest0-21swapper/323:16:053
105237799172,9cyclictest0-21swapper/321:40:573
105237799172,9cyclictest0-21swapper/321:40:573
105237799172,10cyclictest0-21swapper/300:01:053
105237799172,10cyclictest0-21swapper/300:01:053
105237799171,9cyclictest0-21swapper/322:55:583
105237799171,9cyclictest0-21swapper/300:21:043
105237799171,9cyclictest0-21swapper/300:15:583
105237799171,10cyclictest0-21swapper/323:10:563
105237799164,9cyclictest0-21swapper/321:25:563
105237799162,9cyclictest0-21swapper/322:15:573
105237799162,8cyclictest0-21swapper/322:20:593
105237799162,7cyclictest0-21swapper/321:35:553
105237799162,7cyclictest0-21swapper/321:35:553
105237799162,6cyclictest0-21swapper/320:05:593
105237799161,9cyclictest0-21swapper/323:41:013
105237799161,9cyclictest0-21swapper/322:01:053
105237799161,9cyclictest0-21swapper/321:30:493
105237799161,10cyclictest0-21swapper/321:56:083
105237799161,10cyclictest0-21swapper/300:30:563
105237799153,7cyclictest0-21swapper/322:35:583
105237799152,9cyclictest0-21swapper/300:05:593
105237799152,9cyclictest0-21swapper/300:05:593
105237799152,8cyclictest0-21swapper/323:31:003
105237799152,8cyclictest0-21swapper/322:30:573
105237799152,8cyclictest0-21swapper/300:25:573
105237799152,5cyclictest0-21swapper/320:30:573
105237799152,5cyclictest0-21swapper/320:30:573
105237799151,9cyclictest0-21swapper/323:55:513
105237799151,8cyclictest0-21swapper/322:25:573
105237799151,8cyclictest0-21swapper/322:25:573
105237799151,8cyclictest0-21swapper/300:35:493
105237799151,7cyclictest0-21swapper/320:01:003
105237799151,7cyclictest0-21swapper/319:35:563
105237799151,7cyclictest0-21swapper/319:35:563
105237799142,9cyclictest0-21swapper/321:10:563
105237799142,8cyclictest0-21swapper/320:56:053
105237799142,8cyclictest0-21swapper/319:20:013
105237799142,7cyclictest0-21swapper/321:20:563
105237799142,6cyclictest0-21swapper/321:10:033
105237799141,8cyclictest0-21swapper/323:00:573
105237799141,8cyclictest0-21swapper/321:45:513
105237799141,7cyclictest0-21swapper/322:10:463
105237799141,7cyclictest0-21swapper/320:35:593
105237799141,7cyclictest0-21swapper/320:35:593
105237799141,7cyclictest0-21swapper/319:20:553
105237799141,6cyclictest0-21swapper/320:10:553
105237799141,6cyclictest0-21swapper/319:45:563
105237799131,7cyclictest0-21swapper/320:16:043
105237799131,6cyclictest0-21swapper/319:30:573
105237799131,6cyclictest0-21swapper/319:30:573
105237799130,7cyclictest0-21swapper/320:21:023
105237799121,7cyclictest0-21swapper/319:40:573
105237799121,7cyclictest0-21swapper/319:26:003
105237799121,6cyclictest0-21swapper/320:45:573
105237799121,6cyclictest0-21swapper/320:40:583
105237799111,6cyclictest0-21swapper/321:00:563
105237799111,6cyclictest0-21swapper/320:50:563
105237799111,6cyclictest0-21swapper/319:50:563
105237799110,6cyclictest0-21swapper/319:55:533
10523779991,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional