You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 05:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri May 22, 2026 00:43:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1339428995242,7cyclictest1401471-21head21:41:091
1339428994938,8cyclictest1463457-21sendmail23:00:001
1339428994833,11cyclictest1448768-21missed_timers22:40:551
1339429994741,3cyclictest0-21swapper/200:00:002
1339428994738,4cyclictest0-21swapper/123:10:531
133942899473,36cyclictest3738109-21cups-browsed22:00:591
133942799473,19cyclictest1467471-21kworker/u8:7+rpciod00:05:580
133942899464,33cyclictest1512605-21ssh00:00:561
1339428994636,6cyclictest1511301-21cups-browsed00:00:021
1339427994631,12cyclictest1507303-21ssh23:55:020
1339429994526,12cyclictest351ktimers/222:45:582
1339427994540,2cyclictest0-21swapper/023:31:030
1339427994532,10cyclictest1511269-21munin-plugin-st00:00:000
1339429994435,5cyclictest1399445-21munin-run21:40:012
1339429994435,4cyclictest0-21swapper/223:01:002
1339429994432,4cyclictest0-21swapper/222:25:572
1339428994434,6cyclictest3738109-21cups-browsed22:33:501
1339428994431,9cyclictest1511301-21cups-browsed00:05:511
1339428994334,6cyclictest421-21dbus-daemon21:50:021
1339427994334,6cyclictest420-21avahi-daemon22:38:470
1339427994320,10cyclictest1375420-21diskmemload22:51:000
1339427994320,10cyclictest1375420-21diskmemload22:51:000
1339428994236,3cyclictest0-21swapper/119:20:011
1339428994233,6cyclictest1501333-21unixbench_singl23:46:101
1339428994233,6cyclictest1499491-21latency_hist23:45:021
1339429994133,5cyclictest421-21dbus-daemon22:20:012
1339429994132,5cyclictest1517235-21tail00:06:092
1339428994134,4cyclictest1539875-21aten_r2power_vo00:35:481
1339428994134,3cyclictest0-21swapper/120:15:591
1339428994132,6cyclictest1452200-21aten_r2power_en22:45:461
1339428994132,5cyclictest1485033-21mailstats23:26:061
1339428994132,5cyclictest1399445-21munin-run21:40:011
1339427994128,10cyclictest0-21swapper/021:51:040
133942999404,26cyclictest1488724-21ssh23:30:562
1339429994026,10cyclictest1476538-21latency23:15:522
1339429994026,10cyclictest1476538-21latency23:15:522
133942999402,32cyclictest1420832-21ssh22:05:542
133942999402,32cyclictest1420832-21ssh22:05:542
133942899402,30cyclictest1433059-21ssh22:21:021
1339427994033,4cyclictest1-21systemd21:30:010
1339427994032,5cyclictest1511301-21cups-browsed00:22:000
1339427994032,5cyclictest1407445-21cron21:50:010
1339429993931,5cyclictest1349942-21systemd-run19:45:012
1339429993929,6cyclictest1463443-21munin-plugin-st23:00:002
1339429993929,4cyclictest0-21swapper/222:50:572
1339429993929,4cyclictest0-21swapper/222:50:572
133942999392,23cyclictest1496699-21ssh23:40:562
1339428993931,5cyclictest1367914-21cron20:45:011
1339428993927,9cyclictest1489378-21unixbench_singl23:31:081
1339427993932,4cyclictest1429536-21unixbench_singl22:16:080
1339427993931,5cyclictest261-21systemd-journal23:25:010
1339427993929,7cyclictest1340934-21kworker/u8:8-rpciod19:40:010
133942999384,24cyclictest1468856-21ssh23:05:592
1339428993830,5cyclictest1483405-21idleruntime-cro23:25:001
1339428993828,7cyclictest3738109-21cups-browsed22:50:211
1339428993828,7cyclictest3738109-21cups-browsed22:50:211
1339427993829,7cyclictest1375420-21diskmemload23:05:010
1339429993731,3cyclictest0-21swapper/219:40:012
1339429993729,5cyclictest420-21avahi-daemon00:10:592
1339429993727,6cyclictest1416278-21aten_r2power_vo22:00:472
1339428993731,3cyclictest0-21swapper/121:00:011
1339428993729,5cyclictest1471509-21ssh23:10:021
1339428993729,5cyclictest1339438-21systemd-run19:10:021
1339428993728,6cyclictest3738109-21cups-browsed21:55:281
133942899371,28cyclictest1388875-21ssh21:26:011
1339427993733,3cyclictest1339426-21cyclictest23:37:590
1339427993730,5cyclictest1354431-21sendmail-msp20:00:010
1339427993730,4cyclictest0-21swapper/020:40:580
1339427993730,3cyclictest0-21swapper/021:30:560
1339427993723,11cyclictest1435715-21ssh22:25:020
1339427993715,18cyclictest1479406-21munin-run23:20:010
1339427993715,18cyclictest1479406-21munin-run23:20:010
1339427993713,17cyclictest1339426-21cyclictest19:40:550
1339429993631,2cyclictest0-21swapper/221:30:022
1339429993628,5cyclictest1363433-21systemd-run20:30:012
1339429993628,5cyclictest1363433-21systemd-run20:30:012
1339429993625,8cyclictest1540554-21ntpq00:36:052
133942999362,29cyclictest1528616-21ssh00:20:592
133942899365,26cyclictest1504791-21ssh23:51:041
1339428993628,5cyclictest1387523-21munin-plugin-st21:25:001
1339428993628,4cyclictest0-21swapper/121:16:071
1339428993628,4cyclictest0-21swapper/121:16:071
1339428993627,6cyclictest420-21avahi-daemon00:11:201
1339428993626,7cyclictest1431569-21idleruntime-cro22:20:011
1339428993622,11cyclictest1409348-21timerwakeupswit21:51:101
133942799367,7cyclictest0-21swapper/020:35:540
1339427993630,4cyclictest1449169-21mailstats22:41:060
1339427993628,5cyclictest3738109-21cups-browsed22:26:510
1339427993619,10cyclictest191rcuc/022:06:010
1339427993619,10cyclictest191rcuc/022:06:010
1339429993528,3cyclictest0-21swapper/221:10:552
1339429993527,5cyclictest1524146-21fschecks_time00:15:492
1339429993527,5cyclictest1364902-21cron20:35:002
1339429993527,4cyclictest0-21swapper/221:25:012
133942999352,25cyclictest3738109-21cups-browsed21:46:062
1339428993528,4cyclictest1348947-21fschecks_count19:40:481
1339428993526,5cyclictest0-21swapper/120:35:001
133942899352,29cyclictest1444964-21ssh22:35:581
1339427993530,2cyclictest0-21swapper/022:50:010
1339427993528,4cyclictest1380076-21/usr/sbin/munin21:15:530
1339427993528,4cyclictest1380076-21/usr/sbin/munin21:15:530
1339427993527,5cyclictest1511301-21cups-browsed00:02:440
1339427993526,7cyclictest0-21swapper/023:10:500
1339427993522,5cyclictest1373911-21cron21:05:010
133942999346,18cyclictest3850255-21munin-node19:21:102
1339429993426,5cyclictest1443653-21latency_hist22:35:022
1339429993426,5cyclictest1354407-21run-parts20:00:012
1339429993423,8cyclictest1433581-21unixbench_singl22:21:092
133942999342,26cyclictest1413045-21grep21:56:112
1339428993428,3cyclictest0-21swapper/123:16:021
1339428993428,3cyclictest0-21swapper/123:16:021
1339428993427,3cyclictest0-21swapper/123:40:011
1339428993426,5cyclictest1362446-21fschecks_count20:25:491
1339428993426,5cyclictest1362446-21fschecks_count20:25:491
1339428993424,6cyclictest605-21lldpd00:26:151
1339428993424,6cyclictest605-21lldpd00:26:151
1339428993424,6cyclictest420-21avahi-daemon22:08:531
1339428993424,6cyclictest420-21avahi-daemon22:08:531
1339427993428,3cyclictest0-21swapper/000:35:490
1339427993427,5cyclictest1403523-21/usr/sbin/munin21:45:010
1339427993427,4cyclictest1054-21mta-sts-daemon23:28:330
133942799342,21cyclictest1412997-21ssh21:56:100
1339427993417,11cyclictest12350irq/32-eth023:05:590
1339427993415,14cyclictest1384914-21ssh21:20:580
1339429993328,2cyclictest0-21swapper/222:40:012
1339429993325,5cyclictest1448531-21gltestperf22:40:522
133942999332,23cyclictest1424851-21ssh22:10:552
1339428993326,3cyclictest0-21swapper/100:20:561
1339428993324,5cyclictest0-21swapper/121:35:021
1339428993323,6cyclictest1523988-21aten_r2power_vo00:15:461
133942899332,25cyclictest1424941-21ssh22:10:581
1339428993321,8cyclictest0-21swapper/100:35:001
1339427993327,4cyclictest1345569-21latency_hist19:30:020
1339427993327,4cyclictest1055-21mta-sts-daemon00:30:570
1339429993227,3cyclictest1358921-21kworker/u8:9-rpciod20:25:012
1339429993225,5cyclictest351ktimers/221:05:582
1339429993225,4cyclictest1475556-21kworker/u8:9+rpciod23:35:592
1339429993225,4cyclictest1054-21mta-sts-daemon23:29:432
1339429993223,4cyclictest0-21swapper/221:16:012
1339429993223,4cyclictest0-21swapper/221:16:012
1339429993220,7cyclictest1375420-21diskmemload23:45:552
1339428993225,4cyclictest0-21swapper/119:45:551
1339428993224,5cyclictest1361943-21systemd-run20:25:011
1339427993227,2cyclictest0-21swapper/023:41:080
1339427993226,4cyclictest1396812-21ssh21:36:000
1339427993225,4cyclictest1359024-21latency_hist20:15:020
1339429993126,2cyclictest0-21swapper/223:10:552
1339429993124,3cyclictest0-21swapper/200:01:042
1339429993123,6cyclictest0-21swapper/221:30:582
1339428993123,5cyclictest3738109-21cups-browsed23:02:151
1339428993122,6cyclictest420-21avahi-daemon20:40:011
1339427993127,2cyclictest0-21swapper/023:00:000
1339427993125,3cyclictest0-21swapper/022:15:010
1339427993125,3cyclictest0-21swapper/000:15:550
1339427993124,5cyclictest420-21avahi-daemon00:29:310
1339427993124,5cyclictest420-21avahi-daemon00:29:310
1339427993124,5cyclictest191rcuc/020:45:580
1339427993123,5cyclictest1369430-21kworker/u8:12+xprtiod20:55:010
1339429993025,2cyclictest0-21swapper/221:05:012
1339429993024,3cyclictest0-21swapper/220:51:022
1339429993023,3cyclictest0-21swapper/221:40:542
1339429993022,5cyclictest1367909-21cron20:45:012
1339429993022,4cyclictest1440154-21kworker/u8:11-events_unbound23:20:462
133942999302,22cyclictest1536603-21ssh00:31:022
1339428993022,4cyclictest0-21swapper/121:10:011
1339428993020,7cyclictest0-21swapper/122:30:011
1339428993019,8cyclictest1344680-21missed_timers19:25:521
1339428993018,5cyclictest1343608-21switchtime19:21:171
1339427993024,4cyclictest1375420-21diskmemload21:11:090
1339427993024,3cyclictest0-21swapper/023:46:080
1339427993021,7cyclictest1417223-21switchtime22:01:080
1339427993021,6cyclictest1343329-21ntp_kernel_pll_19:21:090
1339427993021,6cyclictest1-21systemd20:10:010
1339429992924,2cyclictest0-21swapper/223:51:032
1339429992922,5cyclictest1357744-21aten_r2power_fr20:10:452
1339429992922,3cyclictest0-21swapper/221:50:592
1339429992921,5cyclictest1366275-21unixbench_multi20:36:082
1339429992920,5cyclictest1-21systemd19:55:022
133942999291,19cyclictest1532591-21ssh00:26:002
133942999291,19cyclictest1532591-21ssh00:26:002
1339428992922,3cyclictest1369429-21kworker/u8:8-xprtiod21:11:031
1339427992924,3cyclictest1055-21mta-sts-daemon20:00:470
1339427992923,3cyclictest0-21swapper/000:11:430
1339427992921,6cyclictest1362735-21needreboot20:25:530
1339427992921,6cyclictest1362735-21needreboot20:25:530
1339427992921,5cyclictest3738109-21cups-browsed20:20:560
1339429992823,2cyclictest0-21swapper/219:20:012
1339429992822,3cyclictest0-21swapper/220:01:022
1339429992821,4cyclictest1356441-21fschecks_count20:05:472
1339428992822,3cyclictest0-21swapper/119:15:011
1339428992821,4cyclictest0-21swapper/120:15:001
1339427992821,4cyclictest1443585-21munin-plugin-st22:35:010
1339429992716,8cyclictest1340257-21ntp_kernel_err19:10:562
1339428992719,5cyclictest1352793-21unixbench_multi19:51:051
1339428992719,5cyclictest0-21swapper/119:40:011
1339428992716,8cyclictest1-21systemd20:04:421
133942799275,6cyclictest0-21swapper/020:30:570
133942799272,7cyclictest1340890-21kworker/0:1+events_freezable_power_19:15:570
1339429992619,4cyclictest1051-21mta-sts-daemon19:31:102
1339429992619,3cyclictest0-21swapper/219:46:022
1339429992617,6cyclictest1372278-21unixbench_multi20:56:062
1339428992621,3cyclictest0-21swapper/120:00:011
1339428992617,6cyclictest1345951-21fschecks_time19:30:491
1339427992618,6cyclictest1371804-21ntp_kernel_pll_20:55:560
133942799261,21cyclictest1375419-21cut21:10:010
1339429992518,4cyclictest1056-21mta-sts-daemon20:15:152
1339428992520,2cyclictest0-21swapper/120:55:011
1339428992517,5cyclictest1373912-21cron21:05:011
133942799250,4cyclictest1351300-21unixbench_multi19:46:070
1339429992416,5cyclictest1344291-21aten_r2power_po19:25:472
133942799241,19cyclictest1345943-21fschecks_time19:30:490
1339429992317,3cyclictest0-21swapper/220:45:592
1339428992317,4cyclictest0-21swapper/120:46:061
133942899231,17cyclictest1356876-21ntpq20:05:581
133942799237,5cyclictest516-21in:imuxsock19:15:010
133942799232,15cyclictest1359885-21ntpq20:16:010
1339427992317,4cyclictest0-21swapper/019:10:010
1339427992315,5cyclictest1349944-21kworker/u8:1-xprtiod19:51:040
1339429992213,5cyclictest1339438-21systemd-run19:10:022
133943099213,9cyclictest0-21swapper/300:21:013
133943099183,8cyclictest0-21swapper/321:30:583
133943099183,7cyclictest0-21swapper/300:01:013
133943099181,10cyclictest0-21swapper/323:10:573
133943099181,10cyclictest0-21swapper/321:56:083
133943099173,10cyclictest0-21swapper/300:36:093
133943099171,8cyclictest0-21swapper/300:05:563
133943099164,8cyclictest0-21swapper/323:15:593
133943099164,8cyclictest0-21swapper/323:15:593
133943099162,9cyclictest0-21swapper/322:41:003
133943099162,9cyclictest0-21swapper/321:40:473
133943099162,8cyclictest0-21swapper/323:35:553
133943099162,8cyclictest0-21swapper/321:50:593
133943099162,8cyclictest0-21swapper/300:31:053
133943099162,7cyclictest0-21swapper/323:45:553
133943099161,9cyclictest0-21swapper/322:35:563
133943099161,6cyclictest0-21swapper/319:21:063
133943099154,6cyclictest0-21swapper/323:50:583
133943099153,7cyclictest0-21swapper/322:26:013
133943099153,7cyclictest0-21swapper/321:35:593
133943099152,8cyclictest0-21swapper/323:55:473
133943099152,8cyclictest0-21swapper/323:05:573
133943099152,8cyclictest0-21swapper/322:50:583
133943099152,8cyclictest0-21swapper/322:50:583
133943099152,8cyclictest0-21swapper/322:05:543
133943099152,8cyclictest0-21swapper/322:05:543
133943099152,8cyclictest0-21swapper/321:20:573
133943099152,7cyclictest0-21swapper/322:30:463
133943099152,6cyclictest0-21swapper/320:50:553
133943099151,9cyclictest0-21swapper/323:25:563
133943099151,8cyclictest0-21swapper/323:31:003
133943099151,8cyclictest0-21swapper/321:10:563
133943099151,8cyclictest0-21swapper/319:15:013
133943099151,7cyclictest0-21swapper/320:05:523
133943099151,6cyclictest0-21swapper/320:15:543
133943099142,8cyclictest0-21swapper/323:00:563
133943099142,8cyclictest0-21swapper/322:00:543
133943099142,8cyclictest0-21swapper/321:17:423
133943099142,8cyclictest0-21swapper/321:17:423
133943099142,7cyclictest0-21swapper/322:10:473
133943099142,7cyclictest0-21swapper/321:46:023
133943099142,7cyclictest0-21swapper/300:25:573
133943099142,7cyclictest0-21swapper/300:25:573
133943099142,6cyclictest0-21swapper/300:15:443
133943099141,8cyclictest0-21swapper/323:20:583
133943099141,8cyclictest0-21swapper/322:55:583
133943099141,8cyclictest0-21swapper/322:21:023
133943099141,8cyclictest0-21swapper/322:15:553
133943099141,7cyclictest0-21swapper/323:40:483
133943099141,7cyclictest0-21swapper/319:30:573
133943099141,7cyclictest0-21swapper/300:10:553
133943099141,6cyclictest0-21swapper/320:20:553
133943099133,4cyclictest0-21swapper/319:15:583
133943099132,6cyclictest0-21swapper/321:25:493
133943099131,8cyclictest0-21swapper/322:45:573
133943099131,7cyclictest0-21swapper/320:35:013
133943099131,6cyclictest0-21swapper/320:46:003
133943099131,6cyclictest0-21swapper/319:40:543
133943099131,6cyclictest0-21swapper/319:25:553
133943099121,6cyclictest0-21swapper/320:01:003
133943099121,6cyclictest0-21swapper/319:45:553
133943099111,6cyclictest0-21swapper/321:10:023
133943099111,6cyclictest0-21swapper/321:00:503
133943099111,6cyclictest0-21swapper/320:55:563
133943099111,6cyclictest0-21swapper/320:35:453
133943099111,6cyclictest0-21swapper/320:15:013
133943099111,6cyclictest0-21swapper/319:10:013
133943099111,5cyclictest0-21swapper/320:40:543
133943099102,5cyclictest0-21swapper/319:56:023
133943099101,6cyclictest0-21swapper/319:50:583
133943099101,6cyclictest0-21swapper/319:36:013
13394309995,3cyclictest0-21swapper/320:25:513
13394309995,3cyclictest0-21swapper/320:25:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional