You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 11:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 02, 2026 00:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1246130995632,21cyclictest753-21ntpd00:15:360
1246130995620,4cyclictest12450irq/33-eth022:50:290
1246132995227,21cyclictest1320786-21grep21:55:412
1246132995125,21cyclictest0-21swapper/223:55:372
1246131995123,25cyclictest1256278-21switchtime19:40:381
1246130995120,28cyclictest421-21dbus-daemon00:03:050
1246131995033,14cyclictest1306875-21kworker/u8:3-xprtiod22:35:351
124613099508,14cyclictest421-21dbus-daemon23:30:040
1246132994927,17cyclictest0-21swapper/223:30:342
1246132994836,8cyclictest421-21dbus-daemon23:40:312
1246131994838,6cyclictest1376357-21latency_hist23:05:011
1246131994838,6cyclictest1376357-21latency_hist23:05:011
1246130994835,9cyclictest420-21avahi-daemon23:40:310
1246131994734,9cyclictest1384449-21sh23:15:001
1246131994734,9cyclictest1384449-21sh23:15:001
1246131994724,19cyclictest1252635-21fschecks_count19:30:211
1246130994723,21cyclictest0-21swapper/021:20:420
1246132994617,26cyclictest1262563-21unixbench_singl20:00:412
1246131994637,6cyclictest1261093-21cron20:00:001
1246130994618,25cyclictest0-21swapper/022:25:270
124613099461,22cyclictest1385538-21ntpq23:15:290
1246132994435,6cyclictest1442077-21/usr/sbin/munin00:25:242
1246132994430,11cyclictest1361039-21sh22:45:292
1246132994422,18cyclictest0-21swapper/200:35:022
1246132994419,22cyclictest1433541-21/usr/sbin/munin00:15:022
1246131994421,20cyclictest0-21swapper/122:10:341
1246130994421,21cyclictest421-21dbus-daemon23:00:020
1246130994416,3cyclictest1294631-21kworker/u8:1-rpciod23:14:350
1246130994416,3cyclictest1294631-21kworker/u8:1-rpciod23:14:350
1246132994335,5cyclictest1290899-21aten_r2power_en21:20:192
124613299431,5cyclictest1260523-21ntpq19:55:312
124613199438,33cyclictest1322696-21ssh21:59:131
1246131994337,4cyclictest1404953-21systemd-run23:40:011
1246130994326,14cyclictest0-21swapper/000:25:330
1246130994321,19cyclictest0-21swapper/022:10:240
1246130994318,2cyclictest0-21swapper/021:25:320
124613099431,14cyclictest1273993-21ntpq20:40:280
1246132994224,15cyclictest1267103-21cron20:20:002
1246132994224,14cyclictest0-21swapper/223:50:302
124613199423,6cyclictest1268040-21ntpq20:20:311
124613199423,5cyclictest1262049-21ntpq20:00:321
124613199422,7cyclictest1265074-21ntpq20:10:311
124613199422,7cyclictest1265074-21ntpq20:10:311
1246131994221,18cyclictest0-21swapper/122:55:011
124613199421,10cyclictest1309416-21ssh21:42:161
1246130994225,14cyclictest3487809-21cups-browsed22:20:120
1246130994222,17cyclictest1-21systemd20:20:180
1246130994222,17cyclictest0-21swapper/022:00:230
1246130994222,17cyclictest0-21swapper/022:00:230
1246130994221,18cyclictest428-21mta-sts-daemon20:00:410
124613299419,29cyclictest1277617-21sh20:55:012
124613299412,14cyclictest1257121-21diskstats19:45:222
124613299411,13cyclictest1411264-21ssh23:46:322
124613199418,30cyclictest1399381-21ssh23:32:171
1246131994132,5cyclictest420-21avahi-daemon00:39:031
124613199412,6cyclictest1348781-21ntpq22:30:301
124613199412,6cyclictest1248528-21ntpq19:15:301
124613199411,9cyclictest1258974-21ntpq19:50:301
124613199411,10cyclictest1448532-21ssh00:32:301
1246130994117,6cyclictest3487809-21cups-browsed20:25:320
1246130994110,10cyclictest0-21swapper/020:10:380
1246130994110,10cyclictest0-21swapper/020:10:380
1246132994032,5cyclictest1345272-21unixbench_multi22:25:412
1246132994032,5cyclictest1298974-21latency_hist21:30:022
1246132994031,6cyclictest3487809-21cups-browsed23:25:012
1246132994027,5cyclictest1343559-21/usr/sbin/munin22:25:022
124613299402,15cyclictest1250043-21ntpq19:20:292
124613299402,14cyclictest1299916-21ntpq21:30:312
1246132994020,17cyclictest0-21swapper/222:30:222
1246132994017,19cyclictest0-21swapper/222:40:422
124613299401,11cyclictest1309943-21ssh21:43:152
124613299401,10cyclictest1404222-21ssh23:38:422
124613199409,28cyclictest1314754-21sh21:49:391
124613199408,3cyclictest1347561-21ssh22:29:521
124613199403,6cyclictest1389742-21ntpq23:20:311
1246131994031,6cyclictest1422315-21needreboot00:00:261
124613199402,5cyclictest1266526-21ntpq20:15:301
124613199402,4cyclictest1271020-21ntpq20:30:301
124613199401,9cyclictest1436486-21ssh00:17:521
124613199401,9cyclictest1387446-21ssh23:17:581
124613199401,9cyclictest1281474-21ntpq21:05:281
124613199401,9cyclictest1274073-21ntp_states20:40:321
124613199401,7cyclictest1249995-21ntpq19:20:271
1246131994017,20cyclictest0-21swapper/121:50:331
1246131994014,23cyclictest1275485-21ntp_kernel_pll_20:45:281
124613199401,12cyclictest1247095-21ntpq19:10:351
124613199401,11cyclictest1443629-21ssh00:25:541
124613199401,10cyclictest1439925-21ssh00:21:371
124613199401,10cyclictest1427900-21ssh00:07:031
124613199401,10cyclictest1417492-21ssh23:55:121
124613199401,10cyclictest1362347-21ssh22:46:521
124613199401,10cyclictest1326268-21ssh22:03:151
124613199401,10cyclictest1326268-21ssh22:03:151
124613199401,10cyclictest1306541-21ssh21:39:371
124613199401,10cyclictest1302143-21ssh21:34:031
124613199401,10cyclictest1298233-21ssh21:29:021
124613099407,6cyclictest1355815-21munin-run22:40:010
124613099402,13cyclictest1307853-21ssh21:40:280
124613099401,9cyclictest1417168-21debian-sa123:55:010
1246130994018,4cyclictest1286299-21systemd-run21:15:010
124613299394,8cyclictest1365117-21ntpq22:50:292
124613299393,9cyclictest1369162-21ntpq22:55:282
124613299392,7cyclictest1254596-21ntpq19:35:312
124613299391,9cyclictest1379711-21ssh23:08:462
124613299391,9cyclictest1379711-21ssh23:08:462
1246132993915,20cyclictest548-21gdbus00:20:532
124613299391,3cyclictest1282111-21diskmemload00:00:152
124613299391,12cyclictest1373398-21ssh23:00:322
124613299391,12cyclictest1373398-21ssh23:00:322
124613299391,10cyclictest1251538-21ntpq19:25:292
1246132993910,2cyclictest0-21swapper/221:12:592
1246131993924,12cyclictest1377169-21latency23:05:261
1246131993924,12cyclictest1377169-21latency23:05:261
124613199391,9cyclictest1329847-21ssh22:07:181
1246131993918,18cyclictest0-21swapper/122:20:351
124613199391,7cyclictest1336055-21/usr/sbin/munin22:15:231
124613199391,6cyclictest1405987-21ntpq23:40:261
1246131993912,24cyclictest3487809-21cups-browsed23:50:361
124613199391,13cyclictest1251613-21ntpq19:25:331
124613199391,11cyclictest1284506-21ssh21:11:451
124613199391,11cyclictest1269495-21ntpq20:25:281
124613199391,10cyclictest1431866-21ssh00:11:511
124613199391,10cyclictest1411617-21ssh23:47:121
124613199391,10cyclictest1370678-21ssh22:57:121
124613199391,10cyclictest1294404-21ssh21:24:391
124613099394,12cyclictest1282111-21diskmemload00:21:150
124613099392,14cyclictest1418408-21ntpq23:55:350
124613099392,14cyclictest1357092-21ntpq22:40:350
1246130993916,3cyclictest0-21swapper/023:10:010
1246130993916,3cyclictest0-21swapper/023:10:010
1246130993916,2cyclictest0-21swapper/020:20:000
124613299382,6cyclictest1393909-21ntpq23:25:342
124613299382,6cyclictest1275588-21ntpq20:45:322
124613299382,14cyclictest1381575-21sh23:10:312
124613299382,14cyclictest1381575-21sh23:10:312
124613299381,9cyclictest1436745-21ssh00:18:222
124613299381,9cyclictest1317928-21ssh21:52:572
124613299381,9cyclictest1290412-21ssh21:20:002
124613299381,9cyclictest1268048-21ntpq20:20:312
1246132993817,18cyclictest0-21swapper/220:30:022
1246132993817,17cyclictest0-21swapper/200:05:332
1246132993816,19cyclictest0-21swapper/200:40:012
1246132993815,20cyclictest0-21swapper/221:00:222
1246132993815,20cyclictest0-21swapper/221:00:222
1246132993814,20cyclictest0-21swapper/221:40:012
1246132993813,22cyclictest0-21swapper/221:05:412
124613299381,10cyclictest1354493-21ssh22:37:322
124613299381,10cyclictest1331560-21ssh22:10:082
124613299381,10cyclictest1330636-21ssh22:08:482
124613199382,5cyclictest1272486-21ntpq20:35:291
124613199382,5cyclictest1257522-21ntpq19:45:301
124613199381,9cyclictest1394571-21ssh23:25:541
124613199381,9cyclictest1357665-21ssh22:40:451
124613199381,8cyclictest1280023-21ntpq21:00:301
124613199381,8cyclictest1280023-21ntpq21:00:301
124613199381,8cyclictest1263521-21ntpq20:05:311
124613199381,12cyclictest1286970-21diskstats21:15:241
124613199381,10cyclictest1255087-21sshd19:37:311
1246130993832,4cyclictest1327859-21cpuspeed22:05:240
124613099382,14cyclictest1427137-21unixbench_singl00:05:400
124613099381,18cyclictest1287434-21ssh21:15:320
124613099381,17cyclictest1247008-21ntpq19:10:300
124613299379,7cyclictest421-21dbus-daemon21:00:012
1246132993729,5cyclictest1273106-21idleruntime-cro20:40:012
124613299371,9cyclictest1326143-21ssh22:03:002
124613299371,9cyclictest1326143-21ssh22:03:002
124613299371,9cyclictest1248520-21ntpq19:15:292
1246132993715,19cyclictest1336523-21sed22:15:312
1246132993712,22cyclictest1263165-21grep20:05:242
124613299371,11cyclictest1311110-21ssh21:45:032
124613299371,10cyclictest1387510-21ssh23:18:052
124613299371,10cyclictest1246656-21/usr/sbin/munin19:10:232
124613199372,6cyclictest1277035-21ntpq20:50:311
124613099378,11cyclictest1450388-21aten_r2power_po00:35:190
1246130993732,3cyclictest0-21swapper/022:15:210
1246130993730,4cyclictest1364047-21kworker/u8:7-rpciod00:10:360
1246130993730,4cyclictest1299480-21fschecks_count21:30:230
1246130993729,5cyclictest420-21avahi-daemon23:23:160
1246130993719,4cyclictest1315474-21aten2.4-expect21:50:200
1246130993718,16cyclictest0-21swapper/023:30:020
1246130993712,8cyclictest1348736-21ssh22:30:280
124613099371,16cyclictest1410177-21ntpq23:45:300
124613099371,16cyclictest1373313-21awk23:00:290
124613099371,16cyclictest1373313-21awk23:00:290
124613299366,7cyclictest1252825-21sort19:30:242
124613299363,12cyclictest1259079-21ntpq19:50:352
124613299362,7cyclictest1265486-21grep20:10:382
124613299362,7cyclictest1265486-21grep20:10:382
124613199361,7cyclictest1278565-21ntpq20:55:341
124613099364,8cyclictest1260478-21ntpq19:55:290
124613099363,9cyclictest1303817-21ntpq21:35:300
1246130993631,3cyclictest1246129-21cyclictest23:35:270
1246130993626,8cyclictest1253639-21kworker/u8:14+rpciod19:50:310
1246130993618,3cyclictest1054-21mta-sts-daemon21:00:310
1246130993618,3cyclictest1054-21mta-sts-daemon21:00:310
124613099361,14cyclictest1446464-21/usr/sbin/munin00:30:230
124613299352,7cyclictest1256095-21ntpq19:40:322
124613299352,11cyclictest1270609-21diskstats20:30:212
1246132993511,2cyclictest0-21swapper/220:40:352
1246130993525,7cyclictest1361396-21switchtime22:45:390
124613099350,17cyclictest1254578-21ntpq19:35:300
124613299341,12cyclictest1246138-21systemd-run19:10:012
1246130993426,5cyclictest421-21dbus-daemon21:45:430
124613099341,14cyclictest1319680-21df21:55:200
1246131993311,18cyclictest0-21swapper/119:10:011
124613099334,8cyclictest1054-21mta-sts-daemon19:25:260
1246130993326,5cyclictest3487809-21cups-browsed20:55:380
1246130993223,7cyclictest1-21systemd20:55:010
1246130993123,6cyclictest1275513-21ntp_kernel_pll_20:45:290
1246130993024,4cyclictest1248138-21sort19:15:210
1246130993014,2cyclictest1253748-21kworker/u8:15-xprtiod19:40:270
1246130992916,11cyclictest0-21swapper/021:05:250
1246130992820,6cyclictest1250235-21mailstats19:20:360
1246130992721,4cyclictest1263083-21cpuspeed20:05:210
1246130992419,2cyclictest0-21swapper/020:35:210
124613099222,15cyclictest1253009-21ntpq19:30:290
1246130992216,3cyclictest0-21swapper/020:30:200
124613099221,17cyclictest1257640-21proc_pri19:45:350
1246130991913,4cyclictest421-21dbus-daemon19:10:010
124613399185,8cyclictest0-21swapper/300:15:303
124613399174,8cyclictest0-21swapper/321:30:293
124613399174,7cyclictest0-21swapper/322:50:283
124613399166,6cyclictest0-21swapper/322:35:353
124613399165,8cyclictest0-21swapper/323:20:343
124613399163,9cyclictest0-21swapper/323:40:313
124613399163,9cyclictest0-21swapper/322:30:283
124613399163,9cyclictest0-21swapper/320:55:313
124613399163,9cyclictest0-21swapper/319:50:373
124613399163,8cyclictest0-21swapper/322:40:293
124613399163,8cyclictest0-21swapper/300:05:283
124613399163,10cyclictest0-21swapper/322:05:313
124613399163,10cyclictest0-21swapper/321:25:323
124613399162,9cyclictest0-21swapper/323:30:333
124613399162,9cyclictest0-21swapper/323:25:313
124613399162,9cyclictest0-21swapper/322:00:233
124613399162,9cyclictest0-21swapper/322:00:233
124613399155,7cyclictest0-21swapper/322:25:353
124613399155,7cyclictest0-21swapper/322:20:343
124613399155,7cyclictest0-21swapper/321:45:323
124613399154,8cyclictest0-21swapper/322:10:323
124613399154,8cyclictest0-21swapper/321:35:273
124613399154,7cyclictest0-21swapper/323:05:343
124613399154,7cyclictest0-21swapper/323:05:343
124613399154,7cyclictest0-21swapper/300:00:303
124613399153,8cyclictest0-21swapper/323:50:253
124613399153,8cyclictest0-21swapper/323:15:333
124613399153,8cyclictest0-21swapper/322:45:283
124613399153,8cyclictest0-21swapper/300:30:363
124613399153,8cyclictest0-21swapper/300:25:273
124613399152,9cyclictest0-21swapper/323:55:293
124613399152,9cyclictest0-21swapper/323:00:283
124613399152,9cyclictest0-21swapper/323:00:283
124613399152,9cyclictest0-21swapper/322:55:313
124613399152,9cyclictest0-21swapper/322:15:303
124613399152,9cyclictest0-21swapper/321:10:223
124613399152,9cyclictest0-21swapper/319:40:013
124613399152,9cyclictest0-21swapper/319:30:333
124613399152,9cyclictest0-21swapper/319:25:283
124613399152,9cyclictest0-21swapper/300:20:303
124613399152,9cyclictest0-21swapper/300:10:203
124613399152,8cyclictest0-21swapper/321:40:343
124613399152,8cyclictest0-21swapper/320:45:283
124613399152,8cyclictest0-21swapper/320:25:303
124613399152,10cyclictest0-21swapper/320:10:283
124613399152,10cyclictest0-21swapper/320:10:283
124613399152,10cyclictest0-21swapper/300:35:273
124613399151,10cyclictest0-21swapper/321:15:353
124613399144,7cyclictest0-21swapper/319:55:283
124613399143,7cyclictest0-21swapper/323:45:313
124613399143,7cyclictest0-21swapper/323:10:193
124613399143,7cyclictest0-21swapper/323:10:193
124613399143,7cyclictest0-21swapper/321:20:263
124613399143,7cyclictest0-21swapper/320:05:293
124613399143,7cyclictest0-21swapper/319:20:333
124613399143,6cyclictest0-21swapper/319:10:283
124613399142,8cyclictest0-21swapper/321:55:283
124613399142,8cyclictest0-21swapper/321:00:243
124613399142,8cyclictest0-21swapper/321:00:243
124613399142,8cyclictest0-21swapper/320:35:213
124613399142,8cyclictest0-21swapper/320:00:313
124613399141,8cyclictest0-21swapper/320:15:303
124613399141,10cyclictest0-21swapper/321:50:343
124613399132,8cyclictest0-21swapper/323:35:253
124613399132,8cyclictest0-21swapper/320:30:303
124613399132,8cyclictest0-21swapper/319:40:303
124613399132,7cyclictest0-21swapper/320:50:283
124613399132,7cyclictest0-21swapper/319:15:273
124613399131,8cyclictest0-21swapper/321:05:353
124613399131,8cyclictest0-21swapper/320:20:273
124613399122,7cyclictest0-21swapper/320:40:223
124613399121,7cyclictest0-21swapper/319:45:223
124613399102,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional