You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 03:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Mon Apr 06, 2026 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1289832995540,7cyclictest0-21swapper/223:55:352
1289832994940,6cyclictest1384662-21systemd-run22:25:012
1289832994838,4cyclictest0-21swapper/223:40:312
1289830994631,12cyclictest1388437-21latency_hist22:30:020
1289831994534,8cyclictest1450721-21ntpq23:50:371
128983199452,36cyclictest1358840-21rm21:50:311
128983099451,20cyclictest1478022-21ssh00:25:340
1289831994437,4cyclictest0-21swapper/100:15:341
1289831994436,5cyclictest1337988-21ssh21:25:011
1289830994414,24cyclictest1439191-21ssh23:35:380
1289832994335,4cyclictest0-21swapper/220:45:322
1289832994333,4cyclictest0-21swapper/221:45:362
1289832994332,8cyclictest261-21systemd-journal23:37:372
1289832994330,10cyclictest420-21avahi-daemon21:25:402
1289831994333,6cyclictest1392625-21aten_r2power_en22:35:211
128983199433,34cyclictest1454625-21ssh23:55:381
1289832994134,4cyclictest0-21swapper/221:20:012
1289832994133,6cyclictest1418842-21systemd-run23:10:012
1289832994133,4cyclictest1380877-21kworker/u8:14-xprtiod22:55:002
1289832994116,16cyclictest421-21dbus-daemon22:55:362
1289831994133,4cyclictest0-21swapper/121:40:341
128983199413,30cyclictest1362709-21ssh21:55:301
128983199412,34cyclictest1400999-21ssh22:45:331
128983199412,34cyclictest1400999-21ssh22:45:331
1289830994131,8cyclictest1355977-21sshd21:46:200
1289830994130,8cyclictest3552651-21cups-browsed22:15:250
1289830994123,12cyclictest12350irq/32-eth000:15:350
1289830994114,20cyclictest1431529-21ssh23:25:370
1289830994114,20cyclictest1431529-21ssh23:25:370
1289830994113,23cyclictest1327033-21ssh21:10:320
1289832994030,7cyclictest1051-21mta-sts-daemon22:13:242
1289832994028,9cyclictest1366667-21ntp_kernel_pll_22:00:382
1289832994027,5cyclictest0-21swapper/222:40:342
1289832994027,5cyclictest0-21swapper/222:40:342
1289832994027,10cyclictest1335646-21timerwakeupswit21:20:442
128983199402,29cyclictest1416277-21ssh23:05:361
128983099408,10cyclictest1454561-21ntpq23:55:360
1289830994029,9cyclictest1382133-21sh22:20:380
1289832993931,5cyclictest1469024-21latency_hist00:15:012
1289832993920,12cyclictest1458396-21sh00:00:322
1289832993920,12cyclictest1458396-21sh00:00:322
1289831993933,3cyclictest0-21swapper/119:20:001
1289831993931,5cyclictest1310824-21idleruntime-cro20:20:001
1289831993931,5cyclictest1310824-21idleruntime-cro20:20:001
1289831993931,4cyclictest1056-21mta-sts-daemon23:21:341
1289831993931,4cyclictest1056-21mta-sts-daemon23:21:341
1289831993925,9cyclictest1482218-21switchtime00:30:431
1289831993925,9cyclictest1482218-21switchtime00:30:431
1289830993933,4cyclictest1306343-21systemd-run20:05:010
1289830993932,4cyclictest1420352-21timerandwakeup23:10:440
1289830993930,6cyclictest3552625-21cupsd21:56:540
1289830993930,6cyclictest0-21swapper/023:30:300
1289830993925,10cyclictest1407372-21cron22:55:000
1289830993922,14cyclictest1369112-21ssh22:04:320
128983299387,27cyclictest1423767-21ssh23:15:322
1289832993831,4cyclictest1297342-21latency_hist19:35:022
1289832993830,5cyclictest1476914-21latency_hist00:25:022
1289832993829,6cyclictest3552651-21cups-browsed23:00:552
1289832993829,6cyclictest1380824-21munin-run22:20:002
1289832993829,5cyclictest1393867-21unixbench-2d22:35:452
1289832993828,7cyclictest341rcuc/220:35:352
1289832993827,7cyclictest3552651-21cups-browsed22:27:102
1289832993826,9cyclictest1343469-21mailstats21:30:412
1289831993831,3cyclictest0-21swapper/122:30:011
1289831993829,6cyclictest1477854-21needreboot00:25:281
1289831993827,7cyclictest420-21avahi-daemon21:13:351
1289830993829,6cyclictest12350irq/32-eth021:25:320
1289830993810,9cyclictest1347441-21ssh21:35:410
1289830993810,9cyclictest1347441-21ssh21:35:410
1289830993810,8cyclictest1393347-21ntpq22:35:340
1289832993729,5cyclictest1321350-21munin-plugin-st20:55:012
1289832993728,6cyclictest1392225-21true22:35:012
1289832993725,8cyclictest420-21avahi-daemon23:22:112
1289832993725,8cyclictest420-21avahi-daemon23:22:112
1289831993731,3cyclictest0-21swapper/122:20:001
1289831993729,5cyclictest1415062-21systemd-run23:05:011
1289831993715,16cyclictest547-21gmain22:50:321
128983099379,11cyclictest1289989-21sed19:10:020
128983099379,11cyclictest1289989-21sed19:10:020
1289830993732,3cyclictest1054-21mta-sts-daemon00:05:390
1289830993732,3cyclictest1054-21mta-sts-daemon00:05:390
1289830993727,6cyclictest1486186-21threads00:35:440
1289830993726,7cyclictest151ktimers/023:05:300
128983099371,28cyclictest1442828-21ssh23:40:300
1289830993712,16cyclictest0-21swapper/022:55:350
1289830993710,19cyclictest1400958-21ssh22:45:310
1289830993710,19cyclictest1400958-21ssh22:45:310
128983299363,7cyclictest0-21swapper/223:30:322
1289832993626,8cyclictest1370786-21sh22:05:422
128983299362,24cyclictest1481827-21ssh00:30:312
128983299362,24cyclictest1481827-21ssh00:30:312
128983199367,14cyclictest1446902-21ssh23:45:371
1289831993630,3cyclictest0-21swapper/123:45:011
1289831993629,4cyclictest1051-21mta-sts-daemon23:30:331
1289831993628,5cyclictest1369382-21systemd-run22:05:011
1289831993628,5cyclictest1325860-21diskmemload22:55:381
1289831993627,6cyclictest1322845-21grep21:00:011
1289831993626,6cyclictest548-21gdbus23:10:401
128983199362,26cyclictest1389600-21ssh22:30:351
1289831993618,14cyclictest271ktimers/100:35:321
1289830993628,3cyclictest0-21swapper/023:20:320
1289830993628,3cyclictest0-21swapper/023:20:320
1289830993620,12cyclictest12350irq/32-eth023:00:350
1289830993619,15cyclictest1445976-21ssh23:45:160
1289830993612,8cyclictest1459365-21sh00:01:210
1289830993612,8cyclictest1459365-21sh00:01:210
1289830993611,7cyclictest1325860-21diskmemload21:43:310
1289830993611,19cyclictest1335097-21ssh21:20:320
128983299354,3cyclictest171rcu_preempt20:55:362
1289832993527,5cyclictest1363355-21unixbench_singl21:55:412
1289832993525,7cyclictest3552651-21cups-browsed21:43:292
1289832993525,6cyclictest1358123-21/usr/sbin/munin21:50:292
1289831993527,5cyclictest1312316-21cron20:25:001
1289831993526,5cyclictest1339135-21ntp_kernel_pll_21:25:321
128983099359,10cyclictest1299027-21cat19:40:030
128983099356,12cyclictest1392428-21sed22:35:030
1289830993526,6cyclictest1304685-21unixbench_multi19:55:410
1289830993522,9cyclictest1302662-21ntpq19:50:290
1289830993521,9cyclictest1331188-21ntpq21:15:390
1289830993521,8cyclictest1301235-21ntpq19:45:330
1289830993521,8cyclictest1292265-21ntpq19:15:350
1289830993521,11cyclictest1375897-21ssh22:12:330
1289830993520,11cyclictest1313246-21ntpq20:25:330
1289832993425,6cyclictest1461192-21kworker/u8:0-xprtiod00:35:442
1289832993425,5cyclictest1310942-21latency_hist20:20:022
1289832993425,5cyclictest1310942-21latency_hist20:20:022
1289832993420,6cyclictest1325860-21diskmemload23:25:332
1289832993420,6cyclictest1325860-21diskmemload23:25:332
128983199346,22cyclictest1347243-21sshd21:35:361
128983199346,22cyclictest1347243-21sshd21:35:361
1289831993427,3cyclictest0-21swapper/123:20:011
1289831993426,5cyclictest1324876-21fschecks_count21:05:241
1289831993426,5cyclictest1318678-21aten_r2power_po20:45:201
1289831993426,5cyclictest1308882-21systemctl20:10:401
1289831993426,5cyclictest1308882-21systemctl20:10:401
1289831993425,6cyclictest261-21systemd-journal20:05:001
1289831993425,4cyclictest0-21swapper/100:05:341
1289831993425,4cyclictest0-21swapper/100:05:341
1289831993423,8cyclictest1384661-21sh22:25:011
128983199342,26cyclictest1439034-21ssh23:35:321
1289830993426,5cyclictest1-21systemd20:15:010
1289830993426,5cyclictest1-21systemd20:15:010
1289830993425,6cyclictest151ktimers/019:40:290
1289830993418,13cyclictest1473167-21ssh00:20:070
1289832993324,6cyclictest0-21swapper/219:55:282
128983299331,22cyclictest1450668-21ssh23:50:342
128983199337,17cyclictest428-21mta-sts-daemon20:25:331
1289831993326,4cyclictest1055-21mta-sts-daemon19:40:331
1289831993326,4cyclictest0-21swapper/100:10:471
1289831993324,7cyclictest1377300-21latency_hist22:15:031
1289830993327,4cyclictest1469008-21ssh00:15:010
1289830993325,5cyclictest1482409-21unixbench-2d00:30:440
1289830993325,5cyclictest1482409-21unixbench-2d00:30:440
1289830993322,7cyclictest1296719-21ntpq19:30:310
1289830993322,7cyclictest1293775-21ntpq19:20:330
1289830993321,8cyclictest1319237-21ntpq20:45:310
1289830993316,14cyclictest1449864-21ssh23:50:190
1289830993311,14cyclictest0-21swapper/021:05:350
1289832993226,3cyclictest0-21swapper/220:45:012
1289832993226,3cyclictest0-21swapper/220:25:012
1289832993225,5cyclictest0-21swapper/200:15:392
1289832993224,5cyclictest1419183-21acpi23:10:202
1289832993223,6cyclictest420-21avahi-daemon20:25:382
1289831993224,5cyclictest1315190-21tail20:30:421
1289830993223,4cyclictest1310163-21ntpq20:15:330
1289830993223,4cyclictest1310163-21ntpq20:15:330
1289830993221,7cyclictest1316156-21ntpq20:35:320
1289830993219,8cyclictest1358850-21ntpq21:50:310
1289830993217,13cyclictest1396245-21ssh22:40:030
1289830993217,13cyclictest1396245-21ssh22:40:030
1289830993217,12cyclictest1311798-21ntpq20:20:350
1289830993216,14cyclictest1345499-21ssh21:34:000
1289830993216,10cyclictest1307370-21ps20:05:370
1289830993216,10cyclictest1307370-21ps20:05:370
1289830993213,15cyclictest1423811-21ntpq23:15:340
1289832993125,3cyclictest0-21swapper/219:50:292
1289832993123,5cyclictest1298821-21idleruntime-cro19:40:012
1289832993123,4cyclictest0-21swapper/222:45:312
1289832993123,4cyclictest0-21swapper/222:45:312
1289832993121,6cyclictest1051-21mta-sts-daemon20:05:292
1289832993121,6cyclictest1051-21mta-sts-daemon20:05:292
128983199315,19cyclictest1431516-21ssh23:25:361
128983199315,19cyclictest1431516-21ssh23:25:361
128983199313,20cyclictest1473975-21ssh00:20:291
1289831993125,3cyclictest0-21swapper/121:15:471
1289831993121,7cyclictest1325860-21diskmemload00:00:371
1289831993121,7cyclictest1325860-21diskmemload00:00:371
1289830993120,8cyclictest1372617-21sh22:08:460
1289830993120,7cyclictest1322299-21ntpq20:55:340
1289830993120,7cyclictest1290693-21ntpq19:10:300
1289830993119,8cyclictest1323712-21ntpq21:00:310
1289830993119,5cyclictest420-21avahi-daemon20:41:280
1289830993117,11cyclictest1320710-21ntpq20:50:300
1289830993117,11cyclictest1295233-21ntpq19:25:330
1289832993022,5cyclictest1315312-21munin-plugin-st20:35:012
1289831993023,4cyclictest1316806-21idleruntime-cro20:40:011
1289831993023,3cyclictest0-21swapper/122:40:351
1289831993023,3cyclictest0-21swapper/122:40:351
1289831993023,3cyclictest0-21swapper/121:30:331
1289831993022,5cyclictest1353927-21kworker/u8:8+xprtiod21:45:371
1289831993021,6cyclictest1321365-21systemd-run20:55:011
1289831993019,7cyclictest1373258-21ssh22:10:011
1289830993021,4cyclictest1314710-21ntpq20:30:320
1289832992922,4cyclictest0-21swapper/221:36:362
1289832992922,4cyclictest0-21swapper/221:36:362
1289831992921,5cyclictest1304812-21idleruntime-cro20:00:011
1289831992920,3cyclictest0-21swapper/119:35:311
128983299284,20cyclictest1462257-21ssh00:05:302
128983299284,20cyclictest1462257-21ssh00:05:302
1289832992822,3cyclictest0-21swapper/219:25:252
1289832992820,5cyclictest1326324-21aten_r2power_en21:10:212
1289832992815,9cyclictest421-21dbus-daemon19:40:322
1289831992822,3cyclictest0-21swapper/121:00:231
1289831992819,7cyclictest1294329-21munin-plugin-st19:25:011
1289831992818,8cyclictest1295194-21ntp_kernel_pll_19:25:311
128983299273,13cyclictest1446756-21ssh23:45:322
128983299272,18cyclictest261-21systemd-journal19:15:302
1289832992720,4cyclictest1305115-21aten_r2power_en20:00:202
128983299271,20cyclictest1290708-21sshd19:10:312
1289831992719,5cyclictest1298853-21kworker/u8:13+xprtiod19:45:311
128983299262,19cyclictest1478031-21sh00:25:342
128983299262,17cyclictest3552651-21cups-browsed19:20:322
1289832992619,5cyclictest0-21swapper/220:10:352
1289832992619,5cyclictest0-21swapper/220:10:352
1289831992621,2cyclictest0-21swapper/119:50:261
1289831992620,3cyclictest1056-21mta-sts-daemon20:05:421
1289831992620,3cyclictest1056-21mta-sts-daemon20:05:421
1289832992519,3cyclictest0-21swapper/221:05:352
1289831992518,4cyclictest1290180-21aten_r2power_po19:10:211
1289831992516,7cyclictest1318313-21latency_hist20:45:021
128983299246,12cyclictest351ktimers/219:45:342
128983299241,19cyclictest421-21dbus-daemon19:10:012
128983299241,19cyclictest421-21dbus-daemon19:10:012
128983199232,16cyclictest1296792-21ntpq19:30:351
1289831992316,5cyclictest0-21swapper/119:10:011
1289831992316,5cyclictest0-21swapper/119:10:011
1289832992113,5cyclictest1323759-21ntpq21:00:342
128983399194,8cyclictest0-21swapper/322:30:313
128983399192,8cyclictest0-21swapper/323:10:323
128983399192,8cyclictest0-21swapper/323:00:333
128983399182,9cyclictest0-21swapper/323:50:403
128983399182,9cyclictest0-21swapper/322:50:293
128983399181,7cyclictest0-21swapper/300:25:313
128983399174,9cyclictest0-21swapper/323:20:323
128983399174,9cyclictest0-21swapper/323:20:323
128983399173,6cyclictest0-21swapper/323:15:313
128983399173,6cyclictest0-21swapper/320:15:343
128983399173,6cyclictest0-21swapper/320:15:343
128983399172,9cyclictest0-21swapper/323:30:353
128983399171,9cyclictest0-21swapper/321:25:333
128983399164,9cyclictest0-21swapper/322:20:313
128983399164,6cyclictest0-21swapper/321:55:353
128983399163,8cyclictest0-21swapper/322:55:363
128983399162,9cyclictest0-21swapper/322:15:303
128983399162,9cyclictest0-21swapper/321:45:343
128983399162,9cyclictest0-21swapper/300:15:203
128983399162,8cyclictest0-21swapper/321:40:303
128983399162,8cyclictest0-21swapper/321:00:343
128983399162,7cyclictest0-21swapper/323:45:303
128983399162,10cyclictest0-21swapper/323:55:383
128983399162,10cyclictest0-21swapper/322:40:343
128983399162,10cyclictest0-21swapper/322:40:343
128983399161,7cyclictest0-21swapper/321:35:233
128983399161,7cyclictest0-21swapper/321:35:233
128983399155,7cyclictest0-21swapper/323:25:373
128983399155,7cyclictest0-21swapper/323:25:373
128983399153,6cyclictest0-21swapper/321:50:293
128983399152,9cyclictest0-21swapper/322:00:383
128983399152,9cyclictest0-21swapper/300:00:433
128983399152,9cyclictest0-21swapper/300:00:433
128983399152,8cyclictest0-21swapper/323:35:213
128983399152,8cyclictest0-21swapper/322:10:323
128983399152,8cyclictest0-21swapper/322:05:333
128983399152,8cyclictest0-21swapper/321:20:333
128983399152,8cyclictest0-21swapper/321:10:303
128983399152,8cyclictest0-21swapper/300:20:323
128983399152,8cyclictest0-21swapper/300:10:243
128983399152,6cyclictest0-21swapper/320:00:293
128983399151,8cyclictest0-21swapper/323:40:293
128983399151,8cyclictest0-21swapper/321:30:223
128983399151,7cyclictest0-21swapper/320:48:383
128983399151,7cyclictest0-21swapper/320:20:293
128983399144,7cyclictest0-21swapper/300:05:333
128983399144,7cyclictest0-21swapper/300:05:333
128983399144,6cyclictest0-21swapper/300:37:223
128983399143,7cyclictest0-21swapper/323:05:213
128983399143,7cyclictest0-21swapper/320:10:343
128983399143,7cyclictest0-21swapper/320:10:343
128983399142,7cyclictest0-21swapper/320:30:373
128983399142,5cyclictest0-21swapper/319:10:283
128983399141,8cyclictest0-21swapper/322:35:213
128983399141,8cyclictest0-21swapper/322:25:283
128983399141,8cyclictest0-21swapper/321:15:343
128983399141,8cyclictest0-21swapper/300:30:303
128983399141,8cyclictest0-21swapper/300:30:303
128983399141,7cyclictest0-21swapper/321:05:363
128983399141,7cyclictest0-21swapper/319:55:323
128983399141,6cyclictest0-21swapper/320:55:353
128983399141,6cyclictest0-21swapper/320:50:333
128983399141,6cyclictest0-21swapper/320:35:363
128983399141,6cyclictest0-21swapper/319:20:303
128983399140,8cyclictest0-21swapper/322:45:303
128983399140,8cyclictest0-21swapper/322:45:303
128983399132,6cyclictest0-21swapper/319:40:313
128983399131,7cyclictest0-21swapper/320:40:403
128983399131,7cyclictest0-21swapper/320:05:313
128983399131,7cyclictest0-21swapper/320:05:313
128983399131,7cyclictest0-21swapper/319:15:303
128983399131,6cyclictest0-21swapper/320:25:283
128983399131,6cyclictest0-21swapper/319:25:293
128983399122,6cyclictest0-21swapper/319:45:313
128983399121,6cyclictest0-21swapper/319:50:323
128983399121,6cyclictest0-21swapper/319:30:313
128983399112,5cyclictest0-21swapper/319:35:243
12898339981,4cyclictest0-21swapper/319:10:023
12898339981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional