You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 01:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu Jul 30, 2026 12:43:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1180427995445,7cyclictest1245343-21sh09:55:560
1180427995042,5cyclictest1271521-21munin-run10:40:010
118042999474,36cyclictest1312324-21ssh11:45:482
118042999474,36cyclictest1312324-21ssh11:45:482
1180429994631,7cyclictest1318644-21mailstats11:55:572
1180427994639,4cyclictest1220648-21ssh09:15:500
118042999455,32cyclictest1272690-21ssh10:40:492
118042999455,32cyclictest1272690-21ssh10:40:492
1180427994538,4cyclictest1320298-21ssh12:00:010
1180429994334,6cyclictest1271521-21munin-run10:40:012
1180428994332,8cyclictest1308111-21idleruntime-cro11:40:011
1180427994332,9cyclictest1278617-21missed_timers10:50:470
1180427994332,9cyclictest1278617-21missed_timers10:50:470
1180429994235,4cyclictest1195528-21kworker/u8:1-rpciod08:15:012
1180429994234,5cyclictest1280677-21cron10:55:002
1180429994234,5cyclictest1280677-21cron10:55:002
1180428994229,9cyclictest1271540-21idleruntime-cro10:40:011
1180429994133,5cyclictest1-21systemd10:00:012
1180429994032,5cyclictest421-21dbus-daemon12:15:012
1180429994032,5cyclictest1326485-21latency_hist12:10:022
1180429994032,5cyclictest1268488-21munin-plugin-st10:35:012
1180429994031,6cyclictest427-21polkitd12:05:012
1180429994027,9cyclictest1-21systemd09:15:372
118042899405,28cyclictest1217617-21ssh09:10:531
1180428994032,5cyclictest421-21dbus-daemon08:25:011
1180428994032,5cyclictest1253395-21latency_hist10:10:021
1180428994028,6cyclictest1312299-21sh11:45:471
1180428994028,6cyclictest1312299-21sh11:45:471
1180427994033,4cyclictest1298979-21cron11:25:010
1180427994032,5cyclictest1332731-21latency_hist12:20:030
118042999393,27cyclictest1233040-21ssh09:35:522
1180429993931,5cyclictest1281534-21kernelversion10:55:412
1180428993931,5cyclictest1274584-21cron10:45:001
1180428993931,5cyclictest1274584-21cron10:45:001
1180428993931,5cyclictest1205923-21systemd-run08:35:011
1180427993933,3cyclictest0-21swapper/011:50:010
1180427993933,3cyclictest0-21swapper/011:50:010
1180427993932,4cyclictest1000-21mta-sts-daemon11:00:560
1180427993931,5cyclictest1216427-21systemd-run09:10:020
1180427993928,8cyclictest1295922-21idleruntime-cro11:20:010
1180429993829,6cyclictest265-21systemd-journal09:33:152
1180429993829,6cyclictest265-21systemd-journal09:33:152
1180427993830,5cyclictest1218045-21ssh09:11:020
1180427993829,6cyclictest485-21in:imuxsock07:50:010
118042799382,31cyclictest1266572-21ssh10:30:500
1180427993822,12cyclictest0-21swapper/010:45:020
1180427993822,12cyclictest0-21swapper/010:45:020
1180429993729,5cyclictest1262381-21cron10:25:012
1180429993729,5cyclictest1247845-21cut10:00:392
1180429993729,5cyclictest1184941-21systemd-run07:25:012
1180429993727,6cyclictest600-21gdbus09:27:012
1180429993727,6cyclictest600-21gdbus09:27:012
1180429993726,8cyclictest1190934-21sh07:45:012
1180428993731,4cyclictest1283743-21munin-run11:00:011
1180428993730,3cyclictest0-21swapper/110:10:381
1180428993729,5cyclictest1194436-21fschecks_time07:55:421
1180428993728,6cyclictest1318916-21unixbench-2d11:56:011
1180428993728,6cyclictest1216403-21munin-run09:10:011
1180428993728,6cyclictest1207393-21idleruntime-cro08:40:001
1180428993728,6cyclictest1207393-21idleruntime-cro08:40:001
1180428993727,7cyclictest1244995-21latency09:55:451
1180428993726,8cyclictest1321526-21ntp_offset12:00:531
1180427993729,5cyclictest1223310-21fschecks_count09:20:430
1180427993723,10cyclictest151ktimers/012:30:510
1180429993630,3cyclictest0-21swapper/207:20:012
1180428993630,3cyclictest0-21swapper/107:55:011
1180428993627,6cyclictest1-21systemd10:30:011
1180428993626,8cyclictest1183411-21/usr/sbin/munin07:20:011
118042799365,29cyclictest1209810-21kworker/0:0+events_power_efficient08:45:530
118042799363,29cyclictest1229978-21ssh09:30:530
118042799363,29cyclictest1229978-21ssh09:30:530
1180427993628,5cyclictest1325071-21unixbench_multi12:06:000
1180427993626,7cyclictest389646-21cups-browsed12:00:180
1180427993626,7cyclictest1-21systemd09:27:010
1180427993626,7cyclictest1-21systemd09:27:010
1180427993624,9cyclictest1216400-21diskmemload10:26:000
1180427993623,10cyclictest1259334-21munin-run10:20:010
1180427993620,13cyclictest0-21swapper/012:25:380
1180429993528,4cyclictest1250911-21fschecks_count10:05:412
1180429993528,4cyclictest1208899-21cron08:45:002
1180429993528,4cyclictest1208899-21cron08:45:002
1180429993528,4cyclictest1206219-21kworker/u8:6-xprtiod09:00:012
1180429993527,5cyclictest389646-21cups-browsed07:50:412
1180429993526,6cyclictest567-21gdbus08:05:542
1180428993527,5cyclictest1250180-21cron10:05:011
1180428993527,5cyclictest1208902-21cron08:45:001
1180428993527,5cyclictest1208902-21cron08:45:001
1180428993527,5cyclictest1186406-21cron07:30:011
1180428993526,6cyclictest1292875-21cron11:15:011
1180428993524,7cyclictest421-21dbus-daemon10:35:021
118042799353,29cyclictest1306145-21ssh11:35:450
1180427993529,4cyclictest1232590-21fschecks_time09:35:420
1180427993529,3cyclictest0-21swapper/011:45:000
1180427993529,3cyclictest0-21swapper/011:45:000
1180427993522,9cyclictest0-21swapper/010:20:390
1180427993520,11cyclictest0-21swapper/011:30:440
118042999349,22cyclictest1002-21mta-sts-daemon11:30:502
118042999345,22cyclictest389643-21cupsd11:05:492
1180429993428,3cyclictest0-21swapper/209:15:012
1180429993427,4cyclictest1338799-21ssh12:30:032
1180429993426,5cyclictest1306029-21missed_timers11:35:432
1180428993427,4cyclictest1231268-21ssh09:33:161
1180428993427,4cyclictest1231268-21ssh09:33:161
1180428993424,7cyclictest439-21systemd-logind08:15:011
1180428993423,8cyclictest1342192-21aten2.4-expect12:35:371
1180428993423,8cyclictest1257866-21timerwakeupswit10:16:021
1180428993423,8cyclictest1226901-21sh09:25:551
1180428993423,8cyclictest1226901-21sh09:25:551
118042799347,22cyclictest1333617-21ssh12:20:490
1180427993427,3cyclictest1262409-21kworker/u8:14-xprtiod12:15:010
118042799342,28cyclictest1290964-21ssh11:10:480
118042799342,25cyclictest1239294-21ssh09:45:570
1180429993328,2cyclictest0-21swapper/212:20:012
1180429993326,3cyclictest0-21swapper/211:15:012
1180429993325,4cyclictest1003-21mta-sts-daemon07:55:492
1180429993323,7cyclictest1000-21mta-sts-daemon11:53:302
1180429993323,7cyclictest1000-21mta-sts-daemon11:53:302
1180428993325,5cyclictest1234923-21munin-plugin-st09:40:001
1180428993325,5cyclictest1225704-21latency_hist09:25:011
1180428993325,4cyclictest1280680-21cron10:55:001
1180428993325,4cyclictest1280680-21cron10:55:001
118042899331,26cyclictest1216400-21diskmemload11:15:531
1180427993326,4cyclictest1199898-21munin-plugin-st08:15:010
1180427993326,4cyclictest1195000-21mailstats07:55:560
1180427993325,5cyclictest1315267-21needreboot11:50:470
1180427993325,5cyclictest1315267-21needreboot11:50:470
1180427993321,3cyclictest0-21swapper/010:06:000
118042999327,14cyclictest1294040-21ssh11:15:502
1180429993226,3cyclictest0-21swapper/212:20:492
1180429993225,4cyclictest1202913-21cron08:25:012
1180429993224,5cyclictest1235689-21fschecks_time09:40:422
1180429993224,5cyclictest1003-21mta-sts-daemon11:44:312
1180429993224,5cyclictest1003-21mta-sts-daemon11:44:312
1180429993222,8cyclictest1254087-21interrupts10:10:432
1180428993226,3cyclictest0-21swapper/109:15:491
1180428993224,5cyclictest1287543-21fschecks_time11:05:421
1180428993224,5cyclictest1201280-21unixbench_multi08:15:591
1180428993223,6cyclictest1003-21mta-sts-daemon09:47:411
1180427993226,4cyclictest151ktimers/010:10:490
1180427993219,3cyclictest0-21swapper/012:39:380
118042999312,24cyclictest1263486-21rm10:25:462
1180428993126,3cyclictest1000-21mta-sts-daemon07:30:411
1180428993124,4cyclictest1002-21mta-sts-daemon09:43:331
1180428993124,4cyclictest0-21swapper/108:50:531
1180428993122,6cyclictest1261054-21unixbench_multi10:21:021
1180427993125,4cyclictest1211897-21munin-run08:55:010
1180427993124,5cyclictest1280706-21kworker/u8:6+xprtiod11:00:000
1180427993122,5cyclictest151ktimers/009:50:550
1180427993122,5cyclictest1192822-21sshd07:50:380
1180427993119,8cyclictest12650irq/32-eth010:00:460
1180429993024,3cyclictest0-21swapper/210:20:012
1180429993023,5cyclictest0-21swapper/209:55:022
1180429993023,4cyclictest1282975-21kworker/u8:9-xprtiod11:00:372
1180429993019,4cyclictest1222841-21ssh09:20:092
1180429993018,9cyclictest1204421-21sh08:30:012
1180428993023,4cyclictest1003-21mta-sts-daemon09:53:301
1180428993021,7cyclictest694-21cron09:00:011
1180428993021,6cyclictest1300718-21unixbench_singl11:25:591
1180428993020,6cyclictest1181933-21systemd-run07:15:011
1180428993019,8cyclictest1317296-21ssh11:55:011
1180428993019,8cyclictest1317296-21ssh11:55:011
1180428993018,8cyclictest1-21systemd12:15:021
1180427993023,4cyclictest1236704-21unixbench_singl09:41:040
1180427993023,3cyclictest0-21swapper/011:08:000
1180427993022,6cyclictest1299395-21/usr/sbin/munin11:26:000
1180427993022,4cyclictest0-21swapper/010:45:540
1180427993022,4cyclictest0-21swapper/010:45:540
1180427993020,7cyclictest1181260-21ntp_kernel_err07:10:460
1180427993010,14cyclictest151ktimers/007:30:480
1180429992923,4cyclictest1180426-21cyclictest12:30:472
1180429992923,3cyclictest0-21swapper/212:40:012
1180429992922,4cyclictest1216400-21sh09:10:012
1180429992922,4cyclictest1205892-21cron08:35:012
1180429992921,5cyclictest1211889-21sh08:55:012
1180428992922,4cyclictest1305075-21cut11:35:011
1180428992921,5cyclictest1286874-21latency_hist11:05:021
1180428992920,6cyclictest1-21systemd07:35:241
1180428992910,16cyclictest271ktimers/112:25:521
1180427992923,4cyclictest1202909-21cron08:25:010
1180427992922,5cyclictest485-21in:imuxsock09:05:010
1180427992920,6cyclictest1213380-21sh09:00:010
118042999282,20cyclictest1275706-21ssh10:45:482
118042999282,20cyclictest1275706-21ssh10:45:482
1180429992821,3cyclictest0-21swapper/208:20:012
1180429992820,5cyclictest0-21swapper/207:40:012
1180428992822,4cyclictest1180426-21cyclictest11:40:391
1180428992822,4cyclictest1180426-21cyclictest11:40:391
1180428992821,4cyclictest1184860-21unixbench_singl07:20:591
118042799287,14cyclictest428-21mta-sts-daemon08:00:510
1180427992823,3cyclictest14-21ksoftirqd/008:40:470
1180427992823,3cyclictest14-21ksoftirqd/008:40:470
1180427992819,7cyclictest1184733-21timerwakeupswit07:20:580
118042799281,19cyclictest151ktimers/008:35:470
118042799281,19cyclictest151ktimers/008:35:470
1180429992720,3cyclictest0-21swapper/211:25:472
1180428992720,3cyclictest0-21swapper/110:45:501
1180428992720,3cyclictest0-21swapper/110:45:501
1180428992718,6cyclictest1330743-21ssh12:15:551
1180427992721,4cyclictest1201327-21unixbench_singl08:16:000
1180427992721,3cyclictest0-21swapper/007:30:020
1180427992720,4cyclictest1188140-21/usr/sbin/munin07:35:500
1180427992713,7cyclictest151ktimers/007:40:520
1180429992619,4cyclictest1187982-21latency_hist07:35:022
1180429992619,4cyclictest1002-21mta-sts-daemon08:38:132
1180429992619,4cyclictest1002-21mta-sts-daemon08:38:132
1180429992618,2cyclictest171rcu_preempt08:45:412
118042999261,22cyclictest627-21lldpd07:45:482
1180428992620,3cyclictest0-21swapper/112:20:381
1180428992620,3cyclictest0-21swapper/112:05:531
1180428992618,5cyclictest1197553-21irqstats08:05:431
1180428992617,6cyclictest1213893-21sshd09:00:381
1180428992616,7cyclictest1339709-21rm12:30:491
118042899261,21cyclictest70-21kswapd011:20:521
1180427992618,6cyclictest265-21systemd-journal08:10:010
1180427992617,6cyclictest1204400-21cron08:30:010
1180429992520,2cyclictest0-21swapper/211:20:392
1180429992519,3cyclictest0-21swapper/208:00:382
1180429992516,7cyclictest1214246-21ntp_kernel_err09:00:452
1180428992519,3cyclictest0-21swapper/107:45:011
118042799252,5cyclictest171rcu_preempt07:15:490
1180429992417,3cyclictest0-21swapper/209:45:562
1180429992415,6cyclictest1186355-21unixbench_singl07:25:592
1180428992418,3cyclictest0-21swapper/108:45:551
1180428992416,5cyclictest1196922-21idleruntime-cro08:05:011
1180428992316,4cyclictest1203251-21aten2.4-expect08:25:361
1180427992316,5cyclictest0-21swapper/007:10:010
1180427992316,5cyclictest0-21swapper/007:10:010
1180427992310,9cyclictest265-21systemd-journal08:35:010
118042999222,15cyclictest1181399-21ntpq07:10:532
118042899225,11cyclictest1003-21mta-sts-daemon07:45:511
118043099214,9cyclictest0-21swapper/310:20:513
118043099212,8cyclictest0-21swapper/307:50:523
118043099212,12cyclictest0-21swapper/309:35:533
118043099204,8cyclictest0-21swapper/311:05:553
118043099202,11cyclictest0-21swapper/311:20:013
118043099201,10cyclictest0-21swapper/311:50:513
118043099201,10cyclictest0-21swapper/311:50:513
118043099201,10cyclictest0-21swapper/311:20:483
118043099193,9cyclictest0-21swapper/310:00:523
118043099193,7cyclictest0-21swapper/310:30:223
118043099193,7cyclictest0-21swapper/308:05:473
118043099192,9cyclictest0-21swapper/309:10:543
118043099192,11cyclictest0-21swapper/312:10:013
118043099192,11cyclictest0-21swapper/311:30:483
118043099192,11cyclictest0-21swapper/311:10:553
1180428991910,6cyclictest1180490-21latency_hist07:10:021
1180428991910,6cyclictest1180490-21latency_hist07:10:021
118043099183,8cyclictest0-21swapper/312:00:493
118043099183,8cyclictest0-21swapper/310:55:483
118043099183,8cyclictest0-21swapper/309:20:523
118043099182,9cyclictest0-21swapper/310:45:493
118043099182,9cyclictest0-21swapper/310:45:493
118043099182,8cyclictest0-21swapper/308:15:503
118043099182,11cyclictest0-21swapper/309:55:563
118043099182,10cyclictest0-21swapper/312:30:503
118043099182,10cyclictest0-21swapper/312:10:563
118043099182,10cyclictest0-21swapper/311:55:413
118043099182,10cyclictest0-21swapper/309:15:503
118043099181,11cyclictest0-21swapper/311:00:493
118043099181,11cyclictest0-21swapper/310:40:523
118043099181,11cyclictest0-21swapper/310:40:523
118043099180,10cyclictest0-21swapper/307:25:473
118043099174,9cyclictest0-21swapper/309:30:463
118043099174,9cyclictest0-21swapper/309:30:463
118043099173,8cyclictest0-21swapper/310:25:533
118043099173,8cyclictest0-21swapper/308:50:533
118043099172,9cyclictest0-21swapper/312:20:443
118043099172,9cyclictest0-21swapper/311:45:473
118043099172,9cyclictest0-21swapper/311:45:473
118043099172,9cyclictest0-21swapper/311:40:493
118043099172,9cyclictest0-21swapper/311:40:493
118043099172,9cyclictest0-21swapper/311:25:483
118043099172,9cyclictest0-21swapper/309:25:483
118043099172,9cyclictest0-21swapper/309:25:483
118043099172,10cyclictest0-21swapper/310:10:503
118043099171,9cyclictest0-21swapper/312:35:413
118043099171,9cyclictest0-21swapper/307:30:503
118043099171,7cyclictest0-21swapper/307:15:513
118043099171,10cyclictest0-21swapper/311:35:473
118043099171,10cyclictest0-21swapper/309:45:523
118042999172,11cyclictest1180382-21/usr/sbin/munin07:10:012
118042999172,11cyclictest1180382-21/usr/sbin/munin07:10:012
118043099163,8cyclictest0-21swapper/309:10:013
118043099163,6cyclictest0-21swapper/308:00:533
118043099162,8cyclictest0-21swapper/312:25:403
118043099162,8cyclictest0-21swapper/310:05:533
118043099162,7cyclictest0-21swapper/307:20:543
118043099161,9cyclictest0-21swapper/310:50:503
118043099161,9cyclictest0-21swapper/310:50:503
118043099153,7cyclictest0-21swapper/310:15:553
118043099152,7cyclictest0-21swapper/307:55:533
118043099152,6cyclictest0-21swapper/308:45:503
118043099151,8cyclictest0-21swapper/309:40:473
118043099151,7cyclictest0-21swapper/308:55:513
118043099151,7cyclictest0-21swapper/308:40:473
118043099151,7cyclictest0-21swapper/308:40:473
118043099151,10cyclictest0-21swapper/312:15:373
118043099151,10cyclictest0-21swapper/310:35:383
118043099146,6cyclictest0-21swapper/309:50:553
118043099143,6cyclictest0-21swapper/307:10:463
118043099142,7cyclictest0-21swapper/308:35:503
118043099142,7cyclictest0-21swapper/308:35:503
118043099141,8cyclictest0-21swapper/308:25:383
118043099141,7cyclictest0-21swapper/307:35:473
118043099141,6cyclictest0-21swapper/309:00:443
118043099141,6cyclictest0-21swapper/308:10:493
118043099132,7cyclictest0-21swapper/308:30:393
118043099132,7cyclictest0-21swapper/307:40:383
118043099131,7cyclictest0-21swapper/307:45:533
118043099112,6cyclictest0-21swapper/308:20:503
11804309982,5cyclictest0-21swapper/307:10:013
11804309982,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional