You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 12:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Apr 04, 2026 00:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1255589995748,6cyclictest1-21systemd21:40:012
125558799561,32cyclictest1292750-21ssh21:10:300
1255587995518,13cyclictest0-21swapper/023:00:370
1255587995518,13cyclictest0-21swapper/023:00:370
1255587995330,20cyclictest1315948-21sendmail21:40:010
1255587995228,16cyclictest151ktimers/023:05:350
1255587995224,4cyclictest1-21systemd21:20:010
1255589995143,5cyclictest1299771-21idleruntime-cro21:20:002
1255589995142,6cyclictest3496356-21cups-browsed21:20:312
125558799514,42cyclictest0-21swapper/019:25:300
1255587995138,4cyclictest1430191-21munin-plugin-st00:00:000
125558799501,25cyclictest1378407-21ssh22:55:370
1255589994739,5cyclictest1352703-21systemd-run22:25:012
1255588994739,5cyclictest1364914-21munin-run22:40:011
1255588994638,5cyclictest1398034-21aten_r2power_fr23:20:191
1255588994638,5cyclictest1328247-21ssh21:55:021
125558799464,23cyclictest1439579-21ssh00:10:340
1255587994624,3cyclictest0-21swapper/022:15:430
1255587994525,17cyclictest1349752-21ssh22:20:320
1255587994514,7cyclictest12350irq/32-eth021:40:360
1255589994436,5cyclictest1381906-21ssh23:00:242
1255589994436,5cyclictest1381906-21ssh23:00:242
1255587994433,7cyclictest151ktimers/019:15:280
1255587994433,7cyclictest151ktimers/019:15:280
1255587994421,7cyclictest1304903-21ssh21:25:300
1255589994335,5cyclictest1272144-21grep20:05:012
1255589994334,6cyclictest1402119-21aten_r2power_po23:25:202
1255588994336,3cyclictest0-21swapper/121:10:231
1255588994334,5cyclictest1291631-21diskmemload23:55:031
1255588994332,8cyclictest1430237-21sh00:00:011
1255588994332,7cyclictest421-21dbus-daemon23:18:101
1255587994331,9cyclictest1418391-21aten_r2power_cu23:45:200
1255589994234,5cyclictest421-21dbus-daemon23:50:022
1255589994233,6cyclictest1255586-21cyclictest23:38:592
1255588994237,3cyclictest0-21swapper/122:30:421
1255588994234,5cyclictest1285647-21systemd-run20:50:011
1255588994233,6cyclictest420-21avahi-daemon00:26:381
1255587994222,12cyclictest12350irq/32-eth000:00:330
1255589994133,5cyclictest1397519-21munin-plugin-st23:20:012
1255589994133,5cyclictest1266109-21systemd-run19:45:012
1255588994133,5cyclictest3496356-21cups-browsed23:37:591
1255588994133,5cyclictest1458692-21cron00:35:011
1255588994133,5cyclictest1446503-21munin-run00:20:011
1255588994133,5cyclictest1296452-21fschecks_time21:15:261
1255588994131,7cyclictest1291649-21ssh21:10:021
1255588994131,6cyclictest3496356-21cups-browsed21:59:031
1255588994131,6cyclictest3496356-21cups-browsed21:59:031
1255588994130,8cyclictest1258624-21sh19:20:011
1255588994130,8cyclictest1258624-21sh19:20:011
1255589994032,5cyclictest421-21dbus-daemon23:15:012
1255589994031,6cyclictest1291743-21latency_hist21:10:022
1255589994030,7cyclictest1354455-21unixbench_singl22:25:422
1255588994031,6cyclictest1353636-21missed_timers22:25:261
1255587994010,11cyclictest12350irq/32-eth022:30:290
1255589993934,3cyclictest0-21swapper/200:20:022
1255588993931,5cyclictest1460433-21unixbench_multi00:35:431
1255588993931,5cyclictest1313573-21unixbench-2d21:35:411
1255587993926,11cyclictest1-21systemd22:10:010
1255587993915,19cyclictest1451833-21ssh00:25:360
1255589993830,5cyclictest421-21dbus-daemon20:20:012
1255589993829,6cyclictest3496356-21cups-browsed23:51:012
1255589993829,5cyclictest1377130-21cron22:55:012
1255588993830,5cyclictest3496356-21cups-browsed22:46:041
1255588993823,11cyclictest421-21dbus-daemon21:40:311
125558799382,29cyclictest659-21snmpd19:55:290
1255589993732,2cyclictest0-21swapper/200:20:342
1255589993731,3cyclictest1055-21mta-sts-daemon21:27:302
1255589993731,3cyclictest0-21swapper/221:45:012
1255589993728,6cyclictest420-21avahi-daemon00:00:002
1255589993727,7cyclictest1341896-21mailstats22:10:402
1255588993729,5cyclictest1337091-21sshd22:05:231
1255588993729,5cyclictest1290209-21latency_hist21:05:021
1255588993729,5cyclictest1261522-21unixbench_singl19:25:401
1255588993728,6cyclictest439-21systemd-logind19:58:201
1255588993728,5cyclictest1-21systemd23:05:011
1255588993728,5cyclictest1-21systemd23:05:011
1255588993726,7cyclictest420-21avahi-daemon00:09:551
1255587993731,3cyclictest0-21swapper/019:40:010
1255587993728,6cyclictest420-21avahi-daemon00:06:140
1255587993728,6cyclictest1328208-21cron21:55:010
1255587993727,7cyclictest1410368-21cpu23:35:200
1255589993628,5cyclictest1434330-21ssh00:05:012
1255589993628,5cyclictest1332340-21sendmail22:00:012
1255589993628,5cyclictest1332340-21sendmail22:00:012
1255588993629,3cyclictest0-21swapper/120:40:291
1255588993628,5cyclictest420-21avahi-daemon19:50:271
1255588993627,6cyclictest421-21dbus-daemon21:30:011
1255588993627,6cyclictest1439660-21wc00:10:371
125558799364,25cyclictest659-21snmpd20:10:310
1255587993621,12cyclictest1450721-21latency_hist00:25:020
1255587993621,11cyclictest0-21swapper/022:00:330
125558799361,28cyclictest1402851-21ssh23:25:340
125558999354,28cyclictest1337685-21ssh22:05:372
1255589993527,5cyclictest3496356-21cups-browsed19:11:012
1255589993527,5cyclictest3496356-21cups-browsed19:11:012
1255589993527,5cyclictest1456268-21timerwakeupswit00:30:412
1255589993526,6cyclictest1-21systemd19:20:012
1255589993526,6cyclictest1-21systemd19:20:012
1255589993525,7cyclictest1055-21mta-sts-daemon22:42:222
1255589993524,8cyclictest1333936-21timerwakeupswit22:00:432
1255588993529,3cyclictest0-21swapper/121:30:331
1255588993527,5cyclictest1-21systemd19:10:011
1255588993527,4cyclictest1360925-21kworker/u8:0-nfsiod23:05:311
1255588993526,6cyclictest3496356-21cups-browsed23:28:161
1255588993525,7cyclictest1381202-21idleruntime-cro23:00:011
1255588993524,7cyclictest420-21avahi-daemon22:42:461
1255587993529,4cyclictest1017-21sendmail-mta23:15:330
1255587993528,4cyclictest1459744-21needreboot00:35:290
1255589993428,3cyclictest0-21swapper/220:00:012
1255589993426,5cyclictest1311886-21cron21:35:012
1255589993426,5cyclictest1264587-21sendmail19:40:012
1255589993425,6cyclictest421-21dbus-daemon00:40:012
1255589993425,6cyclictest420-21avahi-daemon00:14:522
1255589993425,6cyclictest3496356-21cups-browsed22:17:362
1255588993427,4cyclictest1276628-21sh20:20:001
1255588993426,5cyclictest1346317-21unixbench_singl22:15:431
125558899342,29cyclictest1390456-21ssh23:10:281
125558799342,29cyclictest1366014-21ssh22:40:300
1255589993326,5cyclictest1370487-21ssh22:45:422
1255589993326,3cyclictest0-21swapper/223:40:332
1255589993326,3cyclictest0-21swapper/221:50:012
1255589993325,5cyclictest1273579-21grep20:05:432
1255589993325,4cyclictest1404318-21kworker/u8:4-rpciod23:31:122
1255589993323,8cyclictest1451077-21aten_r2power_po00:25:212
1255588993328,2cyclictest0-21swapper/122:25:011
125558799334,21cyclictest1321370-21grep21:45:370
1255589993228,2cyclictest0-21swapper/221:50:322
1255589993224,5cyclictest3496356-21cups-browsed19:25:192
1255589993224,5cyclictest1295851-21latency_hist21:15:032
1255589993223,7cyclictest1282623-21sendmail20:40:012
1255589993222,7cyclictest1362482-21timerwakeupswit22:35:422
125558999322,24cyclictest1386608-21ssh23:05:382
1255588993227,2cyclictest0-21swapper/120:30:011
1255588993226,3cyclictest0-21swapper/121:00:011
1255588993226,3cyclictest0-21swapper/120:05:301
1255588993224,5cyclictest420-21avahi-daemon00:01:291
1255588993224,5cyclictest1272140-21munin-plugin-st20:05:011
1255588993223,6cyclictest420-21avahi-daemon21:20:201
125558799323,23cyclictest753-21ntpd20:40:320
1255587993225,5cyclictest1285643-21kworker/u8:3+xprtiod21:10:010
1255587993223,6cyclictest1282832-21latency_hist20:40:030
1255587993218,11cyclictest12350irq/32-eth022:25:300
1255589993120,7cyclictest420-21avahi-daemon22:30:052
125558999311,26cyclictest659-21snmpd20:25:342
125558999311,21cyclictest1378330-21ssh22:55:342
125558899316,19cyclictest1321147-21ssh21:45:291
125558899314,22cyclictest1333449-21ssh22:00:311
125558899313,23cyclictest1419185-21ssh23:45:331
1255588993125,3cyclictest0-21swapper/123:30:421
125558899312,23cyclictest1374198-21ssh22:50:311
125558799311,24cyclictest1443679-21ssh00:15:360
1255589993023,4cyclictest1269069-21cron19:55:012
1255589993023,4cyclictest1266906-21needreboot19:45:272
125558999302,22cyclictest1398677-21sh23:20:302
1255588993022,5cyclictest1343499-21ssh22:13:011
1255588993022,5cyclictest1280166-21fschecks_time20:30:231
125558799306,14cyclictest12350irq/32-eth021:20:330
1255587993023,5cyclictest421-21dbus-daemon22:40:010
1255587993023,5cyclictest1430195-21cups-browsed00:33:250
125558799302,23cyclictest1415117-21ssh23:40:340
1255587993021,7cyclictest1332312-21munin-run22:00:010
1255587993021,7cyclictest1332312-21munin-run22:00:010
125558999293,21cyclictest659-21snmpd20:20:332
125558999293,21cyclictest659-21snmpd20:20:332
1255589992924,2cyclictest0-21swapper/220:35:012
1255589992922,4cyclictest1288541-21unixbench_singl20:55:412
1255589992921,5cyclictest1-21systemd19:35:022
125558899293,6cyclictest171rcu_preempt19:10:331
125558899293,6cyclictest171rcu_preempt19:10:331
1255588992923,3cyclictest0-21swapper/120:55:011
1255588992923,3cyclictest0-21swapper/119:40:011
1255588992923,3cyclictest0-21swapper/100:20:421
1255588992921,5cyclictest1414788-21latency23:40:251
1255587992923,4cyclictest1289165-21fschecks_time21:00:230
1255587992923,3cyclictest0-21swapper/019:45:300
1255587992922,5cyclictest1315620-21kworker/u8:7+xprtiod22:15:010
125558799292,19cyclictest659-21snmpd20:55:280
1255587992921,5cyclictest1269094-21systemd-run19:55:010
1255587992919,7cyclictest1400030-21sshd23:22:000
1255587992916,9cyclictest12350irq/32-eth021:30:360
1255587992915,10cyclictest1370218-21ssh22:45:340
1255588992821,5cyclictest0-21swapper/119:25:001
1255588992820,5cyclictest1266625-21fschecks_time19:45:231
1255587992822,3cyclictest1423209-21ssh23:50:320
125558799281,23cyclictest3496356-21cups-browsed19:30:270
125558799281,17cyclictest1406823-21ssh23:30:310
125558799280,23cyclictest1390611-21ssh23:10:340
125558999273,15cyclictest1056-21mta-sts-daemon20:40:332
1255589992720,4cyclictest1275024-21unixbench_multi20:10:402
1255589992718,6cyclictest1430195-21cups-browsed00:07:542
125558999271,6cyclictest171rcu_preempt21:00:342
1255588992719,5cyclictest1262108-21fschecks_count19:30:221
1255587992719,4cyclictest151ktimers/019:10:330
1255587992719,4cyclictest151ktimers/019:10:330
125558799271,20cyclictest1374189-21sh22:50:310
1255589992620,3cyclictest0-21swapper/219:25:012
1255587992621,2cyclictest0-21swapper/020:20:010
125558899255,11cyclictest3608118-21munin-node19:40:351
125558799254,15cyclictest659-21snmpd20:00:300
1255587992520,3cyclictest1055-21mta-sts-daemon19:25:020
1255587992517,6cyclictest1265364-21missed_timers19:40:260
125558999241,17cyclictest3496353-21cupsd20:45:282
1255588992415,6cyclictest1277958-21ls20:20:411
1255588992415,6cyclictest1277958-21ls20:20:411
125558999236,12cyclictest428-21mta-sts-daemon20:50:302
1255589992315,5cyclictest1255592-21systemd-run19:10:012
1255588992316,4cyclictest1282552-21unixbench_singl20:35:421
1255588992315,5cyclictest1275319-21latency_hist20:15:021
1255587992316,4cyclictest1051-21mta-sts-daemon20:20:210
1255587992316,4cyclictest1051-21mta-sts-daemon20:20:210
1255587992312,9cyclictest1273125-21ntpq20:05:370
1255587992311,10cyclictest0-21swapper/020:45:390
125558799222,17cyclictest1279519-21tail20:25:410
1255587992214,5cyclictest151ktimers/020:30:310
125558799212,7cyclictest421-21dbus-daemon20:55:010
125559099201,10cyclictest0-21swapper/320:20:313
125559099201,10cyclictest0-21swapper/320:20:313
125559099182,9cyclictest0-21swapper/323:20:333
125559099173,9cyclictest0-21swapper/300:20:343
125559099172,7cyclictest0-21swapper/323:00:383
125559099172,7cyclictest0-21swapper/323:00:383
125559099171,8cyclictest0-21swapper/323:05:323
125559099171,7cyclictest0-21swapper/323:15:273
125559099171,7cyclictest0-21swapper/321:00:293
125559099171,10cyclictest0-21swapper/322:30:363
125559099162,8cyclictest0-21swapper/323:50:333
125559099161,8cyclictest0-21swapper/323:55:293
125559099161,8cyclictest0-21swapper/323:40:293
125559099161,3cyclictest0-21swapper/319:45:283
125559099151,8cyclictest0-21swapper/321:15:363
125559099151,7cyclictest0-21swapper/322:05:313
125559099151,7cyclictest0-21swapper/321:10:233
125559099151,6cyclictest0-21swapper/320:25:323
125559099151,6cyclictest0-21swapper/320:10:343
125559099151,5cyclictest0-21swapper/322:15:283
125559099151,10cyclictest0-21swapper/323:25:303
125558799151,11cyclictest1255544-21/usr/sbin/munin19:10:010
125559099142,7cyclictest0-21swapper/319:40:363
125559099142,6cyclictest0-21swapper/320:30:313
125559099141,8cyclictest0-21swapper/322:25:303
125559099141,8cyclictest0-21swapper/322:20:383
125559099141,8cyclictest0-21swapper/322:10:373
125559099141,8cyclictest0-21swapper/321:50:293
125559099141,8cyclictest0-21swapper/319:15:323
125559099141,8cyclictest0-21swapper/319:15:323
125559099141,8cyclictest0-21swapper/300:35:353
125559099141,7cyclictest0-21swapper/323:35:293
125559099141,7cyclictest0-21swapper/322:50:323
125559099141,7cyclictest0-21swapper/321:45:333
125559099141,7cyclictest0-21swapper/321:30:313
125559099141,7cyclictest0-21swapper/300:25:303
125559099141,6cyclictest0-21swapper/319:55:283
125559099141,6cyclictest0-21swapper/319:25:273
125559099141,6cyclictest0-21swapper/319:20:333
125559099141,6cyclictest0-21swapper/300:15:293
125559099141,5cyclictest0-21swapper/320:55:283
125559099132,6cyclictest0-21swapper/323:00:023
125559099132,6cyclictest0-21swapper/320:55:013
125559099131,8cyclictest0-21swapper/321:40:293
125559099131,7cyclictest0-21swapper/323:10:323
125559099131,7cyclictest0-21swapper/322:40:343
125559099131,7cyclictest0-21swapper/320:05:023
125559099131,6cyclictest0-21swapper/319:30:303
125559099131,6cyclictest0-21swapper/319:10:373
125559099131,6cyclictest0-21swapper/319:10:373
125559099131,5cyclictest0-21swapper/320:15:313
125559099131,5cyclictest0-21swapper/319:35:293
125559099122,6cyclictest0-21swapper/300:15:003
125559099121,6cyclictest0-21swapper/323:30:323
125559099121,6cyclictest0-21swapper/322:45:263
125559099121,6cyclictest0-21swapper/322:35:323
125559099121,6cyclictest0-21swapper/322:00:203
125559099121,6cyclictest0-21swapper/321:25:383
125559099121,6cyclictest0-21swapper/319:50:333
125559099121,5cyclictest0-21swapper/320:40:283
125559099112,6cyclictest0-21swapper/321:10:013
125559099112,6cyclictest0-21swapper/320:05:333
125559099112,6cyclictest0-21swapper/300:00:363
125559099111,7cyclictest0-21swapper/323:45:243
125559099111,7cyclictest0-21swapper/300:05:353
125559099111,6cyclictest0-21swapper/321:55:323
125559099111,6cyclictest0-21swapper/321:55:323
125559099111,6cyclictest0-21swapper/321:35:233
125559099111,6cyclictest0-21swapper/321:20:203
125559099111,6cyclictest0-21swapper/320:35:303
125559099111,5cyclictest0-21swapper/320:45:283
125559099101,6cyclictest0-21swapper/300:30:293
12555909991,4cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional