You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 13:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Fri Apr 24, 2026 12:43:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1221971996153,5cyclictest1306683-21systemd-run10:10:012
122197199583,31cyclictest1328344-21ssh10:35:502
122197199583,31cyclictest1328344-21ssh10:35:502
1221971995828,6cyclictest411063-21cups-browsed09:55:572
122197199565,30cyclictest1324179-21ssh10:30:442
1221970995445,4cyclictest0-21swapper/110:50:411
1221970995344,5cyclictest1247474-21systemd-run08:35:011
1221970995344,5cyclictest1247474-21systemd-run08:35:011
1221969995325,19cyclictest151ktimers/007:25:400
1221969995325,19cyclictest151ktimers/007:25:400
1221970995242,7cyclictest411063-21cups-browsed08:25:011
1221970995242,7cyclictest411063-21cups-browsed08:25:011
122196999523,42cyclictest1263404-21ssh09:15:470
1221969995031,17cyclictest1223469-21kworker/u8:7-events_unbound08:55:400
1221970994840,5cyclictest1391500-21latency_hist11:55:021
1221969994834,9cyclictest0-21swapper/010:55:590
1221970994738,5cyclictest0-21swapper/110:25:411
1221969994737,6cyclictest1270235-21latency_hist09:25:020
1221969994737,6cyclictest1270235-21latency_hist09:25:020
1221971994531,11cyclictest1348621-21switchtime11:00:532
1221971994519,20cyclictest1404880-21sh12:10:462
122197099453,30cyclictest1421024-21ssh12:30:481
1221969994536,6cyclictest1335734-21fschecks_time10:45:350
1221969994535,4cyclictest0-21swapper/009:10:420
1221969994523,15cyclictest151ktimers/007:20:420
1221971994437,4cyclictest0-21swapper/210:40:442
1221971994437,4cyclictest0-21swapper/210:40:442
1221971994425,15cyclictest1257946-21diskmemload10:20:392
1221969994433,7cyclictest411063-21cups-browsed10:14:410
1221971994336,4cyclictest1056-21mta-sts-daemon09:12:432
1221970994337,3cyclictest0-21swapper/110:45:001
1221970994337,3cyclictest0-21swapper/110:45:001
1221970994335,5cyclictest1379260-21basename11:40:011
1221970994316,19cyclictest0-21swapper/109:25:451
1221969994334,6cyclictest411063-21cups-browsed12:00:010
1221969994333,6cyclictest1291666-21ntp_kernel_err09:50:390
122196999431,36cyclictest1279700-21ssh09:35:480
1221971994234,5cyclictest1257946-21diskmemload11:25:372
1221971994233,6cyclictest1388006-21aten_r2power_vo11:50:322
1221971994231,7cyclictest1230972-21sendmail07:40:012
1221970994237,2cyclictest0-21swapper/112:20:011
1221970994233,6cyclictest1304452-21tail10:05:531
1221970994232,6cyclictest0-21swapper/110:15:001
1221970994231,7cyclictest411063-21cups-browsed11:16:111
1221970994230,8cyclictest1278317-21sh09:35:011
1221969994233,6cyclictest420-21avahi-daemon12:29:470
1221971994134,3cyclictest0-21swapper/212:25:432
1221970994134,4cyclictest0-21swapper/111:25:011
1221970994132,6cyclictest421-21dbus-daemon10:35:371
1221970994132,6cyclictest421-21dbus-daemon10:35:371
1221970994132,6cyclictest1416686-21needreboot12:25:381
1221970994132,5cyclictest1286449-21munin-plugin-st09:45:011
1221970994116,8cyclictest1384122-21ssh11:45:361
1221970994116,7cyclictest1281895-21sh09:39:041
1221969994134,4cyclictest1367280-21latency_hist11:25:020
1221969994133,6cyclictest1054-21mta-sts-daemon12:34:320
1221969994133,5cyclictest1413392-21unixbench-2d12:21:000
1221969994131,5cyclictest1286469-21kworker/u8:12-rpciod09:55:430
1221969994131,5cyclictest0-21swapper/012:15:520
1221969994128,9cyclictest1401146-21threads12:05:520
1221969994128,7cyclictest0-21swapper/011:05:400
122196999412,31cyclictest1348322-21ssh11:00:450
1221971994032,5cyclictest1-21systemd12:25:012
1221971994029,7cyclictest420-21avahi-daemon10:49:242
1221970994032,5cyclictest1399510-21munin-plugin-st12:05:011
1221970994032,5cyclictest1399510-21munin-plugin-st12:05:011
1221970994031,6cyclictest1-21systemd11:15:021
1221970994031,6cyclictest1-21systemd11:15:021
1221970994029,8cyclictest695-21cron09:05:011
1221970994022,13cyclictest1368577-21ps11:25:481
1221970994016,7cyclictest1236327-21ntpq07:55:411
1221969994034,4cyclictest12350irq/32-eth008:01:040
1221969994031,5cyclictest0-21swapper/009:35:010
1221969994010,22cyclictest1384541-21ssh11:45:430
122197199393,32cyclictest1283598-21ssh09:40:432
1221971993932,4cyclictest1352876-21unixbench_multi11:05:542
1221971993931,4cyclictest1237030-21latency_hist08:00:022
1221970993931,5cyclictest1257946-21diskmemload10:45:361
1221970993931,5cyclictest1254971-21systemd-run09:00:011
1221970993928,8cyclictest1352393-21ntpq11:05:461
1221970993926,8cyclictest1245344-21ntpq08:25:411
1221970993926,8cyclictest1239310-21ntpq08:05:401
1221970993924,9cyclictest1248363-21ntpq08:35:421
1221970993924,9cyclictest1248363-21ntpq08:35:421
1221970993915,8cyclictest1288815-21sh09:46:471
1221970993911,8cyclictest1382587-21ssh11:43:401
1221969993930,5cyclictest1284718-21sshd09:41:430
1221969993926,9cyclictest1-21systemd10:50:300
1221969993925,9cyclictest0-21swapper/007:30:420
1221971993832,3cyclictest0-21swapper/212:40:012
1221971993830,5cyclictest411063-21cups-browsed09:35:012
1221971993830,3cyclictest0-21swapper/211:40:462
1221971993825,8cyclictest0-21swapper/209:45:442
122197199382,31cyclictest1376373-21ssh11:35:402
122197099383,26cyclictest1312037-21ssh10:15:441
1221970993832,3cyclictest0-21swapper/111:00:011
1221970993831,3cyclictest0-21swapper/110:00:011
1221970993825,8cyclictest1227421-21ntpq07:25:441
1221970993825,8cyclictest1227421-21ntpq07:25:441
1221970993824,8cyclictest1242403-21ntpq08:15:441
1221970993824,8cyclictest1231873-21ntpq07:40:441
1221970993816,7cyclictest1394841-21sh11:58:551
1221970993815,8cyclictest1413142-21sh12:20:571
1221971993728,6cyclictest0-21swapper/210:55:002
1221970993729,5cyclictest1257941-21cron09:10:011
1221970993714,7cyclictest1301634-21sh10:03:081
1221970993714,7cyclictest1301634-21sh10:03:081
1221969993726,6cyclictest0-21swapper/007:40:430
1221971993628,5cyclictest411063-21cups-browsed12:05:332
1221971993627,6cyclictest1250473-21systemd-run08:45:012
1221971993627,5cyclictest411063-21cups-browsed09:54:182
122197199362,28cyclictest659-21snmpd12:30:392
122197199362,26cyclictest1307828-21ssh10:10:392
122197199361,28cyclictest1360523-21grep11:15:502
1221970993627,3cyclictest1348371-21ntpq11:00:471
1221970993625,7cyclictest428-21mta-sts-daemon09:53:371
1221970993625,7cyclictest1222816-21ntpq07:10:431
1221970993622,10cyclictest1224328-21ntpq07:15:421
1221970993614,7cyclictest1374388-21diskmemload11:33:291
1221970993610,8cyclictest1407603-21sh12:15:011
122196999369,24cyclictest753-21ntpd09:25:430
122196999364,25cyclictest1287783-21ssh09:45:460
1221969993630,4cyclictest1315181-21kworker/u8:8-xprtiod10:30:420
1221969993628,6cyclictest1-21systemd11:20:010
1221969993628,5cyclictest1242994-21systemd-run08:20:010
1221969993627,6cyclictest1332041-21cut10:40:430
1221969993627,6cyclictest1332041-21cut10:40:430
1221969993620,12cyclictest12350irq/32-eth010:25:430
122197199353,22cyclictest1356456-21ssh11:10:482
122197199353,22cyclictest1356456-21ssh11:10:482
1221970993527,6cyclictest1270038-21ssh09:24:451
1221970993527,6cyclictest1270038-21ssh09:24:451
1221970993527,5cyclictest1250407-21unixbench_singl08:40:531
1221970993524,7cyclictest1225925-21ntpq07:20:451
1221970993522,9cyclictest1230340-21ntpq07:35:441
1221970993518,12cyclictest1237805-21ntpq08:00:401
1221970993515,4cyclictest1325563-21sh10:32:281
1221970993512,9cyclictest1260739-21sh09:12:341
1221969993525,7cyclictest1318827-21ssh10:25:010
1221969993523,9cyclictest0-21swapper/012:00:520
1221969993523,9cyclictest0-21swapper/012:00:520
1221969993522,6cyclictest12350irq/32-eth010:15:440
122196999352,23cyclictest659-21snmpd11:50:560
1221969993514,14cyclictest1356338-21sshd11:10:430
1221969993514,14cyclictest1356338-21sshd11:10:430
1221971993427,4cyclictest0-21swapper/208:40:002
1221971993427,4cyclictest0-21swapper/208:40:002
1221971993427,3cyclictest0-21swapper/211:30:372
1221971993426,5cyclictest421-21dbus-daemon09:30:012
1221971993426,5cyclictest1266104-21run-parts09:20:012
1221971993426,5cyclictest1051-21mta-sts-daemon08:05:452
1221971993426,5cyclictest0-21swapper/210:25:312
1221971993423,8cyclictest0-21swapper/211:45:432
1221970993427,3cyclictest0-21swapper/112:35:441
1221970993425,6cyclictest1228480-21grep07:30:341
1221970993423,8cyclictest1234836-21ntpq07:50:401
1221970993423,7cyclictest1240861-21ntpq08:10:421
1221970993422,9cyclictest1264607-21ssh09:17:101
1221970993420,9cyclictest1252809-21ntpq08:50:371
1221969993427,3cyclictest0-21swapper/007:50:420
1221969993424,5cyclictest0-21swapper/012:35:450
122196999342,25cyclictest1299972-21ssh10:00:510
122196999342,25cyclictest1299972-21ssh10:00:510
1221969993421,9cyclictest0-21swapper/011:35:430
1221969993420,11cyclictest1380881-21switchtime11:40:520
1221969993412,13cyclictest1056-21mta-sts-daemon11:25:460
122197199334,24cyclictest1408962-21ssh12:15:522
1221971993323,8cyclictest1246797-21ntp_kernel_err08:30:382
1221971993323,8cyclictest1246797-21ntp_kernel_err08:30:382
1221970993324,4cyclictest1400719-21ntpq12:05:421
1221970993323,7cyclictest1316059-21ntpq10:20:441
1221970993322,7cyclictest1251317-21ntpq08:45:411
1221970993318,9cyclictest1233401-21ntpq07:45:431
1221969993325,5cyclictest1372999-21unixbench_multi11:30:540
122197199325,20cyclictest1392609-21ssh11:55:422
1221971993225,3cyclictest0-21swapper/211:20:402
1221971993225,3cyclictest0-21swapper/210:15:482
1221971993224,5cyclictest1280192-21head09:35:542
1221969993224,6cyclictest1051-21mta-sts-daemon08:40:430
1221969993224,5cyclictest1404631-21missed_timers12:10:380
1221969993224,5cyclictest0-21swapper/008:05:400
1221969993223,7cyclictest1252790-21ntp_kernel_err08:50:370
1221969993223,6cyclictest420-21avahi-daemon10:38:590
1221969993223,6cyclictest420-21avahi-daemon10:38:590
1221971993123,5cyclictest1251858-21tail08:45:532
1221971993123,5cyclictest1243494-21fschecks_time08:20:332
1221971993123,5cyclictest1243494-21fschecks_time08:20:332
1221971993123,5cyclictest1237468-21fschecks_count08:00:342
1221971993122,6cyclictest411063-21cups-browsed10:04:572
1221971993122,6cyclictest411063-21cups-browsed10:04:572
1221969993126,3cyclictest1056-21mta-sts-daemon08:20:410
1221969993126,3cyclictest1056-21mta-sts-daemon08:20:410
1221969993122,7cyclictest1255824-21ntp_kernel_pll_09:00:410
1221971993023,4cyclictest1229445-21cron07:35:002
1221971993022,5cyclictest1254826-21unixbench_multi08:55:512
1221969993019,8cyclictest12350irq/32-eth010:05:410
1221971992921,5cyclictest1225495-21fschecks_time07:20:352
1221971992919,7cyclictest411063-21cups-browsed12:04:152
1221971992919,7cyclictest411063-21cups-browsed12:04:152
1221971992912,10cyclictest1344260-21/usr/sbin/munin10:55:472
1221970992920,6cyclictest1221981-21munin-run07:10:011
1221971992823,2cyclictest0-21swapper/207:50:462
122197199282,19cyclictest659-21snmpd09:00:412
1221971992821,4cyclictest0-21swapper/209:20:302
1221971992821,4cyclictest0-21swapper/209:20:302
1221971992819,5cyclictest420-21avahi-daemon07:10:382
1221969992821,4cyclictest1241026-21systemctl08:10:470
1221969992820,5cyclictest411063-21cups-browsed07:15:570
1221969992819,6cyclictest0-21swapper/007:35:430
1221969992813,3cyclictest0-21swapper/007:10:570
1221971992719,5cyclictest411063-21cups-browsed07:44:232
1221971992714,10cyclictest351ktimers/208:25:442
1221969992721,3cyclictest1056-21mta-sts-daemon09:05:070
1221969992718,4cyclictest1233942-21cron07:50:010
1221971992621,2cyclictest0-21swapper/207:45:522
1221971992618,4cyclictest0-21swapper/209:10:012
1221971992520,2cyclictest0-21swapper/207:20:012
1221971992516,5cyclictest1242967-21awk08:20:002
122196999252,18cyclictest1239939-21kworker/0:0+events08:35:390
122196999252,18cyclictest1239939-21kworker/0:0+events08:35:390
1221969992517,6cyclictest1246488-21fschecks_time08:30:330
1221969992517,6cyclictest1246488-21fschecks_time08:30:330
1221971992415,6cyclictest1226731-21tail07:25:302
1221971992415,6cyclictest1226731-21tail07:25:302
1221971992316,3cyclictest0-21swapper/208:50:432
1221971992314,6cyclictest420-21avahi-daemon08:12:392
122196999231,17cyclictest411063-21cups-browsed07:55:470
1221969992216,4cyclictest411063-21cups-browsed08:28:580
122196999211,16cyclictest1251317-21ntpq08:45:410
1221971992014,2cyclictest0-21swapper/207:10:022
122197299192,10cyclictest0-21swapper/311:00:463
122197299183,6cyclictest0-21swapper/310:45:423
122196999182,5cyclictest1055-21mta-sts-daemon07:10:010
122197299172,9cyclictest0-21swapper/310:05:423
122197299172,8cyclictest0-21swapper/310:20:463
122197299163,7cyclictest0-21swapper/312:00:423
122197299163,7cyclictest0-21swapper/312:00:423
122197299162,8cyclictest0-21swapper/311:10:403
122197299162,8cyclictest0-21swapper/311:10:403
122197299161,7cyclictest0-21swapper/309:10:403
122197299153,8cyclictest0-21swapper/312:15:433
122197299153,6cyclictest0-21swapper/310:30:403
122197299151,8cyclictest0-21swapper/312:25:413
122197299151,8cyclictest0-21swapper/312:20:493
122197299151,8cyclictest0-21swapper/311:20:433
122197299151,8cyclictest0-21swapper/310:55:453
122197299151,8cyclictest0-21swapper/309:45:463
122197299151,7cyclictest0-21swapper/312:30:463
122197299151,7cyclictest0-21swapper/312:10:413
122197299151,7cyclictest0-21swapper/308:55:383
122197299151,7cyclictest0-21swapper/308:50:373
122197299151,6cyclictest0-21swapper/307:55:423
122197299151,3cyclictest0-21swapper/307:50:413
122197299144,7cyclictest0-21swapper/311:50:403
122197299143,7cyclictest0-21swapper/310:50:413
122197299142,8cyclictest0-21swapper/309:34:433
122197299142,7cyclictest0-21swapper/310:35:483
122197299142,7cyclictest0-21swapper/310:35:483
122197299142,7cyclictest0-21swapper/310:00:453
122197299142,7cyclictest0-21swapper/310:00:453
122197299142,7cyclictest0-21swapper/309:35:433
122197299142,7cyclictest0-21swapper/309:20:403
122197299142,7cyclictest0-21swapper/309:20:403
122197299142,6cyclictest0-21swapper/309:40:373
122197299142,6cyclictest0-21swapper/308:45:403
122197299142,6cyclictest0-21swapper/308:35:423
122197299142,6cyclictest0-21swapper/308:35:423
122197299142,5cyclictest0-21swapper/308:30:383
122197299142,5cyclictest0-21swapper/308:30:383
122197299141,7cyclictest0-21swapper/311:55:443
122197299141,7cyclictest0-21swapper/311:30:483
122197299141,7cyclictest0-21swapper/310:25:383
122197299141,7cyclictest0-21swapper/310:10:393
122197299141,7cyclictest0-21swapper/309:50:423
122197299141,7cyclictest0-21swapper/309:15:393
122197299141,7cyclictest0-21swapper/308:00:413
122197299141,6cyclictest0-21swapper/309:05:423
122197299141,6cyclictest0-21swapper/308:20:403
122197299141,6cyclictest0-21swapper/308:20:403
122197299141,6cyclictest0-21swapper/308:10:393
122197299141,6cyclictest0-21swapper/307:40:483
122197299141,6cyclictest0-21swapper/307:10:443
122197299133,6cyclictest0-21swapper/311:05:423
122197299133,6cyclictest0-21swapper/309:55:463
122197299131,7cyclictest0-21swapper/311:35:393
122197299131,7cyclictest0-21swapper/310:15:413
122197299131,7cyclictest0-21swapper/309:25:403
122197299131,6cyclictest0-21swapper/312:35:313
122197299131,6cyclictest0-21swapper/308:40:463
122197299131,6cyclictest0-21swapper/308:25:443
122197299131,6cyclictest0-21swapper/307:35:483
122197299124,5cyclictest0-21swapper/310:40:363
122197299124,5cyclictest0-21swapper/310:40:363
122197299122,6cyclictest0-21swapper/311:45:403
122197299122,6cyclictest0-21swapper/311:40:363
122197299121,7cyclictest0-21swapper/312:05:443
122197299121,7cyclictest0-21swapper/311:25:423
122197299121,6cyclictest0-21swapper/307:20:493
122197299112,6cyclictest0-21swapper/307:50:013
122197299112,4cyclictest0-21swapper/307:30:403
122197299111,7cyclictest0-21swapper/311:15:383
122197299111,6cyclictest0-21swapper/308:05:483
122197299111,6cyclictest0-21swapper/307:15:423
122197299101,5cyclictest0-21swapper/309:00:413
122197299101,5cyclictest0-21swapper/307:25:393
122197299101,5cyclictest0-21swapper/307:25:393
12219729991,5cyclictest0-21swapper/308:15:343
12219729991,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional