You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-16 - 18:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Tue Sep 16, 2025 12:40:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1249086998768,15cyclictest0-21swapper/108:05:011
1249085995330,19cyclictest1296611-21ssh09:28:330
1249087995040,7cyclictest422-21dbus-daemon08:03:352
1249086995029,17cyclictest0-21swapper/109:55:011
124908699502,27cyclictest1290405-21ssh09:18:341
1249087994840,5cyclictest1409684-21cron12:25:012
124908699483,22cyclictest1415297-21sshd12:33:361
1249086994827,16cyclictest1280039-21sshd08:53:371
1249086994825,20cyclictest0-21swapper/111:45:011
1249086994823,22cyclictest1419465-21unixbench_multi12:38:471
1249086994823,22cyclictest1419465-21unixbench_multi12:38:471
1249087994735,4cyclictest0-21swapper/209:23:382
1249086994737,6cyclictest1344712-21sed10:43:501
1249086994638,5cyclictest1268606-21cron08:15:011
1249086994627,15cyclictest0-21swapper/111:23:471
1249086994621,3cyclictest0-21swapper/112:13:261
1249085994631,11cyclictest12450irq/32-eth011:33:410
1249086994436,5cyclictest1393295-21sendmail12:00:021
1249086994436,5cyclictest1393295-21sendmail12:00:021
1249086994436,5cyclictest1367136-21systemd-run11:20:011
1249086994422,19cyclictest0-21swapper/108:00:011
1249086994422,19cyclictest0-21swapper/108:00:011
1249086994418,3cyclictest959-21mta-sts-daemon11:52:061
1249085994421,16cyclictest151ktimers/009:33:470
1249087994333,6cyclictest1300110-21unixbench-2d09:33:522
1249086994334,5cyclictest1286903-21aten_r2power_po09:13:251
1249086994333,6cyclictest1267145-21systemd-run08:10:011
1249086994333,6cyclictest1267145-21systemd-run08:10:011
1249086994216,6cyclictest1326773-21aten_r2power_en10:18:241
1249086994216,3cyclictest0-21swapper/108:58:381
1249085994234,5cyclictest422-21dbus-daemon10:50:020
124908699416,17cyclictest1324093-21ssh10:13:351
124908699416,17cyclictest1324093-21ssh10:13:351
1249086994134,4cyclictest1302922-21wc09:38:451
1249086994131,7cyclictest1-21systemd11:40:021
1249086994117,6cyclictest421-21avahi-daemon08:37:031
1249086994111,10cyclictest1296873-21/usr/sbin/munin09:28:421
1249085994135,4cyclictest1-21systemd07:40:020
1249085994132,5cyclictest0-21swapper/011:58:360
1249085994132,5cyclictest0-21swapper/011:58:360
1249087994032,5cyclictest1306574-21cron09:45:012
124908699409,12cyclictest1994697-21sendmail-mta11:48:341
124908699403,30cyclictest1337454-21ssh10:33:331
1249086994031,5cyclictest1373153-21unixbench-2d11:28:461
1249086994029,6cyclictest0-21swapper/109:58:371
1249086994017,5cyclictest1261305-21/usr/sbin/munin07:53:411
1249086994017,5cyclictest1261305-21/usr/sbin/munin07:53:411
1249085994028,9cyclictest1406276-21ssh12:20:010
1249085994026,11cyclictest1268638-21systemd-run08:15:010
1249087993933,3cyclictest0-21swapper/211:00:012
1249087993933,3cyclictest0-21swapper/211:00:012
1249087993931,5cyclictest1302969-21systemctl09:38:472
1249087993929,5cyclictest0-21swapper/211:43:392
1249087993926,10cyclictest1369721-21timerandwakeup11:23:472
124908699393,27cyclictest1411930-21ssh12:28:371
1249086993930,5cyclictest1-21systemd10:10:011
1249086993930,5cyclictest1-21systemd10:10:011
1249086993918,4cyclictest956-21mta-sts-daemon10:04:011
1249086993918,4cyclictest956-21mta-sts-daemon10:04:011
1249085993933,4cyclictest422-21dbus-daemon10:55:020
1249085993930,6cyclictest487177-21cups-browsed12:00:330
1249085993929,7cyclictest1274617-21munin-run08:35:010
124908799383,28cyclictest1398583-21ssh12:08:362
1249087993832,3cyclictest0-21swapper/210:35:012
1249087993830,5cyclictest1267145-21systemd-run08:10:012
1249087993830,5cyclictest1267145-21systemd-run08:10:012
1249087993829,6cyclictest487177-21cups-browsed12:18:562
1249085993829,4cyclictest0-21swapper/009:38:370
1249085993828,7cyclictest1317280-21fschecks_time10:03:290
1249085993828,7cyclictest1317280-21fschecks_time10:03:290
1249085993827,8cyclictest1287397-21needreboot09:13:330
1249085993814,5cyclictest12550irq/33-eth012:08:440
124908599381,27cyclictest1415355-21ssh12:33:380
1249087993729,5cyclictest422-21dbus-daemon11:04:002
1249087993729,5cyclictest1285117-21diskmemload11:53:332
1249086993727,6cyclictest487174-21cupsd11:06:071
124908699371,17cyclictest1405063-21ssh12:18:341
1249085993731,3cyclictest0-21swapper/009:10:010
1249085993730,5cyclictest422-21dbus-daemon08:45:010
1249085993730,4cyclictest1373175-21unixbench_multi11:28:460
1249085993729,5cyclictest487177-21cups-browsed08:38:310
1249085993728,6cyclictest487177-21cups-browsed11:21:560
124908799362,28cyclictest1415403-21ssh12:33:412
124908799361,25cyclictest1379201-21ssh11:38:462
124908699363,28cyclictest1305837-21ssh09:43:451
1249086993625,8cyclictest1330369-21kernelversion10:23:311
124908699361,15cyclictest1375874-21ssh11:33:451
1249085993629,4cyclictest1291519-21munin-run09:20:010
1249085993625,4cyclictest1357485-21latency_hist11:05:020
1249085993617,12cyclictest151ktimers/007:53:420
1249085993617,12cyclictest151ktimers/007:53:420
1249087993527,5cyclictest1392300-21ntp_offset11:58:382
1249087993527,5cyclictest1392300-21ntp_offset11:58:382
1249087993527,5cyclictest1291521-21cut09:20:012
1249087993527,4cyclictest1270117-21munin-run08:20:012
1249087993527,4cyclictest1270117-21munin-run08:20:012
1249087993524,8cyclictest1323829-21kernelversion10:13:302
1249087993524,8cyclictest1323829-21kernelversion10:13:302
1249086993528,3cyclictest0-21swapper/109:10:011
1249086993524,8cyclictest1408960-21switchtime12:23:491
124908599352,6cyclictest171rcu_preempt09:48:410
1249085993525,7cyclictest1354254-21munin-plugin-st11:00:010
1249085993525,7cyclictest1354254-21munin-plugin-st11:00:010
1249085993525,6cyclictest0-21swapper/012:23:360
1249085993522,10cyclictest1323674-21fschecks_count10:13:270
1249085993522,10cyclictest1323674-21fschecks_count10:13:270
1249087993427,4cyclictest959-21mta-sts-daemon11:46:282
1249087993427,4cyclictest956-21mta-sts-daemon09:13:502
1249087993427,3cyclictest0-21swapper/210:08:472
1249087993427,3cyclictest0-21swapper/210:08:472
1249086993428,3cyclictest0-21swapper/107:20:021
1249086993427,4cyclictest0-21swapper/112:00:111
1249086993424,7cyclictest1309352-21/usr/sbin/munin09:50:021
124908599346,23cyclictest151ktimers/010:08:390
124908599346,23cyclictest151ktimers/010:08:390
1249085993427,5cyclictest0-21swapper/011:40:010
1249085993427,4cyclictest957-21mta-sts-daemon10:18:530
1249085993427,4cyclictest1386944-21latency_hist11:50:030
1249085993424,7cyclictest0-21swapper/012:15:020
1249085993423,8cyclictest1337448-21sh10:33:330
1249085993423,8cyclictest12550irq/33-eth011:18:350
1249087993327,3cyclictest0-21swapper/212:39:322
1249087993327,3cyclictest0-21swapper/212:39:322
1249087993327,3cyclictest0-21swapper/207:50:012
1249087993326,4cyclictest1315662-21systemd10:00:022
1249087993326,4cyclictest0-21swapper/210:05:002
1249087993326,4cyclictest0-21swapper/210:05:002
1249087993325,5cyclictest1402041-21wc12:13:442
1249087993325,5cyclictest1331853-21latency_hist10:25:032
1249087993324,6cyclictest453-21systemd-logind07:35:012
1249086993325,5cyclictest1397933-21aten_r2power_cu12:08:261
1249086993324,4cyclictest0-21swapper/107:33:351
1249086993321,9cyclictest1294545-21sh09:25:011
1249086993321,8cyclictest1347781-21timerwakeupswit10:48:541
1249085993328,3cyclictest0-21swapper/008:58:340
1249085993326,3cyclictest0-21swapper/007:45:010
1249087993227,2cyclictest0-21swapper/210:20:012
1249087993226,3cyclictest0-21swapper/210:30:022
1249087993225,4cyclictest1272099-21cpuspeed08:28:242
1249087993225,3cyclictest0-21swapper/209:30:012
1249087993220,7cyclictest1359379-21ntpq11:08:362
1249086993223,6cyclictest487177-21cups-browsed10:27:551
1249086993223,5cyclictest1-21systemd09:05:011
1249086993223,4cyclictest0-21swapper/110:38:361
1249086993222,8cyclictest1277415-21timerwakeupswit08:43:421
1249086993222,7cyclictest1354250-21awk11:00:011
1249086993222,7cyclictest1354250-21awk11:00:011
1249085993224,5cyclictest487177-21cups-browsed09:21:540
1249085993220,9cyclictest1341924-21cron10:40:010
1249085993211,14cyclictest151ktimers/007:23:380
124908799313,22cyclictest1362698-21ssh11:13:362
124908799313,22cyclictest1254505-21sshd07:28:372
124908799313,20cyclictest1372647-21sshd11:28:362
1249087993125,3cyclictest0-21swapper/208:00:012
1249087993125,3cyclictest0-21swapper/208:00:012
1249087993124,3cyclictest0-21swapper/209:52:182
124908799312,24cyclictest1366101-21ssh11:18:412
1249086993123,5cyclictest959-21mta-sts-daemon07:40:011
1249086993123,5cyclictest1258958-21ntp_kernel_err07:43:351
1249086993123,5cyclictest1254973-21tail07:28:471
1249086993122,6cyclictest422-21dbus-daemon07:10:021
1249086993122,5cyclictest474-21in:imuxsock08:20:001
1249086993122,5cyclictest474-21in:imuxsock08:20:001
124908699312,22cyclictest1350469-21ssh10:53:411
124908599314,21cyclictest1344202-21ssh10:43:420
1249085993125,4cyclictest1411348-21cpuspeed12:28:240
1249085993125,3cyclictest0-21swapper/011:55:020
1249085993124,5cyclictest1280603-21latency_hist08:55:010
1249085993123,5cyclictest453-21systemd-logind07:20:020
1249085993122,7cyclictest1250640-21latency_hist07:15:020
124908799303,21cyclictest1347425-21ssh10:48:472
1249087993024,3cyclictest0-21swapper/208:50:012
1249087993023,5cyclictest0-21swapper/210:40:012
1249087993023,4cyclictest0-21swapper/208:25:002
1249087993022,5cyclictest1376440-21unixbench_singl11:33:552
1249087993022,5cyclictest1262581-21kworker/u8:7+xprtiod07:55:012
1249087993022,5cyclictest1262581-21kworker/u8:7+xprtiod07:55:012
1249087993020,7cyclictest1411668-21missed_timers12:28:292
1249087993019,8cyclictest1273658-21fschecks_time08:33:282
1249087993019,5cyclictest0-21swapper/210:55:022
1249086993023,4cyclictest1253583-21idleruntime-cro07:25:011
1249086993022,5cyclictest1357494-21latency_hist11:05:021
124908699301,21cyclictest1299597-21ssh09:33:401
1249085993024,4cyclictest1306219-21grep09:43:500
1249085993024,3cyclictest0-21swapper/008:05:010
1249085993022,5cyclictest487177-21cups-browsed08:48:420
124908799298,17cyclictest487177-21cups-browsed08:58:342
1249087992923,3cyclictest0-21swapper/210:43:502
1249087992922,4cyclictest1258004-21unixbench_singl07:38:422
1249087992921,5cyclictest1251938-21unixbench-2d07:19:062
1249086992920,6cyclictest266-21systemd-journal08:35:011
1249085992924,2cyclictest0-21swapper/012:38:300
1249085992924,2cyclictest0-21swapper/012:38:300
1249085992923,4cyclictest1334435-21ssh10:28:490
1249085992918,8cyclictest422-21dbus-daemon08:25:010
1249085992918,6cyclictest1361993-21/usr/sbin/munin11:13:430
124908599291,24cyclictest1382405-21sh11:43:380
124908599291,22cyclictest1311436-21ssh09:53:360
124908799284,9cyclictest171rcu_preempt09:48:452
1249087992822,3cyclictest0-21swapper/209:10:012
1249086992821,4cyclictest1279265-21latency_hist08:50:021
1249086992817,8cyclictest1362247-21cpuspeed11:13:281
1249085992822,4cyclictest1314171-21cpuspeed09:58:250
1249085992819,7cyclictest1266310-21latency08:08:290
1249085992819,7cyclictest1266310-21latency08:08:290
1249087992718,7cyclictest1396533-21sh12:05:012
1249086992718,6cyclictest1250584-21systemd-run07:15:011
1249085992722,2cyclictest0-21swapper/007:10:010
124908599271,6cyclictest171rcu_preempt07:33:400
1249085992712,10cyclictest266-21systemd-journal10:23:370
1249087992619,5cyclictest266-21systemd-journal09:00:372
1249085992619,5cyclictest487177-21cups-browsed07:59:000
1249085992619,5cyclictest487177-21cups-browsed07:59:000
1249085992619,5cyclictest1-21systemd07:30:010
1249085992616,6cyclictest12550irq/33-eth007:48:410
124908599261,20cyclictest1359334-21ssh11:08:340
124908599261,18cyclictest1269566-21modprobe08:18:370
124908599261,18cyclictest1269566-21modprobe08:18:370
1249087992520,2cyclictest0-21swapper/207:15:012
1249087992518,4cyclictest453-21systemd-logind08:40:012
1249086992516,6cyclictest1272455-21needreboot08:28:311
1249085992519,4cyclictest1283582-21unixbench_singl09:03:490
1249085992515,7cyclictest266-21systemd-journal08:29:370
1249087992416,4cyclictest0-21swapper/208:53:392
1249087992317,4cyclictest1249084-21cyclictest08:43:172
1249087992316,4cyclictest1259807-21kworker/u8:1-xprtiod08:13:442
1249087992313,7cyclictest1252801-21latency07:23:302
124908699232,15cyclictest1271016-21ntpq08:23:371
1249086992316,3cyclictest0-21swapper/107:50:011
1249087992215,5cyclictest1259573-21latency_hist07:45:022
124908899192,8cyclictest0-21swapper/308:05:013
124908899191,9cyclictest0-21swapper/311:18:423
124908899172,7cyclictest0-21swapper/312:33:363
124908899172,7cyclictest0-21swapper/311:38:463
124908899172,7cyclictest0-21swapper/310:38:403
124908899171,7cyclictest0-21swapper/309:18:413
124908799177,6cyclictest266-21systemd-journal07:10:022
124908899162,8cyclictest0-21swapper/312:13:353
124908899162,8cyclictest0-21swapper/311:28:333
124908899162,8cyclictest0-21swapper/311:13:353
124908899162,7cyclictest0-21swapper/307:38:313
124908899162,6cyclictest0-21swapper/309:58:333
124908899161,7cyclictest0-21swapper/309:53:383
124908899153,9cyclictest0-21swapper/312:23:363
124908899153,9cyclictest0-21swapper/310:43:413
124908899153,7cyclictest0-21swapper/310:18:373
124908899152,8cyclictest0-21swapper/311:43:373
124908899152,8cyclictest0-21swapper/311:33:463
124908899152,8cyclictest0-21swapper/309:38:453
124908899152,8cyclictest0-21swapper/309:28:333
124908899152,7cyclictest0-21swapper/309:33:403
124908899152,6cyclictest0-21swapper/308:53:393
124908899152,6cyclictest0-21swapper/307:48:383
124908899151,6cyclictest0-21swapper/307:33:373
124908899143,6cyclictest0-21swapper/311:03:343
124908899143,4cyclictest0-21swapper/309:03:413
124908899142,8cyclictest0-21swapper/310:23:383
124908899142,7cyclictest0-21swapper/312:38:353
124908899142,7cyclictest0-21swapper/312:38:353
124908899142,7cyclictest0-21swapper/310:58:413
124908899142,7cyclictest0-21swapper/310:58:413
124908899142,7cyclictest0-21swapper/309:13:363
124908899142,6cyclictest0-21swapper/311:23:363
124908899142,6cyclictest0-21swapper/308:38:373
124908899141,7cyclictest0-21swapper/312:28:323
124908899141,7cyclictest0-21swapper/312:03:463
124908899141,7cyclictest0-21swapper/310:33:223
124908899141,7cyclictest0-21swapper/310:08:383
124908899141,7cyclictest0-21swapper/310:08:383
124908899141,7cyclictest0-21swapper/310:03:363
124908899141,7cyclictest0-21swapper/310:03:363
124908899141,6cyclictest0-21swapper/307:58:343
124908899141,6cyclictest0-21swapper/307:58:343
124908899141,6cyclictest0-21swapper/307:13:363
124908899134,6cyclictest0-21swapper/312:18:413
124908899133,6cyclictest0-21swapper/310:28:423
124908899132,7cyclictest0-21swapper/312:08:363
124908899132,7cyclictest0-21swapper/311:08:363
124908899132,7cyclictest0-21swapper/310:48:443
124908899132,6cyclictest0-21swapper/308:48:363
124908899131,9cyclictest0-21swapper/309:23:423
124908899131,8cyclictest0-21swapper/311:53:283
124908899131,7cyclictest0-21swapper/311:48:263
124908899131,7cyclictest0-21swapper/309:48:383
124908899122,7cyclictest0-21swapper/309:43:383
124908899122,6cyclictest0-21swapper/310:53:393
124908899122,6cyclictest0-21swapper/310:13:253
124908899122,6cyclictest0-21swapper/310:13:253
124908899122,5cyclictest0-21swapper/307:23:373
124908899121,7cyclictest0-21swapper/312:00:013
124908899121,7cyclictest0-21swapper/312:00:013
124908899121,7cyclictest0-21swapper/308:18:413
124908899121,7cyclictest0-21swapper/308:18:413
124908899121,6cyclictest0-21swapper/308:08:373
124908899121,6cyclictest0-21swapper/308:08:373
124908899121,6cyclictest0-21swapper/307:19:003
124908899121,5cyclictest0-21swapper/308:23:363
124908899121,5cyclictest0-21swapper/308:13:383
124908899112,5cyclictest0-21swapper/308:58:343
124908899111,7cyclictest0-21swapper/308:33:333
124908899111,6cyclictest0-21swapper/308:28:263
124908899111,5cyclictest0-21swapper/308:43:253
124908899102,5cyclictest0-21swapper/309:10:013
124908899101,6cyclictest0-21swapper/307:28:273
124908899101,5cyclictest0-21swapper/307:43:273
12490889981,6cyclictest0-21swapper/307:10:023
12490889981,5cyclictest0-21swapper/307:53:233
12490889981,5cyclictest0-21swapper/307:53:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional