You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-15 - 13:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Wed Oct 15, 2025 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4368992724,2cyclictest9125-21idleruntime-cro19:20:000
4378992422,1cyclictest2564-21cron21:25:003
437099240,23cyclictest31470-21basename22:24:591
436899240,23cyclictest0-21swapper/020:08:510
4378992321,1cyclictest593-21dbus-daemon22:09:593
4378992321,1cyclictest11103-21grep22:50:013
437899230,22cyclictest8681-21cron22:45:013
437899230,22cyclictest8442-21munin-run20:25:013
437899230,22cyclictest7288-21apt19:18:493
437899230,22cyclictest31475-21latency_hist22:25:003
437899230,1cyclictest5561-21dump-pmu-power23:44:593
4374992321,1cyclictest24203-21cron22:10:002
437499230,22cyclictest593-21dbus-daemon20:40:002
437499230,22cyclictest32079-21apt00:39:592
437499230,22cyclictest29087-21munin-plugin-st22:20:002
437499230,1cyclictest313-21systemd-journal23:45:002
436899230,22cyclictest14044-21apt-get22:58:500
436899230,22cyclictest10372-21awk23:54:590
436899230,22cyclictest0-21swapper/021:00:010
436899230,1cyclictest2594-21apt21:24:590
4378992221,0cyclictest0-21swapper/319:35:013
437899220,21cyclictest23636-21basename20:55:003
437899220,21cyclictest18274-21cron23:05:003
437899220,21cyclictest0-21swapper/322:20:003
437899220,21cyclictest0-21swapper/322:03:513
437899220,21cyclictest0-21swapper/320:40:003
437899220,21cyclictest0-21swapper/319:28:493
437899220,21cyclictest0-21swapper/300:33:513
437899220,1cyclictest32070-21turbostat00:39:593
437899220,1cyclictest26622-21munin-run22:15:003
437899220,1cyclictest15290-21munin-run00:05:003
437899220,0cyclictest0-21swapper/322:00:003
4374992221,1cyclictest21716-21apt-get19:48:502
4374992220,1cyclictest30754-21awk23:30:002
4374992220,1cyclictest21242-21munin-run20:49:592
4374992220,1cyclictest0-21swapper/219:58:502
437499220,21cyclictest7950-21apt-get21:38:502
437499220,21cyclictest6587-21apt-get20:23:502
437499220,21cyclictest3159-21apt23:40:002
437499220,21cyclictest30195-21perf21:15:012
437499220,21cyclictest2028-21apt-get22:33:492
437499220,21cyclictest18285-21cron23:05:002
437499220,21cyclictest0-21swapper/219:54:592
437499220,21cyclictest0-21swapper/219:28:502
437499220,1cyclictest31477-21dump-pmu-power22:24:592
437099220,21cyclictest4161-21apt-get20:18:491
437099220,21cyclictest27833-21apt-get00:33:491
437099220,21cyclictest24049-21apt-get23:18:511
437099220,21cyclictest10806-21sh20:30:011
437099220,1cyclictest10367-21apt23:55:001
436899220,21cyclictest0-21swapper/021:58:530
436899220,21cyclictest0-21swapper/020:43:510
436899220,21cyclictest0-21swapper/020:23:490
436899220,21cyclictest0-21swapper/000:03:500
436899220,1cyclictest31321-21apt-get23:33:530
436899220,1cyclictest0-21swapper/019:50:000
4378992120,0cyclictest0-21swapper/319:58:513
4378992119,1cyclictest20063-21turbostat00:15:013
4378992119,1cyclictest0-21swapper/321:03:473
4378992119,1cyclictest0-21swapper/300:00:003
437899210,20cyclictest0-21swapper/322:33:503
437899210,20cyclictest0-21swapper/319:48:503
437899210,0cyclictest0-21swapper/320:48:493
4374992120,0cyclictest0-21swapper/200:13:542
4374992119,1cyclictest12748-21systemctl00:00:002
4370992121,0cyclictest0-21swapper/122:53:491
4370992121,0cyclictest0-21swapper/119:33:511
4370992120,1cyclictest3723-21apt-get23:43:511
4370992120,1cyclictest18229-21apt-get00:13:531
4370992119,1cyclictest31228-21munin-run20:05:001
4370992119,1cyclictest0-21swapper/122:15:001
437099210,20cyclictest0-21swapper/120:38:491
437099210,1cyclictest0-21swapper/100:18:521
437099210,0cyclictest17560-21apt-get21:58:521
4368992121,0cyclictest4437-21apt-get22:38:530
4368992121,0cyclictest0-21swapper/023:23:530
4368992121,0cyclictest0-21swapper/022:28:530
4368992121,0cyclictest0-21swapper/021:00:020
4368992120,1cyclictest0-21swapper/023:05:010
436899210,21cyclictest0-21swapper/021:53:490
436899210,21cyclictest0-21swapper/020:18:490
436899210,20cyclictest26907-21apt-get19:58:510
436899210,20cyclictest0-21swapper/000:33:510
436899210,0cyclictest11379-21apt-get20:33:500
436899210,0cyclictest0-21swapper/022:08:500
4378992020,0cyclictest0-21swapper/323:48:483
4378992020,0cyclictest0-21swapper/323:30:013
4378992020,0cyclictest0-21swapper/323:23:543
4378992020,0cyclictest0-21swapper/320:13:483
4378992020,0cyclictest0-21swapper/319:23:483
4378992018,1cyclictest21099-21perf23:10:003
437899200,0cyclictest0-21swapper/321:18:453
437899200,0cyclictest0-21swapper/319:55:003
437899200,0cyclictest0-21swapper/300:08:533
4374992020,0cyclictest6128-21apt-get23:48:492
4374992020,0cyclictest30732-21apt-get21:18:452
4374992020,0cyclictest1756-21apt-get20:13:482
4374992020,0cyclictest0-21swapper/223:23:532
4374992020,0cyclictest0-21swapper/222:43:542
4374992020,0cyclictest0-21swapper/219:23:492
4374992017,3cyclictest23042-21apt-get00:23:502
437499200,19cyclictest0-21swapper/223:53:502
437499200,0cyclictest0-21swapper/222:03:512
437499200,0cyclictest0-21swapper/221:58:512
437499200,0cyclictest0-21swapper/200:33:502
4370992020,0cyclictest0-21swapper/123:23:531
4370992020,0cyclictest0-21swapper/122:03:511
4370992020,0cyclictest0-21swapper/120:08:501
4370992020,0cyclictest0-21swapper/119:50:001
4370992019,1cyclictest32041-21apt-get22:28:521
4370992019,1cyclictest12104-21apt-get19:28:491
4370992019,0cyclictest10366-21apt-get21:43:481
4370992018,1cyclictest0-21swapper/121:05:011
437099200,0cyclictest0-21swapper/123:58:501
437099200,0cyclictest0-21swapper/123:05:021
437099200,0cyclictest0-21swapper/119:48:491
4368992020,0cyclictest0-21swapper/023:43:510
4368992020,0cyclictest0-21swapper/022:33:490
4368992020,0cyclictest0-21swapper/022:18:500
4368992020,0cyclictest0-21swapper/019:13:480
4368992020,0cyclictest0-21swapper/000:13:530
4368992019,1cyclictest0-21swapper/019:33:500
4368992019,0cyclictest0-21swapper/019:43:490
4368992018,1cyclictest29528-21latency_hist21:10:010
4368992018,1cyclictest0-21swapper/021:29:590
436899200,20cyclictest0-21swapper/022:25:000
436899200,0cyclictest0-21swapper/023:03:510
436899200,0cyclictest0-21swapper/022:48:490
436899200,0cyclictest0-21swapper/021:18:440
436899200,0cyclictest0-21swapper/000:18:530
4378991919,0cyclictest0-21swapper/322:28:533
4378991919,0cyclictest0-21swapper/321:38:513
4378991919,0cyclictest0-21swapper/320:43:513
4378991919,0cyclictest0-21swapper/320:18:483
4378991919,0cyclictest0-21swapper/319:13:483
4378991918,1cyclictest0-21swapper/322:38:523
4374991919,0cyclictest4890-21apt-get19:13:472
4374991919,0cyclictest16595-21apt-get20:43:512
4374991919,0cyclictest0-21swapper/222:28:522
4374991919,0cyclictest0-21swapper/221:03:472
4374991919,0cyclictest0-21swapper/220:18:492
4374991918,1cyclictest0-21swapper/222:38:522
437499190,19cyclictest0-21swapper/200:08:542
4370991919,0cyclictest0-21swapper/122:58:511
4370991919,0cyclictest0-21swapper/121:23:491
4370991918,1cyclictest19384-21apt-get20:48:481
4370991917,1cyclictest23501-21munin-run23:14:591
4368991919,0cyclictest0-21swapper/022:43:540
4368991919,0cyclictest0-21swapper/000:08:520
436899190,0cyclictest0-21swapper/019:28:490
4378991818,0cyclictest0-21swapper/320:28:483
4378991818,0cyclictest0-21swapper/300:28:523
4378991816,1cyclictest24304-21sendmail-msp21:00:003
4378991815,2cyclictest15920-21basename23:00:003
4374991818,0cyclictest16903-21apt-get19:38:492
4374991818,0cyclictest0-21swapper/223:10:002
4374991818,0cyclictest0-21swapper/200:28:512
4374991817,1cyclictest0-21swapper/223:33:522
4374991817,0cyclictest8981-21apt-get20:28:472
4374991816,1cyclictest24259-21apt-get21:00:002
437499180,1cyclictest14609-21perf21:50:002
4370991818,0cyclictest0-21swapper/121:48:491
4370991818,0cyclictest0-21swapper/119:58:511
4370991817,1cyclictest15833-21apt-get00:08:531
4370991816,1cyclictest8707-21munin-run22:45:011
4370991816,1cyclictest15289-21munin-plugin-st00:05:001
437099180,18cyclictest0-21swapper/100:23:491
4368991818,0cyclictest0-21swapper/022:53:480
4368991818,0cyclictest0-21swapper/020:03:500
4368991817,0cyclictest0-21swapper/000:28:510
4368991816,1cyclictest0-21swapper/023:14:590
4378991717,0cyclictest0-21swapper/323:33:533
4378991717,0cyclictest0-21swapper/320:03:503
4378991717,0cyclictest0-21swapper/319:38:493
4378991717,0cyclictest0-21swapper/300:23:503
4378991716,1cyclictest11646-21apt-get22:53:483
4378991715,1cyclictest593-21dbus-daemon21:55:003
4378991715,1cyclictest12203-21turbostat21:45:003
4378991715,1cyclictest0-21swapper/319:45:003
4374991717,0cyclictest0-21swapper/222:48:502
4374991717,0cyclictest0-21swapper/220:03:492
4374991717,0cyclictest0-21swapper/200:03:512
4374991716,1cyclictest3147-21apt-get21:28:512
4374991716,1cyclictest24764-21apt-get22:13:492
4374991715,1cyclictest21148-21awk19:45:002
4370991717,0cyclictest0-21swapper/123:33:531
4370991717,0cyclictest0-21swapper/123:28:501
4370991717,0cyclictest0-21swapper/123:03:521
4370991717,0cyclictest0-21swapper/121:28:501
4370991717,0cyclictest0-21swapper/120:53:511
4370991717,0cyclictest0-21swapper/120:13:491
4370991717,0cyclictest0-21swapper/119:23:491
4370991716,1cyclictest593-21dbus-daemon19:20:001
4370991716,0cyclictest9247-21apt-get22:48:501
4368991717,0cyclictest0-21swapper/023:28:510
4368991717,0cyclictest0-21swapper/021:38:510
4368991716,1cyclictest30231-21apt-get00:38:510
4368991715,1cyclictest12740-21cron23:59:590
4378991616,0cyclictest0-21swapper/321:28:513
4378991616,0cyclictest0-21swapper/321:08:493
4378991614,1cyclictest12761-21apt-get21:48:493
4374991616,0cyclictest27638-21apt-get21:08:502
4374991616,0cyclictest0-21swapper/222:58:522
4374991616,0cyclictest0-21swapper/222:53:492
4374991616,0cyclictest0-21swapper/221:43:492
4374991615,1cyclictest0-21swapper/221:55:002
437499160,15cyclictest25659-21unixbench-2d23:19:012
4370991616,0cyclictest0-21swapper/122:38:531
4370991616,0cyclictest0-21swapper/122:18:501
4370991614,1cyclictest0-21swapper/120:43:551
4368991616,0cyclictest0-21swapper/022:13:480
4368991614,1cyclictest0-21swapper/020:30:000
436899160,15cyclictest0-21swapper/019:23:500
4378991515,0cyclictest0-21swapper/323:13:523
437899150,1cyclictest0-21swapper/321:33:563
437899150,14cyclictest0-21swapper/323:35:583
437899150,14cyclictest0-21swapper/323:16:073
437899150,14cyclictest0-21swapper/300:15:063
4374991515,0cyclictest0-21swapper/223:13:522
4370991515,0cyclictest0-21swapper/121:33:501
437099150,1cyclictest21165-21perf19:44:591
4368991515,0cyclictest0-21swapper/021:48:490
4368991515,0cyclictest0-21swapper/000:23:500
4368991513,1cyclictest12223-21munin-run21:45:000
436899150,1cyclictest0-21swapper/020:53:520
4378991414,0cyclictest0-21swapper/321:13:033
437899140,14cyclictest0-21swapper/320:31:313
4374991414,0cyclictest0-21swapper/221:30:232
437499140,14cyclictest0-21swapper/221:21:052
437499140,14cyclictest0-21swapper/220:31:312
437499140,14cyclictest0-21swapper/220:08:372
437499140,14cyclictest0-21swapper/219:32:092
437499140,14cyclictest0-21swapper/219:19:142
437499140,0cyclictest0-21swapper/220:52:562
437499140,0cyclictest0-21swapper/200:15:062
4370991414,0cyclictest0-21swapper/121:50:361
4370991414,0cyclictest0-21swapper/121:35:421
4370991414,0cyclictest0-21swapper/121:10:241
437099140,14cyclictest0-21swapper/121:05:541
437099140,14cyclictest0-21swapper/119:35:091
437099140,14cyclictest0-21swapper/100:35:211
437099140,0cyclictest0-21swapper/123:45:171
437099140,0cyclictest0-21swapper/123:35:171
437099140,0cyclictest0-21swapper/122:30:141
437099140,0cyclictest0-21swapper/122:05:151
437099140,0cyclictest0-21swapper/121:15:591
437099140,0cyclictest0-21swapper/120:58:401
4368991414,0cyclictest0-21swapper/023:36:320
4368991414,0cyclictest0-21swapper/023:18:140
4368991414,0cyclictest0-21swapper/021:32:570
4368991414,0cyclictest0-21swapper/020:35:060
4368991412,1cyclictest23561-21apt-get19:50:020
436899140,14cyclictest0-21swapper/023:45:090
436899140,14cyclictest0-21swapper/021:10:540
436899140,14cyclictest0-21swapper/020:12:210
436899140,0cyclictest0-21swapper/022:00:280
4378991313,0cyclictest0-21swapper/320:05:233
4368991313,0cyclictest0-21swapper/019:35:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional