You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-14 - 22:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Wed May 14, 2025 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1040754994137,3cyclictest1339389-21kworker/u8:0+flush-259:012:40:000
1040755993529,4cyclictest1040703-21latency_hist07:10:011
1040754993429,4cyclictest1187234-21kworker/u8:1+flush-259:010:35:000
1040759993124,5cyclictest3000860-21systemd-journal08:35:003
1040759993124,5cyclictest3000860-21systemd-journal08:34:593
1040754993127,3cyclictest1312943-21kworker/u8:3+flush-259:011:40:010
1040754993027,2cyclictest1187234-21kworker/u8:1+events_unbound10:40:000
1040754993025,4cyclictest1349796-21apt-get12:10:000
1040755992720,5cyclictest1066711-21latency_hist07:35:011
1040754992724,2cyclictest1339389-21kworker/u8:0+flush-259:012:05:010
1040754992724,2cyclictest1035594-21kworker/u8:1+flush-259:007:20:160
1040754992723,3cyclictest1187234-21kworker/u8:1+flush-259:010:30:000
104075499251,23cyclictest1171430-21munin-run09:15:010
1040759992411,2cyclictest1345250-21irqcore12:05:173
1040759992319,3cyclictest1376477-21/usr/sbin/munin12:35:203
1040759992310,3cyclictest3000860-21systemd-journal11:38:453
1040754992321,1cyclictest1035594-21kworker/u8:1+events_unbound08:00:000
1040754992320,2cyclictest1323074-21kworker/u8:0+flush-259:011:50:240
1040754992318,3cyclictest1181419-21kworker/u8:0+events_unbound09:50:190
1040754992317,4cyclictest1061544-21latency_hist07:30:020
1040756992217,3cyclictest1056325-21latency_hist07:25:012
1040754992220,1cyclictest1355661-21kworker/u8:4+flush-259:012:15:230
1040754992219,2cyclictest1239270-21kworker/u8:2+events_unbound12:20:220
1040754992219,2cyclictest1114331-21kworker/u8:3+flush-259:008:20:240
1040754992219,2cyclictest1082770-21kworker/u8:2+events_unbound08:25:180
1040754992219,2cyclictest1040962-21kworker/u8:0+flush-259:007:10:200
1040754992218,3cyclictest1181419-21kworker/u8:0+events_unbound09:55:240
1040756992117,3cyclictest3000860-21systemd-journal12:30:472
1040756992114,5cyclictest1103459-21apt-get08:10:012
1040755992115,5cyclictest1371265-21/usr/sbin/munin12:30:141
1040755992115,4cyclictest1328072-21ntpq11:45:221
1040754992117,3cyclictest1281292-21kworker/u8:1+events_unbound11:00:160
104075999207,3cyclictest1198019-21cat09:40:023
1040756992017,2cyclictest1082770-21kworker/u8:2+flush-259:007:55:192
1040755992015,3cyclictest1103487-21kworker/u8:0+flush-259:008:50:271
1040755992013,5cyclictest1045956-21latency_hist07:15:021
1040754992018,1cyclictest211rcuc/011:15:000
1040754992017,1cyclictest1229091-21pluginstate10:10:000
1040754992016,2cyclictest1040962-21kworker/u8:0+flush-259:007:30:210
1040754992015,4cyclictest1087915-21latency_hist07:55:010
1040754992013,5cyclictest3000860-21systemd-journal08:10:000
1040754992012,6cyclictest492-21dbus-daemon10:50:000
1040756991915,3cyclictest1350458-21/usr/sbin/munin12:10:252
1040756991913,4cyclictest1365856-21latency_hist12:25:002
1040754991916,2cyclictest3000063-21rs:main0
1040754991916,2cyclictest1040962-21kworker/u8:0+events_unbound07:40:220
1040755991812,3cyclictest1291948-21kworker/u8:1+flush-259:011:40:181
1040755991811,5cyclictest1239758-21/usr/sbin/munin10:20:151
1040754991815,2cyclictest1291948-21kworker/u8:1+flush-259:011:20:260
1040754991815,2cyclictest1291948-21kworker/u8:1+events_unbound11:40:190
1040754991815,2cyclictest1130087-21kworker/u8:1+events_unbound09:20:210
1040756991712,3cyclictest3000860-21systemd-journal09:40:002
1040756991712,3cyclictest1071886-21sh07:39:592
1040755991714,2cyclictest1181419-21kworker/u8:0+flush-259:009:45:151
1040755991713,3cyclictest1291948-21kworker/u8:1+flush-259:011:50:161
1040755991712,4cyclictest1114331-21kworker/u8:3+flush-259:008:40:261
1040755991712,4cyclictest1082770-21kworker/u8:2+flush-259:008:25:131
1040754991714,2cyclictest1291948-21kworker/u8:1+flush-259:011:30:210
1040759991613,2cyclictest1035594-21kworker/u8:1+flush-259:008:15:003
1040759991612,2cyclictest1098556-21apt-get08:05:133
1040759991611,3cyclictest3000860-21systemd-journal07:15:003
1040759991610,4cyclictest1302500-21latency_hist11:25:013
1040756991614,1cyclictest1328967-21idleruntime-cro11:50:002
1040756991612,3cyclictest1239270-21kworker/u8:2+flush-259:012:25:132
1040756991610,4cyclictest1166617-21diskstats09:10:152
104075599169,5cyclictest1092266-21ntpq07:55:241
1040755991613,2cyclictest1132429-21kthreadcore08:35:191
1040754991614,1cyclictest60650irq/133-enp0s29f1:rx-012:25:210
1040754991613,2cyclictest60650irq/133-enp0s29f1:rx-009:41:460
1040754991611,4cyclictest3000860-21systemd-journal08:17:010
1040759991512,2cyclictest1146715-21kthreadcore08:50:203
1040759991512,2cyclictest1113452-21ntp_offset08:15:243
1040759991511,3cyclictest1324034-21apt-get11:45:123
1040759991511,3cyclictest1229398-21cpuspeed_turbos10:10:153
1040759991510,3cyclictest3000063-21rs:main3
1040756991511,3cyclictest1345099-21fschecks_time12:05:152
1040756991511,3cyclictest1239270-21kworker/u8:2+flush-259:011:10:002
104075599159,4cyclictest1350306-21cat12:10:011
104075599159,4cyclictest1307942-21/usr/sbin/munin11:30:221
104075599158,5cyclictest1349791-21latency_hist12:10:011
104075599153,2cyclictest12541942ntp_kernel_err10:30:231
1040755991511,3cyclictest1161041-21kworker/u8:3+flush-259:009:10:161
1040755991511,3cyclictest1072222-21apt-key07:40:001
1040755991511,2cyclictest0-21swapper/111:10:001
104075499159,4cyclictest1271083-21diskstats10:50:140
104075499159,4cyclictest1271083-21diskstats10:50:130
1040754991513,1cyclictest60650irq/133-enp0s29f1:rx-009:15:250
1040754991512,2cyclictest60650irq/133-enp0s29f1:rx-010:41:460
1040754991512,2cyclictest1161041-21kworker/u8:3+flush-259:009:25:260
1040754991512,2cyclictest1035594-21kworker/u8:1+flush-259:007:35:280
1040754991511,3cyclictest1334066-21kworker/u8:3+flush-259:012:00:010
1040754991510,3cyclictest1161002-21latency_hist09:05:000
104075999149,4cyclictest1260727-21diskstats10:40:143
104075999149,3cyclictest1344575-21latency_hist12:05:013
1040759991411,2cyclictest0-21swapper/308:50:003
1040759991411,2cyclictest0-21swapper/308:40:193
1040759991410,3cyclictest1255136-21kworker/u8:3+flush-259:010:50:173
1040759991410,3cyclictest1255136-21kworker/u8:3+flush-259:010:50:173
104075699148,4cyclictest1192217-21latency_hist09:35:012
1040756991411,2cyclictest1035594-21kworker/u8:1+flush-259:007:50:172
104075599148,4cyclictest3000860-21systemd-journal07:53:431
104075599148,4cyclictest1381458-21latency_hist12:40:011
104075599141,11cyclictest3000061-21in:imuxsock08:19:591
1040755991410,3cyclictest1162743-21kthreadcore09:05:181
1040755991410,2cyclictest0-21swapper/107:30:011
1040754991413,1cyclictest60650irq/133-enp0s29f1:rx-011:15:240
1040754991412,2cyclictest60650irq/133-enp0s29f1:rx-009:32:250
1040754991412,2cyclictest60650irq/133-enp0s29f1:rx-009:09:400
1040754991412,2cyclictest60650irq/133-enp0s29f1:rx-008:10:370
1040754991412,1cyclictest60650irq/133-enp0s29f1:rx-011:45:180
1040754991412,1cyclictest60650irq/133-enp0s29f1:rx-011:25:250
1040754991412,1cyclictest60650irq/133-enp0s29f1:rx-009:46:290
1040754991412,1cyclictest60650irq/133-enp0s29f1:rx-008:45:260
1040754991412,1cyclictest60650irq/133-enp0s29f1:rx-007:45:540
1040754991411,2cyclictest60650irq/133-enp0s29f1:rx-008:53:590
1040754991411,2cyclictest60650irq/133-enp0s29f1:rx-007:15:150
104075499141,11cyclictest3000860-21systemd-journal10:15:460
1040754991410,3cyclictest1187234-21kworker/u8:1+events_unbound09:39:460
1040754991410,2cyclictest60650irq/133-enp0s29f1:rx-008:35:130
1040754991410,2cyclictest1066732-21kworker/u8:3+events_unbound08:00:230
1040759991310,2cyclictest3000860-21systemd-journal08:40:013
1040759991310,2cyclictest1338445-21ntp_kernel_err11:55:223
1040759991310,2cyclictest1234767-21fschecks_time10:15:163
1040759991310,2cyclictest1161684-21irqcore09:05:183
1040759991310,2cyclictest1130087-21kworker/u8:1+flush-259:009:15:003
104075699139,3cyclictest1151102-21fschecks_count08:55:152
104075699137,5cyclictest1255555-21uniq10:35:162
104075699137,4cyclictest1081824-21ntpq07:45:222
1040756991310,2cyclictest1281751-21sed11:00:182
104075599138,4cyclictest1040962-21kworker/u8:0+flush-259:007:40:341
104075599137,4cyclictest1050214-21ntpq07:15:251
104075599136,4cyclictest3000860-21systemd-journal11:21:461
1040755991310,2cyclictest1268051-21tr10:45:201
1040755991310,2cyclictest1198318-21cut09:40:161
1040754991311,1cyclictest60650irq/133-enp0s29f1:rx-010:00:200
1040754991310,2cyclictest1239270-21kworker/u8:2+flush-259:010:20:160
104075999129,2cyclictest1292428-21cpuspeed_turbos11:15:143
104075999129,2cyclictest1223439-21unixbench_singl10:00:263
104075999128,3cyclictest1224113-21/usr/sbin/munin10:05:143
104075999127,4cyclictest1072148-21cat07:40:003
104075999127,3cyclictest1180925-21ntp_kernel_pll_09:20:223
104075999120,2cyclictest1061547-21awk07:30:003
104075699129,2cyclictest1376583-21cpuspeed_turbos12:35:112
104075699129,2cyclictest1135376-21cpuspeed_turbos08:40:132
104075699129,2cyclictest1050665-21unixbench_multi07:15:302
104075699129,1cyclictest0-21swapper/209:40:182
104075699127,4cyclictest1302728-21/usr/sbin/munin11:25:212
104075699127,4cyclictest1135051-21latency_hist08:40:002
104075599129,2cyclictest0-21swapper/110:55:191
104075599128,3cyclictest1296871-21unixbench_singl11:15:261
104075599128,3cyclictest1248965-21grep10:25:201
104075599125,5cyclictest1196514-21ntpq09:35:241
104075599120,10cyclictest1-21systemd09:34:451
104075499129,2cyclictest60650irq/133-enp0s29f1:rx-010:59:090
104075499129,2cyclictest1239270-21kworker/u8:2+events_unbound11:08:460
104075499122,1cyclictest1229332-21apt-get10:10:130
1040754991211,1cyclictest60650irq/133-enp0s29f1:rx-008:41:250
1040754991210,1cyclictest60650irq/133-enp0s29f1:rx-008:55:170
104075999119,1cyclictest0-21swapper/311:30:203
104075999119,1cyclictest0-21swapper/311:05:003
104075999118,2cyclictest1210834-21kthreadcore09:50:173
104075999118,2cyclictest1082709-21cron07:50:003
104075699119,1cyclictest1319437-21kthreadcore11:40:192
104075699119,1cyclictest0-21swapper/208:25:172
104075699118,2cyclictest503-21systemd-logind07:15:002
104075699118,2cyclictest3000061-21in:imuxsock10:25:002
104075699118,2cyclictest1270483-21kworker/u8:0+flush-259:011:15:162
104075699118,2cyclictest1218992-21cpuspeed_turbos10:00:132
104075699118,2cyclictest1072653-21cpuspeed_turbos07:40:162
104075699110,8cyclictest1187000-21latency_hist09:30:012
104075599118,2cyclictest1224549-21irqcore10:05:181
104075599118,2cyclictest1098967-21irqcore08:05:191
104075599118,1cyclictest0-21swapper/112:00:271
104075599115,4cyclictest3000860-21systemd-journal10:00:441
104075999106,3cyclictest1363998-21cut12:20:193
104075999106,3cyclictest1250840-21cut10:30:183
104075999106,3cyclictest1187577-21idleruntime09:30:163
104075999106,2cyclictest1118608-21ntpq08:20:223
104075599107,2cyclictest1281395-21cpuspeed_turbos11:00:141
104075599107,2cyclictest1160201-21nvmesmart_nvme009:00:241
104075599100,5cyclictest1365864-21awk12:25:001
10407599997,1cyclictest1051484-21cpuspeed_turbos07:20:143
10407599995,3cyclictest1359772-21ntpq12:15:223
10407599990,2cyclictest1208256-21latency_hist09:50:013
10407569996,2cyclictest1314201-21tr11:35:182
10407569996,2cyclictest1255136-21kworker/u8:3+flush-259:010:55:172
10407569995,2cyclictest1212655-21ntpq09:50:222
10407569984,3cyclictest1254357-21ps10:30:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional