You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 23:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Tue Aug 26, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
224442993024,4cyclictest540321-21latency_hist12:15:000
224456992421,2cyclictest503-21systemd-logind10:35:003
224442992417,5cyclictest3754036-21systemd-journal09:15:010
224456992319,3cyclictest317606-21latency_hist08:40:013
224447992316,5cyclictest457908-21latency_hist10:55:001
224447992216,4cyclictest259667-21ntpq07:40:221
224452992115,4cyclictest447154-21awk10:45:002
224442992116,3cyclictest405845-21apt-get10:05:140
224442992115,4cyclictest3000063-21rs:main0
224452992017,2cyclictest260523-21kworker/u8:4+flush-259:008:05:202
224452992017,2cyclictest170653-21kworker/u8:1+flush-259:007:30:212
224452992014,4cyclictest3754036-21systemd-journal10:20:492
224442992013,5cyclictest472474-21ntpq11:05:220
22444299201,17cyclictest3000063-21rs:main0
224456991916,2cyclictest488845-21idleruntime-cro11:25:003
224456991913,4cyclictest384980-21latency_hist09:45:003
224452991913,4cyclictest170653-21kworker/u8:1+flush-259:007:20:192
224447991915,3cyclictest306928-21kworker/u8:0+flush-259:008:50:011
224456991816,1cyclictest356254-21kthreadcore09:15:193
224456991815,2cyclictest317833-21kworker/u8:3+events_unbound09:25:013
224456991815,2cyclictest306887-21cron08:30:003
224456991814,3cyclictest3754036-21systemd-journal09:05:003
224456991813,4cyclictest300911-21ntpq08:20:213
224452991812,4cyclictest380103-21/usr/sbin/munin09:40:162
224447991815,2cyclictest468413-21kworker/u8:0+flush-259:011:45:281
224447991815,2cyclictest411979-21kthreadcore10:10:181
224447991815,2cyclictest411979-21kthreadcore10:10:181
224447991815,2cyclictest343581-21kworker/u8:4+events_unbound09:20:161
224447991814,3cyclictest348521-21cron09:10:001
224447991813,4cyclictest3754036-21systemd-journal12:35:001
224447991812,4cyclictest3754036-21systemd-journal11:30:011
22444799180,16cyclictest1-21systemd08:30:001
224456991714,2cyclictest525238-21cpuspeed_turbos12:00:123
224456991714,2cyclictest514628-21kworker/u8:1+flush-259:012:10:293
224456991714,2cyclictest514628-21kworker/u8:1+events_unbound12:30:283
224456991714,2cyclictest250004-21kworker/u8:3+flush-259:007:40:223
224456991714,2cyclictest240579-21sed07:25:173
224456991714,2cyclictest229932-21cpuspeed_turbos07:15:163
224452991714,2cyclictest3000061-21in:imuxsock08:05:002
224452991712,4cyclictest3754036-21systemd-journal09:19:362
224447991715,1cyclictest563464-21kthreadcore12:35:181
224447991715,1cyclictest3000063-21rs:main1
224447991714,2cyclictest530041-21sh12:05:001
224447991714,2cyclictest473334-21idleruntime-cro11:10:001
224447991714,2cyclictest317833-21kworker/u8:3+events_unbound09:25:141
224447991712,4cyclictest463312-21apt-get11:00:151
224442991714,2cyclictest60650irq/133-enp0s29f1:rx-008:50:380
22444299171,15cyclictest1-21systemd10:54:340
224456991614,1cyclictest3000063-21rs:main3
224456991613,2cyclictest441132-21unixbench_multi10:35:303
224456991612,3cyclictest317833-21kworker/u8:3+events_unbound08:40:153
224452991612,3cyclictest260524-21kworker/u8:5+flush-259:007:55:182
224452991612,3cyclictest260524-21kworker/u8:5+flush-259:007:55:182
224452991610,4cyclictest410692-21latency_hist10:10:002
224447991613,2cyclictest447209-21kworker/u8:1+events_unbound11:20:151
224447991613,2cyclictest281410-21kworker/u8:1+flush-259:008:05:281
224447991613,2cyclictest229828-21kworker/u8:0+flush-259:007:35:261
224447991613,1cyclictest0-21swapper/108:25:001
224447991612,2cyclictest441599-21latency_hist10:40:001
22444299169,4cyclictest260500-21latency_hist07:45:000
224442991614,1cyclictest60650irq/133-enp0s29f1:rx-009:20:180
224442991614,1cyclictest60650irq/133-enp0s29f1:rx-009:00:150
224442991614,1cyclictest60650irq/133-enp0s29f1:rx-007:36:030
224442991613,2cyclictest60650irq/133-enp0s29f1:rx-012:06:420
224442991613,2cyclictest60650irq/133-enp0s29f1:rx-010:25:250
224442991613,2cyclictest60650irq/133-enp0s29f1:rx-007:51:220
224442991613,2cyclictest60650irq/133-enp0s29f1:rx-007:10:220
224442991613,2cyclictest0-21swapper/011:59:590
224442991612,2cyclictest0-21swapper/008:45:190
224442991611,3cyclictest3754036-21systemd-journal08:57:350
224442991610,4cyclictest488861-21latency_hist11:25:020
224456991513,1cyclictest452348-21apt-key10:50:013
224456991512,2cyclictest540551-21kworker/u8:2+events_unbound12:20:253
224456991512,2cyclictest395250-21kworker/u8:1+flush-259:010:09:403
224456991512,2cyclictest384994-21kworker/u8:0+events_unbound10:00:193
224456991512,2cyclictest328155-21kworker/u8:1+events_unbound08:55:193
224456991512,2cyclictest3000063-21rs:main3
224456991512,2cyclictest276030-21kworker/u8:0+events_unbound08:15:263
224456991511,3cyclictest421226-21kworker/u8:4+flush-259:010:41:403
224456991511,3cyclictest420747-21kworker/u8:3+flush-259:010:19:403
224456991510,4cyclictest290655-21ntpq08:10:253
22445299159,4cyclictest296615-21latency_hist08:20:012
22445299158,4cyclictest415833-21latency_hist10:15:002
224452991512,2cyclictest494248-21kworker/u8:2+events_unbound11:30:242
224452991512,2cyclictest260524-21kworker/u8:5+flush-259:008:20:172
224447991512,2cyclictest494248-21kworker/u8:2+flush-259:011:50:201
224447991512,2cyclictest452373-21kworker/u8:3+events_unbound10:55:201
224447991512,2cyclictest384994-21kworker/u8:0+events_unbound10:15:191
224447991512,2cyclictest328155-21kworker/u8:1+events_unbound09:00:011
224447991512,2cyclictest170653-21kworker/u8:1+events_unbound07:25:201
224447991511,3cyclictest468413-21kworker/u8:0+flush-259:011:10:211
22444299159,4cyclictest565122-21ntpq12:35:200
224442991513,2cyclictest60650irq/133-enp0s29f1:rx-010:31:040
224442991513,2cyclictest60650irq/133-enp0s29f1:rx-010:00:160
224442991513,2cyclictest60650irq/133-enp0s29f1:rx-007:16:150
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-011:35:220
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-010:45:250
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-010:40:210
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-010:15:300
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-009:45:190
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-009:05:180
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-008:40:170
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-008:39:170
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-007:30:200
224442991513,1cyclictest60650irq/133-enp0s29f1:rx-007:20:280
224442991513,1cyclictest374377-21kworker/0:0+events_power_efficient10:20:250
224442991512,2cyclictest60650irq/133-enp0s29f1:rx-011:25:150
224442991512,2cyclictest60650irq/133-enp0s29f1:rx-009:31:180
224442991512,2cyclictest60650irq/133-enp0s29f1:rx-008:00:170
224442991512,2cyclictest60650irq/133-enp0s29f1:rx-007:47:290
224442991512,2cyclictest272134-21kthreadcore07:55:190
224442991512,2cyclictest272134-21kthreadcore07:55:180
224442991511,3cyclictest545698-21kworker/u8:3+flush-259:012:25:250
224442991511,3cyclictest468413-21kworker/u8:0+flush-259:011:10:190
224456991412,1cyclictest0-21swapper/312:40:003
224456991411,2cyclictest374932-21kworker/u8:2+events_unbound10:20:163
224456991411,2cyclictest343581-21kworker/u8:4+flush-259:009:10:213
224456991411,2cyclictest260524-21kworker/u8:5+events_unbound07:55:223
224456991411,2cyclictest260524-21kworker/u8:5+events_unbound07:55:223
224456991410,3cyclictest317833-21kworker/u8:3+flush-259:009:37:403
224456991410,3cyclictest260523-21kworker/u8:4+events_unbound08:00:223
224456991410,3cyclictest229828-21kworker/u8:0+events_unbound07:45:213
22445299149,4cyclictest3754036-21systemd-journal12:30:362
22445299148,4cyclictest441599-21latency_hist10:40:002
22445299148,4cyclictest307143-21/usr/sbin/munin08:30:152
22445299147,5cyclictest353681-21latency_hist09:15:012
224452991411,2cyclictest426808-21irqcore10:25:162
224447991412,1cyclictest535884-21irqcore12:10:201
224447991412,1cyclictest328155-21kworker/u8:1+events_unbound09:00:151
224447991412,1cyclictest0-21swapper/109:39:591
224447991411,2cyclictest540551-21kworker/u8:2+flush-259:012:20:271
224447991411,2cyclictest523980-21ntp_kernel_err11:55:201
224447991411,2cyclictest494248-21kworker/u8:2+flush-259:011:30:401
224447991411,2cyclictest420747-21kworker/u8:3+flush-259:010:25:201
224447991411,2cyclictest374932-21kworker/u8:2+flush-259:010:40:261
224447991411,2cyclictest317833-21kworker/u8:3+flush-259:008:40:211
224447991411,2cyclictest3002-21cron08:00:001
224447991411,2cyclictest3002-21cron08:00:001
224447991411,1cyclictest0-21swapper/109:30:221
224447991410,3cyclictest328155-21kworker/u8:1+events_unbound09:15:201
224447991410,3cyclictest307168-21apt-get08:30:121
224442991413,1cyclictest60650irq/133-enp0s29f1:rx-010:10:280
224442991413,1cyclictest60650irq/133-enp0s29f1:rx-010:10:280
224442991412,2cyclictest60650irq/133-enp0s29f1:rx-011:45:190
224442991412,1cyclictest60650irq/133-enp0s29f1:rx-011:30:240
224442991412,1cyclictest60650irq/133-enp0s29f1:rx-008:25:240
224442991412,1cyclictest60650irq/133-enp0s29f1:rx-008:15:260
224442991411,2cyclictest60650irq/133-enp0s29f1:rx-012:00:510
224442991411,2cyclictest60650irq/133-enp0s29f1:rx-011:02:540
224442991411,2cyclictest60650irq/133-enp0s29f1:rx-009:29:380
224442991411,2cyclictest60650irq/133-enp0s29f1:rx-008:33:130
224442991411,2cyclictest60650irq/133-enp0s29f1:rx-008:10:190
224442991411,2cyclictest60650irq/133-enp0s29f1:rx-007:25:230
224456991311,2cyclictest468413-21kworker/u8:0+events_unbound11:10:293
224456991311,1cyclictest389135-21needreboot09:45:203
224456991310,2cyclictest540551-21kworker/u8:2+flush-259:012:15:233
224456991310,2cyclictest498262-21ntp_kernel_err11:30:223
224456991310,2cyclictest494248-21kworker/u8:2+events_unbound11:50:213
224456991310,2cyclictest468413-21kworker/u8:0+flush-259:011:40:223
224456991310,2cyclictest260523-21kworker/u8:4+events_unbound08:05:173
224456991310,2cyclictest0-21swapper/309:54:593
22445699130,3cyclictest535206-21awk12:10:003
22445299139,3cyclictest468413-21kworker/u8:0+events_unbound11:55:012
224452991310,2cyclictest306484-21unixbench_singl08:25:252
224452991310,2cyclictest265675-21sh07:50:002
22444799139,3cyclictest390122-21latency_hist09:50:001
224447991311,1cyclictest0-21swapper/110:45:171
224447991310,2cyclictest514628-21kworker/u8:1+flush-259:012:29:591
224447991310,2cyclictest468413-21kworker/u8:0+flush-259:011:40:241
224447991310,2cyclictest395464-21kworker/u8:4+flush-259:009:55:201
224447991310,2cyclictest317833-21kworker/u8:3+flush-259:008:50:351
224447991310,2cyclictest229828-21kworker/u8:0+events_unbound07:16:361
224447991310,2cyclictest170653-21kworker/u8:1+events_unbound07:20:281
22444299138,4cyclictest317833-21kworker/u8:3+flush-259:009:40:100
224442991311,2cyclictest60650irq/133-enp0s29f1:rx-011:15:180
224442991311,1cyclictest60650irq/133-enp0s29f1:rx-009:35:150
224442991311,1cyclictest60650irq/133-enp0s29f1:rx-008:20:190
224442991311,1cyclictest60650irq/133-enp0s29f1:rx-008:08:260
224442991310,2cyclictest60650irq/133-enp0s29f1:rx-009:54:240
224442991310,2cyclictest462618-21unixbench_singl10:55:260
22445699129,2cyclictest483228-21unixbench_multi11:15:273
22445699129,2cyclictest453232-21fschecks_time10:50:143
22445699129,2cyclictest384994-21kworker/u8:0+flush-259:009:59:413
22445699129,2cyclictest374932-21kworker/u8:2+events_unbound11:05:143
22445699129,2cyclictest229828-21kworker/u8:0+events_unbound07:35:233
22445699128,3cyclictest224654-21/usr/sbin/munin07:10:153
224456991210,1cyclictest0-21swapper/308:55:023
22445299129,2cyclictest545794-21cpuspeed_turbos12:20:152
22445299129,2cyclictest0-21swapper/208:55:002
22445299128,3cyclictest452373-21kworker/u8:3+flush-259:012:00:242
22445299126,4cyclictest452320-21apt-get10:50:012
22445299121,9cyclictest390113-21awk09:49:592
224452991210,1cyclictest0-21swapper/210:50:012
22444799129,2cyclictest540551-21kworker/u8:2+flush-259:012:15:251
22444799129,2cyclictest431689-21cpuspeed_turbos10:30:201
22444799129,2cyclictest380300-21fschecks_count09:40:161
22444799129,2cyclictest374932-21kworker/u8:2+events_unbound10:20:211
22444799129,2cyclictest317833-21kworker/u8:3+flush-259:009:50:241
224447991210,1cyclictest353667-21idleruntime-cro09:15:001
22444299129,2cyclictest60650irq/133-enp0s29f1:rx-012:32:420
224442991210,2cyclictest60650irq/133-enp0s29f1:rx-011:42:130
224442991210,1cyclictest60650irq/133-enp0s29f1:rx-010:35:200
224442991210,1cyclictest60650irq/133-enp0s29f1:rx-009:15:240
22445699118,2cyclictest447209-21kworker/u8:1+events_unbound11:00:293
22445699118,2cyclictest364020-21kworker/u8:0+events_unbound09:34:363
22445699118,2cyclictest343581-21kworker/u8:4+flush-259:009:05:283
22445699118,2cyclictest317833-21kworker/u8:3+flush-259:009:25:343
22445699110,4cyclictest524914-21latency_hist12:00:013
22445299119,1cyclictest0-21swapper/212:15:162
22445299119,1cyclictest0-21swapper/211:25:222
22445299119,1cyclictest0-21swapper/210:15:212
22445299118,2cyclictest375042-21cpuspeed_turbos09:35:132
22445299118,2cyclictest255861-21fschecks_time07:40:152
22445299118,1cyclictest0-21swapper/209:30:232
22445299117,2cyclictest520012-21apt-get11:55:122
22445299115,4cyclictest228709-21ntpq07:10:212
22445299111,2cyclictest400412-21awk10:00:002
22444799118,2cyclictest266123-21fschecks_count07:50:141
22444799117,3cyclictest170653-21kworker/u8:1+flush-259:007:30:151
22444799111,9cyclictest3000063-21rs:main1
22444799110,3cyclictest296615-21latency_hist08:20:011
22445699108,1cyclictest0-21swapper/311:35:203
22445299108,1cyclictest0-21swapper/210:10:182
22445299108,1cyclictest0-21swapper/210:00:182
22445299108,1cyclictest0-21swapper/207:35:142
22445299107,2cyclictest458580-21irqcore10:55:172
22445299107,2cyclictest361825-21kthreadcore09:20:192
22445299107,2cyclictest0-21swapper/211:00:192
22445299107,2cyclictest0-21swapper/208:45:002
22445299107,1cyclictest0-21swapper/208:45:242
22445299106,3cyclictest447209-21kworker/u8:1+flush-259:011:20:162
22445299105,3cyclictest316508-21sed08:35:192
22445299105,3cyclictest244270-21tr07:25:232
22444799108,1cyclictest0-21swapper/107:45:181
22444799107,2cyclictest374932-21kworker/u8:2+events_unbound10:00:231
22444799103,3cyclictest479629-21sed11:15:181
2244529997,1cyclictest0-21swapper/208:55:152
2244529995,3cyclictest368539-21grep09:25:192
2244529981,6cyclictest482776-21ntpq11:15:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional