You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 23:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Sun Nov 23, 2025 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3084009995224,4cyclictest0-21swapper/207:15:262
3084000995144,5cyclictest3193097-21kworker/u8:0+events_unbound12:35:200
3084013995043,5cyclictest3083804-21kworker/u8:2+events_unbound07:19:283
3084013994936,10cyclictest391ktimers/308:55:123
3084009994837,7cyclictest3122256-21grep08:55:302
3084009994534,6cyclictest0-21swapper/208:39:362
3084004994538,4cyclictest3182810-21idleruntime-cro11:55:001
3084009994328,11cyclictest311ktimers/211:05:132
3084004994335,4cyclictest0-21swapper/107:15:001
3084004994335,4cyclictest0-21swapper/107:15:001
3084004994332,8cyclictest231ktimers/110:30:121
3084000994337,4cyclictest3146510-21kworker/u8:2+events_unbound10:40:050
3084000994231,8cyclictest121ktimers/007:55:160
3084000994231,8cyclictest121ktimers/007:55:150
3084013994034,4cyclictest3152095-21kworker/u8:3+events_unbound10:25:023
3084013994028,7cyclictest0-21swapper/308:45:013
3084004994033,5cyclictest3101424-21kworker/u8:1+events_unbound08:19:341
3084013993931,4cyclictest0-21swapper/307:20:153
3084013993916,11cyclictest0-21swapper/312:18:463
3084000993927,9cyclictest3191750-21cron12:20:010
3084013993831,4cyclictest3164036-21munin-run11:00:003
3084013993819,5cyclictest3134293-21kworker/u8:1-events_unbound09:36:583
3084009993828,6cyclictest0-21swapper/208:00:002
3084009993828,6cyclictest0-21swapper/208:00:002
3084009993827,7cyclictest0-21swapper/208:45:272
3084000993829,6cyclictest3153821-21munin-plugin-st10:30:000
3084013993731,4cyclictest3186311-21kworker/u8:1+events_unbound12:20:183
3084013993731,4cyclictest3144803-21kworker/u8:0+events_unbound10:04:573
3084013993731,4cyclictest3101424-21kworker/u8:1+events_unbound08:14:333
3084013993730,5cyclictest3120614-21kworker/u8:2+events_unbound08:54:403
3084013993730,5cyclictest3118738-21kworker/u8:1+events_unbound08:49:383
3084013993729,5cyclictest3177700-21sendmail11:40:003
3084013993725,7cyclictest3096057-21grep07:40:273
3084009993727,6cyclictest3177682-21sh11:40:012
3084004993730,4cyclictest3157210-21auditctl10:40:001
3084004993729,4cyclictest0-21swapper/109:25:281
3084004993729,4cyclictest0-21swapper/109:25:281
308400499372,4cyclictest131rcu_preempt11:30:131
3084000993728,4cyclictest0-21swapper/011:55:010
3084013993630,4cyclictest3160851-21kworker/u8:2+events_unbound11:20:103
3084013993630,4cyclictest3125983-21kworker/u8:1+events_unbound09:14:413
3084013993622,5cyclictest131rcu_preempt11:55:153
3084013993620,3cyclictest131rcu_preempt12:00:343
3084004993630,4cyclictest3125983-21kworker/u8:1+events_unbound09:19:431
308400099365,5cyclictest131rcu_preempt07:15:140
3084000993630,4cyclictest3193097-21kworker/u8:0+events_unbound12:30:180
3084000993630,4cyclictest3150163-21kworker/u8:1+events_unbound10:20:000
308401399357,20cyclictest40-21ksoftirqd/309:45:013
3084013993529,4cyclictest3089270-21kworker/u8:0+events_unbound07:34:303
3084013993527,4cyclictest0-21swapper/312:40:003
3084013993527,4cyclictest0-21swapper/309:55:253
3084013993525,6cyclictest0-21swapper/309:15:263
3084013993524,8cyclictest3182798-21idleruntime-cro11:55:003
3084013993524,7cyclictest3167071-21grep11:05:403
3084013993520,4cyclictest131rcu_preempt07:27:093
3084004993527,6cyclictest3177481-21kworker/u8:2+events_unbound11:40:141
3084004993527,4cyclictest3191748-21kthreadd12:20:001
3084004993525,7cyclictest3160631-21auditctl10:50:001
3084000993527,4cyclictest0-21swapper/010:25:000
3084013993428,4cyclictest3105097-21kworker/u8:2+events_unbound08:09:333
3084013993428,4cyclictest3089270-21kworker/u8:0+events_unbound07:53:433
3084013993427,5cyclictest3083804-21kworker/u8:2+events_unbound07:13:433
3084013993427,5cyclictest3083804-21kworker/u8:2+events_unbound07:13:433
3084013993426,6cyclictest3188162-21kworker/u8:0+events_unbound12:13:423
3084013993426,4cyclictest0-21swapper/312:30:183
3084013993425,5cyclictest0-21swapper/309:54:533
3084013993420,4cyclictest131rcu_preempt09:22:553
3084009993427,4cyclictest0-21swapper/212:05:162
3084009993426,4cyclictest0-21swapper/210:40:002
3084009993426,4cyclictest0-21swapper/209:45:292
3084009993426,4cyclictest0-21swapper/207:40:282
3084009993424,7cyclictest3153355-21unixbench_multi10:25:262
3084009993423,7cyclictest3170428-21grep11:15:262
3084004993428,4cyclictest3101424-21kworker/u8:1+events_unbound08:04:331
3084004993427,4cyclictest3126109-21idleruntime-cro09:10:001
3084004993427,4cyclictest3105212-21latency_hist08:10:001
3084004993426,4cyclictest0-21swapper/111:55:251
3084004993413,4cyclictest131rcu_preempt10:40:121
3084000993424,6cyclictest0-21swapper/011:35:000
3084013993326,5cyclictest3174213-21kworker/u8:3+events_unbound11:43:423
3084013993324,7cyclictest391ktimers/308:30:133
3084013993319,7cyclictest3146510-21kworker/u8:2+events_unbound10:16:533
3084013993315,11cyclictest131rcu_preempt09:45:133
3084013993313,3cyclictest131rcu_preempt12:28:043
3084013993313,3cyclictest131rcu_preempt12:28:043
3084009993326,4cyclictest3182799-21cron11:55:002
3084009993323,7cyclictest0-21swapper/209:55:252
3084004993325,4cyclictest0-21swapper/109:10:151
3084004993325,4cyclictest0-21swapper/108:10:271
3084004993324,6cyclictest3162572-21kworker/u8:0+events_unbound10:55:271
3084004993322,7cyclictest3137805-21grep09:40:251
308400499331,5cyclictest131rcu_preempt10:00:111
3084000993323,6cyclictest0-21swapper/011:50:000
3084000993315,11cyclictest131rcu_preempt12:20:320
3084013993223,5cyclictest0-21swapper/308:25:263
3084013993223,5cyclictest0-21swapper/308:25:253
3084013993219,5cyclictest131rcu_preempt08:18:063
3084013993219,4cyclictest131rcu_preempt11:10:213
3084013993213,4cyclictest131rcu_preempt09:05:123
3084013993213,3cyclictest131rcu_preempt08:22:383
3084013993212,12cyclictest131rcu_preempt11:35:013
3084013993211,11cyclictest40-21ksoftirqd/310:20:173
3084009993224,4cyclictest0-21swapper/210:40:162
3084004993225,5cyclictest3167374-21kworker/u8:0+events_unbound11:13:411
3084004993224,6cyclictest3136306-21kworker/u8:3+events_unbound09:49:491
3084004993224,4cyclictest0-21swapper/108:45:271
3084004993221,7cyclictest0-21swapper/110:15:001
308400099322,6cyclictest131rcu_preempt09:45:140
3084000993224,3cyclictest0-21swapper/009:55:250
3084013993123,4cyclictest0-21swapper/311:45:253
3084013993117,4cyclictest131rcu_preempt10:05:213
3084013993116,5cyclictest131rcu_preempt10:40:143
3084013993116,4cyclictest131rcu_preempt07:47:293
3084013993115,7cyclictest3134293-21kworker/u8:1+events_unbound09:32:013
3084013993115,5cyclictest131rcu_preempt07:35:163
3084013993112,11cyclictest131rcu_preempt10:10:263
3084013993111,12cyclictest131rcu_preempt10:50:193
3084013993111,12cyclictest131rcu_preempt10:50:193
3084009993123,4cyclictest0-21swapper/207:30:312
3084009993120,7cyclictest0-21swapper/211:00:282
3084004993121,6cyclictest0-21swapper/111:05:401
3084000993124,3cyclictest0-21swapper/008:30:250
308400099311,5cyclictest131rcu_preempt10:05:120
308401399309,14cyclictest131rcu_preempt11:00:153
308401399307,8cyclictest40-21ksoftirqd/308:40:003
308401399307,11cyclictest40-21ksoftirqd/312:10:013
308401399306,12cyclictest40-21ksoftirqd/310:45:043
3084013993024,4cyclictest3124213-21kworker/u8:0+events_unbound09:04:413
3084013993018,5cyclictest131rcu_preempt10:35:003
3084013993014,4cyclictest131rcu_preempt08:00:013
3084013993014,4cyclictest131rcu_preempt08:00:003
3084013993010,11cyclictest40-21ksoftirqd/311:15:213
3084009993019,8cyclictest1-21systemd12:10:152
3084004993023,5cyclictest3196600-21kworker/u8:2+events_unbound12:36:001
3084000993022,4cyclictest0-21swapper/009:40:260
3084000993021,5cyclictest0-21swapper/008:05:000
3084000993020,6cyclictest3123980-21grep09:00:250
3084000993012,16cyclictest0-21swapper/010:30:200
308401399293,9cyclictest40-21ksoftirqd/309:25:283
308401399293,9cyclictest40-21ksoftirqd/309:25:283
3084013992916,5cyclictest131rcu_preempt10:35:203
3084013992916,3cyclictest131rcu_preempt08:01:563
3084009992918,8cyclictest311ktimers/209:15:122
3084009992916,9cyclictest1-21systemd07:29:282
308400499297,8cyclictest3192414-21wc12:20:181
308400499294,5cyclictest131rcu_preempt12:05:131
3084004992911,11cyclictest24-21ksoftirqd/111:35:171
3084000992923,4cyclictest3112380-21kworker/u8:1+events_unbound08:35:020
3084000992922,4cyclictest13650irq/42-30be0000.ethernet11:35:270
3084000992921,4cyclictest0-21swapper/008:20:280
3084000992920,5cyclictest13650irq/42-30be0000.ethernet12:05:250
3084000992918,8cyclictest3172020-21sed11:20:240
3084000992918,7cyclictest3168712-21grep11:10:250
3084000992913,13cyclictest11-21ksoftirqd/008:10:130
308400099290,4cyclictest131rcu_preempt08:05:120
3084013992819,5cyclictest3173806-21unixbench_multi11:25:273
3084009992821,4cyclictest3198576-21munin-plugin-st12:40:002
3084009992821,4cyclictest0-21swapper/212:30:272
3084009992821,4cyclictest0-21swapper/212:00:262
308400499282,6cyclictest131rcu_preempt08:40:141
3084004992822,4cyclictest3098325-21apt-key07:50:011
3084004992821,4cyclictest0-21swapper/108:30:251
3084004992821,3cyclictest0-21swapper/111:00:171
3084004992811,10cyclictest24-21ksoftirqd/112:10:191
3084000992821,3cyclictest0-21swapper/010:50:260
3084000992821,3cyclictest0-21swapper/010:50:260
3084000992820,5cyclictest0-21swapper/007:40:180
3084000992820,4cyclictest0-21swapper/009:35:000
3084009992720,3cyclictest0-21swapper/211:40:162
3084009992719,5cyclictest0-21swapper/209:40:252
3084009992719,4cyclictest0-21swapper/207:20:242
3084009992718,5cyclictest0-21swapper/210:14:572
3084009992717,7cyclictest3117119-21runrttasks08:40:252
3084004992721,4cyclictest3171568-21kworker/u8:1+events_unbound11:24:341
3084004992721,4cyclictest3143389-21idleruntime-cro10:00:011
3084004992721,3cyclictest0-21swapper/108:20:171
3084004992716,8cyclictest3194148-21sshd12:25:171
3084004992716,8cyclictest3194148-21sshd12:25:171
308400099279,5cyclictest3100133-21date07:55:000
308400099272,3cyclictest131rcu_preempt12:25:100
308400099272,3cyclictest131rcu_preempt12:25:100
3084000992719,4cyclictest0-21swapper/010:35:030
3084000992718,6cyclictest3172996-21tune2fs11:25:160
3084000992717,6cyclictest0-21swapper/007:35:240
3084000992715,4cyclictest131rcu_preempt09:27:180
3084000992715,4cyclictest131rcu_preempt09:27:180
3084000992711,5cyclictest3160631-21auditctl10:50:000
3084009992619,4cyclictest0-21swapper/208:30:172
3084009992619,4cyclictest0-21swapper/207:10:282
3084009992619,4cyclictest0-21swapper/207:10:272
3084009992617,5cyclictest0-21swapper/210:00:242
3084009992616,7cyclictest0-21swapper/209:05:022
3084009992613,9cyclictest215-21dbus-daemon11:45:112
308400499268,4cyclictest3120614-21kworker/u8:2+events_unbound08:52:171
3084004992619,5cyclictest3127716-21kworker/u8:2+events_unbound09:21:271
3084004992617,5cyclictest3146510-21kworker/u8:2+events_unbound10:10:001
3084004992616,6cyclictest3153395-21grep10:25:271
3084004992614,3cyclictest24-21ksoftirqd/107:20:201
308400099269,7cyclictest131rcu_preempt07:10:230
308400099269,7cyclictest131rcu_preempt07:10:230
308400099267,16cyclictest151rcuc/010:55:280
308400099266,9cyclictest11-21ksoftirqd/008:20:010
3084000992622,2cyclictest13650irq/42-30be0000.ethernet09:05:130
3084000992620,4cyclictest3188162-21kworker/u8:0+events_unbound12:10:150
3084000992618,4cyclictest13650irq/42-30be0000.ethernet09:10:260
3084000992617,6cyclictest3184316-21head11:55:250
3084000992613,9cyclictest215-21dbus-daemon11:40:110
3084009992519,4cyclictest3138044-21kworker/u8:0+events_unbound09:54:552
3084009992517,5cyclictest0-21swapper/208:10:262
308400499258,6cyclictest24-21ksoftirqd/108:25:121
308400499258,6cyclictest24-21ksoftirqd/108:25:121
3084004992518,3cyclictest0-21swapper/112:30:191
3084004992517,4cyclictest0-21swapper/110:55:011
3084004992517,4cyclictest0-21swapper/110:55:001
3084004992515,6cyclictest0-21swapper/110:20:001
3084004992514,7cyclictest3115880-21tr08:40:001
3084004992512,7cyclictest24-21ksoftirqd/107:50:181
308400099259,2cyclictest0-21swapper/009:35:210
308400099256,7cyclictest131rcu_preempt09:50:200
308400099253,4cyclictest131rcu_preempt07:30:310
308400099251,20cyclictest0-21swapper/008:55:430
308400099251,13cyclictest0-21swapper/010:00:200
3084009992417,5cyclictest3160851-21kworker/u8:2+events_unbound11:31:122
3084009992416,4cyclictest0-21swapper/210:10:002
3084009992415,5cyclictest0-21swapper/211:15:002
3084009992413,6cyclictest3120566-21grep08:50:242
308400499248,9cyclictest24-21ksoftirqd/107:26:491
308400499245,6cyclictest131rcu_preempt11:15:141
3084004992416,5cyclictest0-21swapper/107:55:271
3084004992416,5cyclictest0-21swapper/107:55:271
3084004992416,4cyclictest3134626-21auditctl09:35:011
3084000992417,5cyclictest3120614-21kworker/u8:2+events_unbound08:51:320
3084000992417,5cyclictest3096142-21kworker/u8:1+events_unbound07:45:160
3084000992417,4cyclictest0-21swapper/011:00:170
3084000992417,3cyclictest0-21swapper/010:13:570
3084000992416,4cyclictest0-21swapper/009:20:240
308400099241,21cyclictest0-21swapper/012:00:020
3084009992317,3cyclictest0-21swapper/211:25:182
3084009992316,4cyclictest440-21cron07:40:012
3084009992316,4cyclictest3164038-21munin-plugin-st11:00:002
3084009992316,3cyclictest0-21swapper/207:50:122
3084009992315,4cyclictest0-21swapper/209:40:002
3084009992312,7cyclictest0-21swapper/211:20:252
3084004992315,5cyclictest231ktimers/109:00:111
3084004992314,6cyclictest0-21swapper/111:45:151
3084004992312,8cyclictest3094752-21cron07:40:011
308400099238,8cyclictest121ktimers/011:15:120
308400099231,19cyclictest3088713-21modprobe07:20:190
308400099231,18cyclictest0-21swapper/008:28:240
308400099231,18cyclictest0-21swapper/008:28:240
308400099231,12cyclictest0-21swapper/007:25:120
308400099230,3cyclictest3129512-21munin-plugin-st09:20:010
308400999222,6cyclictest3108362-21unixbench_multi08:15:262
3084009992216,4cyclictest311ktimers/209:20:122
3084009992216,3cyclictest0-21swapper/209:10:232
3084009992216,3cyclictest0-21swapper/208:10:012
3084009992215,4cyclictest3160312-21sshd10:46:122
3084009992215,4cyclictest3133322-21fschecks_count09:30:162
3084009992215,4cyclictest311ktimers/210:18:182
3084009992214,5cyclictest0-21swapper/212:25:232
3084009992214,5cyclictest0-21swapper/212:25:222
3084009992214,5cyclictest0-21swapper/208:20:272
3084009992213,6cyclictest3098311-21sed07:50:012
3084009992213,6cyclictest0-21swapper/211:55:242
3084009992213,5cyclictest3192122-21sshd12:20:162
3084004992216,4cyclictest3089270-21kworker/u8:0+events_unbound07:42:121
3084004992216,3cyclictest0-21swapper/107:15:181
3084004992215,4cyclictest0-21swapper/110:20:271
3084004992212,7cyclictest3185414-21grep12:00:161
308400099225,7cyclictest131rcu_preempt08:40:260
3084000992216,4cyclictest121ktimers/008:47:370
3084000992215,3cyclictest0-21swapper/011:06:250
3084009992114,5cyclictest0-21swapper/210:55:002
3084009992114,5cyclictest0-21swapper/210:55:002
3084009992114,4cyclictest311ktimers/209:06:302
3084009992114,4cyclictest0-21swapper/208:00:252
3084009992110,7cyclictest215-21dbus-daemon10:30:022
3084004992115,3cyclictest0-21swapper/111:25:181
308400499211,4cyclictest0-21swapper/109:00:011
3084009992011,6cyclictest3132191-21/usr/sbin/munin09:25:262
3084009992011,6cyclictest3132191-21/usr/sbin/munin09:25:262
3084009992010,7cyclictest3190511-21cut12:15:172
308400499207,4cyclictest3082066-21kworker/u8:1+events_unbound07:32:441
3084004992013,5cyclictest3138044-21kworker/u8:0+events_unbound09:54:581
3084009991912,4cyclictest0-21swapper/210:20:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional