You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 17:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Thu Dec 04, 2025 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3182780995444,5cyclictest3287529-21kworker/u8:0+events_unbound12:30:143
3182773995144,5cyclictest3273414-21kworker/u8:0+events_unbound11:36:231
3182773995041,5cyclictest3279025-21awk11:45:001
3182776994840,5cyclictest3184401-21munin-plugin-st07:15:012
3182768994741,4cyclictest3282363-21kworker/u8:0+events_unbound11:56:240
3182776994536,6cyclictest3279601-21fschecks_count11:45:162
3182776994536,5cyclictest3185375-21fschecks_count07:15:172
3182768994436,5cyclictest3182760-21cyclictest09:56:100
3182768994423,6cyclictest131rcu_preempt10:15:140
3182768994423,6cyclictest131rcu_preempt10:15:140
3182780994336,4cyclictest3270280-21sendmail11:20:013
3182776994336,4cyclictest3235734-21munin-plugin-st09:40:012
3182776994336,4cyclictest3214200-21apt-get08:40:002
3182776994333,7cyclictest3292874-21cron12:25:002
3182773994231,9cyclictest231ktimers/107:50:121
3182773994135,4cyclictest3240884-21kworker/u8:2+events_unbound09:56:111
3182773994127,11cyclictest231ktimers/108:50:141
3182773994031,5cyclictest0-21swapper/110:35:011
3182768994031,6cyclictest3189706-21unixbench_singl07:25:260
3182768994021,14cyclictest13650irq/42-30be0000.ethernet11:10:130
3182776993929,6cyclictest0-21swapper/208:20:012
3182773993929,7cyclictest3189704-21grep07:25:261
3182773993928,8cyclictest3239201-21apt-key09:50:021
3182780993831,5cyclictest3223231-21kworker/u8:0+events_unbound09:31:083
3182780993831,5cyclictest3223231-21kworker/u8:0+events_unbound09:31:083
3182773993831,4cyclictest3207175-21munin-plugin-st08:20:001
3182773993829,6cyclictest0-21swapper/111:00:011
3182776993727,6cyclictest0-21swapper/210:40:002
3182773993730,4cyclictest3228488-21cron09:20:001
3182773993729,4cyclictest0-21swapper/109:50:251
3182773993728,5cyclictest0-21swapper/107:55:251
3182773993726,7cyclictest3200806-21grep08:00:161
3182773993720,5cyclictest3287529-21kworker/u8:0+events_unbound12:19:531
3182773993719,12cyclictest3285777-21kworker/u8:1+events_unbound12:05:591
3182768993727,6cyclictest0-21swapper/009:15:000
3182768993716,5cyclictest131rcu_preempt12:35:130
3182780993626,6cyclictest0-21swapper/311:40:003
3182776993629,4cyclictest0-21swapper/211:34:592
3182776993627,6cyclictest3238752-21unixbench_singl09:45:272
3182776993627,5cyclictest3225502-21fschecks_time09:10:182
3182776993625,7cyclictest0-21swapper/209:15:272
3182773993623,9cyclictest231ktimers/109:05:131
3182780993527,4cyclictest0-21swapper/307:10:243
3182780993523,7cyclictest0-21swapper/308:35:003
3182776993525,6cyclictest0-21swapper/207:40:272
3182776993525,6cyclictest0-21swapper/207:40:272
3182776993524,5cyclictest3212220-21kworker/u8:3+events_unbound08:54:322
3182768993513,13cyclictest3193335-21kworker/u8:1-events_freezable_power_08:09:300
318278099348,3cyclictest131rcu_preempt12:20:113
3182780993424,6cyclictest0-21swapper/311:20:283
3182776993426,4cyclictest0-21swapper/212:00:172
3182776993426,4cyclictest0-21swapper/210:20:182
3182776993424,7cyclictest0-21swapper/208:00:022
3182776993424,6cyclictest0-21swapper/208:25:242
3182773993427,5cyclictest3223231-21kworker/u8:0+events_unbound09:26:081
3182773993419,6cyclictest3240884-21kworker/u8:2+events_unbound10:39:171
3182773993419,6cyclictest3237207-21kworker/u8:0+events_unbound09:44:091
3182773993418,10cyclictest3265387-21kworker/u8:0+events_unbound11:09:321
3182773993414,5cyclictest131rcu_preempt10:00:101
3182768993427,5cyclictest3193335-21kworker/u8:1+events_unbound08:00:420
3182768993426,5cyclictest3287298-21grep12:05:250
3182768993424,6cyclictest3204992-21grep08:10:270
3182768993423,7cyclictest0-21swapper/012:15:150
3182768993423,5cyclictest0-21swapper/007:55:380
318276899341,9cyclictest11-21ksoftirqd/012:25:140
3182768993417,10cyclictest131rcu_preempt11:40:130
3182780993326,4cyclictest0-21swapper/310:50:153
3182780993325,4cyclictest0-21swapper/308:35:253
3182776993327,3cyclictest0-21swapper/210:30:002
3182776993326,4cyclictest0-21swapper/207:20:272
3182776993325,5cyclictest0-21swapper/211:00:012
3182776993324,5cyclictest0-21swapper/209:35:002
3182776993324,5cyclictest0-21swapper/209:35:002
3182776993322,7cyclictest0-21swapper/210:10:252
3182776993320,5cyclictest3223231-21kworker/u8:0+events_unbound09:03:312
3182776993319,6cyclictest3271654-21kworker/u8:2+events_unbound11:59:332
3182776993318,7cyclictest3263426-21kworker/u8:2+events_unbound11:10:202
3182773993327,3cyclictest0-21swapper/111:15:001
3182773993326,3cyclictest0-21swapper/110:40:151
3182773993324,6cyclictest3265173-21cut11:05:011
3182773993320,5cyclictest3219460-21kworker/u8:1+events_unbound08:55:181
3182773993320,5cyclictest3193335-21kworker/u8:1+events_unbound07:48:541
318276899335,11cyclictest11-21ksoftirqd/010:20:240
3182780993224,4cyclictest3228517-21sendmail-msp09:20:003
3182780993224,4cyclictest0-21swapper/312:00:263
3182780993222,6cyclictest3268146-21grep11:10:263
3182776993223,6cyclictest0-21swapper/210:50:262
3182776993223,6cyclictest0-21swapper/209:10:002
3182776993222,7cyclictest3202842-21sshd08:05:222
3182776993222,6cyclictest0-21swapper/209:55:002
3182776993220,6cyclictest3258245-21kworker/u8:1+events_unbound10:45:232
3182773993225,4cyclictest3270268-21kthreadd11:20:001
3182773993220,6cyclictest3271654-21kworker/u8:2+events_unbound11:24:361
3182773993217,6cyclictest3182400-21kworker/u8:1+events_unbound07:30:231
3182773993217,11cyclictest231ktimers/107:20:141
318276899328,4cyclictest131rcu_preempt08:55:000
318276899324,10cyclictest11-21ksoftirqd/010:10:120
3182768993224,4cyclictest0-21swapper/010:55:000
318276899321,7cyclictest11-21ksoftirqd/010:35:150
3182768993211,7cyclictest131rcu_preempt11:35:120
3182780993125,3cyclictest0-21swapper/308:45:013
3182776993113,10cyclictest3195105-21kworker/u8:2+events_unbound07:48:302
318277399319,15cyclictest0-21swapper/112:35:161
3182773993124,3cyclictest0-21swapper/112:20:161
3182773993123,5cyclictest0-21swapper/109:30:261
3182773993123,5cyclictest0-21swapper/109:30:261
3182773993119,5cyclictest3256497-21kworker/u8:0+events_unbound10:51:191
318276899318,12cyclictest131rcu_preempt07:10:240
3182768993123,4cyclictest0-21swapper/012:10:270
3182768993123,4cyclictest0-21swapper/012:10:270
3182768993123,4cyclictest0-21swapper/011:05:270
3182768993123,4cyclictest0-21swapper/009:00:260
3182768993119,9cyclictest3216127-21cron08:45:010
3182768993115,3cyclictest131rcu_preempt07:15:040
3182780993023,3cyclictest0-21swapper/307:40:003
3182780993022,5cyclictest3216986-21runrttasks08:45:193
3182780993022,4cyclictest0-21swapper/309:55:013
3182780993022,4cyclictest0-21swapper/309:10:273
3182780993022,4cyclictest0-21swapper/307:40:283
3182780993022,4cyclictest0-21swapper/307:40:273
3182780993021,5cyclictest3225143-21cat09:10:023
3182776993020,6cyclictest0-21swapper/210:00:002
3182776993018,4cyclictest3263426-21kworker/u8:2+events_unbound11:09:162
3182776993016,6cyclictest3196823-21kworker/u8:0+events_unbound07:51:172
318277399306,12cyclictest0-21swapper/108:10:241
3182773993021,6cyclictest3203758-21cron08:10:011
3182773993019,5cyclictest3182400-21kworker/u8:1+events_unbound07:15:091
3182773993018,9cyclictest231ktimers/107:10:131
3182773993010,12cyclictest24-21ksoftirqd/109:00:141
3182768993024,3cyclictest13650irq/42-30be0000.ethernet11:00:000
3182768993023,4cyclictest3214222-21sendmail08:40:010
3182768993020,6cyclictest0-21swapper/007:30:250
3182768993010,11cyclictest0-21swapper/009:08:090
3182780992920,6cyclictest3242754-21cron10:00:013
3182780992920,5cyclictest0-21swapper/311:05:253
3182780992920,4cyclictest0-21swapper/311:30:003
3182780992917,9cyclictest215-21dbus-daemon10:06:103
3182780992914,3cyclictest131rcu_preempt12:15:133
318277699298,11cyclictest3193335-21kworker/u8:1+events_unbound08:00:182
3182776992922,5cyclictest3287529-21kworker/u8:0+events_unbound12:30:162
3182776992922,5cyclictest3186358-21kworker/u8:2+events_unbound07:30:342
3182776992921,4cyclictest0-21swapper/208:45:002
3182776992919,6cyclictest0-21swapper/212:35:252
318277399299,13cyclictest221rcuc/109:20:261
318277399294,18cyclictest0-21swapper/108:34:591
3182773992922,4cyclictest0-21swapper/110:45:251
3182773992922,4cyclictest0-21swapper/110:15:011
3182773992922,4cyclictest0-21swapper/107:35:251
3182773992919,6cyclictest0-21swapper/111:30:011
3182773992917,6cyclictest3285777-21kworker/u8:1+events_unbound12:35:011
3182773992917,4cyclictest131rcu_preempt10:29:001
318276899298,11cyclictest11-21ksoftirqd/010:05:260
318276899297,11cyclictest131rcu_preempt11:55:000
318276899296,11cyclictest0-21swapper/011:49:390
3182768992921,4cyclictest0-21swapper/009:41:080
3182768992921,4cyclictest0-21swapper/008:30:260
3182768992920,4cyclictest0-21swapper/008:20:000
3182780992820,5cyclictest0-21swapper/309:00:263
3182776992822,4cyclictest3182760-21cyclictest08:57:412
3182776992820,4cyclictest0-21swapper/208:45:272
3182776992818,6cyclictest0-21swapper/212:10:002
3182776992814,7cyclictest3202010-21kworker/u8:2+events_unbound08:22:562
318277399287,11cyclictest3287986-21apt-get12:10:131
318277399287,11cyclictest3287986-21apt-get12:10:131
318277399287,11cyclictest3244718-21apt-get10:05:141
318277399284,17cyclictest0-21swapper/108:40:191
3182773992819,6cyclictest0-21swapper/111:50:241
318276899283,3cyclictest131rcu_preempt08:24:280
3182768992824,2cyclictest13650irq/42-30be0000.ethernet08:45:060
3182768992822,4cyclictest3285777-21kworker/u8:1+events_unbound12:21:050
3182768992822,3cyclictest0-21swapper/009:00:000
3182768992822,3cyclictest0-21swapper/008:25:240
3182768992821,5cyclictest3267541-21kworker/u8:1+events_unbound11:21:220
3182768992814,3cyclictest131rcu_preempt09:15:240
3182768992811,4cyclictest131rcu_preempt11:01:250
3182768992811,3cyclictest131rcu_preempt10:40:220
3182768992810,10cyclictest0-21swapper/010:04:170
3182780992719,5cyclictest0-21swapper/312:35:273
3182780992719,5cyclictest0-21swapper/311:55:013
3182780992714,9cyclictest1-21systemd10:56:163
318277699275,14cyclictest32-21ksoftirqd/212:14:392
318277699275,14cyclictest32-21ksoftirqd/212:14:392
3182776992720,5cyclictest3235445-21kworker/u8:2+events_unbound09:45:012
3182776992717,6cyclictest0-21swapper/211:50:252
3182776992716,8cyclictest3248451-21awk10:15:182
3182776992716,8cyclictest3248451-21awk10:15:182
3182776992710,9cyclictest3271654-21kworker/u8:2+events_unbound11:35:232
318277399277,10cyclictest24-21ksoftirqd/112:00:161
318277399277,10cyclictest131rcu_preempt12:25:281
318277399274,12cyclictest3240738-21kworker/1:2+mm_percpu_wq11:30:311
318277399271,17cyclictest3226385-21/usr/sbin/munin09:10:261
318277399271,14cyclictest0-21swapper/108:35:141
318277399271,12cyclictest0-21swapper/108:20:181
318276899274,9cyclictest11-21ksoftirqd/011:25:120
3182768992720,4cyclictest0-21swapper/012:30:280
3182768992719,5cyclictest3271654-21kworker/u8:2+events_unbound11:32:020
3182768992718,6cyclictest3253143-21date10:30:000
3182768992718,6cyclictest0-21swapper/007:20:280
3182768992712,7cyclictest11-21ksoftirqd/009:30:000
3182780992620,3cyclictest0-21swapper/312:05:143
3182780992619,3cyclictest3232119-21kthreadd09:30:003
3182780992619,3cyclictest0-21swapper/310:25:203
3182780992615,8cyclictest1-21systemd09:36:073
318277699269,10cyclictest131rcu_preempt10:05:012
318277699265,13cyclictest32-21ksoftirqd/209:25:182
318277699262,3cyclictest131rcu_preempt08:30:132
3182776992620,3cyclictest0-21swapper/210:35:002
3182776992619,4cyclictest0-21swapper/211:20:292
3182776992618,5cyclictest32573152sleep210:40:182
3182776992615,8cyclictest3204415-21sed08:10:182
318277699261,15cyclictest0-21swapper/211:25:132
3182773992611,6cyclictest24-21ksoftirqd/111:45:201
318277399261,14cyclictest0-21swapper/107:40:241
318277399261,14cyclictest0-21swapper/107:40:231
318276899266,10cyclictest11-21ksoftirqd/009:20:140
318276899265,3cyclictest131rcu_preempt07:40:000
318276899265,11cyclictest11-21ksoftirqd/011:20:000
318276899264,10cyclictest0-21swapper/010:34:150
318276899263,7cyclictest11-21ksoftirqd/007:40:150
318276899263,7cyclictest11-21ksoftirqd/007:40:150
318276899263,13cyclictest11-21ksoftirqd/007:50:200
318276899262,12cyclictest0-21swapper/010:48:150
3182768992615,8cyclictest215-21dbus-daemon09:46:080
318276899261,12cyclictest0-21swapper/007:48:440
318276899260,7cyclictest11-21ksoftirqd/009:35:150
3182780992518,3cyclictest0-21swapper/307:25:273
3182780992517,4cyclictest0-21swapper/311:55:153
318277699252,3cyclictest131rcu_preempt07:35:112
3182776992518,5cyclictest3227349-21kworker/u8:2+events_unbound09:21:082
3182776992518,3cyclictest0-21swapper/211:45:012
318277699251,16cyclictest0-21swapper/212:25:142
318277399256,12cyclictest24-21ksoftirqd/110:22:321
318277399254,13cyclictest0-21swapper/108:25:111
318277399253,13cyclictest0-21swapper/109:35:041
3182773992510,8cyclictest131rcu_preempt08:45:151
3182768992519,4cyclictest3238951-21kworker/u8:1+events_unbound09:51:580
3182780992418,4cyclictest3197639-21missed_timers07:50:203
3182780992418,3cyclictest0-21swapper/309:50:003
3182780992418,3cyclictest0-21swapper/309:20:253
3182780992415,6cyclictest0-21swapper/312:30:003
3182780992415,5cyclictest0-21swapper/308:50:283
3182776992418,4cyclictest3263426-21kworker/u8:2+events_unbound11:00:052
3182776992418,3cyclictest0-21swapper/207:25:272
3182776992416,4cyclictest0-21swapper/211:15:192
3182776992410,3cyclictest131rcu_preempt10:05:222
3182773992418,4cyclictest3282363-21kworker/u8:0+events_unbound11:56:561
318276899242,4cyclictest3286041-21cron12:05:000
3182780992316,3cyclictest0-21swapper/310:30:143
3182780992315,5cyclictest0-21swapper/311:40:263
3182780992314,4cyclictest0-21swapper/307:55:243
3182780992313,7cyclictest3220774-21sed08:55:273
3182780992311,8cyclictest1-21systemd08:10:503
318277699238,9cyclictest131rcu_preempt12:15:172
3182780992216,3cyclictest0-21swapper/311:00:183
3182780992215,4cyclictest3210107-21tail08:25:243
3182780992213,6cyclictest3254965-21latency_hist10:35:023
3182780992211,8cyclictest3249599-21run-parts10:20:013
3182780992211,8cyclictest3249599-21run-parts10:20:013
3182780992211,8cyclictest3237467-21basename09:45:003
3182780992116,3cyclictest3207151-21kworker/u8:0-writeback08:20:023
3182780992115,4cyclictest3195473-21/usr/sbin/munin07:45:233
3182780992115,3cyclictest0-21swapper/312:10:183
3182780992115,3cyclictest0-21swapper/312:10:183
3182780992115,3cyclictest0-21swapper/310:10:233
3182780992115,3cyclictest0-21swapper/308:05:223
3182780992113,6cyclictest3191322-21sed07:30:253
3182780992110,8cyclictest3250288-21memory10:20:213
3182780992014,3cyclictest637-21runrttasks10:00:193
3182780992014,3cyclictest0-21swapper/311:30:263
3182780992012,4cyclictest0-21swapper/311:45:283
3182780992011,6cyclictest0-21swapper/310:40:243
318278099199,7cyclictest3184892-21wc07:15:033
3182780991910,6cyclictest3259066-21cat10:45:203
3182780991810,5cyclictest0-21swapper/308:00:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional