You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 04:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot5s.osadl.org (updated Sun Apr 26, 2026 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3188633995448,4cyclictest3239109-21kworker/u8:0+events_unbound22:02:391
3188641995245,4cyclictest3259953-21munin-plugin-st22:35:003
3188628995128,20cyclictest3233953-21idleruntime-cro21:20:010
3188628995128,20cyclictest3233953-21idleruntime-cro21:20:000
318862899501,31cyclictest3220815-21/usr/sbin/munin20:40:160
3188637994935,10cyclictest311ktimers/223:20:132
3188633994833,10cyclictest231ktimers/119:45:121
3188628994639,4cyclictest3265484-21idleruntime-cro22:50:010
3188628994639,4cyclictest3265484-21idleruntime-cro22:50:010
318862899461,40cyclictest3224214-21/usr/sbin/munin20:50:180
318862899451,39cyclictest3215407-21apt-get20:25:120
3188641994433,8cyclictest391ktimers/320:35:113
3188633994438,4cyclictest3251177-21kworker/u8:2+events_unbound22:17:441
3188633994438,4cyclictest3192548-21kworker/u8:2+events_unbound19:27:141
318862899441,38cyclictest3222229-21apt-get20:45:120
318862899441,37cyclictest3298757-21apt-get00:25:100
3188628994332,8cyclictest3275456-21grep23:15:260
318862899433,27cyclictest3214041-21irqstats20:20:180
318862899431,39cyclictest3253652-21/usr/sbin/munin22:15:160
318862899431,39cyclictest3248828-21/usr/sbin/munin22:00:250
318862899431,38cyclictest31928002chrt19:20:040
318862899430,41cyclictest0-21swapper/020:39:530
3188641994229,10cyclictest391ktimers/320:15:123
3188641994229,10cyclictest391ktimers/320:15:123
318862899421,6cyclictest0-21swapper/022:35:010
3188628994214,25cyclictest3300876-21sed00:30:180
3188628994214,25cyclictest3300876-21sed00:30:180
3188628994214,24cyclictest0-21swapper/022:45:000
318862899421,38cyclictest3282058-21/usr/sbin/munin23:35:200
318862899421,38cyclictest3282058-21/usr/sbin/munin23:35:200
318862899421,37cyclictest3249946-21munin-node22:05:120
318862899421,36cyclictest3229156-21df21:05:140
318862899421,36cyclictest3229156-21df21:05:140
318862899421,35cyclictest3203248-21apt-get19:50:120
3188637994129,9cyclictest3291705-21audit_send_reply00:05:012
3188628994131,7cyclictest3211745-21sh20:15:010
318862899411,3cyclictest3283200-21apt-get23:40:120
318862899411,37cyclictest3251953-21/usr/sbin/munin22:10:170
318862899411,37cyclictest3234392-21apt-get21:20:110
318862899411,36cyclictest3196518-21/usr/sbin/munin19:30:180
318862899411,35cyclictest3271071-21/usr/sbin/munin23:05:180
318862899411,35cyclictest3260757-21/usr/sbin/munin22:35:180
318862899411,35cyclictest3237894-21/usr/sbin/munin21:30:160
318862899411,34cyclictest3272624-21/usr/sbin/munin23:10:140
318862899411,34cyclictest3227356-21/usr/sbin/munin21:00:140
318862899411,34cyclictest3208582-21/usr/sbin/munin20:05:180
318862899411,33cyclictest3236096-21apt-get21:25:120
3188637994033,4cyclictest3227123-21idleruntime-cro21:00:002
318862899403,33cyclictest3297620-21ntp_states00:20:220
318862899402,35cyclictest3202066-21/usr/sbin/munin19:45:200
318862899402,31cyclictest3293636-21apt-get00:10:140
3188628994016,21cyclictest0-21swapper/000:40:000
318862899401,5cyclictest3267415-21apt-get22:55:130
318862899401,4cyclictest0-21swapper/021:10:130
318862899401,4cyclictest0-21swapper/021:10:120
318862899401,36cyclictest3296007-21/usr/sbin/munin00:15:240
318862899401,36cyclictest3285179-21/usr/sbin/munin23:45:170
318862899401,35cyclictest3226537-21/usr/sbin/munin20:55:250
318862899401,34cyclictest3292045-21df00:05:140
318862899401,34cyclictest3241491-21/usr/sbin/munin21:40:160
318862899401,34cyclictest3217446-21irqstats20:30:180
318862899401,33cyclictest3288512-21apt-get23:55:120
318862899400,4cyclictest0-21swapper/020:00:150
3188633993928,7cyclictest3303631-21sh00:39:591
318862899392,32cyclictest3194514-21apt-get19:25:120
318862899392,31cyclictest3242917-21apt-get21:45:130
318862899392,23cyclictest3288286-21munin-run23:55:010
318862899391,35cyclictest3290607-21/usr/sbin/munin00:00:180
318862899391,35cyclictest3240573-21users21:35:260
318862899391,35cyclictest3191202-21df_inode19:15:160
318862899391,24cyclictest3246864-21irqstats21:55:170
318862899391,24cyclictest3246864-21irqstats21:55:160
318862899390,2cyclictest0-21swapper/023:20:150
3188641993828,6cyclictest0-21swapper/300:40:003
3188633993817,12cyclictest131rcu_preempt21:10:211
3188633993817,12cyclictest131rcu_preempt21:10:201
318862899382,33cyclictest3269471-21/usr/sbin/munin23:00:190
318862899381,34cyclictest3279583-21date23:30:000
318862899381,34cyclictest3279583-21date23:30:000
318862899381,34cyclictest3257108-21irqstats22:25:190
318862899381,33cyclictest3199836-21apt-get19:40:070
318862899381,33cyclictest3199525-21date19:40:020
318862899381,32cyclictest3244615-21/usr/sbin/munin21:50:120
318862899372,32cyclictest3265763-21/usr/sbin/munin22:50:130
318862899371,30cyclictest3279780-21apt-get23:30:140
318862899371,30cyclictest3279780-21apt-get23:30:140
318862899371,30cyclictest3255072-21apt-get22:20:140
3188628993712,22cyclictest645-21runrttasks19:11:130
318862899370,2cyclictest0-21swapper/019:55:120
3188641993626,6cyclictest3271902-21unixbench_singl23:05:263
3188641993618,5cyclictest131rcu_preempt23:35:553
3188641993618,5cyclictest131rcu_preempt23:35:543
3188633993613,11cyclictest131rcu_preempt19:59:501
318862899361,2cyclictest3211983-21apt-get20:15:120
318862899361,2cyclictest3211983-21apt-get20:15:120
318864199358,6cyclictest40-21ksoftirqd/321:20:163
3188641993527,5cyclictest3218573-21cron20:35:013
3188641993527,4cyclictest0-21swapper/322:20:003
3188641993527,4cyclictest0-21swapper/321:02:283
3188637993524,8cyclictest311ktimers/219:10:122
318863399356,24cyclictest231ktimers/119:40:021
3188633993529,3cyclictest0-21swapper/120:40:001
3188633993527,4cyclictest0-21swapper/120:22:211
3188633993525,6cyclictest0-21swapper/123:25:011
3188633993521,4cyclictest131rcu_preempt22:10:171
3188633993515,10cyclictest131rcu_preempt22:21:061
3188641993424,6cyclictest0-21swapper/322:35:253
3188641993418,4cyclictest131rcu_preempt23:00:273
3188641993412,5cyclictest0-21swapper/319:10:243
3188637993426,4cyclictest0-21swapper/222:00:172
318863399348,7cyclictest24-21ksoftirqd/119:20:001
3188633993428,4cyclictest3239109-21kworker/u8:0+events_unbound21:42:361
3188633993427,5cyclictest3289972-21kworker/u8:1+events_unbound00:08:011
3188633993427,5cyclictest2587691-21kworker/u8:0+events_unbound19:42:421
3188633993427,4cyclictest0-21swapper/119:15:001
3188633993426,6cyclictest3281150-21kworker/u8:3+events_unbound23:34:421
3188633993426,6cyclictest3281150-21kworker/u8:3+events_unbound23:34:411
3188633993422,8cyclictest3291239-21grep00:00:251
3188633993422,4cyclictest131rcu_preempt22:30:211
3188633993418,9cyclictest131rcu_preempt00:20:151
3188633993417,8cyclictest131rcu_preempt20:40:141
3188633993416,10cyclictest131rcu_preempt23:55:181
3188633993414,4cyclictest131rcu_preempt23:08:371
3188633993410,12cyclictest24-21ksoftirqd/121:25:211
318863399340,4cyclictest0-21swapper/121:20:111
3188641993328,3cyclictest381rcuc/323:00:003
3188641993326,4cyclictest3199998-21fschecks_count19:40:173
3188641993324,6cyclictest3198900-21grep19:35:273
3188641993324,6cyclictest0-21swapper/319:15:283
3188641993324,5cyclictest3240989-21sendmail21:40:013
3188641993316,10cyclictest131rcu_preempt23:50:243
3188641993314,5cyclictest131rcu_preempt23:50:003
3188641993314,4cyclictest131rcu_preempt21:10:223
3188641993314,4cyclictest131rcu_preempt21:10:213
3188641993314,3cyclictest131rcu_preempt19:31:353
3188641993312,5cyclictest131rcu_preempt21:50:003
3188637993325,4cyclictest0-21swapper/222:40:012
3188633993326,4cyclictest3216867-21cron20:30:001
3188633993326,4cyclictest0-21swapper/120:20:011
3188633993326,4cyclictest0-21swapper/120:20:001
3188633993322,6cyclictest0-21swapper/121:30:281
3188633993322,4cyclictest131rcu_preempt23:15:191
3188633993322,4cyclictest131rcu_preempt21:50:191
3188633993319,6cyclictest131rcu_preempt21:15:211
3188633993319,6cyclictest131rcu_preempt21:15:211
3188633993319,4cyclictest131rcu_preempt23:38:141
3188633993319,4cyclictest131rcu_preempt23:38:141
3188633993316,5cyclictest131rcu_preempt20:10:011
3188633993316,4cyclictest131rcu_preempt23:41:231
3188633993315,9cyclictest131rcu_preempt22:45:151
3188633993315,9cyclictest131rcu_preempt22:45:151
3188633993315,4cyclictest131rcu_preempt21:35:151
3188633993315,11cyclictest131rcu_preempt21:00:181
3188633993314,9cyclictest131rcu_preempt20:56:541
3188633993314,4cyclictest131rcu_preempt23:49:441
3188633993313,5cyclictest131rcu_preempt22:55:001
3188633993313,3cyclictest131rcu_preempt20:30:251
3188633993313,13cyclictest131rcu_preempt00:30:231
3188633993313,13cyclictest131rcu_preempt00:30:231
3188633993313,12cyclictest3223673-21kworker/u8:1-events_freezable_power_21:06:561
3188633993313,12cyclictest3223673-21kworker/u8:1-events_freezable_power_21:06:551
3188633993313,11cyclictest131rcu_preempt20:02:431
3188633993313,10cyclictest0-21swapper/120:08:571
3188633993311,11cyclictest131rcu_preempt22:05:271
318864199329,7cyclictest40-21ksoftirqd/320:45:153
318864199326,12cyclictest40-21ksoftirqd/322:52:023
318864199326,11cyclictest391ktimers/322:20:323
318864199325,7cyclictest40-21ksoftirqd/319:25:013
3188641993222,7cyclictest3298054-21grep00:20:273
3188641993222,6cyclictest0-21swapper/300:15:263
3188641993220,7cyclictest391ktimers/319:50:123
3188641993217,4cyclictest131rcu_preempt22:45:023
3188641993217,4cyclictest131rcu_preempt22:45:023
3188641993215,5cyclictest131rcu_preempt21:55:483
3188641993215,5cyclictest131rcu_preempt21:55:473
318863799323,5cyclictest131rcu_preempt21:50:132
3188633993224,4cyclictest0-21swapper/119:50:161
3188633993223,6cyclictest3222369-21fschecks_count20:45:171
3188633993219,4cyclictest131rcu_preempt00:10:181
3188633993217,4cyclictest131rcu_preempt23:50:131
3188633993215,10cyclictest131rcu_preempt22:55:191
3188633993214,9cyclictest131rcu_preempt21:49:431
3188633993214,11cyclictest131rcu_preempt23:00:241
3188633993214,10cyclictest131rcu_preempt21:55:151
3188633993214,10cyclictest131rcu_preempt21:55:141
3188633993213,16cyclictest231ktimers/122:29:571
3188633993211,5cyclictest131rcu_preempt23:25:161
3188633993211,5cyclictest131rcu_preempt23:25:161
3188633993211,10cyclictest131rcu_preempt00:25:181
318864199318,15cyclictest40-21ksoftirqd/322:25:263
318864199315,8cyclictest40-21ksoftirqd/320:15:013
318864199313,8cyclictest40-21ksoftirqd/300:25:213
3188641993120,8cyclictest391ktimers/323:30:003
3188641993120,8cyclictest391ktimers/323:30:003
3188641993120,8cyclictest391ktimers/322:45:003
3188641993116,5cyclictest131rcu_preempt23:20:213
3188641993115,5cyclictest131rcu_preempt21:55:003
3188641993115,4cyclictest131rcu_preempt20:25:143
3188641993115,4cyclictest131rcu_preempt20:22:223
3188641993114,10cyclictest131rcu_preempt21:40:193
3188641993113,3cyclictest131rcu_preempt22:05:013
3188641993111,5cyclictest131rcu_preempt19:45:133
3188641993110,9cyclictest131rcu_preempt23:55:223
3188637993125,4cyclictest3301579-21kworker/u8:1+events_unbound00:33:052
3188637993125,4cyclictest3301579-21kworker/u8:1+events_unbound00:33:052
3188637993124,4cyclictest0-21swapper/223:30:262
3188637993124,4cyclictest0-21swapper/223:30:262
3188637993121,7cyclictest3216870-21sh20:30:002
3188633993121,6cyclictest3296392-21grep00:15:251
3188633993120,4cyclictest131rcu_preempt22:40:171
3188633993119,4cyclictest131rcu_preempt20:52:161
3188633993117,5cyclictest131rcu_preempt19:20:141
3188633993117,4cyclictest131rcu_preempt19:32:341
3188633993110,9cyclictest131rcu_preempt23:12:021
318864199308,11cyclictest0-21swapper/320:55:153
318864199307,20cyclictest391ktimers/319:25:243
3188641993020,7cyclictest391ktimers/300:05:003
3188641993017,4cyclictest131rcu_preempt19:55:203
3188641993016,8cyclictest131rcu_preempt00:10:143
3188641993015,4cyclictest131rcu_preempt21:05:173
3188641993015,4cyclictest131rcu_preempt21:05:173
3188641993014,5cyclictest131rcu_preempt20:45:013
318864199301,21cyclictest0-21swapper/320:50:223
3188641993010,5cyclictest131rcu_preempt00:10:003
3188637993022,4cyclictest0-21swapper/223:10:152
3188633993012,4cyclictest131rcu_preempt22:35:031
318864199293,7cyclictest40-21ksoftirqd/321:15:203
318864199293,7cyclictest40-21ksoftirqd/321:15:193
3188641992922,3cyclictest0-21swapper/322:05:173
3188641992918,7cyclictest1-21systemd00:33:033
3188641992918,7cyclictest1-21systemd00:33:033
3188641992917,8cyclictest204-21dbus-daemon20:02:173
3188637992919,6cyclictest0-21swapper/219:55:012
3188641992821,4cyclictest391ktimers/323:14:483
3188641992817,8cyclictest391ktimers/321:25:193
3188641992814,4cyclictest131rcu_preempt20:05:143
3188641992813,4cyclictest131rcu_preempt21:30:243
3188641992811,5cyclictest131rcu_preempt23:40:243
3188637992819,4cyclictest0-21swapper/222:05:182
318864199277,7cyclictest40-21ksoftirqd/323:30:143
318864199277,7cyclictest40-21ksoftirqd/323:30:133
3188641992718,7cyclictest391ktimers/322:10:253
318863799279,3cyclictest131rcu_preempt00:05:122
3188637992721,4cyclictest3286277-21kworker/u8:2+events_unbound23:52:582
3188637992721,3cyclictest0-21swapper/221:40:002
3188637992719,5cyclictest0-21swapper/223:55:162
3188637992718,7cyclictest3272311-21cut23:10:002
3188637992718,6cyclictest0-21swapper/220:05:002
3188637992620,3cyclictest0-21swapper/220:10:252
3188637992619,5cyclictest3195908-21kworker/u8:1+events_unbound19:47:162
3188637992619,3cyclictest0-21swapper/220:20:012
3188637992619,3cyclictest0-21swapper/220:20:002
3188637992617,6cyclictest0-21swapper/219:20:012
318863799253,3cyclictest3294647-21unixbench_multi00:10:262
3188637992519,3cyclictest0-21swapper/200:25:252
3188637992518,4cyclictest3243616-21mailstats21:45:242
3188637992518,4cyclictest0-21swapper/221:30:212
3188637992517,4cyclictest0-21swapper/222:50:262
318863799251,21cyclictest3222415-21fschecks_time20:45:172
3188637992418,3cyclictest0-21swapper/222:00:002
3188637992418,3cyclictest0-21swapper/222:00:002
3188637992417,4cyclictest3295862-21ntpq00:15:202
3188637992417,4cyclictest0-21swapper/221:15:272
3188637992417,4cyclictest0-21swapper/221:15:262
3188637992416,5cyclictest0-21swapper/220:20:252
3188637992416,4cyclictest0-21swapper/223:40:002
3188637992416,4cyclictest0-21swapper/223:40:002
3188637992416,4cyclictest0-21swapper/222:10:172
318863799241,4cyclictest0-21swapper/223:00:182
3188637992317,4cyclictest3298341-21kworker/u8:2+events_unbound00:23:022
3188637992317,4cyclictest3192548-21kworker/u8:2+events_unbound19:22:142
3188637992317,3cyclictest3220275-21latency_hist20:40:012
3188637992317,3cyclictest0-21swapper/200:35:242
3188637992316,4cyclictest3259000-21ntpq22:30:222
3188637992314,6cyclictest3209061-21fgrep20:05:252
3188637992314,6cyclictest0-21swapper/223:40:252
3188637992314,6cyclictest0-21swapper/222:20:002
3188637992314,6cyclictest0-21swapper/221:05:192
3188637992314,6cyclictest0-21swapper/221:05:182
3188637992314,6cyclictest0-21swapper/220:30:172
318863799231,3cyclictest3234963-21ntp_states21:20:222
318863799225,3cyclictest131rcu_preempt21:40:192
318863799223,6cyclictest131rcu_preempt19:55:112
3188637992216,3cyclictest0-21swapper/222:25:202
3188637992216,3cyclictest0-21swapper/222:22:432
3188637992215,3cyclictest0-21swapper/222:55:162
3188637992215,3cyclictest0-21swapper/222:50:002
3188637992215,3cyclictest0-21swapper/222:50:002
3188637992213,7cyclictest3194682-21cut19:25:182
318863799220,7cyclictest3231093-21/usr/sbin/munin21:10:182
318863799220,7cyclictest3231093-21/usr/sbin/munin21:10:182
318863799218,2cyclictest131rcu_preempt19:35:202
318863799217,3cyclictest131rcu_preempt20:40:202
3188637992114,5cyclictest3223673-21kworker/u8:1+events_unbound21:00:202
3188637992113,6cyclictest3285008-21df23:45:142
3188637992113,5cyclictest3237677-21latency_hist21:30:012
3188637992112,6cyclictest0-21swapper/219:30:162
3188637992014,3cyclictest0-21swapper/219:40:252
3188637991913,3cyclictest0-21swapper/220:52:112
3188637991912,5cyclictest311ktimers/223:25:152
3188637991912,5cyclictest311ktimers/223:25:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional