You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 22:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Tue Nov 25, 2025 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
38468995547,5cyclictest69884-21kworker/u8:1+events_unbound08:54:573
38455995344,6cyclictest98042-21fschecks_count10:00:160
38455995037,6cyclictest110928-21kworker/u8:1+events_unbound10:50:210
38455994837,8cyclictest63083-21sh08:20:000
38468994740,4cyclictest84211-21fschecks_time09:20:173
38463994434,6cyclictest81662-21grep09:10:272
38455994436,5cyclictest69911-21munin-plugin-st08:40:000
38468994335,5cyclictest440-21cron12:30:003
38463994332,6cyclictest0-21swapper/211:20:002
38463994332,6cyclictest0-21swapper/211:20:002
38468994235,4cyclictest97431-21sendmail09:59:593
38468994222,6cyclictest78411-21kworker/u8:0+events_unbound09:14:083
38455994234,5cyclictest123004-21unixbench_singl11:10:250
38468994129,8cyclictest141967-21grep12:05:253
38463994131,5cyclictest0-21swapper/207:15:012
38455994131,6cyclictest0-21swapper/007:35:000
38468994032,5cyclictest88875-21cron09:35:003
38468994030,7cyclictest86730-21tail09:25:273
38468994029,8cyclictest47511-21sh07:35:003
38463994031,6cyclictest65174-21fschecks_count08:25:162
38463994031,6cyclictest65174-21fschecks_count08:25:162
38468993928,7cyclictest93601-21unixbench_singl09:45:273
38468993928,7cyclictest93601-21unixbench_singl09:45:273
38468993928,7cyclictest50466-21grep07:40:273
38468993927,8cyclictest112751-21grep10:40:283
38460993929,6cyclictest0-21swapper/107:15:021
38455993929,6cyclictest44115-21apt07:25:010
38455993929,6cyclictest0-21swapper/012:35:010
38468993827,7cyclictest0-21swapper/308:00:283
38463993828,5cyclictest0-21swapper/212:25:452
38460993829,5cyclictest440-21cron12:00:001
38460993828,6cyclictest0-21swapper/109:30:001
38455993827,7cyclictest64811-21cut08:25:000
38468993729,4cyclictest0-21swapper/309:05:163
38468993720,6cyclictest138716-21kworker/u8:1+events_unbound11:58:433
38463993727,6cyclictest0-21swapper/208:55:002
38455993730,4cyclictest90592-21munin-plugin-st09:40:000
38455993729,5cyclictest0-21swapper/009:45:010
38455993728,5cyclictest0-21swapper/011:59:590
38455993727,6cyclictest57507-21grep08:00:280
38468993618,6cyclictest61322-21kworker/u8:0+events_unbound08:38:363
38463993630,3cyclictest80241-21gpgv09:10:012
38463993627,5cyclictest0-21swapper/209:35:162
38463993626,6cyclictest0-21swapper/211:35:002
38463993626,6cyclictest0-21swapper/208:05:012
38463993625,7cyclictest0-21swapper/210:45:252
38460993626,5cyclictest0-21swapper/107:20:011
3846099360,3cyclictest131rcu_preempt08:35:141
38455993629,4cyclictest97418-21munin-plugin-st09:59:590
38455993627,5cyclictest0-21swapper/009:25:010
38455993627,5cyclictest0-21swapper/009:05:000
38455993626,6cyclictest0-21swapper/012:30:000
38455993625,7cyclictest93599-21grep09:45:270
38455993625,7cyclictest93599-21grep09:45:270
38468993529,4cyclictest76678-21kworker/u8:3+events_unbound09:04:573
38468993528,4cyclictest0-21swapper/311:45:423
38468993528,4cyclictest0-21swapper/311:45:423
38468993526,5cyclictest0-21swapper/312:20:013
38468993525,6cyclictest0-21swapper/310:05:003
38468993524,7cyclictest0-21swapper/312:35:003
38468993517,6cyclictest92067-21kworker/u8:0+events_unbound09:42:203
38463993528,4cyclictest88878-21cron09:35:012
38463993528,4cyclictest104236-21kworker/u8:2-events_unbound10:25:132
38463993525,6cyclictest0-21swapper/208:55:262
38463993524,7cyclictest44113-21sh07:25:012
38463993524,7cyclictest0-21swapper/211:55:272
38463993524,6cyclictest64331-21grep08:20:282
3846099353,8cyclictest36761-21kworker/1:0-mm_percpu_wq08:30:121
3846099353,8cyclictest36761-21kworker/1:0-mm_percpu_wq08:30:121
38460993528,5cyclictest119978-21kworker/u8:0+events_unbound11:15:261
38460993528,5cyclictest119978-21kworker/u8:0+events_unbound11:15:251
38455993527,5cyclictest299-21systemd-logind09:50:040
38455993527,5cyclictest0-21swapper/011:54:590
38455993527,4cyclictest0-21swapper/008:30:260
38455993527,4cyclictest0-21swapper/008:30:250
38468993426,4cyclictest0-21swapper/309:00:013
38468993425,5cyclictest0-21swapper/308:45:163
38463993426,4cyclictest0-21swapper/212:20:012
38463993425,5cyclictest0-21swapper/212:20:282
38463993424,6cyclictest0-21swapper/211:30:002
38463993424,6cyclictest0-21swapper/211:30:002
38460993427,3cyclictest0-21swapper/111:40:001
38460993427,3cyclictest0-21swapper/108:05:181
38460993426,4cyclictest0-21swapper/109:00:181
38460993419,12cyclictest231ktimers/107:50:111
38455993425,5cyclictest0-21swapper/012:25:000
38455993424,7cyclictest126224-21timerandwakeup11:20:260
38455993424,7cyclictest126224-21timerandwakeup11:20:260
38455993424,6cyclictest0-21swapper/012:40:010
38468993324,5cyclictest0-21swapper/311:19:593
38468993324,5cyclictest0-21swapper/311:19:593
38468993322,8cyclictest59663-21cron08:10:003
38463993325,4cyclictest0-21swapper/207:34:392
38463993324,5cyclictest109118-21unixbench_multi10:30:262
38460993324,5cyclictest0-21swapper/108:50:011
38460993323,7cyclictest76301-21grep08:55:251
38460993323,6cyclictest0-21swapper/111:10:001
38455993325,4cyclictest0-21swapper/009:15:170
38455993324,6cyclictest0-21swapper/011:39:590
38455993324,6cyclictest0-21swapper/009:00:000
38455993323,6cyclictest0-21swapper/010:40:280
38468993225,4cyclictest0-21swapper/310:35:253
38468993225,4cyclictest0-21swapper/308:10:263
38468993225,4cyclictest0-21swapper/308:10:263
38468993218,9cyclictest62624-21grep08:15:263
38463993224,4cyclictest0-21swapper/212:30:252
38463993222,6cyclictest0-21swapper/211:45:172
38463993222,6cyclictest0-21swapper/211:45:162
38463993222,6cyclictest0-21swapper/211:10:162
38463993222,6cyclictest0-21swapper/210:35:262
38463993221,9cyclictest311ktimers/207:35:142
38460993225,5cyclictest105978-21kworker/u8:0+events_unbound10:35:161
38460993224,4cyclictest0-21swapper/110:05:271
38460993224,4cyclictest0-21swapper/107:20:241
38460993221,6cyclictest0-21swapper/111:10:241
38455993225,5cyclictest105978-21kworker/u8:0+events_unbound10:25:140
38455993223,5cyclictest0-21swapper/011:45:010
38455993223,5cyclictest0-21swapper/007:55:260
38468993125,4cyclictest142289-21kworker/u8:0+events_unbound12:10:423
38468993120,6cyclictest123001-21grep11:10:253
38468993115,5cyclictest128179-21kworker/u8:0+events_unbound12:02:553
3846899311,14cyclictest0-21swapper/310:55:143
38463993123,4cyclictest0-21swapper/210:05:262
38463993122,5cyclictest0-21swapper/211:00:162
38463993121,6cyclictest0-21swapper/208:05:262
38460993125,4cyclictest148951-21kworker/u8:0+events_unbound12:35:431
38460993122,5cyclictest81665-21unixbench_singl09:10:281
38460993121,6cyclictest0-21swapper/112:15:251
38460993121,6cyclictest0-21swapper/110:40:281
38460993121,6cyclictest0-21swapper/108:20:001
38460993119,7cyclictest0-21swapper/108:50:301
38455993124,3cyclictest0-21swapper/007:45:250
38455993123,4cyclictest0-21swapper/011:19:590
38455993123,4cyclictest0-21swapper/011:19:590
38455993122,6cyclictest117845-21sed10:55:260
38455993118,4cyclictest123641-21kworker/u8:2+events_unbound11:31:370
38468993024,4cyclictest92067-21kworker/u8:0+events_unbound09:51:423
38468993014,6cyclictest36523-21kworker/u8:1+events_unbound07:13:253
3846899300,7cyclictest40-21ksoftirqd/310:30:173
38460993024,3cyclictest0-21swapper/109:40:011
38460993022,5cyclictest0-21swapper/109:30:261
38460993022,4cyclictest0-21swapper/112:05:381
38460993022,4cyclictest0-21swapper/110:20:161
38460993020,6cyclictest0-21swapper/111:40:261
38460993020,6cyclictest0-21swapper/109:55:251
38460993020,6cyclictest0-21swapper/109:20:251
38460993019,6cyclictest0-21swapper/108:40:281
38460993014,10cyclictest126457-21kworker/u8:1+events_unbound11:21:041
38460993014,10cyclictest126457-21kworker/u8:1+events_unbound11:21:031
38455993022,5cyclictest0-21swapper/009:05:160
38455993021,5cyclictest0-21swapper/011:50:000
38455993021,5cyclictest0-21swapper/011:49:590
38455993020,6cyclictest42394-21sh07:20:020
38455993010,12cyclictest637-21runrttasks10:14:120
3846899294,3cyclictest131rcu_preempt07:25:123
38468992921,4cyclictest0-21swapper/307:50:163
38468992920,5cyclictest0-21swapper/307:40:003
38468992913,5cyclictest131rcu_preempt10:05:123
38463992921,5cyclictest0-21swapper/210:20:012
38463992921,5cyclictest0-21swapper/210:20:002
38463992920,5cyclictest0-21swapper/208:45:252
3846399291,4cyclictest131rcu_preempt08:35:152
38460992921,5cyclictest134113-21kworker/u8:3+events_unbound12:20:281
38460992920,5cyclictest0-21swapper/110:00:271
38460992919,6cyclictest0-21swapper/107:34:391
38460992914,12cyclictest231ktimers/111:50:111
38455992920,5cyclictest0-21swapper/007:40:270
38455992919,6cyclictest0-21swapper/010:35:270
38455992917,6cyclictest138716-21kworker/u8:1+events_unbound12:05:190
38468992821,4cyclictest0-21swapper/310:15:273
38468992821,4cyclictest0-21swapper/310:15:273
38468992820,4cyclictest0-21swapper/311:20:173
38468992820,4cyclictest0-21swapper/311:20:163
38468992817,8cyclictest131525-21grep11:35:243
3846899281,14cyclictest0-21swapper/312:20:153
38463992821,3cyclictest0-21swapper/212:00:272
38463992820,4cyclictest0-21swapper/209:40:152
38463992820,4cyclictest0-21swapper/208:15:262
38463992819,6cyclictest0-21swapper/210:50:262
38463992817,7cyclictest0-21swapper/209:00:282
38460992821,5cyclictest93798-21kworker/u8:1+events_unbound09:51:051
38460992820,4cyclictest0-21swapper/111:00:171
38460992820,4cyclictest0-21swapper/110:50:251
38460992819,6cyclictest48762-21unixbench_multi07:35:271
38460992819,6cyclictest0-21swapper/108:00:001
38460992819,5cyclictest0-21swapper/109:15:161
38460992818,6cyclictest129773-21unixbench_multi11:30:251
38455992819,6cyclictest0-21swapper/008:30:010
38455992819,6cyclictest0-21swapper/008:30:000
38455992818,6cyclictest0-21swapper/008:10:260
38455992818,6cyclictest0-21swapper/008:10:260
38455992817,7cyclictest119623-21grep11:00:270
38455992810,8cyclictest11-21ksoftirqd/009:25:270
3846899277,9cyclictest131rcu_preempt07:45:243
38468992720,5cyclictest65915-21kworker/u8:1+events_unbound08:26:493
38468992720,5cyclictest65915-21kworker/u8:1+events_unbound08:26:493
38468992720,5cyclictest391ktimers/310:10:183
38468992716,7cyclictest128073-21grep11:25:263
38468992716,7cyclictest128073-21grep11:25:263
38468992716,3cyclictest0-21swapper/312:35:213
38468992715,8cyclictest0-21swapper/311:05:273
38468992712,10cyclictest391ktimers/310:45:133
3846899270,6cyclictest0-21swapper/311:40:133
38463992718,5cyclictest0-21swapper/212:10:262
38463992716,7cyclictest0-21swapper/207:45:002
3845599277,9cyclictest11-21ksoftirqd/010:20:220
38455992721,3cyclictest0-21swapper/007:50:210
38455992720,5cyclictest36523-21kworker/u8:1+events_unbound07:10:160
38455992720,5cyclictest134113-21kworker/u8:3+events_unbound12:12:120
38455992720,5cyclictest112885-21kworker/u8:2+events_unbound10:46:550
38455992720,4cyclictest47263-21kworker/u8:0+events_unbound07:35:160
3845599271,23cyclictest144475-21grep12:15:160
3846899264,16cyclictest42404-21sendmail07:20:023
3846899263,18cyclictest56071-21sendmail08:00:003
3846899262,15cyclictest83782-21idleruntime-cro09:20:013
38468992619,5cyclictest93798-21kworker/u8:1+events_unbound10:20:213
38468992619,5cyclictest61322-21kworker/u8:0+events_unbound08:30:263
38468992619,5cyclictest61322-21kworker/u8:0+events_unbound08:30:263
38468992614,8cyclictest0-21swapper/309:35:203
3846899261,17cyclictest40-21ksoftirqd/311:30:313
38463992619,5cyclictest93798-21kworker/u8:1+events_unbound09:50:212
38463992616,7cyclictest94012-21idleruntime-cro09:50:002
38463992616,7cyclictest94012-21idleruntime-cro09:50:002
38463992615,7cyclictest1-21systemd09:30:012
38463992614,9cyclictest215-21dbus-daemon09:25:002
38460992619,3cyclictest0-21swapper/110:30:271
38460992616,7cyclictest101435-21fschecks_count10:10:151
38455992619,5cyclictest57853-21kworker/u8:2+events_unbound08:09:460
38455992618,5cyclictest75050-21grep08:55:020
38455992618,4cyclictest0-21swapper/010:05:170
3846899258,10cyclictest0-21swapper/310:25:123
38468992518,5cyclictest128179-21kworker/u8:0+events_unbound11:54:043
38468992515,7cyclictest63796-21sort08:20:213
38463992518,5cyclictest93798-21kworker/u8:1+events_unbound10:02:272
38463992518,5cyclictest80123-21kworker/u8:1+events_unbound09:20:002
38463992518,3cyclictest0-21swapper/211:45:002
38463992516,5cyclictest0-21swapper/212:10:002
38460992518,3cyclictest0-21swapper/107:25:251
3846099251,6cyclictest91845-21grep09:40:261
38460992516,5cyclictest0-21swapper/111:25:261
38460992516,5cyclictest0-21swapper/111:25:261
38460992514,8cyclictest49570-21ls07:40:161
38455992517,5cyclictest0-21swapper/010:20:010
38455992517,5cyclictest0-21swapper/010:20:000
3845599251,20cyclictest0-21swapper/009:32:570
3846899247,9cyclictest40-21ksoftirqd/311:00:243
38463992418,4cyclictest114612-21kworker/u8:0+events_unbound10:59:202
38463992417,5cyclictest59593-21kworker/u8:1+events_unbound08:14:472
38463992417,5cyclictest59593-21kworker/u8:1+events_unbound08:14:472
38463992417,3cyclictest0-21swapper/210:10:232
38463992416,5cyclictest45029-21gzip07:25:242
38463992415,6cyclictest68193-21cron08:35:012
38463992415,6cyclictest68193-21cron08:35:002
38463992414,6cyclictest42424-21basename07:20:012
3846399240,3cyclictest131rcu_preempt11:35:152
38460992415,6cyclictest57526-21unixbench_multi08:00:291
38460992414,7cyclictest79073-21cat09:05:191
38460992414,6cyclictest0-21swapper/108:20:281
38455992416,5cyclictest0-21swapper/011:05:270
38455992416,5cyclictest0-21swapper/009:10:230
38455992416,5cyclictest0-21swapper/007:25:210
3845599241,20cyclictest713292sleep008:41:350
3845599241,19cyclictest0-21swapper/012:04:040
3846899231,10cyclictest40-21ksoftirqd/308:40:253
3846899230,14cyclictest0-21swapper/310:50:193
3846399238,5cyclictest110928-21kworker/u8:1+events_unbound11:09:292
38463992316,3cyclictest0-21swapper/207:45:232
38463992312,7cyclictest215-21dbus-daemon11:20:302
38463992312,7cyclictest215-21dbus-daemon11:20:302
38460992317,4cyclictest65030-21/usr/sbin/munin08:25:251
38460992317,4cyclictest65030-21/usr/sbin/munin08:25:251
38460992317,3cyclictest0-21swapper/108:10:231
38460992317,3cyclictest0-21swapper/108:10:221
38460992317,3cyclictest0-21swapper/107:45:141
38460992316,5cyclictest0-21swapper/112:25:201
3846099231,3cyclictest131rcu_preempt12:00:211
38460992313,6cyclictest0-21swapper/112:10:161
38460992313,6cyclictest0-21swapper/110:45:251
38455992314,6cyclictest0-21swapper/008:45:260
38463992215,4cyclictest104839-21fschecks_count10:20:162
38463992214,5cyclictest38448-21cyclictest11:50:262
3846399220,12cyclictest53918-21unixbench_singl07:50:252
38460992216,4cyclictest104236-21kworker/u8:2+events_unbound10:25:031
38460992215,4cyclictest135573-21sh11:49:591
38460992215,4cyclictest135573-21sh11:49:591
38460992215,4cyclictest0-21swapper/109:45:261
38460992215,4cyclictest0-21swapper/109:45:261
38460992214,4cyclictest0-21swapper/110:15:091
38460992214,4cyclictest0-21swapper/110:15:091
38463992114,3cyclictest0-21swapper/207:55:182
38463992113,5cyclictest111727-21/usr/sbin/munin10:40:242
38460992115,3cyclictest0-21swapper/112:30:141
38455992114,3cyclictest0-21swapper/010:31:460
38463992013,4cyclictest311ktimers/208:45:012
3846099196,6cyclictest131rcu_preempt10:55:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional