You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 05:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Nov 21, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
573716994939,7cyclictest636675-21unixbench_multi22:05:252
573716994939,7cyclictest636675-21unixbench_multi22:05:252
573716994842,4cyclictest608686-21kworker/u8:1+events_unbound21:02:392
573712994840,5cyclictest577670-21sendmail19:20:011
573712994840,5cyclictest577670-21sendmail19:20:001
573719994736,7cyclictest687023-21sh00:35:003
573716994739,5cyclictest603443-21unixbench_singl20:30:322
573716994636,4cyclictest642061-21kworker/u8:2+events_unbound22:22:552
573719994534,5cyclictest628205-21kworker/u8:2+events_unbound21:47:483
573719994533,5cyclictest608686-21kworker/u8:1+events_unbound21:17:423
573716994538,5cyclictest679851-21kworker/u8:3+events_unbound00:28:112
573719994436,4cyclictest620925-21tail21:20:283
573716994436,4cyclictest0-21swapper/221:15:002
573712994331,7cyclictest0-21swapper/122:05:261
573712994331,7cyclictest0-21swapper/122:05:251
573716994234,4cyclictest605593-21idleruntime-cro20:40:012
573712994233,5cyclictest215-21dbus-daemon21:57:511
573719994135,4cyclictest636942-21kworker/u8:2+events_unbound22:12:543
573719994130,7cyclictest0-21swapper/323:00:003
573712994134,3cyclictest0-21swapper/122:50:171
573719994032,6cyclictest673280-21kworker/u8:2+events_unbound23:58:113
573719994032,6cyclictest673280-21kworker/u8:2+events_unbound23:58:103
573712994034,3cyclictest0-21swapper/120:50:011
573719993933,4cyclictest598152-21kworker/u8:1+events_unbound20:22:343
573719993919,7cyclictest131rcu_preempt23:50:143
573716993929,6cyclictest626218-21grep21:35:262
573712993932,4cyclictest621375-21munin-plugin-st21:25:001
573708993930,5cyclictest1-21systemd22:27:550
573719993830,4cyclictest688876-21apt-key00:40:013
573719993819,11cyclictest131rcu_preempt19:20:013
573719993819,11cyclictest131rcu_preempt19:20:003
573716993824,7cyclictest642061-21kworker/u8:2+events_unbound22:29:212
573719993731,4cyclictest679851-21kworker/u8:3+events_unbound00:13:113
573719993731,4cyclictest642061-21kworker/u8:2+events_unbound22:27:563
573719993721,5cyclictest657819-21kworker/u8:0-events_freezable_power_23:14:013
573719993720,6cyclictest131rcu_preempt19:15:013
573719993716,5cyclictest131rcu_preempt00:30:003
573716993726,8cyclictest651002-21munin-plugin-st22:50:012
573716993721,4cyclictest131rcu_preempt20:07:052
573716993720,10cyclictest131rcu_preempt00:39:282
573716993717,5cyclictest131rcu_preempt21:15:252
573712993730,4cyclictest619652-21latency_hist21:20:011
573708993723,10cyclictest121ktimers/022:40:140
573719993618,12cyclictest131rcu_preempt00:15:193
573719993617,13cyclictest131rcu_preempt23:45:203
573719993617,13cyclictest131rcu_preempt23:45:193
573716993621,3cyclictest131rcu_preempt19:13:162
573716993619,3cyclictest131rcu_preempt23:20:192
573716993617,5cyclictest131rcu_preempt00:09:592
573716993617,10cyclictest131rcu_preempt22:10:262
573712993628,4cyclictest0-21swapper/122:00:261
573712993627,6cyclictest0-21swapper/121:00:001
573719993529,4cyclictest628277-21kworker/u8:3+events_unbound21:52:503
573719993527,4cyclictest0-21swapper/320:15:293
573719993527,4cyclictest0-21swapper/320:10:003
573719993527,4cyclictest0-21swapper/319:30:013
573719993520,9cyclictest131rcu_preempt00:10:013
573719993519,10cyclictest131rcu_preempt23:35:013
57371699357,15cyclictest32-21ksoftirqd/219:45:022
573716993519,9cyclictest131rcu_preempt22:40:552
573716993519,5cyclictest131rcu_preempt20:50:262
573716993516,5cyclictest131rcu_preempt20:15:282
573716993513,5cyclictest131rcu_preempt22:00:242
573712993527,4cyclictest0-21swapper/120:20:271
573712993524,7cyclictest574866-21grep19:10:261
573712993524,7cyclictest0-21swapper/123:45:001
573708993524,6cyclictest0-21swapper/000:20:160
573719993428,4cyclictest681579-21kworker/u8:0+events_unbound00:23:103
573719993424,7cyclictest603429-21grep20:30:323
573719993424,5cyclictest0-21swapper/320:37:333
573719993422,5cyclictest633485-21kworker/u8:1+events_unbound22:02:533
573719993420,8cyclictest131rcu_preempt22:06:083
573719993420,8cyclictest131rcu_preempt22:06:083
573719993417,6cyclictest608686-21kworker/u8:1+events_unbound21:09:143
573719993414,10cyclictest131rcu_preempt21:45:023
57371699349,16cyclictest32-21ksoftirqd/221:20:292
573716993427,4cyclictest668284-21sendmail23:40:002
573716993427,3cyclictest0-21swapper/222:55:172
573716993423,7cyclictest683620-21sh00:25:012
573716993418,4cyclictest131rcu_preempt00:10:592
573716993417,7cyclictest605226-21kworker/u8:3+events_unbound20:42:172
573716993417,5cyclictest131rcu_preempt21:40:252
573716993416,4cyclictest131rcu_preempt00:15:272
573716993412,5cyclictest131rcu_preempt19:55:242
573712993427,4cyclictest0-21swapper/123:00:011
573712993426,4cyclictest0-21swapper/100:25:261
573708993425,5cyclictest679851-21kworker/u8:3+events_unbound00:33:110
573708993424,7cyclictest0-21swapper/022:40:010
573708993424,6cyclictest0-21swapper/022:05:260
573708993424,6cyclictest0-21swapper/022:05:260
573708993423,6cyclictest0-21swapper/023:39:590
573708993422,7cyclictest626682-21apt21:40:010
57371999337,16cyclictest40-21ksoftirqd/321:30:013
573719993327,4cyclictest650689-21kworker/u8:0+events_unbound22:49:413
573719993320,4cyclictest131rcu_preempt22:20:183
573719993318,8cyclictest643834-21kworker/u8:1-writeback22:35:223
573719993318,4cyclictest131rcu_preempt23:25:003
573719993317,8cyclictest131rcu_preempt21:35:223
573719993315,10cyclictest131rcu_preempt21:55:023
573719993313,13cyclictest131rcu_preempt19:50:153
573719993310,11cyclictest40-21ksoftirqd/320:10:293
57371699338,14cyclictest32-21ksoftirqd/220:55:332
573716993318,9cyclictest131rcu_preempt22:15:332
573716993316,8cyclictest302-21rpcbind21:50:222
573716993316,3cyclictest0-21swapper/221:49:412
573716993313,5cyclictest131rcu_preempt19:40:012
573716993313,4cyclictest131rcu_preempt23:51:262
57371699331,21cyclictest0-21swapper/223:00:162
573716993311,5cyclictest131rcu_preempt23:40:242
573716993311,5cyclictest131rcu_preempt21:30:222
573716993310,9cyclictest131rcu_preempt19:45:122
57371699330,18cyclictest658363-21/usr/sbin/munin23:10:142
573712993322,7cyclictest667851-21grep23:35:281
573708993326,4cyclictest640544-21munin-run22:19:590
573708993326,3cyclictest0-21swapper/021:40:260
573708993324,4cyclictest0-21swapper/023:10:250
573708993322,7cyclictest608588-21grep20:45:290
573708993322,6cyclictest0-21swapper/021:55:270
573719993221,7cyclictest440-21cron19:50:003
573719993217,9cyclictest131rcu_preempt20:44:483
573719993216,4cyclictest131rcu_preempt21:10:113
573719993215,11cyclictest131rcu_preempt23:05:233
573719993214,10cyclictest131rcu_preempt20:00:273
573719993214,10cyclictest131rcu_preempt20:00:273
57371999321,18cyclictest0-21swapper/319:55:133
57371699328,13cyclictest32-21ksoftirqd/223:15:132
573716993218,4cyclictest131rcu_preempt20:20:252
573716993216,7cyclictest622762-21kworker/u8:0+events_unbound21:26:342
573716993214,9cyclictest131rcu_preempt19:25:152
573716993214,5cyclictest131rcu_preempt20:00:212
573716993214,5cyclictest131rcu_preempt20:00:212
573716993214,4cyclictest131rcu_preempt23:55:212
573716993214,4cyclictest131rcu_preempt23:55:212
573716993213,9cyclictest131rcu_preempt19:50:472
573716993213,12cyclictest131rcu_preempt19:30:172
573716993213,10cyclictest131rcu_preempt22:30:162
573716993212,11cyclictest131rcu_preempt19:20:152
573716993211,5cyclictest131rcu_preempt23:45:222
573716993211,5cyclictest131rcu_preempt23:45:222
573716993210,5cyclictest131rcu_preempt23:25:212
573716993210,12cyclictest32-21ksoftirqd/200:30:212
573712993225,4cyclictest0-21swapper/100:10:001
573712993224,4cyclictest0-21swapper/123:25:151
573712993224,4cyclictest0-21swapper/120:50:171
573712993221,7cyclictest0-21swapper/123:15:011
573708993223,4cyclictest0-21swapper/021:10:160
573708993223,4cyclictest0-21swapper/020:05:250
573719993125,4cyclictest584261-21kworker/u8:3+events_unbound19:42:243
573719993121,4cyclictest131rcu_preempt23:40:013
573719993119,4cyclictest131rcu_preempt22:15:203
573719993119,4cyclictest131rcu_preempt19:20:173
573719993116,8cyclictest600109-21kworker/u8:2+events_unbound20:25:103
573719993115,3cyclictest131rcu_preempt19:35:103
573719993113,12cyclictest131rcu_preempt21:35:003
57371999311,17cyclictest40-21ksoftirqd/323:00:263
573719993111,5cyclictest656098-21kworker/u8:1-events_freezable_power_23:27:233
573719993111,11cyclictest131rcu_preempt00:00:213
573716993117,5cyclictest654116-21kworker/u8:2+events_unbound23:07:092
573716993116,5cyclictest586079-21kworker/u8:0+events_unbound20:26:222
573716993115,4cyclictest131rcu_preempt22:53:172
573716993113,4cyclictest131rcu_preempt20:10:242
573716993112,5cyclictest131rcu_preempt20:45:192
573716993111,5cyclictest131rcu_preempt21:57:302
573716993111,5cyclictest131rcu_preempt19:15:192
573716993111,5cyclictest131rcu_preempt19:15:192
573712993124,4cyclictest0-21swapper/123:55:271
573712993124,4cyclictest0-21swapper/123:55:261
573712993123,4cyclictest0-21swapper/121:42:451
573712993123,4cyclictest0-21swapper/121:10:021
573708993121,6cyclictest650990-21sh22:50:000
573708993120,8cyclictest615569-21unixbench_multi21:05:250
573708993120,6cyclictest0-21swapper/000:00:170
57371999307,11cyclictest40-21ksoftirqd/320:50:223
573719993017,7cyclictest657819-21kworker/u8:0+events_unbound23:16:233
573719993016,4cyclictest131rcu_preempt19:30:253
573719993016,10cyclictest609006-21basename20:50:013
573719993015,6cyclictest610932-21kworker/u8:2+events_unbound20:58:343
573719993013,4cyclictest131rcu_preempt21:00:213
573719993013,2cyclictest131rcu_preempt23:44:003
57371999301,17cyclictest644330-21fschecks_count22:30:173
573719993011,11cyclictest131rcu_preempt22:40:203
573716993013,7cyclictest643834-21kworker/u8:1+events_unbound22:36:062
573716993010,5cyclictest131rcu_preempt23:35:012
573716993010,12cyclictest32-21ksoftirqd/200:05:002
573716993010,11cyclictest131rcu_preempt21:05:152
573712993024,3cyclictest658031-21kthreadd23:10:001
573712993022,4cyclictest0-21swapper/120:15:011
573712993020,6cyclictest638408-21grep22:10:311
573712993015,9cyclictest671208-21grep23:45:251
573712993015,9cyclictest671208-21grep23:45:251
573708993024,4cyclictest586079-21kworker/u8:0+events_unbound20:27:330
573708993024,3cyclictest0-21swapper/000:40:010
573708993022,4cyclictest0-21swapper/021:20:280
573708993022,4cyclictest0-21swapper/020:15:190
573712992922,3cyclictest0-21swapper/122:40:001
573712992922,3cyclictest0-21swapper/100:30:271
573712992919,6cyclictest643505-21grep22:25:281
573708992921,5cyclictest0-21swapper/021:50:010
573708992920,5cyclictest0-21swapper/023:55:270
573708992920,5cyclictest0-21swapper/023:55:260
57371999289,12cyclictest131rcu_preempt22:50:163
573712992820,5cyclictest0-21swapper/120:25:271
573708992817,7cyclictest670762-21seq23:45:200
573708992817,7cyclictest670762-21seq23:45:200
573712992721,3cyclictest0-21swapper/100:40:011
573712992719,3cyclictest0-21swapper/120:17:311
573708992721,3cyclictest0-21swapper/020:25:020
573708992720,4cyclictest13650irq/42-30be0000.ethernet20:00:010
573708992720,3cyclictest0-21swapper/021:30:000
573708992719,4cyclictest0-21swapper/021:00:010
573708992719,4cyclictest0-21swapper/019:55:010
573708992718,5cyclictest0-21swapper/019:10:270
573708992716,8cyclictest654437-21http23:00:010
573708992716,7cyclictest0-21swapper/020:15:010
57370899271,4cyclictest131rcu_preempt19:45:110
573708992712,9cyclictest131rcu_preempt21:30:340
573712992619,3cyclictest0-21swapper/121:50:171
573712992618,5cyclictest0-21swapper/121:10:161
573708992618,5cyclictest0-21swapper/020:40:000
57370899261,6cyclictest652851-21latency_hist22:55:010
573712992516,4cyclictest0-21swapper/122:50:011
573712992515,6cyclictest672919-21grep23:50:261
573708992518,4cyclictest619120-21timerwakeupswit21:15:260
573708992518,3cyclictest0-21swapper/020:55:010
57370899251,20cyclictest0-21swapper/022:34:020
573712992418,4cyclictest299-21systemd-logind19:25:421
573712992418,3cyclictest0-21swapper/123:20:291
573712992417,4cyclictest0-21swapper/122:20:001
573712992417,3cyclictest0-21swapper/122:40:171
573712992412,8cyclictest603404-21grep20:30:311
573708992418,3cyclictest0-21swapper/023:00:180
573708992418,3cyclictest0-21swapper/020:30:290
573708992418,3cyclictest0-21swapper/019:25:000
573708992414,7cyclictest585107-21sed19:40:150
573708992413,8cyclictest641649-21sed22:20:270
57370899241,19cyclictest0-21swapper/023:28:340
57371299237,4cyclictest6258862uname21:35:251
573712992317,3cyclictest0-21swapper/120:37:341
573712992316,5cyclictest181-21systemd-udevd21:49:411
573712992315,5cyclictest0-21swapper/119:45:251
573712992315,4cyclictest0-21swapper/119:45:011
573712992313,7cyclictest595186-21sed20:10:011
573712992310,9cyclictest1-21systemd00:23:101
573708992318,3cyclictest0-21swapper/021:50:140
573708992316,4cyclictest606377-21ntpq20:40:250
573708992316,3cyclictest0-21swapper/020:00:300
573708992316,3cyclictest0-21swapper/020:00:290
573708992316,3cyclictest0-21swapper/019:35:240
573708992316,3cyclictest0-21swapper/000:26:020
573708992316,3cyclictest0-21swapper/000:10:180
573708992315,4cyclictest0-21swapper/023:15:180
573708992314,6cyclictest634048-21awk22:00:140
573708992314,5cyclictest581512-21grep19:30:180
573708992313,7cyclictest665650-21ntpq23:30:220
573708992313,7cyclictest576242-21cat19:15:140
573708992313,7cyclictest576242-21cat19:15:140
57370899231,18cyclictest0-21swapper/022:14:480
57370899231,18cyclictest0-21swapper/000:16:260
57370899230,19cyclictest0-21swapper/021:03:080
573712992216,3cyclictest0-21swapper/119:30:211
573712992216,3cyclictest0-21swapper/100:00:261
573712992215,5cyclictest656098-21kworker/u8:1+events_unbound23:33:041
573712992215,5cyclictest591167-21kworker/u8:2+events_unbound19:57:281
573712992215,3cyclictest0-21swapper/121:30:291
573712992214,4cyclictest0-21swapper/100:15:271
573712992212,6cyclictest0-21swapper/119:50:231
573708992216,3cyclictest0-21swapper/019:25:180
57370899221,19cyclictest0-21swapper/023:20:150
57370899221,18cyclictest678320-21sh00:08:010
57370899220,20cyclictest0-21swapper/023:40:210
57370899220,18cyclictest0-21swapper/023:07:040
573712992115,3cyclictest654855-21/usr/sbin/munin23:00:191
573712992115,3cyclictest0-21swapper/123:20:011
573712992115,3cyclictest0-21swapper/120:00:241
573712992115,3cyclictest0-21swapper/120:00:241
573712992114,4cyclictest584464-21cron19:40:001
573712992014,3cyclictest0-21swapper/121:00:161
573712992013,4cyclictest0-21swapper/122:20:281
573712992010,7cyclictest678879-21cut00:10:171
573712991914,3cyclictest578094-21fschecks_time19:20:161
573712991913,4cyclictest231ktimers/122:30:221
57371299191,11cyclictest0-21swapper/120:41:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional