You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-23 - 23:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6.osadl.org (updated Sat Sep 23, 2023 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1811240,0irq_work/0161rcuc/010:44:150
1811240,0irq_work/0161rcuc/010:44:150
2911170,0irq_work/2311rcuc/209:23:432
1811150,0irq_work/0161rcuc/011:58:420
1811150,0irq_work/0161rcuc/011:58:420
1811150,0irq_work/0161rcuc/010:28:430
2911140,0irq_work/2311rcuc/210:46:182
2911140,0irq_work/2311rcuc/210:46:182
2911130,0irq_work/2311rcuc/209:48:432
2911110,0irq_work/2311rcuc/212:14:112
1811110,0irq_work/0161rcuc/008:48:430
2911100,0irq_work/2311rcuc/210:01:462
2911090,0irq_work/2311rcuc/211:00:312
2911090,0irq_work/2311rcuc/208:07:542
2911080,0irq_work/2311rcuc/211:34:382
2911080,0irq_work/2311rcuc/209:03:232
1811080,0irq_work/0161rcuc/011:28:430
1811080,0irq_work/0161rcuc/009:24:200
1811080,0irq_work/0161rcuc/008:04:130
2911070,0irq_work/2311rcuc/212:28:432
2911070,0irq_work/2311rcuc/212:07:272
2911070,0irq_work/2311rcuc/211:14:012
2911070,0irq_work/2311rcuc/209:09:122
1811070,0irq_work/0161rcuc/012:34:180
2911060,0irq_work/2311rcuc/212:02:032
2911060,0irq_work/2311rcuc/212:02:032
1811060,0irq_work/0161rcuc/009:18:430
2911050,0irq_work/2311rcuc/211:49:472
2911050,0irq_work/2311rcuc/210:39:272
2911050,0irq_work/2311rcuc/210:13:442
2911050,0irq_work/2311rcuc/210:06:172
2911050,0irq_work/2311rcuc/209:20:062
2911050,0irq_work/2311rcuc/208:38:042
2911050,0irq_work/2311rcuc/208:25:402
2911050,0irq_work/2311rcuc/207:51:232
2911050,0irq_work/2311rcuc/207:42:052
2911040,0irq_work/2311rcuc/211:44:472
2911040,0irq_work/2311rcuc/211:42:232
2911040,0irq_work/2311rcuc/209:56:492
2911040,0irq_work/2311rcuc/209:04:052
2911030,0irq_work/2311rcuc/212:10:362
2911030,0irq_work/2311rcuc/211:12:042
2911030,0irq_work/2311rcuc/208:22:252
1811030,0irq_work/0161rcuc/010:15:090
2911020,0irq_work/2311rcuc/212:28:342
2911020,0irq_work/2311rcuc/211:07:432
2911020,0irq_work/2311rcuc/209:46:432
2911020,0irq_work/2311rcuc/207:32:352
1811020,0irq_work/0161rcuc/007:54:190
2911010,0irq_work/2311rcuc/209:41:072
2911010,0irq_work/2311rcuc/209:41:072
2911010,0irq_work/2311rcuc/207:13:102
1811010,0irq_work/0161rcuc/011:34:190
1811010,0irq_work/0161rcuc/007:39:160
1811010,0irq_work/0161rcuc/007:15:090
2911000,0irq_work/2311rcuc/212:38:402
2911000,0irq_work/2311rcuc/211:22:252
2911000,0irq_work/2311rcuc/210:13:372
2911000,0irq_work/2311rcuc/208:54:172
2911000,0irq_work/2311rcuc/208:38:432
2911000,0irq_work/2311rcuc/208:32:432
2911000,0irq_work/2311rcuc/208:32:432
2911000,0irq_work/2311rcuc/207:44:322
1811000,0irq_work/0161rcuc/008:18:420
291990,0irq_work/2311rcuc/209:32:002
291990,0irq_work/2311rcuc/207:53:592
181990,0irq_work/0161rcuc/009:19:180
181990,0irq_work/0161rcuc/008:45:090
291980,0irq_work/2311rcuc/211:57:122
291980,0irq_work/2311rcuc/211:57:122
291980,0irq_work/2311rcuc/210:56:542
291980,0irq_work/2311rcuc/210:27:022
291980,0irq_work/2311rcuc/208:47:322
291980,0irq_work/2311rcuc/208:13:362
181980,0irq_work/0161rcuc/009:34:150
291970,0irq_work/2311rcuc/212:21:002
291970,0irq_work/2311rcuc/211:29:172
291970,0irq_work/2311rcuc/210:51:032
291970,0irq_work/2311rcuc/210:51:032
291970,0irq_work/2311rcuc/209:16:482
291970,0irq_work/2311rcuc/208:02:362
291970,0irq_work/2311rcuc/207:25:342
181970,0irq_work/0161rcuc/007:38:430
291960,0irq_work/2311rcuc/210:29:182
291960,0irq_work/2311rcuc/210:20:112
291960,0irq_work/2311rcuc/208:14:122
291960,0irq_work/2311rcuc/207:21:222
291960,0irq_work/2311rcuc/207:21:222
181960,0irq_work/0161rcuc/009:49:190
181960,0irq_work/0161rcuc/009:44:180
181960,0irq_work/0161rcuc/008:23:420
291950,0irq_work/2311rcuc/208:51:532
181950,0irq_work/0161rcuc/007:09:160
181940,0irq_work/0161rcuc/007:28:430
291930,0irq_work/2311rcuc/211:25:162
291930,0irq_work/2311rcuc/207:35:022
291930,0irq_work/2311rcuc/207:15:432
181930,0irq_work/0161rcuc/010:38:420
181920,0irq_work/0161rcuc/012:14:160
181910,0irq_work/0161rcuc/008:38:420
291900,0irq_work/2311rcuc/209:35:482
291880,0irq_work/2311rcuc/210:38:092
181880,0irq_work/0161rcuc/012:19:190
181880,0irq_work/0161rcuc/011:29:060
181880,0irq_work/0161rcuc/011:23:420
181850,0irq_work/0161rcuc/011:04:170
181850,0irq_work/0161rcuc/010:19:060
181850,0irq_work/0161rcuc/008:13:430
181850,0irq_work/0161rcuc/007:43:430
181840,0irq_work/0161rcuc/011:03:370
181840,0irq_work/0161rcuc/009:29:180
181830,0irq_work/0161rcuc/011:39:110
181830,0irq_work/0161rcuc/010:43:370
181830,0irq_work/0161rcuc/009:09:030
181830,0irq_work/0161rcuc/008:58:430
181820,0irq_work/0161rcuc/012:24:190
181820,0irq_work/0161rcuc/010:03:370
181820,0irq_work/0161rcuc/008:33:370
181820,0irq_work/0161rcuc/008:33:370
361810,0irq_work/3381rcuc/309:47:073
181810,0irq_work/0161rcuc/010:29:070
181810,0irq_work/0161rcuc/010:04:180
181800,0irq_work/0161rcuc/009:43:420
181800,0irq_work/0161rcuc/009:43:420
181800,0irq_work/0161rcuc/008:28:370
181790,0irq_work/0161rcuc/010:58:370
181780,0irq_work/0161rcuc/011:59:090
181780,0irq_work/0161rcuc/011:59:090
181780,0irq_work/0161rcuc/009:58:420
181780,0irq_work/0161rcuc/007:49:070
181770,0irq_work/0161rcuc/012:29:070
181770,0irq_work/0161rcuc/011:13:430
181740,0irq_work/0161rcuc/008:43:370
181740,0irq_work/0161rcuc/007:59:210
221730,0irq_work/1241rcuc/108:19:171
181730,0irq_work/0161rcuc/007:19:080
181730,0irq_work/0161rcuc/007:19:080
181710,0irq_work/0161rcuc/012:08:420
181710,0irq_work/0161rcuc/010:53:370
181710,0irq_work/0161rcuc/010:53:370
2616899708,0cyclictest3799migration/309:52:153
181700,0irq_work/0161rcuc/011:44:150
181680,0irq_work/0161rcuc/011:09:160
181670,0irq_work/0161rcuc/011:49:130
2616899653,0cyclictest3799migration/312:09:583
2616899648,0cyclictest3799migration/307:45:073
181640,0irq_work/0161rcuc/009:04:050
181640,0irq_work/0161rcuc/009:03:430
141640,0rcu_preempt39-21ksoftirqd/310:09:103
26168996315,0cyclictest18228-21load09:14:103
181630,0irq_work/0161rcuc/010:08:430
2616899625,0cyclictest3799migration/307:20:313
2616899625,0cyclictest3799migration/307:20:313
181620,0irq_work/0161rcuc/007:29:170
2616899614,0cyclictest3799migration/312:29:573
2616899612,0cyclictest4674-21taskset09:57:043
221610,0irq_work/1241rcuc/110:19:171
221610,0irq_work/1241rcuc/109:59:171
181610,0irq_work/0161rcuc/012:09:020
2616899606,0cyclictest3799migration/310:29:143
2616899605,0cyclictest3799migration/311:53:423
2616899602,0cyclictest3799migration/308:14:033
2616899602,0cyclictest0-21swapper/312:34:063
2616899601,0cyclictest3147-21cpuspeed_turbos07:29:043
141600,0rcu_preempt25029-21grep07:04:063
2616899597,0cyclictest3799migration/310:04:143
2616899596,0cyclictest3799migration/310:19:003
141590,0rcu_preempt39-21ksoftirqd/307:37:533
221580,0irq_work/1241rcuc/111:54:161
221580,0irq_work/1241rcuc/111:54:161
141580,0rcu_preempt39-21ksoftirqd/311:54:183
141580,0rcu_preempt39-21ksoftirqd/311:54:183
2616899575,0cyclictest3799migration/311:44:003
2616899574,0cyclictest3799migration/312:24:183
2616899573,0cyclictest3799migration/311:06:023
221570,0irq_work/1241rcuc/108:38:371
141570,0rcu_preempt39-21ksoftirqd/311:24:133
141570,0rcu_preempt20890-21tr10:34:033
2616899563,0cyclictest3799migration/312:14:233
2616899562,0cyclictest1106-21nscd09:04:183
141560,0rcu_preempt8475-21wc07:39:163
141560,0rcu_preempt15705-21sed11:34:073
361550,0irq_work/3381rcuc/308:38:373
361550,0irq_work/3381rcuc/308:09:253
2616899552,0cyclictest28503-21aten_r1power_cu07:14:013
2616899552,0cyclictest27754-21fschecks_count10:49:073
2616899552,0cyclictest27754-21fschecks_count10:49:073
2615399552,0cyclictest218599rtkit-daemon07:59:131
141550,0rcu_preempt39-21ksoftirqd/308:49:233
2616899544,0cyclictest3799migration/307:49:283
2616899543,0cyclictest3799migration/307:24:503
2616899541,0cyclictest13400-21runrttasks10:14:433
141540,0rcu_preempt25678-21gltestperf10:44:053
141540,0rcu_preempt25678-21gltestperf10:44:053
3099530,0migration/225914-21taskset07:07:462
2616899535,0cyclictest3799migration/308:39:133
2616899533,0cyclictest3799migration/309:30:173
141530,0rcu_preempt9555-21wc08:54:183
141530,0rcu_preempt39-21ksoftirqd/309:34:073
141530,0rcu_preempt39-21ksoftirqd/309:24:173
2616899522,0cyclictest27420-21aten_r1power_fr08:24:033
2616899521,0cyclictest708-21timerandwakeup10:59:233
141520,0rcu_preempt10633-21aten_r1power_cu08:59:023
2616899513,0cyclictest20539-21missed_timers09:19:143
141510,0rcu_preempt999-21kworker/3:211:09:183
141510,0rcu_preempt39-21ksoftirqd/311:43:423
141510,0rcu_preempt39-21ksoftirqd/311:14:273
2616899502,0cyclictest26759-21fschecks_time07:09:053
2615399504,0cyclictest2399migration/108:15:261
2616899493,0cyclictest29460-21needreboot09:39:153
2616899493,0cyclictest29460-21needreboot09:39:153
2616899492,0cyclictest0-21swapper/312:04:123
2616899492,0cyclictest0-21swapper/311:29:183
2616899491,0cyclictest30554-21needreboot10:54:163
221490,0irq_work/1241rcuc/112:29:161
141490,0rcu_preempt39-21ksoftirqd/311:59:003
141490,0rcu_preempt39-21ksoftirqd/311:59:003
141490,0rcu_preempt39-21ksoftirqd/309:59:133
141490,0rcu_preempt39-21ksoftirqd/308:04:043
141490,0rcu_preempt39-21ksoftirqd/308:03:433
141490,0rcu_preempt25565-21fschecks_count08:19:063
141490,0rcu_preempt23497-21sed10:39:053
2616899482,0cyclictest4383-21fschecks_count08:44:073
2616899482,0cyclictest16269-21tail10:24:033
2616899482,0cyclictest15643-21fschecks_time09:09:063
2615399482,0cyclictest218599rtkit-daemon08:54:241
2615399481,0cyclictest218599rtkit-daemon11:01:111
221480,0irq_work/1241rcuc/109:19:181
141480,0rcu_preempt26169-21hostname11:58:591
141480,0rcu_preempt26169-21hostname11:58:591
2616899472,0cyclictest8617-21cpuspeed_turbos11:19:033
2616899472,0cyclictest14282-21cpuspeed_turbos07:54:053
2615399475,0cyclictest2399migration/108:44:081
2615399473,0cyclictest218599rtkit-daemon08:41:341
241470,0rcuc/124039-21munin-node07:03:561
141470,0rcu_preempt25-21ksoftirqd/107:39:051
2616899462,0cyclictest4297-21unixbench-2d12:19:233
2616899462,0cyclictest30139-21fschecks_time08:29:073
2616899462,0cyclictest30139-21fschecks_time08:29:073
221460,0irq_work/1241rcuc/110:49:181
221460,0irq_work/1241rcuc/110:49:181
161460,0rcuc/024971-21munin-node07:04:050
2615399451,0cyclictest218599rtkit-daemon10:54:221
2615399451,0cyclictest218599rtkit-daemon10:10:351
2615399442,0cyclictest218599rtkit-daemon08:53:421
2615399441,0cyclictest218599rtkit-daemon11:25:491
2615399441,0cyclictest218599rtkit-daemon08:27:431
2615399433,0cyclictest2399migration/111:44:101
2615399431,0cyclictest218599rtkit-daemon12:17:561
2615399431,0cyclictest218599rtkit-daemon10:15:061
2615399431,0cyclictest218599rtkit-daemon08:30:281
2615399431,0cyclictest218599rtkit-daemon08:30:281
221430,0irq_work/1241rcuc/107:53:141
2615399421,0cyclictest218599rtkit-daemon09:29:511
2615399421,0cyclictest218599rtkit-daemon08:10:151
2615399421,0cyclictest218599rtkit-daemon07:18:491
2615399421,0cyclictest218599rtkit-daemon07:18:491
221420,0irq_work/1241rcuc/111:39:111
2615399411,0cyclictest218599rtkit-daemon10:04:461
2615399411,0cyclictest218599rtkit-daemon09:53:441
141410,0rcu_preempt25-21ksoftirqd/108:59:101
2615399401,0cyclictest218599rtkit-daemon09:39:051
2615399401,0cyclictest218599rtkit-daemon09:39:051
2615399401,0cyclictest218599rtkit-daemon09:37:271
2615399401,0cyclictest218599rtkit-daemon09:09:331
221400,0irq_work/1241rcuc/110:37:531
2615399395,0cyclictest2399migration/111:04:081
2615399391,0cyclictest218599rtkit-daemon11:10:431
2615399391,0cyclictest218599rtkit-daemon07:24:531
141390,0rcu_preempt25-21ksoftirqd/112:09:201
141390,0rcu_preempt25-21ksoftirqd/107:14:041
2615399381,0cyclictest218599rtkit-daemon10:27:331
221380,0irq_work/1241rcuc/107:57:001
2615399375,0cyclictest2399migration/107:44:061
2615399374,0cyclictest2399migration/109:04:251
26153993726,0cyclictest22457-21hddtemp_smartct09:24:111
2615399371,0cyclictest218599rtkit-daemon11:22:021
241370,0rcuc/110062-21tail12:34:071
221370,0irq_work/1241rcuc/110:44:141
221370,0irq_work/1241rcuc/110:44:141
221370,0irq_work/1241rcuc/109:44:061
2615399365,0cyclictest2399migration/109:49:001
141360,0rcu_preempt22562-21cat11:49:081
2615399354,0cyclictest2399migration/107:09:011
221350,0irq_work/1241rcuc/107:38:281
2615399342,0cyclictest2399migration/112:24:051
2615399342,0cyclictest2399migration/111:34:231
221340,0irq_work/1241rcuc/108:04:141
141340,0rcu_preempt25-21ksoftirqd/110:39:041
141340,0rcu_preempt241rcuc/110:29:061
2615399333,0cyclictest2399migration/111:15:161
2615399332,0cyclictest2399migration/112:05:231
241330,0rcuc/118836-21grep09:14:211
2615399322,0cyclictest4645-21sshd12:22:111
221290,0irq_work/1141rcu_preempt11:29:021
141290,0rcu_preempt25-21ksoftirqd/107:28:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional