You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-25 - 09:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot3.osadl.org (updated Thu Jul 25, 2024 00:46:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1364121770,8sleep11744399cyclictest21:20:141
168982123100,10sleep10-21swapper/119:06:381
15065211390,10sleep20-21swapper/219:05:022
8255911120,4ptp4l401ktimersoftd/319:07:553
57698990,15rtkit-daemon0-21swapper/019:08:520
238942610,6sleep023897-21df_abs00:00:160
170662580,2sleep10-21swapper/123:45:161
170662580,2sleep10-21swapper/123:45:151
17444995238,8cyclictest33-21ksoftirqd/219:10:122
189972510,5sleep233-21ksoftirqd/220:20:242
57698500,2rtkit-daemon0-21swapper/023:30:170
17444995035,8cyclictest33-21ksoftirqd/220:05:102
17443995028,19cyclictest25-21ksoftirqd/120:58:491
57698490,2rtkit-daemon0-21swapper/022:49:130
17444994843,3cyclictest33-21ksoftirqd/200:00:002
17443994625,5cyclictest25-21ksoftirqd/120:25:051
17444994530,8cyclictest33-21ksoftirqd/219:40:122
17445994427,14cyclictest41-21ksoftirqd/320:32:343
17444994439,3cyclictest33-21ksoftirqd/222:20:002
17444994429,9cyclictest33-21ksoftirqd/222:10:102
17444994429,9cyclictest33-21ksoftirqd/222:10:102
1744499441,40cyclictest22883-21cron22:50:002
17443994422,4cyclictest25-21ksoftirqd/122:17:551
17444994338,3cyclictest33-21ksoftirqd/222:30:002
17444994337,4cyclictest33-21ksoftirqd/223:40:012
17444994337,4cyclictest33-21ksoftirqd/219:40:012
17444994337,3cyclictest33-21ksoftirqd/221:20:012
17442994336,4cyclictest32380-21cron23:10:000
1744499426,10cyclictest0-21swapper/221:28:182
17444994236,3cyclictest33-21ksoftirqd/219:50:002
17443994224,4cyclictest2362-21llvmpipe-200:07:591
17445994113,4cyclictest9133-21expr21:10:163
17444994033,4cyclictest33-21ksoftirqd/219:20:232
17444994033,4cyclictest33-21ksoftirqd/219:20:222
17444994025,7cyclictest33-21ksoftirqd/220:45:112
17444994025,7cyclictest33-21ksoftirqd/220:45:102
17443994034,4cyclictest25-21ksoftirqd/100:35:001
17443994021,4cyclictest25-21ksoftirqd/123:16:241
17445993913,3cyclictest28852-21sed00:20:213
17444993933,4cyclictest33-21ksoftirqd/219:20:002
17444993933,4cyclictest33-21ksoftirqd/219:20:002
17444993932,4cyclictest33-21ksoftirqd/223:45:102
17444993932,4cyclictest33-21ksoftirqd/223:45:092
17443993933,3cyclictest25-21ksoftirqd/121:56:201
17442993932,4cyclictest23180-21kworker/0:120:45:000
17445993811,4cyclictest6030-21wc19:55:013
17444993832,3cyclictest33-21ksoftirqd/222:00:142
17444993830,6cyclictest33-21ksoftirqd/219:30:172
17444993824,8cyclictest33-21ksoftirqd/223:40:092
17444993824,8cyclictest33-21ksoftirqd/223:40:092
17444993823,8cyclictest33-21ksoftirqd/222:55:112
17443993832,4cyclictest25-21ksoftirqd/123:10:501
17443993831,4cyclictest25-21ksoftirqd/119:23:501
17443993831,4cyclictest25-21ksoftirqd/119:23:491
17445993731,3cyclictest41-21ksoftirqd/319:20:073
17445993731,3cyclictest41-21ksoftirqd/319:20:073
17445993730,4cyclictest41-21ksoftirqd/322:03:023
17445993729,4cyclictest41-21ksoftirqd/320:56:443
17445993714,3cyclictest26602-21latency_hist19:30:003
17445993713,4cyclictest18136-21bc21:30:103
1744599371,33cyclictest22883-21cron22:54:563
17444993731,4cyclictest33-21ksoftirqd/223:15:292
17444993730,4cyclictest33-21ksoftirqd/223:10:132
17444993730,4cyclictest33-21ksoftirqd/220:53:582
17443993731,4cyclictest25-21ksoftirqd/120:53:471
17443993731,3cyclictest25-21ksoftirqd/123:00:001
17443993731,3cyclictest25-21ksoftirqd/119:51:331
17443993730,4cyclictest25-21ksoftirqd/121:05:371
17443993730,4cyclictest25-21ksoftirqd/119:13:261
17442993731,4cyclictest9-21ksoftirqd/019:30:000
17442993731,3cyclictest9-21ksoftirqd/019:14:390
825591360,11ptp4l0-21swapper/300:13:453
1744599365,8cyclictest16778-21cpuspeed_turbos23:45:123
1744599365,8cyclictest16778-21cpuspeed_turbos23:45:123
17445993630,4cyclictest41-21ksoftirqd/300:34:463
17445993630,3cyclictest41-21ksoftirqd/322:18:353
17445993629,4cyclictest41-21ksoftirqd/322:55:343
17445993629,4cyclictest41-21ksoftirqd/320:53:253
17445993628,5cyclictest41-21ksoftirqd/323:20:593
17444993630,4cyclictest33-21ksoftirqd/220:35:182
17444993628,5cyclictest33-21ksoftirqd/222:09:092
17444993628,5cyclictest33-21ksoftirqd/200:17:482
1744399364,7cyclictest121rcu_preempt22:50:371
17443993629,4cyclictest25-21ksoftirqd/120:49:461
17443993629,4cyclictest25-21ksoftirqd/120:49:461
17443993629,4cyclictest25-21ksoftirqd/100:25:031
17442993631,3cyclictest9-21ksoftirqd/022:59:140
17442993630,3cyclictest9-21ksoftirqd/023:17:180
17442993629,4cyclictest9-21ksoftirqd/021:13:060
17442993628,5cyclictest9-21ksoftirqd/023:48:590
17442993628,5cyclictest9-21ksoftirqd/023:48:590
17442993628,5cyclictest9-21ksoftirqd/023:36:390
17442993628,5cyclictest9-21ksoftirqd/020:19:540
8952350,9sleep11744399cyclictest19:40:331
17445993530,3cyclictest41-21ksoftirqd/322:27:583
17445993529,3cyclictest41-21ksoftirqd/319:15:003
17445993528,4cyclictest41-21ksoftirqd/323:08:313
17445993528,4cyclictest41-21ksoftirqd/322:37:163
17445993528,4cyclictest41-21ksoftirqd/319:20:003
17445993528,4cyclictest41-21ksoftirqd/319:20:003
17445993527,5cyclictest41-21ksoftirqd/323:39:553
1744599352,3cyclictest121rcu_preempt23:55:403
17445993514,4cyclictest2126-21/usr/sbin/munin22:05:153
17445993514,4cyclictest13208-21/usr/sbin/munin20:10:233
17445993513,4cyclictest14897-21if_enp1s023:40:173
17445993513,4cyclictest14897-21if_enp1s023:40:163
17445993511,4cyclictest7877-21smartctl23:25:163
17445993511,4cyclictest22786-21latency_hist21:40:003
17444993529,4cyclictest33-21ksoftirqd/223:27:202
17444993529,4cyclictest33-21ksoftirqd/220:55:232
17444993529,3cyclictest33-21ksoftirqd/223:35:002
17444993529,3cyclictest33-21ksoftirqd/219:30:002
17444993528,4cyclictest33-21ksoftirqd/222:51:242
17444993528,4cyclictest33-21ksoftirqd/221:50:292
17444993528,4cyclictest33-21ksoftirqd/221:21:422
17444993522,6cyclictest33-21ksoftirqd/220:30:102
17444993521,8cyclictest33-21ksoftirqd/221:10:132
17443993530,3cyclictest25-21ksoftirqd/100:39:501
17443993529,4cyclictest25-21ksoftirqd/122:11:161
17443993529,4cyclictest25-21ksoftirqd/122:11:161
17443993529,4cyclictest25-21ksoftirqd/122:06:141
17443993528,4cyclictest25-21ksoftirqd/123:42:471
17443993528,4cyclictest25-21ksoftirqd/123:42:461
17443993528,4cyclictest25-21ksoftirqd/123:02:261
17443993528,4cyclictest25-21ksoftirqd/120:35:551
17443993528,4cyclictest25-21ksoftirqd/120:11:041
17442993529,3cyclictest9-21ksoftirqd/021:25:470
17442993529,3cyclictest9-21ksoftirqd/019:58:370
17442993529,3cyclictest9-21ksoftirqd/019:40:370
17442993528,4cyclictest9-21ksoftirqd/023:50:280
17442993528,4cyclictest9-21ksoftirqd/022:43:050
17442993528,4cyclictest9-21ksoftirqd/022:37:370
17442993528,4cyclictest9-21ksoftirqd/022:25:040
17442993528,4cyclictest9-21ksoftirqd/020:51:460
17442993528,4cyclictest9-21ksoftirqd/019:22:330
17442993528,4cyclictest9-21ksoftirqd/019:22:330
17442993527,5cyclictest9-21ksoftirqd/023:43:140
17442993527,5cyclictest9-21ksoftirqd/023:43:130
17442993526,6cyclictest9-21ksoftirqd/021:01:380
17445993428,4cyclictest41-21ksoftirqd/321:44:163
17445993428,4cyclictest41-21ksoftirqd/320:41:493
17445993427,4cyclictest41-21ksoftirqd/322:41:183
17445993427,4cyclictest41-21ksoftirqd/321:59:333
17445993427,4cyclictest41-21ksoftirqd/321:47:063
17445993427,4cyclictest41-21ksoftirqd/321:24:583
17445993427,4cyclictest41-21ksoftirqd/319:35:083
17445993414,4cyclictest4109-21/usr/sbin/munin21:00:203
17445993413,3cyclictest18871-21/usr/sbin/munin23:50:173
17444993429,3cyclictest33-21ksoftirqd/223:55:002
17444993429,3cyclictest33-21ksoftirqd/221:35:242
17444993429,3cyclictest33-21ksoftirqd/221:05:412
17444993428,4cyclictest33-21ksoftirqd/221:56:202
17444993428,4cyclictest33-21ksoftirqd/200:35:032
17444993428,3cyclictest33-21ksoftirqd/223:25:002
17444993427,4cyclictest33-21ksoftirqd/221:42:062
17444993427,4cyclictest33-21ksoftirqd/221:02:482
17444993427,4cyclictest33-21ksoftirqd/220:44:592
17444993427,4cyclictest33-21ksoftirqd/220:00:392
17443993428,4cyclictest25-21ksoftirqd/121:44:061
17443993428,4cyclictest25-21ksoftirqd/121:14:231
17443993428,4cyclictest25-21ksoftirqd/120:44:441
17443993428,4cyclictest25-21ksoftirqd/100:20:561
17443993428,4cyclictest25-21ksoftirqd/100:00:481
17443993428,3cyclictest25-21ksoftirqd/119:40:001
17443993427,4cyclictest25-21ksoftirqd/123:32:141
17443993427,4cyclictest25-21ksoftirqd/122:36:131
17443993427,4cyclictest25-21ksoftirqd/122:22:201
17443993427,4cyclictest25-21ksoftirqd/121:33:541
17443993427,4cyclictest25-21ksoftirqd/120:32:461
17443993427,4cyclictest25-21ksoftirqd/120:05:541
17443993427,4cyclictest25-21ksoftirqd/100:10:171
17442993429,3cyclictest9-21ksoftirqd/023:26:440
17442993428,3cyclictest9-21ksoftirqd/022:00:540
17442993427,4cyclictest9-21ksoftirqd/022:21:470
17442993427,4cyclictest9-21ksoftirqd/022:20:000
17442993427,4cyclictest9-21ksoftirqd/000:36:290
17442993426,5cyclictest9-21ksoftirqd/021:22:020
17442993426,5cyclictest9-21ksoftirqd/000:05:010
226023325,4sleep09-21ksoftirqd/019:45:230
17445993327,4cyclictest41-21ksoftirqd/322:32:223
17445993327,4cyclictest41-21ksoftirqd/320:39:573
17445993326,4cyclictest41-21ksoftirqd/323:17:403
17445993326,4cyclictest41-21ksoftirqd/323:04:073
17445993326,4cyclictest41-21ksoftirqd/322:24:483
17445993326,4cyclictest41-21ksoftirqd/320:45:433
17445993326,4cyclictest41-21ksoftirqd/320:45:423
17445993326,4cyclictest41-21ksoftirqd/319:49:063
17445993326,4cyclictest41-21ksoftirqd/319:30:093
17445993313,3cyclictest1100-21runrttasks00:28:133
17445993311,4cyclictest9103-21uniq20:00:163
17444993328,3cyclictest33-21ksoftirqd/222:35:142
17444993327,3cyclictest33-21ksoftirqd/223:05:202
17444993327,3cyclictest33-21ksoftirqd/221:31:492
17443993327,4cyclictest25-21ksoftirqd/123:26:151
17443993326,4cyclictest25-21ksoftirqd/122:47:221
17443993326,4cyclictest25-21ksoftirqd/122:31:401
17443993326,4cyclictest25-21ksoftirqd/119:48:041
17443993326,4cyclictest25-21ksoftirqd/119:30:091
17443993325,5cyclictest25-21ksoftirqd/122:29:031
17443993320,10cyclictest25-21ksoftirqd/100:19:231
17442993327,4cyclictest9-21ksoftirqd/021:53:290
17442993327,4cyclictest9-21ksoftirqd/021:06:230
17442993326,4cyclictest9-21ksoftirqd/022:07:390
17442993326,4cyclictest9-21ksoftirqd/000:12:510
17442993325,5cyclictest9-21ksoftirqd/021:37:020
17442993325,5cyclictest9-21ksoftirqd/021:30:440
17445993227,3cyclictest41-21ksoftirqd/323:31:383
17445993226,4cyclictest41-21ksoftirqd/321:53:363
17445993226,4cyclictest41-21ksoftirqd/300:18:343
17445993226,3cyclictest41-21ksoftirqd/320:28:313
17445993225,4cyclictest41-21ksoftirqd/322:14:283
17445993225,4cyclictest41-21ksoftirqd/322:14:273
17445993225,4cyclictest41-21ksoftirqd/319:56:243
17445993211,4cyclictest18301-21df_abs20:20:133
17445993211,4cyclictest12333-21grep21:15:313
17444993226,4cyclictest33-21ksoftirqd/223:00:102
17444993226,3cyclictest33-21ksoftirqd/200:30:002
17444993226,3cyclictest33-21ksoftirqd/200:22:532
17444993225,4cyclictest33-21ksoftirqd/221:45:102
17444993225,4cyclictest33-21ksoftirqd/220:25:532
17444993225,4cyclictest33-21ksoftirqd/219:51:022
17443993226,4cyclictest25-21ksoftirqd/121:52:081
17443993226,4cyclictest25-21ksoftirqd/119:25:101
17443993226,3cyclictest25-21ksoftirqd/121:46:451
17443993226,3cyclictest25-21ksoftirqd/121:25:381
17443993225,4cyclictest25-21ksoftirqd/123:06:321
17443993225,4cyclictest25-21ksoftirqd/121:19:581
17443993225,4cyclictest25-21ksoftirqd/119:59:551
17442993225,5cyclictest9-21ksoftirqd/020:57:240
17442993225,4cyclictest9-21ksoftirqd/022:30:590
17442993225,4cyclictest9-21ksoftirqd/021:19:440
17442993225,4cyclictest9-21ksoftirqd/020:30:260
17442993225,4cyclictest9-21ksoftirqd/020:01:580
17442993225,4cyclictest9-21ksoftirqd/019:17:360
17442993225,4cyclictest9-21ksoftirqd/019:17:350
17442993225,4cyclictest9-21ksoftirqd/000:30:270
17442993225,4cyclictest15291-21cron20:15:000
57698310,2rtkit-daemon0-21swapper/019:51:320
17445993125,4cyclictest41-21ksoftirqd/322:49:403
17445993124,4cyclictest41-21ksoftirqd/300:06:053
17444993126,3cyclictest33-21ksoftirqd/220:15:312
17443993125,3cyclictest25-21ksoftirqd/123:39:391
17443993125,3cyclictest25-21ksoftirqd/120:04:491
17443993124,4cyclictest25-21ksoftirqd/123:24:321
17443993124,4cyclictest25-21ksoftirqd/122:01:371
17443993124,4cyclictest25-21ksoftirqd/121:36:511
17442993124,4cyclictest9-21ksoftirqd/023:57:310
17442993124,4cyclictest9-21ksoftirqd/023:21:290
17442993124,4cyclictest9-21ksoftirqd/022:13:170
17442993124,4cyclictest9-21ksoftirqd/022:13:160
1444323124,3sleep233-21ksoftirqd/222:30:172
17445993024,4cyclictest41-21ksoftirqd/323:13:013
17445993024,3cyclictest41-21ksoftirqd/321:25:423
17445993023,4cyclictest41-21ksoftirqd/319:44:593
17445993023,4cyclictest41-21ksoftirqd/300:35:443
17444993025,3cyclictest33-21ksoftirqd/200:11:212
17444993023,4cyclictest33-21ksoftirqd/220:10:112
17444993023,4cyclictest33-21ksoftirqd/200:00:222
17444993017,4cyclictest33-21ksoftirqd/222:40:162
17443993024,4cyclictest25-21ksoftirqd/123:50:531
17443993023,4cyclictest25-21ksoftirqd/120:22:161
17442993023,4cyclictest9-21ksoftirqd/020:47:120
17442993023,4cyclictest9-21ksoftirqd/020:47:120
17445992923,4cyclictest41-21ksoftirqd/320:06:143
17445992922,4cyclictest41-21ksoftirqd/321:08:553
17445992922,4cyclictest41-21ksoftirqd/320:18:293
17444992923,4cyclictest33-21ksoftirqd/219:55:072
17444992922,4cyclictest33-21ksoftirqd/222:24:542
17444992922,4cyclictest33-21ksoftirqd/200:34:302
17444992922,4cyclictest33-21ksoftirqd/200:05:122
17443992923,4cyclictest25-21ksoftirqd/123:57:441
17443992923,3cyclictest25-21ksoftirqd/119:15:091
17443992923,3cyclictest25-21ksoftirqd/119:15:081
17442992923,4cyclictest9-21ksoftirqd/021:42:530
17442992922,4cyclictest9-21ksoftirqd/023:04:190
17442992921,5cyclictest9-21ksoftirqd/023:10:130
17445992822,4cyclictest41-21ksoftirqd/300:00:593
17442992823,4cyclictest9-21ksoftirqd/000:17:570
17442992823,3cyclictest9-21ksoftirqd/020:21:260
17442992822,3cyclictest9-21ksoftirqd/022:51:490
17442992821,4cyclictest9-21ksoftirqd/020:40:000
17442992820,4cyclictest21835-21grep20:25:300
17443992722,3cyclictest25-21ksoftirqd/122:44:591
17443992721,4cyclictest25-21ksoftirqd/120:19:581
17443992720,4cyclictest25-21ksoftirqd/121:03:271
17442992721,4cyclictest9-21ksoftirqd/020:05:230
17442992721,4cyclictest9-21ksoftirqd/000:20:230
17442992721,3cyclictest9-21ksoftirqd/019:31:510
17442992720,4cyclictest9-21ksoftirqd/021:55:120
17442992720,4cyclictest9-21ksoftirqd/021:47:190
17442992620,3cyclictest9-21ksoftirqd/019:37:050
17442992418,3cyclictest9-21ksoftirqd/000:27:580
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional