You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-19 - 06:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Sat Jul 19, 2025 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3665521550,143pltrace0-21swapper/319:10:003
3665521380,108pltrace0-21swapper/719:05:567
796521250,108pltrace0-21swapper/519:05:165
79652870,74pltrace0-21swapper/119:05:121
369927359,10sleep60-21swapper/619:06:156
368226841,7sleep00-21swapper/019:06:020
363226248,10sleep20-21swapper/219:05:402
265125842,11sleep40-21swapper/419:05:174
3120752280,4pltrace0-21swapper/723:50:247
274712280,1sleep20-21swapper/220:45:212
2006152270,3pltrace0-21swapper/723:30:147
391852250,0pltrace0-21swapper/721:05:207
412599230,12cyclictest0-21swapper/300:19:373
1680552210,13pltrace0-21swapper/620:30:246
565552200,4pltrace0-21swapper/720:10:027
1680552200,4pltrace0-21swapper/720:30:197
1062252200,3pltrace0-21swapper/722:13:257
1340652190,1pltrace0-21swapper/722:19:317
1000652190,15pltrace0-21swapper/100:06:181
440852180,0pltrace0-21swapper/700:00:137
330452180,3pltrace0-21swapper/722:57:097
109652180,1pltrace2324-21expr21:00:147
414199171,9cyclictest0-21swapper/621:34:246
4134991714,1cyclictest0-21swapper/522:12:165
4125991713,2cyclictest0-21swapper/321:24:073
4125991711,5cyclictest0-21swapper/300:27:453
412599170,12cyclictest0-21swapper/319:36:493
283752170,2pltrace0-21swapper/720:05:217
2673252170,5pltrace0-21swapper/700:38:047
2237552170,0pltrace0-21swapper/720:40:157
2115652170,0pltrace0-21swapper/700:30:247
1959052170,3pltrace0-21swapper/720:35:077
1789752170,11pltrace0-21swapper/721:26:127
953852160,10pltrace10218-21/usr/sbin/munin21:15:177
4146991611,4cyclictest4199-21sort23:00:117
414199161,14cyclictest0-21swapper/623:40:516
414199160,8cyclictest0-21swapper/622:23:066
413499161,4cyclictest0-21swapper/519:39:085
413499160,8cyclictest0-21swapper/523:44:535
412599161,8cyclictest0-21swapper/323:20:593
412599161,8cyclictest0-21swapper/300:05:243
412599161,13cyclictest0-21swapper/320:36:413
412599160,8cyclictest0-21swapper/323:59:213
412599160,2cyclictest0-21swapper/322:20:003
412099160,8cyclictest0-21swapper/223:35:082
412099160,8cyclictest0-21swapper/222:55:392
4116991610,5cyclictest0-21swapper/121:55:231
4116991610,5cyclictest0-21swapper/121:55:231
411699161,4cyclictest0-21swapper/123:26:011
411699161,4cyclictest0-21swapper/100:04:271
411699161,3cyclictest0-21swapper/122:49:511
3183652160,4pltrace0-21swapper/721:55:287
3183652160,4pltrace0-21swapper/721:55:287
1619452160,8pltrace17122-21cpuspeed_turbos22:25:106
1558252160,0pltrace0-21swapper/700:16:337
1340652160,2pltrace0-21swapper/722:20:147
1062252160,9pltrace12011-21uniq22:15:142
1062252160,4pltrace11837-21/usr/sbin/munin22:15:126
414699150,8cyclictest0-21swapper/723:17:137
414699150,7cyclictest0-21swapper/722:34:397
414699150,7cyclictest0-21swapper/719:23:067
414699150,2cyclictest0-21swapper/720:45:227
414699150,14cyclictest0-21swapper/721:23:127
414699150,13cyclictest0-21swapper/721:14:277
414199151,3cyclictest0-21swapper/600:20:496
414199150,14cyclictest0-21swapper/623:02:466
414199150,14cyclictest0-21swapper/622:44:546
413499150,8cyclictest0-21swapper/521:24:495
413499150,7cyclictest0-21swapper/519:57:045
413499150,3cyclictest0-21swapper/519:14:355
413499150,14cyclictest0-21swapper/521:46:225
413499150,14cyclictest0-21swapper/521:41:195
412599159,5cyclictest0-21swapper/319:50:183
4125991510,2cyclictest0-21swapper/322:40:563
412599150,8cyclictest0-21swapper/322:30:503
412599150,7cyclictest0-21swapper/321:39:533
412599150,7cyclictest0-21swapper/321:00:383
412599150,14cyclictest0-21swapper/321:16:523
412599150,14cyclictest0-21swapper/319:46:583
412099150,8cyclictest0-21swapper/221:00:062
412099150,8cyclictest0-21swapper/219:19:462
412099150,7cyclictest0-21swapper/221:37:472
412099150,7cyclictest0-21swapper/200:27:432
412099150,13cyclictest0-21swapper/222:36:082
412099150,13cyclictest0-21swapper/200:06:392
411699159,5cyclictest0-21swapper/120:10:561
411699150,8cyclictest0-21swapper/121:30:541
411699150,7cyclictest0-21swapper/121:02:211
411699150,14cyclictest0-21swapper/123:52:571
411699150,14cyclictest0-21swapper/122:57:001
2687152150,2pltrace0-21swapper/719:46:327
2129652150,4pltrace1132-21snmpd19:38:127
1727652150,5pltrace0-21swapper/623:25:226
1062252150,9pltrace12163-21grep22:15:135
891452140,0pltrace0-21swapper/723:10:017
610552140,11pltrace0-21swapper/023:01:360
502752140,2pltrace1132-21snmpd22:03:407
440852140,9pltrace5090-21/usr/sbin/munin00:00:206
440852140,4pltrace6073-21wget00:00:163
4146991411,2cyclictest1132-21snmpd19:51:287
414699140,7cyclictest0-21swapper/723:49:267
414699140,7cyclictest0-21swapper/721:32:547
414699140,7cyclictest0-21swapper/720:01:567
414699140,13cyclictest0-21swapper/723:29:597
414699140,13cyclictest0-21swapper/700:29:497
4141991412,1cyclictest0-21swapper/619:35:506
4141991411,2cyclictest0-21swapper/600:33:426
414199140,8cyclictest0-21swapper/623:15:326
414199140,7cyclictest0-21swapper/621:21:216
4134991413,1cyclictest0-21swapper/523:45:555
413499140,6cyclictest0-21swapper/500:12:115
413499140,3cyclictest0-21swapper/523:57:435
4129991411,2cyclictest28941-21head22:45:154
412599141,12cyclictest0-21swapper/320:57:573
412599140,7cyclictest0-21swapper/320:27:043
412599140,7cyclictest0-21swapper/319:59:463
412599140,6cyclictest0-21swapper/320:18:383
412099140,7cyclictest0-21swapper/222:44:412
412099140,7cyclictest0-21swapper/222:10:012
412099140,7cyclictest0-21swapper/221:05:212
412099140,7cyclictest0-21swapper/220:15:462
412099140,7cyclictest0-21swapper/219:56:392
412099140,6cyclictest0-21swapper/222:28:172
412099140,13cyclictest0-21swapper/222:00:082
412099140,12cyclictest0-21swapper/221:51:592
411699141,12cyclictest0-21swapper/119:40:461
411699140,2cyclictest0-21swapper/120:45:201
411699140,1cyclictest0-21swapper/120:25:201
3012552140,0pltrace0-21swapper/722:48:367
283752140,5pltrace0-21swapper/320:03:173
2347052140,0pltrace0-21swapper/721:40:107
2284752140,0pltrace0-21swapper/723:35:137
1680552140,1pltrace0-21swapper/320:30:213
1279352140,2pltrace12886-21sshd00:12:072
891452130,4pltrace0-21swapper/723:06:217
502752130,6pltrace0-21swapper/622:03:436
440852130,1pltrace0-21swapper/200:00:202
4146991312,1cyclictest0-21swapper/723:40:207
414699130,12cyclictest0-21swapper/721:47:387
414199135,6cyclictest1028-13auditd00:09:596
414199130,12cyclictest0-21swapper/623:39:106
414199130,12cyclictest0-21swapper/622:45:076
414199130,12cyclictest0-21swapper/621:52:226
414199130,12cyclictest0-21swapper/620:15:096
414199130,12cyclictest0-21swapper/619:48:546
413499139,2cyclictest0-21swapper/521:30:135
4134991312,1cyclictest0-21swapper/523:50:075
4134991312,1cyclictest0-21swapper/523:26:105
4134991312,1cyclictest0-21swapper/519:30:255
4134991312,1cyclictest0-21swapper/500:25:155
413499130,12cyclictest0-21swapper/523:31:205
413499130,1cyclictest0-21swapper/522:51:225
4129991310,2cyclictest19318-21tune2fs21:30:144
412599137,5cyclictest0-21swapper/322:24:233
412599130,7cyclictest0-21swapper/320:11:383
412599130,1cyclictest0-21swapper/320:22:083
412599130,1cyclictest0-21swapper/300:30:003
412099137,3cyclictest0-21swapper/221:33:192
412099137,3cyclictest0-21swapper/219:47:132
4120991312,1cyclictest0-21swapper/219:53:092
4120991311,1cyclictest0-21swapper/220:05:092
412099130,6cyclictest0-21swapper/219:41:492
412099130,12cyclictest0-21swapper/223:33:112
412099130,12cyclictest0-21swapper/222:22:122
412099130,1cyclictest0-21swapper/219:10:532
411699132,7cyclictest0-21swapper/120:15:201
411699130,12cyclictest0-21swapper/120:54:561
411699130,12cyclictest0-21swapper/100:23:041
3120752130,5pltrace0-21swapper/223:48:432
2455252130,2pltrace0-21swapper/722:38:007
2347052130,0pltrace0-21swapper/721:36:167
1837252130,6pltrace0-21swapper/200:24:112
1572452130,5pltrace0-21swapper/719:28:097
1170152130,6pltrace0-21swapper/623:14:166
844652120,3pltrace9301-21grep20:15:127
610552120,5pltrace0-21swapper/523:02:345
440852120,3pltrace5090-21/usr/sbin/munin00:00:185
4146991211,1cyclictest0-21swapper/700:05:267
414699120,11cyclictest0-21swapper/722:42:417
414699120,11cyclictest0-21swapper/720:28:457
414699120,11cyclictest0-21swapper/700:10:537
414199127,2cyclictest0-21swapper/623:47:026
414199126,5cyclictest0-21swapper/623:32:226
414199123,8cyclictest0-21swapper/619:30:006
4141991211,1cyclictest0-21swapper/600:15:176
414199120,11cyclictest0-21swapper/623:55:446
413499125,4cyclictest14809-21cpu20:25:145
413499123,8cyclictest0-21swapper/500:31:485
4134991211,1cyclictest0-21swapper/520:44:165
4134991211,1cyclictest0-21swapper/519:15:415
413499120,12cyclictest0-21swapper/522:30:375
413499120,12cyclictest0-21swapper/520:49:565
413499120,11cyclictest0-21swapper/523:06:355
412999129,2cyclictest13784-21ntpdc23:15:194
412999128,3cyclictest54-21ksoftirqd/421:58:384
412999128,3cyclictest54-21ksoftirqd/421:58:384
412999127,4cyclictest13724-21tr00:15:124
412999125,6cyclictest0-21swapper/421:30:014
412999124,5cyclictest0-21swapper/421:15:124
4129991211,1cyclictest0-21swapper/420:20:214
4129991211,1cyclictest0-21swapper/400:26:084
412999120,11cyclictest0-21swapper/421:10:434
412999120,1cyclictest0-21swapper/423:21:584
412599126,5cyclictest0-21swapper/322:14:383
4125991211,1cyclictest0-21swapper/319:30:203
4125991210,1cyclictest0-21swapper/320:54:113
412099127,3cyclictest0-21swapper/223:08:212
4120991211,1cyclictest0-21swapper/223:29:072
412099121,11cyclictest0-21swapper/221:42:052
412099120,11cyclictest0-21swapper/223:51:242
412099120,11cyclictest0-21swapper/220:10:202
412099120,1cyclictest0-21swapper/200:15:072
411699129,2cyclictest0-21swapper/100:18:251
411699126,5cyclictest0-21swapper/121:39:011
411699121,7cyclictest0-21swapper/121:40:121
411699121,5cyclictest20900-21cpuspeed_turbos23:30:121
411699120,5cyclictest0-21swapper/100:36:561
411299129,2cyclictest8627-21fschecks_count19:15:140
411299129,2cyclictest17129-21perl00:20:150
411299129,2cyclictest15214-21expr20:25:150
4112991211,1cyclictest0-21swapper/000:05:230
4112991210,1cyclictest0-21swapper/022:46:040
411299120,2cyclictest7300-21kworker/0:0+events@
delayed_vfree_work
19:30:010
411299120,11cyclictest0-21swapper/019:15:010
411299120,1cyclictest0-21swapper/023:21:570
391852120,5pltrace0-21swapper/521:02:555
3073652120,4pltrace0-21swapper/520:52:175
3012552120,5pltrace0-21swapper/322:50:363
3012552120,0pltrace0-21swapper/722:50:097
2516552120,5pltrace0-21swapper/620:41:196
2455252120,4pltrace0-21swapper/322:38:163
2394652120,5pltrace0-21swapper/200:35:342
2394652120,1pltrace0-21swapper/600:35:236
1789752120,4pltrace0-21swapper/221:27:132
1680552120,1pltrace0-21swapper/220:30:232
1015152120,0pltrace0-21swapper/719:19:587
652552110,3pltrace9893-21timerwakeupswit19:15:226
414699116,3cyclictest0-21swapper/722:29:127
414699112,8cyclictest0-21swapper/723:22:127
414699110,11cyclictest0-21swapper/719:10:367
414199118,2cyclictest1270-21pltrace-run20:50:516
414199118,2cyclictest0-21swapper/621:20:016
414199116,4cyclictest0-21swapper/621:25:396
414199115,5cyclictest0-21swapper/620:01:106
414199112,8cyclictest0-21swapper/623:05:586
414199112,8cyclictest0-21swapper/622:11:226
414199110,2cyclictest0-21swapper/620:45:136
414199110,10cyclictest0-21swapper/622:50:286
414199110,1cyclictest0-21swapper/622:05:366
413499119,1cyclictest0-21swapper/523:10:155
413499119,1cyclictest0-21swapper/522:05:115
413499119,1cyclictest0-21swapper/520:15:205
413499118,2cyclictest24450-21gltestperf23:35:155
413499118,2cyclictest1028-17auditd21:10:015
413499117,1cyclictest0-21swapper/520:35:425
413499116,3cyclictest0-21swapper/519:46:035
4134991110,1cyclictest0-21swapper/522:56:525
4134991110,1cyclictest0-21swapper/521:56:085
4134991110,1cyclictest0-21swapper/521:56:085
4134991110,1cyclictest0-21swapper/500:16:505
413499110,11cyclictest0-21swapper/522:04:045
413499110,11cyclictest0-21swapper/520:32:555
413499110,1cyclictest0-21swapper/522:26:255
413499110,1cyclictest0-21swapper/521:17:125
413499110,1cyclictest0-21swapper/500:23:485
412999119,1cyclictest0-21swapper/423:00:204
412999118,2cyclictest7497-21sed19:15:024
412999118,2cyclictest18891-21unixbench_singl22:25:194
412999114,6cyclictest15577-21proc_pri22:20:204
412999112,8cyclictest27094-21cat00:40:004
4129991110,1cyclictest0-21swapper/423:28:114
4129991110,1cyclictest0-21swapper/422:58:044
4129991110,1cyclictest0-21swapper/422:54:004
4129991110,1cyclictest0-21swapper/422:07:424
4129991110,1cyclictest0-21swapper/419:50:054
4129991110,1cyclictest0-21swapper/419:39:144
4129991110,1cyclictest0-21swapper/400:24:404
412999110,11cyclictest0-21swapper/423:57:104
412999110,11cyclictest0-21swapper/423:11:404
412999110,11cyclictest0-21swapper/422:31:014
412999110,1cyclictest0-21swapper/423:30:564
412999110,1cyclictest0-21swapper/422:04:574
412999110,1cyclictest0-21swapper/421:03:364
412999110,1cyclictest0-21swapper/419:42:184
412599119,1cyclictest0-21swapper/321:13:483
412599118,2cyclictest360-21expr21:55:123
412599118,2cyclictest360-21expr21:55:123
412599118,2cyclictest25111-21head21:40:133
412599118,2cyclictest0-21swapper/322:03:533
412599116,3cyclictest0-21swapper/322:28:323
4125991110,1cyclictest0-21swapper/319:11:433
412599111,8cyclictest0-21swapper/321:25:553
412599110,9cyclictest0-21swapper/323:18:503
412599110,11cyclictest0-21swapper/320:07:153
412599110,1cyclictest0-21swapper/323:42:553
412599110,1cyclictest0-21swapper/323:05:403
412599110,1cyclictest0-21swapper/323:02:543
412599110,1cyclictest0-21swapper/322:08:213
4120991110,1cyclictest0-21swapper/223:22:092
4120991110,1cyclictest0-21swapper/220:57:052
412099110,11cyclictest0-21swapper/223:17:572
412099110,11cyclictest0-21swapper/221:21:512
412099110,11cyclictest0-21swapper/220:37:202
412099110,11cyclictest0-21swapper/220:20:302
412099110,11cyclictest0-21swapper/219:20:462
412099110,1cyclictest0-21swapper/221:12:382
411699118,2cyclictest0-21swapper/119:20:261
411699116,3cyclictest0-21swapper/120:20:411
411699115,5cyclictest0-21swapper/121:25:001
411699115,5cyclictest0-21swapper/121:18:481
411699112,8cyclictest0-21swapper/122:51:411
411699112,3cyclictest20178-21sed19:35:171
4116991110,1cyclictest0-21swapper/119:10:021
411699111,9cyclictest0-21swapper/122:32:121
411699111,9cyclictest0-21swapper/119:25:221
411699111,5cyclictest0-21swapper/122:15:141
411699110,9cyclictest0-21swapper/123:40:161
411699110,9cyclictest0-21swapper/120:09:501
411699110,1cyclictest0-21swapper/123:20:591
411299119,1cyclictest0-21swapper/021:31:440
411299119,1cyclictest0-21swapper/000:33:050
411299118,2cyclictest6640-21tr20:10:120
411299118,2cyclictest30429-21tune2fs21:50:130
411299118,2cyclictest20014-21gltestperf00:25:160
411299115,5cyclictest22676-21crond20:40:000
4112991110,1cyclictest0-21swapper/022:40:440
4112991110,1cyclictest0-21swapper/021:56:020
4112991110,1cyclictest0-21swapper/021:56:010
4112991110,1cyclictest0-21swapper/019:23:110
411299110,9cyclictest5457-21fschecks_time21:05:140
411299110,10cyclictest0-21swapper/000:17:430
411299110,1cyclictest0-21swapper/022:51:250
411299110,1cyclictest0-21swapper/022:17:250
411299110,1cyclictest0-21swapper/021:23:580
411299110,1cyclictest0-21swapper/019:59:570
411299110,1cyclictest0-21swapper/000:14:570
2904652110,0pltrace0-21swapper/221:49:342
1837252110,0pltrace0-21swapper/300:21:463
1680552110,2pltrace16996-21taskset20:28:316
1000652110,1pltrace0-21swapper/600:10:226
414699109,1cyclictest0-21swapper/719:33:557
414699108,1cyclictest0-21swapper/720:55:037
414699107,2cyclictest1132-21snmpd22:08:287
414699105,4cyclictest30359-21grep21:50:137
414699105,2cyclictest0-21swapper/719:56:147
414699103,6cyclictest0-21swapper/720:20:047
414699103,6cyclictest0-21swapper/719:44:567
414699102,7cyclictest0-21swapper/720:50:127
414199109,1cyclictest0-21swapper/619:52:506
414199109,1cyclictest0-21swapper/619:20:576
414199109,1cyclictest0-21swapper/600:27:456
414199107,2cyclictest1065-17auditd22:00:006
414199107,2cyclictest1065-17auditd22:00:006
414199105,3cyclictest0-21swapper/621:00:226
414199104,5cyclictest0-21swapper/623:20:056
414199100,8cyclictest1065-17auditd21:15:016
414199100,10cyclictest0-21swapper/622:59:516
414199100,10cyclictest0-21swapper/621:48:466
414199100,10cyclictest0-21swapper/620:58:456
414199100,1cyclictest0-21swapper/622:39:476
414199100,1cyclictest0-21swapper/619:41:596
413499109,1cyclictest0-21swapper/521:50:125
413499109,1cyclictest0-21swapper/520:23:025
413499109,1cyclictest0-21swapper/520:10:255
413499108,1cyclictest0-21swapper/521:10:185
413499107,2cyclictest1132-21snmpd22:43:005
413499107,2cyclictest0-21swapper/522:45:185
413499107,2cyclictest0-21swapper/522:40:015
413499105,4cyclictest25860-21load00:35:175
413499105,4cyclictest22253-21gltestperf21:35:175
413499105,3cyclictest0-21swapper/519:20:095
412999109,1cyclictest0-21swapper/423:51:404
412999109,1cyclictest0-21swapper/422:44:374
412999109,1cyclictest0-21swapper/422:10:274
412999109,1cyclictest0-21swapper/420:59:224
412999109,1cyclictest0-21swapper/420:30:044
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional