You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 07:20
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot2.osadl.org (updated Tue Dec 02, 2025 00:45:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
136691229168,50phc2sys0-21swapper/319:06:543
283452206172,23sleep20-21swapper/219:06:162
285532203168,23sleep00-21swapper/019:08:510
285192203169,24sleep10-21swapper/119:08:251
2864328462,14sleep10-21swapper/119:10:021
245862710,4sleep224589-21irqrtprio21:15:182
28934997010,19cyclictest11810-21cat00:10:220
45692680,3sleep14572-21needreboot23:55:251
2893499686,18cyclictest28570-21cat23:40:000
314152610,2sleep10-21swapper/122:35:281
314152610,2sleep10-21swapper/122:35:281
2893499610,21cyclictest14025-21sed00:15:200
28937995925,4cyclictest41-21ksoftirqd/322:45:173
28937995831,3cyclictest41-21ksoftirqd/319:35:013
2893499580,21cyclictest4422-21cat23:55:210
28937995725,8cyclictest41-21ksoftirqd/322:35:143
28937995725,8cyclictest41-21ksoftirqd/322:35:143
28937995725,8cyclictest41-21ksoftirqd/320:45:003
28937995721,7cyclictest41-21ksoftirqd/320:35:133
28937995618,9cyclictest41-21ksoftirqd/300:35:143
2893499560,20cyclictest14247-21date22:00:200
2893499560,19cyclictest1785-21basename21:35:160
2893499550,28cyclictest22948-21systemctl21:10:250
2893499550,23cyclictest16256-21seq23:14:110
2893499550,23cyclictest16256-21seq23:14:100
28937995426,8cyclictest41-21ksoftirqd/320:50:003
28937995424,9cyclictest41-21ksoftirqd/300:05:133
28937995424,8cyclictest41-21ksoftirqd/322:20:173
28937995423,4cyclictest41-21ksoftirqd/320:55:303
28937995419,9cyclictest41-21ksoftirqd/323:45:183
28937995419,9cyclictest41-21ksoftirqd/323:45:183
28937995326,8cyclictest41-21ksoftirqd/319:35:163
28937995325,6cyclictest41-21ksoftirqd/323:00:003
28937995323,4cyclictest41-21ksoftirqd/323:00:293
28937995322,5cyclictest41-21ksoftirqd/323:20:193
28937995321,8cyclictest41-21ksoftirqd/300:20:123
28937995321,7cyclictest41-21ksoftirqd/322:50:183
28937995320,9cyclictest41-21ksoftirqd/323:15:003
28937995320,9cyclictest41-21ksoftirqd/323:15:003
28937995319,8cyclictest41-21ksoftirqd/321:50:203
28937995224,3cyclictest41-21ksoftirqd/322:10:263
28937995223,9cyclictest41-21ksoftirqd/321:09:593
28937995223,8cyclictest41-21ksoftirqd/323:50:133
28937995223,7cyclictest41-21ksoftirqd/320:00:243
28937995222,8cyclictest41-21ksoftirqd/321:55:283
28937995222,8cyclictest41-21ksoftirqd/321:55:283
28937995221,9cyclictest41-21ksoftirqd/322:05:133
28937995221,4cyclictest41-21ksoftirqd/321:25:163
28937995220,10cyclictest41-21ksoftirqd/319:55:003
28937995220,10cyclictest41-21ksoftirqd/319:55:003
28937995219,7cyclictest41-21ksoftirqd/321:45:283
28937995218,4cyclictest41-21ksoftirqd/320:25:173
28937995218,10cyclictest41-21ksoftirqd/321:20:133
28937995218,10cyclictest41-21ksoftirqd/321:20:133
28937995212,10cyclictest41-21ksoftirqd/300:30:173
2893499521,20cyclictest29827-21/usr/sbin/munin22:35:180
2893499521,20cyclictest29827-21/usr/sbin/munin22:35:180
28937995126,8cyclictest41-21ksoftirqd/323:15:173
28937995125,7cyclictest41-21ksoftirqd/319:55:283
28937995125,3cyclictest41-21ksoftirqd/300:00:023
28937995123,7cyclictest41-21ksoftirqd/321:30:163
28937995123,3cyclictest41-21ksoftirqd/300:15:223
28937995122,9cyclictest41-21ksoftirqd/322:25:203
28937995121,3cyclictest41-21ksoftirqd/320:10:243
28937995120,7cyclictest41-21ksoftirqd/300:00:153
28937995119,8cyclictest41-21ksoftirqd/320:50:153
28937995023,9cyclictest41-21ksoftirqd/321:39:593
28937995022,7cyclictest41-21ksoftirqd/319:40:273
28937995022,3cyclictest41-21ksoftirqd/323:44:243
28937995021,8cyclictest41-21ksoftirqd/321:40:223
28937995021,4cyclictest41-21ksoftirqd/320:20:003
28937995020,9cyclictest41-21ksoftirqd/323:30:223
28937995018,7cyclictest41-21ksoftirqd/323:30:013
28937995018,4cyclictest41-21ksoftirqd/320:20:163
28937995018,4cyclictest41-21ksoftirqd/320:20:163
28937995017,8cyclictest41-21ksoftirqd/319:15:253
28937995017,4cyclictest41-21ksoftirqd/300:25:143
28937994921,8cyclictest41-21ksoftirqd/320:30:253
28937994921,7cyclictest41-21ksoftirqd/300:10:213
28937994920,7cyclictest41-21ksoftirqd/323:35:273
28937994920,4cyclictest41-21ksoftirqd/322:30:143
28937994919,4cyclictest41-21ksoftirqd/319:30:013
28937994916,10cyclictest41-21ksoftirqd/323:05:163
28937994821,7cyclictest41-21ksoftirqd/321:00:263
28937994815,9cyclictest41-21ksoftirqd/319:10:193
2893499481,19cyclictest27956-21ls21:20:270
2893499481,19cyclictest27956-21ls21:20:270
28937994716,3cyclictest41-21ksoftirqd/319:50:003
28937994715,7cyclictest41-21ksoftirqd/319:20:203
28937994715,4cyclictest41-21ksoftirqd/321:10:143
2893499470,28cyclictest10378-21seq19:36:300
2893799465,3cyclictest41-21ksoftirqd/322:40:233
28937994621,6cyclictest41-21ksoftirqd/322:00:163
2893499460,20cyclictest9087-21runrttasks22:58:340
28937994518,8cyclictest41-21ksoftirqd/322:15:193
28937994515,8cyclictest41-21ksoftirqd/320:05:233
28937994416,8cyclictest41-21ksoftirqd/321:15:233
51698430,8rtkit-daemon0-21swapper/121:29:291
2893499420,27cyclictest6512-21runrttasks22:51:440
2893499420,19cyclictest7790-21snmpd19:18:430
2893499411,26cyclictest0-21swapper/020:05:220
2893499411,20cyclictest18780-21seq21:03:010
51698400,7rtkit-daemon0-21swapper/122:52:251
51698400,4rtkit-daemon18353-21diskstats22:10:141
2893499401,25cyclictest0-21swapper/019:50:260
2893499401,25cyclictest0-21swapper/019:50:260
2893499380,25cyclictest1307-21runrttasks23:54:060
2893499371,22cyclictest25556-21sed21:15:260
2893499370,24cyclictest1307-21runrttasks00:23:110
2893499370,18cyclictest0-21swapper/022:21:150
2893499361,20cyclictest720-21irqstats22:40:210
2893499360,21cyclictest25085-21/usr/sbin/munin22:25:310
51698350,1rtkit-daemon0-21swapper/122:58:231
2893499351,16cyclictest0-21swapper/020:29:420
2893499350,19cyclictest1307-21runrttasks00:30:560
2893499350,18cyclictest0-21swapper/021:56:030
2893499350,18cyclictest0-21swapper/021:56:030
2893499350,18cyclictest0-21swapper/020:55:180
28936993418,13cyclictest33-21ksoftirqd/219:56:272
28935993426,4cyclictest30798-21cron23:45:001
28934993428,4cyclictest9-21ksoftirqd/022:34:080
28934993428,4cyclictest9-21ksoftirqd/000:02:250
2893499341,17cyclictest0-21swapper/019:20:240
2893499340,20cyclictest1307-21runrttasks19:41:250
2893499340,18cyclictest1307-21runrttasks23:47:560
2893499340,18cyclictest1307-21runrttasks23:47:560
2893499340,18cyclictest0-21swapper/023:00:150
51698330,1rtkit-daemon0-21swapper/100:17:421
28936993327,4cyclictest33-21ksoftirqd/223:19:182
28936993326,5cyclictest33-21ksoftirqd/223:05:582
2893499331,16cyclictest0-21swapper/022:15:280
2893499330,19cyclictest0-21swapper/021:49:130
2893499330,17cyclictest1307-21runrttasks00:38:350
2893499330,17cyclictest0-21swapper/021:50:270
28936993227,3cyclictest33-21ksoftirqd/223:00:062
28936993227,3cyclictest33-21ksoftirqd/222:54:152
28936993226,4cyclictest33-21ksoftirqd/222:22:302
28936993226,4cyclictest33-21ksoftirqd/222:13:232
28936993226,4cyclictest33-21ksoftirqd/220:53:562
28936993226,4cyclictest33-21ksoftirqd/220:01:492
28935993227,3cyclictest25-21ksoftirqd/120:14:591
28935993226,4cyclictest25-21ksoftirqd/120:40:061
28935993226,4cyclictest25-21ksoftirqd/119:44:101
28934993226,4cyclictest9-21ksoftirqd/019:25:470
2893499321,16cyclictest0-21swapper/022:05:500
2893499320,19cyclictest2275-21/usr/sbin/munin20:30:260
2893499320,19cyclictest0-21swapper/021:33:540
2893499320,18cyclictest1307-21runrttasks20:13:210
2893499320,16cyclictest0-21swapper/023:25:190
2893499320,16cyclictest0-21swapper/020:35:270
28936993126,3cyclictest33-21ksoftirqd/223:12:032
28936993126,3cyclictest33-21ksoftirqd/223:12:022
28936993126,3cyclictest33-21ksoftirqd/200:35:482
28936993125,4cyclictest33-21ksoftirqd/222:48:422
28936993125,4cyclictest33-21ksoftirqd/221:23:052
28936993125,4cyclictest33-21ksoftirqd/221:23:052
28936993125,4cyclictest33-21ksoftirqd/220:49:262
28936993125,4cyclictest33-21ksoftirqd/200:22:452
28936993124,5cyclictest33-21ksoftirqd/222:57:232
28935993127,3cyclictest25-21ksoftirqd/122:48:251
28935993125,4cyclictest25-21ksoftirqd/121:20:341
28935993125,4cyclictest25-21ksoftirqd/121:20:341
28935993125,4cyclictest25-21ksoftirqd/121:08:471
28934993125,4cyclictest9-21ksoftirqd/021:44:560
2893499310,19cyclictest30568-21iostat_ios20:20:200
2893499310,19cyclictest30568-21iostat_ios20:20:200
2893499310,19cyclictest0-21swapper/000:08:580
2893499310,18cyclictest0-21swapper/019:12:190
2893499310,16cyclictest0-21swapper/021:25:260
28936993025,3cyclictest33-21ksoftirqd/221:49:462
28936993025,3cyclictest33-21ksoftirqd/220:57:172
28936993024,4cyclictest33-21ksoftirqd/222:33:052
28936993024,4cyclictest33-21ksoftirqd/222:29:042
28936993024,4cyclictest33-21ksoftirqd/221:59:512
28936993024,4cyclictest33-21ksoftirqd/221:59:512
28936993024,4cyclictest33-21ksoftirqd/221:43:262
28936993024,4cyclictest33-21ksoftirqd/220:36:182
28936993024,4cyclictest33-21ksoftirqd/219:46:122
28936993024,4cyclictest33-21ksoftirqd/200:00:062
28936993024,3cyclictest33-21ksoftirqd/200:31:332
28935993024,4cyclictest25-21ksoftirqd/123:54:231
28935993024,4cyclictest25-21ksoftirqd/123:48:131
28935993024,4cyclictest25-21ksoftirqd/123:48:131
28935993024,4cyclictest25-21ksoftirqd/123:21:221
28935993024,4cyclictest25-21ksoftirqd/123:14:031
28935993024,4cyclictest25-21ksoftirqd/123:14:031
28935993024,4cyclictest25-21ksoftirqd/121:52:401
28935993024,4cyclictest25-21ksoftirqd/119:17:151
28935993024,4cyclictest25-21ksoftirqd/100:23:041
28934993025,3cyclictest9-21ksoftirqd/020:03:510
28934993025,3cyclictest9-21ksoftirqd/019:59:580
28934993024,4cyclictest9-21ksoftirqd/023:05:520
28934993024,4cyclictest9-21ksoftirqd/022:49:520
2893499300,27cyclictest0-21swapper/022:10:150
2893499300,22cyclictest7790-21snmpd23:32:080
51698290,12rtkit-daemon13877-21iostat22:00:191
28936992924,3cyclictest33-21ksoftirqd/221:32:442
28936992924,3cyclictest33-21ksoftirqd/219:17:132
28936992923,4cyclictest33-21ksoftirqd/220:44:402
28936992923,4cyclictest33-21ksoftirqd/219:42:232
28936992923,4cyclictest33-21ksoftirqd/219:23:002
28936992923,4cyclictest33-21ksoftirqd/200:16:592
28936992923,4cyclictest33-21ksoftirqd/200:06:202
2893699290,28cyclictest9913-21kworker/2:019:25:242
28935992924,3cyclictest25-21ksoftirqd/123:27:171
28935992923,4cyclictest25-21ksoftirqd/122:22:281
28935992923,4cyclictest25-21ksoftirqd/121:31:371
28935992923,4cyclictest25-21ksoftirqd/119:58:421
28935992923,4cyclictest25-21ksoftirqd/119:32:271
28935992923,4cyclictest25-21ksoftirqd/119:27:221
28935992923,4cyclictest25-21ksoftirqd/100:39:161
28935992923,4cyclictest25-21ksoftirqd/100:29:261
28935992923,4cyclictest25-21ksoftirqd/100:06:471
28934992923,4cyclictest9-21ksoftirqd/023:43:470
28934992923,4cyclictest9-21ksoftirqd/019:32:370
28936992823,3cyclictest33-21ksoftirqd/223:20:532
28936992823,3cyclictest33-21ksoftirqd/222:37:252
28936992823,3cyclictest33-21ksoftirqd/222:37:252
28936992823,3cyclictest33-21ksoftirqd/221:37:562
28936992823,3cyclictest33-21ksoftirqd/220:07:352
28936992822,4cyclictest33-21ksoftirqd/221:00:422
28936992821,5cyclictest33-21ksoftirqd/220:11:562
28935992823,3cyclictest25-21ksoftirqd/122:26:301
28935992823,3cyclictest25-21ksoftirqd/119:35:511
28935992822,4cyclictest25-21ksoftirqd/121:35:301
28935992822,4cyclictest25-21ksoftirqd/121:12:401
28935992822,4cyclictest25-21ksoftirqd/120:54:101
28935992822,3cyclictest25-21ksoftirqd/119:46:211
28934992823,3cyclictest9-21ksoftirqd/023:22:290
28934992823,3cyclictest9-21ksoftirqd/023:17:320
28934992823,3cyclictest9-21ksoftirqd/000:26:400
28934992822,4cyclictest9-21ksoftirqd/020:19:270
51698270,2rtkit-daemon0-21swapper/121:40:421
28936992722,3cyclictest33-21ksoftirqd/220:25:112
28936992721,4cyclictest33-21ksoftirqd/223:59:332
28936992721,4cyclictest33-21ksoftirqd/223:43:372
28936992721,4cyclictest33-21ksoftirqd/222:07:172
28936992721,4cyclictest33-21ksoftirqd/219:34:202
28935992722,3cyclictest25-21ksoftirqd/122:31:461
28935992722,3cyclictest25-21ksoftirqd/120:45:051
28935992722,3cyclictest25-21ksoftirqd/120:31:401
28935992721,4cyclictest25-21ksoftirqd/123:05:391
28935992721,4cyclictest25-21ksoftirqd/122:43:411
28935992721,4cyclictest25-21ksoftirqd/120:08:371
28935992721,4cyclictest25-21ksoftirqd/120:04:121
28934992722,3cyclictest9-21ksoftirqd/020:53:170
28934992722,3cyclictest9-21ksoftirqd/019:45:120
28934992721,4cyclictest9-21ksoftirqd/020:42:380
28936992621,3cyclictest33-21ksoftirqd/222:19:042
28936992621,3cyclictest33-21ksoftirqd/219:37:442
28935992621,3cyclictest25-21ksoftirqd/123:35:341
28935992621,3cyclictest25-21ksoftirqd/123:33:571
28935992621,3cyclictest25-21ksoftirqd/120:59:481
28935992621,3cyclictest25-21ksoftirqd/100:32:131
28935992621,3cyclictest25-21ksoftirqd/100:01:061
28935992620,4cyclictest25-21ksoftirqd/121:47:111
28935992620,4cyclictest25-21ksoftirqd/119:23:231
28935992620,3cyclictest25-21ksoftirqd/121:15:101
2893499268,15cyclictest524-21in:imuxsock20:45:140
2893499262,21cyclictest226-21systemd-journal21:08:020
28936992520,3cyclictest33-21ksoftirqd/222:42:592
28936992520,3cyclictest33-21ksoftirqd/221:08:032
28936992519,4cyclictest33-21ksoftirqd/200:11:172
28936992518,5cyclictest33-21ksoftirqd/220:34:502
2893699250,2cyclictest0-21swapper/221:25:262
28935992520,3cyclictest25-21ksoftirqd/123:17:481
28935992520,3cyclictest25-21ksoftirqd/121:55:051
28935992520,3cyclictest25-21ksoftirqd/121:55:051
28935992519,4cyclictest25-21ksoftirqd/122:15:101
28935992519,4cyclictest25-21ksoftirqd/120:26:151
28935992519,4cyclictest25-21ksoftirqd/120:21:501
28935992519,4cyclictest25-21ksoftirqd/120:21:501
28936992419,3cyclictest33-21ksoftirqd/219:14:312
28936992418,4cyclictest33-21ksoftirqd/223:29:132
28936992418,4cyclictest33-21ksoftirqd/220:23:222
28936992418,4cyclictest33-21ksoftirqd/220:23:222
28936992418,4cyclictest33-21ksoftirqd/200:29:312
28936992417,5cyclictest33-21ksoftirqd/220:19:382
28935992419,3cyclictest25-21ksoftirqd/120:37:061
28935992418,4cyclictest25-21ksoftirqd/121:00:371
28935992418,3cyclictest25-21ksoftirqd/122:07:341
51698230,8rtkit-daemon0-21swapper/119:54:541
51698230,8rtkit-daemon0-21swapper/119:54:541
28936992318,3cyclictest33-21ksoftirqd/223:52:052
28936992318,3cyclictest33-21ksoftirqd/223:48:452
28936992318,3cyclictest33-21ksoftirqd/223:48:452
28936992318,3cyclictest33-21ksoftirqd/223:37:072
28936992318,3cyclictest33-21ksoftirqd/223:33:482
28936992318,3cyclictest33-21ksoftirqd/219:51:582
28936992318,3cyclictest33-21ksoftirqd/219:51:582
28935992318,3cyclictest25-21ksoftirqd/123:04:131
28935992318,3cyclictest25-21ksoftirqd/120:16:301
28935992317,3cyclictest25-21ksoftirqd/100:10:131
28936992115,4cyclictest33-21ksoftirqd/222:02:552
2893699190,18cyclictest10334-21kworker/2:121:12:002
28936991812,4cyclictest9599-21head21:50:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional