You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 12:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Sat Jul 12, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
109432325545,7sleep50-21swapper/503:29:285
109423825522,29sleep70-21swapper/703:28:187
109443725040,7sleep60-21swapper/603:31:066
109443524839,6sleep40-21swapper/403:31:044
109483799420,40cyclictest1191425-21latency_hist04:17:277
109483799420,40cyclictest1191425-21latency_hist04:17:277
1094826993230,2cyclictest1200396-21kworker/u16:5+events_unbound05:27:224
1094826993028,2cyclictest1121318-21kworker/u16:2+events_unbound03:54:384
1094826993028,2cyclictest1092227-21kworker/u16:5+events_unbound04:02:494
13326552290,0chrt0-21swapper/505:22:495
1094826992926,2cyclictest1479146-21kworker/u16:3+events_unbound07:07:294
1094830992826,2cyclictest1346808-21kworker/u16:3+flush-8:005:57:465
1094826992826,2cyclictest1686254-21kworker/u16:2+events_unbound08:22:264
1094826992724,2cyclictest1727785-21kworker/u16:1+events_unbound08:52:284
109483799260,25cyclictest1243020-21latency_hist04:42:287
109483799260,24cyclictest1593572-21latency_hist07:32:277
1094826992624,2cyclictest1121318-21kworker/u16:2+events_unbound03:47:444
1094826992623,2cyclictest1479146-21kworker/u16:3+flush-8:009:02:264
1094826992623,2cyclictest1479146-21kworker/u16:3+flush-8:009:02:264
1094830992523,2cyclictest1479146-21kworker/u16:3+flush-8:008:12:455
109483799240,22cyclictest1335773-21latency_hist05:27:277
109483799240,22cyclictest1294539-21latency_hist05:07:277
109483799240,22cyclictest1294539-21latency_hist05:07:277
1094826992422,2cyclictest1407823-21kworker/u16:1+events_unbound06:22:504
1094826992421,2cyclictest1200396-21kworker/u16:5+flush-8:006:07:464
1094830992321,2cyclictest1479146-21kworker/u16:3+events_unbound07:14:415
1094830992321,2cyclictest1064491-21kworker/u16:3+flush-8:003:37:585
1094826992220,2cyclictest820377-21kworker/u16:1+flush-8:004:17:374
1094826992220,2cyclictest1614312-21kworker/u16:4+events_unbound07:47:554
1094826992219,2cyclictest1567576-21kworker/u16:4+events_unbound07:22:404
1094826992219,2cyclictest1567576-21kworker/u16:4+events_unbound07:22:404
15713192210,1chrt1571320-21kthreadcore07:17:497
109483799210,20cyclictest1562648-21sed07:17:267
109483799210,19cyclictest1706795-21latency_hist08:27:277
1094830992117,3cyclictest1121318-21kworker/u16:2+flush-8:005:02:585
1094830992117,3cyclictest1121318-21kworker/u16:2+flush-8:005:02:585
1094826992119,2cyclictest1541611-21kworker/u16:0+flush-8:007:37:224
1094826992118,2cyclictest1440858-21kworker/u16:2+events_unbound06:32:444
1094837992019,0cyclictest0-21swapper/704:07:467
1094832992018,2cyclictest1192159-21kworker/u16:4+events_unbound04:17:466
1094830992018,2cyclictest1200396-21kworker/u16:5+flush-8:006:12:275
1094826992018,2cyclictest1407823-21kworker/u16:1+events_unbound06:43:274
1094826992018,2cyclictest1200396-21kworker/u16:5+flush-8:004:54:504
1094826992018,2cyclictest1192159-21kworker/u16:4+flush-8:005:17:564
1094826992017,2cyclictest1614312-21kworker/u16:4+events_unbound08:32:404
1094826992017,2cyclictest1092227-21kworker/u16:5+events_unbound03:32:394
1094826991917,2cyclictest1200396-21kworker/u16:5+flush-8:005:43:014
1094826991916,3cyclictest1614312-21kworker/u16:4+events_unbound08:08:264
1094837991815,2cyclictest1284288-21latency_hist05:02:277
1094837991815,2cyclictest1129816-21latency_hist03:47:287
1094830991816,2cyclictest1121318-21kworker/u16:2+flush-8:004:12:575
1094830991816,2cyclictest1121318-21kworker/u16:2+flush-8:004:12:575
1094826991816,2cyclictest339226-21kworker/u16:0+events_unbound04:25:374
109483799170,1cyclictest0-21swapper/706:12:537
109483799170,15cyclictest1387411-21latency_hist05:52:277
109483799170,15cyclictest1387411-21latency_hist05:52:277
109483099170,16cyclictest1757243-21latency08:47:495
1094826991715,2cyclictest1604104-21kworker/u16:2+flush-8:007:37:474
1094826991715,2cyclictest1121318-21kworker/u16:2+flush-8:004:07:474
1094826991714,2cyclictest1346808-21kworker/u16:3+flush-8:006:02:374
109482699170,1cyclictest0-21swapper/408:32:054
109482699170,1cyclictest0-21swapper/405:55:254
1094826991614,2cyclictest1479146-21kworker/u16:3+flush-8:008:04:364
1094826991614,2cyclictest1479146-21kworker/u16:3+events_unbound08:37:514
1094826991614,2cyclictest1479146-21kworker/u16:3+events_unbound08:37:514
1094826991614,2cyclictest1407823-21kworker/u16:1+events_unbound06:52:564
1094826991614,2cyclictest1346808-21kworker/u16:3+events_unbound05:57:554
1094826991614,2cyclictest1192159-21kworker/u16:4+events_unbound05:33:494
109482699160,1cyclictest0-21swapper/408:47:264
109482699160,1cyclictest0-21swapper/408:22:524
109482699160,1cyclictest0-21swapper/408:14:404
109482699160,1cyclictest0-21swapper/407:58:174
109482699160,1cyclictest0-21swapper/407:18:214
109482699160,1cyclictest0-21swapper/407:17:204
109482699160,1cyclictest0-21swapper/406:19:594
109482699160,1cyclictest0-21swapper/405:39:024
109482699160,1cyclictest0-21swapper/405:14:274
109482699160,1cyclictest0-21swapper/405:05:144
109482699160,1cyclictest0-21swapper/405:05:144
109482699160,1cyclictest0-21swapper/404:58:044
109482699160,1cyclictest0-21swapper/404:48:514
109482699160,1cyclictest0-21swapper/404:40:404
109482699160,1cyclictest0-21swapper/404:34:314
109482699160,1cyclictest0-21swapper/404:17:074
109482699160,1cyclictest0-21swapper/404:17:074
109482699160,1cyclictest0-21swapper/403:59:424
109482699160,1cyclictest0-21swapper/403:43:194
109483799159,5cyclictest27763-21gvfs-afc-volume08:37:527
109483799159,5cyclictest27763-21gvfs-afc-volume08:37:527
1094837991513,1cyclictest0-21swapper/708:17:527
109483799150,13cyclictest1313-21snmpd08:36:377
1094832991512,2cyclictest1455243-21tr06:22:496
1094832991512,2cyclictest1407823-21kworker/u16:1+flush-8:006:07:426
1094830991513,2cyclictest1407823-21kworker/u16:1+events_unbound06:52:425
1094826991513,2cyclictest1200396-21kworker/u16:5+events_unbound04:27:514
109482699150,1cyclictest0-21swapper/405:47:484
109482699150,1cyclictest0-21swapper/405:47:484
109483799147,6cyclictest27763-21gvfs-afc-volume03:37:537
109483799147,6cyclictest27430-21gmain07:32:497
109483799140,1cyclictest0-21swapper/708:12:547
109483799140,13cyclictest1313-21snmpd07:46:037
109483799140,13cyclictest1150217-21turbostat03:57:277
109483799140,12cyclictest1150939-21exim403:57:287
109483299140,14cyclictest0-21swapper/605:37:446
1094826991412,2cyclictest1272966-21kworker/u16:0+events_unbound05:07:284
1094826991410,3cyclictest1121318-21kworker/u16:2+flush-8:004:42:434
109482699140,1cyclictest0-21swapper/408:48:274
109483799139,2cyclictest1212091-21munin-run04:27:277
109483799130,1cyclictest0-21swapper/706:02:577
109483799130,12cyclictest1313-21snmpd04:49:527
109483799130,11cyclictest1313-21snmpd08:57:347
109483799130,11cyclictest1313-21snmpd08:57:347
1094832991312,1cyclictest0-21swapper/608:27:506
1094832991312,1cyclictest0-21swapper/607:57:526
1094832991311,2cyclictest1172920-21kworker/u16:3+flush-8:004:07:526
109483299130,1cyclictest0-21swapper/604:40:116
109483299130,13cyclictest0-21swapper/606:20:366
109483099130,1cyclictest0-21swapper/508:24:395
109483099130,1cyclictest0-21swapper/505:47:285
109483099130,1cyclictest0-21swapper/505:47:285
109483099130,1cyclictest0-21swapper/504:46:285
109483099130,1cyclictest0-21swapper/504:03:265
109483099130,13cyclictest0-21swapper/508:57:525
109483099130,13cyclictest0-21swapper/508:57:525
109483099130,12cyclictest0-21swapper/508:32:325
1094826991310,2cyclictest1479146-21kworker/u16:3+flush-8:007:27:474
109482699130,13cyclictest0-21swapper/406:47:504
109483799129,2cyclictest1201744-21munin-run04:22:277
1094837991210,1cyclictest1480095-21turbostat06:37:267
1094837991210,1cyclictest0-21swapper/705:42:587
109483799120,2cyclictest1674856-21ntpq08:07:497
109483799120,2cyclictest1480835-21/usr/sbin/munin06:37:557
109483799120,1cyclictest0-21swapper/705:17:527
109483799120,11cyclictest96-21ksoftirqd/705:37:507
109483799120,11cyclictest1313-21snmpd03:51:217
109483799120,11cyclictest1232682-21latency_hist04:37:277
109483799120,10cyclictest1313-21snmpd07:47:547
109483799120,10cyclictest1313-21snmpd06:59:177
109483799120,10cyclictest1313-21snmpd06:50:167
109483799120,10cyclictest0-21swapper/708:47:517
1094832991211,1cyclictest0-21swapper/607:33:266
1094832991211,1cyclictest0-21swapper/607:12:286
1094832991211,1cyclictest0-21swapper/606:40:576
109483299120,1cyclictest0-21swapper/606:32:266
109483299120,1cyclictest0-21swapper/604:53:456
109483299120,1cyclictest0-21swapper/604:05:496
109483299120,1cyclictest0-21swapper/603:46:266
109483099129,2cyclictest1541611-21kworker/u16:0+flush-8:007:27:265
109483099129,2cyclictest1541611-21kworker/u16:0+flush-8:007:27:265
109483099128,3cyclictest1275445-21cat04:57:435
1094830991211,1cyclictest0-21swapper/508:27:395
1094830991211,1cyclictest0-21swapper/507:12:175
1094830991211,1cyclictest0-21swapper/507:02:025
1094830991211,1cyclictest0-21swapper/506:43:365
1094830991211,1cyclictest0-21swapper/506:37:085
1094830991211,1cyclictest0-21swapper/505:38:295
109483099120,1cyclictest0-21swapper/508:01:095
109483099120,1cyclictest0-21swapper/507:55:195
109483099120,1cyclictest0-21swapper/505:35:015
109483099120,12cyclictest0-21swapper/508:11:535
109483099120,12cyclictest0-21swapper/507:28:325
109483099120,12cyclictest0-21swapper/507:06:085
109483099120,12cyclictest0-21swapper/506:38:145
109483099120,12cyclictest0-21swapper/506:03:525
109483099120,12cyclictest0-21swapper/503:49:435
1094826991210,2cyclictest1541611-21kworker/u16:0+events_unbound07:06:584
1094826991210,2cyclictest1437900-21kworker/u16:4+events_unbound06:40:274
1094826991210,2cyclictest1092227-21kworker/u16:5+flush-8:003:38:274
109482699120,1cyclictest0-21swapper/406:29:124
109483799119,1cyclictest1583266-21munin-run07:27:267
109483799119,1cyclictest1583266-21munin-run07:27:267
109483799117,3cyclictest27642-21gmain05:07:537
109483799117,3cyclictest27394-21gmain07:37:457
109483799115,5cyclictest27763-21gvfs-afc-volume05:57:447
1094837991110,0cyclictest0-21swapper/708:06:557
1094837991110,0cyclictest0-21swapper/705:36:127
109483799110,9cyclictest1439603-21sendmail06:17:287
109483799110,4cyclictest1313-21snmpd04:55:527
109483799110,3cyclictest1313-21snmpd08:28:347
109483799110,2cyclictest1654399-21ntpq07:57:577
109483799110,1cyclictest1519869-21kthreadcore06:52:497
109483799110,1cyclictest0-21swapper/706:42:527
109483799110,1cyclictest0-21swapper/703:32:537
109483799110,10cyclictest1537831-21kthreadcore07:02:497
109483799110,10cyclictest1459477-21perf06:27:267
109483799110,10cyclictest1313-21snmpd08:47:197
109483799110,10cyclictest1313-21snmpd06:29:437
109483799110,10cyclictest1313-21snmpd04:29:077
109483299119,1cyclictest1379996-21kthreadcore05:47:476
109483299119,1cyclictest1379996-21kthreadcore05:47:476
109483299117,3cyclictest1739275-21cat08:42:416
109483299117,3cyclictest1512546-21cat06:52:426
109483299114,3cyclictest0-21swapper/604:27:496
1094832991110,1cyclictest0-21swapper/608:54:286
1094832991110,1cyclictest0-21swapper/608:07:296
1094832991110,1cyclictest0-21swapper/607:30:276
1094832991110,1cyclictest0-21swapper/607:00:206
1094832991110,1cyclictest0-21swapper/606:03:546
1094832991110,1cyclictest0-21swapper/605:15:026
1094832991110,1cyclictest0-21swapper/605:00:126
1094832991110,1cyclictest0-21swapper/604:43:206
1094832991110,1cyclictest0-21swapper/604:12:456
1094832991110,1cyclictest0-21swapper/604:12:456
1094832991110,1cyclictest0-21swapper/603:39:496
109483299110,1cyclictest0-21swapper/606:36:416
109483299110,11cyclictest0-21swapper/607:05:236
109483299110,11cyclictest0-21swapper/605:08:436
109483299110,11cyclictest0-21swapper/605:04:286
109483299110,11cyclictest0-21swapper/605:04:286
109483299110,11cyclictest0-21swapper/604:49:296
1094830991110,1cyclictest0-21swapper/507:43:305
1094830991110,1cyclictest0-21swapper/507:39:345
1094830991110,1cyclictest0-21swapper/506:30:405
1094830991110,1cyclictest0-21swapper/505:55:315
1094830991110,1cyclictest0-21swapper/504:57:115
1094830991110,1cyclictest0-21swapper/504:09:255
1094830991110,1cyclictest0-21swapper/503:35:325
109483099110,11cyclictest0-21swapper/505:14:225
109482699119,2cyclictest1593553-21kworker/u16:1+events_unbound07:45:264
109482699110,11cyclictest0-21swapper/407:52:584
109482699110,11cyclictest0-21swapper/407:00:194
1094837991010,0cyclictest0-21swapper/707:11:447
109483799100,9cyclictest1313-21snmpd07:53:307
109483799100,9cyclictest0-21swapper/706:07:547
109483799100,10cyclictest96-21ksoftirqd/704:42:567
109483299109,1cyclictest0-21swapper/607:42:166
109483299109,1cyclictest0-21swapper/603:57:566
109483299108,2cyclictest1192159-21kworker/u16:4+flush-8:005:17:426
109483299108,1cyclictest0-21swapper/606:47:536
109483099109,1cyclictest0-21swapper/508:19:465
109483099109,1cyclictest0-21swapper/506:48:485
109483099109,1cyclictest0-21swapper/505:20:405
109483099109,1cyclictest0-21swapper/504:47:345
109483099109,1cyclictest0-21swapper/504:41:165
109483099109,1cyclictest0-21swapper/504:21:155
109483099100,10cyclictest0-21swapper/504:22:445
10948379990,8cyclictest1313-21snmpd04:04:187
10948379990,8cyclictest0-21swapper/708:52:447
10948379990,7cyclictest1313-21snmpd05:52:287
10948379990,1cyclictest0-21swapper/705:12:397
10948329998,1cyclictest0-21swapper/608:17:066
10948329998,1cyclictest0-21swapper/604:22:496
10948329994,4cyclictest1633785-21ldconfig07:47:516
10948309998,1cyclictest0-21swapper/508:56:295
10948309998,1cyclictest0-21swapper/508:06:035
10948309996,2cyclictest1140644-21/usr/sbin/munin03:52:425
10948269990,1cyclictest0-21swapper/406:12:494
10948379980,7cyclictest1346790-21cat05:32:287
10948329986,1cyclictest0-21swapper/607:22:466
10948329986,1cyclictest0-21swapper/607:22:466
10948329984,3cyclictest1736495-21grep08:37:506
10948329984,3cyclictest1736495-21grep08:37:506
10948329984,3cyclictest1657645-21grep08:02:436
10948329984,3cyclictest1615283-21expr07:42:406
10948329984,3cyclictest1563754-21expr07:17:406
10948329984,3cyclictest1545286-21grep07:07:456
10948329984,3cyclictest1352227-21grep05:32:476
10948329984,3cyclictest1226665-21grep04:32:466
10948329983,4cyclictest1750988-21kthreadcore08:47:456
10948329983,4cyclictest1700975-21cut08:22:466
10948329983,4cyclictest1687645-21sh08:17:436
10948329983,4cyclictest1491457-21expr06:42:406
10948329983,4cyclictest1389687-21cut05:52:446
10948329983,4cyclictest1146067-21kthreadcore03:52:476
10948329983,4cyclictest1134941-21kthreadcore03:47:486
10948329983,4cyclictest1102575-21kthreadcore03:32:486
10948329980,7cyclictest1326639-21cat05:22:416
10948309987,1cyclictest0-21swapper/507:33:345
10948309987,1cyclictest0-21swapper/504:34:575
10948309986,2cyclictest1346808-21kworker/u16:3+events_unbound05:42:465
10948309985,3cyclictest1541611-21kworker/u16:0+flush-8:007:17:525
10948309984,3cyclictest1739549-21irqcore08:42:425
10948309983,4cyclictest1632098-21cut07:47:465
10948309983,4cyclictest1303727-21wc05:07:505
10948309983,4cyclictest1219748-21cut04:27:505
10948309980,7cyclictest1159581-21ntpq03:57:525
10948329974,2cyclictest1636214-21if_err_p4p107:52:426
10948329973,3cyclictest1769874-21cat08:57:406
10948329973,3cyclictest1769874-21cat08:57:406
10948329973,3cyclictest1399767-21cut05:57:466
10948329973,3cyclictest1367889-21expr05:42:446
10948329972,4cyclictest1718333-21cat08:32:406
10948329972,4cyclictest1431519-21kthreadcore06:12:486
10948309975,1cyclictest0-21swapper/508:37:405
10948309975,1cyclictest0-21swapper/508:37:405
10948309973,3cyclictest1451746-21grep06:22:475
10948309973,3cyclictest1441718-21cat06:17:495
10948309973,3cyclictest1429987-21tune2fs06:12:455
10948309950,4cyclictest1120080-21taskset03:42:345
10948329930,3cyclictest0-21swapper/605:32:286
10948309930,1cyclictest991rcuop/705:32:285
10948269920,1cyclictest991rcuop/705:32:284
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional