You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-02 - 11:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Fri May 02, 2025 00:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
80097826049,7sleep60-21swapper/603:35:386
80139399550,53cyclictest1379140-21latency_hist08:17:117
80099725320,29sleep70-21swapper/703:35:557
80094724838,7sleep50-21swapper/503:35:135
801380994340,2cyclictest770901-21kworker/u16:1+flush-8:003:47:124
80107824333,7sleep40-21swapper/403:37:054
801380994139,2cyclictest1124266-21kworker/u16:1+flush-8:006:27:124
9232652380,0sleep7923266-21grep04:32:357
9232652380,0sleep7923266-21grep04:32:357
801389993331,2cyclictest770902-21kworker/u16:3+events_unbound03:56:236
801389993230,2cyclictest1430389-21kworker/u16:1+events_unbound08:57:466
801389993230,2cyclictest1296177-21kworker/u16:2+events_unbound07:50:236
801389993230,2cyclictest1286841-21kworker/u16:1+events_unbound07:36:226
801389993230,2cyclictest1163010-21kworker/u16:5+events_unbound06:31:226
801389993129,2cyclictest1276564-21kworker/u16:0+events_unbound08:56:266
801389993129,2cyclictest1004946-21kworker/u16:3+events_unbound05:41:556
801380993128,2cyclictest1173943-21kworker/u16:1+flush-8:006:37:424
80139399300,28cyclictest1481670-21latency_hist09:07:117
801389993028,2cyclictest781558-21kworker/u16:2+events_unbound04:40:146
801389993028,2cyclictest770901-21kworker/u16:1+events_unbound04:17:296
801389993028,2cyclictest1399515-21kworker/u16:2+events_unbound09:03:226
801389993028,2cyclictest1245946-21kworker/u16:5+events_unbound07:17:466
801389993028,2cyclictest1234716-21kworker/u16:4+events_unbound07:42:476
801389993028,2cyclictest1234716-21kworker/u16:4+events_unbound07:42:476
801389993027,2cyclictest958820-21kworker/u16:1+events_unbound05:02:156
801389992927,2cyclictest1367901-21kworker/u16:5+events_unbound08:31:236
801389992927,2cyclictest1276564-21kworker/u16:0+events_unbound07:29:036
801389992927,2cyclictest1173943-21kworker/u16:1+events_unbound06:52:386
801380992927,2cyclictest1137551-21kworker/u16:3+events_unbound06:27:274
80139399280,26cyclictest1266477-21latency_hist07:22:127
801389992826,2cyclictest908529-21kworker/u16:0+events_unbound04:46:156
801389992826,2cyclictest1367901-21kworker/u16:5+events_unbound08:14:116
801389992826,2cyclictest1296177-21kworker/u16:2+events_unbound07:58:236
801389992826,2cyclictest1276564-21kworker/u16:0+events_unbound08:40:216
801389992826,2cyclictest1082225-21kworker/u16:0+events_unbound06:09:116
801389992826,2cyclictest1082225-21kworker/u16:0+events_unbound06:09:116
801389992825,2cyclictest1082760-21kworker/u16:2+events_unbound06:57:296
80139399270,25cyclictest921-21dbus-daemon07:12:117
801389992725,2cyclictest916736-21kworker/u16:3+events_unbound04:30:146
801389992725,2cyclictest1399515-21kworker/u16:2+events_unbound08:48:156
801389992724,2cyclictest1137551-21kworker/u16:3+flush-8:006:32:276
801389992724,2cyclictest1040991-21kworker/u16:4+events_unbound05:57:156
8882722260,0sleep40-21swapper/404:17:264
801393992622,2cyclictest1019566-21ntpq05:17:427
801389992624,2cyclictest908529-21kworker/u16:0+events_unbound04:58:226
801389992624,2cyclictest867522-21kworker/u16:0+events_unbound04:08:116
801389992624,2cyclictest1124266-21kworker/u16:1+events_unbound06:16:156
801380992624,2cyclictest908529-21kworker/u16:0+flush-8:004:57:124
8783752250,1chrt0-21swapper/604:12:316
801389992523,2cyclictest999146-21kworker/u16:1+flush-8:005:12:466
801389992523,2cyclictest781558-21kworker/u16:2+events_unbound05:11:226
801389992523,2cyclictest770902-21kworker/u16:3+events_unbound03:41:246
801389992523,2cyclictest1430389-21kworker/u16:1+events_unbound08:42:376
801389992523,2cyclictest1173943-21kworker/u16:1+flush-8:007:22:226
801389992523,2cyclictest1124266-21kworker/u16:1+events_unbound06:20:116
801389992522,2cyclictest958820-21kworker/u16:1+flush-8:004:52:296
801380992523,2cyclictest770902-21kworker/u16:3+flush-8:003:42:124
10740372250,1chrt1074039-21kthreadcore05:47:357
801389992422,2cyclictest999146-21kworker/u16:1+events_unbound05:35:126
801389992422,2cyclictest1276564-21kworker/u16:0+events_unbound08:32:206
801389992422,2cyclictest1234716-21kworker/u16:4+events_unbound07:40:116
801389992422,2cyclictest1203984-21kworker/u16:0+events_unbound07:10:216
801389992422,2cyclictest1163010-21kworker/u16:5+events_unbound06:43:506
801389992422,2cyclictest1082760-21kworker/u16:2+events_unbound07:03:466
801389992422,2cyclictest1004946-21kworker/u16:3+flush-8:005:42:396
801380992421,2cyclictest1286841-21kworker/u16:1+flush-8:007:47:424
801380992421,2cyclictest1276564-21kworker/u16:0+flush-8:008:12:294
80139399230,21cyclictest1256193-21latency_hist07:17:127
801389992321,2cyclictest999146-21kworker/u16:1+events_unbound05:31:226
801389992321,2cyclictest916736-21kworker/u16:3+events_unbound04:49:116
801389992321,2cyclictest781558-21kworker/u16:2+events_unbound03:46:156
801389992321,2cyclictest1163010-21kworker/u16:5+events_unbound06:41:556
801389992321,2cyclictest1082760-21kworker/u16:2+flush-8:006:47:406
801389992321,2cyclictest1082760-21kworker/u16:2+events_unbound06:26:196
801389992321,2cyclictest1082225-21kworker/u16:0+events_unbound05:56:156
801389992220,2cyclictest781558-21kworker/u16:2+flush-8:004:22:476
801389992220,2cyclictest1082760-21kworker/u16:2+events_unbound07:13:426
801389992220,2cyclictest1082225-21kworker/u16:0+events_unbound06:03:276
80139399210,20cyclictest1215222-21latency_hist06:57:117
801389992119,2cyclictest999146-21kworker/u16:1+events_unbound05:26:116
801389992119,2cyclictest824923-21kworker/u16:5+events_unbound04:07:106
801389992119,2cyclictest1040991-21kworker/u16:4+events_unbound05:48:226
801389992118,2cyclictest1348120-21kworker/u16:1+events_unbound08:02:286
801389992118,2cyclictest1004946-21kworker/u16:3+flush-8:005:17:226
80139399200,18cyclictest1032119-21tune2fs05:27:317
801389992018,2cyclictest781558-21kworker/u16:2+events_unbound03:49:276
801389992018,2cyclictest781558-21kworker/u16:2+events_unbound03:49:276
801389992018,2cyclictest1276564-21kworker/u16:0+flush-8:008:17:476
801380992018,2cyclictest1040991-21kworker/u16:4+flush-8:005:47:354
80139399190,17cyclictest1286950-21latency_hist07:32:117
801389991917,2cyclictest916736-21kworker/u16:3+events_unbound04:36:196
801389991917,2cyclictest916736-21kworker/u16:3+events_unbound04:36:196
801380991916,2cyclictest1137551-21kworker/u16:3+flush-8:006:32:374
801380991915,3cyclictest999146-21kworker/u16:1+flush-8:005:37:434
801393991815,2cyclictest1061548-21awk05:42:127
80139399180,17cyclictest959105-21grep04:52:127
80139399180,17cyclictest643-21systemd-journal07:52:117
801389991816,2cyclictest1348120-21kworker/u16:1+flush-8:008:24:266
80139399170,15cyclictest1313-21snmpd06:25:377
80139399170,15cyclictest1092286-21latency_hist05:57:117
801389991715,2cyclictest1359339-21kworker/u16:4+events_unbound08:11:316
801389991714,2cyclictest770902-21kworker/u16:3+flush-8:003:57:286
801380991715,2cyclictest1430389-21kworker/u16:1+flush-8:008:57:274
801380991715,2cyclictest1430389-21kworker/u16:1+flush-8:008:52:424
801393991614,1cyclictest1389381-21awk08:22:117
801393991612,2cyclictest1101573-21ntpq05:57:427
80139399160,14cyclictest1313-21snmpd07:01:357
801380991613,3cyclictest1286841-21kworker/u16:1+flush-8:007:32:274
801393991512,2cyclictest811584-21munin-run03:42:117
80139399150,14cyclictest887291-21latency_hist04:17:137
80139399150,13cyclictest918018-21turbostat04:32:117
801384991513,2cyclictest1124266-21kworker/u16:1+flush-8:006:22:375
80139399140,2cyclictest1306491-21ntpq07:37:397
80139399140,12cyclictest1313-21snmpd08:06:277
801389991412,2cyclictest1296177-21kworker/u16:2+flush-8:007:52:266
80138499140,14cyclictest0-21swapper/508:09:225
801380991413,1cyclictest1430389-21kworker/u16:1+events_unbound09:02:374
801380991412,2cyclictest908529-21kworker/u16:0+flush-8:004:32:074
801380991412,2cyclictest1082760-21kworker/u16:2+flush-8:006:42:424
801393991312,0cyclictest0-21swapper/703:57:437
801393991310,2cyclictest1348397-21munin-run08:02:117
80139399130,5cyclictest1415016-21cut08:32:327
80139399130,3cyclictest988770-21ntpq05:02:407
80139399130,3cyclictest975295-21kthreadcore04:57:337
80139399130,2cyclictest999043-21ntpq05:07:467
80139399130,12cyclictest1461158-21munin-run08:57:117
80139399130,12cyclictest1313-21snmpd05:33:547
80139399130,11cyclictest948776-21latency_hist04:47:127
80139399130,11cyclictest1461875-21exim408:57:127
80139399130,11cyclictest1313-21snmpd06:47:297
801384991312,1cyclictest0-21swapper/504:33:245
801384991312,1cyclictest0-21swapper/504:33:245
801384991311,2cyclictest1234716-21kworker/u16:4+flush-8:007:27:375
80139399127,4cyclictest27304-21gsd-housekeepin04:02:357
801393991210,1cyclictest1164012-21awk06:32:127
801393991210,1cyclictest1123033-21awk06:12:117
801393991210,1cyclictest1123033-21awk06:12:117
801393991210,1cyclictest1112795-21awk06:07:117
80139399120,4cyclictest1441307-21cut08:47:127
80139399120,2cyclictest1185250-21/usr/sbin/munin06:42:297
80139399120,11cyclictest835236-21ntpq03:47:397
80139399120,11cyclictest835236-21ntpq03:47:397
80139399120,11cyclictest1313-21snmpd07:52:457
80139399120,11cyclictest1313-21snmpd07:42:277
80139399120,11cyclictest1313-21snmpd07:42:277
80139399120,11cyclictest1313-21snmpd04:11:047
80139399120,11cyclictest1287695-21/usr/sbin/munin07:32:297
80139399120,11cyclictest1165512-21cat06:32:307
80139399120,10cyclictest929799-21cat04:37:317
80139399120,10cyclictest1313-21snmpd06:12:167
80139399120,10cyclictest1022618-21kthreadcore05:22:347
801384991211,1cyclictest0-21swapper/508:22:285
801384991211,1cyclictest0-21swapper/508:13:195
801384991211,1cyclictest0-21swapper/507:17:025
801384991211,1cyclictest0-21swapper/507:07:355
801384991211,1cyclictest0-21swapper/507:06:285
801384991211,1cyclictest0-21swapper/506:34:385
801384991211,1cyclictest0-21swapper/505:40:525
80138499120,12cyclictest0-21swapper/504:45:425
80138499120,12cyclictest0-21swapper/504:23:475
80138499120,12cyclictest0-21swapper/503:55:535
80138099120,1cyclictest0-21swapper/404:26:544
80138099120,1cyclictest0-21swapper/404:16:394
80138099120,12cyclictest0-21swapper/405:20:014
801393991111,0cyclictest0-21swapper/706:37:257
80139399110,3cyclictest1367368-21kthreadcore08:07:357
80139399110,3cyclictest1001847-21sed05:12:347
80139399110,2cyclictest898300-21/usr/sbin/munin04:22:287
80139399110,2cyclictest888036-21/usr/sbin/munin04:17:427
80139399110,2cyclictest1420918-21/usr/sbin/munin08:37:277
80139399110,10cyclictest1441456-21/usr/sbin/munin08:47:457
80139399110,10cyclictest1390098-21/usr/sbin/munin08:22:347
80139399110,10cyclictest1313-21snmpd08:27:587
80139399110,10cyclictest1313-21snmpd06:17:407
80139399110,10cyclictest1313-21snmpd03:54:587
80139399110,10cyclictest1267998-21cat07:22:287
80139399110,10cyclictest1226200-21/usr/sbin/munin07:02:357
80139399110,10cyclictest1062266-21/usr/sbin/munin05:42:387
801384991110,1cyclictest0-21swapper/508:59:495
801384991110,1cyclictest0-21swapper/508:32:425
801384991110,1cyclictest0-21swapper/508:29:525
801384991110,1cyclictest0-21swapper/507:37:225
801384991110,1cyclictest0-21swapper/505:27:385
801384991110,1cyclictest0-21swapper/504:58:475
801384991110,1cyclictest0-21swapper/504:21:545
80138499110,1cyclictest0-21swapper/504:48:325
80138499110,11cyclictest0-21swapper/508:43:075
80138499110,11cyclictest0-21swapper/506:29:265
80138099119,2cyclictest1409215-21kworker/u16:3+events_unbound08:47:114
80138099118,2cyclictest1004946-21kworker/u16:3+flush-8:005:42:434
801380991110,1cyclictest0-21swapper/407:28:024
801380991110,1cyclictest0-21swapper/407:21:434
801380991110,1cyclictest0-21swapper/407:08:104
801380991110,1cyclictest0-21swapper/406:55:334
801380991110,1cyclictest0-21swapper/405:15:554
801380991110,1cyclictest0-21swapper/404:03:434
80138099110,1cyclictest0-21swapper/405:06:474
80138099110,11cyclictest0-21swapper/408:34:144
80138099110,11cyclictest0-21swapper/408:06:204
80138099110,11cyclictest0-21swapper/407:52:474
80138099110,11cyclictest0-21swapper/406:47:594
80138099110,11cyclictest0-21swapper/406:18:214
80138099110,11cyclictest0-21swapper/406:06:324
80138099110,11cyclictest0-21swapper/405:57:514
80138099110,11cyclictest0-21swapper/405:53:264
80138099110,11cyclictest0-21swapper/404:58:444
80139399100,9cyclictest26931-21Xorg03:47:127
80139399100,2cyclictest959736-21/usr/sbin/munin04:52:447
80138499109,1cyclictest0-21swapper/507:21:185
80138499109,1cyclictest0-21swapper/504:56:535
80138499109,1cyclictest0-21swapper/503:57:465
80138099109,1cyclictest0-21swapper/408:22:164
80138099109,1cyclictest0-21swapper/408:21:194
80138099109,1cyclictest0-21swapper/407:05:014
80138099109,1cyclictest0-21swapper/406:14:064
80138099109,1cyclictest0-21swapper/403:56:284
8013849998,1cyclictest0-21swapper/508:38:235
8013849998,1cyclictest0-21swapper/508:05:265
8013849998,1cyclictest0-21swapper/507:22:245
8013849998,1cyclictest0-21swapper/506:20:275
8013849998,1cyclictest0-21swapper/505:47:585
8013849998,1cyclictest0-21swapper/505:45:555
8013849998,1cyclictest0-21swapper/505:11:055
8013849998,1cyclictest0-21swapper/503:48:385
8013849998,1cyclictest0-21swapper/503:48:385
8013849997,1cyclictest0-21swapper/503:37:355
8013809998,1cyclictest0-21swapper/408:50:104
8013809998,1cyclictest0-21swapper/408:27:474
8013809998,1cyclictest0-21swapper/407:59:334
8013809998,1cyclictest0-21swapper/406:08:444
8013809998,1cyclictest0-21swapper/406:08:444
8013809998,1cyclictest0-21swapper/405:10:534
8013809998,1cyclictest0-21swapper/403:57:254
8013809996,2cyclictest908529-21kworker/u16:0+flush-8:004:42:444
8013809996,2cyclictest781558-21kworker/u16:2+flush-8:003:47:274
8013809996,2cyclictest781558-21kworker/u16:2+flush-8:003:47:274
8013849987,1cyclictest0-21swapper/508:21:315
8013849987,1cyclictest0-21swapper/505:17:145
8013849987,1cyclictest0-21swapper/505:15:115
8013809987,1cyclictest0-21swapper/408:09:204
8013809987,1cyclictest0-21swapper/407:22:504
8013809987,1cyclictest0-21swapper/405:27:164
8013809987,1cyclictest0-21swapper/404:07:214
8013849976,1cyclictest0-21swapper/509:02:405
8013849976,1cyclictest0-21swapper/508:01:305
8013849976,1cyclictest0-21swapper/507:51:435
8013849976,1cyclictest0-21swapper/505:35:505
8013849975,1cyclictest0-21swapper/503:42:145
8013849973,3cyclictest1-21systemd08:57:115
8013809976,1cyclictest0-21swapper/407:01:524
8013809976,1cyclictest0-21swapper/404:47:424
8013849965,1cyclictest0-21swapper/507:00:105
8013849965,1cyclictest0-21swapper/506:55:555
8013849965,1cyclictest0-21swapper/504:02:215
8013849964,1cyclictest13379372chrt07:54:575
8013849963,2cyclictest8872342chrt04:16:315
8013849963,2cyclictest1102536-21turbostat06:02:115
8013849961,2cyclictest991rcuop/707:32:305
8013849961,1cyclictest201rcu_preempt06:17:075
8013849960,6cyclictest0-21swapper/505:07:115
8013849960,3cyclictest0-21swapper/505:22:395
8013849960,2cyclictest0-21swapper/506:47:305
8013809965,1cyclictest0-21swapper/408:38:014
8013809965,1cyclictest0-21swapper/405:23:104
8013809960,5cyclictest1050986-21chrt05:33:464
8013809960,1cyclictest0-21swapper/407:13:164
1309350260,1sleep40-21swapper/407:42:314
1309350260,1sleep40-21swapper/407:42:314
8013849953,1cyclictest13081322chrt07:42:155
8013849953,1cyclictest13081322chrt07:42:155
8013849951,1cyclictest201rcu_preempt06:37:155
8013849950,4cyclictest908524-21chrt04:27:215
8013849950,4cyclictest876743-21taskset04:07:585
8013849950,4cyclictest1441439-21taskset08:47:135
8013849950,4cyclictest1185245-21chrt06:42:225
8013849950,4cyclictest1114067-21taskset06:07:285
8013849950,4cyclictest1114067-21taskset06:07:285
8013849950,4cyclictest1103266-21chrt06:02:155
8013849950,4cyclictest1091995-21taskset05:53:105
8013849950,2cyclictest0-21swapper/504:37:295
8013809950,4cyclictest929544-21chrt04:37:294
8013809950,4cyclictest918758-21chrt04:32:244
8013809950,4cyclictest918758-21chrt04:32:244
8013809950,4cyclictest1307094-21chrt07:37:574
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional