You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-15 - 04:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack1slot1.osadl.org (updated Tue Jul 15, 2025 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
107012215164,25sleep00-21swapper/019:05:080
125202212150,21sleep30-21swapper/319:06:173
126992204149,22sleep10-21swapper/119:08:361
124842202163,25sleep20-21swapper/219:05:462
253442630,0sleep00-21swapper/022:16:130
71482600,0sleep25760-21diskmemload23:20:152
12884991717,0cyclictest9-21ksoftirqd/022:46:250
12884991717,0cyclictest9-21ksoftirqd/021:26:350
304932160,0sleep10-21swapper/120:50:241
12884991616,0cyclictest9-21ksoftirqd/022:55:190
12884991515,0cyclictest9-21ksoftirqd/023:56:140
12884991515,0cyclictest9-21ksoftirqd/023:35:190
12884991515,0cyclictest9-21ksoftirqd/021:46:290
12884991515,0cyclictest9-21ksoftirqd/021:36:210
1288499149,4cyclictest9-21ksoftirqd/000:26:210
12884991414,0cyclictest9-21ksoftirqd/023:28:060
12884991414,0cyclictest9-21ksoftirqd/022:41:560
12884991414,0cyclictest9-21ksoftirqd/000:00:010
12884991413,1cyclictest9-21ksoftirqd/023:05:520
1290299131,11cyclictest15458-21munin-node23:56:523
1288499139,3cyclictest9-21ksoftirqd/023:51:310
1288499139,3cyclictest9-21ksoftirqd/023:51:300
1288499135,5cyclictest9-21ksoftirqd/022:52:000
12884991313,0cyclictest9-21ksoftirqd/023:44:050
12884991313,0cyclictest9-21ksoftirqd/023:19:470
12884991313,0cyclictest9-21ksoftirqd/023:12:500
12884991313,0cyclictest9-21ksoftirqd/022:05:490
12884991313,0cyclictest9-21ksoftirqd/022:02:400
12884991313,0cyclictest9-21ksoftirqd/022:02:390
12884991313,0cyclictest9-21ksoftirqd/021:34:250
12884991313,0cyclictest9-21ksoftirqd/021:16:490
12884991313,0cyclictest9-21ksoftirqd/000:21:230
1288499129,2cyclictest6107-21fgrep21:50:230
12884991212,0cyclictest9-21ksoftirqd/023:20:070
12884991212,0cyclictest9-21ksoftirqd/022:26:020
12884991212,0cyclictest9-21ksoftirqd/022:20:080
12884991212,0cyclictest9-21ksoftirqd/000:32:500
1288499119,1cyclictest9-21ksoftirqd/022:32:360
1288499118,2cyclictest16313-21sed20:20:210
1288499118,0cyclictest9-21ksoftirqd/000:36:580
1288499117,4cyclictest9-21ksoftirqd/021:11:590
12884991111,0cyclictest9-21ksoftirqd/022:10:300
12884991111,0cyclictest9-21ksoftirqd/000:08:340
12884991110,1cyclictest9-21ksoftirqd/021:40:320
12884991110,1cyclictest9-21ksoftirqd/021:20:020
12884991110,1cyclictest9-21ksoftirqd/000:10:170
1290299109,1cyclictest35-21ksoftirqd/321:20:003
1290299109,1cyclictest35-21ksoftirqd/300:35:013
1289499101,7cyclictest15458-21munin-node21:02:512
1289499100,10cyclictest0-21swapper/221:15:482
1288499109,1cyclictest9-21ksoftirqd/022:35:130
1288499109,1cyclictest9-21ksoftirqd/022:35:120
1288499109,1cyclictest9-21ksoftirqd/020:35:180
1288499109,1cyclictest9-21ksoftirqd/020:25:010
1288499109,1cyclictest9-21ksoftirqd/019:10:000
1288499109,1cyclictest9-21ksoftirqd/000:18:340
1288499106,4cyclictest9-21ksoftirqd/023:32:140
1288499103,7cyclictest9-21ksoftirqd/019:21:470
12884991010,0cyclictest9-21ksoftirqd/023:45:510
12884991010,0cyclictest9-21ksoftirqd/023:00:110
12884991010,0cyclictest9-21ksoftirqd/019:18:220
1288499100,9cyclictest9779-21gzip21:55:220
129029999,0cyclictest35-21ksoftirqd/300:35:223
129029998,1cyclictest35-21ksoftirqd/323:10:003
129029998,1cyclictest35-21ksoftirqd/321:05:003
129029998,1cyclictest35-21ksoftirqd/300:00:133
129029997,1cyclictest35-21ksoftirqd/323:00:123
129029997,1cyclictest19414-21swap23:35:213
129029994,2cyclictest5760-21diskmemload21:50:583
129029990,5cyclictest0-21swapper/321:20:133
128949990,9cyclictest0-21swapper/222:19:582
128949990,6cyclictest0-21swapper/222:00:132
128949990,6cyclictest0-21swapper/222:00:122
128949990,3cyclictest0-21swapper/221:25:162
128859990,4cyclictest0-21swapper/100:30:131
128849999,0cyclictest9-21ksoftirqd/021:05:110
128849999,0cyclictest9-21ksoftirqd/020:57:010
128849999,0cyclictest9-21ksoftirqd/020:05:230
128849999,0cyclictest9-21ksoftirqd/020:00:140
128849999,0cyclictest9-21ksoftirqd/019:32:380
128849999,0cyclictest9-21ksoftirqd/019:32:380
128849998,1cyclictest9-21ksoftirqd/020:45:220
128849998,1cyclictest9-21ksoftirqd/020:40:150
128849992,7cyclictest9-21ksoftirqd/021:01:030
128849990,1cyclictest9-21ksoftirqd/019:35:160
22498280,0sleep20-21swapper/222:13:402
129029988,0cyclictest35-21ksoftirqd/300:11:003
129029987,1cyclictest35-21ksoftirqd/323:15:153
129029987,1cyclictest35-21ksoftirqd/322:30:153
129029987,1cyclictest35-21ksoftirqd/322:15:183
129029987,1cyclictest35-21ksoftirqd/321:49:333
129029987,1cyclictest35-21ksoftirqd/321:35:003
129029987,1cyclictest35-21ksoftirqd/320:30:003
129029987,1cyclictest35-21ksoftirqd/320:00:003
129029982,5cyclictest14622-21/usr/sbin/munin20:20:223
129029981,5cyclictest5760-21diskmemload22:08:553
129029981,4cyclictest29049-21diskmemload21:39:173
129029980,6cyclictest0-21swapper/323:25:423
129029980,5cyclictest254-21systemd-journal21:29:273
128949987,1cyclictest28-21ksoftirqd/200:27:132
128949986,1cyclictest60950irq/123-eno121:25:012
128949986,1cyclictest60950irq/123-eno100:35:172
128949986,1cyclictest60950irq/123-eno100:25:002
128949986,1cyclictest28-21ksoftirqd/221:30:212
128949986,1cyclictest28-21ksoftirqd/219:55:112
128949986,1cyclictest28-21ksoftirqd/200:00:002
128949980,8cyclictest0-21swapper/221:39:422
128949980,2cyclictest254-21systemd-journal19:40:012
128949980,0cyclictest0-21swapper/223:39:172
128859980,5cyclictest0-21swapper/122:50:171
128849988,0cyclictest9-21ksoftirqd/020:50:000
128849988,0cyclictest9-21ksoftirqd/020:30:030
128849988,0cyclictest9-21ksoftirqd/020:15:280
128849988,0cyclictest9-21ksoftirqd/019:25:170
128849987,1cyclictest9-21ksoftirqd/019:45:110
128849980,8cyclictest9-21ksoftirqd/020:10:010
128849980,8cyclictest9-21ksoftirqd/019:55:120
128849980,8cyclictest9-21ksoftirqd/019:50:030
128849980,7cyclictest9-21ksoftirqd/019:41:050
129029976,1cyclictest35-21ksoftirqd/323:25:013
129029976,1cyclictest19864-21ntpq19:20:193
129029975,1cyclictest430-21ssh22:26:343
129029975,1cyclictest32056-21ssh23:53:393
129029975,1cyclictest32056-21ssh23:53:393
129029975,1cyclictest27315-21ssh23:45:403
129029975,1cyclictest15162-21cat23:30:163
129029975,1cyclictest10729-21ntpq22:40:173
129029973,1cyclictest5760-21diskmemload22:35:493
129029973,1cyclictest5760-21diskmemload22:35:493
129029972,3cyclictest32510-21runrttasks21:43:133
129029971,4cyclictest5760-21diskmemload00:15:483
129029971,3cyclictest5760-21diskmemload23:12:193
129029971,3cyclictest0-21swapper/322:50:123
129029970,5cyclictest341rcuc/322:15:003
129029970,5cyclictest0-21swapper/323:40:263
129029970,4cyclictest19205-21/usr/sbin/munin00:20:143
129029970,2cyclictest0-21swapper/322:20:363
129029970,1cyclictest0-21swapper/319:35:183
128949976,1cyclictest8945-21cat00:05:152
128949976,1cyclictest60950irq/123-eno122:46:072
128949976,1cyclictest60950irq/123-eno122:20:132
128949976,1cyclictest60950irq/123-eno121:58:542
128949976,1cyclictest60950irq/123-eno100:31:272
128949976,1cyclictest60950irq/123-eno100:18:312
128949976,1cyclictest28-21ksoftirqd/220:30:122
128949976,1cyclictest24836-21sensors19:30:232
128949976,1cyclictest24836-21sensors19:30:222
128949976,0cyclictest60950irq/123-eno122:30:072
128949976,0cyclictest60950irq/123-eno121:46:022
128949975,1cyclictest60950irq/123-eno120:05:202
128949975,1cyclictest30041-21cat23:50:182
128949975,1cyclictest30041-21cat23:50:172
128949975,1cyclictest1449-21ntpd22:05:022
128949975,1cyclictest1449-21ntpd21:10:022
128949974,1cyclictest15694-21sensors23:30:192
128949974,1cyclictest0-21swapper/223:15:132
128949973,2cyclictest13445-21sh23:28:332
128949973,1cyclictest254-21systemd-journal21:07:412
128949970,4cyclictest5865-21apt-get22:35:132
128949970,4cyclictest5865-21apt-get22:35:132
128949970,4cyclictest0-21swapper/222:59:042
128859970,7cyclictest0-21swapper/122:07:411
128859970,6cyclictest0-21swapper/122:16:511
128859970,6cyclictest0-21swapper/121:00:191
128859970,5cyclictest0-21swapper/123:52:131
128859970,5cyclictest0-21swapper/123:52:121
2461260,0sleep10-21swapper/123:13:571
129029965,1cyclictest7844-21ssh21:10:253
129029965,1cyclictest35-21ksoftirqd/320:45:003
129029965,1cyclictest35-21ksoftirqd/320:40:013
129029965,1cyclictest35-21ksoftirqd/319:30:013
129029965,1cyclictest24312-21cat19:30:183
129029965,1cyclictest24312-21cat19:30:173
129029965,1cyclictest14741-21ssh22:02:403
129029965,1cyclictest14741-21ssh22:02:393
129029965,1cyclictest10858-21ssh21:57:173
129029964,1cyclictest8000-21ssh00:05:073
129029964,1cyclictest27513-21cat20:45:183
129029964,1cyclictest0-21swapper/321:05:203
129029963,1cyclictest254-21systemd-journal19:45:093
129029963,1cyclictest254-21systemd-journal19:10:083
129029962,2cyclictest5760-21diskmemload22:55:193
129029961,4cyclictest13285-21apt-get22:45:123
129029960,2cyclictest0-21swapper/300:25:173
128949966,0cyclictest60950irq/123-eno123:45:062
128949966,0cyclictest60950irq/123-eno123:40:112
128949966,0cyclictest60950irq/123-eno123:05:122
128949966,0cyclictest60950irq/123-eno122:40:452
128949966,0cyclictest60950irq/123-eno122:25:182
128949966,0cyclictest60950irq/123-eno121:40:152
128949966,0cyclictest60950irq/123-eno120:51:472
128949966,0cyclictest60950irq/123-eno119:36:472
128949966,0cyclictest60950irq/123-eno100:10:422
128949965,1cyclictest60950irq/123-eno123:10:152
128949965,1cyclictest60950irq/123-eno100:00:122
128949965,1cyclictest28-21ksoftirqd/220:22:432
128949965,1cyclictest28-21ksoftirqd/219:45:112
128949965,0cyclictest60950irq/123-eno123:00:142
128949964,1cyclictest60950irq/123-eno120:35:192
128949964,1cyclictest1449-21ntpd20:15:032
128949961,4cyclictest17164-21tail22:50:142
128949961,3cyclictest2185-21users19:50:252
128949961,2cyclictest0-21swapper/221:50:142
128859966,0cyclictest0-21swapper/122:56:551
128859965,1cyclictest0-21swapper/123:06:041
128859962,3cyclictest0-21swapper/123:15:241
128859961,2cyclictest0-21swapper/119:40:151
128859960,6cyclictest0-21swapper/121:15:151
128859960,6cyclictest0-21swapper/120:24:441
128859960,5cyclictest0-21swapper/119:55:151
128859960,4cyclictest0-21swapper/121:35:201
128859960,4cyclictest0-21swapper/120:05:131
128859960,4cyclictest0-21swapper/100:21:191
128859960,3cyclictest0-21swapper/123:45:531
128859960,3cyclictest0-21swapper/123:00:141
128859960,2cyclictest0-21swapper/120:25:121
128859960,2cyclictest0-21swapper/100:04:371
128859960,1cyclictest0-21swapper/123:20:141
129029955,0cyclictest35-21ksoftirqd/320:50:163
129029954,1cyclictest35-21ksoftirqd/320:30:153
129029952,1cyclictest254-21systemd-journal20:58:553
129029951,1cyclictest15948-21smartctl19:15:013
129029950,5cyclictest0-21swapper/319:40:493
129029950,3cyclictest0-21swapper/320:05:183
129029950,1cyclictest254-21systemd-journal20:04:593
129029950,0cyclictest0-21swapper/320:16:313
128949955,0cyclictest60950irq/123-eno120:55:132
128949955,0cyclictest60950irq/123-eno120:40:202
128949954,1cyclictest60950irq/123-eno119:20:212
128949952,1cyclictest28-21ksoftirqd/220:14:092
128949950,4cyclictest21079-21/usr/sbin/munin19:25:162
128949950,4cyclictest0-21swapper/220:25:142
128949950,2cyclictest254-21systemd-journal19:11:092
128949950,2cyclictest0-21swapper/220:45:132
128949950,1cyclictest0-21swapper/220:00:142
128859955,0cyclictest0-21swapper/122:42:061
128859955,0cyclictest0-21swapper/121:25:491
128859953,1cyclictest84322sleep122:37:391
128859953,1cyclictest84322sleep122:37:381
128859953,1cyclictest12142-21chrt22:00:121
128859953,1cyclictest12142-21chrt22:00:121
128859952,0cyclictest0-21swapper/119:35:201
128859951,2cyclictest0-21swapper/122:10:131
128859951,2cyclictest0-21swapper/121:50:141
128859950,5cyclictest0-21swapper/120:10:191
128859950,5cyclictest0-21swapper/100:07:491
128859950,4cyclictest0-21swapper/121:55:321
128859950,4cyclictest0-21swapper/121:43:541
128859950,4cyclictest0-21swapper/100:28:051
128859950,3cyclictest0-21swapper/123:55:191
128859950,3cyclictest0-21swapper/122:49:391
128859950,2cyclictest0-21swapper/121:30:461
128859950,2cyclictest0-21swapper/100:12:181
128859950,0cyclictest0-21swapper/123:31:201
128859950,0cyclictest0-21swapper/119:10:121
129029942,1cyclictest1017-21grep19:50:173
129029940,1cyclictest0-21swapper/320:10:163
128949940,1cyclictest254-21systemd-journal19:15:082
128859943,1cyclictest287162sleep120:49:511
128859943,1cyclictest23874-21chrt20:38:261
128859942,1cyclictest207072sleep123:38:071
128859942,1cyclictest0-21swapper/119:30:141
128859942,1cyclictest0-21swapper/119:30:131
128859941,2cyclictest0-21swapper/123:27:231
128859941,1cyclictest0-21swapper/100:15:121
128859940,4cyclictest0-21swapper/120:01:401
128859940,4cyclictest0-21swapper/119:16:111
128859940,3cyclictest0-21swapper/123:40:161
128859940,3cyclictest0-21swapper/122:30:361
128859940,3cyclictest0-21swapper/122:25:221
128859940,3cyclictest0-21swapper/121:46:261
128859940,3cyclictest0-21swapper/121:21:321
128859940,3cyclictest0-21swapper/119:50:181
128859940,2cyclictest0-21swapper/120:45:001
128859940,2cyclictest0-21swapper/100:37:141
128859940,1cyclictest0-21swapper/121:10:241
128859940,0cyclictest0-21swapper/122:20:201
128859940,0cyclictest0-21swapper/120:55:191
128859940,0cyclictest0-21swapper/120:15:121
128859932,1cyclictest55312sleep121:07:171
128859931,1cyclictest0-21swapper/120:30:131
128859931,1cyclictest0-21swapper/119:45:121
128859930,2cyclictest0-21swapper/119:25:131
128859930,0cyclictest0-21swapper/119:20:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional