You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 03:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackeslot2.osadl.org (updated Tue May 13, 2025 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293422121690,2sleep30-21swapper/319:15:333
302183921070,1sleep10-21swapper/120:30:251
302104121000,2sleep30-21swapper/320:30:233
31239022370,2sleep30-21swapper/321:40:243
2920346992522,2cyclictest2920343-21cyclictest00:20:001
25010912517,3phc2sys0-21swapper/322:18:453
25010912418,2phc2sys0-21swapper/321:48:393
25010912418,2phc2sys0-21swapper/319:54:133
25010912418,2phc2sys0-21swapper/300:11:153
25010912417,3phc2sys0-21swapper/322:08:433
25010912417,2phc2sys0-21swapper/323:05:553
25010912417,2phc2sys0-21swapper/322:50:033
25010912417,2phc2sys0-21swapper/321:57:493
25010912417,2phc2sys0-21swapper/321:05:033
25010912417,2phc2sys0-21swapper/320:25:203
25010912417,2phc2sys0-21swapper/300:40:023
25010912417,2phc2sys0-21swapper/300:30:363
326875622319,2sleep30-21swapper/323:02:303
308631122318,3sleep30-21swapper/321:20:173
2920345992320,2cyclictest3261588-21/usr/sbin/munin23:00:000
2920345992316,6cyclictest3407002-21kthreadcore00:20:260
25010912318,3phc2sys0-21swapper/322:13:433
25010912317,3phc2sys0-21swapper/323:51:553
25010912317,3phc2sys0-21swapper/321:31:313
25010912317,3phc2sys0-21swapper/319:25:203
25010912317,3phc2sys0-21swapper/300:01:573
25010912317,2phc2sys0-21swapper/323:40:573
25010912317,2phc2sys0-21swapper/323:39:373
25010912317,2phc2sys0-21swapper/323:30:233
25010912317,2phc2sys0-21swapper/323:26:553
25010912317,2phc2sys0-21swapper/322:36:453
25010912317,2phc2sys0-21swapper/322:26:233
25010912317,2phc2sys0-21swapper/320:46:323
25010912317,2phc2sys0-21swapper/320:17:343
25010912317,2phc2sys0-21swapper/319:56:383
25010912317,2phc2sys0-21swapper/319:30:573
25010912317,2phc2sys0-21swapper/319:20:353
25010912317,2phc2sys0-21swapper/319:10:093
25010912317,2phc2sys0-21swapper/300:21:413
25010912317,2phc2sys0-21swapper/300:07:073
330364822218,2sleep30-21swapper/323:22:223
321674522217,3sleep30-21swapper/322:33:123
313926522218,2sleep30-21swapper/321:50:073
2920345992219,2cyclictest3121133-21grep21:40:020
2920345992215,6cyclictest3169162-21kthreadcore22:05:260
25010912217,3phc2sys0-21swapper/323:15:393
25010912217,3phc2sys0-21swapper/323:12:513
25010912217,3phc2sys0-21swapper/322:21:293
25010912217,3phc2sys0-21swapper/321:11:553
25010912217,3phc2sys0-21swapper/321:01:533
25010912217,3phc2sys0-21swapper/320:57:413
25010912217,3phc2sys0-21swapper/320:52:363
25010912217,3phc2sys0-21swapper/320:41:583
25010912217,3phc2sys0-21swapper/320:20:053
25010912217,3phc2sys0-21swapper/320:02:213
25010912217,3phc2sys0-21swapper/319:49:043
25010912217,3phc2sys0-21swapper/319:38:023
342963422117,2sleep20-21swapper/200:35:182
342690622117,2sleep20-21swapper/200:31:302
340033622117,2sleep20-21swapper/200:16:342
338522422117,2sleep20-21swapper/200:10:062
338360422117,2sleep20-21swapper/200:08:012
337548222117,2sleep20-21swapper/200:04:372
334689122117,2sleep20-21swapper/223:46:172
333201222117,2sleep20-21swapper/223:40:082
332913822117,2sleep20-21swapper/223:36:022
332094422117,2sleep20-21swapper/223:31:552
330562022117,2sleep20-21swapper/223:25:042
330263022117,2sleep20-21swapper/223:20:402
329526522117,2sleep20-21swapper/223:17:552
328688422117,2sleep20-21swapper/223:13:312
325920722117,2sleep20-21swapper/222:56:102
325174822117,2sleep20-21swapper/222:53:232
324386022117,2sleep20-21swapper/222:49:452
323288522117,2sleep20-21swapper/222:40:562
322522922117,2sleep20-21swapper/222:37:522
320961222117,2sleep20-21swapper/222:30:182
320797422117,2sleep20-21swapper/222:28:142
319689622117,2sleep20-21swapper/222:20:282
319029322117,2sleep20-21swapper/222:18:042
317119322117,2sleep20-21swapper/222:05:332
316371722117,2sleep20-21swapper/222:02:372
316330922117,2sleep30-21swapper/322:01:583
315384322117,2sleep20-21swapper/221:55:512
314529922117,2sleep20-21swapper/221:51:112
313647322117,2sleep20-21swapper/221:46:262
312860922117,2sleep20-21swapper/221:43:172
307744022117,2sleep20-21swapper/221:15:072
307662622117,2sleep20-21swapper/221:14:192
303362022117,2sleep20-21swapper/220:40:252
302699122117,2sleep20-21swapper/220:35:242
298296522117,2sleep20-21swapper/219:58:452
295853322117,2sleep20-21swapper/219:35:452
292601622117,2sleep20-21swapper/219:10:482
2920346992113,3cyclictest0-21swapper/121:10:021
2920345992118,2cyclictest3375897-21apt-key00:05:010
2920345992118,2cyclictest3235379-21turbostat22:45:010
2920345992118,2cyclictest3056324-21/usr/sbin/munin21:00:000
2920345992116,1cyclictest17350irq/127-ahci[0000:00:17.0]22:30:020
25010912117,2phc2sys0-21swapper/323:56:373
25010912117,2phc2sys0-21swapper/323:45:073
25010912117,2phc2sys0-21swapper/322:40:593
25010912117,2phc2sys0-21swapper/321:18:433
25010912117,2phc2sys0-21swapper/300:26:003
25010912117,2phc2sys0-21swapper/300:16:213
25010912116,3phc2sys0-21swapper/320:36:133
25010912115,2phc2sys0-21swapper/320:10:443
306802422017,2sleep20-21swapper/221:06:542
305594522017,2sleep20-21swapper/220:56:522
304379122017,2sleep20-21swapper/220:46:062
302587622017,2sleep20-21swapper/220:33:522
301375122017,2sleep20-21swapper/220:23:182
300738022017,2sleep20-21swapper/220:17:492
299511122017,2sleep20-21swapper/220:09:032
299491522017,2sleep30-21swapper/320:06:323
298899222017,2sleep20-21swapper/220:03:102
297678222017,2sleep20-21swapper/219:51:442
297067022017,2sleep20-21swapper/219:45:552
296489022017,2sleep20-21swapper/219:44:262
295281222017,2sleep20-21swapper/219:34:252
294669422017,2sleep20-21swapper/219:28:362
294059822017,2sleep20-21swapper/219:22:582
2920345992018,1cyclictest2920343-21cyclictest00:15:000
2920345992017,2cyclictest8799-21cron00:20:000
2920345992017,2cyclictest3032048-21sh20:40:010
2920345992017,2cyclictest1726-21systemd-logind21:35:010
2920345992017,2cyclictest0-21swapper/020:10:020
2920345992014,5cyclictest0-21swapper/023:55:020
2920345992011,5cyclictest3188329-21awk22:15:290
25010912017,2phc2sys0-21swapper/322:55:553
25010912017,2phc2sys0-21swapper/321:35:273
25010912017,2phc2sys0-21swapper/319:40:483
335848621916,1sleep20-21swapper/223:55:022
2920349991916,1cyclictest0-21swapper/222:10:262
292034699199,4cyclictest3131126-21fschecks_time21:45:211
2920346991916,2cyclictest3165197-21latency_hist22:05:001
2920346991916,2cyclictest2240602-1atopacctd23:57:211
2920346991912,3cyclictest3332107-21/usr/sbin/munin23:40:241
2920346991911,3cyclictest0-21swapper/122:55:001
292034599197,11cyclictest171ktimers/000:35:180
2920345991916,2cyclictest1695-21dbus-daemon23:05:010
2920345991910,5cyclictest3158242-21kthreadcore22:00:230
924598180,8rtkit-daemon3050301-21munin-plugin-st20:55:010
2920346991815,2cyclictest3121193-21cut21:40:021
292034599189,5cyclictest0-21swapper/022:15:010
292034599188,6cyclictest3422722-21kthreadcore00:30:260
292034599188,5cyclictest0-21swapper/023:25:010
292034599186,7cyclictest0-21swapper/023:40:170
2920345991815,2cyclictest8799-21cron19:40:010
2920345991815,2cyclictest2934788-21cut19:20:010
2920345991812,5cyclictest3376484-21/usr/sbin/munin00:05:270
2920345991812,2cyclictest0-21swapper/023:33:510
2920345991811,6cyclictest0-21swapper/022:55:020
2920345991810,4cyclictest3309243-21kthreadcore23:25:270
292034599180,16cyclictest3367038-21latency_hist00:00:010
2920349991715,1cyclictest0-21swapper/220:55:012
2920349991710,5cyclictest0-21swapper/221:28:442
2920346991714,2cyclictest3019902-21idleruntime-cro20:30:011
292034699171,3cyclictest178850irq/121-eno123:45:171
2920346991713,3cyclictest3057867-21kthreadcore21:00:231
2920346991710,4cyclictest2830101-21kworker/u16:2-xprtiod21:54:031
292034599178,2cyclictest0-21swapper/023:10:180
292034599176,6cyclictest0-21swapper/022:30:170
2920345991714,2cyclictest3013914-21turbostat20:25:010
2920345991710,6cyclictest3130955-21cstates21:45:200
924598160,2rtkit-daemon0-21swapper/120:48:521
31753952160,1sleep13175398-21sed22:10:241
2920349991614,1cyclictest0-21swapper/200:30:002
2920349991613,1cyclictest0-21swapper/223:09:592
2920349991613,1cyclictest0-21swapper/221:20:212
292034699168,4cyclictest3001922-21aten2.4-expect20:15:151
292034699168,3cyclictest0-21swapper/122:48:441
292034699167,4cyclictest0-21swapper/122:20:301
2920346991613,2cyclictest3385142-21sh00:10:011
2920346991613,2cyclictest3331608-21sh23:40:001
2920346991613,2cyclictest3217955-21sh22:35:011
2920346991613,2cyclictest3073422-21kthreadcore21:10:251
2920346991613,2cyclictest2983296-21munin-plugin-st20:00:011
2920346991613,2cyclictest2240602-1atopacctd00:10:251
2920346991613,2cyclictest1695-21dbus-daemon22:00:001
2920346991613,2cyclictest0-21swapper/100:30:021
2920346991613,2cyclictest0-21swapper/100:25:011
2920346991612,3cyclictest3258228-21pmu-power22:55:281
2920346991612,3cyclictest3048773-21kthreadcore20:50:271
2920346991612,3cyclictest2927598-21kthreadcore19:15:021
2920346991612,3cyclictest2240602-1atopacctd23:25:261
2920346991610,2cyclictest3068280-21diskmemload23:51:581
292034599168,4cyclictest3219742-21kthreadcore22:35:270
2920345991613,2cyclictest2240602-1atopacctd21:24:520
2920345991613,2cyclictest1761-21gdbus23:20:010
2920345991613,2cyclictest1726-21systemd-logind20:35:010
2920345991612,3cyclictest3305336-21kworker/u16:2+nfsiod23:49:370
2920345991612,3cyclictest2842082-21kworker/u16:0+nfsiod21:19:010
2920349991513,1cyclictest0-21swapper/223:55:022
2920349991513,1cyclictest0-21swapper/221:35:212
2920349991513,1cyclictest0-21swapper/219:20:012
2920349991512,1cyclictest0-21swapper/220:15:002
2920349991512,1cyclictest0-21swapper/200:20:252
2920349991511,2cyclictest0-21swapper/223:00:272
292034699159,2cyclictest3068280-21diskmemload23:33:361
292034699157,3cyclictest0-21swapper/123:25:021
292034699156,7cyclictest1695-21dbus-daemon20:45:001
2920346991512,2cyclictest3432630-21kthreadcore00:35:281
2920346991512,2cyclictest3281659-21kthreadcore23:10:261
2920346991512,2cyclictest3170818-21sh22:05:291
2920346991512,2cyclictest2240602-1atopacctd22:35:361
2920346991512,2cyclictest2240602-1atopacctd19:10:241
2920346991512,2cyclictest1695-21dbus-daemon21:35:011
2920346991512,2cyclictest0-21swapper/119:50:011
2920346991511,3cyclictest3095241-21aten_repower_po21:25:171
2920346991511,3cyclictest3027030-21tail20:35:231
2920346991511,3cyclictest2949931-21kthreadcore19:30:251
292034699150,13cyclictest3013918-21dump-pmu-power20:25:011
292034599159,5cyclictest0-21swapper/023:40:010
292034599152,6cyclictest1-21systemd22:50:010
2920345991512,2cyclictest770595-21systemd-journal21:05:010
2920345991512,2cyclictest3414611-21sed00:25:260
2920345991512,2cyclictest3038269-21latency_hist20:45:010
2920345991512,2cyclictest2926569-21systemctl19:15:010
2920345991512,2cyclictest0-21swapper/023:10:020
2920345991511,3cyclictest3124247-21kthreadcore21:40:260
2920345991511,3cyclictest3050316-21kworker/u16:1+nfsiod22:21:300
324190821411,2sleep30-21swapper/322:46:313
2920349991411,1cyclictest0-21swapper/220:30:012
292034699141,4cyclictest0-21swapper/100:05:021
2920346991411,2cyclictest3423883-21sh00:30:261
2920346991411,2cyclictest3263217-21ps23:00:261
2920346991411,2cyclictest3209327-21journal-offline22:30:011
2920346991411,2cyclictest2960124-21irqcore19:40:231
2920346991411,2cyclictest2941665-21fschecks_count19:25:191
2920346991411,2cyclictest2240602-1atopacctd21:21:381
2920346991410,3cyclictest3052851-21kthreadcore20:55:221
2920346991410,3cyclictest3001107-21unixbench_singl20:10:331
2920346991410,3cyclictest2956027-21kthreadcore19:35:241
292034699140,5cyclictest3077606-21ssh21:15:191
292034699140,5cyclictest2993145-21tr20:05:251
292034599148,5cyclictest2981496-21tr19:55:250
292034599147,5cyclictest770595-21systemd-journal20:00:450
292034599147,1cyclictest0-21swapper/021:10:580
292034599145,8cyclictest3005350-21tr20:15:220
292034599145,5cyclictest0-21swapper/021:25:010
292034599144,6cyclictest0-21swapper/019:33:450
2920345991411,2cyclictest3148255-21aten_repower_en21:55:190
2920345991411,2cyclictest3144081-21latency21:50:240
2920345991411,2cyclictest3067282-21needreboot21:05:260
2920345991411,2cyclictest2971210-21systemd-run19:50:010
2920345991411,2cyclictest2964994-21sh19:45:010
2920345991411,2cyclictest2935682-21fschecks_time19:20:230
2920345991410,3cyclictest2998165-21kthreadcore20:10:260
2920345991410,3cyclictest2975074-21kthreadcore19:50:260
2920349991311,1cyclictest0-21swapper/221:35:022
292034699139,3cyclictest2976148-21pmu-power19:50:291
292034699139,2cyclictest0-21swapper/119:10:021
292034699138,3cyclictest178850irq/121-eno123:05:231
292034699134,8cyclictest3183112-1ssh22:15:091
292034699131,5cyclictest0-21swapper/122:42:291
2920346991310,2cyclictest3291198-21kthreadcore23:15:281
2920346991310,2cyclictest3122157-21gltestperf21:40:211
2920346991310,2cyclictest2983540-21head20:00:181
2920346991310,2cyclictest2935974-21irqcore19:20:251
292034599139,3cyclictest2920343-21cyclictest20:45:240
292034599134,5cyclictest0-21swapper/019:25:080
2920345991310,2cyclictest3014973-21tail20:25:230
2920345991310,2cyclictest0-21swapper/019:10:020
292034999119,1cyclictest0-21swapper/221:00:292
29203499986,1cyclictest0-21swapper/219:10:022
250109170,2phc2sys0-21swapper/321:25:113
250109170,2phc2sys0-21swapper/319:05:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional