You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 13:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Thu Apr 23, 2026 12:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
33241322930,1sleep50-21swapper/511:54:255
2686182994442,2cyclictest2668039-21kworker/u16:2+events_unbound07:24:256
2686182994341,2cyclictest3394572-21kworker/u16:4+events_unbound12:24:246
26857712424,32sleep70-21swapper/707:07:407
268560923928,8sleep40-21swapper/407:05:214
2686183993833,3cyclictest2795453-21latency_hist08:04:267
268589623727,7sleep50-21swapper/507:09:235
2686183993534,0cyclictest0-21swapper/712:14:207
2686182993433,1cyclictest3066616-21kworker/u16:2+events_unbound12:01:086
2686183993329,2cyclictest3084287-21kthreadcore10:14:447
2686182993330,2cyclictest3066616-21kworker/u16:2+events_unbound10:09:436
2686182993230,2cyclictest2795492-21kworker/u16:3+events_unbound08:09:416
2686182993129,2cyclictest1947538-21kworker/u16:6+flush-8:007:14:416
2686183993029,0cyclictest0-21swapper/709:24:487
2686183993028,1cyclictest2993756-21ssh09:39:217
2686182993028,2cyclictest3101715-21kworker/u16:0+flush-8:011:04:526
2686182993028,2cyclictest3101715-21kworker/u16:0+flush-8:011:04:526
2686182993027,2cyclictest2717895-21kworker/u16:0+events_unbound07:24:446
2686182992926,2cyclictest2725960-21kworker/u16:1+events_unbound07:29:416
31497082280,1chrt0-21swapper/710:39:507
26974442280,1chrt2686172-21cyclictest07:14:297
2686183992827,0cyclictest0-21swapper/710:54:447
2686182992827,1cyclictest3248986-21kworker/u16:3+events_unbound12:34:336
2686182992826,2cyclictest3066616-21kworker/u16:2+events_unbound10:49:056
2686182992826,2cyclictest3017093-21kworker/u16:1+events_unbound10:19:526
2686182992826,2cyclictest3017093-21kworker/u16:1+events_unbound09:48:366
2686183992724,2cyclictest3349700-21ssh12:04:407
2686182992725,2cyclictest2717895-21kworker/u16:0+flush-8:007:49:496
268562022717,7sleep60-21swapper/607:05:316
31141142260,1chrt3114113-21rm10:25:487
31141142260,1chrt3114113-21rm10:25:487
2686182992523,2cyclictest3101715-21kworker/u16:0+events_unbound10:55:366
2686183992422,1cyclictest0-21swapper/708:44:447
2686182992423,1cyclictest2879147-21kworker/u16:2+events_unbound09:15:356
2686182992422,2cyclictest3101715-21kworker/u16:0+events_unbound11:09:276
2686182992422,2cyclictest2853369-21kworker/u16:0+events_unbound09:29:516
2686175992420,3cyclictest3101715-21kworker/u16:0+flush-8:010:40:284
2686183992319,2cyclictest3140623-21munin-run10:39:257
2686182992321,2cyclictest3066616-21kworker/u16:2+events_unbound11:57:376
2686182992321,2cyclictest3066616-21kworker/u16:2+events_unbound10:29:546
2686183992217,4cyclictest3250644-21latency_hist11:24:257
2686182992220,2cyclictest3066616-21kworker/u16:2+events_unbound11:34:496
2686182992220,2cyclictest3066616-21kworker/u16:2+events_unbound11:15:366
2686182992215,6cyclictest3248986-21kworker/u16:3+flush-8:012:09:256
2686183992119,1cyclictest0-21swapper/710:44:567
2686182992119,2cyclictest3248986-21kworker/u16:3+events_unbound12:28:446
2686182992119,2cyclictest3101715-21kworker/u16:0+events_unbound11:19:546
2686182992118,2cyclictest2879147-21kworker/u16:2+events_unbound08:54:366
2686178992118,3cyclictest3017093-21kworker/u16:1+flush-8:009:55:285
2686183992015,4cyclictest3214053-21latency_hist11:09:257
2686183992015,4cyclictest3214053-21latency_hist11:09:257
2686182992018,2cyclictest3357992-21kworker/u16:1+flush-8:012:14:326
2686182992018,2cyclictest3017093-21kworker/u16:1+events_unbound09:58:366
2686182992018,2cyclictest2918854-21kworker/u16:3+events_unbound09:38:366
2686182992018,2cyclictest2918854-21kworker/u16:3+events_unbound09:25:356
2686182992018,2cyclictest2843073-21kworker/u16:4+events_unbound08:39:496
2686182992018,2cyclictest2737969-21kworker/u16:2+flush-8:008:29:446
2686182992017,2cyclictest2717895-21kworker/u16:0+flush-8:007:44:416
2686183991917,1cyclictest0-21swapper/709:29:497
2686183991914,4cyclictest3348379-21latency_hist12:04:257
2686183991913,5cyclictest635-21systemd-journal07:29:257
268618399190,18cyclictest2757112-21awk07:44:257
2686182991917,2cyclictest2795492-21kworker/u16:3+events_unbound08:29:206
2686183991817,0cyclictest0-21swapper/712:19:407
2686183991817,0cyclictest0-21swapper/711:24:407
2686183991816,1cyclictest0-21swapper/711:49:437
2686182991816,2cyclictest2918854-21kworker/u16:3+events_unbound09:09:376
2686182991815,2cyclictest3066616-21kworker/u16:2+events_unbound11:44:326
268618299180,17cyclictest0-21swapper/611:44:016
2686183991713,3cyclictest1234-21snmpd08:22:137
2686183991713,3cyclictest1234-21snmpd08:22:137
2686183991712,4cyclictest635-21systemd-journal09:54:267
268618399171,14cyclictest3222993-21ssh11:09:487
268618399170,15cyclictest5819-21gnome-shell10:32:347
268618299170,3cyclictest3051760-21ldconfig09:59:486
2686183991615,0cyclictest0-21swapper/711:39:057
2686183991615,0cyclictest0-21swapper/709:04:487
2686183991613,2cyclictest1234-21snmpd08:18:257
2686183991613,2cyclictest0-21swapper/707:29:267
2686182991615,1cyclictest3042924-21kworker/u16:4+events_unbound10:28:016
2686182991615,1cyclictest3042924-21kworker/u16:4+events_unbound10:28:016
2686182991614,2cyclictest3138245-21kworker/u16:1+flush-8:010:44:256
2686182991614,2cyclictest2737969-21kworker/u16:2+flush-8:008:19:516
2686182991614,2cyclictest2737969-21kworker/u16:2+flush-8:008:19:516
268618299160,15cyclictest3314536-21kthreadcore11:49:426
2686175991615,1cyclictest0-21swapper/412:28:534
268617599160,15cyclictest2918834-21kthreadcore09:04:464
2686183991514,0cyclictest0-21swapper/710:19:397
2686183991511,3cyclictest1234-21snmpd07:36:397
2686183991511,2cyclictest2695674-21munin-run07:14:267
2686182991513,2cyclictest3248986-21kworker/u16:3+events_unbound11:25:366
2686182991513,2cyclictest2853369-21kworker/u16:0+events_unbound08:39:206
2686175991514,1cyclictest0-21swapper/411:19:534
2686175991514,1cyclictest0-21swapper/410:19:394
268618399149,4cyclictest3238478-21latency_hist11:19:257
2686183991413,0cyclictest0-21swapper/709:44:407
2686183991412,1cyclictest0-21swapper/708:04:507
2686183991410,3cyclictest1234-21snmpd07:21:187
268618399140,13cyclictest0-21swapper/709:59:297
2686182991413,1cyclictest2853369-21kworker/u16:0+flush-8:008:59:446
2686182991412,2cyclictest2737969-21kworker/u16:2+flush-8:008:04:546
268617899140,1cyclictest0-21swapper/508:04:495
268617899140,14cyclictest0-21swapper/511:22:025
268617899140,13cyclictest0-21swapper/511:24:335
2686175991413,1cyclictest0-21swapper/411:16:434
268617599140,14cyclictest65-21ksoftirqd/410:28:124
268617599140,14cyclictest65-21ksoftirqd/410:28:124
268617599140,14cyclictest631rcuc/407:09:504
268617599140,13cyclictest0-21swapper/407:19:274
268618399139,3cyclictest1234-21snmpd11:41:067
268618399138,3cyclictest3002790-21ssh09:39:497
2686183991310,2cyclictest0-21swapper/709:09:397
2686183991310,1cyclictest0-21swapper/711:54:377
268618399130,12cyclictest5819-21gnome-shell11:31:347
2686182991311,2cyclictest2668039-21kworker/u16:2+events_unbound07:09:536
268617899139,3cyclictest3288577-21cat11:39:395
268617899130,1cyclictest0-21swapper/507:09:475
268617899130,0cyclictest0-21swapper/509:54:035
2686175991312,1cyclictest0-21swapper/409:13:544
2686175991310,2cyclictest3294867-21kthreadcore11:39:454
268618399129,2cyclictest3079532-21munin-run10:14:267
268618399129,2cyclictest1234-21snmpd12:15:137
268618399129,2cyclictest1234-21snmpd10:09:107
268618399129,2cyclictest1234-21snmpd09:50:517
268618399129,2cyclictest1234-21snmpd09:15:117
268618399129,2cyclictest1234-21snmpd08:34:467
268618399129,2cyclictest1234-21snmpd07:45:217
268618399128,3cyclictest1234-21snmpd08:49:417
268618399126,5cyclictest2785839-21latency_hist07:59:267
268618399125,6cyclictest6895-21gmain08:09:397
268618399124,6cyclictest6382-21gvfs-afc-volume08:39:487
2686183991211,0cyclictest0-21swapper/708:59:397
2686183991211,0cyclictest0-21swapper/707:49:527
2686182991210,2cyclictest2717895-21kworker/u16:0+flush-8:007:54:506
268617899129,2cyclictest2805681-21cpuspeed_turbos08:09:375
2686178991211,1cyclictest0-21swapper/509:45:145
2686178991211,1cyclictest0-21swapper/507:46:495
268617899120,1cyclictest0-21swapper/508:30:585
268617899120,1cyclictest0-21swapper/507:36:355
268617899120,1cyclictest0-21swapper/507:26:015
268617599128,3cyclictest3264172-21/usr/sbin/munin11:29:394
268617599121,5cyclictest28056321cpuspeed_turbos08:09:374
268617599121,4cyclictest3069331-21grep10:09:414
2686175991211,1cyclictest0-21swapper/411:24:274
268617599120,1cyclictest0-21swapper/409:03:214
268617599120,1cyclictest0-21swapper/407:38:314
268618399118,2cyclictest1234-21snmpd12:24:327
268618399118,2cyclictest0-21swapper/711:44:507
268618399116,5cyclictest6382-21gvfs-afc-volume12:31:317
268618399116,5cyclictest6382-21gvfs-afc-volume12:31:317
268618399115,6cyclictest6382-21gvfs-afc-volume09:22:137
268618399115,4cyclictest0-21swapper/708:54:487
2686183991110,0cyclictest0-21swapper/711:00:047
268618299119,2cyclictest3101715-21kworker/u16:0+events_unbound10:38:526
268618299119,2cyclictest3066616-21kworker/u16:2+flush-8:012:10:166
268618299119,2cyclictest3066616-21kworker/u16:2+events_unbound10:53:416
268618299119,2cyclictest3066616-21kworker/u16:2+events_unbound10:08:446
268618299110,4cyclictest3021291-21kthreadcore09:49:436
268618299110,4cyclictest3002768-21awk09:39:496
268618299110,1cyclictest0-21swapper/608:01:346
268617899117,3cyclictest3190807-21cat10:59:395
268617899116,4cyclictest2970507-21cat09:29:415
2686178991110,1cyclictest0-21swapper/507:15:365
268617599111,6cyclictest28209431kthreadcore08:14:474
268617599111,2cyclictest3418288-21missed_timers12:29:474
268617599111,2cyclictest3418288-21missed_timers12:29:474
2686175991110,1cyclictest0-21swapper/411:12:404
268617599110,11cyclictest0-21swapper/408:27:054
268617599110,11cyclictest0-21swapper/407:48:184
268618399107,2cyclictest1234-21snmpd12:36:417
268618399106,3cyclictest6382-21gvfs-afc-volume08:29:387
268618399106,3cyclictest1234-21snmpd10:51:087
268618399106,3cyclictest1234-21snmpd08:24:407
268618299108,1cyclictest0-21swapper/608:14:396
268618299106,3cyclictest2739081-21cat07:34:406
268618299105,4cyclictest3266800-21cut11:29:426
268618299103,6cyclictest0-21swapper/612:29:456
268618299103,6cyclictest0-21swapper/612:29:456
268618299100,3cyclictest3191084-21irqcore10:59:416
268618299100,3cyclictest3080571-21cut10:14:396
268617899109,1cyclictest0-21swapper/507:57:145
268617899107,2cyclictest3430057-21kthreadcore12:34:455
268617899106,3cyclictest2945973-21cat09:19:415
268617899106,3cyclictest2873304-21cat08:44:415
268617599109,1cyclictest0-21swapper/412:09:384
268617599109,1cyclictest0-21swapper/410:03:544
268617599109,1cyclictest0-21swapper/408:49:424
268617599108,1cyclictest0-21swapper/408:04:424
268617599106,3cyclictest3206047-21kthreadcore11:04:444
268617599106,3cyclictest3206047-21kthreadcore11:04:444
268617599101,8cyclictest2854172-21interrupts08:34:404
268617599101,6cyclictest2868639-21cut08:39:454
268617599100,5cyclictest27689221kthreadcore07:49:424
268617599100,10cyclictest0-21swapper/407:28:544
26861829996,2cyclictest0-21swapper/609:19:436
26861829996,2cyclictest0-21swapper/608:44:386
26861829995,3cyclictest2882889-21cat08:49:406
26861829994,4cyclictest2748409-21df_inode07:39:396
26861789996,2cyclictest3263440-21cat11:29:365
26861789996,2cyclictest0-21swapper/510:29:435
26861789994,4cyclictest2818774-21cut08:14:465
26861789990,2cyclictest681rcuog/412:06:055
26861759998,1cyclictest0-21swapper/411:44:384
26861759998,1cyclictest0-21swapper/411:02:494
26861759998,1cyclictest0-21swapper/408:58:124
26861759998,1cyclictest0-21swapper/407:19:174
26861759997,2cyclictest0-21swapper/407:58:234
26861759997,1cyclictest0-21swapper/412:04:424
26861759996,2cyclictest2729464-21iostat_ios07:29:424
26861759996,2cyclictest0-21swapper/410:49:414
26861759996,1cyclictest681rcuog/409:54:324
26861759995,3cyclictest3063551-21tr10:04:454
26861759995,1cyclictest681rcuog/412:01:034
26861759994,4cyclictest3330453-21kthreadcore11:54:444
26861759994,3cyclictest0-21swapper/412:19:464
26861759991,2cyclictest2949123-21grep09:19:454
26861759990,8cyclictest3279778-21cut11:34:434
26861759990,5cyclictest29838891kthreadcore09:34:424
26861759990,2cyclictest3315057-21kthreadcore11:49:424
26861829980,3cyclictest2919802-21/usr/sbin/munin09:04:546
26861789986,1cyclictest0-21swapper/510:44:455
26861789985,2cyclictest2994292-21/usr/sbin/munin09:39:515
26861789984,3cyclictest3182184-21kthreadcore10:54:445
26861789984,3cyclictest2728892-21cpuspeed_turbos07:29:385
26861789983,4cyclictest3376528-21cut12:14:445
26861789983,4cyclictest3366199-21cut12:09:435
26861789983,4cyclictest3278759-21tr11:34:425
26861789983,4cyclictest3222467-21grep11:09:455
26861789983,4cyclictest3143135-21grep10:39:425
26861789983,4cyclictest3134709-21cut10:34:445
26861789983,4cyclictest3109563-21cut10:24:445
26861789983,4cyclictest3109563-21cut10:24:445
26861789983,4cyclictest3080814-21cat10:14:405
26861789983,4cyclictest3073383-21cut10:09:445
26861789983,4cyclictest3043837-21tune2fs09:59:395
26861789983,4cyclictest2902160-21ip_192.168.115.08:59:415
26861789983,4cyclictest2898012-21cut08:54:445
26861789983,4cyclictest2890173-21latency08:49:465
26861789980,4cyclictest3227118-21expr11:14:375
26861759987,1cyclictest0-21swapper/412:34:384
26861759987,1cyclictest0-21swapper/410:54:414
26861759987,1cyclictest0-21swapper/410:14:374
26861759987,1cyclictest0-21swapper/409:29:324
26861759985,1cyclictest681rcuog/409:52:234
26861759984,3cyclictest2995532-21irqcore09:39:424
26861759984,3cyclictest2787004-21cat07:59:404
26861759984,1cyclictest681rcuog/409:44:484
26861759983,4cyclictest3118392-21cut10:29:414
26861759983,4cyclictest2846835-21cut08:29:424
26861759982,5cyclictest2957604-21tr09:24:374
26861759982,5cyclictest2936890-21cut09:14:454
26861759982,5cyclictest2872568-21cpu08:44:384
26861759981,6cyclictest3377171-21kthreadcore12:14:444
26861759981,6cyclictest3128883-21chrt10:34:344
26861759981,6cyclictest2829512-21cut08:19:454
26861759981,6cyclictest2829512-21cut08:19:454
26861759981,6cyclictest2748395-21expr07:39:384
26861759980,7cyclictest0-21swapper/410:44:464
26861789975,1cyclictest0-21swapper/511:54:405
26861789975,1cyclictest0-21swapper/508:34:365
26861789974,2cyclictest3410980-21irqcore12:29:415
26861789974,2cyclictest3410980-21irqcore12:29:415
26861789974,2cyclictest2958066-21fschecks_time09:24:395
26861789974,2cyclictest2932887-21cpu09:14:375
26861789974,2cyclictest2922340-21kthreadcore09:09:405
26861789973,3cyclictest3398104-21expr12:24:385
26861789973,3cyclictest3385531-21grep12:19:355
26861789973,3cyclictest3339450-21grep11:59:425
26861789973,3cyclictest3300206-21cpuspeed_turbos11:44:385
26861789973,3cyclictest3165628-21grep10:49:355
26861789973,3cyclictest3092879-21grep10:19:385
26861789973,3cyclictest3056824-21grep10:04:415
26861789973,3cyclictest2911378-21expr09:04:375
26861789973,3cyclictest2825121-21expr08:19:385
26861789973,3cyclictest2825121-21expr08:19:385
26861789973,3cyclictest2786846-21fschecks_time07:59:395
26861789973,3cyclictest2768804-21cut07:49:425
26861789973,3cyclictest2748329-21expr07:39:385
26861789973,3cyclictest2710071-21tune2fs07:19:395
26861789972,4cyclictest3208566-21cut11:04:455
26861789972,4cyclictest3208566-21cut11:04:455
26861789972,4cyclictest2985534-21cut09:34:435
26861789972,4cyclictest2866080-21cut08:39:435
26861789970,6cyclictest2834855-21cut08:24:395
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional