You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 01:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Mon May 11, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
284741621020,0sleep50-21swapper/522:53:475
23869632980,1sleep72386964-21grep19:23:387
234861126555,7sleep40-21swapper/419:07:424
2348935996260,2cyclictest2573741-21kworker/u16:0+flush-8:021:08:454
2348935996259,2cyclictest2837855-21kworker/u16:2+flush-8:023:08:334
2348935995452,2cyclictest2684094-21kworker/u16:4+events_unbound21:48:494
2348935995451,2cyclictest2999871-21kworker/u16:0+events_unbound00:08:294
2348935995351,2cyclictest2763100-21kworker/u16:0+events_unbound22:43:304
2348936995047,2cyclictest2850425-21kworker/u16:1+flush-8:022:58:335
2348935994947,2cyclictest2684094-21kworker/u16:4+flush-8:022:13:464
2348935994946,2cyclictest2506542-21kworker/u16:0+events_unbound20:38:314
2348935994845,2cyclictest2381641-21kworker/u16:2+events_unbound19:48:314
2348936994745,2cyclictest2922804-21kworker/u16:5+flush-8:023:33:495
2348935994745,2cyclictest2448765-21kworker/u16:1+events_unbound20:28:314
2348935994744,2cyclictest2684094-21kworker/u16:4+flush-8:022:18:334
2348935994744,2cyclictest2643460-21kworker/u16:0+flush-8:021:43:314
2348935994744,2cyclictest2643460-21kworker/u16:0+flush-8:021:43:314
2348935994744,2cyclictest2448765-21kworker/u16:1+events_unbound20:48:314
2348942994643,1cyclictest2544213-21/usr/sbin/munin20:43:447
2348936994644,1cyclictest2837855-21kworker/u16:2+events_unbound23:38:335
2348935994542,2cyclictest2999871-21kworker/u16:0+events_unbound23:58:384
2348936994442,2cyclictest2684094-21kworker/u16:4+flush-8:022:03:465
2348935994442,2cyclictest2850425-21kworker/u16:1+events_unbound23:03:304
2348935994442,2cyclictest2777778-21kworker/u16:1+events_unbound22:48:484
2348935994442,2cyclictest2525582-21kworker/u16:3+flush-8:020:43:464
2348935994342,1cyclictest3042811-21kworker/u16:2+events_unbound00:33:374
234861224333,7sleep50-21swapper/519:07:435
2348935994240,2cyclictest3068193-21kworker/u16:3+flush-8:000:28:324
2348935994240,2cyclictest3068193-21kworker/u16:3+flush-8:000:28:324
2348935994139,2cyclictest2850425-21kworker/u16:1+flush-8:022:58:314
2348935994138,2cyclictest2400769-21kworker/u16:4+flush-8:019:38:424
23484162415,31sleep70-21swapper/719:04:567
2348942994035,4cyclictest2632045-21latency_hist21:28:167
2348935994038,2cyclictest2716977-21kworker/u16:3+events_unbound22:08:114
2348935994038,2cyclictest2598947-21kworker/u16:2+events_unbound22:25:174
2348935994037,2cyclictest2981917-21kworker/u16:2+events_unbound00:03:284
2348936993936,2cyclictest3042811-21kworker/u16:2+flush-8:000:23:495
2348935993937,2cyclictest2850425-21kworker/u16:1+events_unbound23:53:414
2348935993937,2cyclictest2763100-21kworker/u16:0+events_unbound22:33:014
2348935993937,2cyclictest2558056-21kworker/u16:2+flush-8:021:03:454
2348935993937,2cyclictest2558056-21kworker/u16:2+flush-8:021:03:454
2348935993936,2cyclictest2968281-21kworker/u16:3+flush-8:023:48:374
2348935993936,2cyclictest2968281-21kworker/u16:3+flush-8:023:48:374
2348935993936,2cyclictest2763100-21kworker/u16:0+flush-8:022:53:324
2348936993836,2cyclictest3042811-21kworker/u16:2+events_unbound00:18:295
2348936993836,2cyclictest2598947-21kworker/u16:2+flush-8:021:58:455
2348935993836,2cyclictest2763100-21kworker/u16:0+events_unbound22:40:224
2348935993835,2cyclictest2643460-21kworker/u16:0+events_unbound21:53:504
2348936993735,2cyclictest2999871-21kworker/u16:0+events_unbound00:08:345
2348936993735,1cyclictest2643460-21kworker/u16:0+events_unbound21:48:355
2348935993735,2cyclictest2850425-21kworker/u16:1+events_unbound23:16:064
2348935993735,2cyclictest2448765-21kworker/u16:1+flush-8:020:13:454
2348935993634,2cyclictest2983424-21kworker/u16:4+flush-8:000:23:534
2348935993634,2cyclictest2850425-21kworker/u16:1+flush-8:023:18:474
2348935993634,2cyclictest2656221-21kworker/u16:3+events_unbound21:39:264
2348935993634,2cyclictest2598947-21kworker/u16:2+flush-8:021:58:314
2348936993532,2cyclictest2348705-21kworker/u16:1+events_unbound19:28:305
2348935993533,2cyclictest2922804-21kworker/u16:5+events_unbound23:33:434
2348935993533,2cyclictest2777778-21kworker/u16:1+events_unbound22:36:024
2348936993432,2cyclictest2656221-21kworker/u16:3+events_unbound21:43:305
2348936993432,2cyclictest2656221-21kworker/u16:3+events_unbound21:43:305
2348935993433,1cyclictest2714667-21kworker/u16:1+events_unbound22:10:304
2348935993432,2cyclictest2631124-21kworker/u16:1+flush-8:021:31:374
2348935993432,2cyclictest2598947-21kworker/u16:2+events_unbound21:25:104
2348935993431,2cyclictest2348705-21kworker/u16:1+flush-8:019:28:294
2348935993331,2cyclictest2983424-21kworker/u16:4+flush-8:000:19:334
2348935993331,2cyclictest2983424-21kworker/u16:4+flush-8:000:17:094
2348935993331,2cyclictest2837855-21kworker/u16:2+events_unbound23:29:374
2348935993331,2cyclictest2598947-21kworker/u16:2+flush-8:021:36:344
2348935993330,2cyclictest2448765-21kworker/u16:1+flush-8:020:03:314
2348936993230,2cyclictest3042811-21kworker/u16:2+events_unbound00:28:305
2348936993230,2cyclictest3042811-21kworker/u16:2+events_unbound00:28:305
2348936993230,2cyclictest2850425-21kworker/u16:1+flush-8:023:18:375
2348936993230,2cyclictest2400769-21kworker/u16:4+flush-8:019:33:465
2348936993230,2cyclictest2400769-21kworker/u16:4+flush-8:019:33:465
2348936993229,2cyclictest2789882-21kworker/u16:3+events_unbound22:38:295
2348936993228,3cyclictest2763100-21kworker/u16:0+flush-8:023:28:315
2348935993230,2cyclictest2448765-21kworker/u16:1+flush-8:020:23:314
2348935993229,3cyclictest2850425-21kworker/u16:1+flush-8:023:43:484
2348942993130,0cyclictest0-21swapper/720:35:537
2348936993129,2cyclictest2850425-21kworker/u16:1+flush-8:023:43:345
2348936993129,2cyclictest2583544-21kworker/u16:1+flush-8:021:18:445
2348936993128,3cyclictest2981917-21kworker/u16:2+events_unbound00:03:455
2348935993129,2cyclictest2573741-21kworker/u16:0+flush-8:021:20:424
2348935993129,2cyclictest2525582-21kworker/u16:3+events_unbound21:13:404
2348942993029,0cyclictest0-21swapper/720:42:107
2348936993028,2cyclictest2983424-21kworker/u16:4+events_unbound23:59:175
2348936993028,2cyclictest2598947-21kworker/u16:2+flush-8:021:33:455
2348936993028,2cyclictest2525582-21kworker/u16:3+flush-8:021:13:445
234835623019,8sleep60-21swapper/619:04:076
2348942992928,0cyclictest0-21swapper/723:55:487
2348942992928,0cyclictest0-21swapper/720:27:467
2348942992927,1cyclictest0-21swapper/723:50:287
2348942992927,1cyclictest0-21swapper/723:50:287
2348936992926,3cyclictest2789882-21kworker/u16:3+flush-8:022:43:325
2348935992927,2cyclictest2850425-21kworker/u16:1+flush-8:023:26:184
2348935992927,2cyclictest2400769-21kworker/u16:4+flush-8:019:33:314
2348935992927,2cyclictest2400769-21kworker/u16:4+flush-8:019:33:314
2348935992927,2cyclictest2300526-21kworker/u16:3+events_unbound19:13:294
2348942992827,0cyclictest0-21swapper/721:29:597
2348941992826,2cyclictest2348705-21kworker/u16:1+events_unbound19:08:306
2348936992826,2cyclictest2684094-21kworker/u16:4+flush-8:022:13:475
2348936992826,2cyclictest2598947-21kworker/u16:2+flush-8:021:30:255
2348936992826,2cyclictest2583544-21kworker/u16:1+events_unbound21:08:295
2348936992826,2cyclictest2448765-21kworker/u16:1+flush-8:020:13:455
2348935992826,2cyclictest2448765-21kworker/u16:1+flush-8:020:33:314
2348942992726,0cyclictest0-21swapper/720:06:017
2348942992726,0cyclictest0-21swapper/719:17:547
2348942992723,3cyclictest2457457-21nfs4_client19:58:417
2348942992712,13cyclictest0-21swapper/723:10:297
234894299270,25cyclictest635-21systemd-journal23:43:117
2348936992726,1cyclictest2558056-21kworker/u16:2+events_unbound21:03:405
2348936992726,1cyclictest2558056-21kworker/u16:2+events_unbound21:03:405
2348936992725,2cyclictest2448765-21kworker/u16:1+flush-8:020:03:465
2348936992723,4cyclictest2400769-21kworker/u16:4+flush-8:020:23:165
2348942992625,0cyclictest0-21swapper/723:04:327
2348942992625,0cyclictest0-21swapper/722:13:207
2348942992625,0cyclictest0-21swapper/721:44:447
2348942992625,0cyclictest0-21swapper/721:44:447
2348942992625,0cyclictest0-21swapper/700:37:427
2348942992624,1cyclictest2583597-21diskmemload00:10:267
2348936992624,2cyclictest3017985-21kworker/u16:1+events_unbound00:13:435
2348936992624,2cyclictest2643460-21kworker/u16:0+flush-8:021:53:385
2348936992624,2cyclictest2598947-21kworker/u16:2+flush-8:022:20:265
2348936992624,2cyclictest2448765-21kworker/u16:1+flush-8:020:58:315
2348935992624,2cyclictest2837855-21kworker/u16:2+flush-8:023:40:384
2348942992524,0cyclictest0-21swapper/722:36:247
2348942992524,0cyclictest0-21swapper/720:50:347
2348942992524,0cyclictest0-21swapper/720:32:227
2348936992523,2cyclictest2525582-21kworker/u16:3+events_unbound20:48:415
2348936992523,2cyclictest2448765-21kworker/u16:1+flush-8:020:28:335
2348935992522,2cyclictest2400769-21kworker/u16:4+flush-8:019:43:304
26117952240,1chrt0-21swapper/621:18:356
2348942992423,0cyclictest0-21swapper/723:24:327
2348942992423,0cyclictest0-21swapper/721:39:457
2348942992423,0cyclictest0-21swapper/720:10:427
2348942992423,0cyclictest0-21swapper/719:39:427
2348942992423,0cyclictest0-21swapper/719:19:547
2348942992423,0cyclictest0-21swapper/700:04:487
2348942992422,1cyclictest2594737-21ssh21:11:297
2348936992422,2cyclictest2598947-21kworker/u16:2+events_unbound21:38:415
2348936992422,2cyclictest2598947-21kworker/u16:2+events_unbound21:24:255
2348935992422,2cyclictest2558056-21kworker/u16:2+flush-8:020:58:454
2348942992322,0cyclictest0-21swapper/723:30:307
2348942992322,0cyclictest0-21swapper/722:40:307
2348942992322,0cyclictest0-21swapper/721:35:397
2348942992322,0cyclictest0-21swapper/720:20:507
2348942992322,0cyclictest0-21swapper/719:55:307
2348942992322,0cyclictest0-21swapper/700:27:037
234894299230,21cyclictest2705938-21fschecks_time21:58:317
2348935992320,2cyclictest2348705-21kworker/u16:1+events_unbound19:23:294
2348942992221,0cyclictest0-21swapper/723:15:137
2348942992221,0cyclictest0-21swapper/719:44:157
2348942992221,0cyclictest0-21swapper/719:30:547
2348942992221,0cyclictest0-21swapper/700:21:527
234894299220,20cyclictest2850025-21latency_hist22:58:167
2348936992219,2cyclictest2400769-21kworker/u16:4+flush-8:019:53:295
2348942992120,0cyclictest0-21swapper/722:29:087
2348942992120,0cyclictest0-21swapper/722:25:397
2348942992120,0cyclictest0-21swapper/721:18:317
2348942992119,1cyclictest3006359-21ssh00:01:037
2348942992119,1cyclictest0-21swapper/719:12:477
2348942992119,1cyclictest0-21swapper/700:32:367
2348942992119,1cyclictest0-21swapper/700:32:367
2348936992119,2cyclictest2850425-21kworker/u16:1+events_unbound23:26:265
2348936992119,2cyclictest2763100-21kworker/u16:0+flush-8:022:28:415
2348935992119,2cyclictest2448765-21kworker/u16:1+events_unbound20:57:254
2348942992019,0cyclictest0-21swapper/723:19:147
2348942992019,0cyclictest0-21swapper/720:54:077
2348942992018,1cyclictest2860061-21ssh22:59:277
2348942992014,5cyclictest2410306-21latency_hist19:38:167
2348942992014,5cyclictest2410306-21latency_hist19:38:167
2348936992018,2cyclictest2850425-21kworker/u16:1+flush-8:023:08:155
2348935992017,2cyclictest2429396-21kworker/u16:0+flush-8:019:53:374
2348942991917,1cyclictest2837957-21awk22:53:167
2348942991917,1cyclictest0-21swapper/723:34:047
2348942991913,5cyclictest2439190-21latency_hist19:53:167
2348936991917,2cyclictest2981917-21kworker/u16:2+flush-8:023:52:255
2348936991917,2cyclictest2981917-21kworker/u16:2+flush-8:023:52:255
2348936991917,2cyclictest2981917-21kworker/u16:2+events_unbound23:53:495
2348936991917,2cyclictest2850425-21kworker/u16:1+events_unbound23:15:255
2348935991916,2cyclictest2448765-21kworker/u16:1+flush-8:020:08:254
2348942991817,0cyclictest0-21swapper/722:44:037
2348942991817,0cyclictest0-21swapper/700:15:547
2348936991816,2cyclictest3081975-21kworker/u16:0+flush-8:000:33:375
2348936991816,2cyclictest2684094-21kworker/u16:4+events_unbound22:08:255
2348936991816,2cyclictest2598947-21kworker/u16:2+events_unbound22:26:265
2348936991816,2cyclictest2348705-21kworker/u16:1+flush-8:019:43:355
2348935991816,2cyclictest2448765-21kworker/u16:1+flush-8:020:01:254
2348935991816,2cyclictest2300526-21kworker/u16:3+events_unbound19:09:264
2348942991716,0cyclictest0-21swapper/722:03:577
2348936991715,2cyclictest2777778-21kworker/u16:1+events_unbound22:50:265
2348936991715,2cyclictest2598947-21kworker/u16:2+events_unbound22:33:425
24016712160,0sleep60-21swapper/619:33:316
24016712160,0sleep60-21swapper/619:33:316
2348942991613,2cyclictest0-21swapper/721:53:167
2348936991614,2cyclictest2506542-21kworker/u16:0+flush-8:020:43:325
2348935991614,2cyclictest2348705-21kworker/u16:1+events_unbound19:19:464
238452921513,1sleep50-21swapper/519:23:375
2348942991512,1cyclictest0-21swapper/721:03:457
2348942991512,1cyclictest0-21swapper/721:03:457
234893699150,1cyclictest0-21swapper/523:10:295
2348942991413,0cyclictest0-21swapper/720:58:287
2348942991412,1cyclictest2691474-21ssh21:51:157
2348941991413,1cyclictest0-21swapper/623:09:526
2348941991412,2cyclictest2850425-21kworker/u16:1+flush-8:023:58:266
234894199140,14cyclictest0-21swapper/623:20:456
234894199140,14cyclictest0-21swapper/621:30:146
2348936991412,2cyclictest2348705-21kworker/u16:1+flush-8:019:18:425
234893699140,14cyclictest0-21swapper/519:48:325
234893699140,14cyclictest0-21swapper/519:08:385
234894299139,3cyclictest1234-21snmpd20:13:427
2348941991312,1cyclictest0-21swapper/623:45:216
2348941991312,1cyclictest0-21swapper/622:36:176
2348941991312,1cyclictest0-21swapper/622:10:546
2348941991312,1cyclictest0-21swapper/621:59:056
2348941991312,1cyclictest0-21swapper/600:24:556
2348941991312,1cyclictest0-21swapper/600:13:066
234894199130,13cyclictest0-21swapper/622:58:506
234894199130,13cyclictest0-21swapper/621:48:226
234894199130,13cyclictest0-21swapper/620:37:066
2348936991311,2cyclictest2506542-21kworker/u16:0+flush-8:020:43:165
2348936991311,2cyclictest2400769-21kworker/u16:4+flush-8:019:38:385
234893699131,12cyclictest0-21swapper/520:01:565
234894299125,6cyclictest5822-21gmain22:10:017
2348942991211,0cyclictest0-21swapper/722:18:217
2348941991211,1cyclictest0-21swapper/623:30:506
2348941991211,1cyclictest0-21swapper/619:18:166
234894199120,1cyclictest0-21swapper/621:01:326
2348936991210,2cyclictest2300526-21kworker/u16:3+events_unbound19:13:315
234894299118,2cyclictest1234-21snmpd23:44:387
2348941991110,1cyclictest0-21swapper/622:23:126
2348941991110,1cyclictest0-21swapper/621:39:046
2348941991110,1cyclictest0-21swapper/620:08:056
234894199110,9cyclictest0-21swapper/620:30:166
2348936991110,1cyclictest0-21swapper/520:35:215
2348935991110,1cyclictest0-21swapper/420:22:514
234894299105,4cyclictest2603256-21cut21:13:417
234894199109,1cyclictest0-21swapper/621:12:066
234894199109,1cyclictest0-21swapper/620:49:056
234894199109,1cyclictest0-21swapper/620:22:076
234894199102,1cyclictest0-21swapper/619:13:186
23489419997,2cyclictest86-21ksoftirqd/623:50:596
23489419997,2cyclictest86-21ksoftirqd/623:50:596
23489419997,2cyclictest86-21ksoftirqd/623:26:326
23489419997,2cyclictest86-21ksoftirqd/623:08:076
23489419997,2cyclictest86-21ksoftirqd/622:48:296
23489419997,2cyclictest86-21ksoftirqd/622:44:256
23489419997,2cyclictest86-21ksoftirqd/622:41:066
23489419997,2cyclictest86-21ksoftirqd/622:30:336
23489419997,2cyclictest86-21ksoftirqd/622:15:256
23489419997,2cyclictest86-21ksoftirqd/621:57:236
23489419997,2cyclictest86-21ksoftirqd/621:45:086
23489419997,2cyclictest86-21ksoftirqd/621:45:086
23489419997,2cyclictest86-21ksoftirqd/621:33:416
23489419997,2cyclictest86-21ksoftirqd/621:15:096
23489419997,2cyclictest86-21ksoftirqd/600:33:456
23489419997,2cyclictest86-21ksoftirqd/600:15:316
23489419997,2cyclictest86-21ksoftirqd/600:03:276
23489419995,3cyclictest3044783-21chrt00:18:276
23489369997,2cyclictest2468031-21kworker/u16:2+flush-8:020:08:415
23489369997,2cyclictest2400769-21kworker/u16:4+flush-8:020:23:445
23489419986,2cyclictest86-21ksoftirqd/623:53:276
23489419986,2cyclictest86-21ksoftirqd/623:38:526
23489419986,2cyclictest86-21ksoftirqd/623:33:536
23489419986,2cyclictest86-21ksoftirqd/622:55:356
23489419986,2cyclictest86-21ksoftirqd/622:25:326
23489419986,2cyclictest86-21ksoftirqd/622:03:326
23489419986,2cyclictest86-21ksoftirqd/621:27:496
23489419986,2cyclictest86-21ksoftirqd/600:30:226
23489419986,2cyclictest86-21ksoftirqd/600:30:226
23489419975,2cyclictest86-21ksoftirqd/623:13:476
23489419970,7cyclictest0-21swapper/620:28:056
23489419970,1cyclictest0-21swapper/620:43:376
23489419970,1cyclictest0-21swapper/619:58:326
23489419970,1cyclictest0-21swapper/619:43:556
23489419970,1cyclictest0-21swapper/619:26:226
23489419961,1cyclictest0-21swapper/621:03:466
23489419961,1cyclictest0-21swapper/621:03:466
23489419960,6cyclictest0-21swapper/620:53:296
23489419960,6cyclictest0-21swapper/620:38:316
23489419960,6cyclictest0-21swapper/620:08:336
23489419960,6cyclictest0-21swapper/619:58:166
23489419960,1cyclictest0-21swapper/620:18:066
23489419960,1cyclictest0-21swapper/619:48:306
23489419960,1cyclictest0-21swapper/619:28:366
23489419960,0cyclictest0-21swapper/619:38:426
23489369964,2cyclictest2448765-21kworker/u16:1+flush-8:020:55:265
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional