You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 05:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackeslot2.osadl.org (updated Fri Nov 21, 2025 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39132782330,1sleep00-21swapper/023:40:180
3501998993322,6cyclictest0-21swapper/022:35:010
350200199328,7cyclictest0-21swapper/222:20:012
35428302300,2sleep20-21swapper/219:40:302
3502000992522,2cyclictest3886699-21latency_hist23:25:001
3502000992517,2cyclictest170550irq/121-eno100:10:021
3501998992518,4cyclictest0-21swapper/020:10:370
3501998992411,5cyclictest3899086-21kthreadcore23:30:340
3502000992320,2cyclictest3947923-21latency_hist00:00:011
3502000992320,2cyclictest3825633-21grep22:50:011
3502000992315,3cyclictest3716935-21kthreadcore21:45:361
3501998992314,5cyclictest3783756-21if_err_eno122:25:340
350200099225,8cyclictest3875171-21timerwakeupswit23:15:411
3502000992219,2cyclictest3834279-21cron22:55:011
3502000992219,2cyclictest3712815-21grep21:45:001
3502000992217,1cyclictest170550irq/121-eno123:10:351
350200099221,4cyclictest3930556-21munin-run23:50:011
3501998992217,1cyclictest35535311systemd-run19:50:010
3501998992215,6cyclictest0-21swapper/000:30:010
3501998992212,6cyclictest3739405-21/usr/sbin/munin22:00:410
3501998992212,3cyclictest241rcuc/021:58:100
3501998992211,6cyclictest3652252-21/usr/sbin/munin21:10:340
3501998992210,4cyclictest3909844-21mailstats23:35:410
3502001992119,1cyclictest0-21swapper/221:30:012
3502001992119,1cyclictest0-21swapper/200:00:012
3502001992118,1cyclictest0-21swapper/219:30:012
3502001992118,1cyclictest0-21swapper/200:30:002
350200099216,3cyclictest3800172-21apt22:35:281
3502000992118,2cyclictest3501996-21cyclictest21:40:011
3502000992118,2cyclictest0-21swapper/100:15:011
3502000992115,2cyclictest191rcu_preempt23:25:341
3502000992115,2cyclictest170550irq/121-eno122:10:441
3502000992115,1cyclictest191rcu_preempt23:30:341
3502000992113,3cyclictest3782310-21latency_hist22:25:001
350199899215,6cyclictest0-21swapper/023:25:350
350199899215,6cyclictest0-21swapper/000:10:360
350199899213,2cyclictest4006014-21vmstat00:30:420
3501998992118,2cyclictest3596609-21apt-key20:25:010
3501998992113,4cyclictest0-21swapper/023:05:260
3501998992111,2cyclictest0-21swapper/022:45:340
3501998992110,6cyclictest0-21swapper/000:35:400
302471791210,2ptp4l0-21swapper/321:25:033
350200099204,5cyclictest3730843-21aten2.4-expect21:55:301
3502000992017,2cyclictest0-21swapper/119:55:001
3502000992016,3cyclictest3785686-21kthreadcore22:25:371
3502000992016,1cyclictest191rcu_preempt21:20:011
3502000992015,4cyclictest3851745-21systemd-run23:05:001
3502000992014,2cyclictest191rcu_preempt19:20:331
3502000992014,2cyclictest191rcu_preempt00:00:361
3502000992012,3cyclictest0-21swapper/123:45:011
3502000992010,3cyclictest3907532-21kthreadcore23:35:371
350199899209,5cyclictest3679376-21idleruntime21:25:310
350199899206,2cyclictest191rcu_preempt22:10:450
350199899204,9cyclictest3566366-21cut20:00:290
350199899204,2cyclictest191rcu_preempt21:40:020
3501998992017,2cyclictest3695419-21cut21:35:000
3501998992013,6cyclictest3960089-21kthreadcore00:05:340
350199899200,5cyclictest3575308-21tr20:05:340
835298190,2rtkit-daemon491rcuog/223:24:550
3502001991917,1cyclictest0-21swapper/219:40:002
3502001991916,1cyclictest0-21swapper/220:40:012
350200099199,6cyclictest3935046-21tr23:50:361
350200099194,5cyclictest3603347-21aten2.4-expect20:30:341
3502000991915,3cyclictest3511915-21kthreadcore19:15:351
3502000991914,3cyclictest0-21swapper/120:25:001
3502000991913,1cyclictest170550irq/121-eno123:08:311
3502000991912,2cyclictest170550irq/121-eno120:40:371
3502000991911,3cyclictest3794521-21kthreadcore22:30:351
3502000991910,6cyclictest3687988-21grep21:30:341
3502000991910,2cyclictest3651672-21diskmemload00:27:461
350199899198,2cyclictest0-21swapper/022:15:490
350199899196,5cyclictest0-21swapper/022:40:310
350199899196,1cyclictest0-21swapper/023:10:300
350199899195,1cyclictest191rcu_preempt00:00:020
350199899194,7cyclictest3804133-21kthreadcore22:35:360
3501998991911,4cyclictest0-21swapper/021:23:030
3501998991910,7cyclictest0-21swapper/000:05:000
350199899190,9cyclictest3930487-21cat23:50:010
3502001991816,1cyclictest0-21swapper/223:50:002
3502001991815,1cyclictest0-21swapper/219:35:002
3502001991815,1cyclictest0-21swapper/200:15:012
3502001991811,2cyclictest0-21swapper/222:00:322
350200099189,2cyclictest3626343-21sendmail_mailqu20:45:411
350200099188,6cyclictest0-21swapper/100:30:411
350200099187,7cyclictest3954465-21kworker/u16:3+xprtiod00:19:091
350200099187,7cyclictest3954465-21kworker/u16:3+xprtiod00:19:091
350200099187,3cyclictest0-21swapper/100:35:291
3502000991815,2cyclictest3810207-21kthreadcore22:40:341
3502000991815,2cyclictest3647269-21kthreadcore21:05:331
3502000991815,2cyclictest3544439-21kthreadcore19:40:311
3502000991815,2cyclictest3527478-21kthreadcore19:25:351
3502000991815,2cyclictest1635-21dbus-daemon21:55:011
3502000991815,2cyclictest0-21swapper/120:55:011
3502000991814,3cyclictest3742306-21kthreadcore22:00:351
3502000991814,3cyclictest3656444-21kthreadcore21:10:351
3502000991814,3cyclictest3641696-21kthreadcore21:00:341
3502000991814,3cyclictest3579799-21kernelversion20:10:361
3502000991814,3cyclictest3569265-21kthreadcore20:00:361
3502000991814,3cyclictest3563407-21kthreadcore19:55:351
3502000991813,1cyclictest191rcu_preempt22:15:361
350199899189,2cyclictest3846025-21kthreadcore23:00:340
350199899188,3cyclictest16-21ksoftirqd/019:35:010
350199899187,6cyclictest0-21swapper/022:55:410
350199899187,3cyclictest0-21swapper/021:00:280
350199899187,2cyclictest0-21swapper/021:20:020
350199899187,1cyclictest3465188-21kworker/u16:0-nfsiod22:05:330
3501998991815,2cyclictest3983478-21cstates00:20:300
3501998991815,2cyclictest3983478-21cstates00:20:300
350199899180,8cyclictest3872955-21kthreadcore23:15:350
302487291182,2phc2sys0-21swapper/323:47:253
302487291180,4phc2sys0-21swapper/323:35:293
302487291180,3phc2sys0-21swapper/321:11:483
302487291180,2phc2sys0-21swapper/321:44:283
302487291180,2phc2sys0-21swapper/300:27:323
350200199171,7cyclictest0-21swapper/223:10:012
3502001991715,1cyclictest0-21swapper/221:15:012
3502001991715,1cyclictest0-21swapper/220:55:002
3502001991714,1cyclictest0-21swapper/222:15:002
3502001991714,1cyclictest0-21swapper/222:10:012
3502001991714,1cyclictest0-21swapper/221:30:422
3502001991714,1cyclictest0-21swapper/220:25:012
3502001991714,1cyclictest0-21swapper/219:15:382
350200099179,4cyclictest0-21swapper/119:30:361
350200099178,5cyclictest3965210-21kworker/1:1+events_freezable00:20:331
350200099178,5cyclictest3965210-21kworker/1:1+events_freezable00:20:331
350200099175,5cyclictest1635-21dbus-daemon19:50:021
350200099173,4cyclictest3584907-21cpuspeed_turbos20:15:301
3502000991714,2cyclictest3599869-21kthreadcore20:25:371
3502000991714,2cyclictest3573165-21gltestperf20:05:301
3502000991714,2cyclictest3535794-21cpuspeed_turbos19:35:301
3502000991714,2cyclictest3506275-21kthreadcore19:10:361
3502000991714,2cyclictest0-21swapper/121:30:011
3502000991713,3cyclictest3838205-21kthreadcore22:55:361
3502000991713,3cyclictest3674295-21kthreadcore21:20:371
3502000991713,1cyclictest601rcuop/320:55:371
350199899179,2cyclictest191rcu_preempt19:40:010
350199899176,4cyclictest0-21swapper/022:20:330
350199899176,3cyclictest3415076-21kworker/0:1+events19:45:010
350199899174,4cyclictest3716530-21grep21:45:360
350199899173,9cyclictest0-21swapper/021:41:080
350199899173,6cyclictest3627500-21aten2.4-expect20:50:280
350199899171,7cyclictest3633790-21/usr/sbin/munin20:55:460
3501998991715,1cyclictest0-21swapper/019:30:010
3501998991715,1cyclictest0-21swapper/000:15:300
3501998991715,1cyclictest0-21swapper/000:15:300
3501998991714,2cyclictest3834455-21systemd-run22:55:020
3501998991714,2cyclictest1704-21rs:main0
3501998991713,2cyclictest0-21swapper/020:30:010
3501998991710,6cyclictest3726711-21kthreadcore21:50:390
302487291170,2phc2sys0-21swapper/323:32:193
302487291170,2phc2sys0-21swapper/323:22:463
302487291170,2phc2sys0-21swapper/322:45:553
302487291170,2phc2sys0-21swapper/321:37:233
302487291170,2phc2sys0-21swapper/320:30:013
3502001991613,1cyclictest0-21swapper/223:15:012
3502001991613,1cyclictest0-21swapper/222:50:352
3502001991613,1cyclictest0-21swapper/222:30:312
3502001991613,1cyclictest0-21swapper/222:00:012
3502001991613,1cyclictest0-21swapper/221:25:012
3502001991613,1cyclictest0-21swapper/221:20:012
3502001991613,1cyclictest0-21swapper/221:00:342
3502001991613,1cyclictest0-21swapper/220:10:012
3502001991613,1cyclictest0-21swapper/200:35:322
350200099163,4cyclictest3749028-21cat22:05:321
3502000991611,1cyclictest3553552-21kworker/1:1+events_freezable20:35:281
350199899165,4cyclictest3603161-21/usr/sbin/munin20:30:450
350199899162,10cyclictest0-21swapper/023:50:530
3501998991613,2cyclictest3517529-21cut19:20:300
302487291160,2phc2sys0-21swapper/322:15:543
38517612150,2sleep20-21swapper/223:05:012
3502001991513,1cyclictest0-21swapper/223:25:362
3502001991513,1cyclictest0-21swapper/222:40:022
3502001991513,1cyclictest0-21swapper/222:30:012
3502001991513,1cyclictest0-21swapper/220:50:012
3502001991513,1cyclictest0-21swapper/219:10:022
3502001991512,1cyclictest0-21swapper/221:00:002
3502001991512,1cyclictest0-21swapper/219:50:012
3502001991511,2cyclictest0-21swapper/200:20:412
3502001991511,2cyclictest0-21swapper/200:20:412
350200099154,6cyclictest3501832-21latency_hist19:10:021
350199899155,3cyclictest3553537-21kworker/0:0+rcu_gp20:00:010
350199899155,3cyclictest16-21ksoftirqd/021:10:000
350199899151,5cyclictest3502086-21cat19:10:020
350199899150,7cyclictest3508649-21munin-node19:15:020
302487291152,3phc2sys0-21swapper/300:17:383
302487291152,3phc2sys0-21swapper/300:17:383
302487291150,4phc2sys0-21swapper/323:00:133
302487291150,4phc2sys0-21swapper/322:25:293
302487291150,2phc2sys0-21swapper/321:45:283
350200199142,7cyclictest0-21swapper/221:50:282
3502001991411,1cyclictest0-21swapper/221:45:302
3502001991411,1cyclictest0-21swapper/221:40:342
3502001991411,1cyclictest0-21swapper/221:10:022
3502001991411,1cyclictest0-21swapper/220:40:432
3502001991411,1cyclictest0-21swapper/220:10:322
3502001991411,1cyclictest0-21swapper/219:20:342
3502001991411,1cyclictest0-21swapper/219:15:002
3502001991411,1cyclictest0-21swapper/200:30:412
3502001991410,2cyclictest0-21swapper/222:20:362
3502001991410,2cyclictest0-21swapper/220:00:372
350199899145,7cyclictest3587440-21tr20:15:370
350199899144,2cyclictest170550irq/121-eno119:10:360
3501998991411,2cyclictest3621073-21latency_hist20:45:000
3501998991411,2cyclictest3610565-21kthreadcore20:35:340
350199899140,7cyclictest3621405-21/usr/sbin/munin20:45:360
302487291144,2phc2sys0-21swapper/323:59:413
302487291140,3phc2sys0-21swapper/322:06:483
302487291140,2phc2sys0-21swapper/323:17:573
302487291140,2phc2sys0-21swapper/322:38:103
302487291140,2phc2sys0-21swapper/322:31:593
302487291140,2phc2sys0-21swapper/322:21:523
302487291140,2phc2sys0-21swapper/321:56:063
302487291140,2phc2sys0-21swapper/321:52:593
350200199135,4cyclictest0-21swapper/223:40:272
3502001991311,1cyclictest0-21swapper/223:25:022
3502001991311,1cyclictest0-21swapper/222:40:282
3502001991311,1cyclictest0-21swapper/220:30:342
3502001991310,1cyclictest0-21swapper/223:50:302
3502001991310,1cyclictest0-21swapper/223:35:322
3502001991310,1cyclictest0-21swapper/222:45:292
3502001991310,1cyclictest0-21swapper/220:15:352
3502001991310,1cyclictest0-21swapper/219:55:342
3502001991310,1cyclictest0-21swapper/219:50:332
3502001991310,1cyclictest0-21swapper/200:15:362
3502001991310,1cyclictest0-21swapper/200:15:362
3502001991310,1cyclictest0-21swapper/200:00:362
302487291132,2phc2sys0-21swapper/323:10:363
302487291131,3phc2sys0-21swapper/322:57:543
302487291130,4phc2sys0-21swapper/300:39:393
302487291130,4phc2sys0-21swapper/300:34:583
302487291130,2phc2sys0-21swapper/321:26:413
302487291130,2phc2sys0-21swapper/320:45:363
350200199129,1cyclictest0-21swapper/222:55:352
350200199129,1cyclictest0-21swapper/221:35:352
350200199129,1cyclictest0-21swapper/220:25:352
3502001991210,1cyclictest0-21swapper/223:30:262
3502001991210,1cyclictest0-21swapper/223:20:012
3502001991210,1cyclictest0-21swapper/200:05:342
302487291123,2phc2sys0-21swapper/323:27:553
302487291122,2phc2sys0-21swapper/300:08:363
302487291121,3phc2sys0-21swapper/319:40:243
302487291121,2phc2sys0-21swapper/320:00:373
302487291120,2phc2sys0-21swapper/321:00:273
302487291120,2phc2sys0-21swapper/320:37:143
302487291120,2phc2sys0-21swapper/320:21:593
302487291120,2phc2sys0-21swapper/300:13:413
302487291112,2phc2sys0-21swapper/322:50:323
302487291111,3phc2sys0-21swapper/322:13:253
302487291111,2phc2sys0-21swapper/319:08:153
302487291110,3phc2sys0-21swapper/322:00:273
302487291110,3phc2sys0-21swapper/321:32:143
302487291110,3phc2sys0-21swapper/319:20:013
302487291110,2phc2sys0-21swapper/322:41:113
35289552100,2sleep30-21swapper/319:30:013
302487291102,2phc2sys0-21swapper/320:05:333
302487291101,3phc2sys0-21swapper/300:01:033
302487291101,2phc2sys0-21swapper/321:15:333
302487291101,2phc2sys0-21swapper/319:55:353
302487291101,2phc2sys0-21swapper/319:45:293
302487291101,2phc2sys0-21swapper/319:20:333
302487291100,3phc2sys0-21swapper/323:40:343
302487291100,3phc2sys0-21swapper/321:05:363
302487291100,3phc2sys0-21swapper/319:50:263
302487291100,2phc2sys0-21swapper/323:50:073
302487291100,2phc2sys0-21swapper/323:05:433
302487291100,2phc2sys0-21swapper/320:19:473
302487291100,2phc2sys0-21swapper/320:10:303
302487291100,2phc2sys0-21swapper/319:38:323
302487291100,2phc2sys0-21swapper/319:10:393
30248729191,2phc2sys0-21swapper/320:40:393
30248729191,2phc2sys0-21swapper/300:20:343
30248729191,2phc2sys0-21swapper/300:20:343
30248729190,3phc2sys0-21swapper/319:35:023
30248729190,2phc2sys0-21swapper/320:55:343
30248729190,2phc2sys0-21swapper/320:50:413
30248729190,2phc2sys0-21swapper/320:30:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional