You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 17:02
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackeslot2.osadl.org (updated Sat Nov 29, 2025 12:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
355186999264,2cyclictest3894902-21cat11:00:230
3551869992621,4cyclictest4040426-21idleruntime-cro12:25:000
3551869992621,4cyclictest4040426-21idleruntime-cro12:25:000
38319552250,1sleep10-21swapper/110:25:011
355186999259,6cyclictest0-21swapper/012:00:170
3551869992515,2cyclictest170550irq/121-eno109:15:230
3551869992513,8cyclictest0-21swapper/011:10:190
3551869992511,9cyclictest0-21swapper/011:35:010
3551869992511,6cyclictest0-21swapper/009:55:020
3551870992413,3cyclictest3953492-21munin-plugin-st11:35:001
355186999245,10cyclictest0-21swapper/012:26:460
3551869992418,5cyclictest3823119-21latency_hist10:20:000
3551869992415,8cyclictest3846270-21sort10:30:290
3551869992411,5cyclictest0-21swapper/010:05:010
3551870992313,3cyclictest4045184-21kthreadcore12:25:261
3551870992313,3cyclictest3900441-21runrttasks11:04:051
355186999239,6cyclictest260-21systemd-journal11:49:550
355186999231,9cyclictest3799363-21cut10:05:250
3551869992317,5cyclictest3982831-21kthreadcore11:50:240
3551869992317,5cyclictest3965566-21kthreadcore11:40:270
3551869992317,5cyclictest3959277-21wc11:35:290
3551869992317,5cyclictest3684192-21idleruntime08:55:200
3551869992314,8cyclictest4059820-21irqrtprio12:35:230
3551869992312,10cyclictest3919090-21aten2.4-expect11:15:170
3551869992311,5cyclictest3855600-21runrttasks10:36:150
3551869992311,10cyclictest0-21swapper/009:10:240
3551869992310,5cyclictest3719856-21idleruntime09:20:200
3551869992310,12cyclictest3901447-21/usr/sbin/munin11:05:290
3551870992215,2cyclictest170550irq/121-eno111:20:001
3551870992213,3cyclictest3718998-21/usr/sbin/munin09:20:231
3551869992216,5cyclictest4053214-21kthreadcore12:30:240
3551869992216,5cyclictest3861156-21kthreadcore10:40:260
3551869992216,5cyclictest3690633-21cut09:00:240
3551869992216,5cyclictest3661835-21kthreadcore08:35:250
3551869992216,5cyclictest3655747-21kthreadcore08:30:230
3551869992216,5cyclictest3648158-21kthreadcore08:25:250
3551869992216,5cyclictest3612519-21kthreadcore07:55:240
3551869992215,6cyclictest3834741-21kthreadcore10:25:260
3551869992215,3cyclictest3900808-21kworker/u16:3+nfsiod11:27:490
3551869992215,2cyclictest3109090-1atop09:40:010
3551869992214,1cyclictest3867604-21gltestperf10:45:220
3551869992213,8cyclictest4031739-21latency_hist12:20:010
3551869992211,4cyclictest0-21swapper/008:00:220
3551869992210,11cyclictest0-21swapper/009:45:010
355186999220,8cyclictest1-21systemd08:20:000
3551870992118,2cyclictest1700-21systemd-logind12:20:021
3551870992116,3cyclictest0-21swapper/110:45:271
3551870992113,3cyclictest3750115-21grep09:35:271
355186999219,5cyclictest3809185-21kthreadcore10:10:260
355186999219,5cyclictest3809185-21kthreadcore10:10:260
355186999218,6cyclictest3881693-21diskmemload10:51:390
355186999217,6cyclictest0-21swapper/009:55:280
355186999216,5cyclictest3560849-21tail07:15:170
3551869992116,4cyclictest0-21swapper/007:15:010
3551869992115,5cyclictest4017745-21kthreadcore12:10:240
3551869992115,5cyclictest3630823-21kthreadcore08:10:260
3551869992115,2cyclictest3674037-21kthreadcore08:45:240
3551869992115,2cyclictest3674037-21kthreadcore08:45:240
3551869992115,2cyclictest1652-1atopacctd07:55:010
3551869992114,6cyclictest3929135-21kthreadcore11:20:230
3551869992114,6cyclictest3623214-21kthreadcore08:05:240
3551869992114,6cyclictest3587880-21kthreadcore07:35:280
3551869992114,6cyclictest3581800-21kthreadcore07:30:230
3551869992114,3cyclictest3991440-21kthreadcore11:55:240
3551869992114,3cyclictest3696697-21grep09:05:240
3551869992113,3cyclictest3728579-21gltestperf09:25:210
835298200,3rtkit-daemon8734-21sendmail-mta09:14:431
355187199201,9cyclictest0-21swapper/209:30:012
3551870992017,2cyclictest3939267-21kthreadcore11:25:271
3551870992015,4cyclictest0-21swapper/112:10:031
355186999208,2cyclictest0-21swapper/009:30:340
355186999204,8cyclictest4008148-21cut12:05:260
3551869992015,4cyclictest3578752-21latency_hist07:30:000
3551869992014,5cyclictest3766698-21kthreadcore09:45:240
3551869992014,5cyclictest3683089-21latency_hist08:55:010
3551869992014,2cyclictest3889505-21unixbench-2d10:55:300
355186999200,8cyclictest3825375-21cut10:20:240
302487291201,7phc2sys0-21swapper/307:55:163
302487291200,2phc2sys0-21swapper/311:06:403
3551871991916,1cyclictest0-21swapper/209:15:002
3551871991916,1cyclictest0-21swapper/208:25:002
3551871991916,1cyclictest0-21swapper/207:15:012
355187099199,3cyclictest3787386-21runrttasks09:58:121
3551870991916,2cyclictest4058166-21sh12:35:011
3551870991916,2cyclictest3753990-21aten_repower_po09:40:161
3551870991916,2cyclictest3683768-21cpuspeed_turbos08:55:181
3551870991916,2cyclictest3634007-21cut08:15:011
3551870991916,2cyclictest3597271-21perf07:45:001
3551870991914,3cyclictest0-21swapper/111:55:011
3551870991911,2cyclictest0-21swapper/111:35:241
3551869991912,2cyclictest0-21swapper/007:20:210
3551869991910,4cyclictest0-21swapper/007:45:010
3551871991816,1cyclictest0-21swapper/207:40:002
3551871991813,3cyclictest3577940-21taskset07:25:282
355187099189,3cyclictest4066975-21/usr/sbin/munin12:40:021
355187099185,3cyclictest3727914-21aten2.4-expect09:25:181
3551870991815,2cyclictest3805864-21systemd-run10:10:011
3551870991814,3cyclictest3881132-21sshd10:50:481
3551870991811,3cyclictest3551867-21cyclictest10:15:271
355186999185,5cyclictest0-21swapper/008:42:520
302487291181,2phc2sys0-21swapper/312:29:443
302487291180,3phc2sys0-21swapper/309:15:023
302487291180,3phc2sys0-21swapper/307:30:013
302487291180,2phc2sys0-21swapper/311:57:023
835298170,2rtkit-daemon1652-1atopacctd10:30:541
355187199172,8cyclictest0-21swapper/208:40:012
3551871991714,1cyclictest0-21swapper/209:40:002
3551871991713,2cyclictest0-21swapper/207:45:012
355187099175,2cyclictest170550irq/121-eno111:50:021
355187099171,6cyclictest3639944-21latency_hist08:20:011
3551870991714,2cyclictest4014585-21latency_hist12:10:021
3551870991714,2cyclictest3668356-21kthreadcore08:40:271
3551870991714,2cyclictest3551867-21cyclictest10:00:301
302487291170,3phc2sys0-21swapper/311:32:163
302487291170,3phc2sys0-21swapper/311:24:123
302487291170,2phc2sys0-21swapper/310:37:433
302487291170,2phc2sys0-21swapper/310:04:363
302487291170,2phc2sys0-21swapper/309:35:073
3551871991614,1cyclictest0-21swapper/211:10:262
355187099168,4cyclictest3664097-21/usr/sbin/munin08:35:291
355187099166,3cyclictest0-21swapper/111:58:241
355187099166,3cyclictest0-21swapper/108:53:201
355187099165,6cyclictest3892307-21latency_hist11:00:011
355187099165,2cyclictest170550irq/121-eno110:35:271
355187099164,4cyclictest3615956-21aten2.4-expect08:00:181
3551870991613,2cyclictest3970924-21latency_hist11:45:001
3551870991613,2cyclictest3928116-21cpuspeed_turbos11:20:181
3551870991613,2cyclictest3832186-21/usr/sbin/munin10:25:221
3551870991613,2cyclictest3715747-21timerwakeupswit09:15:291
3551870991613,2cyclictest3658725-21dump-pmu-power08:35:011
3551870991613,2cyclictest3652608-21munin-run08:30:011
3551870991613,2cyclictest3551867-21cyclictest09:09:221
3551870991613,2cyclictest1700-21systemd-logind09:34:551
3551870991612,3cyclictest3861156-21kthreadcore10:40:261
3551870991612,3cyclictest3807395-21irqcore10:10:231
3551870991612,3cyclictest3807395-21irqcore10:10:231
3551870991612,3cyclictest3674514-21kthreadcore08:45:241
3551870991612,3cyclictest3674514-21kthreadcore08:45:241
302487291161,4phc2sys0-21swapper/312:15:183
302487291160,3phc2sys0-21swapper/309:47:253
302487291160,3phc2sys0-21swapper/309:05:003
302487291160,2phc2sys0-21swapper/312:32:243
302487291160,2phc2sys0-21swapper/311:13:003
355187199158,3cyclictest0-21swapper/208:00:172
3551871991513,1cyclictest0-21swapper/212:20:022
3551871991512,1cyclictest0-21swapper/212:25:252
3551871991512,1cyclictest0-21swapper/211:35:202
3551871991512,1cyclictest0-21swapper/210:30:222
3551871991512,1cyclictest0-21swapper/210:20:252
3551871991512,1cyclictest0-21swapper/209:40:522
3551871991512,1cyclictest0-21swapper/208:50:002
3551871991512,1cyclictest0-21swapper/208:50:002
3551871991512,1cyclictest0-21swapper/207:45:232
3551871991512,1cyclictest0-21swapper/207:35:022
3551871991511,2cyclictest0-21swapper/212:07:382
3551871991511,2cyclictest0-21swapper/211:30:202
3551871991511,2cyclictest0-21swapper/210:05:272
355187099156,2cyclictest170550irq/121-eno112:00:291
355187099156,2cyclictest0-21swapper/107:20:021
355187099155,3cyclictest0-21swapper/107:51:351
355187099153,4cyclictest0-21swapper/112:20:221
355187099153,4cyclictest0-21swapper/112:20:221
3551870991513,1cyclictest0-21swapper/107:10:011
3551870991512,2cyclictest3901510-21aten_repower_cu11:05:161
3551870991512,2cyclictest3765462-21kthreadcore09:45:241
3551870991512,2cyclictest3645189-21latency08:20:241
3551870991512,2cyclictest3623319-21kthreadcore08:05:241
3551870991512,2cyclictest3589748-21latency07:35:281
3551870991512,2cyclictest0-21swapper/111:15:021
3551870991512,2cyclictest0-21swapper/107:15:011
3551870991511,3cyclictest3692295-21kthreadcore09:00:261
3551870991511,3cyclictest3615771-21systemd-run08:00:021
355187099150,6cyclictest3603347-21latency_hist07:50:021
3551869991512,2cyclictest3641154-21fschecks_count08:20:190
302487291157,3phc2sys0-21swapper/307:10:013
302487291151,3phc2sys0-21swapper/308:40:163
302487291151,3phc2sys0-21swapper/308:00:173
302487291150,2phc2sys0-21swapper/312:05:043
302487291150,2phc2sys0-21swapper/310:07:523
302487291150,2phc2sys0-21swapper/309:53:553
302487291150,2phc2sys0-21swapper/309:34:273
302487291150,2phc2sys0-21swapper/308:38:243
40008812140,2sleep30-21swapper/312:00:253
355187199141,6cyclictest0-21swapper/211:10:012
3551871991412,1cyclictest0-21swapper/212:11:232
3551871991412,1cyclictest0-21swapper/210:35:302
3551871991411,1cyclictest0-21swapper/212:31:392
3551871991411,1cyclictest0-21swapper/211:45:262
3551871991411,1cyclictest0-21swapper/211:45:002
3551871991411,1cyclictest0-21swapper/210:50:012
3551871991411,1cyclictest0-21swapper/210:45:012
3551871991411,1cyclictest0-21swapper/209:50:012
3551871991411,1cyclictest0-21swapper/209:30:342
3551871991410,2cyclictest0-21swapper/209:55:262
3551870991412,1cyclictest3578714-21cut07:30:001
3551870991411,2cyclictest3551867-21cyclictest09:50:231
3551869991411,2cyclictest0-21swapper/007:50:010
302487291140,3phc2sys0-21swapper/310:42:403
302487291140,3phc2sys0-21swapper/310:32:243
302487291140,3phc2sys0-21swapper/309:20:233
302487291140,2phc2sys0-21swapper/311:19:343
302487291140,2phc2sys0-21swapper/310:47:213
302487291140,2phc2sys0-21swapper/310:20:283
302487291140,2phc2sys0-21swapper/310:10:313
302487291140,2phc2sys0-21swapper/310:10:313
355187199131,7cyclictest0-21swapper/209:15:162
355187199131,6cyclictest0-21swapper/210:50:172
355187199131,4cyclictest0-21swapper/211:00:012
3551871991311,1cyclictest0-21swapper/211:00:232
3551871991311,1cyclictest0-21swapper/209:20:152
3551871991310,1cyclictest0-21swapper/212:00:292
3551871991310,1cyclictest0-21swapper/211:25:272
3551871991310,1cyclictest0-21swapper/211:15:242
3551871991310,1cyclictest0-21swapper/210:25:272
3551871991310,1cyclictest0-21swapper/210:10:242
3551871991310,1cyclictest0-21swapper/210:10:242
3551871991310,1cyclictest0-21swapper/208:50:252
3551871991310,1cyclictest0-21swapper/207:59:562
3551871991310,1cyclictest0-21swapper/207:25:002
3551871991310,1cyclictest0-21swapper/207:20:002
355187099139,3cyclictest3569660-21kthreadcore07:20:261
3551870991311,1cyclictest3579078-21aten_repower_en07:30:161
302487291133,2phc2sys0-21swapper/311:28:163
302487291131,3phc2sys0-21swapper/312:10:173
302487291131,3phc2sys0-21swapper/307:20:183
302487291130,4phc2sys0-21swapper/310:25:183
302487291130,3phc2sys0-21swapper/307:14:553
302487291130,2phc2sys0-21swapper/312:35:103
302487291130,2phc2sys0-21swapper/311:50:333
302487291130,2phc2sys0-21swapper/309:42:303
302487291130,2phc2sys0-21swapper/307:17:013
355187199129,1cyclictest0-21swapper/212:36:062
355187199129,1cyclictest0-21swapper/209:05:192
355187199129,1cyclictest0-21swapper/209:00:242
355187199129,1cyclictest0-21swapper/208:15:232
355187199129,1cyclictest0-21swapper/208:10:242
355187199129,1cyclictest0-21swapper/208:05:302
355187199129,1cyclictest0-21swapper/207:50:232
355187199123,4cyclictest0-21swapper/208:30:152
355187199122,4cyclictest0-21swapper/210:00:162
3551871991210,1cyclictest0-21swapper/210:15:252
3551871991210,1cyclictest0-21swapper/208:45:002
355186999121,9cyclictest1-21systemd07:10:020
302487291125,3phc2sys0-21swapper/308:55:003
302487291121,5phc2sys0-21swapper/308:45:163
302487291121,5phc2sys0-21swapper/308:45:163
302487291120,2phc2sys0-21swapper/311:45:383
302487291120,2phc2sys0-21swapper/307:48:273
355187199119,1cyclictest0-21swapper/211:55:272
355187199119,1cyclictest0-21swapper/211:55:002
355187199119,1cyclictest0-21swapper/209:50:222
355187199118,1cyclictest0-21swapper/211:20:192
355187199118,1cyclictest0-21swapper/208:55:562
355187199118,1cyclictest0-21swapper/208:25:132
355187199116,3cyclictest0-21swapper/212:20:172
355187199116,3cyclictest0-21swapper/212:20:172
302487291113,3phc2sys0-21swapper/311:00:183
302487291112,2phc2sys0-21swapper/310:16:513
302487291112,2phc2sys0-21swapper/307:35:283
302487291111,4phc2sys0-21swapper/311:40:183
302487291111,3phc2sys0-21swapper/311:38:223
302487291111,2phc2sys0-21swapper/309:26:253
302487291111,2phc2sys0-21swapper/307:30:153
302487291110,3phc2sys0-21swapper/310:51:493
302487291110,3phc2sys0-21swapper/308:25:173
302487291110,2phc2sys0-21swapper/309:58:313
302487291110,2phc2sys0-21swapper/308:55:103
302487291101,2phc2sys0-21swapper/309:10:293
302487291101,2phc2sys0-21swapper/308:30:223
302487291100,3phc2sys0-21swapper/312:20:033
302487291100,3phc2sys0-21swapper/312:20:033
302487291100,3phc2sys0-21swapper/308:20:163
302487291100,2phc2sys0-21swapper/310:55:103
302487291100,2phc2sys0-21swapper/309:05:243
302487291100,2phc2sys0-21swapper/308:15:033
302487291100,2phc2sys0-21swapper/308:10:313
302487291100,2phc2sys0-21swapper/307:50:063
302487291100,2phc2sys0-21swapper/307:43:503
35518719996,1cyclictest0-21swapper/207:10:012
30248729190,2phc2sys0-21swapper/308:05:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional