You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 10:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackeslot2.osadl.org (updated Mon Dec 01, 2025 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
283706021010,2sleep12837062-21/usr/sbin/munin00:25:261
27159642300,3sleep30-21swapper/323:15:183
2347176992818,2cyclictest2392136-21idleruntime-cro19:45:011
2347176992815,2cyclictest170550irq/121-eno100:15:011
2347175992616,6cyclictest2732625-21/usr/sbin/munin23:25:160
26768712250,2sleep20-21swapper/222:50:292
26768712250,2sleep20-21swapper/222:50:292
2347175992515,6cyclictest2745870-21kthreadcore23:30:220
2347175992213,7cyclictest0-21swapper/021:55:020
2347175992213,5cyclictest2557854-21cpuspeed_turbos21:45:160
2347180992118,1cyclictest0-21swapper/223:05:012
2347176992118,2cyclictest2539787-21dump-pmu-power21:35:001
2347176992114,3cyclictest2667550-21latency22:45:231
2347175992112,5cyclictest2831334-21true00:25:000
302487291200,4phc2sys0-21swapper/320:15:143
2347180992018,1cyclictest0-21swapper/200:40:022
2347180992017,1cyclictest0-21swapper/222:15:012
234717699203,2cyclictest2407712-21kthreadcore19:55:201
2347176992017,2cyclictest2627420-21apt-key22:25:011
2347176992017,2cyclictest2483766-21apt-key21:00:011
2347176992017,2cyclictest1635-21dbus-daemon20:20:021
234717599201,6cyclictest2513818-21munin-run21:20:010
2347175992012,4cyclictest2690824-21kthreadcore23:00:210
2347175992011,5cyclictest2392792-21tr19:45:150
302487291190,2phc2sys0-21swapper/323:30:473
2347176991916,2cyclictest2644939-21/usr/sbin/munin22:35:011
2347176991916,2cyclictest2609668-21apt-key22:15:001
2347176991916,2cyclictest2361708-21cron19:20:001
2347176991916,2cyclictest2347169-21cyclictest23:40:021
2347176991913,5cyclictest2729019-21sendmail_mailqu23:20:251
2347175991916,2cyclictest2793169-21latency_hist00:00:010
2347175991916,2cyclictest1652-1atopacctd00:10:000
2347175991915,3cyclictest1702-21in:imuxsock20:10:020
2347175991910,8cyclictest0-21swapper/000:35:160
234717599190,18cyclictest2698411-21cstates23:05:180
302487291180,2phc2sys0-21swapper/323:39:453
302487291180,2phc2sys0-21swapper/321:11:123
2347180991815,1cyclictest0-21swapper/223:51:562
2347180991815,1cyclictest0-21swapper/222:40:012
234717699189,4cyclictest0-21swapper/121:20:021
234717699182,5cyclictest2465624-21apt-get20:45:151
2347176991815,2cyclictest2792765-21runrttasks23:59:361
2347176991815,2cyclictest2379970-21munin-run19:35:001
2347175991815,2cyclictest2627498-21apt-key22:25:010
2347175991815,2cyclictest2465267-21cron20:45:000
2347175991811,6cyclictest0-21swapper/022:54:530
2347175991811,6cyclictest0-21swapper/022:54:530
2347175991810,4cyclictest2842569-21kthreadcore00:30:230
302487291170,2phc2sys0-21swapper/322:02:013
25393112170,2sleep20-21swapper/221:34:102
2347180991714,1cyclictest0-21swapper/222:10:002
2347180991714,1cyclictest0-21swapper/219:20:002
2347180991714,1cyclictest0-21swapper/200:25:012
2347180991714,1cyclictest0-21swapper/200:10:002
2347180991714,1cyclictest0-21swapper/200:05:012
2347180991713,2cyclictest0-21swapper/223:25:092
2347180991713,2cyclictest0-21swapper/220:50:232
234717699179,4cyclictest2820118-21runrttasks00:15:481
234717699178,4cyclictest0-21swapper/123:05:021
2347176991714,2cyclictest2653795-21systemd-run22:40:011
2347176991714,2cyclictest2600660-21gpgconf22:10:001
2347176991714,2cyclictest2522585-21systemd-run21:25:011
2347176991714,2cyclictest1-21systemd21:05:271
234717699171,1cyclictest2715190-21apt-get23:15:141
2347176991710,3cyclictest2706656-21aten2.4-expect23:10:161
234717599177,6cyclictest0-21swapper/019:55:490
234717599177,5cyclictest0-21swapper/023:15:230
2347175991714,2cyclictest2609601-21apt-key22:15:000
2347175991714,2cyclictest2379930-21cron19:35:000
2347175991714,2cyclictest2347169-21cyclictest22:35:020
2347175991714,2cyclictest2347169-21cyclictest20:35:290
2347175991714,2cyclictest1635-21dbus-daemon22:45:020
2347175991710,4cyclictest2794407-21if_err_enp1s0.800:00:200
835298160,3rtkit-daemon2353468-21munin-node22:19:281
302487291160,2phc2sys0-21swapper/323:57:353
302487291160,2phc2sys0-21swapper/323:28:073
2347180991614,1cyclictest0-21swapper/222:35:012
2347180991613,1cyclictest0-21swapper/223:15:202
2347180991613,1cyclictest0-21swapper/221:30:012
2347180991613,1cyclictest0-21swapper/200:10:452
2347176991613,2cyclictest2743357-21kthreadcore23:30:201
2347176991613,2cyclictest2683407-21kthreadcore22:55:221
2347176991613,2cyclictest2557352-21systemctl21:45:021
2347176991613,2cyclictest2522980-21aten_repower_po21:25:151
2347176991613,2cyclictest2447124-21cut20:30:011
2347176991612,2cyclictest0-21swapper/120:40:011
2347176991611,4cyclictest2489581-21sshd21:01:501
2347176991611,2cyclictest170550irq/121-eno123:50:001
234717599168,7cyclictest0-21swapper/000:15:180
234717599168,2cyclictest0-21swapper/023:24:560
234717599167,2cyclictest0-21swapper/022:19:070
234717599164,4cyclictest0-21swapper/023:45:300
234717599164,10cyclictest2498019-21cut21:10:200
234717599161,9cyclictest0-21swapper/022:57:380
234717599161,6cyclictest2653674-21ssh22:40:010
234717599161,6cyclictest0-21swapper/000:25:220
2347175991613,2cyclictest2522628-21latency_hist21:25:010
2347175991613,2cyclictest2483778-21apt-key21:00:010
2347175991613,2cyclictest2416439-21perf20:05:000
2347175991613,2cyclictest2404371-21cut19:55:010
2347175991613,2cyclictest0-21swapper/021:40:000
2347175991612,3cyclictest2574664-21kworker/u16:3+nfsiod21:55:590
2347175991612,3cyclictest2385287-21mailstats19:35:260
2347175991612,3cyclictest2009-21runrttasks21:25:220
234717599161,10cyclictest0-21swapper/023:10:550
2347175991610,5cyclictest2368201-21tail19:25:130
27583472150,1sleep02758346-21munin-plugin-st23:40:000
24379152150,2sleep20-21swapper/220:20:242
234718099159,3cyclictest0-21swapper/219:45:242
2347180991513,1cyclictest0-21swapper/222:00:202
2347180991513,1cyclictest0-21swapper/221:10:212
2347180991512,1cyclictest0-21swapper/223:59:552
2347180991512,1cyclictest0-21swapper/223:30:182
2347180991512,1cyclictest0-21swapper/222:55:242
2347180991512,1cyclictest0-21swapper/222:25:392
2347180991512,1cyclictest0-21swapper/221:56:112
2347180991512,1cyclictest0-21swapper/221:50:162
2347180991512,1cyclictest0-21swapper/221:22:242
2347180991512,1cyclictest0-21swapper/221:05:012
2347180991512,1cyclictest0-21swapper/220:40:252
2347180991512,1cyclictest0-21swapper/220:30:022
2347180991512,1cyclictest0-21swapper/220:15:172
2347180991512,1cyclictest0-21swapper/220:10:202
2347180991512,1cyclictest0-21swapper/219:55:202
2347180991510,2cyclictest0-21swapper/222:47:532
234718099150,13cyclictest0-21swapper/221:16:072
234717699159,2cyclictest11systemd22:00:011
2347176991513,1cyclictest2697864-21apt23:05:161
2347176991512,2cyclictest2656439-21kthreadcore22:40:221
2347176991512,2cyclictest1-21systemd23:55:011
2347176991512,2cyclictest0-21swapper/120:25:011
2347176991511,3cyclictest2775771-21kworker/u16:1+nfsiod00:31:581
2347176991511,3cyclictest2775771-21kworker/u16:1+nfsiod00:05:121
2347176991511,3cyclictest2688788-21kworker/u16:0+nfsiod00:01:031
2347176991511,3cyclictest2677093-21kworker/u16:3+nfsiod22:51:111
2347176991511,3cyclictest2677093-21kworker/u16:3+nfsiod22:51:111
2347176991511,3cyclictest1652-1atopacctd23:25:201
2347176991510,4cyclictest2347169-21cyclictest21:12:001
234717599159,2cyclictest2367815-21turbostat19:25:000
234717599158,2cyclictest0-21swapper/022:05:240
234717599157,2cyclictest0-21swapper/021:30:300
2347175991512,2cyclictest2495931-21apt-key21:10:010
2347175991512,2cyclictest2352564-21ptp4l-jitter19:10:260
2347175991512,2cyclictest2347169-21cyclictest20:25:210
2347175991511,3cyclictest2651050-21kworker/u16:0+nfsiod22:45:560
2347175991511,3cyclictest2513883-21kworker/u16:2+nfsiod22:25:360
2347175991511,3cyclictest2448563-21irqrtprio20:30:200
2347175991511,3cyclictest2360960-21mailstats19:15:260
302487291141,4phc2sys0-21swapper/320:40:223
302487291141,2phc2sys0-21swapper/300:31:183
302487291140,3phc2sys0-21swapper/300:25:143
302487291140,2phc2sys0-21swapper/323:52:513
302487291140,2phc2sys0-21swapper/323:46:463
302487291140,2phc2sys0-21swapper/323:42:263
302487291140,2phc2sys0-21swapper/322:20:093
302487291140,2phc2sys0-21swapper/322:19:533
302487291140,2phc2sys0-21swapper/322:11:353
302487291140,2phc2sys0-21swapper/321:51:033
302487291140,2phc2sys0-21swapper/321:40:213
234718099148,3cyclictest0-21swapper/200:15:102
2347180991412,1cyclictest0-21swapper/220:45:182
2347180991411,1cyclictest0-21swapper/223:38:042
2347180991411,1cyclictest0-21swapper/222:25:002
2347180991411,1cyclictest0-21swapper/221:45:212
2347180991411,1cyclictest0-21swapper/220:05:172
2347180991410,2cyclictest0-21swapper/223:10:512
2347180991410,2cyclictest0-21swapper/220:55:272
2347176991411,2cyclictest2850391-21irqrtprio00:35:231
2347176991411,2cyclictest2824393-21kthreadcore00:20:191
2347176991411,2cyclictest2570571-21kthreadcore21:50:221
2347176991411,2cyclictest2540397-21aten2.4-expect21:35:151
2347176991411,2cyclictest2448037-21gltestperf20:30:181
2347176991411,2cyclictest2418306-21kthreadcore20:05:191
2347176991411,2cyclictest1704-21rs:main1
2347176991411,2cyclictest1635-21dbus-daemon20:45:001
2347176991411,2cyclictest0-21swapper/119:55:011
2347176991411,2cyclictest0-21swapper/119:10:011
2347176991410,3cyclictest2759427-21head23:40:181
234717599145,7cyclictest2476077-21tr20:50:230
234717599145,5cyclictest0-21swapper/021:40:200
234717599141,4cyclictest2386359-21/usr/sbin/munin19:40:170
2347175991412,1cyclictest2766742-21runrttasks23:44:410
2347175991411,2cyclictest2810259-21needreboot00:10:190
2347175991411,2cyclictest2777089-21irqcore23:50:200
2347175991411,2cyclictest2487880-21kthreadcore21:00:210
2347175991411,2cyclictest1635-21dbus-daemon22:05:000
302487291131,3phc2sys0-21swapper/322:31:323
302487291131,2phc2sys0-21swapper/300:12:113
302487291130,2phc2sys0-21swapper/323:23:023
302487291130,2phc2sys0-21swapper/323:13:023
302487291130,2phc2sys0-21swapper/323:07:593
302487291130,2phc2sys0-21swapper/322:51:563
302487291130,2phc2sys0-21swapper/322:51:563
302487291130,2phc2sys0-21swapper/321:49:353
302487291130,2phc2sys0-21swapper/319:50:173
302487291130,2phc2sys0-21swapper/300:02:273
234718099139,2cyclictest0-21swapper/222:40:232
234718099131,5cyclictest0-21swapper/220:40:012
2347180991311,1cyclictest0-21swapper/200:30:232
2347180991310,1cyclictest0-21swapper/222:15:242
2347180991310,1cyclictest0-21swapper/221:40:202
2347180991310,1cyclictest0-21swapper/221:05:212
2347180991310,1cyclictest0-21swapper/219:14:012
234718099130,10cyclictest0-21swapper/223:22:572
234717699139,3cyclictest2393765-21kthreadcore19:45:201
234717699135,3cyclictest2583589-21/usr/sbin/munin22:00:221
234717699131,6cyclictest2386031-21latency_hist19:40:021
2347176991310,2cyclictest2629106-21irqrtprio22:25:211
2347176991310,2cyclictest2557704-21head21:45:151
2347176991310,2cyclictest2414393-21kthreadcore20:00:221
2347176991310,2cyclictest2368394-21cpuspeed_turbos19:25:141
2347176991310,2cyclictest2354415-21kthreadcore19:15:021
234717599133,6cyclictest0-21swapper/020:24:310
2347175991310,2cyclictest2428905-21tail20:15:140
234717599130,7cyclictest2423105-21cut20:10:180
302487291123,3phc2sys0-21swapper/319:40:233
302487291121,3phc2sys0-21swapper/321:15:173
302487291121,2phc2sys0-21swapper/320:55:173
302487291120,3phc2sys0-21swapper/321:25:143
302487291120,3phc2sys0-21swapper/320:50:163
302487291120,2phc2sys0-21swapper/322:55:443
302487291120,2phc2sys0-21swapper/322:45:513
302487291120,2phc2sys0-21swapper/319:20:553
302487291120,2phc2sys0-21swapper/300:24:183
302487291120,2phc2sys0-21swapper/300:18:293
234718099129,1cyclictest0-21swapper/221:35:162
234718099129,1cyclictest0-21swapper/219:35:212
234718099129,1cyclictest0-21swapper/219:30:212
2347180991210,1cyclictest0-21swapper/223:40:172
2347180991210,1cyclictest0-21swapper/220:30:212
2347180991210,1cyclictest0-21swapper/220:00:222
2347180991210,1cyclictest0-21swapper/219:50:242
2347180991210,1cyclictest0-21swapper/200:25:132
234717699129,2cyclictest2475887-21kthreadcore20:50:231
234717699127,3cyclictest0-21swapper/120:15:011
234717599129,2cyclictest2468813-21kthreadcore20:45:220
302487291112,3phc2sys0-21swapper/323:03:503
302487291111,2phc2sys0-21swapper/320:30:243
302487291111,2phc2sys0-21swapper/319:45:163
302487291110,4phc2sys0-21swapper/319:55:133
302487291110,3phc2sys0-21swapper/320:20:193
302487291110,2phc2sys0-21swapper/322:35:493
302487291110,2phc2sys0-21swapper/320:15:023
302487291110,2phc2sys0-21swapper/319:30:023
234718099119,1cyclictest0-21swapper/223:46:342
234718099119,1cyclictest0-21swapper/223:06:372
234718099119,1cyclictest0-21swapper/219:25:132
234718099119,1cyclictest0-21swapper/219:20:172
234718099118,1cyclictest0-21swapper/219:40:162
234718099112,5cyclictest0-21swapper/219:10:022
234717599115,1cyclictest2346888-21apt-get19:10:010
302487291101,2phc2sys0-21swapper/321:00:143
302487291100,3phc2sys0-21swapper/322:05:143
302487291100,3phc2sys0-21swapper/322:00:013
302487291100,3phc2sys0-21swapper/321:23:263
302487291100,3phc2sys0-21swapper/319:10:153
302487291100,2phc2sys0-21swapper/322:40:183
302487291100,2phc2sys0-21swapper/322:25:153
302487291100,2phc2sys0-21swapper/321:36:083
302487291100,2phc2sys0-21swapper/321:32:123
302487291100,2phc2sys0-21swapper/300:38:283
302487291100,2phc2sys0-21swapper/300:05:383
30248729190,3phc2sys0-21swapper/319:35:033
30248729190,2phc2sys0-21swapper/321:05:033
30248729190,2phc2sys0-21swapper/320:35:173
30248729190,2phc2sys0-21swapper/320:25:243
30248729190,2phc2sys0-21swapper/320:05:163
30248729190,2phc2sys0-21swapper/320:00:153
30248729190,2phc2sys0-21swapper/319:32:223
30248729190,2phc2sys0-21swapper/319:15:023
30248729190,2phc2sys0-21swapper/319:05:243
30248729180,2phc2sys0-21swapper/320:45:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional