You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 15:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Thu Apr 02, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2894321670,20sleep228944-21fschecks_time06:59:222
2894321670,20sleep228944-21fschecks_time06:59:222
301579911810,24cyclictest16151-21sh09:31:410
301649911464,43cyclictest19591-21threads12:19:331
30157991123,33cyclictest30158-21date07:04:040
30157991123,33cyclictest30158-21date07:04:040
301579911211,21cyclictest27840-21sh09:51:030
416221100,6sleep00-21swapper/011:54:210
301649910953,49cyclictest6227-21ssh10:09:271
301579910952,52cyclictest30149-21cyclictest12:29:440
301579910713,25cyclictest17538-21sh12:16:300
301579910713,22cyclictest20464-21sh12:21:230
301579910584,14cyclictest25980-21ssh09:49:020
30157991054,29cyclictest30287-21runrttasks10:49:260
30157991053,41cyclictest11915-21apt-get10:19:100
30157991053,26cyclictest29036-21sh10:47:510
30157991052,23cyclictest9145-21sh10:14:430
301579910344,53cyclictest15411-21cron12:13:590
30157991032,29cyclictest1440-21ssh11:49:280
30157991032,18cyclictest5095-21latency_hist09:14:040
30157991032,18cyclictest5095-21latency_hist09:14:040
30157991023,32cyclictest3286-21/usr/sbin/munin10:59:160
30157991023,32cyclictest3286-21/usr/sbin/munin10:59:160
30157991023,23cyclictest6261-21/usr/sbin/munin11:04:390
30157991022,21cyclictest23625-21cat10:39:010
30164991013,41cyclictest3045-21latency_hist10:59:011
30157991013,23cyclictest18526-21grep10:29:310
30157991013,23cyclictest18526-21grep10:29:310
3237721000,7sleep00-21swapper/009:59:170
301649910038,8cyclictest0-21swapper/110:39:221
301649910035,11cyclictest0-21swapper/109:29:011
30157991006,87cyclictest12281-21kworker/u19:1+ext4-rsv-conversion11:39:360
30157991003,30cyclictest12109-21latency_hist11:14:030
3016499993,44cyclictest28367-21cpu08:54:161
3015799996,9cyclictest0-21swapper/009:04:360
3015799993,32cyclictest11836-21latency_hist08:04:020
3015799993,30cyclictest352-21runrttasks12:25:230
3015799993,30cyclictest16113-21sed11:19:350
3015799993,30cyclictest16113-21sed11:19:350
3015799993,27cyclictest352-21runrttasks08:19:110
3015799993,25cyclictest18167-21/usr/sbin/munin11:24:100
3015799993,24cyclictest6131-21timerwakeupswit09:14:350
3015799992,25cyclictest3143-21latency_hist10:59:020
3015799983,25cyclictest13893-21cstates08:09:150
3015799983,25cyclictest13893-21cstates08:09:150
3015799974,9cyclictest21902-21apt-get08:34:100
3015799973,25cyclictest8581-21date10:14:020
3015799973,24cyclictest25318-21sh10:40:420
290372979,61sleep129041-21kernelversion06:59:261
290372979,61sleep129041-21kernelversion06:59:261
30164999643,14cyclictest17548-21latency_hist10:29:041
3015799963,24cyclictest20219-21latency_hist09:39:030
3015799963,23cyclictest15431-21runrttasks10:24:260
3015799963,22cyclictest352-21runrttasks12:05:170
30157999610,23cyclictest30998-21diskmemload09:56:270
2925629632,37sleep30-21swapper/306:59:373
2925629632,37sleep30-21swapper/306:59:373
3016499953,44cyclictest26919-21fschecks_count08:49:201
30157999533,54cyclictest0-21swapper/008:49:020
30157999533,54cyclictest0-21swapper/008:49:020
3015799953,30cyclictest3191-21cat10:04:260
3015799953,27cyclictest12206-21latency09:25:310
3015799953,22cyclictest352-21runrttasks11:48:190
3015799953,20cyclictest12430-21fschecks_count08:05:020
3015799952,30cyclictest29951-21/usr/sbin/munin08:59:280
3015799952,30cyclictest29951-21/usr/sbin/munin08:59:280
3015799952,29cyclictest8146-21rm09:19:110
3015799952,25cyclictest3980-21netstat07:29:220
30157999515,25cyclictest7490-21taskset07:45:320
3015799943,30cyclictest2952-21systemd-journal11:37:000
3015799943,26cyclictest352-21runrttasks07:54:310
3015799943,26cyclictest352-21runrttasks07:54:310
3015799943,26cyclictest28445-21cstates08:54:160
30157999336,49cyclictest0-21swapper/007:29:040
30157999336,49cyclictest0-21swapper/007:29:040
3015799933,28cyclictest22642-21ssh11:30:410
3015799933,27cyclictest16241-21timerandwakeup08:14:340
3015799933,21cyclictest297-21systemd-logind07:53:400
3015799932,27cyclictest7642-21/usr/sbin/munin11:59:360
3016499923,38cyclictest21158-21tail08:29:351
30164999220,27cyclictest5080-21sh11:54:571
3015799923,31cyclictest22960-21idleruntime-cro09:44:000
3015799923,31cyclictest22960-21idleruntime-cro09:44:000
3015799923,27cyclictest12533-21grep11:14:230
3015799922,21cyclictest5684-21latency_hist07:39:030
30174999170,13cyclictest17771-21switchtime08:19:342
3015799913,33cyclictest20490-21fschecks_time08:29:210
3015799912,31cyclictest30942-21latency_hist07:09:030
3015799912,31cyclictest30942-21latency_hist07:09:030
3015799912,25cyclictest18922-21fschecks_time08:24:220
3015799912,23cyclictest1505-21grep07:19:180
30164999013,66cyclictest27511-21rm11:39:351
3015799907,8cyclictest0-21swapper/008:49:220
3015799903,31cyclictest31721-21grep07:09:240
3015799902,22cyclictest5771-21/usr/sbin/munin07:39:220
3015799902,19cyclictest23508-21apt-get08:39:110
30164998871,10cyclictest11969-21apt-key08:04:021
3016499883,39cyclictest26529-21latency_hist08:49:011
3016499883,39cyclictest26529-21latency_hist08:49:011
30164998767,14cyclictest10305-21sh12:04:251
30174998659,20cyclictest9492-21fschecks_count11:09:202
30174998658,21cyclictest31322-21diskmemload12:04:232
30157998529,48cyclictest0-21swapper/007:19:000
30174998461,16cyclictest5504-21ssh10:09:032
30174998363,12cyclictest29865-21latency_hist08:59:032
30174998352,20cyclictest338-21idleruntime-cro07:14:002
30174998146,13cyclictest181rcu_preempt12:15:442
30174998058,15cyclictest21911-21cpu08:34:152
30174998054,17cyclictest31322-21diskmemload09:44:302
30174998033,7cyclictest181rcu_preempt09:54:352
3018299793,65cyclictest27521-21chrt11:39:373
30174997946,12cyclictest181rcu_preempt08:34:022
30174997941,11cyclictest181rcu_preempt12:23:022
30164997962,11cyclictest31322-21diskmemload10:04:231
30174997856,7cyclictest181rcu_preempt07:29:192
30174997833,29cyclictest17655-21idleruntime09:34:222
30164997862,11cyclictest30149-21cyclictest09:41:051
30164997862,11cyclictest30149-21cyclictest09:41:051
30174997753,7cyclictest181rcu_preempt09:25:312
30174997749,8cyclictest181rcu_preempt10:53:302
30174997740,9cyclictest499-21ssh11:48:552
30174997740,14cyclictest181rcu_preempt09:15:272
30164997738,14cyclictest181rcu_preempt11:39:021
30174997634,13cyclictest181rcu_preempt12:09:242
30174997555,14cyclictest31322-21diskmemload09:59:162
30174997552,8cyclictest181rcu_preempt08:39:232
30174997546,8cyclictest181rcu_preempt11:50:372
30174997544,13cyclictest181rcu_preempt11:31:052
30174997542,13cyclictest181rcu_preempt10:47:222
30174997539,13cyclictest181rcu_preempt10:29:032
30164997553,16cyclictest19952-21sh09:38:591
30164997552,15cyclictest972-21cpuspeed11:49:171
30164997547,20cyclictest1-21systemd12:17:161
30174997438,13cyclictest181rcu_preempt11:25:162
30174997438,11cyclictest181rcu_preempt11:14:492
30174997349,17cyclictest10834-21apt-key09:24:012
30174997346,12cyclictest181rcu_preempt10:23:502
30174997320,40cyclictest5644-21diskmemload11:03:072
30174997320,40cyclictest5644-21diskmemload11:03:072
3017499727,31cyclictest39-21ksoftirqd/210:14:412
30174997250,8cyclictest181rcu_preempt09:09:342
30174997250,8cyclictest181rcu_preempt09:09:342
30174997249,8cyclictest181rcu_preempt11:04:092
30174997249,8cyclictest181rcu_preempt09:04:242
30174997248,9cyclictest181rcu_preempt07:39:122
30174997246,9cyclictest240-21rpcbind09:49:322
30174997241,10cyclictest181rcu_preempt08:11:072
30174997241,10cyclictest181rcu_preempt08:11:072
30174997230,12cyclictest181rcu_preempt08:28:142
3017499722,38cyclictest39-21ksoftirqd/207:49:102
30174997223,25cyclictest181rcu_preempt09:34:042
30164997254,12cyclictest27323-21modprobe12:33:591
30164997253,12cyclictest31513-21latency_hist09:04:031
30164997253,12cyclictest31513-21latency_hist09:04:031
30164997251,15cyclictest31322-21diskmemload10:54:021
30174997150,14cyclictest58-21kcompactd007:19:352
30174997146,8cyclictest181rcu_preempt11:41:202
3017499714,45cyclictest352-21runrttasks07:07:102
3017499714,45cyclictest352-21runrttasks07:07:102
30174997143,11cyclictest181rcu_preempt08:14:102
30174997142,10cyclictest181rcu_preempt08:46:552
30174997142,10cyclictest181rcu_preempt08:46:552
30174997140,12cyclictest181rcu_preempt10:29:282
30174997140,12cyclictest181rcu_preempt10:29:282
30164997155,10cyclictest9910-21kworker/u17:0-xprtiod12:13:591
30164997152,13cyclictest2476-21sh09:09:221
30164997152,13cyclictest2476-21sh09:09:221
30164997150,15cyclictest24356-21sh12:28:591
30164997149,13cyclictest0-21swapper/110:29:111
30164997149,13cyclictest0-21swapper/110:29:111
30164997148,13cyclictest0-21swapper/109:59:361
30164997141,21cyclictest58-21kcompactd011:18:561
3017499705,37cyclictest12200-21apt-get08:04:152
30174997052,12cyclictest14909-21sh11:19:032
30174997052,12cyclictest14909-21sh11:19:032
30174997052,12cyclictest0-21swapper/210:58:592
30174997046,9cyclictest181rcu_preempt08:49:352
3017499704,44cyclictest3273-21idleruntime-cro07:29:002
3017499704,44cyclictest3273-21idleruntime-cro07:29:002
30174997043,9cyclictest181rcu_preempt10:37:442
30174997040,22cyclictest24344-21cron12:28:592
30174997038,11cyclictest181rcu_preempt07:36:192
30174997030,8cyclictest181rcu_preempt11:59:242
30174997030,12cyclictest181rcu_preempt11:34:252
30174997029,13cyclictest181rcu_preempt10:39:332
3017499702,29cyclictest39-21ksoftirqd/207:14:132
30164997053,11cyclictest580-21/usr/sbin/munin11:49:001
30164997048,13cyclictest11480-21kworker/u17:4-ext4-rsv-conversion09:34:001
30164997043,9cyclictest31-21ksoftirqd/111:04:241
30174996947,12cyclictest0-21swapper/207:59:172
30174996937,9cyclictest352-21runrttasks07:57:322
30174996937,9cyclictest352-21runrttasks07:57:322
30174996935,12cyclictest181rcu_preempt10:11:192
30174996925,14cyclictest181rcu_preempt11:54:052
30164996953,10cyclictest10117-21systemd-run07:59:001
30164996953,10cyclictest10117-21systemd-run07:59:001
30164996946,16cyclictest544-21cron07:54:001
30174996845,8cyclictest181rcu_preempt09:41:062
30174996845,8cyclictest181rcu_preempt09:41:062
30164996827,31cyclictest7124-21modprobe07:43:591
30174996744,14cyclictest0-21swapper/208:59:172
30174996744,14cyclictest0-21swapper/208:59:172
3016499675,37cyclictest9015-21modprobe10:14:211
30164996745,12cyclictest0-21swapper/111:26:531
3016499673,48cyclictest27934-21ssh10:44:431
30164996710,30cyclictest31322-21diskmemload11:00:411
30164996710,30cyclictest31322-21diskmemload11:00:411
30174996640,17cyclictest58-21kcompactd007:47:052
30164996651,9cyclictest58-21kcompactd009:49:151
3016499663,43cyclictest12840-21grep08:05:161
3016499663,42cyclictest6257-21head09:14:361
30182996531,21cyclictest1198-21chrt09:07:073
30174996542,12cyclictest0-21swapper/212:29:222
30164996551,7cyclictest0-21swapper/109:54:351
30164996548,11cyclictest2952-21systemd-journal07:17:561
30164996547,12cyclictest32071-21kernelversion09:04:261
30164996531,17cyclictest0-21swapper/111:59:401
3016499645,26cyclictest0-21swapper/110:34:241
30164996435,8cyclictest181rcu_preempt08:28:141
3016499643,43cyclictest15468-21fschecks_count11:19:211
3016499643,43cyclictest15468-21fschecks_count11:19:211
30164996349,10cyclictest30149-21cyclictest11:09:371
30164996347,10cyclictest18028-21journal-offline08:19:361
30164996327,11cyclictest181rcu_preempt10:21:071
30164996318,32cyclictest3584-21cpuspeed07:29:121
30182996230,20cyclictest21421-21chrt08:31:443
30164996245,9cyclictest0-21swapper/111:33:091
30164996244,12cyclictest3346-21latency_hist07:29:031
30164996244,12cyclictest3346-21latency_hist07:29:031
30164996243,12cyclictest2952-21systemd-journal07:24:011
3016499623,23cyclictest23489-21/usr/sbin/munin08:39:211
30164996215,33cyclictest30389-21sshd07:04:561
30164996215,33cyclictest30389-21sshd07:04:561
3018299613,40cyclictest5503-21chrt09:14:213
30164996142,13cyclictest31322-21diskmemload09:44:201
3016499613,37cyclictest22092-21fschecks_count08:34:201
30182996029,20cyclictest24819-21chrt08:43:503
30182996028,20cyclictest31204-21chrt09:02:423
30182996028,20cyclictest31204-21chrt09:02:423
30164996045,10cyclictest30149-21cyclictest07:39:011
30164996039,14cyclictest544-21cron09:24:001
30164996036,9cyclictest181rcu_preempt08:14:101
30164996036,14cyclictest0-21swapper/108:09:301
30164996036,14cyclictest0-21swapper/108:09:301
30182995928,20cyclictest4571-21chrt07:30:163
30164995939,14cyclictest30149-21cyclictest07:09:371
3018299584,29cyclictest0-21swapper/310:14:203
3018299583,36cyclictest15711-21chrt10:24:333
3018299582,36cyclictest22187-21chrt11:29:363
30182995728,18cyclictest20476-21chrt11:27:583
30182995727,20cyclictest7498-21chrt07:45:393
30164995732,10cyclictest181rcu_preempt07:49:031
3018299563,41cyclictest19261-21taskset10:30:283
3018299563,41cyclictest19261-21taskset10:30:283
30182995626,19cyclictest663-21chrt07:14:293
3018299544,36cyclictest6121-21taskset10:09:253
3018299544,34cyclictest26577-21chrt10:44:023
3018299543,38cyclictest4297-21chrt11:54:243
3018299543,38cyclictest32578-21chrt10:54:033
3018299543,37cyclictest31069-21chrt07:09:063
3018299543,36cyclictest8615-21chrt07:49:283
3018299543,35cyclictest22476-21chrt08:34:313
3018299543,35cyclictest21496-21chrt12:24:023
3018299542,37cyclictest32108-21chrt11:46:303
3018299533,37cyclictest3062-21chrt09:09:353
3018299533,37cyclictest3062-21chrt09:09:353
3018299533,36cyclictest25684-21taskset08:44:313
3018299533,36cyclictest25684-21taskset08:44:313
3018299533,35cyclictest17176-21taskset09:34:033
3018299532,36cyclictest27284-21chrt12:33:553
3018299532,33cyclictest25425-21taskset11:35:033
3018299523,37cyclictest11882-21taskset09:25:163
3018299523,35cyclictest23407-21chrt10:38:433
3018299523,34cyclictest5709-21chrt07:39:033
3018299523,31cyclictest14024-21chrt11:16:313
3018299522,37cyclictest14312-21chrt12:10:533
3018299522,36cyclictest818-21chrt09:59:363
3018299522,34cyclictest3350-21chrt07:29:033
3018299522,34cyclictest3350-21chrt07:29:033
3018299522,34cyclictest24960-21chrt09:46:193
3018299522,34cyclictest15281-21taskset08:14:023
3018299522,34cyclictest15281-21taskset08:14:023
30182995219,22cyclictest4946-21taskset11:01:083
30182995219,22cyclictest4946-21taskset11:01:083
3018299513,32cyclictest29416-21chrt09:54:173
3018299513,32cyclictest17399-21chrt08:19:243
3018299513,31cyclictest7275-21chrt11:59:273
3018299503,31cyclictest13440-21chrt10:20:483
3018299502,34cyclictest10751-21chrt12:04:333
3018299494,34cyclictest1647-21chrt11:49:343
3018299494,34cyclictest15761-21chrt08:14:223
3018299493,35cyclictest28866-21taskset10:47:223
3018299493,31cyclictest663-21chrt10:54:233
30182994931,9cyclictest19563-21chrt09:37:523
3018299492,36cyclictest30859-21taskset07:09:023
3018299492,36cyclictest30859-21taskset07:09:023
3018299492,34cyclictest23422-21chrt12:26:213
3018299486,21cyclictest0-21swapper/308:04:163
3018299484,31cyclictest6261-21chrt07:39:233
3018299483,35cyclictest8449-21chrt11:07:403
3018299483,28cyclictest3900-21chrt10:04:473
3018299483,23cyclictest22623-21chrt09:43:073
3018299483,23cyclictest22623-21chrt09:43:073
3018299482,34cyclictest28846-21chrt08:54:273
3018299482,33cyclictest18652-21taskset08:24:103
3018299473,34cyclictest10179-21taskset09:22:163
30182994727,12cyclictest157212sleep312:14:063
3018299472,31cyclictest28313-21chrt08:54:033
3018299463,30cyclictest16988-21chrt11:21:303
3018299463,30cyclictest16988-21chrt11:21:303
3018299462,34cyclictest1573-21chrt07:19:193
3018299462,32cyclictest10092-21chrt07:58:593
3018299462,32cyclictest10092-21chrt07:58:593
3018299443,32cyclictest28239-21chrt09:52:113
30182994431,6cyclictest98052sleep311:09:263
3018299433,29cyclictest11821-21taskset08:04:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional